summary refs log tree commit diff
path: root/synapse/handlers/user_directory.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Update license headersPatrick Cloke2023-11-211-10/+16
|
* Use full GitHub links instead of bare issue numbers. (#16637)Patrick Cloke2023-11-151-4/+4
|
* Convert state delta processing from a dict to attrs. (#16469)Patrick Cloke2023-10-161-17/+17
| | | For improved type checking & memory usage.
* Implements admin API to lock an user (MSC3939) (#15870)Mathieu Velten2023-08-101-1/+4
|
* Move Spam Checker callbacks to a dedicated file (#15453)Andrew Morgan2023-04-181-2/+4
|
* Refresh remote profiles that have been marked as stale, in order to fill the ↵reivilibre2023-03-161-0/+242
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | user directory. [rei:userdirpriv] (#14756) * Scaffolding for background process to refresh profiles * Add scaffolding for background process to refresh profiles for a given server * Implement the code to select servers to refresh from * Ensure we don't build up multiple looping calls * Make `get_profile` able to respect backoffs * Add logic for refreshing users * When backing off, schedule a refresh when the backoff is over * Wake up the background processes when we receive an interesting state event * Add tests * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Add comment about 1<<62 --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Preparatory work to fix the user directory assuming that any remote ↵reivilibre2023-03-161-34/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | membership state events represent a profile change. [rei:userdirpriv] (#14755) * Remove special-case method for new memberships only, use more generic method * Only collect profiles from state events in public rooms * Add a table to track stale remote user profiles * Add store methods to set and delete rows in this new table * Mark remote profiles as stale when a member state event comes in to a private room * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Simplify by removing Optionality of `event_id` * Replace names and avatars with None if they're set to dodgy things I think this makes more sense anyway. * Move schema delta to 74 (I missed the boat?) * Turns out these can be None after all --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Clarifications in user directory for users who share rooms tracking (#13966)Eric Eastwood2022-09-301-12/+24
| | | | | Spawned while working on [`get_users_in_room` mis-uses](https://github.com/matrix-org/synapse/pull/13958#discussion_r984074897) and thinking we could use `get_local_users_in_room` here but we can't. From first glance, it seemed like this was only using local users from all of the `is_mine_id(user_id)` checks but I see that it does actually use remote users. Just making things a little more clear here what it does and mentions remote users so maybe that will be more obvious in the future.
* Wait for lazy join to complete when getting current state (#12872)Erik Johnston2022-06-011-1/+5
|
* Add the `update_user_directory_from_worker` configuration option ↵reivilibre2022-05-101-1/+1
| | | | | (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. (#12654) Co-authored-by: Shay <hillerys@element.io>
* Correct `check_username_for_spam` annotations and docs (#12246)David Robertson2022-03-181-2/+2
| | | | | | | * Formally type the UserProfile in user searches * export UserProfile in synapse.module_api * Update docs Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Fix AssertionErrors after purging events (#11642)Richard van der Hoff2022-01-041-3/+15
| | | | | | | | | | | * Fix AssertionErrors after purging events If you purged a bunch of events from your database, and then restarted synapse without receiving more events, then you would get a bunch of AssertionErrors on restart. This fixes the situation by rewinding the stream processors. * `check-newsfragment`: ignore deleted newsfiles
* Fix adding excluded users to the private room sharing tables when joining a ↵David Robertson2021-10-211-15/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | room (#11143) * We only need to fetch users in private rooms * Filter out `user_id` at the top * Discard excluded users in the top loop We weren't doing this in the "First, if they're our user" branch so this is a bugfix. * The caller must check that `user_id` is included This is in the docstring. There are two call sites: - one in `_handle_room_publicity_change`, which explicitly checks before calling; - and another in `_handle_room_membership_event`, which returns early if the user is excluded. So this change is safe. * Test joining a private room with an excluded user * Tweak an existing test * Changelog * test docstring * lint
* Don't remove local users from dir when the leave their last room (#11103)David Robertson2021-10-181-5/+8
|
* Correctly exclude users when making a room public or private (#11075)David Robertson2021-10-151-4/+7
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Rearrange the user_directory's `_handle_deltas` function (#11035)David Robertson2021-10-131-57/+78
| | | | | * Pull out `_handle_room_membership_event` * Discard excluded users early * Rearrange logic so the change is membership is effectively switched over. See PR for rationale.
* Fix overwriting profile when making room public (#11003)David Robertson2021-10-081-29/+34
| | | This splits apart `handle_new_user` into a function which adds an entry to the `user_directory` and a function which updates the room sharing tables. I plan to continue doing more of this kind of refactoring to clarify the implementation.
* Don't alter directory entries for local users when setting a per-room ↵David Robertson2021-10-071-7/+13
| | | | | nickname (#11002) Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Don't drop user dir deltas when server leaves room (#10982)David Robertson2021-10-061-1/+1
| | | | | | | | | Fix a long-standing bug where a batch of user directory changes would be silently dropped if the server left a room early in the batch. * Pull out `wait_for_background_update` in tests Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Consistently exclude from user_directory (#10960)David Robertson2021-10-041-18/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Introduce `should_include_local_users_in_dir` We exclude three kinds of local users from the user_directory tables. At present we don't consistently exclude all three in the same places. This commit introduces a new function to gather those exclusion conditions together. Because we have to handle local and remote users in different ways, I've made that function only consider the case of remote users. It's the caller's responsibility to make the local versus remote distinction clear and correct. A test fixup is required. The test now hits a path which makes db queries against the users table. The expected rows were missing, because we were using a dummy user that hadn't actually been registered. We also add new test cases to covert the exclusion logic. ---- By my reading this makes these changes: * When an app service user registers or changes their profile, they will _not_ be added to the user directory. (Previously only support and deactivated users were excluded). This is consistent with the logic that rebuilds the user directory. See also [the discussion here](https://github.com/matrix-org/synapse/pull/10914#discussion_r716859548). * When rebuilding the directory, exclude support and disabled users from room sharing tables. Previously only appservice users were excluded. * Exclude all three categories of local users when rebuilding the directory. Previously `_populate_user_directory_process_users` didn't do any exclusion. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-291-1/+1
|
* Use direct references for configuration variables (part 5). (#10897)Patrick Cloke2021-09-241-1/+1
|
* Require type hints in the handlers module. (#10831)Patrick Cloke2021-09-201-1/+1
| | | | | | | Adds missing type hints to methods in the synapse.handlers module and requires all methods to have type hints there. This also removes the unused construct_auth_difference method from the FederationHandler.
* Easy refactors of the user directory (#10789)David Robertson2021-09-101-18/+28
| | | No functional changes here. This came out as I was working to tackle #5677
* Use inline type hints in `handlers/` and `rest/`. (#10382)Jonathan de Jong2021-07-161-1/+1
|
* User directory: use calculated room membership state instead (#9821)Andrew Morgan2021-04-161-7/+8
| | | | | Fixes: #9797. Should help reduce CPU usage on the user directory, especially when memberships change in rooms with lots of state history.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Import HomeServer from the proper module. (#9665)Patrick Cloke2021-03-231-1/+1
|
* Add back the guard against the user directory stream position not existing. ↵Patrick Cloke2021-02-181-0/+4
| | | | | | (#9428) As the comment says, this guard was there for when the initial user directory update has yet to happen.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-6/+3
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Add type hints to various handlers. (#9223)Patrick Cloke2021-01-261-4/+5
| | | | With this change all handlers except the e2e_* ones have type hints enabled.
* Add type hints to the receipts and user directory handlers. (#8976)Patrick Cloke2021-01-041-21/+42
|
* Fix a bug that deactivated users appear in the directory (#8933)Dirk Klimpel2020-12-171-2/+6
| | | | | | | | | | Fixes a bug that deactivated users appear in the directory when their profile information was updated. To change profile information of deactivated users is neccesary for example you will remove displayname or avatar. But they should not appear in directory. They are deactivated. Co-authored-by: Erik Johnston <erikj@jki.re>
* Do not assume that the contents dictionary includes history_visibility. (#8945)Patrick Cloke2020-12-161-2/+2
|
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-5/+5
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-231-1/+1
|
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-181-1/+1
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Fix errors when updating the user directory with invalid data (#8223)Patrick Cloke2020-09-011-1/+7
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-4/+2
|
* Convert user directory handler and related classes to async/await. (#7640)Patrick Cloke2020-06-051-72/+46
|
* Filter the results of user directory searching via the spam checker (#6888)Patrick Cloke2020-02-141-2/+12
| | | Add a method to the spam checker to filter the user directory results.
* Reduce amount of logging at INFO level. (#6862)Erik Johnston2020-02-061-2/+2
| | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* Fix races in room stats (and other) updates. (#6187)Richard van der Hoff2019-10-101-5/+12
| | | | | | | Hopefully this will fix the occasional failures we were seeing in the room directory. The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8. We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-1/+1
|
* Fix grammar and document get_current_users_in_room (#4998)Andrew Morgan2019-04-031-2/+2
|
* Refactor out state delta handling into its own class (#4917)Amber Brown2019-03-251-46/+5
|
* Migrate the user directory initial population to a background task (#4864)Amber Brown2019-03-191-160/+13
|
* fixupAmber Brown2019-03-131-1/+1
|
* fixupAmber Brown2019-03-121-47/+70
|
* fixesAmber Brown2019-03-121-39/+6
|
* initialAmber Brown2019-03-111-1/+40
|
* Rewrite userdir to be faster (#4537)Amber Brown2019-03-071-185/+37
|
* Run `black` on user directory code (#4635)Amber Brown2019-02-131-44/+50
|
* Add metric for user dir current event stream positionAndrew Morgan2019-01-231-0/+6
|
* create support user (#4141)Neil Johnson2018-12-141-20/+25
| | | | | | Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
* Give some more things logcontexts (#4077)Richard van der Hoff2018-10-231-5/+9
|
* Add some fixmesErik Johnston2018-08-171-0/+4
|
* run isortAmber Brown2018-07-091-3/+4
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-5/+4
|
* Merge pull request #3244 from NotAFile/py3-six-4Amber Brown2018-05-241-1/+2
|\ | | | | replace some iteritems with six
| * replace some iteritems with sixAdrian Tschira2018-05-191-1/+2
| | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Remove users from user directory on deactivateDavid Baker2018-05-241-0/+7
|/
* speed up the rate of initial spam for usersMatthew Hodgson2017-12-041-11/+12
|
* fix pep8 and testsMatthew Hodgson2017-12-041-3/+2
|
* switch to a simpler 'search_all_users' button as per review feedbackMatthew Hodgson2017-12-041-3/+3
|
* fix thinkos galoreMatthew Hodgson2017-11-301-4/+6
|
* untested WIP but might actually workMatthew Hodgson2017-11-291-4/+42
|
* Add user_directory_include_pattern config param to expand search results to ↵Matthew Hodgson2017-11-291-2/+2
| | | | | | | | | | additional users Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive. It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users... ...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too. Committing it here as a WIP
* s/intial/initial/Matthew Hodgson2017-11-041-2/+2
|
* Make some more params configurableErik Johnston2017-06-221-5/+8
|
* Typo. ARGH.Erik Johnston2017-06-221-1/+1
|
* Fix thinko in unhandled user spamErik Johnston2017-06-221-2/+2
|
* Don't work out users who share room with appservice usersErik Johnston2017-06-211-2/+11
|
* Initial worker implErik Johnston2017-06-161-6/+13
|
* Include users who share room with requester in user directoryErik Johnston2017-06-151-2/+2
|
* Implement updating users who share rooms on the flyErik Johnston2017-06-151-23/+125
|
* Implement initial population of users who share rooms tableErik Johnston2017-06-151-2/+76
|
* Fix user directory insertion due to missing room_idErik Johnston2017-06-131-3/+7
|
* Don't assume existance of events when updating user directoryErik Johnston2017-06-131-3/+6
|
* Change is_host_joined to use current_state tableErik Johnston2017-06-091-3/+3
| | | | | | This bypasses a bug where using the state groups to figure out if a host is in a room sometimes errors if the servers isn't in the room. (For example when the server rejected an invite to a remote room)
* Don't start user_directory handling on workersErik Johnston2017-06-071-0/+3
|
* Add some logging to user directoryErik Johnston2017-06-071-0/+7
|
* Handle profile updates in user directoryErik Johnston2017-06-011-0/+25
|
* Split the table in twoErik Johnston2017-06-011-27/+50
|
* CommentsErik Johnston2017-06-011-3/+3
|
* Check if host is still in roomErik Johnston2017-06-011-0/+11
|
* Fix removing usersErik Johnston2017-06-011-0/+5
|
* Bug fixes and loggingErik Johnston2017-06-011-1/+21
| | | | | - Check if room is public when a user joins before adding to user dir - Fix typo of field name "content.join_rules" -> "content.join_rule"
* Comment briefly on how we keep user_directory up to dateErik Johnston2017-05-311-0/+10
|
* Lifts things into separate functionErik Johnston2017-05-311-56/+54
|
* TyposErik Johnston2017-05-311-8/+8
|
* Fix testsErik Johnston2017-05-311-2/+2
|
* Pull max id from correct tableErik Johnston2017-05-311-4/+2
|
* Handle the server leaving a public roomErik Johnston2017-05-311-3/+20
|
* Add commentsErik Johnston2017-05-311-41/+120
|
* Add searchErik Johnston2017-05-311-0/+3
|
* Update room column when room becomes unpublicErik Johnston2017-05-311-2/+21
|
* Add call laterErik Johnston2017-05-311-0/+2
|
* Start laterErik Johnston2017-05-311-1/+4
|
* Don't go round in circlesErik Johnston2017-05-311-4/+2
|
* Add user_directory to databaseErik Johnston2017-05-311-0/+218