summary refs log tree commit diff
path: root/tests/handlers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix style checking due to updated black.Patrick Cloke2021-02-191-1/+4
|
* Be smarter about which hosts to send presence to when processing room joins ↵Andrew Morgan2021-02-191-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | (#9402) This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually. --- When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed. It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw. This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence: * If it was a local user join, send that user's latest presence to all servers in the room * If it was a remote user join, send the presence for all local users in the room to that homeserver We deduplicate by inserting all of those pending updates into a dictionary of the form: ``` { server_name1: {presence_update1, ...}, server_name2: {presence_update1, presence_update2, ...} } ``` Only after building this dict do we then start sending out presence updates.
* Add a config option to prioritise local users in user directory search ↵Andrew Morgan2021-02-191-0/+94
| | | | | | | | | results (#9383) This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default. Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical. This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.
* Add configs to make profile data more private (#9203)AndrewFerr2021-02-191-1/+5
| | | | | | | Add off-by-default configuration settings to: - disable putting an invitee's profile info in invite events - disable profile lookup via federation Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
* Support for form_post in OIDC responses (#9376)Richard van der Hoff2021-02-171-13/+13
| | | Apple want to POST the OIDC auth response back to us rather than using query-params; add the necessary support to make that work.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-1613-77/+101
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Clean up caching/locking of OIDC metadata load (#9362)Richard van der Hoff2021-02-161-26/+45
| | | | Ensure that we lock correctly to prevent multiple concurrent metadata load requests, and generally clean up the way we construct the metadata cache.
* Convert additional test-cases to homeserver test case. (#9396)Patrick Cloke2021-02-161-17/+16
| | | And convert some inlineDeferreds to async-friendly functions.
* Convert some test cases to use HomeserverTestCase. (#9377)Patrick Cloke2021-02-114-488/+301
| | | | This has the side-effect of being able to remove use of `inlineCallbacks` in the test-cases for cleaner tracebacks.
* Combine the CAS & SAML implementations for required attributes. (#9326)Patrick Cloke2021-02-112-3/+105
|
* Honour ratelimit flag for application services for invite ratelimiting (#9302)Erik Johnston2021-02-031-47/+0
|
* Put OIDC callback URI under /_synapse/client. (#9288)Richard van der Hoff2021-02-011-9/+6
|
* Merge branch 'social_login' into developRichard van der Hoff2021-02-013-18/+22
|\
| * Improve styling and wording of SSO redirect confirm template (#9272)Richard van der Hoff2021-02-013-18/+22
| |
* | Ratelimit invites by room and target user (#9258)Erik Johnston2021-01-291-1/+92
|/
* Land support for multiple OIDC providers (#9110)Richard van der Hoff2021-01-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | This is the final step for supporting multiple OIDC providers concurrently. First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before: oidc_config: enabled: true issuer: "https://oidc_provider" # etc After: oidc_providers: - idp_id: prov1 issuer: "https://oidc_provider" - idp_id: prov2 issuer: "https://another_oidc_provider" The old format is still grandfathered in. With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.
* Improve UsernamePickerTestCase (#9112)Richard van der Hoff2021-01-151-118/+2
| | | | | * make the OIDC bits of the test work at a higher level - via the REST api instead of poking the OIDCHandler directly. * Move it to test_login.py, where I think it fits better.
* Store an IdP ID in the OIDC session (#9109)Richard van der Hoff2021-01-151-1/+2
| | | | | Again in preparation for handling more than one OIDC provider, add a new caveat to the macaroon used as an OIDC session cookie, which remembers which OIDC provider we are talking to. In future, when we get a callback, we'll need it to make sure we talk to the right IdP. As part of this, I'm adding an idp_id and idp_name field to the OIDC configuration object. They aren't yet documented, and we'll just use the old values by default.
* Split OidcProvider out of OidcHandler (#9107)Richard van der Hoff2021-01-141-45/+48
| | | | | | | The idea here is that we will have an instance of OidcProvider for each configured IdP, with OidcHandler just doing the marshalling of them. For now it's still hardcoded with a single provider.
* Preparatory refactors of OidcHandler (#9067)Richard van der Hoff2021-01-131-24/+37
| | | | | | | | Some light refactoring of OidcHandler, in preparation for bigger things: * remove inheritance from deprecated BaseHandler * add an object to hold the things that go into a session cookie * factor out a separate class for manipulating said cookies
* Remove user's avatar URL and displayname when deactivated. (#8932)Dirk Klimpel2021-01-121-0/+30
| | | This only applies if the user's data is to be erased.
* Remove SynapseRequest.get_user_agent (#9069)Richard van der Hoff2021-01-123-4/+3
| | | | | | | | | | | SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests", which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the LoggingContext and write the right entries to the request log). Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a SynapseRequest when there is nothing synapse-specific about the Request at all, and any old twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult. In short: move get_user_agent out to a utility function.
* Skip unit tests which require optional dependencies (#9031)Richard van der Hoff2021-01-071-1/+18
| | | If we are lacking an optional dependency, skip the tests that rely on it.
* Use the SSO handler helpers for CAS registration/login. (#8856)Patrick Cloke2021-01-031-0/+121
|
* Implement a username picker for synapse (#8942)Richard van der Hoff2020-12-181-1/+142
| | | | | | | | | | | | | | The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow `UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser back to the client, we redirect to a username-picker resource, which allows the user to enter a username. We *then* complete the SSO flow (including doing the client permission checks). The static resources for the username picker itself (in https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker) are essentially lifted wholesale from https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. As the comment says, we might want to think about making them customisable, but that can be a follow-up. Fixes #8876.
* Fix a bug that deactivated users appear in the directory (#8933)Dirk Klimpel2020-12-171-1/+39
| | | | | | | | | | Fixes a bug that deactivated users appear in the directory when their profile information was updated. To change profile information of deactivated users is neccesary for example you will remove displayname or avatar. But they should not appear in directory. They are deactivated. Co-authored-by: Erik Johnston <erikj@jki.re>
* Push login completion down into SsoHandler (#8941)Richard van der Hoff2020-12-161-4/+4
| | | This is another part of my work towards fixing #8876. It moves some of the logic currently in the SAML and OIDC handlers - in particular the call to `AuthHandler.complete_sso_login` down into the `SsoHandler`.
* Merge pull request #8951 from matrix-org/rav/username_picker_2Richard van der Hoff2020-12-161-81/+104
|\ | | | | More preparatory refactoring of the OidcHandler tests
| * Make `_make_callback_with_userinfo` asyncRichard van der Hoff2020-12-151-68/+83
| | | | | | | | | | | | ... so that we can test its behaviour when it raises. Also pull it out to the top level so that I can use it from other test classes.
| * Remove spurious mocking of complete_sso_loginRichard van der Hoff2020-12-151-2/+0
| | | | | | | | The tests that need this all do it already.
| * Test `get_extra_attributes` fallbackRichard van der Hoff2020-12-151-11/+21
| | | | | | | | | | despite the warnings saying "don't implement get_extra_attributes", we had implemented it, so the tests weren't doing what we thought they were.
* | Remove spurious "SynapseRequest" result from `make_request"Richard van der Hoff2020-12-155-13/+13
|/ | | | This was never used, so let's get rid of it.
* Preparatory refactoring of the SamlHandlerTestCase (#8938)Richard van der Hoff2020-12-152-54/+90
| | | | | | | | | | | | | * move simple_async_mock to test_utils ... so that it can be re-used * Remove references to `SamlHandler._map_saml_response_to_user` from tests This method is going away, so we can no longer use it as a test point. Instead, factor out a higher-level method which takes a SAML object, and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`. * changelog
* Fix startup failure with localdb_enabled: False (#8937)Richard van der Hoff2020-12-141-0/+23
|
* Various clean-ups to the logging context code (#8935)Patrick Cloke2020-12-141-3/+3
|
* Preparatory refactoring of the OidcHandlerTestCase (#8911)Richard van der Hoff2020-12-141-141/+145
| | | | | | | | | | | | | | | | | | | | | | | * Remove references to handler._auth_handler (and replace them with hs.get_auth_handler) * Factor out a utility function for building Requests * Remove mocks of `OidcHandler._map_userinfo_to_user` This method is going away, so mocking it out is no longer a valid approach. Instead, we mock out lower-level methods (eg _remote_id_from_userinfo), or simply allow the regular implementation to proceed and update the expectations accordingly. * Remove references to `OidcHandler._map_userinfo_to_user` from tests This method is going away, so we can no longer use it as a test point. Instead we build mock "callback" requests which we pass into `handle_oidc_callback`, and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`.
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-2/+2
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Skip the SAML tests if xmlsec1 isn't available. (#8905)Patrick Cloke2020-12-091-0/+23
|
* Merge tag 'v1.24.0rc2' into developPatrick Cloke2020-12-042-2/+29
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.24.0rc2 (2020-12-04) ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
| * Fix a regression that mapping providers should be able to redirect users. ↵Patrick Cloke2020-12-042-2/+29
| | | | | | | | | | (#8878) This was broken in #8801.
* | Merge remote-tracking branch 'origin/develop' into rav/remove_unused_mocksRichard van der Hoff2020-12-022-31/+9
|\ \
| * \ Merge pull request #8858 from matrix-org/rav/sso_uiaRichard van der Hoff2020-12-022-19/+9
| |\ \ | | | | | | | | UIA: offer only available auth flows
| | * | Factor out FakeResponse from test_oidcRichard van der Hoff2020-12-021-16/+1
| | | |
| | * | fix up various test casesRichard van der Hoff2020-12-021-3/+8
| | |/ | | | | | | | | | | | | A few test cases were relying on being able to mount non-client servlets on the test resource. it's better to give them their own Resources.
* | | remove unused `resource_for_federation`Richard van der Hoff2020-12-021-1/+13
| | | | | | | | | | | | This is now only used in `test_typing`, so move it there.
* | | Remove redundant mockingRichard van der Hoff2020-12-022-4/+0
|/ /
* / Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-026-9/+9
|/ | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* Fix a regression when grandfathering SAML users. (#8855)Patrick Cloke2020-12-022-1/+41
| | | | | | This was broken in #8801 when abstracting code shared with OIDC. After this change both SAML and OIDC have a concept of grandfathering users, but with different implementations.
* Add basic SAML tests for mapping users. (#8800)Patrick Cloke2020-12-022-17/+153
|
* Create a `PasswordProvider` wrapper object (#8849)Richard van der Hoff2020-12-021-2/+3
| | | | The idea here is to abstract out all the conditional code which tests which methods a given password provider has, to provide a consistent interface.
* Support "identifier" dicts in UIA (#8848)Richard van der Hoff2020-12-011-10/+1
| | | | | | | | | | The spec requires synapse to support `identifier` dicts for `m.login.password` user-interactive auth, which it did not (instead, it required an undocumented `user` parameter.) To fix this properly, we need to pull the code that interprets `identifier` into `AuthHandler.validate_login` so that it can be called from the UIA code. Fixes #5665.
* Don't offer password login when it is disabled (#8835)Richard van der Hoff2020-12-011-3/+105
| | | Fix a minor bug where we would offer "m.login.password" login if a custom auth provider supported it, even if password login was disabled.
* Add some tests for `password_auth_providers` (#8819)Richard van der Hoff2020-12-011-0/+486
| | | | These things seemed to be completely untested, so I added a load of tests for them.
* Simplify the way the `HomeServer` object caches its internal attributes. ↵Jonathan de Jong2020-11-301-3/+3
| | | | | (#8565) Changes `@cache_in_self` to use underscore-prefixed attributes.
* Support trying multiple localparts for OpenID Connect. (#8801)Patrick Cloke2020-11-251-1/+87
| | | | Abstracts the SAML and OpenID Connect code which attempts to regenerate the localpart of a matrix ID if it is already in use.
* Improve error checking for OIDC/SAML mapping providers (#8774)Patrick Cloke2020-11-191-20/+69
| | | | | | Checks that the localpart returned by mapping providers for SAML and OIDC are valid before registering new users. Extends the OIDC tests for existing users and invalid data.
* Consistently use room_id from federation request body (#8776)Richard van der Hoff2020-11-191-1/+0
| | | | | | | | | | | | | * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts".
* Merge pull request #8761 from matrix-org/rav/test_request_renderingRichard van der Hoff2020-11-174-10/+0
|\ | | | | Make `make_request` actually render the request
| * Remove redundant `HomeserverTestCase.render`Richard van der Hoff2020-11-164-10/+0
| |
* | Abstract shared SSO code. (#8765)Patrick Cloke2020-11-171-7/+7
| | | | | | De-duplicates code between the SAML and OIDC implementations.
* | Add admin API for logging in as a user (#8617)Erik Johnston2020-11-171-4/+10
|/
* Block clients from sending server ACLs that lock the local server out. (#8708)Erik Johnston2020-11-031-0/+57
| | | Fixes #4042
* Add ability for access tokens to belong to one user but grant access to ↵Erik Johnston2020-10-292-2/+2
| | | | | | | | | | another user. (#8616) We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't). A future PR will add an API for creating such a token. When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
* Start fewer opentracing spans (#8640)Erik Johnston2020-10-261-12/+8
| | | | | | | #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.
* Fix handling of User-Agent headers with bad utf-8. (#8632)Erik Johnston2020-10-231-6/+18
|
* Merge pull request #8537 from matrix-org/rav/simplify_locally_reject_inviteRichard van der Hoff2020-10-152-2/+1
|\ | | | | Simplify `_locally_reject_invite`
| * Simplify `_locally_reject_invite`Richard van der Hoff2020-10-131-1/+1
| | | | | | | | | | Update `EventCreationHandler.create_event` to accept an auth_events param, and use it in `_locally_reject_invite` instead of reinventing the wheel.
| * Remove redundant `token_id` parameter to create_eventRichard van der Hoff2020-10-131-1/+0
| | | | | | | | this is always the same as requester.access_token_id.
* | Fix not sending events over federation when using sharded event persisters ↵Erik Johnston2020-10-141-3/+10
|/ | | | | | | | | | | | | | | | | (#8536) * Fix outbound federaion with multiple event persisters. We incorrectly notified federation senders that the minimum persisted stream position had advanced when we got an `RDATA` from an event persister. Notifying of federation senders already correctly happens in the notifier, so we just delete the offending line. * Change some interfaces to use RoomStreamToken. By enforcing use of `RoomStreamTokens` we make it less likely that people pass in random ints that they got from somewhere random.
* Move additional tasks to the background worker, part 4 (#8513)Patrick Cloke2020-10-131-34/+14
|
* Fix message duplication if something goes wrong after persisting the event ↵Erik Johnston2020-10-131-0/+157
| | | | | (#8476) Should fix #3365.
* Remove the deprecated Handlers object (#8494)Patrick Cloke2020-10-099-40/+20
| | | All handlers now available via get_*_handler() methods on the HomeServer.
* Invalidate the cache when an olm fallback key is uploaded (#8501)Hubert Chathi2020-10-081-0/+20
|
* Add support for MSC2697: Dehydrated devices (#8380)Hubert Chathi2020-10-071-0/+82
| | | | This allows a user to store an offline device on the server and then restore it at a subsequent login.
* Merge pull request #8463 from matrix-org/rav/clean_up_event_handlingRichard van der Hoff2020-10-071-1/+1
|\ | | | | Reduce inconsistencies between codepaths for membership and non-membership events.
| * kill off `send_nonmember_event`Richard van der Hoff2020-10-051-1/+1
| | | | | | | | This is now redundant, and we can just call `handle_new_client_event` directly.
* | Add support for MSC2732: olm fallback keys (#8312)Hubert Chathi2020-10-061-0/+65
|/
* Add config option for always using "userinfo endpoint" for OIDC (#7658)BBBSnowball2020-10-011-2/+8
| | | This allows for connecting to certain IdPs, e.g. GitLab.
* Allow additional SSO properties to be passed to the client (#8413)Patrick Cloke2020-09-301-55/+105
|
* Allow existing users to login via OpenID Connect. (#8345)Tdxdxoz2020-09-251-0/+35
| | | | | | | Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com> This adds configuration flags that will match a user to pre-existing users when logging in via OpenID Connect. This is useful when switching to an existing SSO system.
* Create function to check for long names in devices (#8364)Dionysis Grigoropoulos2020-09-221-0/+11
| | | | | | | | | * Create a new function to verify that the length of a device name is under a certain threshold. * Refactor old code and tests to use said function. * Verify device name length during registration of device * Add a test for the above Signed-off-by: Dionysis Grigoropoulos <dgrig@erethon.com>
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-182-2/+2
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Catch-up after Federation Outage (split, 4): catch-up loop (#8272)reivilibre2020-09-151-0/+5
|
* Fixup pusher pool notifications (#8287)Erik Johnston2020-09-091-0/+1
| | | | | `pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed. I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
* Revert "Fixup pusher pool notifications"Erik Johnston2020-09-091-1/+0
| | | | This reverts commit e7fd336a53a4ca489cdafc389b494d5477019dc0.
* Fixup pusher pool notificationsErik Johnston2020-09-091-0/+1
|
* Allow for make_awaitable's return value to be re-used. (#8261)Patrick Cloke2020-09-083-16/+16
|
* Rename 'populate_stats_process_rooms_2' background job back to ↵Andrew Morgan2020-09-081-9/+6
| | | | | | | 'populate_stats_process_rooms' again (#8243) Fixes https://github.com/matrix-org/synapse/issues/8238 Alongside the delta file, some changes were also necessary to the codebase to remove references to the now defunct `populate_stats_process_rooms_2` background job. Thankfully the latter doesn't seem to have made it into any documentation yet :)
* Stop sub-classing object (#8249)Patrick Cloke2020-09-042-2/+2
|
* Ensure that the OpenID Connect remote ID is a string. (#8190)Patrick Cloke2020-08-281-2/+39
|
* Do not yield on awaitables in tests. (#8193)Patrick Cloke2020-08-271-1/+3
|
* Convert stats and related calls to async/await (#8192)Patrick Cloke2020-08-272-17/+16
|
* Convert simple_update* and simple_select* to async (#8173)Patrick Cloke2020-08-271-2/+2
|
* Reduce run-times of tests by advancing the reactor less (#7757)Andrew Morgan2020-08-271-1/+1
|
* Do not propagate typing notifications from shadow-banned users. (#8176)Patrick Cloke2020-08-261-6/+20
|
* Convert simple_select_one and simple_select_one_onecol to async (#8162)Patrick Cloke2020-08-262-14/+46
|
* Allow denying or shadow banning registrations via the spam checker (#8034)Patrick Cloke2020-08-203-7/+69
|
* Convert some of the general database methods to async (#8100)Patrick Cloke2020-08-172-3/+3
|
* Convert stream database to async/await. (#8074)Patrick Cloke2020-08-171-1/+1
|
* Convert devices database to async/await. (#8069)Patrick Cloke2020-08-121-1/+1
|
* Convert account data, device inbox, and censor events databases to ↵Patrick Cloke2020-08-121-1/+2
| | | | async/await (#8063)
* Converts event_federation and registration databases to async/await (#8061)Patrick Cloke2020-08-111-5/+6
|
* Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users ↵Patrick Cloke2020-08-071-1/+1
| | | | database to async (#8042)
* Convert synapse.api to async/await (#8031)Patrick Cloke2020-08-061-2/+2
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-052-42/+42
|
* Prevent join->join membership transitions changing member count (#7977)Andrew Morgan2020-08-031-6/+40
| | | | | | | | | | | `StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc. However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information. This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point. Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update). Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
* Convert federation client to async/await. (#7975)Patrick Cloke2020-07-302-4/+4
|
* Convert appservice to async. (#7973)Patrick Cloke2020-07-301-2/+3
|
* Convert device handler to async/await (#7871)Patrick Cloke2020-07-172-12/+11
|
* Convert _base, profile, and _receipts handlers to async/await (#7860)Patrick Cloke2020-07-171-6/+11
|
* Fix testsErik Johnston2020-07-151-2/+2
|
* Convert E2E key and room key handlers to async/await. (#7851)Patrick Cloke2020-07-152-237/+422
|
* Convert the appservice handler to async/await. (#7775)Patrick Cloke2020-07-061-33/+35
|
* isort 5 compatibility (#7786)Will Hunt2020-07-051-3/+1
| | | The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
* Additional configuration options for auto-join rooms (#7763)Patrick Cloke2020-06-301-2/+210
|
* Convert the typing handler to async/await. (#7679)Patrick Cloke2020-06-171-6/+7
|
* Ensure etag is a string for GET room_keys/version response (#7691)Hubert Chathi2020-06-151-0/+1
|
* Add option to enable encryption by default for new rooms (#7639)Andrew Morgan2020-06-102-5/+104
| | | | | | | | | Fixes https://github.com/matrix-org/synapse/issues/2431 Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used. Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637 Based on #7637
* Convert user directory handler and related classes to async/await. (#7640)Patrick Cloke2020-06-051-4/+4
|
* Add an option to disable autojoin for guest accounts (#6637)Travis Ralston2020-06-051-0/+10
| | | | Fixes https://github.com/matrix-org/synapse/issues/3177
* Performance improvements and refactor of Ratelimiter (#7595)Andrew Morgan2020-06-051-5/+1
| | | | | | | | | | While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both: * Rather undocumented, and * causing a *lot* of config checks This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. Best to be reviewed commit-by-commit.
* Fix encryption algorithm typos in tests/comments (#7637)Andrew Morgan2020-06-041-5/+5
| | | | | | | | @uhoreg has confirmed these were both typos. They are only in comments and tests though, rather than anything critical. Introduced in: * https://github.com/matrix-org/synapse/pull/7157 * https://github.com/matrix-org/synapse/pull/5726
* Add ability to wait for replication streams (#7542)Erik Johnston2020-05-221-1/+4
| | | | | | | The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room). Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on. People probably want to look at this commit by commit.
* Support UI Authentication for OpenID Connect accounts (#7457)Patrick Cloke2020-05-151-5/+10
|
* Implement room version 6 (MSC2240). (#7506)Patrick Cloke2020-05-151-3/+3
|
* Strictly enforce canonicaljson requirements in a new room version (#7381)Patrick Cloke2020-05-141-1/+66
|
* Extend spam checker to allow for multiple modules (#7435)Andrew Morgan2020-05-081-2/+2
|
* Implement OpenID Connect-based login (#7256)Quentin Gliech2020-05-081-0/+565
|
* Stop Auth methods from polling the config on every req. (#7420)Andrew Morgan2020-05-062-14/+22
|
* async/await is_server_admin (#7363)Andrew Morgan2020-05-012-36/+53
|
* Convert auth handler to async/await (#7261)Patrick Cloke2020-04-152-29/+55
|
* Allow admins to create aliases when they are not in the room (#7191)Patrick Cloke2020-04-011-0/+62
|
* Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_testsRichard van der Hoff2020-03-301-1/+1
|\ | | | | Add tests for outbound device pokes
| * Remove spurious "name" parameter to `default_config`Richard van der Hoff2020-03-241-1/+1
| | | | | | | | | | this is never set to anything other than "test", and is a source of unnecessary boilerplate.
* | Add options to prevent users from changing their profile. (#7096)Dirk Klimpel2020-03-271-1/+64
|/
* Revert "Add options to disable setting profile info for prevent changes. ↵Richard van der Hoff2020-03-171-32/+1
| | | | | | | (#7053)" This reverts commit 54dd28621b070ca67de9f773fe9a89e1f4dc19da, reversing changes made to 6640460d054e8f4444046a34bdf638921b31c01e.
* Add options to disable setting profile info for prevent changes. (#7053)Brendan Abolivier2020-03-101-1/+32
|\
| * updates after reviewdklimpel2020-03-091-3/+3
| |
| * fix testsdklimpel2020-03-091-2/+2
| |
| * add testsdklimpel2020-03-091-3/+3
| |
| * Add options to disable setting profile info for prevent changes.dklimpel2020-03-081-1/+32
| |
* | Allow deleting an alias if the user has sufficient power level (#6986)Patrick Cloke2020-03-041-24/+104
| |
* | Validate the alt_aliases property of canonical alias events (#6971)Patrick Cloke2020-03-031-36/+30
|/
* Store room version on invite (#6983)Richard van der Hoff2020-02-261-0/+1
| | | | | When we get an invite over federation, store the room version in the rooms table. The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
* Port PresenceHandler to async/await (#6991)Erik Johnston2020-02-261-6/+12
|
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-213-11/+11
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Refactor the membership check methods in AuthRichard van der Hoff2020-02-181-2/+2
| | | | | these were getting a bit unwieldy, so let's combine `check_joined_room` and `check_user_was_in_room` into a single `check_user_in_room`.
* Stop sending events when creating or deleting aliases (#6904)Patrick Cloke2020-02-181-2/+152
| | | Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
* Convert the directory handler tests to use HomeserverTestCase (#6919)Patrick Cloke2020-02-141-24/+17
| | | Convert directory handler tests to use HomeserverTestCase.
* Filter the results of user directory searching via the spam checker (#6888)Patrick Cloke2020-02-141-0/+92
| | | Add a method to the spam checker to filter the user directory results.
* Reject device display names that are too long (#6882)Patrick Cloke2020-02-101-0/+18
| | | | | | | | * Reject device display names that are too long. Too long is currently defined as 100 characters in length. * Add a regression test for rejecting a too long device display name.
* Add typing to synapse.federation.sender (#6871)Erik Johnston2020-02-071-2/+6
|
* Pass room_version into `event_from_pdu_json`Richard van der Hoff2020-02-061-2/+4
| | | | It's called from all over the shop, so this one's a bit messy.
* Merge pull request #6806 from matrix-org/rav/redact_changes/3Richard van der Hoff2020-01-311-2/+2
|\ | | | | Pass room_version into add_hashes_and_signatures
| * Store the room version in EventBuilderRichard van der Hoff2020-01-301-2/+2
| |
* | s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-1/+1
|/ | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* Resync remote device list when detected as stale. (#6786)Erik Johnston2020-01-301-3/+3
|
* Add `local_current_membership` table (#6655)Erik Johnston2020-01-151-2/+2
| | | | | | | Currently we rely on `current_state_events` to figure out what rooms a user was in and their last membership event in there. However, if the server leaves the room then the table may be cleaned up and that information is lost. So lets add a table that separately holds that information.
* Kill off RegistrationError (#6691)Richard van der Hoff2020-01-131-2/+0
| | | This is pretty pointless. Let's just use SynapseError.
* Add database config class (#6513)Erik Johnston2019-12-181-19/+20
| | | | | This encapsulates config for a given database and is the way to get new connections.
* look up cross-signing keys from the DB in bulk (#6486)Hubert Chathi2019-12-121-8/+0
|
* Back out perf regression from get_cross_signing_keys_from_cache. (#6494)Neil Johnson2019-12-091-0/+8
| | | Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
* Merge pull request #6484 from matrix-org/erikj/port_sync_handlerErik Johnston2019-12-092-23/+34
|\ | | | | Port SyncHandler to async/await
| * Fixup functions to consistently return deferredsErik Johnston2019-12-061-6/+18
| |
| * Port SyncHandler to async/awaitErik Johnston2019-12-051-17/+16
| |
* | Move background update handling out of storeErik Johnston2019-12-052-20/+48
| |
* | Move DB pool and helper functions into dedicated Database classErik Johnston2019-12-052-21/+21
|/
* Remove underscore from SQLBaseStore functionsErik Johnston2019-12-042-21/+21
|
* Implementation of MSC2314 (#6176)Amber Brown2019-11-281-0/+3
|
* add etag and count to key backup endpoints (#5858)Hubert Chathi2019-11-271-0/+31
|
* Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-0/+126
| | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
* rename get_devices_by_remote to get_device_updates_by_remoteHubert Chathi2019-10-301-2/+2
|
* Port federation_server to async/awaitErik Johnston2019-10-291-0/+3
|
* Fix testsErik Johnston2019-10-251-0/+2
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-221-1/+294
|\ | | | | | | erikj/refactor_stores
| * Merge branch 'develop' into cross-signing_sig_uploadHubert Chathi2019-10-186-59/+185
| |\
| * | use something that's the right type for user_idHubert Chathi2019-09-061-1/+1
| | |
| * | fix testHubert Chathi2019-09-051-1/+3
| | |
| * | add testHubert Chathi2019-09-041-0/+88
| | |
| * | make isort happyHubert Chathi2019-09-041-0/+1
| | |
| * | make black happyHubert Chathi2019-09-041-85/+62
| | |
| * | allow uploading signatures of master key signed by devicesHubert Chathi2019-09-041-1/+226
| | |
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-221-0/+65
|\ \ \ | | |/ | |/| | | | erikj/refactor_stores
| * | Merge branch 'develop' into uhoreg/e2e_cross-signing_mergedHubert Chathi2019-10-186-59/+185
| |\ \ | | |/ | |/|
| * | Merge branch 'develop' into cross-signing_keysHubert Chathi2019-09-042-112/+533
| |\ \
| * | | fix formattingHubert Chathi2019-08-011-1/+3
| | | |
| * | | Merge branch 'cross-signing_hidden' into cross-signing_keysHubert Chathi2019-08-011-2/+2
| |\ \ \
| * | | | allow uploading keys for cross-signingHubert Chathi2019-07-251-0/+63
| | | | |
* | | | | Move storage classes into a main "data store".Erik Johnston2019-10-211-4/+4
| |_|_|/ |/| | | | | | | | | | | | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* | | | Fix presence timeouts when synchrotron restarts. (#6212)Erik Johnston2019-10-181-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix presence timeouts when synchrotron restarts. Handling timeouts would fail if there was an external process that had timed out, e.g. a synchrotron restarting. This was due to a couple of variable name typoes. Fixes #3715.
* | | | Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_versionHubert Chathi2019-10-111-16/+31
|\ \ \ \ | | | | | | | | | | make version optional in body of e2e backup version update
| * | | | change test name to be uniqueHubert Chathi2019-10-101-1/+1
| | | | |
| * | | | make version optional in body of e2e backup version updateHubert Chathi2019-10-091-16/+31
| | | | | | | | | | | | | | | | | | | | to agree with latest version of the MSC
* | | | | Fix races in room stats (and other) updates. (#6187)Richard van der Hoff2019-10-101-1/+1
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | Hopefully this will fix the occasional failures we were seeing in the room directory. The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8. We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
* | | | Merge pull request #6147 from matrix-org/babolivier/3pid-invite-revokedBrendan Abolivier2019-10-041-0/+81
|\ \ \ \ | | | | | | | | | | Don't 500 when trying to exchange a revoked 3PID invite
| * | | | Lint (again)Brendan Abolivier2019-10-031-3/+1
| | | | |
| * | | | LintBrendan Abolivier2019-10-031-5/+5
| | | | |
| * | | | Add test caseBrendan Abolivier2019-10-031-0/+83
| | | | |
* | | | | Land improved room list based on room stats (#6019)Erik Johnston2019-10-021-39/+0
|/ / / / | | | | | | | | | | | | Use room_stats and room_state for room directory search
* | | | Add 'failure_ts' column to 'destinations' table (#6016)Richard van der Hoff2019-09-171-1/+6
| | | | | | | | | | | | | | | | Track the time that a server started failing at, for general analysis purposes.
* | | | Only count real users when checking for auto-creation of auto-join roomJason Robinson2019-09-091-2/+27
| |_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously if the first registered user was a "support" or "bot" user, when the first real user registers, the auto-join rooms were not created. Fix to exclude non-real (ie users with a special user type) users when counting how many users there are to determine whether we should auto-create a room. Signed-off-by: Jason Robinson <jasonr@matrix.org>
* | | Fix and refactor room and user stats (#5971)Erik Johnston2019-09-041-111/+532
| | | | | | | | | Previously the stats were not being correctly populated.
* | | Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-1/+1
| |/ |/| | | | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* | Remove non-functional 'expire_access_token' setting (#5782)Richard van der Hoff2019-07-301-1/+1
| | | | | | | | | | | | | | | | The `expire_access_token` didn't do what it sounded like it should do. What it actually did was make Synapse enforce the 'time' caveat on macaroons used as access tokens, but since our access token macaroons never contained such a caveat, it was always a no-op. (The code to add 'time' caveats was removed back in v0.18.5, in #1656)
* | Replace returnValue with return (#5736)Amber Brown2019-07-231-1/+1
|/
* Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_dataErik Johnston2019-07-151-0/+210
|\ | | | | Add basic function to get all data for a user out of synapse
| * Fixup from review comments.Erik Johnston2019-07-041-5/+5
| |
| * Add basic function to get all data for a user out of synapseErik Johnston2019-07-021-0/+210
| |
* | Implement access token expiry (#5660)Richard van der Hoff2019-07-122-6/+19
| | | | | | | | Record how long an access token is valid for, and raise a soft-logout once it expires.
* | Remove access-token support from RegistrationStore.register (#5642)Richard van der Hoff2019-07-102-16/+6
| | | | | | | | | | The 'token' param is no longer used anywhere except the tests, so let's kill that off too.
* | Remove access-token support from RegistrationHandler.register (#5641)Richard van der Hoff2019-07-081-25/+28
| | | | | | | | | | | | | | | | Nothing uses this now, so we can remove the dead code, and clean up the API. Since we're changing the shape of the return value anyway, we take the opportunity to give the method a better name.
* | Move get_or_create_user to test code (#5628)Richard van der Hoff2019-07-081-9/+59
|/ | | This is only used in tests, so...
* Run Black. (#5482)Amber Brown2019-06-207-51/+45
|
* Fix background updates to handle redactions/rejections (#5352)Erik Johnston2019-06-061-3/+59
| | | | | | | | * Fix background updates to handle redactions/rejections In background updates based on current state delta stream we need to handle that we may not have all the events (or at least that `get_events` may raise an exception).
* Room Statistics (#4338)Amber Brown2019-05-211-0/+251
|
* Limit UserIds to a length that fits in a state key (#5198)ReidAnderson2019-05-201-0/+7
|
* Migrate all tests to use the dict-based config format instead of hanging ↵Amber Brown2019-05-132-4/+8
| | | | items off HomeserverConfig (#5171)
* Run Black on the tests again (#5170)Amber Brown2019-05-105-262/+213
|
* Add admin api for sending server_notices (#5121)Richard van der Hoff2019-05-021-2/+2
|
* Move admin api impl to its own packageRichard van der Hoff2019-05-011-3/+4
| | | | It doesn't really belong under rest/client/v1 any more.
* Fix grammar and document get_current_users_in_room (#4998)Andrew Morgan2019-04-031-2/+2
|
* Review commentsErik Johnston2019-03-281-5/+9
|
* Add unit testsErik Johnston2019-03-271-1/+171
|
* Make federation endpoints more tolerant of trailing slashes v2 (#4935)Andrew Morgan2019-03-261-3/+3
| | | Redo of https://github.com/matrix-org/synapse/pull/4840
* Some more porting to HomeserverTestCase and remove old RESTHelper (#4913)Amber Brown2019-03-221-66/+58
|
* Remove trailing slashes from outbound federation requests and retry on 400 ↵Andrew Morgan2019-03-211-0/+2
|\ | | | | | | | | (#4840) As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
| * Switch to wrapper function around _send_requestAndrew Morgan2019-03-131-2/+2
| |
| * Correct argument nameAndrew Morgan2019-03-121-2/+2
| |
| * Cleaner way of implementing trailing slashesAndrew Morgan2019-03-121-2/+4
| |
| * and againAndrew Morgan2019-03-111-1/+1
| |
| * Fix syntax errorAndrew Morgan2019-03-111-1/+1
| |
| * We're calling different functions nowAndrew Morgan2019-03-111-2/+2
| |
* | Merge pull request #4896 from matrix-org/erikj/disable_room_directoryErik Johnston2019-03-211-1/+58
|\ \ | | | | | | Add option to disable search room lists
| * | Add testErik Johnston2019-03-201-1/+58
| | |
* | | Add testErik Johnston2019-03-201-0/+52
|/ /
* | Fix RegistrationTestCaseRichard van der Hoff2019-03-191-2/+9
| | | | | | | | turns out this relies on there being a `user_consent_version` set.
* | fix test_auto_create_auto_join_where_no_consent (#4886)Neil Johnson2019-03-191-2/+22
| |
* | Migrate the user directory initial population to a background task (#4864)Amber Brown2019-03-191-35/+74
| |
* | Revert "Make federation endpoints more tolerant of trailing slashes for some ↵Erik Johnston2019-03-141-3/+3
| | | | | | | | | | | | endpoints (#4793)" This reverts commit 290552fd836f4ae2dc1d893a7f72f7fff85365d3.
* | Merge pull request #4846 from matrix-org/hawkowl/userdir-searchErik Johnston2019-03-141-13/+18
|\ \ | |/ |/| Improve searching in the userdir
| * fixupAmber Brown2019-03-131-20/+1
| |
| * fixupAmber Brown2019-03-121-16/+20
| |
| * fixesAmber Brown2019-03-121-4/+12
| |
| * initialAmber Brown2019-03-111-0/+12
| |
* | Make federation endpoints more tolerant of trailing slashes for some ↵Andrew Morgan2019-03-111-3/+3
|/ | | | | endpoints (#4793) Server side of a solution towards #3622.
* Rewrite userdir to be faster (#4537)Amber Brown2019-03-071-41/+225
|
* Add rate-limiting on registration (#4735)Brendan Abolivier2019-03-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test
* Update test_typing to use HomeserverTestCase. (#4771)Richard van der Hoff2019-03-041-157/+133
|
* Fix registration on workers (#4682)Erik Johnston2019-02-201-1/+1
| | | | | | | | | | * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile
* Add configurable room list publishing rulesErik Johnston2019-02-141-0/+1
| | | | | This allows specifying who and what is allowed to be published onto the public room list