summary refs log tree commit diff
path: root/synapse/server.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Refactor config to be an experimental featureHugh Nimmo-Smith2023-05-301-3/+3
| | | | Also enforce you can't combine it with incompatible config options
* Initial MSC3964 support: delegation of auth to OIDC serverQuentin Gliech2023-05-301-0/+4
|
* Make the api.auth.Auth a ProtocolQuentin Gliech2023-05-301-1/+2
|
* Add Unix socket support for Redis connections (#15644)Jason Little2023-05-261-14/+28
| | | | Adds a new configuration setting to connect to Redis via a Unix socket instead of over TCP. Disabled by default.
* Rename blacklist/whitelist internally. (#15620)Patrick Cloke2023-05-191-5/+5
| | | | Avoid renaming configuration settings for now and rename internal code to use blocklist and allowlist instead.
* Do not allow deactivated users to login with JWT. (#15624)Patrick Cloke2023-05-191-0/+7
| | | | | To improve the organization of this code it moves the JWT login checks to a separate handler and then fixes the bug (and a deprecation warning).
* HTTP Replication Client (#15470)Jason Little2023-05-091-1/+12
| | | | | | Separate out a HTTP client for replication in preparation for also supporting using UNIX sockets. The major difference from the base class is that this does not use treq to handle HTTP requests.
* Factor out an `is_mine_server_name` method (#15542)Sean Quah2023-05-051-0/+4
| | | | | | | | | | | | Add an `is_mine_server_name` method, similar to `is_mine_id`. Ideally we would use this consistently, instead of sometimes comparing against `hs.hostname` and other times reaching into `hs.config.server.server_name`. Also fix a bug in the tests where `hs.hostname` would sometimes differ from `hs.config.server.server_name`. Signed-off-by: Sean Quah <seanq@matrix.org>
* Move ThirdPartyEventRules into module_api/callbacks (#15535)Andrew Morgan2023-05-041-5/+0
|
* Merge branch 'release-v1.83' into developSean Quah2023-05-031-21/+0
|\
| * Revert "Reduce the size of the HTTP connection pool for non-pushers" (#15530) v1.83.0rc1Sean Quah2023-05-031-21/+0
| | | | | | | | | | | | | | | | | | #15514 introduced a regression where Synapse would encounter `PartialDownloadError`s when fetching OpenID metadata for certain providers on startup. Due to #8088, this prevents Synapse from starting entirely. Revert the change while we decide what to do about the regression.
* | Add config option to forget rooms automatically when users leave them (#15224)Sean Quah2023-05-031-1/+10
|/ | | | | This is largely based off the stats and user directory updater code. Signed-off-by: Sean Quah <seanq@matrix.org>
* Reduce the size of the HTTP connection pool for non-pushers. (#15514)Patrick Cloke2023-05-021-0/+21
| | | | | | | | | | | | | | Pushers tend to make many connections to the same HTTP host (e.g. a new event comes in, causes events to be pushed, and then the homeserver connects to the same host many times). Due to this the per-host HTTP connection pool size was increased, but this does not make sense for other SimpleHttpClients. Add a parameter for the connection pool and override it for pushers (making a separate SimpleHttpClient for pushers with the increased configuration). This returns the HTTP connection pool settings to the default Twisted ones for non-pusher HTTP clients.
* Experimental support for MSC3970: per-device transaction IDs (#15318)Quentin Gliech2023-04-251-1/+3
|
* Move Spam Checker callbacks to a dedicated file (#15453)Andrew Morgan2023-04-181-6/+1
|
* Move Account Validity callbacks to a dedicated file (#15237)Andrew Morgan2023-03-161-0/+5
|
* Give PyCharm some help with `@cache_in_self` (#15238)David Robertson2023-03-091-3/+26
| | | | | | | * Give PyCharm some help with `@cache_in_self` * Changelog * Fix import for old python versions
* Stop applying edits to event contents (MSC3925). (#15193)Patrick Cloke2023-03-061-1/+1
| | | | | | | | | | | Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.
* Refactor media modules. (#15146)Patrick Cloke2023-02-271-4/+2
| | | | | | | * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
* Support for selecting the Redis logical database. (#15034)999lakhisidhu2023-02-151-0/+1
| | | | Note that this is only used for key-value store (cached values) and not for the pub/sub replication used by Synapse.
* Add final type hint to synapse.server. (#15035)Patrick Cloke2023-02-091-7/+5
|
* Always notify replication when a stream advances (#14877)Erik Johnston2023-01-201-1/+5
| | | This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.
* Remove unnecessary reactor reference from `_PerHostRatelimiter` (#14842)Sean Quah2023-01-161-1/+0
| | | | | Fix up #14812 to avoid introducing a reference to the reactor. Signed-off-by: Sean Quah <seanq@matrix.org>
* Fix stack overflow in `_PerHostRatelimiter` due to synchronous requests (#14812)Sean Quah2023-01-131-0/+1
| | | | | | | | | | | | | | | | | | When there are many synchronous requests waiting on a `_PerHostRatelimiter`, each request will be started recursively just after the previous request has completed. Under the right conditions, this leads to stack exhaustion. A common way for requests to become synchronous is when the remote client disconnects early, because the homeserver is overloaded and slow to respond. Avoid stack exhaustion under these conditions by deferring subsequent requests until the next reactor tick. Fixes #14480. Signed-off-by: Sean Quah <seanq@matrix.org>
* Implement MSC3925: changes to bundling of edits (#14811)Richard van der Hoff2023-01-101-1/+1
| | | | | | | | Two parts to this: * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag. * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.
* Add a type hint for `get_device_handler()` and fix incorrect types. (#14055)Patrick Cloke2022-11-221-1/+1
| | | | | This was the last untyped handler from the HomeServer object. Since it was being treated as Any (and thus unchecked) it was being used incorrectly in a few places.
* Fix typechecking errors introduced in #14128 (#14455)David Robertson2022-11-151-2/+3
| | | | | | | | | * Fix typechecking errors introduced in #14128 * Changelog * Correct annotations so that context_factory works if you don't use TLS
* Bump flake8-bugbear from 22.9.23 to 22.10.27 (#14329)dependabot[bot]2022-11-041-1/+1
| | | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Fix error in `is_mine_id` when encountering a malformed ID (#13746)Sean Quah2022-09-081-1/+11
| | | | | | | | | Previously, `is_mine_id` would raise an exception when passed an ID with no colons. Return `False` instead. Fixes #13040. Signed-off-by: Sean Quah <seanq@matrix.org>
* Share some metrics between the Prometheus exporter and the phone home stats ↵Brendan Abolivier2022-09-051-0/+6
| | | | (#13671)
* Fix rate limit metrics registering twice and misreporting (#13649)Eric Eastwood2022-08-301-1/+3
| | | | | | | | | | | * Fix rate limit metrics registering twice and misreporting Fix https://github.com/matrix-org/synapse/issues/13641 * Fix lints * Add changelog * Document `metrics_name=None`.
* Move the "email unsubscribe" resource, refactor the macaroon generator & ↵Quentin Gliech2022-06-141-2/+5
| | | | | | | | | | | | | | | | | | | | | simplify the access token verification logic. (#12986) This simplifies the access token verification logic by removing the `rights` parameter which was only ever used for the unsubscribe link in email notifications. The latter has been moved under the `/_synapse` namespace, since it is not a standard API. This also makes the email verification link more secure, by embedding the app_id and pushkey in the macaroon and verifying it. This prevents the user from tampering the query parameters of that unsubscribe link. Macaroon generation is refactored: - Centralised all macaroon generation and verification logic to the `MacaroonGenerator` - Moved to `synapse.utils` - Changed the constructor to require only a `Clock`, hostname, and a secret key (instead of a full `Homeserver`). - Added tests for all methods.
* Decouple `synapse.api.auth_blocking.AuthBlocking` from ↵Quentin Gliech2022-06-141-0/+5
| | | | `synapse.api.auth.Auth`. (#13021)
* Rename storage classes (#12913)Erik Johnston2022-05-311-3/+4
|
* Remove backing code for groups/communities (#12558)Patrick Cloke2022-05-261-38/+1
| | | | Including handlers, configuration code, appservice support, and the GroupID construct.
* SpamChecker metrics (#12513)Jess Porter2022-05-131-1/+1
| | | | | | | | | * add Measure blocks all over SpamChecker Signed-off-by: jesopo <github@lolnerd.net> * fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck * better changelog entry
* Remove unneeded `ActionGenerator` class. (#12691)Patrick Cloke2022-05-111-3/+3
| | | | It simply passes through to `BulkPushRuleEvaluator`, which can be called directly instead.
* Add a module API to allow modules to edit push rule actions (#12406)Brendan Abolivier2022-04-271-0/+5
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove references to unstable identifiers from MSC3440. (#12382)Patrick Cloke2022-04-121-1/+1
| | | | | Removes references to unstable thread relation, unstable identifiers for filtering parameters, and the experimental config flag.
* Add a relations handler to avoid duplication. (#12227)Patrick Cloke2022-03-161-0/+5
| | | Adds a handler layer between the REST and datastore layers for relations.
* Remove unnecessary pass statements. (#12206)Patrick Cloke2022-03-111-1/+0
|
* Support stable identifiers for MSC3440: Threading (#12151)Patrick Cloke2022-03-101-1/+1
| | | | The unstable identifiers are still supported if the experimental configuration flag is enabled. The unstable identifiers will be removed in a future release.
* Rename get_tcp_replication to get_replication_command_handler. (#12192)Patrick Cloke2022-03-101-1/+1
| | | | | | Since the object it returns is a ReplicationCommandHandler. This is clean-up from adding support to Redis where the command handler was added as an additional layer of abstraction from the TCP protocol.
* Fix incorrect type hints for txredis. (#12042)Patrick Cloke2022-03-081-2/+2
| | | | Some properties were marked as RedisProtocol instead of ConnectionHandler, which wraps RedisProtocol instance(s).
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-11/+5
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Implement account status endpoints (MSC3720) (#12001)Brendan Abolivier2022-02-221-0/+5
| | | | | See matrix-org/matrix-doc#3720 Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Adds misc missing type hints (#11953)Patrick Cloke2022-02-111-4/+4
|
* Bundle aggregations outside of the serialization method. (#11612)Patrick Cloke2022-01-071-1/+1
| | | | | | | | This makes the serialization of events synchronous (and it no longer access the database), but we must manually calculate and provide the bundled aggregations. Overall this should cause no change in behavior, but is prep work for other improvements.
* Add MSC3030 experimental client and federation API endpoints to get the ↵Eric Eastwood2021-12-021-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | closest event to a given timestamp (#9445) MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030 Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about. ``` GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Federation API endpoint: ``` GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Co-authored-by: Erik Johnston <erik@matrix.org>
* Add missing type hints to `synapse.app`. (#11287)Patrick Cloke2021-11-101-6/+9
|
* Enable passing typing stream writers as a list. (#11237)Nick Barrett2021-11-031-2/+2
| | | | This makes the typing stream writer config match the other stream writers that only currently support a single worker.
* Add type hints for most `HomeServer` parameters (#11095)Sean Quah2021-10-221-3/+8
|
* Port the Password Auth Providers module interface to the new generic ↵Azrenbeth2021-10-131-1/+5
| | | | | | interface (#10548) Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Refactor MSC2716 `/batch_send` endpoint into separate handler functions (#10974)Eric Eastwood2021-10-081-0/+5
|
* Remove the deprecated BaseHandler. (#11005)Patrick Cloke2021-10-081-1/+10
| | | | | | | | The shared ratelimit function was replaced with a dedicated RequestRatelimiter class (accessible from the HomeServer object). Other properties were copied to each sub-class that inherited from BaseHandler.
* Use direct references for some configuration variables (part 2) (#10812)Patrick Cloke2021-09-151-8/+8
|
* Use direct references for some configuration variables (#10798)Patrick Cloke2021-09-131-11/+13
| | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
* Split `FederationHandler` in half (#10692)Richard van der Hoff2021-08-261-0/+5
| | | The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.
* Experimental support for MSC3266 Room Summary API. (#10394)Michael Telatynski2021-08-161-3/+3
|
* Add a setting to disable TLS for sending email (#10546)Richard van der Hoff2021-08-061-6/+0
| | | This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-3/+3
|
* Standardise the module interface (#10062)Brendan Abolivier2021-06-181-4/+35
| | | This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
* Remove support for ACME v1 (#10194)Brendan Abolivier2021-06-171-5/+0
| | | | | Fixes #9778 ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
* Split multiplart email sending into a dedicated handler (#9977)Brendan Abolivier2021-05-171-0/+5
| | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Remove various bits of compatibility code for Python <3.6 (#9879)Andrew Morgan2021-04-271-5/+0
| | | I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
* Improved validation for received requests (#9817)Richard van der Hoff2021-04-231-0/+8
| | | | | | * Simplify `start_listening` callpath * Correctly check the size of uploaded files
* Split presence out of master (#9820)Erik Johnston2021-04-231-3/+3
|
* Check for space membership during a remote join of a restricted room (#9814)Patrick Cloke2021-04-231-0/+5
| | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* Rename handler and config modules which end in handler/config. (#9816)Patrick Cloke2021-04-201-5/+5
|
* remove `HomeServer.get_config` (#9815)Richard van der Hoff2021-04-141-3/+0
| | | | Every single time I want to access the config object, I have to remember whether or not we use `get_config`. Let's just get rid of it.
* Revert "Check for space membership during a remote join of a restricted ↵Patrick Cloke2021-04-141-5/+0
| | | | | | | | room. (#9763)" This reverts commit cc51aaaa7adb0ec2235e027b5184ebda9b660ec4. The PR was prematurely merged and not yet approved.
* Check for space membership during a remote join of a restricted room. (#9763)Patrick Cloke2021-04-141-0/+5
| | | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* Move some replication processing out of generic_worker (#9796)Erik Johnston2021-04-141-3/+10
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Add a Synapse Module for configuring presence update routing (#9491)Andrew Morgan2021-04-061-0/+5
| | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.
* Make RateLimiter class check for ratelimit overrides (#9711)Erik Johnston2021-03-301-0/+1
| | | | | | | This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited. We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits. Fixes #9663
* Add type hints for the federation sender. (#9681)Patrick Cloke2021-03-291-2/+2
| | | | Includes an abstract base class which both the FederationSender and the FederationRemoteSendQueue must implement.
* Add a type hints for service notices to the HomeServer object. (#9675)Patrick Cloke2021-03-241-2/+2
|
* Initial spaces summary API (#9643)Richard van der Hoff2021-03-181-0/+5
| | | This is very bare-bones for now: federation will come soon, while pagination is descoped for now but will come later.
* Fix up types for the typing handler. (#9638)Patrick Cloke2021-03-171-1/+10
| | | | By splitting this to two separate methods the callers know what methods they can expect on the handler.
* Add type hints to the room member handler. (#9631)Patrick Cloke2021-03-171-2/+2
|
* Fix additional type hints from Twisted 21.2.0. (#9591)Patrick Cloke2021-03-121-5/+3
|
* Create a SynapseReactor type which incorporates the necessary reactor ↵Patrick Cloke2021-03-081-3/+2
| | | | | interfaces. (#9528) This helps fix some type hints when running with Twisted 21.2.0.
* Fix additional type hints from Twisted upgrade. (#9518)Patrick Cloke2021-03-031-1/+2
|
* Add support for no_proxy and case insensitive env variables (#9372)Tim Leung2021-02-261-8/+2
| | | | | | | | | | | | | | | ### Changes proposed in this PR - Add support for the `no_proxy` and `NO_PROXY` environment variables - Internally rely on urllib's [`proxy_bypass_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2519) - Extract env variables using urllib's `getproxies`/[`getproxies_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2488) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment This does contain behaviour changes for consumers so making sure these are called out: - `no_proxy`/`NO_PROXY` is now respected - lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY` Related to #9306 which also uses `ProxyAgent` Signed-off-by: Timothy Leung tim95@hotmail.co.uk
* Clean up `ShardedWorkerHandlingConfig` (#9466)Erik Johnston2021-02-241-5/+2
| | | | | | | | | | | | | | | | | * Split ShardedWorkerHandlingConfig This is so that we have a type level understanding of when it is safe to call `get_instance(..)` (as opposed to `should_handle(..)`). * Remove special cases in ShardedWorkerHandlingConfig. `ShardedWorkerHandlingConfig` tried to handle the various different ways it was possible to configure federation senders and pushers. This led to special cases that weren't hit during testing. To fix this the handling of the different cases is moved from there and `generic_worker` into the worker config class. This allows us to have the logic in one place and allows the rest of the code to ignore the different cases.
* Fix deleting pushers when using sharded pushers. (#9465)Erik Johnston2021-02-221-3/+0
|
* Type hints and validation improvements. (#9321)Patrick Cloke2021-02-081-2/+14
| | | | | * Adds type hints to the groups servlet and stringutils code. * Assert the maximum length of some input values for spec compliance.
* Precompute joined hosts and store in Redis (#9198)Erik Johnston2021-01-261-0/+30
|
* Allow moving account data and receipts streams off master (#9104)Erik Johnston2021-01-181-0/+5
|
* Remove user's avatar URL and displayname when deactivated. (#8932)Dirk Klimpel2021-01-121-1/+1
| | | This only applies if the user's data is to be erased.
* Kill off `HomeServer.get_ip_from_request()` (#9080)Richard van der Hoff2021-01-121-4/+0
| | | Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-1/+1
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Default to blacklisting reserved IP ranges and add a whitelist. (#8870)Patrick Cloke2020-12-091-1/+2
| | | | This defaults `ip_range_blacklist` to reserved IP ranges and also adds an `ip_range_whitelist` setting to override it.
* Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-021-7/+29
| | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* Simplify the way the `HomeServer` object caches its internal attributes. ↵Jonathan de Jong2020-11-301-14/+13
| | | | | (#8565) Changes `@cache_in_self` to use underscore-prefixed attributes.
* Add type hints to matrix federation client / agent. (#8806)Patrick Cloke2020-11-251-1/+2
|
* Abstract shared SSO code. (#8765)Patrick Cloke2020-11-171-0/+5
| | | De-duplicates code between the SAML and OIDC implementations.
* Solidify the HomeServer constructor. (#8515)Jonathan de Jong2020-10-151-5/+9
| | | | | | This implements a more standard API for instantiating a homeserver and moves some of the dependency injection into the test suite. More concretely this stops using `setattr` on all `kwargs` passed to `HomeServer`.
* Move additional tasks to the background worker, part 4 (#8513)Patrick Cloke2020-10-131-5/+7
|
* Remove the deprecated Handlers object (#8494)Patrick Cloke2020-10-091-5/+25
| | | All handlers now available via get_*_handler() methods on the HomeServer.
* Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464)Richard van der Hoff2020-10-071-0/+5
| | | | | Lots of different module apis is not easy to maintain. Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
* Allow background tasks to be run on a separate worker. (#8369)Patrick Cloke2020-10-021-4/+13
|
* Fix ratelimiting for federation `/send` requests. (#8342)Erik Johnston2020-09-181-0/+5
| | | c.f. #8295 for rationale
* Add comment explaining castErik Johnston2020-08-111-0/+2
|
* Handle optional dependencies for Oidc and SamlErik Johnston2020-08-111-5/+9
|
* Change HomeServer definition to work with typing.Erik Johnston2020-08-111-224/+231
| | | | | | | | | | Duplicating function signatures between server.py and server.pyi is silly. This commit changes that by changing all `build_*` methods to `get_*` methods and changing the `_make_dependency_method` to work work as a descriptor that caches the produced value. There are some changes in other files that were made to fix the typing in server.py.
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-2/+2
|
* Allow moving typing off master (#7869)Erik Johnston2020-07-161-7/+6
|
* Add some tiny type annotations (#7870)Richard van der Hoff2020-07-161-2/+2
| | | I found these made pycharm have more of a clue as to what was going on in other places.
* Add delete room admin endpoint (#7613)Dirk Klimpel2020-07-141-1/+9
| | | | | | | | | | | | | | | | | | The Delete Room admin API allows server admins to remove rooms from server and block these rooms. `DELETE /_synapse/admin/v1/rooms/<room_id>` It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API. Fixes: #6425 It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`. It should return `None` if the room is unknown. But it returns an `IndexError`. https://github.com/matrix-org/synapse/blob/901b1fa561e3cc661d78aa96d59802cf2078cb0d/synapse/storage/data_stores/main/room.py#L99-L105 Related to: - #5575 - https://github.com/Awesome-Technologies/synapse-admin/issues/17 Signed-off-by: Dirk Klimpel dirk@klimpel.org
* Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-0/+2
| | | ... instead of duplicating `config.signing_key[0]` everywhere
* Performance improvements and refactor of Ratelimiter (#7595)Andrew Morgan2020-06-051-10/+7
| | | | | | | | | | While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both: * Rather undocumented, and * causing a *lot* of config checks This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. Best to be reviewed commit-by-commit.
* Add ability to wait for replication streams (#7542)Erik Johnston2020-05-221-0/+5
| | | | | | | The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room). Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on. People probably want to look at this commit by commit.
* Move EventStream handling into default ReplicationDataHandler (#7493)Erik Johnston2020-05-141-1/+1
| | | This is so that the logic can happen on both master and workers when we move event persistence out.
* Implement OpenID Connect-based login (#7256)Quentin Gliech2020-05-081-0/+6
|
* Add instance name to RDATA/POSITION commands (#7364)Erik Johnston2020-04-291-2/+11
| | | | | This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
* Move client command handling out of TCP protocol (#7185)Erik Johnston2020-04-061-1/+7
| | | The aim here is to move the command handling out of the TCP protocol classes and to also merge the client and server command handling (so that we can reuse them for redis protocol). This PR simply moves the client paths to the new `ReplicationCommandHandler`, a future PR will move the server paths too.
* Fix a bug which could cause incorrect 'cyclic dependency' error. (#7178)Richard van der Hoff2020-03-311-12/+10
| | | | | | | | If there was an exception setting up one of the attributes of the Homeserver god object, then future attempts to fetch that attribute would raise a confusing "Cyclic dependency" error. Let's make sure that we clear the `building` flag so that we just get the original exception. Ref: #7169
* Remove usage of "conn_id" for presence. (#7128)Erik Johnston2020-03-301-0/+11
| | | | | | | | | | | | | | | | * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.
* Refactor the CAS code (move the logic out of the REST layer to a handler) ↵Patrick Cloke2020-03-261-0/+5
| | | | (#7136)
* Allow server admins to define and enforce a password policy (MSC2000). (#7118)Dirk Klimpel2020-03-261-0/+5
|
* Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-0/+5
| | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* Share SSL contexts for non-federation requests (#7094)Richard van der Hoff2020-03-171-3/+3
| | | | | | | Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests. Fixes #7092.
* Allow moving group read APIs to workers (#6866)Erik Johnston2020-02-071-4/+10
|
* Add database config class (#6513)Erik Johnston2019-12-181-37/+4
| | | | | This encapsulates config for a given database and is the way to get new connections.
* Add new config param to docstring and add typesErik Johnston2019-12-111-1/+3
|
* Remove database config parsing from apps.Erik Johnston2019-12-101-1/+9
|
* Pass Database into the data storeErik Johnston2019-12-061-2/+1
|
* Fix startup error when http proxy is defined. (#6421)Richard van der Hoff2019-11-261-2/+2
| | | | | Guess I only tested this on python 2 :/ Fixes #6419.
* Remove the psutil dependency (#6318)Amber Brown2019-11-051-0/+2
| | | * remove psutil and replace with resource
* Support for routing outbound HTTP requests via a proxy (#6239)Richard van der Hoff2019-11-011-0/+9
| | | | | | | | | | | | | | | | | | | | The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy. The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`. The proxy will then be used for * push * url previews * phone-home stats * recaptcha validation * CAS auth validation It will *not* be used for: * Application Services * Identity servers * Outbound federation * In worker configurations, connections from workers to masters Fixes #4198.
* Review commentsErik Johnston2019-10-301-3/+2
|
* Use new EventPersistenceStoreErik Johnston2019-10-231-1/+8
|
* Allow use of different ratelimits for admin redactions.Erik Johnston2019-09-111-0/+4
| | | | | This is useful to allow room admins to quickly deal with a large number of abusive messages.
* cleanupsRichard van der Hoff2019-06-271-4/+4
|
* Move all the saml stuff out to a centralised handlerRichard van der Hoff2019-06-261-6/+6
|
* Merge branch 'develop' into rav/saml2_clientRichard van der Hoff2019-06-261-81/+83
|\
| * Run Black. (#5482)Amber Brown2019-06-201-81/+75
| |
| * Add plugin APIs for implementations of custom event rules.Brendan Abolivier2019-06-141-0/+7
| |
* | Code cleanups and simplifications.Richard van der Hoff2019-06-111-0/+5
|/ | | | Also: share the saml client between redirect and response handlers.
* Room Statistics (#4338)Amber Brown2019-05-211-0/+6
|
* Allow client event serialization to be asyncErik Johnston2019-05-141-0/+5
|
* Send out emails with links to extend an account's validity periodBrendan Abolivier2019-04-171-0/+5
|
* Merge pull request #4855 from matrix-org/rav/refactor_transaction_queueRichard van der Hoff2019-03-151-2/+2
|\ | | | | Split TransactionQueue up
| * Rename and move the classesRichard van der Hoff2019-03-131-2/+2
| |
* | fixupAmber Brown2019-03-131-0/+5
| |
* | master startupAmber Brown2019-03-111-0/+8
|/
* Make registration ratelimiter separate from the main events oneBrendan Abolivier2019-03-061-0/+4
|
* Revert "Split ratelimiters in two (one for events, one for registration)"Brendan Abolivier2019-03-061-7/+3
| | | | This reverts commit d7dbad3526136cfc9fdbd568635be5016fb637db.
* Split ratelimiters in two (one for events, one for registration)Brendan Abolivier2019-03-051-3/+7
|
* Split DeviceHandler into master and workerErik Johnston2019-03-041-2/+5
|
* Fix registration on workers (#4682)Erik Johnston2019-02-201-0/+5
| | | | | | | | | | * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile
* Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)Richard van der Hoff2019-02-111-1/+5
| | | | | It's nothing to do with refreshing the certificates. No idea why it was here.
* ACME Reprovisioning (#4522)Amber Brown2019-02-111-0/+3
|
* Refactor event building into EventBuilderErik Johnston2019-01-291-4/+1
| | | | | This is so that everything is done in one place, making it easier to change the event format based on room version
* Support ACME for certificate provisioning (#4384)Amber Brown2019-01-231-0/+5
|
* Write some tests for the email pusher (#4095)Amber Brown2018-10-301-0/+5
|
* commit transaction before closingNeil Johnson2018-10-241-0/+1
|
* Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backupsDavid Baker2018-10-091-2/+12
|\
| * use abc.abstractpropertyRichard van der Hoff2018-08-281-5/+4
| | | | | | | | This gives clearer messages when someone gets it wrong
| * Make sure that we close db connections opened during initRichard van der Hoff2018-08-281-2/+13
| | | | | | | | | | | | | | | | | | We should explicitly close any db connections we open, because failing to do so can block other transactions as per https://github.com/matrix-org/synapse/issues/3682. Let's also try to factor out some of the boilerplate by having server classes define their datastore class rather than duplicating the whole of `setup`.
* | Merge branch 'develop' into e2e_backupsHubert Chathi2018-08-241-3/+10
|\|
| * Rename WorkerProfileHandler to BaseProfileHandlerErik Johnston2018-08-221-2/+2
| |
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-171-1/+5
| |\ | | | | | | | | | erikj/split_profiles
| | * Add EDU/query handling over replicationErik Johnston2018-08-061-1/+5
| | |
| * | Split ProfileHandler into master and workerErik Johnston2018-08-171-2/+5
| |/
* / make it work and fix pep8Matthew Hodgson2018-08-121-0/+5
|/
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-231-0/+5
|\ | | | | | | erikj/client_apis_move
| * Admin API for creating new users (#3415)Amber Brown2018-07-201-0/+5
| |
* | Move PaginationHandler to its own fileErik Johnston2018-07-201-5/+2
| |
* | Move RoomContextHandler out of HandlersErik Johnston2018-07-181-1/+5
| | | | | | | | This is in preparation for moving GET /context/ to a worker
* | Split MessageHandler into read only and writersErik Johnston2018-07-181-1/+13
|/ | | | | This will let us call the read only parts from workers, and so be able to move some APIs off of master, e.g. the `/state` API.
* run isortAmber Brown2018-07-091-17/+17
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-4/+15
|
* Stub out ServerNoticesSender on the workersRichard van der Hoff2018-05-221-0/+7
| | | | | ... and have the sync endpoints call it directly rather than obsure indirection via PresenceHandler
* Send users a server notice about consentRichard van der Hoff2018-05-221-0/+5
| | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.
* Infrastructure for a server notices roomRichard van der Hoff2018-05-171-0/+5
| | | | | | | Server Notices use a special room which the user can't dismiss. They are created on demand when some other bit of the code calls send_notice. (This doesn't actually do much yet becuse we don't call send_notice anywhere)
* Move RoomCreationHandler out of synapse.handlers.HandlersRichard van der Hoff2018-05-171-0/+5
| | | | | | | Handlers is deprecated nowadays, so let's move this out before I add a new dependency on it. Also fix the docstrings on create_room.
* ConsentResource to gather policy consent from usersRichard van der Hoff2018-05-151-0/+3
| | | | | Hopefully there are enough comments and docs in this that it makes sense on its own.
* Burminate v1authAdrian Tschira2018-04-301-10/+0
| | | | | | | | | | | | | | | | | | This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com>
* Split RoomMemberWorkerHandler to separate fileErik Johnston2018-03-141-3/+2
|
* Implement RoomMemberWorkerHandlerErik Johnston2018-03-131-2/+4
|
* Merge pull request #2987 from matrix-org/erikj/split_room_member_handlerErik Johnston2018-03-131-2/+4
|\ | | | | Split RoomMemberHandler into base and master class
| * Raise, don't return, exceptionErik Johnston2018-03-131-1/+1
| |
| * Split RoomMemberHandler into base and master classErik Johnston2018-03-131-2/+4
| | | | | | | | | | | | | | | | The intention here is to split the class into the bits that can be done on workers and the bits that have to be done on the master. In future there will also be a class that can be run on the worker, which will delegate work to the master when necessary.
* | s/replication_client/federation_client/Erik Johnston2018-03-131-2/+2
| |
* | s/replication_server/federation_serverErik Johnston2018-03-131-2/+2
| |
* | Split replication layer into twoErik Johnston2018-03-131-4/+9
|/
* Split out edu/query registration to a separate classErik Johnston2018-03-131-0/+5
|
* Move RoomMemberHandler out of HandlersErik Johnston2018-03-011-0/+5
|
* Update places where we create eventsErik Johnston2018-02-051-0/+5
|
* Factor out resolve_state_groups to a separate handlerRichard van der Hoff2018-02-011-1/+5
| | | | | | We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.
* Factor out get_db_conn to HomeServer base classRichard van der Hoff2018-01-261-0/+17
| | | | | This function is identical to all subclasses, so we may as well push it up to the base class to reduce duplication (and make use of it in the tests)
* Merge branch 'develop' into matthew/search-all-local-usersMatthew Hodgson2017-11-301-17/+35
|\
| * Move set_password into its own handlerRichard van der Hoff2017-11-291-0/+5
| | | | | | | | | | | | Non-functional refactoring to move set_password. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * Move deactivate_account into its own handlerRichard van der Hoff2017-11-291-0/+5
| | | | | | | | | | | | Non-functional refactoring to move deactivate_account. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * Build MediaRepositoryResource as a homeserver dependencyRichard van der Hoff2017-11-221-1/+10
| | | | | | | | | | | | This avoids the scenario where we have four different PreviewUrlResources configured on a single app, each of which have their own caches and cache clearing jobs.
| * Clean up dependency listRichard van der Hoff2017-11-211-16/+15
| | | | | | | | | | remove those that aren't used at all, and replace the ones that don't have builders with simple getters rather than dynamically-generated methods.
* | Add user_directory_include_pattern config param to expand search results to ↵Matthew Hodgson2017-11-291-2/+2
|/ | | | | | | | | | additional users Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive. It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users... ...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too. Committing it here as a WIP
* Merge branch 'develop' into erikj/groups_mergedDavid Baker2017-10-021-0/+5
|\
| * Make the spam checker a moduleDavid Baker2017-09-261-0/+5
| |
* | Split out profile handler to fix testsErik Johnston2017-08-251-0/+5
| |
* | Add local group server supportErik Johnston2017-07-171-0/+5
| |
* | Initial group server implementationErik Johnston2017-07-101-0/+14
|/
* Add user_directory to databaseErik Johnston2017-05-311-0/+5
|
* Store ActionGenerator in HomeServerErik Johnston2017-05-191-0/+5
|
* Finish implementing RM endpointLuke Barnard2017-04-111-0/+5
| | | | | - This change causes a 405 to be sent if "m.read_marker" is set via /account_data - This also fixes-up the RM endpoint so that it actually Works.
* Update all the workers and master to use TCP replicationErik Johnston2017-04-031-0/+4
|
* Fix email push in pusher workerErik Johnston2017-02-021-1/+5
| | | | | | This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.
* Shuffle receipt handler around so that worker apps don't need to load itErik Johnston2016-11-231-0/+5
|
* Ensure only main or federation_sender process can send federation trafficErik Johnston2016-11-231-2/+11
|
* Hook up the send queue and create a federation sender workerErik Johnston2016-11-161-1/+5
|
* Add transaction queue and transport layer to DIErik Johnston2016-11-161-0/+10
|
* Support /initialSync in synchrotron workerErik Johnston2016-09-211-0/+5
|
* Move RoomListHandler into a separate fileErik Johnston2016-09-141-1/+1
|
* Send device messages over federationMark Haines2016-09-061-0/+5
|
* Make synchrotron accept /eventsErik Johnston2016-08-121-0/+9
|
* Move e2e query logic into a handlerRichard van der Hoff2016-08-011-21/+24
|
* Add device_id support to /loginRichard van der Hoff2016-07-181-0/+5
| | | | | | | | | | | | | Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.
* Implement purge_media_cache admin APIErik Johnston2016-06-291-0/+5
|
* Split out the auth handlerDavid Baker2016-06-021-0/+5
|
* Move the AS handler out of the Handlers object.Mark Haines2016-05-311-0/+15
| | | | | | Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.
* Split out the room list handlerDavid Baker2016-05-311-0/+5
| | | | So I can use it from federation bits without pulling in all the handlers.
* Move typing handler out of the Handlers objectMark Haines2016-05-171-0/+5
|
* Move SyncHandler out of the Handlers objectMark Haines2016-05-161-0/+5
|
* Move the presence handler out of the Handlers objectMark Haines2016-05-161-0/+5
|
* Optionally split out the pushers into a separate processMark Haines2016-04-211-0/+3
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-1/+1
|
* Clean up a bit. Add commentErik Johnston2016-01-281-2/+2
|
* PEP 8Erik Johnston2016-01-271-1/+0
|
* Add a Homeserver.setup method.Erik Johnston2016-01-261-15/+17
| | | | | | This is for setting up dependencies that require work on startup. This is useful for the DataStore that wants to read a bunch from the database before initiliazing.
* Remove redundated BaseHomeServerErik Johnston2016-01-261-49/+57
|
* Use split rather than endswithErik Johnston2016-01-191-2/+1
|
* Add regex cache. Only caculate push actions for users that have sent read ↵Erik Johnston2016-01-191-0/+4
| | | | receipts, and are on that server
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Actually host r0 and unstable prefixesDaniel Wagner-Hall2015-12-081-2/+1
|
* Remove the LockManager class because it wasn't being usedMark Haines2015-11-041-5/+0
|
* Use shorter config key nameDaniel Wagner-Hall2015-09-151-1/+2
|
* Rename context factoryDaniel Wagner-Hall2015-09-151-2/+2
| | | | Mjark is officially no fun.
* Allow configuration to ignore invalid SSL certsDaniel Wagner-Hall2015-09-091-0/+14
| | | | | This will be useful for sytest, and sytest only, hence the aggressive config key name.
* Correctly handle x_forwaded listener optionErik Johnston2015-06-121-10/+2
|
* Manually generate the default config yaml, remove most of the commandline ↵Mark Haines2015-04-301-1/+0
| | | | arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class
* Merge branch 'develop' into key_distributionMark Haines2015-04-291-0/+10
|\ | | | | | | | | Conflicts: synapse/config/homeserver.py
| * Merge branch 'develop' into csauthDavid Baker2015-04-171-1/+0
| |\
| * | 1) Pushers are now associated with an access tokenDavid Baker2015-03-241-0/+10
| | | | | | | | | | | | 2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
* | | Add a version 2 of the key server apiMark Haines2015-04-141-0/+1
| |/ |/|
* | Remove appservice REST servletsKegan Dougal2015-03-311-1/+0
|/
* Build MetricsResource as a specific HomeServer dependencyPaul "LeoNerd" Evans2015-03-121-0/+1
|
* Expose 'config' as a real HomeServer dependency keyPaul "LeoNerd" Evans2015-03-121-0/+1
|