summary refs log tree commit diff
path: root/synapse/config (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Revert "Allow for the configuration of max request retries and min/max retry ↵Mathieu Velten2023-06-141-10/+0
| | | | | | delays in the matrix federation client (#12504)" This reverts commit d84e66144dc12dacf71c987a2ba802dd59c0b68e.
* Allow for the configuration of max request retries and min/max retry delays ↵Shay2023-06-091-0/+10
| | | | | | | in the matrix federation client (#12504) Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org>
* Stabilize support for MSC3952: Intentional mentions. (#15520)Patrick Cloke2023-06-061-5/+0
|
* Implement stable support for MSC3882 to allow an existing device/session to ↵Hugh Nimmo-Smith2023-06-012-10/+13
| | | | | | | | generate a login token for use on a new device/session (#15388) Implements stable support for MSC3882; this involves updating Synapse's support to match the MSC / the spec says. Continue to support the unstable version to allow clients to transition.
* Make the config tests spawn the homeserver only when neededQuentin Gliech2023-05-301-12/+28
|
* Add an admin token for MAS -> Synapse callsQuentin Gliech2023-05-301-0/+9
|
* Refactor config to be an experimental featureHugh Nimmo-Smith2023-05-302-33/+199
| | | | Also enforce you can't combine it with incompatible config options
* Initial MSC3964 support: delegation of auth to OIDC serverQuentin Gliech2023-05-301-1/+29
|
* Add Unix socket support for Redis connections (#15644)Jason Little2023-05-261-0/+1
| | | | Adds a new configuration setting to connect to Redis via a Unix socket instead of over TCP. Disabled by default.
* Merge branch 'master' into developOlivier Wilkinson (reivilibre)2023-05-261-1/+1
|\
| * Fix a bug introduced in Synapse v1.84.0 where workers do not start up when ↵reivilibre2023-05-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | no `instance_map` was provided. (#15672) * Fix #15669: always populate instance map even if it was empty * Fix some tests * Fix more tests * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * CI fix: don't forget to update apt repository sources before installing olddeps deps * Add test testing the backwards compatibility --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* | Remove outdated comment in log config (#15648)Andrew Morgan2023-05-221-3/+1
| |
* | Rename blacklist/whitelist internally. (#15620)Patrick Cloke2023-05-192-16/+16
| | | | | | | | Avoid renaming configuration settings for now and rename internal code to use blocklist and allowlist instead.
* | Remove experimental configuration flags & unstable values for faster joins ↵Patrick Cloke2023-05-191-12/+0
| | | | | | | | | | | | | | (#15625) Synapse will no longer send (or respond to) the unstable flags for faster joins. These were only available behind a configuration flag and handled in parallel with the stable flags.
* | Re-type config paths in `ConfigError`s to be `StrSequence`s (#15615)Sean Quah2023-05-185-11/+13
| | | | | | | | | | | | Part of #14809. Signed-off-by: Sean Quah <seanq@matrix.org>
* | Fix error message when `app_service_config_files` validation fails (#15614)Sean Quah2023-05-181-2/+1
|/ | | | | | | | | | The second argument of `ConfigError` is a path, passed as an optional `Iterable[str]` and not a `str`. If a string is passed directly, Synapse unhelpfully emits "Error in configuration at a.p.p._.s.e.r.v.i.c.e._.c.o.n.f.i.g._.f.i.l.e.s'" when the config option has the wrong data type. Signed-off-by: Sean Quah <seanq@matrix.org>
* Add redis SSL configuration options (#15312)Roel ter Maat2023-05-111-0/+6
| | | | | | | | | | | | | | | | | * Add SSL options to redis config * fix lint issues * Add documentation and changelog file * add missing . at the end of the changelog * Move client context factory to new file * Rename ssl to tls and fix typo * fix lint issues * Added when redis attributes were added
* Remove `worker_replication_*` settings (#15491)Jason Little2023-05-111-17/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add master to the instance_map as part of Complement, have ReplicationEndpoint look at instance_map for master. * Fix typo in drive by. * Remove unnecessary worker_replication_* bits from unit tests and add master to instance_map(hopefully in the right place) * Several updates: 1. Switch from master to main for naming the main process in the instance_map. Add useful constants for easier adjustment of names in the future. 2. Add backwards compatibility for worker_replication_* to allow time to transition to new style. Make sure to prioritize declaring main directly on the instance_map. 3. Clean up old comments/commented out code. 4. Adjust unit tests to match with new code. 5. Adjust Complement setup infrastructure to only add main to the instance_map if workers are used and remove now unused options from the worker.yaml template. * Initial Docs upload * Changelog * Missed some commented out code that can go now * Remove TODO comment that no longer holds true. * Fix links in docs * More docs * Remove debug logging * Apply suggestions from code review Co-authored-by: reivilibre <olivier@librepush.net> * Apply suggestions from code review Co-authored-by: reivilibre <olivier@librepush.net> * Update version to latest, include completeish before/after examples in upgrade notes. * Fix up and docs too --------- Co-authored-by: reivilibre <olivier@librepush.net>
* Stabilize MSC2659 support for AS ping endpoint. (#15528)Tulir Asokan2023-05-091-3/+0
|
* Add config option to prevent media downloads from listed domains. (#15197)Travis Ralston2023-05-091-0/+4
| | | | | | | This stops media (and thumbnails) from being accessed from the listed domains. It does not delete any already locally cached media, but will prevent accessing it. Note that admin APIs are unaffected by this change.
* Error if attempting to set m.push_rules account data, per MSC4010. (#15555)Patrick Cloke2023-05-091-0/+5
| | | | | m.push_rules, like m.fully_read, is a special account data type that cannot be set using the normal /account_data endpoint. Return an error instead of allowing data that will not be used to be stored.
* Implement MSC4009 to widen the allowed Matrix ID grammar (#15536)Patrick Cloke2023-05-051-0/+3
| | | | | Behind a configuration flag this adds + to the list of allowed characters in Matrix IDs. The main feature this enables is using full E.164 phone numbers as Matrix IDs.
* Add config option to forget rooms automatically when users leave them (#15224)Sean Quah2023-05-031-0/+4
| | | | | This is largely based off the stats and user directory updater code. Signed-off-by: Sean Quah <seanq@matrix.org>
* Allow adding random delay to push (#15516)Erik Johnston2023-05-021-2/+8
| | | This is to discourage timing based profiling on the push gateways.
* Initial implementation of MSC3981: recursive relations API (#15315)Patrick Cloke2023-05-021-0/+5
| | | | | | | | | | | Adds an optional keyword argument to the /relations API which will recurse a limited number of event relationships. This will cause the API to return not just the events related to the parent event, but also events related to those related to the parent event, etc. This is disabled by default behind an experimental configuration flag and is currently implemented using prefixed parameters.
* Experimental support for MSC3970: per-device transaction IDs (#15318)Quentin Gliech2023-04-251-0/+3
|
* Switch `InstanceLocationConfig` to a pydantic `BaseModel` (#15431)Jason Little2023-04-172-10/+70
| | | | | * Switch InstanceLocationConfig to a pydantic BaseModel, apply Strict* types and add a few helper methods(that will make more sense in follow up work). Co-authored-by: David Robertson <davidr@element.io>
* Throw if the appservice config list is the wrong type (#15425)Will Hunt2023-04-121-4/+10
| | | | | | | | | | | * raise a ConfigError on an invalid app_service_config_files * changelog * Move config check to read_config * Add test * Ensure list also contains strings
* Experimental Unix socket support (#15353)Jason Little2023-04-032-33/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add IReactorUNIX to ISynapseReactor type hint. * Create listen_unix(). Two options, 'path' to the file and 'mode' of permissions(not umask, recommend 666 as default as nginx/other reverse proxies write to it and it's setup as user www-data) For the moment, leave the option to always create a PID lockfile turned on by default * Create UnixListenerConfig and wire it up. Rename ListenerConfig to TCPListenerConfig, then Union them together into ListenerConfig. This spidered around a bit, but I think I got it all. Metrics and manhole have been placed behind a conditional in case of accidental putting them onto a unix socket. Use new helpers to get if a listener is configured for TLS, and to help create a site tag for logging. There are 2 TODO things in parse_listener_def() to finish up at a later point. * Refactor SynapseRequest to handle logging correctly when using a unix socket. This prevents an exception when an IP address can not be retrieved for a request. * Make the 'Synapse now listening on Unix socket' log line a little prettier. * No silent failures on generic workers when trying to use a unix socket with metrics or manhole. * Inline variables in app/_base.py * Update docstring for listen_unix() to remove reference to a hardcoded permission of 0o666 and add a few comments saying where the default IS declared. * Disallow both a unix socket and a ip/port combo on the same listener resource * Linting * Changelog * review: simplify how listen_unix returns(and get rid of a type: ignore) * review: fix typo from ConfigError in app/homeserver.py * review: roll conditional for http_options.tag into get_site_tag() helper(and add docstring) * review: enhance the conditionals for checking if a port or path is valid, remove a TODO line * review: Try updating comment in get_client_ip_if_available to clarify what is being retrieved and why * Pretty up how 'Synapse now listening on Unix Socket' looks by decoding the byte string. * review: In parse_listener_def(), raise ConfigError if neither socket_path nor port is declared(and fix a typo)
* Implement MSC3984 to proxy /keys/query requests to appservices. (#15321)Patrick Cloke2023-03-301-0/+5
| | | | | If enabled, for users which are exclusively owned by an application service then the appservice will be queried for devices in addition to any information stored in the Synapse database.
* Add the ability to enable/disable registrations when in the OIDC flow (#14978)Warren Bailey2023-03-301-0/+5
| | | Signed-off-by: Warren Bailey <warren@warrenbailey.net>
* Implement MSC3983 to proxy /keys/claim queries to appservices. (#15314)Patrick Cloke2023-03-281-0/+5
| | | | | | Experimental support for MSC3983 is behind a configuration flag. If enabled, for users which are exclusively owned by an application service then the appservice will be queried for one-time keys *if* there are none uploaded to Synapse.
* Implement MSC2659: application service ping endpoint (#15249)Tulir Asokan2023-03-161-0/+3
| | | Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Stabilize support for MSC3873: disambuguated event push keys. (#15190)Patrick Cloke2023-03-071-10/+0
| | | | | | | This removes the experimental configuration option and always escapes the push rule condition keys. Also escapes any (experimental) push rule condition keys in the base rules which contain dot in a field name.
* Stabilize support for MSC3966: event_property_contains push condition. (#15187)Patrick Cloke2023-03-071-8/+2
| | | | This removes the configuration flag & updates the identifiers to use the stable version.
* Stop applying edits to event contents (MSC3925). (#15193)Patrick Cloke2023-03-061-3/+0
| | | | | | | | | | | Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.
* Stabilize support for MSC3758: event_property_is push condition (#15185)Patrick Cloke2023-03-061-7/+1
| | | | This removes the configuration flag & updates the identifiers to use the stable version.
* Update intentional mentions (MSC3952) to depend on ↵Patrick Cloke2023-03-021-1/+7
| | | | | | | `exact_event_property_contains` (MSC3966). (#15051) This replaces the specific `is_user_mention` push rule condition used in MSC3952 with the generic `exact_event_property_contains` push rule condition from MSC3966.
* Implementation of MSC3967: Don't require UIA for initial upload of cross ↵Hugh Nimmo-Smith2023-03-021-0/+3
| | | | signing keys (#15077)
* Refactor media modules. (#15146)Patrick Cloke2023-02-271-5/+7
| | | | | | | * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
* Fix a typo in MSC3873 config option. (#15138)Patrick Cloke2023-02-231-2/+2
| | | | Previously the experimental configuration option referred to the wrong MSC number.
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-227-7/+0
|
* Tighten the default rate limit of creating new devices. (#15135)Patrick Cloke2023-02-221-2/+11
|
* Allow health listener resource to load (#15096)realtyem2023-02-201-0/+1
| | | | | | | * Allow health listener resource to load. * changelog * Update changelog.d/15096.bugfix
* Update intentional mentions (MSC3952) to depend on `exact_event_match` ↵Patrick Cloke2023-02-161-3/+4
| | | | | | | | | | (MSC3758). (#15037) This replaces the specific `is_room_mention` push rule condition used in MSC3952 with the generic `exact_event_match` push rule condition from MSC3758. No functionality changes due to this.
* Support for selecting the Redis logical database. (#15034)999lakhisidhu2023-02-151-0/+1
| | | | Note that this is only used for key-value store (cached values) and not for the pub/sub replication used by Synapse.
* Implement MSC3966: Add a push rule condition to search for a value in an ↵Patrick Cloke2023-02-141-0/+5
| | | | | | array. (#15045) The `exact_event_property_contains` condition can be used to search for a value inside of an array.
* Return read-only collections from `@cached` methods (#13755)Sean Quah2023-02-101-3/+3
| | | | | | | | | | | | | It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
* Support for MSC3758: exact_event_match push condition (#14964)Patrick Cloke2023-02-101-0/+5
| | | | | This specifies to search for an exact value match, instead of string globbing. It only works across non-compound JSON values (null, boolean, integer, and strings).
* MSC3873: Escape keys when flattening dicts. (#15004)Patrick Cloke2023-02-081-0/+5
| | | | | | This disambiguates keys which attempt to match fields with a dot in them (e.g. m.relates_to). Disabled by default behind an experimental configuration flag.
* Implement MSC3958: suppress notifications from edits (#14960)Patrick Cloke2023-02-031-0/+5
| | | | | | | | Co-authored-by: Brad Murray <brad@beeper.com> Co-authored-by: Nick Barrett <nick@beeper.com> Copy the suppress_edits push rule from Beeper to implement MSC3958. https://github.com/beeper/synapse/blame/9415a1284b1bfb558bd66f28c24ca1611e6c6fa2/rust/src/push/base_rules.rs#L98-L114
* Reload the pyo3-log config when the Python logging config changes. (#14976)Patrick Cloke2023-02-031-18/+24
| | | | | | | | | | | Since pyo3-log is initialized very early in the Python start-up it caches the state of the loggers before they're fully initialized (and thus are essentially disabled). Whenever we reload the logging configuration we now also tell pyo3-log to discard any cached logging configuration it has; it will refetch the current logging configuration from Python at the next point it logs. This fixes Rust log lines not appearing in the homeserver logs.
* Prefer `type(x) is int` to `isinstance(x, int)` (#14945)David Robertson2023-01-313-25/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Perfer `type(x) is int` to `isinstance(x, int)` This covered all additional instances I could see where `x` was user-controlled. The remaining cases are ``` $ rg -s 'isinstance.*[^_]int' tests/replication/_base.py 576: if isinstance(obj, int): synapse/util/caches/stream_change_cache.py 136: assert isinstance(stream_pos, int) 214: assert isinstance(stream_pos, int) 246: assert isinstance(stream_pos, int) 267: assert isinstance(stream_pos, int) synapse/replication/tcp/external_cache.py 133: if isinstance(result, int): synapse/metrics/__init__.py 100: if isinstance(calls, (int, float)): synapse/handlers/appservice.py 262: assert isinstance(new_token, int) synapse/config/_util.py 62: if isinstance(p, int): ``` which cover metrics, logic related to `jsonschema`, and replication and data streams. AFAICS these are all internal to Synapse * Changelog
* Implement MSC3952: Intentional mentions (#14823)Patrick Cloke2023-01-271-0/+5
| | | | | | | | MSC3952 defines push rules which searches for mentions in a list of Matrix IDs in the event body, instead of searching the entire event body for display name / local part. This is implemented behind an experimental configuration flag and does not yet implement the backwards compatibility pieces of the MSC.
* Request partial joins by default (#14905)David Robertson2023-01-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | * Request partial joins by default This is a little sloppy, but we are trying to gain confidence in faster joins in the upcoming RC. Admins can still opt out by adding the following to their Synapse config: ```yaml experimental: faster_joins: false ``` We may revert this change before the release proper, depending on how testing in the wild goes. * Changelog * Try to fix the backfill test failures * Upgrade notes * Postgres compat?
* Implement MSC3930: polls push rules (#14787)Andrew Morgan2023-01-191-0/+7
|
* Change default room version to 10. Implements MSC3904 (#14111)Catalan Lover2023-01-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Change Documentation to have v10 as default room version * Change Default Room version to 10 * Add changelog entry for default room version swap * Add changelog entry for v10 default room version in docs * Clarify doc changelog entry Co-authored-by: David Robertson <david.m.robertson1@gmail.com> * Improve Documentation changes. Co-authored-by: David Robertson <david.m.robertson1@gmail.com> * Update Changelog entry to have correct format Co-authored-by: David Robertson <david.m.robertson1@gmail.com> * Update Spec Version to 1.5 * Only need 1 changelog. * Fix test. * Update "Changed in" line Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <patrickc@matrix.org>
* Stabilise serving partial join responses (#14839)David Robertson2023-01-171-1/+5
| | | | | Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true. Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
* Implement MSC3890: Remotely silence local notifications (#14775)Andrew Morgan2023-01-131-0/+15
|
* Re-enable some linting (#14821)Erik Johnston2023-01-121-6/+4
| | | | | | | * Re-enable some linting * Newsfile * Remove comment
* Implement MSC3925: changes to bundling of edits (#14811)Richard van der Hoff2023-01-101-0/+3
| | | | | | | | Two parts to this: * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag. * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.
* Support RFC7636 PKCE in the OAuth 2.0 flow. (#14750)Patrick Cloke2023-01-041-0/+6
| | | | | | | PKCE can protect against certain attacks and is enabled by default. Support can be controlled manually by setting the pkce_method of each oidc_providers entry to 'auto' (default), 'always', or 'never'. This is required by Twitter OAuth 2.0 support.
* Add experimental support for MSC3391: deleting account data (#14714)Andrew Morgan2023-01-011-0/+3
|
* Switch to ruff instead of flake8. (#14633)Patrick Cloke2022-12-211-0/+2
| | | | | ruff is a flake8-compatible Python linter written in Rust. It supports the flake8 plugins that we use and is significantly faster in testing.
* Add missing type hints to tests.config. (#14681)Patrick Cloke2022-12-161-2/+2
|
* Allow selecting "prejoin" events by state keys (#14642)David Robertson2022-12-132-21/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Declare new config * Parse new config * Read new config * Don't use trial/our TestCase where it's not needed Before: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m2.277s user 0m2.186s sys 0m0.083s ``` After: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m0.566s user 0m0.508s sys 0m0.056s ``` * Helper to upsert to event fields without exceeding size limits. * Use helper when adding invite/knock state Now that we allow admins to include events in prejoin room state with arbitrary state keys, be a good Matrix citizen and ensure they don't accidentally create an oversized event. * Changelog * Move StateFilter tests should have done this in #14668 * Add extra methods to StateFilter * Use StateFilter * Ensure test file enforces typed defs; alphabetise * Workaround surprising get_current_state_ids * Whoops, fix mypy
* Add `push.enabled` option to disable push notification calculation (#14551)Will Hunt2022-12-011-0/+1
| | | | | | | * Add initial option * changelog * Some more linting
* Create MSC1767 (extensible events) room version; Implement MSC3932 (#14521)Travis Ralston2022-11-281-0/+5
| | | | | | | | | | | | * Add MSC1767's dedicated room version, based on v10 * Only enable MSC1767 room version if the config flag is on Using a similar technique to knocking: https://github.com/matrix-org/synapse/pull/6739/files#diff-3af529eedb0e00279bafb7369370c9654b37792af8eafa0925400e9281d57f0a * Support MSC3932: Extensible events room version feature flag * Changelog entry
* Initial support for MSC3931: Room version push rule feature flags (#14520)Travis Ralston2022-11-281-0/+3
| | | | | | | * Add support for MSC3931: Room Version Supports push rule condition * Create experimental flag for future work, and use it to gate MSC3931 * Changelog entry
* Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471)Eric Eastwood2022-11-281-3/+0
| | | | | | | | Fix https://github.com/matrix-org/synapse/issues/14390 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` Complement test changes: https://github.com/matrix-org/complement/pull/559
* Remove legacy Prometheus metrics names. They were deprecated in Synapse ↵reivilibre2022-11-241-2/+0
| | | | v1.69.0 and disabled by default in Synapse v1.71.0. (#14538)
* Refactor `federation_sender` and `pusher` configuration loading. (#14496)realtyem2022-11-221-69/+70
| | | | | | To avoid duplicating the same logic for handling legacy configuration settings. This should help in applying similar logic to other worker types.
* Reduce default third party invite rate limit to 216 invites per day (#14487)Sean Quah2022-11-181-4/+1
| | | | | | The previous default was the same as the `rc_message` rate limit, which defaults to 17,280 per day. Signed-off-by: Sean Quah <seanq@matrix.org>
* Fix version that `worker_main_http_uri` is redundant from (#14476)David Robertson2022-11-171-1/+1
| | | | | * Fix version that `worker_main_http_uri` is redundant from * Changelog
* Remove need for `worker_main_http_uri` setting to use /keys/upload. (#14400)realtyem2022-11-161-0/+6
|
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-3/+2
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Support using SSL on worker endpoints. (#14128)Tuomas Ojamies2022-11-151-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix missing SSL support in worker endpoints. * Add changelog * SSL for Replication endpoint * Remove unit test change * Refactor listener creation to reduce duplicated code * Fix the logger message * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add config documentation for new TLS option Co-authored-by: Tuomas Ojamies <tojamies@palantir.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Merge tag 'v1.71.0rc2' into developSean Quah2022-11-041-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.71.0rc2 (2022-11-04) ============================== Please note that, as announced in the release notes for Synapse 1.69.0, legacy Prometheus metric names are now disabled by default. They will be removed altogether in Synapse 1.73.0. If not already done, server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names. See the [upgrade notes](https://matrix-org.github.io/synapse/v1.71/upgrade.html#upgrading-to-v1710) for more details. Improved Documentation ---------------------- - Document the changes to monthly active user metrics due to deprecation of legacy Prometheus metric names. ([\#14358](https://github.com/matrix-org/synapse/issues/14358), [\#14360](https://github.com/matrix-org/synapse/issues/14360)) Deprecations and Removals ------------------------- - Disable legacy Prometheus metric names by default. They can still be re-enabled for now, but they will be removed altogether in Synapse 1.73.0. ([\#14353](https://github.com/matrix-org/synapse/issues/14353)) Internal Changes ---------------- - Run unit tests against Python 3.11. ([\#13812](https://github.com/matrix-org/synapse/issues/13812))
| * Disable legacy Prometheus metric names by default. They can still be ↵reivilibre2022-11-021-1/+1
| | | | | | | | re-enabled for now, but they will be removed altogether in Synapse 1.73.0. (#14353)
* | Implement MSC3912: Relation-based redactions (#14260)Brendan Abolivier2022-11-031-0/+3
|/ | | Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Support OIDC backchannel logouts (#11414)Quentin Gliech2022-10-311-0/+12
| | | | | | | If configured an OIDC IdP can log a user's session out of Synapse when they log out of the identity provider. The IdP sends a request directly to Synapse (and must be configured with an endpoint) when a user logs out.
* Add workers settings to configuration manual (#14086)Dirk Klimpel2022-10-271-1/+1
| | | | | | | | | | | | | | | | | | | | * Add workers settings to configuration manual * Update `pusher_instances` * update url to python logger * update headlines * update links after headline change * remove link from `daemon process` There is no docs in Synapse for this * extend example for `federation_sender_instances` and `pusher_instances` * more infos about stream writers * add link to DAG * update `pusher_instances` * update `worker_listeners` * update `stream_writers` * Update `worker_name` Co-authored-by: David Robertson <davidr@element.io>
* Implementation for MSC3664: Pushrules for relations (#11804)DeepBlueV7.X2022-10-251-0/+3
|
* Implementation of HTTP 307 response for MSC3886 POST endpoint (#14018)Hugh Nimmo-Smith2022-10-182-1/+10
| | | | Co-authored-by: reivilibre <olivier@librepush.net> Co-authored-by: Andrew Morgan <andrewm@element.io>
* Fix `track_memory_usage` on poetry-core 1.3.x installations (#14221)David Robertson2022-10-181-1/+1
| | | | | | | | | | | | | * Fix `track_memory_usage` on poetry-core 1.3.x installations The same kind of problem as discussed in #14085: 1. we defined an extra with an underscore 2. we look it up at runtime with an underscore 3. but poetry-core 1.3.x. installs it with a dash, causing (2) to fail. Fix by using a dash everywhere. * Changelog
* Support filtering the /messages API by relation type (MSC3874). (#14148)Patrick Cloke2022-10-171-0/+3
| | | Gated behind an experimental configuration flag.
* Merge remote-tracking branch 'origin/release-v1.69' into developPatrick Cloke2022-10-141-1/+1
|\
| * Unpin build-system requirements, but impose an upper-bound (#14085)David Robertson2022-10-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Revert to prior build-system requirements This reverts #14080. * Use normalised extra name, which poetry-core 1.3 will generate anyway * Changelog * Upper bound build-system requirements * Remove upgrade note; expand changelog entry a little. * Fix typo in build-system comment Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Stabilize the threads API. (#14175)Patrick Cloke2022-10-141-3/+0
| | | | | | | | | | | | | | Stabilize the threads API (MSC3856) by supporting (only) the v1 path for the endpoint. This also marks the API as safe for workers since it is a read-only API.
* | Add an API for listing threads in a room. (#13394)Patrick Cloke2022-10-131-0/+3
| | | | | | | | | | | | | | | | | | Implement the /threads endpoint from MSC3856. This is currently unstable and behind an experimental configuration flag. It includes a background update to backfill data, results from the /threads endpoint will be partial until that finishes.
* | Remove the experimental implementation of MSC3772. (#14094)Patrick Cloke2022-10-121-2/+0
| | | | | | MSC3772 has been abandoned.
* | Remove the groups config code. (#14142)Patrick Cloke2022-10-111-27/+0
| | | | | | This has been unused for a long time, but missed removal in #11584.
* | Remove support for the unstable dir flag on relations. (#14106)Patrick Cloke2022-10-071-3/+0
| | | | | | | | | | | | From MSC3715, this was unused by clients (and there was no way for clients to know it was supported). Matrix 1.4 defines the stable field.
* | Parse SYNAPSE_ASYNC_IO_REACTOR env variable & log the reactor on startup ↵Quentin Gliech2022-10-071-0/+3
| | | | | | | | (#14092)
* | Use stable identifiers for MSC3771 & MSC3773. (#14050)Patrick Cloke2022-10-071-2/+0
| | | | | | | | | | These are both part of Matrix 1.4 which has now been released. For now, support both the unstable and stable identifiers.
* | Track notification counts per thread (implement MSC3773). (#13776)Patrick Cloke2022-10-041-0/+2
| | | | | | | | | | | | | | | | When retrieving counts of notifications segment the results based on the thread ID, but choose whether to return them as individual threads or as a single summed field by letting the client opt-in via a sync flag. The summarization code is also updated to be per thread, instead of per room.
* | Disable pushing for server ACL events (MSC3786). (#13997)Patrick Cloke2022-10-041-3/+0
|/ | | | | | Switches to the stable identifier for MSC3786 and enables it by default. This disables pushes of m.room.server_acl events.
* Announce that legacy metric names are deprecated, will be turned off by ↵reivilibre2022-10-031-26/+0
| | | | default in Synapse v1.71.0 and removed altogether in Synapse v1.73.0. (#14024)
* Allow admins to require a manual approval process before new accounts can be ↵Brendan Abolivier2022-09-291-0/+19
| | | | used (using MSC3866) (#13556)
* Accept & store thread IDs for receipts (implement MSC3771). (#13782)Patrick Cloke2022-09-231-0/+2
| | | | Updates the `/receipts` endpoint and receipt EDU handler to parse a `thread_id` from the body and insert it in the database.
* Faster Remote Room Joins: tell remote homeservers that we are unable to ↵reivilibre2022-09-231-1/+2
| | | | authorise them if they query a room which has partial state on our server. (#13823)
* Implementation of MSC3882 login token request (#13722)Hugh Nimmo-Smith2022-09-211-0/+7
|
* Support enabling/disabling pushers (from MSC3881) (#13799)Brendan Abolivier2022-09-211-0/+3
| | | Partial implementation of MSC3881
* Be able to correlate timeouts in reverse-proxy layer in front of Synapse ↵Eric Eastwood2022-09-151-3/+10
| | | | | | | | | | | | | | | | | | (pull request ID from header) (#13801) Fix https://github.com/matrix-org/synapse/issues/13685 New config: ```diff listeners: - port: 8008 tls: false type: http x_forwarded: true + request_id_header: "cf-ray" bind_addresses: ['::1', '127.0.0.1', '0.0.0.0'] ```
* Fix a bug where Synapse fails to start if a signing key file contains an ↵reivilibre2022-09-081-1/+12
| | | | empty line. (#13738)
* Remove configuration options for direct TCP replication. (#13647)Patrick Cloke2022-09-062-6/+18
| | | Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
* Disable calculating unread counts unless the config flag is enabled. (#13694)Patrick Cloke2022-09-011-0/+3
| | | | | | | | This avoids doing work that will never be used (since the resulting unread counts will never be sent in a /sync response). The negative of doing this is that unread counts will be incorrect when the feature is initially enabled.
* Remove support for unstable private read receipts (#13653)Šimon Brandner2022-09-011-3/+0
| | | Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
* Generate missing configuration files at startup (#13615)Richard van der Hoff2022-08-261-11/+48
| | | | | | | | If things like the signing key file are missing, let's just try to generate them on startup. Again, this is useful for k8s-like deployments where we just want to generate keys on the first run.
* Support `registration_shared_secret` in a file (#13614)Richard van der Hoff2022-08-251-2/+31
| | | | A new `registration_shared_secret_path` option. This is kinda handy for k8s deployments and things.
* Add experimental configuration option to allow disabling legacy Prometheus ↵reivilibre2022-08-241-0/+29
| | | | | metric names. (#13540) Co-authored-by: David Robertson <davidr@element.io>
* Drop support for delegating email validation, round 2 (#13596)David Robertson2022-08-232-50/+9
|
* Implement MSC3852: Expose `last_seen_user_agent` to users for their own ↵Andrew Morgan2022-08-191-0/+3
| | | | devices; also expose to Admin API (#13549)
* Fix a typo in docs and in some warnings (#13538)Antonin Loubiere2022-08-173-3/+3
|
* Support stable identifiers for MSC2285: private read receipts. (#13273)Šimon Brandner2022-08-051-1/+1
| | | | | This adds support for the stable identifiers of MSC2285 while continuing to support the unstable identifiers behind the configuration flag. These will be removed in a future version.
* Rename `RateLimitConfig` to `RatelimitSettings` (#13442)Dirk Klimpel2022-08-031-21/+21
|
* Merge tag 'v1.64.0rc2' into developRichard van der Hoff2022-07-292-10/+50
|\ | | | | | | | | | | | | Synapse 1.64.0rc2 (2022-07-29) ============================== This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
| * Revert "Drop support for delegating email validation (#13192)" (#13406)3nprob2022-07-292-10/+50
| | | | | | | | | | Reverts commit fa71bb18b527d1a3e2629b48640ea67fff2f8c59, and tweaks documentation. Signed-off-by: 3nprob <git@3n.anonaddy.com>
* | Use stable prefixes for MSC3827: filtering of `/publicRooms` by room type ↵Šimon Brandner2022-07-271-3/+0
| | | | | | | | | | | | (#13370) Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
* | Implement MSC3848: Introduce errcodes for specific event sending failures ↵Will Hunt2022-07-271-0/+3
|/ | | | | (#13343) Implements MSC3848
* Support Implicit TLS for sending emails (#13317)Jan Schär2022-07-251-1/+6
| | | | | | | | | | Previously, TLS could only be used with STARTTLS. Add a new option `force_tls`, where TLS is used from the start. Implicit TLS is recommended over STARTLS, see https://datatracker.ietf.org/doc/html/rfc8314 Fixes #8046. Signed-off-by: Jan Schär <jan@jschaer.ch>
* Merge branch 'master' into developBrendan Abolivier2022-07-191-7/+7
|\
| * Remove 'anonymised' from the phone home stats documentation (#13321)Andrew Morgan2022-07-191-7/+7
| |
* | Rate limit joins per-room (#13276)David Robertson2022-07-191-0/+7
| |
* | Provide more info why we don't have any thumbnails to serve (#13038)Eric Eastwood2022-07-151-7/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix https://github.com/matrix-org/synapse/issues/13016 ## New error code and status ### Before Previously, we returned a `404` for `/thumbnail` which isn't even in the spec. ```json { "errcode": "M_NOT_FOUND", "error": "Not found [b'hs1', b'tefQeZhmVxoiBfuFQUKRzJxc']" } ``` ### After What does the spec say? > 400: The request does not make sense to the server, or the server cannot thumbnail the content. For example, the client requested non-integer dimensions or asked for negatively-sized images. > > *-- https://spec.matrix.org/v1.1/client-server-api/#get_matrixmediav3thumbnailservernamemediaid* Now with this PR, we respond with a `400` when we don't have thumbnails to serve and we explain why we might not have any thumbnails. ```json { "errcode": "M_UNKNOWN", "error": "Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.)", } ``` > Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.) --- We still respond with a 404 in many other places. But we can iterate on those later and maybe keep some in some specific places after spec updates/clarification: https://github.com/matrix-org/matrix-spec/issues/1122 We can also iterate on the bugs where Synapse doesn't thumbnail when it should in other issues/PRs.
* | Drop support for delegating email validation (#13192)Richard van der Hoff2022-07-122-41/+15
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Drop support for delegating email validation Delegating email validation to an IS is insecure (since it allows the owner of the IS to do a password reset on your HS), and has long been deprecated. It will now cause a config error at startup. * Update unit test which checks for email verification Give it an `email` config instead of a threepid delegate * Remove unused method `requestEmailToken` * Simplify config handling for email verification Rather than an enum and a boolean, all we need here is a single bool, which says whether we are or are not doing email verification. * update docs * changelog * upgrade.md: fix typo * update version number this will be in 1.64, not 1.63 * update version number this one too
* Allow dependency errors to pass through (#13113)Jacek Kuśnierz2022-06-307-58/+15
| | | | Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Cleanup references to sample config in the docs and redirect users to ↵Shay2022-06-301-1/+1
| | | | configuration manual (#13077)
* Rate limiting invites per issuer (#13125)David Teller2022-06-301-0/+5
| | | Co-authored-by: reivilibre <oliverw@matrix.org>
* Implement MSC3827: Filtering of `/publicRooms` by room type (#13031)Šimon Brandner2022-06-291-0/+3
| | | | Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
* Add custom well-known (#13035)Jacek Kuśnierz2022-06-161-0/+20
| | | Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
* Change default `sync_response_cache_duration` (#13042)Erik Johnston2022-06-151-1/+1
|
* Replace pyjwt with authlib in `org.matrix.login.jwt` (#13011)Hannes Lerchl2022-06-151-5/+5
|
* Remove code generating comments in configuration file (#12941)Shay2022-06-1438-2873/+53
|
* Move the "email unsubscribe" resource, refactor the macaroon generator & ↵Quentin Gliech2022-06-141-2/+4
| | | | | | | | | | | | | | | | | | | | | simplify the access token verification logic. (#12986) This simplifies the access token verification logic by removing the `rights` parameter which was only ever used for the unsubscribe link in email notifications. The latter has been moved under the `/_synapse` namespace, since it is not a standard API. This also makes the email verification link more secure, by embedding the app_id and pushkey in the macaroon and verifying it. This prevents the user from tampering the query parameters of that unsubscribe link. Macaroon generation is refactored: - Centralised all macaroon generation and verification logic to the `MacaroonGenerator` - Moved to `synapse.utils` - Changed the constructor to require only a `Clock`, hostname, and a secret key (instead of a full `Homeserver`). - Added tests for all methods.
* Move the (unstable) `dir` parameter for /relations behind an experimental ↵Patrick Cloke2022-06-081-0/+3
| | | | | | flag. (#12984) MSC3715 defines this parameter, but the unstable version of it should be behind an experimental flag.
* Fix Synapse git info missing in version strings (#12973)David Robertson2022-06-071-2/+2
|
* Remove remaining bits of groups code. (#12936)Patrick Cloke2022-06-011-1/+0
| | | | | | * Update worker docs to remove group endpoints. * Removes an unused parameter to `ApplicationService`. * Break dependency between media repo and groups. * Avoid copying `m.room.related_groups` state events during room upgrades.
* Add config options for media retention (#12732)Andrew Morgan2022-05-311-0/+16
|
* Add a background job to automatically delete stale devices (#12855)Brendan Abolivier2022-05-271-0/+11
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Merge tag 'v1.60.0rc2' into developSean Quah2022-05-271-2/+15
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.60.0rc2 (2022-05-27) ============================== This release of Synapse adds a unique index to the `state_group_edges` table, in order to prevent accidentally introducing duplicate information (for example, because a database backup was restored multiple times). If your Synapse database already has duplicate rows in this table, this could fail with an error and require manual remediation. Additionally, the signature of the `check_event_for_spam` module callback has changed. The previous signature has been deprecated and remains working for now. Module authors should update their modules to use the new signature where possible. See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600) for more details. Features -------- - Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883)) Bugfixes -------- - Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875)) Internal Changes ---------------- - Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
| * Add an option allowing users to use their password to reauthenticate even ↵reivilibre2022-05-271-2/+15
| | | | | | | | though password authentication is disabled. (#12883)
* | Easy type hints in synapse.logging.opentracing (#12894)David Robertson2022-05-271-2/+4
| |
* | Remove backing code for groups/communities (#12558)Patrick Cloke2022-05-263-31/+0
| | | | | | | | Including handlers, configuration code, appservice support, and the GroupID construct.
* | Remove user-visible groups/communities code (#12553)Patrick Cloke2022-05-252-15/+0
| | | | | | | | | | | | | | | | | | Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove: * All API endpoints (including Client-Server, Server-Server, and admin). * Documented configuration options (and the experimental flag, which is now unused). * Special handling during room upgrades. * The `groups` section of the `/sync` response.
* | Experimental support for MSC3772 (#12740)Patrick Cloke2022-05-241-0/+3
|/ | | | | | | | | | Implements the following behind an experimental configuration flag: * A new push rule kind for mutually related events. * A new default push rule (`.m.rule.thread_reply`) under an unstable prefix. This is missing part of MSC3772: * The `.m.rule.thread_reply_to_me` push rule, this depends on MSC3664 / #11804.
* Properly close providers.json file stream. (#12794)Petr Vaněk2022-05-191-3/+3
|
* Fix typo in listener config (#12742)Dirk Klimpel2022-05-161-1/+1
|
* Add config flags to allow for cache auto-tuning (#12701)Shay2022-05-131-0/+33
|
* add default_power_level_content_override config option. (#12618)Andy Balaam2022-05-121-0/+47
| | | Co-authored-by: Matthew Hodgson <matthew@matrix.org>
* Reload cache factors from disk on SIGHUP (#12673)David Robertson2022-05-113-26/+119
|
* Add the `update_user_directory_from_worker` configuration option ↵reivilibre2022-05-102-4/+7
| | | | | (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. (#12654) Co-authored-by: Shay <hillerys@element.io>
* Implement MSC3786: Add a default push rule to ignore m.room.server_acl ↵Šimon Brandner2022-05-101-0/+3
| | | | | | events (#12601) Fixes vector-im/element-web#20788 Implements matrix-org/matrix-spec-proposals#3786
* Update mypy to 0.950 and fix complaints (#12650)David Robertson2022-05-061-1/+2
|
* Add the `notify_appservices_from_worker` configuration option (superseding ↵reivilibre2022-05-062-2/+108
| | | | `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452)
* Use `private` instead of `hidden` in MSC2285 related code. (#12635)Šimon Brandner2022-05-051-1/+1
|
* Fix typo in some instances of enable_registration_token_3pid_bypass. (#12639)Will Hunt2022-05-051-2/+2
|
* Add `mau_appservice_trial_days` config (#12619)Will Hunt2022-05-041-0/+8
| | | | | | | | | | | | | | | | | | | | | * Add mau_appservice_trial_days * Add a test * Tweaks * changelog * Ensure we sync after the delay * Fix types * Add config statement * Fix test * Reinstate logging that got removed * Fix feature name
* Disable device name lookup over federation by default (#12616)Andrew Morgan2022-05-041-5/+5
|
* Remove special-case for `twisted` logger (#12589)Richard van der Hoff2022-04-291-7/+0
| | | | | | | | | | | | | | | This was originally added when we first added a `MemoryHandler` to the default log config back in https://github.com/matrix-org/synapse/pull/8040, to ensure that we didn't explode with an infinite loop if there was an error formatting the logs. Since then, we made additional improvements to logging which make this workaround redundant. In particular: * we no longer attempt to log un-UTF8-decodable byte sequences, which were the most likely cause of an error in the first place. * https://github.com/matrix-org/synapse/pull/8268 ensures that in the unlikely case that there *is* an error, it won't cause an infinite loop.
* Remove unused `# type: ignore`s (#12531)David Robertson2022-04-271-4/+2
| | | | | | | | | | | | | | | | | | | | | | Over time we've begun to use newer versions of mypy, typeshed, stub packages---and of course we've improved our own annotations. This makes some type ignore comments no longer necessary. I have removed them. There was one exception: a module that imports `select.epoll`. The ignore is redundant on Linux, but I've kept it ignored for those of us who work on the source tree using not-Linux. (#11771) I'm more interested in the config line which enforces this. I want unused ignores to be reported, because I think it's useful feedback when annotating to know when you've fixed a problem you had to previously ignore. * Installing extras before typechecking Lacking an easy way to install all extras generically, let's bite the bullet and make install the hand-maintained `all` extra before typechecking. Now that https://github.com/matrix-org/backend-meta/pull/6 is merged to the release/v1 branch.
* Add option to enable token registration without requiring 3pids (#12526)Will Hunt2022-04-271-0/+9
|
* Implement MSC2815: allow room moderators to view redacted event content (#12427)Tulir Asokan2022-04-201-0/+3
| | | | | | Implements matrix-org/matrix-spec-proposals#2815 Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Process device list updates asynchronously (#12365)Erik Johnston2022-04-121-8/+0
|
* Disable groups/communities by default. (#12344)Patrick Cloke2022-04-121-1/+1
| | | | This disables the endpoints (and sync response fields) for groups/communities by default.
* Remove references to unstable identifiers from MSC3440. (#12382)Patrick Cloke2022-04-121-3/+0
| | | | | Removes references to unstable thread relation, unstable identifiers for filtering parameters, and the experimental config flag.
* Remove experimental configuration flag for MSC3666. (#12436)Patrick Cloke2022-04-111-2/+0
|
* Add missing type hints to config classes. (#12402)Patrick Cloke2022-04-1144-136/+233
|
* Track device list updates per room. (#12321)Erik Johnston2022-04-041-0/+8
| | | | | | | | | | | | | | This is a first step in dealing with #7721. The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things: 1. we can defer calculating the set of remote servers that need to be poked about the change; and 2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed. However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled. There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly). Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table.
* Move MSC2654 support behind an experimental configuration flag. (#12295)Patrick Cloke2022-03-311-0/+3
| | | To match the current thinking on disabling experimental features by default.
* Send device list updates to application services (MSC3202) - part 1 (#11881)Andrew Morgan2022-03-302-2/+4
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add a configuration to exclude rooms from sync response (#12310)Brendan Abolivier2022-03-301-0/+13
|
* Fix typechecker problems exposed by signedjson 1.1.2 (#12326)David Robertson2022-03-291-5/+8
|
* Add restrictions by default to open registration in Synapse (#12091)Shay2022-03-251-1/+13
|
* Optionally include account validity in MSC3720 account status responses (#12266)Brendan Abolivier2022-03-241-0/+4
|
* Refuse to start if DB has an unsafe locale (#12262)Shay2022-03-231-0/+6
|
* Fix dead link in spam checker warning (#12231)David Robertson2022-03-161-2/+2
|
* Deprecate the groups/communities endpoints and add an experimental ↵Patrick Cloke2022-03-121-0/+3
| | | | configuration flag. (#12200)
* Add config settings for background update parameters (#11980)Shay2022-03-113-0/+72
|
* Implement a Jinja2 filter to extract localparts from email addresses (#12212)Brendan Abolivier2022-03-111-1/+2
|
* Update the SSO username picker template to comply with SIWA guidelines (#12210)Brendan Abolivier2022-03-111-2/+7
| | | Fixes https://github.com/matrix-org/synapse/issues/12205
* Move scripts directory inside synapse, exposing as setuptools entry_points ↵David Robertson2022-03-021-1/+1
| | | | | | | | | | | | (#12118) * Two scripts are basically entry_points already * Move and rename scripts/* to synapse/_scripts/*.py * Delete sync_room_to_group.pl * Expose entry points in setup.py * Update linter script and config * Fixup scripts & docs mentioning scripts that moved Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Use importlib.metadata to read requirements (#12088)David Robertson2022-03-017-7/+7
| | | | | | | | | | | | | | | | | | * Pull runtime dep checks into their own module * Reimplement `check_requirements` using `importlib` I've tried to make this clearer. We start by working out which of Synapse's requirements we need to be installed here and now. I was surprised that there wasn't an easier way to see which packages were installed by a given extra. I've pulled out the error messages into functions that deal with "is this for an extra or not". And I've rearranged the loop over two different sets of requirements into one loop with a "must be instaled" flag. I hope you agree that this is clearer. * Test cases
* Add support for MSC3202: sending one-time key counts and fallback key usage ↵reivilibre2022-02-242-6/+23
| | | | | states to Application Services. (#11617) Co-authored-by: Erik Johnston <erik@matrix.org>
* Implement account status endpoints (MSC3720) (#12001)Brendan Abolivier2022-02-221-0/+3
| | | | | See matrix-org/matrix-doc#3720 Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Use room version 9 as the default room version (per MSC3589). (#12058)Patrick Cloke2022-02-221-1/+1
|
* Remove unstable MSC3283 flags (#12018)Erik Johnston2022-02-171-3/+0
| | | Fixes #11962
* Faster joins: parse msc3706 fields in send_join response (#12011)Richard van der Hoff2022-02-171-0/+4
| | | Part of my work on #11249: add code to handle the new fields added in MSC3706.
* Remove support for the legacy structured logging configuration. (#12008)Patrick Cloke2022-02-171-4/+8
|
* remove empty file, reword/rename newsfragment, and add line to upgrade docH. Shay2022-02-141-0/+0
|
* Use version string helper from matrix-common (#11979)David Robertson2022-02-141-3/+6
| | | | * Require latest matrix-common * Use the common function
* Implement MSC3706: partial state in `/send_join` response (#11967)Richard van der Hoff2022-02-121-0/+3
| | | | | | | | | | | | * Make `get_auth_chain_ids` return a Set It has a set internally, and a set is often useful where it gets used, so let's avoid converting to an intermediate list. * Minor refactors in `on_send_join_request` A little bit of non-functional groundwork * Implement MSC3706: partial state in /send_join response
* Enable cache time-based expiry by default (#11849)Shay2022-02-112-7/+33
|
* Experimental support to include bundled aggregations in search results ↵Patrick Cloke2022-02-081-0/+2
| | | | (MSC3666) (#11837)
* Remove support for the webclient listener. (#11895)Patrick Cloke2022-02-031-36/+12
| | | Also remove support for non-HTTP(S) web_client_location.
* Add a ratelimiter for 3pid invite (#11892)Brendan Abolivier2022-02-031-0/+15
|
* Revert experimental push rules from #7997. (#11884)Patrick Cloke2022-02-021-13/+0
| | | Manually reverts the merge from cdbb8e6d6e36e0b6bc36e676d8fe66c96986b399.
* Send to-device messages to application services (#11215)Andrew Morgan2022-02-011-0/+7
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove the obsolete MSC1849 configuration flag. (#11843)Patrick Cloke2022-01-311-2/+0
| | | | | MSC1849 was replaced by MSC2675, which was merged. The configuration flag, which defaulted to true, is no longer useful.
* Configurable limits on avatars (#11846)Brendan Abolivier2022-01-281-0/+27
| | | | | | Only allow files which file size and content types match configured limits to be set as avatar. Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
* Fix some indentation inconsistencies in the sample config (modules) (#11838)Dirk Klimpel2022-01-271-5/+5
|
* Add a config flag to inhibit `M_USER_IN_USE` during registration (#11743)Brendan Abolivier2022-01-261-0/+12
| | | | | | | This is mostly motivated by the tchap use case, where usernames are automatically generated from the user's email address (in a way that allows figuring out the email address from the username). Therefore, it's an issue if we respond to requests on /register and /register/available with M_USER_IN_USE, because it can potentially leak email addresses (which include the user's real name and place of work). This commit adds a flag to inhibit the M_USER_IN_USE errors that are raised both by /register/available, and when providing a username early into the registration process. This error will still be raised if the user completes the registration process but the username conflicts. This is particularly useful when using modules (https://github.com/matrix-org/synapse/pull/11790 adds a module callback to set the username of users at registration) or SSO, since they can ensure the username is unique. More context is available in the PR that introduced this behaviour to synapse-dinsic: matrix-org/synapse-dinsic#48 - as well as the issue in the matrix-dinsic repo: matrix-org/matrix-dinsic#476
* Add deprecation warnings for `webclient` listener and non-HTTP(S) ↵Patrick Cloke2022-01-201-14/+20
| | | | | | | `web_client_location`. (#11774) This changes the behaviour of the root endpoint to redirect directly to the configuration of `web_client_location` if it is given an HTTP(S) URL.
* Fix up docs for `track_puppeted_user_ips` (again) (#11757)Richard van der Hoff2022-01-171-4/+12
| | | Fixes #11741
* Update documentation for configuring facebook login (#11755)Richard van der Hoff2022-01-171-3/+6
| | | ... and a minor thinko fix in the sample config.
* Fix sample_config.yaml in regards track_puppeted_user_ips (#11749)Jason Robinson2022-01-141-5/+6
| | | | | | | * Fix sample_config.yaml in regards track_puppeted_user_ips Closes #11741 Signed-off-by: Jason Robinson <jasonr@matrix.org>
* update room spec url in config files (#11739)qwertyforce2022-01-131-1/+1
| | | | | | | * change spec url in config files * Create 11739.txt * .txt -> .doc
* Use auto_attribs/native type hints for attrs classes. (#11692)Patrick Cloke2022-01-133-29/+23
|
* Allow tracking puppeted users for MAU (#11561)Jason Robinson2022-01-121-0/+10
| | | | | | | | | | | | | | | Currently when puppeting another user, the user doing the puppeting is tracked for client IPs and MAU (if configured). When tracking MAU is important, it becomes necessary to be possible to also track the client IPs and MAU of puppeted users. As an example a client that manages user creation and creation of tokens via the Synapse admin API, passing those tokens for the client to use. This PR adds optional configuration to enable tracking of puppeted users into monthly active users. The default behaviour stays the same. Signed-off-by: Jason Robinson <jasonr@matrix.org>
* Fix link from generated configuration file to documentation (#11678)Philipp Matthias Schäfer2022-01-051-1/+1
| | | | Co-authored-by: reivilibre <olivier@librepush.net> Co-authored-by: reivilibre <oliverw@matrix.org>
* Re-apply: Move glob_to_regex and re_word_boundary to matrix-python-common ↵reivilibre2022-01-052-2/+4
| | | | | #11505 (#11687) Co-authored-by: Sean Quah <seanq@element.io>
* Include the topic event in the prejoin state, per MSC3173. (#11666)Patrick Cloke2022-01-041-0/+2
| | | | Invites and knocks will now include the topic in the stripped state send to clients before joining the room.
* Convert all namedtuples to attrs. (#11665)Patrick Cloke2021-12-301-16/+18
| | | To improve type hints throughout the code.
* Add MSC2716 and MSC3030 to `/versions` -> `unstable_features` (#11582)Eric Eastwood2021-12-161-1/+1
| | | | As suggested in https://github.com/matrix-org/matrix-react-sdk/pull/7372#discussion_r769523369
* Add experimental support for MSC3202: allowing application services to ↵reivilibre2021-12-151-0/+5
| | | | masquerade as specific devices. (#11538)
* Add missing type hints to `synapse.appservice` (#11360)Patrick Cloke2021-12-141-2/+1
|
* Additional type hints for the config module, part 2. (#11480)Patrick Cloke2021-12-094-19/+27
|
* Revert "Move `glob_to_regex` and `re_word_boundary` to ↵Sean Quah2021-12-072-4/+2
| | | | | | `matrix-python-common` (#11505) (#11527) This reverts commit a77c36989785c0d5565ab9a1169f4f88e512ce8a.
* Stabilise support for MSC2918 refresh tokens as they have now been merged ↵reivilibre2021-12-061-0/+38
| | | | into the Matrix specification. (#11435)
* Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)Sean Quah2021-12-062-2/+4
|
* Support configuring the lifetime of non-refreshable access tokens separately ↵reivilibre2021-12-031-0/+49
| | | | to refreshable access tokens. (#11445)
* Add MSC3030 experimental client and federation API endpoints to get the ↵Eric Eastwood2021-12-021-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | closest event to a given timestamp (#9445) MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030 Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about. ``` GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Federation API endpoint: ``` GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Co-authored-by: Erik Johnston <erik@matrix.org>
* Additional type hints for config module. (#11465)Patrick Cloke2021-12-0113-98/+126
| | | | This adds some misc. type hints to helper methods used in the `synapse.config` module.
* Add type annotations to some of the configuration surrounding refresh ↵reivilibre2021-11-291-2/+5
| | | | tokens. (#11428)
* Support expiry of refresh tokens and expiry of the overall session when ↵reivilibre2021-11-261-17/+7
| | | | refresh tokens are in use. (#11425)
* Rename unstable `access_token_lifetime` configuration option to ↵reivilibre2021-11-231-8/+15
| | | | `refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. (#11388)
* Add missing type hints to config base classes (#11377)Patrick Cloke2021-11-237-99/+162
|
* Add config for customizing the claim used for JWT logins. (#11361)Kostas2021-11-221-0/+9
| | | | | Allows specifying a different claim (from the default "sub") to use when calculating the localpart of the Matrix ID used during the JWT login.
* Add an admin API to run background jobs. (#11352)Dirk Klimpel2021-11-191-2/+2
| | | | | | Instead of having admins poke into the database directly. Can currently run jobs to populate stats and to populate the user directory.
* Fix checking whether a room can be published on creation. (#11392)Patrick Cloke2021-11-191-22/+28
| | | | | | | | If `room_list_publication_rules` was configured with a rule with a non-wildcard alias and a room was created with an alias then an internal server error would have been thrown. This fixes the error and properly applies the publication rules during room creation.
* Remove dead code from acme support. (#11393)Patrick Cloke2021-11-191-50/+0
|
* Remove legacy code related to deprecated ↵Shay2021-11-182-29/+8
| | | | | | | | | | | | | | | | | | | | | `trust_identity_server_for_password_resets` config flag (#11333) * remove code legacy code related to deprecated config flag "trust_identity_server_for_password_resets" from synapse/config/emailconfig.py * remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/config/registration.py * remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/handlers/identity.py * add tests to ensure config error is thrown and synapse refuses to start when depreciated config flag is found * add changelog * slightly change behavior to only check for deprecated flag if set to 'true' * Update changelog.d/11333.misc Co-authored-by: reivilibre <oliverw@matrix.org> Co-authored-by: reivilibre <oliverw@matrix.org>
* Docs: Quote wildcard `federation_certificate_verification_whitelist` (#11381)Nicolai Søborg2021-11-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise I get this beautiful stacktrace: ``` python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/root/synapse/synapse/app/homeserver.py", line 455, in <module> main() File "/root/synapse/synapse/app/homeserver.py", line 445, in main hs = setup(sys.argv[1:]) File "/root/synapse/synapse/app/homeserver.py", line 345, in setup config = HomeServerConfig.load_or_generate_config( File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config config_dict = read_config_files(config_files) File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files yaml_config = yaml.safe_load(file_stream) File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load return load(stream, SafeLoader) File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load return loader.get_single_data() File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data node = self.get_single_node() File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node document = self.compose_document() File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document node = self.compose_node(None, None) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node node = self.compose_sequence_node(anchor) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node while not self.check_event(SequenceEndEvent): File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event self.current_event = self.state() File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry return self.parse_block_sequence_entry() File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry if not self.check_token(BlockEntryToken, BlockEndToken): File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token self.fetch_more_tokens() File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens return self.fetch_alias() File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias self.tokens.append(self.scan_anchor(AliasToken)) File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor raise ScannerError("while scanning an %s" % name, start_mark, yaml.scanner.ScannerError: while scanning an alias in "/etc/matrix/homeserver.yaml", line 614, column 5 expected alphabetic or numeric character, but found '.' in "/etc/matrix/homeserver.yaml", line 614, column 6 ``` Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
* Default value for `public_baseurl` (#11210)Richard van der Hoff2021-11-088-54/+53
| | | | | We might as well use a default value for `public_baseurl` based on `server_name` - in many cases, it will be correct.
* Enable passing typing stream writers as a list. (#11237)Nick Barrett2021-11-031-3/+15
| | | | This makes the typing stream writer config match the other stream writers that only currently support a single worker.
* Support for serving server well-known files (#11211)Richard van der Hoff2021-11-011-0/+19
| | | | Fixes https://github.com/matrix-org/synapse/issues/8308
* Add type hints for most `HomeServer` parameters (#11095)Sean Quah2021-10-221-1/+8
|
* Fix synapse.config module "read" command (#11145)Jason Robinson2021-10-221-13/+33
| | | | | | | | | | `synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse. The "read" command broke in https://github.com/matrix-org/synapse/pull/10916 as it now requires passing in "server.server_name" for example. Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok. Signed-off-by: Jason Robinson <jasonr@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Add a thread relation type per MSC3440. (#11088)Patrick Cloke2021-10-211-0/+2
| | | | Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
* Move experimental & retention config out of the server module. (#11070)Patrick Cloke2021-10-155-201/+233
|
* Port the Password Auth Providers module interface to the new generic ↵Azrenbeth2021-10-131-30/+23
| | | | | | interface (#10548) Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Relax `ignore-missing-imports` for modules that have stubs now and update ↵David Robertson2021-10-081-3/+6
| | | | | | | | | | | | mypy (#11006) Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details. We therefore pull in stub packages in setup.py Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them. The rest of this change consists of fixups to make the newer mypy + stubs pass CI. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Require direct references to configuration variables. (#10985)Patrick Cloke2021-10-0611-73/+33
| | | | | | This removes the magic allowing accessing configurable variables directly from the config object. It is now required that a specific configuration class is used (e.g. `config.foo` must be replaced with `config.server.foo`).
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-292-4/+2
|
* Clean-up type hints in server config (#10915)Patrick Cloke2021-09-281-51/+49
| | | | | | By using attrs instead of dicts to store configuration. Also updates some of the attrs classes to use proper type hints and auto_attribs.
* Fix race conditions when creating media store and config directories (#10913)Sean Quah2021-09-271-7/+2
|
* Use direct references for configuration variables (part 5). (#10897)Patrick Cloke2021-09-241-1/+3
|
* Use direct references for configuration variables (part 4). (#10893)Patrick Cloke2021-09-231-1/+1
|
* Use direct references for some configuration variables (part 3) (#10885)Patrick Cloke2021-09-231-3/+6
| | | | | | | | This avoids the overhead of searching through the various configuration classes by directly referencing the class that the attributes are in. It also improves type hints since mypy can now resolve the types of the configuration variables.
* Remove unnecessary parentheses around tuples returned from methods (#10889)Andrew Morgan2021-09-231-1/+1
|
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-28 12:13:18 +0000