summary refs log tree commit diff
path: root/synapse/handlers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes github/matrix-org-hotfixesErik Johnston2024-08-085-279/+480
|\
| * SSS: Implement PREVIOUSLY room tracking (#17535)Erik Johnston2024-08-081-16/+52
| | | | | | | | | | | | Implement tracking of rooms that have had updates that have not been sent down to clients. Simplified Sliding Sync (SSS)
| * Sliding Sync: Use `stream_ordering` based timeline pagination for ↵Eric Eastwood2024-08-075-38/+115
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | incremental sync (#17510) Use `stream_ordering` based `timeline` pagination for incremental `/sync` in Sliding Sync. Previously, we were always using a `topological_ordering` but we should only be using that for historical scenarios (initial `/sync`, newly joined, or haven't sent the room down the connection before). This is slightly different than what the [spec suggests](https://spec.matrix.org/v1.10/client-server-api/#syncing) > Events are ordered in this API according to the arrival time of the event on the homeserver. This can conflict with other APIs which order events based on their partial ordering in the event graph. This can result in duplicate events being received (once per distinct API called). Clients SHOULD de-duplicate events based on the event ID when this happens. But we've had a [discussion below in this PR](https://github.com/element-hq/synapse/pull/17510#discussion_r1699105569) and this matches what Sync v2 already does and seems like it makes sense. Created a spec issue https://github.com/matrix-org/matrix-spec/issues/1917 to clarify this. Related issues: - https://github.com/matrix-org/matrix-spec/issues/1917 - https://github.com/matrix-org/matrix-spec/issues/852 - https://github.com/matrix-org/matrix-spec-proposals/pull/4033
| * Sliding Sync: Add more tracing (#17514)Eric Eastwood2024-08-061-239/+327
| | | | | | | | | | | | | | | | | | | | | | | | Spawning from looking at a couple traces and wanting a little more info. Follow-up to github.com/element-hq/synapse/pull/17501 The changes in this PR allow you to find slow Sliding Sync traces ignoring the `wait_for_events` time. In Jaeger, you can now filter for the `current_sync_for_user` operation with `RESULT.result=true` indicating that it actually returned non-empty results. If you want to find traces for your own user, you can use `RESULT.result=true ARG.sync_config.user="@madlittlemods:matrix.org"`
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-08-064-100/+682
|\|
| * SS: Reset connection if token is unrecognized (#17529)Erik Johnston2024-08-061-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This triggers the client to start a new sliding sync connection. If we don't do this and the client asks for the full range of rooms, we end up sending down all rooms and their state from scratch (which can be very slow) This causes things like https://github.com/element-hq/element-x-ios/issues/3115 after we restart the server --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
| * Sliding Sync: Add typing notification extension (MSC3961) (#17505)Eric Eastwood2024-07-313-1/+93
| | | | | | | | | | | | | | [MSC3961](https://github.com/matrix-org/matrix-spec-proposals/pull/3961): Sliding Sync Extension: Typing Notifications Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * Sliding Sync: Update filters to be robust against remote invite rooms (#17450)Eric Eastwood2024-07-302-38/+351
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update `filters.is_encrypted` and `filters.types`/`filters.not_types` to be robust when dealing with remote invite rooms in Sliding Sync. Part of [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync Follow-up to https://github.com/element-hq/synapse/pull/17434 We now take into account current state, fallback to stripped state for invite/knock rooms, then historical state. If we can't determine the info needed to filter a room (either from state or stripped state), it is filtered out.
| * Sliding Sync: Add receipts extension (MSC3960) (#17489)Eric Eastwood2024-07-302-61/+211
| | | | | | | | | | | | | | [MSC3960](https://github.com/matrix-org/matrix-spec-proposals/pull/3960): Receipts extension Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-302-11/+33
|\|
| * Add some more opentracing to sliding sync (#17501)Erik Johnston2024-07-301-3/+15
| | | | | | This will make it easier to see what it is doing in jaeger.
| * Fix `failures` property in `/keys/query` (#17499)Richard van der Hoff2024-07-301-8/+18
| | | | | | | | Fixes: https://github.com/element-hq/synapse/issues/17498 Fixes: https://github.com/element-hq/element-web/issues/27867
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-301-37/+500
|\|
| * Only send rooms with updates down sliding sync (#17479)Erik Johnston2024-07-301-2/+49
| | | | | | | | | | | | | | | | | | | | | | | | Rather than always including all rooms in range. Also adds a pre-filter to rooms that checks the stream change cache to see if anything might have happened. Based on #17447 --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
| * Sliding Sync: Track whether we have sent rooms down to clients (#17447)Erik Johnston2024-07-291-36/+314
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The basic idea is that we introduce a new token for a sliding sync connection, which stores the mapping of room to room "status" (i.e. have we sent the room down?). This token allows us to handle duplicate requests properly. In future it can be used to store more "per-connection" information safely. In future this should be migrated into the DB, so its important that we try to reduce the number of syncs where we need to update the per-connection information. In this PoC this only happens when we: a) send down a set of room for the first time, or b) we have previously sent down a room and there are updates but we are not sending the room down the sync (due to not falling in a list range) Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
| * Sliding Sync: Add Account Data extension (MSC3959) (#17477)Eric Eastwood2024-07-241-0/+138
| | | | | | | | | | | | Extensions based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-241-14/+26
|\|
| * Sliding Sync: ensure bump stamp ignores backfilled events (#17478)Erik Johnston2024-07-241-2/+8
| | | | | | | | | | | | Backfill events have a negative stream ordering, and so its not useful to use to compare with other (positive) stream orderings. Plus, the Rust SDK currently assumes `bump_stamp` is positive.
| * Use a new token format for sliding sync (#17452)Erik Johnston2024-07-241-12/+18
| | | | | | | | | | | | | | This is in preparation for adding per-connection state. --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-235-54/+157
|\|
| * Speed up SS room sorting (#17468)Erik Johnston2024-07-231-22/+21
| | | | | | | | | | | | | | We do this by bulk fetching the latest stream ordering. --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
| * Fixed presence results not returning offline users on initial sync (#17231)Michael Hollister2024-07-231-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is to address an issue in which `m.presence` results on initial sync are not returning entries of users who are currently offline. The original behaviour was from https://github.com/element-hq/synapse/issues/1535 This change is useful for applications that use the presence system for tracking user profile information/updates (e.g. https://github.com/element-hq/synapse/pull/16992 or for profile status messages). This is gated behind a new configuration option to avoid performance impact for applications that don't need this, as a pragmatic solution for now.
| * Sliding Sync: Add E2EE extension (MSC3884) (#17454)Eric Eastwood2024-07-222-19/+105
| | | | | | | | | | Spec: [MSC3884](https://github.com/matrix-org/matrix-spec-proposals/pull/3884) Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * SS: Implement `$ME` support (#17469)Erik Johnston2024-07-221-1/+5
| | | | | | `$ME` can be used as a substitute for the requester's user ID.
| * Speed up room keys query by using read/write lock (#17461)Erik Johnston2024-07-221-9/+9
| | | | | | | | Linaerizing all access slows things down when devices try and fetch lots of keys on login
| * Generate room sync data concurrently (#17458)Erik Johnston2024-07-191-2/+10
| | | | | | This is also what we do for standard `/sync`.
| * Improve default_power_level_content_override documentation (#17451)Ben Banfield-Zanin2024-07-181-0/+2
| | | | | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-181-221/+533
|\|
| * Add `m.room.create` to default bump event types (#17453)Eric Eastwood2024-07-181-0/+1
| | | | | | | | | | | | Add `m.room.create` to default bump event types This probably helps when no messages have been sent in the room and it was just created.
| * Fix bug where sync could get stuck when using workers (#17438)Erik Johnston2024-07-151-2/+9
| | | | | | | | This is because we serialized the token wrong if the instance map contained entries from before the minimum token.
| * Add room subscriptions to Sliding Sync `/sync` (#17432)Eric Eastwood2024-07-151-115/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | Add room subscriptions to Sliding Sync `/sync` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync Currently, you can only subscribe to rooms you have had *any* membership in before. In the future, we will allow `world_readable` rooms to be subscribed to without joining.
| * Add `is_dm` room field to Sliding Sync `/sync` (#17429)Eric Eastwood2024-07-111-29/+46
| | | | | | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * Add `heroes` and room summary fields to Sliding Sync `/sync` (#17419)Eric Eastwood2024-07-111-83/+197
| | | | | | | | | | | | | | Additional room summary fields: `joined_count`, `invited_count` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-111-68/+208
|\|
| * Fix filtering room types on remote rooms (#17434)Erik Johnston2024-07-111-12/+10
| | | | | | | | | | | | We can only fetch room types for rooms the server is in, so we need to only filter rooms that we're joined to. Also includes a perf fix to bulk fetch room types.
| * Handle to-device extensions to Sliding Sync (#17416)Erik Johnston2024-07-101-1/+102
| | | | | | | | | | | | | | Implements MSC3885 --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
| * Add `rooms` `name` and `avatar` to Sliding Sync `/sync` (#17418)Eric Eastwood2024-07-091-55/+96
| | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesQuentin Gliech2024-07-091-19/+56
|\|
| * Add `rooms.bump_stamp` to Sliding Sync `/sync` for easier client-side ↵Eric Eastwood2024-07-081-19/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | sorting (#17395) `bump_stamp` corresponds to the `stream_ordering` of the latest `DEFAULT_BUMP_EVENT_TYPES` in the room. This helps clients sort more readily without them needing to pull in a bunch of the timeline to determine the last activity. `bump_event_types` is a thing because for example, we don't want display name changes to mark the room as unread and bump it to the top. For encrypted rooms, we just have to consider any activity as a bump because we can't see the content and the client has to figure it out for themselves. Outside of Synapse, `bump_stamp` is just a free-form counter so other implementations could use `received_ts`or `origin_server_ts` (see the [*Security considerations* section in MSC3575 about the potential pitfalls of using `origin_server_ts`](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#security-considerations)). It doesn't have any guarantee about always going up. In the Synapse case, it could go down if an event was redacted/removed (or purged in cases of retention policies). In the future, we could add `bump_event_types` as [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) mentions if people need to customize the event types. --- In the Sliding Sync proxy, a similar [`timestamp` field was added](https://github.com/matrix-org/sliding-sync/pull/247) for the same purpose but the name is not obvious what it pertains to or what it's for. The `timestamp` field was also added to Ruma in https://github.com/ruma/ruma/pull/1622
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-07-083-171/+866
|\|
| * Upon deactivation, forget all of the user's rooms (#17400)Travis Ralston2024-07-051-0/+4
| | | | | | | | | | | | This can help ensure that the rooms are eventually purged if the other local users also forget them. Synapse already clears some of the room information as part of the `_background_remove_left_rooms` background task, but this doesn't catch `events`, `event_json`, etc.
| * Add `rooms.required_state` to Sliding Sync `/sync` (#17342)Eric Eastwood2024-07-042-63/+391
| | | | | | Also handles excluding rooms with partial state when people are asking for room membership events unless it's `$LAZY` room membership.
| * Add `room_types`/`not_room_types` filtering to Sliding Sync `/sync` (#17337)Eric Eastwood2024-07-021-5/+30
| | | | | | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * Return some room data in Sliding Sync `/sync` (#17320)Eric Eastwood2024-07-021-152/+490
| | | | | | | | | | | | - Timeline events - Stripped `invite_state` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-06-251-3/+16
|\|
| * Add `is_invite` filtering to Sliding Sync `/sync` (#17335)Eric Eastwood2024-06-241-3/+16
| | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-06-241-0/+11
|\|
| * Add support for MSC3823 - Account Suspension Part 2 (#17255)Shay2024-06-241-0/+11
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-06-1812-226/+791
|\|
| * Add `is_encrypted` filtering to Sliding Sync `/sync` (#17281)Eric Eastwood2024-06-172-99/+36
| | | | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * Add `stream_ordering` sort to Sliding Sync `/sync` (#17293)Eric Eastwood2024-06-173-34/+136
| | | | | | Sort is no longer configurable and we always sort rooms by the `stream_ordering` of the last event in the room or the point where the user can see up to in cases of leave/ban/invite/knock.
| * Include user membership on events (#17282)Richard van der Hoff2024-06-138-18/+0
| | | | | | | | | | MSC4115 has now completed FCP, so we can enable it by default and switch to the stable identifier.
| * Add `is_dm` filtering to Sliding Sync `/sync` (#17277)Eric Eastwood2024-06-131-6/+112
| | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
| * Fix `newly_left` rooms not appearing if we returned early (Sliding Sync) ↵Eric Eastwood2024-06-131-13/+13
| | | | | | | | | | | | | | | | | | | | | | (#17301) Fix `newly_left` rooms not appearing if we returned early when `membership_snapshot_token.is_before_or_eq(to_token.room_key)`. Introduced in https://github.com/element-hq/synapse/pull/17187 (part of Sliding Sync) The tests didn't catch it because they had a small typo in it `room_id1` vs `room_id2`. Found while working on https://github.com/element-hq/synapse/pull/17293
| * Add `event.internal_metadata.instance_name` (#17300)Eric Eastwood2024-06-131-0/+1
| | | | | | | | | | | | | | | | | | | | Add `event.internal_metadata.instance_name` (the worker instance that persisted the event) to go alongside the existing `event.internal_metadata.stream_ordering`. `instance_name` is useful to properly compare and query for events with a token since you need to compare both the `stream_ordering` and `instance_name` against the vector clock/`instance_map` in the `RoomStreamToken`. This is pre-requisite work and may be used in https://github.com/element-hq/synapse/pull/17293 Adding `event.internal_metadata.instance_name` was first mentioned in the initial Sliding Sync PR while pairing with @erikjohnston, see https://github.com/element-hq/synapse/pull/17187/commits/09609cb0dbca3a4cfd9fbf90cc962e765ec469c0#diff-5cd773fb307aa754bd3948871ba118b1ef0303f4d72d42a2d21e38242bf4e096R405-R410
| * Reorganize Pydantic models and types used in handlers (#17279)Eric Eastwood2024-06-103-176/+5
| | | | | | | | | | | | Spawning from https://github.com/element-hq/synapse/pull/17187#discussion_r1619492779 around wanting to put `SlidingSyncBody` (parse the request in the rest layer), `SlidingSyncConfig` (from the rest layer, pass to the handler), `SlidingSyncResponse` (pass the response from the handler back to the rest layer to respond) somewhere that doesn't contaminate the imports and cause circular import issues. - Moved Pydantic parsing models to `synapse/types/rest` - Moved handler types to `synapse/types/handlers`
| * Add debug logging for when room keys are uploaded, including whether they ↵reivilibre2024-06-071-0/+18
| | | | | | | | | | | | | | | | | | | | | | are replacing other room keys. (#17266) Fixes: #17013 Add logging for whether room keys are replaced This is motivated by the Crypto team who need to diagnose crypto issues. The existing opentracing logging is not enough because it is not enabled for all users.
| * Add Sliding Sync `/sync` endpoint (initial implementation) (#17187)Eric Eastwood2024-06-062-8/+621
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync This iteration only focuses on returning the list of room IDs in the sliding window API (without sorting/filtering). Rooms appear in the Sliding sync response based on: - `invite`, `join`, `knock`, `ban` membership events - Kicks (`leave` membership events where `sender` is different from the `user_id`/`state_key`) - `newly_left` (rooms that were left during the given token range, > `from_token` and <= `to_token`) - In order for bans/kicks to not show up, you need to `/forget` those rooms. This doesn't modify the event itself though and only adds the `forgotten` flag to `room_memberships` in Synapse. There isn't a way to tell when a room was forgotten at the moment so we can't factor it into the from/to range. ### Example request `POST http://localhost:8008/_matrix/client/unstable/org.matrix.msc3575/sync` ```json { "lists": { "foo-list": { "ranges": [ [0, 99] ], "sort": [ "by_notification_level", "by_recency", "by_name" ], "required_state": [ ["m.room.join_rules", ""], ["m.room.history_visibility", ""], ["m.space.child", "*"] ], "timeline_limit": 100 } } } ``` Response: ```json { "next_pos": "s58_224_0_13_10_1_1_16_0_1", "lists": { "foo-list": { "count": 1, "ops": [ { "op": "SYNC", "range": [0, 99], "room_ids": [ "!MmgikIyFzsuvtnbvVG:my.synapse.linux.server" ] } ] } }, "rooms": {}, "extensions": {} } ```
| * Handle OTK uploads off master (#17271)Erik Johnston2024-06-061-29/+55
| | | | | | And fallback keys uploads. Only device keys need handling on master
| * Don't try and resync devices for down hosts (#17273)Erik Johnston2024-06-061-6/+18
| | | | | | | | It's just a waste of time if we won't even query the remote host as its marked as down.
| * Always return OTK counts (#17275)Erik Johnston2024-06-061-4/+29
| | | | | | Broke in https://github.com/element-hq/synapse/pull/17215
| * Use fully-qualified `PersistedEventPosition` when returning `RoomsForUser` ↵Eric Eastwood2024-06-045-69/+8
| | | | | | | | | | | | | | (#17265) Use fully-qualified `PersistedEventPosition` (`instance_name` and `stream_ordering`) when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation. Spawning from https://github.com/element-hq/synapse/pull/17187 where we want to utilize this change
* | Merge remote-tracking branch 'origin/release-v1.109' into matrix-org-hotfixesErik Johnston2024-06-111-1/+1
|\ \
| * | Always return OTK counts (#17275)Erik Johnston2024-06-101-4/+29
| |/ | | | | Broke in https://github.com/element-hq/synapse/pull/17215
* | Don't cacheErik Johnston2024-06-061-0/+1
| |
* | Always return OTK countsErik Johnston2024-06-061-4/+28
| |
* | Merge remote-tracking branch 'origin/release-v1.109' into matrix-org-hotfixesAndrew Morgan2024-06-041-3/+3
|\|
| * Fix bug where typing replication breaks (#17252)Erik Johnston2024-05-311-3/+3
| | | | | | | | This can happen on restarts of the service, due to old rooms being pruned.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-311-0/+5
|\|
| * Fix logging errors when receiving invalid User ID for key querys (#17250)Erik Johnston2024-05-311-0/+5
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-302-35/+32
|\|
| * In sync wait for worker to catch up since token (#17215)Erik Johnston2024-05-301-0/+35
| | | | | | | | | | | | | | Otherwise things will get confused. An alternative would be to make sure that for lagging stream we don't return anything (and make sure the returned next_batch token doesn't go backwards). But that is a faff.
| * Fix deduplicating of membership events to not create unused state groups. ↵Erik Johnston2024-05-302-35/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | (#17164) We try and deduplicate in two places: 1) really early on, and 2) just before we persist the event. The first case was broken due to it occuring before the profile information was added, and so it thought the event contents were different. The second case did catch it and handle it correctly, however doing so creates a redundant state group leading to bloat. Fixes #3791
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-296-46/+292
|\|
| * Don't invalidate all `get_relations_for_event` on history purge (#17083)Erik Johnston2024-05-291-1/+1
| | | | | | | | This is a tree cache already, so may as well move the room ID to the front and use that
| * Ignore attempts to send to-device messages to bad users (#17240)Erik Johnston2024-05-291-0/+7
| | | | | | | | | | | | | | | | Currently sending a to-device message to a user ID with a dodgy destination is accepted, but then ends up spamming the logs when we try and send to the destination. An alternative would be to reject the request, but I'm slightly nervous that could break things.
| * Handle duplicate OTK uploads racing (#17241)Erik Johnston2024-05-291-33/+45
| | | | | | Currently this causes one of then to 500.
| * Add Sliding Sync `/sync/e2ee` endpoint for To-Device messages (#17167)Eric Eastwood2024-05-231-10/+237
| | | | | | | | | | | | | | | | | | | | | | This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa. Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866) for why it may not be as useful as we thought. Based on: - https://github.com/matrix-org/matrix-spec-proposals/pull/3575 - https://github.com/matrix-org/matrix-spec-proposals/pull/3885 - https://github.com/matrix-org/matrix-spec-proposals/pull/3884
| * Log exceptions when failing to auto-join new user according to the ↵reivilibre2024-05-221-1/+1
| | | | | | | | | | | | | | `auto_join_rooms` option. (#17176) Would have been useful for tracking down #16878. Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
| * Reduce work of calculating outbound device pokes (#17211)Erik Johnston2024-05-221-0/+7
| |
| * Bring auto-accept invite logic into Synapse (#17147)devonh2024-05-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR ports the logic from the [synapse_auto_accept_invite](https://github.com/matrix-org/synapse-auto-accept-invite) module into synapse. I went with the naive approach of injecting the "module" next to where third party modules are currently loaded. If there is a better/preferred way to handle this, I'm all ears. It wasn't obvious to me if there was a better location to add this logic that would cleanly apply to all incoming invite events. Relies on https://github.com/element-hq/synapse/pull/17166 to fix linter errors.
| * Improve perf of sync device lists (#17216)Erik Johnston2024-05-212-35/+25
| | | | | | | | | | | | | | | | Re-introduces #17191, and includes #17197 and #17214 The basic idea is to stop calling `get_rooms_for_user` everywhere, and instead use the table `device_lists_changes_in_room`. Commits reviewable one-by-one.
* | Merge branch 'erikj/wait_for_stream_pos' into matrix-org-hotfixesOlivier 'reivilibre2024-05-181-0/+35
|\ \
| * | In sync wait for worker to catch up since tokenErik Johnston2024-05-181-0/+35
| |/ | | | | | | Otherwise things will get confused.
* | Merge branch 'erikj/device_list_sync_perf' into matrix-org-hotfixesOlivier 'reivilibre2024-05-182-35/+25
|\ \
| * | Cap the top stream ID when fetching changed devicesErik Johnston2024-05-182-4/+19
| | |
| * | Improve perf of sync device lists (#17191)Erik Johnston2024-05-181-31/+6
| |/ | | | | | | | | | | It's almost always more efficient to query the rooms that have device list changes, rather than looking at the list of all users whose devices have changed and then look for shared rooms.
* | Merge branch 'develop' into matrix-org-hotfixesOlivier 'reivilibre2024-05-181-122/+154
|\|
| * Refactor `SyncResultBuilder` assembly to its own function (#17202)Eric Eastwood2024-05-161-116/+148
| | | | | | | | | | | | We will re-use `get_sync_result_builder(...)` in https://github.com/element-hq/synapse/pull/17167 Split out from https://github.com/element-hq/synapse/pull/17167
| * Fix `joined_rooms`/`joined_room_ids` usage (#17208)Eric Eastwood2024-05-161-1/+1
| | | | | | | | | | | | | | | | This change was introduced in https://github.com/element-hq/synapse/pull/17203 But then https://github.com/element-hq/synapse/pull/17207 was reverted which brought back usage `joined_rooms` that needed to be updated. Wasn't caught because `develop` wasn't up to date before merging.
| * Rename to be obvious: `joined_rooms` -> `joined_room_ids` (#17203)Eric Eastwood2024-05-161-2/+2
| | | | | | Split out from https://github.com/element-hq/synapse/pull/17167
| * Removed `request_key` from the `SyncConfig` (moved outside as its own ↵Eric Eastwood2024-05-161-3/+3
| | | | | | | | | | | | | | | | | | function parameter) (#17201) Removed `request_key` from the `SyncConfig` (moved outside as its own function parameter) so it doesn't have to flow into `_generate_sync_entry_for_xxx` methods. This way we can separate the concerns of caching from generating the response and reuse the `_generate_sync_entry_for_xxx` functions as we see fit. Plus caching doesn't really have anything to do with the config of sync. Split from https://github.com/element-hq/synapse/pull/17167 Spawning from https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279
* | Merge branch 'erikj/redundant_outbound_pokes' into matrix-org-hotfixesErik Johnston2024-05-171-0/+7
|\ \
| * | Reduce work of calculating outbound device pokesErik Johnston2024-05-171-0/+7
| |/
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-161-25/+97
|\|
| * Revert "Improve perf of sync device lists" (#17207)Erik Johnston2024-05-161-6/+31
| | | | | | Reverts element-hq/synapse#17191
| * Fix bug where push rules would be empty in `/sync` (#17142)Erik Johnston2024-05-161-12/+8
| | | | | | | | | | | | Fixes #16987 Some old accounts seem to have an entry in global account data table for push rules, which we should ignore
| * Refactor Sync handler to be able to return different sync responses ↵Eric Eastwood2024-05-161-7/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (`SyncVersion`) (#17200) Refactor Sync handler to be able to be able to return different sync responses (`SyncVersion`). Preparation to be able support sync v2 and a new Sliding Sync `/sync/e2ee` endpoint which returns a subset of sync v2. Split upon request: https://github.com/element-hq/synapse/pull/17167#discussion_r1601497279 Split from https://github.com/element-hq/synapse/pull/17167 where we will add `SyncVersion.E2EE_SYNC` and a new type of sync response.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-142-31/+11
|\|
| * Improve perf of sync device lists (#17191)Erik Johnston2024-05-141-31/+6
| | | | | | | | | | It's almost always more efficient to query the rooms that have device list changes, rather than looking at the list of all users whose devices have changed and then look for shared rooms.
| * Allows CAS SSO flow to provide user IDs composed of numbers only (#17098)Aurélien Grimpard2024-05-141-0/+5
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-131-3/+11
|\|
| * Fix undiscovered linter errors (#17166)devonh2024-05-081-3/+11
| | | | | | | | | | | | Linter errors are showing up in #17147 that are unrelated to that PR. The errors do not currently show up on develop. This PR aims to resolve the linter errors separately from #17147.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-05-0711-7/+83
|\|
| * Add support for MSC3823 - Account Suspension (#17051)Shay2024-05-011-0/+30
| |
| * Apply user `email` & `picture` during OIDC registration if present & ↵devonh2024-04-291-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | selected (#17120) This change will apply the `email` & `picture` provided by OIDC to the new user account when registering a new user via OIDC. If the user is directed to the account details form, this change makes sure they have been selected before applying them, otherwise they are omitted. In particular, this change ensures the values are carried through when Synapse has consent configured, and the redirect to the consent form/s are followed. I have tested everything manually. Including: - with/without consent configured - allowing/not allowing the use of email/avatar (via `sso_auth_account_details.html`) - with/without automatic account detail population (by un/commenting the `localpart_template` option in synapse config). ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
| * Add support for MSC4115 (#17104)Richard van der Hoff2024-04-298-7/+40
| | | | | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
| * Ensure that incoming to-device messages are not dropped (#17127)Richard van der Hoff2024-04-291-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... when workers are unreachable, etc. Fixes https://github.com/element-hq/synapse/issues/17117. The general principle is just to make sure that we propagate any exceptions to the JsonResource, so that we return an error code to the sending server. That means that the sending server no longer considers the message safely sent, so it will retry later. In the issue, Erik mentions that an alternative solution would be to persist the to-device messages into a table so that they can be retried. This might be an improvement for performance, but even if we did that, we still need this mechanism, since we might be unable to reach the database. So, if we want to do that, it can be a later follow-up. --------- Co-authored-by: Erik Johnston <erik@matrix.org>
* | Merge remote-tracking branch 'origin/release-v1.106' into matrix-org-hotfixesOlivier 'reivilibre2024-04-251-1/+12
|\|
| * Redact membership events if the user requested erasure upon deactivating ↵Till2024-04-251-1/+12
| | | | | | | | | | (#17076) Fixes #15355 by redacting all membership events before leaving rooms.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-04-181-0/+36
|\|
| * bugfix: make msc3967 idempotent (#16943)Kegan Dougal2024-04-151-0/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MSC3967 was updated recently to make it more robust to network failures: > there is an existing cross-signing master key and it exactly matches the cross-signing master key provided in the request body. If there are any additional keys provided in the request (self signing key, user signing key) they MUST also match the existing keys stored on the server. In other words, the request contains no new keys. If there are new keys, UIA MUST be performed. https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/device-signing-upload-uia/proposals/3967-device-signing-upload-uia.md#proposal This covers the case where the 200 OK is lost in transit so the client retries the upload, only to then get UIA'd. Complement tests: https://github.com/matrix-org/complement/pull/713 - passing example https://github.com/element-hq/synapse/actions/runs/7976948122/job/21778795094?pr=16943#step:7:8820 ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [x] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [x] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: reivilibre <oliverw@matrix.org>
* | Revert "block non-admins from publishing to room directory"Andrew Morgan2024-04-151-3/+1
| | | | | | | | This reverts commit e34f406cfd1ded69fb5c7d66321631679dac7ff6.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-04-102-7/+14
|\|
| * Also check if first event matches the last in prev batch (#17066)Erik Johnston2024-04-091-7/+13
| | | | | | | | | | Refinement of #17064 cc @richvdh
| * Fix PR #16677, a parameter was missing in a function call (#17033)Mathieu Velten2024-04-091-0/+1
| | | | | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-04-081-0/+3
|\|
| * Fixups to #17064 (#17065)Erik Johnston2024-04-081-0/+3
| | | | | | | | | | Forget a line, and an empty batch is trivially linear. c.f. #17064
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-04-081-0/+36
|\|
| * Add back fast path for non-gappy syncs (#17064)Erik Johnston2024-04-081-0/+36
| | | | | | | | | | | | | | | | | | | | | | PR #16942 removed an invalid optimisation that avoided pulling out state for non-gappy syncs. This causes a large increase in DB usage. c.f. #16941 for why that optimisation was wrong. However, we can still optimise in the simple case where the events in the timeline are a linear chain without any branching/merging of the DAG. cc. @richvdh
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-04-081-91/+139
|\|
| * Fix bug in calculating state for non-gappy syncs (#16942)Richard van der Hoff2024-04-041-54/+37
| | | | | | | | | | | | | | | | | | Unfortunately, the optimisation we applied here for non-gappy syncs is not actually valid. Fixes https://github.com/element-hq/synapse/issues/16941. ~~Based on https://github.com/element-hq/synapse/pull/16930.~~ Requires https://github.com/matrix-org/sytest/pull/1374.
| * `/sync`: fix bug in calculating `state` response (#16930)Richard van der Hoff2024-04-041-41/+13
| | | | | | | | | | | | | | Fix a long-standing issue which could cause state to be omitted from the sync response if the last event was filtered out. Fixes: https://github.com/element-hq/synapse/issues/16928
| * Fix bug in `/sync` response for archived rooms (#16932)Richard van der Hoff2024-04-041-14/+107
| | | | | | | | | | | | | | | | | | | | | | | | This PR fixes a very, very niche edge-case, but I've got some more work coming which will otherwise make the problem worse. The bug happens when the syncing user leaves a room, and has a sync filter which includes "left" rooms, but sets the timeline limit to 0. In that case, the state returned in the `state` section is calculated incorrectly. The fix is to pass a token corresponding to the point that the user leaves the room through to `compute_state_delta`.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-03-281-3/+18
|\|
| * Fixups to new push stream (#17038)Erik Johnston2024-03-281-2/+4
| | | | | | Follow on from #17037
| * Add support for moving `/push_rules` off of main process (#17037)Erik Johnston2024-03-281-3/+16
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-03-2617-232/+406
|\|
| * Add OIDC config to add extra parameters to the authorize URL (#16971)Mathieu Velten2024-03-221-6/+14
| |
| * Do not refuse to set read_marker if previous event_id is in wrong room (#16990)SpiritCroc2024-03-211-2/+2
| |
| * Fix reject knocks on deactivating account (#17010)Hanadi2024-03-211-10/+15
| |
| * OIDC: try to JWT decode userinfo response if JSON parsing failed (#16972)Mathieu Velten2024-03-211-4/+28
| |
| * Update power level default for public rooms (#16907)Shay2024-03-191-1/+1
| |
| * Improve event validation (#16908)Shay2024-03-192-1/+24
| | | | | | As the title states.
| * Pass module API to OIDC mapping provider (#16974)Mathieu Velten2024-03-191-3/+14
| | | | | | | | As done for SAML mapping provider, let's pass the module API to the OIDC one so the mapper can do more logic in its code.
| * `/sync`: Fix edge-case in calculating the "device_lists" response (#16949)Richard van der Hoff2024-03-141-2/+9
| | | | | | | | | | Fixes https://github.com/element-hq/synapse/issues/16948. If the `join` and the `leave` are in the same sync response, we need to count them as a "left" user.
| * Split up `SyncHandler.compute_state_delta` (#16929)Richard van der Hoff2024-03-141-145/+236
| | | | | | | | | | | | This is a huge method, which melts my brain. This is a non-functional change which lays some groundwork for future work in this area.
| * Improve lock performance when a lot of locks are waiting (#16840)Mathieu Velten2024-03-141-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | When a lot of locks are waiting for a single lock, notifying all locks independently with `call_later` on each release is really costly and incurs some kind of async contention, where the CPU is spinning a lot for not much. The included test is taking around 30s before the change, and 0.5s after. It was found following failing tests with https://github.com/element-hq/synapse/pull/16827.
| * Bump mypy from 1.5.1 to 1.8.0 (#16901)dependabot[bot]2024-03-132-3/+3
| |
| * Bump black from 23.10.1 to 24.2.0 (#16936)dependabot[bot]2024-03-1311-49/+51
| |
* | Merge branch 'develop' into matrix-org-hotfixesAndrew Morgan2024-02-201-9/+17
|\|
| * Don't lock up when joining large rooms (#16903)Erik Johnston2024-02-201-9/+17
| | | | | | | | Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-02-131-2/+11
|\|
| * Add a config to not send out device list updates for specific users (#16909)Erik Johnston2024-02-131-2/+11
| | | | | | | | | | | | | | | | | | List of users not to send out device list updates for when they register new devices. This is useful to handle bot accounts. This is undocumented as its mostly a hack to test on matrix.org. Note: This will still send out device list updates if the device is later updated, e.g. end to end keys are added.
* | Merge branch 'release-v1.100' into matrix-org-hotfixesOlivier Wilkinson (reivilibre)2024-01-2347-4/+62
|\|
| * Correctly mention previous copyright (#16820)Erik Johnston2024-01-2347-0/+60
| | | | | | | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
| * Allow room creation but not publishing to continue if room publication rules ↵Shay2024-01-221-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | are violated when creating a new room. (#16811) Prior to this PR, if a request to create a public (public as in published to the rooms directory) room violated the room list publication rules set in the [config](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#room_list_publication_rules), the request to create the room was denied and the room was not created. This PR changes the behavior such that when a request to create a room published to the directory violates room list publication rules, the room is still created but the room is not published to the directory.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes matrix-org-hotfixesErik Johnston2024-01-123-65/+61
|\|
| * Faster load recents for sync (#16783)Erik Johnston2024-01-101-6/+8
| | | | | | This hopefully reduces the amount of state we need to keep in memory
| * Pull less state out if we fail to backfill (#16788)Erik Johnston2024-01-101-9/+12
| | | | | | | | | | | | | | | | | | | | | | Sometimes we fail to fetch events during backfill due to missing state, and we often end up querying the same bad events periodically (as people backpaginate). In such cases its likely we will continue to fail to get the state, and therefore we should try *before* loading the state that we have from the DB (as otherwise it's wasted DB and memory). --------- Co-authored-by: reivilibre <oliverw@matrix.org>
| * Reduce amount of state pulled out when querying federation hierachy (#16785)Erik Johnston2024-01-101-1/+11
| | | | | | | | | | | | | | | | | | | | | | There are two changes here: 1. Only pull out the required state when handling the request. 2. Change the get filtered state return type to check that we're only querying state that was requested --------- Co-authored-by: reivilibre <oliverw@matrix.org>
| * Faster partial join to room with complex auth graph (#7)Erik Johnston2024-01-101-49/+30
| | | | | | | | | | | | | | | | Instead of persisting outliers in a bunch of batches, let's just do them all at once. This is fine because all `_auth_and_persist_outliers_inner` is doing is checking the auth rules for each event, which requires the events to be topologically sorted by the auth graph.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2024-01-0950-578/+964
|\|
| * Filter out rooms from the room directory being served to other homeservers ↵reivilibre2024-01-081-50/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when those rooms block that homeserver by their Access Control Lists. (#16759) The idea here being that the directory server shouldn't advertise rooms to a requesting server is the requesting server would not be allowed to join or participate in the room. <!-- Fixes: # <!-- --> <!-- Supersedes: # <!-- --> <!-- Follows: # <!-- --> <!-- Part of: # <!-- --> Base: `develop` <!-- git-stack-base-branch:develop --> <!-- This pull request is commit-by-commit review friendly. <!-- --> <!-- This pull request is intended for commit-by-commit review. <!-- --> Original commit schedule, with full messages: <ol> <li> Pass `from_federation_origin` down into room list retrieval code </li> <li> Don't cache /publicRooms response for inbound federated requests </li> <li> fixup! Don't cache /publicRooms response for inbound federated requests </li> <li> Cap the number of /publicRooms entries to 100 </li> <li> Simplify code now that you can't request unlimited rooms </li> <li> Filter out rooms from federated requests that don't have the correct ACL </li> <li> Request a handful more when filtering ACLs so that we can try to avoid shortchanging the requester </li> </ol> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
| * Fix linting (#16780)Erik Johnston2024-01-051-1/+11
| | | | | | Introduced in #16762
| * Simplify internal metadata class. (#16762)Erik Johnston2024-01-053-5/+22
| | | | | | | | | | | | | | | | | | We remove these fields as they're just duplicating data the event already stores, and (for reasons :shushing_face:) I'd like to simplify the class to only store simple types. I'm not entirely convinced that we shouldn't instead add helper methods to the event class to generate stream tokens, but I don't really think that's where they belong either
| * Add recursion_depth to /relations if recursing (#16775)David Baker2024-01-041-0/+4
| | | | | | | | | | This is an extra response parameter just added to MSC3981. In the current impl, the recursion depth is always 3, so this just returns a static 3 if the recurse parameter is supplied.
| * Merge remote-tracking branch 'gitlab/clokep/license-license' into new_developErik Johnston2023-12-1350-522/+800
| |\
| | * Update license headersPatrick Cloke2023-11-2150-522/+800
| | |
* | | Merge remote-tracking branch 'shay/shay/freeze_directory' into ↵Erik Johnston2023-12-121-1/+3
|\ \ \ | | | | | | | | | | | | matrix-org-hotfixes
| * | | block non-admins from publishing to room directoryH. Shay2023-12-111-1/+3
| |/ /
* | | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-12-052-3/+9
|\| |
| * | Fix upgrading a room without `events` field in power levels (#16725)David Robertson2023-12-051-1/+1
| | |
| * | Implement MSC4069: Inhibit profile propagation (#16636)Travis Ralston2023-12-041-2/+8
| | | | | | | | | MSC: https://github.com/matrix-org/matrix-spec-proposals/pull/4069
* | | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-12-045-10/+41
|\| |
| * | ModuleAPI SSO auth callbacks (#15207)Andrew Yasinishyn2023-12-012-0/+24
| | | | | | | | | Signed-off-by: Andrii Yasynyshyn yasinishyn.a.n@gmail.com
| * | Drop unused tables & unneeded access token ID for events. (#16522)Patrick Cloke2023-12-011-6/+2
| | |
| * | Reduce DB load when forget on leave setting is disabled (#16668)Erik Johnston2023-11-291-3/+8
| | | | | | | | | | | | | | | * Reduce DB load when forget on leave setting is disabled * Newsfile
| * | Ignore `encryption_enabled_by_default_for_room_type` for notices room (#16677)Mathieu Velten2023-11-281-1/+7
| |/
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-11-201-1/+1
|\|
| * Reduce task concurrency (#16656)Erik Johnston2023-11-171-1/+1
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-11-166-29/+33
|\|
| * Speed up deleting device messages (#16643)Erik Johnston2023-11-161-3/+5
| | | | | | Keeping track of a lower bound of stream ID where we've deleted everything below makes the queries much faster. Otherwise, every time we scan for rows to delete we'd re-scan across all the rows that have previously deleted (until the next table VACUUM).
| * Speed up persisting large number of outliers (#16649)Erik Johnston2023-11-161-11/+7
| | | | | | Recalculating the roots tuple every iteration could be very expensive, so instead let's do a topological sort.
| * Add an Admin API to temporarily grant the ability to update an existing ↵David Robertson2023-11-151-7/+13
| | | | | | | | cross-signing key without UIA (#16634)
| * Use full GitHub links instead of bare issue numbers. (#16637)Patrick Cloke2023-11-154-8/+8
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesPatrick Cloke2023-11-094-11/+15
|\|
| * Convert simple_select_one_txn and simple_select_one to return tuples. (#16612)Patrick Cloke2023-11-092-4/+5
| |
| * Return attrs for more media repo APIs. (#16611)Patrick Cloke2023-11-092-7/+10
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesPatrick Cloke2023-11-035-36/+44
|\|
| * Remove remaining usage of cursor_to_dict. (#16564)Patrick Cloke2023-10-313-36/+35
| |
| * Merge pull request from GHSA-mp92-3jfm-3575Patrick Cloke2023-10-312-0/+9
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-10-311-0/+10
|\|
| * Claim local one-time-keys in bulk (#16565)David Robertson2023-10-301-0/+10
| | | | | | | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-10-301-0/+14
|\|
| * Claim fallback keys in bulk (#16570)David Robertson2023-10-301-0/+14
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-10-301-3/+4
|\|
| * Ensure local invited & knocking users leave before purge. (#16559)Patrick Cloke2023-10-271-3/+4
| | | | | | | | | | This is mostly useful for federated rooms where some users would get stuck in the invite or knock state when the room was purged from their homeserver.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesPatrick Cloke2023-10-2715-123/+227
|\|
| * Fix cross-worker ratelimiting (#16558)Erik Johnston2023-10-271-16/+57
| | | | | | c.f. #16481
| * Add new module API for adding custom fields to events `unsigned` section ↵Erik Johnston2023-10-276-18/+20
| | | | | | | | (#16549)
| * Remove more usages of cursor_to_dict. (#16551)Patrick Cloke2023-10-262-12/+12
| | | | | | Mostly to improve type safety.
| * Add a new module API to update user presence state. (#16544)Patrick Cloke2023-10-263-33/+49
| | | | | | | | | | | | | | | | | | | | This adds a module API which allows a module to update a user's presence state/status message. This is useful for controlling presence from an external system. To fully control presence from the module the presence.enabled config parameter gains a new state of "untracked" which disables internal tracking of presence changes via user actions, etc. Only updates from the module will be persisted and sent down sync properly).
| * Convert simple_select_list and simple_select_list_txn to return lists of ↵Patrick Cloke2023-10-262-6/+3
| | | | | | | | | | tuples (#16505) This should use fewer allocations and improves type hints.
| * Allow multiple workers to write to receipts stream. (#16432)Erik Johnston2023-10-254-30/+40
| | | | | | Fixes #16417
| * Mark sync as limited if there is a gap in the timeline (#16485)Erik Johnston2023-10-191-8/+44
| | | | | | | | | | | | | | | | This splits thinsg into two queries, but most of the time we won't have new event backwards extremities so this shouldn't actually add an extra RTT for the majority of cases. Note this removes the check for events with no prev events, but that was part of MSC2716 work that has since been removed.
| * Improve performance of delete device messages query (#16492)Mathieu Velten2023-10-181-0/+2
| |
* | Merge branch 'release-v1.95' into matrix-org-hotfixesPatrick Cloke2023-10-1716-185/+133
|\|
| * Convert DeviceLastConnectionInfo to attrs. (#16507)Patrick Cloke2023-10-171-16/+7
| | | | | | To improve type safety & memory usage.
| * Convert state delta processing from a dict to attrs. (#16469)Patrick Cloke2023-10-164-78/+73
| | | | | | For improved type checking & memory usage.
| * Remove useless async job to delete device messages on sync (#16491)Mathieu Velten2023-10-161-22/+0
| |
| * Revert "Drop unused tables & unneeded access token ID for events. (#16268)" ↵Patrick Cloke2023-10-121-2/+6
| | | | | | | | | | | | | | | | (#16465) This reverts commit cabd57746004fe2dacc11aa8d373854a3d25e306. There are additional usages of these tables which need to be removed first.
| * Convert user_get_threepids response to attrs. (#16468)Patrick Cloke2023-10-113-5/+7
| | | | | | This improves type annotations by not having a dictionary of Any values.
| * Bump ruff from 0.0.290 to 0.0.292 (#16449)dependabot[bot]2023-10-095-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump ruff from 0.0.290 to 0.0.292 Bumps [ruff](https://github.com/astral-sh/ruff) from 0.0.290 to 0.0.292. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/BREAKING_CHANGES.md) - [Commits](https://github.com/astral-sh/ruff/compare/v0.0.290...v0.0.292) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Fix up lint --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
| * Apply join rate limiter outside the lineariser (#16441)David Robertson2023-10-061-20/+23
| |
| * Drop unused tables & unneeded access token ID for events. (#16268)Patrick Cloke2023-10-061-6/+2
| | | | | | | | Drop the event_txn_id table and the tables related to MSC2716, which is no longer supported in Synapse.
| * Stop sending incorrect knock_state_events. (#16403)Patrick Cloke2023-10-061-11/+2
| | | | | | | | | | | | | | | | | | Synapse was incorrectly implemented with a knock_state_events property on some APIs (instead of knock_room_state). This was correct in Synapse 1.70.0, but *both* fields were sent to also be compatible with Synapse versions expecting the wrong field. Enough time has passed that only the correct field needs to be included/handled.
| * Factor out `MultiWriter` token from `RoomStreamToken` (#16427)Erik Johnston2023-10-054-6/+5
| |
| * Some refactors around receipts stream (#16426)Erik Johnston2023-10-043-19/+16
| |
* | Merge remote-tracking branch 'origin/release-v1.94' into matrix-org-hotfixesDavid Robertson2023-10-031-2/+2
|\|
| * mypy plugin to check `@cached` return types (#14911)David Robertson2023-10-021-2/+2
| | | | | | | | | | | | | | | | Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Patrick Cloke <patrickc@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org> Assert that the return type of callables wrapped in @cached and @cachedList are cachable (aka immutable).
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesPatrick Cloke2023-10-0210-51/+79
|\|
| * Add a cache around server ACL checking (#16360)Patrick Cloke2023-09-262-0/+11
| | | | | | | | | | * Pre-compiles the server ACLs onto an object per room and invalidates them when new events come in. * Converts the server ACL checking into Rust.
| * Reduce calls to `send_presence_to_destinations` (#16385)Erik Johnston2023-09-261-15/+18
| |
| * Convert more cached return values to immutable types (#16356)Patrick Cloke2023-09-202-3/+13
| |
| * Return immutable objects for cachedList decorators (#16350)Patrick Cloke2023-09-196-33/+37
| |
* | Merge remote-tracking branch 'origin/release-v1.93' into matrix-org-hotfixesPatrick Cloke2023-09-198-419/+335
|\|
| * Make cached account data/tags/admin types immutable (#16325)Patrick Cloke2023-09-182-20/+25
| |
| * Return an immutable value from get_latest_event_ids_in_room. (#16326)Patrick Cloke2023-09-181-5/+3
| |
| * Additional validation of receipts (#16327)Patrick Cloke2023-09-181-2/+24
| | | | | | | | Reject invalid receipts with a reasonable error message & expands tests for receipts.
| * Set email charset as utf-8 rather than utf8 (#16329)José Joaquín Atria2023-09-181-2/+2
| |
| * Add automatic purge after all users forget a room (#15488)Mathieu Velten2023-09-153-390/+281
| | | | | | | | | | | | Also add restore of purge/shutdown rooms after a synapse restart. Co-authored-by: Eric Eastwood <erice@matrix.org> Co-authored-by: Erik Johnston <erikj@matrix.org>
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-09-144-47/+44
|\|
| * Speed up deleting to-device messages task (#16318)Erik Johnston2023-09-141-13/+14
| |
| * Refactor `get_user_by_id` (#16316)Erik Johnston2023-09-143-31/+26
| |
| * Fix using dehydrated devices (MSC2697) & refresh tokens (#16288)Hanadi2023-09-131-3/+4
| | | | | | | | | | Refresh tokens were not correctly moved to the rehydrated device (similar to how the access token is currently handled). This resulted in invalid refresh tokens after rehydration.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-09-131-11/+26
|\|
| * Don't schedule an async task on every sync (#16312)Erik Johnston2023-09-131-11/+26
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-09-131-31/+31
|\|
| * Fix deleting device inbox when using background worker (#16311)Erik Johnston2023-09-131-31/+31
| | | | | | | | | | Introduced in #16240 The action for the task was only defined on the "master" handler, rather than the base worker one.
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesErik Johnston2023-09-138-70/+372
|\|
| * Add the List-Unsubscribe header for notification emails. (#16274)Patrick Cloke2023-09-111-1/+9
| | | | | | | | | | | | | | | | Adds both the List-Unsubscribe (RFC2369) and List-Unsubscribe-Post (RFC8058) headers to push notification emails, which together should: * Show an "Unsubscribe" link in the MUA UI when viewing Synapse notification emails. * Enable "one-click" unsubscribe (the user never leaves their MUA, which automatically makes a POST request to the specified endpoint).
| * Filter out down hosts when retrying fetching device lists (#16298)Erik Johnston2023-09-111-1/+14
| |
| * Recheck if remote device is cached before requesting it (#16252)Erik Johnston2023-09-071-6/+15
| | | | | | | | This fixes a bug where we could get stuck re-requesting the device over replication again and again.
| * Handle "registration_enabled" parameter for CAS (#16262)Aurélien Grimpard2023-09-061-0/+2
| | | | | | | | Similar to OIDC, CAS providers can now disable registration such that only existing users are able to login via SSO.
| * Allow modules to delete rooms. (#15997)reivilibre2023-09-062-3/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Allow user_id to be optional for room deletion * Add module API method to delete a room * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Don't worry about the case block=True && requester_user_id is None --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
| * Delete device messages asynchronously and in staged batches (#16240)Mathieu Velten2023-09-063-6/+62
| |
| * Time out busy presence status & test multi-device busy (#16174)Patrick Cloke2023-09-051-1/+18
| | | | | | | | | | | | | | Add a (long) timeout to when a "busy" device is considered not online. This does *not* match MSC3026, but is a reasonable thing for an implementation to do. Expands tests for the (unstable) busy presence with multiple devices.
| * Track presence state per-device and combine to a user state. (#16066)Patrick Cloke2023-09-051-47/+232
| | | | | | | | | | | | | | | | | | | | | | Tracks presence on an individual per-device basis and combine the per-device state into a per-user state. This should help in situations where a user has multiple devices with conflicting status (e.g. one is syncing with unavailable and one is syncing with online). The tie-breaking is done by priority: BUSY > ONLINE > UNAVAILABLE > OFFLINE
| * Bump twisted from 22.10.0 to 23.8.0 (#16235)dependabot[bot]2023-09-051-6/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump twisted from 22.10.0 to 23.8.0 Bumps [twisted](https://github.com/twisted/twisted) from 22.10.0 to 23.8.0. - [Release notes](https://github.com/twisted/twisted/releases) - [Changelog](https://github.com/twisted/twisted/blob/trunk/NEWS.rst) - [Commits](https://github.com/twisted/twisted/compare/twisted-22.10.0...twisted-23.8.0) --- updated-dependencies: - dependency-name: twisted dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Fix types * Fix lint * Newsfile --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
* | Merge remote-tracking branch 'origin/release-v1.92' into matrix-org-hotfixesDavid Robertson2023-09-0513-253/+289
|\|
| * Don't wake up destination transaction queue if they're not due for retry. ↵Erik Johnston2023-09-044-27/+36
| | | | | | | | (#16223)
| * Add last_seen_ts to the admin users API (#16218)Mathieu Velten2023-09-041-0/+1
| |
| * Stop writing to the event_txn_id table (#16175)Patrick Cloke2023-08-301-13/+0
| |
| * Describe which rate limiter was hit in logs (#16135)David Robertson2023-08-305-26/+17
| |
| * Track currently syncing users by device for presence (#16172)Patrick Cloke2023-08-291-55/+100
| | | | | | | | | | | | | | Refactoring to use both the user ID & the device ID when tracking the currently syncing users in the presence handler. This is done both locally and over replication. Note that the device ID is discarded but will be used in a future change.
| * Bump ruff from 0.0.277 to 0.0.286 (#16198)dependabot[bot]2023-08-291-1/+1
| | | | | | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
| * Support IPv6-only SMTP servers (#16155)Evilham2023-08-291-17/+11
| | | | | | | | Use Twisted HostnameEndpoint to connect to SMTP servers (instead of connectTCP/connectSSL) which properly supports IPv6-only servers.
| * Pass the device ID around in the presence handler (#16171)Patrick Cloke2023-08-283-11/+45
| | | | | | | | | | | | Refactoring to pass the device ID (in addition to the user ID) through the presence handler (specifically the `user_syncing`, `set_state`, and `bump_presence_active_time` methods and their replication versions).
| * Combine logic about not overriding BUSY presence. (#16170)Patrick Cloke2023-08-281-92/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | Simplify some of the presence code by reducing duplicated code between worker & non-worker modes. The main change is to push some of the logic from `user_syncing` into `set_state`. This is done by passing whether the user is setting the presence via a `/sync` with a new `is_sync` flag to `set_state`. If this is `true` some additional logic is performed: * Don't override `busy` presence. * Update the `last_user_sync_ts`. * Never update the status message.
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-07-16 12:47:18 +0000