summary refs log tree commit diff
path: root/tests/rest/client/test_sync.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Sliding Sync: Split and move tests (#17504)Eric Eastwood2024-07-311-5860/+3
| | | | Split and move Sliding Sync tests so we have some more sane test file sizes
* Sliding Sync: Update filters to be robust against remote invite rooms (#17450)Eric Eastwood2024-07-301-0/+145
| | | | | | | | | | | | | | | Update `filters.is_encrypted` and `filters.types`/`filters.not_types` to be robust when dealing with remote invite rooms in Sliding Sync. Part of [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync Follow-up to https://github.com/element-hq/synapse/pull/17434 We now take into account current state, fallback to stripped state for invite/knock rooms, then historical state. If we can't determine the info needed to filter a room (either from state or stripped state), it is filtered out.
* Sliding Sync: Add receipts extension (MSC3960) (#17489)Eric Eastwood2024-07-301-205/+816
| | | | | | | [MSC3960](https://github.com/matrix-org/matrix-spec-proposals/pull/3960): Receipts extension Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Only send rooms with updates down sliding sync (#17479)Erik Johnston2024-07-301-27/+62
| | | | | | | | | | | | Rather than always including all rooms in range. Also adds a pre-filter to rooms that checks the stream change cache to see if anything might have happened. Based on #17447 --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
* Sliding Sync: Track whether we have sent rooms down to clients (#17447)Erik Johnston2024-07-291-3/+449
| | | | | | | | | | | | | | | | The basic idea is that we introduce a new token for a sliding sync connection, which stores the mapping of room to room "status" (i.e. have we sent the room down?). This token allows us to handle duplicate requests properly. In future it can be used to store more "per-connection" information safely. In future this should be migrated into the DB, so its important that we try to reduce the number of syncs where we need to update the per-connection information. In this PoC this only happens when we: a) send down a set of room for the first time, or b) we have previously sent down a room and there are updates but we are not sending the room down the sync (due to not falling in a list range) Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
* Refactor Sliding Sync tests to better utilize the ↵Eric Eastwood2024-07-251-1021/+725
| | | | | | | | `SlidingSyncBase.do_sync(...)` (pt. 2) (#17482) `SlidingSyncBase.do_sync()` for tests was first introduced in https://github.com/element-hq/synapse/pull/17452 Part 1: https://github.com/element-hq/synapse/pull/17481
* Refactor Sliding Sync tests to better utilize the `SlidingSyncBase` (pt. 1) ↵Eric Eastwood2024-07-251-228/+106
| | | | | | | | (#17481) `SlidingSyncBase` for tests was first introduced in https://github.com/element-hq/synapse/pull/17452 Part 2: https://github.com/element-hq/synapse/pull/17482
* Sliding Sync: Add Account Data extension (MSC3959) (#17477)Eric Eastwood2024-07-241-0/+786
| | | | | | Extensions based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Sliding Sync: ensure bump stamp ignores backfilled events (#17478)Erik Johnston2024-07-241-1/+121
| | | | | | Backfill events have a negative stream ordering, and so its not useful to use to compare with other (positive) stream orderings. Plus, the Rust SDK currently assumes `bump_stamp` is positive.
* Use a new token format for sliding sync (#17452)Erik Johnston2024-07-241-191/+225
| | | | | | | This is in preparation for adding per-connection state. --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
* Sliding Sync: Add E2EE extension (MSC3884) (#17454)Eric Eastwood2024-07-221-4/+821
| | | | | Spec: [MSC3884](https://github.com/matrix-org/matrix-spec-proposals/pull/3884) Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* SS: Implement `$ME` support (#17469)Erik Johnston2024-07-221-0/+74
| | | `$ME` can be used as a substitute for the requester's user ID.
* Order `heroes` by `stream_ordering` (as spec'ed) (#17435)Eric Eastwood2024-07-171-12/+9
| | | | | | | | | | | | | The spec specifically mentions `stream_ordering` but that's a Synapse specific concept. In any case, the essence of the spec is basically the first 5 members of the room which `stream_ordering` accomplishes. Split off from https://github.com/element-hq/synapse/pull/17419#discussion_r1671342794 ## Spec compliance > This should be the first 5 members of the room, **ordered by stream ordering**, which are joined or invited. The list must never include the client’s own user ID. When no joined or invited members are available, this should consist of the banned and left users. > > *-- https://spec.matrix.org/v1.10/client-server-api/#_matrixclientv3sync_roomsummary* Related to https://github.com/matrix-org/matrix-spec/issues/1334
* Add room subscriptions to Sliding Sync `/sync` (#17432)Eric Eastwood2024-07-151-63/+284
| | | | | | | | | | | | | Add room subscriptions to Sliding Sync `/sync` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync Currently, you can only subscribe to rooms you have had *any* membership in before. In the future, we will allow `world_readable` rooms to be subscribed to without joining.
* Add `is_dm` room field to Sliding Sync `/sync` (#17429)Eric Eastwood2024-07-111-0/+23
| | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Add `heroes` and room summary fields to Sliding Sync `/sync` (#17419)Eric Eastwood2024-07-111-12/+292
| | | | | | | Additional room summary fields: `joined_count`, `invited_count` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Handle to-device extensions to Sliding Sync (#17416)Erik Johnston2024-07-101-2/+198
| | | | | | | Implements MSC3885 --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>
* Add `rooms` `name` and `avatar` to Sliding Sync `/sync` (#17418)Eric Eastwood2024-07-091-0/+208
| | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Add `rooms.bump_stamp` to Sliding Sync `/sync` for easier client-side ↵Eric Eastwood2024-07-081-0/+96
| | | | | | | | | | | | | | | sorting (#17395) `bump_stamp` corresponds to the `stream_ordering` of the latest `DEFAULT_BUMP_EVENT_TYPES` in the room. This helps clients sort more readily without them needing to pull in a bunch of the timeline to determine the last activity. `bump_event_types` is a thing because for example, we don't want display name changes to mark the room as unread and bump it to the top. For encrypted rooms, we just have to consider any activity as a bump because we can't see the content and the client has to figure it out for themselves. Outside of Synapse, `bump_stamp` is just a free-form counter so other implementations could use `received_ts`or `origin_server_ts` (see the [*Security considerations* section in MSC3575 about the potential pitfalls of using `origin_server_ts`](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#security-considerations)). It doesn't have any guarantee about always going up. In the Synapse case, it could go down if an event was redacted/removed (or purged in cases of retention policies). In the future, we could add `bump_event_types` as [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) mentions if people need to customize the event types. --- In the Sliding Sync proxy, a similar [`timestamp` field was added](https://github.com/matrix-org/sliding-sync/pull/247) for the same purpose but the name is not obvious what it pertains to or what it's for. The `timestamp` field was also added to Ruma in https://github.com/ruma/ruma/pull/1622
* Add `rooms.required_state` to Sliding Sync `/sync` (#17342)Eric Eastwood2024-07-041-1/+712
| | | Also handles excluding rooms with partial state when people are asking for room membership events unless it's `$LAZY` room membership.
* Return some room data in Sliding Sync `/sync` (#17320)Eric Eastwood2024-07-021-8/+1071
| | | | | | - Timeline events - Stripped `invite_state` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Fix sync waiting for an invalid token from the "future" (#17386)Erik Johnston2024-07-021-1/+3
| | | | | | | | | | | | | Fixes https://github.com/element-hq/synapse/issues/17274, hopefully. Basically, old versions of Synapse could advance streams without persisting anything in the DB (fixed in #17229). On restart those updates would get lost, and so the position of the stream would revert to an older position. If this happened across an upgrade to a later Synapse version which included #17215, then sync could get blocked indefinitely (until the stream advanced to the position in the token). We fix this by bounding the stream positions we'll wait for to the maximum position of the underlying stream ID generator.
* Add `is_invite` filtering to Sliding Sync `/sync` (#17335)Eric Eastwood2024-06-241-25/+123
| | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Register sliding sync under a different path (#17331)Erik Johnston2024-06-191-1/+3
| | | As the API is slightly incompatible.
* Add `stream_ordering` sort to Sliding Sync `/sync` (#17293)Eric Eastwood2024-06-171-4/+57
| | | Sort is no longer configurable and we always sort rooms by the `stream_ordering` of the last event in the room or the point where the user can see up to in cases of leave/ban/invite/knock.
* Add `is_dm` filtering to Sliding Sync `/sync` (#17277)Eric Eastwood2024-06-131-0/+127
| | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Add Sliding Sync `/sync` endpoint (initial implementation) (#17187)Eric Eastwood2024-06-061-1/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync This iteration only focuses on returning the list of room IDs in the sliding window API (without sorting/filtering). Rooms appear in the Sliding sync response based on: - `invite`, `join`, `knock`, `ban` membership events - Kicks (`leave` membership events where `sender` is different from the `user_id`/`state_key`) - `newly_left` (rooms that were left during the given token range, > `from_token` and <= `to_token`) - In order for bans/kicks to not show up, you need to `/forget` those rooms. This doesn't modify the event itself though and only adds the `forgotten` flag to `room_memberships` in Synapse. There isn't a way to tell when a room was forgotten at the moment so we can't factor it into the from/to range. ### Example request `POST http://localhost:8008/_matrix/client/unstable/org.matrix.msc3575/sync` ```json { "lists": { "foo-list": { "ranges": [ [0, 99] ], "sort": [ "by_notification_level", "by_recency", "by_name" ], "required_state": [ ["m.room.join_rules", ""], ["m.room.history_visibility", ""], ["m.space.child", "*"] ], "timeline_limit": 100 } } } ``` Response: ```json { "next_pos": "s58_224_0_13_10_1_1_16_0_1", "lists": { "foo-list": { "count": 1, "ops": [ { "op": "SYNC", "range": [0, 99], "room_ids": [ "!MmgikIyFzsuvtnbvVG:my.synapse.linux.server" ] } ] } }, "rooms": {}, "extensions": {} } ```
* Add Sliding Sync `/sync/e2ee` endpoint for To-Device messages (#17167)Eric Eastwood2024-05-231-10/+389
| | | | | | | | | | | This is being introduced as part of Sliding Sync but doesn't have any sliding window component. It's just a way to get E2EE events without having to sit through a big initial sync (`/sync` v2). And we can avoid encryption events being backed up by the main sync response or vice-versa. Part of some Sliding Sync simplification/experimentation. See [this discussion](https://github.com/element-hq/synapse/pull/17167#discussion_r1610495866) for why it may not be as useful as we thought. Based on: - https://github.com/matrix-org/matrix-spec-proposals/pull/3575 - https://github.com/matrix-org/matrix-spec-proposals/pull/3885 - https://github.com/matrix-org/matrix-spec-proposals/pull/3884
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Update license headersPatrick Cloke2023-11-211-11/+16
|
* Use full GitHub links instead of bare issue numbers. (#16637)Patrick Cloke2023-11-151-1/+1
|
* Additional validation of receipts (#16327)Patrick Cloke2023-09-181-153/+1
| | | | Reject invalid receipts with a reasonable error message & expands tests for receipts.
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-221-3/+0
|
* Fix type hints in knocking tests. (#14887)Andrew Morgan2023-01-251-3/+1
|
* Faster joins: omit partial rooms from eager syncs until the resync completes ↵David Robertson2023-01-231-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#14870) * Allow `AbstractSet` in `StrCollection` Or else frozensets are excluded. This will be useful in an upcoming commit where I plan to change a function that accepts `List[str]` to accept `StrCollection` instead. * `rooms_to_exclude` -> `rooms_to_exclude_globally` I am about to make use of this exclusion mechanism to exclude rooms for a specific user and a specific sync. This rename helps to clarify the distinction between the global config and the rooms to exclude for a specific sync. * Better function names for internal sync methods * Track a list of excluded rooms on SyncResultBuilder I plan to feed a list of partially stated rooms for this sync to ignore * Exclude partial state rooms during eager sync using the mechanism established in the previous commit * Track un-partial-state stream in sync tokens So that we can work out which rooms have become fully-stated during a given sync period. * Fix mutation of `@cached` return value This was fouling up a complement test added alongside this PR. Excluding a room would mean the set of forgotten rooms in the cache would be extended. This means that room could be erroneously considered forgotten in the future. Introduced in #12310, Synapse 1.57.0. I don't think this had any user-visible side effects (until now). * SyncResultBuilder: track rooms to force as newly joined Similar plan as before. We've omitted rooms from certain sync responses; now we establish the mechanism to reintroduce them into future syncs. * Read new field, to present rooms as newly joined * Force un-partial-stated rooms to be newly-joined for eager incremental syncs only, provided they're still fully stated * Notify user stream listeners to wake up long polling syncs * Changelog * Typo fix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Unnecessary list cast Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Rephrase comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Another comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Fixup merge(?) * Poke notifier when receiving un-partial-stated msg over replication * Fixup merge whoops Thanks MV :) Co-authored-by: Mathieu Velen <mathieuv@matrix.org> Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Remove support for unstable private read receipts (#13653)Šimon Brandner2022-09-011-26/+11
| | | Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
* Support stable identifiers for MSC2285: private read receipts. (#13273)Šimon Brandner2022-08-051-19/+39
| | | | | This adds support for the stable identifiers of MSC2285 while continuing to support the unstable identifiers behind the configuration flag. These will be removed in a future version.
* Fix rooms not being properly excluded from incremental sync (#13408)Brendan Abolivier2022-08-041-0/+21
|
* Remove unnecessary `json.dumps` from tests (#13303)Dirk Klimpel2022-07-171-2/+1
|
* Additional constants for EDU types. (#12884)Patrick Cloke2022-05-271-1/+2
| | | Instead of hard-coding strings in many places.
* Consolidate logic for parsing relations. (#12693)Patrick Cloke2022-05-161-2/+6
| | | | | | | | | | | | | Parse the `m.relates_to` event content field (which describes relations) in a single place, this is used during: * Event persistence. * Validation of the Client-Server API. * Fetching bundled aggregations. * Processing of push rules. Each of these separately implement the logic and each made slightly different assumptions about what was valid. Some had minor / potential bugs.
* No longer permit empty body when sending receipts (#12709)David Robertson2022-05-111-26/+4
|
* Use `private` instead of `hidden` in MSC2285 related code. (#12635)Šimon Brandner2022-05-051-2/+2
|
* Implement changes to MSC2285 (hidden read receipts) (#12168)Šimon Brandner2022-05-041-17/+144
| | | | | * Changes hidden read receipts to be a separate receipt type (instead of a field on `m.read`). * Updates the `/receipts` endpoint to accept `m.fully_read`.
* Use constants for receipt types in tests. (#12582)Šimon Brandner2022-04-281-1/+2
|
* Move MSC2654 support behind an experimental configuration flag. (#12295)Patrick Cloke2022-03-311-0/+5
| | | To match the current thinking on disabling experimental features by default.
* Remove references to "msc2403" (#12165)Andrew Morgan2022-03-301-1/+0
|
* Add a configuration to exclude rooms from sync response (#12310)Brendan Abolivier2022-03-301-0/+62
|
* Replace assertEquals and friends with non-deprecated versions. (#12092)Patrick Cloke2022-02-281-8/+8
|
* Add type hints to `tests/rest/client` (#12066)Dirk Klimpel2022-02-231-20/+27
|
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Add a unit test for users receiving their own device list updates (#11909)Andrew Morgan2022-02-071-1/+56
|
* Require body for read receipts with user-agent exceptions (#11157)rogersheu2021-11-091-2/+28
| | | Co-authored-by: reivilibre <olivier@librepush.net>
* Flatten tests/rest/client/{v1,v2_alpha} too (#10667)David Robertson2021-08-201-0/+686
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-05 19:43:02 +0000