summary refs log tree commit diff
path: root/tests/storage/test_stream.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Sliding Sync: Avoid fetching left rooms and add back `newly_left` rooms (#17725)Eric Eastwood2024-09-191-2/+83
| | | | | | | | | | | | | | | Performance optimization: We can avoid fetching rooms that the user has left themselves (which could be a significant amount), then only add back rooms that the user has `newly_left` (left in the token range of an incremental sync). It's a lot faster to fetch less rooms than fetch them all and throw them away in most cases. Since the user only leaves a room (or is state reset out) once in a blue moon, we can avoid a lot of work. Based on @erikjohnston's branch, erikj/ss_perf --------- Co-authored-by: Erik Johnston <erik@matrix.org>
* Sliding Sync: Retrieve fewer events from DB in sync (#17688)Erik Johnston2024-09-101-1/+1
| | | | When using timeline limit of 1 we end up fetching 2 events from the DB purely to tell if the response was "limited" or not. Lets not do that.
* Sliding Sync: Use `stream_ordering` based timeline pagination for ↵Eric Eastwood2024-08-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | incremental sync (#17510) Use `stream_ordering` based `timeline` pagination for incremental `/sync` in Sliding Sync. Previously, we were always using a `topological_ordering` but we should only be using that for historical scenarios (initial `/sync`, newly joined, or haven't sent the room down the connection before). This is slightly different than what the [spec suggests](https://spec.matrix.org/v1.10/client-server-api/#syncing) > Events are ordered in this API according to the arrival time of the event on the homeserver. This can conflict with other APIs which order events based on their partial ordering in the event graph. This can result in duplicate events being received (once per distinct API called). Clients SHOULD de-duplicate events based on the event ID when this happens. But we've had a [discussion below in this PR](https://github.com/element-hq/synapse/pull/17510#discussion_r1699105569) and this matches what Sync v2 already does and seems like it makes sense. Created a spec issue https://github.com/matrix-org/matrix-spec/issues/1917 to clarify this. Related issues: - https://github.com/matrix-org/matrix-spec/issues/1917 - https://github.com/matrix-org/matrix-spec/issues/852 - https://github.com/matrix-org/matrix-spec-proposals/pull/4033
* Add `rooms.bump_stamp` to Sliding Sync `/sync` for easier client-side ↵Eric Eastwood2024-07-081-0/+41
| | | | | | | | | | | | | | | sorting (#17395) `bump_stamp` corresponds to the `stream_ordering` of the latest `DEFAULT_BUMP_EVENT_TYPES` in the room. This helps clients sort more readily without them needing to pull in a bunch of the timeline to determine the last activity. `bump_event_types` is a thing because for example, we don't want display name changes to mark the room as unread and bump it to the top. For encrypted rooms, we just have to consider any activity as a bump because we can't see the content and the client has to figure it out for themselves. Outside of Synapse, `bump_stamp` is just a free-form counter so other implementations could use `received_ts`or `origin_server_ts` (see the [*Security considerations* section in MSC3575 about the potential pitfalls of using `origin_server_ts`](https://github.com/matrix-org/matrix-spec-proposals/blob/kegan/sync-v3/proposals/3575-sync.md#security-considerations)). It doesn't have any guarantee about always going up. In the Synapse case, it could go down if an event was redacted/removed (or purged in cases of retention policies). In the future, we could add `bump_event_types` as [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575) mentions if people need to customize the event types. --- In the Sliding Sync proxy, a similar [`timestamp` field was added](https://github.com/matrix-org/sliding-sync/pull/247) for the same purpose but the name is not obvious what it pertains to or what it's for. The `timestamp` field was also added to Ruma in https://github.com/ruma/ruma/pull/1622
* Return some room data in Sliding Sync `/sync` (#17320)Eric Eastwood2024-07-021-3/+871
| | | | | | - Timeline events - Stripped `invite_state` Based on [MSC3575](https://github.com/matrix-org/matrix-spec-proposals/pull/3575): Sliding Sync
* Add `stream_ordering` sort to Sliding Sync `/sync` (#17293)Eric Eastwood2024-06-171-23/+33
| | | Sort is no longer configurable and we always sort rooms by the `stream_ordering` of the last event in the room or the point where the user can see up to in cases of leave/ban/invite/knock.
* Fix `get_last_event_in_room_before_stream_ordering(...)` finding the wrong ↵Eric Eastwood2024-06-131-2/+267
| | | | | | | | | | | last event (#17295) PR where this was introduced: https://github.com/matrix-org/synapse/pull/14817 ### What does this affect? `get_last_event_in_room_before_stream_ordering(...)` is used in Sync v2 in a lot of different state calculations. `get_last_event_in_room_before_stream_ordering(...)` is also used in `/rooms/{roomId}/members`
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Update license headersPatrick Cloke2023-11-211-10/+16
|
* Add final type hint to tests.unittest. (#15072)Patrick Cloke2023-02-141-2/+2
| | | | Adds a return type to HomeServerTestCase.make_homeserver and deal with any variables which are no longer Any.
* Require types in tests.storage. (#14646)Patrick Cloke2022-12-091-6/+12
| | | | Adds missing type hints to `tests.storage` package and does not allow untyped definitions.
* Support filtering the /messages API by relation type (MSC3874). (#14148)Patrick Cloke2022-10-171-33/+85
| | | Gated behind an experimental configuration flag.
* Generate historic pagination token for `/messages` when no `?from` token ↵Eric Eastwood2022-04-061-1/+3
| | | | provided (#12370)
* Remove redundant `get_success` calls in test code (#12346)Richard van der Hoff2022-04-011-3/+1
| | | There are a bunch of places we call get_success on an immediate value, which is unnecessary. Let's rip them out, and remove the redundant functionality in get_success and friends.
* Support stable identifiers for MSC3440: Threading (#12151)Patrick Cloke2022-03-101-11/+9
| | | | The unstable identifiers are still supported if the experimental configuration flag is enabled. The unstable identifiers will be removed in a future release.
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Support filtering by relations per MSC3440 (#11236)Patrick Cloke2021-11-091-0/+207
| | | | Adds experimental support for `relation_types` and `relation_senders` fields for filters.
* Remove some unused functions (#711)Mark Haines2016-04-081-185/+0
| | | | | | | | * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want
* Fix flake8 warnings for testsMark Haines2016-02-191-2/+2
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Remove more unused parametersDaniel Wagner-Hall2015-11-021-4/+0
|
* Implement configurable stats reportingDaniel Wagner-Hall2015-09-221-53/+15
| | | | | | | | | | SYN-287 This requires that HS owners either opt in or out of stats reporting. When --generate-config is passed, --report-stats must be specified If an already-generated config is used, and doesn't have the report_stats key, it is requested to be set.
* Factor out some of the common homeserver setup code into aMark Haines2015-02-111-12/+2
| | | | setup_test_homeserver function in utils.
* Replace hs.parse_roomid with RoomID.from_stringMark Haines2015-01-231-3/+3
|
* Replace hs.parse_userid with UserID.from_stringMark Haines2015-01-231-2/+3
|
* Kill off synapse.api.events.*Erik Johnston2014-12-161-6/+5
|
* Fix prev_contentErik Johnston2014-12-111-3/+2
|
* Fix some testsErik Johnston2014-12-101-39/+34
|
* Fix stream test.Erik Johnston2014-11-101-5/+12
|
* Fix a couple more storage testsErik Johnston2014-11-051-7/+17
|
* Test that prev_content get's added if there is a prev_state key (in the ↵Erik Johnston2014-09-191-17/+70
| | | | event stream).
* SYN-47: Fix bug where we still returned events for rooms we had left.Erik Johnston2014-09-191-0/+173
SYN-47 #resolve