summary refs log tree commit diff
path: root/synapse/storage/databases/main/relations.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Update license headersPatrick Cloke2023-11-211-10/+16
|
* Convert simple_select_list and simple_select_list_txn to return lists of ↵Patrick Cloke2023-10-261-6/+9
| | | | | tuples (#16505) This should use fewer allocations and improves type hints.
* Allow multiple workers to write to receipts stream. (#16432)Erik Johnston2023-10-251-2/+2
| | | Fixes #16417
* Convert simple_select_many_batch, simple_select_many_txn to tuples. (#16444)Patrick Cloke2023-10-111-8/+11
|
* mypy plugin to check `@cached` return types (#14911)David Robertson2023-10-021-5/+7
| | | | | | | | Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Patrick Cloke <patrickc@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org> Assert that the return type of callables wrapped in @cached and @cachedList are cachable (aka immutable).
* Convert more cached return values to immutable types (#16356)Patrick Cloke2023-09-201-2/+2
|
* Return immutable objects for cachedList decorators (#16350)Patrick Cloke2023-09-191-3/+3
|
* Add a catch-all * to the supported relation types when redacting (#15705)Mathieu Velten2023-06-021-0/+30
| | | This is an update to MSC3912 implementation
* Initial implementation of MSC3981: recursive relations API (#15315)Patrick Cloke2023-05-021-17/+48
| | | | | | | | | | | Adds an optional keyword argument to the /relations API which will recurse a limited number of event relationships. This will cause the API to return not just the events related to the parent event, but also events related to those related to the parent event, etc. This is disabled by default behind an experimental configuration flag and is currently implemented using prefixed parameters.
* Apply & bundle edits for non-message events. (#15295)Patrick Cloke2023-03-211-7/+4
|
* Remove support for aggregating reactions (#15172)Richard van der Hoff2023-02-281-137/+0
| | | | | | | | | | It turns out that no clients rely on server-side aggregation of `m.annotation` relationships: it's just not very useful as currently implemented. It's also non-trivial to calculate. I want to remove it from MSC2677, so to keep the implementation in line, let's remove it here.
* Return read-only collections from `@cached` methods (#13755)Sean Quah2023-02-101-2/+5
| | | | | | | | | | | | | It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
* Use an enum for direction. (#14927)Patrick Cloke2023-01-271-4/+4
| | | | For better type safety we use an enum instead of strings to configure direction (backwards or forwards).
* Fix paginating /relations with a live token (#14866)Patrick Cloke2023-01-261-21/+17
| | | | | The `/relations` endpoint was not properly handle "live tokens" (i.e sync tokens), to do this properly we abstract the code that `/messages` has and re-use it.
* Faster joins: omit partial rooms from eager syncs until the resync completes ↵David Robertson2023-01-231-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#14870) * Allow `AbstractSet` in `StrCollection` Or else frozensets are excluded. This will be useful in an upcoming commit where I plan to change a function that accepts `List[str]` to accept `StrCollection` instead. * `rooms_to_exclude` -> `rooms_to_exclude_globally` I am about to make use of this exclusion mechanism to exclude rooms for a specific user and a specific sync. This rename helps to clarify the distinction between the global config and the rooms to exclude for a specific sync. * Better function names for internal sync methods * Track a list of excluded rooms on SyncResultBuilder I plan to feed a list of partially stated rooms for this sync to ignore * Exclude partial state rooms during eager sync using the mechanism established in the previous commit * Track un-partial-state stream in sync tokens So that we can work out which rooms have become fully-stated during a given sync period. * Fix mutation of `@cached` return value This was fouling up a complement test added alongside this PR. Excluding a room would mean the set of forgotten rooms in the cache would be extended. This means that room could be erroneously considered forgotten in the future. Introduced in #12310, Synapse 1.57.0. I don't think this had any user-visible side effects (until now). * SyncResultBuilder: track rooms to force as newly joined Similar plan as before. We've omitted rooms from certain sync responses; now we establish the mechanism to reintroduce them into future syncs. * Read new field, to present rooms as newly joined * Force un-partial-stated rooms to be newly-joined for eager incremental syncs only, provided they're still fully stated * Notify user stream listeners to wake up long polling syncs * Changelog * Typo fix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Unnecessary list cast Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Rephrase comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Another comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Fixup merge(?) * Poke notifier when receiving un-partial-stated msg over replication * Fixup merge whoops Thanks MV :) Co-authored-by: Mathieu Velen <mathieuv@matrix.org> Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Batch fetch bundled references (#14508)Patrick Cloke2022-11-221-8/+66
| | | | | | | | | Avoid an n+1 query problem and fetch the bundled aggregations for m.reference relations in a single query instead of a query per event. This applies similar logic for as was previously done for edits in 8b309adb436c162510ed1402f33b8741d71fc058 (#11660; threads in b65acead428653b988351ae8d7b22127a22039cd (#11752); and annotations in 1799a54a545618782840a60950ef4b64da9ee24d (#14491).
* Batch fetch bundled annotations (#14491)Patrick Cloke2022-11-221-54/+85
| | | | | | | | Avoid an n+1 query problem and fetch the bundled aggregations for m.annotation relations in a single query instead of a query per event. This applies similar logic for as was previously done for edits in 8b309adb436c162510ed1402f33b8741d71fc058 (#11660) and threads in b65acead428653b988351ae8d7b22127a22039cd (#11752).
* Implement MSC3912: Relation-based redactions (#14260)Brendan Abolivier2022-11-031-0/+36
| | | Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Revert behavior change for bundling edits of non-message events (#14283)Patrick Cloke2022-10-241-4/+7
|
* Accept threaded receipts for events related to the root event. (#14174)Patrick Cloke2022-10-141-6/+92
| | | | | | | | | The root node of a thread (and events related to it) are considered "part of a thread" when validating receipts. This allows clients which show the root node in both the main timeline and the threaded timeline to easily send receipts in either. Note that threaded notifications are not created for these events, these events created notifications on the main timeline.
* Fix sqlite syntax for upserts. (#14171)Patrick Cloke2022-10-131-1/+1
|
* Add an API for listing threads in a room. (#13394)Patrick Cloke2022-10-131-1/+165
| | | | | | | | | Implement the /threads endpoint from MSC3856. This is currently unstable and behind an experimental configuration flag. It includes a background update to backfill data, results from the /threads endpoint will be partial until that finishes.
* Return the main timeline for events which are not part of a thread. (#14140)Patrick Cloke2022-10-121-5/+7
| | | | Fixes a bug where threaded receipts could not be sent for the main timeline.
* Remove the experimental implementation of MSC3772. (#14094)Patrick Cloke2022-10-121-53/+0
| | | MSC3772 has been abandoned.
* Apply & bundle edits for non-message events. (#14034)Patrick Cloke2022-10-071-7/+4
| | | | | | | Fixes two related bugs: * No edit information was bundled for events which aren't `m.room.message`. * `m.new_content` was not applied for those events.
* Recursively fetch the thread for receipts & notifications. (#13824)Patrick Cloke2022-10-041-0/+36
| | | | | | Consider an event to be part of a thread if you can follow a chain of relations up to a thread root. Part of MSC3773 & MSC3771.
* Properly paginate forward in the /relations API. (#13840)Patrick Cloke2022-09-221-10/+28
| | | | | This fixes a bug where the `/relations` API with `dir=f` would skip the first item of each page (except the first page), causing incomplete data to be returned to the client.
* Remove unused argument for get_relations_for_event. (#13383)Patrick Cloke2022-07-261-6/+0
|
* Fix caching behavior for relations push rules. (#12859)Patrick Cloke2022-05-251-2/+3
| | | | | By always returning all requested values from the function wrapped by cachedList. Otherwise implicit None values get added into the cache, which are unexpected.
* Experimental support for MSC3772 (#12740)Patrick Cloke2022-05-241-0/+52
| | | | | | | | | | Implements the following behind an experimental configuration flag: * A new push rule kind for mutually related events. * A new default push rule (`.m.rule.thread_reply`) under an unstable prefix. This is missing part of MSC3772: * The `.m.rule.thread_reply_to_me` push rule, this depends on MSC3664 / #11804.
* Add `StreamKeyType` class and replace string literals with constants (#12567)Andrew Morgan2022-05-161-2/+4
|
* Include bundled aggregations for the latest event in a thread. (#12273)Patrick Cloke2022-05-041-8/+3
| | | | | | The `latest_event` field of the bundled aggregations for `m.thread` relations did not include bundled aggregations itself. This resulted in clients needing to immediately request the event from the server (and thus making it useless that the latest event itself was serialized instead of just including an event ID).
* Remove references to unstable identifiers from MSC3440. (#12382)Patrick Cloke2022-04-121-62/+16
| | | | | Removes references to unstable thread relation, unstable identifiers for filtering parameters, and the experimental config flag.
* Add some type hints to datastore (#12423)Dirk Klimpel2022-04-121-1/+1
| | | | | | | | | | | | | | | | | | | * Add some type hints to datastore * newsfile * change `Collection` to `List` * refactor return type of `select_users_txn` * correct type hint in `stream.py` * Remove `Optional` in `select_users_txn` * remove not needed return type in `__init__` * Revert change in `get_stream_id_for_event_txn` * Remove import from `Literal`
* Do not consider events by ignored users for bundled aggregations (#12235)Patrick Cloke2022-04-111-9/+143
| | | | | | | Consider the requester's ignored users when calculating the bundled aggregations. See #12285 / 4df10d32148ae29f792afc68ff774bcbd1915cea for corresponding changes for the `/relations` endpoint.
* Remove an unnecessary class from the relations code. (#12338)Patrick Cloke2022-03-311-9/+8
| | | | | The PaginationChunk class attempted to bundle some properties together, but really just caused callers to jump through hoops and hid implementation details.
* Remove the unused and unstable `/aggregations` endpoint. (#12293)Patrick Cloke2022-03-301-65/+13
| | | | | | | | | This endpoint was removed from MSC2675 before it was approved. It is currently unspecified (even in any MSCs) and therefore subject to removal. It is not implemented by any known clients. This also changes the bundled aggregation format for `m.annotation`, which previously included pagination tokens for the `/aggregations` endpoint, which are no longer useful.
* Move get_bundled_aggregations to relations handler. (#12237)Patrick Cloke2022-03-181-146/+5
| | | | | The get_bundled_aggregations code is fairly high-level and uses a lot of store methods, we move it into the handler as that seems like a better fit.
* Only fetch thread participation for events with threads. (#12228)Patrick Cloke2022-03-181-1/+3
| | | | | | | | | We fetch the thread summary in two phases: 1. The summary that is shared by all users (count of messages and latest event). 2. Whether the requesting user has participated in the thread. There's no use in attempting step 2 for events which did not return a summary from step 1.
* Support stable identifiers for MSC3440: Threading (#12151)Patrick Cloke2022-03-101-30/+47
| | | | The unstable identifiers are still supported if the experimental configuration flag is enabled. The unstable identifiers will be removed in a future release.
* Allow retrieving the relations of a redacted event. (#12130)Patrick Cloke2022-03-101-28/+32
| | | | | | | | | This is allowed per MSC2675, although the original implementation did not allow for it and would return an empty chunk / not bundle aggregations. The main thing to improve is that the various caches get cleared properly when an event is redacted, and that edits must not leak if the original event is redacted (as that would presumably leak something similar to the original event content).
* Fix a typo in a comment.Patrick Cloke2022-02-161-1/+1
|
* Fix incorrect thread summaries when the latest event is edited. (#11992)Patrick Cloke2022-02-151-6/+18
| | | | | If the latest event in a thread was edited than the original event content was included in bundled aggregation for threads instead of the edited event content.
* Fetch thread summaries for multiple events in a single query (#11752)Patrick Cloke2022-02-111-73/+149
| | | | | This should reduce database usage when fetching bundled aggregations as the number of individual queries (and round trips to the database) are reduced.
* Support pagination tokens from /sync and /messages in the relations API. ↵Patrick Cloke2022-02-101-15/+31
| | | | (#11952)
* Experimental support to include bundled aggregations in search results ↵Patrick Cloke2022-02-081-2/+11
| | | | (MSC3666) (#11837)
* Fetch edits for multiple events in a single query. (#11660)Patrick Cloke2022-02-081-51/+99
| | | | | This should reduce database usage when fetching bundled aggregations as the number of individual queries (and round trips to the database) are reduced.
* Remove the obsolete MSC1849 configuration flag. (#11843)Patrick Cloke2022-01-311-4/+0
| | | | | MSC1849 was replaced by MSC2675, which was merged. The configuration flag, which defaulted to true, is no longer useful.
* Improvements to bundling aggregations. (#11815)Patrick Cloke2022-01-261-23/+38
| | | | | | | | | | | This is some odds and ends found during the review of #11791 and while continuing to work in this code: * Return attrs classes instead of dictionaries from some methods to improve type safety. * Call `get_bundled_aggregations` fewer times. * Adds a missing assertion in the tests. * Do not return empty bundled aggregations for an event (preferring to not include the bundle at all, as the docstring states).
* Include whether the requesting user has participated in a thread. (#11577)Patrick Cloke2022-01-181-11/+55
| | | | | | Per updates to MSC3440. This is implement as a separate method since it needs to be cached on a per-user basis, instead of a per-thread basis.
* Bundle aggregations outside of the serialization method. (#11612)Patrick Cloke2022-01-071-3/+125
| | | | | | | | This makes the serialization of events synchronous (and it no longer access the database), but we must manually calculate and provide the bundled aggregations. Overall this should cause no change in behavior, but is prep work for other improvements.
* Improve type hints in storage classes. (#11652)Dirk Klimpel2021-12-291-2/+2
| | | By using cast and making ignores more specific.
* Remove redundant `COALESCE()`s around `COUNT()`s in database queries (#11570)Sean Quah2021-12-141-1/+1
| | | | | `COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a `COUNT(NULL)` is also 0.
* Do not allow cross-room relations, per MSC2674. (#11516)Patrick Cloke2021-12-091-10/+26
|
* Do not allow MSC3440 threads to fork threads (#11161)Patrick Cloke2021-11-181-2/+65
| | | | | | | | | | | Adds validation to the Client-Server API to ensure that the potential thread head does not relate to another event already. This results in not allowing a thread to "fork" into other threads. If the target event is unknown for some reason (maybe it isn't visible to your homeserver), but is the target of other events it is assumed that the thread can be created from it. Otherwise, it is rejected as an unknown event.
* Support filtering by relations per MSC3440 (#11236)Patrick Cloke2021-11-091-1/+57
| | | | Adds experimental support for `relation_types` and `relation_senders` fields for filters.
* Additional type hints for relations database class. (#11205)Patrick Cloke2021-10-281-15/+23
|
* Add a thread relation type per MSC3440. (#11088)Patrick Cloke2021-10-211-1/+58
| | | | Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Convert additional databases to async/await (#8199)Patrick Cloke2020-09-011-55/+48
|
* Convert appservice, group server, profile and more databases to async (#8066)Patrick Cloke2020-08-121-9/+10
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-0/+327
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-03 20:56:47 +0000