summary refs log tree commit diff
path: root/synapse/handlers/sync.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Implement changes to MSC2285 (hidden read receipts) (#12168)Šimon Brandner2022-05-041-1/+1
| | | | | * Changes hidden read receipts to be a separate receipt type (instead of a field on `m.read`). * Updates the `/receipts` endpoint to accept `m.fully_read`.
* Fix missing sync events during historical batch imports (#12319)Nick Mills-Barrett2022-04-131-8/+7
| | | | | | | | Discovered after much in-depth investigation in #12281. Closes: #12281 Closes: #3305 Signed off by: Nick Mills-Barrett nick@beeper.com
* Fix rejecting invites over federation (#12409)Erik Johnston2022-04-071-20/+33
| | | | | Currently causes future incremental syncs to fail. Broke by #12191
* Do not add groups to sync results if disabled. (#12408)Patrick Cloke2022-04-071-2/+3
|
* Send device list updates to application services (MSC3202) - part 1 (#11881)Andrew Morgan2022-03-301-31/+9
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add a configuration to exclude rooms from sync response (#12310)Brendan Abolivier2022-03-301-6/+17
|
* Move get_bundled_aggregations to relations handler. (#12237)Patrick Cloke2022-03-181-3/+6
| | | | | The get_bundled_aggregations code is fairly high-level and uses a lot of store methods, we move it into the handler as that seems like a better fit.
* Use the ignored_users table to test event visibility & sync. (#12225)Patrick Cloke2022-03-151-28/+2
| | | | | Instead of fetching the raw account data and re-parsing it. The ignored_users table is a denormalised version of the account data for quick searching.
* Add logging to `/sync` for debugging #11916 (#12068)Erik Johnston2022-02-231-0/+9
|
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Optimise calculating device_list changes in `/sync`. (#11974)Erik Johnston2022-02-151-15/+53
| | | | | | For users with large accounts it is inefficient to calculate the set of users they share a room with (and takes a lot of space in the cache). Instead we can look at users whose devices have changed since the last sync and check if they share a room with the syncing user.
* Send to-device messages to application services (#11215)Andrew Morgan2022-02-011-2/+2
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Improvements to bundling aggregations. (#11815)Patrick Cloke2022-01-261-1/+2
| | | | | | | | | | | This is some odds and ends found during the review of #11791 and while continuing to work in this code: * Return attrs classes instead of dictionaries from some methods to improve type safety. * Call `get_bundled_aggregations` fewer times. * Adds a missing assertion in the tests. * Do not return empty bundled aggregations for an event (preferring to not include the bundle at all, as the docstring states).
* Make the `get_global_account_data_by_type_for_user` cache be a tree-cache ↵reivilibre2022-01-211-1/+1
| | | | whose key is prefixed with the user ID (#11788)
* Include whether the requesting user has participated in a thread. (#11577)Patrick Cloke2022-01-181-1/+3
| | | | | | Per updates to MSC3440. This is implement as a separate method since it needs to be cached on a per-user basis, instead of a per-thread basis.
* Include bundled aggregations in the sync response cache. (#11659)Patrick Cloke2022-01-131-0/+10
|
* Remove debug logging for #4422 (#11693)David Robertson2022-01-061-37/+1
| | | | as per https://github.com/matrix-org/synapse/pull/11532#discussion_r769123269
* Add type hints to event_push_actions. (#11594)Patrick Cloke2021-12-211-6/+6
|
* Various opentracing enhancements (#11619)Richard van der Hoff2021-12-211-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | * Wrap `auth.get_user_by_req` in an opentracing span give `get_user_by_req` its own opentracing span, since it can result in a non-trivial number of sub-spans which it is useful to group together. This requires a bit of reorganisation because it also sets some tags (and may force tracing) on the servlet span. * Emit opentracing span for encoding json responses This can be a significant time sink. * Rename all sync spans with a prefix * Write an opentracing span for encoding sync response * opentracing span to group generate_room_entries * opentracing spans within sync.encode_response * changelog * Use the `trace` decorator instead of context managers
* Adjust _get_rooms_changed comments (#11550)David Robertson2021-12-101-22/+29
| | | C.f. https://github.com/matrix-org/synapse/pull/11494#pullrequestreview-827780886
* Add a constant for receipt types (m.read). (#11531)Patrick Cloke2021-12-081-2/+2
| | | And expand some type hints in the receipts storage module.
* Refactors in `_generate_sync_entry_for_rooms` (#11515)David Robertson2021-12-071-45/+77
| | | | | | | | * Move sync_token up to the top * Pull out _get_ignored_users * Try to signpost the body of `_generate_sync_entry_for_rooms` * Pull out _calculate_user_changes Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Correctly ignore invites from ignored users (#11511)David Robertson2021-12-071-5/+6
|
* Comments on the /sync tentacles (#11494)David Robertson2021-12-021-39/+117
| | | | | This mainly consists of docstrings and inline comments. There are one or two type annotations and variable renames thrown in while I was here. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Support filtering by relations per MSC3440 (#11236)Patrick Cloke2021-11-091-8/+10
| | | | Adds experimental support for `relation_types` and `relation_senders` fields for filters.
* Add type hints for event streams. (#10856)Patrick Cloke2021-09-211-3/+3
|
* Require type hints in the handlers module. (#10831)Patrick Cloke2021-09-201-5/+6
| | | | | | | Adds missing type hints to methods in the synapse.handlers module and requires all methods to have type hints there. This also removes the unused construct_auth_difference method from the FederationHandler.
* Merge branch 'release-v1.43' into developDavid Robertson2021-09-171-126/+141
|\
| * Cleanup opentracing logging for syncs (#10828)Erik Johnston2021-09-151-126/+141
| | | | | | | | | | We added a bunch of spans in #10704, but this ended up adding a lot of redundant spans for rooms where nothing changed, so instead we only start the span if there might be something interesting going on.
* | Use direct references for some configuration variables (part 2) (#10812)Patrick Cloke2021-09-151-1/+1
|/
* Add logging to help debug #9424 (#10704)Azrenbeth2021-09-061-13/+54
|
* Additional type hints for the sync REST servlet. (#10666)Patrick Cloke2021-08-231-10/+11
|
* Do not include rooms with an unknown room version in a sync response. (#10644)Patrick Cloke2021-08-191-2/+5
| | | | A user will still see this room if it is in a local cache, but it will not reappear if clearing the cache and reloading.
* Convert room member storage tuples to attrs. (#10629)Patrick Cloke2021-08-181-8/+10
| | | | Instead of using namedtuples. This helps with asserting type hints and code completion.
* Use auto-attribs for attrs classes for sync. (#10630)Patrick Cloke2021-08-181-78/+78
|
* Make sync response cache time configurable. (#10513)Richard van der Hoff2021-08-031-3/+11
|
* Always communicate device OTK counts to clients (#10485)Andrew Morgan2021-07-271-0/+4
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* [pyupgrade] `synapse/` (#10348)Jonathan de Jong2021-07-191-1/+1
| | | | | | | | | This PR is tantamount to running ``` pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"` ``` Part of #9744
* Use inline type hints in `handlers/` and `rest/`. (#10382)Jonathan de Jong2021-07-161-15/+17
|
* Ensure that we do not cache empty sync responses after a timeout (#10158)Richard van der Hoff2021-06-171-10/+26
| | | Fixes #8518 by telling the ResponseCache not to cache the /sync response if the next_batch param is the same as the since token.
* Implement knock feature (#6739)Sorunome2021-06-091-20/+67
| | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io
* Delete completes to-device messages earlier in /sync (#10124)Richard van der Hoff2021-06-071-10/+11
| | | | I hope this will improve https://github.com/matrix-org/synapse/issues/9564.
* Make /sync do less state res (#10102)Erik Johnston2021-06-021-2/+2
|
* Use get_current_users_in_room from store and not StateHandler (#9910)Erik Johnston2021-05-051-3/+3
|
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-221-2/+11
| | | This is no longer required, since we have dropped support for Python 3.5.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-4/+4
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Add type hints to expiring cache. (#9730)Patrick Cloke2021-04-061-4/+6
|
* Improve tracing for to device messages (#9686)Erik Johnston2021-04-011-1/+17
|
* Add a type hints for service notices to the HomeServer object. (#9675)Patrick Cloke2021-03-241-2/+4
|
* Fix bad naming of storage function (#9637)Erik Johnston2021-03-171-2/+4
| | | | | | We had two functions named `get_forward_extremities_for_room` and `get_forward_extremeties_for_room` that took different paramters. We rename one of them to avoid confusion.
* Add ResponseCache tests. (#9458)Jonathan de Jong2021-03-081-1/+1
|
* Revert "Fix #8518 (sync requests being cached wrongly on timeout) (#9358)"Patrick Cloke2021-03-021-2/+1
| | | | | | | This reverts commit f5c93fc9931e4029bbd8000f398b6f39d67a8c46. This is being backed out due to a regression (#9507) and additional review feedback being provided.
* Fix #8518 (sync requests being cached wrongly on timeout) (#9358)Jonathan de Jong2021-02-241-1/+2
| | | | | | | This fixes #8518 by adding a conditional check on `SyncResult` in a function when `prev_stream_token == current_stream_token`, as a sanity check. In `CachedResponse.set.<remove>()`, the result is immediately popped from the cache if the conditional function returns "false". This prevents the caching of a timed-out `SyncResult` (that has `next_key` as the stream key that produced that `SyncResult`). The cache is prevented from returning a `SyncResult` that makes the client request the same stream key over and over again, effectively making it stuck in a loop of requesting and getting a response immediately for as long as the cache keeps those values. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-17/+20
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Add additional type hints to the storage module. (#8980)Patrick Cloke2020-12-301-1/+1
|
* Add admin API for logging in as a user (#8617)Erik Johnston2020-11-171-1/+3
|
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-231-2/+2
|
* Send some ephemeral events to appservices (#8437)Will Hunt2020-10-151-1/+0
| | | Optionally sends typing, presence, and read receipt information to appservices.
* Add type hints to response cache. (#8507)Patrick Cloke2020-10-091-1/+3
|
* Fix returning incorrect prev_batch token in incremental sync (#8486)Erik Johnston2020-10-071-1/+6
|
* Add support for MSC2732: olm fallback keys (#8312)Hubert Chathi2020-10-061-0/+8
|
* Do not assume that account data is of the correct form. (#8454)Patrick Cloke2020-10-051-8/+11
| | | | This fixes a bug where `m.ignored_user_list` was assumed to be a dict, leading to odd behavior for users who set it to something else.
* Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-1/+1
|
* Various clean ups to room stream tokens. (#8423)Erik Johnston2020-09-291-6/+14
|
* Add EventStreamPosition type (#8388)Erik Johnston2020-09-241-5/+5
| | | | | | | | | | | | | | The idea is to remove some of the places we pass around `int`, where it can represent one of two things: 1. the position of an event in the stream; or 2. a token that partitions the stream, used as part of the stream tokens. The valid operations are then: 1. did a position happen before or after a token; 2. get all events that happened before or after a token; and 3. get all events between two tokens. (Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
* Use slots in attrs classes where possible (#8296)Patrick Cloke2020-09-141-24/+10
| | | | | slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.
* Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)Erik Johnston2020-09-111-5/+6
|
* Add types to StreamToken and RoomStreamToken (#8279)Erik Johnston2020-09-081-3/+2
| | | The intention here is to change `StreamToken.room_key` to be a `RoomStreamToken` in a future PR, but that is a big enough change without this refactoring too.
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-2/+2
|
* Fix typing for SyncHandler (#8237)Erik Johnston2020-09-031-5/+7
|
* Re-implement unread counts (again) (#8059)Brendan Abolivier2020-09-021-15/+18
|
* Define StateMap as immutable and add a MutableStateMap type. (#8183)Patrick Cloke2020-08-281-2/+3
|
* Revert #7736 (#8039)Brendan Abolivier2020-08-061-6/+0
|
* Convert streams to async. (#8014)Patrick Cloke2020-08-041-1/+1
|
* Re-implement unread counts (#7736)Brendan Abolivier2020-07-291-0/+6
|
* Fix up types and comments that refer to Deferreds. (#7945)Patrick Cloke2020-07-241-4/+0
|
* Add debugging to sync response generation (#7929)Richard van der Hoff2020-07-221-2/+15
|
* Back out MSC2625 implementation (#7761)Brendan Abolivier2020-07-011-3/+0
|
* Merge branch 'develop' into babolivier/mark_unreadBrendan Abolivier2020-06-151-18/+16
|\
| * Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-18/+16
| |
* | Appease mypyBrendan Abolivier2020-06-101-1/+1
| |
* | LintBrendan Abolivier2020-06-101-3/+3
| |
* | Use temporary prefixes as per the MSCBrendan Abolivier2020-06-101-1/+3
| |
* | Add a new unread_counter to sync responsesBrendan Abolivier2020-06-101-0/+1
|/
* Typo fixes.Patrick Cloke2020-06-051-1/+1
|
* Speed up fetching device lists changes in sync.Erik Johnston2020-05-051-4/+8
| | | | | Currently we copy `users_who_share_room` needlessly about three times, which is expensive when the set is large (which it can easily be).
* typosMatthew Hodgson2020-04-111-1/+1
|
* Add explanatory commentAndrew Morgan2020-03-301-0/+2
|
* Just add own user ID to the list we track device changes forDavid Baker2020-03-301-4/+6
|
* Always send the user updates to their own device listDavid Baker2020-03-271-3/+4
| | | | | This will allow clients to notify users about new devices even if the user isn't in any rooms (yet).
* Clean up some LoggingContext stuff (#7120)Richard van der Hoff2020-03-241-2/+2
| | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-12/+10
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Merge branch 'release-v1.10.0' into developRichard van der Hoff2020-02-111-0/+1
|\
| * filter out m.room.aliases from /sync state blocks (#6884)Matthew Hodgson2020-02-101-0/+1
| | | | | | We forgot to filter out aliases from /sync state blocks as well as the timeline.
* | Reduce amount of logging at INFO level. (#6862)Erik Johnston2020-02-061-3/+3
| | | | | | | | | | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* | Add typing to SyncHandler (#6821)Erik Johnston2020-02-031-341/+364
|/ | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Add `local_current_membership` table (#6655)Erik Johnston2020-01-151-1/+1
| | | | | | | Currently we rely on `current_state_events` to figure out what rooms a user was in and their last membership event in there. However, if the server leaves the room then the table may be cleaned up and that information is lost. So lets add a table that separately holds that information.
* Fixup functions to consistently return deferredsErik Johnston2019-12-061-3/+3
|
* Port SyncHandler to async/awaitErik Johnston2019-12-051-134/+117
|
* Update black to 19.10b0 (#6304)Amber Brown2019-11-011-7/+9
| | | * update version of black and also fix the mypy config being overridden
* Port to use state storageErik Johnston2019-10-301-9/+11
|
* Merge branch 'develop' into cross-signing_keysHubert Chathi2019-09-041-5/+4
|\
| * Remove double return statements (#5962)Andrew Morgan2019-09-031-1/+0
| | | | | | | | | | | | | | | | | | | | Remove all the "double return" statements which were a result of us removing all the instances of ``` defer.returnValue(...) return ``` statements when we switched to python3 fully.
| * Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-4/+4
| | | | | | | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* | Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keysHubert Chathi2019-08-281-6/+25
|\|
| * Remove logging for #5407 and update commentsErik Johnston2019-08-151-17/+5
| |
| * Handle TimelineBatch being limited and empty.Erik Johnston2019-08-061-6/+37
| | | | | | | | | | | | This hopefully addresses #5407 by gracefully handling an empty but limited TimelineBatch. We also add some logging to figure out how this is happening.
* | Merge branch 'cross-signing_hidden' into cross-signing_keysHubert Chathi2019-08-011-61/+51
|\|
| * Replace returnValue with return (#5736)Amber Brown2019-07-231-61/+51
| |
* | allow uploading keys for cross-signingHubert Chathi2019-07-251-1/+6
|/
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Review commentsErik Johnston2019-06-271-4/+4
|
* Refactor and comment sync device list codeErik Johnston2019-06-261-17/+53
|
* Rename get_users_whose_devices_changedErik Johnston2019-06-261-1/+1
|
* Refactor get_user_ids_changed to pull less from DBErik Johnston2019-06-261-12/+10
| | | | | | | | | | | When a client asks for users whose devices have changed since a token we used to pull *all* users from the database since the token, which could easily be thousands of rows for old tokens. This PR changes this to only check for changes for users the client is actually interested in. Fixes #5553
* Run Black. (#5482)Amber Brown2019-06-201-343/+351
|
* Merge pull request #5089 from dnaf/m-heroes-empty-room-nameBrendan Abolivier2019-06-061-5/+4
|\ | | | | Make /sync return heroes if room name or canonical alias are empty
| * Simplify conditionBrendan Abolivier2019-06-061-2/+2
| |
| * Simplify conditionBrendan Abolivier2019-06-051-3/+2
| |
| * Merge branch 'develop' into m-heroes-empty-room-nameBrendan Abolivier2019-06-051-19/+25
| |\
| * | LintKatie Wolfe2019-04-241-1/+1
| | | | | | | | | | | | | | | | | | I probably should've just run autopep8 in the first place... Signed-off-by: Katie Wolfe <katie@dnaf.moe>
| * | Clean up codeKatie Wolfe2019-04-241-2/+3
| | | | | | | | | | | | Signed-off-by: Katie Wolfe <katie@dnaf.moe>
| * | Show heroes if room name or canonical alias are emptyKatie Wolfe2019-04-241-3/+4
| | | | | | | | | | | | | | | | | | Fixes #4194 Signed-off-by: Katie Wolfe <katie@dnaf.moe>
| * | Show heroes if room name or canonical alias are emptyKatie Wolfe2019-04-241-5/+3
| | | | | | | | | | | | | | | | | | Fixes #4194 Signed-off-by: Katie Wolfe <katie@dnaf.moe>
* | | Do user_id != me checks before deciding whether we should pick heroes from ↵Brendan Abolivier2019-06-051-15/+19
| |/ |/| | | | | the joined members or the parted ones
* | Make /sync attempt to return device updates for both joined and invited ↵Matthew Hodgson2019-05-161-19/+25
|/ | | | users (#3484)
* Fix grammar and document get_current_users_in_room (#4998)Andrew Morgan2019-04-031-4/+4
|
* Merge pull request #4815 from matrix-org/erikj/docstringsErik Johnston2019-03-061-7/+26
|\ | | | | Add docstrings from matrix-org-hotfixes
| * Add docstrings from matrix-org-hotfixesErik Johnston2019-03-061-7/+26
| |
* | Port #4422 debug logging from hotfixesErik Johnston2019-03-061-7/+46
|/
* Add some debug for membership syncing issues (#4538)Richard van der Hoff2019-01-311-2/+11
| | | I can't figure out what's going on with #4422 and #4436; perhaps this will help.
* Fix remote invite rejections not comming down syncErik Johnston2019-01-301-1/+13
| | | | | | | | | | This was broken in PR #4405, commit 886e5ac, where we changed remote rejections to be outliers. The fix is to explicitly add the leave event in when we know its an out of band invite. We can't always add the event as if the server is/was in the room there might be more events to send down the sync than just the leave.
* Fixed line lengthNathan Pennie2018-12-191-1/+5
|
* Fix the variable names used for account_dataNathan Pennie2018-12-191-3/+3
|
* Refactor state group lookup to reduce DB hits (#4011)Erik Johnston2018-10-251-54/+43
| | | | | | | | Currently when fetching state groups from the data store we make two hits two the database: once for members and once for non-members (unless request is filtered to one or the other). This adds needless load to the datbase, so this PR refactors the lookup to make only a single database hit.
* Add description to counter metricErik Johnston2018-10-101-1/+5
|
* Track number of non-empty sync responses insteadErik Johnston2018-10-101-9/+15
|
* Append _total to metric and fix up spellingErik Johnston2018-10-101-2/+2
|
* Add metric to count lazy member sync requestsErik Johnston2018-10-091-6/+20
|
* Fix lazy loaded sync with rejected state eventsErik Johnston2018-10-011-2/+2
| | | | | | | | In particular, we assume that the name and canonical alias events in the state have not been rejected. In practice this may not be the case (though we should probably think about fixing that) so lets ensure that we gracefully handle that case, rather than 404'ing the sync request like we do now.
* Only lazy load self-members on initial syncMatthew Hodgson2018-09-251-5/+8
| | | Given we have disabled lazy loading for incr syncs in #3840, we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
* Always LL ourselves if we're in a room to simplify clients (#3916)Matthew Hodgson2018-09-201-0/+4
| | | | Should fix https://github.com/vector-im/riot-web/issues/7209
* show heroes if a room has a 'deleted' name/canonical_alias (#3851)Matthew Hodgson2018-09-121-2/+15
|
* argh pep8Matthew Hodgson2018-09-121-1/+1
|
* correctly log gappy sync metricsMatthew Hodgson2018-09-121-1/+1
|
* fix loglineMatthew Hodgson2018-09-121-1/+1
|
* Speed up lazy loading (#3827)Matthew Hodgson2018-09-121-30/+88
| | | | | * speed up room summaries by pulling their data from room_memberships rather than room state * disable LL for incr syncs, and log incr sync stats (#3840)
* Don't send empty tags list down syncErik Johnston2018-09-061-0/+5
|
* Send existing room tags down sync on joinErik Johnston2018-09-061-0/+8
| | | | | | | | When a user joined a room any existing tags were not sent down the sync stream. Ordinarily this isn't a problem because the user needs to be in the room to have set tags in it, however synapse will sometimes add tags for a user to a room, e.g. for server notices, which need to come down sync.
* Merge pull request #3795 from matrix-org/erikj/faster_sync_stateAmber Brown2018-09-071-8/+8
|\ | | | | User iter* during sync state calculations
| * User iter* during sync state calculationsErik Johnston2018-09-051-8/+8
| |
* | Port handlers/ to Python 3 (#3803)Amber Brown2018-09-071-3/+3
|/
* don't return non-LL-member state in incremental sync state blocks (#3760)Matthew Hodgson2018-08-281-1/+8
| | | don't return non-LL-member state in incremental sync state blocks
* Integrate presence from hotfixes (#3694)Amber Brown2018-08-181-1/+2
|
* initial cut at a room summary API (#3574)Matthew Hodgson2018-08-161-10/+149
|
* Merge pull request #3670 from matrix-org/neilj/mau_sync_blockNeil Johnson2018-08-141-2/+11
|\ | | | | Block ability to read via sync if mau limit exceeded
| * only block on sync where user is not part of the mau cohortNeil Johnson2018-08-091-1/+6
| |
| * block sync if auth checks failNeil Johnson2018-08-091-7/+5
| |
| * sync auth blockingNeil Johnson2018-08-081-5/+11
| |
* | Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-1/+1
|/
* Deduplicate redundant lazy-loaded members (#3331)Matthew Hodgson2018-07-261-25/+62
| | | | | | | | | | | | | | | | | | | | * attempt at deduplicating lazy-loaded members as per the proposal; we can deduplicate redundant lazy-loaded members which are sent in the same sync sequence. we do this heuristically rather than requiring the client to somehow tell us which members it has chosen to cache, by instead caching the last N members sent to a client, and not sending them again. For now we hardcode N to 100. Each cache for a given (user,device) tuple is in turn cached for up to X minutes (to avoid the caches building up). For now we hardcode X to 30. * add include_redundant_members filter option & make it work * remove stale todo * add tests for _get_some_state_from_cache * incorporate review
* clarify comment on p_idsMatthew Hodgson2018-07-241-1/+2
|
* rather than adding ll_ids, remove them from p_idsMatthew Hodgson2018-07-241-5/+6
|
* incorporate more reviewMatthew Hodgson2018-07-241-3/+9
|
* incorporate reviewMatthew Hodgson2018-07-231-40/+27
|
* incorporate reviewMatthew Hodgson2018-07-191-17/+27
|
* add a filtered_types param to limit filtering to specific typesMatthew Hodgson2018-07-191-29/+36
|
* merge develop pydoc for _get_state_for_groupsMatthew Hodgson2018-07-191-13/+13
|\
| * run isortAmber Brown2018-07-091-12/+12
| |
| * typoMatthew Hodgson2018-06-251-1/+1
| |
* | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-06-041-4/+14
|\|
| * Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-3/+3
| | | | | | | | they're not meant to be lazy (#3307)
| * Update some comments and docstrings in SyncHandlerRichard van der Hoff2018-05-291-1/+11
| |
* | more commentsMatthew Hodgson2018-06-041-0/+7
| |
* | add pydocMatthew Hodgson2018-05-291-5/+13
| |
* | merge developMatthew Hodgson2018-05-291-24/+41
|\|
| * replace some iteritems with sixAdrian Tschira2018-05-191-6/+8
| | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * Add comment to sync as to why code path is splitErik Johnston2018-05-091-0/+5
| |
| * Refactor sync APIs to reuse pagination APIErik Johnston2018-05-091-6/+13
| | | | | | | | | | | | The sync API often returns events in a topological rather than stream ordering, e.g. when the user joined the room or on initial sync. When this happens we can reuse existing pagination storage functions.
| * Refactor get_recent_events_for_room return typeErik Johnston2018-05-091-1/+1
| | | | | | | | | | | | There is no reason to return a tuple of tokens when the last token is always the token passed as an argument. Changing it makes it consistent with other storage APIs
| * Merge pull request #3107 from NotAFile/py3-bool-nonzeroRichard van der Hoff2018-04-201-0/+7
| |\ | | | | | | add __bool__ alias to __nonzero__ methods
| | * add __bool__ alias to __nonzero__ methodsAdrian Tschira2018-04-151-0/+7
| | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * | Refactor ResponseCache usageRichard van der Hoff2018-04-121-10/+6
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production.
| * Add metrics for ResponseCacheRichard van der Hoff2018-04-101-1/+1
| |
* | make lazy_load_members configurable in filtersMatthew Hodgson2018-03-191-9/+9
| |
* | reshuffle todo & commentsMatthew Hodgson2018-03-191-2/+10
| |
* | only get member state IDs for incremental syncs if we're filteringMatthew Hodgson2018-03-181-5/+5
| |
* | remove debugMatthew Hodgson2018-03-161-4/+0
| |
* | fix tsm search againMatthew Hodgson2018-03-161-5/+4
| |
* | add state_ids for timeline entriesMatthew Hodgson2018-03-161-4/+14
| |
* | simplify timeline_start_membersMatthew Hodgson2018-03-161-14/+13
| |
* | make incr syncs workMatthew Hodgson2018-03-161-3/+4
| |
* | add copyright to nudge CIMatthew Hodgson2018-03-141-1/+2
| |
* | oopsMatthew Hodgson2018-03-131-1/+1
| |
* | make it workMatthew Hodgson2018-03-131-3/+3
| |
* | ensure we always include the members for a given timeline blockMatthew Hodgson2018-03-131-4/+32
| |
* | typosMatthew Hodgson2018-03-111-2/+2
| |
* | WIP experiment in lazyloading room membersMatthew Hodgson2018-03-111-12/+31
|/
* Fix up log messageErik Johnston2018-03-071-1/+1
|
* Fix typoErik Johnston2018-03-071-2/+2
|
* Fix race in sync when joining roomErik Johnston2018-03-071-28/+75
| | | | | | | | | | | The race happens when the user joins a room at the same time as doing a sync. We fetch the current token and then get the rooms the user is in. If the join happens after the current token, but before we get the rooms we end up sending down a partial room entry in the sync. This is fixed by looking at the stream ordering of the membership returned by get_rooms_for_user, and handling the case when that stream ordering is after the current token.
* Remove ability for AS users to call /events and /syncErik Johnston2018-03-051-4/+6
| | | | | | | This functionality has been deprecated for a while as well as being broken for a while. Instead of fixing it lets just remove it entirely. See: https://github.com/matrix-org/matrix-doc/issues/1144
* Do logcontexts outside ResponseCacheErik Johnston2017-10-251-3/+3
|
* Merge branch 'develop' into erikj/groups_mergedDavid Baker2017-10-021-7/+32
|\
| * Refactor to speed up incremental syncsErik Johnston2017-09-251-5/+18
| |
| * Fix device list when rejoining room (#2461)Erik Johnston2017-09-221-2/+14
| |
* | Ensure that creator of group sees group down /syncErik Johnston2017-09-191-0/+1
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_mergedErik Johnston2017-09-191-21/+105
|\|
| * Don't filter out current state events from timelineErik Johnston2017-09-181-0/+7
| |
| * Actually hook leave notifs upErik Johnston2017-09-141-2/+3
| |
| * Get left roomsErik Johnston2017-09-131-3/+32
| |
| * In sync handle device lists for newly joined/left roomsErik Johnston2017-09-121-2/+10
| |
| * Add left section to /keys/changesErik Johnston2017-09-081-1/+1
| |
| * Send down device list change notif when member leaves/rejoins roomErik Johnston2017-09-071-12/+52
| |
| * Use less DB for device list handling in syncErik Johnston2017-07-121-8/+7
| |
* | Correctly add joins to correct segmentErik Johnston2017-07-241-1/+1
| |
* | Remember to cast to boolErik Johnston2017-07-201-1/+1
| |
* | Add groups to sync streamErik Johnston2017-07-201-1/+63
|/
* Add count of one time keys to sync streamErik Johnston2017-05-191-0/+11
|
* CommentsErik Johnston2017-03-161-2/+2
|
* CommentErik Johnston2017-03-161-3/+8
|
* Don't recreate so many setsErik Johnston2017-03-161-11/+7
|
* Implement no op for room stream in syncErik Johnston2017-03-161-4/+47
|
* Format presence events on the edges instead of reformatting them multiple timesErik Johnston2017-03-151-7/+7
|
* Noop repated delete device inbox calls from /syncErik Johnston2017-03-101-3/+3
|
* Measure new device list stuffErik Johnston2017-02-021-1/+2
|
* Wake sync up for device changesErik Johnston2017-02-011-1/+2
|
* Hook device list updates to replicationErik Johnston2017-01-271-10/+25
|
* Add basic implementation of local device list changesErik Johnston2017-01-251-0/+13
|
* Add some logging for syncing to_device eventsRichard van der Hoff2016-12-151-4/+8
| | | | | Attempt to track down the loss of to_device events (https://github.com/vector-im/riot-web/issues/2711 etc).
* Fix rare notifier bug where listeners dont timeoutErik Johnston2016-12-091-0/+1
| | | | | | There was a race condition that caused the notifier to 'miss' the timeout notification, since there were no other checks for the timeout this caused listeners to get stuck in a loop until something happened.
* Return early on /sync code paths if a '*' filter is usedKegan Dougal2016-11-221-9/+22
| | | | | | | | This is currently very conservative in that it only does this if there is no `since` token. This limits the risk to clients likely to be doing one-off syncs (like bridges), but does mean that normal human clients won't benefit from the time savings here. If the savings are large enough, I would consider generalising this to just check the filter.
* storage/appservice: make appservice methods only relying on the cache ↵Patrik Oldsberg2016-10-061-1/+1
| | | | synchronous
* Only return new device messages in /syncMark Haines2016-09-021-1/+1
|
* Comment about message deletionErik Johnston2016-08-301-0/+3
|
* Fix checkErik Johnston2016-08-301-2/+1
|
* Noop get_new_messages_for_device if token hasn't changedErik Johnston2016-08-301-9/+12
|
* Merge pull request #1049 from matrix-org/erikj/presence_users_in_roomErik Johnston2016-08-301-1/+2
|\ | | | | Use state handler instead of get_users_in_room/get_joined_hosts
| * Use state handler instead of get_users_in_room/get_joined_hostsErik Johnston2016-08-261-1/+2
| |
* | Merge remote-tracking branch 'origin/develop' into markjh/direct_to_deviceMark Haines2016-08-261-34/+41
|\|
| * Make sync not pull out full stateErik Johnston2016-08-251-34/+41
| |
* | Add store-and-forward direct-to-device messagingMark Haines2016-08-251-1/+42
|/
* defer.returnValue must not be called within MeasureErik Johnston2016-08-191-7/+7
|
* Cache federation state responsesErik Johnston2016-07-211-1/+1
|
* Load push rules in storage layer, so that they get cachedErik Johnston2016-06-031-3/+2
|
* Deduplicate presence entries in sync (#818)Mark Haines2016-06-021-0/+3
|
* Change short circuit pathErik Johnston2016-05-241-16/+26
|
* Don't send down all ephemeral eventsErik Johnston2016-05-241-1/+3
|
* Only work out newly_joined_users for incremental syncErik Johnston2016-05-241-6/+9
|