summary refs log tree commit diff
path: root/synapse/state (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Revert "Revert accidental fast-forward merge from v1.49.0rc1"Olivier Wilkinson (reivilibre)2021-12-142-2/+3
| | | | This reverts commit 158d73ebdd61eef33831ae5f6990acf07244fc55.
* Revert accidental fast-forward merge from v1.49.0rc1Sean Quah2021-12-072-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Revert "Sort internal changes in changelog" Revert "Update CHANGES.md" Revert "1.49.0rc1" Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)" Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515)" Revert "Correctly register shutdown handler for presence workers (#11518)" Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524)" Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523)" Revert "Correctly ignore invites from ignored users (#11511)" Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522)" Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435)" Revert "Save the OIDC session ID (sid) with the device on login (#11482)" Revert "Add admin API to get some information about federation status (#11407)" Revert "Include bundled aggregations in /sync and related fixes (#11478)" Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)" Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469)" Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)" Revert "Add type hints to `synapse/tests/rest/admin` (#11501)" Revert "Revert accidental commits to develop." Revert "Newsfile" Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour" Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`" Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`" Revert "Disambiguate queries on `state_key` (#11497)" Revert "Comments on the /sync tentacles (#11494)" Revert "Clean up tests.storage.test_appservice (#11492)" Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493)" Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495)" Revert "Minor cleanup on recently ported doc pages (#11466)" Revert "Add most of the missing type hints to `synapse.federation`. (#11483)" Revert "Avoid waiting for zombie processes in `synctl stop` (#11490)" Revert "Fix media repository failing when media store path contains symlinks (#11446)" Revert "Add type annotations to `tests.storage.test_appservice`. (#11488)" Revert "`scripts-dev/sign_json`: support for signing events (#11486)" Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)" Revert "Port wiki pages to documentation website (#11402)" Revert "Add a license header and comment. (#11479)" Revert "Clean-up get_version_string (#11468)" Revert "Link background update controller docs to summary (#11475)" Revert "Additional type hints for config module. (#11465)" Revert "Register the login redirect endpoint for v3. (#11451)" Revert "Update openid.md" Revert "Remove mention of OIDC certification from Dex (#11470)" Revert "Add a note about huge pages to our Postgres doc (#11467)" Revert "Don't start Synapse master process if `worker_app` is set (#11416)" Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449)" Revert "Bundle relations of relations into the `/relations` result. (#11284)" Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454)" Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453)" Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461)" Revert "Merge branch 'master' into develop" This reverts commit 26b5d2320f62b5eb6262c7614fbdfc364a4dfc02. This reverts commit bce4220f387bf5448387f0ed7d14ed1e41e40747. This reverts commit 966b5d0fa0893c3b628c942dfc232e285417f46d. This reverts commit 088d748f2cb51f03f3bcacc0fb3af1e0f9607737. This reverts commit 14d593f72d10b4d8cb67e3288bb3131ee30ccf59. This reverts commit 2a3ec6facf79f6aae011d9fb6f9ed5e43c7b6bec. This reverts commit eccc49d7554d1fab001e1fefb0fda8ffb254b630. This reverts commit b1ecd19c5d19815b69e425d80f442bf2877cab76. This reverts commit 9c55dedc8c4484e6269451a8c3c10b3e314aeb4a. This reverts commit 2d42e586a8c54be1a83643148358b1651c1ca666. This reverts commit 2f053f3f82ca174cc1c858c75afffae51af8ce0d. This reverts commit a15a893df8428395df7cb95b729431575001c38a. This reverts commit 8b4b153c9e86c04c7db8c74fde4b6a04becbc461. This reverts commit 494ebd7347ba52d702802fba4c3bb13e7bfbc2cf. This reverts commit a77c36989785c0d5565ab9a1169f4f88e512ce8a. This reverts commit 4eb77965cd016181d2111f37d93526e9bb0434f0. This reverts commit 637df95de63196033a6da4a6e286e1d58ea517b6. This reverts commit e5f426cd54609e7f05f8241d845e6e36c5f10d9a. This reverts commit 8cd68b8102eeab1b525712097c1b2e9679c11896. This reverts commit 6cae125e20865c52d770b24278bb7ab8fde5bc0d. This reverts commit 7be88fbf48156b36b6daefb228e1258e7d48cae4. This reverts commit b3fd99b74a3f6f42a9afd1b19ee4c60e38e8e91a. This reverts commit f7ec6e7d9e0dc360d9fb41f3a1afd7bdba1475c7. This reverts commit 5640992d176a499204a0756b1677c9b1575b0a49. This reverts commit d26808dd854006bd26a2366c675428ce0737238c. This reverts commit f91624a5950e14ba9007eed9bfa1c828676d4745. This reverts commit 16d39a5490ce74c901c7a8dbb990c6e83c379207. This reverts commit 8a4c2969874c0b7d72003f2523883eba8a348e83. This reverts commit 49e1356ee3d5d72929c91f778b3a231726c1413c. This reverts commit d2279f471ba8f44d9f578e62b286897a338d8aa1. This reverts commit b50e39df578adc3f86c5efa16bee9035cfdab61b. This reverts commit 858d80bf0f9f656a03992794874081b806e49222. This reverts commit 435f04480728c5d982e1a63c1b2777784bf9cd26. This reverts commit f61462e1be36a51dbf571076afa8e1930cb182f4. This reverts commit a6f1a3abecf8e8fd3e1bff439a06b853df18f194. This reverts commit 84dc50e160a2ec6590813374b5a1e58b97f7a18d. This reverts commit ed635d32853ee0a3e5ec1078679b27e7844a4ac7. This reverts commit 7b62791e001d6a4f8897ed48b3232d7f8fe6aa48. This reverts commit 153194c7717d8016b0eb974c81b1baee7dc1917d. This reverts commit f44d729d4ccae61bc0cdd5774acb3233eb5f7c13. This reverts commit a265fbd397ae72b2d3ea4c9310591ff1d0f3e05c. This reverts commit b9fef1a7cdfcc128fa589a32160e6aa7ed8964d7. This reverts commit b0eb64ff7bf6bde42046e091f8bdea9b7aab5f04. This reverts commit f1795463bf503a6fca909d77f598f641f9349f56. This reverts commit 70cbb1a5e311f609b624e3fae1a1712db639c51e. This reverts commit 42bf0204635213e2c75188b19ee66dc7e7d8a35e. This reverts commit 379f2650cf875f50c59524147ec0e33cfd5ef60c. This reverts commit 7ff22d6da41cd5ca80db95c18b409aea38e49fcd. This reverts commit 5a0b652d36ae4b6d423498c1f2c82c97a49c6f75. This reverts commit 432a174bc192740ac7a0a755009f6099b8363ad9. This reverts commit b14f8a1baf6f500997ae4c1d6a6d72094ce14270, reversing changes made to e713855dca17a7605bae99ea8d71bc7f8657e4b8.
* Add type hints to `synapse/storage/databases/main/events_worker.py` (#11411)Sean Quah2021-11-262-2/+3
| | | | Also refactor the stream ID trackers/generators a bit and try to document them better.
* Add remaining type hints to `synapse.events`. (#11098)Patrick Cloke2021-11-021-1/+1
|
* Annotate `log_function` decorator (#10943)reivilibre2021-10-271-2/+3
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Check *all* auth events for room id and rejection (#11009)Richard van der Hoff2021-10-182-3/+3
| | | | | | | | | | | This fixes a bug where we would accept an event whose `auth_events` include rejected events, if the rejected event was shadowed by another `auth_event` with same `(type, state_key)`. The approach is to pass a list of auth events into `check_auth_rules_for_event` instead of a dict, which of course means updating the call sites. This is an extension of #10956.
* `disallow-untyped-defs` for `synapse.state` (#11004)David Robertson2021-10-063-4/+4
| | | | | * `disallow-untyped-defs` for `synapse.state` Much smaller than I was expecting!
* Split `event_auth.check` into two parts (#10940)Richard van der Hoff2021-09-292-9/+3
| | | | | | | | | | | | | Broadly, the existing `event_auth.check` function has two parts: * a validation section: checks that the event isn't too big, that it has the rught signatures, etc. This bit is independent of the rest of the state in the room, and so need only be done once for each event. * an auth section: ensures that the event is allowed, given the rest of the state in the room. This gets done multiple times, against various sets of room state, because it forms part of the state res algorithm. Currently, this is implemented with `do_sig_check` and `do_size_check` parameters, but I think that makes everything hard to follow. Instead, we split the function in two and call each part separately where it is needed.
* Factor out a separate `EventContext.for_outlier` (#10883)Richard van der Hoff2021-09-221-30/+4
| | | | | | Constructing an EventContext for an outlier is actually really simple, and there's no sense in going via an `async` method in the `StateHandler`. This also means that we can resolve a bunch of FIXMEs.
* Add type hints to state handler. (#10482)Patrick Cloke2021-07-261-11/+15
|
* Update the MSC3083 support to verify if joins are from an authorized server. ↵Patrick Cloke2021-07-263-21/+42
| | | | (#10254)
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-153-19/+21
|
* Use get_current_users_in_room from store and not StateHandler (#9910)Erik Johnston2021-05-051-3/+7
|
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-222-2/+4
| | | This is no longer required, since we have dropped support for Python 3.5.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-143-3/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Add type hints to expiring cache. (#9730)Patrick Cloke2021-04-061-2/+3
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-163-8/+26
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Fix some typos.Patrick Cloke2021-02-121-1/+1
|
* Precompute joined hosts and store in Redis (#9198)Erik Johnston2021-01-261-3/+8
|
* Fix mainline ordering in state res v2 (#8971)Erik Johnston2020-12-181-1/+1
| | | | | | This had two effects 1) it'd give the wrong answer and b) would iterate *all* power levels in the auth chain of each event. The latter of which can be *very* expensive for certain types of IRC bridge rooms that have large numbers of power level changes.
* Pass room_id to get_auth_chain_difference (#8879)Erik Johnston2020-12-042-4/+9
| | | | | This is so that we can choose which algorithm to use based on the room ID.
* Correctly handle unpersisted events when calculating auth chain difference. ↵Erik Johnston2020-12-021-4/+83
| | | | | | | | | (#8827) We do state res with unpersisted events when calculating the new current state of the room, so that should be the only thing impacted. I don't think this is tooooo big of a deal as: 1. the next time a state event happens in the room the current state should correct itself; 2. in the common case all the unpersisted events' auth events will be pulled in by other state, so will still return the correct result (or one which is sufficiently close to not affect the result); and 3. we mostly use the state at an event to do important operations, which isn't affected by this.
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-233-3/+3
|
* Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-1/+1
|
* Report state res metrics to Prometheus and logRichard van der Hoff2020-09-291-20/+124
|
* Move Measure calls into `resolve_events_with_store`Richard van der Hoff2020-09-291-32/+31
|
* Move `resolve_events_with_store` into StateResolutionHandlerRichard van der Hoff2020-09-291-45/+47
|
* Improve logging of state resolution (#8371)Richard van der Hoff2020-09-231-48/+16
| | | | | | | | | | I'd like to get a better insight into what we are doing with respect to state res. The list of state groups we are resolving across should be short (if it isn't, that's a massive problem in itself), so it should be fine to log it in ite entiretly. I've done some grepping and found approximately zero cases in which the "shortcut" code delivered the result, so I've ripped that out too.
* Use slots in attrs classes where possible (#8296)Patrick Cloke2020-09-141-1/+1
| | | | | slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-4/+4
|
* Define StateMap as immutable and add a MutableStateMap type. (#8183)Patrick Cloke2020-08-283-20/+28
|
* Add type hints for state. (#8140)Patrick Cloke2020-08-243-186/+348
|
* Convert events worker database to async/await. (#8071)Patrick Cloke2020-08-181-1/+1
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-1/+1
|
* Convert state resolution to async/await (#7942)Patrick Cloke2020-07-243-120/+97
|
* Add another yield point to state res v2 (#7746)Erik Johnston2020-06-261-2/+10
|
* Yield during large v2 state res. (#7735)Erik Johnston2020-06-242-11/+51
| | | | | | | | | | | State res v2 across large data sets can be very CPU intensive, and if all the relevant events are in the cache the algorithm will run from start to finish within a single reactor tick. This can result in blocking the reactor tick for several seconds, which can have major repercussions on other requests. To fix this we simply add the occaisonal `sleep(0)` during iterations to yield execution until the next reactor tick. The aim is to only do this for large data sets so that we don't impact otherwise quick resolutions.=
* Speed up state res v2 across large state differences. (#7725)Erik Johnston2020-06-191-1/+2
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-153-30/+24
|
* Allow configuration of Synapse's cache without using synctl or environment ↵Amber Brown2020-05-111-3/+1
| | | | variables (#6391)
* Improve get auth chain difference algorithm. (#7095)Erik Johnston2020-03-182-48/+12
| | | | | | | | | | | It was originally implemented by pulling the full auth chain of all state sets out of the database and doing set comparison. However, that can take a lot work if the state and auth chains are large. Instead, lets try and fetch the auth chains at the same time and calculate the difference on the fly, allowing us to bail early if all the auth chains converge. Assuming that the auth chains do converge more often than not, this should improve performance. Hopefully.
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-212-9/+9
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Reduce auth chains fetched during v2 state res. (#6952)Erik Johnston2020-02-192-6/+11
| | | | | | The state res v2 algorithm only cares about the difference between auth chains, so we can pass in the known common state to the `get_auth_chain` storage function so that it can ignore those events.
* s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-1/+1
| | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* Pass room version object into event_auth.check and check_redaction (#6788)Richard van der Hoff2020-01-282-3/+5
| | | | | | | These are easier to work with than the strings and we normally have one around. This fixes `FederationHander._persist_auth_tree` which was passing a RoomVersion object into event_auth.check instead of a string.
* Add StateMap type alias (#6715)Erik Johnston2020-01-163-8/+11
|
* Merge branch 'master' into developRichard van der Hoff2019-12-311-15/+37
|\
| * Hacks to work around #6605 (#6608)Richard van der Hoff2019-12-311-15/+37
| | | | | | | | | | When we have an event which refers to non-existent auth_events, ignore said events rather than exploding in a ball of fire. Fixes #6605.
| * sanity-checking for events used in state res (#6531)Richard van der Hoff2019-12-163-43/+123
| | | | | | | | | | When we perform state resolution, check that all of the events involved are in the right room.
* | sanity-checking for events used in state res (#6531)Richard van der Hoff2019-12-133-43/+123
| | | | | | | | | | | | | | When we perform state resolution, check that all of the events involved are in the right room.
* | Prevent redacted events from appearing in message search (#6377)Andrew Morgan2019-12-111-1/+2
|/
* Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-89/+83
| | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
* Improve documentation for EventContext fields (#6319)Richard van der Hoff2019-11-051-0/+3
|
* Port to use state storageErik Johnston2019-10-301-5/+8
|
* add some metrics on the federation sender (#6160)Richard van der Hoff2019-10-031-6/+18
|
* Remove double return statements (#5962)Andrew Morgan2019-09-031-1/+0
| | | | | | | | | | Remove all the "double return" statements which were a result of us removing all the instances of ``` defer.returnValue(...) return ``` statements when we switched to python3 fully.
* Replace returnValue with return (#5736)Amber Brown2019-07-233-38/+34
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-06-203-151/+104
|\ | | | | | | erikj/histogram_extremities
| * Run Black. (#5482)Amber Brown2019-06-203-151/+104
| |
* | Add descriptions and remove redundant set(..)Erik Johnston2019-06-191-1/+2
| |
* | Only count non-cache state resolutionErik Johnston2019-06-181-3/+3
| |
* | Use consistent bucketsErik Johnston2019-06-181-1/+1
| |
* | Add metric fo number of state groups in resolutionErik Johnston2019-06-171-0/+10
|/
* Fix grammar and document get_current_users_in_room (#4998)Andrew Morgan2019-04-031-2/+13
|
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-012-14/+13
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* Enable room version v3Erik Johnston2019-01-291-1/+1
|
* Drop vdh supportErik Johnston2019-01-291-1/+1
|
* Pass through room version to event authErik Johnston2019-01-253-9/+21
|
* Merge pull request #4307 from matrix-org/erikj/v2_roomsRichard van der Hoff2018-12-241-1/+3
|\ | | | | Add v2 room version
| * Add v2 room versionErik Johnston2018-12-181-1/+3
| |
* | Workaround for non-ascii event ids (#4241)Richard van der Hoff2018-12-031-1/+3
|/ | | | | | It turns out that we accept events with non-ascii IDs, which would later cause an explosion during state res. Fixes #4226
* Add helpers for getting prev and auth events (#4139)Erik Johnston2018-11-062-9/+9
| | | | | | | * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions.
* Fix None exception in state res v2Erik Johnston2018-11-021-0/+4
|
* Add STATE_V2_TEST room versionErik Johnston2018-11-011-1/+1
|
* Rename resolve_events_with_factoryErik Johnston2018-10-243-9/+9
|
* Add v2 state res algorithm.Erik Johnston2018-10-162-18/+613
| | | | We hook this up to the vdh test room version.
* Logging improvementsRichard van der Hoff2018-09-261-2/+12
| | | | Some logging tweaks to help with debugging incoming federation transactions
* Fix some instances of ExpiringCache not expiring cache itemsErik Johnston2018-09-211-9/+0
| | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.
* Remove unnecessary resolve_events_with_state_mapErik Johnston2018-08-222-56/+6
| | | | | We only ever used the synchronous resolve_events_with_state_map in one place, which is trivial to replace with the async version.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-201-3/+3
| | | | erikj/refactor_state_handler
* Remove redundant room_version checksErik Johnston2018-08-201-14/+3
|
* Add fast path in state res for zero prev eventsErik Johnston2018-08-091-1/+6
|
* Choose state algorithm based on room versionErik Johnston2018-08-091-11/+93
|
* Refactor state moduleErik Johnston2018-08-092-0/+895
We split out the actual state resolution algorithm to prepare for having multiple versions.