summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* Make cleaning up pushers depend on the device_id instead of the token_id ↵Quentin Gliech2023-03-2415-65/+142
| | | | | | | | | | | | | | (#15280) This makes it so that we rely on the `device_id` to delete pushers on logout, instead of relying on the `access_token_id`. This ensures we're not removing pushers on token refresh, and prepares for a world without access token IDs (also known as the OIDC). This actually runs the `set_device_id_for_pushers` background update, which was forgotten in #13831. Note that for backwards compatibility it still deletes pushers based on the `access_token` until the background update finishes.
* Reject mentions on the C-S API which are invalid. (#15311)Patrick Cloke2023-03-244-54/+105
| | | | | | Invalid mentions data received over the Client-Server API should be rejected with a 400 error. This will hopefully stop clients from sending invalid data, although does not help with data received over federation.
* Reintroduce membership tables event stream ordering (#15128)Nick Mills-Barrett2023-03-246-12/+131
| | | | | | | | | | | | | | | | | | * Add `event_stream_ordering` column to membership state tables Specifically this adds the column to `current_state_events`, `local_current_membership` and `room_memberships`. Each of these tables is regularly joined with the `events` table to get the stream ordering and denormalising this into each table will yield significant query performance improvements once used. * Make denormalised `event_stream_ordering` columns foreign keys * Add comment in schema file explaining new denormalised columns * Add triggers to enforce consistency of `event_stream_ordering` columns * Re-order purge room tables to account for foreign keys * Bump schema version to 75 Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Add a primitive helper script for listing worker endpoints. (#15243)reivilibre2023-03-2331-12/+424
| | | | Co-authored-by: Patrick Cloke <patrickc@matrix.org>
* Use immutabledict instead of frozendict (#15113)David Robertson2023-03-2220-244/+124
| | | | | | | | | | Additionally: * Consistently use `freeze()` in test --------- Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: 6543 <6543@obermui.de>
* Merge branch 'release-v1.80' into developH. Shay2023-03-2249-49/+91
|\
| * 1.80.0rc2 v1.80.0rc2H. Shay2023-03-226-6/+18
| |
| * Merge commit '96bcc5d902' into release-v1.80Richard van der Hoff2023-03-219-11/+76
| |\
| * \ Merge branch 'release-v1.80' of https://github.com/matrix-org/synapse into ↵ v1.80.0rc1H. Shay2023-03-211-1/+1
| |\ \ | | | | | | | | | | | | release-v1.80
| | * | Update CHANGES.mdShay2023-03-211-1/+1
| | | |
| * | | update changelogH. Shay2023-03-211-8/+9
| |/ /
| * | 1.80.0rc1H. Shay2023-03-2146-44/+73
| | |
* | | Add a check to SQLite port DB script to ensure that the sqlite database ↵Shay2023-03-222-0/+10
| | | | | | | | | | | | passed to the script exists before trying to port from it (#15306)
* | | Bump dtolnay/rust-toolchain from e12eda571dc9a5ee5d58eecf4738ec291c66f295 to ↵dependabot[bot]2023-03-224-15/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fc3253060d0c959bea12a59f10f8391454a0b02d (#15304) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from e12eda571dc9a5ee5d58eecf4738ec291c66f295 to fc3253060d0c959bea12a59f10f8391454a0b02d. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/e12eda571dc9a5ee5d58eecf4738ec291c66f295...fc3253060d0c959bea12a59f10f8391454a0b02d) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | | Have replication clients remove _INT_STREAM_POS (#15309)David Robertson2023-03-222-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Have replication clients remove _INT_STREAM_POS Suppose worker A makes an internal http request from worker B. B may make changes that A later learns about over replication. We want A's request to block until it has seen those changes—mainly to ensure A's caches are invalidated promptly. This helps provide read-after-write consistency, eliminating entire categories of races and test flakes. To implement this, B includes a top-level field `_INT_STREAM_POS` in its response JSON. Roughly speaking, the field's value tells A what to wait for. But we weren't removing that internal field before A's request completed! Introduced in https://github.com/matrix-org/synapse/pull/14820. Fixes #15308. * Changelog
* | | Change the parameter `immediate` of `send_device_messages` to default to ↵Shay2023-03-213-2/+3
| | | | | | | | | | | | `True` (#15297)
* | | Bump sentry-sdk from 1.15.0 to 1.17.0 (#15285)dependabot[bot]2023-03-212-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump sentry-sdk from 1.15.0 to 1.17.0 Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.15.0 to 1.17.0. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.15.0...1.17.0) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
* | | Update 15295.bugfixPatrick Cloke2023-03-211-1/+1
| | |
* | | Allow running twisted trunk against other branches (#15302)David Robertson2023-03-212-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Allow running twisted trunk against other branches I would like to do this so we can try Synapse's typechecking against a specific branch that the project solicited tests for, see https://mail.python.org/archives/list/twisted@python.org/message/GGO5JHA5S475AK6JZ3GCC3GIHGKQYM6Y/ * Changelog
* | | Remind maintainer to ask #synapse-dev for changelog (#15303)David Robertson2023-03-212-1/+2
| | | | | | | | | | | | | | | * Remind maintainer to ask #synapse-dev for changelog * Changelog
* | | Apply & bundle edits for non-message events. (#15295)Patrick Cloke2023-03-212-7/+5
| |/ |/|
* | Revert "check sqlite database file exists before porting/#14692" (#15301)Shay2023-03-212-1/+4
| |
* | Make `POST /_matrix/client/v3/rooms/{roomId}/report/{eventId}` endpoint ↵Andrew Morgan2023-03-215-6/+61
| | | | | | | | return 404 if event exists, but the user lacks access (#15300)
* | Make `EventHandler.get_event` return `None` when the requested event is not ↵Andrew Morgan2023-03-213-4/+11
|/ | | | found (#15298)
* Document that our Docker images are mirrored to GHCR. (#15282)reivilibre2023-03-212-2/+3
| | | Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Fix error when sending message into deleted room. (#15235)Erik Johnston2023-03-213-2/+31
| | | | | | | | When a room is deleted in Synapse we remove the event forward extremities in the room, so if (say a bot) tries to send a message into the room we error out due to not being able to calculate prev events for the new event *before* we check if the sender is in the room. Fixes #8094
* Separate HTTP preview code and URL previewer. (#15269)Patrick Cloke2023-03-204-810/+854
| | | Separates REST layer code from the actual URL previewing.
* Add Synapse-Trace-Id to access-control-expose-headers header (#14974)Shay2023-03-203-0/+9
|
* Mirror images to the GitHub Container Registry ↵reivilibre2023-03-202-1/+12
| | | | (`ghcr.io/matrix-org/synapse`). (#15281)
* Add /versions flag for MSC3952. (#15293)Patrick Cloke2023-03-202-0/+3
|
* Bump pygithub from 1.57 to 1.58.1 (#15290)dependabot[bot]2023-03-202-7/+8
|
* Bump serde from 1.0.155 to 1.0.157 (#15287)dependabot[bot]2023-03-202-11/+23
|
* Bump anyhow from 1.0.69 to 1.0.70 (#15288)dependabot[bot]2023-03-202-2/+3
|
* Bump pydantic from 1.10.4 to 1.10.6 (#15286)dependabot[bot]2023-03-202-37/+38
|
* Bump txredisapi from 1.4.7 to 1.4.9 (#15289)dependabot[bot]2023-03-202-3/+4
|
* Bump types-requests from 2.28.11.12 to 2.28.11.15 (#15291)dependabot[bot]2023-03-202-3/+4
|
* Load `/register/available` endpoint on workers (#15268)Jason Little2023-03-174-1/+4
|
* Make `configure_workers_and_start` script used in Complement tests ↵reivilibre2023-03-172-1/+2
| | | | compatible with older versions of Python. (#15275)
* Remove no-op send_command for Redis replication. (#15274)Patrick Cloke2023-03-164-89/+3
| | | | | With Redis commands do not need to be re-issued by the main process (they fan-out to all processes at once) and thus it is no longer necessary to worry about them reflecting recursively forever.
* Implement MSC2659: application service ping endpoint (#15249)Tulir Asokan2023-03-167-0/+141
| | | Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Refresh remote profiles that have been marked as stale, in order to fill the ↵reivilibre2023-03-165-4/+504
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | user directory. [rei:userdirpriv] (#14756) * Scaffolding for background process to refresh profiles * Add scaffolding for background process to refresh profiles for a given server * Implement the code to select servers to refresh from * Ensure we don't build up multiple looping calls * Make `get_profile` able to respect backoffs * Add logic for refreshing users * When backing off, schedule a refresh when the backoff is over * Wake up the background processes when we receive an interesting state event * Add tests * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Add comment about 1<<62 --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Move Account Validity callbacks to a dedicated file (#15237)Andrew Morgan2023-03-168-106/+154
|
* Preparatory work to fix the user directory assuming that any remote ↵reivilibre2023-03-164-34/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | membership state events represent a profile change. [rei:userdirpriv] (#14755) * Remove special-case method for new memberships only, use more generic method * Only collect profiles from state events in public rooms * Add a table to track stale remote user profiles * Add store methods to set and delete rows in this new table * Mark remote profiles as stale when a member state event comes in to a private room * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Simplify by removing Optionality of `event_id` * Replace names and avatars with None if they're set to dodgy things I think this makes more sense anyway. * Move schema delta to 74 (I missed the boat?) * Turns out these can be None after all --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Remove unused class: DirectTcpReplicationClientFactory. (#15272)Patrick Cloke2023-03-152-51/+1
|
* Enable running tests & release artifacts on merge queue. (#15244)Patrick Cloke2023-03-153-1/+5
|
* Add schema comments about the `destinations` and `destination_rooms` tables. ↵reivilibre2023-03-152-0/+53
| | | | (#15247)
* Remove unused store method `_set_destination_retry_timings_emulated`. (#15266)reivilibre2023-03-142-54/+3
|
* Merge branch 'master' into developMathieu Velten2023-03-143-1/+13
|\
| * 1.79.0 v1.79.0Mathieu Velten2023-03-143-1/+13
| |
* | Additional functionality for declaring worker types in Complement (#14921)Jason Little2023-03-143-115/+413
| | | | | | | | Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* | Avoid unneeded work if auto-join rooms aren't configured. (#15262)Patrick Cloke2023-03-142-2/+9
| | | | | | | | | | | | It is not necessary to reach out to the database to check some parameters if the auto-join rooms are not configured, or (in some cases) if auto-create rooms is not configured.
* | Merge branch 'release-v1.79' into developDavid Robertson2023-03-1311-44/+271
|\|
| * 1.79.0rc2 v1.79.0rc2David Robertson2023-03-136-4/+23
| |
| * Ensure fed-sender catchup does not block for full state (#15248)David Robertson2023-03-134-3/+125
| | | | | | | | * Reproduce bad scenario in test * Avoid catchup optimisation for partial state rooms
| * Refactor `filter_events_for_server` (#15240)David Robertson2023-03-105-30/+109
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Tweak docstring and type hint * Flip logic and provide better name * Separate decision from action * Track a set of strings, not EventBases * Require explicit boolean options from callers * Add explicit option for partial state rooms * Changelog * Rename param
| * Fix missing conditional for registering ↵Andrew Morgan2023-03-103-9/+16
| | | | | | | | `on_remove_user_third_party_identifier` module api callbacks (#15227
| * Fix typo in changelogDavid Robertson2023-03-071-1/+1
| |
* | Bump cryptography from 39.0.1 to 39.0.2 (#15257)dependabot[bot]2023-03-132-24/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump cryptography from 39.0.1 to 39.0.2 Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 39.0.2. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/39.0.1...39.0.2) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Install rust during Stage 0 of docker build (#15239)David Robertson2023-03-132-1/+17
| | | | | | | | | | | | | | | | * Install rust during Stage 0 of docker build Thanks to @atomdmac for spotting the fix. Fixes #15179. * Changelog
* | Bump hiredis from 2.2.1 to 2.2.2 (#15252)dependabot[bot]2023-03-132-90/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump hiredis from 2.2.1 to 2.2.2 Bumps [hiredis](https://github.com/redis/hiredis-py) from 2.2.1 to 2.2.2. - [Release notes](https://github.com/redis/hiredis-py/releases) - [Changelog](https://github.com/redis/hiredis-py/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/hiredis-py/compare/v2.2.1...v2.2.2) --- updated-dependencies: - dependency-name: hiredis dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump serde from 1.0.152 to 1.0.155 (#15253)dependabot[bot]2023-03-132-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump serde from 1.0.152 to 1.0.155 Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.155. - [Release notes](https://github.com/serde-rs/serde/releases) - [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.155) --- updated-dependencies: - dependency-name: serde dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump pysaml2 from 7.2.1 to 7.3.1 (#15254)dependabot[bot]2023-03-132-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump pysaml2 from 7.2.1 to 7.3.1 Bumps [pysaml2](https://github.com/IdentityPython/pysaml2) from 7.2.1 to 7.3.1. - [Release notes](https://github.com/IdentityPython/pysaml2/releases) - [Changelog](https://github.com/IdentityPython/pysaml2/blob/v7.3.1/CHANGELOG.md) - [Commits](https://github.com/IdentityPython/pysaml2/compare/v7.2.1...v7.3.1) --- updated-dependencies: - dependency-name: pysaml2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump msgpack from 1.0.4 to 1.0.5 (#15255)dependabot[bot]2023-03-132-53/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump msgpack from 1.0.4 to 1.0.5 Bumps [msgpack](https://github.com/msgpack/msgpack-python) from 1.0.4 to 1.0.5. - [Release notes](https://github.com/msgpack/msgpack-python/releases) - [Changelog](https://github.com/msgpack/msgpack-python/blob/main/ChangeLog.rst) - [Commits](https://github.com/msgpack/msgpack-python/compare/v1.0.4...v1.0.5) --- updated-dependencies: - dependency-name: msgpack dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump gitpython from 3.1.30 to 3.1.31 (#15256)dependabot[bot]2023-03-132-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump gitpython from 3.1.30 to 3.1.31 Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.31. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.31) --- updated-dependencies: - dependency-name: gitpython dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Give PyCharm some help with `@cache_in_self` (#15238)David Robertson2023-03-092-3/+27
| | | | | | | | | | | | | | * Give PyCharm some help with `@cache_in_self` * Changelog * Fix import for old python versions
* | Faster joins: Fix spurious errors on incremental sync (#15232)Sean Quah2023-03-092-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When pushing events in partial state rooms down incremental /sync, we try to find the `m.room.member` state event for their senders by digging through their auth events, so that we can present the membership to the client. Events usually have a membership event in their auth events, with the exception of the `m.room.create` event and a user's first join into the room. When implementing #13477, we took the case of a user's first join into account, but forgot to handle the `m.room.create` case. This change fixes that. Signed-off-by: Sean Quah <seanq@matrix.org>
* | Add missing type hints to `synapse.storage.database`. (#15230)Patrick Cloke2023-03-093-8/+17
| |
* | Pull in netaddr type hints. (#15231)Patrick Cloke2023-03-097-12/+23
| | | | | | And fix any issues from having those type hints.
* | Add topic and name events to group of events that are batch persisted when ↵Shay2023-03-082-56/+53
| | | | | | | | creating a room. (#15229)
* | Include the room ID in more purge room log lines. (#15222)Patrick Cloke2023-03-085-15/+23
| |
* | Add a missing endpoint to the workers documentation. (#15223)Patrick Cloke2023-03-082-0/+2
| |
* | More speedups/fixes to creating batched events (#15195)Shay2023-03-075-9/+33
| |
* | Stabilize support for MSC3873: disambuguated event push keys. (#15190)Patrick Cloke2023-03-075-45/+15
| | | | | | | | | | | | | | This removes the experimental configuration option and always escapes the push rule condition keys. Also escapes any (experimental) push rule condition keys in the base rules which contain dot in a field name.
* | Pass the Requester down to the HttpTransactionCache. (#15200)Quentin Gliech2023-03-076-129/+215
| |
* | Stabilize support for MSC3966: event_property_contains push condition. (#15187)Patrick Cloke2023-03-079-55/+13
|/ | | | This removes the configuration flag & updates the identifiers to use the stable version.
* Remove duplicate entry from changelog v1.79.0rc1David Robertson2023-03-071-1/+0
|
* Update changelogDavid Robertson2023-03-071-21/+24
|
* 1.79.0rc1David Robertson2023-03-0763-61/+102
|
* Hack to rebuild the complement editable image (#15184)David Robertson2023-03-072-1/+12
| | | | | * Hack to rebuild the complement editable image * Changelog
* Bump types-pyopenssl from 22.1.0.2 to 23.0.0.4 (#15213)dependabot[bot]2023-03-072-44/+5
|
* Bump types-pillow from 9.4.0.13 to 9.4.0.17 (#15211)dependabot[bot]2023-03-072-3/+4
|
* Bump types-setuptools from 67.4.0.3 to 67.5.0.0 (#15212)dependabot[bot]2023-03-072-3/+4
|
* Bump types-psycopg2 from 2.9.21.4 to 2.9.21.8 (#15210)dependabot[bot]2023-03-072-3/+4
|
* Bump types-commonmark from 0.9.2.1 to 0.9.2.2 (#15209)dependabot[bot]2023-03-072-3/+4
|
* Split up txn for fetching device keys (#15215)Erik Johnston2023-03-073-9/+26
| | | | We look up keys in batches, but we should do that outside of the transaction to avoid starving the database pool.
* Pass the requester during event serialization. (#15174)Quentin Gliech2023-03-0612-75/+151
| | | | This allows Synapse to properly include the transaction ID in the unsigned data of events.
* Stop applying edits to event contents (MSC3925). (#15193)Patrick Cloke2023-03-066-109/+15
| | | | | | | | | | | Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.
* Stabilize support for MSC3758: event_property_is push condition (#15185)Patrick Cloke2023-03-0610-81/+39
| | | | This removes the configuration flag & updates the identifiers to use the stable version.
* Bump serde_json from 1.0.93 to 1.0.94 (#15214)Co-authored-by: ↵dependabot[bot]2023-03-062-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> * Bump serde_json from 1.0.93 to 1.0.94 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.93 to 1.0.94. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.93...v1.0.94) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Use nightly rustfmt in CI (#15188)Erik Johnston2023-03-034-2/+5
| | | | As we use some nightly only options, e.g. to group and sort imports consistently.
* Fix ICU tests on alpine / macOS. (#15177)65432023-03-032-0/+3
| | | | The word boundary behaviour is slightly different, consider it acceptable for the tests.
* Combine AbstractStreamIdTracker and AbstractStreamIdGenerator. (#15192)Patrick Cloke2023-03-037-27/+15
| | | | | AbstractStreamIdTracker (now) has only a single sub-class: AbstractStreamIdGenerator, combine them to simplify some code and remove any direct references to AbstractStreamIdTracker.
* Remove unspecced and buggy `PUT` method on the unstable ↵Quentin Gliech2023-03-032-15/+2
| | | | `/rooms/<room_id>/batch_send` endpoint. (#15199)
* Configure ruff to automatically fix issues. (#15194)Patrick Cloke2023-03-032-1/+2
|
* Experimental MSC3890 Implementation: Fix deleting account data when using an ↵Andrew Morgan2023-03-033-26/+16
| | | | account data writer worker (#14869)
* Add a `get_next_txn` method to `StreamIdGenerator` to match ↵Andrew Morgan2023-03-024-11/+48
| | | | `MultiWriterIdGenerator` (#15191
* Add support for knocking to workers. (#15133)Dirk Klimpel2023-03-029-16/+15
|
* Remove the unspecced and bugged PUT /knock/{roomIdOrAlias} endpoint (#15189)Quentin Gliech2023-03-022-15/+2
|
* Dockerfile-workers: spell out when config isn't generated (#15186)David Robertson2023-03-022-1/+6
| | | | | * Complement: Spell out when config isn't generated * Changelog
* Update intentional mentions (MSC3952) to depend on ↵Patrick Cloke2023-03-0211-94/+73
| | | | | | | `exact_event_property_contains` (MSC3966). (#15051) This replaces the specific `is_user_mention` push rule condition used in MSC3952 with the generic `exact_event_property_contains` push rule condition from MSC3966.
* Fix conflicting URLs for dehydrated devices. (#15180)Patrick Cloke2023-03-022-1/+2
|
* Add Sytest jobs with the asyncio reactor enabled (#14101)Quentin Gliech2023-03-023-0/+17
|
* Move event_reports to `RoomWorkerStore` (#15165)Dirk Klimpel2023-03-022-177/+178
|
* Implementation of MSC3967: Don't require UIA for initial upload of cross ↵Hugh Nimmo-Smith2023-03-025-9/+182
| | | | signing keys (#15077)
* Remove support for aggregating reactions (#15172)Richard van der Hoff2023-02-288-377/+30
| | | | | | | | | | It turns out that no clients rely on server-side aggregation of `m.annotation` relationships: it's just not very useful as currently implemented. It's also non-trivial to calculate. I want to remove it from MSC2677, so to keep the implementation in line, let's remove it here.
* Merge branch 'master' into developH. Shay2023-02-285-1/+95
|\
| * 1.78.0 v1.78.0H. Shay2023-02-284-2/+16
| |
| * Fix bug where 5s delays would occasionally happen. (#15150)Erik Johnston2023-02-243-0/+80
| | | | | | This only affects deployments using workers.
* | Update spam checker documentation for moved media modules. (#15175)Patrick Cloke2023-02-282-2/+3
| |
* | Fix a long-standing bug where an initial sync would not respond to changes ↵reivilibre2023-02-284-2/+77
| | | | | | | | to the list of ignored users if there was an initial sync cached. (#15163)
* | Allow use of the `/filter` Client-Server APIs on workers. (#15134)reivilibre2023-02-286-8/+27
| |
* | Add missing type hints to InsecureInterceptableContextFactory. (#15164)Patrick Cloke2023-02-283-5/+4
| |
* | Do not accept pattern_type from user input in push rules. (#15088)Patrick Cloke2023-02-286-186/+244
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Internally the push rules module uses a `pattern_type` property for `event_match` conditions (and `related_event_match`) to mark the condition as matching the current user's Matrix ID or localpart. This is leaky to the Client-Server API where a user can successfully set a condition which provides `pattern_type` instead of `pattern` (note that there's no benefit to doing this -- the user can just use their own Matrix ID or localpart instead). When serializing back to the client the `pattern_type` property is converted into a proper `pattern`. The following changes are made to avoid this: * Separate the `KnownCondition::EventMatch` enum value into `EventMatch` and `EventMatchType`, each with their own expected properties. (Note that a similar change is made for `RelatedEventMatch`.) * Make it such that the `pattern_type` variants serialize to the same condition kind, but cannot be deserialized (since they're only provided by base rules). * As a final tweak, convert `user_id` vs. `user_localpart` values into an enum.
* | Add documentation for caching in a module (#14026)Brendan Abolivier2023-02-282-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add documentation for caching in a module * Changelog * Formatting * Wrap lines at a length that mdbook is happier with * Typo fix Co-authored-by: Erik Johnston <erik@matrix.org> * Link to recent version of the API In the longer term I'd like to see us generate markdown with Sphinx. * Refer to public `cached` decorator * Mark caching as being added in 1.74 Some of the underlying infrastructure was added in 1.69, but the public-facing `cached` decorator was only added in 1.74. It is the latter that I think we should be advertising. * Update docs/modules/writing_a_module.md Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> --------- Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Erik Johnston <erik@matrix.org> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* | Admin API endpoint to delete a reported event (#15116)Dirk Klimpel2023-02-285-11/+224
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Admin api to delete event report * lint + tests * newsfile * Apply suggestions from code review Co-authored-by: David Robertson <david.m.robertson1@gmail.com> * revert changes - move to WorkerStore * update unit test * Note that timestamp is in millseconds --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
* | Correct documentation about registration_shared_secret_path (#15168)Evan Krall2023-02-282-2/+3
| | | | | | | | | | | | | | | | | | | | | | * Correct documentation about registration_shared_secret_path * Create 15168.doc * Update changelog.d/15168.doc --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
* | Remove dangling reference to being a reference implementation (#15167)Travis Ralston2023-02-272-2/+4
| | | | | | | | | | * Remove dangling reference to being a reference implementation * Create 15167.misc
* | Add module API callbacks for adding and deleting local 3PID associations (#15044Andrew Morgan2023-02-2712-48/+324
| |
* | Refactor media modules. (#15146)Patrick Cloke2023-02-2731-1123/+1190
| | | | | | | | | | | | | | * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
* | Small fixes to `MatrixFederationHttpClient` docstrings (#15148)Andrew Morgan2023-02-272-7/+10
| |
* | Bump ruff from 0.0.237 to 0.0.252 (#15159)dependabot[bot]2023-02-273-19/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump ruff from 0.0.237 to 0.0.252 Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.237 to 0.0.252. - [Release notes](https://github.com/charliermarsh/ruff/releases) - [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md) - [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.237...v0.0.252) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump types-setuptools from 67.3.0.1 to 67.4.0.3 (#15160)dependabot[bot]2023-02-272-18/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump types-setuptools from 67.3.0.1 to 67.4.0.3 Bumps [types-setuptools](https://github.com/python/typeshed) from 67.3.0.1 to 67.4.0.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump types-opentracing from 2.4.10.1 to 2.4.10.3 (#15158)dependabot[bot]2023-02-272-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump types-opentracing from 2.4.10.1 to 2.4.10.3 Bumps [types-opentracing](https://github.com/python/typeshed) from 2.4.10.1 to 2.4.10.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-opentracing dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump typing-extensions from 4.4.0 to 4.5.0 (#15157)dependabot[bot]2023-02-272-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump typing-extensions from 4.4.0 to 4.5.0 Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.4.0 to 4.5.0. - [Release notes](https://github.com/python/typing_extensions/releases) - [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md) - [Commits](https://github.com/python/typing_extensions/compare/4.4.0...4.5.0) --- updated-dependencies: - dependency-name: typing-extensions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump matrix-org/backend-meta from 1 to 2 (#15156)dependabot[bot]2023-02-272-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump matrix-org/backend-meta from 1 to 2 Bumps [matrix-org/backend-meta](https://github.com/matrix-org/backend-meta) from 1 to 2. - [Release notes](https://github.com/matrix-org/backend-meta/releases) - [Commits](https://github.com/matrix-org/backend-meta/compare/v1...v2) --- updated-dependencies: - dependency-name: matrix-org/backend-meta dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump actions/checkout from 2 to 3 (#15155)dependabot[bot]2023-02-272-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump actions/checkout from 2 to 3 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump docker/login-action from 1 to 2 (#15154)dependabot[bot]2023-02-272-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump docker/login-action from 1 to 2 Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump dawidd6/action-download-artifact from 2.25.0 to 2.26.0 (#15152)dependabot[bot]2023-02-272-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump dawidd6/action-download-artifact from 2.25.0 to 2.26.0 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.25.0 to 2.26.0. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/b59d8c6a6c5c6c6437954f470d963c0b20ea7415...5e780fc7bbd0cac69fc73271ed86edf5dcb72d67) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Batch up storing state groups when creating new room (#14918)Shay2023-02-2414-49/+371
| |
* | Improve handling of non-ASCII characters in user directory search (#15143)Sean Quah2023-02-243-2/+184
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix a long-standing bug where non-ASCII characters in search terms, including accented letters, would not match characters in a different case. * Fix a long-standing bug where search terms using combining accents would not match display names using precomposed accents and vice versa. To fully take effect, the user directory must be rebuilt after this change. Fixes #14630. Signed-off-by: Sean Quah <seanq@matrix.org>
* | Do not fail completely if oEmbed autodiscovery fails. (#15092)Patrick Cloke2023-02-233-13/+65
| | | | | | | | | | | | Previously if an autodiscovered oEmbed request failed (e.g. the oEmbed endpoint is down or does not exist) then the entire URL preview would fail. Instead we now return everything we can, even if this additional request fails.
* | Stop parsing the unspecced type parameter on thumbnail requests. (#15137)Patrick Cloke2023-02-232-1/+3
| | | | | | | | | | | | | | | | Ideally we would replace this with parsing of the Accept header or something else, but for now just make Synapse spec compliant by ignoring the unspecced parameter. It does not seem that this is ever sent by a client, and even if it is there's a reasonable fallback.
* | Fix a typo in MSC3873 config option. (#15138)Patrick Cloke2023-02-234-9/+10
| | | | | | | | Previously the experimental configuration option referred to the wrong MSC number.
* | Fix typo in federation_verify_certificates in config documentation. (#15139)Centzilius2023-02-232-1/+2
| |
* | Add information on uploaded media to user export command. (#15107)Dirk Klimpel2023-02-235-16/+136
| |
* | Documentation using Shibboleth with OIDC Plugin for SSO. (#15112)Fly2023-02-232-0/+42
| |
* | Update .git-blame-ignore-revs for #15103.Patrick Cloke2023-02-221-1/+5
| |
* | Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-22117-218/+108
| |
* | Tighten the default rate limit of creating new devices. (#15135)Patrick Cloke2023-02-223-5/+15
| |
* | Use `json.dump` in `FileExfiltrationWriter` (#15095)Dirk Klimpel2023-02-222-8/+9
| | | | | | | | To directly write to the open file, instead of writing to an in-memory string first.
* | Clarify the workers that the ThirdPartyRules' `on_new_event` callback will ↵Andrew Morgan2023-02-222-0/+4
| | | | | | | | run on (#15071)
* | Remove unused `room_alias` field from `/createRoom` response (#15093)David Robertson2023-02-2210-39/+40
|/ | | | | | | | | | | | | | * Change `create_room` return type * Don't return room alias from /createRoom * Update other callsites * Fix up mypy complaints It looks like new_room_user_id is None iff new_room_id is None. It's a shame we haven't expressed this in a way that mypy can understand. * Changelog
* Tweak changelog v1.78.0rc1Olivier Wilkinson (reivilibre)2023-02-211-4/+3
|
* 1.78.0rc1Olivier Wilkinson (reivilibre)2023-02-2171-73/+94
|
* Tweak logging for when a worker waits for its view of a replication stream ↵reivilibre2023-02-212-2/+11
| | | | | | | | | | | | | | | | | | to catch up. (#15120)Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Improve logging messages for the 'wait for repl stream' read-after-write consistency feature * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Update synapse/replication/tcp/client.py Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Upper-bound frozendict dependency (#15114)David Robertson2023-02-213-2/+5
| | | | | | | | | | | * Upper-bound frozendict dependency This is an ugly kludge to solve https://github.com/matrix-org/synapse/issues/15109. It is not the most friendly thing to do for downstream packagers (apologies), but we are a) running low on time at the moment, and b) seeking to remove frozendict anyway. * Changelog
* Add `matrix-org-archive-keyring` package as `Recommends` (#15110)Erik Johnston2023-02-202-0/+7
| | | | This is so installations will pull in the keyring package, allowing us to update the expiry time of the `packages.matrix.org` repository.
* Update database_maintenance_tools.md (#15083)jahway6032023-02-202-2/+3
| | | | | | | | | | * Update database_maintenance_tools.md Included a blog post by Jackson Chen, which DID work when I followed it to perform Matrix Synapse Maintenance, versus the 2020 blog post by Victor Berger, which DID NOT work when performining maintenance. * Update database_maintenance_tools.md * Rephrasing
* Teach portdb about `un_partial_stated_event_stream` (#15108)David Robertson2023-02-202-33/+53
| | | | | | | | | | | | * Sort BOOLEAN_COLUMNS and APPEND_ONLY_TABLES So I can see if a given table is present in logarithmic time, rather than linear. * Teach portdb about `un_partial_stated_event_streams` * Comments comments comments * Changelog
* Allow health listener resource to load (#15096)realtyem2023-02-202-0/+2
| | | | | | | * Allow health listener resource to load. * changelog * Update changelog.d/15096.bugfix
* Fix a bug introduced in Synapse v1.74.0 where searching with colons when ↵reivilibre2023-02-204-5/+90
| | | | | | using ICU for search term tokenisation would fail with an error. (#15079) Co-authored-by: David Robertson <davidr@element.io>
* Bump types-setuptools from 67.1.0.0 to 67.3.0.1 (#15105)dependabot[bot]2023-02-202-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump types-setuptools from 67.1.0.0 to 67.3.0.1 Bumps [types-setuptools](https://github.com/python/typeshed) from 67.1.0.0 to 67.3.0.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump types-pillow from 9.4.0.10 to 9.4.0.13 (#15104)dependabot[bot]2023-02-202-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump types-pillow from 9.4.0.10 to 9.4.0.13 Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.10 to 9.4.0.13. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump dawidd6/action-download-artifact from 2.24.3 to 2.25.0 (#15102)dependabot[bot]2023-02-202-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | * Bump dawidd6/action-download-artifact from 2.24.3 to 2.25.0 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.24.3 to 2.25.0. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/bd10f381a96414ce2b13a11bfa89902ba7cea07f...b59d8c6a6c5c6c6437954f470d963c0b20ea7415) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump dtolnay/rust-toolchain from 25dc93b901a87e864900a8aec6c12e9aa794c0c3 to ↵dependabot[bot]2023-02-204-15/+16
| | | | | | | | | | | | | | | | | | | | | | | | | e12eda571dc9a5ee5d58eecf4738ec291c66f295 (#15101) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from 25dc93b901a87e864900a8aec6c12e9aa794c0c3 to e12eda571dc9a5ee5d58eecf4738ec291c66f295. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/25dc93b901a87e864900a8aec6c12e9aa794c0c3...e12eda571dc9a5ee5d58eecf4738ec291c66f295) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump types-bleach from 5.0.3.1 to 6.0.0.0 (#15100)dependabot[bot]2023-02-202-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump types-bleach from 5.0.3.1 to 6.0.0.0 Bumps [types-bleach](https://github.com/python/typeshed) from 5.0.3.1 to 6.0.0.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-bleach dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump types-jsonschema from 4.17.0.3 to 4.17.0.5 (#15099)dependabot[bot]2023-02-202-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | * Bump types-jsonschema from 4.17.0.3 to 4.17.0.5 Bumps [types-jsonschema](https://github.com/python/typeshed) from 4.17.0.3 to 4.17.0.5. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-jsonschema dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Fix-up type hints in tests/server.py. (#15084)Patrick Cloke2023-02-179-129/+226
| | | | | This file was being ignored by mypy, we remove that and add the missing type hints & deal with any fallout.
* Add account data to export command (#14969)Dirk Klimpel2023-02-175-16/+79
| | | | | | | | | | | * Add account data to to export command * newsfile * remove not needed function * update newsfile * adopt #14973
* Mitigate a race where /make_join could 403 for restricted rooms (#15080)Sean Quah2023-02-172-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | Previously, when creating a join event in /make_join, we would decide whether to include additional fields to satisfy restricted room checks based on the current state of the room. Then, when building the event, we would capture the forward extremities of the room to use as prev events. This is subject to race conditions. For example, when leaving and rejoining a room, the following sequence of events leads to a misleading 403 response: 1. /make_join reads the current state of the room and sees that the user is still in the room. It decides to omit the field required for restricted room joins. 2. The leave event is persisted and the room's forward extremities are updated. 3. /make_join builds the event, using the post-leave forward extremities. The event then fails the restricted room checks. To mitigate the race, we move the read of the forward extremities closer to the read of the current state. Ideally, we would compute the state based off the chosen prev events, but that can involve state resolution, which is expensive. Signed-off-by: Sean Quah <seanq@matrix.org>
* Document how to start Synapse with Poetry (#14892)ZAID BIN TARIQ2023-02-162-0/+12
| | | | | | | | | | | | | | | | * Add Start Synapse with Poetry * Create 14892.doc * Apply suggestions from code review Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Update docs/workers.md --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Use mypy 1.0 (#15052)David Robertson2023-02-1617-104/+209
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update mypy and mypy-zope * Remove unused ignores These used to suppress ``` synapse/storage/engines/__init__.py:28: error: "__new__" must return a class instance (got "NoReturn") [misc] ``` and ``` synapse/http/matrixfederationclient.py:1270: error: "BaseException" has no attribute "reasons" [attr-defined] ``` (note that we check `hasattr(e, "reasons")` above) * Avoid empty body warnings, sometimes by marking methods as abstract E.g. ``` tests/handlers/test_register.py:58: error: Missing return statement [empty-body] tests/handlers/test_register.py:108: error: Missing return statement [empty-body] ``` * Suppress false positive about `JaegerConfig` Complaint was ``` synapse/logging/opentracing.py:450: error: Function "Type[Config]" could always be true in boolean context [truthy-function] ``` * Fix not calling `is_state()` Oops! ``` tests/rest/client/test_third_party_rules.py:428: error: Function "Callable[[], bool]" could always be true in boolean context [truthy-function] ``` * Suppress false positives from ParamSpecs ```` synapse/logging/opentracing.py:971: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] synapse/logging/opentracing.py:1017: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] ```` * Drive-by improvement to `wrapping_logic` annotation * Workaround false "unreachable" positives See https://github.com/Shoobx/mypy-zope/issues/91 ``` tests/http/test_proxyagent.py:626: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:762: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:826: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:838: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:845: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:151: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:452: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:60: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:93: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:127: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:152: error: Statement is unreachable [unreachable] ``` * Changelog * Tweak DBAPI2 Protocol to be accepted by mypy 1.0 Some extra context in: - https://github.com/matrix-org/python-canonicaljson/pull/57 - https://github.com/python/mypy/issues/6002 - https://mypy.readthedocs.io/en/latest/common_issues.html#covariant-subtyping-of-mutable-protocol-members-is-rejected * Pull in updated canonicaljson lib so the protocol check just works * Improve comments in opentracing I tried to workaround the ignores but found it too much trouble. I think the corresponding issue is https://github.com/python/mypy/issues/12909. The mypy repo has a PR claiming to fix this (https://github.com/python/mypy/pull/14677) which might mean this gets resolved soon? * Better annotation for INTERACTIVE_AUTH_CHECKERS * Drive-by AUTH_TYPE annotation, to remove an ignore
* Update intentional mentions (MSC3952) to depend on `exact_event_match` ↵Patrick Cloke2023-02-1610-59/+26
| | | | | | | | | | (MSC3758). (#15037) This replaces the specific `is_room_mention` push rule condition used in MSC3952 with the generic `exact_event_match` push rule condition from MSC3758. No functionality changes due to this.
* Fix a mistake in registration_shared_secret_path docs (#15078)saddfox2023-02-152-1/+2
| | | | | | * fix a typo in registration_shared_secret_path docs Signed-off-by: Filip Rutar <filip.rutar@gmail.com> * changelog
* Fix federated joins when the first server in the list is not in the room ↵Sean Quah2023-02-152-6/+6
| | | | | | | | (#15074) Previously we would give up upon receiving a 404 from the first server, instead of trying the rest of the servers in the list. Signed-off-by: Sean Quah <seanq@matrix.org>
* Make it easier to use DataGrip w/ Synapse's schema (#14982)David Robertson2023-02-158-3/+50
| | | | | | Also tweak the schema dump script: - add a note explaining myself how to use it -Explicitly call `poetry run`, because not everyone uses direnv :(
* Support for selecting the Redis logical database. (#15034)999lakhisidhu2023-02-155-0/+8
| | | | Note that this is only used for key-value store (cached values) and not for the pub/sub replication used by Synapse.
* Update the error code for duplicate annotation (#15075)Richard van der Hoff2023-02-153-1/+11
|
* Fix order of partial state tables when purging (#15068)David Robertson2023-02-143-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix order of partial state tables when purging `partial_state_rooms` has an FK on `events` pointing to the join event we get from `/send_join`, so we must delete from that table before deleting from `events`. **NB:** It would be nice to cancel any resync processes for the room being purged. We do not do this at present. To do so reliably we'd need an internal HTTP "replication" endpoint, because the worker doing the resync process may be different to that handling the purge request. The first time the resync process tries to write data after the deletion it will fail because we have deleted necessary data e.g. auth events. AFAICS it will not retry the resync, so the only downside to not cancelling the resync is a scary-looking traceback. (This is presumably extremely race-sensitive.) * Changelog * admist(?) -> between * Warn about a race * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Add final type hint to tests.unittest. (#15072)Patrick Cloke2023-02-1455-320/+433
| | | | Adds a return type to HomeServerTestCase.make_homeserver and deal with any variables which are no longer Any.
* Implement MSC3966: Add a push rule condition to search for a value in an ↵Patrick Cloke2023-02-149-42/+176
| | | | | | array. (#15045) The `exact_event_property_contains` condition can be used to search for a value inside of an array.
* Remove spurious `dont_notify` action from `.m.rule.reaction` (#15073)Richard van der Hoff2023-02-142-1/+2
| | | This does nothing and I want to remove it from the MSC.
* Prevent clients from reporting nonexistent events. (#13779)reivilibre2023-02-143-1/+23
|
* Merge branch 'master' into developMathieu Velten2023-02-143-1/+13
|\
| * 1.77.0 v1.77.0Mathieu Velten2023-02-143-1/+13
| |
* | Faster joins: Omit device list updates from partial state rooms in /sync ↵Sean Quah2023-02-142-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | (#15069) ...when lazy loading of members is not enabled. It's weird to notify a client that another user's device list has changed when the client doesn't think that they share a room. Note that when a room is un-partial stated, device list updates are emitted for every member in that room over /sync. Signed-off-by: Sean Quah <seanq@matrix.org>
* | Fix clashing DB txn name (#15070)Erik Johnston2023-02-142-1/+2
| | | | | | | | | | * Fix clashing DB txn name * Newsfile
* | Skip calculating unread push actions in `/sync` when `enable_push` is false. ↵Erik Johnston2023-02-143-0/+16
| | | | | | | | (#14980)
* | Change collection[str] to StrCollection in event_auth code (#14929)Harishankar Kumar2023-02-145-24/+19
| | | | | | Signed-off-by: Harishankar Kumar <hari01584@gmail.com>
* | Add check to ensure locked dependencies have source distributions available. ↵reivilibre2023-02-135-1/+87
| | | | | | | | | | | | (#14742)
* | Tweak comment on `_is_local_room_accessible` as part of room visibility in ↵reivilibre2023-02-132-2/+3
| | | | | | | | `/hierarchy` to clarify the condition for a room being visible. (#14834)
* | Update pyo3-log to v0.8.1 (#15043)V024602023-02-133-3/+4
| | | | | | Signed-off-by: Kai A. Hiller <V02460@gmail.com>
* | Refactor arguments of `try_unbind_threepid(_with_id_server)` from dict to ↵Andrew Morgan2023-02-135-39/+28
| | | | | | | | separate args (#15053)
* | Apply logging from hotfixes branch to develop (#15054)David Robertson2023-02-132-0/+4
| | | | | | | | | | | | | | | | | | * Apply logging from hotfixes branch to develop Part of #4826. Originally added in #11882. * Changelog
* | Bump sentry-sdk from 1.13.0 to 1.15.0 (#15065)dependabot[bot]2023-02-132-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump sentry-sdk from 1.13.0 to 1.15.0 Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.13.0 to 1.15.0. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.13.0...1.15.0) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump types-pillow from 9.4.0.5 to 9.4.0.10 (#15064)dependabot[bot]2023-02-132-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump types-pillow from 9.4.0.5 to 9.4.0.10 Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.5 to 9.4.0.10. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump types-requests from 2.28.11.8 to 2.28.11.12 (#15063)dependabot[bot]2023-02-132-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump types-requests from 2.28.11.8 to 2.28.11.12 Bumps [types-requests](https://github.com/python/typeshed) from 2.28.11.8 to 2.28.11.12. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-requests dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump serde_json from 1.0.92 to 1.0.93 (#15062)dependabot[bot]2023-02-132-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump serde_json from 1.0.92 to 1.0.93 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.92 to 1.0.93. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump systemd-python from 234 to 235 (#15061)dependabot[bot]2023-02-132-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump systemd-python from 234 to 235 Bumps [systemd-python](https://github.com/systemd/python-systemd) from 234 to 235. - [Release notes](https://github.com/systemd/python-systemd/releases) - [Changelog](https://github.com/systemd/python-systemd/blob/main/NEWS) - [Commits](https://github.com/systemd/python-systemd/compare/v234...v235) --- updated-dependencies: - dependency-name: systemd-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump dtolnay/rust-toolchain from 9cd00a88a73addc8617065438eff914dd08d0955 to ↵dependabot[bot]2023-02-134-15/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 25dc93b901a87e864900a8aec6c12e9aa794c0c3 (#15060) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from 9cd00a88a73addc8617065438eff914dd08d0955 to 25dc93b901a87e864900a8aec6c12e9aa794c0c3. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/9cd00a88a73addc8617065438eff914dd08d0955...25dc93b901a87e864900a8aec6c12e9aa794c0c3) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Bump bleach from 5.0.1 to 6.0.0 (#15059)dependabot[bot]2023-02-132-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump bleach from 5.0.1 to 6.0.0 Bumps [bleach](https://github.com/mozilla/bleach) from 5.0.1 to 6.0.0. - [Release notes](https://github.com/mozilla/bleach/releases) - [Changelog](https://github.com/mozilla/bleach/blob/main/CHANGES) - [Commits](https://github.com/mozilla/bleach/compare/v5.0.1...v6.0.0) --- updated-dependencies: - dependency-name: bleach dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Faster joins: don't stall when a user joins during a fast join (#14606)Mathieu Velten2023-02-1012-94/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #12801. Complement tests are at https://github.com/matrix-org/complement/pull/567. Avoid blocking on full state when handling a subsequent join into a partial state room. Also always perform a remote join into partial state rooms, since we do not know whether the joining user has been banned and want to avoid leaking history to banned users. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
* | Return read-only collections from `@cached` methods (#13755)Sean Quah2023-02-1027-77/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
* | Support for MSC3758: exact_event_match push condition (#14964)Patrick Cloke2023-02-109-41/+356
| | | | | | | | | | This specifies to search for an exact value match, instead of string globbing. It only works across non-compound JSON values (null, boolean, integer, and strings).
* | Avoid fetching unused account data in sync. (#14973)Patrick Cloke2023-02-108-90/+176
| | | | | | | | | | | | | | The per-room account data is no longer unconditionally fetched, even if all rooms will be filtered out. Global account data will not be fetched if it will all be filtered out.
* | Merge branch 'release-v1.77' into developDavid Robertson2023-02-105-3/+31
|\|
| * 1.77.0rc2 v1.77.0rc2David Robertson2023-02-105-3/+22
| |
| * Bump schema version (#15036)David Robertson2023-02-092-2/+8
| | | | | | | | | | | | | | | | * Bump schema version This should have been included in f10caa73eee0caa91cf373966104d1ededae2aee (and #14979). * Changelog
| * Fix bug in replication where response is cached (#15024)Erik Johnston2023-02-082-0/+3
| |
* | Avoid mutating cached values in `_generate_sync_entry_for_account_data` (#15047)Sean Quah2023-02-102-0/+3
| |
* | Refactor get_user_devices_from_cache to avoid mutating cached values. (#15040)Patrick Cloke2023-02-103-18/+25
| | | | | | | | | | | | | | | | The previous version of the code could mutate a cached value, but only if the input requested all devices of a user *and* a specific device. To avoid this nonsensical situation we no longer fetch a specific device ID if all of a user's devices are returned.
* | Fix exception on start up about device lists (#15041)Erik Johnston2023-02-102-0/+2
| | | | | | Fixes #15010.
* | Tag federation request spans with the worker name (#15042)David Robertson2023-02-093-8/+10
| | | | | | | | | | | | | | * Systematically include worker name as process info * Changelog * don't bother with inner setdefault
* | Add a class UnpersistedEventContext to allow for the batching up of storing ↵Shay2023-02-0914-162/+359
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | state groups (#14675) * add class UnpersistedEventContext * modify create new client event to create unpersistedeventcontexts * persist event contexts after creation * fix tests to persist unpersisted event contexts * cleanup * misc lints + cleanup * changelog + fix comments * lints * fix batch insertion? * reduce redundant calculation * add unpersisted event classes * rework compute_event_context, split into function that returns unpersisted event context and then persists it * use calculate_context_info to create unpersisted event contexts * update typing * $%#^&* * black * fix comments and consolidate classes, use attr.s for class * requested changes * lint * requested changes * requested changes * refactor to be stupidly explicit * clearer renaming and flow * make partial state non-optional * update docstrings --------- Co-authored-by: Erik Johnston <erik@matrix.org>
* | Do not always start a db txn on Postgres (#14840)Andrew Morgan2023-02-092-4/+10
| |
* | Bump ruff version from 0.0.230 to 0.0.237. (#15033)Andrew Morgan2023-02-093-19/+20
| |
* | Respond correctly to unknown methods on known endpoints (#14605)Patrick Cloke2023-02-098-51/+89
| | | | | | | | Respond with a 405 error if a request is received on a known endpoint, but to an unknown method, per MSC3743.
* | Avoid mutating cached room aliases. (#15038)Patrick Cloke2023-02-092-1/+3
| | | | | | | | | | This might cause incorrect data in other callers which are not expecting the canonical alias to be added into the response.
* | Add final type hint to synapse.server. (#15035)Patrick Cloke2023-02-097-12/+11
| |
* | Proper types for `tests.module_api` (#15031)David Robertson2023-02-094-54/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | * -> None for test methods * A first batch of type fixes * Introduce common parent test case * Fixup that big test method * tests.module_api passes mypy * Changelog
* | Add more missing type hints to tests. (#15028)Patrick Cloke2023-02-0814-111/+124
| |
* | Add missing type hints to tests. (#15027)Patrick Cloke2023-02-088-76/+70
| |
* | Limit concurrent event creation for a room to avoid state resolution when ↵Shay2023-02-082-3/+4
| | | | | | | | sending bursts of events to a local room (#14977)
* | Explicit disabling of disallowed_untyped_defs. (#15026)Patrick Cloke2023-02-082-58/+26
| | | | | | To make it easier to see which files still need to be fixed.
* | MSC3873: Escape keys when flattening dicts. (#15004)Patrick Cloke2023-02-084-4/+40
| | | | | | | | | | | | This disambiguates keys which attempt to match fields with a dot in them (e.g. m.relates_to). Disabled by default behind an experimental configuration flag.
* | Document how to run Synapse (#15022)Andy Balaam2023-02-082-0/+14
| | | | | | | | | | | | | | | | * Document how to run Synapse * Changelog for 15022 * Update docs/development/contributing_guide.md
* | Merge branch 'release-v1.77' into developDavid Robertson2023-02-0869-205/+88
|\|
| * Permalink to the docs v1.77.0rc1David Robertson2023-02-081-1/+1
| |
| * Note the revert (that I didn't PR)David Robertson2023-02-081-1/+1
| |
| * Merge the osx wheel fix with poetry-core 1.5.0David Robertson2023-02-081-2/+1
| |
| * Rearrange itemsDavid Robertson2023-02-081-23/+24
| |
| * Manually add new news fragmentsDavid Robertson2023-02-083-2/+2
| |
| * Hacky fix to make mac wheels (#15019)David Robertson2023-02-073-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Skip testing PyPy wheels One of the test builds on #15015 failed to install a pp38-* wheel because it didn't have access to the openssl headers to build `cryptography` from source. We don't run CI against PyPy so I'm going to be a meanie and skip testing the wheels. (And I've no idea why 3.8 was special in the first place, either.) * Hack the name of the wheel so cibw can test it I hate hate hate hate hate hate hate hate hate this * Changelog * Apply suggestions from code review Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> --------- Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
| * Include no actions instead of dont_notify for suppressing edits. (#15016)Patrick Cloke2023-02-072-1/+2
| |
| * Disambiguate `get_ex_outlier_stream_rows` queryDavid Robertson2023-02-071-4/+4
| | | | | | | | A backwards-compatible piece of #14979 that's safe to land now.
| * Revert "Add `event_stream_ordering` column to membership state tables (#14979)"David Robertson2023-02-074-145/+11
| | | | | | | | This reverts commit 5fdc12f482c68e2cdbb78d7db5de2cfe621720d4.
| * 1.77.0rc1David Robertson2023-02-0763-61/+67
| |
* | Bump cryptography from 38.0.4 to 39.0.1 (#15020)dependabot[bot]2023-02-082-30/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump cryptography from 38.0.4 to 39.0.1 Bumps [cryptography](https://github.com/pyca/cryptography) from 38.0.4 to 39.0.1. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/38.0.4...39.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* | Clarify limitations of SRV delegation in documentation (#14959)William Kray2023-02-082-0/+10
| | | | | | | | | | | | | | | | This PR just clarifies in the SRV DNS delegation document that there are still cases a user may have to serve files from `.well-known` endpoints, and this may not be a valid case for using SRV delegation. This has caused some confusion in a few cases. Signed-off-by: William Kray <github@williamkray.com>
* | Typecheck tests.rest.media.v1.test_media_storage (#15008)David Robertson2023-02-074-23/+35
|/ | | | | | | | | | | * Fix MediaStorage type hint * Typecheck tests.rest.media.v1.test_media_storage * Changelog * Remove assert and make the comment succinct * Fix syntax for olddeps
* Proper types for tests.test_terms_auth (#15007)David Robertson2023-02-073-7/+17
| | | | | * Proper types for tests.test_terms_auth * Changelog
* Add tests for using _flatten_dict with an event. (#15002)Patrick Cloke2023-02-073-9/+68
|
* Properly typecheck types.http (#14988)David Robertson2023-02-0717-191/+298
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Tweak http types in Synapse AFACIS these are correct, and they make mypy happier on tests.http. * Type hints for test_proxyagent * type hints for test_srv_resolver * test_matrix_federation_agent * tests.http.server._base * tests.http.__init__ * tests.http.test_additional_resource * tests.http.test_client * tests.http.test_endpoint * tests.http.test_matrixfederationclient * tests.http.test_servlet * tests.http.test_simple_client * tests.http.test_site * One fixup in tests.server * Untyped defs * Changelog * Fixup syntax for Python 3.7 * Fix olddeps syntax * Use a twisted IPv4 addr for dummy_address * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Remove redundant `Optional` --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Add `event_stream_ordering` column to membership state tables (#14979)Nick Mills-Barrett2023-02-075-11/+146
| | | | | | | | | | | | This adds an `event_stream_ordering` column to `current_state_events`, `local_current_membership` and `room_memberships`. Each of these tables is regularly joined with the `events` table to get the stream ordering and denormalising this into each table will yield significant query performance improvements once used. Includes a background job to populate these values from the `events` table. Same idea as https://github.com/matrix-org/synapse/pull/13703. Signed off by Nick @ Beeper (@fizzadar).
* Allow poetry-core 1.5.0 (#14949)icp2023-02-062-2/+5
|
* Add missing types to test_state. (#14985)Patrick Cloke2023-02-063-62/+141
|
* Type hints for tests.federation (#14991)David Robertson2023-02-0610-94/+127
| | | | | | | | | | | | | * Make tests.federation pass mypy * Untyped defs in tests.federation.transport * test methods return None * Remaining type hints in tests.federation * Changelog * Avoid an uncessary type-ignore
* Add missing type hints to tests.replication. (#14987)Patrick Cloke2023-02-0621-149/+193
|
* Expect type stubs from canonicaljson (#14992)David Robertson2023-02-062-3/+1
| | | | | | | | | * canonicaljson has stubs now since https://github.com/matrix-org/python-canonicaljson/pull/52 which is included in the lockfile version we use for type checking. * Changelog
* Type hints for tests.appservice (#14990)David Robertson2023-02-069-59/+132
| | | | | | | | | | | | | | | | | | | | * Accept a Sequence of events in synapse.appservice This avoids some casts/ignores in the tests I'm about to fixup. It seems that `List[Mock]` is not a subtype of `List[EventBase]`, but `Sequence[Mock]` is a subtype of `Sequence[EventBase]`. So presumably `Mock` is considered a subtype of anything, much like `Any`. * make tests.appservice.test_scheduler pass mypy * Extra hints in tests.appservice.test_scheduler * Extra hints in tests.appservice.test_api * Extra hints in tests.appservice.test_appservice * Disallow untyped defs * Changelog
* Bump anyhow from 1.0.68 to 1.0.69 (#14996)dependabot[bot]2023-02-062-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | * Bump anyhow from 1.0.68 to 1.0.69 Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.68 to 1.0.69. - [Release notes](https://github.com/dtolnay/anyhow/releases) - [Commits](https://github.com/dtolnay/anyhow/compare/1.0.68...1.0.69) --- updated-dependencies: - dependency-name: anyhow dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump phonenumbers from 8.13.4 to 8.13.5 (#14999)dependabot[bot]2023-02-062-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump phonenumbers from 8.13.4 to 8.13.5 Bumps [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) from 8.13.4 to 8.13.5. - [Release notes](https://github.com/daviddrysdale/python-phonenumbers/releases) - [Commits](https://github.com/daviddrysdale/python-phonenumbers/compare/v8.13.4...v8.13.5) --- updated-dependencies: - dependency-name: phonenumbers dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump isort from 5.11.4 to 5.11.5 (#14998)dependabot[bot]2023-02-062-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | * Bump isort from 5.11.4 to 5.11.5 Bumps [isort](https://github.com/pycqa/isort) from 5.11.4 to 5.11.5. - [Release notes](https://github.com/pycqa/isort/releases) - [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md) - [Commits](https://github.com/pycqa/isort/compare/5.11.4...5.11.5) --- updated-dependencies: - dependency-name: isort dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump serde_json from 1.0.91 to 1.0.92 (#14997)dependabot[bot]2023-02-062-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | * Bump serde_json from 1.0.91 to 1.0.92 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.91 to 1.0.92. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump prometheus-client from 0.15.0 to 0.16.0 (#14995)dependabot[bot]2023-02-062-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump prometheus-client from 0.15.0 to 0.16.0 Bumps [prometheus-client](https://github.com/prometheus/client_python) from 0.15.0 to 0.16.0. - [Release notes](https://github.com/prometheus/client_python/releases) - [Commits](https://github.com/prometheus/client_python/compare/v0.15.0...v0.16.0) --- updated-dependencies: - dependency-name: prometheus-client dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump types-setuptools from 65.6.0.3 to 67.1.0.0 (#14994)dependabot[bot]2023-02-062-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | * Bump types-setuptools from 65.6.0.3 to 67.1.0.0 Bumps [types-setuptools](https://github.com/python/typeshed) from 65.6.0.3 to 67.1.0.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Bump hiredis from 2.1.1 to 2.2.1 (#14993)dependabot[bot]2023-02-062-90/+91
| | | | | | | | | | | | | | | | | | | | | | | | | * Bump hiredis from 2.1.1 to 2.2.1 Bumps [hiredis](https://github.com/redis/hiredis-py) from 2.1.1 to 2.2.1. - [Release notes](https://github.com/redis/hiredis-py/releases) - [Changelog](https://github.com/redis/hiredis-py/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/hiredis-py/compare/v2.1.1...v2.2.1) --- updated-dependencies: - dependency-name: hiredis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
* Only notify the target of a membership event (#14971)David Robertson2023-02-062-8/+31
| | | | | * Only notify the target of a membership event Naughty, but should be a big speedup in large rooms
* Properly typecheck tests.api (#14983)David Robertson2023-02-037-110/+140
|
* Implement MSC3958: suppress notifications from edits (#14960)Patrick Cloke2023-02-039-2/+76
| | | | | | | | Co-authored-by: Brad Murray <brad@beeper.com> Co-authored-by: Nick Barrett <nick@beeper.com> Copy the suppress_edits push rule from Beeper to implement MSC3958. https://github.com/beeper/synapse/blame/9415a1284b1bfb558bd66f28c24ca1611e6c6fa2/rust/src/push/base_rules.rs#L98-L114