summary refs log tree commit diff
path: root/docs/tcp_replication.rst (unfollow)
Commit message (Collapse)AuthorFilesLines
2023-03-08Avoid state lookups for events we'll ignoreDavid Robertson1-5/+8
2023-03-08Track a set of strings, not EventBasesDavid Robertson1-4/+14
2023-03-08Separate decision from actionDavid Robertson1-3/+6
2023-03-08Unconditionally omit remote events in partial state roomsDavid Robertson1-8/+7
I can't see why erased senders makes a difference here?
2023-03-08Flip logic and provide better nameDavid Robertson2-7/+6
2023-03-08Tweak docstring and type hintDavid Robertson1-7/+19
2023-03-08TestDavid Robertson2-1/+107
2023-03-07Fix typo in changelogDavid Robertson1-1/+1
2023-03-07Remove duplicate entry from changelog v1.79.0rc1David Robertson1-1/+0
2023-03-07Update changelogDavid Robertson1-21/+24
2023-03-071.79.0rc1David Robertson63-61/+102
2023-03-07Hack to rebuild the complement editable image (#15184)David Robertson2-1/+12
* Hack to rebuild the complement editable image * Changelog
2023-03-07Bump types-pyopenssl from 22.1.0.2 to 23.0.0.4 (#15213)dependabot[bot]2-44/+5
2023-03-07Bump types-pillow from 9.4.0.13 to 9.4.0.17 (#15211)dependabot[bot]2-3/+4
2023-03-07Bump types-setuptools from 67.4.0.3 to 67.5.0.0 (#15212)dependabot[bot]2-3/+4
2023-03-07Bump types-psycopg2 from 2.9.21.4 to 2.9.21.8 (#15210)dependabot[bot]2-3/+4
2023-03-07Bump types-commonmark from 0.9.2.1 to 0.9.2.2 (#15209)dependabot[bot]2-3/+4
2023-03-07Split up txn for fetching device keys (#15215)Erik Johnston3-9/+26
We look up keys in batches, but we should do that outside of the transaction to avoid starving the database pool.
2023-03-06Pass the requester during event serialization. (#15174)Quentin Gliech12-75/+151
This allows Synapse to properly include the transaction ID in the unsigned data of events.
2023-03-06Stop applying edits to event contents (MSC3925). (#15193)Patrick Cloke6-109/+15
Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.
2023-03-06Stabilize support for MSC3758: event_property_is push condition (#15185)Patrick Cloke10-81/+39
This removes the configuration flag & updates the identifiers to use the stable version.
2023-03-06Bump serde_json from 1.0.93 to 1.0.94 (#15214)Co-authored-by: ↵dependabot[bot]2-2/+3
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> * Bump serde_json from 1.0.93 to 1.0.94 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.93 to 1.0.94. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.93...v1.0.94) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-03-03Use nightly rustfmt in CI (#15188)Erik Johnston4-2/+5
As we use some nightly only options, e.g. to group and sort imports consistently.
2023-03-03Fix ICU tests on alpine / macOS. (#15177)65432-0/+3
The word boundary behaviour is slightly different, consider it acceptable for the tests.
2023-03-03Combine AbstractStreamIdTracker and AbstractStreamIdGenerator. (#15192)Patrick Cloke7-27/+15
AbstractStreamIdTracker (now) has only a single sub-class: AbstractStreamIdGenerator, combine them to simplify some code and remove any direct references to AbstractStreamIdTracker.
2023-03-03Remove unspecced and buggy `PUT` method on the unstable ↵Quentin Gliech2-15/+2
`/rooms/<room_id>/batch_send` endpoint. (#15199)
2023-03-03Configure ruff to automatically fix issues. (#15194)Patrick Cloke2-1/+2
2023-03-03Experimental MSC3890 Implementation: Fix deleting account data when using an ↵Andrew Morgan3-26/+16
account data writer worker (#14869)
2023-03-02Add a `get_next_txn` method to `StreamIdGenerator` to match ↵Andrew Morgan4-11/+48
`MultiWriterIdGenerator` (#15191
2023-03-02Add support for knocking to workers. (#15133)Dirk Klimpel9-16/+15
2023-03-02Remove the unspecced and bugged PUT /knock/{roomIdOrAlias} endpoint (#15189)Quentin Gliech2-15/+2
2023-03-02Dockerfile-workers: spell out when config isn't generated (#15186)David Robertson2-1/+6
* Complement: Spell out when config isn't generated * Changelog
2023-03-02Update intentional mentions (MSC3952) to depend on ↵Patrick Cloke11-94/+73
`exact_event_property_contains` (MSC3966). (#15051) This replaces the specific `is_user_mention` push rule condition used in MSC3952 with the generic `exact_event_property_contains` push rule condition from MSC3966.
2023-03-02Fix conflicting URLs for dehydrated devices. (#15180)Patrick Cloke2-1/+2
2023-03-02Add Sytest jobs with the asyncio reactor enabled (#14101)Quentin Gliech3-0/+17
2023-03-02Move event_reports to `RoomWorkerStore` (#15165)Dirk Klimpel2-177/+178
2023-03-02Implementation of MSC3967: Don't require UIA for initial upload of cross ↵Hugh Nimmo-Smith5-9/+182
signing keys (#15077)
2023-02-28Remove support for aggregating reactions (#15172)Richard van der Hoff8-377/+30
It turns out that no clients rely on server-side aggregation of `m.annotation` relationships: it's just not very useful as currently implemented. It's also non-trivial to calculate. I want to remove it from MSC2677, so to keep the implementation in line, let's remove it here.
2023-02-28Update spam checker documentation for moved media modules. (#15175)Patrick Cloke2-2/+3
2023-02-28Fix a long-standing bug where an initial sync would not respond to changes ↵reivilibre4-2/+77
to the list of ignored users if there was an initial sync cached. (#15163)
2023-02-281.78.0 v1.78.0H. Shay4-2/+16
2023-02-28Allow use of the `/filter` Client-Server APIs on workers. (#15134)reivilibre6-8/+27
2023-02-28Add missing type hints to InsecureInterceptableContextFactory. (#15164)Patrick Cloke3-5/+4
2023-02-28Do not accept pattern_type from user input in push rules. (#15088)Patrick Cloke6-186/+244
Internally the push rules module uses a `pattern_type` property for `event_match` conditions (and `related_event_match`) to mark the condition as matching the current user's Matrix ID or localpart. This is leaky to the Client-Server API where a user can successfully set a condition which provides `pattern_type` instead of `pattern` (note that there's no benefit to doing this -- the user can just use their own Matrix ID or localpart instead). When serializing back to the client the `pattern_type` property is converted into a proper `pattern`. The following changes are made to avoid this: * Separate the `KnownCondition::EventMatch` enum value into `EventMatch` and `EventMatchType`, each with their own expected properties. (Note that a similar change is made for `RelatedEventMatch`.) * Make it such that the `pattern_type` variants serialize to the same condition kind, but cannot be deserialized (since they're only provided by base rules). * As a final tweak, convert `user_id` vs. `user_localpart` values into an enum.
2023-02-28Add documentation for caching in a module (#14026)Brendan Abolivier2-0/+57
* Add documentation for caching in a module * Changelog * Formatting * Wrap lines at a length that mdbook is happier with * Typo fix Co-authored-by: Erik Johnston <erik@matrix.org> * Link to recent version of the API In the longer term I'd like to see us generate markdown with Sphinx. * Refer to public `cached` decorator * Mark caching as being added in 1.74 Some of the underlying infrastructure was added in 1.69, but the public-facing `cached` decorator was only added in 1.74. It is the latter that I think we should be advertising. * Update docs/modules/writing_a_module.md Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> --------- Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Erik Johnston <erik@matrix.org> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2023-02-28Admin API endpoint to delete a reported event (#15116)Dirk Klimpel5-11/+224
* Admin api to delete event report * lint + tests * newsfile * Apply suggestions from code review Co-authored-by: David Robertson <david.m.robertson1@gmail.com> * revert changes - move to WorkerStore * update unit test * Note that timestamp is in millseconds --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2023-02-28Correct documentation about registration_shared_secret_path (#15168)Evan Krall2-2/+3
* Correct documentation about registration_shared_secret_path * Create 15168.doc * Update changelog.d/15168.doc --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2023-02-27Remove dangling reference to being a reference implementation (#15167)Travis Ralston2-2/+4
* Remove dangling reference to being a reference implementation * Create 15167.misc
2023-02-27Add module API callbacks for adding and deleting local 3PID associations (#15044Andrew Morgan12-48/+324
2023-02-27Refactor media modules. (#15146)Patrick Cloke31-1123/+1190
* Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
2023-02-27Small fixes to `MatrixFederationHttpClient` docstrings (#15148)Andrew Morgan2-7/+10
2023-02-27Bump ruff from 0.0.237 to 0.0.252 (#15159)dependabot[bot]3-19/+21
* Bump ruff from 0.0.237 to 0.0.252 Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.237 to 0.0.252. - [Release notes](https://github.com/charliermarsh/ruff/releases) - [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md) - [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.237...v0.0.252) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump types-setuptools from 67.3.0.1 to 67.4.0.3 (#15160)dependabot[bot]2-18/+4
* Bump types-setuptools from 67.3.0.1 to 67.4.0.3 Bumps [types-setuptools](https://github.com/python/typeshed) from 67.3.0.1 to 67.4.0.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump types-opentracing from 2.4.10.1 to 2.4.10.3 (#15158)dependabot[bot]2-3/+4
* Bump types-opentracing from 2.4.10.1 to 2.4.10.3 Bumps [types-opentracing](https://github.com/python/typeshed) from 2.4.10.1 to 2.4.10.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-opentracing dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump typing-extensions from 4.4.0 to 4.5.0 (#15157)dependabot[bot]2-3/+4
* Bump typing-extensions from 4.4.0 to 4.5.0 Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.4.0 to 4.5.0. - [Release notes](https://github.com/python/typing_extensions/releases) - [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md) - [Commits](https://github.com/python/typing_extensions/compare/4.4.0...4.5.0) --- updated-dependencies: - dependency-name: typing-extensions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump matrix-org/backend-meta from 1 to 2 (#15156)dependabot[bot]2-1/+2
* Bump matrix-org/backend-meta from 1 to 2 Bumps [matrix-org/backend-meta](https://github.com/matrix-org/backend-meta) from 1 to 2. - [Release notes](https://github.com/matrix-org/backend-meta/releases) - [Commits](https://github.com/matrix-org/backend-meta/compare/v1...v2) --- updated-dependencies: - dependency-name: matrix-org/backend-meta dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump actions/checkout from 2 to 3 (#15155)dependabot[bot]2-2/+3
* Bump actions/checkout from 2 to 3 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump docker/login-action from 1 to 2 (#15154)dependabot[bot]2-1/+2
* Bump docker/login-action from 1 to 2 Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-27Bump dawidd6/action-download-artifact from 2.25.0 to 2.26.0 (#15152)dependabot[bot]2-1/+2
* Bump dawidd6/action-download-artifact from 2.25.0 to 2.26.0 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.25.0 to 2.26.0. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/b59d8c6a6c5c6c6437954f470d963c0b20ea7415...5e780fc7bbd0cac69fc73271ed86edf5dcb72d67) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-24Batch up storing state groups when creating new room (#14918)Shay14-49/+371
2023-02-24Fix bug where 5s delays would occasionally happen. (#15150)Erik Johnston3-0/+80
This only affects deployments using workers.
2023-02-24Improve handling of non-ASCII characters in user directory search (#15143)Sean Quah3-2/+184
* Fix a long-standing bug where non-ASCII characters in search terms, including accented letters, would not match characters in a different case. * Fix a long-standing bug where search terms using combining accents would not match display names using precomposed accents and vice versa. To fully take effect, the user directory must be rebuilt after this change. Fixes #14630. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-23Do not fail completely if oEmbed autodiscovery fails. (#15092)Patrick Cloke3-13/+65
Previously if an autodiscovered oEmbed request failed (e.g. the oEmbed endpoint is down or does not exist) then the entire URL preview would fail. Instead we now return everything we can, even if this additional request fails.
2023-02-23Stop parsing the unspecced type parameter on thumbnail requests. (#15137)Patrick Cloke2-1/+3
Ideally we would replace this with parsing of the Accept header or something else, but for now just make Synapse spec compliant by ignoring the unspecced parameter. It does not seem that this is ever sent by a client, and even if it is there's a reasonable fallback.
2023-02-23Fix a typo in MSC3873 config option. (#15138)Patrick Cloke4-9/+10
Previously the experimental configuration option referred to the wrong MSC number.
2023-02-23Fix typo in federation_verify_certificates in config documentation. (#15139)Centzilius2-1/+2
2023-02-23Add information on uploaded media to user export command. (#15107)Dirk Klimpel5-16/+136
2023-02-23Documentation using Shibboleth with OIDC Plugin for SSO. (#15112)Fly2-0/+42
2023-02-22Update .git-blame-ignore-revs for #15103.Patrick Cloke1-1/+5
2023-02-22Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]117-218/+108
2023-02-22Tighten the default rate limit of creating new devices. (#15135)Patrick Cloke3-5/+15
2023-02-22Use `json.dump` in `FileExfiltrationWriter` (#15095)Dirk Klimpel2-8/+9
To directly write to the open file, instead of writing to an in-memory string first.
2023-02-22Clarify the workers that the ThirdPartyRules' `on_new_event` callback will ↵Andrew Morgan2-0/+4
run on (#15071)
2023-02-22Remove unused `room_alias` field from `/createRoom` response (#15093)David Robertson10-39/+40
* Change `create_room` return type * Don't return room alias from /createRoom * Update other callsites * Fix up mypy complaints It looks like new_room_user_id is None iff new_room_id is None. It's a shame we haven't expressed this in a way that mypy can understand. * Changelog
2023-02-21Tweak changelog v1.78.0rc1Olivier Wilkinson (reivilibre)1-4/+3
2023-02-211.78.0rc1Olivier Wilkinson (reivilibre)71-73/+94
2023-02-21Tweak logging for when a worker waits for its view of a replication stream ↵reivilibre2-2/+11
to catch up. (#15120)Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Improve logging messages for the 'wait for repl stream' read-after-write consistency feature * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Update synapse/replication/tcp/client.py Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-02-21Upper-bound frozendict dependency (#15114)David Robertson3-2/+5
* Upper-bound frozendict dependency This is an ugly kludge to solve https://github.com/matrix-org/synapse/issues/15109. It is not the most friendly thing to do for downstream packagers (apologies), but we are a) running low on time at the moment, and b) seeking to remove frozendict anyway. * Changelog
2023-02-20Add `matrix-org-archive-keyring` package as `Recommends` (#15110)Erik Johnston2-0/+7
This is so installations will pull in the keyring package, allowing us to update the expiry time of the `packages.matrix.org` repository.
2023-02-20Update database_maintenance_tools.md (#15083)jahway6032-2/+3
* Update database_maintenance_tools.md Included a blog post by Jackson Chen, which DID work when I followed it to perform Matrix Synapse Maintenance, versus the 2020 blog post by Victor Berger, which DID NOT work when performining maintenance. * Update database_maintenance_tools.md * Rephrasing
2023-02-20Teach portdb about `un_partial_stated_event_stream` (#15108)David Robertson2-33/+53
* Sort BOOLEAN_COLUMNS and APPEND_ONLY_TABLES So I can see if a given table is present in logarithmic time, rather than linear. * Teach portdb about `un_partial_stated_event_streams` * Comments comments comments * Changelog
2023-02-20Allow health listener resource to load (#15096)realtyem2-0/+2
* Allow health listener resource to load. * changelog * Update changelog.d/15096.bugfix
2023-02-20Fix a bug introduced in Synapse v1.74.0 where searching with colons when ↵reivilibre4-5/+90
using ICU for search term tokenisation would fail with an error. (#15079) Co-authored-by: David Robertson <davidr@element.io>
2023-02-20Bump types-setuptools from 67.1.0.0 to 67.3.0.1 (#15105)dependabot[bot]2-3/+4
* Bump types-setuptools from 67.1.0.0 to 67.3.0.1 Bumps [types-setuptools](https://github.com/python/typeshed) from 67.1.0.0 to 67.3.0.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-20Bump types-pillow from 9.4.0.10 to 9.4.0.13 (#15104)dependabot[bot]2-3/+4
* Bump types-pillow from 9.4.0.10 to 9.4.0.13 Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.10 to 9.4.0.13. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-20Bump dawidd6/action-download-artifact from 2.24.3 to 2.25.0 (#15102)dependabot[bot]2-1/+2
* Bump dawidd6/action-download-artifact from 2.24.3 to 2.25.0 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.24.3 to 2.25.0. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/bd10f381a96414ce2b13a11bfa89902ba7cea07f...b59d8c6a6c5c6c6437954f470d963c0b20ea7415) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-20Bump dtolnay/rust-toolchain from 25dc93b901a87e864900a8aec6c12e9aa794c0c3 to ↵dependabot[bot]4-15/+16
e12eda571dc9a5ee5d58eecf4738ec291c66f295 (#15101) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from 25dc93b901a87e864900a8aec6c12e9aa794c0c3 to e12eda571dc9a5ee5d58eecf4738ec291c66f295. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/25dc93b901a87e864900a8aec6c12e9aa794c0c3...e12eda571dc9a5ee5d58eecf4738ec291c66f295) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-20Bump types-bleach from 5.0.3.1 to 6.0.0.0 (#15100)dependabot[bot]2-3/+4
* Bump types-bleach from 5.0.3.1 to 6.0.0.0 Bumps [types-bleach](https://github.com/python/typeshed) from 5.0.3.1 to 6.0.0.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-bleach dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-20Bump types-jsonschema from 4.17.0.3 to 4.17.0.5 (#15099)dependabot[bot]2-4/+5
* Bump types-jsonschema from 4.17.0.3 to 4.17.0.5 Bumps [types-jsonschema](https://github.com/python/typeshed) from 4.17.0.3 to 4.17.0.5. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-jsonschema dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-17Fix-up type hints in tests/server.py. (#15084)Patrick Cloke9-129/+226
This file was being ignored by mypy, we remove that and add the missing type hints & deal with any fallout.
2023-02-17Add account data to export command (#14969)Dirk Klimpel5-16/+79
* Add account data to to export command * newsfile * remove not needed function * update newsfile * adopt #14973
2023-02-17Mitigate a race where /make_join could 403 for restricted rooms (#15080)Sean Quah2-1/+16
Previously, when creating a join event in /make_join, we would decide whether to include additional fields to satisfy restricted room checks based on the current state of the room. Then, when building the event, we would capture the forward extremities of the room to use as prev events. This is subject to race conditions. For example, when leaving and rejoining a room, the following sequence of events leads to a misleading 403 response: 1. /make_join reads the current state of the room and sees that the user is still in the room. It decides to omit the field required for restricted room joins. 2. The leave event is persisted and the room's forward extremities are updated. 3. /make_join builds the event, using the post-leave forward extremities. The event then fails the restricted room checks. To mitigate the race, we move the read of the forward extremities closer to the read of the current state. Ideally, we would compute the state based off the chosen prev events, but that can involve state resolution, which is expensive. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-16Document how to start Synapse with Poetry (#14892)ZAID BIN TARIQ2-0/+12
* Add Start Synapse with Poetry * Create 14892.doc * Apply suggestions from code review Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Update docs/workers.md --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-02-16Use mypy 1.0 (#15052)David Robertson17-104/+209
* Update mypy and mypy-zope * Remove unused ignores These used to suppress ``` synapse/storage/engines/__init__.py:28: error: "__new__" must return a class instance (got "NoReturn") [misc] ``` and ``` synapse/http/matrixfederationclient.py:1270: error: "BaseException" has no attribute "reasons" [attr-defined] ``` (note that we check `hasattr(e, "reasons")` above) * Avoid empty body warnings, sometimes by marking methods as abstract E.g. ``` tests/handlers/test_register.py:58: error: Missing return statement [empty-body] tests/handlers/test_register.py:108: error: Missing return statement [empty-body] ``` * Suppress false positive about `JaegerConfig` Complaint was ``` synapse/logging/opentracing.py:450: error: Function "Type[Config]" could always be true in boolean context [truthy-function] ``` * Fix not calling `is_state()` Oops! ``` tests/rest/client/test_third_party_rules.py:428: error: Function "Callable[[], bool]" could always be true in boolean context [truthy-function] ``` * Suppress false positives from ParamSpecs ```` synapse/logging/opentracing.py:971: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] synapse/logging/opentracing.py:1017: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] ```` * Drive-by improvement to `wrapping_logic` annotation * Workaround false "unreachable" positives See https://github.com/Shoobx/mypy-zope/issues/91 ``` tests/http/test_proxyagent.py:626: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:762: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:826: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:838: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:845: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:151: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:452: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:60: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:93: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:127: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:152: error: Statement is unreachable [unreachable] ``` * Changelog * Tweak DBAPI2 Protocol to be accepted by mypy 1.0 Some extra context in: - https://github.com/matrix-org/python-canonicaljson/pull/57 - https://github.com/python/mypy/issues/6002 - https://mypy.readthedocs.io/en/latest/common_issues.html#covariant-subtyping-of-mutable-protocol-members-is-rejected * Pull in updated canonicaljson lib so the protocol check just works * Improve comments in opentracing I tried to workaround the ignores but found it too much trouble. I think the corresponding issue is https://github.com/python/mypy/issues/12909. The mypy repo has a PR claiming to fix this (https://github.com/python/mypy/pull/14677) which might mean this gets resolved soon? * Better annotation for INTERACTIVE_AUTH_CHECKERS * Drive-by AUTH_TYPE annotation, to remove an ignore
2023-02-16Update intentional mentions (MSC3952) to depend on `exact_event_match` ↵Patrick Cloke10-59/+26
(MSC3758). (#15037) This replaces the specific `is_room_mention` push rule condition used in MSC3952 with the generic `exact_event_match` push rule condition from MSC3758. No functionality changes due to this.
2023-02-15Fix a mistake in registration_shared_secret_path docs (#15078)saddfox2-1/+2
* fix a typo in registration_shared_secret_path docs Signed-off-by: Filip Rutar <filip.rutar@gmail.com> * changelog
2023-02-15Fix federated joins when the first server in the list is not in the room ↵Sean Quah2-6/+6
(#15074) Previously we would give up upon receiving a 404 from the first server, instead of trying the rest of the servers in the list. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-15Make it easier to use DataGrip w/ Synapse's schema (#14982)David Robertson8-3/+50
Also tweak the schema dump script: - add a note explaining myself how to use it -Explicitly call `poetry run`, because not everyone uses direnv :(
2023-02-15Support for selecting the Redis logical database. (#15034)999lakhisidhu5-0/+8
Note that this is only used for key-value store (cached values) and not for the pub/sub replication used by Synapse.
2023-02-15Update the error code for duplicate annotation (#15075)Richard van der Hoff3-1/+11
2023-02-14Fix order of partial state tables when purging (#15068)David Robertson3-2/+10
* Fix order of partial state tables when purging `partial_state_rooms` has an FK on `events` pointing to the join event we get from `/send_join`, so we must delete from that table before deleting from `events`. **NB:** It would be nice to cancel any resync processes for the room being purged. We do not do this at present. To do so reliably we'd need an internal HTTP "replication" endpoint, because the worker doing the resync process may be different to that handling the purge request. The first time the resync process tries to write data after the deletion it will fail because we have deleted necessary data e.g. auth events. AFAICS it will not retry the resync, so the only downside to not cancelling the resync is a scary-looking traceback. (This is presumably extremely race-sensitive.) * Changelog * admist(?) -> between * Warn about a race * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-02-14Add final type hint to tests.unittest. (#15072)Patrick Cloke55-320/+433
Adds a return type to HomeServerTestCase.make_homeserver and deal with any variables which are no longer Any.
2023-02-14Implement MSC3966: Add a push rule condition to search for a value in an ↵Patrick Cloke9-42/+176
array. (#15045) The `exact_event_property_contains` condition can be used to search for a value inside of an array.
2023-02-14Remove spurious `dont_notify` action from `.m.rule.reaction` (#15073)Richard van der Hoff2-1/+2
This does nothing and I want to remove it from the MSC.
2023-02-14Prevent clients from reporting nonexistent events. (#13779)reivilibre3-1/+23
2023-02-14Faster joins: Omit device list updates from partial state rooms in /sync ↵Sean Quah2-0/+6
(#15069) ...when lazy loading of members is not enabled. It's weird to notify a client that another user's device list has changed when the client doesn't think that they share a room. Note that when a room is un-partial stated, device list updates are emitted for every member in that room over /sync. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-141.77.0 v1.77.0Mathieu Velten3-1/+13
2023-02-14Fix clashing DB txn name (#15070)Erik Johnston2-1/+2
* Fix clashing DB txn name * Newsfile
2023-02-14Skip calculating unread push actions in `/sync` when `enable_push` is false. ↵Erik Johnston3-0/+16
(#14980)
2023-02-14Change collection[str] to StrCollection in event_auth code (#14929)Harishankar Kumar5-24/+19
Signed-off-by: Harishankar Kumar <hari01584@gmail.com>
2023-02-13Add check to ensure locked dependencies have source distributions available. ↵reivilibre5-1/+87
(#14742)
2023-02-13Tweak comment on `_is_local_room_accessible` as part of room visibility in ↵reivilibre2-2/+3
`/hierarchy` to clarify the condition for a room being visible. (#14834)
2023-02-13Update pyo3-log to v0.8.1 (#15043)V024603-3/+4
Signed-off-by: Kai A. Hiller <V02460@gmail.com>
2023-02-13Refactor arguments of `try_unbind_threepid(_with_id_server)` from dict to ↵Andrew Morgan5-39/+28
separate args (#15053)
2023-02-13Apply logging from hotfixes branch to develop (#15054)David Robertson2-0/+4
* Apply logging from hotfixes branch to develop Part of #4826. Originally added in #11882. * Changelog
2023-02-13Bump sentry-sdk from 1.13.0 to 1.15.0 (#15065)dependabot[bot]2-4/+6
* Bump sentry-sdk from 1.13.0 to 1.15.0 Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.13.0 to 1.15.0. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.13.0...1.15.0) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump types-pillow from 9.4.0.5 to 9.4.0.10 (#15064)dependabot[bot]2-3/+4
* Bump types-pillow from 9.4.0.5 to 9.4.0.10 Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.5 to 9.4.0.10. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump types-requests from 2.28.11.8 to 2.28.11.12 (#15063)dependabot[bot]2-3/+4
* Bump types-requests from 2.28.11.8 to 2.28.11.12 Bumps [types-requests](https://github.com/python/typeshed) from 2.28.11.8 to 2.28.11.12. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-requests dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump serde_json from 1.0.92 to 1.0.93 (#15062)dependabot[bot]2-2/+3
* Bump serde_json from 1.0.92 to 1.0.93 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.92 to 1.0.93. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump systemd-python from 234 to 235 (#15061)dependabot[bot]2-2/+3
* Bump systemd-python from 234 to 235 Bumps [systemd-python](https://github.com/systemd/python-systemd) from 234 to 235. - [Release notes](https://github.com/systemd/python-systemd/releases) - [Changelog](https://github.com/systemd/python-systemd/blob/main/NEWS) - [Commits](https://github.com/systemd/python-systemd/compare/v234...v235) --- updated-dependencies: - dependency-name: systemd-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump dtolnay/rust-toolchain from 9cd00a88a73addc8617065438eff914dd08d0955 to ↵dependabot[bot]4-15/+16
25dc93b901a87e864900a8aec6c12e9aa794c0c3 (#15060) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from 9cd00a88a73addc8617065438eff914dd08d0955 to 25dc93b901a87e864900a8aec6c12e9aa794c0c3. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/9cd00a88a73addc8617065438eff914dd08d0955...25dc93b901a87e864900a8aec6c12e9aa794c0c3) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-13Bump bleach from 5.0.1 to 6.0.0 (#15059)dependabot[bot]2-4/+6
* Bump bleach from 5.0.1 to 6.0.0 Bumps [bleach](https://github.com/mozilla/bleach) from 5.0.1 to 6.0.0. - [Release notes](https://github.com/mozilla/bleach/releases) - [Changelog](https://github.com/mozilla/bleach/blob/main/CHANGES) - [Commits](https://github.com/mozilla/bleach/compare/v5.0.1...v6.0.0) --- updated-dependencies: - dependency-name: bleach dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-10Faster joins: don't stall when a user joins during a fast join (#14606)Mathieu Velten12-94/+196
Fixes #12801. Complement tests are at https://github.com/matrix-org/complement/pull/567. Avoid blocking on full state when handling a subsequent join into a partial state room. Also always perform a remote join into partial state rooms, since we do not know whether the joining user has been banned and want to avoid leaking history to banned users. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
2023-02-10Return read-only collections from `@cached` methods (#13755)Sean Quah27-77/+98
It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
2023-02-10Support for MSC3758: exact_event_match push condition (#14964)Patrick Cloke9-41/+356
This specifies to search for an exact value match, instead of string globbing. It only works across non-compound JSON values (null, boolean, integer, and strings).
2023-02-10Avoid fetching unused account data in sync. (#14973)Patrick Cloke8-90/+176
The per-room account data is no longer unconditionally fetched, even if all rooms will be filtered out. Global account data will not be fetched if it will all be filtered out.
2023-02-10Avoid mutating cached values in `_generate_sync_entry_for_account_data` (#15047)Sean Quah2-0/+3
2023-02-10Refactor get_user_devices_from_cache to avoid mutating cached values. (#15040)Patrick Cloke3-18/+25
The previous version of the code could mutate a cached value, but only if the input requested all devices of a user *and* a specific device. To avoid this nonsensical situation we no longer fetch a specific device ID if all of a user's devices are returned.
2023-02-101.77.0rc2 v1.77.0rc2David Robertson5-3/+22
2023-02-10Fix exception on start up about device lists (#15041)Erik Johnston2-0/+2
Fixes #15010.
2023-02-09Tag federation request spans with the worker name (#15042)David Robertson3-8/+10
* Systematically include worker name as process info * Changelog * don't bother with inner setdefault
2023-02-09Add a class UnpersistedEventContext to allow for the batching up of storing ↵Shay14-162/+359
state groups (#14675) * add class UnpersistedEventContext * modify create new client event to create unpersistedeventcontexts * persist event contexts after creation * fix tests to persist unpersisted event contexts * cleanup * misc lints + cleanup * changelog + fix comments * lints * fix batch insertion? * reduce redundant calculation * add unpersisted event classes * rework compute_event_context, split into function that returns unpersisted event context and then persists it * use calculate_context_info to create unpersisted event contexts * update typing * $%#^&* * black * fix comments and consolidate classes, use attr.s for class * requested changes * lint * requested changes * requested changes * refactor to be stupidly explicit * clearer renaming and flow * make partial state non-optional * update docstrings --------- Co-authored-by: Erik Johnston <erik@matrix.org>
2023-02-09Do not always start a db txn on Postgres (#14840)Andrew Morgan2-4/+10
2023-02-09Bump ruff version from 0.0.230 to 0.0.237. (#15033)Andrew Morgan3-19/+20
2023-02-09Respond correctly to unknown methods on known endpoints (#14605)Patrick Cloke8-51/+89
Respond with a 405 error if a request is received on a known endpoint, but to an unknown method, per MSC3743.
2023-02-09Avoid mutating cached room aliases. (#15038)Patrick Cloke2-1/+3
This might cause incorrect data in other callers which are not expecting the canonical alias to be added into the response.
2023-02-09Bump schema version (#15036)David Robertson2-2/+8
* Bump schema version This should have been included in f10caa73eee0caa91cf373966104d1ededae2aee (and #14979). * Changelog
2023-02-09Add final type hint to synapse.server. (#15035)Patrick Cloke7-12/+11
2023-02-09Proper types for `tests.module_api` (#15031)David Robertson4-54/+80
* -> None for test methods * A first batch of type fixes * Introduce common parent test case * Fixup that big test method * tests.module_api passes mypy * Changelog
2023-02-08Add more missing type hints to tests. (#15028)Patrick Cloke14-111/+124
2023-02-08Add missing type hints to tests. (#15027)Patrick Cloke8-76/+70
2023-02-08Limit concurrent event creation for a room to avoid state resolution when ↵Shay2-3/+4
sending bursts of events to a local room (#14977)
2023-02-08Explicit disabling of disallowed_untyped_defs. (#15026)Patrick Cloke2-58/+26
To make it easier to see which files still need to be fixed.
2023-02-08MSC3873: Escape keys when flattening dicts. (#15004)Patrick Cloke4-4/+40
This disambiguates keys which attempt to match fields with a dot in them (e.g. m.relates_to). Disabled by default behind an experimental configuration flag.
2023-02-08Fix bug in replication where response is cached (#15024)Erik Johnston2-0/+3
2023-02-08Document how to run Synapse (#15022)Andy Balaam2-0/+14
* Document how to run Synapse * Changelog for 15022 * Update docs/development/contributing_guide.md
2023-02-08Bump cryptography from 38.0.4 to 39.0.1 (#15020)dependabot[bot]2-30/+28
* Bump cryptography from 38.0.4 to 39.0.1 Bumps [cryptography](https://github.com/pyca/cryptography) from 38.0.4 to 39.0.1. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/38.0.4...39.0.1) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-08Clarify limitations of SRV delegation in documentation (#14959)William Kray2-0/+10
This PR just clarifies in the SRV DNS delegation document that there are still cases a user may have to serve files from `.well-known` endpoints, and this may not be a valid case for using SRV delegation. This has caused some confusion in a few cases. Signed-off-by: William Kray <github@williamkray.com>
2023-02-08Permalink to the docs v1.77.0rc1David Robertson1-1/+1
2023-02-08Note the revert (that I didn't PR)David Robertson1-1/+1
2023-02-08Merge the osx wheel fix with poetry-core 1.5.0David Robertson1-2/+1
2023-02-08Rearrange itemsDavid Robertson1-23/+24
2023-02-08Manually add new news fragmentsDavid Robertson3-2/+2
2023-02-07Hacky fix to make mac wheels (#15019)David Robertson3-2/+12
* Skip testing PyPy wheels One of the test builds on #15015 failed to install a pp38-* wheel because it didn't have access to the openssl headers to build `cryptography` from source. We don't run CI against PyPy so I'm going to be a meanie and skip testing the wheels. (And I've no idea why 3.8 was special in the first place, either.) * Hack the name of the wheel so cibw can test it I hate hate hate hate hate hate hate hate hate this * Changelog * Apply suggestions from code review Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> --------- Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2023-02-07Include no actions instead of dont_notify for suppressing edits. (#15016)Patrick Cloke2-1/+2
2023-02-07Disambiguate `get_ex_outlier_stream_rows` queryDavid Robertson1-4/+4
A backwards-compatible piece of #14979 that's safe to land now.
2023-02-07Revert "Add `event_stream_ordering` column to membership state tables (#14979)"David Robertson4-145/+11
This reverts commit 5fdc12f482c68e2cdbb78d7db5de2cfe621720d4.
2023-02-07Typecheck tests.rest.media.v1.test_media_storage (#15008)David Robertson4-23/+35
* Fix MediaStorage type hint * Typecheck tests.rest.media.v1.test_media_storage * Changelog * Remove assert and make the comment succinct * Fix syntax for olddeps
2023-02-071.77.0rc1David Robertson63-61/+67
2023-02-07Proper types for tests.test_terms_auth (#15007)David Robertson3-7/+17
* Proper types for tests.test_terms_auth * Changelog
2023-02-07Add tests for using _flatten_dict with an event. (#15002)Patrick Cloke3-9/+68
2023-02-07Properly typecheck types.http (#14988)David Robertson17-191/+298
* Tweak http types in Synapse AFACIS these are correct, and they make mypy happier on tests.http. * Type hints for test_proxyagent * type hints for test_srv_resolver * test_matrix_federation_agent * tests.http.server._base * tests.http.__init__ * tests.http.test_additional_resource * tests.http.test_client * tests.http.test_endpoint * tests.http.test_matrixfederationclient * tests.http.test_servlet * tests.http.test_simple_client * tests.http.test_site * One fixup in tests.server * Untyped defs * Changelog * Fixup syntax for Python 3.7 * Fix olddeps syntax * Use a twisted IPv4 addr for dummy_address * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Remove redundant `Optional` --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-02-07Add `event_stream_ordering` column to membership state tables (#14979)Nick Mills-Barrett5-11/+146
This adds an `event_stream_ordering` column to `current_state_events`, `local_current_membership` and `room_memberships`. Each of these tables is regularly joined with the `events` table to get the stream ordering and denormalising this into each table will yield significant query performance improvements once used. Includes a background job to populate these values from the `events` table. Same idea as https://github.com/matrix-org/synapse/pull/13703. Signed off by Nick @ Beeper (@fizzadar).
2023-02-06Allow poetry-core 1.5.0 (#14949)icp2-2/+5
2023-02-06Add missing types to test_state. (#14985)Patrick Cloke3-62/+141
2023-02-06Type hints for tests.federation (#14991)David Robertson10-94/+127
* Make tests.federation pass mypy * Untyped defs in tests.federation.transport * test methods return None * Remaining type hints in tests.federation * Changelog * Avoid an uncessary type-ignore
2023-02-06Add missing type hints to tests.replication. (#14987)Patrick Cloke21-149/+193
2023-02-06Expect type stubs from canonicaljson (#14992)David Robertson2-3/+1
* canonicaljson has stubs now since https://github.com/matrix-org/python-canonicaljson/pull/52 which is included in the lockfile version we use for type checking. * Changelog
2023-02-06Type hints for tests.appservice (#14990)David Robertson9-59/+132
* Accept a Sequence of events in synapse.appservice This avoids some casts/ignores in the tests I'm about to fixup. It seems that `List[Mock]` is not a subtype of `List[EventBase]`, but `Sequence[Mock]` is a subtype of `Sequence[EventBase]`. So presumably `Mock` is considered a subtype of anything, much like `Any`. * make tests.appservice.test_scheduler pass mypy * Extra hints in tests.appservice.test_scheduler * Extra hints in tests.appservice.test_api * Extra hints in tests.appservice.test_appservice * Disallow untyped defs * Changelog
2023-02-06Bump anyhow from 1.0.68 to 1.0.69 (#14996)dependabot[bot]2-2/+3
* Bump anyhow from 1.0.68 to 1.0.69 Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.68 to 1.0.69. - [Release notes](https://github.com/dtolnay/anyhow/releases) - [Commits](https://github.com/dtolnay/anyhow/compare/1.0.68...1.0.69) --- updated-dependencies: - dependency-name: anyhow dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump phonenumbers from 8.13.4 to 8.13.5 (#14999)dependabot[bot]2-3/+4
* Bump phonenumbers from 8.13.4 to 8.13.5 Bumps [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) from 8.13.4 to 8.13.5. - [Release notes](https://github.com/daviddrysdale/python-phonenumbers/releases) - [Commits](https://github.com/daviddrysdale/python-phonenumbers/compare/v8.13.4...v8.13.5) --- updated-dependencies: - dependency-name: phonenumbers dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump isort from 5.11.4 to 5.11.5 (#14998)dependabot[bot]2-4/+5
* Bump isort from 5.11.4 to 5.11.5 Bumps [isort](https://github.com/pycqa/isort) from 5.11.4 to 5.11.5. - [Release notes](https://github.com/pycqa/isort/releases) - [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md) - [Commits](https://github.com/pycqa/isort/compare/5.11.4...5.11.5) --- updated-dependencies: - dependency-name: isort dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump serde_json from 1.0.91 to 1.0.92 (#14997)dependabot[bot]2-2/+3
* Bump serde_json from 1.0.91 to 1.0.92 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.91 to 1.0.92. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump prometheus-client from 0.15.0 to 0.16.0 (#14995)dependabot[bot]2-3/+4
* Bump prometheus-client from 0.15.0 to 0.16.0 Bumps [prometheus-client](https://github.com/prometheus/client_python) from 0.15.0 to 0.16.0. - [Release notes](https://github.com/prometheus/client_python/releases) - [Commits](https://github.com/prometheus/client_python/compare/v0.15.0...v0.16.0) --- updated-dependencies: - dependency-name: prometheus-client dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump types-setuptools from 65.6.0.3 to 67.1.0.0 (#14994)dependabot[bot]2-3/+4
* Bump types-setuptools from 65.6.0.3 to 67.1.0.0 Bumps [types-setuptools](https://github.com/python/typeshed) from 65.6.0.3 to 67.1.0.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Bump hiredis from 2.1.1 to 2.2.1 (#14993)dependabot[bot]2-90/+91
* Bump hiredis from 2.1.1 to 2.2.1 Bumps [hiredis](https://github.com/redis/hiredis-py) from 2.1.1 to 2.2.1. - [Release notes](https://github.com/redis/hiredis-py/releases) - [Changelog](https://github.com/redis/hiredis-py/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/hiredis-py/compare/v2.1.1...v2.2.1) --- updated-dependencies: - dependency-name: hiredis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-06Only notify the target of a membership event (#14971)David Robertson2-8/+31
* Only notify the target of a membership event Naughty, but should be a big speedup in large rooms
2023-02-03Properly typecheck tests.api (#14983)David Robertson7-110/+140
2023-02-03Implement MSC3958: suppress notifications from edits (#14960)Patrick Cloke9-2/+76
Co-authored-by: Brad Murray <brad@beeper.com> Co-authored-by: Nick Barrett <nick@beeper.com> Copy the suppress_edits push rule from Beeper to implement MSC3958. https://github.com/beeper/synapse/blame/9415a1284b1bfb558bd66f28c24ca1611e6c6fa2/rust/src/push/base_rules.rs#L98-L114
2023-02-03Properly typecheck tests.app (#14984David Robertson5-15/+29
2023-02-03Add a docstring & tests for _flatten_dict. (#14981)Patrick Cloke3-1/+49
2023-02-03Support the backwards compatibility features in MSC3952. (#14958)Patrick Cloke7-59/+184
If the feature is enabled and the event has a `m.mentions` property, skip processing of the legacy mentions rules.
2023-02-03Faster joins: Refactor handling of servers in room (#14954)Sean Quah9-37/+77
Ensure that the list of servers in a partial state room always contains the server we joined off. Also refactor `get_partial_state_servers_at_join` to return `None` when the given room is no longer partial stated, to explicitly indicate when the room has partial state. Otherwise it's not clear whether an empty list means that the room has full state, or the room is partial stated, but the server we joined off told us that there are no servers in the room. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-02-03Reload the pyo3-log config when the Python logging config changes. (#14976)Patrick Cloke5-20/+44
Since pyo3-log is initialized very early in the Python start-up it caches the state of the loggers before they're fully initialized (and thus are essentially disabled). Whenever we reload the logging configuration we now also tell pyo3-log to discard any cached logging configuration it has; it will refetch the current logging configuration from Python at the next point it logs. This fixes Rust log lines not appearing in the homeserver logs.
2023-02-02Skip unused calculations in sync handler. (#14908)Patrick Cloke3-125/+137
If a sync request does not need to calculate per-room entries & is not generating presence & is not generating device list data (e.g. during initial sync) avoid the expensive calculation of room specific data. This is a micro-optimisation for clients syncing simply to receive to-device information.
2023-02-02Do not calculate presence or ephemeral events when they are filtered out ↵Patrick Cloke2-10/+10
(#14970) This expands the previous optimisation from being only for initial sync to being for all sync requests. It also inverts some of the logic to be inclusive instead of exclusive.
2023-02-02Fetch fewer events when getting hosts in room (#14962)David Robertson2-2/+45
2023-02-02Bump dtolnay/rust-toolchain from e645b0cf01249a964ec099494d38d2da0f0b349f to ↵dependabot[bot]4-15/+16
9cd00a88a73addc8617065438eff914dd08d0955 (#14968)
2023-02-01Make extension-module optional, but default. (#14965)Patrick Cloke2-1/+6
2023-02-01Allow enabling the asyncio reactor in complement (#14858)realtyem6-4/+42
Signed-off-by: Jason Little realtyem@gmail.com
2023-02-01Bump hiredis from 2.0.0 to 2.1.1 (#14939)dependabot[bot]2-43/+92
* Bump hiredis from 2.0.0 to 2.1.1 Bumps [hiredis](https://github.com/redis/hiredis-py) from 2.0.0 to 2.1.1. - [Release notes](https://github.com/redis/hiredis-py/releases) - [Changelog](https://github.com/redis/hiredis-py/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/hiredis-py/compare/v2.0.0...v2.1.1) --- updated-dependencies: - dependency-name: hiredis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-01Add helper to parse an enum from query args & use it. (#14956)Patrick Cloke25-96/+176
The `parse_enum` helper pulls an enum value from the query string (by delegating down to the parse_string helper with values generated from the enum). This is used to pull out "f" and "b" in most places and then we thread the resulting Direction enum throughout more code.
2023-02-01Attempt to delete more duplicate rows in receipts_linearized table. (#14915)Patrick Cloke3-9/+30
The previous assumption was that the stream_id column was unique (for a room ID, receipt type, user ID tuple), but this turned out to be incorrect. Now find the max stream ID, then map this back to a database-specific row identifier and delete other rows which match the (room ID, receipt type, user ID) tuple, but *not* the row ID.
2023-02-01Bump docker/build-push-action from 3 to 4 (#14952)dependabot[bot]2-1/+2
* Bump docker/build-push-action from 3 to 4 Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v3...v4) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-02-01Add more user information to export-data command. (#14894)Dirk Klimpel6-20/+206
* The user's profile information. * The user's devices. * The user's connections / IP address information.
2023-02-01lnav config for synpase logs (#14953)David Robertson3-0/+115
2023-02-01Fix inconsistencies between MSC3952 and implementation. (#14957)Patrick Cloke2-3/+4
* Correct the push rule IDs. * Removes the sound tweak for room notifications.
2023-01-31update changelog and upgrade notes v1.76.0H. Shay2-2/+2
2023-01-31update changelogH. Shay1-9/+10
2023-01-311.76.0H. Shay4-2/+16
2023-01-31Fix running cargo bench & test in CI. (#14943)Patrick Cloke3-0/+36
2023-01-31Tag /send_join responses to detect faster joins (#14950)David Robertson3-0/+12
* Tag /send_join responses to detect faster joins * Changelog * Define a proper SynapseTag * isort
2023-01-31Fix "Re-starting finished log context" spam when creating events (#14947)Sean Quah2-1/+4
`run_in_background` calls re-use the current logging context. When they are not awaited, they can complete after the current logging context has been marked as finished, which leads to log spam. Use `run_as_background_process` instead. Fixes one of the instances of #13090. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-31Make sqlite database migrations transactional again, part two (#14926)Sean Quah5-5/+109
#14910 fixed the regression introduced by #13873 where sqlite database migrations would no longer run inside a transaction. However, it committed the transaction before Synapse updated its bookkeeping of which migrations have been run, which means that migrations may be run again after they have completed successfully. Leave the transaction open at the end of `executescript`, to restore the old, correct behaviour. Also make the PostgreSQL behaviour consistent with SQLite. Fixes #14909. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-31Reject boolean power levels (#14944)David Robertson4-6/+7
* Better test for bad values in power levels events The previous test only checked that Synapse didn't raise an exception, but didn't check that we had correctly interpreted the value of the dodgy power level. It also conflated two things: bad room notification levels, and bad user levels. There _is_ logic for converting the latter to integers, but we should test it separately. * Check we ignore types that don't convert to int * Handle `None` values in `notifications.room` * Changelog * Also test that bad values are rejected by event auth * Docstring * linter scripttttttttt * Test boolean values in PL content * Reject boolean power levels * Changelog
2023-01-31Prefer `type(x) is int` to `isinstance(x, int)` (#14945)David Robertson14-47/+75
* Perfer `type(x) is int` to `isinstance(x, int)` This covered all additional instances I could see where `x` was user-controlled. The remaining cases are ``` $ rg -s 'isinstance.*[^_]int' tests/replication/_base.py 576: if isinstance(obj, int): synapse/util/caches/stream_change_cache.py 136: assert isinstance(stream_pos, int) 214: assert isinstance(stream_pos, int) 246: assert isinstance(stream_pos, int) 267: assert isinstance(stream_pos, int) synapse/replication/tcp/external_cache.py 133: if isinstance(result, int): synapse/metrics/__init__.py 100: if isinstance(calls, (int, float)): synapse/handlers/appservice.py 262: assert isinstance(new_token, int) synapse/config/_util.py 62: if isinstance(p, int): ``` which cover metrics, logic related to `jsonschema`, and replication and data streams. AFAICS these are all internal to Synapse * Changelog
2023-01-30Handle malformed values of `notification.room` in power level events (#14942)David Robertson4-17/+128
* Better test for bad values in power levels events The previous test only checked that Synapse didn't raise an exception, but didn't check that we had correctly interpreted the value of the dodgy power level. It also conflated two things: bad room notification levels, and bad user levels. There _is_ logic for converting the latter to integers, but we should test it separately. * Check we ignore types that don't convert to int * Handle `None` values in `notifications.room` * Changelog * Also test that bad values are rejected by event auth * Docstring * linter scripttttttttt
2023-01-30Initial batch of notes on faster joins (#14677)David Robertson3-0/+377
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Shay <hillerys@element.io>
2023-01-30Bump types-pillow from 9.4.0.3 to 9.4.0.5 (#14938)dependabot[bot]2-3/+4
* Bump types-pillow from 9.4.0.3 to 9.4.0.5 Bumps [types-pillow](https://github.com/python/typeshed) from 9.4.0.3 to 9.4.0.5. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-01-30Bump types-jsonschema from 4.17.0.2 to 4.17.0.3 (#14937)dependabot[bot]2-3/+4
* Bump types-jsonschema from 4.17.0.2 to 4.17.0.3 Bumps [types-jsonschema](https://github.com/python/typeshed) from 4.17.0.2 to 4.17.0.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-jsonschema dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-01-30Bump types-pyyaml from 6.0.12.2 to 6.0.12.3 (#14936)dependabot[bot]2-3/+4
* Bump types-pyyaml from 6.0.12.2 to 6.0.12.3 Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.12.2 to 6.0.12.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-01-30Bump ijson from 3.1.4 to 3.2.0.post0 (#14935)dependabot[bot]2-63/+80
* Bump ijson from 3.1.4 to 3.2.0.post0 Bumps [ijson](https://github.com/ICRAR/ijson) from 3.1.4 to 3.2.0.post0. - [Release notes](https://github.com/ICRAR/ijson/releases) - [Changelog](https://github.com/ICRAR/ijson/blob/master/CHANGELOG.md) - [Commits](https://github.com/ICRAR/ijson/compare/v3.1.4...v3.2.0.post0) --- updated-dependencies: - dependency-name: ijson dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2023-01-27Implement MSC3952: Intentional mentions (#14823)Patrick Cloke11-11/+263
MSC3952 defines push rules which searches for mentions in a list of Matrix IDs in the event body, instead of searching the entire event body for display name / local part. This is implemented behind an experimental configuration flag and does not yet implement the backwards compatibility pieces of the MSC.
2023-01-27Describe faster joinsDavid Robertson1-0/+17
2023-01-27Use an enum for direction. (#14927)Patrick Cloke9-44/+76
For better type safety we use an enum instead of strings to configure direction (backwards or forwards).
2023-01-271.76.0rc2 v1.76.0rc2David Robertson5-5/+25
2023-01-26Add missing type hints in tests (#14879)Patrick Cloke8-42/+75
* FIx-up type hints in tests.logging. * Add missing type hints to test_transactions.
2023-01-26Fix paginating /relations with a live token (#14866)Patrick Cloke3-70/+123
The `/relations` endpoint was not properly handle "live tokens" (i.e sync tokens), to do this properly we abstract the code that `/messages` has and re-use it.
2023-01-26Use StrCollection in place of Collection[str] in (most) handlers code. (#14922)Patrick Cloke13-58/+43
Due to the increased safety of StrCollection over Collection[str] and Sequence[str].
2023-01-26Batch look-ups to see if rooms are partial stated. (#14917)Patrick Cloke4-11/+43
* Batch look-ups to see if rooms are partial stated. * Fix issues found in linting. * Fix typo. * Apply suggestions from code review Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Clarify comments. Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Also improve the cache size while we're at it * is_partial_state_rooms -> is_partial_state_room_batched * Run `black` * Improve annotation for `simple_select_many_batch` * Fix is_partial_state_room_batched impl * Okay, _actually_ fix impl * Update description. * Update synapse/storage/databases/main/room.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Run black. Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: David Robertson <davidr@element.io>
2023-01-26Fix typo in release script (#14920)David Robertson2-1/+2
* Fix typo in release script * Changelog
2023-01-26Fix initialization of `_device_list_id_gen` (#14914)Sean Quah2-0/+2
On startup, the `_device_list_id_gen` stream id generator is initialized using the maximum stream id seen in a list of tables. When we started populating the `device_list_remote_pending` table in #13913, we forgot to add it to the aforementioned list of tables, so the stream id generator can hand out old stream ids after a restart. The end result is that Synapse can fail to handle device list update EDUs after a restart when a partial state join is in progress. Add the `device_list_remote_pending` table to the list of tables to consider when initializing the `_device_list_id_gen` stream id generator. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-26Fix type hints in typing edu unit tests (#14886)Andrew Morgan5-43/+66
2023-01-25Fix a bug in the send_local_online_presence_to module API (#14880)Patrick Cloke5-11/+19
Destination was being used incorrectly (a single destination instead of a list of destinations was being passed). This also updates some of the types in the area to not use Collection[str], which is a footgun.
2023-01-25Add missing type hints for tests.events. (#14904)Patrick Cloke6-64/+91
2023-01-25Document how to handle Dependabot pull requests. (#14916)Patrick Cloke2-0/+15
2023-01-25Fix type hints in knocking tests. (#14887)Andrew Morgan4-7/+5
2023-01-25Fix link syntax in changelogDavid Robertson1-1/+1
2023-01-25Refer to upgrade notesDavid Robertson1-0/+6
2023-01-25Touch-up the features sectionDavid Robertson1-3/+3
2023-01-25Group dependabot linesDavid Robertson1-4/+4
2023-01-251.76.0rc1 v1.76.0rc1David Robertson63-63/+91
2023-01-25Bump the client-side timeout for /state (#14912)David Robertson2-0/+5
* Bump the client-side timeout for /state to allow faster joins resyncs the chance to complete for large rooms. We have seen this fair poorly (~90s for Matrix HQ's /state) in testing, causing the resync to advance to another HS who hasn't seen our join yet. * Changelog * Milliseconds!!!!
2023-01-25Make sqlite database migrations transactional again (#14910)Sean Quah3-2/+7
#13873 introduced a regression which causes sqlite database migrations to no longer run inside a transaction. Wrap them in a transaction again, to avoid database corruption when migrations are interrupted. Fixes #14909. Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-25Document the export user data command. (#14883)ZAID BIN TARIQ2-0/+9
2023-01-24Request partial joins by default (#14905)David Robertson4-7/+49
* Request partial joins by default This is a little sloppy, but we are trying to gain confidence in faster joins in the upcoming RC. Admins can still opt out by adding the following to their Synapse config: ```yaml experimental: faster_joins: false ``` We may revert this change before the release proper, depending on how testing in the wild goes. * Changelog * Try to fix the backfill test failures * Upgrade notes * Postgres compat?
2023-01-23Faster joins: omit partial rooms from eager syncs until the resync completes ↵David Robertson13-44/+170
(#14870) * Allow `AbstractSet` in `StrCollection` Or else frozensets are excluded. This will be useful in an upcoming commit where I plan to change a function that accepts `List[str]` to accept `StrCollection` instead. * `rooms_to_exclude` -> `rooms_to_exclude_globally` I am about to make use of this exclusion mechanism to exclude rooms for a specific user and a specific sync. This rename helps to clarify the distinction between the global config and the rooms to exclude for a specific sync. * Better function names for internal sync methods * Track a list of excluded rooms on SyncResultBuilder I plan to feed a list of partially stated rooms for this sync to ignore * Exclude partial state rooms during eager sync using the mechanism established in the previous commit * Track un-partial-state stream in sync tokens So that we can work out which rooms have become fully-stated during a given sync period. * Fix mutation of `@cached` return value This was fouling up a complement test added alongside this PR. Excluding a room would mean the set of forgotten rooms in the cache would be extended. This means that room could be erroneously considered forgotten in the future. Introduced in #12310, Synapse 1.57.0. I don't think this had any user-visible side effects (until now). * SyncResultBuilder: track rooms to force as newly joined Similar plan as before. We've omitted rooms from certain sync responses; now we establish the mechanism to reintroduce them into future syncs. * Read new field, to present rooms as newly joined * Force un-partial-stated rooms to be newly-joined for eager incremental syncs only, provided they're still fully stated * Notify user stream listeners to wake up long polling syncs * Changelog * Typo fix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Unnecessary list cast Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Rephrase comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Another comment Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Fixup merge(?) * Poke notifier when receiving un-partial-stated msg over replication * Fixup merge whoops Thanks MV :) Co-authored-by: Mathieu Velen <mathieuv@matrix.org> Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-23Bump ruff from 0.0.224 to 0.0.230 (#14897)dependabot[bot]3-19/+20
2023-01-23Bump types-opentracing from 2.4.10 to 2.4.10.1 (#14896)dependabot[bot]2-3/+4
2023-01-23Bump types-commonmark from 0.9.2 to 0.9.2.1 (#14901)dependabot[bot]2-3/+4
2023-01-23Bump types-psycopg2 from 2.9.21.2 to 2.9.21.4 (#14900)dependabot[bot]2-3/+4
2023-01-23Bump types-requests from 2.28.11.7 to 2.28.11.8 (#14899)dependabot[bot]2-3/+4
2023-01-23Skip processing stats for broken rooms. (#14873)Patrick Cloke4-36/+72
* Skip processing stats for broken rooms. * Newsfragment * Use a custom exception.
2023-01-23Faster joins: Update room stats and the user directory on workers when ↵Sean Quah4-5/+11
finishing join (#14874) * Faster joins: Update room stats and user directory on workers when done When finishing a partial state join to a room, we update the current state of the room without persisting additional events. Workers receive notice of the current state update over replication, but neglect to wake the room stats and user directory updaters, which then get incidentally triggered the next time an event is persisted or an unrelated event persister sends out a stream position update. We wake the room stats and user directory updaters at the appropriate time in this commit. Part of #12814 and #12815. Signed-off-by: Sean Quah <seanq@matrix.org> * fixup comment Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-22Enable Faster Remote Room Joins against worker-mode Synapse. (#14752)reivilibre10-36/+35
* Enable Complement tests for Faster Remote Room Joins on worker-mode * (dangerous) Add an override to allow Complement to use FRRJ under workers * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Fix race where we didn't send out replication notification * MORE HACKS * Fix get_un_partial_stated_rooms_token to take instance_name * Fix bad merge * Remove warning * Correctly advance un_partial_stated_room_stream * Fix merge * Add another notify_replication * Fixups * Create a separate ReplicationNotifier * Fix test * Fix portdb * Create a separate ReplicationNotifier * Fix test * Fix portdb * Fix presence test * Newsfile * Apply suggestions from code review * Update changelog.d/14752.misc Co-authored-by: Erik Johnston <erik@matrix.org> * lint Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org>
2023-01-22Faster joins: Fix incompatibility with restricted joins (#14882)Sean Quah3-81/+140
* Avoid clearing out forward extremities when doing a second remote join When joining a restricted room where the local homeserver does not have a user able to issue invites, we perform a second remote join. We want to avoid clearing out forward extremities in this case because the forward extremities we have are up to date and clearing out forward extremities creates a window in which the room can get bricked if Synapse crashes. Signed-off-by: Sean Quah <seanq@matrix.org> * Do a full join when doing a second remote join into a full state room We cannot persist a partial state join event into a joined full state room, so we perform a full state join for such rooms instead. As a future optimization, we could always perform a partial state join and compute or retrieve the full state ourselves if necessary. Signed-off-by: Sean Quah <seanq@matrix.org> * Add lock around partial state flag for rooms Signed-off-by: Sean Quah <seanq@matrix.org> * Preserve partial state info when doing a second partial state join Signed-off-by: Sean Quah <seanq@matrix.org> * Add newsfile * Add a TODO(faster_joins) marker Signed-off-by: Sean Quah <seanq@matrix.org>
2023-01-22Fix type hints for Monthly Active Users tests (#14889)Andrew Morgan3-7/+8
2023-01-21Add type hints to `TestRatelimiter` (#14885)Andrew Morgan3-16/+52
2023-01-20Reduce max time we wait for stream positions (#14881)Erik Johnston3-12/+12
Now that we wait for stream positions whenever we do a HTTP replication hit, we need to be less brutal in the case where we do timeout (as we have bugs around this).
2023-01-20Always notify replication when a stream advances (#14877)Erik Johnston19-29/+104
This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-02 10:33:47 +0000