diff options
| author | David Robertson <davidr@element.io> | 2022-04-26 18:07:15 +0100 |
|---|---|---|
| committer | David Robertson <davidr@element.io> | 2022-04-26 18:07:15 +0100 |
| commit | 9986621bc8eea30f7605f74dd9f78050a97918ae (patch) | |
| tree | 72cb48e44913e5305af9bb5371a524a2ba863aa6 | |
| parent | Mark Dockerfile as requiring BuildKit (#12541) (diff) | |
| parent | Adjust changelog (diff) | |
| download | synapse-9986621bc8eea30f7605f74dd9f78050a97918ae.tar.xz | |
Merge tag 'v1.58.0rc2' into develop
Synapse 1.58.0rc2 (2022-04-26) ============================== This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates. Bugfixes -------- - Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](https://github.com/matrix-org/synapse/issues/12554)) - Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](https://github.com/matrix-org/synapse/issues/12555)) Internal Changes ---------------- - Reduce unnecessary work when handling remote device list updates. ([\#12557](https://github.com/matrix-org/synapse/issues/12557))
59 files changed, 144 insertions, 172 deletions
diff --git a/CHANGES.md b/CHANGES.md index a7d2529b55..1fbe0815de 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,3 +1,93 @@ +Synapse 1.58.0rc2 (2022-04-26) +============================== + +This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates. + +Bugfixes +-------- + +- Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](https://github.com/matrix-org/synapse/issues/12554)) +- Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](https://github.com/matrix-org/synapse/issues/12555)) + + +Internal Changes +---------------- + +- Reduce unnecessary work when handling remote device list updates. ([\#12557](https://github.com/matrix-org/synapse/issues/12557)) + + +Synapse 1.58.0rc1 (2022-04-26) +============================== + +As of this release, the groups/communities feature in Synapse is now disabled by default. See [\#11584](https://github.com/matrix-org/synapse/issues/11584) for details. As mentioned in [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1580), this feature will be removed in Synapse 1.61. + +Features +-------- + +- Implement [MSC3383](https://github.com/matrix-org/matrix-spec-proposals/pull/3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly. ([\#11398](https://github.com/matrix-org/synapse/issues/11398)) +- Docker images and Debian packages from matrix.org now contain a locked set of Python dependencies, greatly improving build reproducibility. ([Board](https://github.com/orgs/matrix-org/projects/54), [\#11537](https://github.com/matrix-org/synapse/issues/11537)) +- Enable processing of device list updates asynchronously. ([\#12365](https://github.com/matrix-org/synapse/issues/12365), [\#12465](https://github.com/matrix-org/synapse/issues/12465)) +- Implement [MSC2815](https://github.com/matrix-org/matrix-spec-proposals/pull/2815) to allow room moderators to view redacted event content. Contributed by @tulir @ Beeper. ([\#12427](https://github.com/matrix-org/synapse/issues/12427)) +- Build Debian packages for Ubuntu 22.04 "Jammy Jellyfish". ([\#12543](https://github.com/matrix-org/synapse/issues/12543)) + + +Bugfixes +-------- + +- Prevent a sync request from removing a user's busy presence status. ([\#12213](https://github.com/matrix-org/synapse/issues/12213)) +- Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper. ([\#12319](https://github.com/matrix-org/synapse/issues/12319)) +- Fix a long-standing bug which incorrectly caused `GET /_matrix/client/v3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original. ([\#12476](https://github.com/matrix-org/synapse/issues/12476)) +- Fix a bug introduced in Synapse 1.27.0 where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1, no matter how many extremities were deleted. ([\#12496](https://github.com/matrix-org/synapse/issues/12496)) +- Fix a long-standing bug where the image thumbnails embedded into email notifications were broken. ([\#12510](https://github.com/matrix-org/synapse/issues/12510)) +- Fix a bug in the implementation of [MSC3202](https://github.com/matrix-org/matrix-spec-proposals/pull/3202) where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`. ([\#12520](https://github.com/matrix-org/synapse/issues/12520)) +- Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room. ([\#12522](https://github.com/matrix-org/synapse/issues/12522)) + + +Improved Documentation +---------------------- + +- Fix rendering of the documentation site when using the 'print' feature. ([\#12340](https://github.com/matrix-org/synapse/issues/12340)) +- Add a manual documenting config file options. ([\#12368](https://github.com/matrix-org/synapse/issues/12368), [\#12527](https://github.com/matrix-org/synapse/issues/12527)) +- Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental. ([\#12451](https://github.com/matrix-org/synapse/issues/12451)) +- Update worker documentation and replace old `federation_reader` with `generic_worker`. ([\#12457](https://github.com/matrix-org/synapse/issues/12457)) +- Strongly recommend [Poetry](https://python-poetry.org/) for development. ([\#12475](https://github.com/matrix-org/synapse/issues/12475)) +- Add some example configurations for workers and update architectural diagram. ([\#12492](https://github.com/matrix-org/synapse/issues/12492)) +- Fix a broken link in `README.rst`. ([\#12495](https://github.com/matrix-org/synapse/issues/12495)) +- Add HAProxy delegation example with CORS headers to docs. ([\#12501](https://github.com/matrix-org/synapse/issues/12501)) +- Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari. ([\#12533](https://github.com/matrix-org/synapse/issues/12533)) + + +Deprecations and Removals +------------------------- + +- The groups/communities feature in Synapse is now disabled by default. ([\#12344](https://github.com/matrix-org/synapse/issues/12344)) +- Remove unstable identifiers from [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440). ([\#12382](https://github.com/matrix-org/synapse/issues/12382)) + + +Internal Changes +---------------- + +- Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join. ([\#12394](https://github.com/matrix-org/synapse/issues/12394)) +- Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive. ([\#12399](https://github.com/matrix-org/synapse/issues/12399)) +- Remove an unstable identifier from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). ([\#12395](https://github.com/matrix-org/synapse/issues/12395)) +- Run CI in the locked [Poetry](https://python-poetry.org/) environment, and remove corresponding `tox` jobs. ([\#12425](https://github.com/matrix-org/synapse/issues/12425), [\#12434](https://github.com/matrix-org/synapse/issues/12434), [\#12438](https://github.com/matrix-org/synapse/issues/12438), [\#12441](https://github.com/matrix-org/synapse/issues/12441), [\#12449](https://github.com/matrix-org/synapse/issues/12449), [\#12478](https://github.com/matrix-org/synapse/issues/12478), [\#12514](https://github.com/matrix-org/synapse/issues/12514), [\#12472](https://github.com/matrix-org/synapse/issues/12472)) +- Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current iteration of [MSC2666](https://github.com/matrix-org/matrix-spec-proposals/pull/2666). ([\#12445](https://github.com/matrix-org/synapse/issues/12445)) +- Fix typo in the release script help string. ([\#12450](https://github.com/matrix-org/synapse/issues/12450)) +- Fix a minor typo in the Debian changelogs generated by the release script. ([\#12497](https://github.com/matrix-org/synapse/issues/12497)) +- Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development. ([\#12455](https://github.com/matrix-org/synapse/issues/12455)) +- Limit length of `device_id` to less than 512 characters. ([\#12454](https://github.com/matrix-org/synapse/issues/12454)) +- Dockerfile-workers: reduce the amount we install in the image. ([\#12464](https://github.com/matrix-org/synapse/issues/12464)) +- Dockerfile-workers: give the master its own log config. ([\#12466](https://github.com/matrix-org/synapse/issues/12466)) +- complement-synapse-workers: factor out separate entry point script. ([\#12467](https://github.com/matrix-org/synapse/issues/12467)) +- Back out experimental implementation of [MSC2314](https://github.com/matrix-org/matrix-spec-proposals/pull/2314). ([\#12474](https://github.com/matrix-org/synapse/issues/12474)) +- Fix grammatical error in federation error response when the room version of a room is unknown. ([\#12483](https://github.com/matrix-org/synapse/issues/12483)) +- Remove unnecessary configuration overrides in tests. ([\#12511](https://github.com/matrix-org/synapse/issues/12511)) +- Refactor the relations code for clarity. ([\#12519](https://github.com/matrix-org/synapse/issues/12519)) +- Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`. ([\#12528](https://github.com/matrix-org/synapse/issues/12528)) +- Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s. ([\#12468](https://github.com/matrix-org/synapse/issues/12468)) +- Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`. ([\#12529](https://github.com/matrix-org/synapse/issues/12529)) + + Synapse 1.57.1 (2022-04-20) =========================== diff --git a/changelog.d/11398.feature b/changelog.d/11398.feature deleted file mode 100644 index a910f4da14..0000000000 --- a/changelog.d/11398.feature +++ /dev/null @@ -1 +0,0 @@ -Implement [MSC3383](https://github.com/matrix-org/matrix-spec-proposals/pull/3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly GmbH. diff --git a/changelog.d/12213.bugfix b/changelog.d/12213.bugfix deleted file mode 100644 index 9278e3a9c1..0000000000 --- a/changelog.d/12213.bugfix +++ /dev/null @@ -1 +0,0 @@ -Prevent a sync request from removing a user's busy presence status. diff --git a/changelog.d/12319.bugfix b/changelog.d/12319.bugfix deleted file mode 100644 index a50191feaa..0000000000 --- a/changelog.d/12319.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper. diff --git a/changelog.d/12337.feature b/changelog.d/12337.feature deleted file mode 100644 index 6c4444c707..0000000000 --- a/changelog.d/12337.feature +++ /dev/null @@ -1 +0,0 @@ -Use poetry to manage Synapse's dependencies. \ No newline at end of file diff --git a/changelog.d/12340.doc b/changelog.d/12340.doc deleted file mode 100644 index 8354f2259e..0000000000 --- a/changelog.d/12340.doc +++ /dev/null @@ -1 +0,0 @@ -Fix rendering of the documentation site when using the 'print' feature. diff --git a/changelog.d/12344.removal b/changelog.d/12344.removal deleted file mode 100644 index ecefa76d8e..0000000000 --- a/changelog.d/12344.removal +++ /dev/null @@ -1 +0,0 @@ -The groups/communities feature in Synapse has been disabled by default. diff --git a/changelog.d/12365.feature b/changelog.d/12365.feature deleted file mode 100644 index 642dea966c..0000000000 --- a/changelog.d/12365.feature +++ /dev/null @@ -1 +0,0 @@ -Enable processing of device list updates asynchronously. diff --git a/changelog.d/12368.doc b/changelog.d/12368.doc deleted file mode 100644 index 62e4cb2c7e..0000000000 --- a/changelog.d/12368.doc +++ /dev/null @@ -1 +0,0 @@ -Add a manual documenting config file options. \ No newline at end of file diff --git a/changelog.d/12382.removal b/changelog.d/12382.removal deleted file mode 100644 index eb91186340..0000000000 --- a/changelog.d/12382.removal +++ /dev/null @@ -1 +0,0 @@ -Remove unstable identifiers from [MSC3440](https://github.com/matrix-org/matrix-doc/pull/3440). diff --git a/changelog.d/12394.misc b/changelog.d/12394.misc deleted file mode 100644 index 69109fcc37..0000000000 --- a/changelog.d/12394.misc +++ /dev/null @@ -1 +0,0 @@ -Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join. diff --git a/changelog.d/12395.misc b/changelog.d/12395.misc deleted file mode 100644 index 0a2123b294..0000000000 --- a/changelog.d/12395.misc +++ /dev/null @@ -1 +0,0 @@ -Remove an unstable identifier from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). diff --git a/changelog.d/12399.misc b/changelog.d/12399.misc deleted file mode 100644 index cd2e09626d..0000000000 --- a/changelog.d/12399.misc +++ /dev/null @@ -1 +0,0 @@ -Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive. diff --git a/changelog.d/12425.misc b/changelog.d/12425.misc deleted file mode 100644 index 3b076be0bd..0000000000 --- a/changelog.d/12425.misc +++ /dev/null @@ -1 +0,0 @@ -Run twisted trunk CI job in the locked poetry environment. diff --git a/changelog.d/12427.feature b/changelog.d/12427.feature deleted file mode 100644 index e6913c8c09..0000000000 --- a/changelog.d/12427.feature +++ /dev/null @@ -1 +0,0 @@ -Implement [MSC2815](https://github.com/matrix-org/matrix-spec-proposals/pull/2815) to allow room moderators to view redacted event content. Contributed by @tulir. diff --git a/changelog.d/12434.misc b/changelog.d/12434.misc deleted file mode 100644 index 88dab428d2..0000000000 --- a/changelog.d/12434.misc +++ /dev/null @@ -1 +0,0 @@ -Run lints under poetry in CI, and remove corresponding tox lint jobs. diff --git a/changelog.d/12438.misc b/changelog.d/12438.misc deleted file mode 100644 index f2c07a56da..0000000000 --- a/changelog.d/12438.misc +++ /dev/null @@ -1 +0,0 @@ -Run "main" trial tests under `poetry`. diff --git a/changelog.d/12441.misc b/changelog.d/12441.misc deleted file mode 100644 index c2619f1654..0000000000 --- a/changelog.d/12441.misc +++ /dev/null @@ -1 +0,0 @@ -Bump twisted version in `poetry.lock` to work around [pip bug #9644](https://github.com/pypa/pip/issues/9644). diff --git a/changelog.d/12445.misc b/changelog.d/12445.misc deleted file mode 100644 index 954248115a..0000000000 --- a/changelog.d/12445.misc +++ /dev/null @@ -1 +0,0 @@ -Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current MSC iteration. \ No newline at end of file diff --git a/changelog.d/12449.misc b/changelog.d/12449.misc deleted file mode 100644 index 03e08aace4..0000000000 --- a/changelog.d/12449.misc +++ /dev/null @@ -1 +0,0 @@ -Use `poetry` to manage the virtualenv in debian packages. diff --git a/changelog.d/12450.misc b/changelog.d/12450.misc deleted file mode 100644 index 4b1c8cba87..0000000000 --- a/changelog.d/12450.misc +++ /dev/null @@ -1 +0,0 @@ -Fix typo in the release script help string. diff --git a/changelog.d/12451.doc b/changelog.d/12451.doc deleted file mode 100644 index c8b23c1285..0000000000 --- a/changelog.d/12451.doc +++ /dev/null @@ -1 +0,0 @@ -Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental. diff --git a/changelog.d/12454.misc b/changelog.d/12454.misc deleted file mode 100644 index cb7ff74b4c..0000000000 --- a/changelog.d/12454.misc +++ /dev/null @@ -1 +0,0 @@ -Limit length of device_id to less than 512 characters. diff --git a/changelog.d/12455.misc b/changelog.d/12455.misc deleted file mode 100644 index 9b19945673..0000000000 --- a/changelog.d/12455.misc +++ /dev/null @@ -1 +0,0 @@ -Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development. diff --git a/changelog.d/12457.doc b/changelog.d/12457.doc deleted file mode 100644 index a4871622cf..0000000000 --- a/changelog.d/12457.doc +++ /dev/null @@ -1 +0,0 @@ -Update worker documentation and replace old `federation_reader` with `generic_worker`. \ No newline at end of file diff --git a/changelog.d/12464.misc b/changelog.d/12464.misc deleted file mode 100644 index 7a8cc6ba51..0000000000 --- a/changelog.d/12464.misc +++ /dev/null @@ -1 +0,0 @@ -Dockerfile-workers: reduce the amount we install in the image. diff --git a/changelog.d/12465.feature b/changelog.d/12465.feature deleted file mode 100644 index 642dea966c..0000000000 --- a/changelog.d/12465.feature +++ /dev/null @@ -1 +0,0 @@ -Enable processing of device list updates asynchronously. diff --git a/changelog.d/12466.misc b/changelog.d/12466.misc deleted file mode 100644 index b0c2c950fe..0000000000 --- a/changelog.d/12466.misc +++ /dev/null @@ -1 +0,0 @@ -Dockerfile-workers: give the master its own log config. diff --git a/changelog.d/12467.misc b/changelog.d/12467.misc deleted file mode 100644 index fbf415f707..0000000000 --- a/changelog.d/12467.misc +++ /dev/null @@ -1 +0,0 @@ -complement-synapse-workers: factor out separate entry point script. diff --git a/changelog.d/12468.misc b/changelog.d/12468.misc deleted file mode 100644 index 3d5d25247f..0000000000 --- a/changelog.d/12468.misc +++ /dev/null @@ -1 +0,0 @@ -Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s. diff --git a/changelog.d/12472.misc b/changelog.d/12472.misc deleted file mode 100644 index ed306209cc..0000000000 --- a/changelog.d/12472.misc +++ /dev/null @@ -1 +0,0 @@ -Add a CI job which tests Synapse against the latest version of all dependencies. diff --git a/changelog.d/12474.misc b/changelog.d/12474.misc deleted file mode 100644 index 5292108b39..0000000000 --- a/changelog.d/12474.misc +++ /dev/null @@ -1 +0,0 @@ -Back out experimental implementation of [MSC2314](https://github.com/matrix-org/matrix-spec-proposals/pull/2314). diff --git a/changelog.d/12475.doc b/changelog.d/12475.doc deleted file mode 100644 index f4481d0613..0000000000 --- a/changelog.d/12475.doc +++ /dev/null @@ -1 +0,0 @@ -Strongly recommend `poetry` for development. diff --git a/changelog.d/12476.bugfix b/changelog.d/12476.bugfix deleted file mode 100644 index 9ad6a71abd..0000000000 --- a/changelog.d/12476.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug which incorrectly caused `GET /_matrix/client/r3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original. diff --git a/changelog.d/12478.misc b/changelog.d/12478.misc deleted file mode 100644 index 061a604a1e..0000000000 --- a/changelog.d/12478.misc +++ /dev/null @@ -1 +0,0 @@ -Use poetry-core instead of setuptools to build wheels. diff --git a/changelog.d/12483.misc b/changelog.d/12483.misc deleted file mode 100644 index 88c6e3e465..0000000000 --- a/changelog.d/12483.misc +++ /dev/null @@ -1 +0,0 @@ -Fix grammatical error in federation error response when the room version of a room is unknown. diff --git a/changelog.d/12492.doc b/changelog.d/12492.doc deleted file mode 100644 index 4a3e2f4f06..0000000000 --- a/changelog.d/12492.doc +++ /dev/null @@ -1 +0,0 @@ -Add some example configurations for workers and update architectural diagram. diff --git a/changelog.d/12495.doc b/changelog.d/12495.doc deleted file mode 100644 index afa0111675..0000000000 --- a/changelog.d/12495.doc +++ /dev/null @@ -1 +0,0 @@ -Fix a broken link in `README.rst`. diff --git a/changelog.d/12496.bugfix b/changelog.d/12496.bugfix deleted file mode 100644 index a68df7c96a..0000000000 --- a/changelog.d/12496.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix bug where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1 no matter how many extremities were deleted. Broke in v1.27.0. diff --git a/changelog.d/12497.misc b/changelog.d/12497.misc deleted file mode 100644 index 17a661ec61..0000000000 --- a/changelog.d/12497.misc +++ /dev/null @@ -1 +0,0 @@ -Fix a minor typo in the Debian changelogs generated by the release script. diff --git a/changelog.d/12501.doc b/changelog.d/12501.doc deleted file mode 100644 index 278193a69a..0000000000 --- a/changelog.d/12501.doc +++ /dev/null @@ -1 +0,0 @@ -Add HAProxy delegation example with CORS headers to docs. diff --git a/changelog.d/12510.bugfix b/changelog.d/12510.bugfix deleted file mode 100644 index d5856e982a..0000000000 --- a/changelog.d/12510.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a long-standing bug where the image thumbanils embedded into email notifications were broken. diff --git a/changelog.d/12511.misc b/changelog.d/12511.misc deleted file mode 100644 index a314bedfc4..0000000000 --- a/changelog.d/12511.misc +++ /dev/null @@ -1 +0,0 @@ -Remove unnecessary configuration overrides in tests. diff --git a/changelog.d/12514.misc b/changelog.d/12514.misc deleted file mode 100644 index 061a604a1e..0000000000 --- a/changelog.d/12514.misc +++ /dev/null @@ -1 +0,0 @@ -Use poetry-core instead of setuptools to build wheels. diff --git a/changelog.d/12519.misc b/changelog.d/12519.misc deleted file mode 100644 index 9c023d8e3e..0000000000 --- a/changelog.d/12519.misc +++ /dev/null @@ -1 +0,0 @@ -Refactor the relations code for clarity. diff --git a/changelog.d/12520.bugfix b/changelog.d/12520.bugfix deleted file mode 100644 index c73005fde8..0000000000 --- a/changelog.d/12520.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a bug in the implementation of MSC3202 where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`. \ No newline at end of file diff --git a/changelog.d/12522.bugfix b/changelog.d/12522.bugfix deleted file mode 100644 index 2220f05ceb..0000000000 --- a/changelog.d/12522.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room. diff --git a/changelog.d/12527.doc b/changelog.d/12527.doc deleted file mode 100644 index e6907321e7..0000000000 --- a/changelog.d/12527.doc +++ /dev/null @@ -1,2 +0,0 @@ -Add an index to the configuration manual. - diff --git a/changelog.d/12528.misc b/changelog.d/12528.misc deleted file mode 100644 index f64b5d24b0..0000000000 --- a/changelog.d/12528.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`. diff --git a/changelog.d/12529.misc b/changelog.d/12529.misc deleted file mode 100644 index 5427108742..0000000000 --- a/changelog.d/12529.misc +++ /dev/null @@ -1 +0,0 @@ -Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`. diff --git a/changelog.d/12533.doc b/changelog.d/12533.doc deleted file mode 100644 index 2c15488111..0000000000 --- a/changelog.d/12533.doc +++ /dev/null @@ -1 +0,0 @@ -Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari. \ No newline at end of file diff --git a/changelog.d/12543.misc b/changelog.d/12543.misc deleted file mode 100644 index eed7a6973c..0000000000 --- a/changelog.d/12543.misc +++ /dev/null @@ -1 +0,0 @@ -Build debian packages for Ubuntu 22.04 "Jammy Jellyfish". diff --git a/debian/changelog b/debian/changelog index 05e6bd75a7..5f1bf872bb 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,8 +1,15 @@ -matrix-synapse-py3 (1.58.0+nmu1) UNRELEASED; urgency=medium +matrix-synapse-py3 (1.58.0~rc2) stable; urgency=medium + + * New Synapse release 1.58.0rc2. + + -- Synapse Packaging team <packages@matrix.org> Tue, 26 Apr 2022 17:14:56 +0100 + +matrix-synapse-py3 (1.58.0~rc1) stable; urgency=medium * Use poetry to manage the bundled virtualenv included with this package. + * New Synapse release 1.58.0rc1. - -- Synapse Packaging team <packages@matrix.org> Wed, 30 Mar 2022 12:21:43 +0100 + -- Synapse Packaging team <packages@matrix.org> Tue, 26 Apr 2022 11:15:20 +0100 matrix-synapse-py3 (1.57.1) stable; urgency=medium diff --git a/poetry.lock b/poetry.lock index 95c1afc077..8c7af1fa1e 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,12 +1,4 @@ [[package]] -name = "appdirs" -version = "1.4.4" -description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." -category = "dev" -optional = false -python-versions = "*" - -[[package]] name = "attrs" version = "21.4.0" description = "Classes Without Boilerplate" @@ -50,17 +42,6 @@ six = "*" visualize = ["graphviz (>0.5.1)", "Twisted (>=16.1.1)"] [[package]] -name = "baron" -version = "0.10.1" -description = "Full Syntax Tree for python to make writing refactoring code a realist task" -category = "dev" -optional = false -python-versions = "*" - -[package.dependencies] -rply = "*" - -[[package]] name = "bcrypt" version = "3.2.0" description = "Modern password hashing for your software and your servers" @@ -985,20 +966,6 @@ Pygments = ">=2.5.1" md = ["cmarkgfm (>=0.8.0)"] [[package]] -name = "redbaron" -version = "0.9.2" -description = "Abstraction on top of baron, a FST for python to make writing refactoring code a realistic task" -category = "dev" -optional = false -python-versions = "*" - -[package.dependencies] -baron = ">=0.7" - -[package.extras] -notebook = ["pygments"] - -[[package]] name = "requests" version = "2.27.1" description = "Python HTTP for Humans." @@ -1039,17 +1006,6 @@ python-versions = ">=3.7" idna2008 = ["idna"] [[package]] -name = "rply" -version = "0.7.8" -description = "A pure Python Lex/Yacc that works with RPython" -category = "dev" -optional = false -python-versions = "*" - -[package.dependencies] -appdirs = "*" - -[[package]] name = "secretstorage" version = "3.3.1" description = "Python bindings to FreeDesktop.org Secret Service API" @@ -1597,13 +1553,9 @@ url_preview = ["lxml"] [metadata] lock-version = "1.1" python-versions = "^3.7" -content-hash = "964ad29eaf7fd02749a4e735818f3bc0ba729c2f4b9e3213f0daa02643508b16" +content-hash = "f482a4f594a165dfe01ce253a22510d5faf38647ab0dcebc35789350cafd9bf0" [metadata.files] -appdirs = [ - {file = "appdirs-1.4.4-py2.py3-none-any.whl", hash = "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"}, - {file = "appdirs-1.4.4.tar.gz", hash = "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41"}, -] attrs = [ {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"}, {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"}, @@ -1616,10 +1568,6 @@ automat = [ {file = "Automat-20.2.0-py2.py3-none-any.whl", hash = "sha256:b6feb6455337df834f6c9962d6ccf771515b7d939bca142b29c20c2376bc6111"}, {file = "Automat-20.2.0.tar.gz", hash = "sha256:7979803c74610e11ef0c0d68a2942b152df52da55336e0c9d58daf1831cbdf33"}, ] -baron = [ - {file = "baron-0.10.1-py2.py3-none-any.whl", hash = "sha256:befb33f4b9e832c7cd1e3cf0eafa6dd3cb6ed4cb2544245147c019936f4e0a8a"}, - {file = "baron-0.10.1.tar.gz", hash = "sha256:af822ad44d4eb425c8516df4239ac4fdba9fdb398ef77e4924cd7c9b4045bc2f"}, -] bcrypt = [ {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:b589229207630484aefe5899122fb938a5b017b0f4349f769b8c13e78d99a8fd"}, {file = "bcrypt-3.2.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c95d4cbebffafcdd28bd28bb4e25b31c50f6da605c81ffd9ad8a3d1b2ab7b1b6"}, @@ -2412,10 +2360,6 @@ readme-renderer = [ {file = "readme_renderer-33.0-py3-none-any.whl", hash = "sha256:f02cee0c4de9636b5a62b6be50c9742427ba1b956aad1d938bfb087d0d72ccdf"}, {file = "readme_renderer-33.0.tar.gz", hash = "sha256:e3b53bc84bd6af054e4cc1fe3567dc1ae19f554134221043a3f8c674e22209db"}, ] -redbaron = [ - {file = "redbaron-0.9.2-py2.py3-none-any.whl", hash = "sha256:d01032b6a848b5521a8d6ef72486315c2880f420956870cdd742e2b5a09b9bab"}, - {file = "redbaron-0.9.2.tar.gz", hash = "sha256:472d0739ca6b2240bb2278ae428604a75472c9c12e86c6321e8c016139c0132f"}, -] requests = [ {file = "requests-2.27.1-py2.py3-none-any.whl", hash = "sha256:f22fa1e554c9ddfd16e6e41ac79759e17be9e492b3587efa038054674760e72d"}, {file = "requests-2.27.1.tar.gz", hash = "sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61"}, @@ -2428,10 +2372,6 @@ rfc3986 = [ {file = "rfc3986-2.0.0-py2.py3-none-any.whl", hash = "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd"}, {file = "rfc3986-2.0.0.tar.gz", hash = "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"}, ] -rply = [ - {file = "rply-0.7.8-py2.py3-none-any.whl", hash = "sha256:28ffd11d656c48aeb8c508eb382acd6a0bd906662624b34388751732a27807e7"}, - {file = "rply-0.7.8.tar.gz", hash = "sha256:2a808ac25a4580a9991fc304d64434e299a8fc75760574492f242cbb5bb301c9"}, -] secretstorage = [ {file = "SecretStorage-3.3.1-py3-none-any.whl", hash = "sha256:422d82c36172d88d6a0ed5afdec956514b189ddbfb72fefab0c8a1cee4eaf71f"}, {file = "SecretStorage-3.3.1.tar.gz", hash = "sha256:fd666c51a6bf200643495a04abb261f83229dcb6fd8472ec393df7ffc8b6f195"}, diff --git a/pyproject.toml b/pyproject.toml index c7f3e20fed..bdded78434 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -54,7 +54,7 @@ skip_gitignore = true [tool.poetry] name = "matrix-synapse" -version = "1.57.1" +version = "1.58.0rc2" description = "Homeserver for the Matrix decentralised comms protocol" authors = ["Matrix.org Team and Contributors <packages@matrix.org>"] license = "Apache-2.0" @@ -270,7 +270,6 @@ idna = ">=2.5" # The following are used by the release script click = "==8.1.0" -redbaron = "==0.9.2" GitPython = "==3.1.14" commonmark = "==0.9.1" pygithub = "==1.55" diff --git a/scripts-dev/release.py b/scripts-dev/release.py index 6f7cf6888d..9d7c7c445f 100755 --- a/scripts-dev/release.py +++ b/scripts-dev/release.py @@ -25,13 +25,12 @@ import sys import urllib.request from os import path from tempfile import TemporaryDirectory -from typing import List, Optional, Tuple +from typing import List, Optional import attr import click import commonmark import git -import redbaron from click.exceptions import ClickException from github import Github from packaging import version @@ -100,7 +99,7 @@ def prepare(): repo.remote().fetch() # Get the current version and AST from root Synapse module. - current_version, parsed_synapse_ast, version_node = parse_version_from_module() + current_version = get_package_version() # Figure out what sort of release we're doing and calcuate the new version. rc = click.confirm("RC", default=True) @@ -162,7 +161,7 @@ def prepare(): click.get_current_context().abort() # Switch to the release branch. - parsed_new_version = version.parse(new_version) + parsed_new_version: version.Version = version.parse(new_version) # We assume for debian changelogs that we only do RCs or full releases. assert not parsed_new_version.is_devrelease @@ -207,17 +206,15 @@ def prepare(): # Create the new release branch release_branch = repo.create_head(release_branch_name, commit=base_branch) - # Switch to the release branch and ensure its up to date. + # Switch to the release branch and ensure it's up to date. repo.git.checkout(release_branch_name) update_branch(repo) - # Update the `__version__` variable and write it back to the file. - version_node.value = '"' + new_version + '"' - with open("synapse/__init__.py", "w") as f: - f.write(parsed_synapse_ast.dumps()) + # Update the version specified in pyproject.toml. + subprocess.check_output(["poetry", "version", new_version]) # Generate changelogs. - generate_and_write_changelog(current_version) + generate_and_write_changelog(current_version, new_version) # Generate debian changelogs if parsed_new_version.pre is not None: @@ -284,7 +281,7 @@ def tag(gh_token: Optional[str]): repo.remote().fetch() # Find out the version and tag name. - current_version, _, _ = parse_version_from_module() + current_version = get_package_version() tag_name = f"v{current_version}" # Check we haven't released this version. @@ -362,7 +359,7 @@ def publish(gh_token: str): if repo.is_dirty(): raise click.ClickException("Uncommitted changes exist.") - current_version, _, _ = parse_version_from_module() + current_version = get_package_version() tag_name = f"v{current_version}" if not click.confirm(f"Publish {tag_name}?", default=True): @@ -396,7 +393,7 @@ def publish(gh_token: str): def upload(): """Upload release to pypi.""" - current_version, _, _ = parse_version_from_module() + current_version = get_package_version() tag_name = f"v{current_version}" pypi_asset_names = [ @@ -424,7 +421,7 @@ def upload(): def announce(): """Generate markdown to announce the release.""" - current_version, _, _ = parse_version_from_module() + current_version = get_package_version() tag_name = f"v{current_version}" click.echo( @@ -455,37 +452,11 @@ Announce the release in ) -def parse_version_from_module() -> Tuple[ - version.Version, redbaron.RedBaron, redbaron.Node -]: - # Parse the AST and load the `__version__` node so that we can edit it - # later. - with open("synapse/__init__.py") as f: - red = redbaron.RedBaron(f.read()) - - version_node = None - for node in red: - if node.type != "assignment": - continue - - if node.target.type != "name": - continue - - if node.target.value != "__version__": - continue - - version_node = node - break - - if not version_node: - print("Failed to find '__version__' definition in synapse/__init__.py") - sys.exit(1) - - # Parse the current version. - current_version = version.parse(version_node.value.value.strip('"')) - assert isinstance(current_version, version.Version) - - return current_version, red, version_node +def get_package_version() -> version.Version: + version_string = subprocess.check_output(["poetry", "version", "--short"]).decode( + "utf-8" + ) + return version.Version(version_string) def find_ref(repo: git.Repo, ref_name: str) -> Optional[git.HEAD]: @@ -565,11 +536,13 @@ def get_changes_for_version(wanted_version: version.Version) -> str: return "\n".join(version_changelog) -def generate_and_write_changelog(current_version: version.Version): +def generate_and_write_changelog(current_version: version.Version, new_version: str): # We do this by getting a draft so that we can edit it before writing to the # changelog. result = run_until_successful( - "python3 -m towncrier --draft", shell=True, capture_output=True + f"python3 -m towncrier build --draft --version {new_version}", + shell=True, + capture_output=True, ) new_changes = result.stdout.decode("utf-8") new_changes = new_changes.replace( diff --git a/synapse/handlers/device.py b/synapse/handlers/device.py index 3c0fc756d4..a91b1ee4d5 100644 --- a/synapse/handlers/device.py +++ b/synapse/handlers/device.py @@ -505,8 +505,9 @@ class DeviceHandler(DeviceWorkerHandler): "device_list_key", position, users={user_id}, rooms=room_ids ) - # We may need to do some processing asynchronously. - self._handle_new_device_update_async() + # We may need to do some processing asynchronously for local user IDs. + if self.hs.is_mine_id(user_id): + self._handle_new_device_update_async() async def notify_user_signature_update( self, from_user_id: str, user_ids: List[str] @@ -683,9 +684,12 @@ class DeviceHandler(DeviceWorkerHandler): self.federation_sender.send_device_messages( host, immediate=False ) - log_kv( - {"message": "sent device update to host", "host": host} - ) + # TODO: when called, this isn't in a logging context. + # This leads to log spam, sentry event spam, and massive + # memory usage. See #12552. + # log_kv( + # {"message": "sent device update to host", "host": host} + # ) if current_stream_id != stream_id: # Clear the set of hosts we've already sent to as we're diff --git a/synapse/storage/databases/main/devices.py b/synapse/storage/databases/main/devices.py index 318e4df376..483dd80406 100644 --- a/synapse/storage/databases/main/devices.py +++ b/synapse/storage/databases/main/devices.py @@ -1748,7 +1748,8 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): device_id, room_id, stream_id, - False, + # We only need to calculate outbound pokes for local users + not self.hs.is_mine_id(user_id), encoded_context, ) for room_id in room_ids @@ -1776,7 +1777,17 @@ class DeviceStore(DeviceWorkerStore, DeviceBackgroundUpdateStore): def get_uncoverted_outbound_room_pokes_txn(txn): txn.execute(sql, (limit,)) - return txn.fetchall() + + return [ + ( + user_id, + device_id, + room_id, + stream_id, + db_to_json(opentracing_context), + ) + for user_id, device_id, room_id, stream_id, opentracing_context in txn + ] return await self.db_pool.runInteraction( "get_uncoverted_outbound_room_pokes", get_uncoverted_outbound_room_pokes_txn diff --git a/tests/storage/test_devices.py b/tests/storage/test_devices.py index ccc3893869..bbf079b25b 100644 --- a/tests/storage/test_devices.py +++ b/tests/storage/test_devices.py @@ -29,7 +29,7 @@ class DeviceStoreTestCase(HomeserverTestCase): for device_id in device_ids: stream_id = self.get_success( self.store.add_device_change_to_streams( - "user_id", [device_id], ["!some:room"] + user_id, [device_id], ["!some:room"] ) ) |