Merge tag 'v1.39.0rc1' into develop

Synapse 1.39.0rc1 (2021-07-20)
==============================

The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.

Features
--------

- Add the ability to override the account validity feature with a module. ([\#9884](https://github.com/matrix-org/synapse/issues/9884))
- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](https://github.com/matrix-org/synapse/issues/10298), [\#10305](https://github.com/matrix-org/synapse/issues/10305))
- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](https://github.com/matrix-org/synapse/issues/10332), [\#10427](https://github.com/matrix-org/synapse/issues/10427))
- Allow providing credentials to `http_proxy`. ([\#10360](https://github.com/matrix-org/synapse/issues/10360))

Bugfixes
--------

- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](https://github.com/matrix-org/synapse/issues/10433))
- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](https://github.com/matrix-org/synapse/issues/10250))
- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](https://github.com/matrix-org/synapse/issues/10276))
- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](https://github.com/matrix-org/synapse/issues/10317))
- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](https://github.com/matrix-org/synapse/issues/10343))
- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](https://github.com/matrix-org/synapse/issues/10344))
- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](https://github.com/matrix-org/synapse/issues/10355))
- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](https://github.com/matrix-org/synapse/issues/10359))
- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](https://github.com/matrix-org/synapse/issues/10367))
- Fix a number of logged errors caused by remote servers being down. ([\#10400](https://github.com/matrix-org/synapse/issues/10400), [\#10414](https://github.com/matrix-org/synapse/issues/10414))
- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](https://github.com/matrix-org/synapse/issues/10404))

Improved Documentation
----------------------

- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](https://github.com/matrix-org/synapse/issues/9971))
- Simplify structure of room admin API. ([\#10313](https://github.com/matrix-org/synapse/issues/10313))
- Refresh the logcontext dev documentation. ([\#10353](https://github.com/matrix-org/synapse/issues/10353)), ([\#10337](https://github.com/matrix-org/synapse/issues/10337))
- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](https://github.com/matrix-org/synapse/issues/10368))
- Fix and clarify some links in `docs` and `contrib`. ([\#10370](https://github.com/matrix-org/synapse/issues/10370)), ([\#10322](https://github.com/matrix-org/synapse/issues/10322)), ([\#10399](https://github.com/matrix-org/synapse/issues/10399))
- Make deprecation notice of the spam checker doc more obvious. ([\#10395](https://github.com/matrix-org/synapse/issues/10395))
- Add instructions on installing Debian packages for release candidates. ([\#10396](https://github.com/matrix-org/synapse/issues/10396))

Deprecations and Removals
-------------------------

- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](https://github.com/matrix-org/synapse/issues/9721))
- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](https://github.com/matrix-org/synapse/issues/10386))

Internal Changes
----------------

- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](https://github.com/matrix-org/synapse/issues/10289))
- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](https://github.com/matrix-org/synapse/issues/10315))
- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](https://github.com/matrix-org/synapse/issues/10316))
- Minor change to the code that populates `user_daily_visits`. ([\#10324](https://github.com/matrix-org/synapse/issues/10324))
- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](https://github.com/matrix-org/synapse/issues/10345), [\#10357](https://github.com/matrix-org/synapse/issues/10357))
- Run `pyupgrade` on the codebase. ([\#10347](https://github.com/matrix-org/synapse/issues/10347), [\#10348](https://github.com/matrix-org/synapse/issues/10348))
- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](https://github.com/matrix-org/synapse/issues/10349))
- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](https://github.com/matrix-org/synapse/issues/10350), [\#10380](https://github.com/matrix-org/synapse/issues/10380), [\#10381](https://github.com/matrix-org/synapse/issues/10381), [\#10382](https://github.com/matrix-org/synapse/issues/10382), [\#10418](https://github.com/matrix-org/synapse/issues/10418))
- Make the Github Actions workflow configuration more efficient. ([\#10383](https://github.com/matrix-org/synapse/issues/10383))
- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](https://github.com/matrix-org/synapse/issues/10385))
- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](https://github.com/matrix-org/synapse/issues/10391))
- Add type hints and comments to event auth code. ([\#10393](https://github.com/matrix-org/synapse/issues/10393))
- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](https://github.com/matrix-org/synapse/issues/10398))
- Remove unused `events_by_room` code (tech debt). ([\#10421](https://github.com/matrix-org/synapse/issues/10421))
- Add a github actions job which records success of other jobs. ([\#10430](https://github.com/matrix-org/synapse/issues/10430))

55 files changed, 94 insertions, 52 deletions

diff --git a/CHANGES.md b/CHANGES.md
index 82baaa2d1f..066f798a95 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,3 +1,72 @@
+Synapse 1.39.0rc1 (2021-07-20)
+==============================
+
+The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.
+
+Features
+--------
+
+- Add the ability to override the account validity feature with a module. ([\#9884](https://github.com/matrix-org/synapse/issues/9884))
+- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](https://github.com/matrix-org/synapse/issues/10298), [\#10305](https://github.com/matrix-org/synapse/issues/10305))
+- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](https://github.com/matrix-org/synapse/issues/10332), [\#10427](https://github.com/matrix-org/synapse/issues/10427))
+- Allow providing credentials to `http_proxy`. ([\#10360](https://github.com/matrix-org/synapse/issues/10360))
+
+
+Bugfixes
+--------
+
+- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](https://github.com/matrix-org/synapse/issues/10433))
+- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](https://github.com/matrix-org/synapse/issues/10250))
+- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](https://github.com/matrix-org/synapse/issues/10276))
+- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](https://github.com/matrix-org/synapse/issues/10317))
+- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](https://github.com/matrix-org/synapse/issues/10343))
+- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](https://github.com/matrix-org/synapse/issues/10344))
+- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](https://github.com/matrix-org/synapse/issues/10355))
+- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](https://github.com/matrix-org/synapse/issues/10359))
+- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](https://github.com/matrix-org/synapse/issues/10367))
+- Fix a number of logged errors caused by remote servers being down. ([\#10400](https://github.com/matrix-org/synapse/issues/10400), [\#10414](https://github.com/matrix-org/synapse/issues/10414))
+- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](https://github.com/matrix-org/synapse/issues/10404))
+
+
+Improved Documentation
+----------------------
+
+- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](https://github.com/matrix-org/synapse/issues/9971))
+- Simplify structure of room admin API. ([\#10313](https://github.com/matrix-org/synapse/issues/10313))
+- Refresh the logcontext dev documentation. ([\#10353](https://github.com/matrix-org/synapse/issues/10353)), ([\#10337](https://github.com/matrix-org/synapse/issues/10337))
+- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](https://github.com/matrix-org/synapse/issues/10368))
+- Fix and clarify some links in `docs` and `contrib`. ([\#10370](https://github.com/matrix-org/synapse/issues/10370)), ([\#10322](https://github.com/matrix-org/synapse/issues/10322)), ([\#10399](https://github.com/matrix-org/synapse/issues/10399))
+- Make deprecation notice of the spam checker doc more obvious. ([\#10395](https://github.com/matrix-org/synapse/issues/10395))
+- Add instructions on installing Debian packages for release candidates. ([\#10396](https://github.com/matrix-org/synapse/issues/10396))
+
+
+Deprecations and Removals
+-------------------------
+
+- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](https://github.com/matrix-org/synapse/issues/9721))
+- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](https://github.com/matrix-org/synapse/issues/10386))
+
+
+Internal Changes
+----------------
+
+- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](https://github.com/matrix-org/synapse/issues/10289))
+- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](https://github.com/matrix-org/synapse/issues/10315))
+- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](https://github.com/matrix-org/synapse/issues/10316))
+- Minor change to the code that populates `user_daily_visits`. ([\#10324](https://github.com/matrix-org/synapse/issues/10324))
+- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](https://github.com/matrix-org/synapse/issues/10345), [\#10357](https://github.com/matrix-org/synapse/issues/10357))
+- Run `pyupgrade` on the codebase. ([\#10347](https://github.com/matrix-org/synapse/issues/10347), [\#10348](https://github.com/matrix-org/synapse/issues/10348))
+- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](https://github.com/matrix-org/synapse/issues/10349))
+- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](https://github.com/matrix-org/synapse/issues/10350), [\#10380](https://github.com/matrix-org/synapse/issues/10380), [\#10381](https://github.com/matrix-org/synapse/issues/10381), [\#10382](https://github.com/matrix-org/synapse/issues/10382), [\#10418](https://github.com/matrix-org/synapse/issues/10418))
+- Make the Github Actions workflow configuration more efficient. ([\#10383](https://github.com/matrix-org/synapse/issues/10383))
+- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](https://github.com/matrix-org/synapse/issues/10385))
+- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](https://github.com/matrix-org/synapse/issues/10391))
+- Add type hints and comments to event auth code. ([\#10393](https://github.com/matrix-org/synapse/issues/10393))
+- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](https://github.com/matrix-org/synapse/issues/10398))
+- Remove unused `events_by_room` code (tech debt). ([\#10421](https://github.com/matrix-org/synapse/issues/10421))
+- Add a github actions job which records success of other jobs. ([\#10430](https://github.com/matrix-org/synapse/issues/10430))
+
+
 Synapse 1.38.0 (2021-07-13)
 ===========================
 
diff --git a/changelog.d/10250.bugfix b/changelog.d/10250.bugfix
deleted file mode 100644
index a8107dafb2..0000000000
--- a/changelog.d/10250.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Add base starting insertion event when no chunk ID is specified in the historical batch send API.
diff --git a/changelog.d/10276.bugfix b/changelog.d/10276.bugfix
deleted file mode 100644
index 42adc57ad1..0000000000
--- a/changelog.d/10276.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders.
diff --git a/changelog.d/10289.misc b/changelog.d/10289.misc
deleted file mode 100644
index 2df30e7a7a..0000000000
--- a/changelog.d/10289.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert `room_depth.min_depth` column to a `BIGINT`.
diff --git a/changelog.d/10298.feature b/changelog.d/10298.feature
deleted file mode 100644
index 7059db5075..0000000000
--- a/changelog.d/10298.feature
+++ /dev/null
@@ -1 +0,0 @@
-The spaces summary API now returns any joinable rooms, not only rooms which are world-readable.
diff --git a/changelog.d/10305.feature b/changelog.d/10305.feature
deleted file mode 100644
index 7059db5075..0000000000
--- a/changelog.d/10305.feature
+++ /dev/null
@@ -1 +0,0 @@
-The spaces summary API now returns any joinable rooms, not only rooms which are world-readable.
diff --git a/changelog.d/10313.doc b/changelog.d/10313.doc
deleted file mode 100644
index 44086e3d9d..0000000000
--- a/changelog.d/10313.doc
+++ /dev/null
@@ -1 +0,0 @@
-Simplify structure of room admin API.
\ No newline at end of file
diff --git a/changelog.d/10315.misc b/changelog.d/10315.misc
deleted file mode 100644
index 2c78644e20..0000000000
--- a/changelog.d/10315.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add tests to characterise the current behaviour of R30 phone-home metrics.
diff --git a/changelog.d/10316.misc b/changelog.d/10316.misc
deleted file mode 100644
index 1fd0810fde..0000000000
--- a/changelog.d/10316.misc
+++ /dev/null
@@ -1 +0,0 @@
-Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules.
diff --git a/changelog.d/10317.bugfix b/changelog.d/10317.bugfix
deleted file mode 100644
index 826c269eff..0000000000
--- a/changelog.d/10317.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari.
diff --git a/changelog.d/10322.doc b/changelog.d/10322.doc
deleted file mode 100644
index db604cf2aa..0000000000
--- a/changelog.d/10322.doc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a broken link in the admin api docs.
diff --git a/changelog.d/10324.misc b/changelog.d/10324.misc
deleted file mode 100644
index 3c3ee6d6fc..0000000000
--- a/changelog.d/10324.misc
+++ /dev/null
@@ -1 +0,0 @@
-Minor change to the code that populates `user_daily_visits`.
diff --git a/changelog.d/10332.feature b/changelog.d/10332.feature
deleted file mode 100644
index 091947ff22..0000000000
--- a/changelog.d/10332.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric.
diff --git a/changelog.d/10337.doc b/changelog.d/10337.doc
deleted file mode 100644
index f305bdb3ba..0000000000
--- a/changelog.d/10337.doc
+++ /dev/null
@@ -1 +0,0 @@
-Fix formatting in the logcontext documentation.
diff --git a/changelog.d/10343.bugfix b/changelog.d/10343.bugfix
deleted file mode 100644
index 53ccf79a81..0000000000
--- a/changelog.d/10343.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir).
diff --git a/changelog.d/10344.bugfix b/changelog.d/10344.bugfix
deleted file mode 100644
index ab6eb4999f..0000000000
--- a/changelog.d/10344.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use.
diff --git a/changelog.d/10345.misc b/changelog.d/10345.misc
deleted file mode 100644
index 7424486e87..0000000000
--- a/changelog.d/10345.misc
+++ /dev/null
@@ -1 +0,0 @@
-Re-enable Sytests that were disabled for the 1.37.1 release.
diff --git a/changelog.d/10347.misc b/changelog.d/10347.misc
deleted file mode 100644
index b2275a1350..0000000000
--- a/changelog.d/10347.misc
+++ /dev/null
@@ -1 +0,0 @@
-Run `pyupgrade` on the codebase.
\ No newline at end of file
diff --git a/changelog.d/10348.misc b/changelog.d/10348.misc
deleted file mode 100644
index b2275a1350..0000000000
--- a/changelog.d/10348.misc
+++ /dev/null
@@ -1 +0,0 @@
-Run `pyupgrade` on the codebase.
\ No newline at end of file
diff --git a/changelog.d/10349.misc b/changelog.d/10349.misc
deleted file mode 100644
index 5b014e7416..0000000000
--- a/changelog.d/10349.misc
+++ /dev/null
@@ -1 +0,0 @@
-Switch `application_services_txns.txn_id` database column to `BIGINT`.
diff --git a/changelog.d/10350.misc b/changelog.d/10350.misc
deleted file mode 100644
index eed2d8552a..0000000000
--- a/changelog.d/10350.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert internal type variable syntax to reflect wider ecosystem use.
\ No newline at end of file
diff --git a/changelog.d/10353.doc b/changelog.d/10353.doc
deleted file mode 100644
index 274ac83549..0000000000
--- a/changelog.d/10353.doc
+++ /dev/null
@@ -1 +0,0 @@
-Refresh the logcontext dev documentation.
diff --git a/changelog.d/10355.bugfix b/changelog.d/10355.bugfix
deleted file mode 100644
index 92df612011..0000000000
--- a/changelog.d/10355.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp.
diff --git a/changelog.d/10357.misc b/changelog.d/10357.misc
deleted file mode 100644
index 7424486e87..0000000000
--- a/changelog.d/10357.misc
+++ /dev/null
@@ -1 +0,0 @@
-Re-enable Sytests that were disabled for the 1.37.1 release.
diff --git a/changelog.d/10359.bugfix b/changelog.d/10359.bugfix
deleted file mode 100644
index d318f8fa08..0000000000
--- a/changelog.d/10359.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO.
diff --git a/changelog.d/10360.feature b/changelog.d/10360.feature
deleted file mode 100644
index 904221cb6d..0000000000
--- a/changelog.d/10360.feature
+++ /dev/null
@@ -1 +0,0 @@
-Allow providing credentials to `http_proxy`.
\ No newline at end of file
diff --git a/changelog.d/10367.bugfix b/changelog.d/10367.bugfix
deleted file mode 100644
index b445556084..0000000000
--- a/changelog.d/10367.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Bugfix `make_room_admin` fails for users that have left a private room.
\ No newline at end of file
diff --git a/changelog.d/10368.doc b/changelog.d/10368.doc
deleted file mode 100644
index 10297aa424..0000000000
--- a/changelog.d/10368.doc
+++ /dev/null
@@ -1 +0,0 @@
-Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz.
diff --git a/changelog.d/10370.doc b/changelog.d/10370.doc
deleted file mode 100644
index 8c59d98ee8..0000000000
--- a/changelog.d/10370.doc
+++ /dev/null
@@ -1 +0,0 @@
-Fix some links in `docs` and `contrib`.
\ No newline at end of file
diff --git a/changelog.d/10380.misc b/changelog.d/10380.misc
deleted file mode 100644
index eed2d8552a..0000000000
--- a/changelog.d/10380.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert internal type variable syntax to reflect wider ecosystem use.
\ No newline at end of file
diff --git a/changelog.d/10381.misc b/changelog.d/10381.misc
deleted file mode 100644
index eed2d8552a..0000000000
--- a/changelog.d/10381.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert internal type variable syntax to reflect wider ecosystem use.
\ No newline at end of file
diff --git a/changelog.d/10382.misc b/changelog.d/10382.misc
deleted file mode 100644
index eed2d8552a..0000000000
--- a/changelog.d/10382.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert internal type variable syntax to reflect wider ecosystem use.
\ No newline at end of file
diff --git a/changelog.d/10383.misc b/changelog.d/10383.misc
deleted file mode 100644
index 952c1e77a8..0000000000
--- a/changelog.d/10383.misc
+++ /dev/null
@@ -1 +0,0 @@
-Make the Github Actions workflow configuration more efficient.
diff --git a/changelog.d/10385.misc b/changelog.d/10385.misc
deleted file mode 100644
index e515ac09fd..0000000000
--- a/changelog.d/10385.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add type hints to `get_{domain,localpart}_from_id`.
diff --git a/changelog.d/10386.removal b/changelog.d/10386.removal
deleted file mode 100644
index 800a6143d7..0000000000
--- a/changelog.d/10386.removal
+++ /dev/null
@@ -1 +0,0 @@
-The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information.
diff --git a/changelog.d/10391.misc b/changelog.d/10391.misc
deleted file mode 100644
index 3f191b520a..0000000000
--- a/changelog.d/10391.misc
+++ /dev/null
@@ -1 +0,0 @@
-When building Debian packages for prerelease versions, set the Section accordingly.
diff --git a/changelog.d/10393.misc b/changelog.d/10393.misc
deleted file mode 100644
index e80f16d607..0000000000
--- a/changelog.d/10393.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add type hints and comments to event auth code.
diff --git a/changelog.d/10395.doc b/changelog.d/10395.doc
deleted file mode 100644
index 4bdaea76c5..0000000000
--- a/changelog.d/10395.doc
+++ /dev/null
@@ -1 +0,0 @@
-Make deprecation notice of the spam checker doc more obvious.
diff --git a/changelog.d/10396.doc b/changelog.d/10396.doc
deleted file mode 100644
index b521ad9cbf..0000000000
--- a/changelog.d/10396.doc
+++ /dev/null
@@ -1 +0,0 @@
-Add instructructions on installing Debian packages for release candidates.
diff --git a/changelog.d/10398.misc b/changelog.d/10398.misc
deleted file mode 100644
index 326e54655a..0000000000
--- a/changelog.d/10398.misc
+++ /dev/null
@@ -1 +0,0 @@
-Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once.
diff --git a/changelog.d/10399.doc b/changelog.d/10399.doc
deleted file mode 100644
index b596ac5627..0000000000
--- a/changelog.d/10399.doc
+++ /dev/null
@@ -1 +0,0 @@
-Rewrite the text of links to be clearer in the documentation.
diff --git a/changelog.d/10400.bugfix b/changelog.d/10400.bugfix
deleted file mode 100644
index bfebed8d29..0000000000
--- a/changelog.d/10400.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a number of logged errors caused by remote servers being down.
diff --git a/changelog.d/10404.bugfix b/changelog.d/10404.bugfix
deleted file mode 100644
index 2e095b6402..0000000000
--- a/changelog.d/10404.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`.
diff --git a/changelog.d/10414.bugfix b/changelog.d/10414.bugfix
deleted file mode 100644
index bfebed8d29..0000000000
--- a/changelog.d/10414.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a number of logged errors caused by remote servers being down.
diff --git a/changelog.d/10418.misc b/changelog.d/10418.misc
deleted file mode 100644
index eed2d8552a..0000000000
--- a/changelog.d/10418.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert internal type variable syntax to reflect wider ecosystem use.
\ No newline at end of file
diff --git a/changelog.d/10421.misc b/changelog.d/10421.misc
deleted file mode 100644
index 385cbe07af..0000000000
--- a/changelog.d/10421.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove unused `events_by_room` code (tech debt).
diff --git a/changelog.d/10427.feature b/changelog.d/10427.feature
deleted file mode 100644
index 091947ff22..0000000000
--- a/changelog.d/10427.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric.
diff --git a/changelog.d/10430.misc b/changelog.d/10430.misc
deleted file mode 100644
index a017cf4ac9..0000000000
--- a/changelog.d/10430.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add a github actions job recording success of other jobs.
diff --git a/changelog.d/9721.removal b/changelog.d/9721.removal
deleted file mode 100644
index da2ba48c84..0000000000
--- a/changelog.d/9721.removal
+++ /dev/null
@@ -1 +0,0 @@
-Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz.
diff --git a/changelog.d/9884.feature b/changelog.d/9884.feature
deleted file mode 100644
index 525fd2f93c..0000000000
--- a/changelog.d/9884.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a module type for the account validity feature.
diff --git a/changelog.d/9971.doc b/changelog.d/9971.doc
deleted file mode 100644
index ada68f70ca..0000000000
--- a/changelog.d/9971.doc
+++ /dev/null
@@ -1 +0,0 @@
-Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh.
diff --git a/debian/changelog b/debian/changelog
index 43d26fc133..4d214c23b6 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+matrix-synapse-py3 (1.39.0~rc1) stable; urgency=medium
+
+  * New synapse release 1.39.0rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 20 Jul 2021 14:28:34 +0100
+
 matrix-synapse-py3 (1.38.0) stable; urgency=medium
 
   * New synapse release 1.38.0.
diff --git a/synapse/__init__.py b/synapse/__init__.py
index 5ecce24eee..46902adab5 100644
--- a/synapse/__init__.py
+++ b/synapse/__init__.py
@@ -47,7 +47,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.38.0"
+__version__ = "1.39.0rc1"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when
diff --git a/synapse/storage/databases/main/lock.py b/synapse/storage/databases/main/lock.py
index 774861074c..3d1dff660b 100644
--- a/synapse/storage/databases/main/lock.py
+++ b/synapse/storage/databases/main/lock.py
@@ -78,7 +78,11 @@ class LockStore(SQLBaseStore):
         """Called when the server is shutting down"""
         logger.info("Dropping held locks due to shutdown")
 
-        for (lock_name, lock_key), token in self._live_tokens.items():
+        # We need to take a copy of the tokens dict as dropping the locks will
+        # cause the dictionary to change.
+        tokens = dict(self._live_tokens)
+
+        for (lock_name, lock_key), token in tokens.items():
             await self._drop_lock(lock_name, lock_key, token)
 
         logger.info("Dropped locks due to shutdown")
diff --git a/tests/storage/databases/main/test_lock.py b/tests/storage/databases/main/test_lock.py
index 9ca70e7367..d326a1d6a6 100644
--- a/tests/storage/databases/main/test_lock.py
+++ b/tests/storage/databases/main/test_lock.py
@@ -98,3 +98,16 @@ class LockTestCase(unittest.HomeserverTestCase):
 
         lock2 = self.get_success(self.store.try_acquire_lock("name", "key"))
         self.assertIsNotNone(lock2)
+
+    def test_shutdown(self):
+        """Test that shutting down Synapse releases the locks"""
+        # Acquire two locks
+        lock = self.get_success(self.store.try_acquire_lock("name", "key1"))
+        self.assertIsNotNone(lock)
+        lock2 = self.get_success(self.store.try_acquire_lock("name", "key2"))
+        self.assertIsNotNone(lock2)
+
+        # Now call the shutdown code
+        self.get_success(self.store._on_shutdown())
+
+        self.assertEqual(self.store._live_tokens, {})