Merge remote-tracking branch 'origin/release-v1.45' into matrix-org-hotfixes

70 files changed, 176 insertions, 93 deletions

diff --git a/CHANGES.md b/CHANGES.md
index 3f048ba881..5acc8b537e 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,3 +1,85 @@
+Synapse 1.45.0rc1 (2021-10-12)
+==============================
+
+**Note:** We are aware of [a performance issue](https://github.com/matrix-org/synapse/issues/11049) introduced in Synapse v1.44.0, as well as [a bug](https://github.com/matrix-org/synapse/issues/11025) with the user directory when using application services. While this release candidate doesn't fix either of those issues, a second release candidate is expected to come out in a few days to address them.
+
+Media storage providers module that read from Synapse's configuration need changes as of this version, see the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1450) for more information.
+
+Features
+--------
+
+- Add [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069) support to `/account/whoami`. ([\#9655](https://github.com/matrix-org/synapse/issues/9655))
+- Support autodiscovery of oEmbed previews. ([\#10822](https://github.com/matrix-org/synapse/issues/10822))
+- Add a `user_may_send_3pid_invite` spam checker callback for modules to allow or deny 3PID invites. ([\#10894](https://github.com/matrix-org/synapse/issues/10894))
+- Add a spam checker callback to allow or deny room joins. ([\#10910](https://github.com/matrix-org/synapse/issues/10910))
+- Include an `update_synapse_database` script in the distribution. Contributed by @Fizzadar at Beeper. ([\#10954](https://github.com/matrix-org/synapse/issues/10954))
+- Include exception information in JSON logging output. Contributed by @Fizzadar at Beeper. ([\#11028](https://github.com/matrix-org/synapse/issues/11028))
+
+
+Bugfixes
+--------
+
+- Fix a minor bug in the response to `/_matrix/client/r0/voip/turnServer`. Contributed by @lukaslihotzki. ([\#10922](https://github.com/matrix-org/synapse/issues/10922))
+- Fix a bug where empty `yyyy-mm-dd/` directories would be left behind in the media store's `url_cache_thumbnails/` directory. ([\#10924](https://github.com/matrix-org/synapse/issues/10924))
+- Fix a bug introduced in Synapse v1.40.0 where the signature checks for room version 8 and 9 could be applied to earlier room versions in some situations. ([\#10927](https://github.com/matrix-org/synapse/issues/10927))
+- Fix a long-standing bug wherein deactivated users still count towards the monthly active users limit. ([\#10947](https://github.com/matrix-org/synapse/issues/10947))
+- Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state. ([\#10956](https://github.com/matrix-org/synapse/issues/10956))
+- Fix a long-standing bug where rebuilding the user directory wouldn't exclude support and deactivated users. ([\#10960](https://github.com/matrix-org/synapse/issues/10960))
+- Fix [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint rejecting subsequent batches with unknown batch ID error in existing room versions from the room creator. ([\#10962](https://github.com/matrix-org/synapse/issues/10962))
+- Fix a bug that could leak local users' per-room nicknames and avatars when the user directory is rebuilt. ([\#10981](https://github.com/matrix-org/synapse/issues/10981))
+- Fix a long-standing bug where the remainder of a batch of user directory changes would be silently dropped if the server left a room early in the batch. ([\#10982](https://github.com/matrix-org/synapse/issues/10982))
+- Correct a bugfix introduced in Synapse v1.44.0 that would catch the wrong error if a connection is lost before a response could be written to it. ([\#10995](https://github.com/matrix-org/synapse/issues/10995))
+- Fix a long-standing bug where local users' per-room nicknames/avatars were visible to anyone who could see you in the user directory. ([\#11002](https://github.com/matrix-org/synapse/issues/11002))
+- Fix a long-standing bug where a user's per-room nickname/avatar would overwrite their profile in the user directory when a room was made public. ([\#11003](https://github.com/matrix-org/synapse/issues/11003))
+- Work around a regression, introduced in Synapse v1.39.0, that caused `SynapseError`s raised by the experimental third-party rules module callback `check_event_allowed` to be ignored. ([\#11042](https://github.com/matrix-org/synapse/issues/11042))
+- Fix a bug in [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) insertion events in rooms that could cause cross-talk/conflicts between batches. ([\#10877](https://github.com/matrix-org/synapse/issues/10877))
+
+
+Improved Documentation
+----------------------
+
+- Change wording ("reference homeserver") in Synapse repository documentation. Contributed by @maxkratz. ([\#10971](https://github.com/matrix-org/synapse/issues/10971))
+- Fix a dead URL in development documentation (SAML) and change wording from "Riot" to "Element". Contributed by @maxkratz. ([\#10973](https://github.com/matrix-org/synapse/issues/10973))
+- Add additional content to the Welcome and Overview page of the documentation. ([\#10990](https://github.com/matrix-org/synapse/issues/10990))
+- Update links to MSCs in documentation. Contributed by @dklimpel. ([\#10991](https://github.com/matrix-org/synapse/issues/10991))
+
+
+Internal Changes
+----------------
+
+- Improve type hinting in `synapse.util`. ([\#10888](https://github.com/matrix-org/synapse/issues/10888))
+- Add further type hints to `synapse.storage.util`. ([\#10892](https://github.com/matrix-org/synapse/issues/10892))
+- Fix type hints to be compatible with an upcoming change to Twisted. ([\#10895](https://github.com/matrix-org/synapse/issues/10895))
+- Update utility code to handle C implementations of frozendict. ([\#10902](https://github.com/matrix-org/synapse/issues/10902))
+- Drop old functionality which maintained database compatibility with Synapse versions before v1.31. ([\#10903](https://github.com/matrix-org/synapse/issues/10903))
+- Clean-up configuration helper classes for the `ServerConfig` class. ([\#10915](https://github.com/matrix-org/synapse/issues/10915))
+- Use direct references to config flags. ([\#10916](https://github.com/matrix-org/synapse/issues/10916), [\#10959](https://github.com/matrix-org/synapse/issues/10959), [\#10985](https://github.com/matrix-org/synapse/issues/10985))
+- Clean up some of the federation event authentication code for clarity. ([\#10926](https://github.com/matrix-org/synapse/issues/10926), [\#10940](https://github.com/matrix-org/synapse/issues/10940), [\#10986](https://github.com/matrix-org/synapse/issues/10986), [\#10987](https://github.com/matrix-org/synapse/issues/10987), [\#10988](https://github.com/matrix-org/synapse/issues/10988), [\#11010](https://github.com/matrix-org/synapse/issues/11010), [\#11011](https://github.com/matrix-org/synapse/issues/11011))
+- Refactor various parts of the codebase to use `RoomVersion` objects instead of room version identifier strings. ([\#10934](https://github.com/matrix-org/synapse/issues/10934))
+- Refactor user directory tests in preparation for upcoming changes. ([\#10935](https://github.com/matrix-org/synapse/issues/10935))
+- Include the event id in the logcontext when handling PDUs received over federation. ([\#10936](https://github.com/matrix-org/synapse/issues/10936))
+- Fix logged errors in unit tests. ([\#10939](https://github.com/matrix-org/synapse/issues/10939))
+- Fix a broken test to ensure that consent configuration works during registration. ([\#10945](https://github.com/matrix-org/synapse/issues/10945))
+- Add type hints to filtering classes. ([\#10958](https://github.com/matrix-org/synapse/issues/10958))
+- Add type-hint to `HomeserverTestcase.setup_test_homeserver`. ([\#10961](https://github.com/matrix-org/synapse/issues/10961))
+- Fix the test utility function `create_room_as` so that `is_public=True` will explicitly set the `visibility` parameter of room creation requests to `public`. Contributed by @AndrewFerr. ([\#10963](https://github.com/matrix-org/synapse/issues/10963))
+- Make the release script more robust and transparent. ([\#10966](https://github.com/matrix-org/synapse/issues/10966))
+- Refactor [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` mega function into smaller handler functions. ([\#10974](https://github.com/matrix-org/synapse/issues/10974))
+- Log stack traces when a missing opentracing span is detected. ([\#10983](https://github.com/matrix-org/synapse/issues/10983))
+- Update GHA config to run tests against Python 3.10 and PostgreSQL 14. ([\#10992](https://github.com/matrix-org/synapse/issues/10992))
+- Fix a long-standing bug where `ReadWriteLock`s could drop logging contexts on exit. ([\#10993](https://github.com/matrix-org/synapse/issues/10993))
+- Add a `CODEOWNERS` file to automatically request reviews from the `@matrix-org/synapse-core` team on new pull requests. ([\#10994](https://github.com/matrix-org/synapse/issues/10994))
+- Add further type hints to `synapse.state`. ([\#11004](https://github.com/matrix-org/synapse/issues/11004))
+- Remove the deprecated `BaseHandler` object. ([\#11005](https://github.com/matrix-org/synapse/issues/11005))
+- Bump mypy version for CI to 0.910, and pull in new type stubs for dependencies. ([\#11006](https://github.com/matrix-org/synapse/issues/11006))
+- Fix CI to run the unit tests without optional deps. ([\#11017](https://github.com/matrix-org/synapse/issues/11017))
+- Ensure that cache config tests do not share state. ([\#11019](https://github.com/matrix-org/synapse/issues/11019))
+- Add additional type hints to `synapse.server_notices`. ([\#11021](https://github.com/matrix-org/synapse/issues/11021))
+- Add additional type hints for `synapse.push`. ([\#11023](https://github.com/matrix-org/synapse/issues/11023))
+- When installing the optional developer dependencies, also include the dependencies needed for type-checking and unit testing. ([\#11034](https://github.com/matrix-org/synapse/issues/11034))
+- Remove unnecessary list comprehension from `synapse_port_db` to satisfy code style requirements. ([\#11043](https://github.com/matrix-org/synapse/issues/11043))
+
+
 Synapse 1.44.0 (2021-10-05)
 ===========================
 
diff --git a/changelog.d/10822.feature b/changelog.d/10822.feature
deleted file mode 100644
index 72566e31ec..0000000000
--- a/changelog.d/10822.feature
+++ /dev/null
@@ -1 +0,0 @@
-Support autodiscovery of oEmbed previews.
diff --git a/changelog.d/10877.feature b/changelog.d/10877.feature
deleted file mode 100644
index 06a246c108..0000000000
--- a/changelog.d/10877.feature
+++ /dev/null
@@ -1 +0,0 @@
-Ensure `(room_id, next_batch_id)` is unique across [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) insertion events in rooms to avoid cross-talk/conflicts between batches.
diff --git a/changelog.d/10888.misc b/changelog.d/10888.misc
deleted file mode 100644
index d9c9917881..0000000000
--- a/changelog.d/10888.misc
+++ /dev/null
@@ -1 +0,0 @@
-Improve type hinting in `synapse.util`.
\ No newline at end of file
diff --git a/changelog.d/10892.misc b/changelog.d/10892.misc
deleted file mode 100644
index c8c471159b..0000000000
--- a/changelog.d/10892.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add further type hints to `synapse.storage.util`.
diff --git a/changelog.d/10894.feature b/changelog.d/10894.feature
deleted file mode 100644
index a4f968bed1..0000000000
--- a/changelog.d/10894.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a `user_may_send_3pid_invite` spam checker callback for modules to allow or deny 3PID invites.
diff --git a/changelog.d/10895.misc b/changelog.d/10895.misc
deleted file mode 100644
index d1c8224980..0000000000
--- a/changelog.d/10895.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix type hints to be compatible with an upcoming change to Twisted.
\ No newline at end of file
diff --git a/changelog.d/10902.misc b/changelog.d/10902.misc
deleted file mode 100644
index 2cd79887f6..0000000000
--- a/changelog.d/10902.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update utility code to handle C implementations of frozendict.
\ No newline at end of file
diff --git a/changelog.d/10903.misc b/changelog.d/10903.misc
deleted file mode 100644
index 2716ccb08c..0000000000
--- a/changelog.d/10903.misc
+++ /dev/null
@@ -1 +0,0 @@
-Drop old functionality which maintained database compatibility with Synapse versions before 1.31.
diff --git a/changelog.d/10910.feature b/changelog.d/10910.feature
deleted file mode 100644
index aee139f8b6..0000000000
--- a/changelog.d/10910.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a spam checker callback to allow or deny room joins.
diff --git a/changelog.d/10915.misc b/changelog.d/10915.misc
deleted file mode 100644
index 1ce2910ffa..0000000000
--- a/changelog.d/10915.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean-up configuration helper classes for the `ServerConfig` class.
diff --git a/changelog.d/10916.misc b/changelog.d/10916.misc
deleted file mode 100644
index 586a0b3a96..0000000000
--- a/changelog.d/10916.misc
+++ /dev/null
@@ -1 +0,0 @@
-Use direct references to config flags.
diff --git a/changelog.d/10922.bugfix b/changelog.d/10922.bugfix
deleted file mode 100644
index b7315514e0..0000000000
--- a/changelog.d/10922.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a minor bug in the response to `/_matrix/client/r0/voip/turnServer`. Contributed by @lukaslihotzki.
diff --git a/changelog.d/10924.bugfix b/changelog.d/10924.bugfix
deleted file mode 100644
index c73a51e32f..0000000000
--- a/changelog.d/10924.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a bug where empty `yyyy-mm-dd/` directories would be left behind in the media store's `url_cache_thumbnails/` directory.
diff --git a/changelog.d/10926.misc b/changelog.d/10926.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/10926.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/10927.bugfix b/changelog.d/10927.bugfix
deleted file mode 100644
index fd24288c54..0000000000
--- a/changelog.d/10927.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a bug introduced in Synapse v1.40.0 where the signature checks for room version 8/9 could be applied to earlier room versions in some situations.
diff --git a/changelog.d/10934.misc b/changelog.d/10934.misc
deleted file mode 100644
index 56c640ec9e..0000000000
--- a/changelog.d/10934.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactor various parts of the codebase to use `RoomVersion` objects instead of room version identifier strings.
diff --git a/changelog.d/10935.misc b/changelog.d/10935.misc
deleted file mode 100644
index 80529c04ca..0000000000
--- a/changelog.d/10935.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactor user directory tests in preparation for upcoming changes.
diff --git a/changelog.d/10936.misc b/changelog.d/10936.misc
deleted file mode 100644
index 9d1d6e5b02..0000000000
--- a/changelog.d/10936.misc
+++ /dev/null
@@ -1 +0,0 @@
-Include the event id in the logcontext when handling PDUs received over federation.
diff --git a/changelog.d/10939.misc b/changelog.d/10939.misc
deleted file mode 100644
index a7cecf8a5b..0000000000
--- a/changelog.d/10939.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix logged errors in unit tests.
diff --git a/changelog.d/10940.misc b/changelog.d/10940.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/10940.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/10945.misc b/changelog.d/10945.misc
deleted file mode 100644
index 7cf1f02ad6..0000000000
--- a/changelog.d/10945.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a broken test to ensure that consent configuration works during registration.
diff --git a/changelog.d/10947.bugfix b/changelog.d/10947.bugfix
deleted file mode 100644
index 40c70d3ece..0000000000
--- a/changelog.d/10947.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fixes a long-standing bug wherin deactivated users still count towards the mau limit.
\ No newline at end of file
diff --git a/changelog.d/10954.feature b/changelog.d/10954.feature
deleted file mode 100644
index 94dfa7175c..0000000000
--- a/changelog.d/10954.feature
+++ /dev/null
@@ -1 +0,0 @@
-Include an `update_synapse_database` script in the distribution. Contributed by @Fizzadar at Beeper.
diff --git a/changelog.d/10956.bugfix b/changelog.d/10956.bugfix
deleted file mode 100644
index 13b8e5983b..0000000000
--- a/changelog.d/10956.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug which meant that events received over federation were sometimes incorrectly accepted into the room state.
diff --git a/changelog.d/10958.misc b/changelog.d/10958.misc
deleted file mode 100644
index 409ecc35cb..0000000000
--- a/changelog.d/10958.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add type hints to filtering classes.
diff --git a/changelog.d/10959.misc b/changelog.d/10959.misc
deleted file mode 100644
index 586a0b3a96..0000000000
--- a/changelog.d/10959.misc
+++ /dev/null
@@ -1 +0,0 @@
-Use direct references to config flags.
diff --git a/changelog.d/10960.bugfix b/changelog.d/10960.bugfix
deleted file mode 100644
index b4f1c228ea..0000000000
--- a/changelog.d/10960.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug where rebuilding the user directory wouldn't exclude support and disabled users.
\ No newline at end of file
diff --git a/changelog.d/10961.misc b/changelog.d/10961.misc
deleted file mode 100644
index 0e35813488..0000000000
--- a/changelog.d/10961.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add type-hint to `HomeserverTestcase.setup_test_homeserver`.
\ No newline at end of file
diff --git a/changelog.d/10962.bugfix b/changelog.d/10962.bugfix
deleted file mode 100644
index 9b0760d731..0000000000
--- a/changelog.d/10962.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint rejecting subsequent batches with unknown batch ID error in existing room versions from the room creator.
diff --git a/changelog.d/10963.misc b/changelog.d/10963.misc
deleted file mode 100644
index daf40155de..0000000000
--- a/changelog.d/10963.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix the test utility function `create_room_as` so that `is_public=True` will explicitly set the `visibility` parameter of room creation requests to `public`. Contributed by @AndrewFerr.
diff --git a/changelog.d/10966.misc b/changelog.d/10966.misc
deleted file mode 100644
index 095b9d5689..0000000000
--- a/changelog.d/10966.misc
+++ /dev/null
@@ -1 +0,0 @@
-Make the release script more robust and transparent.
diff --git a/changelog.d/10971.doc b/changelog.d/10971.doc
deleted file mode 100644
index cc6cfe4164..0000000000
--- a/changelog.d/10971.doc
+++ /dev/null
@@ -1 +0,0 @@
-Change wording ("reference homeserver") in Synapse repository documentation. Contributed by @maxkratz.
diff --git a/changelog.d/10973.doc b/changelog.d/10973.doc
deleted file mode 100644
index d7429a9da6..0000000000
--- a/changelog.d/10973.doc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a dead URL in development documentation (SAML) and change wording from "Riot" to "Element". Contributed by @maxkratz.
diff --git a/changelog.d/10974.misc b/changelog.d/10974.misc
deleted file mode 100644
index 8695b378aa..0000000000
--- a/changelog.d/10974.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactor [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` mega function into smaller handler functions.
diff --git a/changelog.d/10981.bugfix b/changelog.d/10981.bugfix
deleted file mode 100644
index d7bf660348..0000000000
--- a/changelog.d/10981.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a bug that could leak local users' per-room nicknames and avatars when the user directory is rebuilt.
\ No newline at end of file
diff --git a/changelog.d/10982.bugfix b/changelog.d/10982.bugfix
deleted file mode 100644
index 5c9e15eeaa..0000000000
--- a/changelog.d/10982.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug where the remainder of a batch of user directory changes would be silently dropped if the server left a room early in the batch.
\ No newline at end of file
diff --git a/changelog.d/10983.misc b/changelog.d/10983.misc
deleted file mode 100644
index 235899d14f..0000000000
--- a/changelog.d/10983.misc
+++ /dev/null
@@ -1 +0,0 @@
-Log stack traces when a missing opentracing span is detected.
diff --git a/changelog.d/10985.misc b/changelog.d/10985.misc
deleted file mode 100644
index 586a0b3a96..0000000000
--- a/changelog.d/10985.misc
+++ /dev/null
@@ -1 +0,0 @@
-Use direct references to config flags.
diff --git a/changelog.d/10986.misc b/changelog.d/10986.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/10986.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/10987.misc b/changelog.d/10987.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/10987.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/10988.misc b/changelog.d/10988.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/10988.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/10990.doc b/changelog.d/10990.doc
deleted file mode 100644
index 51290d6200..0000000000
--- a/changelog.d/10990.doc
+++ /dev/null
@@ -1 +0,0 @@
-Add additional content to the Welcome and Overview page of the documentation.
diff --git a/changelog.d/10991.doc b/changelog.d/10991.doc
deleted file mode 100644
index 2f9bb24ca7..0000000000
--- a/changelog.d/10991.doc
+++ /dev/null
@@ -1 +0,0 @@
-Update links to MSCs in documentation. Contributed by @dklimpel.
\ No newline at end of file
diff --git a/changelog.d/10992.misc b/changelog.d/10992.misc
deleted file mode 100644
index 60432a559c..0000000000
--- a/changelog.d/10992.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update GHA config to run tests against Python 3.10 and PostgreSQL 14.
diff --git a/changelog.d/10993.misc b/changelog.d/10993.misc
deleted file mode 100644
index 23c73dbac5..0000000000
--- a/changelog.d/10993.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug where `ReadWriteLock`s could drop logging contexts on exit.
diff --git a/changelog.d/10994.misc b/changelog.d/10994.misc
deleted file mode 100644
index 0a8538b01e..0000000000
--- a/changelog.d/10994.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add a `CODEOWNERS` file to automatically request reviews from the `@matrix-org/synapse-core` team on new pull requests.
diff --git a/changelog.d/10995.bugfix b/changelog.d/10995.bugfix
deleted file mode 100644
index 3eef96f3db..0000000000
--- a/changelog.d/10995.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Correct a bugfix introduced in Synapse v1.44.0 that wouldn't catch every error of the connection breaks before a response could be written to it.
diff --git a/changelog.d/11002.bugfix b/changelog.d/11002.bugfix
deleted file mode 100644
index cf894a6314..0000000000
--- a/changelog.d/11002.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug where local users' per-room nicknames/avatars were visible to anyone who could see you in the user_directory.
diff --git a/changelog.d/11003.bugfix b/changelog.d/11003.bugfix
deleted file mode 100644
index 0786f1b886..0000000000
--- a/changelog.d/11003.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a long-standing bug where a user's per-room nickname/avatar would overwrite their profile in the user directory when a room was made public.
\ No newline at end of file
diff --git a/changelog.d/11004.misc b/changelog.d/11004.misc
deleted file mode 100644
index 821033710a..0000000000
--- a/changelog.d/11004.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add further type hints to `synapse.state`.
\ No newline at end of file
diff --git a/changelog.d/11005.misc b/changelog.d/11005.misc
deleted file mode 100644
index a893591971..0000000000
--- a/changelog.d/11005.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove the deprecated `BaseHandler` object.
diff --git a/changelog.d/11006.misc b/changelog.d/11006.misc
deleted file mode 100644
index 7b4abae76a..0000000000
--- a/changelog.d/11006.misc
+++ /dev/null
@@ -1 +0,0 @@
-Bump mypy version for CI to 0.910, and pull in new type stubs for dependencies.
\ No newline at end of file
diff --git a/changelog.d/11010.misc b/changelog.d/11010.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/11010.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/11011.misc b/changelog.d/11011.misc
deleted file mode 100644
index 9a765435db..0000000000
--- a/changelog.d/11011.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up some of the federation event authentication code for clarity.
diff --git a/changelog.d/11017.misc b/changelog.d/11017.misc
deleted file mode 100644
index f05530ac94..0000000000
--- a/changelog.d/11017.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix CI to run the unit tests without optional deps.
diff --git a/changelog.d/11021.misc b/changelog.d/11021.misc
deleted file mode 100644
index 8ac1bfcf22..0000000000
--- a/changelog.d/11021.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add additional type hints to `synapse.server_notices`.
\ No newline at end of file
diff --git a/changelog.d/11023.misc b/changelog.d/11023.misc
deleted file mode 100644
index ecc0467529..0000000000
--- a/changelog.d/11023.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add additional type hints for `synapse.push`.
\ No newline at end of file
diff --git a/changelog.d/11028.feature b/changelog.d/11028.feature
deleted file mode 100644
index 48798356b7..0000000000
--- a/changelog.d/11028.feature
+++ /dev/null
@@ -1 +0,0 @@
-Include exception information in JSON logging output. Contributed by @Fizzadar at Beeper.
diff --git a/changelog.d/11034.misc b/changelog.d/11034.misc
deleted file mode 100644
index b15fd66ac3..0000000000
--- a/changelog.d/11034.misc
+++ /dev/null
@@ -1 +0,0 @@
-When installing the optional developer dependencies, also include the dependencies needed for type-checking and unit testing.
diff --git a/changelog.d/11019.misc b/changelog.d/11036.misc
index aae5ee62b2..aae5ee62b2 100644
--- a/changelog.d/11019.misc
+++ b/changelog.d/11036.misc
diff --git a/changelog.d/11042.bugfix b/changelog.d/11042.bugfix
deleted file mode 100644
index 536c47417d..0000000000
--- a/changelog.d/11042.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Work around a regression, introduced in Synapse 1.39.0, that caused `SynapseError`s raised by the experimental third-party rules module callback `check_event_allowed` to be ignored.
diff --git a/changelog.d/11043.misc b/changelog.d/11043.misc
deleted file mode 100644
index c5f127bb46..0000000000
--- a/changelog.d/11043.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove unnecessary list comprehension from `synapse_port_db` to satisfy code style requirements.
diff --git a/changelog.d/11045.bugfix b/changelog.d/11045.bugfix
new file mode 100644
index 0000000000..d712dc946a
--- /dev/null
+++ b/changelog.d/11045.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug when using multiple event persister workers where events were not correctly sent down `/sync` due to a race.
diff --git a/changelog.d/9655.feature b/changelog.d/9655.feature
deleted file mode 100644
index 70cac230d8..0000000000
--- a/changelog.d/9655.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069) support to `/account/whoami`.
\ No newline at end of file
diff --git a/debian/changelog b/debian/changelog
index 8e80c78ee7..0d5db739e7 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,9 +1,12 @@
-matrix-synapse-py3 (1.44.0~rc2+nmu1) UNRELEASED; urgency=medium
+matrix-synapse-py3 (1.45.0~rc1) stable; urgency=medium
 
   [ Nick @ Beeper ]
   * Include an `update_synapse_database` script in the distribution.
 
- -- root <root@f7b8a71098d3>  Mon, 04 Oct 2021 13:29:26 +0000
+  [ Synapse Packaging team ]
+  * New synapse release 1.45.0~rc1.
+
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 12 Oct 2021 10:46:27 +0100
 
 matrix-synapse-py3 (1.44.0) stable; urgency=medium
 
diff --git a/docs/upgrade.md b/docs/upgrade.md
index a8221372df..18ecb2678e 100644
--- a/docs/upgrade.md
+++ b/docs/upgrade.md
@@ -85,6 +85,15 @@ process, for example:
     dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
     ```
 
+# Upgrading to v1.45.0
+
+## Changes required to media storage provider modules when reading from the Synapse configuration object
+
+Media storage provider modules that read from the Synapse configuration object (i.e. that
+read the value of `hs.config.[...]`) now need to specify the configuration section they're
+reading from. This means that if a module reads the value of e.g. `hs.config.media_store_path`,
+it needs to replace it with `hs.config.media.media_store_path`.
+
 # Upgrading to v1.44.0
 
 ## The URL preview cache is no longer mirrored to storage providers
diff --git a/synapse/__init__.py b/synapse/__init__.py
index b8979c365e..6b109ccffa 100644
--- a/synapse/__init__.py
+++ b/synapse/__init__.py
@@ -47,7 +47,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.44.0"
+__version__ = "1.45.0rc1"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when
diff --git a/synapse/storage/util/id_generators.py b/synapse/storage/util/id_generators.py
index 852bd79fee..670811611f 100644
--- a/synapse/storage/util/id_generators.py
+++ b/synapse/storage/util/id_generators.py
@@ -36,7 +36,7 @@ from typing import (
 )
 
 import attr
-from sortedcontainers import SortedSet
+from sortedcontainers import SortedList, SortedSet
 
 from synapse.metrics.background_process_metrics import run_as_background_process
 from synapse.storage.database import (
@@ -265,6 +265,15 @@ class MultiWriterIdGenerator:
         # should be less than the minimum of this set (if not empty).
         self._unfinished_ids: SortedSet[int] = SortedSet()
 
+        # We also need to track when we've requested some new stream IDs but
+        # they haven't yet been added to the `_unfinished_ids` set. Every time
+        # we request a new stream ID we add the current max stream ID to the
+        # list, and remove it once we've added the newly allocated IDs to the
+        # `_unfinished_ids` set. This means that we *may* be allocated stream
+        # IDs above those in the list, and so we can't advance the local current
+        # position beyond the minimum stream ID in this list.
+        self._in_flight_fetches: SortedList[int] = SortedList()
+
         # Set of local IDs that we've processed that are larger than the current
         # position, due to there being smaller unpersisted IDs.
         self._finished_ids: Set[int] = set()
@@ -290,6 +299,9 @@ class MultiWriterIdGenerator:
         )
         self._known_persisted_positions: List[int] = []
 
+        # The maximum stream ID that we have seen been allocated across any writer.
+        self._max_seen_allocated_stream_id = 1
+
         self._sequence_gen = PostgresSequenceGenerator(sequence_name)
 
         # We check that the table and sequence haven't diverged.
@@ -305,6 +317,10 @@ class MultiWriterIdGenerator:
         # This goes and fills out the above state from the database.
         self._load_current_ids(db_conn, tables)
 
+        self._max_seen_allocated_stream_id = max(
+            self._current_positions.values(), default=1
+        )
+
     def _load_current_ids(
         self,
         db_conn: LoggingDatabaseConnection,
@@ -411,10 +427,32 @@ class MultiWriterIdGenerator:
         cur.close()
 
     def _load_next_id_txn(self, txn: Cursor) -> int:
-        return self._sequence_gen.get_next_id_txn(txn)
+        stream_ids = self._load_next_mult_id_txn(txn, 1)
+        return stream_ids[0]
 
     def _load_next_mult_id_txn(self, txn: Cursor, n: int) -> List[int]:
-        return self._sequence_gen.get_next_mult_txn(txn, n)
+        # We need to track that we've requested some more stream IDs, and what
+        # the current max allocated stream ID is. This is to prevent a race
+        # where we've been allocated stream IDs but they have not yet been added
+        # to the `_unfinished_ids` set, allowing the current position to advance
+        # past them.
+        with self._lock:
+            current_max = self._max_seen_allocated_stream_id
+            self._in_flight_fetches.add(current_max)
+
+        try:
+            stream_ids = self._sequence_gen.get_next_mult_txn(txn, n)
+
+            with self._lock:
+                self._unfinished_ids.update(stream_ids)
+                self._max_seen_allocated_stream_id = max(
+                    self._max_seen_allocated_stream_id, self._unfinished_ids[-1]
+                )
+        finally:
+            with self._lock:
+                self._in_flight_fetches.remove(current_max)
+
+        return stream_ids
 
     def get_next(self) -> AsyncContextManager[int]:
         """
@@ -463,9 +501,6 @@ class MultiWriterIdGenerator:
 
         next_id = self._load_next_id_txn(txn)
 
-        with self._lock:
-            self._unfinished_ids.add(next_id)
-
         txn.call_after(self._mark_id_as_finished, next_id)
         txn.call_on_exception(self._mark_id_as_finished, next_id)
 
@@ -497,15 +532,27 @@ class MultiWriterIdGenerator:
 
             new_cur: Optional[int] = None
 
-            if self._unfinished_ids:
+            if self._unfinished_ids or self._in_flight_fetches:
                 # If there are unfinished IDs then the new position will be the
-                # largest finished ID less than the minimum unfinished ID.
+                # largest finished ID strictly less than the minimum unfinished
+                # ID.
+
+                # The minimum unfinished ID needs to take account of both
+                # `_unfinished_ids` and `_in_flight_fetches`.
+                if self._unfinished_ids and self._in_flight_fetches:
+                    # `_in_flight_fetches` stores the maximum safe stream ID, so
+                    # we add one to make it equivalent to the minimum unsafe ID.
+                    min_unfinished = min(
+                        self._unfinished_ids[0], self._in_flight_fetches[0] + 1
+                    )
+                elif self._in_flight_fetches:
+                    min_unfinished = self._in_flight_fetches[0] + 1
+                else:
+                    min_unfinished = self._unfinished_ids[0]
 
                 finished = set()
-
-                min_unfinshed = self._unfinished_ids[0]
                 for s in self._finished_ids:
-                    if s < min_unfinshed:
+                    if s < min_unfinished:
                         if new_cur is None or new_cur < s:
                             new_cur = s
                     else:
@@ -575,6 +622,10 @@ class MultiWriterIdGenerator:
                 new_id, self._current_positions.get(instance_name, 0)
             )
 
+            self._max_seen_allocated_stream_id = max(
+                self._max_seen_allocated_stream_id, new_id
+            )
+
             self._add_persisted_position(new_id)
 
     def get_persisted_upto_position(self) -> int:
@@ -605,7 +656,11 @@ class MultiWriterIdGenerator:
         # to report a recent position when asked, rather than a potentially old
         # one (if this instance hasn't written anything for a while).
         our_current_position = self._current_positions.get(self._instance_name)
-        if our_current_position and not self._unfinished_ids:
+        if (
+            our_current_position
+            and not self._unfinished_ids
+            and not self._in_flight_fetches
+        ):
             self._current_positions[self._instance_name] = max(
                 our_current_position, new_id
             )
@@ -697,9 +752,6 @@ class _MultiWriterCtxManager:
             db_autocommit=True,
         )
 
-        with self.id_gen._lock:
-            self.id_gen._unfinished_ids.update(self.stream_ids)
-
         if self.multiple_ids is None:
             return self.stream_ids[0] * self.id_gen._return_factor
         else:
diff --git a/tests/config/test_cache.py b/tests/config/test_cache.py
index 79d417568d..4bb82e810e 100644
--- a/tests/config/test_cache.py
+++ b/tests/config/test_cache.py
@@ -12,25 +12,23 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from unittest.mock import patch
-
 from synapse.config.cache import CacheConfig, add_resizable_cache
 from synapse.util.caches.lrucache import LruCache
 
 from tests.unittest import TestCase
 
 
-# Patch the global _CACHES so that each test runs against its own state.
-@patch("synapse.config.cache._CACHES", new_callable=dict)
 class CacheConfigTests(TestCase):
     def setUp(self):
-        # Reset caches before each test
+        # Reset caches before each test since there's global state involved.
         self.config = CacheConfig()
+        self.config.reset()
 
     def tearDown(self):
+        # Also reset the caches after each test to leave state pristine.
         self.config.reset()
 
-    def test_individual_caches_from_environ(self, _caches):
+    def test_individual_caches_from_environ(self):
         """
         Individual cache factors will be loaded from the environment.
         """
@@ -43,7 +41,7 @@ class CacheConfigTests(TestCase):
 
         self.assertEqual(dict(self.config.cache_factors), {"something_or_other": 2.0})
 
-    def test_config_overrides_environ(self, _caches):
+    def test_config_overrides_environ(self):
         """
         Individual cache factors defined in the environment will take precedence
         over those in the config.
@@ -60,7 +58,7 @@ class CacheConfigTests(TestCase):
             {"foo": 1.0, "bar": 3.0, "something_or_other": 2.0},
         )
 
-    def test_individual_instantiated_before_config_load(self, _caches):
+    def test_individual_instantiated_before_config_load(self):
         """
         If a cache is instantiated before the config is read, it will be given
         the default cache size in the interim, and then resized once the config
@@ -76,7 +74,7 @@ class CacheConfigTests(TestCase):
 
         self.assertEqual(cache.max_size, 300)
 
-    def test_individual_instantiated_after_config_load(self, _caches):
+    def test_individual_instantiated_after_config_load(self):
         """
         If a cache is instantiated after the config is read, it will be
         immediately resized to the correct size given the per_cache_factor if
@@ -89,7 +87,7 @@ class CacheConfigTests(TestCase):
         add_resizable_cache("foo", cache_resize_callback=cache.set_cache_factor)
         self.assertEqual(cache.max_size, 200)
 
-    def test_global_instantiated_before_config_load(self, _caches):
+    def test_global_instantiated_before_config_load(self):
         """
         If a cache is instantiated before the config is read, it will be given
         the default cache size in the interim, and then resized to the new
@@ -104,7 +102,7 @@ class CacheConfigTests(TestCase):
 
         self.assertEqual(cache.max_size, 400)
 
-    def test_global_instantiated_after_config_load(self, _caches):
+    def test_global_instantiated_after_config_load(self):
         """
         If a cache is instantiated after the config is read, it will be
         immediately resized to the correct size given the global factor if there
@@ -117,7 +115,7 @@ class CacheConfigTests(TestCase):
         add_resizable_cache("foo", cache_resize_callback=cache.set_cache_factor)
         self.assertEqual(cache.max_size, 150)
 
-    def test_cache_with_asterisk_in_name(self, _caches):
+    def test_cache_with_asterisk_in_name(self):
         """Some caches have asterisks in their name, test that they are set correctly."""
 
         config = {
@@ -143,7 +141,7 @@ class CacheConfigTests(TestCase):
         add_resizable_cache("*cache_c*", cache_resize_callback=cache_c.set_cache_factor)
         self.assertEqual(cache_c.max_size, 200)
 
-    def test_apply_cache_factor_from_config(self, _caches):
+    def test_apply_cache_factor_from_config(self):
         """Caches can disable applying cache factor updates, mainly used by
         event cache size.
         """