Merge branch 'develop' into cross-signing_hidden

81 files changed, 30 insertions, 54 deletions

diff --git a/changelog.d/5397.doc b/changelog.d/5397.doc
deleted file mode 100644
index c2b500b482..0000000000
--- a/changelog.d/5397.doc
+++ /dev/null
@@ -1 +0,0 @@
-Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks!
diff --git a/changelog.d/5544.feature b/changelog.d/5544.feature
deleted file mode 100644
index 7d3459129d..0000000000
--- a/changelog.d/5544.feature
+++ /dev/null
@@ -1,2 +0,0 @@
-Add support for opentracing.
-
diff --git a/changelog.d/5589.feature b/changelog.d/5589.feature
deleted file mode 100644
index a87e669dd4..0000000000
--- a/changelog.d/5589.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add ability to pull all locally stored events out of synapse that a particular user can see.
diff --git a/changelog.d/5597.feature b/changelog.d/5597.feature
deleted file mode 100644
index 6f92748885..0000000000
--- a/changelog.d/5597.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance.
diff --git a/changelog.d/5606.misc b/changelog.d/5606.misc
deleted file mode 100644
index bb3c028167..0000000000
--- a/changelog.d/5606.misc
+++ /dev/null
@@ -1 +0,0 @@
-Move logging code out of `synapse.util` and into `synapse.logging`.
diff --git a/changelog.d/5609.bugfix b/changelog.d/5609.bugfix
deleted file mode 100644
index 534ee22a1b..0000000000
--- a/changelog.d/5609.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix 'utime went backwards' errors on daemonization.
diff --git a/changelog.d/5611.misc b/changelog.d/5611.misc
deleted file mode 100644
index a2d1695139..0000000000
--- a/changelog.d/5611.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add a blacklist file to the repo to blacklist certain sytests from failing CI.
diff --git a/changelog.d/5613.feature b/changelog.d/5613.feature
deleted file mode 100644
index 4b7bb2745c..0000000000
--- a/changelog.d/5613.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add `sender` and `origin_server_ts` fields to `m.replace`.
diff --git a/changelog.d/5616.misc b/changelog.d/5616.misc
deleted file mode 100644
index 9f94be6778..0000000000
--- a/changelog.d/5616.misc
+++ /dev/null
@@ -1 +0,0 @@
-Make runtime errors surrounding password reset emails much clearer.
diff --git a/changelog.d/5617.misc b/changelog.d/5617.misc
deleted file mode 100644
index bb3c028167..0000000000
--- a/changelog.d/5617.misc
+++ /dev/null
@@ -1 +0,0 @@
-Move logging code out of `synapse.util` and into `synapse.logging`.
diff --git a/changelog.d/5619.docker b/changelog.d/5619.docker
deleted file mode 100644
index b69e5cc57c..0000000000
--- a/changelog.d/5619.docker
+++ /dev/null
@@ -1 +0,0 @@
-Base Docker image on a newer Alpine Linux version (3.8 -> 3.10).
diff --git a/changelog.d/5620.docker b/changelog.d/5620.docker
deleted file mode 100644
index cbb5a75d6a..0000000000
--- a/changelog.d/5620.docker
+++ /dev/null
@@ -1 +0,0 @@
-Add missing space in default logging file format generated by the Docker image.
diff --git a/changelog.d/5621.bugfix b/changelog.d/5621.bugfix
deleted file mode 100644
index f1a2851f45..0000000000
--- a/changelog.d/5621.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Various minor fixes to the federation request rate limiter.
diff --git a/changelog.d/5622.misc b/changelog.d/5622.misc
deleted file mode 100644
index 9f0a87311c..0000000000
--- a/changelog.d/5622.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove dead code for persiting outgoing federation transactions.
diff --git a/changelog.d/5623.feature b/changelog.d/5623.feature
deleted file mode 100644
index b73080e88d..0000000000
--- a/changelog.d/5623.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add default push rule to ignore reactions.
diff --git a/changelog.d/5625.removal b/changelog.d/5625.removal
deleted file mode 100644
index d33a778d69..0000000000
--- a/changelog.d/5625.removal
+++ /dev/null
@@ -1 +0,0 @@
-Remove support for the `invite_3pid_guest` configuration setting.
diff --git a/changelog.d/5626.feature b/changelog.d/5626.feature
deleted file mode 100644
index 5ef793b943..0000000000
--- a/changelog.d/5626.feature
+++ /dev/null
@@ -1 +0,0 @@
-Include the original event when asking for its relations.
diff --git a/changelog.d/5627.misc b/changelog.d/5627.misc
deleted file mode 100644
index 730721b5ef..0000000000
--- a/changelog.d/5627.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI.
diff --git a/changelog.d/5628.misc b/changelog.d/5628.misc
deleted file mode 100644
index fec8446793..0000000000
--- a/changelog.d/5628.misc
+++ /dev/null
@@ -1 +0,0 @@
-Move RegistrationHandler.get_or_create_user to test code.
diff --git a/changelog.d/5629.bugfix b/changelog.d/5629.bugfix
deleted file mode 100644
index 672eabad40..0000000000
--- a/changelog.d/5629.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Forbid viewing relations on an event once it has been redacted.
diff --git a/changelog.d/5630.misc b/changelog.d/5630.misc
deleted file mode 100644
index f112d873eb..0000000000
--- a/changelog.d/5630.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add some more common python virtual-environment paths to the black exclusion list.
diff --git a/changelog.d/5636.misc b/changelog.d/5636.misc
deleted file mode 100644
index 3add990283..0000000000
--- a/changelog.d/5636.misc
+++ /dev/null
@@ -1 +0,0 @@
-Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details.
\ No newline at end of file
diff --git a/changelog.d/5637.misc b/changelog.d/5637.misc
deleted file mode 100644
index f18d6197e5..0000000000
--- a/changelog.d/5637.misc
+++ /dev/null
@@ -1 +0,0 @@
-Unblacklist some user_directory sytests.
diff --git a/changelog.d/5638.bugfix b/changelog.d/5638.bugfix
deleted file mode 100644
index 66781ad9e6..0000000000
--- a/changelog.d/5638.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format.
diff --git a/changelog.d/5639.misc b/changelog.d/5639.misc
deleted file mode 100644
index 413b13128c..0000000000
--- a/changelog.d/5639.misc
+++ /dev/null
@@ -1 +0,0 @@
-Factor out some redundant code in the login implementation.
diff --git a/changelog.d/5640.misc b/changelog.d/5640.misc
deleted file mode 100644
index 7d69a1b3b6..0000000000
--- a/changelog.d/5640.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update ModuleApi to avoid register(generate_token=True).
diff --git a/changelog.d/5641.misc b/changelog.d/5641.misc
deleted file mode 100644
index 1899bc963d..0000000000
--- a/changelog.d/5641.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove access-token support from RegistrationHandler.register, and rename it.
diff --git a/changelog.d/5642.misc b/changelog.d/5642.misc
deleted file mode 100644
index e7f8e214a4..0000000000
--- a/changelog.d/5642.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove access-token support from `RegistrationStore.register`, and rename it.
diff --git a/changelog.d/5643.misc b/changelog.d/5643.misc
deleted file mode 100644
index 2b2316469e..0000000000
--- a/changelog.d/5643.misc
+++ /dev/null
@@ -1 +0,0 @@
-Improve logging for auto-join when a new user is created.
diff --git a/changelog.d/5644.bugfix b/changelog.d/5644.bugfix
deleted file mode 100644
index f6302fd08d..0000000000
--- a/changelog.d/5644.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix newly-registered users not being able to lookup their own profile without joining a room.
diff --git a/changelog.d/5645.misc b/changelog.d/5645.misc
deleted file mode 100644
index 4fa9699e4f..0000000000
--- a/changelog.d/5645.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure.
\ No newline at end of file
diff --git a/changelog.d/5651.doc b/changelog.d/5651.doc
deleted file mode 100644
index e2d5a8dc8a..0000000000
--- a/changelog.d/5651.doc
+++ /dev/null
@@ -1 +0,0 @@
---no-pep517 should be --no-use-pep517 in the documentation to setup the development environment.
diff --git a/changelog.d/5654.bugfix b/changelog.d/5654.bugfix
deleted file mode 100644
index 5f76b041cd..0000000000
--- a/changelog.d/5654.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`.
\ No newline at end of file
diff --git a/changelog.d/5655.misc b/changelog.d/5655.misc
deleted file mode 100644
index acab6aee92..0000000000
--- a/changelog.d/5655.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a small typo in a code comment.
\ No newline at end of file
diff --git a/changelog.d/5656.misc b/changelog.d/5656.misc
deleted file mode 100644
index a8de20a7d0..0000000000
--- a/changelog.d/5656.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clean up exception handling around client access tokens.
diff --git a/changelog.d/5657.misc b/changelog.d/5657.misc
deleted file mode 100644
index bdec9ae4c0..0000000000
--- a/changelog.d/5657.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add a mechanism for per-test homeserver configuration in the unit tests.
diff --git a/changelog.d/5658.bugfix b/changelog.d/5658.bugfix
deleted file mode 100644
index f6ae906a9a..0000000000
--- a/changelog.d/5658.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated.
diff --git a/changelog.d/5659.misc b/changelog.d/5659.misc
deleted file mode 100644
index 686001295c..0000000000
--- a/changelog.d/5659.misc
+++ /dev/null
@@ -1 +0,0 @@
-Inline issue_access_token.
diff --git a/changelog.d/5660.feature b/changelog.d/5660.feature
deleted file mode 100644
index 82889fdaf1..0000000000
--- a/changelog.d/5660.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement `session_lifetime` configuration option, after which access tokens will expire.
diff --git a/changelog.d/5661.doc b/changelog.d/5661.doc
deleted file mode 100644
index c70e62014e..0000000000
--- a/changelog.d/5661.doc
+++ /dev/null
@@ -1 +0,0 @@
-Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks!
diff --git a/changelog.d/5664.misc b/changelog.d/5664.misc
deleted file mode 100644
index 0ca7a0fbd0..0000000000
--- a/changelog.d/5664.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update the sytest BuildKite configuration to checkout Synapse in `/src`.
diff --git a/changelog.d/5673.misc b/changelog.d/5673.misc
deleted file mode 100644
index 1942256358..0000000000
--- a/changelog.d/5673.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add a `docker` type to the towncrier configuration.
diff --git a/changelog.d/5674.feature b/changelog.d/5674.feature
deleted file mode 100644
index 04bdfa4ad5..0000000000
--- a/changelog.d/5674.feature
+++ /dev/null
@@ -1 +0,0 @@
-Return "This account has been deactivated" when a deactivated user tries to login.
diff --git a/changelog.d/5675.doc b/changelog.d/5675.doc
deleted file mode 100644
index 4cd4d0be1a..0000000000
--- a/changelog.d/5675.doc
+++ /dev/null
@@ -1 +0,0 @@
-Minor tweaks to postgres documentation.
diff --git a/changelog.d/5689.misc b/changelog.d/5689.misc
deleted file mode 100644
index 8aa3e3f6a2..0000000000
--- a/changelog.d/5689.misc
+++ /dev/null
@@ -1 +0,0 @@
-Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces.
\ No newline at end of file
diff --git a/changelog.d/5693.bugfix b/changelog.d/5693.bugfix
new file mode 100644
index 0000000000..d6f4e590ae
--- /dev/null
+++ b/changelog.d/5693.bugfix
@@ -0,0 +1 @@
+Fix UISIs during homeserver outage.
diff --git a/changelog.d/5694.misc b/changelog.d/5694.misc
new file mode 100644
index 0000000000..3b12dcc849
--- /dev/null
+++ b/changelog.d/5694.misc
@@ -0,0 +1 @@
+Make Jaeger fully configurable.
diff --git a/changelog.d/5699.bugfix b/changelog.d/5699.bugfix
deleted file mode 100644
index 30d5e67f67..0000000000
--- a/changelog.d/5699.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix some problems with authenticating redactions in recent room versions.
\ No newline at end of file
diff --git a/changelog.d/5700.bugfix b/changelog.d/5700.bugfix
deleted file mode 100644
index 51bce8d441..0000000000
--- a/changelog.d/5700.bugfix
+++ /dev/null
@@ -1,2 +0,0 @@
-Fix some problems with authenticating redactions in recent room versions.
-
diff --git a/changelog.d/5701.bugfix b/changelog.d/5701.bugfix
deleted file mode 100644
index fd2866e16a..0000000000
--- a/changelog.d/5701.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Ignore redactions of m.room.create events.
diff --git a/changelog.d/5707.bugfix b/changelog.d/5707.bugfix
deleted file mode 100644
index aa3046c5e1..0000000000
--- a/changelog.d/5707.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix some problems with authenticating redactions in recent room versions.
diff --git a/changelog.d/5712.feature b/changelog.d/5712.feature
deleted file mode 100644
index 7d3459129d..0000000000
--- a/changelog.d/5712.feature
+++ /dev/null
@@ -1,2 +0,0 @@
-Add support for opentracing.
-
diff --git a/changelog.d/5714.feature b/changelog.d/5714.feature
deleted file mode 100644
index 2fd32e5e38..0000000000
--- a/changelog.d/5714.feature
+++ /dev/null
@@ -1 +0,0 @@
-Enable aggregations support by default
diff --git a/changelog.d/5722.misc b/changelog.d/5722.misc
new file mode 100644
index 0000000000..f2d236188d
--- /dev/null
+++ b/changelog.d/5722.misc
@@ -0,0 +1 @@
+Add a set of opentracing utils.
diff --git a/changelog.d/5724.bugfix b/changelog.d/5724.bugfix
new file mode 100644
index 0000000000..1b3683daf6
--- /dev/null
+++ b/changelog.d/5724.bugfix
@@ -0,0 +1 @@
+Fix stack overflow in server key lookup code.
\ No newline at end of file
diff --git a/changelog.d/5725.bugfix b/changelog.d/5725.bugfix
new file mode 100644
index 0000000000..73ef419727
--- /dev/null
+++ b/changelog.d/5725.bugfix
@@ -0,0 +1 @@
+start.sh no longer uses deprecated cli option.
diff --git a/changelog.d/5729.removal b/changelog.d/5729.removal
new file mode 100644
index 0000000000..3af5198e6b
--- /dev/null
+++ b/changelog.d/5729.removal
@@ -0,0 +1 @@
+ Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration.
diff --git a/changelog.d/5730.misc b/changelog.d/5730.misc
new file mode 100644
index 0000000000..a99677f5e7
--- /dev/null
+++ b/changelog.d/5730.misc
@@ -0,0 +1 @@
+Cache result of get_version_string to reduce overhead of `/version` federation requests.
diff --git a/changelog.d/5731.misc b/changelog.d/5731.misc
new file mode 100644
index 0000000000..dffae5d874
--- /dev/null
+++ b/changelog.d/5731.misc
@@ -0,0 +1 @@
+Return 'user_type' in admin API user endpoints results.
diff --git a/changelog.d/5732.feature b/changelog.d/5732.feature
new file mode 100644
index 0000000000..9021864350
--- /dev/null
+++ b/changelog.d/5732.feature
@@ -0,0 +1 @@
+Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify.
diff --git a/changelog.d/5733.misc b/changelog.d/5733.misc
new file mode 100644
index 0000000000..a2a8c26383
--- /dev/null
+++ b/changelog.d/5733.misc
@@ -0,0 +1 @@
+Don't package the sytest test blacklist file.
diff --git a/changelog.d/5736.misc b/changelog.d/5736.misc
new file mode 100644
index 0000000000..5713b8b32d
--- /dev/null
+++ b/changelog.d/5736.misc
@@ -0,0 +1 @@
+Replace uses of returnValue with plain return, as returnValue is not needed on Python 3.
diff --git a/changelog.d/5738.misc b/changelog.d/5738.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5738.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5740.misc b/changelog.d/5740.misc
new file mode 100644
index 0000000000..97a476bef5
--- /dev/null
+++ b/changelog.d/5740.misc
@@ -0,0 +1 @@
+Blacklist some flakey tests in worker mode.
diff --git a/changelog.d/5743.bugfix b/changelog.d/5743.bugfix
new file mode 100644
index 0000000000..65728ff079
--- /dev/null
+++ b/changelog.d/5743.bugfix
@@ -0,0 +1 @@
+Log when we receive an event receipt from an unexpected origin.
diff --git a/changelog.d/5746.misc b/changelog.d/5746.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5746.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5749.misc b/changelog.d/5749.misc
new file mode 100644
index 0000000000..48dd61f461
--- /dev/null
+++ b/changelog.d/5749.misc
@@ -0,0 +1 @@
+Fix some error cases in the caching layer.
diff --git a/changelog.d/5750.misc b/changelog.d/5750.misc
new file mode 100644
index 0000000000..6beaa460a5
--- /dev/null
+++ b/changelog.d/5750.misc
@@ -0,0 +1 @@
+Add a prometheus metric for pending cache lookups.
\ No newline at end of file
diff --git a/changelog.d/5752.misc b/changelog.d/5752.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5752.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5753.misc b/changelog.d/5753.misc
new file mode 100644
index 0000000000..22bba9ce3c
--- /dev/null
+++ b/changelog.d/5753.misc
@@ -0,0 +1 @@
+Stop trying to fetch events with event_id=None.
diff --git a/changelog.d/5768.misc b/changelog.d/5768.misc
new file mode 100644
index 0000000000..7a9c88b4c2
--- /dev/null
+++ b/changelog.d/5768.misc
@@ -0,0 +1 @@
+Convert RedactionTestCase to modern test style.
diff --git a/changelog.d/5770.misc b/changelog.d/5770.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5770.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5774.misc b/changelog.d/5774.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5774.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5775.bugfix b/changelog.d/5775.bugfix
new file mode 100644
index 0000000000..b124897d80
--- /dev/null
+++ b/changelog.d/5775.bugfix
@@ -0,0 +1 @@
+Fix debian packaging scripts to correctly build sid packages.
diff --git a/changelog.d/5780.misc b/changelog.d/5780.misc
new file mode 100644
index 0000000000..b7eb56e625
--- /dev/null
+++ b/changelog.d/5780.misc
@@ -0,0 +1 @@
+Allow looping calls to be given arguments.
diff --git a/changelog.d/5782.removal b/changelog.d/5782.removal
new file mode 100644
index 0000000000..658bf923ab
--- /dev/null
+++ b/changelog.d/5782.removal
@@ -0,0 +1 @@
+Remove non-functional 'expire_access_token' setting.
diff --git a/changelog.d/5783.feature b/changelog.d/5783.feature
new file mode 100644
index 0000000000..18f5a3cb28
--- /dev/null
+++ b/changelog.d/5783.feature
@@ -0,0 +1 @@
+Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers.
diff --git a/changelog.d/5785.misc b/changelog.d/5785.misc
new file mode 100644
index 0000000000..0691222c42
--- /dev/null
+++ b/changelog.d/5785.misc
@@ -0,0 +1 @@
+Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO.
diff --git a/changelog.d/5789.bugfix b/changelog.d/5789.bugfix
new file mode 100644
index 0000000000..d6f4e590ae
--- /dev/null
+++ b/changelog.d/5789.bugfix
@@ -0,0 +1 @@
+Fix UISIs during homeserver outage.
diff --git a/changelog.d/5792.misc b/changelog.d/5792.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5792.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.
diff --git a/changelog.d/5793.misc b/changelog.d/5793.misc
new file mode 100644
index 0000000000..5e15dfd5fa
--- /dev/null
+++ b/changelog.d/5793.misc
@@ -0,0 +1 @@
+Reduce database IO usage by optimising queries for current membership.