Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable

90 files changed, 151 insertions, 84 deletions

diff --git a/CHANGES.md b/CHANGES.md
index 6fdd31daab..e08b8771b8 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,3 +1,86 @@
+Synapse 0.99.0rc2 (2019-01-30)
+==============================
+
+Bugfixes
+--------
+
+- Fix bug when rejecting remote invites. ([\#4527](https://github.com/matrix-org/synapse/issues/4527))
+- Fix incorrect rendering of server capabilities. ([81b7e7eed](https://github.com/matrix-org/synapse/commit/81b7e7eed323f55d6550e7a270a9dc2c4c7b0fe0))
+
+Improved Documentation
+----------------------
+
+- Add documentation on enabling ACME support when upgrading to v0.99. ([\#4528](https://github.com/matrix-org/synapse/issues/4528))
+
+
+Synapse 0.99.0rc1 (2019-01-30)
+==============================
+
+Synapse v0.99.x is a precursor to the upcoming Synapse v1.0 release. It contains foundational changes to room architecture and the federation security model necessary to support the upcoming r0 release of the Server to Server API.
+
+Features
+--------
+
+- Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored. ([\#4229](https://github.com/matrix-org/synapse/issues/4229))
+- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt). ([\#4384](https://github.com/matrix-org/synapse/issues/4384), [\#4492](https://github.com/matrix-org/synapse/issues/4492), [\#4525](https://github.com/matrix-org/synapse/issues/4525))
+- Implement MSC1708 (.well-known routing for server-server federation) ([\#4408](https://github.com/matrix-org/synapse/issues/4408), [\#4409](https://github.com/matrix-org/synapse/issues/4409), [\#4426](https://github.com/matrix-org/synapse/issues/4426), [\#4427](https://github.com/matrix-org/synapse/issues/4427), [\#4428](https://github.com/matrix-org/synapse/issues/4428), [\#4464](https://github.com/matrix-org/synapse/issues/4464), [\#4468](https://github.com/matrix-org/synapse/issues/4468), [\#4487](https://github.com/matrix-org/synapse/issues/4487), [\#4488](https://github.com/matrix-org/synapse/issues/4488), [\#4489](https://github.com/matrix-org/synapse/issues/4489), [\#4497](https://github.com/matrix-org/synapse/issues/4497), [\#4511](https://github.com/matrix-org/synapse/issues/4511), [\#4516](https://github.com/matrix-org/synapse/issues/4516), [\#4520](https://github.com/matrix-org/synapse/issues/4520), [\#4521](https://github.com/matrix-org/synapse/issues/4521))
+- Search now includes results from predecessor rooms after a room upgrade. ([\#4415](https://github.com/matrix-org/synapse/issues/4415))
+- Config option to disable requesting MSISDN on registration. ([\#4423](https://github.com/matrix-org/synapse/issues/4423))
+- Add a metric for tracking event stream position of the user directory. ([\#4445](https://github.com/matrix-org/synapse/issues/4445))
+- Support exposing server capabilities in CS API (MSC1753, MSC1804) ([\#4472](https://github.com/matrix-org/synapse/issues/4472))
+- Add support for room version 3 ([\#4483](https://github.com/matrix-org/synapse/issues/4483), [\#4499](https://github.com/matrix-org/synapse/issues/4499), [\#4515](https://github.com/matrix-org/synapse/issues/4515), [\#4523](https://github.com/matrix-org/synapse/issues/4523))
+- Synapse will now reload TLS certificates from disk upon SIGHUP. ([\#4495](https://github.com/matrix-org/synapse/issues/4495), [\#4524](https://github.com/matrix-org/synapse/issues/4524))
+
+
+Bugfixes
+--------
+
+- Prevent users with access tokens predating the introduction of device IDs from creating spurious entries in the user_ips table. ([\#4369](https://github.com/matrix-org/synapse/issues/4369))
+- Fix typo in ALL_USER_TYPES definition to ensure type is a tuple ([\#4392](https://github.com/matrix-org/synapse/issues/4392))
+- Fix high CPU usage due to remote devicelist updates ([\#4397](https://github.com/matrix-org/synapse/issues/4397))
+- Fix potential bug where creating or joining a room could fail ([\#4404](https://github.com/matrix-org/synapse/issues/4404))
+- Fix bug when rejecting remote invites ([\#4405](https://github.com/matrix-org/synapse/issues/4405))
+- Fix incorrect logcontexts after a Deferred was cancelled ([\#4407](https://github.com/matrix-org/synapse/issues/4407))
+- Ensure encrypted room state is persisted across room upgrades. ([\#4411](https://github.com/matrix-org/synapse/issues/4411))
+- Copy over whether a room is a direct message and any associated room tags on room upgrade. ([\#4412](https://github.com/matrix-org/synapse/issues/4412))
+- Fix None guard in calling config.server.is_threepid_reserved ([\#4435](https://github.com/matrix-org/synapse/issues/4435))
+- Don't send IP addresses as SNI ([\#4452](https://github.com/matrix-org/synapse/issues/4452))
+- Fix UnboundLocalError in post_urlencoded_get_json ([\#4460](https://github.com/matrix-org/synapse/issues/4460))
+- Add a timeout to filtered room directory queries. ([\#4461](https://github.com/matrix-org/synapse/issues/4461))
+- Workaround for login error when using both LDAP and internal authentication. ([\#4486](https://github.com/matrix-org/synapse/issues/4486))
+- Fix a bug where setting a relative consent directory path would cause a crash. ([\#4512](https://github.com/matrix-org/synapse/issues/4512))
+
+
+Deprecations and Removals
+-------------------------
+
+- Synapse no longer generates self-signed TLS certificates when generating a configuration file. ([\#4509](https://github.com/matrix-org/synapse/issues/4509))
+
+
+Internal Changes
+----------------
+
+- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+. ([\#4306](https://github.com/matrix-org/synapse/issues/4306), [\#4459](https://github.com/matrix-org/synapse/issues/4459), [\#4466](https://github.com/matrix-org/synapse/issues/4466), [\#4471](https://github.com/matrix-org/synapse/issues/4471), [\#4477](https://github.com/matrix-org/synapse/issues/4477), [\#4505](https://github.com/matrix-org/synapse/issues/4505))
+- Update README to use the new virtualenv everywhere ([\#4342](https://github.com/matrix-org/synapse/issues/4342))
+- Add better logging for unexpected errors while sending transactions ([\#4368](https://github.com/matrix-org/synapse/issues/4368))
+- Apply a unique index to the user_ips table, preventing duplicates. ([\#4370](https://github.com/matrix-org/synapse/issues/4370), [\#4432](https://github.com/matrix-org/synapse/issues/4432), [\#4434](https://github.com/matrix-org/synapse/issues/4434))
+- Silence travis-ci build warnings by removing non-functional python3.6 ([\#4377](https://github.com/matrix-org/synapse/issues/4377))
+- Fix a comment in the generated config file ([\#4387](https://github.com/matrix-org/synapse/issues/4387))
+- Add ground work for implementing future federation API versions ([\#4390](https://github.com/matrix-org/synapse/issues/4390))
+- Update dependencies on msgpack and pymacaroons to use the up-to-date packages. ([\#4399](https://github.com/matrix-org/synapse/issues/4399))
+- Tweak codecov settings to make them less loud. ([\#4400](https://github.com/matrix-org/synapse/issues/4400))
+- Implement server support for MSC1794 - Federation v2 Invite API ([\#4402](https://github.com/matrix-org/synapse/issues/4402))
+- debian package: symlink to explicit python version ([\#4433](https://github.com/matrix-org/synapse/issues/4433))
+- Add infrastructure to support different event formats ([\#4437](https://github.com/matrix-org/synapse/issues/4437), [\#4447](https://github.com/matrix-org/synapse/issues/4447), [\#4448](https://github.com/matrix-org/synapse/issues/4448), [\#4470](https://github.com/matrix-org/synapse/issues/4470), [\#4481](https://github.com/matrix-org/synapse/issues/4481), [\#4482](https://github.com/matrix-org/synapse/issues/4482), [\#4493](https://github.com/matrix-org/synapse/issues/4493), [\#4494](https://github.com/matrix-org/synapse/issues/4494), [\#4496](https://github.com/matrix-org/synapse/issues/4496), [\#4510](https://github.com/matrix-org/synapse/issues/4510), [\#4514](https://github.com/matrix-org/synapse/issues/4514))
+- Generate the debian config during build ([\#4444](https://github.com/matrix-org/synapse/issues/4444))
+- Clarify documentation for the `public_baseurl` config param ([\#4458](https://github.com/matrix-org/synapse/issues/4458), [\#4498](https://github.com/matrix-org/synapse/issues/4498))
+- Fix quoting for allowed_local_3pids example config ([\#4476](https://github.com/matrix-org/synapse/issues/4476))
+- Remove deprecated --process-dependency-links option from UPGRADE.rst ([\#4485](https://github.com/matrix-org/synapse/issues/4485))
+- Make it possible to set the log level for tests via an environment variable ([\#4506](https://github.com/matrix-org/synapse/issues/4506))
+- Reduce the log level of linearizer lock acquirement to DEBUG. ([\#4507](https://github.com/matrix-org/synapse/issues/4507))
+- Fix code to comply with linting in PyFlakes 3.7.1. ([\#4519](https://github.com/matrix-org/synapse/issues/4519))
+
+
 Synapse 0.34.1.1 (2019-01-11)
 =============================
 
diff --git a/UPGRADE.rst b/UPGRADE.rst
index 47a8cb9c88..c46f70f699 100644
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -48,6 +48,38 @@ returned by the Client-Server API:
     # configured on port 443.
     curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
 
+Upgrading to v0.99.0
+====================
+
+In preparation for Synapse v1.0, you must update your TLS certificates from
+self-signed ones to verifiable ones signed by a trusted root CA.
+
+If you do not already have a certificate for your domain, the easiest way to get
+one is with Synapse's new ACME support, which will use the ACME protocol to
+provision a certificate automatically. By default, certificates will be obtained
+from the publicly trusted CA Let's Encrypt.
+
+For a sample configuration, please inspect the new ACME section in the example
+generated config by running the ``generate-config`` executable. For example::
+
+  ~/synapse/env3/bin/generate-config
+
+You will need to provide Let's Encrypt (or other ACME provider) access to your
+Synapse ACME challenge responder on port 80, at the domain of your homeserver.
+This requires you either change the port of the ACME listener provided by
+Synapse to a high port and reverse proxy to it, or use a tool like authbind to
+allow Synapse to listen on port 80 without root access. (Do not run Synapse with
+root permissions!)
+
+You will need to back up or delete your self signed TLS certificate
+(``example.com.tls.crt`` and ``example.com.tls.key``), Synapse's ACME
+implementation will not overwrite them.
+
+You may wish to use alternate methods such as Certbot to obtain a certificate
+from Let's Encrypt, depending on your server configuration. Of course, if you
+already have a valid certificate for your homeserver's domain, that can be
+placed in Synapse's config directory without the need for ACME.
+
 Upgrading to v0.34.0
 ====================
 
diff --git a/changelog.d/3902.feature b/changelog.d/3902.feature
new file mode 100644
index 0000000000..eb8d9f2393
--- /dev/null
+++ b/changelog.d/3902.feature
@@ -0,0 +1 @@
+Include m.room.encryption on invites by default
diff --git a/changelog.d/4229.feature b/changelog.d/4229.feature
deleted file mode 100644
index 0d1996c7e8..0000000000
--- a/changelog.d/4229.feature
+++ /dev/null
@@ -1 +0,0 @@
-Synapse's cipher string has been updated to require ECDH key exchange. Configuring and generating dh_params is no longer required, and they will be ignored.
diff --git a/changelog.d/4306.misc b/changelog.d/4306.misc
deleted file mode 100644
index 58130b6190..0000000000
--- a/changelog.d/4306.misc
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4342.misc b/changelog.d/4342.misc
deleted file mode 100644
index a4fda47c20..0000000000
--- a/changelog.d/4342.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update README to use the new virtualenv everywhere
\ No newline at end of file
diff --git a/changelog.d/4368.misc b/changelog.d/4368.misc
deleted file mode 100644
index 020dacb547..0000000000
--- a/changelog.d/4368.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add better logging for unexpected errors while sending transactions
diff --git a/changelog.d/4369.bugfix b/changelog.d/4369.bugfix
deleted file mode 100644
index a78d557932..0000000000
--- a/changelog.d/4369.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Prevent users with access tokens predating the introduction of device IDs from creating spurious entries in the user_ips table.
diff --git a/changelog.d/4370.misc b/changelog.d/4370.misc
deleted file mode 100644
index 047061ed3c..0000000000
--- a/changelog.d/4370.misc
+++ /dev/null
@@ -1 +0,0 @@
-Apply a unique index to the user_ips table, preventing duplicates.
diff --git a/changelog.d/4377.misc b/changelog.d/4377.misc
deleted file mode 100644
index 06273023fc..0000000000
--- a/changelog.d/4377.misc
+++ /dev/null
@@ -1 +0,0 @@
-Silence travis-ci build warnings by removing non-functional python3.6
\ No newline at end of file
diff --git a/changelog.d/4384.feature b/changelog.d/4384.feature
deleted file mode 100644
index daedcd58c4..0000000000
--- a/changelog.d/4384.feature
+++ /dev/null
@@ -1 +0,0 @@
-Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt).
diff --git a/changelog.d/4387.misc b/changelog.d/4387.misc
deleted file mode 100644
index 0c04a0fa9b..0000000000
--- a/changelog.d/4387.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix a comment in the generated config file
diff --git a/changelog.d/4390.misc b/changelog.d/4390.misc
deleted file mode 100644
index c05a9609cf..0000000000
--- a/changelog.d/4390.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add ground work for implementing future federation API versions
diff --git a/changelog.d/4392.bugfix b/changelog.d/4392.bugfix
deleted file mode 100644
index 2223f7dcd6..0000000000
--- a/changelog.d/4392.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix typo in ALL_USER_TYPES definition to ensure type is a tuple
diff --git a/changelog.d/4397.bugfix b/changelog.d/4397.bugfix
deleted file mode 100644
index e7526d4454..0000000000
--- a/changelog.d/4397.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix high CPU usage due to remote devicelist updates
diff --git a/changelog.d/4399.misc b/changelog.d/4399.misc
deleted file mode 100644
index 2f77a8fa54..0000000000
--- a/changelog.d/4399.misc
+++ /dev/null
@@ -1 +0,0 @@
-Update dependencies on msgpack and pymacaroons to use the up-to-date packages.
diff --git a/changelog.d/4400.misc b/changelog.d/4400.misc
deleted file mode 100644
index 3d299dfe95..0000000000
--- a/changelog.d/4400.misc
+++ /dev/null
@@ -1 +0,0 @@
-Tweak codecov settings to make them less loud.
diff --git a/changelog.d/4402.misc b/changelog.d/4402.misc
deleted file mode 100644
index 4a0063ed34..0000000000
--- a/changelog.d/4402.misc
+++ /dev/null
@@ -1 +0,0 @@
-Implement server support for MSC1794 - Federation v2 Invite API
diff --git a/changelog.d/4404.bugfix b/changelog.d/4404.bugfix
deleted file mode 100644
index 5d40a3a60b..0000000000
--- a/changelog.d/4404.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix potential bug where creating or joining a room could fail
diff --git a/changelog.d/4405.bugfix b/changelog.d/4405.bugfix
deleted file mode 100644
index 974d799b88..0000000000
--- a/changelog.d/4405.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix bug when rejecting remote invites
diff --git a/changelog.d/4407.bugfix b/changelog.d/4407.bugfix
deleted file mode 100644
index 54c5e76d1f..0000000000
--- a/changelog.d/4407.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix incorrect logcontexts after a Deferred was cancelled
diff --git a/changelog.d/4408.feature b/changelog.d/4408.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4408.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4409.feature b/changelog.d/4409.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4409.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4411.bugfix b/changelog.d/4411.bugfix
deleted file mode 100644
index 219e98a924..0000000000
--- a/changelog.d/4411.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Ensure encrypted room state is persisted across room upgrades.
\ No newline at end of file
diff --git a/changelog.d/4412.bugfix b/changelog.d/4412.bugfix
deleted file mode 100644
index 007be1b7db..0000000000
--- a/changelog.d/4412.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Copy over whether a room is a direct message and any associated room tags on room upgrade.
\ No newline at end of file
diff --git a/changelog.d/4415.feature b/changelog.d/4415.feature
deleted file mode 100644
index 1fb1d58f8f..0000000000
--- a/changelog.d/4415.feature
+++ /dev/null
@@ -1 +0,0 @@
-Search now includes results from predecessor rooms after a room upgrade.
\ No newline at end of file
diff --git a/changelog.d/4423.feature b/changelog.d/4423.feature
deleted file mode 100644
index 74aeab6d39..0000000000
--- a/changelog.d/4423.feature
+++ /dev/null
@@ -1 +0,0 @@
-Config option to disable requesting MSISDN on registration.
diff --git a/changelog.d/4426.feature b/changelog.d/4426.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4426.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4427.feature b/changelog.d/4427.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4427.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4428.feature b/changelog.d/4428.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4428.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4432.misc b/changelog.d/4432.misc
deleted file mode 100644
index 047061ed3c..0000000000
--- a/changelog.d/4432.misc
+++ /dev/null
@@ -1 +0,0 @@
-Apply a unique index to the user_ips table, preventing duplicates.
diff --git a/changelog.d/4433.misc b/changelog.d/4433.misc
deleted file mode 100644
index 30f2912db2..0000000000
--- a/changelog.d/4433.misc
+++ /dev/null
@@ -1 +0,0 @@
-debian package: symlink to explicit python version
diff --git a/changelog.d/4434.misc b/changelog.d/4434.misc
deleted file mode 100644
index 047061ed3c..0000000000
--- a/changelog.d/4434.misc
+++ /dev/null
@@ -1 +0,0 @@
-Apply a unique index to the user_ips table, preventing duplicates.
diff --git a/changelog.d/4435.bugfix b/changelog.d/4435.bugfix
deleted file mode 100644
index 4ea9a5df02..0000000000
--- a/changelog.d/4435.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix None guard in calling config.server.is_threepid_reserved
diff --git a/changelog.d/4437.misc b/changelog.d/4437.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4437.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4444.misc b/changelog.d/4444.misc
deleted file mode 100644
index 1be84188c6..0000000000
--- a/changelog.d/4444.misc
+++ /dev/null
@@ -1 +0,0 @@
-Generate the debian config during build
diff --git a/changelog.d/4445.feature b/changelog.d/4445.feature
deleted file mode 100644
index a6f9b7bbac..0000000000
--- a/changelog.d/4445.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add a metric for tracking event stream position of the user directory.
\ No newline at end of file
diff --git a/changelog.d/4447.misc b/changelog.d/4447.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4447.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4448.misc b/changelog.d/4448.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4448.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4452.bugfix b/changelog.d/4452.bugfix
deleted file mode 100644
index a715ca3788..0000000000
--- a/changelog.d/4452.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Don't send IP addresses as SNI
diff --git a/changelog.d/4458.misc b/changelog.d/4458.misc
deleted file mode 100644
index 8b3bc94a34..0000000000
--- a/changelog.d/4458.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clarify documentation for the `public_baseurl` config param
diff --git a/changelog.d/4459.misc b/changelog.d/4459.misc
deleted file mode 100644
index 58130b6190..0000000000
--- a/changelog.d/4459.misc
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4460.bugfix b/changelog.d/4460.bugfix
deleted file mode 100644
index 8c5d5b4e0e..0000000000
--- a/changelog.d/4460.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix UnboundLocalError in post_urlencoded_get_json
diff --git a/changelog.d/4461.bugfix b/changelog.d/4461.bugfix
deleted file mode 100644
index 92062a2bfb..0000000000
--- a/changelog.d/4461.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Add a timeout to filtered room directory queries.
\ No newline at end of file
diff --git a/changelog.d/4464.feature b/changelog.d/4464.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4464.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4466.misc b/changelog.d/4466.misc
deleted file mode 100644
index 58130b6190..0000000000
--- a/changelog.d/4466.misc
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4468.feature b/changelog.d/4468.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4468.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4470.misc b/changelog.d/4470.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4470.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4471.misc b/changelog.d/4471.misc
deleted file mode 100644
index 994801fd1e..0000000000
--- a/changelog.d/4471.misc
+++ /dev/null
@@ -1 +0,0 @@
- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4472.feature b/changelog.d/4472.feature
deleted file mode 100644
index 3413c33d48..0000000000
--- a/changelog.d/4472.feature
+++ /dev/null
@@ -1 +0,0 @@
-Support exposing server capabilities in CS API (MSC1753, MSC1804)
diff --git a/changelog.d/4476.misc b/changelog.d/4476.misc
deleted file mode 100644
index a070e10c7c..0000000000
--- a/changelog.d/4476.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix quoting for allowed_local_3pids example config
diff --git a/changelog.d/4477.misc b/changelog.d/4477.misc
deleted file mode 100644
index 58130b6190..0000000000
--- a/changelog.d/4477.misc
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4481.misc b/changelog.d/4481.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4481.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4482.misc b/changelog.d/4482.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4482.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4483.feature b/changelog.d/4483.feature
deleted file mode 100644
index 9538c64f08..0000000000
--- a/changelog.d/4483.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add support for room version 3
diff --git a/changelog.d/4485.misc b/changelog.d/4485.misc
deleted file mode 100644
index 8aa0aeab2a..0000000000
--- a/changelog.d/4485.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove deprecated --process-dependency-links option from UPGRADE.rst
diff --git a/changelog.d/4486.bugfix b/changelog.d/4486.bugfix
deleted file mode 100644
index 64588d509b..0000000000
--- a/changelog.d/4486.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Workaround for login error when using both LDAP and internal authentication.
diff --git a/changelog.d/4487.feature b/changelog.d/4487.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4487.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4488.feature b/changelog.d/4488.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4488.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4489.feature b/changelog.d/4489.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4489.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4492.feature b/changelog.d/4492.feature
deleted file mode 100644
index c7f595cec2..0000000000
--- a/changelog.d/4492.feature
+++ /dev/null
@@ -1 +0,0 @@
- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt).
diff --git a/changelog.d/4493.misc b/changelog.d/4493.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4493.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4494.misc b/changelog.d/4494.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4494.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4495.feature b/changelog.d/4495.feature
deleted file mode 100644
index fc2b5daf63..0000000000
--- a/changelog.d/4495.feature
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now reload TLS certificates from disk upon SIGHUP.
diff --git a/changelog.d/4496.misc b/changelog.d/4496.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4496.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4497.feature b/changelog.d/4497.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4497.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4498.misc b/changelog.d/4498.misc
deleted file mode 100644
index 8b3bc94a34..0000000000
--- a/changelog.d/4498.misc
+++ /dev/null
@@ -1 +0,0 @@
-Clarify documentation for the `public_baseurl` config param
diff --git a/changelog.d/4499.feature b/changelog.d/4499.feature
deleted file mode 100644
index 9538c64f08..0000000000
--- a/changelog.d/4499.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add support for room version 3
diff --git a/changelog.d/4505.misc b/changelog.d/4505.misc
deleted file mode 100644
index 994801fd1e..0000000000
--- a/changelog.d/4505.misc
+++ /dev/null
@@ -1 +0,0 @@
- Synapse will now take advantage of native UPSERT functionality in PostgreSQL 9.5+ and SQLite 3.24+.
diff --git a/changelog.d/4506.misc b/changelog.d/4506.misc
deleted file mode 100644
index ea0e7d9580..0000000000
--- a/changelog.d/4506.misc
+++ /dev/null
@@ -1 +0,0 @@
-Make it possible to set the log level for tests via an environment variable
\ No newline at end of file
diff --git a/changelog.d/4507.misc b/changelog.d/4507.misc
deleted file mode 100644
index baf45b2bec..0000000000
--- a/changelog.d/4507.misc
+++ /dev/null
@@ -1 +0,0 @@
-Reduce the log level of linearizer lock acquirement to DEBUG.
diff --git a/changelog.d/4509.removal b/changelog.d/4509.removal
deleted file mode 100644
index 9165009813..0000000000
--- a/changelog.d/4509.removal
+++ /dev/null
@@ -1 +0,0 @@
-Synapse no longer generates self-signed TLS certificates when generating a configuration file.
diff --git a/changelog.d/4510.misc b/changelog.d/4510.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4510.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4511.feature b/changelog.d/4511.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4511.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4512.bugfix b/changelog.d/4512.bugfix
deleted file mode 100644
index 7a1345c4ac..0000000000
--- a/changelog.d/4512.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix a bug where setting a relative consent directory path would cause a crash.
\ No newline at end of file
diff --git a/changelog.d/4513.misc b/changelog.d/4513.misc
new file mode 100644
index 0000000000..1f64a96465
--- /dev/null
+++ b/changelog.d/4513.misc
@@ -0,0 +1 @@
+Reject federation transactions if they include more than 50 PDUs or 100 EDUs.
\ No newline at end of file
diff --git a/changelog.d/4514.misc b/changelog.d/4514.misc
deleted file mode 100644
index 43f8963614..0000000000
--- a/changelog.d/4514.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add infrastructure to support different event formats
diff --git a/changelog.d/4515.feature b/changelog.d/4515.feature
deleted file mode 100644
index 9538c64f08..0000000000
--- a/changelog.d/4515.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add support for room version 3
diff --git a/changelog.d/4516.feature b/changelog.d/4516.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4516.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4519.misc b/changelog.d/4519.misc
deleted file mode 100644
index 897e783d28..0000000000
--- a/changelog.d/4519.misc
+++ /dev/null
@@ -1 +0,0 @@
-Fix code to comply with linting in PyFlakes 3.7.1.
diff --git a/changelog.d/4520.feature b/changelog.d/4520.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4520.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4521.feature b/changelog.d/4521.feature
deleted file mode 100644
index bda713adf9..0000000000
--- a/changelog.d/4521.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement MSC1708 (.well-known routing for server-server federation)
\ No newline at end of file
diff --git a/changelog.d/4523.feature b/changelog.d/4523.feature
deleted file mode 100644
index 9538c64f08..0000000000
--- a/changelog.d/4523.feature
+++ /dev/null
@@ -1 +0,0 @@
-Add support for room version 3
diff --git a/changelog.d/4524.feature b/changelog.d/4524.feature
deleted file mode 100644
index fc2b5daf63..0000000000
--- a/changelog.d/4524.feature
+++ /dev/null
@@ -1 +0,0 @@
-Synapse will now reload TLS certificates from disk upon SIGHUP.
diff --git a/changelog.d/4525.feature b/changelog.d/4525.feature
deleted file mode 100644
index c7f595cec2..0000000000
--- a/changelog.d/4525.feature
+++ /dev/null
@@ -1 +0,0 @@
- Synapse can now automatically provision TLS certificates via ACME (the protocol used by CAs like Let's Encrypt).
diff --git a/synapse/__init__.py b/synapse/__init__.py
index d6a191ccc6..5da59aa924 100644
--- a/synapse/__init__.py
+++ b/synapse/__init__.py
@@ -27,4 +27,4 @@ try:
 except ImportError:
     pass
 
-__version__ = "0.34.1.1"
+__version__ = "0.99.0rc2"
diff --git a/synapse/api/constants.py b/synapse/api/constants.py
index 0cbae9429b..f47c33a074 100644
--- a/synapse/api/constants.py
+++ b/synapse/api/constants.py
@@ -73,6 +73,7 @@ class EventTypes(object):
     RoomHistoryVisibility = "m.room.history_visibility"
     CanonicalAlias = "m.room.canonical_alias"
     RoomAvatar = "m.room.avatar"
+    RoomEncryption = "m.room.encryption"
     GuestAccess = "m.room.guest_access"
 
     # These are used for validation
@@ -109,7 +110,7 @@ class RoomVersions(object):
 
 
 class RoomDisposition(object):
-    STABLE = "stable",
+    STABLE = "stable"
     UNSTABLE = "unstable"
 
 
diff --git a/synapse/config/api.py b/synapse/config/api.py
index 403d96ba76..9f25bbc5cb 100644
--- a/synapse/config/api.py
+++ b/synapse/config/api.py
@@ -24,6 +24,7 @@ class ApiConfig(Config):
             EventTypes.JoinRules,
             EventTypes.CanonicalAlias,
             EventTypes.RoomAvatar,
+            EventTypes.RoomEncryption,
             EventTypes.Name,
         ])
 
@@ -36,5 +37,6 @@ class ApiConfig(Config):
             - "{JoinRules}"
             - "{CanonicalAlias}"
             - "{RoomAvatar}"
+            - "{RoomEncryption}"
             - "{Name}"
         """.format(**vars(EventTypes))
diff --git a/synapse/federation/federation_server.py b/synapse/federation/federation_server.py
index aeadc9c564..3da86d4ba6 100644
--- a/synapse/federation/federation_server.py
+++ b/synapse/federation/federation_server.py
@@ -148,6 +148,22 @@ class FederationServer(FederationBase):
 
         logger.debug("[%s] Transaction is new", transaction.transaction_id)
 
+        # Reject if PDU count > 50 and EDU count > 100
+        if (len(transaction.pdus) > 50
+                or (hasattr(transaction, "edus") and len(transaction.edus) > 100)):
+
+            logger.info(
+                "Transaction PDU or EDU count too large. Returning 400",
+            )
+
+            response = {}
+            yield self.transaction_actions.set_response(
+                origin,
+                transaction,
+                400, response
+            )
+            defer.returnValue((400, response))
+
         received_pdus_counter.inc(len(transaction.pdus))
 
         origin_host, _ = parse_server_name(origin)
diff --git a/synapse/handlers/sync.py b/synapse/handlers/sync.py
index f7f768f751..28857bfc1c 100644
--- a/synapse/handlers/sync.py
+++ b/synapse/handlers/sync.py
@@ -1473,10 +1473,22 @@ class SyncHandler(object):
                 if since_token and since_token.is_after(leave_token):
                     continue
 
+                # If this is an out of band message, like a remote invite
+                # rejection, we include it in the recents batch. Otherwise, we
+                # let _load_filtered_recents handle fetching the correct
+                # batches.
+                #
+                # This is all screaming out for a refactor, as the logic here is
+                # subtle and the moving parts numerous.
+                if leave_event.internal_metadata.is_out_of_band_membership():
+                    batch_events = [leave_event]
+                else:
+                    batch_events = None
+
                 room_entries.append(RoomSyncResultBuilder(
                     room_id=room_id,
                     rtype="archived",
-                    events=None,
+                    events=batch_events,
                     newly_joined=room_id in newly_joined_rooms,
                     full_state=False,
                     since_token=since_token,