Merge tag 'v0.33.3'

Features
--------

- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439))
- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184))
- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568))
- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574))
- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589))
- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633))
- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653))
- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654))
- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655))
- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662))
- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664))
- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670))
- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687))
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694))
- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708))

Bugfixes
--------

- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658))
- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661))
- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676))
- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677))
- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681))
- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689))
- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692))
- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700))
- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701))
- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705), [\#3713](https://github.com/matrix-org/synapse/issues/3713))
- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710))
- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719))
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723))
- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732))

Deprecations and Removals
-------------------------

- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst). ([\#3703](https://github.com/matrix-org/synapse/issues/3703))

Internal Changes
----------------

- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423))
- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632))
- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647))
- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660))
- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668))
- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669))
- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678))
- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679))
- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684))
- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690))
- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707))
- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709))
- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712))

1 files changed, 17 insertions, 38 deletions

diff --git a/tests/test_preview.py b/tests/test_preview.py

index 446843367e..84ef5e5ba4 100644
--- a/tests/test_preview.py
+++ b/tests/test_preview.py
@@ -22,7 +22,6 @@ from . import unittest
 
 
 class PreviewTestCase(unittest.TestCase):
-
     def test_long_summarize(self):
         example_paras = [
             u"""Tromsø (Norwegian pronunciation: [ˈtrʊmsœ] ( listen); Northern Sami:
@@ -32,7 +31,6 @@ class PreviewTestCase(unittest.TestCase):
             alternative spellings of the city.Tromsø is considered the northernmost
             city in the world with a population above 50,000. The most populous town
             north of it is Alta, Norway, with a population of 14,272 (2013).""",
-
             u"""Tromsø lies in Northern Norway. The municipality has a population of
             (2015) 72,066, but with an annual influx of students it has over 75,000
             most of the year. It is the largest urban area in Northern Norway and the
@@ -46,7 +44,6 @@ class PreviewTestCase(unittest.TestCase):
             Sandnessund Bridge. Tromsø Airport connects the city to many destinations
             in Europe. The city is warmer than most other places located on the same
             latitude, due to the warming effect of the Gulf Stream.""",
-
             u"""The city centre of Tromsø contains the highest number of old wooden
             houses in Northern Norway, the oldest house dating from 1789. The Arctic
             Cathedral, a modern church from 1965, is probably the most famous landmark
@@ -67,7 +64,7 @@ class PreviewTestCase(unittest.TestCase):
             u" the city of Tromsø. Outside of Norway, Tromso and Tromsö are"
             u" alternative spellings of the city.Tromsø is considered the northernmost"
             u" city in the world with a population above 50,000. The most populous town"
-            u" north of it is Alta, Norway, with a population of 14,272 (2013)."
+            u" north of it is Alta, Norway, with a population of 14,272 (2013).",
         )
 
         desc = summarize_paragraphs(example_paras[1:], min_size=200, max_size=500)
@@ -80,7 +77,7 @@ class PreviewTestCase(unittest.TestCase):
             u" third largest north of the Arctic Circle (following Murmansk and Norilsk)."
             u" Most of Tromsø, including the city centre, is located on the island of"
             u" Tromsøya, 350 kilometres (217 mi) north of the Arctic Circle. In 2012,"
-            u" Tromsøya had a population of 36,088. Substantial parts of the urban…"
+            u" Tromsøya had a population of 36,088. Substantial parts of the urban…",
         )
 
     def test_short_summarize(self):
@@ -88,11 +85,9 @@ class PreviewTestCase(unittest.TestCase):
             u"Tromsø (Norwegian pronunciation: [ˈtrʊmsœ] ( listen); Northern Sami:"
             u" Romsa; Finnish: Tromssa[2] Kven: Tromssa) is a city and municipality in"
             u" Troms county, Norway.",
-
             u"Tromsø lies in Northern Norway. The municipality has a population of"
             u" (2015) 72,066, but with an annual influx of students it has over 75,000"
             u" most of the year.",
-
             u"The city centre of Tromsø contains the highest number of old wooden"
             u" houses in Northern Norway, the oldest house dating from 1789. The Arctic"
             u" Cathedral, a modern church from 1965, is probably the most famous landmark"
@@ -109,7 +104,7 @@ class PreviewTestCase(unittest.TestCase):
             u"\n"
             u"Tromsø lies in Northern Norway. The municipality has a population of"
             u" (2015) 72,066, but with an annual influx of students it has over 75,000"
-            u" most of the year."
+            u" most of the year.",
         )
 
     def test_small_then_large_summarize(self):
@@ -117,7 +112,6 @@ class PreviewTestCase(unittest.TestCase):
             u"Tromsø (Norwegian pronunciation: [ˈtrʊmsœ] ( listen); Northern Sami:"
             u" Romsa; Finnish: Tromssa[2] Kven: Tromssa) is a city and municipality in"
             u" Troms county, Norway.",
-
             u"Tromsø lies in Northern Norway. The municipality has a population of"
             u" (2015) 72,066, but with an annual influx of students it has over 75,000"
             u" most of the year."
@@ -138,7 +132,7 @@ class PreviewTestCase(unittest.TestCase):
             u" (2015) 72,066, but with an annual influx of students it has over 75,000"
             u" most of the year. The city centre of Tromsø contains the highest number"
             u" of old wooden houses in Northern Norway, the oldest house dating from"
-            u" 1789. The Arctic Cathedral, a modern church from…"
+            u" 1789. The Arctic Cathedral, a modern church from…",
         )
 
 
@@ -155,10 +149,7 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": u"Foo",
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": u"Foo", u"og:description": u"Some text."})
 
     def test_comment(self):
         html = u"""
@@ -173,10 +164,7 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": u"Foo",
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": u"Foo", u"og:description": u"Some text."})
 
     def test_comment2(self):
         html = u"""
@@ -194,10 +182,13 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": u"Foo",
-            u"og:description": u"Some text.\n\nSome more text.\n\nText\n\nMore text"
-        })
+        self.assertEquals(
+            og,
+            {
+                u"og:title": u"Foo",
+                u"og:description": u"Some text.\n\nSome more text.\n\nText\n\nMore text",
+            },
+        )
 
     def test_script(self):
         html = u"""
@@ -212,10 +203,7 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": u"Foo",
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": u"Foo", u"og:description": u"Some text."})
 
     def test_missing_title(self):
         html = u"""
@@ -228,10 +216,7 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": None,
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": None, u"og:description": u"Some text."})
 
     def test_h1_as_title(self):
         html = u"""
@@ -245,10 +230,7 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": u"Title",
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": u"Title", u"og:description": u"Some text."})
 
     def test_missing_title_and_broken_h1(self):
         html = u"""
@@ -262,7 +244,4 @@ class PreviewUrlTestCase(unittest.TestCase):
 
         og = decode_and_calc_og(html, "http://example.com/test.html")
 
-        self.assertEquals(og, {
-            u"og:title": None,
-            u"og:description": u"Some text."
-        })
+        self.assertEquals(og, {u"og:title": None, u"og:description": u"Some text."})