diff --git a/CHANGES.md b/CHANGES.md
index f794c585b7..09a4b236c2 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,12 +1,52 @@
-Synapse 1.12.0 (2020-03-23)
+Synapse 1.12.3 (2020-04-03)
+===========================
+
+- Remove the the pin to Pillow 7.0 which was introduced in Synapse 1.12.2, and
+correctly fix the issue with building the Debian packages. ([\#7212](https://github.com/matrix-org/synapse/issues/7212))
+
+Synapse 1.12.2 (2020-04-02)
===========================
-No significant changes since 1.12.0rc1.
+This release works around [an
+issue](https://github.com/matrix-org/synapse/issues/7208) with building the
+debian packages.
+
+No other significant changes since 1.12.1.
+
+
+Synapse 1.12.1 (2020-04-02)
+===========================
+
+No significant changes since 1.12.1rc1.
+
+
+Synapse 1.12.1rc1 (2020-03-31)
+==============================
+
+Bugfixes
+--------
+
+- Fix starting workers when federation sending not split out. ([\#7133](https://github.com/matrix-org/synapse/issues/7133)). Introduced in v1.12.0.
+- Avoid importing `sqlite3` when using the postgres backend. Contributed by David Vo. ([\#7155](https://github.com/matrix-org/synapse/issues/7155)). Introduced in v1.12.0rc1.
+- Fix a bug which could cause outbound federation traffic to stop working if a client uploaded an incorrect e2e device signature. ([\#7177](https://github.com/matrix-org/synapse/issues/7177)). Introduced in v1.11.0.
+
+
+Synapse 1.12.0 (2020-03-23)
+===========================
Debian packages and Docker images are rebuilt using the latest versions of
dependency libraries, including Twisted 20.3.0. **Please see security advisory
below**.
+Potential slow database update during upgrade
+---------------------------------------------
+
+Synapse 1.12.0 includes a database update which is run as part of the upgrade,
+and which may take some time (several hours in the case of a large
+server). Synapse will not respond to HTTP requests while this update is taking
+place. For imformation on seeing if you are affected, and workaround if you
+are, see the [upgrade notes](UPGRADE.rst#upgrading-to-v1120).
+
Security advisory
-----------------
diff --git a/INSTALL.md b/INSTALL.md
index c0926ba590..8ded6e9092 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -36,7 +36,7 @@ that your email address is probably `user@example.com` rather than
System requirements:
- POSIX-compliant system (tested on Linux & OS X)
-- Python 3.5, 3.6, 3.7 or 3.8.
+- Python 3.5.2 or later, up to Python 3.8.
- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
Synapse is written in Python but some of the libraries it uses are written in
@@ -427,8 +427,8 @@ so, you will need to edit `homeserver.yaml`, as follows:
for having Synapse automatically provision and renew federation
certificates through ACME can be found at [ACME.md](docs/ACME.md).
Note that, as pointed out in that document, this feature will not
- work with installs set up after November 2019.
-
+ work with installs set up after November 2019.
+
If you are using your own certificate, be sure to use a `.pem` file that
includes the full certificate chain including any intermediate certificates
(for instance, if using certbot, use `fullchain.pem` as your certificate, not
diff --git a/UPGRADE.rst b/UPGRADE.rst
index 3cad8c2837..768d94a393 100644
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -75,6 +75,71 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
+Upgrading to v1.12.0
+====================
+
+This version includes a database update which is run as part of the upgrade,
+and which may take some time (several hours in the case of a large
+server). Synapse will not respond to HTTP requests while this update is taking
+place.
+
+This is only likely to be a problem in the case of a server which is
+participating in many rooms.
+
+0. As with all upgrades, it is recommended that you have a recent backup of
+ your database which can be used for recovery in the event of any problems.
+
+1. As an initial check to see if you will be affected, you can try running the
+ following query from the `psql` or `sqlite3` console. It is safe to run it
+ while Synapse is still running.
+
+ .. code:: sql
+
+ SELECT MAX(q.v) FROM (
+ SELECT (
+ SELECT ej.json AS v
+ FROM state_events se INNER JOIN event_json ej USING (event_id)
+ WHERE se.room_id=rooms.room_id AND se.type='m.room.create' AND se.state_key=''
+ LIMIT 1
+ ) FROM rooms WHERE rooms.room_version IS NULL
+ ) q;
+
+ This query will take about the same amount of time as the upgrade process: ie,
+ if it takes 5 minutes, then it is likely that Synapse will be unresponsive for
+ 5 minutes during the upgrade.
+
+ If you consider an outage of this duration to be acceptable, no further
+ action is necessary and you can simply start Synapse 1.12.0.
+
+ If you would prefer to reduce the downtime, continue with the steps below.
+
+2. The easiest workaround for this issue is to manually
+ create a new index before upgrading. On PostgreSQL, his can be done as follows:
+
+ .. code:: sql
+
+ CREATE INDEX CONCURRENTLY tmp_upgrade_1_12_0_index
+ ON state_events(room_id) WHERE type = 'm.room.create';
+
+ The above query may take some time, but is also safe to run while Synapse is
+ running.
+
+ We assume that no SQLite users have databases large enough to be
+ affected. If you *are* affected, you can run a similar query, omitting the
+ ``CONCURRENTLY`` keyword. Note however that this operation may in itself cause
+ Synapse to stop running for some time. Synapse admins are reminded that
+ `SQLite is not recommended for use outside a test
+ environment <https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql>`_.
+
+3. Once the index has been created, the ``SELECT`` query in step 1 above should
+ complete quickly. It is therefore safe to upgrade to Synapse 1.12.0.
+
+4. Once Synapse 1.12.0 has successfully started and is responding to HTTP
+ requests, the temporary index can be removed:
+
+ .. code:: sql
+
+ DROP INDEX tmp_upgrade_1_12_0_index;
Upgrading to v1.10.0
====================
diff --git a/debian/changelog b/debian/changelog
index 39ec9da7ab..642115fc5a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,26 @@
+matrix-synapse-py3 (1.12.3) stable; urgency=medium
+
+ [ Richard van der Hoff ]
+ * Update the Debian build scripts to handle the new installation paths
+ for the support libraries introduced by Pillow 7.1.1.
+
+ [ Synapse Packaging team ]
+ * New synapse release 1.12.3.
+
+ -- Synapse Packaging team <packages@matrix.org> Fri, 03 Apr 2020 10:55:03 +0100
+
+matrix-synapse-py3 (1.12.2) stable; urgency=medium
+
+ * New synapse release 1.12.2.
+
+ -- Synapse Packaging team <packages@matrix.org> Mon, 02 Apr 2020 19:02:17 +0000
+
+matrix-synapse-py3 (1.12.1) stable; urgency=medium
+
+ * New synapse release 1.12.1.
+
+ -- Synapse Packaging team <packages@matrix.org> Mon, 02 Apr 2020 11:30:47 +0000
+
matrix-synapse-py3 (1.12.0) stable; urgency=medium
* New synapse release 1.12.0.
diff --git a/debian/rules b/debian/rules
index a4d2ce2ba4..c744060a57 100755
--- a/debian/rules
+++ b/debian/rules
@@ -15,17 +15,38 @@ override_dh_installinit:
# we don't really want to strip the symbols from our object files.
override_dh_strip:
+# dh_shlibdeps calls dpkg-shlibdeps, which finds all the binary files
+# (executables and shared libs) in the package, and looks for the shared
+# libraries that they depend on. It then adds a dependency on the package that
+# contains that library to the package.
+#
+# We make two modifications to that process...
+#
override_dh_shlibdeps:
- # make the postgres package's dependencies a recommendation
- # rather than a hard dependency.
+ # Firstly, postgres is not a hard dependency for us, so we want to make
+ # the things that psycopg2 depends on (such as libpq) be
+ # recommendations rather than hard dependencies. We do so by
+ # running dpkg-shlibdeps manually on psycopg2's libs.
+ #
find debian/$(PACKAGE_NAME)/ -path '*/site-packages/psycopg2/*.so' | \
xargs dpkg-shlibdeps -Tdebian/$(PACKAGE_NAME).substvars \
-pshlibs1 -dRecommends
- # all the other dependencies can be normal 'Depends' requirements,
- # except for PIL's, which is self-contained and which confuses
- # dpkg-shlibdeps.
- dh_shlibdeps -X site-packages/PIL/.libs -X site-packages/psycopg2
+ # secondly, we exclude PIL's libraries from the process. They are known
+ # to be self-contained, but they have interdependencies and
+ # dpkg-shlibdeps doesn't know how to resolve them.
+ #
+ # As of Pillow 7.1.0, these libraries are in
+ # site-packages/Pillow.libs. Previously, they were in
+ # site-packages/PIL/.libs.
+ #
+ # (we also need to exclude psycopg2, of course, since we've already
+ # dealt with that.)
+ #
+ dh_shlibdeps \
+ -X site-packages/PIL/.libs \
+ -X site-packages/Pillow.libs \
+ -X site-packages/psycopg2
override_dh_virtualenv:
./debian/build_virtualenv
diff --git a/docs/postgres.md b/docs/postgres.md
index e0793ecee8..ca7ef1cf3a 100644
--- a/docs/postgres.md
+++ b/docs/postgres.md
@@ -72,8 +72,7 @@ underneath the database, or if a different version of the locale is used on any
replicas.
The safest way to fix the issue is to take a dump and recreate the database with
-the correct `COLLATE` and `CTYPE` parameters (as per
-[docs/postgres.md](docs/postgres.md)). It is also possible to change the
+the correct `COLLATE` and `CTYPE` parameters (as shown above). It is also possible to change the
parameters on a live database and run a `REINDEX` on the entire database,
however extreme care must be taken to avoid database corruption.
diff --git a/docs/turn-howto.md b/docs/turn-howto.md
index 1bd3943f54..1e121ead6a 100644
--- a/docs/turn-howto.md
+++ b/docs/turn-howto.md
@@ -113,7 +113,7 @@ Your home server configuration file needs the following extra keys:
As an example, here is the relevant section of the config file for matrix.org:
turn_uris: [ "turn:turn.matrix.org:3478?transport=udp", "turn:turn.matrix.org:3478?transport=tcp" ]
- turn_shared_secret: n0t4ctuAllymatr1Xd0TorgSshar3d5ecret4obvIousreAsons
+ turn_shared_secret: "n0t4ctuAllymatr1Xd0TorgSshar3d5ecret4obvIousreAsons"
turn_user_lifetime: 86400000
turn_allow_guests: True
diff --git a/synapse/__init__.py b/synapse/__init__.py
index 5b86008945..3bf2d02450 100644
--- a/synapse/__init__.py
+++ b/synapse/__init__.py
@@ -36,7 +36,7 @@ try:
except ImportError:
pass
-__version__ = "1.12.0"
+__version__ = "1.12.3"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when
diff --git a/synapse/app/generic_worker.py b/synapse/app/generic_worker.py
index b2c764bfe8..5363642d64 100644
--- a/synapse/app/generic_worker.py
+++ b/synapse/app/generic_worker.py
@@ -860,6 +860,9 @@ def start(config_options):
# Force the appservice to start since they will be disabled in the main config
config.notify_appservices = True
+ else:
+ # For other worker types we force this to off.
+ config.notify_appservices = False
if config.worker_app == "synapse.app.pusher":
if config.start_pushers:
@@ -873,6 +876,9 @@ def start(config_options):
# Force the pushers to start since they will be disabled in the main config
config.start_pushers = True
+ else:
+ # For other worker types we force this to off.
+ config.start_pushers = False
if config.worker_app == "synapse.app.user_dir":
if config.update_user_directory:
@@ -886,6 +892,9 @@ def start(config_options):
# Force the pushers to start since they will be disabled in the main config
config.update_user_directory = True
+ else:
+ # For other worker types we force this to off.
+ config.update_user_directory = False
if config.worker_app == "synapse.app.federation_sender":
if config.send_federation:
@@ -899,6 +908,9 @@ def start(config_options):
# Force the pushers to start since they will be disabled in the main config
config.send_federation = True
+ else:
+ # For other worker types we force this to off.
+ config.send_federation = False
synapse.events.USE_FROZEN_DICTS = config.use_frozen_dicts
diff --git a/synapse/storage/data_stores/main/devices.py b/synapse/storage/data_stores/main/devices.py
index d55733a4cd..8af5f7de54 100644
--- a/synapse/storage/data_stores/main/devices.py
+++ b/synapse/storage/data_stores/main/devices.py
@@ -317,14 +317,16 @@ class DeviceWorkerStore(SQLBaseStore):
key_json = device.get("key_json", None)
if key_json:
result["keys"] = db_to_json(key_json)
+
+ if "signatures" in device:
+ for sig_user_id, sigs in device["signatures"].items():
+ result["keys"].setdefault("signatures", {}).setdefault(
+ sig_user_id, {}
+ ).update(sigs)
+
device_display_name = device.get("device_display_name", None)
if device_display_name:
result["device_display_name"] = device_display_name
- if "signatures" in device:
- for sig_user_id, sigs in device["signatures"].items():
- result["keys"].setdefault("signatures", {}).setdefault(
- sig_user_id, {}
- ).update(sigs)
else:
result["deleted"] = True
@@ -525,14 +527,16 @@ class DeviceWorkerStore(SQLBaseStore):
key_json = device.get("key_json", None)
if key_json:
result["keys"] = db_to_json(key_json)
+
+ if "signatures" in device:
+ for sig_user_id, sigs in device["signatures"].items():
+ result["keys"].setdefault("signatures", {}).setdefault(
+ sig_user_id, {}
+ ).update(sigs)
+
device_display_name = device.get("device_display_name", None)
if device_display_name:
result["device_display_name"] = device_display_name
- if "signatures" in device:
- for sig_user_id, sigs in device["signatures"].items():
- result["keys"].setdefault("signatures", {}).setdefault(
- sig_user_id, {}
- ).update(sigs)
results.append(result)
diff --git a/synapse/storage/engines/sqlite.py b/synapse/storage/engines/sqlite.py
index 2bfeefd54e..3bc2e8b986 100644
--- a/synapse/storage/engines/sqlite.py
+++ b/synapse/storage/engines/sqlite.py
@@ -12,14 +12,17 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-import sqlite3
import struct
import threading
+import typing
from synapse.storage.engines import BaseDatabaseEngine
+if typing.TYPE_CHECKING:
+ import sqlite3 # noqa: F401
-class Sqlite3Engine(BaseDatabaseEngine[sqlite3.Connection]):
+
+class Sqlite3Engine(BaseDatabaseEngine["sqlite3.Connection"]):
def __init__(self, database_module, database_config):
super().__init__(database_module, database_config)
|
