diff --git a/CHANGES.md b/CHANGES.md
index 4c413b72ee..ab6fce3e7d 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -1,3 +1,50 @@
+Synapse 1.10.0rc1 (2020-01-31)
+==============================
+
+**WARNING**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details.
+
+
+Features
+--------
+
+- Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794))
+
+
+Bugfixes
+--------
+
+- Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734))
+- Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761))
+- Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767))
+- Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771))
+- Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795))
+- Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796))
+- Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800))
+- Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801))
+- Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811))
+
+
+Improved Documentation
+----------------------
+
+- Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775))
+
+
+Internal Changes
+----------------
+
+- Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810))
+- Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748))
+- Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751))
+- Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757))
+- Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819))
+- Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786))
+- Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792))
+- When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797))
+- Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816))
+- Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820))
+
+
Synapse 1.9.1 (2020-01-28)
==========================
diff --git a/UPGRADE.rst b/UPGRADE.rst
index 470246f128..3cad8c2837 100644
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -76,13 +76,13 @@ for example:
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
-Upgrading to **<NEXT_VERSION>**
-===============================
+Upgrading to v1.10.0
+====================
Synapse will now log a warning on start up if used with a PostgreSQL database
that has a non-recommended locale set.
-See [docs/postgres.md](docs/postgres.md) for details.
+See `docs/postgres.md <docs/postgres.md>`_ for details.
Upgrading to v1.8.0
diff --git a/changelog.d/6729.misc b/changelog.d/6729.misc
deleted file mode 100644
index 5537355bea..0000000000
--- a/changelog.d/6729.misc
+++ /dev/null
@@ -1 +0,0 @@
-Record room versions in the `rooms` table.
diff --git a/changelog.d/6734.bugfix b/changelog.d/6734.bugfix
deleted file mode 100644
index 79c6bab4d1..0000000000
--- a/changelog.d/6734.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS).
diff --git a/changelog.d/6748.misc b/changelog.d/6748.misc
deleted file mode 100644
index de320d4cd9..0000000000
--- a/changelog.d/6748.misc
+++ /dev/null
@@ -1 +0,0 @@
-Propagate cache invalidates from workers to other workers.
diff --git a/changelog.d/6751.misc b/changelog.d/6751.misc
deleted file mode 100644
index 7222520528..0000000000
--- a/changelog.d/6751.misc
+++ /dev/null
@@ -1 +0,0 @@
-Remove some unnecessary admin handler abstraction methods.
\ No newline at end of file
diff --git a/changelog.d/6757.misc b/changelog.d/6757.misc
deleted file mode 100644
index a50c5e974a..0000000000
--- a/changelog.d/6757.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add some debugging for media storage providers.
diff --git a/changelog.d/6761.bugfix b/changelog.d/6761.bugfix
deleted file mode 100644
index 1c664c02df..0000000000
--- a/changelog.d/6761.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Minor fixes to `PUT /_synapse/admin/v2/users` admin api.
diff --git a/changelog.d/6767.bugfix b/changelog.d/6767.bugfix
deleted file mode 100644
index 63c7c63315..0000000000
--- a/changelog.d/6767.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release.
diff --git a/changelog.d/6771.bugfix b/changelog.d/6771.bugfix
deleted file mode 100644
index 623ba24acb..0000000000
--- a/changelog.d/6771.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key).
diff --git a/changelog.d/6775.doc b/changelog.d/6775.doc
deleted file mode 100644
index c6078ef82d..0000000000
--- a/changelog.d/6775.doc
+++ /dev/null
@@ -1 +0,0 @@
-Clarify documentation related to `user_dir` and `federation_reader` workers.
diff --git a/changelog.d/6776.misc b/changelog.d/6776.misc
deleted file mode 100644
index 4f9a4ac7a5..0000000000
--- a/changelog.d/6776.misc
+++ /dev/null
@@ -1 +0,0 @@
-Detect unknown remote devices and mark cache as stale.
diff --git a/changelog.d/6786.misc b/changelog.d/6786.misc
deleted file mode 100644
index 94c692e53a..0000000000
--- a/changelog.d/6786.misc
+++ /dev/null
@@ -1 +0,0 @@
-Attempt to resync remote users' devices when detected as stale.
diff --git a/changelog.d/6787.feature b/changelog.d/6787.feature
deleted file mode 100644
index df9e4b77ab..0000000000
--- a/changelog.d/6787.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260).
diff --git a/changelog.d/6788.misc b/changelog.d/6788.misc
deleted file mode 100644
index 5537355bea..0000000000
--- a/changelog.d/6788.misc
+++ /dev/null
@@ -1 +0,0 @@
-Record room versions in the `rooms` table.
diff --git a/changelog.d/6790.feature b/changelog.d/6790.feature
deleted file mode 100644
index df9e4b77ab..0000000000
--- a/changelog.d/6790.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260).
diff --git a/changelog.d/6792.misc b/changelog.d/6792.misc
deleted file mode 100644
index fa31d509b3..0000000000
--- a/changelog.d/6792.misc
+++ /dev/null
@@ -1 +0,0 @@
-Delete current state from the database when server leaves a room.
diff --git a/changelog.d/6794.feature b/changelog.d/6794.feature
deleted file mode 100644
index df9e4b77ab..0000000000
--- a/changelog.d/6794.feature
+++ /dev/null
@@ -1 +0,0 @@
-Implement updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260).
diff --git a/changelog.d/6795.bugfix b/changelog.d/6795.bugfix
deleted file mode 100644
index d1585653b1..0000000000
--- a/changelog.d/6795.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix outbound federation request metrics.
diff --git a/changelog.d/6796.bugfix b/changelog.d/6796.bugfix
deleted file mode 100644
index 206a157311..0000000000
--- a/changelog.d/6796.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device.
diff --git a/changelog.d/6797.misc b/changelog.d/6797.misc
deleted file mode 100644
index e9127bac51..0000000000
--- a/changelog.d/6797.misc
+++ /dev/null
@@ -1 +0,0 @@
-When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale.
diff --git a/changelog.d/6799.bugfix b/changelog.d/6799.bugfix
deleted file mode 100644
index 322a2758af..0000000000
--- a/changelog.d/6799.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix race in federation sender worker that delayed sending of device updates.
diff --git a/changelog.d/6800.bugfix b/changelog.d/6800.bugfix
deleted file mode 100644
index 322a2758af..0000000000
--- a/changelog.d/6800.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix race in federation sender worker that delayed sending of device updates.
diff --git a/changelog.d/6801.bugfix b/changelog.d/6801.bugfix
deleted file mode 100644
index f401fa5d69..0000000000
--- a/changelog.d/6801.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room.
diff --git a/changelog.d/6802.misc b/changelog.d/6802.misc
deleted file mode 100644
index a77ba1d7a5..0000000000
--- a/changelog.d/6802.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add background update to clean out left rooms from current state.
diff --git a/changelog.d/6803.misc b/changelog.d/6803.misc
deleted file mode 100644
index 08aa80bcd9..0000000000
--- a/changelog.d/6803.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactoring work in preparation for changing the event redaction algorithm.
diff --git a/changelog.d/6805.misc b/changelog.d/6805.misc
deleted file mode 100644
index 08aa80bcd9..0000000000
--- a/changelog.d/6805.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactoring work in preparation for changing the event redaction algorithm.
diff --git a/changelog.d/6806.misc b/changelog.d/6806.misc
deleted file mode 100644
index 08aa80bcd9..0000000000
--- a/changelog.d/6806.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactoring work in preparation for changing the event redaction algorithm.
diff --git a/changelog.d/6807.misc b/changelog.d/6807.misc
deleted file mode 100644
index 08aa80bcd9..0000000000
--- a/changelog.d/6807.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactoring work in preparation for changing the event redaction algorithm.
diff --git a/changelog.d/6810.misc b/changelog.d/6810.misc
deleted file mode 100644
index 5537355bea..0000000000
--- a/changelog.d/6810.misc
+++ /dev/null
@@ -1 +0,0 @@
-Record room versions in the `rooms` table.
diff --git a/changelog.d/6811.bugfix b/changelog.d/6811.bugfix
deleted file mode 100644
index 361f2fc2e8..0000000000
--- a/changelog.d/6811.bugfix
+++ /dev/null
@@ -1 +0,0 @@
-Fix waking up other workers when remote server is detected to have come back online.
diff --git a/changelog.d/6816.misc b/changelog.d/6816.misc
deleted file mode 100644
index a77ba1d7a5..0000000000
--- a/changelog.d/6816.misc
+++ /dev/null
@@ -1 +0,0 @@
-Add background update to clean out left rooms from current state.
diff --git a/changelog.d/6819.misc b/changelog.d/6819.misc
deleted file mode 100644
index 4f9a4ac7a5..0000000000
--- a/changelog.d/6819.misc
+++ /dev/null
@@ -1 +0,0 @@
-Detect unknown remote devices and mark cache as stale.
diff --git a/changelog.d/6820.misc b/changelog.d/6820.misc
deleted file mode 100644
index 08aa80bcd9..0000000000
--- a/changelog.d/6820.misc
+++ /dev/null
@@ -1 +0,0 @@
-Refactoring work in preparation for changing the event redaction algorithm.
diff --git a/changelog.d/6844.bugfix b/changelog.d/6844.bugfix
new file mode 100644
index 0000000000..e84aa1029f
--- /dev/null
+++ b/changelog.d/6844.bugfix
@@ -0,0 +1 @@
+Fix an issue with cross-signing where device signatures were not sent to remote servers.
diff --git a/changelog.d/6848.bugfix b/changelog.d/6848.bugfix
new file mode 100644
index 0000000000..65688e5d57
--- /dev/null
+++ b/changelog.d/6848.bugfix
@@ -0,0 +1 @@
+Fix detecting unknown devices from remote encrypted events.
diff --git a/changelog.d/6850.misc b/changelog.d/6850.misc
new file mode 100644
index 0000000000..418569113f
--- /dev/null
+++ b/changelog.d/6850.misc
@@ -0,0 +1 @@
+Detect unexpected sender keys on inbound encrypted events and resync device lists.
diff --git a/synapse/__init__.py b/synapse/__init__.py
index a236888d3c..bd942d3e1c 100644
--- a/synapse/__init__.py
+++ b/synapse/__init__.py
@@ -36,7 +36,7 @@ try:
except ImportError:
pass
-__version__ = "1.9.1"
+__version__ = "1.10.0rc1"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when
diff --git a/synapse/api/constants.py b/synapse/api/constants.py
index 0ade47e624..cc8577552b 100644
--- a/synapse/api/constants.py
+++ b/synapse/api/constants.py
@@ -77,12 +77,11 @@ class EventTypes(object):
Aliases = "m.room.aliases"
Redaction = "m.room.redaction"
ThirdPartyInvite = "m.room.third_party_invite"
- Encryption = "m.room.encryption"
RelatedGroups = "m.room.related_groups"
RoomHistoryVisibility = "m.room.history_visibility"
CanonicalAlias = "m.room.canonical_alias"
- Encryption = "m.room.encryption"
+ Encrypted = "m.room.encrypted"
RoomAvatar = "m.room.avatar"
RoomEncryption = "m.room.encryption"
GuestAccess = "m.room.guest_access"
diff --git a/synapse/handlers/device.py b/synapse/handlers/device.py
index 26ef5e150c..a9bd431486 100644
--- a/synapse/handlers/device.py
+++ b/synapse/handlers/device.py
@@ -598,7 +598,13 @@ class DeviceListUpdater(object):
# happens if we've missed updates.
resync = yield self._need_to_do_resync(user_id, pending_updates)
- logger.debug("Need to re-sync devices for %r? %r", user_id, resync)
+ if logger.isEnabledFor(logging.INFO):
+ logger.info(
+ "Received device list update for %s, requiring resync: %s. Devices: %s",
+ user_id,
+ resync,
+ ", ".join(u[0] for u in pending_updates),
+ )
if resync:
yield self.user_device_resync(user_id)
diff --git a/synapse/handlers/federation.py b/synapse/handlers/federation.py
index c86d3177e9..e9441bbeff 100644
--- a/synapse/handlers/federation.py
+++ b/synapse/handlers/federation.py
@@ -752,29 +752,75 @@ class FederationHandler(BaseHandler):
# For encrypted messages we check that we know about the sending device,
# if we don't then we mark the device cache for that user as stale.
- if event.type == EventTypes.Encryption:
+ if event.type == EventTypes.Encrypted:
device_id = event.content.get("device_id")
+ sender_key = event.content.get("sender_key")
+
+ cached_devices = await self.store.get_cached_devices_for_user(event.sender)
+
+ resync = False # Whether we should resync device lists.
+
+ device = None
if device_id is not None:
- cached_devices = await self.store.get_cached_devices_for_user(
- event.sender
- )
- if device_id not in cached_devices:
+ device = cached_devices.get(device_id)
+ if device is None:
logger.info(
"Received event from remote device not in our cache: %s %s",
event.sender,
device_id,
)
- await self.store.mark_remote_user_device_cache_as_stale(
- event.sender
+ resync = True
+
+ # We also check if the `sender_key` matches what we expect.
+ if sender_key is not None:
+ # Figure out what sender key we're expecting. If we know the
+ # device and recognize the algorithm then we can work out the
+ # exact key to expect. Otherwise check it matches any key we
+ # have for that device.
+ if device:
+ keys = device.get("keys", {}).get("keys", {})
+
+ if event.content.get("algorithm") == "m.megolm.v1.aes-sha2":
+ # For this algorithm we expect a curve25519 key.
+ key_name = "curve25519:%s" % (device_id,)
+ current_keys = [keys.get(key_name)]
+ else:
+ # We don't know understand the algorithm, so we just
+ # check it matches a key for the device.
+ current_keys = keys.values()
+ elif device_id:
+ # We don't have any keys for the device ID.
+ current_keys = []
+ else:
+ # The event didn't include a device ID, so we just look for
+ # keys across all devices.
+ current_keys = (
+ key
+ for device in cached_devices
+ for key in device.get("keys", {}).get("keys", {}).values()
)
- # Immediately attempt a resync in the background
- if self.config.worker_app:
- return run_in_background(self._user_device_resync, event.sender)
- else:
- return run_in_background(
- self._device_list_updater.user_device_resync, event.sender
- )
+ # We now check that the sender key matches (one of) the expected
+ # keys.
+ if sender_key not in current_keys:
+ logger.info(
+ "Received event from remote device with unexpected sender key: %s %s: %s",
+ event.sender,
+ device_id or "<no device_id>",
+ sender_key,
+ )
+ resync = True
+
+ if resync:
+ await self.store.mark_remote_user_device_cache_as_stale(event.sender)
+
+ # Immediately attempt a resync in the background
+ if self.config.worker_app:
+ return run_in_background(self._user_device_resync, event.sender)
+ else:
+ return run_in_background(
+ self._device_list_updater.user_device_resync, event.sender
+ )
@log_function
async def backfill(self, dest, room_id, limit, extremities):
diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py
index 1382399557..b609a65f47 100644
--- a/synapse/handlers/room.py
+++ b/synapse/handlers/room.py
@@ -360,7 +360,7 @@ class RoomCreationHandler(BaseHandler):
(EventTypes.RoomHistoryVisibility, ""),
(EventTypes.GuestAccess, ""),
(EventTypes.RoomAvatar, ""),
- (EventTypes.Encryption, ""),
+ (EventTypes.RoomEncryption, ""),
(EventTypes.ServerACL, ""),
(EventTypes.RelatedGroups, ""),
(EventTypes.PowerLevels, ""),
diff --git a/synapse/handlers/stats.py b/synapse/handlers/stats.py
index 7f7d56390e..68e6edace5 100644
--- a/synapse/handlers/stats.py
+++ b/synapse/handlers/stats.py
@@ -286,7 +286,7 @@ class StatsHandler(StateDeltasHandler):
room_state["history_visibility"] = event_content.get(
"history_visibility"
)
- elif typ == EventTypes.Encryption:
+ elif typ == EventTypes.RoomEncryption:
room_state["encryption"] = event_content.get("algorithm")
elif typ == EventTypes.Name:
room_state["name"] = event_content.get("name")
diff --git a/synapse/storage/data_stores/main/devices.py b/synapse/storage/data_stores/main/devices.py
index ea0503476f..b7617efb80 100644
--- a/synapse/storage/data_stores/main/devices.py
+++ b/synapse/storage/data_stores/main/devices.py
@@ -320,6 +320,11 @@ class DeviceWorkerStore(SQLBaseStore):
device_display_name = device.get("device_display_name", None)
if device_display_name:
result["device_display_name"] = device_display_name
+ if "signatures" in device:
+ for sig_user_id, sigs in device["signatures"].items():
+ result["keys"].setdefault("signatures", {}).setdefault(
+ sig_user_id, {}
+ ).update(sigs)
else:
result["deleted"] = True
@@ -524,6 +529,11 @@ class DeviceWorkerStore(SQLBaseStore):
device_display_name = device.get("device_display_name", None)
if device_display_name:
result["device_display_name"] = device_display_name
+ if "signatures" in device:
+ for sig_user_id, sigs in device["signatures"].items():
+ result["keys"].setdefault("signatures", {}).setdefault(
+ sig_user_id, {}
+ ).update(sigs)
results.append(result)
diff --git a/synapse/storage/data_stores/main/stats.py b/synapse/storage/data_stores/main/stats.py
index 7bc186e9a1..7af1495e47 100644
--- a/synapse/storage/data_stores/main/stats.py
+++ b/synapse/storage/data_stores/main/stats.py
@@ -744,7 +744,7 @@ class StatsStore(StateDeltasStore):
EventTypes.Create,
EventTypes.JoinRules,
EventTypes.RoomHistoryVisibility,
- EventTypes.Encryption,
+ EventTypes.RoomEncryption,
EventTypes.Name,
EventTypes.Topic,
EventTypes.RoomAvatar,
@@ -816,7 +816,7 @@ class StatsStore(StateDeltasStore):
room_state["history_visibility"] = event.content.get(
"history_visibility"
)
- elif event.type == EventTypes.Encryption:
+ elif event.type == EventTypes.RoomEncryption:
room_state["encryption"] = event.content.get("algorithm")
elif event.type == EventTypes.Name:
room_state["name"] = event.content.get("name")
|
