summary refs log tree commit diff
path: root/synapse/storage
diff options
context:
space:
mode:
authorErik Johnston <erik@matrix.org>2018-10-29 17:34:34 +0000
committerErik Johnston <erik@matrix.org>2018-10-29 17:34:34 +0000
commit88e5ffe6fe816e54a5471728e93fde63353d9a70 (patch)
tree1a0c1eaf976ee3316f213e7c3948c3df09384c01 /synapse/storage
parentDon't truncate command name in metrics (diff)
downloadsynapse-88e5ffe6fe816e54a5471728e93fde63353d9a70.tar.xz
Deduplicate device updates sent over replication
We currently send several kHz of device list updates over replication
occisonally, which often causes the replications streams to lag and then
get dropped.

A lot of those updates will actually be duplicates, since we don't send
e.g. device_ids across replication, so let's deduplicate it when we pull
them out of the database.
Diffstat (limited to 'synapse/storage')
-rw-r--r--synapse/storage/devices.py6
1 files changed, 5 insertions, 1 deletions
diff --git a/synapse/storage/devices.py b/synapse/storage/devices.py
index d10ff9e4b9..62497ab63f 100644
--- a/synapse/storage/devices.py
+++ b/synapse/storage/devices.py
@@ -589,10 +589,14 @@ class DeviceStore(SQLBaseStore):
         combined list of changes to devices, and which destinations need to be
         poked. `destination` may be None if no destinations need to be poked.
         """
+        # We do a group by here as there can be a large number of duplicate
+        # entries, since we throw away device IDs.
         sql = """
-            SELECT stream_id, user_id, destination FROM device_lists_stream
+            SELECT MAX(stream_id) AS stream_id, user_id, destination
+            FROM device_lists_stream
             LEFT JOIN device_lists_outbound_pokes USING (stream_id, user_id, device_id)
             WHERE ? < stream_id AND stream_id <= ?
+            GROUP BY user_id, destination
         """
         return self._execute(
             "get_all_device_list_changes_for_remotes", None,