summary refs log tree commit diff
path: root/docs/development/synapse_architecture/replication.md
diff options
context:
space:
mode:
authorAndrew Morgan <andrew@amorgan.xyz>2021-11-21 02:39:02 +0000
committerAndrew Morgan <andrew@amorgan.xyz>2021-11-21 02:39:02 +0000
commit1a8406f42ac3a7f63690c13cb8d8fd5547928632 (patch)
treeb87d0c0ca33f6f2eea33cf953aef9767183c702f /docs/development/synapse_architecture/replication.md
parentUpdate README.md (diff)
downloadsynapse-1a8406f42ac3a7f63690c13cb8d8fd5547928632.tar.xz
Move documentation files to their place in the hierarchy; update SUMMARY.md github/anoa/doc_hierarchy anoa/doc_hierarchy
Diffstat (limited to 'docs/development/synapse_architecture/replication.md')
-rw-r--r--docs/development/synapse_architecture/replication.md37
1 files changed, 37 insertions, 0 deletions
diff --git a/docs/development/synapse_architecture/replication.md b/docs/development/synapse_architecture/replication.md
new file mode 100644

index 0000000000..e82df0de8a --- /dev/null +++ b/docs/development/synapse_architecture/replication.md
@@ -0,0 +1,37 @@ +# Replication Architecture + +## Motivation + +We'd like to be able to split some of the work that synapse does into +multiple python processes. In theory multiple synapse processes could +share a single postgresql database and we\'d scale up by running more +synapse processes. However much of synapse assumes that only one process +is interacting with the database, both for assigning unique identifiers +when inserting into tables, notifying components about new updates, and +for invalidating its caches. + +So running multiple copies of the current code isn't an option. One way +to run multiple processes would be to have a single writer process and +multiple reader processes connected to the same database. In order to do +this we'd need a way for the reader process to invalidate its in-memory +caches when an update happens on the writer. One way to do this is for +the writer to present an append-only log of updates which the readers +can consume to invalidate their caches and to push updates to listening +clients or pushers. + +Synapse already stores much of its data as an append-only log so that it +can correctly respond to `/sync` requests so the amount of code changes +needed to expose the append-only log to the readers should be fairly +minimal. + +## Architecture + +### The Replication Protocol + +See [the TCP replication documentation](tcp_replication.md). + +### The Slaved DataStore + +There are read-only version of the synapse storage layer in +`synapse/replication/slave/storage` that use the response of the +replication API to invalidate their caches.