summary refs log tree commit diff
path: root/docs/development/synapse_architecture/replication.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/development/synapse_architecture/replication.md')
-rw-r--r--docs/development/synapse_architecture/replication.md42
1 files changed, 42 insertions, 0 deletions
diff --git a/docs/development/synapse_architecture/replication.md b/docs/development/synapse_architecture/replication.md
new file mode 100644

index 0000000000..108da9a065 --- /dev/null +++ b/docs/development/synapse_architecture/replication.md
@@ -0,0 +1,42 @@ +# Replication Architecture + +## Motivation + +We'd like to be able to split some of the work that synapse does into +multiple python processes. In theory multiple synapse processes could +share a single postgresql database and we\'d scale up by running more +synapse processes. However much of synapse assumes that only one process +is interacting with the database, both for assigning unique identifiers +when inserting into tables, notifying components about new updates, and +for invalidating its caches. + +So running multiple copies of the current code isn't an option. One way +to run multiple processes would be to have a single writer process and +multiple reader processes connected to the same database. In order to do +this we'd need a way for the reader process to invalidate its in-memory +caches when an update happens on the writer. One way to do this is for +the writer to present an append-only log of updates which the readers +can consume to invalidate their caches and to push updates to listening +clients or pushers. + +Synapse already stores much of its data as an append-only log so that it +can correctly respond to `/sync` requests so the amount of code changes +needed to expose the append-only log to the readers should be fairly +minimal. + +## Architecture + +### The Replication Protocol + +See [the TCP replication documentation](tcp_replication.md). + +### The Slaved DataStore + +There are read-only version of the synapse storage layer in +`synapse/replication/slave/storage` that use the response of the +replication API to invalidate their caches. + +### The TCP Replication Module +Information about how the tcp replication module is structured, including how +the classes interact, can be found in +`synapse/replication/tcp/__init__.py`