From 30272b99d01e5239602de0c165a99daad2fb79ca Mon Sep 17 00:00:00 2001 From: David Robertson Date: Mon, 24 Oct 2022 14:14:10 +0100 Subject: Move old architecture docs --- docs/ancient_architecture_notes.md | 81 ---------------------- docs/architecture.md | 65 ----------------- .../old/ancient_architecture_notes.md | 81 ++++++++++++++++++++++ .../synapse_architecture/old/architecture.md | 65 +++++++++++++++++ 4 files changed, 146 insertions(+), 146 deletions(-) delete mode 100644 docs/ancient_architecture_notes.md delete mode 100644 docs/architecture.md create mode 100644 docs/development/synapse_architecture/old/ancient_architecture_notes.md create mode 100644 docs/development/synapse_architecture/old/architecture.md diff --git a/docs/ancient_architecture_notes.md b/docs/ancient_architecture_notes.md deleted file mode 100644 index 07bb199d7a..0000000000 --- a/docs/ancient_architecture_notes.md +++ /dev/null @@ -1,81 +0,0 @@ -> **Warning** -> These architecture notes are spectacularly old, and date back -> to when Synapse was just federation code in isolation. This should be -> merged into the main spec. - -# Server to Server - -## Server to Server Stack - -To use the server to server stack, homeservers should only need to -interact with the Messaging layer. - -The server to server side of things is designed into 4 distinct layers: - -1. Messaging Layer -2. Pdu Layer -3. Transaction Layer -4. Transport Layer - -Where the bottom (the transport layer) is what talks to the internet via -HTTP, and the top (the messaging layer) talks to the rest of the Home -Server with a domain specific API. - -1. **Messaging Layer** - - This is what the rest of the homeserver hits to send messages, join rooms, - etc. It also allows you to register callbacks for when it get's notified by - lower levels that e.g. a new message has been received. - - It is responsible for serializing requests to send to the data - layer, and to parse requests received from the data layer. - -2. **PDU Layer** - - This layer handles: - - - duplicate `pdu_id`'s - i.e., it makes sure we ignore them. - - responding to requests for a given `pdu_id` - - responding to requests for all metadata for a given context (i.e. room) - - handling incoming backfill requests - - So it has to parse incoming messages to discover which are metadata and - which aren't, and has to correctly clobber existing metadata where - appropriate. - - For incoming PDUs, it has to check the PDUs it references to see - if we have missed any. If we have go and ask someone (another - homeserver) for it. - -3. **Transaction Layer** - - This layer makes incoming requests idempotent. i.e., it stores - which transaction id's we have seen and what our response were. - If we have already seen a message with the given transaction id, - we do not notify higher levels but simply respond with the - previous response. - - `transaction_id` is from "`GET /send//`" - - It's also responsible for batching PDUs into single transaction for - sending to remote destinations, so that we only ever have one - transaction in flight to a given destination at any one time. - - This is also responsible for answering requests for things after a - given set of transactions, i.e., ask for everything after 'ver' X. - -4. **Transport Layer** - - This is responsible for starting a HTTP server and hitting the - correct callbacks on the Transaction layer, as well as sending - both data and requests for data. - -## Persistence - -We persist things in a single sqlite3 database. All database queries get -run on a separate, dedicated thread. This that we only ever have one -query running at a time, making it a lot easier to do things in a safe -manner. - -The queries are located in the `synapse.persistence.transactions` module, -and the table information in the `synapse.persistence.tables` module. diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 0c7f315f3f..0000000000 --- a/docs/architecture.md +++ /dev/null @@ -1,65 +0,0 @@ -# Synapse Architecture - -As of the end of Oct 2014, Synapse's overall architecture looks like: - - synapse - .-----------------------------------------------------. - | Notifier | - | ^ | | - | | | | - | .------------|------. | - | | handlers/ | | | - | | v | | - | | Event*Handler <--------> rest/* <=> Client - | | Rooms*Handler | | - HS <=> federation/* <==> FederationHandler | | - | | | PresenceHandler | | - | | | TypingHandler | | - | | '-------------------' | - | | | | | - | | state/* | | - | | | | | - | | v v | - | `--------------> storage/* | - | | | - '--------------------------|--------------------------' - v - .----. - | DB | - '----' - -- Handlers: business logic of synapse itself. Follows a set contract of BaseHandler: - - BaseHandler gives us onNewRoomEvent which: (TODO: flesh this out and make it less cryptic): - - handle_state(event) - - auth(event) - - persist_event(event) - - notify notifier or federation(event) - - PresenceHandler: use distributor to get EDUs out of Federation. - Very lightweight logic built on the distributor - - TypingHandler: use distributor to get EDUs out of Federation. - Very lightweight logic built on the distributor - - EventsHandler: handles the events stream... - - FederationHandler: - gets PDU from Federation Layer; turns into - an event; follows basehandler functionality. - - RoomsHandler: does all the room logic, including members - lots - of classes in RoomsHandler. - - ProfileHandler: talks to the storage to store/retrieve profile - info. -- EventFactory: generates events of particular event types. -- Notifier: Backs the events handler -- REST: Interfaces handlers and events to the outside world via - HTTP/JSON. Converts events back and forth from JSON. -- Federation: holds the HTTP client & server to talk to other servers. - Does replication to make sure there's nothing missing in the graph. - Handles reliability. Handles txns. -- Distributor: generic event bus. used for presence & typing only - currently. Notifier could be implemented using Distributor - so far - we are only using for things which actually /require/ dynamic - pluggability however as it can obfuscate the actual flow of control. -- Auth: helper singleton to say whether a given event is allowed to do - a given thing (TODO: put this on the diagram) -- State: helper singleton: does state conflict resolution. You give it - an event and it tells you if it actually updates the state or not, - and annotates the event up properly and handles merge conflict - resolution. -- Storage: abstracts the storage engine. diff --git a/docs/development/synapse_architecture/old/ancient_architecture_notes.md b/docs/development/synapse_architecture/old/ancient_architecture_notes.md new file mode 100644 index 0000000000..07bb199d7a --- /dev/null +++ b/docs/development/synapse_architecture/old/ancient_architecture_notes.md @@ -0,0 +1,81 @@ +> **Warning** +> These architecture notes are spectacularly old, and date back +> to when Synapse was just federation code in isolation. This should be +> merged into the main spec. + +# Server to Server + +## Server to Server Stack + +To use the server to server stack, homeservers should only need to +interact with the Messaging layer. + +The server to server side of things is designed into 4 distinct layers: + +1. Messaging Layer +2. Pdu Layer +3. Transaction Layer +4. Transport Layer + +Where the bottom (the transport layer) is what talks to the internet via +HTTP, and the top (the messaging layer) talks to the rest of the Home +Server with a domain specific API. + +1. **Messaging Layer** + + This is what the rest of the homeserver hits to send messages, join rooms, + etc. It also allows you to register callbacks for when it get's notified by + lower levels that e.g. a new message has been received. + + It is responsible for serializing requests to send to the data + layer, and to parse requests received from the data layer. + +2. **PDU Layer** + + This layer handles: + + - duplicate `pdu_id`'s - i.e., it makes sure we ignore them. + - responding to requests for a given `pdu_id` + - responding to requests for all metadata for a given context (i.e. room) + - handling incoming backfill requests + + So it has to parse incoming messages to discover which are metadata and + which aren't, and has to correctly clobber existing metadata where + appropriate. + + For incoming PDUs, it has to check the PDUs it references to see + if we have missed any. If we have go and ask someone (another + homeserver) for it. + +3. **Transaction Layer** + + This layer makes incoming requests idempotent. i.e., it stores + which transaction id's we have seen and what our response were. + If we have already seen a message with the given transaction id, + we do not notify higher levels but simply respond with the + previous response. + + `transaction_id` is from "`GET /send//`" + + It's also responsible for batching PDUs into single transaction for + sending to remote destinations, so that we only ever have one + transaction in flight to a given destination at any one time. + + This is also responsible for answering requests for things after a + given set of transactions, i.e., ask for everything after 'ver' X. + +4. **Transport Layer** + + This is responsible for starting a HTTP server and hitting the + correct callbacks on the Transaction layer, as well as sending + both data and requests for data. + +## Persistence + +We persist things in a single sqlite3 database. All database queries get +run on a separate, dedicated thread. This that we only ever have one +query running at a time, making it a lot easier to do things in a safe +manner. + +The queries are located in the `synapse.persistence.transactions` module, +and the table information in the `synapse.persistence.tables` module. diff --git a/docs/development/synapse_architecture/old/architecture.md b/docs/development/synapse_architecture/old/architecture.md new file mode 100644 index 0000000000..0c7f315f3f --- /dev/null +++ b/docs/development/synapse_architecture/old/architecture.md @@ -0,0 +1,65 @@ +# Synapse Architecture + +As of the end of Oct 2014, Synapse's overall architecture looks like: + + synapse + .-----------------------------------------------------. + | Notifier | + | ^ | | + | | | | + | .------------|------. | + | | handlers/ | | | + | | v | | + | | Event*Handler <--------> rest/* <=> Client + | | Rooms*Handler | | + HS <=> federation/* <==> FederationHandler | | + | | | PresenceHandler | | + | | | TypingHandler | | + | | '-------------------' | + | | | | | + | | state/* | | + | | | | | + | | v v | + | `--------------> storage/* | + | | | + '--------------------------|--------------------------' + v + .----. + | DB | + '----' + +- Handlers: business logic of synapse itself. Follows a set contract of BaseHandler: + - BaseHandler gives us onNewRoomEvent which: (TODO: flesh this out and make it less cryptic): + - handle_state(event) + - auth(event) + - persist_event(event) + - notify notifier or federation(event) + - PresenceHandler: use distributor to get EDUs out of Federation. + Very lightweight logic built on the distributor + - TypingHandler: use distributor to get EDUs out of Federation. + Very lightweight logic built on the distributor + - EventsHandler: handles the events stream... + - FederationHandler: - gets PDU from Federation Layer; turns into + an event; follows basehandler functionality. + - RoomsHandler: does all the room logic, including members - lots + of classes in RoomsHandler. + - ProfileHandler: talks to the storage to store/retrieve profile + info. +- EventFactory: generates events of particular event types. +- Notifier: Backs the events handler +- REST: Interfaces handlers and events to the outside world via + HTTP/JSON. Converts events back and forth from JSON. +- Federation: holds the HTTP client & server to talk to other servers. + Does replication to make sure there's nothing missing in the graph. + Handles reliability. Handles txns. +- Distributor: generic event bus. used for presence & typing only + currently. Notifier could be implemented using Distributor - so far + we are only using for things which actually /require/ dynamic + pluggability however as it can obfuscate the actual flow of control. +- Auth: helper singleton to say whether a given event is allowed to do + a given thing (TODO: put this on the diagram) +- State: helper singleton: does state conflict resolution. You give it + an event and it tells you if it actually updates the state or not, + and annotates the event up properly and handles merge conflict + resolution. +- Storage: abstracts the storage engine. -- cgit 1.5.1