Reference Matrix Home Server

72 files changed, 3801 insertions, 0 deletions

diff --git a/docs/client-server/specification b/docs/client-server/specification
new file mode 100644
index 0000000000..7df2bb14c5
--- /dev/null
+++ b/docs/client-server/specification
@@ -0,0 +1,1264 @@
+=========================
+Synapse Client-Server API
+=========================
+
+The following specification outlines how a client can send and receive data from 
+a home server.
+
+[[TODO(kegan): 4/7/14 Grilling
+- Mechanism for getting historical state changes (e.g. topic updates) - add 
+  query param flag?
+- Generic mechanism for linking first class events (e.g. feedback) with other s
+  first class events (e.g. messages)?
+- Generic mechanism for updating 'stuff about the room' (e.g. favourite coffee) 
+  AND specifying clobbering rules (clobber/add to list/etc)?
+- How to ensure a consistent view for clients paginating through room lists? 
+  They aren't really ordered in any way, and if you're paginating
+  through them, how can you show them a consistent result set? Temporary 'room 
+  list versions' akin to event version? How does that work?
+]]
+
+[[TODO(kegan):
+Outstanding problems / missing spec:
+- Push
+- Typing notifications
+]]
+
+Terminology
+-----------
+Stream Tokens: 
+An opaque token used to make further streaming requests. When using any 
+pagination streaming API, responses will contain a start and end stream token. 
+When reconnecting to the stream, these tokens can be used to tell the server 
+where the client got up to in the stream.
+
+Event ID:
+Every event that comes down the event stream or that is returned from the REST
+API has an associated event ID (event_id). This ID will be the same between the 
+REST API and the event stream, so any duplicate events can be clobbered 
+correctly without knowing anything else about the event.
+
+Message ID:
+The ID of a message sent by a client in a room. Clients send IMs to each other 
+in rooms. Each IM sent by a client must have a unique message ID which is unique
+for that particular client.
+
+User ID:
+The @username:host style ID of the client. When registering for an account, the 
+client specifies their username. The user_id is this username along with the 
+home server's unique hostname. When federating between home servers, the user_id
+is used to uniquely identify users across multiple home servers.
+
+Room ID:
+The room_id@host style ID for the room. When rooms are created, the client either
+specifies or is allocated a room ID. This room ID must be used to send messages 
+in that room. Like with clients, there may be multiple rooms with the same ID 
+across multiple home servers. The room_id is used to uniquely identify a room 
+when federating.
+
+Global message ID:
+The globally unique ID for a message. This ID is formed from the msg_id, the 
+client's user_id and the room_id. This uniquely identifies any 
+message. It is represented with '-' as the delimeter between IDs. The 
+global_msg_id is of the form: room_id-user_id-msg_id
+
+
+REST API and the Event Stream
+-----------------------------
+Clients send data to the server via a RESTful API. They can receive data via 
+this API or from an event stream. An event stream is a special path which 
+streams all events the client may be interested in. This makes it easy to 
+immediately receive updates from the REST API. All data is represented as JSON.
+
+Pagination streaming API
+========================
+Clients are often interested in very large datasets. The data itself could
+be 1000s of messages in a given room, 1000s of rooms in a public room list, or
+1000s of events (presence, typing, messages, etc) in the system. It is not
+practical to send vast quantities of data to the client every time they
+request a list of public rooms for example. There needs to be a way to show a
+subset of this data, and apply various filters to it. This is what the pagination
+streaming API is. This API defines standard request/response parameters which 
+can be used when navigating this stream of data.
+
+Pagination Request Query Parameters
+-----------------------------------
+Clients may wish to paginate results from the event stream, or other sources of 
+information where the amount of information may be a problem,
+e.g. in a room with 10,000s messages. The pagination query parameters provide a 
+way to navigate a 'window' around a large set of data. These
+parameters are only valid for GET requests.
+       
+        S e r v e r - s i d e   d a t a
+ |-------------------------------------------------|
+START      ^               ^                      END
+           |_______________|
+                   |
+            Client-extraction
+
+'START' and 'END' are magic token values which specify the start and end of the 
+dataset respectively.
+
+Query parameters:
+  from : $streamtoken - The opaque token to start streaming from.
+  to : $streamtoken - The opaque token to end streaming at. Typically,
+       clients will not know the item of data to end at, so this will usually be 
+       START or END.
+  limit : integer - An integer representing the maximum number of items to 
+          return.
+
+For example, the event stream has events E1 -> E15. The client wants the last 5 
+events and doesn't know any previous events:
+
+S                                                    E
+|-E1-E2-E3-E4-E5-E6-E7-E8-E9-E10-E11-E12-E13-E14-E15-|
+|                               |                    |
+|                          _____|                    |
+|__________________       |       ___________________|
+                   |      |      |
+ GET /events?to=START&limit=5&from=END
+ Returns:
+   E15,E14,E13,E12,E11
+
+
+Another example: a public room list has rooms R1 -> R17. The client is showing 5 
+rooms at a time on screen, and is on page 2. They want to
+now show page 3 (rooms R11 -> 15):
+
+S                                                           E
+|  0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16 | stream token
+|-R1-R2-R3-R4-R5-R6-R7-R8-R9-R10-R11-R12-R13-R14-R15-R16-R17| room
+                  |____________| |________________|
+                        |                |
+                    Currently            |
+                    viewing              |
+                                         |
+                         GET /rooms/list?from=9&to=END&limit=5
+                         Returns: R11,R12,R13,R14,R15
+                         
+Note that tokens are treated in an *exclusive*, not inclusive, manner. The end 
+token from the intial request was '9' which corresponded to R10. When the 2nd
+request was made, R10 did not appear again, even though from=9 was specified. If
+you know the token, you already have the data.
+
+Pagination Response
+-------------------
+Responses to pagination requests MUST follow the format:
+{
+  "chunk": [ ... , Responses , ... ],
+  "start" : $streamtoken,
+  "end" : $streamtoken
+}
+Where $streamtoken is an opaque token which can be used in another query to
+get the next set of results. The "start" and "end" keys can only be omitted if
+the complete dataset is provided in "chunk".
+
+If the client wants earlier results, they should use from=$start_streamtoken,
+to=START. Likewise, if the client wants later results, they should use
+from=$end_streamtoken, to=END.
+
+Unless specified, the default pagination parameters are from=START, to=END, 
+without a limit set. This allows you to hit an API like
+/events without any query parameters to get everything.
+
+The Event Stream
+----------------
+The event stream returns events using the pagination streaming API. When the 
+client disconnects for a while and wants to reconnect to the event stream, they 
+should specify from=$end_streamtoken. This lets the server know where in the 
+event stream the client is. These tokens are completely opaque, and the client 
+cannot infer anything from them.
+
+  GET /events?from=$LAST_STREAM_TOKEN
+  REST Path: /events
+  Returns (success): A JSON array of Event Data.
+  Returns (failure): An Error Response
+
+LAST_STREAM_TOKEN is the last stream token obtained from the event stream. If the 
+client is connecting for the first time and does not know any stream tokens,
+they can use "START" to request all events from the start. For more information 
+on this, see "Pagination Request Query Parameters".
+
+The event stream supports shortpoll and longpoll with the "timeout" query
+parameter. This parameter specifies the number of milliseconds the server should
+hold onto the connection waiting for incoming events. If no events occur in this
+period, the connection will be closed and an empty chunk will be returned. To
+use shortpoll, specify "timeout=0".
+
+Event Data
+----------
+This is a JSON object which looks like:
+{
+  "event_id" : $EVENT_ID,
+  "type" : $EVENT_TYPE,
+  $URL_ARGS,
+  "content" : {
+    $EVENT_CONTENT
+  }
+}
+
+EVENT_ID
+  An ID identifying this event. This is so duplicate events can be suppressed on
+  the client.
+
+EVENT_TYPE
+  The namespaced event type (m.*)
+
+URL_ARGS
+  Path specific data from the REST API.
+
+EVENT_CONTENT
+  The event content, matching the REST content PUT previously.
+
+Events are differentiated via the event type "type" key. This is the type of 
+event being received. This can be expanded upon by using different namespaces. 
+Every event MUST have a 'type' key.
+
+Most events will have a corresponding REST URL. This URL will generally have 
+data in it to represent the resource being modified,
+e.g. /rooms/$room_id. The event data will contain extra top-level keys to expose 
+this information to clients listening on an event
+stream. The event content maps directly to the contents submitted via the REST 
+API.
+
+For example:
+  Event Type: m.example.room.members
+  REST Path: /examples/room/$room_id/members/$user_id
+  REST Content: { "membership" : "invited" }
+  
+is represented in the event stream as:
+
+{
+  "event_id" : "e_some_event_id",
+  "type" : "m.example.room.members",
+  "room_id" : $room_id,
+  "user_id" : $user_id,
+  "content" : {
+    "membership" : "invited"
+  }
+}
+
+As convention, the URL variable "$varname" will map directly onto the name 
+of the JSON key "varname".
+
+Error Responses
+---------------
+If the client sends an invalid request, the server MAY respond with an error 
+response. This is of the form:
+{
+  "error" : "string",
+  "errcode" : "string"
+}
+The 'error' string will be a human-readable error message, usually a sentence
+explaining what went wrong. 
+
+The 'errcode' string will be a unique string which can be used to handle an 
+error message e.g. "M_FORBIDDEN". These error codes should have their namespace 
+first in ALL CAPS, followed by a single _. For example, if there was a custom
+namespace com.mydomain.here, and a "FORBIDDEN" code, the error code should look
+like "COM.MYDOMAIN.HERE_FORBIDDEN". There may be additional keys depending on 
+the error, but the keys 'error' and 'errcode' will always be present. 
+
+Some standard error codes are below:
+
+M_FORBIDDEN:
+Forbidden access, e.g. bad access token, failed login.
+
+M_BAD_JSON:
+Request contained valid JSON, but it was malformed in some way, e.g. missing
+required keys, invalid values for keys.
+
+M_NOT_JSON:
+Request did not contain valid JSON.
+
+M_NOT_FOUND:
+No resource was found for this request.
+
+Some requests have unique error codes:
+
+M_USER_IN_USE:
+Encountered when trying to register a user ID which has been taken.
+
+M_ROOM_IN_USE:
+Encountered when trying to create a room which has been taken.
+
+M_BAD_PAGINATION:
+Encountered when specifying bad pagination values to a Pagination Streaming API.
+
+
+========
+REST API
+========
+
+All content must be application/json. Some keys are required, while others are 
+optional. Unless otherwise specified,
+all HTTP PUT/POST/DELETEs will return a 200 OK with an empty response body on 
+success, and a 4xx/5xx with an optional Error Response on failure. When sending 
+data, if there are no keys to send, an empty JSON object should be sent.
+
+All POST/PUT/GET/DELETE requests MUST have an 'access_token' query parameter to 
+allow the server to authenticate the client. All
+POST requests MUST be submitted as application/json. 
+
+All paths MUST be namespaced by the version of the API being used. This should
+be:
+
+/matrix/client/api/v1
+
+All REST paths in this section MUST be prefixed with this. E.g.
+  REST Path: /rooms/$room_id
+  Absolute Path: /matrix/client/api/v1/rooms/$room_id
+
+Registration
+============
+Clients must register with the server in order to use the service. After 
+registering, the client will be given an
+access token which must be used in ALL requests as a query parameter 
+'access_token'.
+
+Registering for an account
+--------------------------
+  POST /register
+  With: A JSON object containing the key "user_id" which contains the desired 
+        user_id, or an empty JSON object to have the server allocate a user_id 
+        automatically.
+  Returns (success): 200 OK with a JSON object:
+                     {
+                       "user_id" : "string [user_id]",
+                       "access_token" : "string"
+                     }
+  Returns (failure): An Error Response. M_USER_IN_USE if the user ID is taken.
+                     
+
+Unregistering an account
+------------------------
+  POST /unregister
+  With query parameters: access_token=$ACCESS_TOKEN
+  Returns (success): 200 OK
+  Returns (failure): An Error Response.
+  
+  
+Logging in to an existing account
+=================================
+If the client has already registered, they need to be able to login to their
+account. The home server may provide many different ways of logging in, such
+as user/password auth, login via a social network (OAuth), login by confirming 
+a token sent to their email address, etc. This section does NOT define how home 
+servers should authorise their users who want to login to their existing 
+accounts. This section defines the standard interface which implementations 
+should follow so that ANY client can login to ANY home server.
+
+The login process breaks down into the following:
+  1: Get login process info.
+  2: Submit the login stage credentials.
+  3: Get access token or be told the next stage in the login process and repeat 
+     step 2.
+     
+Getting login process info:
+  GET /login
+  Returns (success): 200 OK with LoginInfo.
+  Returns (failure): An Error Response.
+  
+Submitting the login stage credentials:
+  POST /login
+  With: LoginSubmission
+  Returns (success): 200 OK with LoginResult
+  Returns (failure): An Error Response
+  
+Where LoginInfo is a JSON object which MUST have a "type" key which denotes the 
+login type. If there are multiple login stages, this object MUST also contain a 
+"stages" key, which has a JSON array of login types denoting all the steps in 
+order to login, including the first stage which is in "type". This allows the 
+client to make an informed decision as to whether or not they can natively
+handle the entire login process, or whether they should fallback (see below).
+
+Where LoginSubmission is a JSON object which MUST have a "type" key.
+
+Where LoginResult is a JSON object which MUST have either a "next" key OR an
+"access_token" key, depending if the login process is over or not. This object
+MUST have a "session" key if multiple POSTs need to be sent to /login.
+
+Fallback
+--------
+If the client does NOT know how to handle the given type, they should:
+  GET /login/fallback
+This MUST return an HTML page which can perform the entire login process.
+
+Password-based
+--------------
+Type: "m.login.password"
+LoginSubmission:
+{
+  "type": "m.login.password",
+  "user": <user_id>,
+  "password": <password>
+}
+
+Example:
+Assume you are @bob:matrix.org and you wish to login on another mobile device.
+First, you GET /login which returns:
+{
+  "type": "m.login.password"
+}
+Your client knows how to handle this, so your client prompts the user to enter
+their username and password. This is then submitted:
+{
+  "type": "m.login.password",
+  "user": "@bob:matrix.org",
+  "password": "monkey"
+}
+The server checks this, finds it is valid, and returns:
+{
+  "access_token": "abcdef0123456789"
+}
+
+OAuth2-based
+------------
+Type: "m.login.oauth2"
+This is a multi-stage login.
+
+LoginSubmission:
+{
+  "type": "m.login.oauth2",
+  "user": <user_id>
+}
+Returns:
+{
+  "uri": <Authorization Request uri OR service selection uri>
+}
+
+The home server acts as a 'confidential' Client for the purposes of OAuth2.
+
+If the uri is a "sevice selection uri", it is a simple page which prompts the 
+user to choose which service to authorize with. On selection of a service, they
+link through to Authorization Request URIs. If there is only 1 service which the
+home server accepts when logging in, this indirection can be skipped and the
+"uri" key can be the Authorization Request URI. 
+
+The client visits the Authorization Request URI, which then shows the OAuth2 
+Allow/Deny prompt. Hitting 'Allow' returns the redirect URI with the auth code. 
+Home servers can choose any path for the redirect URI. The client should visit 
+the redirect URI, which will then finish the OAuth2 login process, granting the 
+home server an access token for the chosen service. When the home server gets 
+this access token, it knows that the cilent has authed with the 3rd party, and 
+so can return a LoginResult.
+
+The OAuth redirect URI (with auth code) MUST return a LoginResult.
+    
+Example:
+Assume you are @bob:matrix.org and you wish to login on another mobile device.
+First, you GET /login which returns:
+{
+  "type": "m.login.oauth2"
+}
+Your client knows how to handle this, so your client prompts the user to enter
+their username. This is then submitted:
+{
+  "type": "m.login.oauth2",
+  "user": "@bob:matrix.org"
+}
+The server only accepts auth from Google, so returns the Authorization Request
+URI for Google:
+{
+  "uri": "https://accounts.google.com/o/oauth2/auth?response_type=code&
+  client_id=CLIENT_ID&redirect_uri=REDIRECT_URI&scope=photos"
+}
+The client then visits this URI and authorizes the home server. The client then
+visits the REDIRECT_URI with the auth code= query parameter which returns:
+{
+  "access_token": "0123456789abcdef"
+}
+
+Email-based (code)
+------------------
+Type: "m.login.email.code"
+This is a multi-stage login.
+
+First LoginSubmission:
+{
+  "type": "m.login.email.code",
+  "user": <user_id>
+  "email": <email address>
+}
+Returns:
+{
+  "session": <session id>
+}
+
+The email contains a code which must be sent in the next LoginSubmission:
+{
+  "type": "m.login.email.code",
+  "session": <session id>,
+  "code": <code in email sent>
+}
+Returns:
+{
+  "access_token": <access token>
+}
+
+Example:
+Assume you are @bob:matrix.org and you wish to login on another mobile device.
+First, you GET /login which returns:
+{
+  "type": "m.login.email.code"
+}
+Your client knows how to handle this, so your client prompts the user to enter
+their email address. This is then submitted:
+{
+  "type": "m.login.email.code",
+  "user": "@bob:matrix.org",
+  "email": "bob@mydomain.com"
+}
+The server confirms that bob@mydomain.com is linked to @bob:matrix.org, then 
+sends an email to this address and returns:
+{
+  "session": "ewuigf7462"
+}
+The client's screen changes to a code submission page. The email arrives and it 
+says something to the effect of "please enter 2348623 into the app". This is
+the submitted along with the session:
+{
+  "type": "m.login.email.code",
+  "session": "ewuigf7462",
+  "code": "2348623"
+}
+The server accepts this and returns:
+{
+  "access_token": "abcdef0123456789"
+}
+
+Email-based (url)
+-----------------
+Type: "m.login.email.url"
+This is a multi-stage login.
+
+First LoginSubmission:
+{
+  "type": "m.login.email.url",
+  "user": <user_id>
+  "email": <email address>
+}
+Returns:
+{
+  "session": <session id>
+}
+
+The email contains a URL which must be clicked. After it has been clicked, the
+client should perform a request:
+{
+  "type": "m.login.email.code",
+  "session": <session id>
+}
+Returns:
+{
+  "access_token": <access token>
+}
+
+Example:
+Assume you are @bob:matrix.org and you wish to login on another mobile device.
+First, you GET /login which returns:
+{
+  "type": "m.login.email.url"
+}
+Your client knows how to handle this, so your client prompts the user to enter
+their email address. This is then submitted:
+{
+  "type": "m.login.email.url",
+  "user": "@bob:matrix.org",
+  "email": "bob@mydomain.com"
+}
+The server confirms that bob@mydomain.com is linked to @bob:matrix.org, then 
+sends an email to this address and returns:
+{
+  "session": "ewuigf7462"
+}
+The client then starts polling the server with the following:
+{
+  "type": "m.login.email.url",
+  "session": "ewuigf7462"
+}
+(Alternatively, the server could send the device a push notification when the
+email has been validated). The email arrives and it contains a URL to click on.
+The user clicks on the which completes the login process with the server. The
+next time the client polls, it returns:
+{
+  "access_token": "abcdef0123456789"
+}
+
+N-Factor auth
+-------------
+Multiple login stages can be combined with the "next" key in the LoginResult.
+
+Example:
+A server demands an email.code then password auth before logging in. First, the
+client performs a GET /login which returns:
+{
+  "type": "m.login.email.code",
+  "stages": ["m.login.email.code", "m.login.password"]
+}
+The client performs the email login (See "Email-based (code)"), but instead of
+returning an access_token, it returns:
+{
+  "next": "m.login.password"
+}
+The client then presents a user/password screen and the login continues until
+this is complete (See "Password-based"), which then returns the "access_token".
+
+Rooms
+=====
+A room is a conceptual place where users can send and receive messages. Rooms 
+can be created, joined and left. Messages are sent
+to a room, and all participants in that room will receive the message. Rooms are 
+uniquely identified via the room_id.
+
+Creating a room (with a room ID)
+--------------------------------
+  Event Type: m.room.create [TODO(kegan): Do we generate events for this?]
+  REST Path: /rooms/$room_id
+  Valid methods: PUT
+  Required keys: None.
+  Optional keys:
+    visibility : [public|private] - Set whether this room shows up in the public 
+    room list.
+  Returns:
+    On Failure: MAY return a suggested alternative room ID if this room ID is 
+    taken.
+    {
+      suggested_room_id : $new_room_id
+      error : "Room already in use."
+      errcode : "M_ROOM_IN_USE"
+    }
+    
+
+Creating a room (without a room ID)
+-----------------------------------
+  Event Type: m.room.create [TODO(kegan): Do we generate events for this?]
+  REST Path: /rooms
+  Valid methods: POST
+  Required keys: None.
+  Optional keys:
+    visibility : [public|private] - Set whether this room shows up in the public 
+    room list.
+  Returns:
+    On Success: The allocated room ID. Additional information about the room
+    such as the visibility MAY be included as extra keys in this response.
+    {
+      room_id : $room_id
+    }
+
+Setting the topic for a room
+----------------------------
+  Event Type: m.room.topic
+  REST Path: /rooms/$room_id/topic
+  Valid methods: GET/PUT
+  Required keys: 
+    topic : $topicname - Set the topic to $topicname in room $room_id.
+
+
+See a list of public rooms
+--------------------------
+  REST Path: /public/rooms?pagination_query_parameters
+  Valid methods: GET
+  This API can use pagination query parameters.
+  Returns:
+    {
+      "chunk" : JSON array of RoomInfo JSON objects - Required.
+      "start" : "string (start token)" - See Pagination Response.
+      "end" : "string (end token)" - See Pagination Response.
+      "total" : integer - Optional. The total number of rooms.
+    }
+
+RoomInfo: Information about a single room.
+  Servers MUST send the key: room_id
+  Servers MAY send the keys: topic, num_members
+  {
+    "room_id" : "string",
+    "topic" : "string",
+    "num_members" : integer
+  }
+
+Room Members
+============
+
+Invite/Joining/Leaving a room
+-----------------------------
+  Event Type: m.room.member
+  REST Path: /rooms/$room_id/members/$user_id/state
+  Valid methods: PUT/GET/DELETE
+  Required keys:
+    membership : [join|invite] - The membership state of $user_id in room 
+                                 $room_id.
+
+Where:
+  join - Indicate you ($user_id) are joining the room $room_id.
+  invite - Indicate that $user_id has been invited to room $room_id.
+
+User $user_id can leave room $room_id by DELETEing this path.
+
+Checking the user list of a room
+--------------------------------
+  REST Path: /rooms/$room_id/members/list
+  This API can use pagination query parameters.
+  Valid methods: GET
+  Returns:
+    A pagination response with chunk data as m.room.member events.
+
+Messages
+========
+Users send messages to other users in rooms. These messages may be text, images, 
+video, etc. Clients may also want to acknowledge messages by sending feedback, 
+in the form of delivery/read receipts.
+
+Server-attached keys
+--------------------
+The server MAY attach additional keys to messages and feedback. If a client 
+submits keys with the same name, they will be clobbered by
+the server.
+
+Required keys:
+from : "string [user_id]"
+  The user_id of the user who sent the message/feedback.
+
+Optional keys:
+hsob_ts : integer
+  A timestamp (ms resolution) representing when the message/feedback got to the 
+  sender's home server ("home server outbound timestamp").
+
+hsib_ts : integer
+  A timestamp (ms resolution) representing when the 
+  message/feedback got to the receiver's home server ("home server inbound 
+  timestamp"). This may be the same as hsob_ts if the sender/receiver are on the 
+  same home server.
+
+Sending messages
+----------------
+  Event Type: m.room.message
+  REST Path: /rooms/$room_id/messages/$from/$msg_id
+  Valid methods: GET/PUT
+  URL parameters:
+    $from : user_id - The sender's user_id. This value will be clobbered by the 
+    server before sending.
+  Required keys: 
+    msgtype: [m.text|m.emote|m.image|m.audio|m.video|m.location|m.file] - 
+             The type of message. Not to be confused with the Event 'type'.
+  Optional keys:
+    sender_ts : integer - A timestamp (ms resolution) representing the 
+                wall-clock time when the message was sent from the client.
+  Reserved keys:
+    body : "string" - The human readable string for compatibility with clients 
+           which cannot process a given msgtype. This key is optional, but
+           if it is included, it MUST be human readable text 
+           describing the message. See individual msgtypes for more 
+           info on what this means in practice.
+
+Each msgtype may have required fields of their own.
+
+msgtype: m.text
+----------------
+Required keys:
+  body : "string" - The body of the message.
+Optional keys:
+  None.
+
+msgtype: m.emote
+-----------------
+Required keys:
+  body : "string" - *tries to come up with a witty explanation*.
+Optional keys:
+  None.
+
+msgtype: m.image
+-----------------
+Required keys:
+  url : "string" - The URL to the image.
+Optional keys:
+  body : "string" - info : JSON object (ImageInfo) - The image info for image 
+         referred to in 'url'.
+  thumbnail_url : "string" - The URL to the thumbnail.
+  thumbnail_info : JSON object (ImageInfo) - The image info for the image 
+                   referred to in 'thumbnail_url'.
+
+ImageInfo: Information about an image.
+{
+  "size" : integer (size of image in bytes),
+  "w" : integer (width of image in pixels),
+  "h" : integer (height of image in pixels),
+  "mimetype" : "string (e.g. image/jpeg)"
+}
+
+Interpretation of 'body' key: The alt text of the image, or some kind of content 
+description for accessibility e.g. "image attachment".
+
+msgtype: m.audio
+-----------------
+Required keys:
+  url : "string" - The URL to the audio.
+Optional keys:
+  info : JSON object (AudioInfo) - The audio info for the audio referred to in 
+         'url'.
+
+AudioInfo: Information about a piece of audio. 
+{
+  "mimetype" : "string (e.g. audio/aac)",
+  "size" : integer (size of audio in bytes),
+  "duration" : integer (duration of audio in milliseconds)
+}
+
+Interpretation of 'body' key: A description of the audio e.g. "Bee Gees - 
+Stayin' Alive", or some kind of content description for accessibility e.g. 
+"audio attachment".
+
+msgtype: m.video
+-----------------
+Required keys:
+  url : "string" - The URL to the video.
+Optional keys:
+  info : JSON object (VideoInfo) - The video info for the video referred to in 
+         'url'.
+
+VideoInfo: Information about a video.
+{
+  "mimetype" : "string (e.g. video/mp4)",
+  "size" : integer (size of video in bytes),
+  "duration" : integer (duration of video in milliseconds),
+  "w" : integer (width of video in pixels),
+  "h" : integer (height of video in pixels),
+  "thumbnail_url" : "string (URL to image)",
+  "thumbanil_info" : JSON object (ImageInfo)
+}
+
+Interpretation of 'body' key: A description of the video e.g. "Gangnam style", 
+or some kind of content description for accessibility e.g. "video attachment".
+
+msgtype: m.location
+--------------------
+Required keys:
+  geo_uri : "string" - The geo URI representing the location.
+Optional keys:
+  thumbnail_url : "string" - The URL to a thumnail of the location being 
+                  represented.
+  thumbnail_info : JSON object (ImageInfo) - The image info for the image 
+                   referred to in 'thumbnail_url'.
+
+Interpretation of 'body' key: A description of the location e.g. "Big Ben, 
+London, UK", or some kind of content description for accessibility e.g. 
+"location attachment".
+
+
+Sending feedback
+----------------
+When you receive a message, you may want to send delivery receipt to let the 
+sender know that the message arrived. You may also want to send a read receipt 
+when the user has read the message. These receipts are collectively known as 
+'feedback'.
+
+  Event Type: m.room.message.feedback
+  REST Path: /rooms/$room_id/messages/$msgfrom/$msg_id/feedback/$from/$feedback
+  Valid methods: GET/PUT
+  URL parameters:
+    $msgfrom - The sender of the message's user_id.
+    $from : user_id - The sender of the feedback's user_id. This value will be 
+    clobbered by the server before sending.
+    $feedback : [d|r] - Specify if this is a [d]elivery or [r]ead receipt.
+  Required keys:
+    None.
+  Optional keys:
+    sender_ts : integer - A timestamp (ms resolution) representing the 
+    wall-clock time when the receipt was sent from the client.
+
+Receiving messages (bulk/pagination)
+------------------------------------
+  Event Type: m.room.message
+  REST Path: /rooms/$room_id/messages/list
+  Valid methods: GET
+  Query Parameters:
+    feedback : [true|false] - Specify if feedback should be bundled with each 
+    message.
+  This API can use pagination query parameters.
+  Returns:
+    A JSON array of Event Data in "chunk" (see Pagination Response). If the 
+    "feedback" parameter was set, the Event Data will also contain a "feedback" 
+    key which contains a JSON array of feedback, with each element as Event Data 
+    with compressed feedback for this message.
+
+Event Data with compressed feedback is a special type of feedback with 
+contextual keys removed. It is designed to limit the amount of redundant data 
+being sent for feedback. This removes the type, event_id, room ID, 
+message sender ID and message ID keys.
+
+     ORIGINAL (via event streaming)
+{
+  "event_id":"e1247632487",
+  "type":"m.room.message.feedback",
+  "from":"string [user_id]",
+  "feedback":"string [d|r]",
+  "room_id":"$room_id",
+  "msg_id":"$msg_id",
+  "msgfrom":"$msgfromid",
+  "content":{
+    "sender_ts":139880943
+  }
+}
+
+     COMPRESSED (via /messages/list)
+{
+  "from":"string [user_id]",
+  "feedback":"string [d|r]",
+  "content":{
+    "sender_ts":139880943
+  }
+}
+
+When you join a room $room_id, you may want the last 10 messages with feedback. 
+This is represented as:
+  GET 
+  /rooms/$room_id/messages/list?from=END&to=START&limit=10&feedback=true
+
+You may want to get 10 messages even earlier than that without feedback. If the 
+start stream token from the previous request was stok_019173, this request would 
+be:
+  GET 
+  /rooms/$room_id/messages/list?from=stok_019173&to=START&limit=10&
+                               feedback=false
+  
+NOTE: Care must be taken when using this API in conjunction with event 
+      streaming. It is possible that this will return a message which will
+      then come down the event stream, resulting in a duplicate message. Clients 
+      should clobber based on the global message ID, or event ID.
+
+
+Get current state for all rooms (aka IM Initial Sync API)
+-------------------------------
+  REST Path: /im/sync
+  Valid methods: GET
+  This API can use pagination query parameters. Pagination is applied on a per
+  *room* basis. E.g. limit=1 means "get 1 message for each room" and not "get 1
+  room's messages". If there is no limit, all messages for all rooms will be
+  returned.
+  If you want 1 room's messages, see "Receiving messages (bulk/pagination)".
+  Additional query parameters: 
+    feedback: [true] - Bundles feedback with messages.
+  Returns:
+    An array of RoomStateInfo.
+
+RoomStateInfo: A snapshot of information about a single room.
+  {
+    "room_id" : "string",
+    "membership" : "string [join|invite]",
+    "messages" : {
+      "start": "string",
+      "end": "string",
+      "chunk":
+      m.room.message pagination stream events (with feedback if specified),
+      this is the same as "Receiving messages (bulk/pagination)".
+    }
+  }
+The "membership" key is the calling user's membership state in the given 
+"room_id". The "messages" key may be omitted if the "membership" value is 
+"invite". Additional keys may be added to the top-level object, such as:
+  "topic" : "string" - The topic for the room in question.
+  "room_image_url" : "string" - The URL of the room image if specified.
+  "num_members" : integer - The number of members in the room.
+
+
+Profiles
+========
+
+Getting/Setting your own displayname
+------------------------------------
+  REST Path: /profile/$user_id/displayname
+  Valid methods: GET/PUT
+  Required keys:
+    displayname : The displayname text
+
+Getting/Setting your own avatar image URL
+-----------------------------------------
+The homeserver does not currently store the avatar image itself, but offers
+storage for the user to specify a web URL that points at the required image,
+leaving it up to clients to fetch it themselves.
+  REST Path: /profile/$user_id/avatar_url
+  Valid methods: GET/PUT
+  Required keys:
+    avatar_url : The URL path to the required image
+
+Getting other user's profile information
+----------------------------------------
+Either of the above REST methods may be used to fetch other user's profile
+information by the client, either on other local users on the same homeserver or
+for users from other servers entirely.
+
+
+Presence
+========
+
+In the following messages, the presence state is an integer enumeration of the
+following states:
+  0 : OFFLINE
+  1 : BUSY
+  2 : ONLINE
+  3 : FREE_TO_CHAT
+
+Aside from OFFLINE, the protocol doesn't assign any special meaning to these
+states; they are provided as an approximate signal for users to give to other
+users and for clients to present them in some way that may be useful. Clients
+could have different behaviours for different states of the user's presence, for
+example to decide how much prominence or sound to use for incoming event
+notifications.
+
+Getting/Setting your own presence state
+---------------------------------------
+  REST Path: /presence/$user_id/status
+  Valid methods: GET/PUT
+  Required keys:
+    state : [0|1|2|3] - The user's new presence state
+  Optional keys:
+    status_msg : text string provided by the user to explain their status
+
+Fetching your presence list
+---------------------------
+  REST Path: /presence_list/$user_id
+  Valid methods: GET/(post)
+  Returns:
+    An array of presence list entries. Each entry is an object with the
+    following keys:
+      {
+        "user_id" : string giving the observed user's ID
+        "state" : int giving their status
+        "status_msg" : optional text string
+        "displayname" : optional text string from the user's profile
+        "avatar_url" : optional text string from the user's profile
+      }
+
+Maintaining your presence list
+------------------------------
+  REST Path: /presence_list/$user_id
+  Valid methods: POST/(get)
+  With: A JSON object optionally containing either of the following keys:
+    "invite" : a list of strings giving user IDs to invite for presence
+      subscription
+    "drop" : a list of strings giving user IDs to remove from your presence
+      list
+
+Receiving presence update events
+--------------------------------
+  Event Type: m.presence
+  Keys of the event's content are the same as those returned by the presence
+    list.
+
+Examples
+========
+
+The following example is the story of "bob", who signs up at "sy.org" and joins 
+the public room "room_beta@sy.org". They get the 2 most recent
+messages (with feedback) in that room and then send a message in that room. 
+
+For context, here is the complete chat log for room_beta@sy.org:
+
+Room: "Hello world" (room_beta@sy.org)
+Members: (2) alice@randomhost.org, friend_of_alice@randomhost.org
+Messages:
+  alice@randomhost.org : hi friend!                     
+  [friend_of_alice@randomhost.org DELIVERED]
+  alice@randomhost.org : you're my only friend          
+  [friend_of_alice@randomhost.org DELIVERED]
+  alice@randomhost.org : afk                            
+  [friend_of_alice@randomhost.org DELIVERED]
+  [ bob@sy.org joins ]
+  bob@sy.org : Hi everyone
+  [ alice@randomhost.org changes the topic to "FRIENDS ONLY" ]
+  alice@randomhost.org : Hello!!!!
+  alice@randomhost.org : Let's go to another room
+  alice@randomhost.org : You're not my friend
+  [ alice@randomhost.org invites bob@sy.org to the room 
+  commoners@randomhost.org]
+
+
+REGISTER FOR AN ACCOUNT
+POST: /register
+Content: {}
+Returns: { "user_id" : "bob@sy.org" , "access_token" : "abcdef0123456789" }
+
+GET PUBLIC ROOM LIST
+GET: /rooms/list?access_token=abcdef0123456789
+Returns: 
+{ 
+  "total":3,
+  "chunk":
+  [
+    { "room_id":"room_alpha@sy.org", "topic":"I am a fish" },
+    { "room_id":"room_beta@sy.org", "topic":"Hello world" },
+    { "room_id":"room_xyz@sy.org", "topic":"Goodbye cruel world" }
+  ]
+}
+
+JOIN ROOM room_beta@sy.org
+PUT 
+/rooms/room_beta%40sy.org/members/bob%40sy.org/state?
+                                    access_token=abcdef0123456789
+Content: { "membership" : "join" }
+Returns: 200 OK
+
+GET LATEST 2 MESSAGES WITH FEEDBACK
+GET 
+/rooms/room_beta%40sy.org/messages/list?from=END&to=START&limit=2&
+                                    feedback=true&access_token=abcdef0123456789
+Returns:
+{
+  "chunk":
+    [
+      { 
+        "event_id":"01948374", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"avefifu",
+        "from":"alice@randomhost.org",
+        "hs_ts":139985736,
+        "content":{
+          "msgtype":"m.text",
+          "body":"afk"
+        }
+        "feedback": [
+          {
+            "from":"friend_of_alice@randomhost.org",
+            "feedback":"d",
+            "hs_ts":139985850,
+            "content":{
+              "sender_ts":139985843
+            }
+          }
+        ]
+      },
+      { 
+        "event_id":"028dfe8373", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"afhgfff",
+        "from":"alice@randomhost.org",
+        "hs_ts":139970006,
+        "content":{
+          "msgtype":"m.text",
+          "body":"you're my only friend"
+        }
+        "feedback": [
+          {
+            "from":"friend_of_alice@randomhost.org",
+            "feedback":"d",
+            "hs_ts":139970144,
+            "content":{
+              "sender_ts":139970122
+            }
+          }
+        ]
+      },
+    ],
+  "start": "stok_04823947",
+  "end": "etok_1426425"
+}
+
+SEND MESSAGE IN ROOM
+PUT 
+/rooms/room_beta%40sy.org/messages/bob%40sy.org/m0001?
+                            access_token=abcdef0123456789
+Content: { "msgtype" : "text" , "body" : "Hi everyone" }
+Returns: 200 OK
+
+
+Checking the event stream for this user:
+GET: /events?from=START&access_token=abcdef0123456789
+Returns:
+{
+  "chunk": 
+    [
+      { 
+        "event_id":"e10f3d2b", 
+        "type":"m.room.member",
+        "room_id":"room_beta@sy.org",
+        "user_id":"bob@sy.org",
+        "content":{
+          "membership":"join"
+        }
+      },
+      { 
+        "event_id":"1b352d32", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"m0001",
+        "from":"bob@sy.org",
+        "hs_ts":140193857,
+        "content":{
+          "msgtype":"m.text",
+          "body":"Hi everyone"
+        }
+      }
+    ],
+  "start": "stok_9348635",
+  "end": "etok_1984723"
+}
+
+Client disconnects for a while and the topic is updated in this room, 3 new 
+messages arrive whilst offline, and bob is invited to another room.
+
+GET /events?from=etok_1984723&access_token=abcdef0123456789
+Returns:
+{
+  "chunk": 
+    [
+      { 
+        "event_id":"feee0294", 
+        "type":"m.room.topic",
+        "room_id":"room_beta@sy.org",
+        "from":"alice@randomhost.org",
+        "content":{
+          "topic":"FRIENDS ONLY",
+        }
+      },
+      { 
+        "event_id":"a028bd9e", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"z839409",
+        "from":"alice@randomhost.org",
+        "hs_ts":140195000,
+        "content":{
+          "msgtype":"m.text",
+          "body":"Hello!!!"
+        }
+      },
+      { 
+        "event_id":"49372d9e", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"z839410",
+        "from":"alice@randomhost.org",
+        "hs_ts":140196000,
+        "content":{
+          "msgtype":"m.text",
+          "body":"Let's go to another room"
+        }
+      },
+      { 
+        "event_id":"10abdd01", 
+        "type":"m.room.message",
+        "room_id":"room_beta@sy.org",
+        "msg_id":"z839411",
+        "from":"alice@randomhost.org",
+        "hs_ts":140197000,
+        "content":{
+          "msgtype":"m.text",
+          "body":"You're not my friend"
+        }
+      },
+      { 
+        "event_id":"0018453d", 
+        "type":"m.room.member",
+        "room_id":"commoners@randomhost.org",
+        "from":"alice@randomhost.org",
+        "user_id":"bob@sy.org",
+        "content":{
+          "membership":"invite"
+        }
+      },
+    ],
+  "start": "stok_0184288",
+  "end": "etok_1348723"
+}
diff --git a/docs/client-server/urls b/docs/client-server/urls
new file mode 100644
index 0000000000..c0d2eaa80c
--- /dev/null
+++ b/docs/client-server/urls
@@ -0,0 +1,92 @@
+=========================
+Client-Server URL Summary
+=========================
+
+A brief overview of the URL scheme involved in the Synapse Client-Server API.
+
+
+URLs
+====
+
+Fetch events:
+  GET /events
+
+Registering an account
+  POST /register
+
+Unregistering an account
+  POST /unregister
+
+Rooms
+-----
+
+Creating a room by ID
+  PUT /rooms/$roomid
+
+Creating an anonymous room
+  POST /rooms
+
+Room topic
+  GET /rooms/$roomid/topic
+  PUT /rooms/$roomid/topic
+
+List rooms
+  GET /rooms/list
+
+Invite/Join/Leave
+  GET    /rooms/$roomid/members/$userid/state
+  PUT    /rooms/$roomid/members/$userid/state
+  DELETE /rooms/$roomid/members/$userid/state
+
+List members
+  GET  /rooms/$roomid/members/list
+
+Sending/reading messages
+  PUT /rooms/$roomid/messages/$sender/$msgid
+
+Feedback
+  GET /rooms/$roomid/messages/$sender/$msgid/feedback/$feedbackuser/$feedback
+  PUT /rooms/$roomid/messages/$sender/$msgid/feedback/$feedbackuser/$feedback
+
+Paginating messages
+  GET /rooms/$roomid/messages/list
+
+Profiles
+--------
+
+Display name
+  GET /profile/$userid/displayname
+  PUT /profile/$userid/displayname
+
+Avatar URL
+  GET /profile/$userid/avatar_url
+  PUT /profile/$userid/avatar_url
+
+Metadata
+  GET  /profile/$userid/metadata
+  POST /profile/$userid/metadata
+
+Presence
+--------
+
+My state or status message
+  GET /presence/$userid/status
+  PUT /presence/$userid/status
+    also 'GET' for fetching others
+
+TODO(paul): per-device idle time, device type; similar to above
+
+My presence list
+  GET  /presence_list/$myuserid
+  POST /presence_list/$myuserid
+    body is JSON-encoded dict of keys:
+      invite: list of UserID strings to invite
+      drop: list of UserID strings to remove
+      TODO(paul): define other ops: accept, group management, ordering?
+
+Presence polling start/stop
+  POST /presence_list/$myuserid?op=start
+  POST /presence_list/$myuserid?op=stop
+
+Presence invite
+  POST /presence_list/$myuserid/invite/$targetuserid
diff --git a/docs/code_style b/docs/code_style
new file mode 100644
index 0000000000..d7e2d5e69e
--- /dev/null
+++ b/docs/code_style
@@ -0,0 +1,18 @@
+Basically, PEP8
+
+- Max line width: 80 chars.
+- Use camel case for class and type names
+- Use underscores for functions and variables.
+- Use double quotes.
+- Use parentheses instead of '\' for line continuation where ever possible (which is pretty much everywhere)
+- There should be max a single new line between:
+    - statements
+    - functions in a class
+- There should be two new lines between:
+    - definitions in a module (e.g., between different classes)
+- There should be spaces where spaces should be and not where there shouldn't be:
+    - a single space after a comma
+    - a single space before and after for '=' when used as assignment
+    - no spaces before and after for '=' for default values and keyword arguments.
+
+Comments should follow the google code style. This is so that we can generate documentation with sphinx (http://sphinxcontrib-napoleon.readthedocs.org/en/latest/) 
diff --git a/docs/documentation_style b/docs/documentation_style
new file mode 100644
index 0000000000..c365d09dff
--- /dev/null
+++ b/docs/documentation_style
@@ -0,0 +1,43 @@
+===================
+Documentation Style
+===================
+
+A brief single sentence to describe what this file contains; in this case a
+description of the style to write documentation in.
+
+
+Sections
+========
+
+Each section should be separated from the others by two blank lines. Headings
+should be underlined using a row of equals signs (===). Paragraphs should be
+separated by a single blank line, and wrap to no further than 80 columns.
+
+[[TODO(username): if you want to leave some unanswered questions, notes for
+further consideration, or other kinds of comment, use a TODO section. Make sure
+to notate it with your name so we know who to ask about it!]]
+
+Subsections
+-----------
+
+If required, subsections can use a row of dashes to underline their header. A
+single blank line between subsections of a single section.
+
+
+Bullet Lists
+============
+
+ * Bullet lists can use asterisks with a single space either side.
+ 
+ * Another blank line between list elements.
+
+
+Definition Lists
+================
+
+Terms:
+  Start in the first column, ending with a colon
+
+Definitions:
+  Take a two space indent, following immediately from the term without a blank
+  line before it, but having a blank line afterwards.
diff --git a/docs/model/presence b/docs/model/presence
new file mode 100644
index 0000000000..7e54505364
--- /dev/null
+++ b/docs/model/presence
@@ -0,0 +1,249 @@
+========
+Presence
+========
+
+A description of presence information and visibility between users.
+
+Overview
+========
+
+Each user has the concept of Presence information. This encodes a sense of the
+"availability" of that user, suitable for display on other user's clients.
+
+
+Presence Information
+====================
+
+The basic piece of presence information is an enumeration of a small set of
+state; such as "free to chat", "online", "busy", or "offline". The default state
+unless the user changes it is "online". Lower states suggest some amount of
+decreased availability from normal, which might have some client-side effect
+like muting notification sounds and suggests to other users not to bother them
+unless it is urgent. Equally, the "free to chat" state exists to let the user
+announce their general willingness to receive messages moreso than default.
+
+Home servers should also allow a user to set their state as "hidden" - a state
+which behaves as offline, but allows the user to see the client state anyway and
+generally interact with client features such as reading message history or
+accessing contacts in the address book.
+
+This basic state field applies to the user as a whole, regardless of how many
+client devices they have connected. The home server should synchronise this
+status choice among multiple devices to ensure the user gets a consistent
+experience.
+
+Idle Time
+---------
+
+As well as the basic state field, the presence information can also show a sense
+of an "idle timer". This should be maintained individually by the user's
+clients, and the homeserver can take the highest reported time as that to
+report. Likely this should be presented in fairly coarse granularity; possibly
+being limited to letting the home server automatically switch from a "free to
+chat" or "online" mode into "idle".
+
+When a user is offline, the Home Server can still report when the user was last
+seen online, again perhaps in a somewhat coarse manner.
+
+Device Type
+-----------
+
+Client devices that may limit the user experience somewhat (such as "mobile"
+devices with limited ability to type on a real keyboard or read large amounts of
+text) should report this to the home server, as this is also useful information
+to report as "presence" if the user cannot be expected to provide a good typed
+response to messages.
+
+
+Presence List
+=============
+
+Each user's home server stores a "presence list" for that user. This stores a
+list of other user IDs the user has chosen to add to it (remembering any ACL
+Pointer if appropriate).
+
+To be added to a contact list, the user being added must grant permission. Once
+granted, both user's HS(es) store this information, as it allows the user who
+has added the contact some more abilities; see below. Since such subscriptions
+are likely to be bidirectional, HSes may wish to automatically accept requests
+when a reverse subscription already exists.
+
+As a convenience, presence lists should support the ability to collect users
+into groups, which could allow things like inviting the entire group to a new
+("ad-hoc") chat room, or easy interaction with the profile information ACL
+implementation of the HS.
+
+
+Presence and Permissions
+========================
+
+For a viewing user to be allowed to see the presence information of a target
+user, either
+
+ * The target user has allowed the viewing user to add them to their presence
+   list, or
+
+ * The two users share at least one room in common
+
+In the latter case, this allows for clients to display some minimal sense of
+presence information in a user list for a room.
+
+Home servers can also use the user's choice of presence state as a signal for
+how to handle new private one-to-one chat message requests. For example, it
+might decide:
+
+  "free to chat": accept anything
+  "online": accept from anyone in my addres book list
+  "busy": accept from anyone in this "important people" group in my address
+    book list
+
+
+API Efficiency
+==============
+
+A simple implementation of presence messaging has the ability to cause a large
+amount of Internet traffic relating to presence updates. In order to minimise
+the impact of such a feature, the following observations can be made:
+
+ * There is no point in a Home Server polling status for peers in a user's
+   presence list if the user has no clients connected that care about it.
+
+ * It is highly likely that most presence subscriptions will be symmetric - a
+   given user watching another is likely to in turn be watched by that user.
+
+ * It is likely that most subscription pairings will be between users who share
+   at least one Room in common, and so their Home Servers are actively
+   exchanging message PDUs or transactions relating to that Room.
+
+ * Presence update messages do not need realtime guarantees. It is acceptable to
+   delay delivery of updates for some small amount of time (10 seconds to a
+   minute).
+
+The general model of presence information is that of a HS registering its
+interest in receiving presence status updates from other HSes, which then
+promise to send them when required. Rather than actively polling for the
+currentt state all the time, HSes can rely on their relative stability to only
+push updates when required.
+
+A Home Server should not rely on the longterm validity of this presence
+information, however, as this would not cover such cases as a user's server
+crashing and thus failing to inform their peers that users it used to host are
+no longer available online. Therefore, each promise of future updates should
+carry with a timeout value (whether explicit in the message, or implicit as some
+defined default in the protocol), after which the receiving HS should consider
+the information potentially stale and request it again.
+
+However, because of the likelyhood that two home servers are exchanging messages
+relating to chat traffic in a room common to both of them, the ongoing receipt
+of these messages can be taken by each server as an implicit notification that
+the sending server is still up and running, and therefore that no status changes
+have happened; because if they had the server would have sent them. A second,
+larger timeout should be applied to this implicit inference however, to protect
+against implementation bugs or other reasons that the presence state cache may
+become invalid; eventually the HS should re-enquire the current state of users
+and update them with its own.
+
+The following workflows can therefore be used to handle presence updates:
+
+ 1 When a user first appears online their HS sends a message to each other HS
+   containing at least one user to be watched; each message carrying both a
+   notification of the sender's new online status, and a request to obtain and
+   watch the target users' presence information. This message implicitly
+   promises the sending HS will now push updates to the target HSes.
+
+ 2 The target HSes then respond a single message each, containing the current
+   status of the requested user(s). These messages too implicitly promise the
+   target HSes will themselves push updates to the sending HS.
+
+   As these messages arrive at the sending user's HS they can be pushed to the
+   user's client(s), possibly batched again to ensure not too many small
+   messages which add extra protocol overheads.
+
+At this point, all the user's clients now have the current presence status
+information for this moment in time, and have promised to send each other
+updates in future.
+
+ 3 The HS maintains two watchdog timers per peer HS it is exchanging presence
+   information with. The first timer should have a relatively small expiry
+   (perhaps 1 minute), and the second timer should have a much longer time
+   (perhaps 1 hour).
+
+ 4 Any time any kind of message is received from a peer HS, the short-term
+   presence timer associated with it is reset.
+
+ 5 Whenever either of these timers expires, an HS should push a status reminder
+   to the target HS whose timer has now expired, and request again from that
+   server the status of the subscribed users.
+
+ 6 On receipt of one of these presence status reminders, an HS can reset both
+   of its presence watchdog timers.
+
+To avoid bursts of traffic, implementations should attempt to stagger the expiry
+of the longer-term watchdog timers for different peer HSes.
+
+When individual users actively change their status (either by explicit requests
+from clients, or inferred changes due to idle timers or client timeouts), the HS
+should batch up any status changes for some reasonable amount of time (10
+seconds to a minute). This allows for reduced protocol overheads in the case of
+multiple messages needing to be sent to the same peer HS; as is the likely
+scenario in many cases, such as a given human user having multiple user
+accounts.
+
+
+API Requirements
+================
+
+The data model presented here puts the following requirements on the APIs:
+
+Client-Server
+-------------
+
+Requests that a client can make to its Home Server
+
+ * get/set current presence state
+   Basic enumeration + ability to set a custom piece of text
+
+ * report per-device idle time
+   After some (configurable?) idle time the device should send a single message
+   to set the idle duration. The HS can then infer a "start of idle" instant and
+   use that to keep the device idleness up to date. At some later point the
+   device can cancel this idleness.
+
+ * report per-device type
+   Inform the server that this device is a "mobile" device, or perhaps some
+   other to-be-defined category of reduced capability that could be presented to
+   other users.
+
+ * start/stop presence polling for my presence list
+   It is likely that these messages could be implicitly inferred by other
+   messages, though having explicit control is always useful.
+
+ * get my presence list
+   [implicit poll start?]
+   It is possible that the HS doesn't yet have current presence information when
+   the client requests this. There should be a "don't know" type too.
+
+ * add/remove a user to my presence list
+
+Server-Server
+-------------
+
+Requests that Home Servers make to others
+
+ * request permission to add a user to presence list
+
+ * allow/deny a request to add to a presence list
+
+ * perform a combined presence state push and subscription request
+   For each sending user ID, the message contains their new status.
+   For each receiving user ID, the message should contain an indication on
+   whether the sending server is also interested in receiving status from that
+   user; either as an immediate update response now, or as a promise to send
+   future updates.
+
+Server to Client
+----------------
+
+[[TODO(paul): There also needs to be some way for a user's HS to push status
+updates of the presence list to clients, but the general server-client event
+model currently lacks a space to do that.]]
diff --git a/docs/model/profiles b/docs/model/profiles
new file mode 100644
index 0000000000..f7d6bd5679
--- /dev/null
+++ b/docs/model/profiles
@@ -0,0 +1,232 @@
+========
+Profiles
+========
+
+A description of Synapse user profile metadata support.
+
+
+Overview
+========
+
+Internally within Synapse users are referred to by an opaque ID, which consists
+of some opaque localpart combined with the domain name of their home server.
+Obviously this does not yield a very nice user experience; users would like to
+see readable names for other users that are in some way meaningful to them.
+Additionally, users like to be able to publish "profile" details to inform other
+users of other information about them.
+
+It is also conceivable that since we are attempting to provide a
+worldwide-applicable messaging system, that users may wish to present different
+subsets of information in their profile to different other people, from a
+privacy and permissions perspective.
+
+A Profile consists of a display name, an (optional?) avatar picture, and a set
+of other metadata fields that the user may wish to publish (email address, phone
+numbers, website URLs, etc...). We put no requirements on the display name other
+than it being a valid Unicode string. Since it is likely that users will end up
+having multiple accounts (perhaps by necessity of being hosted in multiple
+places, perhaps by choice of wanting multiple distinct identifies), it would be
+useful that a metadata field type exists that can refer to another Synapse User
+ID, so that clients and HSes can make use of this information.
+
+Metadata Fields
+---------------
+
+[[TODO(paul): Likely this list is incomplete; more fields can be defined as we
+think of them. At the very least, any sort of supported ID for the 3rd Party ID
+servers should be accounted for here.]]
+
+ * Synapse Directory Server username(s)
+
+ * Email address
+
+ * Phone number - classify "home"/"work"/"mobile"/custom?
+ 
+ * Twitter/Facebook/Google+/... social networks
+
+ * Location - keep this deliberately vague to allow people to choose how
+     granular it is
+ 
+ * "Bio" information - date of birth, etc...
+
+ * Synapse User ID of another account
+
+ * Web URL
+
+ * Freeform description text
+
+
+Visibility Permissions
+======================
+
+A home server implementation could offer the ability to set permissions on
+limited visibility of those fields. When another user requests access to the
+target user's profile, their own identity should form part of that request. The
+HS implementation can then decide which fields to make available to the
+requestor.
+
+A particular detail of implementation could allow the user to create one or more
+ACLs; where each list is granted permission to see a given set of non-public
+fields (compare to Google+ Circles) and contains a set of other people allowed
+to use it. By giving these ACLs strong identities within the HS, they can be
+referenced in communications with it, granting other users who encounter these
+the "ACL Token" to use the details in that ACL.
+
+If we further allow an ACL Token to be present on Room join requests or stored
+by 3PID servers, then users of these ACLs gain the extra convenience of not
+having to manually curate people in the access list; anyone in the room or with
+knowledge of the 3rd Party ID is automatically granted access. Every HS and
+client implementation would have to be aware of the existence of these ACL
+Token, and include them in requests if present, but not every HS implementation
+needs to actually provide the full permissions model. This can be used as a
+distinguishing feature among competing implementations. However, servers MUST
+NOT serve profile information from a cache if there is a chance that its limited
+understanding could lead to information leakage.
+
+
+Client Concerns of Multiple Accounts
+====================================
+
+Because a given person may want to have multiple Synapse User accounts, client
+implementations should allow the use of multiple accounts simultaneously
+(especially in the field of mobile phone clients, which generally don't support
+running distinct instances of the same application). Where features like address
+books, presence lists or rooms are presented, the client UI should remember to
+make distinct with user account is in use for each.
+
+
+Directory Servers
+=================
+
+Directory Servers can provide a forward mapping from human-readable names to
+User IDs. These can provide a service similar to giving domain-namespaced names
+for Rooms; in this case they can provide a way for a user to reference their
+User ID in some external form (e.g. that can be printed on a business card).
+
+The format for Synapse user name will consist of a localpart specific to the
+directory server, and the domain name of that directory server:
+
+  @localname:some.domain.name
+
+The localname is separated from the domain name using a colon, so as to ensure
+the localname can still contain periods, as users may want this for similarity
+to email addresses or the like, which typically can contain them. The format is
+also visually quite distinct from email addresses, phone numbers, etc... so
+hopefully reasonably "self-describing" when written on e.g. a business card
+without surrounding context.
+
+[[TODO(paul): we might have to think about this one - too close to email?
+  Twitter? Also it suggests a format scheme for room names of
+  #localname:domain.name, which I quite like]]
+
+Directory server administrators should be able to make some kind of policy
+decision on how these are allocated. Servers within some "closed" domain (such
+as company-specific ones) may wish to verify the validity of a mapping using
+their own internal mechanisms; "public" naming servers can operate on a FCFS
+basis. There are overlapping concerns here with the idea of the 3rd party
+identity servers as well, though in this specific case we are creating a new
+namespace to allocate names into.
+
+It would also be nice from a user experience perspective if the profile that a
+given name links to can also declare that name as part of its metadata.
+Furthermore as a security and consistency perspective it would be nice if each
+end (the directory server and the user's home server) check the validity of the
+mapping in some way. This needs investigation from a security perspective to
+ensure against spoofing.
+
+One such model may be that the user starts by declaring their intent to use a
+given user name link to their home server, which then contacts the directory
+service. At some point later (maybe immediately for "public open FCFS servers",
+maybe after some kind of human intervention for verification) the DS decides to
+honour this link, and includes it in its served output. It should also tell the
+HS of this fact, so that the HS can present this as fact when requested for the
+profile information. For efficiency, it may further wish to provide the HS with
+a cryptographically-signed certificate as proof, so the HS serving the profile
+can provide that too when asked, avoiding requesting HSes from constantly having
+to contact the DS to verify this mapping. (Note: This is similar to the security
+model often applied in DNS to verify PTR <-> A bidirectional mappings).
+
+
+Identity Servers
+================
+
+The identity servers should support the concept of pointing a 3PID being able to
+store an ACL Token as well as the main User ID. It is however, beyond scope to
+do any kind of verification that any third-party IDs that the profile is
+claiming match up to the 3PID mappings.
+
+
+User Interface and Expectations Concerns
+========================================
+
+Given the weak "security" of some parts of this model as compared to what users
+might expect, some care should be taken on how it is presented to users,
+specifically in the naming or other wording of user interface components.
+
+Most notably mere knowledge of an ACL Pointer is enough to read the information
+stored in it. It is possible that Home or Identity Servers could leak this
+information, allowing others to see it. This is a security-vs-convenience
+balancing choice on behalf of the user who would choose, or not, to make use of
+such a feature to publish their information.
+
+Additionally, unless some form of strong end-to-end user-based encryption is
+used, a user of ACLs for information privacy has to trust other home servers not
+to lie about the identify of the user requesting access to the Profile.
+
+
+API Requirements
+================
+
+The data model presented here puts the following requirements on the APIs:
+
+Client-Server
+-------------
+
+Requests that a client can make to its Home Server
+
+ * get/set my Display Name
+   This should return/take a simple "text/plain" field
+
+ * get/set my Avatar URL
+   The avatar image data itself is not stored by this API; we'll just store a
+   URL to let the clients fetch it. Optionally HSes could integrate this with
+   their generic content attacmhent storage service, allowing a user to set
+   upload their profile Avatar and update the URL to point to it.
+
+ * get/add/remove my metadata fields
+   Also we need to actually define types of metadata
+
+ * get another user's Display Name / Avatar / metadata fields
+
+[[TODO(paul): At some later stage we should consider the API for:
+
+ * get/set ACL permissions on my metadata fields
+
+ * manage my ACL tokens
+]]
+
+Server-Server
+-------------
+
+Requests that Home Servers make to others
+
+ * get a user's Display Name / Avatar
+
+ * get a user's full profile - name/avatar + MD fields
+   This request must allow for specifying the User ID of the requesting user,
+   for permissions purposes. It also needs to take into account any ACL Tokens
+   the requestor has.
+
+ * push a change of Display Name to observers (overlaps with the presence API)
+
+Room Event PDU Types
+--------------------
+
+Events that are pushed from Home Servers to other Home Servers or clients.
+
+ * user Display Name change
+ 
+ * user Avatar change
+   [[TODO(paul): should the avatar image itself be stored in all the room
+   histories? maybe this event should just be a hint to clients that they should
+   re-fetch the avatar image]]
diff --git a/docs/model/room-join-workflow b/docs/model/room-join-workflow
new file mode 100644
index 0000000000..c321a64fab
--- /dev/null
+++ b/docs/model/room-join-workflow
@@ -0,0 +1,113 @@
+==================
+Room Join Workflow
+==================
+
+An outline of the workflows required when a user joins a room.
+
+Discovery
+=========
+
+To join a room, a user has to discover the room by some mechanism in order to
+obtain the (opaque) Room ID and a candidate list of likely home servers that
+contain it.
+
+Sending an Invitation
+---------------------
+
+The most direct way a user discovers the existence of a room is from a
+invitation from some other user who is a member of that room.
+
+The inviter's HS sets the membership status of the invitee to "invited" in the
+"m.members" state key by sending a state update PDU. The HS then broadcasts this
+PDU among the existing members in the usual way. An invitation message is also
+sent to the invited user, containing the Room ID and the PDU ID of this
+invitation state change and potentially a list of some other home servers to use
+to accept the invite. The user's client can then choose to display it in some
+way to alert the user.
+
+[[TODO(paul): At present, no API has been designed or described to actually send
+that invite to the invited user. Likely it will be some facet of the larger
+user-user API required for presence, profile management, etc...]]
+
+Directory Service
+-----------------
+
+Alternatively, the user may discover the channel via a directory service; either
+by performing a name lookup, or some kind of browse or search acitivty. However
+this is performed, the end result is that the user's home server requests the
+Room ID and candidate list from the directory service.
+
+[[TODO(paul): At present, no API has been designed or described for this
+directory service]]
+
+
+Joining
+=======
+
+Once the ID and home servers are obtained, the user can then actually join the
+room.
+
+Accepting an Invite
+-------------------
+
+If a user has received and accepted an invitation to join a room, the invitee's
+home server can now send an invite acceptance message to a chosen candidate
+server from the list given in the invitation, citing also the PDU ID of the
+invitation as "proof" of their invite. (This is required as due to late message
+propagation it could be the case that the acceptance is received before the
+invite by some servers). If this message is allowed by the candidate server, it
+generates a new PDU that updates the invitee's membership status to "joined",
+referring back to the acceptance PDU, and broadcasts that as a state change in
+the usual way. The newly-invited user is now a full member of the room, and
+state propagation proceeds as usual.
+
+Joining a Public Room
+---------------------
+
+If a user has discovered the existence of a room they wish to join but does not
+have an active invitation, they can request to join it directly by sending a
+join message to a candidate server on the list provided by the directory
+service. As this list may be out of date, the HS should be prepared to retry
+other candidates if the chosen one is no longer aware of the room, because it
+has no users as members in it.
+
+Once a candidate server that is aware of the room has been found, it can
+broadcast an update PDU to add the member into the "m.members" key setting their
+state directly to "joined" (i.e. bypassing the two-phase invite semantics),
+remembering to include the new user's HS in that list.
+
+Knocking on a Semi-Public Room
+------------------------------
+
+If a user requests to join a room but the join mode of the room is "knock", the
+join is not immediately allowed. Instead, if the user wishes to proceed, they
+can instead post a "knock" message, which informs other members of the room that
+the would-be joiner wishes to become a member and sets their membership value to
+"knocked". If any of them wish to accept this, they can then send an invitation
+in the usual way described above. Knowing that the user has already knocked and
+expressed an interest in joining, the invited user's home server should
+immediately accept that invitation on the user's behalf, and go on to join the
+room in the usual way.
+
+[[NOTE(Erik): Though this may confuse users who expect 'X has joined' to
+actually be a user initiated action, i.e. they may expect that 'X' is actually
+looking at synapse right now?]]
+
+[[NOTE(paul): Yes, a fair point maybe we should suggest HSes don't do that, and
+just offer an invite to the user as normal]]
+
+Private and Non-Existent Rooms
+------------------------------
+
+If a user requests to join a room but the room is either unknown by the home
+server receiving the request, or is known by the join mode is "invite" and the
+user has not been invited, the server must respond that the room does not exist.
+This is to prevent leaking information about the existence and identity of
+private rooms.
+
+
+Outstanding Questions
+=====================
+
+ * Do invitations or knocks time out and expire at some point? If so when? Time
+   is hard in distributed systems.
diff --git a/docs/model/rooms b/docs/model/rooms
new file mode 100644
index 0000000000..0007e48e30
--- /dev/null
+++ b/docs/model/rooms
@@ -0,0 +1,274 @@
+===========
+Rooms Model
+===========
+
+A description of the general data model used to implement Rooms, and the
+user-level visible effects and implications.
+
+
+Overview
+========
+
+"Rooms" in Synapse are shared messaging channels over which all the participant
+users can exchange messages. Rooms have an opaque persistent identify, a
+globally-replicated set of state (consisting principly of a membership set of
+users, and other management and miscellaneous metadata), and a message history.
+
+
+Room Identity and Naming
+========================
+
+Rooms can be arbitrarily created by any user on any home server; at which point
+the home server will sign the message that creates the channel, and the
+fingerprint of this signature becomes the strong persistent identify of the
+room. This now identifies the room to any home server in the network regardless
+of its original origin. This allows the identify of the room to outlive any
+particular server. Subject to appropriate permissions [to be discussed later],
+any current member of a room can invite others to join it, can post messages
+that become part of its history, and can change the persistent state of the room
+(including its current set of permissions).
+
+Home servers can provide a directory service, allowing a lookup from a
+convenient human-readable form of room label to a room ID. This mapping is
+scoped to the particular home server domain and so simply represents that server
+administrator's opinion of what room should take that label; it does not have to
+be globally replicated and does not form part of the stored state of that room.
+
+This room name takes the form
+
+  #localname:some.domain.name
+
+for similarity and consistency with user names on directories.
+
+To join a room (and therefore to be allowed to inspect past history, post new
+messages to it, and read its state), a user must become aware of the room's
+fingerprint ID. There are two mechanisms to allow this:
+
+ * An invite message from someone else in the room
+
+ * A referral from a room directory service
+
+As room IDs are opaque and ephemeral, they can serve as a mechanism to create
+"ad-hoc" rooms deliberately unnamed, for small group-chats or even private
+one-to-one message exchange.
+
+
+Stored State and Permissions
+============================
+
+Every room has a globally-replicated set of stored state. This state is a set of
+key/value or key/subkey/value pairs. The value of every (sub)key is a
+JSON-representable object. The main key of a piece of stored state establishes
+its meaning; some keys store sub-keys to allow a sub-structure within them [more
+detail below]. Some keys have special meaning to Synapse, as they relate to
+management details of the room itself, storing such details as user membership,
+and permissions of users to alter the state of the room itself. Other keys may
+store information to present to users, which the system does not directly rely
+on. The key space itself is namespaced, allowing 3rd party extensions, subject
+to suitable permission.
+
+Permission management is based on the concept of "power-levels". Every user
+within a room has an integer assigned, being their "power-level" within that
+room. Along with its actual data value, each key (or subkey) also stores the
+minimum power-level a user must have in order to write to that key, the
+power-level of the last user who actually did write to it, and the PDU ID of
+that state change.
+
+To be accepted as valid, a change must NOT:
+
+ * Be made by a user having a power-level lower than required to write to the
+   state key
+
+ * Alter the required power-level for that state key to a value higher than the
+   user has
+
+ * Increase that user's own power-level
+
+ * Grant any other user a power-level higher than the level of the user making
+   the change
+
+[[TODO(paul): consider if relaxations should be allowed; e.g. is the current
+outright-winner allowed to raise their own level, to allow for "inflation"?]]
+
+
+Room State Keys
+===============
+
+[[TODO(paul): if this list gets too big it might become necessary to move it
+into its own doc]]
+
+The following keys have special semantics or meaning to Synapse itself:
+
+m.member (has subkeys)
+  Stores a sub-key for every Synapse User ID which is currently a member of
+  this room. Its value gives the membership type ("knocked", "invited",
+  "joined").
+
+m.power_levels
+  Stores a mapping from Synapse User IDs to their power-level in the room. If
+  they are not present in this mapping, the default applies.
+
+  The reason to store this as a single value rather than a value with subkeys
+  is that updates to it are atomic; allowing a number of colliding-edit
+  problems to be avoided.
+
+m.default_level
+  Gives the default power-level for members of the room that do not have one
+  specified in their membership key.
+
+m.invite_level
+  If set, gives the minimum power-level required for members to invite others
+  to join, or to accept knock requests from non-members requesting access. If
+  absent, then invites are not allowed. An invitation involves setting their
+  membership type to "invited", in addition to sending the invite message.
+
+m.join_rules
+  Encodes the rules on how non-members can join the room. Has the following
+  possibilities:
+    "public" - a non-member can join the room directly
+    "knock" - a non-member cannot join the room, but can post a single "knock"
+        message requesting access, which existing members may approve or deny
+    "invite" - non-members cannot join the room without an invite from an
+        existing member
+    "private" - nobody who is not in the 'may_join' list or already a member
+        may join by any mechanism
+
+  In any of the first three modes, existing members with sufficient permission
+  can send invites to non-members if allowed by the "m.invite_level" key. A
+  "private" room is not allowed to have the "m.invite_level" set.
+
+  A client may use the value of this key to hint at the user interface
+  expectations to provide; in particular, a private chat with one other use
+  might warrant specific handling in the client.
+
+m.may_join
+  A list of User IDs that are always allowed to join the room, regardless of any
+  of the prevailing join rules and invite levels. These apply even to private
+  rooms. These are stored in a single list with normal update-powerlevel
+  permissions applied; users cannot arbitrarily remove themselves from the list.
+
+m.add_state_level
+  The power-level required for a user to be able to add new state keys.
+
+m.public_history
+  If set and true, anyone can request the history of the room, without needing
+  to be a member of the room.
+
+m.archive_servers
+  For "public" rooms with public history, gives a list of home servers that
+  should be included in message distribution to the room, even if no users on
+  that server are present. These ensure that a public room can still persist
+  even if no users are currently members of it. This list should be consulted by
+  the dirctory servers as the candidate list they respond with.
+
+The following keys are provided by Synapse for user benefit, but their value is
+not otherwise used by Synapse.
+
+m.name
+  Stores a short human-readable name for the room, such that clients can display
+  to a user to assist in identifying which room is which.
+  
+  This name specifically is not the strong ID used by the message transport
+  system to refer to the room, because it may be changed from time to time.
+
+m.topic
+  Stores the current human-readable topic
+
+
+Room Creation Templates
+=======================
+
+A client (or maybe home server?) could offer a few templates for the creation of
+new rooms. For example, for a simple private one-to-one chat the channel could
+assign the creator a power-level of 1, requiring a level of 1 to invite, and
+needing an invite before members can join. An invite is then sent to the other
+party, and if accepted and the other user joins, the creator's power-level can
+now be reduced to 0. This now leaves a room with two participants in it being
+unable to add more.
+
+
+Rooms that Continue History
+===========================
+
+An option that could be considered for room creation, is that when a new room is
+created the creator could specify a PDU ID into an existing room, as the history
+continuation point. This would be stored as an extra piece of meta-data on the
+initial PDU of the room's creation. (It does not appear in the normal previous
+PDU linkage).
+
+This would allow users in rooms to "fork" a room, if it is considered that the
+conversations in the room no longer fit its original purpose, and wish to
+diverge. Existing permissions on the original room would continue to apply of
+course, for viewing that history. If both rooms are considered "public" we might
+also want to define a message to post into the original room to represent this
+fork point, and give a reference to the new room.
+
+
+User Direct Message Rooms
+=========================
+
+There is no need to build a mechanism for directly sending messages between
+users, because a room can handle this ability. To allow direct user-to-user chat
+messaging we simply need to be able to create rooms with specific set of
+permissions to allow this direct messaging.
+
+Between any given pair of user IDs that wish to exchange private messages, there
+will exist a single shared Room, created lazily by either side. These rooms will
+need a certain amount of special handling in both home servers and display on
+clients, but as much as possible should be treated by the lower layers of code
+the same as other rooms.
+
+Specially, a client would likely offer a special menu choice associated with
+another user (in room member lists, presence list, etc..) as "direct chat". That
+would perform all the necessary steps to create the private chat room. Receiving
+clients should display these in a special way too as the room name is not
+important; instead it should distinguish them on the Display Name of the other
+party.
+
+Home Servers will need a client-API option to request setting up a new user-user
+chat room, which will then need special handling within the server. It will
+create a new room with the following 
+
+  m.member: the proposing user
+  m.join_rules: "private"
+  m.may_join: both users
+  m.power_levels: empty
+  m.default_level: 0
+  m.add_state_level: 0
+  m.public_history: False
+
+Having created the room, it can send an invite message to the other user in the
+normal way - the room permissions state that no users can be set to the invited
+state, but because they're in the may_join list then they'd be allowed to join
+anyway.
+
+In this arrangement there is now a room with both users may join but neither has
+the power to invite any others. Both users now have the confidence that (at
+least within the messaging system itself) their messages remain private and
+cannot later be provably leaked to a third party. They can freely set the topic
+or name if they choose and add or edit any other state of the room. The update
+powerlevel of each of these fixed properties should be 1, to lock out the users
+from being able to alter them.
+
+
+Anti-Glare
+==========
+
+There exists the possibility of a race condition if two users who have no chat
+history with each other simultaneously create a room and invite the other to it.
+This is called a "glare" situation. There are two possible ideas for how to
+resolve this:
+
+ * Each Home Server should persist the mapping of (user ID pair) to room ID, so
+   that duplicate requests can be suppressed. On receipt of a room creation
+   request that the HS thinks there already exists a room for, the invitation to
+   join can be rejected if:
+      a) the HS believes the sending user is already a member of the room (and
+         maybe their HS has forgotten this fact), or
+      b) the proposed room has a lexicographically-higher ID than the existing
+         room (to resolve true race condition conflicts)
+      
+ * The room ID for a private 1:1 chat has a special form, determined by
+   concatenting the User IDs of both members in a deterministic order, such that
+   it doesn't matter which side creates it first; the HSes can just ignore
+   (or merge?) received PDUs that create the room twice.
diff --git a/docs/model/third-party-id b/docs/model/third-party-id
new file mode 100644
index 0000000000..1f8138ddf7
--- /dev/null
+++ b/docs/model/third-party-id
@@ -0,0 +1,108 @@
+======================
+Third Party Identities
+======================
+
+A description of how email addresses, mobile phone numbers and other third
+party identifiers can be used to authenticate and discover users in Matrix.
+
+
+Overview
+========
+
+New users need to authenticate their account. An email or SMS text message can 
+be a convenient form of authentication. Users already have email addresses 
+and phone numbers for contacts in their address book. They want to communicate
+with those contacts in Matrix without manually exchanging a Matrix User ID with 
+them.
+
+Third Party IDs
+---------------
+
+[[TODO(markjh): Describe the format of a 3PID]]
+
+
+Third Party ID Associations
+---------------------------
+
+An Associaton is a binding between a Matrix User ID and a Third Party ID (3PID).
+Each 3PID can be associated with one Matrix User ID at a time.
+
+[[TODO(markjh): JSON format of the association.]]
+
+Verification 
+------------
+
+An Assocation must be verified by a trusted Verification Server. Email 
+addresses and phone numbers can be verified by sending a token to the address 
+which a client can supply to the verifier to confirm ownership.
+
+An email Verification Server may be capable of verifying all email 3PIDs or may
+be restricted to verifying addresses for a particular domain. A phone number
+Verification Server may be capable of verifying all phone numbers or may be
+restricted to verifying numbers for a given country or phone prefix.
+
+Verification Servers fulfil a similar role to Certificate Authorities in PKI so
+a similar level of vetting should be required before clients trust their
+signatures.
+
+A Verification Server may wish to check for existing Associations for a 3PID 
+before creating a new Association.
+
+Discovery
+---------
+
+Users can discover Associations using a trusted Identity Server. Each 
+Association will be signed by the Identity Server. An Identity Server may store
+the entire space of Associations or may delegate to other Identity Servers when
+looking up Associations.
+
+Each Association returned from an Identity Server must be signed by a 
+Verification Server. Clients should check these signatures.
+
+Identity Servers fulfil a similar role to DNS servers.
+
+Privacy
+-------
+
+A User may publish the association between their phone number and Matrix User ID
+on the Identity Server without publishing the number in their Profile hosted on
+their Home Server.
+
+Identity Servers should refrain from publishing reverse mappings and should 
+take steps, such as rate limiting, to prevent attackers enumerating the space of
+mappings.
+
+Federation
+==========
+
+Delegation
+----------
+
+Verification Servers could delegate signing to another server by issuing 
+certificate to that server allowing it to verify and sign a subset of 3PID on 
+its behalf. It would be necessary to provide a language for describing which
+subset of 3PIDs that server had authority to validate. Alternatively it could 
+delegate the verification step to another server but sign the resulting
+association itself.
+
+The 3PID space will have a heirachical structure like DNS so Identity Servers
+can delegate lookups to other servers. An Identity Server should be prepared 
+to host or delegate any valid association within the subset of the 3PIDs it is 
+resonsible for.
+
+Multiple Root Verification Servers
+----------------------------------
+
+There can be multiple root Verification Servers and an Association could be
+signed by multiple servers if different clients trust different subsets of
+the verification servers.
+
+Multiple Root Identity Servers
+------------------------------
+
+There can be be multiple root Identity Servers. Clients will add each
+Association to all root Identity Servers.
+
+[[TODO(markjh): Describe how clients find the list of root Identity Servers]]
+
+
diff --git a/docs/protocol_examples b/docs/protocol_examples
new file mode 100644
index 0000000000..61a599b432
--- /dev/null
+++ b/docs/protocol_examples
@@ -0,0 +1,64 @@
+PUT /send/abc/ HTTP/1.1
+Host: ...
+Content-Length: ...
+Content-Type: application/json
+
+{
+    "origin": "localhost:5000",
+    "pdus": [
+        {
+            "content": {},
+            "context": "tng",
+            "depth": 12,
+            "is_state": false,
+            "origin": "localhost:5000",
+            "pdu_id": 1404381396854,
+            "pdu_type": "feedback",
+            "prev_pdus": [
+                [
+                    "1404381395883",
+                    "localhost:6000"
+                ]
+            ],
+            "ts": 1404381427581
+        }
+    ],
+    "prev_ids": [
+        "1404381396852"
+    ],
+    "ts": 1404381427823
+}
+
+HTTP/1.1 200 OK
+...
+
+======================================
+
+GET /pull/-1/ HTTP/1.1
+Host: ...
+Content-Length: 0
+
+HTTP/1.1 200 OK
+Content-Length: ...
+Content-Type: application/json
+
+{
+    origin: ...,
+    prev_ids: ...,
+    data: [
+        {
+            data_id: ...,
+            prev_pdus: [...],
+            depth: ...,
+            ts: ...,
+            context: ...,
+            origin: ...,
+            content: {
+                ...
+            }
+        },
+        ...,
+    ]
+}
+
+
diff --git a/docs/python_architecture b/docs/python_architecture
new file mode 100644
index 0000000000..eca36a1902
--- /dev/null
+++ b/docs/python_architecture
@@ -0,0 +1,53 @@
+= Server to Server =
+
+== Server to Server Stack ==
+
+To use the server to server stack, home servers should only need to interact with the Messaging layer.
+
+The server to server side of things is designed into 4 distinct layers:
+
+    1. Messaging Layer
+    2. Pdu Layer
+    3. Transaction Layer
+    4. Transport Layer
+
+Where the bottom (the transport layer) is what talks to the internet via HTTP, and the top (the messaging layer) talks to the rest of the Home Server with a domain specific API.
+
+1. Messaging Layer
+    This is what the rest of the Home Server hits to send messages, join rooms, etc. It also allows you to register callbacks for when it get's notified by lower levels that e.g. a new message has been received.
+
+    It is responsible for serializing requests to send to the data layer, and to parse requests received from the data layer.
+
+
+2. PDU Layer
+    This layer handles: 
+        * duplicate pdu_id's - i.e., it makes sure we ignore them. 
+        * responding to requests for a given pdu_id
+        * responding to requests for all metadata for a given context (i.e. room)
+        * handling incoming pagination requests
+
+    So it has to parse incoming messages to discover which are metadata and which aren't, and has to correctly clobber existing metadata where appropriate.
+
+    For incoming PDUs, it has to check the PDUs it references to see if we have missed any. If we have go and ask someone (another home server) for it.    
+
+
+3. Transaction Layer
+    This layer makes incoming requests idempotent. I.e., it stores which transaction id's we have seen and what our response were. If we have already seen a message with the given transaction id, we do not notify higher levels but simply respond with the previous response.
+
+transaction_id is from "GET /send/<tx_id>/"
+
+    It's also responsible for batching PDUs into single transaction for sending to remote destinations, so that we only ever have one transaction in flight to a given destination at any one time.
+
+    This is also responsible for answering requests for things after a given set of transactions, i.e., ask for everything after 'ver' X.
+
+
+4. Transport Layer
+    This is responsible for starting a HTTP server and hitting the correct callbacks on the Transaction layer, as well as sending both data and requests for data.
+
+
+== Persistence ==
+
+We persist things in a single sqlite3 database. All database queries get run on a separate, dedicated thread. This that we only ever have one query running at a time, making it a lot easier to do things in a safe manner.
+
+The queries are located in the synapse.persistence.transactions module, and the table information in the synapse.persistence.tables module.
+
diff --git a/docs/server-server/protocol-format b/docs/server-server/protocol-format
new file mode 100644
index 0000000000..2838253ab7
--- /dev/null
+++ b/docs/server-server/protocol-format
@@ -0,0 +1,59 @@
+
+Transaction
+===========
+
+Required keys:
+
+============ =================== ===============================================
+    Key            Type                         Description
+============ =================== ===============================================
+origin       String              DNS name of homeserver making this transaction.
+ts           Integer             Timestamp in milliseconds on originating 
+                                 homeserver when this transaction started.
+previous_ids List of Strings     List of transactions that were sent immediately
+                                 prior to this transaction.
+pdus         List of Objects     List of updates contained in this transaction.
+============ =================== ===============================================
+
+
+PDU
+===
+
+Required keys:
+
+============ ================== ================================================
+    Key            Type                         Description
+============ ================== ================================================
+context      String             Event context identifier
+origin       String             DNS name of homeserver that created this PDU.
+pdu_id       String             Unique identifier for PDU within the context for
+                                the originating homeserver.
+ts           Integer            Timestamp in milliseconds on originating 
+                                homeserver when this PDU was created.
+pdu_type     String             PDU event type.
+prev_pdus    List of Pairs      The originating homeserver and PDU ids of the
+             of Strings         most recent PDUs the homeserver was aware of for
+                                this context when it made this PDU.
+depth        Integer            The maximum depth of the previous PDUs plus one.
+============ ================== ================================================
+
+Keys for state updates:
+
+================== ============ ================================================
+    Key               Type                      Description
+================== ============ ================================================
+is_state           Boolean      True if this PDU is updating state.
+state_key          String       Optional key identifying the updated state within
+                                the context.
+power_level        Integer      The asserted power level of the user performing
+                                the update.
+min_update         Integer      The required power level needed to replace this
+                                update.
+prev_state_id      String       The homeserver of the update this replaces
+prev_state_origin  String       The PDU id of the update this replaces.
+user               String       The user updating the state.
+================== ============ ================================================
+
+
+
+
diff --git a/docs/server-server/security-threat-model b/docs/server-server/security-threat-model
new file mode 100644
index 0000000000..cf0430e43d
--- /dev/null
+++ b/docs/server-server/security-threat-model
@@ -0,0 +1,141 @@
+Overview
+========
+
+Scope
+-----
+
+This document considers threats specific to the server to server federation 
+synapse protocol.
+
+
+Attacker
+--------
+
+It is assumed that the attacker can see and manipulate all network traffic 
+between any of the servers and may be in control of one or more homeservers 
+participating in the federation protocol.
+
+Threat Model
+============
+
+Denial of Service
+-----------------
+
+The attacker could attempt to prevent delivery of messages to or from the 
+victim in order to:
+
+    * Disrupt service or marketing campaign of a commercial competitor.
+    * Censor a discussion or censor a participant in a discussion.
+    * Perform general vandalism.
+
+Threat: Resource Exhaustion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could cause the victims server to exhaust a particular resource 
+(e.g. open TCP connections, CPU, memory, disk storage)
+
+Threat: Unrecoverable Consistency Violations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could send messages which created an unrecoverable "split-brain"
+state in the cluster such that the victim's servers could no longer dervive a
+consistent view of the chatroom state.
+
+Threat: Bad History
+~~~~~~~~~~~~~~~~~~~
+
+An attacker could convince the victim to accept invalid messages which the 
+victim would then include in their view of the chatroom history. Other servers
+in the chatroom would reject the invalid messages and potentially reject the
+victims messages as well since they depended on the invalid messages.
+
+Threat: Block Network Traffic
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to firewall traffic between the victim's server and some
+or all of the other servers in the chatroom.
+
+Threat: High Volume of Messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could send large volumes of messages to a chatroom with the victim
+making the chatroom unusable.
+
+Threat: Banning users without necessary authorisation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could attempt to ban a user from a chatroom with the necessary
+authorisation.
+
+Spoofing
+--------
+
+An attacker could try to send a message claiming to be from the victim without 
+the victim having sent the message in order to:
+
+    * Impersonate the victim while performing illict activity.
+    * Obtain privileges of the victim.
+
+Threat: Altering Message Contents
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to alter the contents of an existing message from the 
+victim.
+
+Threat: Fake Message "origin" Field
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to send a new message purporting to be from the victim
+with a phony "origin" field.
+
+Spamming
+--------
+
+The attacker could try to send a high volume of solicicted or unsolicted 
+messages to the victim in order to:
+    
+    * Find victims for scams.
+    * Market unwanted products.
+
+Threat: Unsoliticted Messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to send messages to victims who do not wish to receive 
+them.
+
+Threat: Abusive Messages
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could send abusive or threatening messages to the victim
+
+Spying
+------
+
+The attacker could try to access message contents or metadata for messages sent
+by the victim or to the victim that were not intended to reach the attacker in
+order to:
+
+    * Gain sensitive personal or commercial information.
+    * Impersonate the victim using credentials contained in the messages.
+      (e.g. password reset messages)
+    * Discover who the victim was talking to and when.
+
+Threat: Disclosure during Transmission
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to expose the message contents or metadata during 
+transmission between the servers.
+
+Threat: Disclosure to Servers Outside Chatroom
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could try to convince servers within a chatroom to send messages to
+a server it controls that was not authorised to be within the chatroom.
+
+Threat: Disclosure to Servers Within Chatroom
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+An attacker could take control of a server within a chatroom to expose message
+contents or metadata for messages in that room.
+
+
diff --git a/docs/server-server/specification b/docs/server-server/specification
new file mode 100644
index 0000000000..e1e49cc069
--- /dev/null
+++ b/docs/server-server/specification
@@ -0,0 +1,177 @@
+============================
+Synapse Server-to-Server API
+============================
+
+A description of the protocol used to communicate between Synapse home servers;
+also known as Federation.
+
+
+Overview
+========
+
+The server-server API is a mechanism by which two home servers can exchange
+Synapse event messages, both as a real-time push of current events, and as a
+historic fetching mechanism to synchronise past history for clients to view. It
+uses HTTP connections between each pair of servers involved as the underlying
+transport. Messages are exchanged between servers in real-time by active pushing
+from each server's HTTP client into the server of the other. Queries to fetch
+historic data for the purpose of back-filling scrollback buffers and the like
+can also be performed.
+
+
+ { Synapse entities }                            { Synapse entities }
+     ^          |                                    ^          |
+     |  events  |                                    |  events  |
+     |          V                                    |          V
+ +------------------+                            +------------------+
+ |                  |---------( HTTP )---------->|                  |
+ |   Home Server    |                            |   Home Server    |
+ |                  |<--------( HTTP )-----------|                  |
+ +------------------+                            +------------------+
+
+
+Transactions and PDUs
+=====================
+
+The communication between home servers is performed by a bidirectional exchange
+of messages. These messages are called Transactions, and are encoded as JSON
+objects with a dict as the top-level element, passed over HTTP. A Transaction is
+meaningful only to the pair of home servers that exchanged it; they are not
+globally-meaningful.
+
+Each transaction has an opaque ID and timestamp (UNIX epoch time in miliseconds)
+generated by its origin server, an origin and destination server name, a list of
+"previous IDs", and a list of PDUs - the actual message payload that the
+Transaction carries.
+
+ {"transaction_id":"916d630ea616342b42e98a3be0b74113",
+  "ts":1404835423000,
+  "origin":"red",
+  "destination":"blue",
+  "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
+  "pdus":[...]}
+
+The "previous IDs" field will contain a list of previous transaction IDs that
+the origin server has sent to this destination. Its purpose is to act as a
+sequence checking mechanism - the destination server can check whether it has
+successfully received that Transaction, or ask for a retransmission if not.
+
+The "pdus" field of a transaction is a list, containing zero or more PDUs.[*]
+Each PDU is itself a dict containing a number of keys, the exact details of
+which will vary depending on the type of PDU.
+
+(* Normally the PDU list will be non-empty, but the server should cope with
+receiving an "empty" transaction, as this is useful for informing peers of other
+transaction IDs they should be aware of. This effectively acts as a push
+mechanism to encourage peers to continue to replicate content.)
+
+All PDUs have an ID, a context, a declaration of their type, a list of other PDU
+IDs that have been seen recently on that context (regardless of which origin
+sent them), and a nested content field containing the actual event content.
+
+[[TODO(paul): Update this structure so that 'pdu_id' is a two-element
+[origin,ref] pair like the prev_pdus are]]
+
+ {"pdu_id":"a4ecee13e2accdadf56c1025af232176",
+  "context":"#example.green",
+  "origin":"green",
+  "ts":1404838188000,
+  "pdu_type":"m.text",
+  "prev_pdus":[["blue","99d16afbc857975916f1d73e49e52b65"]],
+  "content":...
+  "is_state":false}
+
+In contrast to the transaction layer, it is important to note that the prev_pdus
+field of a PDU refers to PDUs that any origin server has sent, rather than
+previous IDs that this origin has sent. This list may refer to other PDUs sent
+by the same origin as the current one, or other origins.
+
+Because of the distributed nature of participants in a Synapse conversation, it
+is impossible to establish a globally-consistent total ordering on the events.
+However, by annotating each outbound PDU at its origin with IDs of other PDUs it
+has received, a partial ordering can be constructed allowing causallity
+relationships to be preserved. A client can then display these messages to the
+end-user in some order consistent with their content and ensure that no message
+that is semantically in reply of an earlier one is ever displayed before it.
+
+PDUs fall into two main categories: those that deliver Events, and those that
+synchronise State. For PDUs that relate to State synchronisation, additional
+keys exist to support this:
+
+ {...,
+  "is_state":true,
+  "state_key":TODO
+  "power_level":TODO
+  "prev_state_id":TODO
+  "prev_state_origin":TODO}
+
+[[TODO(paul): At this point we should probably have a long description of how
+State management works, with descriptions of clobbering rules, power levels, etc
+etc... But some of that detail is rather up-in-the-air, on the whiteboard, and
+so on. This part needs refining. And writing in its own document as the details
+relate to the server/system as a whole, not specifically to server-server
+federation.]]
+
+
+Protocol URLs
+=============
+
+For active pushing of messages representing live activity "as it happens":
+
+  PUT /send/:transaction_id/
+    Body: JSON encoding of a single Transaction
+
+    Response: [[TODO(paul): I don't actually understand what
+    ReplicationLayer.on_transaction() is doing here, so I'm not sure what the
+    response ought to be]]
+
+  The transaction_id path argument will override any ID given in the JSON body.
+  The destination name will be set to that of the receiving server itself. Each
+  embedded PDU in the transaction body will be processed.
+
+
+To fetch a particular PDU:
+
+  GET /pdu/:origin/:pdu_id/
+
+    Response: JSON encoding of a single Transaction containing one PDU
+
+  Retrieves a given PDU from the server. The response will contain a single new
+  Transaction, inside which will be the requested PDU.
+  
+
+To fetch all the state of a given context:
+
+  GET /state/:context/
+
+    Response: JSON encoding of a single Transaction containing multiple PDUs
+
+  Retrieves a snapshot of the entire current state of the given context. The
+  response will contain a single Transaction, inside which will be a list of
+  PDUs that encode the state.
+
+
+To paginate events on a given context:
+
+  GET /paginate/:context/
+    Query args: v, limit
+
+    Response: JSON encoding of a single Transaction containing multiple PDUs
+
+  Retrieves a sliding-window history of previous PDUs that occurred on the
+  given context. Starting from the PDU ID(s) given in the "v" argument, the
+  PDUs that preceeded it are retrieved, up to a total number given by the
+  "limit" argument. These are then returned in a new Transaction containing all
+  off the PDUs.
+
+
+To stream events all the events:
+
+  GET /pull/
+    Query args: origin, v
+
+  Response: JSON encoding of a single Transaction consisting of multiple PDUs
+
+  Retrieves all of the transactions later than any version given by the "v"
+  arguments. [[TODO(paul): I'm not sure what the "origin" argument does because
+  I think at some point in the code it's got swapped around.]]
diff --git a/docs/sphinx/conf.py b/docs/sphinx/conf.py
new file mode 100644
index 0000000000..15c19834fc
--- /dev/null
+++ b/docs/sphinx/conf.py
@@ -0,0 +1,271 @@
+# -*- coding: utf-8 -*-
+#
+# Synapse documentation build configuration file, created by
+# sphinx-quickstart on Tue Jun 10 17:31:02 2014.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.insert(0, os.path.abspath('..'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.coverage',
+    'sphinx.ext.ifconfig',
+    'sphinxcontrib.napoleon',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix of source filenames.
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'Synapse'
+copyright = u'2014, TNG'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = '1.0'
+# The full version, including alpha/beta/rc tags.
+release = '1.0'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+html_theme = 'default'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Synapsedoc'
+
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+  ('index', 'Synapse.tex', u'Synapse Documentation',
+   u'TNG', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    ('index', 'synapse', u'Synapse Documentation',
+     [u'TNG'], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+  ('index', 'Synapse', u'Synapse Documentation',
+   u'TNG', 'Synapse', 'One line description of project.',
+   'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+
+# Example configuration for intersphinx: refer to the Python standard library.
+intersphinx_mapping = {'http://docs.python.org/': None}
+
+napoleon_include_special_with_doc = True
+napoleon_use_ivar = True
diff --git a/docs/sphinx/index.rst b/docs/sphinx/index.rst
new file mode 100644
index 0000000000..76a4c0c7bf
--- /dev/null
+++ b/docs/sphinx/index.rst
@@ -0,0 +1,20 @@
+.. Synapse documentation master file, created by
+   sphinx-quickstart on Tue Jun 10 17:31:02 2014.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to Synapse's documentation!
+===================================
+
+Contents:
+
+.. toctree::
+   synapse
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
diff --git a/docs/sphinx/modules.rst b/docs/sphinx/modules.rst
new file mode 100644
index 0000000000..1c7f70bd13
--- /dev/null
+++ b/docs/sphinx/modules.rst
@@ -0,0 +1,7 @@
+synapse
+=======
+
+.. toctree::
+   :maxdepth: 4
+
+   synapse
diff --git a/docs/sphinx/synapse.api.auth.rst b/docs/sphinx/synapse.api.auth.rst
new file mode 100644
index 0000000000..931eb59836
--- /dev/null
+++ b/docs/sphinx/synapse.api.auth.rst
@@ -0,0 +1,7 @@
+synapse.api.auth module
+=======================
+
+.. automodule:: synapse.api.auth
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.constants.rst b/docs/sphinx/synapse.api.constants.rst
new file mode 100644
index 0000000000..a1e3c47f68
--- /dev/null
+++ b/docs/sphinx/synapse.api.constants.rst
@@ -0,0 +1,7 @@
+synapse.api.constants module
+============================
+
+.. automodule:: synapse.api.constants
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.dbobjects.rst b/docs/sphinx/synapse.api.dbobjects.rst
new file mode 100644
index 0000000000..e9d31167e0
--- /dev/null
+++ b/docs/sphinx/synapse.api.dbobjects.rst
@@ -0,0 +1,7 @@
+synapse.api.dbobjects module
+============================
+
+.. automodule:: synapse.api.dbobjects
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.errors.rst b/docs/sphinx/synapse.api.errors.rst
new file mode 100644
index 0000000000..f1c6881478
--- /dev/null
+++ b/docs/sphinx/synapse.api.errors.rst
@@ -0,0 +1,7 @@
+synapse.api.errors module
+=========================
+
+.. automodule:: synapse.api.errors
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.event_stream.rst b/docs/sphinx/synapse.api.event_stream.rst
new file mode 100644
index 0000000000..9291cb2dbc
--- /dev/null
+++ b/docs/sphinx/synapse.api.event_stream.rst
@@ -0,0 +1,7 @@
+synapse.api.event_stream module
+===============================
+
+.. automodule:: synapse.api.event_stream
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.events.factory.rst b/docs/sphinx/synapse.api.events.factory.rst
new file mode 100644
index 0000000000..2e71ff6070
--- /dev/null
+++ b/docs/sphinx/synapse.api.events.factory.rst
@@ -0,0 +1,7 @@
+synapse.api.events.factory module
+=================================
+
+.. automodule:: synapse.api.events.factory
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.events.room.rst b/docs/sphinx/synapse.api.events.room.rst
new file mode 100644
index 0000000000..6cd5998599
--- /dev/null
+++ b/docs/sphinx/synapse.api.events.room.rst
@@ -0,0 +1,7 @@
+synapse.api.events.room module
+==============================
+
+.. automodule:: synapse.api.events.room
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.events.rst b/docs/sphinx/synapse.api.events.rst
new file mode 100644
index 0000000000..b762da55ee
--- /dev/null
+++ b/docs/sphinx/synapse.api.events.rst
@@ -0,0 +1,18 @@
+synapse.api.events package
+==========================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.api.events.factory
+   synapse.api.events.room
+
+Module contents
+---------------
+
+.. automodule:: synapse.api.events
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.events.rst b/docs/sphinx/synapse.api.handlers.events.rst
new file mode 100644
index 0000000000..d2e1b54ac0
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.events.rst
@@ -0,0 +1,7 @@
+synapse.api.handlers.events module
+==================================
+
+.. automodule:: synapse.api.handlers.events
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.factory.rst b/docs/sphinx/synapse.api.handlers.factory.rst
new file mode 100644
index 0000000000..b04a93f740
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.factory.rst
@@ -0,0 +1,7 @@
+synapse.api.handlers.factory module
+===================================
+
+.. automodule:: synapse.api.handlers.factory
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.federation.rst b/docs/sphinx/synapse.api.handlers.federation.rst
new file mode 100644
index 0000000000..61a6542210
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.federation.rst
@@ -0,0 +1,7 @@
+synapse.api.handlers.federation module
+======================================
+
+.. automodule:: synapse.api.handlers.federation
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.register.rst b/docs/sphinx/synapse.api.handlers.register.rst
new file mode 100644
index 0000000000..388f144eca
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.register.rst
@@ -0,0 +1,7 @@
+synapse.api.handlers.register module
+====================================
+
+.. automodule:: synapse.api.handlers.register
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.room.rst b/docs/sphinx/synapse.api.handlers.room.rst
new file mode 100644
index 0000000000..8ca156c7ff
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.room.rst
@@ -0,0 +1,7 @@
+synapse.api.handlers.room module
+================================
+
+.. automodule:: synapse.api.handlers.room
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.handlers.rst b/docs/sphinx/synapse.api.handlers.rst
new file mode 100644
index 0000000000..e84f563fcb
--- /dev/null
+++ b/docs/sphinx/synapse.api.handlers.rst
@@ -0,0 +1,21 @@
+synapse.api.handlers package
+============================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.api.handlers.events
+   synapse.api.handlers.factory
+   synapse.api.handlers.federation
+   synapse.api.handlers.register
+   synapse.api.handlers.room
+
+Module contents
+---------------
+
+.. automodule:: synapse.api.handlers
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.notifier.rst b/docs/sphinx/synapse.api.notifier.rst
new file mode 100644
index 0000000000..631b42a497
--- /dev/null
+++ b/docs/sphinx/synapse.api.notifier.rst
@@ -0,0 +1,7 @@
+synapse.api.notifier module
+===========================
+
+.. automodule:: synapse.api.notifier
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.register_events.rst b/docs/sphinx/synapse.api.register_events.rst
new file mode 100644
index 0000000000..79ad4ce211
--- /dev/null
+++ b/docs/sphinx/synapse.api.register_events.rst
@@ -0,0 +1,7 @@
+synapse.api.register_events module
+==================================
+
+.. automodule:: synapse.api.register_events
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.room_events.rst b/docs/sphinx/synapse.api.room_events.rst
new file mode 100644
index 0000000000..bead1711f5
--- /dev/null
+++ b/docs/sphinx/synapse.api.room_events.rst
@@ -0,0 +1,7 @@
+synapse.api.room_events module
+==============================
+
+.. automodule:: synapse.api.room_events
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.rst b/docs/sphinx/synapse.api.rst
new file mode 100644
index 0000000000..f4d39ff331
--- /dev/null
+++ b/docs/sphinx/synapse.api.rst
@@ -0,0 +1,30 @@
+synapse.api package
+===================
+
+Subpackages
+-----------
+
+.. toctree::
+
+    synapse.api.events
+    synapse.api.handlers
+    synapse.api.streams
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.api.auth
+   synapse.api.constants
+   synapse.api.errors
+   synapse.api.notifier
+   synapse.api.storage
+
+Module contents
+---------------
+
+.. automodule:: synapse.api
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.server.rst b/docs/sphinx/synapse.api.server.rst
new file mode 100644
index 0000000000..b01600235e
--- /dev/null
+++ b/docs/sphinx/synapse.api.server.rst
@@ -0,0 +1,7 @@
+synapse.api.server module
+=========================
+
+.. automodule:: synapse.api.server
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.storage.rst b/docs/sphinx/synapse.api.storage.rst
new file mode 100644
index 0000000000..afa40685c4
--- /dev/null
+++ b/docs/sphinx/synapse.api.storage.rst
@@ -0,0 +1,7 @@
+synapse.api.storage module
+==========================
+
+.. automodule:: synapse.api.storage
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.stream.rst b/docs/sphinx/synapse.api.stream.rst
new file mode 100644
index 0000000000..0d5e3f01bf
--- /dev/null
+++ b/docs/sphinx/synapse.api.stream.rst
@@ -0,0 +1,7 @@
+synapse.api.stream module
+=========================
+
+.. automodule:: synapse.api.stream
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.streams.event.rst b/docs/sphinx/synapse.api.streams.event.rst
new file mode 100644
index 0000000000..2ac45a35c8
--- /dev/null
+++ b/docs/sphinx/synapse.api.streams.event.rst
@@ -0,0 +1,7 @@
+synapse.api.streams.event module
+================================
+
+.. automodule:: synapse.api.streams.event
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.api.streams.rst b/docs/sphinx/synapse.api.streams.rst
new file mode 100644
index 0000000000..72eb205caf
--- /dev/null
+++ b/docs/sphinx/synapse.api.streams.rst
@@ -0,0 +1,17 @@
+synapse.api.streams package
+===========================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.api.streams.event
+
+Module contents
+---------------
+
+.. automodule:: synapse.api.streams
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.app.homeserver.rst b/docs/sphinx/synapse.app.homeserver.rst
new file mode 100644
index 0000000000..54b93da8fe
--- /dev/null
+++ b/docs/sphinx/synapse.app.homeserver.rst
@@ -0,0 +1,7 @@
+synapse.app.homeserver module
+=============================
+
+.. automodule:: synapse.app.homeserver
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.app.rst b/docs/sphinx/synapse.app.rst
new file mode 100644
index 0000000000..4535b79827
--- /dev/null
+++ b/docs/sphinx/synapse.app.rst
@@ -0,0 +1,17 @@
+synapse.app package
+===================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.app.homeserver
+
+Module contents
+---------------
+
+.. automodule:: synapse.app
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.db.rst b/docs/sphinx/synapse.db.rst
new file mode 100644
index 0000000000..83df6c03db
--- /dev/null
+++ b/docs/sphinx/synapse.db.rst
@@ -0,0 +1,10 @@
+synapse.db package
+==================
+
+Module contents
+---------------
+
+.. automodule:: synapse.db
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.handler.rst b/docs/sphinx/synapse.federation.handler.rst
new file mode 100644
index 0000000000..5597f5c46d
--- /dev/null
+++ b/docs/sphinx/synapse.federation.handler.rst
@@ -0,0 +1,7 @@
+synapse.federation.handler module
+=================================
+
+.. automodule:: synapse.federation.handler
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.messaging.rst b/docs/sphinx/synapse.federation.messaging.rst
new file mode 100644
index 0000000000..4bbaabf3ef
--- /dev/null
+++ b/docs/sphinx/synapse.federation.messaging.rst
@@ -0,0 +1,7 @@
+synapse.federation.messaging module
+===================================
+
+.. automodule:: synapse.federation.messaging
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.pdu_codec.rst b/docs/sphinx/synapse.federation.pdu_codec.rst
new file mode 100644
index 0000000000..8f0b15a63c
--- /dev/null
+++ b/docs/sphinx/synapse.federation.pdu_codec.rst
@@ -0,0 +1,7 @@
+synapse.federation.pdu_codec module
+===================================
+
+.. automodule:: synapse.federation.pdu_codec
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.persistence.rst b/docs/sphinx/synapse.federation.persistence.rst
new file mode 100644
index 0000000000..db7ab8ade1
--- /dev/null
+++ b/docs/sphinx/synapse.federation.persistence.rst
@@ -0,0 +1,7 @@
+synapse.federation.persistence module
+=====================================
+
+.. automodule:: synapse.federation.persistence
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.replication.rst b/docs/sphinx/synapse.federation.replication.rst
new file mode 100644
index 0000000000..49e26e0928
--- /dev/null
+++ b/docs/sphinx/synapse.federation.replication.rst
@@ -0,0 +1,7 @@
+synapse.federation.replication module
+=====================================
+
+.. automodule:: synapse.federation.replication
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.rst b/docs/sphinx/synapse.federation.rst
new file mode 100644
index 0000000000..7240c7901b
--- /dev/null
+++ b/docs/sphinx/synapse.federation.rst
@@ -0,0 +1,22 @@
+synapse.federation package
+==========================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.federation.handler
+   synapse.federation.pdu_codec
+   synapse.federation.persistence
+   synapse.federation.replication
+   synapse.federation.transport
+   synapse.federation.units
+
+Module contents
+---------------
+
+.. automodule:: synapse.federation
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.transport.rst b/docs/sphinx/synapse.federation.transport.rst
new file mode 100644
index 0000000000..877956b3c9
--- /dev/null
+++ b/docs/sphinx/synapse.federation.transport.rst
@@ -0,0 +1,7 @@
+synapse.federation.transport module
+===================================
+
+.. automodule:: synapse.federation.transport
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.federation.units.rst b/docs/sphinx/synapse.federation.units.rst
new file mode 100644
index 0000000000..8f9212b07d
--- /dev/null
+++ b/docs/sphinx/synapse.federation.units.rst
@@ -0,0 +1,7 @@
+synapse.federation.units module
+===============================
+
+.. automodule:: synapse.federation.units
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.persistence.rst b/docs/sphinx/synapse.persistence.rst
new file mode 100644
index 0000000000..37c0c23720
--- /dev/null
+++ b/docs/sphinx/synapse.persistence.rst
@@ -0,0 +1,19 @@
+synapse.persistence package
+===========================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.persistence.service
+   synapse.persistence.tables
+   synapse.persistence.transactions
+
+Module contents
+---------------
+
+.. automodule:: synapse.persistence
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.persistence.service.rst b/docs/sphinx/synapse.persistence.service.rst
new file mode 100644
index 0000000000..3514d3c76f
--- /dev/null
+++ b/docs/sphinx/synapse.persistence.service.rst
@@ -0,0 +1,7 @@
+synapse.persistence.service module
+==================================
+
+.. automodule:: synapse.persistence.service
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.persistence.tables.rst b/docs/sphinx/synapse.persistence.tables.rst
new file mode 100644
index 0000000000..907b02769d
--- /dev/null
+++ b/docs/sphinx/synapse.persistence.tables.rst
@@ -0,0 +1,7 @@
+synapse.persistence.tables module
+=================================
+
+.. automodule:: synapse.persistence.tables
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.persistence.transactions.rst b/docs/sphinx/synapse.persistence.transactions.rst
new file mode 100644
index 0000000000..475c02a8c5
--- /dev/null
+++ b/docs/sphinx/synapse.persistence.transactions.rst
@@ -0,0 +1,7 @@
+synapse.persistence.transactions module
+=======================================
+
+.. automodule:: synapse.persistence.transactions
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rest.base.rst b/docs/sphinx/synapse.rest.base.rst
new file mode 100644
index 0000000000..84d2d9b31d
--- /dev/null
+++ b/docs/sphinx/synapse.rest.base.rst
@@ -0,0 +1,7 @@
+synapse.rest.base module
+========================
+
+.. automodule:: synapse.rest.base
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rest.events.rst b/docs/sphinx/synapse.rest.events.rst
new file mode 100644
index 0000000000..ebbe26c746
--- /dev/null
+++ b/docs/sphinx/synapse.rest.events.rst
@@ -0,0 +1,7 @@
+synapse.rest.events module
+==========================
+
+.. automodule:: synapse.rest.events
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rest.register.rst b/docs/sphinx/synapse.rest.register.rst
new file mode 100644
index 0000000000..a4a48a8a8f
--- /dev/null
+++ b/docs/sphinx/synapse.rest.register.rst
@@ -0,0 +1,7 @@
+synapse.rest.register module
+============================
+
+.. automodule:: synapse.rest.register
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rest.room.rst b/docs/sphinx/synapse.rest.room.rst
new file mode 100644
index 0000000000..63fc5c2840
--- /dev/null
+++ b/docs/sphinx/synapse.rest.room.rst
@@ -0,0 +1,7 @@
+synapse.rest.room module
+========================
+
+.. automodule:: synapse.rest.room
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rest.rst b/docs/sphinx/synapse.rest.rst
new file mode 100644
index 0000000000..016af926b2
--- /dev/null
+++ b/docs/sphinx/synapse.rest.rst
@@ -0,0 +1,20 @@
+synapse.rest package
+====================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.rest.base
+   synapse.rest.events
+   synapse.rest.register
+   synapse.rest.room
+
+Module contents
+---------------
+
+.. automodule:: synapse.rest
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.rst b/docs/sphinx/synapse.rst
new file mode 100644
index 0000000000..e7869e0e5d
--- /dev/null
+++ b/docs/sphinx/synapse.rst
@@ -0,0 +1,30 @@
+synapse package
+===============
+
+Subpackages
+-----------
+
+.. toctree::
+
+    synapse.api
+    synapse.app
+    synapse.federation
+    synapse.persistence
+    synapse.rest
+    synapse.util
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.server
+   synapse.state
+
+Module contents
+---------------
+
+.. automodule:: synapse
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.server.rst b/docs/sphinx/synapse.server.rst
new file mode 100644
index 0000000000..7f33f084d7
--- /dev/null
+++ b/docs/sphinx/synapse.server.rst
@@ -0,0 +1,7 @@
+synapse.server module
+=====================
+
+.. automodule:: synapse.server
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.state.rst b/docs/sphinx/synapse.state.rst
new file mode 100644
index 0000000000..744be2a8be
--- /dev/null
+++ b/docs/sphinx/synapse.state.rst
@@ -0,0 +1,7 @@
+synapse.state module
+====================
+
+.. automodule:: synapse.state
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.async.rst b/docs/sphinx/synapse.util.async.rst
new file mode 100644
index 0000000000..542bb54444
--- /dev/null
+++ b/docs/sphinx/synapse.util.async.rst
@@ -0,0 +1,7 @@
+synapse.util.async module
+=========================
+
+.. automodule:: synapse.util.async
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.dbutils.rst b/docs/sphinx/synapse.util.dbutils.rst
new file mode 100644
index 0000000000..afaa9eb749
--- /dev/null
+++ b/docs/sphinx/synapse.util.dbutils.rst
@@ -0,0 +1,7 @@
+synapse.util.dbutils module
+===========================
+
+.. automodule:: synapse.util.dbutils
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.http.rst b/docs/sphinx/synapse.util.http.rst
new file mode 100644
index 0000000000..344af5a490
--- /dev/null
+++ b/docs/sphinx/synapse.util.http.rst
@@ -0,0 +1,7 @@
+synapse.util.http module
+========================
+
+.. automodule:: synapse.util.http
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.lockutils.rst b/docs/sphinx/synapse.util.lockutils.rst
new file mode 100644
index 0000000000..16ee26cabd
--- /dev/null
+++ b/docs/sphinx/synapse.util.lockutils.rst
@@ -0,0 +1,7 @@
+synapse.util.lockutils module
+=============================
+
+.. automodule:: synapse.util.lockutils
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.logutils.rst b/docs/sphinx/synapse.util.logutils.rst
new file mode 100644
index 0000000000..2b79fa7a4b
--- /dev/null
+++ b/docs/sphinx/synapse.util.logutils.rst
@@ -0,0 +1,7 @@
+synapse.util.logutils module
+============================
+
+.. automodule:: synapse.util.logutils
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.rst b/docs/sphinx/synapse.util.rst
new file mode 100644
index 0000000000..01a0c3a591
--- /dev/null
+++ b/docs/sphinx/synapse.util.rst
@@ -0,0 +1,21 @@
+synapse.util package
+====================
+
+Submodules
+----------
+
+.. toctree::
+
+   synapse.util.async
+   synapse.util.http
+   synapse.util.lockutils
+   synapse.util.logutils
+   synapse.util.stringutils
+
+Module contents
+---------------
+
+.. automodule:: synapse.util
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/sphinx/synapse.util.stringutils.rst b/docs/sphinx/synapse.util.stringutils.rst
new file mode 100644
index 0000000000..ec626eee28
--- /dev/null
+++ b/docs/sphinx/synapse.util.stringutils.rst
@@ -0,0 +1,7 @@
+synapse.util.stringutils module
+===============================
+
+.. automodule:: synapse.util.stringutils
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/terminology b/docs/terminology
new file mode 100644
index 0000000000..575cc0c808
--- /dev/null
+++ b/docs/terminology
@@ -0,0 +1,86 @@
+===========
+Terminology
+===========
+
+A list of definitions of specific terminology used among these documents.
+These terms were originally taken from the server-server documentation, and may
+not currently match the exact meanings used in other places; though as a
+medium-term goal we should encourage the unification of this terminology.
+
+
+Terms
+=====
+
+Context:
+  A single human-level entity of interest (currently, a chat room)
+
+EDU (Ephemeral Data Unit):
+  A message that relates directly to a given pair of home servers that are
+  exchanging it. EDUs are short-lived messages that related only to one single
+  pair of servers; they are not persisted for a long time and are not forwarded
+  on to other servers. Because of this, they have no internal ID nor previous
+  EDUs reference chain.
+
+Event:
+  A record of activity that records a single thing that happened on to a context
+  (currently, a chat room). These are the "chat messages" that Synapse makes
+  available.
+  [[NOTE(paul): The current server-server implementation calls these simply
+  "messages" but the term is too ambiguous here; I've called them Events]]
+
+Pagination:
+  The process of synchronising historic state from one home server to another,
+  to backfill the event storage so that scrollback can be presented to the
+  client(s).
+
+PDU (Persistent Data Unit):
+  A message that relates to a single context, irrespective of the server that
+  is communicating it. PDUs either encode a single Event, or a single State
+  change. A PDU is referred to by its PDU ID; the pair of its origin server
+  and local reference from that server.
+
+PDU ID:
+  The pair of PDU Origin and PDU Reference, that together globally uniquely
+  refers to a specific PDU.
+
+PDU Origin:
+  The name of the origin server that generated a given PDU. This may not be the
+  server from which it has been received, due to the way they are copied around
+  from server to server. The origin always records the original server that
+  created it.
+
+PDU Reference:
+  A local ID used to refer to a specific PDU from a given origin server. These
+  references are opaque at the protocol level, but may optionally have some
+  structured meaning within a given origin server or implementation.
+
+Presence:
+  The concept of whether a user is currently online, how available they declare
+  they are, and so on. See also: doc/model/presence
+
+Profile:
+  A set of metadata about a user, such as a display name, provided for the
+  benefit of other users. See also: doc/model/profiles
+
+Room ID:
+  An opaque string (of as-yet undecided format) that identifies a particular
+  room and used in PDUs referring to it.
+
+Room Alias:
+  A human-readable string of the form #name:some.domain that users can use as a
+  pointer to identify a room; a Directory Server will map this to its Room ID
+
+State:
+  A set of metadata maintained about a Context, which is replicated among the
+  servers in addition to the history of Events.
+
+User ID:
+  A string of the form @localpart:domain.name that identifies a user for
+  wire-protocol purposes. The localpart is meaningless outside of a particular
+  home server. This takes a human-readable form that end-users can use directly
+  if they so wish, avoiding the 3PIDs.
+
+Transaction:
+  A message which relates to the communication between a given pair of servers.
+  A transaction contains possibly-empty lists of PDUs and EDUs.
+
diff --git a/docs/versioning b/docs/versioning
new file mode 100644
index 0000000000..2f94bb6ef6
--- /dev/null
+++ b/docs/versioning
@@ -0,0 +1,11 @@
+Versioning is, like, hard for paginating backwards because of the number of Home Servers involved.
+
+The way we solve this is by doing versioning as an acyclic directed graph of PDUs. For pagination purposes, this is done on a per context basis. 
+When we send a PDU we include all PDUs that have been received for that context that hasn't been subsequently listed in a later PDU. The trivial case is a simple list of PDUs, e.g. A <- B <- C. However, if two servers send out a PDU at the same to, both B and C would point at A - a later PDU would then list both B and C.
+
+Problems with opaque version strings:
+    - How do you do clustering without mandating that a cluster can only have one transaction in flight to a given remote home server at a time. 
+      If you have multiple transactions sent at once, then you might drop one transaction, receive anotherwith a version that is later than the dropped transaction and which point ARGH WE LOST A TRANSACTION.
+    - How do you do pagination? A version string defines a point in a stream w.r.t. a single home server, not a point in the context.
+
+We only need to store the ends of the directed graph, we DO NOT need to do the whole one table of nodes and one of edges.