Merge branch 'develop' into room-initial-sync

Conflicts:
	synapse/handlers/message.py

7 files changed, 0 insertions, 1026 deletions

diff --git a/docs/client-server/model/presence.rst b/docs/client-server/model/presence.rst
deleted file mode 100644
index 811bac3fab..0000000000
--- a/docs/client-server/model/presence.rst
+++ /dev/null
@@ -1,149 +0,0 @@
-API Efficiency
-==============
-
-A simple implementation of presence messaging has the ability to cause a large
-amount of Internet traffic relating to presence updates. In order to minimise
-the impact of such a feature, the following observations can be made:
-
- * There is no point in a Home Server polling status for peers in a user's
-   presence list if the user has no clients connected that care about it.
-
- * It is highly likely that most presence subscriptions will be symmetric - a
-   given user watching another is likely to in turn be watched by that user.
-
- * It is likely that most subscription pairings will be between users who share
-   at least one Room in common, and so their Home Servers are actively
-   exchanging message PDUs or transactions relating to that Room.
-
- * Presence update messages do not need realtime guarantees. It is acceptable to
-   delay delivery of updates for some small amount of time (10 seconds to a
-   minute).
-
-The general model of presence information is that of a HS registering its
-interest in receiving presence status updates from other HSes, which then
-promise to send them when required. Rather than actively polling for the
-currentt state all the time, HSes can rely on their relative stability to only
-push updates when required.
-
-A Home Server should not rely on the longterm validity of this presence
-information, however, as this would not cover such cases as a user's server
-crashing and thus failing to inform their peers that users it used to host are
-no longer available online. Therefore, each promise of future updates should
-carry with a timeout value (whether explicit in the message, or implicit as some
-defined default in the protocol), after which the receiving HS should consider
-the information potentially stale and request it again.
-
-However, because of the likelyhood that two home servers are exchanging messages
-relating to chat traffic in a room common to both of them, the ongoing receipt
-of these messages can be taken by each server as an implicit notification that
-the sending server is still up and running, and therefore that no status changes
-have happened; because if they had the server would have sent them. A second,
-larger timeout should be applied to this implicit inference however, to protect
-against implementation bugs or other reasons that the presence state cache may
-become invalid; eventually the HS should re-enquire the current state of users
-and update them with its own.
-
-The following workflows can therefore be used to handle presence updates:
-
- 1 When a user first appears online their HS sends a message to each other HS
-   containing at least one user to be watched; each message carrying both a
-   notification of the sender's new online status, and a request to obtain and
-   watch the target users' presence information. This message implicitly
-   promises the sending HS will now push updates to the target HSes.
-
- 2 The target HSes then respond a single message each, containing the current
-   status of the requested user(s). These messages too implicitly promise the
-   target HSes will themselves push updates to the sending HS.
-
-   As these messages arrive at the sending user's HS they can be pushed to the
-   user's client(s), possibly batched again to ensure not too many small
-   messages which add extra protocol overheads.
-
-At this point, all the user's clients now have the current presence status
-information for this moment in time, and have promised to send each other
-updates in future.
-
- 3 The HS maintains two watchdog timers per peer HS it is exchanging presence
-   information with. The first timer should have a relatively small expiry
-   (perhaps 1 minute), and the second timer should have a much longer time
-   (perhaps 1 hour).
-
- 4 Any time any kind of message is received from a peer HS, the short-term
-   presence timer associated with it is reset.
-
- 5 Whenever either of these timers expires, an HS should push a status reminder
-   to the target HS whose timer has now expired, and request again from that
-   server the status of the subscribed users.
-
- 6 On receipt of one of these presence status reminders, an HS can reset both
-   of its presence watchdog timers.
-
-To avoid bursts of traffic, implementations should attempt to stagger the expiry
-of the longer-term watchdog timers for different peer HSes.
-
-When individual users actively change their status (either by explicit requests
-from clients, or inferred changes due to idle timers or client timeouts), the HS
-should batch up any status changes for some reasonable amount of time (10
-seconds to a minute). This allows for reduced protocol overheads in the case of
-multiple messages needing to be sent to the same peer HS; as is the likely
-scenario in many cases, such as a given human user having multiple user
-accounts.
-
-
-API Requirements
-================
-
-The data model presented here puts the following requirements on the APIs:
-
-Client-Server
--------------
-
-Requests that a client can make to its Home Server
-
- * get/set current presence state
-   Basic enumeration + ability to set a custom piece of text
-
- * report per-device idle time
-   After some (configurable?) idle time the device should send a single message
-   to set the idle duration. The HS can then infer a "start of idle" instant and
-   use that to keep the device idleness up to date. At some later point the
-   device can cancel this idleness.
-
- * report per-device type
-   Inform the server that this device is a "mobile" device, or perhaps some
-   other to-be-defined category of reduced capability that could be presented to
-   other users.
-
- * start/stop presence polling for my presence list
-   It is likely that these messages could be implicitly inferred by other
-   messages, though having explicit control is always useful.
-
- * get my presence list
-   [implicit poll start?]
-   It is possible that the HS doesn't yet have current presence information when
-   the client requests this. There should be a "don't know" type too.
-
- * add/remove a user to my presence list
-
-Server-Server
--------------
-
-Requests that Home Servers make to others
-
- * request permission to add a user to presence list
-
- * allow/deny a request to add to a presence list
-
- * perform a combined presence state push and subscription request
-   For each sending user ID, the message contains their new status.
-   For each receiving user ID, the message should contain an indication on
-   whether the sending server is also interested in receiving status from that
-   user; either as an immediate update response now, or as a promise to send
-   future updates.
-
-Server to Client
-----------------
-
-[[TODO(paul): There also needs to be some way for a user's HS to push status
-updates of the presence list to clients, but the general server-client event
-model currently lacks a space to do that.]]
diff --git a/docs/client-server/model/profiles.rst b/docs/client-server/model/profiles.rst
deleted file mode 100644
index f7d6bd5679..0000000000
--- a/docs/client-server/model/profiles.rst
+++ /dev/null
@@ -1,232 +0,0 @@
-========
-Profiles
-========
-
-A description of Synapse user profile metadata support.
-
-
-Overview
-========
-
-Internally within Synapse users are referred to by an opaque ID, which consists
-of some opaque localpart combined with the domain name of their home server.
-Obviously this does not yield a very nice user experience; users would like to
-see readable names for other users that are in some way meaningful to them.
-Additionally, users like to be able to publish "profile" details to inform other
-users of other information about them.
-
-It is also conceivable that since we are attempting to provide a
-worldwide-applicable messaging system, that users may wish to present different
-subsets of information in their profile to different other people, from a
-privacy and permissions perspective.
-
-A Profile consists of a display name, an (optional?) avatar picture, and a set
-of other metadata fields that the user may wish to publish (email address, phone
-numbers, website URLs, etc...). We put no requirements on the display name other
-than it being a valid Unicode string. Since it is likely that users will end up
-having multiple accounts (perhaps by necessity of being hosted in multiple
-places, perhaps by choice of wanting multiple distinct identifies), it would be
-useful that a metadata field type exists that can refer to another Synapse User
-ID, so that clients and HSes can make use of this information.
-
-Metadata Fields
----------------
-
-[[TODO(paul): Likely this list is incomplete; more fields can be defined as we
-think of them. At the very least, any sort of supported ID for the 3rd Party ID
-servers should be accounted for here.]]
-
- * Synapse Directory Server username(s)
-
- * Email address
-
- * Phone number - classify "home"/"work"/"mobile"/custom?
- 
- * Twitter/Facebook/Google+/... social networks
-
- * Location - keep this deliberately vague to allow people to choose how
-     granular it is
- 
- * "Bio" information - date of birth, etc...
-
- * Synapse User ID of another account
-
- * Web URL
-
- * Freeform description text
-
-
-Visibility Permissions
-======================
-
-A home server implementation could offer the ability to set permissions on
-limited visibility of those fields. When another user requests access to the
-target user's profile, their own identity should form part of that request. The
-HS implementation can then decide which fields to make available to the
-requestor.
-
-A particular detail of implementation could allow the user to create one or more
-ACLs; where each list is granted permission to see a given set of non-public
-fields (compare to Google+ Circles) and contains a set of other people allowed
-to use it. By giving these ACLs strong identities within the HS, they can be
-referenced in communications with it, granting other users who encounter these
-the "ACL Token" to use the details in that ACL.
-
-If we further allow an ACL Token to be present on Room join requests or stored
-by 3PID servers, then users of these ACLs gain the extra convenience of not
-having to manually curate people in the access list; anyone in the room or with
-knowledge of the 3rd Party ID is automatically granted access. Every HS and
-client implementation would have to be aware of the existence of these ACL
-Token, and include them in requests if present, but not every HS implementation
-needs to actually provide the full permissions model. This can be used as a
-distinguishing feature among competing implementations. However, servers MUST
-NOT serve profile information from a cache if there is a chance that its limited
-understanding could lead to information leakage.
-
-
-Client Concerns of Multiple Accounts
-====================================
-
-Because a given person may want to have multiple Synapse User accounts, client
-implementations should allow the use of multiple accounts simultaneously
-(especially in the field of mobile phone clients, which generally don't support
-running distinct instances of the same application). Where features like address
-books, presence lists or rooms are presented, the client UI should remember to
-make distinct with user account is in use for each.
-
-
-Directory Servers
-=================
-
-Directory Servers can provide a forward mapping from human-readable names to
-User IDs. These can provide a service similar to giving domain-namespaced names
-for Rooms; in this case they can provide a way for a user to reference their
-User ID in some external form (e.g. that can be printed on a business card).
-
-The format for Synapse user name will consist of a localpart specific to the
-directory server, and the domain name of that directory server:
-
-  @localname:some.domain.name
-
-The localname is separated from the domain name using a colon, so as to ensure
-the localname can still contain periods, as users may want this for similarity
-to email addresses or the like, which typically can contain them. The format is
-also visually quite distinct from email addresses, phone numbers, etc... so
-hopefully reasonably "self-describing" when written on e.g. a business card
-without surrounding context.
-
-[[TODO(paul): we might have to think about this one - too close to email?
-  Twitter? Also it suggests a format scheme for room names of
-  #localname:domain.name, which I quite like]]
-
-Directory server administrators should be able to make some kind of policy
-decision on how these are allocated. Servers within some "closed" domain (such
-as company-specific ones) may wish to verify the validity of a mapping using
-their own internal mechanisms; "public" naming servers can operate on a FCFS
-basis. There are overlapping concerns here with the idea of the 3rd party
-identity servers as well, though in this specific case we are creating a new
-namespace to allocate names into.
-
-It would also be nice from a user experience perspective if the profile that a
-given name links to can also declare that name as part of its metadata.
-Furthermore as a security and consistency perspective it would be nice if each
-end (the directory server and the user's home server) check the validity of the
-mapping in some way. This needs investigation from a security perspective to
-ensure against spoofing.
-
-One such model may be that the user starts by declaring their intent to use a
-given user name link to their home server, which then contacts the directory
-service. At some point later (maybe immediately for "public open FCFS servers",
-maybe after some kind of human intervention for verification) the DS decides to
-honour this link, and includes it in its served output. It should also tell the
-HS of this fact, so that the HS can present this as fact when requested for the
-profile information. For efficiency, it may further wish to provide the HS with
-a cryptographically-signed certificate as proof, so the HS serving the profile
-can provide that too when asked, avoiding requesting HSes from constantly having
-to contact the DS to verify this mapping. (Note: This is similar to the security
-model often applied in DNS to verify PTR <-> A bidirectional mappings).
-
-
-Identity Servers
-================
-
-The identity servers should support the concept of pointing a 3PID being able to
-store an ACL Token as well as the main User ID. It is however, beyond scope to
-do any kind of verification that any third-party IDs that the profile is
-claiming match up to the 3PID mappings.
-
-
-User Interface and Expectations Concerns
-========================================
-
-Given the weak "security" of some parts of this model as compared to what users
-might expect, some care should be taken on how it is presented to users,
-specifically in the naming or other wording of user interface components.
-
-Most notably mere knowledge of an ACL Pointer is enough to read the information
-stored in it. It is possible that Home or Identity Servers could leak this
-information, allowing others to see it. This is a security-vs-convenience
-balancing choice on behalf of the user who would choose, or not, to make use of
-such a feature to publish their information.
-
-Additionally, unless some form of strong end-to-end user-based encryption is
-used, a user of ACLs for information privacy has to trust other home servers not
-to lie about the identify of the user requesting access to the Profile.
-
-
-API Requirements
-================
-
-The data model presented here puts the following requirements on the APIs:
-
-Client-Server
--------------
-
-Requests that a client can make to its Home Server
-
- * get/set my Display Name
-   This should return/take a simple "text/plain" field
-
- * get/set my Avatar URL
-   The avatar image data itself is not stored by this API; we'll just store a
-   URL to let the clients fetch it. Optionally HSes could integrate this with
-   their generic content attacmhent storage service, allowing a user to set
-   upload their profile Avatar and update the URL to point to it.
-
- * get/add/remove my metadata fields
-   Also we need to actually define types of metadata
-
- * get another user's Display Name / Avatar / metadata fields
-
-[[TODO(paul): At some later stage we should consider the API for:
-
- * get/set ACL permissions on my metadata fields
-
- * manage my ACL tokens
-]]
-
-Server-Server
--------------
-
-Requests that Home Servers make to others
-
- * get a user's Display Name / Avatar
-
- * get a user's full profile - name/avatar + MD fields
-   This request must allow for specifying the User ID of the requesting user,
-   for permissions purposes. It also needs to take into account any ACL Tokens
-   the requestor has.
-
- * push a change of Display Name to observers (overlaps with the presence API)
-
-Room Event PDU Types
---------------------
-
-Events that are pushed from Home Servers to other Home Servers or clients.
-
- * user Display Name change
- 
- * user Avatar change
-   [[TODO(paul): should the avatar image itself be stored in all the room
-   histories? maybe this event should just be a hint to clients that they should
-   re-fetch the avatar image]]
diff --git a/docs/client-server/model/protocol_examples.rst b/docs/client-server/model/protocol_examples.rst
deleted file mode 100644
index 61a599b432..0000000000
--- a/docs/client-server/model/protocol_examples.rst
+++ /dev/null
@@ -1,64 +0,0 @@
-PUT /send/abc/ HTTP/1.1
-Host: ...
-Content-Length: ...
-Content-Type: application/json
-
-{
-    "origin": "localhost:5000",
-    "pdus": [
-        {
-            "content": {},
-            "context": "tng",
-            "depth": 12,
-            "is_state": false,
-            "origin": "localhost:5000",
-            "pdu_id": 1404381396854,
-            "pdu_type": "feedback",
-            "prev_pdus": [
-                [
-                    "1404381395883",
-                    "localhost:6000"
-                ]
-            ],
-            "ts": 1404381427581
-        }
-    ],
-    "prev_ids": [
-        "1404381396852"
-    ],
-    "ts": 1404381427823
-}
-
-HTTP/1.1 200 OK
-...
-
-======================================
-
-GET /pull/-1/ HTTP/1.1
-Host: ...
-Content-Length: 0
-
-HTTP/1.1 200 OK
-Content-Length: ...
-Content-Type: application/json
-
-{
-    origin: ...,
-    prev_ids: ...,
-    data: [
-        {
-            data_id: ...,
-            prev_pdus: [...],
-            depth: ...,
-            ts: ...,
-            context: ...,
-            origin: ...,
-            content: {
-                ...
-            }
-        },
-        ...,
-    ]
-}
-
-
diff --git a/docs/client-server/model/room-join-workflow.rst b/docs/client-server/model/room-join-workflow.rst
deleted file mode 100644
index c321a64fab..0000000000
--- a/docs/client-server/model/room-join-workflow.rst
+++ /dev/null
@@ -1,113 +0,0 @@
-==================
-Room Join Workflow
-==================
-
-An outline of the workflows required when a user joins a room.
-
-Discovery
-=========
-
-To join a room, a user has to discover the room by some mechanism in order to
-obtain the (opaque) Room ID and a candidate list of likely home servers that
-contain it.
-
-Sending an Invitation
----------------------
-
-The most direct way a user discovers the existence of a room is from a
-invitation from some other user who is a member of that room.
-
-The inviter's HS sets the membership status of the invitee to "invited" in the
-"m.members" state key by sending a state update PDU. The HS then broadcasts this
-PDU among the existing members in the usual way. An invitation message is also
-sent to the invited user, containing the Room ID and the PDU ID of this
-invitation state change and potentially a list of some other home servers to use
-to accept the invite. The user's client can then choose to display it in some
-way to alert the user.
-
-[[TODO(paul): At present, no API has been designed or described to actually send
-that invite to the invited user. Likely it will be some facet of the larger
-user-user API required for presence, profile management, etc...]]
-
-Directory Service
------------------
-
-Alternatively, the user may discover the channel via a directory service; either
-by performing a name lookup, or some kind of browse or search acitivty. However
-this is performed, the end result is that the user's home server requests the
-Room ID and candidate list from the directory service.
-
-[[TODO(paul): At present, no API has been designed or described for this
-directory service]]
-
-
-Joining
-=======
-
-Once the ID and home servers are obtained, the user can then actually join the
-room.
-
-Accepting an Invite
--------------------
-
-If a user has received and accepted an invitation to join a room, the invitee's
-home server can now send an invite acceptance message to a chosen candidate
-server from the list given in the invitation, citing also the PDU ID of the
-invitation as "proof" of their invite. (This is required as due to late message
-propagation it could be the case that the acceptance is received before the
-invite by some servers). If this message is allowed by the candidate server, it
-generates a new PDU that updates the invitee's membership status to "joined",
-referring back to the acceptance PDU, and broadcasts that as a state change in
-the usual way. The newly-invited user is now a full member of the room, and
-state propagation proceeds as usual.
-
-Joining a Public Room
----------------------
-
-If a user has discovered the existence of a room they wish to join but does not
-have an active invitation, they can request to join it directly by sending a
-join message to a candidate server on the list provided by the directory
-service. As this list may be out of date, the HS should be prepared to retry
-other candidates if the chosen one is no longer aware of the room, because it
-has no users as members in it.
-
-Once a candidate server that is aware of the room has been found, it can
-broadcast an update PDU to add the member into the "m.members" key setting their
-state directly to "joined" (i.e. bypassing the two-phase invite semantics),
-remembering to include the new user's HS in that list.
-
-Knocking on a Semi-Public Room
-------------------------------
-
-If a user requests to join a room but the join mode of the room is "knock", the
-join is not immediately allowed. Instead, if the user wishes to proceed, they
-can instead post a "knock" message, which informs other members of the room that
-the would-be joiner wishes to become a member and sets their membership value to
-"knocked". If any of them wish to accept this, they can then send an invitation
-in the usual way described above. Knowing that the user has already knocked and
-expressed an interest in joining, the invited user's home server should
-immediately accept that invitation on the user's behalf, and go on to join the
-room in the usual way.
-
-[[NOTE(Erik): Though this may confuse users who expect 'X has joined' to
-actually be a user initiated action, i.e. they may expect that 'X' is actually
-looking at synapse right now?]]
-
-[[NOTE(paul): Yes, a fair point maybe we should suggest HSes don't do that, and
-just offer an invite to the user as normal]]
-
-Private and Non-Existent Rooms
-------------------------------
-
-If a user requests to join a room but the room is either unknown by the home
-server receiving the request, or is known by the join mode is "invite" and the
-user has not been invited, the server must respond that the room does not exist.
-This is to prevent leaking information about the existence and identity of
-private rooms.
-
-
-Outstanding Questions
-=====================
-
- * Do invitations or knocks time out and expire at some point? If so when? Time
-   is hard in distributed systems.
diff --git a/docs/client-server/model/rooms.rst b/docs/client-server/model/rooms.rst
deleted file mode 100644
index 0007e48e30..0000000000
--- a/docs/client-server/model/rooms.rst
+++ /dev/null
@@ -1,274 +0,0 @@
-===========
-Rooms Model
-===========
-
-A description of the general data model used to implement Rooms, and the
-user-level visible effects and implications.
-
-
-Overview
-========
-
-"Rooms" in Synapse are shared messaging channels over which all the participant
-users can exchange messages. Rooms have an opaque persistent identify, a
-globally-replicated set of state (consisting principly of a membership set of
-users, and other management and miscellaneous metadata), and a message history.
-
-
-Room Identity and Naming
-========================
-
-Rooms can be arbitrarily created by any user on any home server; at which point
-the home server will sign the message that creates the channel, and the
-fingerprint of this signature becomes the strong persistent identify of the
-room. This now identifies the room to any home server in the network regardless
-of its original origin. This allows the identify of the room to outlive any
-particular server. Subject to appropriate permissions [to be discussed later],
-any current member of a room can invite others to join it, can post messages
-that become part of its history, and can change the persistent state of the room
-(including its current set of permissions).
-
-Home servers can provide a directory service, allowing a lookup from a
-convenient human-readable form of room label to a room ID. This mapping is
-scoped to the particular home server domain and so simply represents that server
-administrator's opinion of what room should take that label; it does not have to
-be globally replicated and does not form part of the stored state of that room.
-
-This room name takes the form
-
-  #localname:some.domain.name
-
-for similarity and consistency with user names on directories.
-
-To join a room (and therefore to be allowed to inspect past history, post new
-messages to it, and read its state), a user must become aware of the room's
-fingerprint ID. There are two mechanisms to allow this:
-
- * An invite message from someone else in the room
-
- * A referral from a room directory service
-
-As room IDs are opaque and ephemeral, they can serve as a mechanism to create
-"ad-hoc" rooms deliberately unnamed, for small group-chats or even private
-one-to-one message exchange.
-
-
-Stored State and Permissions
-============================
-
-Every room has a globally-replicated set of stored state. This state is a set of
-key/value or key/subkey/value pairs. The value of every (sub)key is a
-JSON-representable object. The main key of a piece of stored state establishes
-its meaning; some keys store sub-keys to allow a sub-structure within them [more
-detail below]. Some keys have special meaning to Synapse, as they relate to
-management details of the room itself, storing such details as user membership,
-and permissions of users to alter the state of the room itself. Other keys may
-store information to present to users, which the system does not directly rely
-on. The key space itself is namespaced, allowing 3rd party extensions, subject
-to suitable permission.
-
-Permission management is based on the concept of "power-levels". Every user
-within a room has an integer assigned, being their "power-level" within that
-room. Along with its actual data value, each key (or subkey) also stores the
-minimum power-level a user must have in order to write to that key, the
-power-level of the last user who actually did write to it, and the PDU ID of
-that state change.
-
-To be accepted as valid, a change must NOT:
-
- * Be made by a user having a power-level lower than required to write to the
-   state key
-
- * Alter the required power-level for that state key to a value higher than the
-   user has
-
- * Increase that user's own power-level
-
- * Grant any other user a power-level higher than the level of the user making
-   the change
-
-[[TODO(paul): consider if relaxations should be allowed; e.g. is the current
-outright-winner allowed to raise their own level, to allow for "inflation"?]]
-
-
-Room State Keys
-===============
-
-[[TODO(paul): if this list gets too big it might become necessary to move it
-into its own doc]]
-
-The following keys have special semantics or meaning to Synapse itself:
-
-m.member (has subkeys)
-  Stores a sub-key for every Synapse User ID which is currently a member of
-  this room. Its value gives the membership type ("knocked", "invited",
-  "joined").
-
-m.power_levels
-  Stores a mapping from Synapse User IDs to their power-level in the room. If
-  they are not present in this mapping, the default applies.
-
-  The reason to store this as a single value rather than a value with subkeys
-  is that updates to it are atomic; allowing a number of colliding-edit
-  problems to be avoided.
-
-m.default_level
-  Gives the default power-level for members of the room that do not have one
-  specified in their membership key.
-
-m.invite_level
-  If set, gives the minimum power-level required for members to invite others
-  to join, or to accept knock requests from non-members requesting access. If
-  absent, then invites are not allowed. An invitation involves setting their
-  membership type to "invited", in addition to sending the invite message.
-
-m.join_rules
-  Encodes the rules on how non-members can join the room. Has the following
-  possibilities:
-    "public" - a non-member can join the room directly
-    "knock" - a non-member cannot join the room, but can post a single "knock"
-        message requesting access, which existing members may approve or deny
-    "invite" - non-members cannot join the room without an invite from an
-        existing member
-    "private" - nobody who is not in the 'may_join' list or already a member
-        may join by any mechanism
-
-  In any of the first three modes, existing members with sufficient permission
-  can send invites to non-members if allowed by the "m.invite_level" key. A
-  "private" room is not allowed to have the "m.invite_level" set.
-
-  A client may use the value of this key to hint at the user interface
-  expectations to provide; in particular, a private chat with one other use
-  might warrant specific handling in the client.
-
-m.may_join
-  A list of User IDs that are always allowed to join the room, regardless of any
-  of the prevailing join rules and invite levels. These apply even to private
-  rooms. These are stored in a single list with normal update-powerlevel
-  permissions applied; users cannot arbitrarily remove themselves from the list.
-
-m.add_state_level
-  The power-level required for a user to be able to add new state keys.
-
-m.public_history
-  If set and true, anyone can request the history of the room, without needing
-  to be a member of the room.
-
-m.archive_servers
-  For "public" rooms with public history, gives a list of home servers that
-  should be included in message distribution to the room, even if no users on
-  that server are present. These ensure that a public room can still persist
-  even if no users are currently members of it. This list should be consulted by
-  the dirctory servers as the candidate list they respond with.
-
-The following keys are provided by Synapse for user benefit, but their value is
-not otherwise used by Synapse.
-
-m.name
-  Stores a short human-readable name for the room, such that clients can display
-  to a user to assist in identifying which room is which.
-  
-  This name specifically is not the strong ID used by the message transport
-  system to refer to the room, because it may be changed from time to time.
-
-m.topic
-  Stores the current human-readable topic
-
-
-Room Creation Templates
-=======================
-
-A client (or maybe home server?) could offer a few templates for the creation of
-new rooms. For example, for a simple private one-to-one chat the channel could
-assign the creator a power-level of 1, requiring a level of 1 to invite, and
-needing an invite before members can join. An invite is then sent to the other
-party, and if accepted and the other user joins, the creator's power-level can
-now be reduced to 0. This now leaves a room with two participants in it being
-unable to add more.
-
-
-Rooms that Continue History
-===========================
-
-An option that could be considered for room creation, is that when a new room is
-created the creator could specify a PDU ID into an existing room, as the history
-continuation point. This would be stored as an extra piece of meta-data on the
-initial PDU of the room's creation. (It does not appear in the normal previous
-PDU linkage).
-
-This would allow users in rooms to "fork" a room, if it is considered that the
-conversations in the room no longer fit its original purpose, and wish to
-diverge. Existing permissions on the original room would continue to apply of
-course, for viewing that history. If both rooms are considered "public" we might
-also want to define a message to post into the original room to represent this
-fork point, and give a reference to the new room.
-
-
-User Direct Message Rooms
-=========================
-
-There is no need to build a mechanism for directly sending messages between
-users, because a room can handle this ability. To allow direct user-to-user chat
-messaging we simply need to be able to create rooms with specific set of
-permissions to allow this direct messaging.
-
-Between any given pair of user IDs that wish to exchange private messages, there
-will exist a single shared Room, created lazily by either side. These rooms will
-need a certain amount of special handling in both home servers and display on
-clients, but as much as possible should be treated by the lower layers of code
-the same as other rooms.
-
-Specially, a client would likely offer a special menu choice associated with
-another user (in room member lists, presence list, etc..) as "direct chat". That
-would perform all the necessary steps to create the private chat room. Receiving
-clients should display these in a special way too as the room name is not
-important; instead it should distinguish them on the Display Name of the other
-party.
-
-Home Servers will need a client-API option to request setting up a new user-user
-chat room, which will then need special handling within the server. It will
-create a new room with the following 
-
-  m.member: the proposing user
-  m.join_rules: "private"
-  m.may_join: both users
-  m.power_levels: empty
-  m.default_level: 0
-  m.add_state_level: 0
-  m.public_history: False
-
-Having created the room, it can send an invite message to the other user in the
-normal way - the room permissions state that no users can be set to the invited
-state, but because they're in the may_join list then they'd be allowed to join
-anyway.
-
-In this arrangement there is now a room with both users may join but neither has
-the power to invite any others. Both users now have the confidence that (at
-least within the messaging system itself) their messages remain private and
-cannot later be provably leaked to a third party. They can freely set the topic
-or name if they choose and add or edit any other state of the room. The update
-powerlevel of each of these fixed properties should be 1, to lock out the users
-from being able to alter them.
-
-
-Anti-Glare
-==========
-
-There exists the possibility of a race condition if two users who have no chat
-history with each other simultaneously create a room and invite the other to it.
-This is called a "glare" situation. There are two possible ideas for how to
-resolve this:
-
- * Each Home Server should persist the mapping of (user ID pair) to room ID, so
-   that duplicate requests can be suppressed. On receipt of a room creation
-   request that the HS thinks there already exists a room for, the invitation to
-   join can be rejected if:
-      a) the HS believes the sending user is already a member of the room (and
-         maybe their HS has forgotten this fact), or
-      b) the proposed room has a lexicographically-higher ID than the existing
-         room (to resolve true race condition conflicts)
-      
- * The room ID for a private 1:1 chat has a special form, determined by
-   concatenting the User IDs of both members in a deterministic order, such that
-   it doesn't matter which side creates it first; the HSes can just ignore
-   (or merge?) received PDUs that create the room twice.
diff --git a/docs/client-server/model/terminology.rst b/docs/client-server/model/terminology.rst
deleted file mode 100644
index cc6e6760ac..0000000000
--- a/docs/client-server/model/terminology.rst
+++ /dev/null
@@ -1,86 +0,0 @@
-===========
-Terminology
-===========
-
-A list of definitions of specific terminology used among these documents.
-These terms were originally taken from the server-server documentation, and may
-not currently match the exact meanings used in other places; though as a
-medium-term goal we should encourage the unification of this terminology.
-
-
-Terms
-=====
-
-Backfilling:
-  The process of synchronising historic state from one home server to another,
-  to backfill the event storage so that scrollback can be presented to the
-  client(s).  (Formerly, and confusingly, called 'pagination')
-
-Context:
-  A single human-level entity of interest (currently, a chat room)
-
-EDU (Ephemeral Data Unit):
-  A message that relates directly to a given pair of home servers that are
-  exchanging it. EDUs are short-lived messages that related only to one single
-  pair of servers; they are not persisted for a long time and are not forwarded
-  on to other servers. Because of this, they have no internal ID nor previous
-  EDUs reference chain.
-
-Event:
-  A record of activity that records a single thing that happened on to a context
-  (currently, a chat room). These are the "chat messages" that Synapse makes
-  available.
-  [[NOTE(paul): The current server-server implementation calls these simply
-  "messages" but the term is too ambiguous here; I've called them Events]]
-
-PDU (Persistent Data Unit):
-  A message that relates to a single context, irrespective of the server that
-  is communicating it. PDUs either encode a single Event, or a single State
-  change. A PDU is referred to by its PDU ID; the pair of its origin server
-  and local reference from that server.
-
-PDU ID:
-  The pair of PDU Origin and PDU Reference, that together globally uniquely
-  refers to a specific PDU.
-
-PDU Origin:
-  The name of the origin server that generated a given PDU. This may not be the
-  server from which it has been received, due to the way they are copied around
-  from server to server. The origin always records the original server that
-  created it.
-
-PDU Reference:
-  A local ID used to refer to a specific PDU from a given origin server. These
-  references are opaque at the protocol level, but may optionally have some
-  structured meaning within a given origin server or implementation.
-
-Presence:
-  The concept of whether a user is currently online, how available they declare
-  they are, and so on. See also: doc/model/presence
-
-Profile:
-  A set of metadata about a user, such as a display name, provided for the
-  benefit of other users. See also: doc/model/profiles
-
-Room ID:
-  An opaque string (of as-yet undecided format) that identifies a particular
-  room and used in PDUs referring to it.
-
-Room Alias:
-  A human-readable string of the form #name:some.domain that users can use as a
-  pointer to identify a room; a Directory Server will map this to its Room ID
-
-State:
-  A set of metadata maintained about a Context, which is replicated among the
-  servers in addition to the history of Events.
-
-User ID:
-  A string of the form @localpart:domain.name that identifies a user for
-  wire-protocol purposes. The localpart is meaningless outside of a particular
-  home server. This takes a human-readable form that end-users can use directly
-  if they so wish, avoiding the 3PIDs.
-
-Transaction:
-  A message which relates to the communication between a given pair of servers.
-  A transaction contains possibly-empty lists of PDUs and EDUs.
-
diff --git a/docs/client-server/model/third-party-id.rst b/docs/client-server/model/third-party-id.rst
deleted file mode 100644
index 1f8138ddf7..0000000000
--- a/docs/client-server/model/third-party-id.rst
+++ /dev/null
@@ -1,108 +0,0 @@
-======================
-Third Party Identities
-======================
-
-A description of how email addresses, mobile phone numbers and other third
-party identifiers can be used to authenticate and discover users in Matrix.
-
-
-Overview
-========
-
-New users need to authenticate their account. An email or SMS text message can 
-be a convenient form of authentication. Users already have email addresses 
-and phone numbers for contacts in their address book. They want to communicate
-with those contacts in Matrix without manually exchanging a Matrix User ID with 
-them.
-
-Third Party IDs
----------------
-
-[[TODO(markjh): Describe the format of a 3PID]]
-
-
-Third Party ID Associations
----------------------------
-
-An Associaton is a binding between a Matrix User ID and a Third Party ID (3PID).
-Each 3PID can be associated with one Matrix User ID at a time.
-
-[[TODO(markjh): JSON format of the association.]]
-
-Verification 
-------------
-
-An Assocation must be verified by a trusted Verification Server. Email 
-addresses and phone numbers can be verified by sending a token to the address 
-which a client can supply to the verifier to confirm ownership.
-
-An email Verification Server may be capable of verifying all email 3PIDs or may
-be restricted to verifying addresses for a particular domain. A phone number
-Verification Server may be capable of verifying all phone numbers or may be
-restricted to verifying numbers for a given country or phone prefix.
-
-Verification Servers fulfil a similar role to Certificate Authorities in PKI so
-a similar level of vetting should be required before clients trust their
-signatures.
-
-A Verification Server may wish to check for existing Associations for a 3PID 
-before creating a new Association.
-
-Discovery
----------
-
-Users can discover Associations using a trusted Identity Server. Each 
-Association will be signed by the Identity Server. An Identity Server may store
-the entire space of Associations or may delegate to other Identity Servers when
-looking up Associations.
-
-Each Association returned from an Identity Server must be signed by a 
-Verification Server. Clients should check these signatures.
-
-Identity Servers fulfil a similar role to DNS servers.
-
-Privacy
--------
-
-A User may publish the association between their phone number and Matrix User ID
-on the Identity Server without publishing the number in their Profile hosted on
-their Home Server.
-
-Identity Servers should refrain from publishing reverse mappings and should 
-take steps, such as rate limiting, to prevent attackers enumerating the space of
-mappings.
-
-Federation
-==========
-
-Delegation
-----------
-
-Verification Servers could delegate signing to another server by issuing 
-certificate to that server allowing it to verify and sign a subset of 3PID on 
-its behalf. It would be necessary to provide a language for describing which
-subset of 3PIDs that server had authority to validate. Alternatively it could 
-delegate the verification step to another server but sign the resulting
-association itself.
-
-The 3PID space will have a heirachical structure like DNS so Identity Servers
-can delegate lookups to other servers. An Identity Server should be prepared 
-to host or delegate any valid association within the subset of the 3PIDs it is 
-resonsible for.
-
-Multiple Root Verification Servers
-----------------------------------
-
-There can be multiple root Verification Servers and an Association could be
-signed by multiple servers if different clients trust different subsets of
-the verification servers.
-
-Multiple Root Identity Servers
-------------------------------
-
-There can be be multiple root Identity Servers. Clients will add each
-Association to all root Identity Servers.
-
-[[TODO(markjh): Describe how clients find the list of root Identity Servers]]
-
-