Start fleshing out architecture section. Moar .rst formatting! Reword some copypastaed sections to be terser.

1 files changed, 146 insertions, 106 deletions

diff --git a/docs/specification.rst b/docs/specification.rst
index fa085bac27..35fbd4dd22 100644
--- a/docs/specification.rst
+++ b/docs/specification.rst
@@ -5,16 +5,18 @@ TODO(Introduction) : Matthew
  - Similar to intro paragraph from README.
  - Explaining the overall mission, what this spec describes...
  - "What is Matrix?"
+ - Draw parallels with email?
 
 Architecture
 ============
-
-- Basic structure: What are clients/home servers and what are their 
-  responsibilities? What are events.
+- Sending a message from A to B
 
 ::
 
-        { Matrix clients }                              { Matrix clients }
+                         How data flows between clients
+                         ==============================
+
+       { Matrix client A }                             { Matrix client B }
            ^          |                                    ^          |
            |  events  |                                    |  events  |
            |          V                                    |          V
@@ -22,22 +24,87 @@ Architecture
        |                  |---------( HTTP )---------->|                  |
        |   Home Server    |                            |   Home Server    |
        |                  |<--------( HTTP )-----------|                  |
-       +------------------+                            +------------------+
-       
-- How do identity servers fit in? 3PIDs? Users? Aliases
-- Pattern of the APIs (HTTP/JSON, REST + txns)
-- Standard error response format.
-- C-S Event stream
-
-Rooms
-=====
+       +------------------+        Federation          +------------------+
+
+- Client is an end-user (web app, mobile app) which uses C-S APIs to talk to the home server.
+  A given client is typically responsible for a single user.
+- Home server provides C-S APIs and has the ability to federate with other HSes.
+  Typically responsible for N clients.
+- Federation's purpose is to share content between interested HSes; no SPOF. 
+- Events are actions within the system. Typically each action (e.g. sending a message)
+  correlates with exactly one event. Each event has a ``type`` string. 
+- ``type`` values SHOULD be namespaced according to standard Java package naming conventions, 
+  with a ``.`` delimiter e.g. ``com.example.myapp.event``
+- Events are typically send in the context of a room.
+
+Room structure
+--------------
 
 A room is a conceptual place where users can send and receive messages. Rooms 
 can be created, joined and left. Messages are sent to a room, and all 
 participants in that room will receive the message. Rooms are uniquely 
-identified via a room ID. There is exactly one room ID for each room.
+identified via a room ID. There is exactly one room ID for each room. Each
+room can also have an alias. Each room can have many aliases.
+
+::
 
-- Aliases
+                            How events flow in rooms
+                            ========================
+
+       { @alice:matrix.org }                             { @bob:domain.com }
+               |                                                 ^
+               |                                                 |
+      Room ID: !qporfwt:matrix.org                 Room ID: !qporfwt:matrix.org
+      Event type: m.room.message                   Event type: m.room.message
+      Content: { JSON object }                     Content: { JSON object }
+               |                                                 |
+               V                                                 |
+       +------------------+                            +------------------+
+       |   Home Server    |                            |   Home Server    |
+       |   matrix.org     |<-------Federation--------->|   domain.com     |
+       +------------------+                            +------------------+
+      Room ID: !qporfwt:matrix.org                    Room ID: !qporfwt:matrix.org
+      Servers: matrix.org, domain.com                 Servers: matrix.org, domain.com
+      Members:                                        Members:
+        - @alice:matrix.org                             - @alice:matrix.org
+        - @bob:domain.com                               - @bob:domain.com
+                  
+
+- Room IDs MUST have ! prefix; looks like !foo:domain - domain is simply for namespacing,
+  the room does NOT reside on domain. NOT human readable.
+- Room Aliases MUST have # prefix; looks like #foo:domain - domain indicates where this
+  alias can be mapped to a room ID. Key point: human readable / friendly.
+- User IDs MUST have @ prefix; looks like @foo:domain - domain indicates the user's home
+  server. 
+- Aliases can be queried on the domain they specify, which will return a room ID if a
+  mapping exists. These mappings can change.
+       
+Identity
+--------
+- Identity in relation to 3PIDs. Discovery of users based on 3PIDs.
+- Identity servers; trusted clique of servers which replicate content.
+- They govern the mapping of 3PIDs to user IDs and the creation of said mappings.
+- Not strictly required in order to communicate.
+
+
+API Standards
+-------------
+- All HTTP[S]
+- Uses JSON as HTTP bodies
+- Standard error response format { errcode: M_WHATEVER, error: "some message" }
+- C-S API provides POST for operations, or PUT with txn IDs. Explain txn IDs.
+
+Receiving live updates on a client
+----------------------------------
+- C-S longpoll event stream
+- Concept of start/end tokens.
+- Mention /initialSync to get token.
+
+
+Rooms
+=====
+- How are they created?
+- Adding / removing aliases.
 - Invite/join dance
 - State and non-state data (+extensibility)
 
@@ -46,10 +113,8 @@ TODO : Room permissions / config / power levels.
 Messages
 ========
 
-This specification outlines several standard message types, all of which are
-prefixed with "m.".
-
-- Namespacing?
+This specification outlines several standard event types, all of which are
+prefixed with ``m.``
 
 State messages
 --------------
@@ -174,88 +239,59 @@ The following keys can be attached to any ``m.room.message``:
 Presence
 ========
 
-Each user has the concept of Presence information. This encodes a sense of the
-"availability" of that user, suitable for display on other user's clients.
-
-The basic piece of presence information is an enumeration of a small set of
-state; such as "free to chat", "online", "busy", or "offline". The default state
-unless the user changes it is "online". Lower states suggest some amount of
-decreased availability from normal, which might have some client-side effect
-like muting notification sounds and suggests to other users not to bother them
-unless it is urgent. Equally, the "free to chat" state exists to let the user
-announce their general willingness to receive messages moreso than default.
-
-Home servers should also allow a user to set their state as "hidden" - a state
-which behaves as offline, but allows the user to see the client state anyway and
-generally interact with client features such as reading message history or
-accessing contacts in the address book.
-
-This basic state field applies to the user as a whole, regardless of how many
+Each user has the concept of presence information. This encodes the
+"availability" of that user, suitable for display on other user's clients. This
+is transmitted as an ``m.presence`` event and is one of the few events which
+are sent *outside the context of a room*. The basic piece of presence information 
+is represented by the ``state`` key, which is an enum of one of the following:
+
+  - ``online`` : The default state when the user is connected to an event stream.
+  - ``unavailable`` : The user is not reachable at this time.
+  - ``offline`` : The user is not connected to an event stream.
+  - ``free_for_chat`` : The user is generally willing to receive messages 
+    moreso than default.
+  - ``hidden`` : TODO. Behaves as offline, but allows the user to see the client 
+    state anyway and generally interact with client features.
+
+This basic ``state`` field applies to the user as a whole, regardless of how many
 client devices they have connected. The home server should synchronise this
 status choice among multiple devices to ensure the user gets a consistent
 experience.
 
 Idle Time
 ---------
-As well as the basic state field, the presence information can also show a sense
+As well as the basic ``state`` field, the presence information can also show a sense
 of an "idle timer". This should be maintained individually by the user's
-clients, and the homeserver can take the highest reported time as that to
-report. Likely this should be presented in fairly coarse granularity; possibly
-being limited to letting the home server automatically switch from a "free to
-chat" or "online" mode into "idle".
-
-When a user is offline, the Home Server can still report when the user was last
-seen online, again perhaps in a somewhat coarse manner.
+clients, and the home server can take the highest reported time as that to
+report. When a user is offline, the home server can still report when the user was last
+seen online.
 
-Device Type
------------
-Client devices that may limit the user experience somewhat (such as "mobile"
-devices with limited ability to type on a real keyboard or read large amounts of
-text) should report this to the home server, as this is also useful information
-to report as "presence" if the user cannot be expected to provide a good typed
-response to messages.
-
-- m.presence and enums (when should they be used)
+Transmission
+------------
+- Transmitted as an EDU.
+- Presence lists determine who to send to.
 
 Presence List
 -------------
 Each user's home server stores a "presence list" for that user. This stores a
-list of other user IDs the user has chosen to add to it (remembering any ACL
-Pointer if appropriate).
-
-To be added to a contact list, the user being added must grant permission. Once
-granted, both user's HS(es) store this information, as it allows the user who
-has added the contact some more abilities; see below. Since such subscriptions
+list of other user IDs the user has chosen to add to it. To be added to this 
+list, the user being added must receive permission from the list owner. Once
+granted, both user's HS(es) store this information. Since such subscriptions
 are likely to be bidirectional, HSes may wish to automatically accept requests
 when a reverse subscription already exists.
 
-As a convenience, presence lists should support the ability to collect users
-into groups, which could allow things like inviting the entire group to a new
-("ad-hoc") chat room, or easy interaction with the profile information ACL
-implementation of the HS.
-
 Presence and Permissions
 ------------------------
 For a viewing user to be allowed to see the presence information of a target
-user, either
+user, either:
 
- * The target user has allowed the viewing user to add them to their presence
+ - The target user has allowed the viewing user to add them to their presence
    list, or
-
- * The two users share at least one room in common
+ - The two users share at least one room in common
 
 In the latter case, this allows for clients to display some minimal sense of
 presence information in a user list for a room.
 
-Home servers can also use the user's choice of presence state as a signal for
-how to handle new private one-to-one chat message requests. For example, it
-might decide:
-
- - "free to chat": accept anything
- - "online": accept from anyone in my address book list
- - "busy": accept from anyone in this "important people" group in my address
-    book list
-
 Typing notifications
 ====================
 
@@ -274,18 +310,14 @@ human-friendly string. Profiles grant users the ability to see human-readable
 names for other users that are in some way meaningful to them. Additionally, 
 profiles can publish additional information, such as the user's age or location.
 
-It is also conceivable that since we are attempting to provide a
-worldwide-applicable messaging system, that users may wish to present different
-subsets of information in their profile to different other people, from a
-privacy and permissions perspective.
-
 A Profile consists of a display name, an avatar picture, and a set of other 
 metadata fields that the user may wish to publish (email address, phone
 numbers, website URLs, etc...). This specification puts no requirements on the 
-display name other than it being a valid Unicode string.
+display name other than it being a valid unicode string.
 
 - Metadata extensibility
 - Bundled with which events? e.g. m.room.member
+- Generate own events? What type?
 
 Registration and login
 ======================
@@ -312,8 +344,8 @@ The login process breaks down into the following:
      step 2.
      
 As each home server may have different ways of logging in, the client needs to know how
-they should login. All distinct login stages MUST have a corresponding ``'type'``.
-A ``'type'`` is a namespaced string which details the mechanism for logging in.
+they should login. All distinct login stages MUST have a corresponding ``type``.
+A ``type`` is a namespaced string which details the mechanism for logging in.
 
 A client may be able to login via multiple valid login flows, and should choose a single
 flow when logging in. A flow is a series of login stages. The home server MUST respond 
@@ -359,17 +391,17 @@ subsequent requests until the login is completed::
   }
 
 This specification defines the following login types:
- - m.login.password
- - m.login.oauth2
- - m.login.email.code
- - m.login.email.url
+ - ``m.login.password``
+ - ``m.login.oauth2``
+ - ``m.login.email.code``
+ - ``m.login.email.url``
 
 
 Password-based
 --------------
-Type: 
-  "m.login.password"
-Description:
+:Type: 
+  m.login.password
+:Description: 
   Login is supported via a username and password.
 
 To respond to this type, reply with::
@@ -385,9 +417,9 @@ process, or a standard error response.
 
 OAuth2-based
 ------------
-Type: 
-  "m.login.oauth2"
-Description:
+:Type: 
+  m.login.oauth2
+:Description:
   Login is supported via OAuth2 URLs. This login consists of multiple requests.
 
 To respond to this type, reply with::
@@ -438,9 +470,9 @@ visits the REDIRECT_URI with the auth code= query parameter which returns::
 
 Email-based (code)
 ------------------
-Type: 
-  "m.login.email.code"
-Description:
+:Type: 
+  m.login.email.code
+:Description:
   Login is supported by typing in a code which is sent in an email. This login 
   consists of multiple requests.
 
@@ -473,9 +505,9 @@ the login process, or a standard error response.
 
 Email-based (url)
 -----------------
-Type: 
-  "m.login.email.url"
-Description:
+:Type: 
+  m.login.email.url
+:Description:
   Login is supported by clicking on a URL in an email. This login consists of 
   multiple requests.
 
@@ -515,7 +547,7 @@ N-Factor Authentication
 -----------------------
 Multiple login stages can be combined to create N-factor authentication during login.
 
-This can be achieved by responding with the ``'next'`` login type on completion of a 
+This can be achieved by responding with the ``next`` login type on completion of a 
 previous login stage::
 
   {
@@ -523,7 +555,7 @@ previous login stage::
   }
 
 If a home server implements N-factor authentication, it MUST respond with all 
-``'stages'`` when initially queried for their login requirements::
+``stages`` when initially queried for their login requirements::
 
   {
     "type": "<1st login type>",
@@ -592,19 +624,19 @@ can also be performed.
 
 There are three main kinds of communication that occur between home servers:
 
- * Queries
+ - Queries
    These are single request/response interactions between a given pair of
    servers, initiated by one side sending an HTTP request to obtain some
    information, and responded by the other. They are not persisted and contain
    no long-term significant history. They simply request a snapshot state at the
    instant the query is made.
 
- * EDUs - Ephemeral Data Units
+ - EDUs - Ephemeral Data Units
    These are notifications of events that are pushed from one home server to
    another. They are not persisted and contain no long-term significant history,
    nor does the receiving home server have to reply to them.
 
- * PDUs - Persisted Data Units
+ - PDUs - Persisted Data Units
    These are notifications of events that are broadcast from one home server to
    any others that are interested in the same "context" (namely, a Room ID).
    They are persisted to long-term storage and form the record of history for
@@ -629,6 +661,8 @@ milliseconds) generated by its origin server, an origin and destination server
 name, a list of "previous IDs", and a list of PDUs - the actual message payload
 that the Transaction carries.
 
+::
+
  {"transaction_id":"916d630ea616342b42e98a3be0b74113",
   "ts":1404835423000,
   "origin":"red",
@@ -660,6 +694,8 @@ sent them), and a nested content field containing the actual event content.
 [[TODO(paul): Update this structure so that 'pdu_id' is a two-element
 [origin,ref] pair like the prev_pdus are]]
 
+::
+
  {"pdu_id":"a4ecee13e2accdadf56c1025af232176",
   "context":"#example.green",
   "origin":"green",
@@ -686,6 +722,8 @@ PDUs fall into two main categories: those that deliver Events, and those that
 synchronise State. For PDUs that relate to State synchronisation, additional
 keys exist to support this:
 
+::
+
  {...,
   "is_state":true,
   "state_key":TODO
@@ -704,6 +742,8 @@ EDUs, by comparison to PDUs, do not have an ID, a context, or a list of
 "previous" IDs. The only mandatory fields for these are the type, origin and
 destination home server names, and the actual nested content.
 
+::
+
  {"edu_type":"m.presence",
   "origin":"blue",
   "destination":"orange",