Merge branch 'develop' into server2server_tls

11 files changed, 166 insertions, 38 deletions

diff --git a/docs/client-server/howto.rst b/docs/client-server/howto.rst
index 9ef4cb5f78..3660c73d36 100644
--- a/docs/client-server/howto.rst
+++ b/docs/client-server/howto.rst
@@ -30,7 +30,7 @@ Registration
 The aim of registration is to get a user ID and access token which you will need
 when accessing other APIs::
 
-    curl -XPOST -d '{"user_id":"example", "password":"wordpass"}' "http://localhost:8080/matrix/client/api/v1/register"
+    curl -XPOST -d '{"user_id":"example", "password":"wordpass"}' "http://localhost:8080/_matrix/client/api/v1/register"
 
     {
         "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.AqdSzFmFYrLrTmteXc", 
@@ -51,13 +51,13 @@ Login
 -----
 The aim when logging in is to get an access token for your existing user ID::
 
-    curl -XGET "http://localhost:8080/matrix/client/api/v1/login"
+    curl -XGET "http://localhost:8080/_matrix/client/api/v1/login"
 
     {
         "type": "m.login.password"
     }
 
-    curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8080/matrix/client/api/v1/login"
+    curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8080/_matrix/client/api/v1/login"
 
     {
         "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd", 
@@ -87,7 +87,7 @@ Creating a room
 If you want to send a message to someone, you have to be in a room with them. To
 create a room::
 
-    curl -XPOST -d '{"room_alias_name":"tutorial"}' "http://localhost:8080/matrix/client/api/v1/rooms?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
+    curl -XPOST -d '{"room_alias_name":"tutorial"}' "http://localhost:8080/_matrix/client/api/v1/rooms?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
 
     {
         "room_alias": "#tutorial:localhost", 
@@ -105,7 +105,7 @@ Sending messages
 ----------------
 You can now send messages to this room::
 
-    curl -XPUT -d '{"msgtype":"m.text", "body":"hello"}' "http://localhost:8080/matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/messages/%40example%3Alocalhost/msgid1?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
+    curl -XPUT -d '{"msgtype":"m.text", "body":"hello"}' "http://localhost:8080/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/messages/%40example%3Alocalhost/msgid1?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
     
 NB: There are no limitations to the types of messages which can be exchanged.
 The only requirement is that ``"msgtype"`` is specified.
@@ -127,7 +127,7 @@ Inviting a user to a room
 -------------------------
 You can directly invite a user to a room like so::
 
-    curl -XPUT -d '{"membership":"invite"}' "http://localhost:8080/matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/members/%40myfriend%3Alocalhost/state?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
+    curl -XPUT -d '{"membership":"invite"}' "http://localhost:8080/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/members/%40myfriend%3Alocalhost/state?access_token=QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd"
     
 This informs ``@myfriend:localhost`` of the room ID 
 ``!CvcvRuDYDzTOzfKKgh:localhost`` and allows them to join the room.
@@ -137,7 +137,7 @@ Joining a room via an invite
 If you receive an invite, you can join the room by changing the membership to
 join::
 
-    curl -XPUT -d '{"membership":"join"}' "http://localhost:8080/matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/members/%40myfriend%3Alocalhost/state?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
+    curl -XPUT -d '{"membership":"join"}' "http://localhost:8080/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh:localhost/members/%40myfriend%3Alocalhost/state?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
     
 NB: Only the person invited (``@myfriend:localhost``) can change the membership
 state to ``"join"``.
@@ -147,7 +147,7 @@ Joining a room via an alias
 Alternatively, if you know the room alias for this room and the room config 
 allows it, you can directly join a room via the alias::
 
-    curl -XPUT -d '{}' "http://localhost:8080/matrix/client/api/v1/join/%23tutorial%3Alocalhost?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
+    curl -XPUT -d '{}' "http://localhost:8080/_matrix/client/api/v1/join/%23tutorial%3Alocalhost?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
     
     {
         "room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
@@ -173,7 +173,7 @@ Getting all state
 If the client doesn't know any information on the rooms the user is 
 invited/joined on, they can get all the user's state for all rooms::
 
-    curl -XGET "http://localhost:8080/matrix/client/api/v1/im/sync?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
+    curl -XGET "http://localhost:8080/_matrix/client/api/v1/im/sync?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK"
     
     [
         {
@@ -236,7 +236,7 @@ all of the messages and feedback for these rooms. This can be a LOT of data. You
 may just want the most recent message for each room. This can be achieved by 
 applying pagination stream parameters to this request::
 
-    curl -XGET "http://localhost:8080/matrix/client/api/v1/im/sync?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK&from=END&to=START&limit=1"
+    curl -XGET "http://localhost:8080/_matrix/client/api/v1/im/sync?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK&from=END&to=START&limit=1"
     
     [
         {
@@ -271,7 +271,7 @@ Getting live state
 Once you know which rooms the client has previously interacted with, you need to
 listen for incoming events. This can be done like so::
 
-    curl -XGET "http://localhost:8080/matrix/client/api/v1/events?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK&from=END"
+    curl -XGET "http://localhost:8080/_matrix/client/api/v1/events?access_token=QG15ZnJpZW5kOmxvY2FsaG9zdA...XKuGdVsovHmwMyDDvK&from=END"
     
     {
         "chunk": [], 
diff --git a/docs/client-server/specification.rst b/docs/client-server/specification.rst
index 4c1a3d4456..ee8bb5c420 100644
--- a/docs/client-server/specification.rst
+++ b/docs/client-server/specification.rst
@@ -306,11 +306,11 @@ POST requests MUST be submitted as application/json.
 All paths MUST be namespaced by the version of the API being used. This should
 be:
 
-/matrix/client/api/v1
+/_matrix/client/api/v1
 
 All REST paths in this section MUST be prefixed with this. E.g.
   REST Path: /rooms/$room_id
-  Absolute Path: /matrix/client/api/v1/rooms/$room_id
+  Absolute Path: /_matrix/client/api/v1/rooms/$room_id
 
 Registration
 ============
diff --git a/docs/client-server/swagger_matrix/directory b/docs/client-server/swagger_matrix/directory
index 3f3bef9c11..98109a0fbc 100644
--- a/docs/client-server/swagger_matrix/directory
+++ b/docs/client-server/swagger_matrix/directory
@@ -1,7 +1,7 @@
 {
   "apiVersion": "1.0.0",
   "swaggerVersion": "1.2",
-  "basePath": "http://localhost:8080/matrix/client/api/v1",
+  "basePath": "http://localhost:8080/_matrix/client/api/v1",
   "resourcePath": "/directory",
   "produces": [
     "application/json"
diff --git a/docs/client-server/swagger_matrix/events b/docs/client-server/swagger_matrix/events
index ca69d34db5..d22c68cc78 100644
--- a/docs/client-server/swagger_matrix/events
+++ b/docs/client-server/swagger_matrix/events
@@ -1,7 +1,7 @@
 {
   "apiVersion": "1.0.0",
   "swaggerVersion": "1.2",
-  "basePath": "http://localhost:8080/matrix/client/api/v1",
+  "basePath": "http://localhost:8080/_matrix/client/api/v1",
   "resourcePath": "/events",
   "produces": [
     "application/json"
diff --git a/docs/client-server/swagger_matrix/login b/docs/client-server/swagger_matrix/login
index 4410d3c887..8cc598b3c1 100644
--- a/docs/client-server/swagger_matrix/login
+++ b/docs/client-server/swagger_matrix/login
@@ -40,7 +40,7 @@
       "path": "/login"
     }
   ], 
-  "basePath": "http://localhost:8080/matrix/client/api/v1", 
+  "basePath": "http://localhost:8080/_matrix/client/api/v1", 
   "consumes": [
     "application/json"
   ], 
diff --git a/docs/client-server/swagger_matrix/presence b/docs/client-server/swagger_matrix/presence
index 1b4c7323aa..d52ce2164a 100644
--- a/docs/client-server/swagger_matrix/presence
+++ b/docs/client-server/swagger_matrix/presence
@@ -1,7 +1,7 @@
 {
   "apiVersion": "1.0.0",
   "swaggerVersion": "1.2",
-  "basePath": "http://localhost:8080/matrix/client/api/v1",
+  "basePath": "http://localhost:8080/_matrix/client/api/v1",
   "resourcePath": "/presence",
   "produces": [
     "application/json"
diff --git a/docs/client-server/swagger_matrix/profile b/docs/client-server/swagger_matrix/profile
index 1ebde62e20..188259fa3d 100644
--- a/docs/client-server/swagger_matrix/profile
+++ b/docs/client-server/swagger_matrix/profile
@@ -1,7 +1,7 @@
 {
   "apiVersion": "1.0.0",
   "swaggerVersion": "1.2",
-  "basePath": "http://localhost:8080/matrix/client/api/v1",
+  "basePath": "http://localhost:8080/_matrix/client/api/v1",
   "resourcePath": "/profile",
   "produces": [
     "application/json"
diff --git a/docs/client-server/swagger_matrix/registration b/docs/client-server/swagger_matrix/registration
index ccd542d11e..2048aec1d2 100644
--- a/docs/client-server/swagger_matrix/registration
+++ b/docs/client-server/swagger_matrix/registration
@@ -37,7 +37,7 @@
       "path": "/register"
     }
   ], 
-  "basePath": "http://localhost:8080/matrix/client/api/v1", 
+  "basePath": "http://localhost:8080/_matrix/client/api/v1", 
   "consumes": [
     "application/json"
   ], 
diff --git a/docs/client-server/swagger_matrix/rooms b/docs/client-server/swagger_matrix/rooms
index 1c32d135aa..7d3341f097 100644
--- a/docs/client-server/swagger_matrix/rooms
+++ b/docs/client-server/swagger_matrix/rooms
@@ -1,7 +1,7 @@
 {
   "apiVersion": "1.0.0",
   "swaggerVersion": "1.2",
-  "basePath": "http://localhost:8080/matrix/client/api/v1", 
+  "basePath": "http://localhost:8080/_matrix/client/api/v1", 
   "resourcePath": "/rooms",
   "produces": [
     "application/json"
diff --git a/docs/server-server/specification.rst b/docs/server-server/specification.rst
index a9ab9bff66..17cffafdd4 100644
--- a/docs/server-server/specification.rst
+++ b/docs/server-server/specification.rst
@@ -155,7 +155,7 @@ Protocol URLs
 
 All these URLs are namespaced within a prefix of 
 
-  /matrix/federation/v1/...
+  /_matrix/federation/v1/...
 
 For active pushing of messages representing live activity "as it happens":
 
diff --git a/docs/specification.rst b/docs/specification.rst
index d4a01a3fc2..3288aac019 100644
--- a/docs/specification.rst
+++ b/docs/specification.rst
@@ -35,8 +35,8 @@ namespaced to the home server which allocated the account and looks like::
 
   @localpart:domain
 
-The ``localpart`` of a user ID may be a user name, or an opaque ID identifying this user.
-
+The ``localpart`` of a user ID may be a user name, or an opaque ID identifying this user. They are
+case-insensitive.
 
 A "Home Server" is a server which provides C-S APIs and has the ability to federate with other HSes.
 It is typically responsible for multiple clients. "Federation" is the term used to describe the
@@ -60,7 +60,8 @@ identified via a "Room ID", which look like::
 
 There is exactly one room ID for each room. Whilst the room ID does contain a
 domain, it is simply for namespacing room IDs. The room does NOT reside on the
-domain specified. Room IDs are not meant to be human readable.
+domain specified. Room IDs are not meant to be human readable. They ARE
+case-sensitive.
 
 The following diagram shows an ``m.room.message`` event being sent in the room 
 ``!qporfwt:matrix.org``::
@@ -102,10 +103,10 @@ Each room can also have multiple "Room Aliases", which looks like::
 
 A room alias "points" to a room ID. The room ID the alias is pointing to can be obtained
 by visiting the domain specified. Room aliases are designed to be human readable strings
-which can be used to publicise rooms. Note that the mapping from a room alias to a 
-room ID is not fixed, and may change over time to point to a different room ID. For this
-reason, Clients SHOULD resolve the room alias to a room ID once and then use that ID on
-subsequent requests.
+which can be used to publicise rooms. They are case-insensitive. Note that the mapping 
+from a room alias to a room ID is not fixed, and may change over time to point to a 
+different room ID. For this reason, Clients SHOULD resolve the room alias to a room ID 
+once and then use that ID on subsequent requests.
 
 ::
 
@@ -214,24 +215,150 @@ In contrast, these are invalid requests::
       "key": "This is a put but it is missing a txnId."
     }
 
+
+
+- TODO: All strings everywhere are UTF-8
+
+
+
 Receiving live updates on a client
 ----------------------------------
-- C-S longpoll event stream
-- Concept of start/end tokens.
-- Mention /initialSync to get token.
+Clients can receive new events by long-polling the home server. This will hold open the
+HTTP connection for a short period of time waiting for new events, returning early if an
+event occurs. This is called the "Event Stream". All events which the client is authorised 
+to view will appear in the event stream. When the stream is closed, an ``end`` token is 
+returned. This token can be used in the next request to continue where the client left off.
 
+When the client first logs in, they will need to initially synchronise with their home
+server. This is achieved via the ``/initialSync`` API. This API also returns an ``end``
+token which can be used with the event stream.
 
 Rooms
 =====
-- How are they created? PDU anchor point: "root of the tree".
+
+Creation
+--------
+To create a room, a client has to use the ``/createRoom`` API. There are various options
+which can be set when creating a room:
+
+``visibility``
+  Type: 
+    String
+  Optional: 
+    Yes
+  Value:
+    Either ``public`` or ``private``.
+  Description:
+    A ``public`` visibility indicates that the room will be shown in the public room list. A
+    ``private`` visibility will hide the room from the public room list. Rooms default to
+    ``public`` visibility if this key is not included.
+
+``room_alias_name``
+  Type: 
+    String
+  Optional: 
+    Yes
+  Value:
+    The room alias localpart.
+  Description:
+    If this is included, a room alias will be created and mapped to the newly created room.
+    The alias will belong on the same home server which created the room, e.g.
+    ``!qadnasoi:domain.com >>> #room_alias_name:domain.com``
+
+Example::
+
+  {
+    "visibility": "public", 
+    "room_alias_name": "the pub"
+  }
+
+- TODO: This creates a room creation event which serves as the root of the PDU graph for this room.
+
+Modifying aliases
+-----------------
 - Adding / removing aliases.
-- Invite/join dance
-- State and non-state data (+extensibility)
 
-TODO : Room permissions / config / power levels.
+Permissions
+-----------
+- TODO : Room permissions / config / power levels. What they are. How do they work. Examples.
+
+Joining rooms
+-------------
+- What is joining? What permissions / access does it give you? How does this affect /initialSync?
+- API to hit (``/join/$alias or id``). Explain how alias joining works (auto-resolving).  See "Room events" for more info.
+- What does the home server have to do?
+- Rooms that DON'T need an invite to join. This follows through onto inviting users section.
+- Outline invite join dance?
 
-Messages
-========
+
+Inviting users
+--------------
+- Can invite users to a room if the room config key TODO is set to TODO. Must have required power level.
+- Outline invite join dance. What is it? Why is it required? How does it work?
+- What does the home server have to do?
+
+The purpose of inviting users to a room is to notify them that the room exists 
+so they can choose to become a member of that room. Some rooms require that all 
+users who join a room are previously invited to it (an "invite-only" room). 
+Whether a given room is an "invite-only" room is determined by the room config 
+key ``TODO``. It can have one of the following values:
+
+ - TODO Room config invite only value explanation
+ - TODO Room config free-to-join value explanation
+
+Only users who have a membership state of ``join`` in a room can invite new 
+users to said room. The person being invited must not be in the ``join`` state 
+in the room. The fully-qualified user ID must be specified when inviting a user, 
+as the user may reside on a different home server. To invite a user, send the 
+following request to ``/rooms/<room id>/invite``, which will manage the 
+entire invitation process::
+
+  {
+    "user_id": "<user id to invite>"
+  }
+
+Alternatively, the membership state for this user in this room can be modified 
+directly by sending the following request to 
+``/rooms/<room id>/state/m.room.member/<url encoded user id>``::
+
+  {
+    "membership": "invite"
+  }
+
+See the "Room events" section for more information on ``m.room.member``.
+
+- TODO: In what circumstances will this NOT be equivalent to ``/invite``?
+
+Leaving rooms
+-------------
+- API to hit (``$roomid/leave``). See "Room events" for more info.
+- Must be joined to leave. How does this affect /initialSync?
+- Not ever being in a room is NOT equivalent to have left it (due to membership: leave).
+- Need to be re-invited if invite-only room.
+- If no more HSes in room, can delete room?
+- Is there a dance?
+
+Events in a room
+----------------
+- Split into state and non-state data
+- Explain what they are, semantics, give examples of clobbering / not, use cases (msgs vs room names).
+  Not too much detail on the actual event contents.
+- API to hit.
+- Extensibility provided by the API for custom events. Examples.
+- How this hooks into ``initialSync``.
+- See the "Room Events" section for actual spec on each type.
+
+Syncing a room
+--------------
+- Single room initial sync. API to hit. Why it might be used (lazy loading)
+
+Getting grouped state events
+----------------------------
+- ``/members`` and ``/messages`` and the events they return.
+- ``/state`` and it returns ALL THE THINGS.
+
+Room Events
+===========
 
 This specification outlines several standard event types, all of which are
 prefixed with ``m.``
@@ -244,7 +371,8 @@ State messages
 - m.room.config
 - m.room.invite_join
 
-What are they, when are they used, what do they contain, how should they be used
+What are they, when are they used, what do they contain, how should they be used.
+Link back to explanatory sections (e.g. invite/join/leave sections for m.room.member)
 
 Non-state messages
 ------------------