From 550e8f32ac7a9bc56b57b515c515f85bf264e891 Mon Sep 17 00:00:00 2001 From: Kegan Dougal Date: Tue, 9 Sep 2014 13:51:03 -0700 Subject: Move model to client-server for now. --- docs/client-server/model/presence.rst | 249 +++++++++++++++++++++ docs/client-server/model/profiles.rst | 232 ++++++++++++++++++++ docs/client-server/model/protocol_examples.rst | 64 ++++++ docs/client-server/model/room-join-workflow.rst | 113 ++++++++++ docs/client-server/model/rooms.rst | 274 ++++++++++++++++++++++++ docs/client-server/model/terminology.rst | 86 ++++++++ docs/client-server/model/third-party-id.rst | 108 ++++++++++ docs/model/presence.rst | 249 --------------------- docs/model/profiles.rst | 232 -------------------- docs/model/protocol_examples.rst | 64 ------ docs/model/room-join-workflow.rst | 113 ---------- docs/model/rooms.rst | 274 ------------------------ docs/model/terminology.rst | 86 -------- docs/model/third-party-id.rst | 108 ---------- 14 files changed, 1126 insertions(+), 1126 deletions(-) create mode 100644 docs/client-server/model/presence.rst create mode 100644 docs/client-server/model/profiles.rst create mode 100644 docs/client-server/model/protocol_examples.rst create mode 100644 docs/client-server/model/room-join-workflow.rst create mode 100644 docs/client-server/model/rooms.rst create mode 100644 docs/client-server/model/terminology.rst create mode 100644 docs/client-server/model/third-party-id.rst delete mode 100644 docs/model/presence.rst delete mode 100644 docs/model/profiles.rst delete mode 100644 docs/model/protocol_examples.rst delete mode 100644 docs/model/room-join-workflow.rst delete mode 100644 docs/model/rooms.rst delete mode 100644 docs/model/terminology.rst delete mode 100644 docs/model/third-party-id.rst (limited to 'docs') diff --git a/docs/client-server/model/presence.rst b/docs/client-server/model/presence.rst new file mode 100644 index 0000000000..7e54505364 --- /dev/null +++ b/docs/client-server/model/presence.rst @@ -0,0 +1,249 @@ +======== +Presence +======== + +A description of presence information and visibility between users. + +Overview +======== + +Each user has the concept of Presence information. This encodes a sense of the +"availability" of that user, suitable for display on other user's clients. + + +Presence Information +==================== + +The basic piece of presence information is an enumeration of a small set of +state; such as "free to chat", "online", "busy", or "offline". The default state +unless the user changes it is "online". Lower states suggest some amount of +decreased availability from normal, which might have some client-side effect +like muting notification sounds and suggests to other users not to bother them +unless it is urgent. Equally, the "free to chat" state exists to let the user +announce their general willingness to receive messages moreso than default. + +Home servers should also allow a user to set their state as "hidden" - a state +which behaves as offline, but allows the user to see the client state anyway and +generally interact with client features such as reading message history or +accessing contacts in the address book. + +This basic state field applies to the user as a whole, regardless of how many +client devices they have connected. The home server should synchronise this +status choice among multiple devices to ensure the user gets a consistent +experience. + +Idle Time +--------- + +As well as the basic state field, the presence information can also show a sense +of an "idle timer". This should be maintained individually by the user's +clients, and the homeserver can take the highest reported time as that to +report. Likely this should be presented in fairly coarse granularity; possibly +being limited to letting the home server automatically switch from a "free to +chat" or "online" mode into "idle". + +When a user is offline, the Home Server can still report when the user was last +seen online, again perhaps in a somewhat coarse manner. + +Device Type +----------- + +Client devices that may limit the user experience somewhat (such as "mobile" +devices with limited ability to type on a real keyboard or read large amounts of +text) should report this to the home server, as this is also useful information +to report as "presence" if the user cannot be expected to provide a good typed +response to messages. + + +Presence List +============= + +Each user's home server stores a "presence list" for that user. This stores a +list of other user IDs the user has chosen to add to it (remembering any ACL +Pointer if appropriate). + +To be added to a contact list, the user being added must grant permission. Once +granted, both user's HS(es) store this information, as it allows the user who +has added the contact some more abilities; see below. Since such subscriptions +are likely to be bidirectional, HSes may wish to automatically accept requests +when a reverse subscription already exists. + +As a convenience, presence lists should support the ability to collect users +into groups, which could allow things like inviting the entire group to a new +("ad-hoc") chat room, or easy interaction with the profile information ACL +implementation of the HS. + + +Presence and Permissions +======================== + +For a viewing user to be allowed to see the presence information of a target +user, either + + * The target user has allowed the viewing user to add them to their presence + list, or + + * The two users share at least one room in common + +In the latter case, this allows for clients to display some minimal sense of +presence information in a user list for a room. + +Home servers can also use the user's choice of presence state as a signal for +how to handle new private one-to-one chat message requests. For example, it +might decide: + + "free to chat": accept anything + "online": accept from anyone in my addres book list + "busy": accept from anyone in this "important people" group in my address + book list + + +API Efficiency +============== + +A simple implementation of presence messaging has the ability to cause a large +amount of Internet traffic relating to presence updates. In order to minimise +the impact of such a feature, the following observations can be made: + + * There is no point in a Home Server polling status for peers in a user's + presence list if the user has no clients connected that care about it. + + * It is highly likely that most presence subscriptions will be symmetric - a + given user watching another is likely to in turn be watched by that user. + + * It is likely that most subscription pairings will be between users who share + at least one Room in common, and so their Home Servers are actively + exchanging message PDUs or transactions relating to that Room. + + * Presence update messages do not need realtime guarantees. It is acceptable to + delay delivery of updates for some small amount of time (10 seconds to a + minute). + +The general model of presence information is that of a HS registering its +interest in receiving presence status updates from other HSes, which then +promise to send them when required. Rather than actively polling for the +currentt state all the time, HSes can rely on their relative stability to only +push updates when required. + +A Home Server should not rely on the longterm validity of this presence +information, however, as this would not cover such cases as a user's server +crashing and thus failing to inform their peers that users it used to host are +no longer available online. Therefore, each promise of future updates should +carry with a timeout value (whether explicit in the message, or implicit as some +defined default in the protocol), after which the receiving HS should consider +the information potentially stale and request it again. + +However, because of the likelyhood that two home servers are exchanging messages +relating to chat traffic in a room common to both of them, the ongoing receipt +of these messages can be taken by each server as an implicit notification that +the sending server is still up and running, and therefore that no status changes +have happened; because if they had the server would have sent them. A second, +larger timeout should be applied to this implicit inference however, to protect +against implementation bugs or other reasons that the presence state cache may +become invalid; eventually the HS should re-enquire the current state of users +and update them with its own. + +The following workflows can therefore be used to handle presence updates: + + 1 When a user first appears online their HS sends a message to each other HS + containing at least one user to be watched; each message carrying both a + notification of the sender's new online status, and a request to obtain and + watch the target users' presence information. This message implicitly + promises the sending HS will now push updates to the target HSes. + + 2 The target HSes then respond a single message each, containing the current + status of the requested user(s). These messages too implicitly promise the + target HSes will themselves push updates to the sending HS. + + As these messages arrive at the sending user's HS they can be pushed to the + user's client(s), possibly batched again to ensure not too many small + messages which add extra protocol overheads. + +At this point, all the user's clients now have the current presence status +information for this moment in time, and have promised to send each other +updates in future. + + 3 The HS maintains two watchdog timers per peer HS it is exchanging presence + information with. The first timer should have a relatively small expiry + (perhaps 1 minute), and the second timer should have a much longer time + (perhaps 1 hour). + + 4 Any time any kind of message is received from a peer HS, the short-term + presence timer associated with it is reset. + + 5 Whenever either of these timers expires, an HS should push a status reminder + to the target HS whose timer has now expired, and request again from that + server the status of the subscribed users. + + 6 On receipt of one of these presence status reminders, an HS can reset both + of its presence watchdog timers. + +To avoid bursts of traffic, implementations should attempt to stagger the expiry +of the longer-term watchdog timers for different peer HSes. + +When individual users actively change their status (either by explicit requests +from clients, or inferred changes due to idle timers or client timeouts), the HS +should batch up any status changes for some reasonable amount of time (10 +seconds to a minute). This allows for reduced protocol overheads in the case of +multiple messages needing to be sent to the same peer HS; as is the likely +scenario in many cases, such as a given human user having multiple user +accounts. + + +API Requirements +================ + +The data model presented here puts the following requirements on the APIs: + +Client-Server +------------- + +Requests that a client can make to its Home Server + + * get/set current presence state + Basic enumeration + ability to set a custom piece of text + + * report per-device idle time + After some (configurable?) idle time the device should send a single message + to set the idle duration. The HS can then infer a "start of idle" instant and + use that to keep the device idleness up to date. At some later point the + device can cancel this idleness. + + * report per-device type + Inform the server that this device is a "mobile" device, or perhaps some + other to-be-defined category of reduced capability that could be presented to + other users. + + * start/stop presence polling for my presence list + It is likely that these messages could be implicitly inferred by other + messages, though having explicit control is always useful. + + * get my presence list + [implicit poll start?] + It is possible that the HS doesn't yet have current presence information when + the client requests this. There should be a "don't know" type too. + + * add/remove a user to my presence list + +Server-Server +------------- + +Requests that Home Servers make to others + + * request permission to add a user to presence list + + * allow/deny a request to add to a presence list + + * perform a combined presence state push and subscription request + For each sending user ID, the message contains their new status. + For each receiving user ID, the message should contain an indication on + whether the sending server is also interested in receiving status from that + user; either as an immediate update response now, or as a promise to send + future updates. + +Server to Client +---------------- + +[[TODO(paul): There also needs to be some way for a user's HS to push status +updates of the presence list to clients, but the general server-client event +model currently lacks a space to do that.]] diff --git a/docs/client-server/model/profiles.rst b/docs/client-server/model/profiles.rst new file mode 100644 index 0000000000..f7d6bd5679 --- /dev/null +++ b/docs/client-server/model/profiles.rst @@ -0,0 +1,232 @@ +======== +Profiles +======== + +A description of Synapse user profile metadata support. + + +Overview +======== + +Internally within Synapse users are referred to by an opaque ID, which consists +of some opaque localpart combined with the domain name of their home server. +Obviously this does not yield a very nice user experience; users would like to +see readable names for other users that are in some way meaningful to them. +Additionally, users like to be able to publish "profile" details to inform other +users of other information about them. + +It is also conceivable that since we are attempting to provide a +worldwide-applicable messaging system, that users may wish to present different +subsets of information in their profile to different other people, from a +privacy and permissions perspective. + +A Profile consists of a display name, an (optional?) avatar picture, and a set +of other metadata fields that the user may wish to publish (email address, phone +numbers, website URLs, etc...). We put no requirements on the display name other +than it being a valid Unicode string. Since it is likely that users will end up +having multiple accounts (perhaps by necessity of being hosted in multiple +places, perhaps by choice of wanting multiple distinct identifies), it would be +useful that a metadata field type exists that can refer to another Synapse User +ID, so that clients and HSes can make use of this information. + +Metadata Fields +--------------- + +[[TODO(paul): Likely this list is incomplete; more fields can be defined as we +think of them. At the very least, any sort of supported ID for the 3rd Party ID +servers should be accounted for here.]] + + * Synapse Directory Server username(s) + + * Email address + + * Phone number - classify "home"/"work"/"mobile"/custom? + + * Twitter/Facebook/Google+/... social networks + + * Location - keep this deliberately vague to allow people to choose how + granular it is + + * "Bio" information - date of birth, etc... + + * Synapse User ID of another account + + * Web URL + + * Freeform description text + + +Visibility Permissions +====================== + +A home server implementation could offer the ability to set permissions on +limited visibility of those fields. When another user requests access to the +target user's profile, their own identity should form part of that request. The +HS implementation can then decide which fields to make available to the +requestor. + +A particular detail of implementation could allow the user to create one or more +ACLs; where each list is granted permission to see a given set of non-public +fields (compare to Google+ Circles) and contains a set of other people allowed +to use it. By giving these ACLs strong identities within the HS, they can be +referenced in communications with it, granting other users who encounter these +the "ACL Token" to use the details in that ACL. + +If we further allow an ACL Token to be present on Room join requests or stored +by 3PID servers, then users of these ACLs gain the extra convenience of not +having to manually curate people in the access list; anyone in the room or with +knowledge of the 3rd Party ID is automatically granted access. Every HS and +client implementation would have to be aware of the existence of these ACL +Token, and include them in requests if present, but not every HS implementation +needs to actually provide the full permissions model. This can be used as a +distinguishing feature among competing implementations. However, servers MUST +NOT serve profile information from a cache if there is a chance that its limited +understanding could lead to information leakage. + + +Client Concerns of Multiple Accounts +==================================== + +Because a given person may want to have multiple Synapse User accounts, client +implementations should allow the use of multiple accounts simultaneously +(especially in the field of mobile phone clients, which generally don't support +running distinct instances of the same application). Where features like address +books, presence lists or rooms are presented, the client UI should remember to +make distinct with user account is in use for each. + + +Directory Servers +================= + +Directory Servers can provide a forward mapping from human-readable names to +User IDs. These can provide a service similar to giving domain-namespaced names +for Rooms; in this case they can provide a way for a user to reference their +User ID in some external form (e.g. that can be printed on a business card). + +The format for Synapse user name will consist of a localpart specific to the +directory server, and the domain name of that directory server: + + @localname:some.domain.name + +The localname is separated from the domain name using a colon, so as to ensure +the localname can still contain periods, as users may want this for similarity +to email addresses or the like, which typically can contain them. The format is +also visually quite distinct from email addresses, phone numbers, etc... so +hopefully reasonably "self-describing" when written on e.g. a business card +without surrounding context. + +[[TODO(paul): we might have to think about this one - too close to email? + Twitter? Also it suggests a format scheme for room names of + #localname:domain.name, which I quite like]] + +Directory server administrators should be able to make some kind of policy +decision on how these are allocated. Servers within some "closed" domain (such +as company-specific ones) may wish to verify the validity of a mapping using +their own internal mechanisms; "public" naming servers can operate on a FCFS +basis. There are overlapping concerns here with the idea of the 3rd party +identity servers as well, though in this specific case we are creating a new +namespace to allocate names into. + +It would also be nice from a user experience perspective if the profile that a +given name links to can also declare that name as part of its metadata. +Furthermore as a security and consistency perspective it would be nice if each +end (the directory server and the user's home server) check the validity of the +mapping in some way. This needs investigation from a security perspective to +ensure against spoofing. + +One such model may be that the user starts by declaring their intent to use a +given user name link to their home server, which then contacts the directory +service. At some point later (maybe immediately for "public open FCFS servers", +maybe after some kind of human intervention for verification) the DS decides to +honour this link, and includes it in its served output. It should also tell the +HS of this fact, so that the HS can present this as fact when requested for the +profile information. For efficiency, it may further wish to provide the HS with +a cryptographically-signed certificate as proof, so the HS serving the profile +can provide that too when asked, avoiding requesting HSes from constantly having +to contact the DS to verify this mapping. (Note: This is similar to the security +model often applied in DNS to verify PTR <-> A bidirectional mappings). + + +Identity Servers +================ + +The identity servers should support the concept of pointing a 3PID being able to +store an ACL Token as well as the main User ID. It is however, beyond scope to +do any kind of verification that any third-party IDs that the profile is +claiming match up to the 3PID mappings. + + +User Interface and Expectations Concerns +======================================== + +Given the weak "security" of some parts of this model as compared to what users +might expect, some care should be taken on how it is presented to users, +specifically in the naming or other wording of user interface components. + +Most notably mere knowledge of an ACL Pointer is enough to read the information +stored in it. It is possible that Home or Identity Servers could leak this +information, allowing others to see it. This is a security-vs-convenience +balancing choice on behalf of the user who would choose, or not, to make use of +such a feature to publish their information. + +Additionally, unless some form of strong end-to-end user-based encryption is +used, a user of ACLs for information privacy has to trust other home servers not +to lie about the identify of the user requesting access to the Profile. + + +API Requirements +================ + +The data model presented here puts the following requirements on the APIs: + +Client-Server +------------- + +Requests that a client can make to its Home Server + + * get/set my Display Name + This should return/take a simple "text/plain" field + + * get/set my Avatar URL + The avatar image data itself is not stored by this API; we'll just store a + URL to let the clients fetch it. Optionally HSes could integrate this with + their generic content attacmhent storage service, allowing a user to set + upload their profile Avatar and update the URL to point to it. + + * get/add/remove my metadata fields + Also we need to actually define types of metadata + + * get another user's Display Name / Avatar / metadata fields + +[[TODO(paul): At some later stage we should consider the API for: + + * get/set ACL permissions on my metadata fields + + * manage my ACL tokens +]] + +Server-Server +------------- + +Requests that Home Servers make to others + + * get a user's Display Name / Avatar + + * get a user's full profile - name/avatar + MD fields + This request must allow for specifying the User ID of the requesting user, + for permissions purposes. It also needs to take into account any ACL Tokens + the requestor has. + + * push a change of Display Name to observers (overlaps with the presence API) + +Room Event PDU Types +-------------------- + +Events that are pushed from Home Servers to other Home Servers or clients. + + * user Display Name change + + * user Avatar change + [[TODO(paul): should the avatar image itself be stored in all the room + histories? maybe this event should just be a hint to clients that they should + re-fetch the avatar image]] diff --git a/docs/client-server/model/protocol_examples.rst b/docs/client-server/model/protocol_examples.rst new file mode 100644 index 0000000000..61a599b432 --- /dev/null +++ b/docs/client-server/model/protocol_examples.rst @@ -0,0 +1,64 @@ +PUT /send/abc/ HTTP/1.1 +Host: ... +Content-Length: ... +Content-Type: application/json + +{ + "origin": "localhost:5000", + "pdus": [ + { + "content": {}, + "context": "tng", + "depth": 12, + "is_state": false, + "origin": "localhost:5000", + "pdu_id": 1404381396854, + "pdu_type": "feedback", + "prev_pdus": [ + [ + "1404381395883", + "localhost:6000" + ] + ], + "ts": 1404381427581 + } + ], + "prev_ids": [ + "1404381396852" + ], + "ts": 1404381427823 +} + +HTTP/1.1 200 OK +... + +====================================== + +GET /pull/-1/ HTTP/1.1 +Host: ... +Content-Length: 0 + +HTTP/1.1 200 OK +Content-Length: ... +Content-Type: application/json + +{ + origin: ..., + prev_ids: ..., + data: [ + { + data_id: ..., + prev_pdus: [...], + depth: ..., + ts: ..., + context: ..., + origin: ..., + content: { + ... + } + }, + ..., + ] +} + + diff --git a/docs/client-server/model/room-join-workflow.rst b/docs/client-server/model/room-join-workflow.rst new file mode 100644 index 0000000000..c321a64fab --- /dev/null +++ b/docs/client-server/model/room-join-workflow.rst @@ -0,0 +1,113 @@ +================== +Room Join Workflow +================== + +An outline of the workflows required when a user joins a room. + +Discovery +========= + +To join a room, a user has to discover the room by some mechanism in order to +obtain the (opaque) Room ID and a candidate list of likely home servers that +contain it. + +Sending an Invitation +--------------------- + +The most direct way a user discovers the existence of a room is from a +invitation from some other user who is a member of that room. + +The inviter's HS sets the membership status of the invitee to "invited" in the +"m.members" state key by sending a state update PDU. The HS then broadcasts this +PDU among the existing members in the usual way. An invitation message is also +sent to the invited user, containing the Room ID and the PDU ID of this +invitation state change and potentially a list of some other home servers to use +to accept the invite. The user's client can then choose to display it in some +way to alert the user. + +[[TODO(paul): At present, no API has been designed or described to actually send +that invite to the invited user. Likely it will be some facet of the larger +user-user API required for presence, profile management, etc...]] + +Directory Service +----------------- + +Alternatively, the user may discover the channel via a directory service; either +by performing a name lookup, or some kind of browse or search acitivty. However +this is performed, the end result is that the user's home server requests the +Room ID and candidate list from the directory service. + +[[TODO(paul): At present, no API has been designed or described for this +directory service]] + + +Joining +======= + +Once the ID and home servers are obtained, the user can then actually join the +room. + +Accepting an Invite +------------------- + +If a user has received and accepted an invitation to join a room, the invitee's +home server can now send an invite acceptance message to a chosen candidate +server from the list given in the invitation, citing also the PDU ID of the +invitation as "proof" of their invite. (This is required as due to late message +propagation it could be the case that the acceptance is received before the +invite by some servers). If this message is allowed by the candidate server, it +generates a new PDU that updates the invitee's membership status to "joined", +referring back to the acceptance PDU, and broadcasts that as a state change in +the usual way. The newly-invited user is now a full member of the room, and +state propagation proceeds as usual. + +Joining a Public Room +--------------------- + +If a user has discovered the existence of a room they wish to join but does not +have an active invitation, they can request to join it directly by sending a +join message to a candidate server on the list provided by the directory +service. As this list may be out of date, the HS should be prepared to retry +other candidates if the chosen one is no longer aware of the room, because it +has no users as members in it. + +Once a candidate server that is aware of the room has been found, it can +broadcast an update PDU to add the member into the "m.members" key setting their +state directly to "joined" (i.e. bypassing the two-phase invite semantics), +remembering to include the new user's HS in that list. + +Knocking on a Semi-Public Room +------------------------------ + +If a user requests to join a room but the join mode of the room is "knock", the +join is not immediately allowed. Instead, if the user wishes to proceed, they +can instead post a "knock" message, which informs other members of the room that +the would-be joiner wishes to become a member and sets their membership value to +"knocked". If any of them wish to accept this, they can then send an invitation +in the usual way described above. Knowing that the user has already knocked and +expressed an interest in joining, the invited user's home server should +immediately accept that invitation on the user's behalf, and go on to join the +room in the usual way. + +[[NOTE(Erik): Though this may confuse users who expect 'X has joined' to +actually be a user initiated action, i.e. they may expect that 'X' is actually +looking at synapse right now?]] + +[[NOTE(paul): Yes, a fair point maybe we should suggest HSes don't do that, and +just offer an invite to the user as normal]] + +Private and Non-Existent Rooms +------------------------------ + +If a user requests to join a room but the room is either unknown by the home +server receiving the request, or is known by the join mode is "invite" and the +user has not been invited, the server must respond that the room does not exist. +This is to prevent leaking information about the existence and identity of +private rooms. + + +Outstanding Questions +===================== + + * Do invitations or knocks time out and expire at some point? If so when? Time + is hard in distributed systems. diff --git a/docs/client-server/model/rooms.rst b/docs/client-server/model/rooms.rst new file mode 100644 index 0000000000..0007e48e30 --- /dev/null +++ b/docs/client-server/model/rooms.rst @@ -0,0 +1,274 @@ +=========== +Rooms Model +=========== + +A description of the general data model used to implement Rooms, and the +user-level visible effects and implications. + + +Overview +======== + +"Rooms" in Synapse are shared messaging channels over which all the participant +users can exchange messages. Rooms have an opaque persistent identify, a +globally-replicated set of state (consisting principly of a membership set of +users, and other management and miscellaneous metadata), and a message history. + + +Room Identity and Naming +======================== + +Rooms can be arbitrarily created by any user on any home server; at which point +the home server will sign the message that creates the channel, and the +fingerprint of this signature becomes the strong persistent identify of the +room. This now identifies the room to any home server in the network regardless +of its original origin. This allows the identify of the room to outlive any +particular server. Subject to appropriate permissions [to be discussed later], +any current member of a room can invite others to join it, can post messages +that become part of its history, and can change the persistent state of the room +(including its current set of permissions). + +Home servers can provide a directory service, allowing a lookup from a +convenient human-readable form of room label to a room ID. This mapping is +scoped to the particular home server domain and so simply represents that server +administrator's opinion of what room should take that label; it does not have to +be globally replicated and does not form part of the stored state of that room. + +This room name takes the form + + #localname:some.domain.name + +for similarity and consistency with user names on directories. + +To join a room (and therefore to be allowed to inspect past history, post new +messages to it, and read its state), a user must become aware of the room's +fingerprint ID. There are two mechanisms to allow this: + + * An invite message from someone else in the room + + * A referral from a room directory service + +As room IDs are opaque and ephemeral, they can serve as a mechanism to create +"ad-hoc" rooms deliberately unnamed, for small group-chats or even private +one-to-one message exchange. + + +Stored State and Permissions +============================ + +Every room has a globally-replicated set of stored state. This state is a set of +key/value or key/subkey/value pairs. The value of every (sub)key is a +JSON-representable object. The main key of a piece of stored state establishes +its meaning; some keys store sub-keys to allow a sub-structure within them [more +detail below]. Some keys have special meaning to Synapse, as they relate to +management details of the room itself, storing such details as user membership, +and permissions of users to alter the state of the room itself. Other keys may +store information to present to users, which the system does not directly rely +on. The key space itself is namespaced, allowing 3rd party extensions, subject +to suitable permission. + +Permission management is based on the concept of "power-levels". Every user +within a room has an integer assigned, being their "power-level" within that +room. Along with its actual data value, each key (or subkey) also stores the +minimum power-level a user must have in order to write to that key, the +power-level of the last user who actually did write to it, and the PDU ID of +that state change. + +To be accepted as valid, a change must NOT: + + * Be made by a user having a power-level lower than required to write to the + state key + + * Alter the required power-level for that state key to a value higher than the + user has + + * Increase that user's own power-level + + * Grant any other user a power-level higher than the level of the user making + the change + +[[TODO(paul): consider if relaxations should be allowed; e.g. is the current +outright-winner allowed to raise their own level, to allow for "inflation"?]] + + +Room State Keys +=============== + +[[TODO(paul): if this list gets too big it might become necessary to move it +into its own doc]] + +The following keys have special semantics or meaning to Synapse itself: + +m.member (has subkeys) + Stores a sub-key for every Synapse User ID which is currently a member of + this room. Its value gives the membership type ("knocked", "invited", + "joined"). + +m.power_levels + Stores a mapping from Synapse User IDs to their power-level in the room. If + they are not present in this mapping, the default applies. + + The reason to store this as a single value rather than a value with subkeys + is that updates to it are atomic; allowing a number of colliding-edit + problems to be avoided. + +m.default_level + Gives the default power-level for members of the room that do not have one + specified in their membership key. + +m.invite_level + If set, gives the minimum power-level required for members to invite others + to join, or to accept knock requests from non-members requesting access. If + absent, then invites are not allowed. An invitation involves setting their + membership type to "invited", in addition to sending the invite message. + +m.join_rules + Encodes the rules on how non-members can join the room. Has the following + possibilities: + "public" - a non-member can join the room directly + "knock" - a non-member cannot join the room, but can post a single "knock" + message requesting access, which existing members may approve or deny + "invite" - non-members cannot join the room without an invite from an + existing member + "private" - nobody who is not in the 'may_join' list or already a member + may join by any mechanism + + In any of the first three modes, existing members with sufficient permission + can send invites to non-members if allowed by the "m.invite_level" key. A + "private" room is not allowed to have the "m.invite_level" set. + + A client may use the value of this key to hint at the user interface + expectations to provide; in particular, a private chat with one other use + might warrant specific handling in the client. + +m.may_join + A list of User IDs that are always allowed to join the room, regardless of any + of the prevailing join rules and invite levels. These apply even to private + rooms. These are stored in a single list with normal update-powerlevel + permissions applied; users cannot arbitrarily remove themselves from the list. + +m.add_state_level + The power-level required for a user to be able to add new state keys. + +m.public_history + If set and true, anyone can request the history of the room, without needing + to be a member of the room. + +m.archive_servers + For "public" rooms with public history, gives a list of home servers that + should be included in message distribution to the room, even if no users on + that server are present. These ensure that a public room can still persist + even if no users are currently members of it. This list should be consulted by + the dirctory servers as the candidate list they respond with. + +The following keys are provided by Synapse for user benefit, but their value is +not otherwise used by Synapse. + +m.name + Stores a short human-readable name for the room, such that clients can display + to a user to assist in identifying which room is which. + + This name specifically is not the strong ID used by the message transport + system to refer to the room, because it may be changed from time to time. + +m.topic + Stores the current human-readable topic + + +Room Creation Templates +======================= + +A client (or maybe home server?) could offer a few templates for the creation of +new rooms. For example, for a simple private one-to-one chat the channel could +assign the creator a power-level of 1, requiring a level of 1 to invite, and +needing an invite before members can join. An invite is then sent to the other +party, and if accepted and the other user joins, the creator's power-level can +now be reduced to 0. This now leaves a room with two participants in it being +unable to add more. + + +Rooms that Continue History +=========================== + +An option that could be considered for room creation, is that when a new room is +created the creator could specify a PDU ID into an existing room, as the history +continuation point. This would be stored as an extra piece of meta-data on the +initial PDU of the room's creation. (It does not appear in the normal previous +PDU linkage). + +This would allow users in rooms to "fork" a room, if it is considered that the +conversations in the room no longer fit its original purpose, and wish to +diverge. Existing permissions on the original room would continue to apply of +course, for viewing that history. If both rooms are considered "public" we might +also want to define a message to post into the original room to represent this +fork point, and give a reference to the new room. + + +User Direct Message Rooms +========================= + +There is no need to build a mechanism for directly sending messages between +users, because a room can handle this ability. To allow direct user-to-user chat +messaging we simply need to be able to create rooms with specific set of +permissions to allow this direct messaging. + +Between any given pair of user IDs that wish to exchange private messages, there +will exist a single shared Room, created lazily by either side. These rooms will +need a certain amount of special handling in both home servers and display on +clients, but as much as possible should be treated by the lower layers of code +the same as other rooms. + +Specially, a client would likely offer a special menu choice associated with +another user (in room member lists, presence list, etc..) as "direct chat". That +would perform all the necessary steps to create the private chat room. Receiving +clients should display these in a special way too as the room name is not +important; instead it should distinguish them on the Display Name of the other +party. + +Home Servers will need a client-API option to request setting up a new user-user +chat room, which will then need special handling within the server. It will +create a new room with the following + + m.member: the proposing user + m.join_rules: "private" + m.may_join: both users + m.power_levels: empty + m.default_level: 0 + m.add_state_level: 0 + m.public_history: False + +Having created the room, it can send an invite message to the other user in the +normal way - the room permissions state that no users can be set to the invited +state, but because they're in the may_join list then they'd be allowed to join +anyway. + +In this arrangement there is now a room with both users may join but neither has +the power to invite any others. Both users now have the confidence that (at +least within the messaging system itself) their messages remain private and +cannot later be provably leaked to a third party. They can freely set the topic +or name if they choose and add or edit any other state of the room. The update +powerlevel of each of these fixed properties should be 1, to lock out the users +from being able to alter them. + + +Anti-Glare +========== + +There exists the possibility of a race condition if two users who have no chat +history with each other simultaneously create a room and invite the other to it. +This is called a "glare" situation. There are two possible ideas for how to +resolve this: + + * Each Home Server should persist the mapping of (user ID pair) to room ID, so + that duplicate requests can be suppressed. On receipt of a room creation + request that the HS thinks there already exists a room for, the invitation to + join can be rejected if: + a) the HS believes the sending user is already a member of the room (and + maybe their HS has forgotten this fact), or + b) the proposed room has a lexicographically-higher ID than the existing + room (to resolve true race condition conflicts) + + * The room ID for a private 1:1 chat has a special form, determined by + concatenting the User IDs of both members in a deterministic order, such that + it doesn't matter which side creates it first; the HSes can just ignore + (or merge?) received PDUs that create the room twice. diff --git a/docs/client-server/model/terminology.rst b/docs/client-server/model/terminology.rst new file mode 100644 index 0000000000..cc6e6760ac --- /dev/null +++ b/docs/client-server/model/terminology.rst @@ -0,0 +1,86 @@ +=========== +Terminology +=========== + +A list of definitions of specific terminology used among these documents. +These terms were originally taken from the server-server documentation, and may +not currently match the exact meanings used in other places; though as a +medium-term goal we should encourage the unification of this terminology. + + +Terms +===== + +Backfilling: + The process of synchronising historic state from one home server to another, + to backfill the event storage so that scrollback can be presented to the + client(s). (Formerly, and confusingly, called 'pagination') + +Context: + A single human-level entity of interest (currently, a chat room) + +EDU (Ephemeral Data Unit): + A message that relates directly to a given pair of home servers that are + exchanging it. EDUs are short-lived messages that related only to one single + pair of servers; they are not persisted for a long time and are not forwarded + on to other servers. Because of this, they have no internal ID nor previous + EDUs reference chain. + +Event: + A record of activity that records a single thing that happened on to a context + (currently, a chat room). These are the "chat messages" that Synapse makes + available. + [[NOTE(paul): The current server-server implementation calls these simply + "messages" but the term is too ambiguous here; I've called them Events]] + +PDU (Persistent Data Unit): + A message that relates to a single context, irrespective of the server that + is communicating it. PDUs either encode a single Event, or a single State + change. A PDU is referred to by its PDU ID; the pair of its origin server + and local reference from that server. + +PDU ID: + The pair of PDU Origin and PDU Reference, that together globally uniquely + refers to a specific PDU. + +PDU Origin: + The name of the origin server that generated a given PDU. This may not be the + server from which it has been received, due to the way they are copied around + from server to server. The origin always records the original server that + created it. + +PDU Reference: + A local ID used to refer to a specific PDU from a given origin server. These + references are opaque at the protocol level, but may optionally have some + structured meaning within a given origin server or implementation. + +Presence: + The concept of whether a user is currently online, how available they declare + they are, and so on. See also: doc/model/presence + +Profile: + A set of metadata about a user, such as a display name, provided for the + benefit of other users. See also: doc/model/profiles + +Room ID: + An opaque string (of as-yet undecided format) that identifies a particular + room and used in PDUs referring to it. + +Room Alias: + A human-readable string of the form #name:some.domain that users can use as a + pointer to identify a room; a Directory Server will map this to its Room ID + +State: + A set of metadata maintained about a Context, which is replicated among the + servers in addition to the history of Events. + +User ID: + A string of the form @localpart:domain.name that identifies a user for + wire-protocol purposes. The localpart is meaningless outside of a particular + home server. This takes a human-readable form that end-users can use directly + if they so wish, avoiding the 3PIDs. + +Transaction: + A message which relates to the communication between a given pair of servers. + A transaction contains possibly-empty lists of PDUs and EDUs. + diff --git a/docs/client-server/model/third-party-id.rst b/docs/client-server/model/third-party-id.rst new file mode 100644 index 0000000000..1f8138ddf7 --- /dev/null +++ b/docs/client-server/model/third-party-id.rst @@ -0,0 +1,108 @@ +====================== +Third Party Identities +====================== + +A description of how email addresses, mobile phone numbers and other third +party identifiers can be used to authenticate and discover users in Matrix. + + +Overview +======== + +New users need to authenticate their account. An email or SMS text message can +be a convenient form of authentication. Users already have email addresses +and phone numbers for contacts in their address book. They want to communicate +with those contacts in Matrix without manually exchanging a Matrix User ID with +them. + +Third Party IDs +--------------- + +[[TODO(markjh): Describe the format of a 3PID]] + + +Third Party ID Associations +--------------------------- + +An Associaton is a binding between a Matrix User ID and a Third Party ID (3PID). +Each 3PID can be associated with one Matrix User ID at a time. + +[[TODO(markjh): JSON format of the association.]] + +Verification +------------ + +An Assocation must be verified by a trusted Verification Server. Email +addresses and phone numbers can be verified by sending a token to the address +which a client can supply to the verifier to confirm ownership. + +An email Verification Server may be capable of verifying all email 3PIDs or may +be restricted to verifying addresses for a particular domain. A phone number +Verification Server may be capable of verifying all phone numbers or may be +restricted to verifying numbers for a given country or phone prefix. + +Verification Servers fulfil a similar role to Certificate Authorities in PKI so +a similar level of vetting should be required before clients trust their +signatures. + +A Verification Server may wish to check for existing Associations for a 3PID +before creating a new Association. + +Discovery +--------- + +Users can discover Associations using a trusted Identity Server. Each +Association will be signed by the Identity Server. An Identity Server may store +the entire space of Associations or may delegate to other Identity Servers when +looking up Associations. + +Each Association returned from an Identity Server must be signed by a +Verification Server. Clients should check these signatures. + +Identity Servers fulfil a similar role to DNS servers. + +Privacy +------- + +A User may publish the association between their phone number and Matrix User ID +on the Identity Server without publishing the number in their Profile hosted on +their Home Server. + +Identity Servers should refrain from publishing reverse mappings and should +take steps, such as rate limiting, to prevent attackers enumerating the space of +mappings. + +Federation +========== + +Delegation +---------- + +Verification Servers could delegate signing to another server by issuing +certificate to that server allowing it to verify and sign a subset of 3PID on +its behalf. It would be necessary to provide a language for describing which +subset of 3PIDs that server had authority to validate. Alternatively it could +delegate the verification step to another server but sign the resulting +association itself. + +The 3PID space will have a heirachical structure like DNS so Identity Servers +can delegate lookups to other servers. An Identity Server should be prepared +to host or delegate any valid association within the subset of the 3PIDs it is +resonsible for. + +Multiple Root Verification Servers +---------------------------------- + +There can be multiple root Verification Servers and an Association could be +signed by multiple servers if different clients trust different subsets of +the verification servers. + +Multiple Root Identity Servers +------------------------------ + +There can be be multiple root Identity Servers. Clients will add each +Association to all root Identity Servers. + +[[TODO(markjh): Describe how clients find the list of root Identity Servers]] + + diff --git a/docs/model/presence.rst b/docs/model/presence.rst deleted file mode 100644 index 7e54505364..0000000000 --- a/docs/model/presence.rst +++ /dev/null @@ -1,249 +0,0 @@ -======== -Presence -======== - -A description of presence information and visibility between users. - -Overview -======== - -Each user has the concept of Presence information. This encodes a sense of the -"availability" of that user, suitable for display on other user's clients. - - -Presence Information -==================== - -The basic piece of presence information is an enumeration of a small set of -state; such as "free to chat", "online", "busy", or "offline". The default state -unless the user changes it is "online". Lower states suggest some amount of -decreased availability from normal, which might have some client-side effect -like muting notification sounds and suggests to other users not to bother them -unless it is urgent. Equally, the "free to chat" state exists to let the user -announce their general willingness to receive messages moreso than default. - -Home servers should also allow a user to set their state as "hidden" - a state -which behaves as offline, but allows the user to see the client state anyway and -generally interact with client features such as reading message history or -accessing contacts in the address book. - -This basic state field applies to the user as a whole, regardless of how many -client devices they have connected. The home server should synchronise this -status choice among multiple devices to ensure the user gets a consistent -experience. - -Idle Time ---------- - -As well as the basic state field, the presence information can also show a sense -of an "idle timer". This should be maintained individually by the user's -clients, and the homeserver can take the highest reported time as that to -report. Likely this should be presented in fairly coarse granularity; possibly -being limited to letting the home server automatically switch from a "free to -chat" or "online" mode into "idle". - -When a user is offline, the Home Server can still report when the user was last -seen online, again perhaps in a somewhat coarse manner. - -Device Type ------------ - -Client devices that may limit the user experience somewhat (such as "mobile" -devices with limited ability to type on a real keyboard or read large amounts of -text) should report this to the home server, as this is also useful information -to report as "presence" if the user cannot be expected to provide a good typed -response to messages. - - -Presence List -============= - -Each user's home server stores a "presence list" for that user. This stores a -list of other user IDs the user has chosen to add to it (remembering any ACL -Pointer if appropriate). - -To be added to a contact list, the user being added must grant permission. Once -granted, both user's HS(es) store this information, as it allows the user who -has added the contact some more abilities; see below. Since such subscriptions -are likely to be bidirectional, HSes may wish to automatically accept requests -when a reverse subscription already exists. - -As a convenience, presence lists should support the ability to collect users -into groups, which could allow things like inviting the entire group to a new -("ad-hoc") chat room, or easy interaction with the profile information ACL -implementation of the HS. - - -Presence and Permissions -======================== - -For a viewing user to be allowed to see the presence information of a target -user, either - - * The target user has allowed the viewing user to add them to their presence - list, or - - * The two users share at least one room in common - -In the latter case, this allows for clients to display some minimal sense of -presence information in a user list for a room. - -Home servers can also use the user's choice of presence state as a signal for -how to handle new private one-to-one chat message requests. For example, it -might decide: - - "free to chat": accept anything - "online": accept from anyone in my addres book list - "busy": accept from anyone in this "important people" group in my address - book list - - -API Efficiency -============== - -A simple implementation of presence messaging has the ability to cause a large -amount of Internet traffic relating to presence updates. In order to minimise -the impact of such a feature, the following observations can be made: - - * There is no point in a Home Server polling status for peers in a user's - presence list if the user has no clients connected that care about it. - - * It is highly likely that most presence subscriptions will be symmetric - a - given user watching another is likely to in turn be watched by that user. - - * It is likely that most subscription pairings will be between users who share - at least one Room in common, and so their Home Servers are actively - exchanging message PDUs or transactions relating to that Room. - - * Presence update messages do not need realtime guarantees. It is acceptable to - delay delivery of updates for some small amount of time (10 seconds to a - minute). - -The general model of presence information is that of a HS registering its -interest in receiving presence status updates from other HSes, which then -promise to send them when required. Rather than actively polling for the -currentt state all the time, HSes can rely on their relative stability to only -push updates when required. - -A Home Server should not rely on the longterm validity of this presence -information, however, as this would not cover such cases as a user's server -crashing and thus failing to inform their peers that users it used to host are -no longer available online. Therefore, each promise of future updates should -carry with a timeout value (whether explicit in the message, or implicit as some -defined default in the protocol), after which the receiving HS should consider -the information potentially stale and request it again. - -However, because of the likelyhood that two home servers are exchanging messages -relating to chat traffic in a room common to both of them, the ongoing receipt -of these messages can be taken by each server as an implicit notification that -the sending server is still up and running, and therefore that no status changes -have happened; because if they had the server would have sent them. A second, -larger timeout should be applied to this implicit inference however, to protect -against implementation bugs or other reasons that the presence state cache may -become invalid; eventually the HS should re-enquire the current state of users -and update them with its own. - -The following workflows can therefore be used to handle presence updates: - - 1 When a user first appears online their HS sends a message to each other HS - containing at least one user to be watched; each message carrying both a - notification of the sender's new online status, and a request to obtain and - watch the target users' presence information. This message implicitly - promises the sending HS will now push updates to the target HSes. - - 2 The target HSes then respond a single message each, containing the current - status of the requested user(s). These messages too implicitly promise the - target HSes will themselves push updates to the sending HS. - - As these messages arrive at the sending user's HS they can be pushed to the - user's client(s), possibly batched again to ensure not too many small - messages which add extra protocol overheads. - -At this point, all the user's clients now have the current presence status -information for this moment in time, and have promised to send each other -updates in future. - - 3 The HS maintains two watchdog timers per peer HS it is exchanging presence - information with. The first timer should have a relatively small expiry - (perhaps 1 minute), and the second timer should have a much longer time - (perhaps 1 hour). - - 4 Any time any kind of message is received from a peer HS, the short-term - presence timer associated with it is reset. - - 5 Whenever either of these timers expires, an HS should push a status reminder - to the target HS whose timer has now expired, and request again from that - server the status of the subscribed users. - - 6 On receipt of one of these presence status reminders, an HS can reset both - of its presence watchdog timers. - -To avoid bursts of traffic, implementations should attempt to stagger the expiry -of the longer-term watchdog timers for different peer HSes. - -When individual users actively change their status (either by explicit requests -from clients, or inferred changes due to idle timers or client timeouts), the HS -should batch up any status changes for some reasonable amount of time (10 -seconds to a minute). This allows for reduced protocol overheads in the case of -multiple messages needing to be sent to the same peer HS; as is the likely -scenario in many cases, such as a given human user having multiple user -accounts. - - -API Requirements -================ - -The data model presented here puts the following requirements on the APIs: - -Client-Server -------------- - -Requests that a client can make to its Home Server - - * get/set current presence state - Basic enumeration + ability to set a custom piece of text - - * report per-device idle time - After some (configurable?) idle time the device should send a single message - to set the idle duration. The HS can then infer a "start of idle" instant and - use that to keep the device idleness up to date. At some later point the - device can cancel this idleness. - - * report per-device type - Inform the server that this device is a "mobile" device, or perhaps some - other to-be-defined category of reduced capability that could be presented to - other users. - - * start/stop presence polling for my presence list - It is likely that these messages could be implicitly inferred by other - messages, though having explicit control is always useful. - - * get my presence list - [implicit poll start?] - It is possible that the HS doesn't yet have current presence information when - the client requests this. There should be a "don't know" type too. - - * add/remove a user to my presence list - -Server-Server -------------- - -Requests that Home Servers make to others - - * request permission to add a user to presence list - - * allow/deny a request to add to a presence list - - * perform a combined presence state push and subscription request - For each sending user ID, the message contains their new status. - For each receiving user ID, the message should contain an indication on - whether the sending server is also interested in receiving status from that - user; either as an immediate update response now, or as a promise to send - future updates. - -Server to Client ----------------- - -[[TODO(paul): There also needs to be some way for a user's HS to push status -updates of the presence list to clients, but the general server-client event -model currently lacks a space to do that.]] diff --git a/docs/model/profiles.rst b/docs/model/profiles.rst deleted file mode 100644 index f7d6bd5679..0000000000 --- a/docs/model/profiles.rst +++ /dev/null @@ -1,232 +0,0 @@ -======== -Profiles -======== - -A description of Synapse user profile metadata support. - - -Overview -======== - -Internally within Synapse users are referred to by an opaque ID, which consists -of some opaque localpart combined with the domain name of their home server. -Obviously this does not yield a very nice user experience; users would like to -see readable names for other users that are in some way meaningful to them. -Additionally, users like to be able to publish "profile" details to inform other -users of other information about them. - -It is also conceivable that since we are attempting to provide a -worldwide-applicable messaging system, that users may wish to present different -subsets of information in their profile to different other people, from a -privacy and permissions perspective. - -A Profile consists of a display name, an (optional?) avatar picture, and a set -of other metadata fields that the user may wish to publish (email address, phone -numbers, website URLs, etc...). We put no requirements on the display name other -than it being a valid Unicode string. Since it is likely that users will end up -having multiple accounts (perhaps by necessity of being hosted in multiple -places, perhaps by choice of wanting multiple distinct identifies), it would be -useful that a metadata field type exists that can refer to another Synapse User -ID, so that clients and HSes can make use of this information. - -Metadata Fields ---------------- - -[[TODO(paul): Likely this list is incomplete; more fields can be defined as we -think of them. At the very least, any sort of supported ID for the 3rd Party ID -servers should be accounted for here.]] - - * Synapse Directory Server username(s) - - * Email address - - * Phone number - classify "home"/"work"/"mobile"/custom? - - * Twitter/Facebook/Google+/... social networks - - * Location - keep this deliberately vague to allow people to choose how - granular it is - - * "Bio" information - date of birth, etc... - - * Synapse User ID of another account - - * Web URL - - * Freeform description text - - -Visibility Permissions -====================== - -A home server implementation could offer the ability to set permissions on -limited visibility of those fields. When another user requests access to the -target user's profile, their own identity should form part of that request. The -HS implementation can then decide which fields to make available to the -requestor. - -A particular detail of implementation could allow the user to create one or more -ACLs; where each list is granted permission to see a given set of non-public -fields (compare to Google+ Circles) and contains a set of other people allowed -to use it. By giving these ACLs strong identities within the HS, they can be -referenced in communications with it, granting other users who encounter these -the "ACL Token" to use the details in that ACL. - -If we further allow an ACL Token to be present on Room join requests or stored -by 3PID servers, then users of these ACLs gain the extra convenience of not -having to manually curate people in the access list; anyone in the room or with -knowledge of the 3rd Party ID is automatically granted access. Every HS and -client implementation would have to be aware of the existence of these ACL -Token, and include them in requests if present, but not every HS implementation -needs to actually provide the full permissions model. This can be used as a -distinguishing feature among competing implementations. However, servers MUST -NOT serve profile information from a cache if there is a chance that its limited -understanding could lead to information leakage. - - -Client Concerns of Multiple Accounts -==================================== - -Because a given person may want to have multiple Synapse User accounts, client -implementations should allow the use of multiple accounts simultaneously -(especially in the field of mobile phone clients, which generally don't support -running distinct instances of the same application). Where features like address -books, presence lists or rooms are presented, the client UI should remember to -make distinct with user account is in use for each. - - -Directory Servers -================= - -Directory Servers can provide a forward mapping from human-readable names to -User IDs. These can provide a service similar to giving domain-namespaced names -for Rooms; in this case they can provide a way for a user to reference their -User ID in some external form (e.g. that can be printed on a business card). - -The format for Synapse user name will consist of a localpart specific to the -directory server, and the domain name of that directory server: - - @localname:some.domain.name - -The localname is separated from the domain name using a colon, so as to ensure -the localname can still contain periods, as users may want this for similarity -to email addresses or the like, which typically can contain them. The format is -also visually quite distinct from email addresses, phone numbers, etc... so -hopefully reasonably "self-describing" when written on e.g. a business card -without surrounding context. - -[[TODO(paul): we might have to think about this one - too close to email? - Twitter? Also it suggests a format scheme for room names of - #localname:domain.name, which I quite like]] - -Directory server administrators should be able to make some kind of policy -decision on how these are allocated. Servers within some "closed" domain (such -as company-specific ones) may wish to verify the validity of a mapping using -their own internal mechanisms; "public" naming servers can operate on a FCFS -basis. There are overlapping concerns here with the idea of the 3rd party -identity servers as well, though in this specific case we are creating a new -namespace to allocate names into. - -It would also be nice from a user experience perspective if the profile that a -given name links to can also declare that name as part of its metadata. -Furthermore as a security and consistency perspective it would be nice if each -end (the directory server and the user's home server) check the validity of the -mapping in some way. This needs investigation from a security perspective to -ensure against spoofing. - -One such model may be that the user starts by declaring their intent to use a -given user name link to their home server, which then contacts the directory -service. At some point later (maybe immediately for "public open FCFS servers", -maybe after some kind of human intervention for verification) the DS decides to -honour this link, and includes it in its served output. It should also tell the -HS of this fact, so that the HS can present this as fact when requested for the -profile information. For efficiency, it may further wish to provide the HS with -a cryptographically-signed certificate as proof, so the HS serving the profile -can provide that too when asked, avoiding requesting HSes from constantly having -to contact the DS to verify this mapping. (Note: This is similar to the security -model often applied in DNS to verify PTR <-> A bidirectional mappings). - - -Identity Servers -================ - -The identity servers should support the concept of pointing a 3PID being able to -store an ACL Token as well as the main User ID. It is however, beyond scope to -do any kind of verification that any third-party IDs that the profile is -claiming match up to the 3PID mappings. - - -User Interface and Expectations Concerns -======================================== - -Given the weak "security" of some parts of this model as compared to what users -might expect, some care should be taken on how it is presented to users, -specifically in the naming or other wording of user interface components. - -Most notably mere knowledge of an ACL Pointer is enough to read the information -stored in it. It is possible that Home or Identity Servers could leak this -information, allowing others to see it. This is a security-vs-convenience -balancing choice on behalf of the user who would choose, or not, to make use of -such a feature to publish their information. - -Additionally, unless some form of strong end-to-end user-based encryption is -used, a user of ACLs for information privacy has to trust other home servers not -to lie about the identify of the user requesting access to the Profile. - - -API Requirements -================ - -The data model presented here puts the following requirements on the APIs: - -Client-Server -------------- - -Requests that a client can make to its Home Server - - * get/set my Display Name - This should return/take a simple "text/plain" field - - * get/set my Avatar URL - The avatar image data itself is not stored by this API; we'll just store a - URL to let the clients fetch it. Optionally HSes could integrate this with - their generic content attacmhent storage service, allowing a user to set - upload their profile Avatar and update the URL to point to it. - - * get/add/remove my metadata fields - Also we need to actually define types of metadata - - * get another user's Display Name / Avatar / metadata fields - -[[TODO(paul): At some later stage we should consider the API for: - - * get/set ACL permissions on my metadata fields - - * manage my ACL tokens -]] - -Server-Server -------------- - -Requests that Home Servers make to others - - * get a user's Display Name / Avatar - - * get a user's full profile - name/avatar + MD fields - This request must allow for specifying the User ID of the requesting user, - for permissions purposes. It also needs to take into account any ACL Tokens - the requestor has. - - * push a change of Display Name to observers (overlaps with the presence API) - -Room Event PDU Types --------------------- - -Events that are pushed from Home Servers to other Home Servers or clients. - - * user Display Name change - - * user Avatar change - [[TODO(paul): should the avatar image itself be stored in all the room - histories? maybe this event should just be a hint to clients that they should - re-fetch the avatar image]] diff --git a/docs/model/protocol_examples.rst b/docs/model/protocol_examples.rst deleted file mode 100644 index 61a599b432..0000000000 --- a/docs/model/protocol_examples.rst +++ /dev/null @@ -1,64 +0,0 @@ -PUT /send/abc/ HTTP/1.1 -Host: ... -Content-Length: ... -Content-Type: application/json - -{ - "origin": "localhost:5000", - "pdus": [ - { - "content": {}, - "context": "tng", - "depth": 12, - "is_state": false, - "origin": "localhost:5000", - "pdu_id": 1404381396854, - "pdu_type": "feedback", - "prev_pdus": [ - [ - "1404381395883", - "localhost:6000" - ] - ], - "ts": 1404381427581 - } - ], - "prev_ids": [ - "1404381396852" - ], - "ts": 1404381427823 -} - -HTTP/1.1 200 OK -... - -====================================== - -GET /pull/-1/ HTTP/1.1 -Host: ... -Content-Length: 0 - -HTTP/1.1 200 OK -Content-Length: ... -Content-Type: application/json - -{ - origin: ..., - prev_ids: ..., - data: [ - { - data_id: ..., - prev_pdus: [...], - depth: ..., - ts: ..., - context: ..., - origin: ..., - content: { - ... - } - }, - ..., - ] -} - - diff --git a/docs/model/room-join-workflow.rst b/docs/model/room-join-workflow.rst deleted file mode 100644 index c321a64fab..0000000000 --- a/docs/model/room-join-workflow.rst +++ /dev/null @@ -1,113 +0,0 @@ -================== -Room Join Workflow -================== - -An outline of the workflows required when a user joins a room. - -Discovery -========= - -To join a room, a user has to discover the room by some mechanism in order to -obtain the (opaque) Room ID and a candidate list of likely home servers that -contain it. - -Sending an Invitation ---------------------- - -The most direct way a user discovers the existence of a room is from a -invitation from some other user who is a member of that room. - -The inviter's HS sets the membership status of the invitee to "invited" in the -"m.members" state key by sending a state update PDU. The HS then broadcasts this -PDU among the existing members in the usual way. An invitation message is also -sent to the invited user, containing the Room ID and the PDU ID of this -invitation state change and potentially a list of some other home servers to use -to accept the invite. The user's client can then choose to display it in some -way to alert the user. - -[[TODO(paul): At present, no API has been designed or described to actually send -that invite to the invited user. Likely it will be some facet of the larger -user-user API required for presence, profile management, etc...]] - -Directory Service ------------------ - -Alternatively, the user may discover the channel via a directory service; either -by performing a name lookup, or some kind of browse or search acitivty. However -this is performed, the end result is that the user's home server requests the -Room ID and candidate list from the directory service. - -[[TODO(paul): At present, no API has been designed or described for this -directory service]] - - -Joining -======= - -Once the ID and home servers are obtained, the user can then actually join the -room. - -Accepting an Invite -------------------- - -If a user has received and accepted an invitation to join a room, the invitee's -home server can now send an invite acceptance message to a chosen candidate -server from the list given in the invitation, citing also the PDU ID of the -invitation as "proof" of their invite. (This is required as due to late message -propagation it could be the case that the acceptance is received before the -invite by some servers). If this message is allowed by the candidate server, it -generates a new PDU that updates the invitee's membership status to "joined", -referring back to the acceptance PDU, and broadcasts that as a state change in -the usual way. The newly-invited user is now a full member of the room, and -state propagation proceeds as usual. - -Joining a Public Room ---------------------- - -If a user has discovered the existence of a room they wish to join but does not -have an active invitation, they can request to join it directly by sending a -join message to a candidate server on the list provided by the directory -service. As this list may be out of date, the HS should be prepared to retry -other candidates if the chosen one is no longer aware of the room, because it -has no users as members in it. - -Once a candidate server that is aware of the room has been found, it can -broadcast an update PDU to add the member into the "m.members" key setting their -state directly to "joined" (i.e. bypassing the two-phase invite semantics), -remembering to include the new user's HS in that list. - -Knocking on a Semi-Public Room ------------------------------- - -If a user requests to join a room but the join mode of the room is "knock", the -join is not immediately allowed. Instead, if the user wishes to proceed, they -can instead post a "knock" message, which informs other members of the room that -the would-be joiner wishes to become a member and sets their membership value to -"knocked". If any of them wish to accept this, they can then send an invitation -in the usual way described above. Knowing that the user has already knocked and -expressed an interest in joining, the invited user's home server should -immediately accept that invitation on the user's behalf, and go on to join the -room in the usual way. - -[[NOTE(Erik): Though this may confuse users who expect 'X has joined' to -actually be a user initiated action, i.e. they may expect that 'X' is actually -looking at synapse right now?]] - -[[NOTE(paul): Yes, a fair point maybe we should suggest HSes don't do that, and -just offer an invite to the user as normal]] - -Private and Non-Existent Rooms ------------------------------- - -If a user requests to join a room but the room is either unknown by the home -server receiving the request, or is known by the join mode is "invite" and the -user has not been invited, the server must respond that the room does not exist. -This is to prevent leaking information about the existence and identity of -private rooms. - - -Outstanding Questions -===================== - - * Do invitations or knocks time out and expire at some point? If so when? Time - is hard in distributed systems. diff --git a/docs/model/rooms.rst b/docs/model/rooms.rst deleted file mode 100644 index 0007e48e30..0000000000 --- a/docs/model/rooms.rst +++ /dev/null @@ -1,274 +0,0 @@ -=========== -Rooms Model -=========== - -A description of the general data model used to implement Rooms, and the -user-level visible effects and implications. - - -Overview -======== - -"Rooms" in Synapse are shared messaging channels over which all the participant -users can exchange messages. Rooms have an opaque persistent identify, a -globally-replicated set of state (consisting principly of a membership set of -users, and other management and miscellaneous metadata), and a message history. - - -Room Identity and Naming -======================== - -Rooms can be arbitrarily created by any user on any home server; at which point -the home server will sign the message that creates the channel, and the -fingerprint of this signature becomes the strong persistent identify of the -room. This now identifies the room to any home server in the network regardless -of its original origin. This allows the identify of the room to outlive any -particular server. Subject to appropriate permissions [to be discussed later], -any current member of a room can invite others to join it, can post messages -that become part of its history, and can change the persistent state of the room -(including its current set of permissions). - -Home servers can provide a directory service, allowing a lookup from a -convenient human-readable form of room label to a room ID. This mapping is -scoped to the particular home server domain and so simply represents that server -administrator's opinion of what room should take that label; it does not have to -be globally replicated and does not form part of the stored state of that room. - -This room name takes the form - - #localname:some.domain.name - -for similarity and consistency with user names on directories. - -To join a room (and therefore to be allowed to inspect past history, post new -messages to it, and read its state), a user must become aware of the room's -fingerprint ID. There are two mechanisms to allow this: - - * An invite message from someone else in the room - - * A referral from a room directory service - -As room IDs are opaque and ephemeral, they can serve as a mechanism to create -"ad-hoc" rooms deliberately unnamed, for small group-chats or even private -one-to-one message exchange. - - -Stored State and Permissions -============================ - -Every room has a globally-replicated set of stored state. This state is a set of -key/value or key/subkey/value pairs. The value of every (sub)key is a -JSON-representable object. The main key of a piece of stored state establishes -its meaning; some keys store sub-keys to allow a sub-structure within them [more -detail below]. Some keys have special meaning to Synapse, as they relate to -management details of the room itself, storing such details as user membership, -and permissions of users to alter the state of the room itself. Other keys may -store information to present to users, which the system does not directly rely -on. The key space itself is namespaced, allowing 3rd party extensions, subject -to suitable permission. - -Permission management is based on the concept of "power-levels". Every user -within a room has an integer assigned, being their "power-level" within that -room. Along with its actual data value, each key (or subkey) also stores the -minimum power-level a user must have in order to write to that key, the -power-level of the last user who actually did write to it, and the PDU ID of -that state change. - -To be accepted as valid, a change must NOT: - - * Be made by a user having a power-level lower than required to write to the - state key - - * Alter the required power-level for that state key to a value higher than the - user has - - * Increase that user's own power-level - - * Grant any other user a power-level higher than the level of the user making - the change - -[[TODO(paul): consider if relaxations should be allowed; e.g. is the current -outright-winner allowed to raise their own level, to allow for "inflation"?]] - - -Room State Keys -=============== - -[[TODO(paul): if this list gets too big it might become necessary to move it -into its own doc]] - -The following keys have special semantics or meaning to Synapse itself: - -m.member (has subkeys) - Stores a sub-key for every Synapse User ID which is currently a member of - this room. Its value gives the membership type ("knocked", "invited", - "joined"). - -m.power_levels - Stores a mapping from Synapse User IDs to their power-level in the room. If - they are not present in this mapping, the default applies. - - The reason to store this as a single value rather than a value with subkeys - is that updates to it are atomic; allowing a number of colliding-edit - problems to be avoided. - -m.default_level - Gives the default power-level for members of the room that do not have one - specified in their membership key. - -m.invite_level - If set, gives the minimum power-level required for members to invite others - to join, or to accept knock requests from non-members requesting access. If - absent, then invites are not allowed. An invitation involves setting their - membership type to "invited", in addition to sending the invite message. - -m.join_rules - Encodes the rules on how non-members can join the room. Has the following - possibilities: - "public" - a non-member can join the room directly - "knock" - a non-member cannot join the room, but can post a single "knock" - message requesting access, which existing members may approve or deny - "invite" - non-members cannot join the room without an invite from an - existing member - "private" - nobody who is not in the 'may_join' list or already a member - may join by any mechanism - - In any of the first three modes, existing members with sufficient permission - can send invites to non-members if allowed by the "m.invite_level" key. A - "private" room is not allowed to have the "m.invite_level" set. - - A client may use the value of this key to hint at the user interface - expectations to provide; in particular, a private chat with one other use - might warrant specific handling in the client. - -m.may_join - A list of User IDs that are always allowed to join the room, regardless of any - of the prevailing join rules and invite levels. These apply even to private - rooms. These are stored in a single list with normal update-powerlevel - permissions applied; users cannot arbitrarily remove themselves from the list. - -m.add_state_level - The power-level required for a user to be able to add new state keys. - -m.public_history - If set and true, anyone can request the history of the room, without needing - to be a member of the room. - -m.archive_servers - For "public" rooms with public history, gives a list of home servers that - should be included in message distribution to the room, even if no users on - that server are present. These ensure that a public room can still persist - even if no users are currently members of it. This list should be consulted by - the dirctory servers as the candidate list they respond with. - -The following keys are provided by Synapse for user benefit, but their value is -not otherwise used by Synapse. - -m.name - Stores a short human-readable name for the room, such that clients can display - to a user to assist in identifying which room is which. - - This name specifically is not the strong ID used by the message transport - system to refer to the room, because it may be changed from time to time. - -m.topic - Stores the current human-readable topic - - -Room Creation Templates -======================= - -A client (or maybe home server?) could offer a few templates for the creation of -new rooms. For example, for a simple private one-to-one chat the channel could -assign the creator a power-level of 1, requiring a level of 1 to invite, and -needing an invite before members can join. An invite is then sent to the other -party, and if accepted and the other user joins, the creator's power-level can -now be reduced to 0. This now leaves a room with two participants in it being -unable to add more. - - -Rooms that Continue History -=========================== - -An option that could be considered for room creation, is that when a new room is -created the creator could specify a PDU ID into an existing room, as the history -continuation point. This would be stored as an extra piece of meta-data on the -initial PDU of the room's creation. (It does not appear in the normal previous -PDU linkage). - -This would allow users in rooms to "fork" a room, if it is considered that the -conversations in the room no longer fit its original purpose, and wish to -diverge. Existing permissions on the original room would continue to apply of -course, for viewing that history. If both rooms are considered "public" we might -also want to define a message to post into the original room to represent this -fork point, and give a reference to the new room. - - -User Direct Message Rooms -========================= - -There is no need to build a mechanism for directly sending messages between -users, because a room can handle this ability. To allow direct user-to-user chat -messaging we simply need to be able to create rooms with specific set of -permissions to allow this direct messaging. - -Between any given pair of user IDs that wish to exchange private messages, there -will exist a single shared Room, created lazily by either side. These rooms will -need a certain amount of special handling in both home servers and display on -clients, but as much as possible should be treated by the lower layers of code -the same as other rooms. - -Specially, a client would likely offer a special menu choice associated with -another user (in room member lists, presence list, etc..) as "direct chat". That -would perform all the necessary steps to create the private chat room. Receiving -clients should display these in a special way too as the room name is not -important; instead it should distinguish them on the Display Name of the other -party. - -Home Servers will need a client-API option to request setting up a new user-user -chat room, which will then need special handling within the server. It will -create a new room with the following - - m.member: the proposing user - m.join_rules: "private" - m.may_join: both users - m.power_levels: empty - m.default_level: 0 - m.add_state_level: 0 - m.public_history: False - -Having created the room, it can send an invite message to the other user in the -normal way - the room permissions state that no users can be set to the invited -state, but because they're in the may_join list then they'd be allowed to join -anyway. - -In this arrangement there is now a room with both users may join but neither has -the power to invite any others. Both users now have the confidence that (at -least within the messaging system itself) their messages remain private and -cannot later be provably leaked to a third party. They can freely set the topic -or name if they choose and add or edit any other state of the room. The update -powerlevel of each of these fixed properties should be 1, to lock out the users -from being able to alter them. - - -Anti-Glare -========== - -There exists the possibility of a race condition if two users who have no chat -history with each other simultaneously create a room and invite the other to it. -This is called a "glare" situation. There are two possible ideas for how to -resolve this: - - * Each Home Server should persist the mapping of (user ID pair) to room ID, so - that duplicate requests can be suppressed. On receipt of a room creation - request that the HS thinks there already exists a room for, the invitation to - join can be rejected if: - a) the HS believes the sending user is already a member of the room (and - maybe their HS has forgotten this fact), or - b) the proposed room has a lexicographically-higher ID than the existing - room (to resolve true race condition conflicts) - - * The room ID for a private 1:1 chat has a special form, determined by - concatenting the User IDs of both members in a deterministic order, such that - it doesn't matter which side creates it first; the HSes can just ignore - (or merge?) received PDUs that create the room twice. diff --git a/docs/model/terminology.rst b/docs/model/terminology.rst deleted file mode 100644 index cc6e6760ac..0000000000 --- a/docs/model/terminology.rst +++ /dev/null @@ -1,86 +0,0 @@ -=========== -Terminology -=========== - -A list of definitions of specific terminology used among these documents. -These terms were originally taken from the server-server documentation, and may -not currently match the exact meanings used in other places; though as a -medium-term goal we should encourage the unification of this terminology. - - -Terms -===== - -Backfilling: - The process of synchronising historic state from one home server to another, - to backfill the event storage so that scrollback can be presented to the - client(s). (Formerly, and confusingly, called 'pagination') - -Context: - A single human-level entity of interest (currently, a chat room) - -EDU (Ephemeral Data Unit): - A message that relates directly to a given pair of home servers that are - exchanging it. EDUs are short-lived messages that related only to one single - pair of servers; they are not persisted for a long time and are not forwarded - on to other servers. Because of this, they have no internal ID nor previous - EDUs reference chain. - -Event: - A record of activity that records a single thing that happened on to a context - (currently, a chat room). These are the "chat messages" that Synapse makes - available. - [[NOTE(paul): The current server-server implementation calls these simply - "messages" but the term is too ambiguous here; I've called them Events]] - -PDU (Persistent Data Unit): - A message that relates to a single context, irrespective of the server that - is communicating it. PDUs either encode a single Event, or a single State - change. A PDU is referred to by its PDU ID; the pair of its origin server - and local reference from that server. - -PDU ID: - The pair of PDU Origin and PDU Reference, that together globally uniquely - refers to a specific PDU. - -PDU Origin: - The name of the origin server that generated a given PDU. This may not be the - server from which it has been received, due to the way they are copied around - from server to server. The origin always records the original server that - created it. - -PDU Reference: - A local ID used to refer to a specific PDU from a given origin server. These - references are opaque at the protocol level, but may optionally have some - structured meaning within a given origin server or implementation. - -Presence: - The concept of whether a user is currently online, how available they declare - they are, and so on. See also: doc/model/presence - -Profile: - A set of metadata about a user, such as a display name, provided for the - benefit of other users. See also: doc/model/profiles - -Room ID: - An opaque string (of as-yet undecided format) that identifies a particular - room and used in PDUs referring to it. - -Room Alias: - A human-readable string of the form #name:some.domain that users can use as a - pointer to identify a room; a Directory Server will map this to its Room ID - -State: - A set of metadata maintained about a Context, which is replicated among the - servers in addition to the history of Events. - -User ID: - A string of the form @localpart:domain.name that identifies a user for - wire-protocol purposes. The localpart is meaningless outside of a particular - home server. This takes a human-readable form that end-users can use directly - if they so wish, avoiding the 3PIDs. - -Transaction: - A message which relates to the communication between a given pair of servers. - A transaction contains possibly-empty lists of PDUs and EDUs. - diff --git a/docs/model/third-party-id.rst b/docs/model/third-party-id.rst deleted file mode 100644 index 1f8138ddf7..0000000000 --- a/docs/model/third-party-id.rst +++ /dev/null @@ -1,108 +0,0 @@ -====================== -Third Party Identities -====================== - -A description of how email addresses, mobile phone numbers and other third -party identifiers can be used to authenticate and discover users in Matrix. - - -Overview -======== - -New users need to authenticate their account. An email or SMS text message can -be a convenient form of authentication. Users already have email addresses -and phone numbers for contacts in their address book. They want to communicate -with those contacts in Matrix without manually exchanging a Matrix User ID with -them. - -Third Party IDs ---------------- - -[[TODO(markjh): Describe the format of a 3PID]] - - -Third Party ID Associations ---------------------------- - -An Associaton is a binding between a Matrix User ID and a Third Party ID (3PID). -Each 3PID can be associated with one Matrix User ID at a time. - -[[TODO(markjh): JSON format of the association.]] - -Verification ------------- - -An Assocation must be verified by a trusted Verification Server. Email -addresses and phone numbers can be verified by sending a token to the address -which a client can supply to the verifier to confirm ownership. - -An email Verification Server may be capable of verifying all email 3PIDs or may -be restricted to verifying addresses for a particular domain. A phone number -Verification Server may be capable of verifying all phone numbers or may be -restricted to verifying numbers for a given country or phone prefix. - -Verification Servers fulfil a similar role to Certificate Authorities in PKI so -a similar level of vetting should be required before clients trust their -signatures. - -A Verification Server may wish to check for existing Associations for a 3PID -before creating a new Association. - -Discovery ---------- - -Users can discover Associations using a trusted Identity Server. Each -Association will be signed by the Identity Server. An Identity Server may store -the entire space of Associations or may delegate to other Identity Servers when -looking up Associations. - -Each Association returned from an Identity Server must be signed by a -Verification Server. Clients should check these signatures. - -Identity Servers fulfil a similar role to DNS servers. - -Privacy -------- - -A User may publish the association between their phone number and Matrix User ID -on the Identity Server without publishing the number in their Profile hosted on -their Home Server. - -Identity Servers should refrain from publishing reverse mappings and should -take steps, such as rate limiting, to prevent attackers enumerating the space of -mappings. - -Federation -========== - -Delegation ----------- - -Verification Servers could delegate signing to another server by issuing -certificate to that server allowing it to verify and sign a subset of 3PID on -its behalf. It would be necessary to provide a language for describing which -subset of 3PIDs that server had authority to validate. Alternatively it could -delegate the verification step to another server but sign the resulting -association itself. - -The 3PID space will have a heirachical structure like DNS so Identity Servers -can delegate lookups to other servers. An Identity Server should be prepared -to host or delegate any valid association within the subset of the 3PIDs it is -resonsible for. - -Multiple Root Verification Servers ----------------------------------- - -There can be multiple root Verification Servers and an Association could be -signed by multiple servers if different clients trust different subsets of -the verification servers. - -Multiple Root Identity Servers ------------------------------- - -There can be be multiple root Identity Servers. Clients will add each -Association to all root Identity Servers. - -[[TODO(markjh): Describe how clients find the list of root Identity Servers]] - - -- cgit 1.4.1