deploy: 9f7c038272318bab09535e85e6bb4345ed2f1368

1 files changed, 123 insertions, 46 deletions

diff --git a/latest/print.html b/latest/print.html
index a0e119ad03..f784e0f0f0 100644
--- a/latest/print.html
+++ b/latest/print.html
@@ -101,7 +101,7 @@
 
         <nav id="sidebar" class="sidebar" aria-label="Table of contents">
             <div class="sidebar-scrollbox">
-                <ol class="chapter"><li class="chapter-item expanded affix "><li class="part-title">Introduction</li><li class="chapter-item expanded "><a href="welcome_and_overview.html">Welcome and Overview</a></li><li class="chapter-item expanded affix "><li class="part-title">Setup</li><li class="chapter-item expanded "><a href="setup/installation.html">Installation</a></li><li class="chapter-item expanded "><a href="postgres.html">Using Postgres</a></li><li class="chapter-item expanded "><a href="reverse_proxy.html">Configuring a Reverse Proxy</a></li><li class="chapter-item expanded "><a href="turn-howto.html">Configuring a Turn Server</a></li><li class="chapter-item expanded "><a href="delegate.html">Delegation</a></li><li class="chapter-item expanded affix "><li class="part-title">Upgrading</li><li class="chapter-item expanded "><a href="upgrade.html">Upgrading between Synapse Versions</a></li><li class="chapter-item expanded "><a href="MSC1711_certificates_FAQ.html">Upgrading from pre-Synapse 1.0</a></li><li class="chapter-item expanded affix "><li class="part-title">Usage</li><li class="chapter-item expanded "><a href="federate.html">Federation</a></li><li class="chapter-item expanded "><a href="usage/configuration/index.html">Configuration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/configuration/homeserver_sample_config.html">Homeserver Sample Config File</a></li><li class="chapter-item expanded "><a href="usage/configuration/logging_sample_config.html">Logging Sample Config File</a></li><li class="chapter-item expanded "><a href="structured_logging.html">Structured Logging</a></li><li class="chapter-item expanded "><a href="usage/configuration/user_authentication/index.html">User Authentication</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single-Sign On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="openid.html">OpenID Connect</a></li><li class="chapter-item expanded "><div>SAML</div></li><li class="chapter-item expanded "><div>CAS</div></li><li class="chapter-item expanded "><a href="sso_mapping_providers.html">SSO Mapping Providers</a></li></ol></li><li class="chapter-item expanded "><a href="password_auth_providers.html">Password Auth Providers</a></li><li class="chapter-item expanded "><a href="jwt.html">JSON Web Tokens</a></li></ol></li><li class="chapter-item expanded "><a href="CAPTCHA_SETUP.html">Registration Captcha</a></li><li class="chapter-item expanded "><a href="application_services.html">Application Services</a></li><li class="chapter-item expanded "><a href="server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="consent_tracking.html">Consent Tracking</a></li><li class="chapter-item expanded "><a href="url_previews.html">URL Previews</a></li><li class="chapter-item expanded "><a href="user_directory.html">User Directory</a></li><li class="chapter-item expanded "><a href="message_retention_policies.html">Message Retention Policies</a></li><li class="chapter-item expanded "><a href="modules.html">Pluggable Modules</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Third Party Rules</div></li><li class="chapter-item expanded "><a href="spam_checker.html">Spam Checker</a></li><li class="chapter-item expanded "><a href="presence_router_module.html">Presence Router</a></li><li class="chapter-item expanded "><div>Media Storage Providers</div></li></ol></li><li class="chapter-item expanded "><a href="workers.html">Workers</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="synctl_workers.html">Using synctl with Workers</a></li><li class="chapter-item expanded "><a href="systemd-with-workers/index.html">Systemd</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="usage/administration/index.html">Administration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/administration/admin_api/index.html">Admin API</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="admin_api/account_validity.html">Account Validity</a></li><li class="chapter-item expanded "><a href="admin_api/delete_group.html">Delete Group</a></li><li class="chapter-item expanded "><a href="admin_api/event_reports.html">Event Reports</a></li><li class="chapter-item expanded "><a href="admin_api/media_admin_api.html">Media</a></li><li class="chapter-item expanded "><a href="admin_api/purge_history_api.html">Purge History</a></li><li class="chapter-item expanded "><a href="admin_api/purge_room.html">Purge Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/register_api.html">Register Users</a></li><li class="chapter-item expanded "><a href="admin_api/room_membership.html">Manipulate Room Membership</a></li><li class="chapter-item expanded "><a href="admin_api/rooms.html">Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="admin_api/shutdown_room.html">Shutdown Room</a></li><li class="chapter-item expanded "><a href="admin_api/statistics.html">Statistics</a></li><li class="chapter-item expanded "><a href="admin_api/user_admin_api.html">Users</a></li><li class="chapter-item expanded "><a href="admin_api/version_api.html">Server Version</a></li></ol></li><li class="chapter-item expanded "><a href="manhole.html">Manhole</a></li><li class="chapter-item expanded "><a href="metrics-howto.html">Monitoring</a></li><li class="chapter-item expanded "><a href="usage/administration/request_log.html">Request log format</a></li><li class="chapter-item expanded "><div>Scripts</div></li></ol></li><li class="chapter-item expanded "><li class="part-title">Development</li><li class="chapter-item expanded "><a href="development/contributing_guide.html">Contributing Guide</a></li><li class="chapter-item expanded "><a href="code_style.html">Code Style</a></li><li class="chapter-item expanded "><a href="dev/git.html">Git Usage</a></li><li class="chapter-item expanded "><div>Testing</div></li><li class="chapter-item expanded "><a href="opentracing.html">OpenTracing</a></li><li class="chapter-item expanded "><a href="development/database_schema.html">Database Schemas</a></li><li class="chapter-item expanded "><div>Synapse Architecture</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="log_contexts.html">Log Contexts</a></li><li class="chapter-item expanded "><a href="replication.html">Replication</a></li><li class="chapter-item expanded "><a href="tcp_replication.html">TCP Replication</a></li></ol></li><li class="chapter-item expanded "><a href="development/internal_documentation/index.html">Internal Documentation</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single Sign-On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="dev/saml.html">SAML</a></li><li class="chapter-item expanded "><a href="dev/cas.html">CAS</a></li></ol></li><li class="chapter-item expanded "><div>State Resolution</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="auth_chain_difference_algorithm.html">The Auth Chain Difference Algorithm</a></li></ol></li><li class="chapter-item expanded "><a href="media_repository.html">Media Repository</a></li><li class="chapter-item expanded "><a href="room_and_user_statistics.html">Room and User Statistics</a></li></ol></li><li class="chapter-item expanded "><div>Scripts</div></li><li class="chapter-item expanded affix "><li class="part-title">Other</li><li class="chapter-item expanded "><a href="deprecation_policy.html">Dependency Deprecation Policy</a></li></ol>
+                <ol class="chapter"><li class="chapter-item expanded affix "><li class="part-title">Introduction</li><li class="chapter-item expanded "><a href="welcome_and_overview.html">Welcome and Overview</a></li><li class="chapter-item expanded affix "><li class="part-title">Setup</li><li class="chapter-item expanded "><a href="setup/installation.html">Installation</a></li><li class="chapter-item expanded "><a href="postgres.html">Using Postgres</a></li><li class="chapter-item expanded "><a href="reverse_proxy.html">Configuring a Reverse Proxy</a></li><li class="chapter-item expanded "><a href="turn-howto.html">Configuring a Turn Server</a></li><li class="chapter-item expanded "><a href="delegate.html">Delegation</a></li><li class="chapter-item expanded affix "><li class="part-title">Upgrading</li><li class="chapter-item expanded "><a href="upgrade.html">Upgrading between Synapse Versions</a></li><li class="chapter-item expanded "><a href="MSC1711_certificates_FAQ.html">Upgrading from pre-Synapse 1.0</a></li><li class="chapter-item expanded affix "><li class="part-title">Usage</li><li class="chapter-item expanded "><a href="federate.html">Federation</a></li><li class="chapter-item expanded "><a href="usage/configuration/index.html">Configuration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/configuration/homeserver_sample_config.html">Homeserver Sample Config File</a></li><li class="chapter-item expanded "><a href="usage/configuration/logging_sample_config.html">Logging Sample Config File</a></li><li class="chapter-item expanded "><a href="structured_logging.html">Structured Logging</a></li><li class="chapter-item expanded "><a href="usage/configuration/user_authentication/index.html">User Authentication</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single-Sign On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="openid.html">OpenID Connect</a></li><li class="chapter-item expanded "><div>SAML</div></li><li class="chapter-item expanded "><div>CAS</div></li><li class="chapter-item expanded "><a href="sso_mapping_providers.html">SSO Mapping Providers</a></li></ol></li><li class="chapter-item expanded "><a href="password_auth_providers.html">Password Auth Providers</a></li><li class="chapter-item expanded "><a href="jwt.html">JSON Web Tokens</a></li></ol></li><li class="chapter-item expanded "><a href="CAPTCHA_SETUP.html">Registration Captcha</a></li><li class="chapter-item expanded "><a href="application_services.html">Application Services</a></li><li class="chapter-item expanded "><a href="server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="consent_tracking.html">Consent Tracking</a></li><li class="chapter-item expanded "><a href="url_previews.html">URL Previews</a></li><li class="chapter-item expanded "><a href="user_directory.html">User Directory</a></li><li class="chapter-item expanded "><a href="message_retention_policies.html">Message Retention Policies</a></li><li class="chapter-item expanded "><a href="modules.html">Pluggable Modules</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Third Party Rules</div></li><li class="chapter-item expanded "><a href="spam_checker.html">Spam Checker</a></li><li class="chapter-item expanded "><a href="presence_router_module.html">Presence Router</a></li><li class="chapter-item expanded "><div>Media Storage Providers</div></li></ol></li><li class="chapter-item expanded "><a href="workers.html">Workers</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="synctl_workers.html">Using synctl with Workers</a></li><li class="chapter-item expanded "><a href="systemd-with-workers/index.html">Systemd</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="usage/administration/index.html">Administration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/administration/admin_api/index.html">Admin API</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="admin_api/account_validity.html">Account Validity</a></li><li class="chapter-item expanded "><a href="admin_api/delete_group.html">Delete Group</a></li><li class="chapter-item expanded "><a href="admin_api/event_reports.html">Event Reports</a></li><li class="chapter-item expanded "><a href="admin_api/media_admin_api.html">Media</a></li><li class="chapter-item expanded "><a href="admin_api/purge_history_api.html">Purge History</a></li><li class="chapter-item expanded "><a href="admin_api/purge_room.html">Purge Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/register_api.html">Register Users</a></li><li class="chapter-item expanded "><a href="admin_api/room_membership.html">Manipulate Room Membership</a></li><li class="chapter-item expanded "><a href="admin_api/rooms.html">Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="admin_api/shutdown_room.html">Shutdown Room</a></li><li class="chapter-item expanded "><a href="admin_api/statistics.html">Statistics</a></li><li class="chapter-item expanded "><a href="admin_api/user_admin_api.html">Users</a></li><li class="chapter-item expanded "><a href="admin_api/version_api.html">Server Version</a></li></ol></li><li class="chapter-item expanded "><a href="manhole.html">Manhole</a></li><li class="chapter-item expanded "><a href="metrics-howto.html">Monitoring</a></li><li class="chapter-item expanded "><a href="usage/administration/request_log.html">Request log format</a></li><li class="chapter-item expanded "><div>Scripts</div></li></ol></li><li class="chapter-item expanded "><li class="part-title">Development</li><li class="chapter-item expanded "><a href="development/contributing_guide.html">Contributing Guide</a></li><li class="chapter-item expanded "><a href="code_style.html">Code Style</a></li><li class="chapter-item expanded "><a href="development/git.html">Git Usage</a></li><li class="chapter-item expanded "><div>Testing</div></li><li class="chapter-item expanded "><a href="opentracing.html">OpenTracing</a></li><li class="chapter-item expanded "><a href="development/database_schema.html">Database Schemas</a></li><li class="chapter-item expanded "><div>Synapse Architecture</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="log_contexts.html">Log Contexts</a></li><li class="chapter-item expanded "><a href="replication.html">Replication</a></li><li class="chapter-item expanded "><a href="tcp_replication.html">TCP Replication</a></li></ol></li><li class="chapter-item expanded "><a href="development/internal_documentation/index.html">Internal Documentation</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single Sign-On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="development/saml.html">SAML</a></li><li class="chapter-item expanded "><a href="development/cas.html">CAS</a></li></ol></li><li class="chapter-item expanded "><a href="development/room-dag-concepts.html">Room DAG concepts</a></li><li class="chapter-item expanded "><div>State Resolution</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="auth_chain_difference_algorithm.html">The Auth Chain Difference Algorithm</a></li></ol></li><li class="chapter-item expanded "><a href="media_repository.html">Media Repository</a></li><li class="chapter-item expanded "><a href="room_and_user_statistics.html">Room and User Statistics</a></li></ol></li><li class="chapter-item expanded "><div>Scripts</div></li><li class="chapter-item expanded affix "><li class="part-title">Other</li><li class="chapter-item expanded "><a href="deprecation_policy.html">Dependency Deprecation Policy</a></li></ol>
             </div>
             <div id="sidebar-resize-handle" class="sidebar-resize-handle"></div>
         </nav>
@@ -1504,9 +1504,9 @@ particular server:</p>
 <h1 id="upgrading-to-v1370"><a class="header" href="#upgrading-to-v1370">Upgrading to v1.37.0</a></h1>
 <h2 id="deprecation-of-the-current-spam-checker-interface"><a class="header" href="#deprecation-of-the-current-spam-checker-interface">Deprecation of the current spam checker interface</a></h2>
 <p>The current spam checker interface is deprecated in favour of a new generic modules system.
-Authors of spam checker modules can refer to <a href="https://matrix-org.github.io/synapse/develop/modules.html#porting-an-existing-module-that-uses-the-old-interface">this
+Authors of spam checker modules can refer to <a href="modules.html#porting-an-existing-module-that-uses-the-old-interface">this
 documentation</a>
-to update their modules. Synapse administrators can refer to <a href="https://matrix-org.github.io/synapse/develop/modules.html#using-modules">this
+to update their modules. Synapse administrators can refer to <a href="modules.html#using-modules">this
 documentation</a>
 to update their configuration once the modules they are using have been updated.</p>
 <p>We plan to remove support for the current spam checker interface in August 2021.</p>
@@ -1559,8 +1559,7 @@ to remove excess writeahead logs in order for Prometheus to recover.
 Instructions for doing so are provided
 <a href="https://github.com/matrix-org/synapse/pull/9854#issuecomment-823472183">here</a>.</p>
 <h2 id="dropping-support-for-old-python-postgres-and-sqlite-versions"><a class="header" href="#dropping-support-for-old-python-postgres-and-sqlite-versions">Dropping support for old Python, Postgres and SQLite versions</a></h2>
-<p>In line with our <a href="https://github.com/matrix-org/synapse/blob/release-v1.32.0/docs/deprecation_policy.md">deprecation
-policy</a>,
+<p>In line with our <a href="deprecation_policy.html">deprecation policy</a>,
 we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no
 longer supported upstream.</p>
 <p>This release of Synapse requires Python 3.6+ and PostgresSQL 9.6+ or
@@ -1569,8 +1568,7 @@ SQLite 3.22+.</p>
 <p>The deprecated v1 &quot;list accounts&quot; admin API
 (<code>GET /_synapse/admin/v1/users/&lt;user_id&gt;</code>) has been removed in this
 version.</p>
-<p>The <a href="https://github.com/matrix-org/synapse/blob/master/docs/admin_api/user_admin_api.rst#list-accounts">v2 list accounts
-API</a>
+<p>The <a href="admin_api/user_admin_api.html#list-accounts">v2 list accounts API</a>
 has been available since Synapse 1.7.0 (2019-12-13), and is accessible
 under <code>GET /_synapse/admin/v2/users</code>.</p>
 <p>The deprecation of the old endpoint was announced with Synapse 1.28.0
@@ -1595,7 +1593,7 @@ that the reverse proxy sets [X-Forwarded-Proto]{.title-ref} header to
 [https]{.title-ref} or [http]{.title-ref} to indicate the protocol used
 by the client.</p>
 <p>Synapse also requires the [Host]{.title-ref} header to be preserved.</p>
-<p>See the <a href="../reverse_proxy.html">reverse proxy documentation</a>, where the
+<p>See the <a href="reverse_proxy.html">reverse proxy documentation</a>, where the
 example configurations have been updated to show how to set these
 headers.</p>
 <p>(Users of <a href="https://caddyserver.com/">Caddy</a> are unaffected, since we
@@ -1610,7 +1608,7 @@ identity providers:</p>
 Connect or OAuth2 identity provider, you will need to add
 <code>[synapse public baseurl]/_synapse/client/oidc/callback</code> to the list
 of permitted &quot;redirect URIs&quot; at the identity provider.</p>
-<p>See the <a href="../openid.html">OpenID docs</a> for more information on setting
+<p>See the <a href="openid.html">OpenID docs</a> for more information on setting
 up OpenID Connect.</p>
 </li>
 <li>
@@ -1779,8 +1777,7 @@ lock down external access to the Admin API endpoints.</p>
 <p>This release deprecates use of the <code>structured: true</code> logging
 configuration for structured logging. If your logging configuration
 contains <code>structured: true</code> then it should be modified based on the
-<a href="../structured_logging.html">structured logging
-documentation</a>.</p>
+<a href="structured_logging.html">structured logging documentation</a>.</p>
 <p>The <code>structured</code> and <code>drains</code> logging options are now deprecated and
 should be replaced by standard logging configuration of <code>handlers</code> and
 <code>formatters</code>.</p>
@@ -1801,13 +1798,12 @@ acts the same as the <code>http_client</code> argument previously passed to
 <code>ThirdPartyEventRules</code> modules.</p>
 <h1 id="upgrading-to-v1210"><a class="header" href="#upgrading-to-v1210">Upgrading to v1.21.0</a></h1>
 <h2 id="forwarding-_synapseclient-through-your-reverse-proxy"><a class="header" href="#forwarding-_synapseclient-through-your-reverse-proxy">Forwarding <code>/_synapse/client</code> through your reverse proxy</a></h2>
-<p>The <a href="https://github.com/matrix-org/synapse/blob/develop/docs/reverse_proxy.md">reverse proxy
-documentation</a>
+<p>The <a href="reverse_proxy.html">reverse proxy documentation</a>
 has been updated to include reverse proxy directives for
 <code>/_synapse/client/*</code> endpoints. As the user password reset flow now uses
 endpoints under this prefix, <strong>you must update your reverse proxy
 configurations for user password reset to work</strong>.</p>
-<p>Additionally, note that the <a href="https://github.com/matrix-org/synapse/blob/develop/docs/workers.md">Synapse worker documentation</a> has been updated to</p>
+<p>Additionally, note that the <a href="workers.html">Synapse worker documentation</a> has been updated to</p>
 <p>:   state that the <code>/_synapse/client/password_reset/email/submit_token</code>
 endpoint can be handled</p>
 <p>by all workers. If you make use of Synapse's worker feature, please
@@ -1856,7 +1852,7 @@ updated.</p>
 <p>When setting up worker processes, we now recommend the use of a Redis
 server for replication. <strong>The old direct TCP connection method is
 deprecated and will be removed in a future release.</strong> See
-<a href="../workers.html">workers</a> for more details.</p>
+<a href="workers.html">workers</a> for more details.</p>
 <h1 id="upgrading-to-v1140"><a class="header" href="#upgrading-to-v1140">Upgrading to v1.14.0</a></h1>
 <p>This version includes a database update which is run as part of the
 upgrade, and which may take a couple of minutes in the case of a large
@@ -1969,9 +1965,8 @@ Synapse is running.</p>
 affected. If you <em>are</em> affected, you can run a similar query,
 omitting the <code>CONCURRENTLY</code> keyword. Note however that this
 operation may in itself cause Synapse to stop running for some time.
-Synapse admins are reminded that <a href="https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql">SQLite is not recommended for use
-outside a test
-environment</a>.</p>
+Synapse admins are reminded that <a href="postgres.html">SQLite is not recommended for use
+outside a test environment</a>.</p>
 </li>
 <li>
 <p>Once the index has been created, the <code>SELECT</code> query in step 1 above
@@ -1988,7 +1983,7 @@ HTTP requests, the temporary index can be removed:</p>
 <h1 id="upgrading-to-v1100"><a class="header" href="#upgrading-to-v1100">Upgrading to v1.10.0</a></h1>
 <p>Synapse will now log a warning on start up if used with a PostgreSQL
 database that has a non-recommended locale set.</p>
-<p>See <a href="../postgres.html">Postgres</a> for details.</p>
+<p>See <a href="postgres.html">Postgres</a> for details.</p>
 <h1 id="upgrading-to-v180"><a class="header" href="#upgrading-to-v180">Upgrading to v1.8.0</a></h1>
 <p>Specifying a <code>log_file</code> config option will now cause Synapse to refuse
 to start, and should be replaced by with the <code>log_config</code> option.
@@ -2082,8 +2077,8 @@ section headed <code>email</code>, and be sure to have at least the
 <code>smtp_host, smtp_port</code> and <code>notif_from</code> fields filled out.</p>
 <p>You may also need to set <code>smtp_user</code>, <code>smtp_pass</code>, and
 <code>require_transport_security</code>.</p>
-<p>See the <a href="docs/sample_config.yaml">sample configuration file</a> for more
-details on these settings.</p>
+<p>See the <a href="usage/configuration/homeserver_sample_config.html">sample configuration file</a>
+for more details on these settings.</p>
 <h4 id="delegate-email-to-an-identity-server"><a class="header" href="#delegate-email-to-an-identity-server">Delegate email to an identity server</a></h4>
 <p>Some admins will wish to continue using email verification as part of
 the registration process, but will not immediately have an appropriate
@@ -2163,7 +2158,7 @@ for its use in the room directory!</p>
 </ul>
 <h1 id="upgrading-to-v120"><a class="header" href="#upgrading-to-v120">Upgrading to v1.2.0</a></h1>
 <p>Some counter metrics have been renamed, with the old names deprecated.
-See <a href="../metrics-howto.html#renaming-of-metrics--deprecation-of-old-names-in-12">the metrics
+See <a href="metrics-howto.html#renaming-of-metrics--deprecation-of-old-names-in-12">the metrics
 documentation</a>
 for details.</p>
 <h1 id="upgrading-to-v110"><a class="header" href="#upgrading-to-v110">Upgrading to v1.1.0</a></h1>
@@ -2189,7 +2184,7 @@ more details on upgrading your database.</p>
 <p>Synapse v1.0 is the first release to enforce validation of TLS
 certificates for the federation API. It is therefore essential that your
 certificates are correctly configured. See the
-<a href="../MSC1711_certificates_FAQ.html">FAQ</a> for more information.</p>
+<a href="MSC1711_certificates_FAQ.html">FAQ</a> for more information.</p>
 <p>Note, v1.0 installations will also no longer be able to federate with
 servers that have not correctly configured their certificates.</p>
 <p>In rare cases, it may be desirable to disable certificate checking: for
@@ -2202,8 +2197,8 @@ ways:-</p>
 <li>Configure a whitelist of server domains to trust via
 <code>federation_certificate_verification_whitelist</code>.</li>
 </ul>
-<p>See the <a href="docs/sample_config.yaml">sample configuration file</a> for more
-details on these settings.</p>
+<p>See the <a href="usage/configuration/homeserver_sample_config.html">sample configuration file</a>
+for more details on these settings.</p>
 <h2 id="email-2"><a class="header" href="#email-2">Email</a></h2>
 <p>When a user requests a password reset, Synapse will send an email to the
 user to confirm the request.</p>
@@ -2221,8 +2216,8 @@ section headed <code>email</code>, and be sure to have at least the <code>smtp_h
 <p>If you are absolutely certain that you wish to continue using an
 identity server for password resets, set
 <code>trust_identity_server_for_password_resets</code> to <code>true</code>.</p>
-<p>See the <a href="docs/sample_config.yaml">sample configuration file</a> for more
-details on these settings.</p>
+<p>See the <a href="usage/configuration/homeserver_sample_config.html">sample configuration file</a>
+for more details on these settings.</p>
 <h2 id="new-email-templates"><a class="header" href="#new-email-templates">New email templates</a></h2>
 <p>Some new templates have been added to the default template directory for the purpose of
 the homeserver sending its own password reset emails. If you have configured a
@@ -2237,10 +2232,10 @@ sent to them.</p>
 <h1 id="upgrading-to-v0990"><a class="header" href="#upgrading-to-v0990">Upgrading to v0.99.0</a></h1>
 <p>Please be aware that, before Synapse v1.0 is released around March 2019,
 you will need to replace any self-signed certificates with those
-verified by a root CA. Information on how to do so can be found at <a href="../ACME.html">the
-ACME docs</a>.</p>
+verified by a root CA. Information on how to do so can be found at the
+ACME docs.</p>
 <p>For more information on configuring TLS certificates see the
-<a href="../MSC1711_certificates_FAQ.html">FAQ</a>.</p>
+<a href="MSC1711_certificates_FAQ.html">FAQ</a>.</p>
 <h1 id="upgrading-to-v0340"><a class="header" href="#upgrading-to-v0340">Upgrading to v0.34.0</a></h1>
 <ol>
 <li>
@@ -3590,6 +3585,9 @@ caches:
 # 'name' gives the database engine to use: either 'sqlite3' (for SQLite) or
 # 'psycopg2' (for PostgreSQL).
 #
+# 'txn_limit' gives the maximum number of transactions to run per connection
+# before reconnecting. Defaults to 0, which means no limit.
+#
 # 'args' gives options which are passed through to the database engine,
 # except for options starting 'cp_', which are used to configure the Twisted
 # connection pool. For a reference to valid arguments, see:
@@ -3610,6 +3608,7 @@ caches:
 #
 #database:
 #  name: psycopg2
+#  txn_limit: 10000
 #  args:
 #    user: synapse_user
 #    password: secretpassword
@@ -6239,7 +6238,7 @@ does not return a <code>sub</code> property, an alternative <code>subject_claim<
         localpart_template: &quot;{{ user.preferred_username }}&quot;
         display_name_template: &quot;{{ user.name }}&quot;
 </code></pre>
-<h2 id="apple"><a class="header" href="#apple">Apple</a></h2>
+<h3 id="apple"><a class="header" href="#apple">Apple</a></h3>
 <p>Configuring &quot;Sign in with Apple&quot; (SiWA) requires an Apple Developer account.</p>
 <p>You will need to create a new &quot;Services ID&quot; for SiWA, and create and download a
 private key with &quot;SiWA&quot; enabled.</p>
@@ -9903,7 +9902,8 @@ server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a
             &quot;deactivated&quot;: 0,
             &quot;shadow_banned&quot;: 0,
             &quot;displayname&quot;: &quot;&lt;User One&gt;&quot;,
-            &quot;avatar_url&quot;: null
+            &quot;avatar_url&quot;: null,
+            &quot;creation_ts&quot;: 1560432668000
         }, {
             &quot;name&quot;: &quot;&lt;user_id2&gt;&quot;,
             &quot;is_guest&quot;: 0,
@@ -9912,7 +9912,8 @@ server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a
             &quot;deactivated&quot;: 0,
             &quot;shadow_banned&quot;: 0,
             &quot;displayname&quot;: &quot;&lt;User Two&gt;&quot;,
-            &quot;avatar_url&quot;: &quot;&lt;avatar_url&gt;&quot;
+            &quot;avatar_url&quot;: &quot;&lt;avatar_url&gt;&quot;,
+            &quot;creation_ts&quot;: 1561550621000
         }
     ],
     &quot;next_token&quot;: &quot;100&quot;,
@@ -9965,6 +9966,7 @@ which guarantees a stable ordering. Valid values are:</p>
 <li><code>shadow_banned</code> - Users are ordered by <code>shadow_banned</code> status.</li>
 <li><code>displayname</code> - Users are ordered alphabetically by <code>displayname</code>.</li>
 <li><code>avatar_url</code> - Users are ordered alphabetically by avatar URL.</li>
+<li><code>creation_ts</code> - Users are ordered by when the users was created in ms.</li>
 </ul>
 </li>
 <li>
@@ -9972,7 +9974,7 @@ which guarantees a stable ordering. Valid values are:</p>
 Setting this value to <code>b</code> will reverse the above sort order. Defaults to <code>f</code>.</p>
 </li>
 </ul>
-<p>Caution. The database only has indexes on the columns <code>name</code> and <code>created_ts</code>.
+<p>Caution. The database only has indexes on the columns <code>name</code> and <code>creation_ts</code>.
 This means that if a different sort order is used (<code>is_guest</code>, <code>admin</code>,
 <code>user_type</code>, <code>deactivated</code>, <code>shadow_banned</code>, <code>avatar_url</code> or <code>displayname</code>),
 this can cause a large load on the database, especially for large environments.</p>
@@ -9992,6 +9994,7 @@ This allows user type specific behaviour. There are also types <code>support</co
 <li><code>shadow_banned</code> - bool - Status if that user has been marked as shadow banned.</li>
 <li><code>displayname</code> - string - The user's display name if they have set one.</li>
 <li><code>avatar_url</code> - string -  The user's avatar URL if they have set one.</li>
+<li><code>creation_ts</code> - integer - The user's creation timestamp in ms.</li>
 </ul>
 </li>
 <li>
@@ -11109,7 +11112,7 @@ Make sure that you have saved all your files.</p>
 <pre><code class="language-sh">source ./env/bin/activate
 ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
 </code></pre>
-<h2 id="run-the-unit-tests"><a class="header" href="#run-the-unit-tests">Run the unit tests.</a></h2>
+<h2 id="run-the-unit-tests-twisted-trial"><a class="header" href="#run-the-unit-tests-twisted-trial">Run the unit tests (Twisted trial).</a></h2>
 <p>The unit tests run parts of Synapse, including your changes, to see if anything
 was broken. They are slower than the linters but will typically catch more errors.</p>
 <pre><code class="language-sh">source ./env/bin/activate
@@ -11126,7 +11129,7 @@ trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_i
 <p>To increase the log level for the tests, set <code>SYNAPSE_TEST_LOG_LEVEL</code>:</p>
 <pre><code class="language-sh">SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests
 </code></pre>
-<h2 id="run-the-integration-tests"><a class="header" href="#run-the-integration-tests">Run the integration tests.</a></h2>
+<h2 id="run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgsytestsytesta"><a class="header" href="#run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgsytestsytesta">Run the integration tests (<a href="https://github.com/matrix-org/sytest">Sytest</a>).</a></h2>
 <p>The integration tests are a more comprehensive suite of tests. They
 run a full version of Synapse, including your changes, to check if
 anything was broken. They are slower than the unit tests but will
@@ -11136,6 +11139,29 @@ configuration:</p>
 <pre><code class="language-sh">$ docker run --rm -it -v /path/where/you/have/cloned/the/repository\:/src:ro -v /path/to/where/you/want/logs\:/logs matrixdotorg/sytest-synapse:py37
 </code></pre>
 <p>This configuration should generally cover  your needs. For more details about other configurations, see <a href="https://github.com/matrix-org/sytest/blob/develop/docker/README.md">documentation in the SyTest repo</a>.</p>
+<h2 id="run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgcomplementcomplementa"><a class="header" href="#run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgcomplementcomplementa">Run the integration tests (<a href="https://github.com/matrix-org/complement">Complement</a>).</a></h2>
+<p><a href="https://github.com/matrix-org/complement">Complement</a> is a suite of black box tests that can be run on any homeserver implementation. It can also be thought of as end-to-end (e2e) tests.</p>
+<p>It's often nice to develop on Synapse and write Complement tests at the same time.
+Here is how to run your local Synapse checkout against your local Complement checkout.</p>
+<p>(checkout <a href="https://github.com/matrix-org/complement"><code>complement</code></a> alongside your <code>synapse</code> checkout)</p>
+<pre><code class="language-sh">COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh
+</code></pre>
+<p>To run a specific test file, you can pass the test name at the end of the command. The name passed comes from the naming structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output:</p>
+<pre><code class="language-sh">COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory
+</code></pre>
+<p>To run a specific test, you can specify the whole name structure:</p>
+<pre><code class="language-sh">COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/Backfilled_historical_events_resolve_with_proper_state_in_correct_order
+</code></pre>
+<h3 id="access-database-for-homeserver-after-complement-test-runs"><a class="header" href="#access-database-for-homeserver-after-complement-test-runs">Access database for homeserver after Complement test runs.</a></h3>
+<p>If you're curious what the database looks like after you run some tests, here are some steps to get you going in Synapse:</p>
+<ol>
+<li>In your Complement test comment out <code>defer deployment.Destroy(t)</code> and replace with <code>defer time.Sleep(2 * time.Hour)</code> to keep the homeserver running after the tests complete</li>
+<li>Start the Complement tests</li>
+<li>Find the name of the container, <code>docker ps -f name=complement_</code> (this will filter for just the Compelement related Docker containers)</li>
+<li>Access the container replacing the name with what you found in the previous step: <code>docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash</code></li>
+<li>Install sqlite (database driver), <code>apt-get update &amp;&amp; apt-get install -y sqlite3</code></li>
+<li>Then run <code>sqlite3</code> and open the database <code>.open /conf/homeserver.db</code> (this db path comes from the Synapse homeserver.yaml)</li>
+</ol>
 <h1 id="9-submit-your-patch"><a class="header" href="#9-submit-your-patch">9. Submit your patch.</a></h1>
 <p>Once you're happy with your patch, it's time to prepare a Pull Request.</p>
 <p>To prepare a Pull Request, please:</p>
@@ -11298,7 +11324,7 @@ flag to <code>git commit</code>, which uses the name and email set in your
 <p>By now, you know the drill!</p>
 <h1 id="notes-for-maintainers-on-merging-prs-etc"><a class="header" href="#notes-for-maintainers-on-merging-prs-etc">Notes for maintainers on merging PRs etc</a></h1>
 <p>There are some notes for those with commit access to the project on how we
-manage git <a href="development/docs/dev/git.html">here</a>.</p>
+manage git <a href="development/docs/development/git.html">here</a>.</p>
 <h1 id="conclusion"><a class="header" href="#conclusion">Conclusion</a></h1>
 <p>That's it! Matrix is a very open and collaborative project as you might expect
 given our obsession with open communication. If we're going to successfully
@@ -11489,7 +11515,7 @@ is left as an exercise for the reader!</p>
 <p>In an ideal world, our git commit history would be a linear progression of
 commits each of which contains a single change building on what came
 before. Here, by way of an arbitrary example, is the top of <code>git log --graph b2dba0607</code>:</p>
-<img src="dev/git/clean.png" alt="clean git graph" width="500px">
+<img src="development/img/git/clean.png" alt="clean git graph" width="500px">
 <p>Note how the commit comment explains clearly what is changing and why. Also
 note the <em>absence</em> of merge commits, as well as the absence of commits called
 things like (to pick a few culprits):
@@ -11537,10 +11563,10 @@ towards.</p>
 <p>Ok, so that's what we'd like to achieve. How do we achieve it?</p>
 <p>The TL;DR is: when you come to merge a pull request, you <em>probably</em> want to
 “squash and merge”:</p>
-<p><img src="dev/git/squash.png" alt="squash and merge" />.</p>
+<p><img src="development/img/git/squash.png" alt="squash and merge" />.</p>
 <p>(This applies whether you are merging your own PR, or that of another
 contributor.)</p>
-<p>“Squash and merge”<sup id="a1"><a href="dev/git.html#f1">1</a></sup> takes all of the changes in the
+<p>“Squash and merge”<sup id="a1"><a href="development/git.html#f1">1</a></sup> takes all of the changes in the
 PR, and bundles them into a single commit. GitHub gives you the opportunity to
 edit the commit message before you confirm, and normally you should do so,
 because the default will be useless (again: <code>* woops typo</code> is not a useful
@@ -11571,10 +11597,10 @@ lot</a>). I tend to
 think the whole thing is overblown. Fundamentally, it's not that
 complicated. Here's how we do it.</p>
 <p>Let's start with a picture:</p>
-<p><img src="dev/git/branches.jpg" alt="branching model" /></p>
+<p><img src="development/img/git/branches.jpg" alt="branching model" /></p>
 <p>It looks complicated, but it's really not. There's one basic rule: <em>anyone</em> is
 free to merge from <em>any</em> more-stable branch to <em>any</em> less-stable branch at
-<em>any</em> time<sup id="a2"><a href="dev/git.html#f2">2</a></sup>. (The principle behind this is that if a
+<em>any</em> time<sup id="a2"><a href="development/git.html#f2">2</a></sup>. (The principle behind this is that if a
 change is good enough for the more-stable branch, then it's also good enough go
 put in a less-stable branch.)</p>
 <p>Meanwhile, merging (or squashing, as per the above) from a less-stable to a
@@ -11586,7 +11612,7 @@ that our active branches are ordered thus, from more-stable to less-stable:</p>
 <ul>
 <li><code>master</code> (tracks our last release).</li>
 <li><code>release-vX.Y</code> (the branch where we prepare the next release)<sup
-   id="a3"><a href="dev/git.html#f3">3</a></sup>.</li>
+   id="a3"><a href="development/git.html#f3">3</a></sup>.</li>
 <li>PR branches which are targeting the release.</li>
 <li><code>develop</code> (our &quot;mainline&quot; branch containing our bleeding-edge).</li>
 <li>regular PR branches.</li>
@@ -11600,11 +11626,11 @@ branch first helps reduce the chance of annoying conflicts.)</p>
 <hr />
 <p><b id="f1">[1]</b>: “Squash and merge” is GitHub's term for this
 operation. Given that there is no merge involved, I'm not convinced it's the
-most intuitive name. <a href="dev/git.html#a1">^</a></p>
-<p><b id="f2">[2]</b>: Well, anyone with commit access.<a href="dev/git.html#a2">^</a></p>
+most intuitive name. <a href="development/git.html#a1">^</a></p>
+<p><b id="f2">[2]</b>: Well, anyone with commit access.<a href="development/git.html#a2">^</a></p>
 <p><b id="f3">[3]</b>: Very, very occasionally (I think this has happened once in
 the history of Synapse), we've had two releases in flight at once. Obviously,
-<code>release-v1.2</code> is more-stable than <code>release-v1.3</code>. <a href="dev/git.html#a3">^</a></p>
+<code>release-v1.2</code> is more-stable than <code>release-v1.3</code>. <a href="development/git.html#a3">^</a></p>
 <div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="opentracing"><a class="header" href="#opentracing">OpenTracing</a></h1>
 <h2 id="background"><a class="header" href="#background">Background</a></h2>
 <p>OpenTracing is a semi-standard being adopted by a number of distributed
@@ -12428,6 +12454,57 @@ and that the CAS server is on port 8000, both on localhost.</p>
 <li>http://localhost:8000/admin/</li>
 <li>Click &quot;logout&quot; in the top right.</li>
 </ol>
+<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="room-dag-concepts"><a class="header" href="#room-dag-concepts">Room DAG concepts</a></h1>
+<h2 id="edges"><a class="header" href="#edges">Edges</a></h2>
+<p>The word &quot;edge&quot; comes from graph theory lingo. An edge is just a connection
+between two events. In Synapse, we connect events by specifying their
+<code>prev_events</code>. A subsequent event points back at a previous event.</p>
+<pre><code>A (oldest) &lt;---- B &lt;---- C (most recent)
+</code></pre>
+<h2 id="depth-and-stream-ordering"><a class="header" href="#depth-and-stream-ordering">Depth and stream ordering</a></h2>
+<p>Events are normally sorted by <code>(topological_ordering, stream_ordering)</code> where
+<code>topological_ordering</code> is just <code>depth</code>. In other words, we first sort by <code>depth</code>
+and then tie-break based on <code>stream_ordering</code>. <code>depth</code> is incremented as new
+messages are added to the DAG. Normally, <code>stream_ordering</code> is an auto
+incrementing integer, but backfilled events start with <code>stream_ordering=-1</code> and decrement.</p>
+<hr />
+<ul>
+<li><code>/sync</code> returns things in the order they arrive at the server (<code>stream_ordering</code>).</li>
+<li><code>/messages</code> (and <code>/backfill</code> in the federation API) return them in the order determined by the event graph <code>(topological_ordering, stream_ordering)</code>.</li>
+</ul>
+<p>The general idea is that, if you're following a room in real-time (i.e.
+<code>/sync</code>), you probably want to see the messages as they arrive at your server,
+rather than skipping any that arrived late; whereas if you're looking at a
+historical section of timeline (i.e. <code>/messages</code>), you want to see the best
+representation of the state of the room as others were seeing it at the time.</p>
+<h2 id="forward-extremity"><a class="header" href="#forward-extremity">Forward extremity</a></h2>
+<p>Most-recent-in-time events in the DAG which are not referenced by any other events' <code>prev_events</code> yet.</p>
+<p>The forward extremities of a room are used as the <code>prev_events</code> when the next event is sent.</p>
+<h2 id="backwards-extremity"><a class="header" href="#backwards-extremity">Backwards extremity</a></h2>
+<p>The current marker of where we have backfilled up to and will generally be the
+oldest-in-time events we know of in the DAG.</p>
+<p>This is an event where we haven't fetched all of the <code>prev_events</code> for.</p>
+<p>Once we have fetched all of its <code>prev_events</code>, it's unmarked as a backwards
+extremity (although we may have formed new backwards extremities from the prev
+events during the backfilling process).</p>
+<h2 id="outliers"><a class="header" href="#outliers">Outliers</a></h2>
+<p>We mark an event as an <code>outlier</code> when we haven't figured out the state for the
+room at that point in the DAG yet.</p>
+<p>We won't <em>necessarily</em> have the <code>prev_events</code> of an <code>outlier</code> in the database,
+but it's entirely possible that we <em>might</em>. The status of whether we have all of
+the <code>prev_events</code> is marked as a <a href="development/room-dag-concepts.html#backwards-extremity">backwards extremity</a>.</p>
+<p>For example, when we fetch the event auth chain or state for a given event, we
+mark all of those claimed auth events as outliers because we haven't done the
+state calculation ourself.</p>
+<h2 id="state-groups"><a class="header" href="#state-groups">State groups</a></h2>
+<p>For every non-outlier event we need to know the state at that event. Instead of
+storing the full state for each event in the DB (i.e. a <code>event_id -&gt; state</code>
+mapping), which is <em>very</em> space inefficient when state doesn't change, we
+instead assign each different set of state a &quot;state group&quot; and then have
+mappings of <code>event_id -&gt; state_group</code> and <code>state_group -&gt; state</code>.</p>
+<h3 id="stage-group-edges"><a class="header" href="#stage-group-edges">Stage group edges</a></h3>
+<p>TODO: <code>state_group_edges</code> is a further optimization...
+notes from @Azrenbeth, https://pastebin.com/seUGVGeT</p>
 <div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="auth-chain-difference-algorithm"><a class="header" href="#auth-chain-difference-algorithm">Auth Chain Difference Algorithm</a></h1>
 <p>The auth chain difference algorithm is used by V2 state resolution, where a
 naive implementation can be a significant source of CPU and DB usage.</p>