diff --git a/latest/print.html b/latest/print.html
index f784e0f0f0..53931652ab 100644
--- a/latest/print.html
+++ b/latest/print.html
@@ -101,7 +101,7 @@
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<div class="sidebar-scrollbox">
- <ol class="chapter"><li class="chapter-item expanded affix "><li class="part-title">Introduction</li><li class="chapter-item expanded "><a href="welcome_and_overview.html">Welcome and Overview</a></li><li class="chapter-item expanded affix "><li class="part-title">Setup</li><li class="chapter-item expanded "><a href="setup/installation.html">Installation</a></li><li class="chapter-item expanded "><a href="postgres.html">Using Postgres</a></li><li class="chapter-item expanded "><a href="reverse_proxy.html">Configuring a Reverse Proxy</a></li><li class="chapter-item expanded "><a href="turn-howto.html">Configuring a Turn Server</a></li><li class="chapter-item expanded "><a href="delegate.html">Delegation</a></li><li class="chapter-item expanded affix "><li class="part-title">Upgrading</li><li class="chapter-item expanded "><a href="upgrade.html">Upgrading between Synapse Versions</a></li><li class="chapter-item expanded "><a href="MSC1711_certificates_FAQ.html">Upgrading from pre-Synapse 1.0</a></li><li class="chapter-item expanded affix "><li class="part-title">Usage</li><li class="chapter-item expanded "><a href="federate.html">Federation</a></li><li class="chapter-item expanded "><a href="usage/configuration/index.html">Configuration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/configuration/homeserver_sample_config.html">Homeserver Sample Config File</a></li><li class="chapter-item expanded "><a href="usage/configuration/logging_sample_config.html">Logging Sample Config File</a></li><li class="chapter-item expanded "><a href="structured_logging.html">Structured Logging</a></li><li class="chapter-item expanded "><a href="usage/configuration/user_authentication/index.html">User Authentication</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single-Sign On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="openid.html">OpenID Connect</a></li><li class="chapter-item expanded "><div>SAML</div></li><li class="chapter-item expanded "><div>CAS</div></li><li class="chapter-item expanded "><a href="sso_mapping_providers.html">SSO Mapping Providers</a></li></ol></li><li class="chapter-item expanded "><a href="password_auth_providers.html">Password Auth Providers</a></li><li class="chapter-item expanded "><a href="jwt.html">JSON Web Tokens</a></li></ol></li><li class="chapter-item expanded "><a href="CAPTCHA_SETUP.html">Registration Captcha</a></li><li class="chapter-item expanded "><a href="application_services.html">Application Services</a></li><li class="chapter-item expanded "><a href="server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="consent_tracking.html">Consent Tracking</a></li><li class="chapter-item expanded "><a href="url_previews.html">URL Previews</a></li><li class="chapter-item expanded "><a href="user_directory.html">User Directory</a></li><li class="chapter-item expanded "><a href="message_retention_policies.html">Message Retention Policies</a></li><li class="chapter-item expanded "><a href="modules.html">Pluggable Modules</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Third Party Rules</div></li><li class="chapter-item expanded "><a href="spam_checker.html">Spam Checker</a></li><li class="chapter-item expanded "><a href="presence_router_module.html">Presence Router</a></li><li class="chapter-item expanded "><div>Media Storage Providers</div></li></ol></li><li class="chapter-item expanded "><a href="workers.html">Workers</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="synctl_workers.html">Using synctl with Workers</a></li><li class="chapter-item expanded "><a href="systemd-with-workers/index.html">Systemd</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="usage/administration/index.html">Administration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/administration/admin_api/index.html">Admin API</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="admin_api/account_validity.html">Account Validity</a></li><li class="chapter-item expanded "><a href="admin_api/delete_group.html">Delete Group</a></li><li class="chapter-item expanded "><a href="admin_api/event_reports.html">Event Reports</a></li><li class="chapter-item expanded "><a href="admin_api/media_admin_api.html">Media</a></li><li class="chapter-item expanded "><a href="admin_api/purge_history_api.html">Purge History</a></li><li class="chapter-item expanded "><a href="admin_api/purge_room.html">Purge Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/register_api.html">Register Users</a></li><li class="chapter-item expanded "><a href="admin_api/room_membership.html">Manipulate Room Membership</a></li><li class="chapter-item expanded "><a href="admin_api/rooms.html">Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="admin_api/shutdown_room.html">Shutdown Room</a></li><li class="chapter-item expanded "><a href="admin_api/statistics.html">Statistics</a></li><li class="chapter-item expanded "><a href="admin_api/user_admin_api.html">Users</a></li><li class="chapter-item expanded "><a href="admin_api/version_api.html">Server Version</a></li></ol></li><li class="chapter-item expanded "><a href="manhole.html">Manhole</a></li><li class="chapter-item expanded "><a href="metrics-howto.html">Monitoring</a></li><li class="chapter-item expanded "><a href="usage/administration/request_log.html">Request log format</a></li><li class="chapter-item expanded "><div>Scripts</div></li></ol></li><li class="chapter-item expanded "><li class="part-title">Development</li><li class="chapter-item expanded "><a href="development/contributing_guide.html">Contributing Guide</a></li><li class="chapter-item expanded "><a href="code_style.html">Code Style</a></li><li class="chapter-item expanded "><a href="development/git.html">Git Usage</a></li><li class="chapter-item expanded "><div>Testing</div></li><li class="chapter-item expanded "><a href="opentracing.html">OpenTracing</a></li><li class="chapter-item expanded "><a href="development/database_schema.html">Database Schemas</a></li><li class="chapter-item expanded "><div>Synapse Architecture</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="log_contexts.html">Log Contexts</a></li><li class="chapter-item expanded "><a href="replication.html">Replication</a></li><li class="chapter-item expanded "><a href="tcp_replication.html">TCP Replication</a></li></ol></li><li class="chapter-item expanded "><a href="development/internal_documentation/index.html">Internal Documentation</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single Sign-On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="development/saml.html">SAML</a></li><li class="chapter-item expanded "><a href="development/cas.html">CAS</a></li></ol></li><li class="chapter-item expanded "><a href="development/room-dag-concepts.html">Room DAG concepts</a></li><li class="chapter-item expanded "><div>State Resolution</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="auth_chain_difference_algorithm.html">The Auth Chain Difference Algorithm</a></li></ol></li><li class="chapter-item expanded "><a href="media_repository.html">Media Repository</a></li><li class="chapter-item expanded "><a href="room_and_user_statistics.html">Room and User Statistics</a></li></ol></li><li class="chapter-item expanded "><div>Scripts</div></li><li class="chapter-item expanded affix "><li class="part-title">Other</li><li class="chapter-item expanded "><a href="deprecation_policy.html">Dependency Deprecation Policy</a></li></ol>
+ <ol class="chapter"><li class="chapter-item expanded affix "><li class="part-title">Introduction</li><li class="chapter-item expanded "><a href="welcome_and_overview.html">Welcome and Overview</a></li><li class="chapter-item expanded affix "><li class="part-title">Setup</li><li class="chapter-item expanded "><a href="setup/installation.html">Installation</a></li><li class="chapter-item expanded "><a href="postgres.html">Using Postgres</a></li><li class="chapter-item expanded "><a href="reverse_proxy.html">Configuring a Reverse Proxy</a></li><li class="chapter-item expanded "><a href="setup/forward_proxy.html">Configuring a Forward/Outbound Proxy</a></li><li class="chapter-item expanded "><a href="turn-howto.html">Configuring a Turn Server</a></li><li class="chapter-item expanded "><a href="delegate.html">Delegation</a></li><li class="chapter-item expanded affix "><li class="part-title">Upgrading</li><li class="chapter-item expanded "><a href="upgrade.html">Upgrading between Synapse Versions</a></li><li class="chapter-item expanded "><a href="MSC1711_certificates_FAQ.html">Upgrading from pre-Synapse 1.0</a></li><li class="chapter-item expanded affix "><li class="part-title">Usage</li><li class="chapter-item expanded "><a href="federate.html">Federation</a></li><li class="chapter-item expanded "><a href="usage/configuration/index.html">Configuration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/configuration/homeserver_sample_config.html">Homeserver Sample Config File</a></li><li class="chapter-item expanded "><a href="usage/configuration/logging_sample_config.html">Logging Sample Config File</a></li><li class="chapter-item expanded "><a href="structured_logging.html">Structured Logging</a></li><li class="chapter-item expanded "><a href="templates.html">Templates</a></li><li class="chapter-item expanded "><a href="usage/configuration/user_authentication/index.html">User Authentication</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single-Sign On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="openid.html">OpenID Connect</a></li><li class="chapter-item expanded "><div>SAML</div></li><li class="chapter-item expanded "><div>CAS</div></li><li class="chapter-item expanded "><a href="sso_mapping_providers.html">SSO Mapping Providers</a></li></ol></li><li class="chapter-item expanded "><a href="password_auth_providers.html">Password Auth Providers</a></li><li class="chapter-item expanded "><a href="jwt.html">JSON Web Tokens</a></li></ol></li><li class="chapter-item expanded "><a href="CAPTCHA_SETUP.html">Registration Captcha</a></li><li class="chapter-item expanded "><a href="application_services.html">Application Services</a></li><li class="chapter-item expanded "><a href="server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="consent_tracking.html">Consent Tracking</a></li><li class="chapter-item expanded "><a href="url_previews.html">URL Previews</a></li><li class="chapter-item expanded "><a href="user_directory.html">User Directory</a></li><li class="chapter-item expanded "><a href="message_retention_policies.html">Message Retention Policies</a></li><li class="chapter-item expanded "><a href="modules.html">Pluggable Modules</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Third Party Rules</div></li><li class="chapter-item expanded "><a href="spam_checker.html">Spam Checker</a></li><li class="chapter-item expanded "><a href="presence_router_module.html">Presence Router</a></li><li class="chapter-item expanded "><div>Media Storage Providers</div></li></ol></li><li class="chapter-item expanded "><a href="workers.html">Workers</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="synctl_workers.html">Using synctl with Workers</a></li><li class="chapter-item expanded "><a href="systemd-with-workers/index.html">Systemd</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="usage/administration/index.html">Administration</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="usage/administration/admin_api/index.html">Admin API</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="admin_api/account_validity.html">Account Validity</a></li><li class="chapter-item expanded "><a href="admin_api/delete_group.html">Delete Group</a></li><li class="chapter-item expanded "><a href="admin_api/event_reports.html">Event Reports</a></li><li class="chapter-item expanded "><a href="admin_api/media_admin_api.html">Media</a></li><li class="chapter-item expanded "><a href="admin_api/purge_history_api.html">Purge History</a></li><li class="chapter-item expanded "><a href="admin_api/purge_room.html">Purge Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/register_api.html">Register Users</a></li><li class="chapter-item expanded "><a href="admin_api/room_membership.html">Manipulate Room Membership</a></li><li class="chapter-item expanded "><a href="admin_api/rooms.html">Rooms</a></li><li class="chapter-item expanded "><a href="admin_api/server_notices.html">Server Notices</a></li><li class="chapter-item expanded "><a href="admin_api/shutdown_room.html">Shutdown Room</a></li><li class="chapter-item expanded "><a href="admin_api/statistics.html">Statistics</a></li><li class="chapter-item expanded "><a href="admin_api/user_admin_api.html">Users</a></li><li class="chapter-item expanded "><a href="admin_api/version_api.html">Server Version</a></li></ol></li><li class="chapter-item expanded "><a href="manhole.html">Manhole</a></li><li class="chapter-item expanded "><a href="metrics-howto.html">Monitoring</a></li><li class="chapter-item expanded "><a href="usage/administration/request_log.html">Request log format</a></li><li class="chapter-item expanded "><div>Scripts</div></li></ol></li><li class="chapter-item expanded "><li class="part-title">Development</li><li class="chapter-item expanded "><a href="development/contributing_guide.html">Contributing Guide</a></li><li class="chapter-item expanded "><a href="code_style.html">Code Style</a></li><li class="chapter-item expanded "><a href="development/git.html">Git Usage</a></li><li class="chapter-item expanded "><div>Testing</div></li><li class="chapter-item expanded "><a href="opentracing.html">OpenTracing</a></li><li class="chapter-item expanded "><a href="development/database_schema.html">Database Schemas</a></li><li class="chapter-item expanded "><div>Synapse Architecture</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="log_contexts.html">Log Contexts</a></li><li class="chapter-item expanded "><a href="replication.html">Replication</a></li><li class="chapter-item expanded "><a href="tcp_replication.html">TCP Replication</a></li></ol></li><li class="chapter-item expanded "><a href="development/internal_documentation/index.html">Internal Documentation</a></li><li><ol class="section"><li class="chapter-item expanded "><div>Single Sign-On</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="development/saml.html">SAML</a></li><li class="chapter-item expanded "><a href="development/cas.html">CAS</a></li></ol></li><li class="chapter-item expanded "><a href="development/room-dag-concepts.html">Room DAG concepts</a></li><li class="chapter-item expanded "><div>State Resolution</div></li><li><ol class="section"><li class="chapter-item expanded "><a href="auth_chain_difference_algorithm.html">The Auth Chain Difference Algorithm</a></li></ol></li><li class="chapter-item expanded "><a href="media_repository.html">Media Repository</a></li><li class="chapter-item expanded "><a href="room_and_user_statistics.html">Room and User Statistics</a></li></ol></li><li class="chapter-item expanded "><div>Scripts</div></li><li class="chapter-item expanded affix "><li class="part-title">Other</li><li class="chapter-item expanded "><a href="deprecation_policy.html">Dependency Deprecation Policy</a></li></ol>
</div>
<div id="sidebar-resize-handle" class="sidebar-resize-handle"></div>
</nav>
@@ -864,6 +864,14 @@ Beware that Apache <em>will</em> canonicalise URIs unless you specify
<code>https://matrix.example.com</code>, and other servers to connect at
<code>https://example.com:8448</code>. The following sections detail the configuration of
the reverse proxy and the homeserver.</p>
+<h2 id="homeserver-configuration"><a class="header" href="#homeserver-configuration">Homeserver Configuration</a></h2>
+<p>The HTTP configuration will need to be updated for Synapse to correctly record
+client IP addresses and generate redirect URLs while behind a reverse proxy. </p>
+<p>In <code>homeserver.yaml</code> set <code>x_forwarded: true</code> in the port 8008 section and
+consider setting <code>bind_addresses: ['127.0.0.1']</code> so that the server only
+listens to traffic on localhost. (Do not change <code>bind_addresses</code> to <code>127.0.0.1</code>
+when using a containerized Synapse, as that will prevent it from responding
+to proxied traffic.)</p>
<h2 id="reverse-proxy-configuration-examples"><a class="header" href="#reverse-proxy-configuration-examples">Reverse-proxy configuration examples</a></h2>
<p><strong>NOTE</strong>: You only need one of these.</p>
<h3 id="nginx"><a class="header" href="#nginx">nginx</a></h3>
@@ -1043,13 +1051,6 @@ relay "matrix_federation" {
forward to <matrixserver> port 8008 check tcp
}
</code></pre>
-<h2 id="homeserver-configuration"><a class="header" href="#homeserver-configuration">Homeserver Configuration</a></h2>
-<p>You will also want to set <code>bind_addresses: ['127.0.0.1']</code> and
-<code>x_forwarded: true</code> for port 8008 in <code>homeserver.yaml</code> to ensure that
-client IP addresses are recorded correctly.</p>
-<p>Having done so, you can then use <code>https://matrix.example.com</code> (instead
-of <code>https://matrix.example.com:8448</code>) as the "Custom server" when
-connecting to Synapse from a client.</p>
<h2 id="health-check-endpoint"><a class="header" href="#health-check-endpoint">Health check endpoint</a></h2>
<p>Synapse exposes a health check endpoint for use by reverse proxies.
Each configured HTTP listener has a <code>/health</code> endpoint which always returns
@@ -1059,6 +1060,81 @@ Each configured HTTP listener has a <code>/health</code> endpoint which always r
<code>/_synapse/admin</code>. These require authentication through an access token of an
admin user. However as access to these endpoints grants the caller a lot of power,
we do not recommend exposing them to the public internet without good reason.</p>
+<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="using-a-forward-proxy-with-synapse"><a class="header" href="#using-a-forward-proxy-with-synapse">Using a forward proxy with Synapse</a></h1>
+<p>You can use Synapse with a forward or outbound proxy. An example of when
+this is necessary is in corporate environments behind a DMZ (demilitarized zone).
+Synapse supports routing outbound HTTP(S) requests via a proxy. Only HTTP(S)
+proxy is supported, not SOCKS proxy or anything else.</p>
+<h2 id="configure"><a class="header" href="#configure">Configure</a></h2>
+<p>The <code>http_proxy</code>, <code>https_proxy</code>, <code>no_proxy</code> environment variables are used to
+specify proxy settings. The environment variable is not case sensitive.</p>
+<ul>
+<li><code>http_proxy</code>: Proxy server to use for HTTP requests.</li>
+<li><code>https_proxy</code>: Proxy server to use for HTTPS requests.</li>
+<li><code>no_proxy</code>: Comma-separated list of hosts, IP addresses, or IP ranges in CIDR
+format which should not use the proxy. Synapse will directly connect to these hosts.</li>
+</ul>
+<p>The <code>http_proxy</code> and <code>https_proxy</code> environment variables have the form: <code>[scheme://][<username>:<password>@]<host>[:<port>]</code></p>
+<ul>
+<li>
+<p>Supported schemes are <code>http://</code> and <code>https://</code>. The default scheme is <code>http://</code>
+for compatibility reasons; it is recommended to set a scheme. If scheme is set
+to <code>https://</code> the connection uses TLS between Synapse and the proxy.</p>
+<p><strong>NOTE</strong>: Synapse validates the certificates. If the certificate is not
+valid, then the connection is dropped.</p>
+</li>
+<li>
+<p>Default port if not given is <code>1080</code>.</p>
+</li>
+<li>
+<p>Username and password are optional and will be used to authenticate against
+the proxy.</p>
+</li>
+</ul>
+<p><strong>Examples</strong></p>
+<ul>
+<li>HTTP_PROXY=http://USERNAME:PASSWORD@10.0.1.1:8080/</li>
+<li>HTTPS_PROXY=http://USERNAME:PASSWORD@proxy.example.com:8080/</li>
+<li>NO_PROXY=master.hostname.example.com,10.1.0.0/16,172.30.0.0/16</li>
+</ul>
+<p><strong>NOTE</strong>:
+Synapse does not apply the IP blacklist to connections through the proxy (since
+the DNS resolution is done by the proxy). It is expected that the proxy or firewall
+will apply blacklisting of IP addresses.</p>
+<h2 id="connection-types"><a class="header" href="#connection-types">Connection types</a></h2>
+<p>The proxy will be <strong>used</strong> for:</p>
+<ul>
+<li>push</li>
+<li>url previews</li>
+<li>phone-home stats</li>
+<li>recaptcha validation</li>
+<li>CAS auth validation</li>
+<li>OpenID Connect</li>
+<li>Outbound federation</li>
+<li>Federation (checking public key revocation)</li>
+<li>Fetching public keys of other servers</li>
+<li>Downloading remote media</li>
+</ul>
+<p>It will <strong>not be used</strong> for:</p>
+<ul>
+<li>Application Services</li>
+<li>Identity servers</li>
+<li>In worker configurations
+<ul>
+<li>connections between workers</li>
+<li>connections from workers to Redis</li>
+</ul>
+</li>
+</ul>
+<h2 id="troubleshooting-1"><a class="header" href="#troubleshooting-1">Troubleshooting</a></h2>
+<p>If a proxy server is used with TLS (HTTPS) and no connections are established,
+it is most likely due to the proxy's certificates. To test this, the validation
+in Synapse can be deactivated.</p>
+<p><strong>NOTE</strong>: This has an impact on security and is for testing purposes only!</p>
+<p>To deactivate the certificate validation, the following setting must be made in
+<a href="setup/../usage/configuration/homeserver_sample_config.html">homserver.yaml</a>.</p>
+<pre><code class="language-yaml">use_insecure_ssl_client_just_for_testing_do_not_use: true
+</code></pre>
<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="overview-1"><a class="header" href="#overview-1">Overview</a></h1>
<p>This document explains how to enable VoIP relaying on your Home Server with
TURN.</p>
@@ -1240,7 +1316,7 @@ turn_allow_guests: True
</ul>
<p>... and then reload any clients (or wait an hour for them to refresh their
settings).</p>
-<h2 id="troubleshooting-1"><a class="header" href="#troubleshooting-1">Troubleshooting</a></h2>
+<h2 id="troubleshooting-2"><a class="header" href="#troubleshooting-2">Troubleshooting</a></h2>
<p>The normal symptoms of a misconfigured TURN server are that calls between
devices on different networks ring, but get stuck at "call
connecting". Unfortunately, troubleshooting this can be tricky.</p>
@@ -1469,6 +1545,43 @@ dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
</code></pre>
</li>
</ul>
+<h1 id="upgrading-to-v1410"><a class="header" href="#upgrading-to-v1410">Upgrading to v1.41.0</a></h1>
+<h2 id="add-support-for-routing-outbound-http-requests-via-a-proxy-for-federation"><a class="header" href="#add-support-for-routing-outbound-http-requests-via-a-proxy-for-federation">Add support for routing outbound HTTP requests via a proxy for federation</a></h2>
+<p>Since Synapse 1.6.0 (2019-11-26) you can set a proxy for outbound HTTP requests via
+http_proxy/https_proxy environment variables. This proxy was set for:</p>
+<ul>
+<li>push</li>
+<li>url previews</li>
+<li>phone-home stats</li>
+<li>recaptcha validation</li>
+<li>CAS auth validation</li>
+<li>OpenID Connect</li>
+<li>Federation (checking public key revocation)</li>
+</ul>
+<p>In this version we have added support for outbound requests for:</p>
+<ul>
+<li>Outbound federation</li>
+<li>Downloading remote media</li>
+<li>Fetching public keys of other servers</li>
+</ul>
+<p>These requests use the same proxy configuration. If you have a proxy configuration we
+recommend to verify the configuration. It may be necessary to adjust the <code>no_proxy</code>
+environment variable.</p>
+<p>See <a href="setup/forward_proxy.html">using a forward proxy with Synapse documentation</a> for
+details.</p>
+<h2 id="deprecation-of-template_dir"><a class="header" href="#deprecation-of-template_dir">Deprecation of <code>template_dir</code></a></h2>
+<p>The <code>template_dir</code> settings in the <code>sso</code>, <code>account_validity</code> and <code>email</code> sections of the
+configuration file are now deprecated. Server admins should use the new
+<code>templates.custom_template_directory</code> setting in the configuration file and use one single
+custom template directory for all aforementioned features. Template file names remain
+unchanged. See <a href="https://matrix-org.github.io/synapse/latest/templates.html">the related documentation</a>
+for more information and examples.</p>
+<p>We plan to remove support for these settings in October 2021.</p>
+<h2 id="_synapseadminv1usersuseridmedia-must-be-handled-by-media-workers"><a class="header" href="#_synapseadminv1usersuseridmedia-must-be-handled-by-media-workers"><code>/_synapse/admin/v1/users/{userId}/media</code> must be handled by media workers</a></h2>
+<p>The <a href="https://matrix-org.github.io/synapse/latest/workers.html#synapseappmedia_repository">media repository worker documentation</a>
+has been updated to reflect that calls to <code>/_synapse/admin/v1/users/{userId}/media</code>
+must now be handled by media repository workers. This is due to the new <code>DELETE</code> method
+of this endpoint modifying the media store.</p>
<h1 id="upgrading-to-v1390"><a class="header" href="#upgrading-to-v1390">Upgrading to v1.39.0</a></h1>
<h2 id="deprecation-of-the-current-third-party-rules-module-interface"><a class="header" href="#deprecation-of-the-current-third-party-rules-module-interface">Deprecation of the current third-party rules module interface</a></h2>
<p>The current third-party rules module interface is deprecated in favour of the new generic
@@ -2824,7 +2937,7 @@ be sent. See <a href="delegate.html">the delegation documentation</a> for instru
<p>Once federation has been configured, you should be able to join a room over
federation. A good place to start is <code>#synapse:matrix.org</code> - a room for
Synapse admins.</p>
-<h2 id="troubleshooting-2"><a class="header" href="#troubleshooting-2">Troubleshooting</a></h2>
+<h2 id="troubleshooting-3"><a class="header" href="#troubleshooting-3">Troubleshooting</a></h2>
<p>You can use the <a href="https://matrix.org/federationtester">federation tester</a>
to check if your homeserver is configured correctly. Alternatively try the
<a href="https://matrix.org/federationtester/api/report?server_name=DOMAIN">JSON API used by the federation tester</a>.
@@ -3075,6 +3188,8 @@ presence:
#
# This option replaces federation_ip_range_blacklist in Synapse v1.25.0.
#
+# Note: The value is ignored when an HTTP proxy is in use
+#
#ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -3428,6 +3543,19 @@ retention:
#
#next_link_domain_whitelist: ["matrix.org"]
+# Templates to use when generating email or HTML page contents.
+#
+templates:
+ # Directory in which Synapse will try to find template files to use to generate
+ # email or HTML page contents.
+ # If not set, or a file is not found within the template directory, a default
+ # template from within the Synapse package will be used.
+ #
+ # See https://matrix-org.github.io/synapse/latest/templates.html for more
+ # information about using custom templates.
+ #
+ #custom_template_directory: /path/to/custom/templates/
+
## TLS ##
@@ -3576,6 +3704,15 @@ caches:
#
#expiry_time: 30m
+ # Controls how long the results of a /sync request are cached for after
+ # a successful response is returned. A higher duration can help clients with
+ # intermittent connections, at the cost of higher memory usage.
+ #
+ # By default, this is zero, which means that sync responses are not cached
+ # at all.
+ #
+ #sync_response_cache_duration: 2m
+
## Database ##
@@ -3828,6 +3965,8 @@ media_store_path: "DATADIR/media_store"
# This must be specified if url_preview_enabled is set. It is recommended that
# you uncomment the following list as a starting point.
#
+# Note: The value is ignored when an HTTP proxy is in use
+#
#url_preview_ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -4747,6 +4886,9 @@ cas_config:
# Additional settings to use with single-sign on systems such as OpenID Connect,
# SAML2 and CAS.
#
+# Server admins can configure custom templates for pages related to SSO. See
+# https://matrix-org.github.io/synapse/latest/templates.html for more information.
+#
sso:
# A list of client URLs which are whitelisted so that the user does not
# have to confirm giving access to their account to the URL. Any client
@@ -4779,169 +4921,6 @@ sso:
#
#update_profile_information: true
- # Directory in which Synapse will try to find the template files below.
- # If not set, or the files named below are not found within the template
- # directory, default templates from within the Synapse package will be used.
- #
- # Synapse will look for the following templates in this directory:
- #
- # * HTML page to prompt the user to choose an Identity Provider during
- # login: 'sso_login_idp_picker.html'.
- #
- # This is only used if multiple SSO Identity Providers are configured.
- #
- # When rendering, this template is given the following variables:
- # * redirect_url: the URL that the user will be redirected to after
- # login.
- #
- # * server_name: the homeserver's name.
- #
- # * providers: a list of available Identity Providers. Each element is
- # an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # The rendered HTML page should contain a form which submits its results
- # back as a GET request, with the following query parameters:
- #
- # * redirectUrl: the client redirect URI (ie, the `redirect_url` passed
- # to the template)
- #
- # * idp: the 'idp_id' of the chosen IDP.
- #
- # * HTML page to prompt new users to enter a userid and confirm other
- # details: 'sso_auth_account_details.html'. This is only shown if the
- # SSO implementation (with any user_mapping_provider) does not return
- # a localpart.
- #
- # When rendering, this template is given the following variables:
- #
- # * server_name: the homeserver's name.
- #
- # * idp: details of the SSO Identity Provider that the user logged in
- # with: an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # * user_attributes: an object containing details about the user that
- # we received from the IdP. May have the following attributes:
- #
- # * display_name: the user's display_name
- # * emails: a list of email addresses
- #
- # The template should render a form which submits the following fields:
- #
- # * username: the localpart of the user's chosen user id
- #
- # * HTML page allowing the user to consent to the server's terms and
- # conditions. This is only shown for new users, and only if
- # `user_consent.require_at_registration` is set.
- #
- # When rendering, this template is given the following variables:
- #
- # * server_name: the homeserver's name.
- #
- # * user_id: the user's matrix proposed ID.
- #
- # * user_profile.display_name: the user's proposed display name, if any.
- #
- # * consent_version: the version of the terms that the user will be
- # shown
- #
- # * terms_url: a link to the page showing the terms.
- #
- # The template should render a form which submits the following fields:
- #
- # * accepted_version: the version of the terms accepted by the user
- # (ie, 'consent_version' from the input variables).
- #
- # * HTML page for a confirmation step before redirecting back to the client
- # with the login token: 'sso_redirect_confirm.html'.
- #
- # When rendering, this template is given the following variables:
- #
- # * redirect_url: the URL the user is about to be redirected to.
- #
- # * display_url: the same as `redirect_url`, but with the query
- # parameters stripped. The intention is to have a
- # human-readable URL to show to users, not to use it as
- # the final address to redirect to.
- #
- # * server_name: the homeserver's name.
- #
- # * new_user: a boolean indicating whether this is the user's first time
- # logging in.
- #
- # * user_id: the user's matrix ID.
- #
- # * user_profile.avatar_url: an MXC URI for the user's avatar, if any.
- # None if the user has not set an avatar.
- #
- # * user_profile.display_name: the user's display name. None if the user
- # has not set a display name.
- #
- # * HTML page which notifies the user that they are authenticating to confirm
- # an operation on their account during the user interactive authentication
- # process: 'sso_auth_confirm.html'.
- #
- # When rendering, this template is given the following variables:
- # * redirect_url: the URL the user is about to be redirected to.
- #
- # * description: the operation which the user is being asked to confirm
- #
- # * idp: details of the Identity Provider that we will use to confirm
- # the user's identity: an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # * HTML page shown after a successful user interactive authentication session:
- # 'sso_auth_success.html'.
- #
- # Note that this page must include the JavaScript which notifies of a successful authentication
- # (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
- #
- # This template has no additional variables.
- #
- # * HTML page shown after a user-interactive authentication session which
- # does not map correctly onto the expected user: 'sso_auth_bad_user.html'.
- #
- # When rendering, this template is given the following variables:
- # * server_name: the homeserver's name.
- # * user_id_to_verify: the MXID of the user that we are trying to
- # validate.
- #
- # * HTML page shown during single sign-on if a deactivated user (according to Synapse's database)
- # attempts to login: 'sso_account_deactivated.html'.
- #
- # This template has no additional variables.
- #
- # * HTML page to display to users if something goes wrong during the
- # OpenID Connect authentication process: 'sso_error.html'.
- #
- # When rendering, this template is given two variables:
- # * error: the technical name of the error
- # * error_description: a human-readable message for the error
- #
- # You can see the default templates at:
- # https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
- #
- #template_dir: "res/templates"
-
# JSON web token integration. The following settings can be used to make
# Synapse JSON web tokens for authentication, instead of its internal
@@ -5072,6 +5051,9 @@ ui_auth:
# Configuration for sending emails from Synapse.
#
+# Server admins can configure custom templates for email content. See
+# https://matrix-org.github.io/synapse/latest/templates.html for more information.
+#
email:
# The hostname of the outgoing SMTP server to use. Defaults to 'localhost'.
#
@@ -5094,6 +5076,14 @@ email:
#
#require_transport_security: true
+ # Uncomment the following to disable TLS for SMTP.
+ #
+ # By default, if the server supports TLS, it will be used, and the server
+ # must present a certificate that is valid for 'smtp_host'. If this option
+ # is set to false, TLS will not be used.
+ #
+ #enable_tls: false
+
# notif_from defines the "From" address to use when sending emails.
# It must be set if email sending is enabled.
#
@@ -5140,49 +5130,6 @@ email:
#
#invite_client_location: https://app.element.io
- # Directory in which Synapse will try to find the template files below.
- # If not set, or the files named below are not found within the template
- # directory, default templates from within the Synapse package will be used.
- #
- # Synapse will look for the following templates in this directory:
- #
- # * The contents of email notifications of missed events: 'notif_mail.html' and
- # 'notif_mail.txt'.
- #
- # * The contents of account expiry notice emails: 'notice_expiry.html' and
- # 'notice_expiry.txt'.
- #
- # * The contents of password reset emails sent by the homeserver:
- # 'password_reset.html' and 'password_reset.txt'
- #
- # * An HTML page that a user will see when they follow the link in the password
- # reset email. The user will be asked to confirm the action before their
- # password is reset: 'password_reset_confirmation.html'
- #
- # * HTML pages for success and failure that a user will see when they confirm
- # the password reset flow using the page above: 'password_reset_success.html'
- # and 'password_reset_failure.html'
- #
- # * The contents of address verification emails sent during registration:
- # 'registration.html' and 'registration.txt'
- #
- # * HTML pages for success and failure that a user will see when they follow
- # the link in an address verification email sent during registration:
- # 'registration_success.html' and 'registration_failure.html'
- #
- # * The contents of address verification emails sent when an address is added
- # to a Matrix account: 'add_threepid.html' and 'add_threepid.txt'
- #
- # * HTML pages for success and failure that a user will see when they follow
- # the link in an address verification email sent when an address is added
- # to a Matrix account: 'add_threepid_success.html' and
- # 'add_threepid_failure.html'
- #
- # You can see the default templates at:
- # https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
- #
- #template_dir: "res/templates"
-
# Subjects to use when sending emails from Synapse.
#
# The placeholder '%(app)s' will be replaced with the value of the 'app_name'
@@ -5670,18 +5617,31 @@ handlers:
backupCount: 3 # Does not include the current log file.
encoding: utf8
- # Default to buffering writes to log file for efficiency. This means that
- # will be a delay for INFO/DEBUG logs to get written, but WARNING/ERROR
- # logs will still be flushed immediately.
+ # Default to buffering writes to log file for efficiency.
+ # WARNING/ERROR logs will still be flushed immediately, but there will be a
+ # delay (of up to `period` seconds, or until the buffer is full with
+ # `capacity` messages) before INFO/DEBUG logs get written.
buffer:
- class: logging.handlers.MemoryHandler
+ class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
- # The capacity is the number of log lines that are buffered before
- # being written to disk. Increasing this will lead to better
+
+ # The capacity is the maximum number of log lines that are buffered
+ # before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
+ # This parameter is required.
capacity: 10
- flushLevel: 30 # Flush for WARNING logs as well
+
+ # Logs with a level at or above the flush level will cause the buffer to
+ # be flushed immediately.
+ # Default value: 40 (ERROR)
+ # Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
+ flushLevel: 30 # Flush immediately for WARNING logs and higher
+
+ # The period of time, in seconds, between forced flushes.
+ # Messages will not be delayed for longer than this time.
+ # Default value: 5 seconds
+ period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
@@ -5856,6 +5816,276 @@ loggers:
flexible. It allows for configuration that were not previously possible, such as
sending plain logs over the network, or using different handlers for different
modules.</p>
+<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="templates"><a class="header" href="#templates">Templates</a></h1>
+<p>Synapse uses parametrised templates to generate the content of emails it sends and
+webpages it shows to users.</p>
+<p>By default, Synapse will use the templates listed <a href="https://github.com/matrix-org/synapse/tree/master/synapse/res/templates">here</a>.
+Server admins can configure an additional directory for Synapse to look for templates
+in, allowing them to specify custom templates:</p>
+<pre><code class="language-yaml">templates:
+ custom_templates_directory: /path/to/custom/templates/
+</code></pre>
+<p>If this setting is not set, or the files named below are not found within the directory,
+default templates from within the Synapse package will be used.</p>
+<p>Templates that are given variables when being rendered are rendered using <a href="https://jinja.palletsprojects.com/en/2.11.x/">Jinja 2</a>.
+Templates rendered by Jinja 2 can also access two functions on top of the functions
+already available as part of Jinja 2:</p>
+<pre><code class="language-python">format_ts(value: int, format: str) -> str
+</code></pre>
+<p>Formats a timestamp in milliseconds.</p>
+<p>Example: <code>reason.last_sent_ts|format_ts("%c")</code></p>
+<pre><code class="language-python">mxc_to_http(value: str, width: int, height: int, resize_method: str = "crop") -> str
+</code></pre>
+<p>Turns a <code>mxc://</code> URL for media content into an HTTP(S) one using the homeserver's
+<code>public_baseurl</code> configuration setting as the URL's base.</p>
+<p>Example: <code>message.sender_avatar_url|mxc_to_http(32,32)</code></p>
+<h2 id="email-templates"><a class="header" href="#email-templates">Email templates</a></h2>
+<p>Below are the templates Synapse will look for when generating the content of an email:</p>
+<ul>
+<li><code>notif_mail.html</code> and <code>notif_mail.txt</code>: The contents of email notifications of missed
+events.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>user_display_name</code>: the display name for the user receiving the notification</li>
+<li><code>unsubscribe_link</code>: the link users can click to unsubscribe from email notifications</li>
+<li><code>summary_text</code>: a summary of the notification(s). The text used can be customised
+by configuring the various settings in the <code>email.subjects</code> section of the
+configuration file.</li>
+<li><code>rooms</code>: a list of rooms containing events to include in the email. Each element is
+an object with the following attributes:
+<ul>
+<li><code>title</code>: a human-readable name for the room</li>
+<li><code>hash</code>: a hash of the ID of the room</li>
+<li><code>invite</code>: a boolean, which is <code>True</code> if the room is an invite the user hasn't
+accepted yet, <code>False</code> otherwise</li>
+<li><code>notifs</code>: a list of events, or an empty list if <code>invite</code> is <code>True</code>. Each element
+is an object with the following attributes:
+<ul>
+<li><code>link</code>: a <code>matrix.to</code> link to the event</li>
+<li><code>ts</code>: the time in milliseconds at which the event was received</li>
+<li><code>messages</code>: a list of messages containing one message before the event, the
+message in the event, and one message after the event. Each element is an
+object with the following attributes:
+<ul>
+<li><code>event_type</code>: the type of the event</li>
+<li><code>is_historical</code>: a boolean, which is <code>False</code> if the message is the one
+that triggered the notification, <code>True</code> otherwise</li>
+<li><code>id</code>: the ID of the event</li>
+<li><code>ts</code>: the time in milliseconds at which the event was sent</li>
+<li><code>sender_name</code>: the display name for the event's sender</li>
+<li><code>sender_avatar_url</code>: the avatar URL (as a <code>mxc://</code> URL) for the event's
+sender</li>
+<li><code>sender_hash</code>: a hash of the user ID of the sender</li>
+</ul>
+</li>
+</ul>
+</li>
+<li><code>link</code>: a <code>matrix.to</code> link to the room</li>
+</ul>
+</li>
+<li><code>reason</code>: information on the event that triggered the email to be sent. It's an
+object with the following attributes:
+<ul>
+<li><code>room_id</code>: the ID of the room the event was sent in</li>
+<li><code>room_name</code>: a human-readable name for the room the event was sent in</li>
+<li><code>now</code>: the current time in milliseconds</li>
+<li><code>received_at</code>: the time in milliseconds at which the event was received</li>
+<li><code>delay_before_mail_ms</code>: the amount of time in milliseconds Synapse always waits
+before ever emailing about a notification (to give the user a chance to respond
+to other push or notice the window)</li>
+<li><code>last_sent_ts</code>: the time in milliseconds at which a notification was last sent
+for an event in this room</li>
+<li><code>throttle_ms</code>: the minimum amount of time in milliseconds between two
+notifications can be sent for this room</li>
+</ul>
+</li>
+</ul>
+</li>
+<li><code>password_reset.html</code> and <code>password_reset.txt</code>: The contents of password reset emails
+sent by the homeserver.
+When rendering, these templates are given a <code>link</code> variable which contains the link the
+user must click in order to reset their password.</li>
+<li><code>registration.html</code> and <code>registration.txt</code>: The contents of address verification emails
+sent during registration.
+When rendering, these templates are given a <code>link</code> variable which contains the link the
+user must click in order to validate their email address.</li>
+<li><code>add_threepid.html</code> and <code>add_threepid.txt</code>: The contents of address verification emails
+sent when an address is added to a Matrix account.
+When rendering, these templates are given a <code>link</code> variable which contains the link the
+user must click in order to validate their email address.</li>
+</ul>
+<h2 id="html-page-templates-for-registration-and-password-reset"><a class="header" href="#html-page-templates-for-registration-and-password-reset">HTML page templates for registration and password reset</a></h2>
+<p>Below are the templates Synapse will look for when generating pages related to
+registration and password reset:</p>
+<ul>
+<li><code>password_reset_confirmation.html</code>: An HTML page that a user will see when they follow
+the link in the password reset email. The user will be asked to confirm the action
+before their password is reset.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>sid</code>: the session ID for the password reset</li>
+<li><code>token</code>: the token for the password reset</li>
+<li><code>client_secret</code>: the client secret for the password reset</li>
+</ul>
+</li>
+<li><code>password_reset_success.html</code> and <code>password_reset_failure.html</code>: HTML pages for success
+and failure that a user will see when they confirm the password reset flow using the
+page above.
+When rendering, <code>password_reset_success.html</code> is given no variable, and
+<code>password_reset_failure.html</code> is given a <code>failure_reason</code>, which contains the reason
+for the password reset failure. </li>
+<li><code>registration_success.html</code> and <code>registration_failure.html</code>: HTML pages for success and
+failure that a user will see when they follow the link in an address verification email
+sent during registration.
+When rendering, <code>registration_success.html</code> is given no variable, and
+<code>registration_failure.html</code> is given a <code>failure_reason</code>, which contains the reason
+for the registration failure.</li>
+<li><code>add_threepid_success.html</code> and <code>add_threepid_failure.html</code>: HTML pages for success and
+failure that a user will see when they follow the link in an address verification email
+sent when an address is added to a Matrix account.
+When rendering, <code>add_threepid_success.html</code> is given no variable, and
+<code>add_threepid_failure.html</code> is given a <code>failure_reason</code>, which contains the reason
+for the registration failure.</li>
+</ul>
+<h2 id="html-page-templates-for-single-sign-on-sso"><a class="header" href="#html-page-templates-for-single-sign-on-sso">HTML page templates for Single Sign-On (SSO)</a></h2>
+<p>Below are the templates Synapse will look for when generating pages related to SSO:</p>
+<ul>
+<li><code>sso_login_idp_picker.html</code>: HTML page to prompt the user to choose an
+Identity Provider during login.
+This is only used if multiple SSO Identity Providers are configured.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>redirect_url</code>: the URL that the user will be redirected to after
+login.</li>
+<li><code>server_name</code>: the homeserver's name.</li>
+<li><code>providers</code>: a list of available Identity Providers. Each element is
+an object with the following attributes:
+<ul>
+<li><code>idp_id</code>: unique identifier for the IdP</li>
+<li><code>idp_name</code>: user-facing name for the IdP</li>
+<li><code>idp_icon</code>: if specified in the IdP config, an MXC URI for an icon
+for the IdP</li>
+<li><code>idp_brand</code>: if specified in the IdP config, a textual identifier
+for the brand of the IdP
+The rendered HTML page should contain a form which submits its results
+back as a GET request, with the following query parameters:</li>
+</ul>
+</li>
+<li><code>redirectUrl</code>: the client redirect URI (ie, the <code>redirect_url</code> passed
+to the template)</li>
+<li><code>idp</code>: the 'idp_id' of the chosen IDP.</li>
+</ul>
+</li>
+<li><code>sso_auth_account_details.html</code>: HTML page to prompt new users to enter a
+userid and confirm other details. This is only shown if the
+SSO implementation (with any <code>user_mapping_provider</code>) does not return
+a localpart.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>server_name</code>: the homeserver's name.</li>
+<li><code>idp</code>: details of the SSO Identity Provider that the user logged in
+with: an object with the following attributes:
+<ul>
+<li><code>idp_id</code>: unique identifier for the IdP</li>
+<li><code>idp_name</code>: user-facing name for the IdP</li>
+<li><code>idp_icon</code>: if specified in the IdP config, an MXC URI for an icon
+for the IdP</li>
+<li><code>idp_brand</code>: if specified in the IdP config, a textual identifier
+for the brand of the IdP</li>
+</ul>
+</li>
+<li><code>user_attributes</code>: an object containing details about the user that
+we received from the IdP. May have the following attributes:
+<ul>
+<li>display_name: the user's display_name</li>
+<li>emails: a list of email addresses
+The template should render a form which submits the following fields:</li>
+</ul>
+</li>
+<li><code>username</code>: the localpart of the user's chosen user id</li>
+</ul>
+</li>
+<li><code>sso_new_user_consent.html</code>: HTML page allowing the user to consent to the
+server's terms and conditions. This is only shown for new users, and only if
+<code>user_consent.require_at_registration</code> is set.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>server_name</code>: the homeserver's name.</li>
+<li><code>user_id</code>: the user's matrix proposed ID.</li>
+<li><code>user_profile.display_name</code>: the user's proposed display name, if any.</li>
+<li>consent_version: the version of the terms that the user will be
+shown</li>
+<li><code>terms_url</code>: a link to the page showing the terms.
+The template should render a form which submits the following fields:</li>
+<li><code>accepted_version</code>: the version of the terms accepted by the user
+(ie, 'consent_version' from the input variables).</li>
+</ul>
+</li>
+<li><code>sso_redirect_confirm.html</code>: HTML page for a confirmation step before redirecting back
+to the client with the login token.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>redirect_url</code>: the URL the user is about to be redirected to.</li>
+<li><code>display_url</code>: the same as <code>redirect_url</code>, but with the query
+parameters stripped. The intention is to have a
+human-readable URL to show to users, not to use it as
+the final address to redirect to.</li>
+<li><code>server_name</code>: the homeserver's name.</li>
+<li><code>new_user</code>: a boolean indicating whether this is the user's first time
+logging in.</li>
+<li><code>user_id</code>: the user's matrix ID.</li>
+<li><code>user_profile.avatar_url</code>: an MXC URI for the user's avatar, if any.
+<code>None</code> if the user has not set an avatar.</li>
+<li><code>user_profile.display_name</code>: the user's display name. <code>None</code> if the user
+has not set a display name.</li>
+</ul>
+</li>
+<li><code>sso_auth_confirm.html</code>: HTML page which notifies the user that they are authenticating
+to confirm an operation on their account during the user interactive authentication
+process.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>redirect_url</code>: the URL the user is about to be redirected to.</li>
+<li><code>description</code>: the operation which the user is being asked to confirm</li>
+<li><code>idp</code>: details of the Identity Provider that we will use to confirm
+the user's identity: an object with the following attributes:
+<ul>
+<li><code>idp_id</code>: unique identifier for the IdP</li>
+<li><code>idp_name</code>: user-facing name for the IdP</li>
+<li><code>idp_icon</code>: if specified in the IdP config, an MXC URI for an icon
+for the IdP</li>
+<li><code>idp_brand</code>: if specified in the IdP config, a textual identifier
+for the brand of the IdP</li>
+</ul>
+</li>
+</ul>
+</li>
+<li><code>sso_auth_success.html</code>: HTML page shown after a successful user interactive
+authentication session.
+Note that this page must include the JavaScript which notifies of a successful
+authentication (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
+This template has no additional variables.</li>
+<li><code>sso_auth_bad_user.html</code>: HTML page shown after a user-interactive authentication
+session which does not map correctly onto the expected user.
+When rendering, this template is given the following variables:
+<ul>
+<li><code>server_name</code>: the homeserver's name.</li>
+<li><code>user_id_to_verify</code>: the MXID of the user that we are trying to
+validate.</li>
+</ul>
+</li>
+<li><code>sso_account_deactivated.html</code>: HTML page shown during single sign-on if a deactivated
+user (according to Synapse's database) attempts to login.
+This template has no additional variables.</li>
+<li><code>sso_error.html</code>: HTML page to display to users if something goes wrong during the
+OpenID Connect authentication process.
+When rendering, this template is given two variables:
+<ul>
+<li><code>error</code>: the technical name of the error</li>
+<li><code>error_description</code>: a human-readable message for the error</li>
+</ul>
+</li>
+</ul>
<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="user-authentication"><a class="header" href="#user-authentication">User Authentication</a></h1>
<p>Synapse supports multiple methods of authenticating users, either out-of-the-box or through custom pluggable
authentication modules.</p>
@@ -7998,6 +8228,7 @@ expressions:</p>
^/_matrix/federation/v1/send/
# Client API requests
+^/_matrix/client/(api/v1|r0|unstable)/createRoom$
^/_matrix/client/(api/v1|r0|unstable)/publicRooms$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/joined_members$
^/_matrix/client/(api/v1|r0|unstable)/rooms/.*/context/.*$
@@ -8163,10 +8394,12 @@ For example:</p>
^/_synapse/admin/v1/user/.*/media.*$
^/_synapse/admin/v1/media/.*$
^/_synapse/admin/v1/quarantine_media/.*$
+^/_synapse/admin/v1/users/.*/media$
</code></pre>
<p>You should also set <code>enable_media_repo: False</code> in the shared configuration
file to stop the main synapse running background jobs related to managing the
-media repository.</p>
+media repository. Note that doing so will prevent the main process from being
+able to handle the above endpoints.</p>
<p>In the <code>media_repository</code> worker configuration file, configure the http listener to
expose the <code>media</code> resource. For example:</p>
<pre><code class="language-yaml"> worker_listeners:
@@ -8611,6 +8844,7 @@ have a canonical alias set.</li>
<ul>
<li><a href="admin_api/media_admin_api.html#delete-a-specific-local-media">Delete a specific local media</a></li>
<li><a href="admin_api/media_admin_api.html#delete-local-media-by-date-or-size">Delete local media by date or size</a></li>
+<li><a href="admin_api/media_admin_api.html#delete-media-uploaded-by-a-user">Delete media uploaded by a user</a></li>
</ul>
</li>
<li><a href="admin_api/media_admin_api.html#purge-remote-media-api">Purge Remote Media API</a></li>
@@ -8639,7 +8873,8 @@ server admin: see <a href="admin_api/../usage/administration/admin_api">Admin AP
</code></pre>
<h2 id="list-all-media-uploaded-by-a-user"><a class="header" href="#list-all-media-uploaded-by-a-user">List all media uploaded by a user</a></h2>
<p>Listing all media that has been uploaded by a local user can be achieved through
-the use of the <a href="admin_api/user_admin_api.html#list-media-of-a-user">List media of a user</a>
+the use of the
+<a href="admin_api/user_admin_api.html#list-media-uploaded-by-a-user">List media uploaded by a user</a>
Admin API.</p>
<h1 id="quarantine-media"><a class="header" href="#quarantine-media">Quarantine media</a></h1>
<p>Quarantining media means that it is marked as inaccessible by users. It applies
@@ -8799,6 +9034,9 @@ If <code>false</code> these files will be deleted. Defaults to <code>true</code>
<li><code>deleted_media</code>: an array of strings - List of deleted <code>media_id</code></li>
<li><code>total</code>: integer - Total number of deleted <code>media_id</code></li>
</ul>
+<h2 id="delete-media-uploaded-by-a-user"><a class="header" href="#delete-media-uploaded-by-a-user">Delete media uploaded by a user</a></h2>
+<p>You can find details of how to delete multiple media uploaded by a user in
+<a href="admin_api/user_admin_api.html#delete-media-uploaded-by-a-user">User Admin API</a>.</p>
<h1 id="purge-remote-media-api"><a class="header" href="#purge-remote-media-api">Purge Remote Media API</a></h1>
<p>The purge remote media API allows server admins to purge old cached remote media.</p>
<p>The API is:</p>
@@ -9843,6 +10081,16 @@ specific <code>user_id</code>.</p>
"address": "<user_mail_2>"
}
],
+ "external_ids": [
+ {
+ "auth_provider": "<provider1>",
+ "external_id": "<user_id_provider_1>"
+ },
+ {
+ "auth_provider": "<provider2>",
+ "external_id": "<user_id_provider_2>"
+ }
+ ],
"avatar_url": "<avatar_url>",
"admin": false,
"deactivated": false
@@ -9850,36 +10098,44 @@ specific <code>user_id</code>.</p>
</code></pre>
<p>To use it, you will need to authenticate by providing an <code>access_token</code> for a
server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a></p>
+<p>Returns HTTP status code:</p>
+<ul>
+<li><code>201</code> - When a new user object was created.</li>
+<li><code>200</code> - When a user was modified.</li>
+</ul>
<p>URL parameters:</p>
<ul>
<li><code>user_id</code>: fully-qualified user id: for example, <code>@user:server.com</code>.</li>
</ul>
<p>Body parameters:</p>
<ul>
-<li>
-<p><code>password</code>, optional. If provided, the user's password is updated and all
-devices are logged out.</p>
-</li>
-<li>
-<p><code>displayname</code>, optional, defaults to the value of <code>user_id</code>.</p>
-</li>
-<li>
-<p><code>threepids</code>, optional, allows setting the third-party IDs (email, msisdn)
-belonging to a user.</p>
-</li>
-<li>
-<p><code>avatar_url</code>, optional, must be a
-<a href="https://matrix.org/docs/spec/client_server/r0.6.0#matrix-content-mxc-uris">MXC URI</a>.</p>
+<li><code>password</code> - string, optional. If provided, the user's password is updated and all
+devices are logged out.</li>
+<li><code>displayname</code> - string, optional, defaults to the value of <code>user_id</code>.</li>
+<li><code>threepids</code> - array, optional, allows setting the third-party IDs (email, msisdn)
+<ul>
+<li><code>medium</code> - string. Kind of third-party ID, either <code>email</code> or <code>msisdn</code>.</li>
+<li><code>address</code> - string. Value of third-party ID.
+belonging to a user.</li>
+</ul>
</li>
-<li>
-<p><code>admin</code>, optional, defaults to <code>false</code>.</p>
+<li><code>external_ids</code> - array, optional. Allow setting the identifier of the external identity
+provider for SSO (Single sign-on). Details in
+<a href="admin_api/../usage/configuration/homeserver_sample_config.html">Sample Configuration File</a>
+section <code>sso</code> and <code>oidc_providers</code>.
+<ul>
+<li><code>auth_provider</code> - string. ID of the external identity provider. Value of <code>idp_id</code>
+in homeserver configuration.</li>
+<li><code>external_id</code> - string, user ID in the external identity provider.</li>
+</ul>
</li>
-<li>
-<p><code>deactivated</code>, optional. If unspecified, deactivation state will be left
+<li><code>avatar_url</code> - string, optional, must be a
+<a href="https://matrix.org/docs/spec/client_server/r0.6.0#matrix-content-mxc-uris">MXC URI</a>.</li>
+<li><code>admin</code> - bool, optional, defaults to <code>false</code>.</li>
+<li><code>deactivated</code> - bool, optional. If unspecified, deactivation state will be left
unchanged on existing accounts and set to <code>false</code> for new accounts.
A user cannot be erased by deactivating with this API. For details on
-deactivating users see <a href="admin_api/user_admin_api.html#deactivate-account">Deactivate Account</a>.</p>
-</li>
+deactivating users see <a href="admin_api/user_admin_api.html#deactivate-account">Deactivate Account</a>.</li>
</ul>
<p>If the user already exists then optional parameters default to the current value.</p>
<p>In order to re-activate an account <code>deactivated</code> must be set to <code>false</code>. If
@@ -10148,7 +10404,8 @@ member are returned.</p>
<li><code>joined_rooms</code> - An array of <code>room_id</code>.</li>
<li><code>total</code> - Number of rooms.</li>
</ul>
-<h2 id="list-media-of-a-user"><a class="header" href="#list-media-of-a-user">List media of a user</a></h2>
+<h2 id="user-media"><a class="header" href="#user-media">User media</a></h2>
+<h3 id="list-media-uploaded-by-a-user"><a class="header" href="#list-media-uploaded-by-a-user">List media uploaded by a user</a></h3>
<p>Gets a list of all local media that a specific <code>user_id</code> has created.
By default, the response is ordered by descending creation date and ascending media ID.
The newest media is on top. You can change the order with parameters
@@ -10241,44 +10498,56 @@ database, especially for large environments.</p>
<p><strong>Response</strong></p>
<p>The following fields are returned in the JSON response body:</p>
<ul>
-<li>
-<p><code>media</code> - An array of objects, each containing information about a media.
-Media objects contain the following fields:</p>
+<li><code>media</code> - An array of objects, each containing information about a media.
+Media objects contain the following fields:
<ul>
-<li>
-<p><code>created_ts</code> - integer - Timestamp when the content was uploaded in ms.</p>
-</li>
-<li>
-<p><code>last_access_ts</code> - integer - Timestamp when the content was last accessed in ms.</p>
-</li>
-<li>
-<p><code>media_id</code> - string - The id used to refer to the media.</p>
-</li>
-<li>
-<p><code>media_length</code> - integer - Length of the media in bytes.</p>
-</li>
-<li>
-<p><code>media_type</code> - string - The MIME-type of the media.</p>
-</li>
-<li>
-<p><code>quarantined_by</code> - string - The user ID that initiated the quarantine request
-for this media.</p>
-</li>
-<li>
-<p><code>safe_from_quarantine</code> - bool - Status if this media is safe from quarantining.</p>
-</li>
-<li>
-<p><code>upload_name</code> - string - The name the media was uploaded with.</p>
-</li>
+<li><code>created_ts</code> - integer - Timestamp when the content was uploaded in ms.</li>
+<li><code>last_access_ts</code> - integer - Timestamp when the content was last accessed in ms.</li>
+<li><code>media_id</code> - string - The id used to refer to the media.</li>
+<li><code>media_length</code> - integer - Length of the media in bytes.</li>
+<li><code>media_type</code> - string - The MIME-type of the media.</li>
+<li><code>quarantined_by</code> - string - The user ID that initiated the quarantine request
+for this media.</li>
+<li><code>safe_from_quarantine</code> - bool - Status if this media is safe from quarantining.</li>
+<li><code>upload_name</code> - string - The name the media was uploaded with.</li>
</ul>
</li>
-<li>
-<p><code>next_token</code>: integer - Indication for pagination. See above.</p>
-</li>
-<li>
-<p><code>total</code> - integer - Total number of media.</p>
-</li>
+<li><code>next_token</code>: integer - Indication for pagination. See above.</li>
+<li><code>total</code> - integer - Total number of media.</li>
+</ul>
+<h3 id="delete-media-uploaded-by-a-user-1"><a class="header" href="#delete-media-uploaded-by-a-user-1">Delete media uploaded by a user</a></h3>
+<p>This API deletes the <em>local</em> media from the disk of your own server
+that a specific <code>user_id</code> has created. This includes any local thumbnails.</p>
+<p>This API will not affect media that has been uploaded to external
+media repositories (e.g https://github.com/turt2live/matrix-media-repo/).</p>
+<p>By default, the API deletes media ordered by descending creation date and ascending media ID.
+The newest media is deleted first. You can change the order with parameters
+<code>order_by</code> and <code>dir</code>. If no <code>limit</code> is set the API deletes <code>100</code> files per request.</p>
+<p>The API is:</p>
+<pre><code>DELETE /_synapse/admin/v1/users/<user_id>/media
+</code></pre>
+<p>To use it, you will need to authenticate by providing an <code>access_token</code> for a
+server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a></p>
+<p>A response body like the following is returned:</p>
+<pre><code class="language-json">{
+ "deleted_media": [
+ "abcdefghijklmnopqrstuvwx"
+ ],
+ "total": 1
+}
+</code></pre>
+<p>The following fields are returned in the JSON response body:</p>
+<ul>
+<li><code>deleted_media</code>: an array of strings - List of deleted <code>media_id</code></li>
+<li><code>total</code>: integer - Total number of deleted <code>media_id</code></li>
</ul>
+<p><strong>Note</strong>: There is no <code>next_token</code>. This is not useful for deleting media, because
+after deleting media the remaining media have a new order.</p>
+<p><strong>Parameters</strong></p>
+<p>This API has the same parameters as
+<a href="admin_api/user_admin_api.html#list-media-uploaded-by-a-user">List media uploaded by a user</a>.
+With the parameters you can for example limit the number of files to delete at once or
+delete largest/smallest or newest/oldest files first.</p>
<h2 id="login-as-a-user"><a class="header" href="#login-as-a-user">Login as a user</a></h2>
<p>Get an access token that can be used to authenticate as that user. Useful for
when admins wish to do actions on behalf of a user.</p>
@@ -10647,6 +10916,18 @@ server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a
<li><code>user_id</code> - The fully qualified MXID: for example, <code>@user:server.com</code>. The user must
be local.</li>
</ul>
+<h3 id="check-username-availability"><a class="header" href="#check-username-availability">Check username availability</a></h3>
+<p>Checks to see if a username is available, and valid, for the server. See <a href="https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available">the client-server
+API</a>
+for more information.</p>
+<p>This endpoint will work even if registration is disabled on the server, unlike
+<code>/_matrix/client/r0/register/available</code>.</p>
+<p>The API is:</p>
+<pre><code>POST /_synapse/admin/v1/username_availabile?username=$localpart
+</code></pre>
+<p>The request and response format is the same as the <a href="https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available">/_matrix/client/r0/register/available</a> API.</p>
+<p>To use it, you will need to authenticate by providing an <code>access_token</code> for a
+server admin: <a href="admin_api/../usage/administration/admin_api">Admin API</a></p>
<div id="chapter_begin" style="break-before: page; page-break-before: always;"></div><h1 id="version-api"><a class="header" href="#version-api">Version API</a></h1>
<p>This API returns the running Synapse version and the Python version
on which Synapse is being run. This is useful when a Synapse instance
@@ -10706,7 +10987,7 @@ the username <code>matrix</code>:</p>
<p>This gives a Python REPL in which <code>hs</code> gives access to the
<code>synapse.server.HomeServer</code> object - which in turn gives access to many other
parts of the process.</p>
-<p>Note that any call which returns a coroutine will need to be wrapped in <code>ensureDeferred</code>.</p>
+<p>Note that, prior to Synapse 1.41, any call which returns a coroutine will need to be wrapped in <code>ensureDeferred</code>.</p>
<p>As a simple example, retrieving an event from the database:</p>
<pre><code class="language-pycon">>>> from twisted.internet import defer
>>> defer.ensureDeferred(hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org'))
@@ -11010,8 +11291,9 @@ small processing times.</p>
<li><a href="development/contributing_guide.html#8-test-test-test">8. Test, test, test!</a>
<ul>
<li><a href="development/contributing_guide.html#run-the-linters">Run the linters.</a></li>
-<li><a href="development/contributing_guide.html#run-the-unit-tests">Run the unit tests.</a></li>
-<li><a href="development/contributing_guide.html#run-the-integration-tests">Run the integration tests.</a></li>
+<li><a href="development/contributing_guide.html#run-the-unit-tests-twisted-trial">Run the unit tests.</a></li>
+<li><a href="development/contributing_guide.html#run-the-integration-tests-sytest">Run the integration tests (SyTest).</a></li>
+<li><a href="development/contributing_guide.html#run-the-integration-tests-complement">Run the integration tests (Complement).</a></li>
</ul>
</li>
<li><a href="development/contributing_guide.html#9-submit-your-patch">9. Submit your patch.</a>
@@ -11136,7 +11418,7 @@ anything was broken. They are slower than the unit tests but will
typically catch more errors.</p>
<p>The following command will let you run the integration test with the most common
configuration:</p>
-<pre><code class="language-sh">$ docker run --rm -it -v /path/where/you/have/cloned/the/repository\:/src:ro -v /path/to/where/you/want/logs\:/logs matrixdotorg/sytest-synapse:py37
+<pre><code class="language-sh">$ docker run --rm -it -v /path/where/you/have/cloned/the/repository\:/src:ro -v /path/to/where/you/want/logs\:/logs matrixdotorg/sytest-synapse:buster
</code></pre>
<p>This configuration should generally cover your needs. For more details about other configurations, see <a href="https://github.com/matrix-org/sytest/blob/develop/docker/README.md">documentation in the SyTest repo</a>.</p>
<h2 id="run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgcomplementcomplementa"><a class="header" href="#run-the-integration-tests-a-hrefhttpsgithubcommatrix-orgcomplementcomplementa">Run the integration tests (<a href="https://github.com/matrix-org/complement">Complement</a>).</a></h2>
@@ -11172,6 +11454,7 @@ Here is how to run your local Synapse checkout against your local Complement che
<li>on GitHub, <a href="https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request">create the Pull Request</a>;</li>
<li>add a <a href="development/contributing_guide.html#changelog">changelog entry</a> and push it to your Pull Request;</li>
<li>for most contributors, that's all - however, if you are a member of the organization <code>matrix-org</code>, on GitHub, please request a review from <code>matrix.org / Synapse Core</code>.</li>
+<li>if you need to update your PR, please avoid rebasing and just add new commits to your branch.</li>
</ol>
<h2 id="changelog"><a class="header" href="#changelog">Changelog</a></h2>
<p>All changes, even minor ones, need a corresponding changelog / newsfragment
|
