diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 10be12d638..56e0141c2b 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -7,6 +7,7 @@
- [Installation](setup/installation.md)
- [Using Postgres](postgres.md)
- [Configuring a Reverse Proxy](reverse_proxy.md)
+ - [Configuring a Forward/Outbound Proxy](setup/forward_proxy.md)
- [Configuring a Turn Server](turn-howto.md)
- [Delegation](delegate.md)
@@ -20,6 +21,7 @@
- [Homeserver Sample Config File](usage/configuration/homeserver_sample_config.md)
- [Logging Sample Config File](usage/configuration/logging_sample_config.md)
- [Structured Logging](structured_logging.md)
+ - [Templates](templates.md)
- [User Authentication](usage/configuration/user_authentication/README.md)
- [Single-Sign On]()
- [OpenID Connect](openid.md)
diff --git a/docs/admin_api/media_admin_api.md b/docs/admin_api/media_admin_api.md
index 61bed1e0d5..ea05bd6e44 100644
--- a/docs/admin_api/media_admin_api.md
+++ b/docs/admin_api/media_admin_api.md
@@ -12,6 +12,7 @@
- [Delete local media](#delete-local-media)
* [Delete a specific local media](#delete-a-specific-local-media)
* [Delete local media by date or size](#delete-local-media-by-date-or-size)
+ * [Delete media uploaded by a user](#delete-media-uploaded-by-a-user)
- [Purge Remote Media API](#purge-remote-media-api)
# Querying media
@@ -47,7 +48,8 @@ The API returns a JSON body like the following:
## List all media uploaded by a user
Listing all media that has been uploaded by a local user can be achieved through
-the use of the [List media of a user](user_admin_api.md#list-media-of-a-user)
+the use of the
+[List media uploaded by a user](user_admin_api.md#list-media-uploaded-by-a-user)
Admin API.
# Quarantine media
@@ -281,6 +283,11 @@ The following fields are returned in the JSON response body:
* `deleted_media`: an array of strings - List of deleted `media_id`
* `total`: integer - Total number of deleted `media_id`
+## Delete media uploaded by a user
+
+You can find details of how to delete multiple media uploaded by a user in
+[User Admin API](user_admin_api.md#delete-media-uploaded-by-a-user).
+
# Purge Remote Media API
The purge remote media API allows server admins to purge old cached remote media.
diff --git a/docs/admin_api/user_admin_api.md b/docs/admin_api/user_admin_api.md
index 160899754e..6a9335d6ec 100644
--- a/docs/admin_api/user_admin_api.md
+++ b/docs/admin_api/user_admin_api.md
@@ -81,6 +81,16 @@ with a body of:
"address": "<user_mail_2>"
}
],
+ "external_ids": [
+ {
+ "auth_provider": "<provider1>",
+ "external_id": "<user_id_provider_1>"
+ },
+ {
+ "auth_provider": "<provider2>",
+ "external_id": "<user_id_provider_2>"
+ }
+ ],
"avatar_url": "<avatar_url>",
"admin": false,
"deactivated": false
@@ -90,26 +100,34 @@ with a body of:
To use it, you will need to authenticate by providing an `access_token` for a
server admin: [Admin API](../usage/administration/admin_api)
+Returns HTTP status code:
+- `201` - When a new user object was created.
+- `200` - When a user was modified.
+
URL parameters:
- `user_id`: fully-qualified user id: for example, `@user:server.com`.
Body parameters:
-- `password`, optional. If provided, the user's password is updated and all
+- `password` - string, optional. If provided, the user's password is updated and all
devices are logged out.
-
-- `displayname`, optional, defaults to the value of `user_id`.
-
-- `threepids`, optional, allows setting the third-party IDs (email, msisdn)
+- `displayname` - string, optional, defaults to the value of `user_id`.
+- `threepids` - array, optional, allows setting the third-party IDs (email, msisdn)
+ - `medium` - string. Kind of third-party ID, either `email` or `msisdn`.
+ - `address` - string. Value of third-party ID.
belonging to a user.
-
-- `avatar_url`, optional, must be a
+- `external_ids` - array, optional. Allow setting the identifier of the external identity
+ provider for SSO (Single sign-on). Details in
+ [Sample Configuration File](../usage/configuration/homeserver_sample_config.html)
+ section `sso` and `oidc_providers`.
+ - `auth_provider` - string. ID of the external identity provider. Value of `idp_id`
+ in homeserver configuration.
+ - `external_id` - string, user ID in the external identity provider.
+- `avatar_url` - string, optional, must be a
[MXC URI](https://matrix.org/docs/spec/client_server/r0.6.0#matrix-content-mxc-uris).
-
-- `admin`, optional, defaults to `false`.
-
-- `deactivated`, optional. If unspecified, deactivation state will be left
+- `admin` - bool, optional, defaults to `false`.
+- `deactivated` - bool, optional. If unspecified, deactivation state will be left
unchanged on existing accounts and set to `false` for new accounts.
A user cannot be erased by deactivating with this API. For details on
deactivating users see [Deactivate Account](#deactivate-account).
@@ -443,8 +461,9 @@ The following fields are returned in the JSON response body:
- `joined_rooms` - An array of `room_id`.
- `total` - Number of rooms.
+## User media
-## List media of a user
+### List media uploaded by a user
Gets a list of all local media that a specific `user_id` has created.
By default, the response is ordered by descending creation date and ascending media ID.
The newest media is on top. You can change the order with parameters
@@ -543,7 +562,6 @@ The following fields are returned in the JSON response body:
- `media` - An array of objects, each containing information about a media.
Media objects contain the following fields:
-
- `created_ts` - integer - Timestamp when the content was uploaded in ms.
- `last_access_ts` - integer - Timestamp when the content was last accessed in ms.
- `media_id` - string - The id used to refer to the media.
@@ -551,13 +569,58 @@ The following fields are returned in the JSON response body:
- `media_type` - string - The MIME-type of the media.
- `quarantined_by` - string - The user ID that initiated the quarantine request
for this media.
-
- `safe_from_quarantine` - bool - Status if this media is safe from quarantining.
- `upload_name` - string - The name the media was uploaded with.
-
- `next_token`: integer - Indication for pagination. See above.
- `total` - integer - Total number of media.
+### Delete media uploaded by a user
+
+This API deletes the *local* media from the disk of your own server
+that a specific `user_id` has created. This includes any local thumbnails.
+
+This API will not affect media that has been uploaded to external
+media repositories (e.g https://github.com/turt2live/matrix-media-repo/).
+
+By default, the API deletes media ordered by descending creation date and ascending media ID.
+The newest media is deleted first. You can change the order with parameters
+`order_by` and `dir`. If no `limit` is set the API deletes `100` files per request.
+
+The API is:
+
+```
+DELETE /_synapse/admin/v1/users/<user_id>/media
+```
+
+To use it, you will need to authenticate by providing an `access_token` for a
+server admin: [Admin API](../usage/administration/admin_api)
+
+A response body like the following is returned:
+
+```json
+{
+ "deleted_media": [
+ "abcdefghijklmnopqrstuvwx"
+ ],
+ "total": 1
+}
+```
+
+The following fields are returned in the JSON response body:
+
+* `deleted_media`: an array of strings - List of deleted `media_id`
+* `total`: integer - Total number of deleted `media_id`
+
+**Note**: There is no `next_token`. This is not useful for deleting media, because
+after deleting media the remaining media have a new order.
+
+**Parameters**
+
+This API has the same parameters as
+[List media uploaded by a user](#list-media-uploaded-by-a-user).
+With the parameters you can for example limit the number of files to delete at once or
+delete largest/smallest or newest/oldest files first.
+
## Login as a user
Get an access token that can be used to authenticate as that user. Useful for
@@ -1013,3 +1076,22 @@ The following parameters should be set in the URL:
- `user_id` - The fully qualified MXID: for example, `@user:server.com`. The user must
be local.
+### Check username availability
+
+Checks to see if a username is available, and valid, for the server. See [the client-server
+API](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available)
+for more information.
+
+This endpoint will work even if registration is disabled on the server, unlike
+`/_matrix/client/r0/register/available`.
+
+The API is:
+
+```
+POST /_synapse/admin/v1/username_availabile?username=$localpart
+```
+
+The request and response format is the same as the [/_matrix/client/r0/register/available](https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available) API.
+
+To use it, you will need to authenticate by providing an `access_token` for a
+server admin: [Admin API](../usage/administration/admin_api)
diff --git a/docs/manhole.md b/docs/manhole.md
index 37d1d7823c..db92df88dc 100644
--- a/docs/manhole.md
+++ b/docs/manhole.md
@@ -67,7 +67,7 @@ This gives a Python REPL in which `hs` gives access to the
`synapse.server.HomeServer` object - which in turn gives access to many other
parts of the process.
-Note that any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
+Note that, prior to Synapse 1.41, any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
As a simple example, retrieving an event from the database:
diff --git a/docs/reverse_proxy.md b/docs/reverse_proxy.md
index 76bb45aff2..5f8d20129e 100644
--- a/docs/reverse_proxy.md
+++ b/docs/reverse_proxy.md
@@ -33,6 +33,19 @@ Let's assume that we expect clients to connect to our server at
`https://example.com:8448`. The following sections detail the configuration of
the reverse proxy and the homeserver.
+
+## Homeserver Configuration
+
+The HTTP configuration will need to be updated for Synapse to correctly record
+client IP addresses and generate redirect URLs while behind a reverse proxy.
+
+In `homeserver.yaml` set `x_forwarded: true` in the port 8008 section and
+consider setting `bind_addresses: ['127.0.0.1']` so that the server only
+listens to traffic on localhost. (Do not change `bind_addresses` to `127.0.0.1`
+when using a containerized Synapse, as that will prevent it from responding
+to proxied traffic.)
+
+
## Reverse-proxy configuration examples
**NOTE**: You only need one of these.
@@ -239,16 +252,6 @@ relay "matrix_federation" {
}
```
-## Homeserver Configuration
-
-You will also want to set `bind_addresses: ['127.0.0.1']` and
-`x_forwarded: true` for port 8008 in `homeserver.yaml` to ensure that
-client IP addresses are recorded correctly.
-
-Having done so, you can then use `https://matrix.example.com` (instead
-of `https://matrix.example.com:8448`) as the "Custom server" when
-connecting to Synapse from a client.
-
## Health check endpoint
diff --git a/docs/sample_config.yaml b/docs/sample_config.yaml
index 1a217f35db..3ec76d5abf 100644
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@@ -210,6 +210,8 @@ presence:
#
# This option replaces federation_ip_range_blacklist in Synapse v1.25.0.
#
+# Note: The value is ignored when an HTTP proxy is in use
+#
#ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -563,6 +565,19 @@ retention:
#
#next_link_domain_whitelist: ["matrix.org"]
+# Templates to use when generating email or HTML page contents.
+#
+templates:
+ # Directory in which Synapse will try to find template files to use to generate
+ # email or HTML page contents.
+ # If not set, or a file is not found within the template directory, a default
+ # template from within the Synapse package will be used.
+ #
+ # See https://matrix-org.github.io/synapse/latest/templates.html for more
+ # information about using custom templates.
+ #
+ #custom_template_directory: /path/to/custom/templates/
+
## TLS ##
@@ -711,6 +726,15 @@ caches:
#
#expiry_time: 30m
+ # Controls how long the results of a /sync request are cached for after
+ # a successful response is returned. A higher duration can help clients with
+ # intermittent connections, at the cost of higher memory usage.
+ #
+ # By default, this is zero, which means that sync responses are not cached
+ # at all.
+ #
+ #sync_response_cache_duration: 2m
+
## Database ##
@@ -963,6 +987,8 @@ media_store_path: "DATADIR/media_store"
# This must be specified if url_preview_enabled is set. It is recommended that
# you uncomment the following list as a starting point.
#
+# Note: The value is ignored when an HTTP proxy is in use
+#
#url_preview_ip_range_blacklist:
# - '127.0.0.0/8'
# - '10.0.0.0/8'
@@ -1882,6 +1908,9 @@ cas_config:
# Additional settings to use with single-sign on systems such as OpenID Connect,
# SAML2 and CAS.
#
+# Server admins can configure custom templates for pages related to SSO. See
+# https://matrix-org.github.io/synapse/latest/templates.html for more information.
+#
sso:
# A list of client URLs which are whitelisted so that the user does not
# have to confirm giving access to their account to the URL. Any client
@@ -1914,169 +1943,6 @@ sso:
#
#update_profile_information: true
- # Directory in which Synapse will try to find the template files below.
- # If not set, or the files named below are not found within the template
- # directory, default templates from within the Synapse package will be used.
- #
- # Synapse will look for the following templates in this directory:
- #
- # * HTML page to prompt the user to choose an Identity Provider during
- # login: 'sso_login_idp_picker.html'.
- #
- # This is only used if multiple SSO Identity Providers are configured.
- #
- # When rendering, this template is given the following variables:
- # * redirect_url: the URL that the user will be redirected to after
- # login.
- #
- # * server_name: the homeserver's name.
- #
- # * providers: a list of available Identity Providers. Each element is
- # an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # The rendered HTML page should contain a form which submits its results
- # back as a GET request, with the following query parameters:
- #
- # * redirectUrl: the client redirect URI (ie, the `redirect_url` passed
- # to the template)
- #
- # * idp: the 'idp_id' of the chosen IDP.
- #
- # * HTML page to prompt new users to enter a userid and confirm other
- # details: 'sso_auth_account_details.html'. This is only shown if the
- # SSO implementation (with any user_mapping_provider) does not return
- # a localpart.
- #
- # When rendering, this template is given the following variables:
- #
- # * server_name: the homeserver's name.
- #
- # * idp: details of the SSO Identity Provider that the user logged in
- # with: an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # * user_attributes: an object containing details about the user that
- # we received from the IdP. May have the following attributes:
- #
- # * display_name: the user's display_name
- # * emails: a list of email addresses
- #
- # The template should render a form which submits the following fields:
- #
- # * username: the localpart of the user's chosen user id
- #
- # * HTML page allowing the user to consent to the server's terms and
- # conditions. This is only shown for new users, and only if
- # `user_consent.require_at_registration` is set.
- #
- # When rendering, this template is given the following variables:
- #
- # * server_name: the homeserver's name.
- #
- # * user_id: the user's matrix proposed ID.
- #
- # * user_profile.display_name: the user's proposed display name, if any.
- #
- # * consent_version: the version of the terms that the user will be
- # shown
- #
- # * terms_url: a link to the page showing the terms.
- #
- # The template should render a form which submits the following fields:
- #
- # * accepted_version: the version of the terms accepted by the user
- # (ie, 'consent_version' from the input variables).
- #
- # * HTML page for a confirmation step before redirecting back to the client
- # with the login token: 'sso_redirect_confirm.html'.
- #
- # When rendering, this template is given the following variables:
- #
- # * redirect_url: the URL the user is about to be redirected to.
- #
- # * display_url: the same as `redirect_url`, but with the query
- # parameters stripped. The intention is to have a
- # human-readable URL to show to users, not to use it as
- # the final address to redirect to.
- #
- # * server_name: the homeserver's name.
- #
- # * new_user: a boolean indicating whether this is the user's first time
- # logging in.
- #
- # * user_id: the user's matrix ID.
- #
- # * user_profile.avatar_url: an MXC URI for the user's avatar, if any.
- # None if the user has not set an avatar.
- #
- # * user_profile.display_name: the user's display name. None if the user
- # has not set a display name.
- #
- # * HTML page which notifies the user that they are authenticating to confirm
- # an operation on their account during the user interactive authentication
- # process: 'sso_auth_confirm.html'.
- #
- # When rendering, this template is given the following variables:
- # * redirect_url: the URL the user is about to be redirected to.
- #
- # * description: the operation which the user is being asked to confirm
- #
- # * idp: details of the Identity Provider that we will use to confirm
- # the user's identity: an object with the following attributes:
- #
- # * idp_id: unique identifier for the IdP
- # * idp_name: user-facing name for the IdP
- # * idp_icon: if specified in the IdP config, an MXC URI for an icon
- # for the IdP
- # * idp_brand: if specified in the IdP config, a textual identifier
- # for the brand of the IdP
- #
- # * HTML page shown after a successful user interactive authentication session:
- # 'sso_auth_success.html'.
- #
- # Note that this page must include the JavaScript which notifies of a successful authentication
- # (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
- #
- # This template has no additional variables.
- #
- # * HTML page shown after a user-interactive authentication session which
- # does not map correctly onto the expected user: 'sso_auth_bad_user.html'.
- #
- # When rendering, this template is given the following variables:
- # * server_name: the homeserver's name.
- # * user_id_to_verify: the MXID of the user that we are trying to
- # validate.
- #
- # * HTML page shown during single sign-on if a deactivated user (according to Synapse's database)
- # attempts to login: 'sso_account_deactivated.html'.
- #
- # This template has no additional variables.
- #
- # * HTML page to display to users if something goes wrong during the
- # OpenID Connect authentication process: 'sso_error.html'.
- #
- # When rendering, this template is given two variables:
- # * error: the technical name of the error
- # * error_description: a human-readable message for the error
- #
- # You can see the default templates at:
- # https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
- #
- #template_dir: "res/templates"
-
# JSON web token integration. The following settings can be used to make
# Synapse JSON web tokens for authentication, instead of its internal
@@ -2207,6 +2073,9 @@ ui_auth:
# Configuration for sending emails from Synapse.
#
+# Server admins can configure custom templates for email content. See
+# https://matrix-org.github.io/synapse/latest/templates.html for more information.
+#
email:
# The hostname of the outgoing SMTP server to use. Defaults to 'localhost'.
#
@@ -2229,6 +2098,14 @@ email:
#
#require_transport_security: true
+ # Uncomment the following to disable TLS for SMTP.
+ #
+ # By default, if the server supports TLS, it will be used, and the server
+ # must present a certificate that is valid for 'smtp_host'. If this option
+ # is set to false, TLS will not be used.
+ #
+ #enable_tls: false
+
# notif_from defines the "From" address to use when sending emails.
# It must be set if email sending is enabled.
#
@@ -2275,49 +2152,6 @@ email:
#
#invite_client_location: https://app.element.io
- # Directory in which Synapse will try to find the template files below.
- # If not set, or the files named below are not found within the template
- # directory, default templates from within the Synapse package will be used.
- #
- # Synapse will look for the following templates in this directory:
- #
- # * The contents of email notifications of missed events: 'notif_mail.html' and
- # 'notif_mail.txt'.
- #
- # * The contents of account expiry notice emails: 'notice_expiry.html' and
- # 'notice_expiry.txt'.
- #
- # * The contents of password reset emails sent by the homeserver:
- # 'password_reset.html' and 'password_reset.txt'
- #
- # * An HTML page that a user will see when they follow the link in the password
- # reset email. The user will be asked to confirm the action before their
- # password is reset: 'password_reset_confirmation.html'
- #
- # * HTML pages for success and failure that a user will see when they confirm
- # the password reset flow using the page above: 'password_reset_success.html'
- # and 'password_reset_failure.html'
- #
- # * The contents of address verification emails sent during registration:
- # 'registration.html' and 'registration.txt'
- #
- # * HTML pages for success and failure that a user will see when they follow
- # the link in an address verification email sent during registration:
- # 'registration_success.html' and 'registration_failure.html'
- #
- # * The contents of address verification emails sent when an address is added
- # to a Matrix account: 'add_threepid.html' and 'add_threepid.txt'
- #
- # * HTML pages for success and failure that a user will see when they follow
- # the link in an address verification email sent when an address is added
- # to a Matrix account: 'add_threepid_success.html' and
- # 'add_threepid_failure.html'
- #
- # You can see the default templates at:
- # https://github.com/matrix-org/synapse/tree/master/synapse/res/templates
- #
- #template_dir: "res/templates"
-
# Subjects to use when sending emails from Synapse.
#
# The placeholder '%(app)s' will be replaced with the value of the 'app_name'
diff --git a/docs/sample_log_config.yaml b/docs/sample_log_config.yaml
index 669e600081..2485ad25ed 100644
--- a/docs/sample_log_config.yaml
+++ b/docs/sample_log_config.yaml
@@ -24,18 +24,31 @@ handlers:
backupCount: 3 # Does not include the current log file.
encoding: utf8
- # Default to buffering writes to log file for efficiency. This means that
- # will be a delay for INFO/DEBUG logs to get written, but WARNING/ERROR
- # logs will still be flushed immediately.
+ # Default to buffering writes to log file for efficiency.
+ # WARNING/ERROR logs will still be flushed immediately, but there will be a
+ # delay (of up to `period` seconds, or until the buffer is full with
+ # `capacity` messages) before INFO/DEBUG logs get written.
buffer:
- class: logging.handlers.MemoryHandler
+ class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
- # The capacity is the number of log lines that are buffered before
- # being written to disk. Increasing this will lead to better
+
+ # The capacity is the maximum number of log lines that are buffered
+ # before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
+ # This parameter is required.
capacity: 10
- flushLevel: 30 # Flush for WARNING logs as well
+
+ # Logs with a level at or above the flush level will cause the buffer to
+ # be flushed immediately.
+ # Default value: 40 (ERROR)
+ # Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
+ flushLevel: 30 # Flush immediately for WARNING logs and higher
+
+ # The period of time, in seconds, between forced flushes.
+ # Messages will not be delayed for longer than this time.
+ # Default value: 5 seconds
+ period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
diff --git a/docs/setup/forward_proxy.md b/docs/setup/forward_proxy.md
new file mode 100644
index 0000000000..494c14893b
--- /dev/null
+++ b/docs/setup/forward_proxy.md
@@ -0,0 +1,74 @@
+# Using a forward proxy with Synapse
+
+You can use Synapse with a forward or outbound proxy. An example of when
+this is necessary is in corporate environments behind a DMZ (demilitarized zone).
+Synapse supports routing outbound HTTP(S) requests via a proxy. Only HTTP(S)
+proxy is supported, not SOCKS proxy or anything else.
+
+## Configure
+
+The `http_proxy`, `https_proxy`, `no_proxy` environment variables are used to
+specify proxy settings. The environment variable is not case sensitive.
+- `http_proxy`: Proxy server to use for HTTP requests.
+- `https_proxy`: Proxy server to use for HTTPS requests.
+- `no_proxy`: Comma-separated list of hosts, IP addresses, or IP ranges in CIDR
+ format which should not use the proxy. Synapse will directly connect to these hosts.
+
+The `http_proxy` and `https_proxy` environment variables have the form: `[scheme://][<username>:<password>@]<host>[:<port>]`
+- Supported schemes are `http://` and `https://`. The default scheme is `http://`
+ for compatibility reasons; it is recommended to set a scheme. If scheme is set
+ to `https://` the connection uses TLS between Synapse and the proxy.
+
+ **NOTE**: Synapse validates the certificates. If the certificate is not
+ valid, then the connection is dropped.
+- Default port if not given is `1080`.
+- Username and password are optional and will be used to authenticate against
+ the proxy.
+
+**Examples**
+- HTTP_PROXY=http://USERNAME:PASSWORD@10.0.1.1:8080/
+- HTTPS_PROXY=http://USERNAME:PASSWORD@proxy.example.com:8080/
+- NO_PROXY=master.hostname.example.com,10.1.0.0/16,172.30.0.0/16
+
+**NOTE**:
+Synapse does not apply the IP blacklist to connections through the proxy (since
+the DNS resolution is done by the proxy). It is expected that the proxy or firewall
+will apply blacklisting of IP addresses.
+
+## Connection types
+
+The proxy will be **used** for:
+
+- push
+- url previews
+- phone-home stats
+- recaptcha validation
+- CAS auth validation
+- OpenID Connect
+- Outbound federation
+- Federation (checking public key revocation)
+- Fetching public keys of other servers
+- Downloading remote media
+
+It will **not be used** for:
+
+- Application Services
+- Identity servers
+- In worker configurations
+ - connections between workers
+ - connections from workers to Redis
+
+## Troubleshooting
+
+If a proxy server is used with TLS (HTTPS) and no connections are established,
+it is most likely due to the proxy's certificates. To test this, the validation
+in Synapse can be deactivated.
+
+**NOTE**: This has an impact on security and is for testing purposes only!
+
+To deactivate the certificate validation, the following setting must be made in
+[homserver.yaml](../usage/configuration/homeserver_sample_config.md).
+
+```yaml
+use_insecure_ssl_client_just_for_testing_do_not_use: true
+```
diff --git a/docs/templates.md b/docs/templates.md
new file mode 100644
index 0000000000..a240f58b54
--- /dev/null
+++ b/docs/templates.md
@@ -0,0 +1,239 @@
+# Templates
+
+Synapse uses parametrised templates to generate the content of emails it sends and
+webpages it shows to users.
+
+By default, Synapse will use the templates listed [here](https://github.com/matrix-org/synapse/tree/master/synapse/res/templates).
+Server admins can configure an additional directory for Synapse to look for templates
+in, allowing them to specify custom templates:
+
+```yaml
+templates:
+ custom_templates_directory: /path/to/custom/templates/
+```
+
+If this setting is not set, or the files named below are not found within the directory,
+default templates from within the Synapse package will be used.
+
+Templates that are given variables when being rendered are rendered using [Jinja 2](https://jinja.palletsprojects.com/en/2.11.x/).
+Templates rendered by Jinja 2 can also access two functions on top of the functions
+already available as part of Jinja 2:
+
+```python
+format_ts(value: int, format: str) -> str
+```
+
+Formats a timestamp in milliseconds.
+
+Example: `reason.last_sent_ts|format_ts("%c")`
+
+```python
+mxc_to_http(value: str, width: int, height: int, resize_method: str = "crop") -> str
+```
+
+Turns a `mxc://` URL for media content into an HTTP(S) one using the homeserver's
+`public_baseurl` configuration setting as the URL's base.
+
+Example: `message.sender_avatar_url|mxc_to_http(32,32)`
+
+
+## Email templates
+
+Below are the templates Synapse will look for when generating the content of an email:
+
+* `notif_mail.html` and `notif_mail.txt`: The contents of email notifications of missed
+ events.
+ When rendering, this template is given the following variables:
+ * `user_display_name`: the display name for the user receiving the notification
+ * `unsubscribe_link`: the link users can click to unsubscribe from email notifications
+ * `summary_text`: a summary of the notification(s). The text used can be customised
+ by configuring the various settings in the `email.subjects` section of the
+ configuration file.
+ * `rooms`: a list of rooms containing events to include in the email. Each element is
+ an object with the following attributes:
+ * `title`: a human-readable name for the room
+ * `hash`: a hash of the ID of the room
+ * `invite`: a boolean, which is `True` if the room is an invite the user hasn't
+ accepted yet, `False` otherwise
+ * `notifs`: a list of events, or an empty list if `invite` is `True`. Each element
+ is an object with the following attributes:
+ * `link`: a `matrix.to` link to the event
+ * `ts`: the time in milliseconds at which the event was received
+ * `messages`: a list of messages containing one message before the event, the
+ message in the event, and one message after the event. Each element is an
+ object with the following attributes:
+ * `event_type`: the type of the event
+ * `is_historical`: a boolean, which is `False` if the message is the one
+ that triggered the notification, `True` otherwise
+ * `id`: the ID of the event
+ * `ts`: the time in milliseconds at which the event was sent
+ * `sender_name`: the display name for the event's sender
+ * `sender_avatar_url`: the avatar URL (as a `mxc://` URL) for the event's
+ sender
+ * `sender_hash`: a hash of the user ID of the sender
+ * `link`: a `matrix.to` link to the room
+ * `reason`: information on the event that triggered the email to be sent. It's an
+ object with the following attributes:
+ * `room_id`: the ID of the room the event was sent in
+ * `room_name`: a human-readable name for the room the event was sent in
+ * `now`: the current time in milliseconds
+ * `received_at`: the time in milliseconds at which the event was received
+ * `delay_before_mail_ms`: the amount of time in milliseconds Synapse always waits
+ before ever emailing about a notification (to give the user a chance to respond
+ to other push or notice the window)
+ * `last_sent_ts`: the time in milliseconds at which a notification was last sent
+ for an event in this room
+ * `throttle_ms`: the minimum amount of time in milliseconds between two
+ notifications can be sent for this room
+* `password_reset.html` and `password_reset.txt`: The contents of password reset emails
+ sent by the homeserver.
+ When rendering, these templates are given a `link` variable which contains the link the
+ user must click in order to reset their password.
+* `registration.html` and `registration.txt`: The contents of address verification emails
+ sent during registration.
+ When rendering, these templates are given a `link` variable which contains the link the
+ user must click in order to validate their email address.
+* `add_threepid.html` and `add_threepid.txt`: The contents of address verification emails
+ sent when an address is added to a Matrix account.
+ When rendering, these templates are given a `link` variable which contains the link the
+ user must click in order to validate their email address.
+
+
+## HTML page templates for registration and password reset
+
+Below are the templates Synapse will look for when generating pages related to
+registration and password reset:
+
+* `password_reset_confirmation.html`: An HTML page that a user will see when they follow
+ the link in the password reset email. The user will be asked to confirm the action
+ before their password is reset.
+ When rendering, this template is given the following variables:
+ * `sid`: the session ID for the password reset
+ * `token`: the token for the password reset
+ * `client_secret`: the client secret for the password reset
+* `password_reset_success.html` and `password_reset_failure.html`: HTML pages for success
+ and failure that a user will see when they confirm the password reset flow using the
+ page above.
+ When rendering, `password_reset_success.html` is given no variable, and
+ `password_reset_failure.html` is given a `failure_reason`, which contains the reason
+ for the password reset failure.
+* `registration_success.html` and `registration_failure.html`: HTML pages for success and
+ failure that a user will see when they follow the link in an address verification email
+ sent during registration.
+ When rendering, `registration_success.html` is given no variable, and
+ `registration_failure.html` is given a `failure_reason`, which contains the reason
+ for the registration failure.
+* `add_threepid_success.html` and `add_threepid_failure.html`: HTML pages for success and
+ failure that a user will see when they follow the link in an address verification email
+ sent when an address is added to a Matrix account.
+ When rendering, `add_threepid_success.html` is given no variable, and
+ `add_threepid_failure.html` is given a `failure_reason`, which contains the reason
+ for the registration failure.
+
+
+## HTML page templates for Single Sign-On (SSO)
+
+Below are the templates Synapse will look for when generating pages related to SSO:
+
+* `sso_login_idp_picker.html`: HTML page to prompt the user to choose an
+ Identity Provider during login.
+ This is only used if multiple SSO Identity Providers are configured.
+ When rendering, this template is given the following variables:
+ * `redirect_url`: the URL that the user will be redirected to after
+ login.
+ * `server_name`: the homeserver's name.
+ * `providers`: a list of available Identity Providers. Each element is
+ an object with the following attributes:
+ * `idp_id`: unique identifier for the IdP
+ * `idp_name`: user-facing name for the IdP
+ * `idp_icon`: if specified in the IdP config, an MXC URI for an icon
+ for the IdP
+ * `idp_brand`: if specified in the IdP config, a textual identifier
+ for the brand of the IdP
+ The rendered HTML page should contain a form which submits its results
+ back as a GET request, with the following query parameters:
+ * `redirectUrl`: the client redirect URI (ie, the `redirect_url` passed
+ to the template)
+ * `idp`: the 'idp_id' of the chosen IDP.
+* `sso_auth_account_details.html`: HTML page to prompt new users to enter a
+ userid and confirm other details. This is only shown if the
+ SSO implementation (with any `user_mapping_provider`) does not return
+ a localpart.
+ When rendering, this template is given the following variables:
+ * `server_name`: the homeserver's name.
+ * `idp`: details of the SSO Identity Provider that the user logged in
+ with: an object with the following attributes:
+ * `idp_id`: unique identifier for the IdP
+ * `idp_name`: user-facing name for the IdP
+ * `idp_icon`: if specified in the IdP config, an MXC URI for an icon
+ for the IdP
+ * `idp_brand`: if specified in the IdP config, a textual identifier
+ for the brand of the IdP
+ * `user_attributes`: an object containing details about the user that
+ we received from the IdP. May have the following attributes:
+ * display_name: the user's display_name
+ * emails: a list of email addresses
+ The template should render a form which submits the following fields:
+ * `username`: the localpart of the user's chosen user id
+* `sso_new_user_consent.html`: HTML page allowing the user to consent to the
+ server's terms and conditions. This is only shown for new users, and only if
+ `user_consent.require_at_registration` is set.
+ When rendering, this template is given the following variables:
+ * `server_name`: the homeserver's name.
+ * `user_id`: the user's matrix proposed ID.
+ * `user_profile.display_name`: the user's proposed display name, if any.
+ * consent_version: the version of the terms that the user will be
+ shown
+ * `terms_url`: a link to the page showing the terms.
+ The template should render a form which submits the following fields:
+ * `accepted_version`: the version of the terms accepted by the user
+ (ie, 'consent_version' from the input variables).
+* `sso_redirect_confirm.html`: HTML page for a confirmation step before redirecting back
+ to the client with the login token.
+ When rendering, this template is given the following variables:
+ * `redirect_url`: the URL the user is about to be redirected to.
+ * `display_url`: the same as `redirect_url`, but with the query
+ parameters stripped. The intention is to have a
+ human-readable URL to show to users, not to use it as
+ the final address to redirect to.
+ * `server_name`: the homeserver's name.
+ * `new_user`: a boolean indicating whether this is the user's first time
+ logging in.
+ * `user_id`: the user's matrix ID.
+ * `user_profile.avatar_url`: an MXC URI for the user's avatar, if any.
+ `None` if the user has not set an avatar.
+ * `user_profile.display_name`: the user's display name. `None` if the user
+ has not set a display name.
+* `sso_auth_confirm.html`: HTML page which notifies the user that they are authenticating
+ to confirm an operation on their account during the user interactive authentication
+ process.
+ When rendering, this template is given the following variables:
+ * `redirect_url`: the URL the user is about to be redirected to.
+ * `description`: the operation which the user is being asked to confirm
+ * `idp`: details of the Identity Provider that we will use to confirm
+ the user's identity: an object with the following attributes:
+ * `idp_id`: unique identifier for the IdP
+ * `idp_name`: user-facing name for the IdP
+ * `idp_icon`: if specified in the IdP config, an MXC URI for an icon
+ for the IdP
+ * `idp_brand`: if specified in the IdP config, a textual identifier
+ for the brand of the IdP
+* `sso_auth_success.html`: HTML page shown after a successful user interactive
+ authentication session.
+ Note that this page must include the JavaScript which notifies of a successful
+ authentication (see https://matrix.org/docs/spec/client_server/r0.6.0#fallback).
+ This template has no additional variables.
+* `sso_auth_bad_user.html`: HTML page shown after a user-interactive authentication
+ session which does not map correctly onto the expected user.
+ When rendering, this template is given the following variables:
+ * `server_name`: the homeserver's name.
+ * `user_id_to_verify`: the MXID of the user that we are trying to
+ validate.
+* `sso_account_deactivated.html`: HTML page shown during single sign-on if a deactivated
+ user (according to Synapse's database) attempts to login.
+ This template has no additional variables.
+* `sso_error.html`: HTML page to display to users if something goes wrong during the
+ OpenID Connect authentication process.
+ When rendering, this template is given two variables:
+ * `error`: the technical name of the error
+ * `error_description`: a human-readable message for the error
diff --git a/docs/upgrade.md b/docs/upgrade.md
index ce9167e6de..1c459d8e2b 100644
--- a/docs/upgrade.md
+++ b/docs/upgrade.md
@@ -86,6 +86,44 @@ process, for example:
```
+# Upgrading to v1.xx.0
+
+## Add support for routing outbound HTTP requests via a proxy for federation
+
+Since Synapse 1.6.0 (2019-11-26) you can set a proxy for outbound HTTP requests via
+http_proxy/https_proxy environment variables. This proxy was set for:
+- push
+- url previews
+- phone-home stats
+- recaptcha validation
+- CAS auth validation
+- OpenID Connect
+- Federation (checking public key revocation)
+
+In this version we have added support for outbound requests for:
+- Outbound federation
+- Downloading remote media
+- Fetching public keys of other servers
+
+These requests use the same proxy configuration. If you have a proxy configuration we
+recommend to verify the configuration. It may be necessary to adjust the `no_proxy`
+environment variable.
+
+See [using a forward proxy with Synapse documentation](setup/forward_proxy.md) for
+details.
+
+## Deprecation of `template_dir`
+
+The `template_dir` settings in the `sso`, `account_validity` and `email` sections of the
+configuration file are now deprecated. Server admins should use the new
+`templates.custom_template_directory` setting in the configuration file and use one single
+custom template directory for all aforementioned features. Template file names remain
+unchanged. See [the related documentation](https://matrix-org.github.io/synapse/latest/templates.html)
+for more information and examples.
+
+We plan to remove support for these settings in October 2021.
+
+
# Upgrading to v1.39.0
## Deprecation of the current third-party rules module interface
|
