diff --git a/docs/.sample_config_header.yaml b/docs/.sample_config_header.yaml
index 8c9b31acdb..09e86ca0ca 100644
--- a/docs/.sample_config_header.yaml
+++ b/docs/.sample_config_header.yaml
@@ -8,7 +8,8 @@
#
# It is *not* intended to be copied and used as the basis for a real
# homeserver.yaml. Instead, if you are starting from scratch, please generate
-# a fresh config using Synapse by following the instructions in INSTALL.md.
+# a fresh config using Synapse by following the instructions in
+# https://matrix-org.github.io/synapse/latest/setup/installation.html.
# Configuration options that take a time period can be set using a number
# followed by a letter. Letters have the following meanings:
diff --git a/docs/MSC1711_certificates_FAQ.md b/docs/MSC1711_certificates_FAQ.md
index ce8189d4ed..283f288aaf 100644
--- a/docs/MSC1711_certificates_FAQ.md
+++ b/docs/MSC1711_certificates_FAQ.md
@@ -14,7 +14,7 @@ upgraded, however it may be of use to those with old installs returning to the
project.
If you are setting up a server from scratch you almost certainly should look at
-the [installation guide](../INSTALL.md) instead.
+the [installation guide](setup/installation.md) instead.
## Introduction
The goal of Synapse 0.99.0 is to act as a stepping stone to Synapse 1.0.0. It
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 98969bdd2d..db4ef1a44e 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -11,7 +11,7 @@
- [Delegation](delegate.md)
# Upgrading
- - [Upgrading between Synapse Versions](upgrading/README.md)
+ - [Upgrading between Synapse Versions](upgrade.md)
- [Upgrading from pre-Synapse 1.0](MSC1711_certificates_FAQ.md)
# Usage
diff --git a/docs/admin_api/user_admin_api.md b/docs/admin_api/user_admin_api.md
index ef1e735e33..4a65d0c3bc 100644
--- a/docs/admin_api/user_admin_api.md
+++ b/docs/admin_api/user_admin_api.md
@@ -36,7 +36,17 @@ It returns a JSON body like the following:
"creation_ts": 1560432506,
"appservice_id": null,
"consent_server_notice_sent": null,
- "consent_version": null
+ "consent_version": null,
+ "external_ids": [
+ {
+ "auth_provider": "<provider1>",
+ "external_id": "<user_id_provider_1>"
+ },
+ {
+ "auth_provider": "<provider2>",
+ "external_id": "<user_id_provider_2>"
+ }
+ ]
}
```
diff --git a/docs/modules.md b/docs/modules.md
index 3a9fab61b8..bec1c06d15 100644
--- a/docs/modules.md
+++ b/docs/modules.md
@@ -194,7 +194,7 @@ In order to port a module that uses Synapse's old module interface, its author n
* ensure the module's callbacks are all asynchronous.
* register their callbacks using one or more of the `register_[...]_callbacks` methods
- from the `ModuleApi` class in the module's `__init__` method (see [this section](#registering-a-web-resource)
+ from the `ModuleApi` class in the module's `__init__` method (see [this section](#registering-a-callback)
for more info).
Additionally, if the module is packaged with an additional web resource, the module
diff --git a/docs/postgres.md b/docs/postgres.md
index f83155e52a..2c0a5b803a 100644
--- a/docs/postgres.md
+++ b/docs/postgres.md
@@ -8,14 +8,14 @@ Synapse will require the python postgres client library in order to
connect to a postgres database.
- If you are using the [matrix.org debian/ubuntu
- packages](../INSTALL.md#matrixorg-packages), the necessary python
+ packages](setup/installation.md#matrixorg-packages), the necessary python
library will already be installed, but you will need to ensure the
low-level postgres library is installed, which you can do with
`apt install libpq5`.
- For other pre-built packages, please consult the documentation from
the relevant package.
- If you installed synapse [in a
- virtualenv](../INSTALL.md#installing-from-source), you can install
+ virtualenv](setup/installation.md#installing-from-source), you can install
the library with:
~/synapse/env/bin/pip install "matrix-synapse[postgres]"
diff --git a/docs/presence_router_module.md b/docs/presence_router_module.md
index bf859e4254..4a3e720240 100644
--- a/docs/presence_router_module.md
+++ b/docs/presence_router_module.md
@@ -222,7 +222,9 @@ Synapse, amend your homeserver config file with the following.
```yaml
presence:
- routing_module:
+ enabled: true
+
+ presence_router:
module: my_module.ExamplePresenceRouter
config:
# Any configuration options for your module. The below is an example.
diff --git a/docs/sample_config.yaml b/docs/sample_config.yaml
index f826e4f2e6..0b2ca70b53 100644
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@@ -8,7 +8,8 @@
#
# It is *not* intended to be copied and used as the basis for a real
# homeserver.yaml. Instead, if you are starting from scratch, please generate
-# a fresh config using Synapse by following the instructions in INSTALL.md.
+# a fresh config using Synapse by following the instructions in
+# https://matrix-org.github.io/synapse/latest/setup/installation.html.
# Configuration options that take a time period can be set using a number
# followed by a letter. Letters have the following meanings:
@@ -36,7 +37,7 @@
# Server admins can expand Synapse's functionality with external modules.
#
-# See https://matrix-org.github.io/synapse/develop/modules.html for more
+# See https://matrix-org.github.io/synapse/latest/modules.html for more
# documentation on how to configure or create custom modules for Synapse.
#
modules:
@@ -58,7 +59,7 @@ modules:
# In most cases you should avoid using a matrix specific subdomain such as
# matrix.example.com or synapse.example.com as the server_name for the same
# reasons you wouldn't use user@email.example.com as your email address.
-# See https://github.com/matrix-org/synapse/blob/master/docs/delegate.md
+# See https://matrix-org.github.io/synapse/latest/delegate.html
# for information on how to host Synapse on a subdomain while preserving
# a clean server_name.
#
@@ -253,9 +254,9 @@ presence:
# 'all local interfaces'.
#
# type: the type of listener. Normally 'http', but other valid options are:
-# 'manhole' (see docs/manhole.md),
-# 'metrics' (see docs/metrics-howto.md),
-# 'replication' (see docs/workers.md).
+# 'manhole' (see https://matrix-org.github.io/synapse/latest/manhole.html),
+# 'metrics' (see https://matrix-org.github.io/synapse/latest/metrics-howto.html),
+# 'replication' (see https://matrix-org.github.io/synapse/latest/workers.html).
#
# tls: set to true to enable TLS for this listener. Will use the TLS
# key/cert specified in tls_private_key_path / tls_certificate_path.
@@ -280,8 +281,8 @@ presence:
# client: the client-server API (/_matrix/client), and the synapse admin
# API (/_synapse/admin). Also implies 'media' and 'static'.
#
-# consent: user consent forms (/_matrix/consent). See
-# docs/consent_tracking.md.
+# consent: user consent forms (/_matrix/consent).
+# See https://matrix-org.github.io/synapse/latest/consent_tracking.html.
#
# federation: the server-server API (/_matrix/federation). Also implies
# 'media', 'keys', 'openid'
@@ -290,12 +291,13 @@ presence:
#
# media: the media API (/_matrix/media).
#
-# metrics: the metrics interface. See docs/metrics-howto.md.
+# metrics: the metrics interface.
+# See https://matrix-org.github.io/synapse/latest/metrics-howto.html.
#
# openid: OpenID authentication.
#
-# replication: the HTTP replication API (/_synapse/replication). See
-# docs/workers.md.
+# replication: the HTTP replication API (/_synapse/replication).
+# See https://matrix-org.github.io/synapse/latest/workers.html.
#
# static: static resources under synapse/static (/_matrix/static). (Mostly
# useful for 'fallback authentication'.)
@@ -319,7 +321,7 @@ listeners:
# that unwraps TLS.
#
# If you plan to use a reverse proxy, please see
- # https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md.
+ # https://matrix-org.github.io/synapse/latest/reverse_proxy.html.
#
- port: 8008
tls: false
@@ -741,35 +743,41 @@ retention:
#event_cache_size: 10K
caches:
- # Controls the global cache factor, which is the default cache factor
- # for all caches if a specific factor for that cache is not otherwise
- # set.
- #
- # This can also be set by the "SYNAPSE_CACHE_FACTOR" environment
- # variable. Setting by environment variable takes priority over
- # setting through the config file.
- #
- # Defaults to 0.5, which will half the size of all caches.
- #
- #global_factor: 1.0
+ # Controls the global cache factor, which is the default cache factor
+ # for all caches if a specific factor for that cache is not otherwise
+ # set.
+ #
+ # This can also be set by the "SYNAPSE_CACHE_FACTOR" environment
+ # variable. Setting by environment variable takes priority over
+ # setting through the config file.
+ #
+ # Defaults to 0.5, which will half the size of all caches.
+ #
+ #global_factor: 1.0
- # A dictionary of cache name to cache factor for that individual
- # cache. Overrides the global cache factor for a given cache.
- #
- # These can also be set through environment variables comprised
- # of "SYNAPSE_CACHE_FACTOR_" + the name of the cache in capital
- # letters and underscores. Setting by environment variable
- # takes priority over setting through the config file.
- # Ex. SYNAPSE_CACHE_FACTOR_GET_USERS_WHO_SHARE_ROOM_WITH_USER=2.0
- #
- # Some caches have '*' and other characters that are not
- # alphanumeric or underscores. These caches can be named with or
- # without the special characters stripped. For example, to specify
- # the cache factor for `*stateGroupCache*` via an environment
- # variable would be `SYNAPSE_CACHE_FACTOR_STATEGROUPCACHE=2.0`.
- #
- per_cache_factors:
- #get_users_who_share_room_with_user: 2.0
+ # A dictionary of cache name to cache factor for that individual
+ # cache. Overrides the global cache factor for a given cache.
+ #
+ # These can also be set through environment variables comprised
+ # of "SYNAPSE_CACHE_FACTOR_" + the name of the cache in capital
+ # letters and underscores. Setting by environment variable
+ # takes priority over setting through the config file.
+ # Ex. SYNAPSE_CACHE_FACTOR_GET_USERS_WHO_SHARE_ROOM_WITH_USER=2.0
+ #
+ # Some caches have '*' and other characters that are not
+ # alphanumeric or underscores. These caches can be named with or
+ # without the special characters stripped. For example, to specify
+ # the cache factor for `*stateGroupCache*` via an environment
+ # variable would be `SYNAPSE_CACHE_FACTOR_STATEGROUPCACHE=2.0`.
+ #
+ per_cache_factors:
+ #get_users_who_share_room_with_user: 2.0
+
+ # Controls how long an entry can be in a cache without having been
+ # accessed before being evicted. Defaults to None, which means
+ # entries are never evicted based on time.
+ #
+ #expiry_time: 30m
## Database ##
@@ -809,7 +817,8 @@ caches:
# cp_min: 5
# cp_max: 10
#
-# For more information on using Synapse with Postgres, see `docs/postgres.md`.
+# For more information on using Synapse with Postgres,
+# see https://matrix-org.github.io/synapse/latest/postgres.html.
#
database:
name: sqlite3
@@ -968,7 +977,7 @@ media_store_path: "DATADIR/media_store"
#
# If you are using a reverse proxy you may also need to set this value in
# your reverse proxy's config. Notably Nginx has a small max body size by default.
-# See https://matrix-org.github.io/synapse/develop/reverse_proxy.html.
+# See https://matrix-org.github.io/synapse/latest/reverse_proxy.html.
#
#max_upload_size: 50M
@@ -2075,7 +2084,7 @@ saml2_config:
#
# module: The class name of a custom mapping module. Default is
# 'synapse.handlers.oidc.JinjaOidcMappingProvider'.
-# See https://github.com/matrix-org/synapse/blob/master/docs/sso_mapping_providers.md#openid-mapping-providers
+# See https://matrix-org.github.io/synapse/latest/sso_mapping_providers.html#openid-mapping-providers
# for information on implementing a custom mapping provider.
#
# config: Configuration for the mapping provider module. This section will
@@ -2126,7 +2135,7 @@ saml2_config:
# - attribute: groups
# value: "admin"
#
-# See https://github.com/matrix-org/synapse/blob/master/docs/openid.md
+# See https://matrix-org.github.io/synapse/latest/openid.html
# for information on how to configure these options.
#
# For backwards compatibility, it is also possible to configure a single OIDC
@@ -2404,7 +2413,7 @@ sso:
# Note that this is a non-standard login type and client support is
# expected to be non-existent.
#
-# See https://github.com/matrix-org/synapse/blob/master/docs/jwt.md.
+# See https://matrix-org.github.io/synapse/latest/jwt.html.
#
#jwt_config:
# Uncomment the following to enable authorization using JSON web
@@ -2704,7 +2713,7 @@ email:
# ex. LDAP, external tokens, etc.
#
# For more information and known implementations, please see
-# https://github.com/matrix-org/synapse/blob/master/docs/password_auth_providers.md
+# https://matrix-org.github.io/synapse/latest/password_auth_providers.html
#
# Note: instances wishing to use SAML or CAS authentication should
# instead use the `saml2_config` or `cas_config` options,
@@ -2806,7 +2815,7 @@ user_directory:
#
# If you set it true, you'll have to rebuild the user_directory search
# indexes, see:
- # https://github.com/matrix-org/synapse/blob/master/docs/user_directory.md
+ # https://matrix-org.github.io/synapse/latest/user_directory.html
#
# Uncomment to return search results containing all known users, even if that
# user does not share a room with the requester.
@@ -2832,7 +2841,7 @@ user_directory:
# User Consent configuration
#
# for detailed instructions, see
-# https://github.com/matrix-org/synapse/blob/master/docs/consent_tracking.md
+# https://matrix-org.github.io/synapse/latest/consent_tracking.html
#
# Parts of this section are required if enabling the 'consent' resource under
# 'listeners', in particular 'template_dir' and 'version'.
@@ -2882,7 +2891,7 @@ user_directory:
# Settings for local room and user statistics collection. See
-# docs/room_and_user_statistics.md.
+# https://matrix-org.github.io/synapse/latest/room_and_user_statistics.html.
#
stats:
# Uncomment the following to disable room and user statistics. Note that doing
@@ -3009,7 +3018,7 @@ opentracing:
#enabled: true
# The list of homeservers we wish to send and receive span contexts and span baggage.
- # See docs/opentracing.rst.
+ # See https://matrix-org.github.io/synapse/latest/opentracing.html.
#
# This is a list of regexes which are matched against the server_name of the
# homeserver.
diff --git a/docs/sample_log_config.yaml b/docs/sample_log_config.yaml
index ff3c747180..669e600081 100644
--- a/docs/sample_log_config.yaml
+++ b/docs/sample_log_config.yaml
@@ -7,7 +7,7 @@
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
-# [2]: https://github.com/matrix-org/synapse/blob/master/docs/structured_logging.md
+# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
diff --git a/docs/setup/installation.md b/docs/setup/installation.md
index 8bb1cffd3d..d041d08333 100644
--- a/docs/setup/installation.md
+++ b/docs/setup/installation.md
@@ -1,7 +1,596 @@
-<!--
- Include the contents of INSTALL.md from the project root without moving it, which may
- break links around the internet. Additionally, note that SUMMARY.md is unable to
- directly link to content outside of the docs/ directory. So we use this file as a
- redirection.
--->
-{{#include ../../INSTALL.md}}
\ No newline at end of file
+# Installation Instructions
+
+There are 3 steps to follow under **Installation Instructions**.
+
+- [Installation Instructions](#installation-instructions)
+ - [Choosing your server name](#choosing-your-server-name)
+ - [Installing Synapse](#installing-synapse)
+ - [Installing from source](#installing-from-source)
+ - [Platform-specific prerequisites](#platform-specific-prerequisites)
+ - [Debian/Ubuntu/Raspbian](#debianubunturaspbian)
+ - [ArchLinux](#archlinux)
+ - [CentOS/Fedora](#centosfedora)
+ - [macOS](#macos)
+ - [OpenSUSE](#opensuse)
+ - [OpenBSD](#openbsd)
+ - [Windows](#windows)
+ - [Prebuilt packages](#prebuilt-packages)
+ - [Docker images and Ansible playbooks](#docker-images-and-ansible-playbooks)
+ - [Debian/Ubuntu](#debianubuntu)
+ - [Matrix.org packages](#matrixorg-packages)
+ - [Downstream Debian packages](#downstream-debian-packages)
+ - [Downstream Ubuntu packages](#downstream-ubuntu-packages)
+ - [Fedora](#fedora)
+ - [OpenSUSE](#opensuse-1)
+ - [SUSE Linux Enterprise Server](#suse-linux-enterprise-server)
+ - [ArchLinux](#archlinux-1)
+ - [Void Linux](#void-linux)
+ - [FreeBSD](#freebsd)
+ - [OpenBSD](#openbsd-1)
+ - [NixOS](#nixos)
+ - [Setting up Synapse](#setting-up-synapse)
+ - [Using PostgreSQL](#using-postgresql)
+ - [TLS certificates](#tls-certificates)
+ - [Client Well-Known URI](#client-well-known-uri)
+ - [Email](#email)
+ - [Registering a user](#registering-a-user)
+ - [Setting up a TURN server](#setting-up-a-turn-server)
+ - [URL previews](#url-previews)
+ - [Troubleshooting Installation](#troubleshooting-installation)
+
+
+## Choosing your server name
+
+It is important to choose the name for your server before you install Synapse,
+because it cannot be changed later.
+
+The server name determines the "domain" part of user-ids for users on your
+server: these will all be of the format `@user:my.domain.name`. It also
+determines how other matrix servers will reach yours for federation.
+
+For a test configuration, set this to the hostname of your server. For a more
+production-ready setup, you will probably want to specify your domain
+(`example.com`) rather than a matrix-specific hostname here (in the same way
+that your email address is probably `user@example.com` rather than
+`user@email.example.com`) - but doing so may require more advanced setup: see
+[Setting up Federation](../federate.md).
+
+## Installing Synapse
+
+### Installing from source
+
+(Prebuilt packages are available for some platforms - see [Prebuilt packages](#prebuilt-packages).)
+
+When installing from source please make sure that the [Platform-specific prerequisites](#platform-specific-prerequisites) are already installed.
+
+System requirements:
+
+- POSIX-compliant system (tested on Linux & OS X)
+- Python 3.5.2 or later, up to Python 3.9.
+- At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
+
+
+To install the Synapse homeserver run:
+
+```sh
+mkdir -p ~/synapse
+virtualenv -p python3 ~/synapse/env
+source ~/synapse/env/bin/activate
+pip install --upgrade pip
+pip install --upgrade setuptools
+pip install matrix-synapse
+```
+
+This will download Synapse from [PyPI](https://pypi.org/project/matrix-synapse)
+and install it, along with the python libraries it uses, into a virtual environment
+under `~/synapse/env`. Feel free to pick a different directory if you
+prefer.
+
+This Synapse installation can then be later upgraded by using pip again with the
+update flag:
+
+```sh
+source ~/synapse/env/bin/activate
+pip install -U matrix-synapse
+```
+
+Before you can start Synapse, you will need to generate a configuration
+file. To do this, run (in your virtualenv, as before):
+
+```sh
+cd ~/synapse
+python -m synapse.app.homeserver \
+ --server-name my.domain.name \
+ --config-path homeserver.yaml \
+ --generate-config \
+ --report-stats=[yes|no]
+```
+
+... substituting an appropriate value for `--server-name`.
+
+This command will generate you a config file that you can then customise, but it will
+also generate a set of keys for you. These keys will allow your homeserver to
+identify itself to other homeserver, so don't lose or delete them. It would be
+wise to back them up somewhere safe. (If, for whatever reason, you do need to
+change your homeserver's keys, you may find that other homeserver have the
+old key cached. If you update the signing key, you should change the name of the
+key in the `<server name>.signing.key` file (the second word) to something
+different. See the [spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys) for more information on key management).
+
+To actually run your new homeserver, pick a working directory for Synapse to
+run (e.g. `~/synapse`), and:
+
+```sh
+cd ~/synapse
+source env/bin/activate
+synctl start
+```
+
+#### Platform-specific prerequisites
+
+Synapse is written in Python but some of the libraries it uses are written in
+C. So before we can install Synapse itself we need a working C compiler and the
+header files for Python C extensions.
+
+##### Debian/Ubuntu/Raspbian
+
+Installing prerequisites on Ubuntu or Debian:
+
+```sh
+sudo apt install build-essential python3-dev libffi-dev \
+ python3-pip python3-setuptools sqlite3 \
+ libssl-dev virtualenv libjpeg-dev libxslt1-dev
+```
+
+##### ArchLinux
+
+Installing prerequisites on ArchLinux:
+
+```sh
+sudo pacman -S base-devel python python-pip \
+ python-setuptools python-virtualenv sqlite3
+```
+
+##### CentOS/Fedora
+
+Installing prerequisites on CentOS or Fedora Linux:
+
+```sh
+sudo dnf install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
+ libwebp-devel libxml2-devel libxslt-devel libpq-devel \
+ python3-virtualenv libffi-devel openssl-devel python3-devel
+sudo dnf groupinstall "Development Tools"
+```
+
+##### macOS
+
+Installing prerequisites on macOS:
+
+```sh
+xcode-select --install
+sudo easy_install pip
+sudo pip install virtualenv
+brew install pkg-config libffi
+```
+
+On macOS Catalina (10.15) you may need to explicitly install OpenSSL
+via brew and inform `pip` about it so that `psycopg2` builds:
+
+```sh
+brew install openssl@1.1
+export LDFLAGS="-L/usr/local/opt/openssl/lib"
+export CPPFLAGS="-I/usr/local/opt/openssl/include"
+```
+
+##### OpenSUSE
+
+Installing prerequisites on openSUSE:
+
+```sh
+sudo zypper in -t pattern devel_basis
+sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
+ python-devel libffi-devel libopenssl-devel libjpeg62-devel
+```
+
+##### OpenBSD
+
+A port of Synapse is available under `net/synapse`. The filesystem
+underlying the homeserver directory (defaults to `/var/synapse`) has to be
+mounted with `wxallowed` (cf. `mount(8)`), so creating a separate filesystem
+and mounting it to `/var/synapse` should be taken into consideration.
+
+To be able to build Synapse's dependency on python the `WRKOBJDIR`
+(cf. `bsd.port.mk(5)`) for building python, too, needs to be on a filesystem
+mounted with `wxallowed` (cf. `mount(8)`).
+
+Creating a `WRKOBJDIR` for building python under `/usr/local` (which on a
+default OpenBSD installation is mounted with `wxallowed`):
+
+```sh
+doas mkdir /usr/local/pobj_wxallowed
+```
+
+Assuming `PORTS_PRIVSEP=Yes` (cf. `bsd.port.mk(5)`) and `SUDO=doas` are
+configured in `/etc/mk.conf`:
+
+```sh
+doas chown _pbuild:_pbuild /usr/local/pobj_wxallowed
+```
+
+Setting the `WRKOBJDIR` for building python:
+
+```sh
+echo WRKOBJDIR_lang/python/3.7=/usr/local/pobj_wxallowed \\nWRKOBJDIR_lang/python/2.7=/usr/local/pobj_wxallowed >> /etc/mk.conf
+```
+
+Building Synapse:
+
+```sh
+cd /usr/ports/net/synapse
+make install
+```
+
+##### Windows
+
+If you wish to run or develop Synapse on Windows, the Windows Subsystem For
+Linux provides a Linux environment on Windows 10 which is capable of using the
+Debian, Fedora, or source installation methods. More information about WSL can
+be found at <https://docs.microsoft.com/en-us/windows/wsl/install-win10> for
+Windows 10 and <https://docs.microsoft.com/en-us/windows/wsl/install-on-server>
+for Windows Server.
+
+### Prebuilt packages
+
+As an alternative to installing from source, prebuilt packages are available
+for a number of platforms.
+
+#### Docker images and Ansible playbooks
+
+There is an official synapse image available at
+<https://hub.docker.com/r/matrixdotorg/synapse> which can be used with
+the docker-compose file available at
+[contrib/docker](https://github.com/matrix-org/synapse/tree/develop/contrib/docker).
+Further information on this including configuration options is available in the README
+on hub.docker.com.
+
+Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
+Dockerfile to automate a synapse server in a single Docker image, at
+<https://hub.docker.com/r/avhost/docker-matrix/tags/>
+
+Slavi Pantaleev has created an Ansible playbook,
+which installs the offical Docker image of Matrix Synapse
+along with many other Matrix-related services (Postgres database, Element, coturn,
+ma1sd, SSL support, etc.).
+For more details, see
+<https://github.com/spantaleev/matrix-docker-ansible-deploy>
+
+#### Debian/Ubuntu
+
+##### Matrix.org packages
+
+Matrix.org provides Debian/Ubuntu packages of the latest stable version of
+Synapse via <https://packages.matrix.org/debian/>. They are available for Debian
+9 (Stretch), Ubuntu 16.04 (Xenial), and later. To use them:
+
+```sh
+sudo apt install -y lsb-release wget apt-transport-https
+sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
+echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
+ sudo tee /etc/apt/sources.list.d/matrix-org.list
+sudo apt update
+sudo apt install matrix-synapse-py3
+```
+
+**Note**: if you followed a previous version of these instructions which
+recommended using `apt-key add` to add an old key from
+`https://matrix.org/packages/debian/`, you should note that this key has been
+revoked. You should remove the old key with `sudo apt-key remove
+C35EB17E1EAE708E6603A9B3AD0592FE47F0DF61`, and follow the above instructions to
+update your configuration.
+
+The fingerprint of the repository signing key (as shown by `gpg
+/usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
+`AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.
+
+##### Downstream Debian packages
+
+We do not recommend using the packages from the default Debian `buster`
+repository at this time, as they are old and suffer from known security
+vulnerabilities. You can install the latest version of Synapse from
+[our repository](#matrixorg-packages) or from `buster-backports`. Please
+see the [Debian documentation](https://backports.debian.org/Instructions/)
+for information on how to use backports.
+
+If you are using Debian `sid` or testing, Synapse is available in the default
+repositories and it should be possible to install it simply with:
+
+```sh
+sudo apt install matrix-synapse
+```
+
+##### Downstream Ubuntu packages
+
+We do not recommend using the packages in the default Ubuntu repository
+at this time, as they are old and suffer from known security vulnerabilities.
+The latest version of Synapse can be installed from [our repository](#matrixorg-packages).
+
+#### Fedora
+
+Synapse is in the Fedora repositories as `matrix-synapse`:
+
+```sh
+sudo dnf install matrix-synapse
+```
+
+Oleg Girko provides Fedora RPMs at
+<https://obs.infoserver.lv/project/monitor/matrix-synapse>
+
+#### OpenSUSE
+
+Synapse is in the OpenSUSE repositories as `matrix-synapse`:
+
+```sh
+sudo zypper install matrix-synapse
+```
+
+#### SUSE Linux Enterprise Server
+
+Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
+<https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/>
+
+#### ArchLinux
+
+The quickest way to get up and running with ArchLinux is probably with the community package
+<https://www.archlinux.org/packages/community/any/matrix-synapse/>, which should pull in most of
+the necessary dependencies.
+
+pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
+
+```sh
+sudo pip install --upgrade pip
+```
+
+If you encounter an error with lib bcrypt causing an Wrong ELF Class:
+ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
+compile it under the right architecture. (This should not be needed if
+installing under virtualenv):
+
+```sh
+sudo pip uninstall py-bcrypt
+sudo pip install py-bcrypt
+```
+
+#### Void Linux
+
+Synapse can be found in the void repositories as 'synapse':
+
+```sh
+xbps-install -Su
+xbps-install -S synapse
+```
+
+#### FreeBSD
+
+Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
+
+- Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
+- Packages: `pkg install py37-matrix-synapse`
+
+#### OpenBSD
+
+As of OpenBSD 6.7 Synapse is available as a pre-compiled binary. The filesystem
+underlying the homeserver directory (defaults to `/var/synapse`) has to be
+mounted with `wxallowed` (cf. `mount(8)`), so creating a separate filesystem
+and mounting it to `/var/synapse` should be taken into consideration.
+
+Installing Synapse:
+
+```sh
+doas pkg_add synapse
+```
+
+#### NixOS
+
+Robin Lambertz has packaged Synapse for NixOS at:
+<https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix>
+
+## Setting up Synapse
+
+Once you have installed synapse as above, you will need to configure it.
+
+### Using PostgreSQL
+
+By default Synapse uses an [SQLite](https://sqlite.org/) database and in doing so trades
+performance for convenience. Almost all installations should opt to use [PostgreSQL](https://www.postgresql.org)
+instead. Advantages include:
+
+- significant performance improvements due to the superior threading and
+ caching model, smarter query optimiser
+- allowing the DB to be run on separate hardware
+
+For information on how to install and use PostgreSQL in Synapse, please see
+[docs/postgres.md](../postgres.md)
+
+SQLite is only acceptable for testing purposes. SQLite should not be used in
+a production server. Synapse will perform poorly when using
+SQLite, especially when participating in large rooms.
+
+### TLS certificates
+
+The default configuration exposes a single HTTP port on the local
+interface: `http://localhost:8008`. It is suitable for local testing,
+but for any practical use, you will need Synapse's APIs to be served
+over HTTPS.
+
+The recommended way to do so is to set up a reverse proxy on port
+`8448`. You can find documentation on doing so in
+[docs/reverse_proxy.md](../reverse_proxy.md).
+
+Alternatively, you can configure Synapse to expose an HTTPS port. To do
+so, you will need to edit `homeserver.yaml`, as follows:
+
+- First, under the `listeners` section, uncomment the configuration for the
+ TLS-enabled listener. (Remove the hash sign (`#`) at the start of
+ each line). The relevant lines are like this:
+
+```yaml
+ - port: 8448
+ type: http
+ tls: true
+ resources:
+ - names: [client, federation]
+ ```
+
+- You will also need to uncomment the `tls_certificate_path` and
+ `tls_private_key_path` lines under the `TLS` section. You will need to manage
+ provisioning of these certificates yourself.
+
+ If you are using your own certificate, be sure to use a `.pem` file that
+ includes the full certificate chain including any intermediate certificates
+ (for instance, if using certbot, use `fullchain.pem` as your certificate, not
+ `cert.pem`).
+
+For a more detailed guide to configuring your server for federation, see
+[federate.md](../federate.md).
+
+### Client Well-Known URI
+
+Setting up the client Well-Known URI is optional but if you set it up, it will
+allow users to enter their full username (e.g. `@user:<server_name>`) into clients
+which support well-known lookup to automatically configure the homeserver and
+identity server URLs. This is useful so that users don't have to memorize or think
+about the actual homeserver URL you are using.
+
+The URL `https://<server_name>/.well-known/matrix/client` should return JSON in
+the following format.
+
+```json
+{
+ "m.homeserver": {
+ "base_url": "https://<matrix.example.com>"
+ }
+}
+```
+
+It can optionally contain identity server information as well.
+
+```json
+{
+ "m.homeserver": {
+ "base_url": "https://<matrix.example.com>"
+ },
+ "m.identity_server": {
+ "base_url": "https://<identity.example.com>"
+ }
+}
+```
+
+To work in browser based clients, the file must be served with the appropriate
+Cross-Origin Resource Sharing (CORS) headers. A recommended value would be
+`Access-Control-Allow-Origin: *` which would allow all browser based clients to
+view it.
+
+In nginx this would be something like:
+
+```nginx
+location /.well-known/matrix/client {
+ return 200 '{"m.homeserver": {"base_url": "https://<matrix.example.com>"}}';
+ default_type application/json;
+ add_header Access-Control-Allow-Origin *;
+}
+```
+
+You should also ensure the `public_baseurl` option in `homeserver.yaml` is set
+correctly. `public_baseurl` should be set to the URL that clients will use to
+connect to your server. This is the same URL you put for the `m.homeserver`
+`base_url` above.
+
+```yaml
+public_baseurl: "https://<matrix.example.com>"
+```
+
+### Email
+
+It is desirable for Synapse to have the capability to send email. This allows
+Synapse to send password reset emails, send verifications when an email address
+is added to a user's account, and send email notifications to users when they
+receive new messages.
+
+To configure an SMTP server for Synapse, modify the configuration section
+headed `email`, and be sure to have at least the `smtp_host`, `smtp_port`
+and `notif_from` fields filled out. You may also need to set `smtp_user`,
+`smtp_pass`, and `require_transport_security`.
+
+If email is not configured, password reset, registration and notifications via
+email will be disabled.
+
+### Registering a user
+
+The easiest way to create a new user is to do so from a client like [Element](https://element.io/).
+
+Alternatively, you can do so from the command line. This can be done as follows:
+
+ 1. If synapse was installed via pip, activate the virtualenv as follows (if Synapse was
+ installed via a prebuilt package, `register_new_matrix_user` should already be
+ on the search path):
+ ```sh
+ cd ~/synapse
+ source env/bin/activate
+ synctl start # if not already running
+ ```
+ 2. Run the following command:
+ ```sh
+ register_new_matrix_user -c homeserver.yaml http://localhost:8008
+ ```
+
+This will prompt you to add details for the new user, and will then connect to
+the running Synapse to create the new user. For example:
+```
+New user localpart: erikj
+Password:
+Confirm password:
+Make admin [no]:
+Success!
+```
+
+This process uses a setting `registration_shared_secret` in
+`homeserver.yaml`, which is shared between Synapse itself and the
+`register_new_matrix_user` script. It doesn't matter what it is (a random
+value is generated by `--generate-config`), but it should be kept secret, as
+anyone with knowledge of it can register users, including admin accounts,
+on your server even if `enable_registration` is `false`.
+
+### Setting up a TURN server
+
+For reliable VoIP calls to be routed via this homeserver, you MUST configure
+a TURN server. See
+[docs/turn-howto.md](../turn-howto.md)
+for details.
+
+### URL previews
+
+Synapse includes support for previewing URLs, which is disabled by default. To
+turn it on you must enable the `url_preview_enabled: True` config parameter
+and explicitly specify the IP ranges that Synapse is not allowed to spider for
+previewing in the `url_preview_ip_range_blacklist` configuration parameter.
+This is critical from a security perspective to stop arbitrary Matrix users
+spidering 'internal' URLs on your network. At the very least we recommend that
+your loopback and RFC1918 IP addresses are blacklisted.
+
+This also requires the optional `lxml` python dependency to be installed. This
+in turn requires the `libxml2` library to be available - on Debian/Ubuntu this
+means `apt-get install libxml2-dev`, or equivalent for your OS.
+
+### Troubleshooting Installation
+
+`pip` seems to leak *lots* of memory during installation. For instance, a Linux
+host with 512MB of RAM may run out of memory whilst installing Twisted. If this
+happens, you will have to individually install the dependencies which are
+failing, e.g.:
+
+```sh
+pip install twisted
+```
+
+If you have any other problems, feel free to ask in
+[#synapse:matrix.org](https://matrix.to/#/#synapse:matrix.org).
diff --git a/docs/upgrade.md b/docs/upgrade.md
new file mode 100644
index 0000000000..db0450f563
--- /dev/null
+++ b/docs/upgrade.md
@@ -0,0 +1,1391 @@
+# Upgrading Synapse
+
+Before upgrading check if any special steps are required to upgrade from
+the version you currently have installed to the current version of
+Synapse. The extra instructions that may be required are listed later in
+this document.
+
+- Check that your versions of Python and PostgreSQL are still
+ supported.
+
+ Synapse follows upstream lifecycles for [Python](https://endoflife.date/python) and
+ [PostgreSQL](https://endoflife.date/postgresql), and removes support for versions
+ which are no longer maintained.
+
+ The website <https://endoflife.date> also offers convenient
+ summaries.
+
+- If Synapse was installed using [prebuilt
+ packages](setup/installation.md#prebuilt-packages), you will need to follow the
+ normal process for upgrading those packages.
+
+- If Synapse was installed from source, then:
+
+ 1. Activate the virtualenv before upgrading. For example, if
+ Synapse is installed in a virtualenv in `~/synapse/env` then
+ run:
+
+ ```bash
+ source ~/synapse/env/bin/activate
+ ```
+
+ 2. If Synapse was installed using pip then upgrade to the latest
+ version by running:
+
+ ```bash
+ pip install --upgrade matrix-synapse
+ ```
+
+ If Synapse was installed using git then upgrade to the latest
+ version by running:
+
+ ```bash
+ git pull
+ pip install --upgrade .
+ ```
+
+ 3. Restart Synapse:
+
+ ```bash
+ ./synctl restart
+ ```
+
+To check whether your update was successful, you can check the running
+server version with:
+
+```bash
+# you may need to replace 'localhost:8008' if synapse is not configured
+# to listen on port 8008.
+
+curl http://localhost:8008/_synapse/admin/v1/server_version
+```
+
+## Rolling back to older versions
+
+Rolling back to previous releases can be difficult, due to database
+schema changes between releases. Where we have been able to test the
+rollback process, this will be noted below.
+
+In general, you will need to undo any changes made during the upgrade
+process, for example:
+
+- pip:
+
+ ```bash
+ source env/bin/activate
+ # replace `1.3.0` accordingly:
+ pip install matrix-synapse==1.3.0
+ ```
+
+- Debian:
+
+ ```bash
+ # replace `1.3.0` and `stretch` accordingly:
+ wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
+ dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
+ ```
+
+
+# Upgrading to v1.38.0
+
+## Re-indexing of `events` table on Postgres databases
+
+This release includes a database schema update which requires re-indexing one of
+the larger tables in the database, `events`. This could result in increased
+disk I/O for several hours or days after upgrading while the migration
+completes. Furthermore, because we have to keep the old indexes until the new
+indexes are ready, it could result in a significant, temporary, increase in
+disk space.
+
+To get a rough idea of the disk space required, check the current size of one
+of the indexes. For example, from a `psql` shell, run the following sql:
+
+```sql
+SELECT pg_size_pretty(pg_relation_size('events_order_room'));
+```
+
+We need to rebuild **four** indexes, so you will need to multiply this result
+by four to give an estimate of the disk space required. For example, on one
+particular server:
+
+```
+synapse=# select pg_size_pretty(pg_relation_size('events_order_room'));
+ pg_size_pretty
+----------------
+ 288 MB
+(1 row)
+```
+
+On this server, it would be wise to ensure that at least 1152MB are free.
+
+The additional disk space will be freed once the migration completes.
+
+SQLite databases are unaffected by this change.
+
+
+# Upgrading to v1.37.0
+
+## Deprecation of the current spam checker interface
+
+The current spam checker interface is deprecated in favour of a new generic modules system.
+Authors of spam checker modules can refer to [this
+documentation](https://matrix-org.github.io/synapse/develop/modules.html#porting-an-existing-module-that-uses-the-old-interface)
+to update their modules. Synapse administrators can refer to [this
+documentation](https://matrix-org.github.io/synapse/develop/modules.html#using-modules)
+to update their configuration once the modules they are using have been updated.
+
+We plan to remove support for the current spam checker interface in August 2021.
+
+More module interfaces will be ported over to this new generic system in future versions
+of Synapse.
+
+
+# Upgrading to v1.34.0
+
+## `room_invite_state_types` configuration setting
+
+The `room_invite_state_types` configuration setting has been deprecated
+and replaced with `room_prejoin_state`. See the [sample configuration
+file](https://github.com/matrix-org/synapse/blob/v1.34.0/docs/sample_config.yaml#L1515).
+
+If you have set `room_invite_state_types` to the default value you
+should simply remove it from your configuration file. The default value
+used to be:
+
+```yaml
+room_invite_state_types:
+ - "m.room.join_rules"
+ - "m.room.canonical_alias"
+ - "m.room.avatar"
+ - "m.room.encryption"
+ - "m.room.name"
+```
+
+If you have customised this value, you should remove
+`room_invite_state_types` and configure `room_prejoin_state` instead.
+
+# Upgrading to v1.33.0
+
+## Account Validity HTML templates can now display a user's expiration date
+
+This may affect you if you have enabled the account validity feature,
+and have made use of a custom HTML template specified by the
+`account_validity.template_dir` or
+`account_validity.account_renewed_html_path` Synapse config options.
+
+The template can now accept an `expiration_ts` variable, which
+represents the unix timestamp in milliseconds for the future date of
+which their account has been renewed until. See the [default
+template](https://github.com/matrix-org/synapse/blob/release-v1.33.0/synapse/res/templates/account_renewed.html)
+for an example of usage.
+
+ALso note that a new HTML template, `account_previously_renewed.html`,
+has been added. This is is shown to users when they attempt to renew
+their account with a valid renewal token that has already been used
+before. The default template contents can been found
+[here](https://github.com/matrix-org/synapse/blob/release-v1.33.0/synapse/res/templates/account_previously_renewed.html),
+and can also accept an `expiration_ts` variable. This template replaces
+the error message users would previously see upon attempting to use a
+valid renewal token more than once.
+
+# Upgrading to v1.32.0
+
+## Regression causing connected Prometheus instances to become overwhelmed
+
+This release introduces [a
+regression](https://github.com/matrix-org/synapse/issues/9853) that can
+overwhelm connected Prometheus instances. This issue is not present in
+Synapse v1.32.0rc1.
+
+If you have been affected, please downgrade to 1.31.0. You then may need
+to remove excess writeahead logs in order for Prometheus to recover.
+Instructions for doing so are provided
+[here](https://github.com/matrix-org/synapse/pull/9854#issuecomment-823472183).
+
+## Dropping support for old Python, Postgres and SQLite versions
+
+In line with our [deprecation
+policy](https://github.com/matrix-org/synapse/blob/release-v1.32.0/docs/deprecation_policy.md),
+we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no
+longer supported upstream.
+
+This release of Synapse requires Python 3.6+ and PostgresSQL 9.6+ or
+SQLite 3.22+.
+
+## Removal of old List Accounts Admin API
+
+The deprecated v1 "list accounts" admin API
+(`GET /_synapse/admin/v1/users/<user_id>`) has been removed in this
+version.
+
+The [v2 list accounts
+API](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/user_admin_api.rst#list-accounts)
+has been available since Synapse 1.7.0 (2019-12-13), and is accessible
+under `GET /_synapse/admin/v2/users`.
+
+The deprecation of the old endpoint was announced with Synapse 1.28.0
+(released on 2021-02-25).
+
+## Application Services must use type `m.login.application_service` when registering users
+
+In compliance with the [Application Service
+spec](https://matrix.org/docs/spec/application_service/r0.1.2#server-admin-style-permissions),
+Application Services are now required to use the
+`m.login.application_service` type when registering users via the
+`/_matrix/client/r0/register` endpoint. This behaviour was deprecated in
+Synapse v1.30.0.
+
+Please ensure your Application Services are up to date.
+
+# Upgrading to v1.29.0
+
+## Requirement for X-Forwarded-Proto header
+
+When using Synapse with a reverse proxy (in particular, when using the
+[x_forwarded]{.title-ref} option on an HTTP listener), Synapse now
+expects to receive an [X-Forwarded-Proto]{.title-ref} header on incoming
+HTTP requests. If it is not set, Synapse will log a warning on each
+received request.
+
+To avoid the warning, administrators using a reverse proxy should ensure
+that the reverse proxy sets [X-Forwarded-Proto]{.title-ref} header to
+[https]{.title-ref} or [http]{.title-ref} to indicate the protocol used
+by the client.
+
+Synapse also requires the [Host]{.title-ref} header to be preserved.
+
+See the [reverse proxy documentation](../reverse_proxy.md), where the
+example configurations have been updated to show how to set these
+headers.
+
+(Users of [Caddy](https://caddyserver.com/) are unaffected, since we
+believe it sets [X-Forwarded-Proto]{.title-ref} by default.)
+
+# Upgrading to v1.27.0
+
+## Changes to callback URI for OAuth2 / OpenID Connect and SAML2
+
+This version changes the URI used for callbacks from OAuth2 and SAML2
+identity providers:
+
+- If your server is configured for single sign-on via an OpenID
+ Connect or OAuth2 identity provider, you will need to add
+ `[synapse public baseurl]/_synapse/client/oidc/callback` to the list
+ of permitted "redirect URIs" at the identity provider.
+
+ See the [OpenID docs](../openid.md) for more information on setting
+ up OpenID Connect.
+
+- If your server is configured for single sign-on via a SAML2 identity
+ provider, you will need to add
+ `[synapse public baseurl]/_synapse/client/saml2/authn_response` as a
+ permitted "ACS location" (also known as "allowed callback URLs")
+ at the identity provider.
+
+ The "Issuer" in the "AuthnRequest" to the SAML2 identity
+ provider is also updated to
+ `[synapse public baseurl]/_synapse/client/saml2/metadata.xml`. If
+ your SAML2 identity provider uses this property to validate or
+ otherwise identify Synapse, its configuration will need to be
+ updated to use the new URL. Alternatively you could create a new,
+ separate "EntityDescriptor" in your SAML2 identity provider with
+ the new URLs and leave the URLs in the existing "EntityDescriptor"
+ as they were.
+
+## Changes to HTML templates
+
+The HTML templates for SSO and email notifications now have [Jinja2's
+autoescape](https://jinja.palletsprojects.com/en/2.11.x/api/#autoescaping)
+enabled for files ending in `.html`, `.htm`, and `.xml`. If you have
+customised these templates and see issues when viewing them you might
+need to update them. It is expected that most configurations will need
+no changes.
+
+If you have customised the templates *names* for these templates, it is
+recommended to verify they end in `.html` to ensure autoescape is
+enabled.
+
+The above applies to the following templates:
+
+- `add_threepid.html`
+- `add_threepid_failure.html`
+- `add_threepid_success.html`
+- `notice_expiry.html`
+- `notice_expiry.html`
+- `notif_mail.html` (which, by default, includes `room.html` and
+ `notif.html`)
+- `password_reset.html`
+- `password_reset_confirmation.html`
+- `password_reset_failure.html`
+- `password_reset_success.html`
+- `registration.html`
+- `registration_failure.html`
+- `registration_success.html`
+- `sso_account_deactivated.html`
+- `sso_auth_bad_user.html`
+- `sso_auth_confirm.html`
+- `sso_auth_success.html`
+- `sso_error.html`
+- `sso_login_idp_picker.html`
+- `sso_redirect_confirm.html`
+
+# Upgrading to v1.26.0
+
+## Rolling back to v1.25.0 after a failed upgrade
+
+v1.26.0 includes a lot of large changes. If something problematic
+occurs, you may want to roll-back to a previous version of Synapse.
+Because v1.26.0 also includes a new database schema version, reverting
+that version is also required alongside the generic rollback
+instructions mentioned above. In short, to roll back to v1.25.0 you need
+to:
+
+1. Stop the server
+
+2. Decrease the schema version in the database:
+
+ ```sql
+ UPDATE schema_version SET version = 58;
+ ```
+
+3. Delete the ignored users & chain cover data:
+
+ ```sql
+ DROP TABLE IF EXISTS ignored_users;
+ UPDATE rooms SET has_auth_chain_index = false;
+ ```
+
+ For PostgreSQL run:
+
+ ```sql
+ TRUNCATE event_auth_chain_links;
+ TRUNCATE event_auth_chains;
+ ```
+
+ For SQLite run:
+
+ ```sql
+ DELETE FROM event_auth_chain_links;
+ DELETE FROM event_auth_chains;
+ ```
+
+4. Mark the deltas as not run (so they will re-run on upgrade).
+
+ ```sql
+ DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/01ignored_user.py";
+ DELETE FROM applied_schema_deltas WHERE version = 59 AND file = "59/06chain_cover_index.sql";
+ ```
+
+5. Downgrade Synapse by following the instructions for your
+ installation method in the "Rolling back to older versions"
+ section above.
+
+# Upgrading to v1.25.0
+
+## Last release supporting Python 3.5
+
+This is the last release of Synapse which guarantees support with Python
+3.5, which passed its upstream End of Life date several months ago.
+
+We will attempt to maintain support through March 2021, but without
+guarantees.
+
+In the future, Synapse will follow upstream schedules for ending support
+of older versions of Python and PostgreSQL. Please upgrade to at least
+Python 3.6 and PostgreSQL 9.6 as soon as possible.
+
+## Blacklisting IP ranges
+
+Synapse v1.25.0 includes new settings, `ip_range_blacklist` and
+`ip_range_whitelist`, for controlling outgoing requests from Synapse for
+federation, identity servers, push, and for checking key validity for
+third-party invite events. The previous setting,
+`federation_ip_range_blacklist`, is deprecated. The new
+`ip_range_blacklist` defaults to private IP ranges if it is not defined.
+
+If you have never customised `federation_ip_range_blacklist` it is
+recommended that you remove that setting.
+
+If you have customised `federation_ip_range_blacklist` you should update
+the setting name to `ip_range_blacklist`.
+
+If you have a custom push server that is reached via private IP space
+you may need to customise `ip_range_blacklist` or `ip_range_whitelist`.
+
+# Upgrading to v1.24.0
+
+## Custom OpenID Connect mapping provider breaking change
+
+This release allows the OpenID Connect mapping provider to perform
+normalisation of the localpart of the Matrix ID. This allows for the
+mapping provider to specify different algorithms, instead of the
+[default
+way](<https://matrix.org/docs/spec/appendices#mapping-from-other-character-sets>).
+
+If your Synapse configuration uses a custom mapping provider
+([oidc_config.user_mapping_provider.module]{.title-ref} is specified and
+not equal to
+[synapse.handlers.oidc_handler.JinjaOidcMappingProvider]{.title-ref})
+then you *must* ensure that [map_user_attributes]{.title-ref} of the
+mapping provider performs some normalisation of the
+[localpart]{.title-ref} returned. To match previous behaviour you can
+use the [map_username_to_mxid_localpart]{.title-ref} function provided
+by Synapse. An example is shown below:
+
+```python
+from synapse.types import map_username_to_mxid_localpart
+
+class MyMappingProvider:
+ def map_user_attributes(self, userinfo, token):
+ # ... your custom logic ...
+ sso_user_id = ...
+ localpart = map_username_to_mxid_localpart(sso_user_id)
+
+ return {"localpart": localpart}
+```
+
+## Removal historical Synapse Admin API
+
+Historically, the Synapse Admin API has been accessible under:
+
+- `/_matrix/client/api/v1/admin`
+- `/_matrix/client/unstable/admin`
+- `/_matrix/client/r0/admin`
+- `/_synapse/admin/v1`
+
+The endpoints with `/_matrix/client/*` prefixes have been removed as of
+v1.24.0. The Admin API is now only accessible under:
+
+- `/_synapse/admin/v1`
+
+The only exception is the [/admin/whois]{.title-ref} endpoint, which is
+[also available via the client-server
+API](https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-admin-whois-userid).
+
+The deprecation of the old endpoints was announced with Synapse 1.20.0
+(released on 2020-09-22) and makes it easier for homeserver admins to
+lock down external access to the Admin API endpoints.
+
+# Upgrading to v1.23.0
+
+## Structured logging configuration breaking changes
+
+This release deprecates use of the `structured: true` logging
+configuration for structured logging. If your logging configuration
+contains `structured: true` then it should be modified based on the
+[structured logging
+documentation](../structured_logging.md).
+
+The `structured` and `drains` logging options are now deprecated and
+should be replaced by standard logging configuration of `handlers` and
+`formatters`.
+
+A future will release of Synapse will make using `structured: true` an
+error.
+
+# Upgrading to v1.22.0
+
+## ThirdPartyEventRules breaking changes
+
+This release introduces a backwards-incompatible change to modules
+making use of `ThirdPartyEventRules` in Synapse. If you make use of a
+module defined under the `third_party_event_rules` config option, please
+make sure it is updated to handle the below change:
+
+The `http_client` argument is no longer passed to modules as they are
+initialised. Instead, modules are expected to make use of the
+`http_client` property on the `ModuleApi` class. Modules are now passed
+a `module_api` argument during initialisation, which is an instance of
+`ModuleApi`. `ModuleApi` instances have a `http_client` property which
+acts the same as the `http_client` argument previously passed to
+`ThirdPartyEventRules` modules.
+
+# Upgrading to v1.21.0
+
+## Forwarding `/_synapse/client` through your reverse proxy
+
+The [reverse proxy
+documentation](https://github.com/matrix-org/synapse/blob/develop/docs/reverse_proxy.md)
+has been updated to include reverse proxy directives for
+`/_synapse/client/*` endpoints. As the user password reset flow now uses
+endpoints under this prefix, **you must update your reverse proxy
+configurations for user password reset to work**.
+
+Additionally, note that the [Synapse worker documentation](https://github.com/matrix-org/synapse/blob/develop/docs/workers.md) has been updated to
+
+: state that the `/_synapse/client/password_reset/email/submit_token`
+ endpoint can be handled
+
+by all workers. If you make use of Synapse's worker feature, please
+update your reverse proxy configuration to reflect this change.
+
+## New HTML templates
+
+A new HTML template,
+[password_reset_confirmation.html](https://github.com/matrix-org/synapse/blob/develop/synapse/res/templates/password_reset_confirmation.html),
+has been added to the `synapse/res/templates` directory. If you are
+using a custom template directory, you may want to copy the template
+over and modify it.
+
+Note that as of v1.20.0, templates do not need to be included in custom
+template directories for Synapse to start. The default templates will be
+used if a custom template cannot be found.
+
+This page will appear to the user after clicking a password reset link
+that has been emailed to them.
+
+To complete password reset, the page must include a way to make a
+[POST]{.title-ref} request to
+`/_synapse/client/password_reset/{medium}/submit_token` with the query
+parameters from the original link, presented as a URL-encoded form. See
+the file itself for more details.
+
+## Updated Single Sign-on HTML Templates
+
+The `saml_error.html` template was removed from Synapse and replaced
+with the `sso_error.html` template. If your Synapse is configured to use
+SAML and a custom `sso_redirect_confirm_template_dir` configuration then
+any customisations of the `saml_error.html` template will need to be
+merged into the `sso_error.html` template. These templates are similar,
+but the parameters are slightly different:
+
+- The `msg` parameter should be renamed to `error_description`.
+- There is no longer a `code` parameter for the response code.
+- A string `error` parameter is available that includes a short hint
+ of why a user is seeing the error page.
+
+# Upgrading to v1.18.0
+
+## Docker [-py3]{.title-ref} suffix will be removed in future versions
+
+From 10th August 2020, we will no longer publish Docker images with the
+[-py3]{.title-ref} tag suffix. The images tagged with the
+[-py3]{.title-ref} suffix have been identical to the non-suffixed tags
+since release 0.99.0, and the suffix is obsolete.
+
+On 10th August, we will remove the [latest-py3]{.title-ref} tag.
+Existing per-release tags (such as [v1.18.0-py3]{.title-ref}) will not
+be removed, but no new [-py3]{.title-ref} tags will be added.
+
+Scripts relying on the [-py3]{.title-ref} suffix will need to be
+updated.
+
+## Redis replication is now recommended in lieu of TCP replication
+
+When setting up worker processes, we now recommend the use of a Redis
+server for replication. **The old direct TCP connection method is
+deprecated and will be removed in a future release.** See
+[workers](../workers.md) for more details.
+
+# Upgrading to v1.14.0
+
+This version includes a database update which is run as part of the
+upgrade, and which may take a couple of minutes in the case of a large
+server. Synapse will not respond to HTTP requests while this update is
+taking place.
+
+# Upgrading to v1.13.0
+
+## Incorrect database migration in old synapse versions
+
+A bug was introduced in Synapse 1.4.0 which could cause the room
+directory to be incomplete or empty if Synapse was upgraded directly
+from v1.2.1 or earlier, to versions between v1.4.0 and v1.12.x.
+
+This will *not* be a problem for Synapse installations which were:
+
+: - created at v1.4.0 or later,
+ - upgraded via v1.3.x, or
+ - upgraded straight from v1.2.1 or earlier to v1.13.0 or later.
+
+If completeness of the room directory is a concern, installations which
+are affected can be repaired as follows:
+
+1. Run the following sql from a [psql]{.title-ref} or
+ [sqlite3]{.title-ref} console:
+
+ ```sql
+ INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES
+ ('populate_stats_process_rooms', '{}', 'current_state_events_membership');
+
+ INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES
+ ('populate_stats_process_users', '{}', 'populate_stats_process_rooms');
+ ```
+
+2. Restart synapse.
+
+## New Single Sign-on HTML Templates
+
+New templates (`sso_auth_confirm.html`, `sso_auth_success.html`, and
+`sso_account_deactivated.html`) were added to Synapse. If your Synapse
+is configured to use SSO and a custom
+`sso_redirect_confirm_template_dir` configuration then these templates
+will need to be copied from
+[synapse/res/templates](synapse/res/templates) into that directory.
+
+## Synapse SSO Plugins Method Deprecation
+
+Plugins using the `complete_sso_login` method of
+`synapse.module_api.ModuleApi` should update to using the async/await
+version `complete_sso_login_async` which includes additional checks. The
+non-async version is considered deprecated.
+
+## Rolling back to v1.12.4 after a failed upgrade
+
+v1.13.0 includes a lot of large changes. If something problematic
+occurs, you may want to roll-back to a previous version of Synapse.
+Because v1.13.0 also includes a new database schema version, reverting
+that version is also required alongside the generic rollback
+instructions mentioned above. In short, to roll back to v1.12.4 you need
+to:
+
+1. Stop the server
+
+2. Decrease the schema version in the database:
+
+ ```sql
+ UPDATE schema_version SET version = 57;
+ ```
+
+3. Downgrade Synapse by following the instructions for your
+ installation method in the "Rolling back to older versions"
+ section above.
+
+# Upgrading to v1.12.0
+
+This version includes a database update which is run as part of the
+upgrade, and which may take some time (several hours in the case of a
+large server). Synapse will not respond to HTTP requests while this
+update is taking place.
+
+This is only likely to be a problem in the case of a server which is
+participating in many rooms.
+
+0. As with all upgrades, it is recommended that you have a recent
+ backup of your database which can be used for recovery in the event
+ of any problems.
+
+1. As an initial check to see if you will be affected, you can try
+ running the following query from the [psql]{.title-ref} or
+ [sqlite3]{.title-ref} console. It is safe to run it while Synapse is
+ still running.
+
+ ```sql
+ SELECT MAX(q.v) FROM (
+ SELECT (
+ SELECT ej.json AS v
+ FROM state_events se INNER JOIN event_json ej USING (event_id)
+ WHERE se.room_id=rooms.room_id AND se.type='m.room.create' AND se.state_key=''
+ LIMIT 1
+ ) FROM rooms WHERE rooms.room_version IS NULL
+ ) q;
+ ```
+
+ This query will take about the same amount of time as the upgrade
+ process: ie, if it takes 5 minutes, then it is likely that Synapse
+ will be unresponsive for 5 minutes during the upgrade.
+
+ If you consider an outage of this duration to be acceptable, no
+ further action is necessary and you can simply start Synapse 1.12.0.
+
+ If you would prefer to reduce the downtime, continue with the steps
+ below.
+
+2. The easiest workaround for this issue is to manually create a new
+ index before upgrading. On PostgreSQL, his can be done as follows:
+
+ ```sql
+ CREATE INDEX CONCURRENTLY tmp_upgrade_1_12_0_index
+ ON state_events(room_id) WHERE type = 'm.room.create';
+ ```
+
+ The above query may take some time, but is also safe to run while
+ Synapse is running.
+
+ We assume that no SQLite users have databases large enough to be
+ affected. If you *are* affected, you can run a similar query,
+ omitting the `CONCURRENTLY` keyword. Note however that this
+ operation may in itself cause Synapse to stop running for some time.
+ Synapse admins are reminded that [SQLite is not recommended for use
+ outside a test
+ environment](https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql).
+
+3. Once the index has been created, the `SELECT` query in step 1 above
+ should complete quickly. It is therefore safe to upgrade to Synapse
+ 1.12.0.
+
+4. Once Synapse 1.12.0 has successfully started and is responding to
+ HTTP requests, the temporary index can be removed:
+
+ ```sql
+ DROP INDEX tmp_upgrade_1_12_0_index;
+ ```
+
+# Upgrading to v1.10.0
+
+Synapse will now log a warning on start up if used with a PostgreSQL
+database that has a non-recommended locale set.
+
+See [Postgres](../postgres.md) for details.
+
+# Upgrading to v1.8.0
+
+Specifying a `log_file` config option will now cause Synapse to refuse
+to start, and should be replaced by with the `log_config` option.
+Support for the `log_file` option was removed in v1.3.0 and has since
+had no effect.
+
+# Upgrading to v1.7.0
+
+In an attempt to configure Synapse in a privacy preserving way, the
+default behaviours of `allow_public_rooms_without_auth` and
+`allow_public_rooms_over_federation` have been inverted. This means that
+by default, only authenticated users querying the Client/Server API will
+be able to query the room directory, and relatedly that the server will
+not share room directory information with other servers over federation.
+
+If your installation does not explicitly set these settings one way or
+the other and you want either setting to be `true` then it will
+necessary to update your homeserver configuration file accordingly.
+
+For more details on the surrounding context see our
+[explainer](https://matrix.org/blog/2019/11/09/avoiding-unwelcome-visitors-on-private-matrix-servers).
+
+# Upgrading to v1.5.0
+
+This release includes a database migration which may take several
+minutes to complete if there are a large number (more than a million or
+so) of entries in the `devices` table. This is only likely to a be a
+problem on very large installations.
+
+# Upgrading to v1.4.0
+
+## New custom templates
+
+If you have configured a custom template directory with the
+`email.template_dir` option, be aware that there are new templates
+regarding registration and threepid management (see below) that must be
+included.
+
+- `registration.html` and `registration.txt`
+- `registration_success.html` and `registration_failure.html`
+- `add_threepid.html` and `add_threepid.txt`
+- `add_threepid_failure.html` and `add_threepid_success.html`
+
+Synapse will expect these files to exist inside the configured template
+directory, and **will fail to start** if they are absent. To view the
+default templates, see
+[synapse/res/templates](https://github.com/matrix-org/synapse/tree/master/synapse/res/templates).
+
+## 3pid verification changes
+
+**Note: As of this release, users will be unable to add phone numbers or
+email addresses to their accounts, without changes to the Synapse
+configuration. This includes adding an email address during
+registration.**
+
+It is possible for a user to associate an email address or phone number
+with their account, for a number of reasons:
+
+- for use when logging in, as an alternative to the user id.
+- in the case of email, as an alternative contact to help with account
+ recovery.
+- in the case of email, to receive notifications of missed messages.
+
+Before an email address or phone number can be added to a user's
+account, or before such an address is used to carry out a
+password-reset, Synapse must confirm the operation with the owner of the
+email address or phone number. It does this by sending an email or text
+giving the user a link or token to confirm receipt. This process is
+known as '3pid verification'. ('3pid', or 'threepid', stands for
+third-party identifier, and we use it to refer to external identifiers
+such as email addresses and phone numbers.)
+
+Previous versions of Synapse delegated the task of 3pid verification to
+an identity server by default. In most cases this server is `vector.im`
+or `matrix.org`.
+
+In Synapse 1.4.0, for security and privacy reasons, the homeserver will
+no longer delegate this task to an identity server by default. Instead,
+the server administrator will need to explicitly decide how they would
+like the verification messages to be sent.
+
+In the medium term, the `vector.im` and `matrix.org` identity servers
+will disable support for delegated 3pid verification entirely. However,
+in order to ease the transition, they will retain the capability for a
+limited period. Delegated email verification will be disabled on Monday
+2nd December 2019 (giving roughly 2 months notice). Disabling delegated
+SMS verification will follow some time after that once SMS verification
+support lands in Synapse.
+
+Once delegated 3pid verification support has been disabled in the
+`vector.im` and `matrix.org` identity servers, all Synapse versions that
+depend on those instances will be unable to verify email and phone
+numbers through them. There are no imminent plans to remove delegated
+3pid verification from Sydent generally. (Sydent is the identity server
+project that backs the `vector.im` and `matrix.org` instances).
+
+### Email
+
+Following upgrade, to continue verifying email (e.g. as part of the
+registration process), admins can either:-
+
+- Configure Synapse to use an email server.
+- Run or choose an identity server which allows delegated email
+ verification and delegate to it.
+
+#### Configure SMTP in Synapse
+
+To configure an SMTP server for Synapse, modify the configuration
+section headed `email`, and be sure to have at least the
+`smtp_host, smtp_port` and `notif_from` fields filled out.
+
+You may also need to set `smtp_user`, `smtp_pass`, and
+`require_transport_security`.
+
+See the [sample configuration file](docs/sample_config.yaml) for more
+details on these settings.
+
+#### Delegate email to an identity server
+
+Some admins will wish to continue using email verification as part of
+the registration process, but will not immediately have an appropriate
+SMTP server at hand.
+
+To this end, we will continue to support email verification delegation
+via the `vector.im` and `matrix.org` identity servers for two months.
+Support for delegated email verification will be disabled on Monday 2nd
+December.
+
+The `account_threepid_delegates` dictionary defines whether the
+homeserver should delegate an external server (typically an [identity
+server](https://matrix.org/docs/spec/identity_service/r0.2.1)) to handle
+sending confirmation messages via email and SMS.
+
+So to delegate email verification, in `homeserver.yaml`, set
+`account_threepid_delegates.email` to the base URL of an identity
+server. For example:
+
+```yaml
+account_threepid_delegates:
+ email: https://example.com # Delegate email sending to example.com
+```
+
+Note that `account_threepid_delegates.email` replaces the deprecated
+`email.trust_identity_server_for_password_resets`: if
+`email.trust_identity_server_for_password_resets` is set to `true`, and
+`account_threepid_delegates.email` is not set, then the first entry in
+`trusted_third_party_id_servers` will be used as the
+`account_threepid_delegate` for email. This is to ensure compatibility
+with existing Synapse installs that set up external server handling for
+these tasks before v1.4.0. If
+`email.trust_identity_server_for_password_resets` is `true` and no
+trusted identity server domains are configured, Synapse will report an
+error and refuse to start.
+
+If `email.trust_identity_server_for_password_resets` is `false` or
+absent and no `email` delegate is configured in
+`account_threepid_delegates`, then Synapse will send email verification
+messages itself, using the configured SMTP server (see above). that
+type.
+
+### Phone numbers
+
+Synapse does not support phone-number verification itself, so the only
+way to maintain the ability for users to add phone numbers to their
+accounts will be by continuing to delegate phone number verification to
+the `matrix.org` and `vector.im` identity servers (or another identity
+server that supports SMS sending).
+
+The `account_threepid_delegates` dictionary defines whether the
+homeserver should delegate an external server (typically an [identity
+server](https://matrix.org/docs/spec/identity_service/r0.2.1)) to handle
+sending confirmation messages via email and SMS.
+
+So to delegate phone number verification, in `homeserver.yaml`, set
+`account_threepid_delegates.msisdn` to the base URL of an identity
+server. For example:
+
+```yaml
+account_threepid_delegates:
+ msisdn: https://example.com # Delegate sms sending to example.com
+```
+
+The `matrix.org` and `vector.im` identity servers will continue to
+support delegated phone number verification via SMS until such time as
+it is possible for admins to configure their servers to perform phone
+number verification directly. More details will follow in a future
+release.
+
+## Rolling back to v1.3.1
+
+If you encounter problems with v1.4.0, it should be possible to roll
+back to v1.3.1, subject to the following:
+
+- The 'room statistics' engine was heavily reworked in this release
+ (see [#5971](https://github.com/matrix-org/synapse/pull/5971)),
+ including significant changes to the database schema, which are not
+ easily reverted. This will cause the room statistics engine to stop
+ updating when you downgrade.
+
+ The room statistics are essentially unused in v1.3.1 (in future
+ versions of Synapse, they will be used to populate the room
+ directory), so there should be no loss of functionality. However,
+ the statistics engine will write errors to the logs, which can be
+ avoided by setting the following in `homeserver.yaml`:
+
+ ```yaml
+ stats:
+ enabled: false
+ ```
+
+ Don't forget to re-enable it when you upgrade again, in preparation
+ for its use in the room directory!
+
+# Upgrading to v1.2.0
+
+Some counter metrics have been renamed, with the old names deprecated.
+See [the metrics
+documentation](../metrics-howto.md#renaming-of-metrics--deprecation-of-old-names-in-12)
+for details.
+
+# Upgrading to v1.1.0
+
+Synapse v1.1.0 removes support for older Python and PostgreSQL versions,
+as outlined in [our deprecation
+notice](https://matrix.org/blog/2019/04/08/synapse-deprecating-postgres-9-4-and-python-2-x).
+
+## Minimum Python Version
+
+Synapse v1.1.0 has a minimum Python requirement of Python 3.5. Python
+3.6 or Python 3.7 are recommended as they have improved internal string
+handling, significantly reducing memory usage.
+
+If you use current versions of the Matrix.org-distributed Debian
+packages or Docker images, action is not required.
+
+If you install Synapse in a Python virtual environment, please see
+"Upgrading to v0.34.0" for notes on setting up a new virtualenv under
+Python 3.
+
+## Minimum PostgreSQL Version
+
+If using PostgreSQL under Synapse, you will need to use PostgreSQL 9.5
+or above. Please see the [PostgreSQL
+documentation](https://www.postgresql.org/docs/11/upgrading.html) for
+more details on upgrading your database.
+
+# Upgrading to v1.0
+
+## Validation of TLS certificates
+
+Synapse v1.0 is the first release to enforce validation of TLS
+certificates for the federation API. It is therefore essential that your
+certificates are correctly configured. See the
+[FAQ](../MSC1711_certificates_FAQ.md) for more information.
+
+Note, v1.0 installations will also no longer be able to federate with
+servers that have not correctly configured their certificates.
+
+In rare cases, it may be desirable to disable certificate checking: for
+example, it might be essential to be able to federate with a given
+legacy server in a closed federation. This can be done in one of two
+ways:-
+
+- Configure the global switch `federation_verify_certificates` to
+ `false`.
+- Configure a whitelist of server domains to trust via
+ `federation_certificate_verification_whitelist`.
+
+See the [sample configuration file](docs/sample_config.yaml) for more
+details on these settings.
+
+## Email
+
+When a user requests a password reset, Synapse will send an email to the
+user to confirm the request.
+
+Previous versions of Synapse delegated the job of sending this email to
+an identity server. If the identity server was somehow malicious or
+became compromised, it would be theoretically possible to hijack an
+account through this means.
+
+Therefore, by default, Synapse v1.0 will send the confirmation email
+itself. If Synapse is not configured with an SMTP server, password reset
+via email will be disabled.
+
+To configure an SMTP server for Synapse, modify the configuration
+section headed `email`, and be sure to have at least the `smtp_host`,
+`smtp_port` and `notif_from` fields filled out. You may also need to set
+`smtp_user`, `smtp_pass`, and `require_transport_security`.
+
+If you are absolutely certain that you wish to continue using an
+identity server for password resets, set
+`trust_identity_server_for_password_resets` to `true`.
+
+See the [sample configuration file](docs/sample_config.yaml) for more
+details on these settings.
+
+## New email templates
+
+Some new templates have been added to the default template directory for the purpose of
+the homeserver sending its own password reset emails. If you have configured a
+custom `template_dir` in your Synapse config, these files will need to be added.
+
+`password_reset.html` and `password_reset.txt` are HTML and plain text
+templates respectively that contain the contents of what will be emailed
+to the user upon attempting to reset their password via email.
+`password_reset_success.html` and `password_reset_failure.html` are HTML
+files that the content of which (assuming no redirect URL is set) will
+be shown to the user after they attempt to click the link in the email
+sent to them.
+
+# Upgrading to v0.99.0
+
+Please be aware that, before Synapse v1.0 is released around March 2019,
+you will need to replace any self-signed certificates with those
+verified by a root CA. Information on how to do so can be found at [the
+ACME docs](../ACME.md).
+
+For more information on configuring TLS certificates see the
+[FAQ](../MSC1711_certificates_FAQ.md).
+
+# Upgrading to v0.34.0
+
+1. This release is the first to fully support Python 3. Synapse will
+ now run on Python versions 3.5, or 3.6 (as well as 2.7). We
+ recommend switching to Python 3, as it has been shown to give
+ performance improvements.
+
+ For users who have installed Synapse into a virtualenv, we recommend
+ doing this by creating a new virtualenv. For example:
+
+ virtualenv -p python3 ~/synapse/env3
+ source ~/synapse/env3/bin/activate
+ pip install matrix-synapse
+
+ You can then start synapse as normal, having activated the new
+ virtualenv:
+
+ cd ~/synapse
+ source env3/bin/activate
+ synctl start
+
+ Users who have installed from distribution packages should see the
+ relevant package documentation. See below for notes on Debian
+ packages.
+
+ - When upgrading to Python 3, you **must** make sure that your log
+ files are configured as UTF-8, by adding `encoding: utf8` to the
+ `RotatingFileHandler` configuration (if you have one) in your
+ `<server>.log.config` file. For example, if your `log.config`
+ file contains:
+
+ handlers:
+ file:
+ class: logging.handlers.RotatingFileHandler
+ formatter: precise
+ filename: homeserver.log
+ maxBytes: 104857600
+ backupCount: 10
+ filters: [context]
+ console:
+ class: logging.StreamHandler
+ formatter: precise
+ filters: [context]
+
+ Then you should update this to be:
+
+ handlers:
+ file:
+ class: logging.handlers.RotatingFileHandler
+ formatter: precise
+ filename: homeserver.log
+ maxBytes: 104857600
+ backupCount: 10
+ filters: [context]
+ encoding: utf8
+ console:
+ class: logging.StreamHandler
+ formatter: precise
+ filters: [context]
+
+ There is no need to revert this change if downgrading to
+ Python 2.
+
+ We are also making available Debian packages which will run Synapse
+ on Python 3. You can switch to these packages with
+ `apt-get install matrix-synapse-py3`, however, please read
+ [debian/NEWS](https://github.com/matrix-org/synapse/blob/release-v0.34.0/debian/NEWS)
+ before doing so. The existing `matrix-synapse` packages will
+ continue to use Python 2 for the time being.
+
+2. This release removes the `riot.im` from the default list of trusted
+ identity servers.
+
+ If `riot.im` is in your homeserver's list of
+ `trusted_third_party_id_servers`, you should remove it. It was added
+ in case a hypothetical future identity server was put there. If you
+ don't remove it, users may be unable to deactivate their accounts.
+
+3. This release no longer installs the (unmaintained) Matrix Console
+ web client as part of the default installation. It is possible to
+ re-enable it by installing it separately and setting the
+ `web_client_location` config option, but please consider switching
+ to another client.
+
+# Upgrading to v0.33.7
+
+This release removes the example email notification templates from
+`res/templates` (they are now internal to the python package). This
+should only affect you if you (a) deploy your Synapse instance from a
+git checkout or a github snapshot URL, and (b) have email notifications
+enabled.
+
+If you have email notifications enabled, you should ensure that
+`email.template_dir` is either configured to point at a directory where
+you have installed customised templates, or leave it unset to use the
+default templates.
+
+# Upgrading to v0.27.3
+
+This release expands the anonymous usage stats sent if the opt-in
+`report_stats` configuration is set to `true`. We now capture RSS memory
+and cpu use at a very coarse level. This requires administrators to
+install the optional `psutil` python module.
+
+We would appreciate it if you could assist by ensuring this module is
+available and `report_stats` is enabled. This will let us see if
+performance changes to synapse are having an impact to the general
+community.
+
+# Upgrading to v0.15.0
+
+If you want to use the new URL previewing API
+(`/_matrix/media/r0/preview_url`) then you have to explicitly enable it
+in the config and update your dependencies dependencies. See README.rst
+for details.
+
+# Upgrading to v0.11.0
+
+This release includes the option to send anonymous usage stats to
+matrix.org, and requires that administrators explictly opt in or out by
+setting the `report_stats` option to either `true` or `false`.
+
+We would really appreciate it if you could help our project out by
+reporting anonymized usage statistics from your homeserver. Only very
+basic aggregate data (e.g. number of users) will be reported, but it
+helps us to track the growth of the Matrix community, and helps us to
+make Matrix a success, as well as to convince other networks that they
+should peer with us.
+
+# Upgrading to v0.9.0
+
+Application services have had a breaking API change in this version.
+
+They can no longer register themselves with a home server using the AS
+HTTP API. This decision was made because a compromised application
+service with free reign to register any regex in effect grants full
+read/write access to the home server if a regex of `.*` is used. An
+attack where a compromised AS re-registers itself with `.*` was deemed
+too big of a security risk to ignore, and so the ability to register
+with the HS remotely has been removed.
+
+It has been replaced by specifying a list of application service
+registrations in `homeserver.yaml`:
+
+ app_service_config_files: ["registration-01.yaml", "registration-02.yaml"]
+
+Where `registration-01.yaml` looks like:
+
+ url: <String> # e.g. "https://my.application.service.com"
+ as_token: <String>
+ hs_token: <String>
+ sender_localpart: <String> # This is a new field which denotes the user_id localpart when using the AS token
+ namespaces:
+ users:
+ - exclusive: <Boolean>
+ regex: <String> # e.g. "@prefix_.*"
+ aliases:
+ - exclusive: <Boolean>
+ regex: <String>
+ rooms:
+ - exclusive: <Boolean>
+ regex: <String>
+
+# Upgrading to v0.8.0
+
+Servers which use captchas will need to add their public key to:
+
+ static/client/register/register_config.js
+
+ window.matrixRegistrationConfig = {
+ recaptcha_public_key: "YOUR_PUBLIC_KEY"
+ };
+
+This is required in order to support registration fallback (typically
+used on mobile devices).
+
+# Upgrading to v0.7.0
+
+New dependencies are:
+
+- pydenticon
+- simplejson
+- syutil
+- matrix-angular-sdk
+
+To pull in these dependencies in a virtual env, run:
+
+ python synapse/python_dependencies.py | xargs -n 1 pip install
+
+# Upgrading to v0.6.0
+
+To pull in new dependencies, run:
+
+ python setup.py develop --user
+
+This update includes a change to the database schema. To upgrade you
+first need to upgrade the database by running:
+
+ python scripts/upgrade_db_to_v0.6.0.py <db> <server_name> <signing_key>
+
+Where [<db>]{.title-ref} is the location of the database,
+[<server_name>]{.title-ref} is the server name as specified in the
+synapse configuration, and [<signing_key>]{.title-ref} is the location
+of the signing key as specified in the synapse configuration.
+
+This may take some time to complete. Failures of signatures and content
+hashes can safely be ignored.
+
+# Upgrading to v0.5.1
+
+Depending on precisely when you installed v0.5.0 you may have ended up
+with a stale release of the reference matrix webclient installed as a
+python module. To uninstall it and ensure you are depending on the
+latest module, please run:
+
+ $ pip uninstall syweb
+
+# Upgrading to v0.5.0
+
+The webclient has been split out into a seperate repository/pacakage in
+this release. Before you restart your homeserver you will need to pull
+in the webclient package by running:
+
+ python setup.py develop --user
+
+This release completely changes the database schema and so requires
+upgrading it before starting the new version of the homeserver.
+
+The script "database-prepare-for-0.5.0.sh" should be used to upgrade
+the database. This will save all user information, such as logins and
+profiles, but will otherwise purge the database. This includes messages,
+which rooms the home server was a member of and room alias mappings.
+
+If you would like to keep your history, please take a copy of your
+database file and ask for help in #matrix:matrix.org. The upgrade
+process is, unfortunately, non trivial and requires human intervention
+to resolve any resulting conflicts during the upgrade process.
+
+Before running the command the homeserver should be first completely
+shutdown. To run it, simply specify the location of the database, e.g.:
+
+> ./scripts/database-prepare-for-0.5.0.sh "homeserver.db"
+
+Once this has successfully completed it will be safe to restart the
+homeserver. You may notice that the homeserver takes a few seconds
+longer to restart than usual as it reinitializes the database.
+
+On startup of the new version, users can either rejoin remote rooms
+using room aliases or by being reinvited. Alternatively, if any other
+homeserver sends a message to a room that the homeserver was previously
+in the local HS will automatically rejoin the room.
+
+# Upgrading to v0.4.0
+
+This release needs an updated syutil version. Run:
+
+ python setup.py develop
+
+You will also need to upgrade your configuration as the signing key
+format has changed. Run:
+
+ python -m synapse.app.homeserver --config-path <CONFIG> --generate-config
+
+# Upgrading to v0.3.0
+
+This registration API now closely matches the login API. This introduces
+a bit more backwards and forwards between the HS and the client, but
+this improves the overall flexibility of the API. You can now GET on
+/register to retrieve a list of valid registration flows. Upon choosing
+one, they are submitted in the same way as login, e.g:
+
+ {
+ type: m.login.password,
+ user: foo,
+ password: bar
+ }
+
+The default HS supports 2 flows, with and without Identity Server email
+authentication. Enabling captcha on the HS will add in an extra step to
+all flows: `m.login.recaptcha` which must be completed before you can
+transition to the next stage. There is a new login type:
+`m.login.email.identity` which contains the `threepidCreds` key which
+were previously sent in the original register request. For more
+information on this, see the specification.
+
+## Web Client
+
+The VoIP specification has changed between v0.2.0 and v0.3.0. Users
+should refresh any browser tabs to get the latest web client code. Users
+on v0.2.0 of the web client will not be able to call those on v0.3.0 and
+vice versa.
+
+# Upgrading to v0.2.0
+
+The home server now requires setting up of SSL config before it can run.
+To automatically generate default config use:
+
+ $ python synapse/app/homeserver.py \
+ --server-name machine.my.domain.name \
+ --bind-port 8448 \
+ --config-path homeserver.config \
+ --generate-config
+
+This config can be edited if desired, for example to specify a different
+SSL certificate to use. Once done you can run the home server using:
+
+ $ python synapse/app/homeserver.py --config-path homeserver.config
+
+See the README.rst for more information.
+
+Also note that some config options have been renamed, including:
+
+- "host" to "server-name"
+- "database" to "database-path"
+- "port" to "bind-port" and "unsecure-port"
+
+# Upgrading to v0.0.1
+
+This release completely changes the database schema and so requires
+upgrading it before starting the new version of the homeserver.
+
+The script "database-prepare-for-0.0.1.sh" should be used to upgrade
+the database. This will save all user information, such as logins and
+profiles, but will otherwise purge the database. This includes messages,
+which rooms the home server was a member of and room alias mappings.
+
+Before running the command the homeserver should be first completely
+shutdown. To run it, simply specify the location of the database, e.g.:
+
+> ./scripts/database-prepare-for-0.0.1.sh "homeserver.db"
+
+Once this has successfully completed it will be safe to restart the
+homeserver. You may notice that the homeserver takes a few seconds
+longer to restart than usual as it reinitializes the database.
+
+On startup of the new version, users can either rejoin remote rooms
+using room aliases or by being reinvited. Alternatively, if any other
+homeserver sends a message to a room that the homeserver was previously
+in the local HS will automatically rejoin the room.
diff --git a/docs/upgrading/README.md b/docs/upgrading/README.md
deleted file mode 100644
index 258e58cf15..0000000000
--- a/docs/upgrading/README.md
+++ /dev/null
@@ -1,7 +0,0 @@
-<!--
- Include the contents of UPGRADE.rst from the project root without moving it, which may
- break links around the internet. Additionally, note that SUMMARY.md is unable to
- directly link to content outside of the docs/ directory. So we use this file as a
- redirection.
--->
-{{#include ../../UPGRADE.rst}}
\ No newline at end of file
|
