diff options
Diffstat (limited to 'UPGRADE.rst')
| -rw-r--r-- | UPGRADE.rst | 468 |
1 files changed, 406 insertions, 62 deletions
diff --git a/UPGRADE.rst b/UPGRADE.rst index 5aaf804902..3b5627e852 100644 --- a/UPGRADE.rst +++ b/UPGRADE.rst @@ -2,102 +2,446 @@ Upgrading Synapse ================= Before upgrading check if any special steps are required to upgrade from the -what you currently have installed to current version of synapse. The extra +version you currently have installed to the current version of Synapse. The extra instructions that may be required are listed later in this document. -1. If synapse was installed in a virtualenv then activate that virtualenv before - upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then - run: +* If Synapse was installed using `prebuilt packages + <INSTALL.md#prebuilt-packages>`_, you will need to follow the normal process + for upgrading those packages. - .. code:: bash +* If Synapse was installed from source, then: - source ~/synapse/env/bin/activate + 1. Activate the virtualenv before upgrading. For example, if Synapse is + installed in a virtualenv in ``~/synapse/env`` then run: -2. If synapse was installed using pip then upgrade to the latest version by - running: + .. code:: bash - .. code:: bash + source ~/synapse/env/bin/activate - pip install --upgrade matrix-synapse[all] + 2. If Synapse was installed using pip then upgrade to the latest version by + running: - # restart synapse - synctl restart + .. code:: bash + pip install --upgrade matrix-synapse - If synapse was installed using git then upgrade to the latest version by - running: + If Synapse was installed using git then upgrade to the latest version by + running: - .. code:: bash + .. code:: bash - # Pull the latest version of the master branch. git pull + pip install --upgrade . - # Update synapse and its python dependencies. - pip install --upgrade .[all] + 3. Restart Synapse: - # restart synapse - ./synctl restart + .. code:: bash + ./synctl restart -To check whether your update was successful, you can check the Server header -returned by the Client-Server API: +To check whether your update was successful, you can check the running server +version with: .. code:: bash - # replace <host.name> with the hostname of your synapse homeserver. - # You may need to specify a port (eg, :8448) if your server is not - # configured on port 443. - curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:" + # you may need to replace 'localhost:8008' if synapse is not configured + # to listen on port 8008. + + curl http://localhost:8008/_synapse/admin/v1/server_version + +Rolling back to older versions +------------------------------ + +Rolling back to previous releases can be difficult, due to database schema +changes between releases. Where we have been able to test the rollback process, +this will be noted below. + +In general, you will need to undo any changes made during the upgrade process, +for example: + +* pip: + + .. code:: bash + + source env/bin/activate + # replace `1.3.0` accordingly: + pip install matrix-synapse==1.3.0 + +* Debian: + + .. code:: bash + + # replace `1.3.0` and `stretch` accordingly: + wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb + dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb + +Upgrading to v1.14.0 +==================== + +This version includes a database update which is run as part of the upgrade, +and which may take a couple of minutes in the case of a large server. Synapse +will not respond to HTTP requests while this update is taking place. + +Upgrading to v1.13.0 +==================== + +Incorrect database migration in old synapse versions +---------------------------------------------------- + +A bug was introduced in Synapse 1.4.0 which could cause the room directory to +be incomplete or empty if Synapse was upgraded directly from v1.2.1 or +earlier, to versions between v1.4.0 and v1.12.x. + +This will *not* be a problem for Synapse installations which were: + * created at v1.4.0 or later, + * upgraded via v1.3.x, or + * upgraded straight from v1.2.1 or earlier to v1.13.0 or later. + +If completeness of the room directory is a concern, installations which are +affected can be repaired as follows: + +1. Run the following sql from a `psql` or `sqlite3` console: + + .. code:: sql + + INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES + ('populate_stats_process_rooms', '{}', 'current_state_events_membership'); + + INSERT INTO background_updates (update_name, progress_json, depends_on) VALUES + ('populate_stats_process_users', '{}', 'populate_stats_process_rooms'); + +2. Restart synapse. + +New Single Sign-on HTML Templates +--------------------------------- + +New templates (``sso_auth_confirm.html``, ``sso_auth_success.html``, and +``sso_account_deactivated.html``) were added to Synapse. If your Synapse is +configured to use SSO and a custom ``sso_redirect_confirm_template_dir`` +configuration then these templates will need to be copied from +`synapse/res/templates <synapse/res/templates>`_ into that directory. + +Synapse SSO Plugins Method Deprecation +-------------------------------------- + +Plugins using the ``complete_sso_login`` method of +``synapse.module_api.ModuleApi`` should update to using the async/await +version ``complete_sso_login_async`` which includes additional checks. The +non-async version is considered deprecated. + +Rolling back to v1.12.4 after a failed upgrade +---------------------------------------------- + +v1.13.0 includes a lot of large changes. If something problematic occurs, you +may want to roll-back to a previous version of Synapse. Because v1.13.0 also +includes a new database schema version, reverting that version is also required +alongside the generic rollback instructions mentioned above. In short, to roll +back to v1.12.4 you need to: + +1. Stop the server +2. Decrease the schema version in the database: + + .. code:: sql + + UPDATE schema_version SET version = 57; + +3. Downgrade Synapse by following the instructions for your installation method + in the "Rolling back to older versions" section above. + + +Upgrading to v1.12.0 +==================== + +This version includes a database update which is run as part of the upgrade, +and which may take some time (several hours in the case of a large +server). Synapse will not respond to HTTP requests while this update is taking +place. + +This is only likely to be a problem in the case of a server which is +participating in many rooms. + +0. As with all upgrades, it is recommended that you have a recent backup of + your database which can be used for recovery in the event of any problems. + +1. As an initial check to see if you will be affected, you can try running the + following query from the `psql` or `sqlite3` console. It is safe to run it + while Synapse is still running. + + .. code:: sql + + SELECT MAX(q.v) FROM ( + SELECT ( + SELECT ej.json AS v + FROM state_events se INNER JOIN event_json ej USING (event_id) + WHERE se.room_id=rooms.room_id AND se.type='m.room.create' AND se.state_key='' + LIMIT 1 + ) FROM rooms WHERE rooms.room_version IS NULL + ) q; + + This query will take about the same amount of time as the upgrade process: ie, + if it takes 5 minutes, then it is likely that Synapse will be unresponsive for + 5 minutes during the upgrade. + + If you consider an outage of this duration to be acceptable, no further + action is necessary and you can simply start Synapse 1.12.0. + + If you would prefer to reduce the downtime, continue with the steps below. + +2. The easiest workaround for this issue is to manually + create a new index before upgrading. On PostgreSQL, his can be done as follows: + + .. code:: sql + + CREATE INDEX CONCURRENTLY tmp_upgrade_1_12_0_index + ON state_events(room_id) WHERE type = 'm.room.create'; + + The above query may take some time, but is also safe to run while Synapse is + running. + + We assume that no SQLite users have databases large enough to be + affected. If you *are* affected, you can run a similar query, omitting the + ``CONCURRENTLY`` keyword. Note however that this operation may in itself cause + Synapse to stop running for some time. Synapse admins are reminded that + `SQLite is not recommended for use outside a test + environment <https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql>`_. + +3. Once the index has been created, the ``SELECT`` query in step 1 above should + complete quickly. It is therefore safe to upgrade to Synapse 1.12.0. + +4. Once Synapse 1.12.0 has successfully started and is responding to HTTP + requests, the temporary index can be removed: + + .. code:: sql + + DROP INDEX tmp_upgrade_1_12_0_index; + +Upgrading to v1.10.0 +==================== + +Synapse will now log a warning on start up if used with a PostgreSQL database +that has a non-recommended locale set. + +See `docs/postgres.md <docs/postgres.md>`_ for details. + + +Upgrading to v1.8.0 +=================== + +Specifying a ``log_file`` config option will now cause Synapse to refuse to +start, and should be replaced by with the ``log_config`` option. Support for +the ``log_file`` option was removed in v1.3.0 and has since had no effect. + + +Upgrading to v1.7.0 +=================== + +In an attempt to configure Synapse in a privacy preserving way, the default +behaviours of ``allow_public_rooms_without_auth`` and +``allow_public_rooms_over_federation`` have been inverted. This means that by +default, only authenticated users querying the Client/Server API will be able +to query the room directory, and relatedly that the server will not share +room directory information with other servers over federation. + +If your installation does not explicitly set these settings one way or the other +and you want either setting to be ``true`` then it will necessary to update +your homeserver configuration file accordingly. + +For more details on the surrounding context see our `explainer +<https://matrix.org/blog/2019/11/09/avoiding-unwelcome-visitors-on-private-matrix-servers>`_. + + +Upgrading to v1.5.0 +=================== + +This release includes a database migration which may take several minutes to +complete if there are a large number (more than a million or so) of entries in +the ``devices`` table. This is only likely to a be a problem on very large +installations. + Upgrading to v1.4.0 =================== -Config options --------------- +New custom templates +-------------------- -**Note: Registration by email address or phone number will not work in this release unless -some config options are changed from their defaults.** +If you have configured a custom template directory with the +``email.template_dir`` option, be aware that there are new templates regarding +registration and threepid management (see below) that must be included. + +* ``registration.html`` and ``registration.txt`` +* ``registration_success.html`` and ``registration_failure.html`` +* ``add_threepid.html`` and ``add_threepid.txt`` +* ``add_threepid_failure.html`` and ``add_threepid_success.html`` + +Synapse will expect these files to exist inside the configured template +directory, and **will fail to start** if they are absent. +To view the default templates, see `synapse/res/templates +<https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_. -This is due to Synapse v1.4.0 now defaulting to sending registration and password reset tokens -itself. This is for security reasons as well as putting less reliance on identity servers. -However, currently Synapse only supports sending emails, and does not have support for -phone-based password reset or account registration. If Synapse is configured to handle these on -its own, phone-based password resets and registration will be disabled. For Synapse to send -emails, the ``email`` block of the config must be filled out. If not, then password resets and -registration via email will be disabled entirely. +3pid verification changes +------------------------- + +**Note: As of this release, users will be unable to add phone numbers or email +addresses to their accounts, without changes to the Synapse configuration. This +includes adding an email address during registration.** + +It is possible for a user to associate an email address or phone number +with their account, for a number of reasons: + +* for use when logging in, as an alternative to the user id. +* in the case of email, as an alternative contact to help with account recovery. +* in the case of email, to receive notifications of missed messages. + +Before an email address or phone number can be added to a user's account, +or before such an address is used to carry out a password-reset, Synapse must +confirm the operation with the owner of the email address or phone number. +It does this by sending an email or text giving the user a link or token to confirm +receipt. This process is known as '3pid verification'. ('3pid', or 'threepid', +stands for third-party identifier, and we use it to refer to external +identifiers such as email addresses and phone numbers.) + +Previous versions of Synapse delegated the task of 3pid verification to an +identity server by default. In most cases this server is ``vector.im`` or +``matrix.org``. + +In Synapse 1.4.0, for security and privacy reasons, the homeserver will no +longer delegate this task to an identity server by default. Instead, +the server administrator will need to explicitly decide how they would like the +verification messages to be sent. + +In the medium term, the ``vector.im`` and ``matrix.org`` identity servers will +disable support for delegated 3pid verification entirely. However, in order to +ease the transition, they will retain the capability for a limited +period. Delegated email verification will be disabled on Monday 2nd December +2019 (giving roughly 2 months notice). Disabling delegated SMS verification +will follow some time after that once SMS verification support lands in +Synapse. + +Once delegated 3pid verification support has been disabled in the ``vector.im`` and +``matrix.org`` identity servers, all Synapse versions that depend on those +instances will be unable to verify email and phone numbers through them. There +are no imminent plans to remove delegated 3pid verification from Sydent +generally. (Sydent is the identity server project that backs the ``vector.im`` and +``matrix.org`` instances). + +Email +~~~~~ +Following upgrade, to continue verifying email (e.g. as part of the +registration process), admins can either:- + +* Configure Synapse to use an email server. +* Run or choose an identity server which allows delegated email verification + and delegate to it. + +Configure SMTP in Synapse ++++++++++++++++++++++++++ + +To configure an SMTP server for Synapse, modify the configuration section +headed ``email``, and be sure to have at least the ``smtp_host, smtp_port`` +and ``notif_from`` fields filled out. -This release also deprecates the ``email.trust_identity_server_for_password_resets`` option and -replaces it with the ``account_threepid_delegates`` dictionary. This option defines whether the -homeserver should delegate an external server (typically an `identity server -<https://matrix.org/docs/spec/identity_service/r0.2.1>`_) to handle sending password reset or -registration messages via email and SMS. +You may also need to set ``smtp_user``, ``smtp_pass``, and +``require_transport_security``. -If ``email.trust_identity_server_for_password_resets`` is set to ``true``, and +See the `sample configuration file <docs/sample_config.yaml>`_ for more details +on these settings. + +Delegate email to an identity server +++++++++++++++++++++++++++++++++++++ + +Some admins will wish to continue using email verification as part of the +registration process, but will not immediately have an appropriate SMTP server +at hand. + +To this end, we will continue to support email verification delegation via the +``vector.im`` and ``matrix.org`` identity servers for two months. Support for +delegated email verification will be disabled on Monday 2nd December. + +The ``account_threepid_delegates`` dictionary defines whether the homeserver +should delegate an external server (typically an `identity server +<https://matrix.org/docs/spec/identity_service/r0.2.1>`_) to handle sending +confirmation messages via email and SMS. + +So to delegate email verification, in ``homeserver.yaml``, set +``account_threepid_delegates.email`` to the base URL of an identity server. For +example: + +.. code:: yaml + + account_threepid_delegates: + email: https://example.com # Delegate email sending to example.com + +Note that ``account_threepid_delegates.email`` replaces the deprecated +``email.trust_identity_server_for_password_resets``: if +``email.trust_identity_server_for_password_resets`` is set to ``true``, and ``account_threepid_delegates.email`` is not set, then the first entry in -``trusted_third_party_id_servers`` will be used as the account threepid delegate for email. -This is to ensure compatibility with existing Synapse installs that set up external server -handling for these tasks before v1.4.0. If ``email.trust_identity_server_for_password_resets`` -is ``true`` and no trusted identity server domains are configured, Synapse will throw an error. +``trusted_third_party_id_servers`` will be used as the +``account_threepid_delegate`` for email. This is to ensure compatibility with +existing Synapse installs that set up external server handling for these tasks +before v1.4.0. If ``email.trust_identity_server_for_password_resets`` is +``true`` and no trusted identity server domains are configured, Synapse will +report an error and refuse to start. + +If ``email.trust_identity_server_for_password_resets`` is ``false`` or absent +and no ``email`` delegate is configured in ``account_threepid_delegates``, +then Synapse will send email verification messages itself, using the configured +SMTP server (see above). +that type. + +Phone numbers +~~~~~~~~~~~~~ + +Synapse does not support phone-number verification itself, so the only way to +maintain the ability for users to add phone numbers to their accounts will be +by continuing to delegate phone number verification to the ``matrix.org`` and +``vector.im`` identity servers (or another identity server that supports SMS +sending). + +The ``account_threepid_delegates`` dictionary defines whether the homeserver +should delegate an external server (typically an `identity server +<https://matrix.org/docs/spec/identity_service/r0.2.1>`_) to handle sending +confirmation messages via email and SMS. + +So to delegate phone number verification, in ``homeserver.yaml``, set +``account_threepid_delegates.msisdn`` to the base URL of an identity +server. For example: + +.. code:: yaml + + account_threepid_delegates: + msisdn: https://example.com # Delegate sms sending to example.com + +The ``matrix.org`` and ``vector.im`` identity servers will continue to support +delegated phone number verification via SMS until such time as it is possible +for admins to configure their servers to perform phone number verification +directly. More details will follow in a future release. + +Rolling back to v1.3.1 +---------------------- -If ``email.trust_identity_server_for_password_resets`` is ``false`` or absent and a threepid -type in ``account_threepid_delegates`` is not set to a domain, then Synapse will attempt to -send password reset and registration messages for that type. +If you encounter problems with v1.4.0, it should be possible to roll back to +v1.3.1, subject to the following: -Email templates ---------------- +* The 'room statistics' engine was heavily reworked in this release (see + `#5971 <https://github.com/matrix-org/synapse/pull/5971>`_), including + significant changes to the database schema, which are not easily + reverted. This will cause the room statistics engine to stop updating when + you downgrade. -If you have configured a custom template directory with the ``email.template_dir`` option, be -aware that there are new templates regarding registration. ``registration.html`` and -``registration.txt`` have been added and contain the content that is sent to a client upon -registering via an email address. + The room statistics are essentially unused in v1.3.1 (in future versions of + Synapse, they will be used to populate the room directory), so there should + be no loss of functionality. However, the statistics engine will write errors + to the logs, which can be avoided by setting the following in + `homeserver.yaml`: -``registration_success.html`` and ``registration_failure.html`` are also new HTML templates -that will be shown to the user when they click the link in their registration emai , either -showing them a success or failure page (assuming a redirect URL is not configured). + .. code:: yaml -Synapse will expect these files to exist inside the configured template directory. To view the -default templates, see `synapse/res/templates -<https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_. + stats: + enabled: false + + Don't forget to re-enable it when you upgrade again, in preparation for its + use in the room directory! Upgrading to v1.2.0 =================== |