From e6862512dc0d78dc069e8799c06b47024ed167d3 Mon Sep 17 00:00:00 2001 From: erikjohnston Date: Tue, 22 Feb 2022 12:26:55 +0000 Subject: deploy: 07f82ac29be42098ab5b135cb173437f9cd7c754 --- latest/print.html | 388 +++++++++++++++++------------------------------------- 1 file changed, 122 insertions(+), 266 deletions(-) (limited to 'latest/print.html') diff --git a/latest/print.html b/latest/print.html index a034ec9dc0..8b5ea5623c 100644 --- a/latest/print.html +++ b/latest/print.html @@ -101,7 +101,7 @@ @@ -1638,6 +1638,52 @@ dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb +

Upgrading to v1.53.0

+

Dropping support for webclient listeners and non-HTTP(S) web_client_location

+

Per the deprecation notice in Synapse v1.51.0, listeners of type webclient +are no longer supported and configuring them is a now a configuration error.

+

Configuring a non-HTTP(S) web_client_location configuration is is now a +configuration error. Since the webclient listener is no longer supported, this +setting only applies to the root path / of Synapse's web server and no longer +the /_matrix/client/ path.

+

Stablisation of MSC3231

+

The unstable validity-check endpoint for the +Registration Tokens +feature has been stabilised and moved from:

+

/_matrix/client/unstable/org.matrix.msc3231/register/org.matrix.msc3231.login.registration_token/validity

+

to:

+

/_matrix/client/v1/register/m.login.registration_token/validity

+

Please update any relevant reverse proxy or firewall configurations appropriately.

+

Time-based cache expiry is now enabled by default

+

Formerly, entries in the cache were not evicted regardless of whether they were accessed after storing. +This behavior has now changed. By default entries in the cache are now evicted after 30m of not being accessed. +To change the default behavior, go to the caches section of the config and change the expire_caches and +cache_entry_ttl flags as necessary. Please note that these flags replace the expiry_time flag in the config.
+The expiry_time flag will still continue to work, but it has been deprecated and will be removed in the future.

+

Deprecation of capability org.matrix.msc3283.*

+

The capabilities of MSC3283 from the REST API /_matrix/client/r0/capabilities +becomes stable.

+

The old capabilities

+ +

are deprecated and scheduled to be removed in Synapse v1.54.0.

+

The new capabilities

+ +

are now active by default.

+

Removal of user_may_create_room_with_invites

+

As announced with the release of Synapse 1.47.0, +the deprecated user_may_create_room_with_invites module callback has been removed.

+

Modules relying on it can instead implement user_may_invite +and use the get_room_state +module API to infer whether the invite is happening while creating a room (see this function +as an example). Alternately, modules can also implement on_create_room.

Upgrading to v1.52.0

Twisted security release

Note that Twisted 22.1.0 @@ -2463,8 +2509,7 @@ more details on upgrading your database.

Validation of TLS certificates

Synapse v1.0 is the first release to enforce validation of TLS certificates for the federation API. It is therefore essential that your -certificates are correctly configured. See the -FAQ for more information.

+certificates are correctly configured.

Note, v1.0 installations will also no longer be able to federate with servers that have not correctly configured their certificates.

In rare cases, it may be desirable to disable certificate checking: for @@ -2514,8 +2559,6 @@ sent to them.

you will need to replace any self-signed certificates with those verified by a root CA. Information on how to do so can be found at the ACME docs.

-

For more information on configuring TLS certificates see the -FAQ.

Upgrading to v0.34.0

  1. @@ -2804,257 +2847,6 @@ longer to restart than usual as it reinitializes the database.

    using room aliases or by being reinvited. Alternatively, if any other homeserver sends a message to a room that the homeserver was previously in the local HS will automatically rejoin the room.

    -

    MSC1711 Certificates FAQ

    -

    Historical Note

    -

    This document was originally written to guide server admins through the upgrade -path towards Synapse 1.0. Specifically, -MSC1711 -required that all servers present valid TLS certificates on their federation -API. Admins were encouraged to achieve compliance from version 0.99.0 (released -in February 2019) ahead of version 1.0 (released June 2019) enforcing the -certificate checks.

    -

    Much of what follows is now outdated since most admins will have already -upgraded, however it may be of use to those with old installs returning to the -project.

    -

    If you are setting up a server from scratch you almost certainly should look at -the installation guide instead.

    -

    Introduction

    -

    The goal of Synapse 0.99.0 is to act as a stepping stone to Synapse 1.0.0. It -supports the r0.1 release of the server to server specification, but is -compatible with both the legacy Matrix federation behaviour (pre-r0.1) as well -as post-r0.1 behaviour, in order to allow for a smooth upgrade across the -federation.

    -

    The most important thing to know is that Synapse 1.0.0 will require a valid TLS -certificate on federation endpoints. Self signed certificates will not be -sufficient.

    -

    Synapse 0.99.0 makes it easy to configure TLS certificates and will -interoperate with both >= 1.0.0 servers as well as existing servers yet to -upgrade.

    -

    It is critical that all admins upgrade to 0.99.0 and configure a valid TLS -certificate. Admins will have 1 month to do so, after which 1.0.0 will be -released and those servers without a valid certificate will not longer be able -to federate with >= 1.0.0 servers.

    -

    Full details on how to carry out this configuration change is given -below. A -timeline and some frequently asked questions are also given below.

    -

    For more details and context on the release of the r0.1 Server/Server API and -imminent Matrix 1.0 release, you can also see our -main talk from FOSDEM 2019.

    -

    Timeline

    -

    5th Feb 2019 - Synapse 0.99.0 is released.

    -

    All server admins are encouraged to upgrade.

    -

    0.99.0:

    -
      -
    • -

      provides support for ACME to make setting up Let's Encrypt certs easy, as -well as .well-known support.

      -
      • -
    • -

      does not enforce that a valid CA cert is present on the federation API, but -rather makes it easy to set one up.

      -
      • -
    • -

      provides support for .well-known

      -
      • -
    -

    Admins should upgrade and configure a valid CA cert. Homeservers that require a -.well-known entry (see below), should retain their SRV record and use it -alongside their .well-known record.

    -

    10th June 2019 - Synapse 1.0.0 is released

    -

    1.0.0 is scheduled for release on 10th June. In -accordance with the the S2S spec -1.0.0 will enforce certificate validity. This means that any homeserver without a -valid certificate after this point will no longer be able to federate with -1.0.0 servers.

    -

    Configuring certificates for compatibility with Synapse 1.0.0

    -

    If you do not currently have an SRV record

    -

    In this case, your server_name points to the host where your Synapse is -running. There is no need to create a .well-known URI or an SRV record, but -you will need to give Synapse a valid, signed, certificate.

    -

    If you do have an SRV record currently

    -

    If you are using an SRV record, your matrix domain (server_name) may not -point to the same host that your Synapse is running on (the 'target -domain'). (If it does, you can follow the recommendation above; otherwise, read -on.)

    -

    Let's assume that your server_name is example.com, and your Synapse is -hosted at a target domain of customer.example.net. Currently you should have -an SRV record which looks like:

    -
    _matrix._tcp.example.com. IN SRV 10 5 8000 customer.example.net.
-
    -

    In this situation, you have three choices for how to proceed:

    -

    Option 1: give Synapse a certificate for your matrix domain

    -

    Synapse 1.0 will expect your server to present a TLS certificate for your -server_name (example.com in the above example). You can achieve this by acquiring a -certificate for the server_name yourself (for example, using certbot), and giving it -and the key to Synapse via tls_certificate_path and tls_private_key_path.

    -

    Option 2: run Synapse behind a reverse proxy

    -

    If you have an existing reverse proxy set up with correct TLS certificates for -your domain, you can simply route all traffic through the reverse proxy by -updating the SRV record appropriately (or removing it, if the proxy listens on -8448).

    -

    See the reverse proxy documentation for information on setting up a -reverse proxy.

    -

    Option 3: add a .well-known file to delegate your matrix traffic

    -

    This will allow you to keep Synapse on a separate domain, without having to -give it a certificate for the matrix domain.

    -

    You can do this with a .well-known file as follows:

    -
      -
    1. -

      Keep the SRV record in place - it is needed for backwards compatibility -with Synapse 0.34 and earlier.

      -
      2. -
    2. -

      Give Synapse a certificate corresponding to the target domain -(customer.example.net in the above example). You can do this by acquire a -certificate for the target domain and giving it to Synapse via tls_certificate_path -and tls_private_key_path.

      -
      3. -
    3. -

      Restart Synapse to ensure the new certificate is loaded.

      -
      4. -
    4. -

      Arrange for a .well-known file at -https://<server_name>/.well-known/matrix/server with contents:

      -
      {"m.server": "<target server name>"}
-
      -

      where the target server name is resolved as usual (i.e. SRV lookup, falling -back to talking to port 8448).

      -

      In the above example, where synapse is listening on port 8000, -https://example.com/.well-known/matrix/server should have m.server set to one of:

      -
        -
      1. -

        customer.example.net ─ with a SRV record on -_matrix._tcp.customer.example.com pointing to port 8000, or:

        -
        2. -
      2. -

        customer.example.net ─ updating synapse to listen on the default port -8448, or:

        -
        3. -
      3. -

        customer.example.net:8000 ─ ensuring that if there is a reverse proxy -on customer.example.net:8000 it correctly handles HTTP requests with -Host header set to customer.example.net:8000.

        -
        4. -
      -
      5. -
    -

    FAQ

    -

    Synapse 0.99.0 has just been released, what do I need to do right now?

    -

    Upgrade as soon as you can in preparation for Synapse 1.0.0, and update your -TLS certificates as above.

    -

    What will happen if I do not set up a valid federation certificate immediately?

    -

    Nothing initially, but once 1.0.0 is in the wild it will not be possible to -federate with 1.0.0 servers.

    -

    What will happen if I do nothing at all?

    -

    If the admin takes no action at all, and remains on a Synapse < 0.99.0 then the -homeserver will be unable to federate with those who have implemented -.well-known. Then, as above, once the month upgrade window has expired the -homeserver will not be able to federate with any Synapse >= 1.0.0

    -

    When do I need a SRV record or .well-known URI?

    -

    If your homeserver listens on the default federation port (8448), and your -server_name points to the host that your homeserver runs on, you do not need an -SRV record or .well-known/matrix/server URI.

    -

    For instance, if you registered example.com and pointed its DNS A record at a -fresh Upcloud VPS or similar, you could install Synapse 0.99 on that host, -giving it a server_name of example.com, and it would automatically generate a -valid TLS certificate for you via Let's Encrypt and no SRV record or -.well-known URI would be needed.

    -

    This is the common case, although you can add an SRV record or -.well-known/matrix/server URI for completeness if you wish.

    -

    However, if your server does not listen on port 8448, or if your server_name -does not point to the host that your homeserver runs on, you will need to let -other servers know how to find it.

    -

    In this case, you should see "If you do have an SRV record -currently" above.

    -

    Can I still use an SRV record?

    -

    Firstly, if you didn't need an SRV record before (because your server is -listening on port 8448 of your server_name), you certainly don't need one now: -the defaults are still the same.

    -

    If you previously had an SRV record, you can keep using it provided you are -able to give Synapse a TLS certificate corresponding to your server name. For -example, suppose you had the following SRV record, which directs matrix traffic -for example.com to matrix.example.com:443:

    -
    _matrix._tcp.example.com. IN SRV 10 5 443 matrix.example.com
-
    -

    In this case, Synapse must be given a certificate for example.com - or be -configured to acquire one from Let's Encrypt.

    -

    If you are unable to give Synapse a certificate for your server_name, you will -also need to use a .well-known URI instead. However, see also "I have created a -.well-known URI. Do I still need an SRV record?".

    -

    I have created a .well-known URI. Do I still need an SRV record?

    -

    As of Synapse 0.99, Synapse will first check for the existence of a .well-known -URI and follow any delegation it suggests. It will only then check for the -existence of an SRV record.

    -

    That means that the SRV record will often be redundant. However, you should -remember that there may still be older versions of Synapse in the federation -which do not understand .well-known URIs, so if you removed your SRV record you -would no longer be able to federate with them.

    -

    It is therefore best to leave the SRV record in place for now. Synapse 0.34 and -earlier will follow the SRV record (and not care about the invalid -certificate). Synapse 0.99 and later will follow the .well-known URI, with the -correct certificate chain.

    -

    It used to work just fine, why are you breaking everything?

    -

    We have always wanted Matrix servers to be as easy to set up as possible, and -so back when we started federation in 2014 we didn't want admins to have to go -through the cumbersome process of buying a valid TLS certificate to run a -server. This was before Let's Encrypt came along and made getting a free and -valid TLS certificate straightforward. So instead, we adopted a system based on -Perspectives: an approach -where you check a set of "notary servers" (in practice, homeservers) to vouch -for the validity of a certificate rather than having it signed by a CA. As long -as enough different notaries agree on the certificate's validity, then it is -trusted.

    -

    However, in practice this has never worked properly. Most people only use the -default notary server (matrix.org), leading to inadvertent centralisation which -we want to eliminate. Meanwhile, we never implemented the full consensus -algorithm to query the servers participating in a room to determine consensus -on whether a given certificate is valid. This is fiddly to get right -(especially in face of sybil attacks), and we found ourselves questioning -whether it was worth the effort to finish the work and commit to maintaining a -secure certificate validation system as opposed to focusing on core Matrix -development.

    -

    Meanwhile, Let's Encrypt came along in 2016, and put the final nail in the -coffin of the Perspectives project (which was already pretty dead). So, the -Spec Core Team decided that a better approach would be to mandate valid TLS -certificates for federation alongside the rest of the Web. More details can be -found in -MSC1711.

    -

    This results in a breaking change, which is disruptive, but absolutely critical -for the security model. However, the existence of Let's Encrypt as a trivial -way to replace the old self-signed certificates with valid CA-signed ones helps -smooth things over massively, especially as Synapse can now automate Let's -Encrypt certificate generation if needed.

    -

    Can I manage my own certificates rather than having Synapse renew certificates itself?

    -

    Yes, you are welcome to manage your certificates yourself. Synapse will only -attempt to obtain certificates from Let's Encrypt if you configure it to do -so.The only requirement is that there is a valid TLS cert present for -federation end points.

    -

    Do you still recommend against using a reverse proxy on the federation port?

    -

    We no longer actively recommend against using a reverse proxy. Many admins will -find it easier to direct federation traffic to a reverse proxy and manage their -own TLS certificates, and this is a supported configuration.

    -

    See the reverse proxy documentation for information on setting up a -reverse proxy.

    -

    Do I still need to give my TLS certificates to Synapse if I am using a reverse proxy?

    -

    Practically speaking, this is no longer necessary.

    -

    If you are using a reverse proxy for all of your TLS traffic, then you can set -no_tls: True. In that case, the only reason Synapse needs the certificate is -to populate a legacy 'tls_fingerprints' field in the federation API. This is -ignored by Synapse 0.99.0 and later, and the only time pre-0.99 Synapses will -check it is when attempting to fetch the server keys - and generally this is -delegated via matrix.org, which is on 0.99.0.

    -

    However, there is a bug in Synapse 0.99.0 -4554 which prevents -Synapse from starting if you do not give it a TLS certificate. To work around -this, you can give it any TLS certificate at all. This will be fixed soon.

    -

    Do I need the same certificate for the client and federation port?

    -

    No. There is nothing stopping you from using different certificates, -particularly if you are using a reverse proxy. However, Synapse will use the -same certificate on any ports where TLS is configured.

    -

    How do I tell Synapse to reload my keys/certificates after I replace them?

    -

    Synapse will reload the keys and certificates when it receives a SIGHUP - for -example kill -HUP $(cat homeserver.pid). Alternatively, simply restart -Synapse, though this will result in downtime while it restarts.

    Setting up federation

    Federation is the process by which users on different servers can participate in the same room. For this to work, those other servers must be able to contact @@ -3871,11 +3663,16 @@ caches: per_cache_factors: #get_users_who_share_room_with_user: 2.0 - # Controls how long an entry can be in a cache without having been - # accessed before being evicted. Defaults to None, which means - # entries are never evicted based on time. + # Controls whether cache entries are evicted after a specified time + # period. Defaults to true. Uncomment to disable this feature. + # + #expire_caches: false + + # If expire_caches is enabled, this flag controls how long an entry can + # be in a cache without having been accessed before being evicted. + # Defaults to 30m. Uncomment to set a different time to live for cache entries. # - #expiry_time: 30m + #cache_entry_ttl: 30m # Controls how long the results of a /sync request are cached for after # a successful response is returned. A higher duration can help clients with @@ -3977,6 +3774,9 @@ log_config: "CONFDIR/SERVERNAME.log.config" # - one for ratelimiting how often a user or IP can attempt to validate a 3PID. # - two for ratelimiting how often invites can be sent in a room or to a # specific user. +# - one for ratelimiting 3PID invites (i.e. invites sent to a third-party ID +# such as an email address or a phone number) based on the account that's +# sending the invite. # # The defaults are as shown below. # @@ -4026,6 +3826,10 @@ log_config: "CONFDIR/SERVERNAME.log.config" # per_user: # per_second: 0.003 # burst_count: 5 +# +#rc_third_party_invite: +# per_second: 0.2 +# burst_count: 10 # Ratelimiting settings for incoming federation # @@ -6054,7 +5858,7 @@ formatters: handlers: console: class: logging.StreamHandler - location: ext://sys.stdout + stream: ext://sys.stdout file: class: logging.FileHandler formatter: json @@ -8702,7 +8506,7 @@ Here's an example featuring all currently supported keys:

    "address": "33123456789", "validated_at": 1642701357084, }, - "org.matrix.msc3231.login.registration_token": "sometoken", # User has registered through the flow described in MSC3231 + "m.login.registration_token": "sometoken", # User has registered through a registration token }

    The second dictionary contains the parameters provided by the user's client in the request @@ -8716,6 +8520,20 @@ callback that does not return None will be used. If this happens, S any of the subsequent implementations of this callback. If every callback return None, the username provided by the user is used, if any (otherwise one is automatically generated).

    +

    is_3pid_allowed

    +

    First introduced in Synapse v1.53.0

    +
    async def is_3pid_allowed(self, medium: str, address: str, registration: bool) -> bool
+
    +

    Called when attempting to bind a third-party identifier (i.e. an email address or a phone +number). The module is given the medium of the third-party identifier (which is email if +the identifier is an email address, or msisdn if the identifier is a phone number) and +its address, as well as a boolean indicating whether the attempt to bind is happening as +part of registering a new user. The module must return a boolean indicating whether the +identifier can be allowed to be bound to an account on the local homeserver.

    +

    If multiple modules implement this callback, they will be considered in order. If a +callback returns True, Synapse falls through to the next one. The value of the first +callback that does not return True will be used. If this happens, Synapse will not call +any of the subsequent implementations of this callback.

    Example

    The example module below implements authentication checkers for two different login types:

      @@ -9049,7 +8867,7 @@ expressions:

      # Registration/login requests ^/_matrix/client/(api/v1|r0|v3|unstable)/login$ ^/_matrix/client/(r0|v3|unstable)/register$ -^/_matrix/client/unstable/org.matrix.msc3231/register/org.matrix.msc3231.login.registration_token/validity$ +^/_matrix/client/v1/register/m.login.registration_token/validity$ # Event sending requests ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/redact @@ -9694,6 +9512,8 @@ have a canonical alias set.

    Querying media

    These APIs allow extracting media information from the homeserver.

    +

    Details about the format of the media_id and storage of the media in the file system +are documented under media repository.

    To use it, you will need to authenticate by providing an access_token for a server admin: see Admin API.

    List all media in a room

    @@ -11499,7 +11319,7 @@ were sent, but hidden from users joining the room afterwards.

    An empty body may be passed for backwards compatibility.

    The following actions are performed when deactivating an user:

      -
    • Try to unpind 3PIDs from the identity server
      • +
    • Try to unbind 3PIDs from the identity server
    • Remove all 3PIDs from the homeserver
    • Delete all devices and E2EE keys
    • Delete all access tokens
      • @@ -11666,7 +11486,11 @@ member are returned.

      User media

      List media uploaded by a user

      Gets a list of all local media that a specific user_id has created. -By default, the response is ordered by descending creation date and ascending media ID. +These are media that the user has uploaded themselves +(local media), as well as +URL preview images requested by the user if the +feature is enabled.

      +

      By default, the response is ordered by descending creation date and ascending media ID. The newest media is on top. You can change the order with parameters order_by and dir.

      The API is:

      @@ -11760,7 +11584,9 @@ Media objects contain the following fields:
      • created_ts - integer - Timestamp when the content was uploaded in ms.
      • last_access_ts - integer - Timestamp when the content was last accessed in ms.
        • -
      • media_id - string - The id used to refer to the media.
        • +
      • media_id - string - The id used to refer to the media. Details about the format +are documented under +media repository.
      • media_length - integer - Length of the media in bytes.
      • media_type - string - The MIME-type of the media.
      • quarantined_by - string - The user ID that initiated the quarantine request @@ -13507,6 +13333,36 @@ frobber: and is maintained by a script, scripts-dev/generate_sample_config. Making sure that the output from this script matches the desired format is left as an exercise for the reader!

        +

        Synapse Release Cycle

        +

        Releases of Synapse follow a two week release cycle with new releases usually +occurring on Tuesdays:

        +
          +
        • Day 0: Synapse N - 1 is released.
          • +
        • Day 7: Synapse N release candidate 1 is released.
          • +
        • Days 7 - 13: Synapse N release candidates 2+ are released, if bugs are found.
          • +
        • Day 14: Synapse N is released.
          • +
        +

        Note that this schedule might be modified depending on the availability of the +Synapse team, e.g. releases may be skipped to avoid holidays.

        +

        Release announcements can be found in the +release category of the Matrix blog.

        +

        Bugfix releases

        +

        If a bug is found after release that is deemed severe enough (by a combination +of the impacted users and the impact on those users) then a bugfix release may +be issued. This may be at any point in the release cycle.

        +

        Security releases

        +

        Security will sometimes be backported to the previous version and released +immediately before the next release candidate. An example of this might be:

        +
          +
        • Day 0: Synapse N - 1 is released.
          • +
        • Day 7: Synapse (N - 1).1 is released as Synapse N - 1 + the security fix.
          • +
        • Day 7: Synapse N release candidate 1 is released (including the security fix).
          • +
        +

        Depending on the impact and complexity of security fixes, multiple fixes might +be held to be released together.

        +

        In some cases, a pre-disclosure of a security release will be issued as a notice +to Synapse operators that there is an upcoming security release. These can be +found in the security category of the Matrix blog.

        Some notes on how we use git

        On keeping the commit history clean

        In an ideal world, our git commit history would be a linear progression of -- cgit 1.5.1