From d225cfe2cabb01eb6982d26cd6d410b90934d1c2 Mon Sep 17 00:00:00 2001 From: babolivier Date: Tue, 5 Oct 2021 13:13:48 +0000 Subject: deploy: b2c5e79291b9f93cdb39c9a6f7de50e62f45e64e --- latest/print.html | 360 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 235 insertions(+), 125 deletions(-) (limited to 'latest/print.html') diff --git a/latest/print.html b/latest/print.html index dc2b03ddb4..1b32b73ca4 100644 --- a/latest/print.html +++ b/latest/print.html @@ -101,7 +101,7 @@ @@ -197,13 +197,118 @@ that your email address is probably user@example.com rather than user@email.example.com) - but doing so may require more advanced setup: see Setting up Federation.

Installing Synapse

-

Installing from source

-

(Prebuilt packages are available for some platforms - see Prebuilt packages.)

-

When installing from source please make sure that the Platform-specific prerequisites are already installed.

+

Prebuilt packages

+

Prebuilt packages are available for a number of platforms. These are recommended +for most users.

+

Docker images and Ansible playbooks

+

There is an official synapse image available at +https://hub.docker.com/r/matrixdotorg/synapse which can be used with +the docker-compose file available at +contrib/docker. +Further information on this including configuration options is available in the README +on hub.docker.com.

+

Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a +Dockerfile to automate a synapse server in a single Docker image, at +https://hub.docker.com/r/avhost/docker-matrix/tags/

+

Slavi Pantaleev has created an Ansible playbook, +which installs the offical Docker image of Matrix Synapse +along with many other Matrix-related services (Postgres database, Element, coturn, +ma1sd, SSL support, etc.). +For more details, see +https://github.com/spantaleev/matrix-docker-ansible-deploy

+

Debian/Ubuntu

+
Matrix.org packages
+

Matrix.org provides Debian/Ubuntu packages of Synapse, for the amd64 +architecture via https://packages.matrix.org/debian/.

+

To install the latest release:

+
sudo apt install -y lsb-release wget apt-transport-https
+sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
+echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
+    sudo tee /etc/apt/sources.list.d/matrix-org.list
+sudo apt update
+sudo apt install matrix-synapse-py3
+
+

Packages are also published for release candidates. To enable the prerelease +channel, add prerelease to the sources.list line. For example:

+
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
+echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main prerelease" |
+    sudo tee /etc/apt/sources.list.d/matrix-org.list
+sudo apt update
+sudo apt install matrix-synapse-py3
+
+

The fingerprint of the repository signing key (as shown by gpg /usr/share/keyrings/matrix-org-archive-keyring.gpg) is +AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058.

+
Downstream Debian packages
+

We do not recommend using the packages from the default Debian buster +repository at this time, as they are old and suffer from known security +vulnerabilities. You can install the latest version of Synapse from +our repository or from buster-backports. Please +see the Debian documentation +for information on how to use backports.

+

If you are using Debian sid or testing, Synapse is available in the default +repositories and it should be possible to install it simply with:

+
sudo apt install matrix-synapse
+
+
Downstream Ubuntu packages
+

We do not recommend using the packages in the default Ubuntu repository +at this time, as they are old and suffer from known security vulnerabilities. +The latest version of Synapse can be installed from our repository.

+

Fedora

+

Synapse is in the Fedora repositories as matrix-synapse:

+
sudo dnf install matrix-synapse
+
+

Oleg Girko provides Fedora RPMs at +https://obs.infoserver.lv/project/monitor/matrix-synapse

+

OpenSUSE

+

Synapse is in the OpenSUSE repositories as matrix-synapse:

+
sudo zypper install matrix-synapse
+
+

SUSE Linux Enterprise Server

+

Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at +https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/

+

ArchLinux

+

The quickest way to get up and running with ArchLinux is probably with the community package +https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of +the necessary dependencies.

+

pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):

+
sudo pip install --upgrade pip
+
+

If you encounter an error with lib bcrypt causing an Wrong ELF Class: +ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly +compile it under the right architecture. (This should not be needed if +installing under virtualenv):

+
sudo pip uninstall py-bcrypt
+sudo pip install py-bcrypt
+
+

Void Linux

+

Synapse can be found in the void repositories as 'synapse':

+
xbps-install -Su
+xbps-install -S synapse
+
+

FreeBSD

+

Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:

+ +

OpenBSD

+

As of OpenBSD 6.7 Synapse is available as a pre-compiled binary. The filesystem +underlying the homeserver directory (defaults to /var/synapse) has to be +mounted with wxallowed (cf. mount(8)), so creating a separate filesystem +and mounting it to /var/synapse should be taken into consideration.

+

Installing Synapse:

+
doas pkg_add synapse
+
+

NixOS

+

Robin Lambertz has packaged Synapse for NixOS at: +https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix

+

Installing as a Python module from PyPI

+

It's also possible to install Synapse as a Python module from PyPI.

+

When following this route please make sure that the Platform-specific prerequisites are already installed.

System requirements:

To install the Synapse homeserver run:

@@ -257,7 +362,7 @@ header files for Python C extensions.

python3-pip python3-setuptools sqlite3 \ libssl-dev virtualenv libjpeg-dev libxslt1-dev -
ArchLinux
+
ArchLinux

Installing prerequisites on ArchLinux:

sudo pacman -S base-devel python python-pip \
                python-setuptools python-virtualenv sqlite3
@@ -283,13 +388,13 @@ via brew and inform pip about it so that psycopg2 buil
 export LDFLAGS="-L/usr/local/opt/openssl/lib"
 export CPPFLAGS="-I/usr/local/opt/openssl/include"
-
OpenSUSE
+
OpenSUSE

Installing prerequisites on openSUSE:

sudo zypper in -t pattern devel_basis
 sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
                python-devel libffi-devel libopenssl-devel libjpeg62-devel
-
OpenBSD
+
OpenBSD

A port of Synapse is available under net/synapse. The filesystem underlying the homeserver directory (defaults to /var/synapse) has to be mounted with wxallowed (cf. mount(8)), so creating a separate filesystem @@ -319,110 +424,6 @@ Debian, Fedora, or source installation methods. More information about WSL can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server for Windows Server.

-

Prebuilt packages

-

As an alternative to installing from source, prebuilt packages are available -for a number of platforms.

-

Docker images and Ansible playbooks

-

There is an official synapse image available at -https://hub.docker.com/r/matrixdotorg/synapse which can be used with -the docker-compose file available at -contrib/docker. -Further information on this including configuration options is available in the README -on hub.docker.com.

-

Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a -Dockerfile to automate a synapse server in a single Docker image, at -https://hub.docker.com/r/avhost/docker-matrix/tags/

-

Slavi Pantaleev has created an Ansible playbook, -which installs the offical Docker image of Matrix Synapse -along with many other Matrix-related services (Postgres database, Element, coturn, -ma1sd, SSL support, etc.). -For more details, see -https://github.com/spantaleev/matrix-docker-ansible-deploy

-

Debian/Ubuntu

-
Matrix.org packages
-

Matrix.org provides Debian/Ubuntu packages of Synapse via -https://packages.matrix.org/debian/. To install the latest release:

-
sudo apt install -y lsb-release wget apt-transport-https
-sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
-echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
-    sudo tee /etc/apt/sources.list.d/matrix-org.list
-sudo apt update
-sudo apt install matrix-synapse-py3
-
-

Packages are also published for release candidates. To enable the prerelease -channel, add prerelease to the sources.list line. For example:

-
sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
-echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main prerelease" |
-    sudo tee /etc/apt/sources.list.d/matrix-org.list
-sudo apt update
-sudo apt install matrix-synapse-py3
-
-

The fingerprint of the repository signing key (as shown by gpg /usr/share/keyrings/matrix-org-archive-keyring.gpg) is -AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058.

-
Downstream Debian packages
-

We do not recommend using the packages from the default Debian buster -repository at this time, as they are old and suffer from known security -vulnerabilities. You can install the latest version of Synapse from -our repository or from buster-backports. Please -see the Debian documentation -for information on how to use backports.

-

If you are using Debian sid or testing, Synapse is available in the default -repositories and it should be possible to install it simply with:

-
sudo apt install matrix-synapse
-
-
Downstream Ubuntu packages
-

We do not recommend using the packages in the default Ubuntu repository -at this time, as they are old and suffer from known security vulnerabilities. -The latest version of Synapse can be installed from our repository.

-

Fedora

-

Synapse is in the Fedora repositories as matrix-synapse:

-
sudo dnf install matrix-synapse
-
-

Oleg Girko provides Fedora RPMs at -https://obs.infoserver.lv/project/monitor/matrix-synapse

-

OpenSUSE

-

Synapse is in the OpenSUSE repositories as matrix-synapse:

-
sudo zypper install matrix-synapse
-
-

SUSE Linux Enterprise Server

-

Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at -https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/

-

ArchLinux

-

The quickest way to get up and running with ArchLinux is probably with the community package -https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of -the necessary dependencies.

-

pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):

-
sudo pip install --upgrade pip
-
-

If you encounter an error with lib bcrypt causing an Wrong ELF Class: -ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly -compile it under the right architecture. (This should not be needed if -installing under virtualenv):

-
sudo pip uninstall py-bcrypt
-sudo pip install py-bcrypt
-
-

Void Linux

-

Synapse can be found in the void repositories as 'synapse':

-
xbps-install -Su
-xbps-install -S synapse
-
-

FreeBSD

-

Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:

- -

OpenBSD

-

As of OpenBSD 6.7 Synapse is available as a pre-compiled binary. The filesystem -underlying the homeserver directory (defaults to /var/synapse) has to be -mounted with wxallowed (cf. mount(8)), so creating a separate filesystem -and mounting it to /var/synapse should be taken into consideration.

-

Installing Synapse:

-
doas pkg_add synapse
-
-

NixOS

-

Robin Lambertz has packaged Synapse for NixOS at: -https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix

Setting up Synapse

Once you have installed synapse as above, you will need to configure it.

Using PostgreSQL

@@ -1487,6 +1488,11 @@ dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb +

Upgrading to v1.44.0

+

The URL preview cache is no longer mirrored to storage providers

+

The url_cache/ and url_cache_thumbnails/ directories in the media store are +no longer mirrored to storage providers. These two directories can be safely +deleted from any configured storage providers to reclaim space.

Upgrading to v1.43.0

The spaces summary APIs can now be handled by workers

The available worker applications documentation @@ -5307,12 +5313,16 @@ user_directory: #enabled: false # Defines whether to search all users visible to your HS when searching - # the user directory, rather than limiting to users visible in public - # rooms. Defaults to false. + # the user directory. If false, search results will only contain users + # visible in public rooms and users sharing a room with the requester. + # Defaults to false. # - # If you set it true, you'll have to rebuild the user_directory search - # indexes, see: - # https://matrix-org.github.io/synapse/latest/user_directory.html + # NB. If you set this to true, and the last time the user_directory search + # indexes were (re)built was before Synapse 1.44, you'll have to + # rebuild the indexes in order to search through all known users. + # These indexes are built the first time Synapse starts; admins can + # manually trigger a rebuild following the instructions at + # https://matrix-org.github.io/synapse/latest/user_directory.html # # Uncomment to return search results containing all known users, even if that # user does not share a room with the requester. @@ -7300,12 +7310,10 @@ is also used to de-duplicate processing of multiple in-flight requests at once.)

  1. Checks the database cache by URL and timestamp and returns the result if it has not expired and was successful (a 2xx return code).
    2. -
  2. Checks if the URL matches an oEmbed pattern. If it does, fetch the oEmbed -response. If this is an image, replace the URL to fetch and continue. If -if it is HTML content, use the HTML as the document and continue.
    3. -
  3. If it doesn't match an oEmbed pattern, downloads the URL and stores it -into a file via the media storage provider and saves the local media -metadata.
    4. +
  4. Checks if the URL matches an oEmbed pattern. If it +does, update the URL to download.
    5. +
  5. Downloads the URL and stores it into a file via the media storage provider +and saves the local media metadata.
  6. If the media is an image:
    1. Generates thumbnails.
      2. @@ -7326,6 +7334,19 @@ provider and saves the local media metadata.
    7. +
  7. If the media is JSON and an oEmbed URL was found: +
      +
    1. Convert the oEmbed response to an Open Graph response.
      2. +
    2. If a thumbnail or image is in the oEmbed response: +
        +
      1. Downloads the URL and stores it into a file via the media storage +provider and saves the local media metadata.
        2. +
      2. Generates thumbnails.
        3. +
      3. Updates the Open Graph response based on image properties.
        4. +
      +
      3. +
    +
  8. Stores the result in the database cache.
@@ -7645,6 +7666,27 @@ represented by their Matrix user ID (e.g. @alice:example.com).

Called when processing a room creation request. The module must return a bool indicating whether the given user (represented by their Matrix user ID) is allowed to create a room.

+

user_may_create_room_with_invites

+
async def user_may_create_room_with_invites(
+    user: str,
+    invites: List[str],
+    threepid_invites: List[Dict[str, str]],
+) -> bool
+
+

Called when processing a room creation request (right after user_may_create_room). +The module is given the Matrix user ID of the user trying to create a room, as well as a +list of Matrix users to invite and a list of third-party identifiers (3PID, e.g. email +addresses) to invite.

+

An invited Matrix user to invite is represented by their Matrix user IDs, and an invited +3PIDs is represented by a dict that includes the 3PID medium (e.g. "email") through its +medium key and its address (e.g. "alice@example.com") through its address key.

+

See the Matrix specification for more +information regarding third-party identifiers.

+

If no invite and/or 3PID invite were specified in the room creation request, the +corresponding list(s) will be empty.

+

Note: This callback is not called when a room is cloned (e.g. during a room upgrade) +since no invites are sent when cloning a room. To cover this case, modules also need to +implement user_may_create_room.

user_may_create_room_alias

async def user_may_create_room_alias(user: str, room_alias: "synapse.types.RoomAlias") -> bool
@@ -7720,9 +7762,9 @@ class IsUserEvilResource(Resource): self.evil_users = config.get("evil_users") or [] def render_GET(self, request: Request): - user = request.args.get(b"user")[0] + user = request.args.get(b"user")[0].decode() request.setHeader(b"Content-Type", b"application/json") - return json.dumps({"evil": user in self.evil_users}) + return json.dumps({"evil": user in self.evil_users}).encode() class ListSpamChecker: @@ -11445,6 +11487,41 @@ trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_i

To increase the log level for the tests, set SYNAPSE_TEST_LOG_LEVEL:

SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests
+

Running tests under PostgreSQL

+

Invoking trial as above will use an in-memory SQLite database. This is great for +quick development and testing. However, we recommend using a PostgreSQL database +in production (and indeed, we have some code paths specific to each database). +This means that we need to run our unit tests against PostgreSQL too. Our CI does +this automatically for pull requests and release candidates, but it's sometimes +useful to reproduce this locally.

+

To do so, configure Postgres and run trial with the +following environment variables matching your configuration:

+ +

For example:

+
export SYNAPSE_POSTGRES=1
+export SYNAPSE_POSTGRES_HOST=localhost
+export SYNAPSE_POSTGRES_USER=postgres
+export SYNAPSE_POSTGRES_PASSWORD=mydevenvpassword
+trial
+
+

Prebuilt container

+

Since configuring PostgreSQL can be fiddly, we can make use of a pre-made +Docker container to set up PostgreSQL and run our tests for us. To do so, run

+
scripts-dev/test_postgresql.sh
+
+

Any extra arguments to the script will be passed to tox and then to trial, +so we can run a specific test in this container with e.g.

+
scripts-dev/test_postgresql.sh tests.replication.test_sharded_event_persister.EventPersisterShardTestCase
+
+

The container creates a folder in your Synapse checkout called +.tox-pg-container and uses this as a tox environment. The output of any +trial runs goes into _trial_temp in your synapse source directory — the same +as running trial directly on your host machine.

Run the integration tests (Sytest).

The integration tests are a more comprehensive suite of tests. They run a full version of Synapse, including your changes, to check if @@ -12142,6 +12219,39 @@ default value is the string "FALSE" - wh in Python, evaluates to True.

+

Implementing experimental features in Synapse

+

It can be desirable to implement "experimental" features which are disabled by +default and must be explicitly enabled via the Synapse configuration. This is +applicable for features which:

+ +

Note that this only really applies to features which are expected to be desirable +to a broad audience. The module infrastructure should +instead be investigated for non-standard features.

+

Guarding experimental features behind configuration flags should help with some +of the following scenarios:

+ +

Experimental configuration flags should be disabled by default (requiring Synapse +administrators to explicitly opt-in), although there are situations where it makes +sense (from a product point-of-view) to enable features by default. This is +expected and not an issue.

+

It is not a requirement for experimental features to be behind a configuration flag, +but one should be used if unsure.

+

New experimental configuration flags should be added under the experimental +configuration key (see the synapse.config.experimental file) and either explain +(briefly) what is being enabled, or include the MSC number.

Log Contexts

To help track the processing of individual requests, synapse uses a 'log context' to track which request it is handling at any given -- cgit 1.5.1