diff --git a/docs/ACME.md b/docs/ACME.md
deleted file mode 100644
index a7a498f575..0000000000
--- a/docs/ACME.md
+++ /dev/null
@@ -1,161 +0,0 @@
-# ACME
-
-From version 1.0 (June 2019) onwards, Synapse requires valid TLS
-certificates for communication between servers (by default on port
-`8448`) in addition to those that are client-facing (port `443`). To
-help homeserver admins fulfil this new requirement, Synapse v0.99.0
-introduced support for automatically provisioning certificates through
-[Let's Encrypt](https://letsencrypt.org/) using the ACME protocol.
-
-## Deprecation of ACME v1
-
-In [March 2019](https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430),
-Let's Encrypt announced that they were deprecating version 1 of the ACME
-protocol, with the plan to disable the use of it for new accounts in
-November 2019, for new domains in June 2020, and for existing accounts and
-domains in June 2021.
-
-Synapse doesn't currently support version 2 of the ACME protocol, which
-means that:
-
-* for existing installs, Synapse's built-in ACME support will continue
- to work until June 2021.
-* for new installs, this feature will not work at all.
-
-Either way, it is recommended to move from Synapse's ACME support
-feature to an external automated tool such as [certbot](https://github.com/certbot/certbot)
-(or browse [this list](https://letsencrypt.org/fr/docs/client-options/)
-for an alternative ACME client).
-
-It's also recommended to use a reverse proxy for the server-facing
-communications (more documentation about this can be found
-[here](/docs/reverse_proxy.md)) as well as the client-facing ones and
-have it serve the certificates.
-
-In case you can't do that and need Synapse to serve them itself, make
-sure to set the `tls_certificate_path` configuration setting to the path
-of the certificate (make sure to use the certificate containing the full
-certification chain, e.g. `fullchain.pem` if using certbot) and
-`tls_private_key_path` to the path of the matching private key. Note
-that in this case you will need to restart Synapse after each
-certificate renewal so that Synapse stops using the old certificate.
-
-If you still want to use Synapse's built-in ACME support, the rest of
-this document explains how to set it up.
-
-## Initial setup
-
-In the case that your `server_name` config variable is the same as
-the hostname that the client connects to, then the same certificate can be
-used between client and federation ports without issue.
-
-If your configuration file does not already have an `acme` section, you can
-generate an example config by running the `generate_config` executable. For
-example:
-
-```
-~/synapse/env3/bin/generate_config
-```
-
-You will need to provide Let's Encrypt (or another ACME provider) access to
-your Synapse ACME challenge responder on port 80, at the domain of your
-homeserver. This requires you to either change the port of the ACME listener
-provided by Synapse to a high port and reverse proxy to it, or use a tool
-like `authbind` to allow Synapse to listen on port 80 without root access.
-(Do not run Synapse with root permissions!) Detailed instructions are
-available under "ACME setup" below.
-
-If you already have certificates, you will need to back up or delete them
-(files `example.com.tls.crt` and `example.com.tls.key` in Synapse's root
-directory), Synapse's ACME implementation will not overwrite them.
-
-## ACME setup
-
-The main steps for enabling ACME support in short summary are:
-
-1. Allow Synapse to listen for incoming ACME challenges.
-1. Enable ACME support in `homeserver.yaml`.
-1. Move your old certificates (files `example.com.tls.crt` and `example.com.tls.key` out of the way if they currently exist at the paths specified in `homeserver.yaml`.
-1. Restart Synapse.
-
-Detailed instructions for each step are provided below.
-
-### Listening on port 80
-
-In order for Synapse to complete the ACME challenge to provision a
-certificate, it needs access to port 80. Typically listening on port 80 is
-only granted to applications running as root. There are thus two solutions to
-this problem.
-
-#### Using a reverse proxy
-
-A reverse proxy such as Apache or nginx allows a single process (the web
-server) to listen on port 80 and proxy traffic to the appropriate program
-running on your server. It is the recommended method for setting up ACME as
-it allows you to use your existing webserver while also allowing Synapse to
-provision certificates as needed.
-
-For nginx users, add the following line to your existing `server` block:
-
-```
-location /.well-known/acme-challenge {
- proxy_pass http://localhost:8009;
-}
-```
-
-For Apache, add the following to your existing webserver config:
-
-```
-ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
-```
-
-Make sure to restart/reload your webserver after making changes.
-
-Now make the relevant changes in `homeserver.yaml` to enable ACME support:
-
-```
-acme:
- enabled: true
- port: 8009
-```
-
-#### Authbind
-
-`authbind` allows a program which does not run as root to bind to
-low-numbered ports in a controlled way. The setup is simpler, but requires a
-webserver not to already be running on port 80. **This includes every time
-Synapse renews a certificate**, which may be cumbersome if you usually run a
-web server on port 80. Nevertheless, if you're sure port 80 is not being used
-for any other purpose then all that is necessary is the following:
-
-Install `authbind`. For example, on Debian/Ubuntu:
-
-```
-sudo apt-get install authbind
-```
-
-Allow `authbind` to bind port 80:
-
-```
-sudo touch /etc/authbind/byport/80
-sudo chmod 777 /etc/authbind/byport/80
-```
-
-When Synapse is started, use the following syntax:
-
-```
-authbind --deep <synapse start command>
-```
-
-Make the relevant changes in `homeserver.yaml` to enable ACME support:
-
-```
-acme:
- enabled: true
-```
-
-### (Re)starting synapse
-
-Ensure that the certificate paths specified in `homeserver.yaml` (`tls_certificate_path` and `tls_private_key_path`) do not currently point to any files. Synapse will not provision certificates if files exist, as it does not want to overwrite existing certificates.
-
-Finally, start/restart Synapse.
diff --git a/docs/MSC1711_certificates_FAQ.md b/docs/MSC1711_certificates_FAQ.md
index 80bd1294c7..ce8189d4ed 100644
--- a/docs/MSC1711_certificates_FAQ.md
+++ b/docs/MSC1711_certificates_FAQ.md
@@ -101,15 +101,6 @@ In this case, your `server_name` points to the host where your Synapse is
running. There is no need to create a `.well-known` URI or an SRV record, but
you will need to give Synapse a valid, signed, certificate.
-The easiest way to do that is with Synapse's built-in ACME (Let's Encrypt)
-support. Full details are in [ACME.md](./ACME.md) but, in a nutshell:
-
- 1. Allow Synapse to listen on port 80 with `authbind`, or forward it from a
- reverse proxy.
- 2. Enable acme support in `homeserver.yaml`.
- 3. Move your old certificates out of the way.
- 4. Restart Synapse.
-
### If you do have an SRV record currently
If you are using an SRV record, your matrix domain (`server_name`) may not
@@ -130,15 +121,9 @@ In this situation, you have three choices for how to proceed:
#### Option 1: give Synapse a certificate for your matrix domain
Synapse 1.0 will expect your server to present a TLS certificate for your
-`server_name` (`example.com` in the above example). You can achieve this by
-doing one of the following:
-
- * Acquire a certificate for the `server_name` yourself (for example, using
- `certbot`), and give it and the key to Synapse via `tls_certificate_path`
- and `tls_private_key_path`, or:
-
- * Use Synapse's [ACME support](./ACME.md), and forward port 80 on the
- `server_name` domain to your Synapse instance.
+`server_name` (`example.com` in the above example). You can achieve this by acquiring a
+certificate for the `server_name` yourself (for example, using `certbot`), and giving it
+and the key to Synapse via `tls_certificate_path` and `tls_private_key_path`.
#### Option 2: run Synapse behind a reverse proxy
@@ -161,10 +146,9 @@ You can do this with a `.well-known` file as follows:
with Synapse 0.34 and earlier.
2. Give Synapse a certificate corresponding to the target domain
- (`customer.example.net` in the above example). You can either use Synapse's
- built-in [ACME support](./ACME.md) for this (via the `domain` parameter in
- the `acme` section), or acquire a certificate yourself and give it to
- Synapse via `tls_certificate_path` and `tls_private_key_path`.
+ (`customer.example.net` in the above example). You can do this by acquire a
+ certificate for the target domain and giving it to Synapse via `tls_certificate_path`
+ and `tls_private_key_path`.
3. Restart Synapse to ensure the new certificate is loaded.
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 01ef4ff600..98969bdd2d 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -35,7 +35,7 @@
- [URL Previews](url_previews.md)
- [User Directory](user_directory.md)
- [Message Retention Policies](message_retention_policies.md)
- - [Pluggable Modules]()
+ - [Pluggable Modules](modules.md)
- [Third Party Rules]()
- [Spam Checker](spam_checker.md)
- [Presence Router](presence_router_module.md)
diff --git a/docs/modules.md b/docs/modules.md
new file mode 100644
index 0000000000..3a9fab61b8
--- /dev/null
+++ b/docs/modules.md
@@ -0,0 +1,258 @@
+# Modules
+
+Synapse supports extending its functionality by configuring external modules.
+
+## Using modules
+
+To use a module on Synapse, add it to the `modules` section of the configuration file:
+
+```yaml
+modules:
+ - module: my_super_module.MySuperClass
+ config:
+ do_thing: true
+ - module: my_other_super_module.SomeClass
+ config: {}
+```
+
+Each module is defined by a path to a Python class as well as a configuration. This
+information for a given module should be available in the module's own documentation.
+
+**Note**: When using third-party modules, you effectively allow someone else to run
+custom code on your Synapse homeserver. Server admins are encouraged to verify the
+provenance of the modules they use on their homeserver and make sure the modules aren't
+running malicious code on their instance.
+
+Also note that we are currently in the process of migrating module interfaces to this
+system. While some interfaces might be compatible with it, others still require
+configuring modules in another part of Synapse's configuration file. Currently, only the
+spam checker interface is compatible with this new system.
+
+## Writing a module
+
+A module is a Python class that uses Synapse's module API to interact with the
+homeserver. It can register callbacks that Synapse will call on specific operations, as
+well as web resources to attach to Synapse's web server.
+
+When instantiated, a module is given its parsed configuration as well as an instance of
+the `synapse.module_api.ModuleApi` class. The configuration is a dictionary, and is
+either the output of the module's `parse_config` static method (see below), or the
+configuration associated with the module in Synapse's configuration file.
+
+See the documentation for the `ModuleApi` class
+[here](https://github.com/matrix-org/synapse/blob/master/synapse/module_api/__init__.py).
+
+### Handling the module's configuration
+
+A module can implement the following static method:
+
+```python
+@staticmethod
+def parse_config(config: dict) -> dict
+```
+
+This method is given a dictionary resulting from parsing the YAML configuration for the
+module. It may modify it (for example by parsing durations expressed as strings (e.g.
+"5d") into milliseconds, etc.), and return the modified dictionary. It may also verify
+that the configuration is correct, and raise an instance of
+`synapse.module_api.errors.ConfigError` if not.
+
+### Registering a web resource
+
+Modules can register web resources onto Synapse's web server using the following module
+API method:
+
+```python
+def ModuleApi.register_web_resource(path: str, resource: IResource)
+```
+
+The path is the full absolute path to register the resource at. For example, if you
+register a resource for the path `/_synapse/client/my_super_module/say_hello`, Synapse
+will serve it at `http(s)://[HS_URL]/_synapse/client/my_super_module/say_hello`. Note
+that Synapse does not allow registering resources for several sub-paths in the `/_matrix`
+namespace (such as anything under `/_matrix/client` for example). It is strongly
+recommended that modules register their web resources under the `/_synapse/client`
+namespace.
+
+The provided resource is a Python class that implements Twisted's [IResource](https://twistedmatrix.com/documents/current/api/twisted.web.resource.IResource.html)
+interface (such as [Resource](https://twistedmatrix.com/documents/current/api/twisted.web.resource.Resource.html)).
+
+Only one resource can be registered for a given path. If several modules attempt to
+register a resource for the same path, the module that appears first in Synapse's
+configuration file takes priority.
+
+Modules **must** register their web resources in their `__init__` method.
+
+### Registering a callback
+
+Modules can use Synapse's module API to register callbacks. Callbacks are functions that
+Synapse will call when performing specific actions. Callbacks must be asynchronous, and
+are split in categories. A single module may implement callbacks from multiple categories,
+and is under no obligation to implement all callbacks from the categories it registers
+callbacks for.
+
+#### Spam checker callbacks
+
+To register one of the callbacks described in this section, a module needs to use the
+module API's `register_spam_checker_callbacks` method. The callback functions are passed
+to `register_spam_checker_callbacks` as keyword arguments, with the callback name as the
+argument name and the function as its value. This is demonstrated in the example below.
+
+The available spam checker callbacks are:
+
+```python
+async def check_event_for_spam(event: "synapse.events.EventBase") -> Union[bool, str]
+```
+
+Called when receiving an event from a client or via federation. The module can return
+either a `bool` to indicate whether the event must be rejected because of spam, or a `str`
+to indicate the event must be rejected because of spam and to give a rejection reason to
+forward to clients.
+
+```python
+async def user_may_invite(inviter: str, invitee: str, room_id: str) -> bool
+```
+
+Called when processing an invitation. The module must return a `bool` indicating whether
+the inviter can invite the invitee to the given room. Both inviter and invitee are
+represented by their Matrix user ID (i.e. `@alice:example.com`).
+
+```python
+async def user_may_create_room(user: str) -> bool
+```
+
+Called when processing a room creation request. The module must return a `bool` indicating
+whether the given user (represented by their Matrix user ID) is allowed to create a room.
+
+```python
+async def user_may_create_room_alias(user: str, room_alias: "synapse.types.RoomAlias") -> bool
+```
+
+Called when trying to associate an alias with an existing room. The module must return a
+`bool` indicating whether the given user (represented by their Matrix user ID) is allowed
+to set the given alias.
+
+```python
+async def user_may_publish_room(user: str, room_id: str) -> bool
+```
+
+Called when trying to publish a room to the homeserver's public rooms directory. The
+module must return a `bool` indicating whether the given user (represented by their
+Matrix user ID) is allowed to publish the given room.
+
+```python
+async def check_username_for_spam(user_profile: Dict[str, str]) -> bool
+```
+
+Called when computing search results in the user directory. The module must return a
+`bool` indicating whether the given user profile can appear in search results. The profile
+is represented as a dictionary with the following keys:
+
+* `user_id`: The Matrix ID for this user.
+* `display_name`: The user's display name.
+* `avatar_url`: The `mxc://` URL to the user's avatar.
+
+The module is given a copy of the original dictionary, so modifying it from within the
+module cannot modify a user's profile when included in user directory search results.
+
+```python
+async def check_registration_for_spam(
+ email_threepid: Optional[dict],
+ username: Optional[str],
+ request_info: Collection[Tuple[str, str]],
+ auth_provider_id: Optional[str] = None,
+) -> "synapse.spam_checker_api.RegistrationBehaviour"
+```
+
+Called when registering a new user. The module must return a `RegistrationBehaviour`
+indicating whether the registration can go through or must be denied, or whether the user
+may be allowed to register but will be shadow banned.
+
+The arguments passed to this callback are:
+
+* `email_threepid`: The email address used for registering, if any.
+* `username`: The username the user would like to register. Can be `None`, meaning that
+ Synapse will generate one later.
+* `request_info`: A collection of tuples, which first item is a user agent, and which
+ second item is an IP address. These user agents and IP addresses are the ones that were
+ used during the registration process.
+* `auth_provider_id`: The identifier of the SSO authentication provider, if any.
+
+```python
+async def check_media_file_for_spam(
+ file_wrapper: "synapse.rest.media.v1.media_storage.ReadableFileWrapper",
+ file_info: "synapse.rest.media.v1._base.FileInfo"
+) -> bool
+```
+
+Called when storing a local or remote file. The module must return a boolean indicating
+whether the given file can be stored in the homeserver's media store.
+
+### Porting an existing module that uses the old interface
+
+In order to port a module that uses Synapse's old module interface, its author needs to:
+
+* ensure the module's callbacks are all asynchronous.
+* register their callbacks using one or more of the `register_[...]_callbacks` methods
+ from the `ModuleApi` class in the module's `__init__` method (see [this section](#registering-a-web-resource)
+ for more info).
+
+Additionally, if the module is packaged with an additional web resource, the module
+should register this resource in its `__init__` method using the `register_web_resource`
+method from the `ModuleApi` class (see [this section](#registering-a-web-resource) for
+more info).
+
+The module's author should also update any example in the module's configuration to only
+use the new `modules` section in Synapse's configuration file (see [this section](#using-modules)
+for more info).
+
+### Example
+
+The example below is a module that implements the spam checker callback
+`user_may_create_room` to deny room creation to user `@evilguy:example.com`, and registers
+a web resource to the path `/_synapse/client/demo/hello` that returns a JSON object.
+
+```python
+import json
+
+from twisted.web.resource import Resource
+from twisted.web.server import Request
+
+from synapse.module_api import ModuleApi
+
+
+class DemoResource(Resource):
+ def __init__(self, config):
+ super(DemoResource, self).__init__()
+ self.config = config
+
+ def render_GET(self, request: Request):
+ name = request.args.get(b"name")[0]
+ request.setHeader(b"Content-Type", b"application/json")
+ return json.dumps({"hello": name})
+
+
+class DemoModule:
+ def __init__(self, config: dict, api: ModuleApi):
+ self.config = config
+ self.api = api
+
+ self.api.register_web_resource(
+ path="/_synapse/client/demo/hello",
+ resource=DemoResource(self.config),
+ )
+
+ self.api.register_spam_checker_callbacks(
+ user_may_create_room=self.user_may_create_room,
+ )
+
+ @staticmethod
+ def parse_config(config):
+ return config
+
+ async def user_may_create_room(self, user: str) -> bool:
+ if user == "@evilguy:example.com":
+ return False
+
+ return True
+```
diff --git a/docs/sample_config.yaml b/docs/sample_config.yaml
index f8925a5e24..6fcc022b47 100644
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@@ -31,6 +31,22 @@
#
# [1] https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
+
+## Modules ##
+
+# Server admins can expand Synapse's functionality with external modules.
+#
+# See https://matrix-org.github.io/synapse/develop/modules.html for more
+# documentation on how to configure or create custom modules for Synapse.
+#
+modules:
+ # - module: my_super_module.MySuperClass
+ # config:
+ # do_thing: true
+ # - module: my_other_super_module.SomeClass
+ # config: {}
+
+
## Server ##
# The public-facing domain of the server
@@ -552,13 +568,9 @@ retention:
# This certificate, as of Synapse 1.0, will need to be a valid and verifiable
# certificate, signed by a recognised Certificate Authority.
#
-# See 'ACME support' below to enable auto-provisioning this certificate via
-# Let's Encrypt.
-#
-# If supplying your own, be sure to use a `.pem` file that includes the
-# full certificate chain including any intermediate certificates (for
-# instance, if using certbot, use `fullchain.pem` as your certificate,
-# not `cert.pem`).
+# Be sure to use a `.pem` file that includes the full certificate chain including
+# any intermediate certificates (for instance, if using certbot, use
+# `fullchain.pem` as your certificate, not `cert.pem`).
#
#tls_certificate_path: "CONFDIR/SERVERNAME.tls.crt"
@@ -609,80 +621,6 @@ retention:
# - myCA2.pem
# - myCA3.pem
-# ACME support: This will configure Synapse to request a valid TLS certificate
-# for your configured `server_name` via Let's Encrypt.
-#
-# Note that ACME v1 is now deprecated, and Synapse currently doesn't support
-# ACME v2. This means that this feature currently won't work with installs set
-# up after November 2019. For more info, and alternative solutions, see
-# https://github.com/matrix-org/synapse/blob/master/docs/ACME.md#deprecation-of-acme-v1
-#
-# Note that provisioning a certificate in this way requires port 80 to be
-# routed to Synapse so that it can complete the http-01 ACME challenge.
-# By default, if you enable ACME support, Synapse will attempt to listen on
-# port 80 for incoming http-01 challenges - however, this will likely fail
-# with 'Permission denied' or a similar error.
-#
-# There are a couple of potential solutions to this:
-#
-# * If you already have an Apache, Nginx, or similar listening on port 80,
-# you can configure Synapse to use an alternate port, and have your web
-# server forward the requests. For example, assuming you set 'port: 8009'
-# below, on Apache, you would write:
-#
-# ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
-#
-# * Alternatively, you can use something like `authbind` to give Synapse
-# permission to listen on port 80.
-#
-acme:
- # ACME support is disabled by default. Set this to `true` and uncomment
- # tls_certificate_path and tls_private_key_path above to enable it.
- #
- enabled: false
-
- # Endpoint to use to request certificates. If you only want to test,
- # use Let's Encrypt's staging url:
- # https://acme-staging.api.letsencrypt.org/directory
- #
- #url: https://acme-v01.api.letsencrypt.org/directory
-
- # Port number to listen on for the HTTP-01 challenge. Change this if
- # you are forwarding connections through Apache/Nginx/etc.
- #
- port: 80
-
- # Local addresses to listen on for incoming connections.
- # Again, you may want to change this if you are forwarding connections
- # through Apache/Nginx/etc.
- #
- bind_addresses: ['::', '0.0.0.0']
-
- # How many days remaining on a certificate before it is renewed.
- #
- reprovision_threshold: 30
-
- # The domain that the certificate should be for. Normally this
- # should be the same as your Matrix domain (i.e., 'server_name'), but,
- # by putting a file at 'https://<server_name>/.well-known/matrix/server',
- # you can delegate incoming traffic to another server. If you do that,
- # you should give the target of the delegation here.
- #
- # For example: if your 'server_name' is 'example.com', but
- # 'https://example.com/.well-known/matrix/server' delegates to
- # 'matrix.example.com', you should put 'matrix.example.com' here.
- #
- # If not set, defaults to your 'server_name'.
- #
- domain: matrix.example.com
-
- # file to use for the account key. This will be generated if it doesn't
- # exist.
- #
- # If unspecified, we will use CONFDIR/client.key.
- #
- account_key_file: DATADIR/acme_account.key
-
## Federation ##
@@ -2037,6 +1975,17 @@ sso:
# - https://riot.im/develop
# - https://my.custom.client/
+ # Uncomment to keep a user's profile fields in sync with information from
+ # the identity provider. Currently only syncing the displayname is
+ # supported. Fields are checked on every SSO login, and are updated
+ # if necessary.
+ #
+ # Note that enabling this option will override user profile information,
+ # regardless of whether users have opted-out of syncing that
+ # information when first signing in. Defaults to false.
+ #
+ #update_profile_information: true
+
# Directory in which Synapse will try to find the template files below.
# If not set, or the files named below are not found within the template
# directory, default templates from within the Synapse package will be used.
@@ -2318,6 +2267,10 @@ ui_auth:
# the user-interactive authentication process, by allowing for multiple
# (and potentially different) operations to use the same validation session.
#
+ # This is ignored for potentially "dangerous" operations (including
+ # deactivating an account, modifying an account password, and
+ # adding a 3PID).
+ #
# Uncomment below to allow for credential validation to last for 15
# seconds.
#
@@ -2565,19 +2518,6 @@ push:
#group_unread_count_by_room: false
-# Spam checkers are third-party modules that can block specific actions
-# of local users, such as creating rooms and registering undesirable
-# usernames, as well as remote users by redacting incoming events.
-#
-spam_checker:
- #- module: "my_custom_project.SuperSpamChecker"
- # config:
- # example_option: 'things'
- #- module: "some_other_project.BadEventStopper"
- # config:
- # example_stop_events_from: ['@bad:example.com']
-
-
## Rooms ##
# Controls whether locally-created rooms should be end-to-end encrypted by
diff --git a/docs/spam_checker.md b/docs/spam_checker.md
index 52947f605e..c16914e61d 100644
--- a/docs/spam_checker.md
+++ b/docs/spam_checker.md
@@ -1,3 +1,7 @@
+**Note: this page of the Synapse documentation is now deprecated. For up to date
+documentation on setting up or writing a spam checker module, please see
+[this page](https://matrix-org.github.io/synapse/develop/modules.html).**
+
# Handling spam in Synapse
Synapse has support to customize spam checking behavior. It can plug into a
|
