Merge pull request #4570 from matrix-org/anoa/self_signed_upgrade

Add ACME docs and link to it from README and INSTALL

4 files changed, 151 insertions, 120 deletions

diff --git a/INSTALL.md b/INSTALL.md
index a04524cdc7..cbe4bda120 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -40,7 +40,7 @@ pip install matrix-synapse[all]
 
 This will download Synapse from [PyPI](https://pypi.org/project/matrix-synapse)
 and install it, along with the python libraries it uses, into a virtual environment
-under ``~/synapse/env``.  Feel free to pick a different directory if you
+under `~/synapse/env`.  Feel free to pick a different directory if you
 prefer.
 
 This Synapse installation can then be later upgraded by using pip again with the
@@ -88,7 +88,7 @@ You will need to give Synapse a TLS certficate before it will start - see [TLS
 certificates](#tls-certificates).
 
 To actually run your new homeserver, pick a working directory for Synapse to
-run (e.g. ``~/synapse``), and::
+run (e.g. `~/synapse`), and::
 
     cd ~/synapse
     source env/bin/activate
@@ -197,19 +197,19 @@ may need to manually upgrade it::
 
     sudo pip install --upgrade pip
 
-Installing may fail with ``Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)``.
+Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
 You can fix this by manually upgrading pip and virtualenv::
 
     sudo pip install --upgrade virtualenv
 
-You can next rerun ``virtualenv -p python3 synapse`` to update the virtual env.
+You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
 
-Installing may fail during installing virtualenv with ``InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.``
+Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
 You can fix this  by manually installing ndg-httpsclient::
 
     pip install --upgrade ndg-httpsclient
 
-Installing may fail with ``mock requires setuptools>=17.1. Aborting installation``.
+Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
 You can fix this by upgrading setuptools::
 
     pip install --upgrade setuptools
@@ -355,90 +355,13 @@ configured without TLS; it should be behind a reverse proxy for TLS/SSL
 termination on port 443 which in turn should be used for clients. Port 8448
 is configured to use TLS for Federation with a self-signed or verified
 certificate, but please be aware that a valid certificate will be required in
-Synapse v1.0.
+Synapse v1.0. Instructions for having Synapse automatically provision and renew federation certificates through ACME can be found at [ACME.md](docs/ACME.md).
 
 If you would like to use your own certificates, you can do so by changing
 `tls_certificate_path` and `tls_private_key_path` in `homeserver.yaml`;
 alternatively, you can use a reverse-proxy. Apart from port 8448 using TLS,
 both ports are the same in the default configuration.
 
-### ACME setup
-
-Synapse v1.0 will require valid TLS certificates for communication between servers
-(port `8448` by default) in addition to those that are client-facing (port
-`443`). In the case that your `server_name` config variable is the same as
-the hostname that the client connects to, then the same certificate can be
-used between client and federation ports without issue. Synapse v0.99.0+
-**will provision server-to-server certificates automatically for you for
-free** through [Let's Encrypt](https://letsencrypt.org/) if you tell it to.
-
-In order for Synapse to complete the ACME challenge to provision a
-certificate, it needs access to port 80. Typically listening on port 80 is
-only granted to applications running as root. There are thus two solutions to
-this problem.
-
-#### Using a reverse proxy
-
-A reverse proxy such as Apache or nginx allows a single process (the web
-server) to listen on port 80 and proxy traffic to the appropriate program
-running on your server. It is the recommended method for setting up ACME as
-it allows you to use your existing webserver while also allowing Synapse to
-provision certificates as needed.
-
-For nginx users, add the following line to your existing `server` block:
-
-```
-location /.well-known/acme-challenge {
-    proxy_pass http://localhost:8009/;
-}
-```
-
-For Apache, add the following to your existing webserver config::
-
-```
-ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
-```
-
-Make sure to restart/reload your webserver after making changes.
-
-
-#### Authbind
-
-`authbind` allows a program which does not run as root to bind to
-low-numbered ports in a controlled way. The setup is simpler, but requires a
-webserver not to already be running on port 80. **This includes every time
-Synapse renews a certificate**, which may be cumbersome if you usually run a
-web server on port 80. Nevertheless, if you're sure port 80 is not being used
-for any other purpose then all that is necessary is the following:
-
-Install `authbind`. For example, on Debian/Ubuntu:
-
-```
-sudo apt-get install authbind
-```
-
-Allow `authbind` to bind port 80:
-
-```
-sudo touch /etc/authbind/byport/80
-sudo chmod 777 /etc/authbind/byport/80
-```
-
-When Synapse is started, use the following syntax::
-
-```
-authbind --deep <synapse start command>
-```
-
-Finally, once Synapse is able to listen on port 80 for ACME challenge
-requests, it must be told to perform ACME provisioning by setting `enabled`
-to true under the `acme` section in `homeserver.yaml`:
-
-```
-acme:
-    enabled: true
-```
-
 ## Registering a user
 
 You will need at least one user on your server in order to use a Matrix
@@ -459,12 +382,12 @@ Make admin [no]:
 Success!
 ```
 
-This process uses a setting ``registration_shared_secret`` in
-``homeserver.yaml``, which is shared between Synapse itself and the
-``register_new_matrix_user`` script. It doesn't matter what it is (a random
-value is generated by ``--generate-config``), but it should be kept secret, as
+This process uses a setting `registration_shared_secret` in
+`homeserver.yaml`, which is shared between Synapse itself and the
+`register_new_matrix_user` script. It doesn't matter what it is (a random
+value is generated by `--generate-config`), but it should be kept secret, as
 anyone with knowledge of it can register users on your server even if
-``enable_registration`` is ``false``.
+`enable_registration` is `false`.
 
 ## Setting up a TURN server
 
@@ -474,14 +397,14 @@ a TURN server.  See [docs/turn-howto.rst](docs/turn-howto.rst) for details.
 ## URL previews
 
 Synapse includes support for previewing URLs, which is disabled by default.  To
-turn it on you must enable the ``url_preview_enabled: True`` config parameter
+turn it on you must enable the `url_preview_enabled: True` config parameter
 and explicitly specify the IP ranges that Synapse is not allowed to spider for
-previewing in the ``url_preview_ip_range_blacklist`` configuration parameter.
+previewing in the `url_preview_ip_range_blacklist` configuration parameter.
 This is critical from a security perspective to stop arbitrary Matrix users
 spidering 'internal' URLs on your network.  At the very least we recommend that
 your loopback and RFC1918 IP addresses are blacklisted.
 
 This also requires the optional lxml and netaddr python dependencies to be
 installed.  This in turn requires the libxml2 library to be available - on
-Debian/Ubuntu this means ``apt-get install libxml2-dev``, or equivalent for
+Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
 your OS.
diff --git a/README.rst b/README.rst
index 8c5220be88..e666b3b427 100644
--- a/README.rst
+++ b/README.rst
@@ -127,6 +127,12 @@ create the account. Your name will take the form of::
 As when logging in, you will need to specify a "Custom server".  Specify your
 desired ``localpart`` in the 'User name' box.
 
+ACME setup
+==========
+
+For details on having Synapse manage your federation TLS certificates
+automatically, please see `<docs/ACME.md>`_.
+
 
 Security Note
 =============
diff --git a/UPGRADE.rst b/UPGRADE.rst
index 7bd631f14c..75aef366bd 100644
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -51,34 +51,10 @@ returned by the Client-Server API:
 Upgrading to v0.99.0
 ====================
 
-In preparation for Synapse v1.0, you must update your TLS certificates from
-self-signed ones to verifiable ones signed by a trusted root CA.
-
-If you do not already have a certificate for your domain, the easiest way to get
-one is with Synapse's new ACME support, which will use the ACME protocol to
-provision a certificate automatically. By default, certificates will be obtained
-from the publicly trusted CA Let's Encrypt.
-
-For a sample configuration, please inspect the new ACME section in the example
-generated config by running the ``generate-config`` executable. For example::
-
-  ~/synapse/env3/bin/generate-config
-
-You will need to provide Let's Encrypt (or other ACME provider) access to your
-Synapse ACME challenge responder on port 80, at the domain of your homeserver.
-This requires you either change the port of the ACME listener provided by
-Synapse to a high port and reverse proxy to it, or use a tool like authbind to
-allow Synapse to listen on port 80 without root access. (Do not run Synapse with
-root permissions!)
-
-You will need to back up or delete your self signed TLS certificate
-(``example.com.tls.crt`` and ``example.com.tls.key``), Synapse's ACME
-implementation will not overwrite them.
-
-You may wish to use alternate methods such as Certbot to obtain a certificate
-from Let's Encrypt, depending on your server configuration. Of course, if you
-already have a valid certificate for your homeserver's domain, that can be
-placed in Synapse's config directory without the need for ACME.
+Please be aware that, before Synapse v1.0 is released around March 2019, you
+will need to replace any self-signed certificates with those verified by a
+root CA. Information on how to do so can be found at `the ACME docs
+<docs/ACME.md>`_.
 
 For more information on configuring TLS certificates see the `FAQ <https://github.com/matrix-org/synapse/blob/master/docs/MSC1711_certificates_FAQ.md>`_
 
diff --git a/docs/ACME.md b/docs/ACME.md
new file mode 100644
index 0000000000..8fb2bd66a9
--- /dev/null
+++ b/docs/ACME.md
@@ -0,0 +1,126 @@
+# ACME
+
+Synapse v1.0 will require valid TLS certificates for communication between
+servers (port `8448` by default) in addition to those that are client-facing
+(port `443`). If you do not already have a valid certificate for your domain,
+the easiest way to get one is with Synapse's new ACME support, which will use
+the ACME protocol to provision a certificate automatically. Synapse v0.99.0+
+will provision server-to-server certificates automatically for you for free
+through [Let's Encrypt](https://letsencrypt.org/) if you tell it to.
+
+In the case that your `server_name` config variable is the same as
+the hostname that the client connects to, then the same certificate can be
+used between client and federation ports without issue. 
+
+For a sample configuration, please inspect the new ACME section in the example
+generated config by running the `generate-config` executable. For example:
+
+```
+~/synapse/env3/bin/generate-config
+```
+
+You will need to provide Let's Encrypt (or another ACME provider) access to
+your Synapse ACME challenge responder on port 80, at the domain of your
+homeserver. This requires you to either change the port of the ACME listener
+provided by Synapse to a high port and reverse proxy to it, or use a tool
+like `authbind` to allow Synapse to listen on port 80 without root access.
+(Do not run Synapse with root permissions!) Detailed instructions are
+available under "ACME setup" below.
+
+If you are already using self-signed certificates, you will need to back up
+or delete them (files `example.com.tls.crt` and `example.com.tls.key` in
+Synapse's root directory), Synapse's ACME implementation will not overwrite
+them.
+
+You may wish to use alternate methods such as Certbot to obtain a certificate
+from Let's Encrypt, depending on your server configuration. Of course, if you
+already have a valid certificate for your homeserver's domain, that can be
+placed in Synapse's config directory without the need for any ACME setup.
+
+## ACME setup
+
+The main steps for enabling ACME support in short summary are:
+
+1. Allow Synapse to listen on port 80 with authbind, or forward it from a reverse-proxy.
+1. Set `acme:enabled` to `true` in homeserver.yaml.
+1. Move your old certificates (files `example.com.tls.crt` and `example.com.tls.key` out of the way if they currently exist at the paths specified in `homeserver.yaml`.
+1. Restart Synapse
+
+Detailed instructions for each step are provided below.
+
+### Listening on port 80
+
+In order for Synapse to complete the ACME challenge to provision a
+certificate, it needs access to port 80. Typically listening on port 80 is
+only granted to applications running as root. There are thus two solutions to
+this problem.
+
+#### Using a reverse proxy
+
+A reverse proxy such as Apache or nginx allows a single process (the web
+server) to listen on port 80 and proxy traffic to the appropriate program
+running on your server. It is the recommended method for setting up ACME as
+it allows you to use your existing webserver while also allowing Synapse to
+provision certificates as needed.
+
+For nginx users, add the following line to your existing `server` block:
+
+```
+location /.well-known/acme-challenge {
+    proxy_pass http://localhost:8009/;
+}
+```
+
+For Apache, add the following to your existing webserver config::
+
+```
+ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge
+```
+
+Make sure to restart/reload your webserver after making changes.
+
+
+#### Authbind
+
+`authbind` allows a program which does not run as root to bind to
+low-numbered ports in a controlled way. The setup is simpler, but requires a
+webserver not to already be running on port 80. **This includes every time
+Synapse renews a certificate**, which may be cumbersome if you usually run a
+web server on port 80. Nevertheless, if you're sure port 80 is not being used
+for any other purpose then all that is necessary is the following:
+
+Install `authbind`. For example, on Debian/Ubuntu:
+
+```
+sudo apt-get install authbind
+```
+
+Allow `authbind` to bind port 80:
+
+```
+sudo touch /etc/authbind/byport/80
+sudo chmod 777 /etc/authbind/byport/80
+```
+
+When Synapse is started, use the following syntax::
+
+```
+authbind --deep <synapse start command>
+```
+
+### Config file editing
+
+Once Synapse is able to listen on port 80 for ACME challenge
+requests, it must be told to perform ACME provisioning by setting `enabled`
+to true under the `acme` section in `homeserver.yaml`:
+
+```
+acme:
+    enabled: true
+```
+
+### Starting synapse
+
+Ensure that the certificate paths specified in `homeserver.yaml` (`tls_certificate_path` and `tls_private_key_path`) do not currently point to any files. Synapse will not provision certificates if files exist, as it does not want to overwrite existing certificates.
+
+Finally, start/restart Synapse.
\ No newline at end of file