diff --git a/docs/code_style.rst b/docs/code_style.rst
index e3ca626bfd..39ac4ebedc 100644
--- a/docs/code_style.rst
+++ b/docs/code_style.rst
@@ -1,4 +1,8 @@
-# Code Style
+Code Style
+==========
+
+Formatting tools
+----------------
The Synapse codebase uses a number of code formatting tools in order to
quickly and automatically check for formatting (and sometimes logical) errors
@@ -6,20 +10,20 @@ in code.
The necessary tools are detailed below.
-## Formatting tools
+- **black**
-The Synapse codebase uses [black](https://pypi.org/project/black/) as an
-opinionated code formatter, ensuring all comitted code is properly
-formatted.
+ The Synapse codebase uses `black <https://pypi.org/project/black/>`_ as an
+ opinionated code formatter, ensuring all comitted code is properly
+ formatted.
-First install ``black`` with::
+ First install ``black`` with::
- pip install --upgrade black
+ pip install --upgrade black
-Have ``black`` auto-format your code (it shouldn't change any
-functionality) with::
+ Have ``black`` auto-format your code (it shouldn't change any functionality)
+ with::
- black . --exclude="\.tox|build|env"
+ black . --exclude="\.tox|build|env"
- **flake8**
@@ -54,17 +58,16 @@ functionality is supported in your editor for a more convenient development
workflow. It is not, however, recommended to run ``flake8`` on save as it
takes a while and is very resource intensive.
-## General rules
+General rules
+-------------
- **Naming**:
- Use camel case for class and type names
- Use underscores for functions and variables.
-- Use double quotes ``"foo"`` rather than single quotes ``'foo'``.
-
-- **Comments**: should follow the `google code style
- <http://google.github.io/styleguide/pyguide.html?showone=Comments#Comments>`_.
+- **Docstrings**: should follow the `google code style
+ <https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
This is so that we can generate documentation with `sphinx
<http://sphinxcontrib-napoleon.readthedocs.org/en/latest/>`_. See the
`examples
@@ -73,6 +76,8 @@ takes a while and is very resource intensive.
- **Imports**:
+ - Imports should be sorted by ``isort`` as described above.
+
- Prefer to import classes and functions rather than packages or modules.
Example::
@@ -92,25 +97,84 @@ takes a while and is very resource intensive.
This goes against the advice in the Google style guide, but it means that
errors in the name are caught early (at import time).
- - Multiple imports from the same package can be combined onto one line::
+ - Avoid wildcard imports (``from synapse.types import *``) and relative
+ imports (``from .types import UserID``).
- from synapse.types import GroupID, RoomID, UserID
+Configuration file format
+-------------------------
- An effort should be made to keep the individual imports in alphabetical
- order.
+The `sample configuration file <./sample_config.yaml>`_ acts as a reference to
+Synapse's configuration options for server administrators. Remember that many
+readers will be unfamiliar with YAML and server administration in general, so
+that it is important that the file be as easy to understand as possible, which
+includes following a consistent format.
- If the list becomes long, wrap it with parentheses and split it over
- multiple lines.
+Some guidelines follow:
- - As per `PEP-8 <https://www.python.org/dev/peps/pep-0008/#imports>`_,
- imports should be grouped in the following order, with a blank line between
- each group:
+* Sections should be separated with a heading consisting of a single line
+ prefixed and suffixed with ``##``. There should be **two** blank lines
+ before the section header, and **one** after.
- 1. standard library imports
- 2. related third party imports
- 3. local application/library specific imports
+* Each option should be listed in the file with the following format:
- - Imports within each group should be sorted alphabetically by module name.
+ * A comment describing the setting. Each line of this comment should be
+ prefixed with a hash (``#``) and a space.
- - Avoid wildcard imports (``from synapse.types import *``) and relative
- imports (``from .types import UserID``).
+ The comment should describe the default behaviour (ie, what happens if
+ the setting is omitted), as well as what the effect will be if the
+ setting is changed.
+
+ Often, the comment end with something like "uncomment the
+ following to \<do action>".
+
+ * A line consisting of only ``#``.
+
+ * A commented-out example setting, prefixed with only ``#``.
+
+ For boolean (on/off) options, convention is that this example should be
+ the *opposite* to the default (so the comment will end with "Uncomment
+ the following to enable [or disable] \<feature\>." For other options,
+ the example should give some non-default value which is likely to be
+ useful to the reader.
+
+* There should be a blank line between each option.
+
+* Where several settings are grouped into a single dict, *avoid* the
+ convention where the whole block is commented out, resulting in comment
+ lines starting ``# #``, as this is hard to read and confusing to
+ edit. Instead, leave the top-level config option uncommented, and follow
+ the conventions above for sub-options. Ensure that your code correctly
+ handles the top-level option being set to ``None`` (as it will be if no
+ sub-options are enabled).
+
+* Lines should be wrapped at 80 characters.
+
+Example::
+
+ ## Frobnication ##
+
+ # The frobnicator will ensure that all requests are fully frobnicated.
+ # To enable it, uncomment the following.
+ #
+ #frobnicator_enabled: true
+
+ # By default, the frobnicator will frobnicate with the default frobber.
+ # The following will make it use an alternative frobber.
+ #
+ #frobincator_frobber: special_frobber
+
+ # Settings for the frobber
+ #
+ frobber:
+ # frobbing speed. Defaults to 1.
+ #
+ #speed: 10
+
+ # frobbing distance. Defaults to 1000.
+ #
+ #distance: 100
+
+Note that the sample configuration is generated from the synapse code and is
+maintained by a script, ``scripts-dev/generate_sample_config``. Making sure
+that the output from this script matches the desired format is left as an
+exercise for the reader!
diff --git a/docs/metrics-howto.rst b/docs/metrics-howto.rst
index 32b064e2da..973641f3dc 100644
--- a/docs/metrics-howto.rst
+++ b/docs/metrics-howto.rst
@@ -59,6 +59,108 @@ How to monitor Synapse metrics using Prometheus
Restart Prometheus.
+Renaming of metrics & deprecation of old names in 1.2
+-----------------------------------------------------
+
+Synapse 1.2 updates the Prometheus metrics to match the naming convention of the
+upstream ``prometheus_client``. The old names are considered deprecated and will
+be removed in a future version of Synapse.
+
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| New Name | Old Name |
++=============================================================================+=======================================================================+
+| python_gc_objects_collected_total | python_gc_objects_collected |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| python_gc_objects_uncollectable_total | python_gc_objects_uncollectable |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| python_gc_collections_total | python_gc_collections |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| process_cpu_seconds_total | process_cpu_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_client_sent_transactions_total | synapse_federation_client_sent_transactions |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_client_events_processed_total | synapse_federation_client_events_processed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_event_processing_loop_count_total | synapse_event_processing_loop_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_event_processing_loop_room_count_total | synapse_event_processing_loop_room_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_count_total | synapse_util_metrics_block_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_time_seconds_total | synapse_util_metrics_block_time_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_ru_utime_seconds_total | synapse_util_metrics_block_ru_utime_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_ru_stime_seconds_total | synapse_util_metrics_block_ru_stime_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_db_txn_count_total | synapse_util_metrics_block_db_txn_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_db_txn_duration_seconds_total | synapse_util_metrics_block_db_txn_duration_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_util_metrics_block_db_sched_duration_seconds_total | synapse_util_metrics_block_db_sched_duration_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_start_count_total | synapse_background_process_start_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_ru_utime_seconds_total | synapse_background_process_ru_utime_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_ru_stime_seconds_total | synapse_background_process_ru_stime_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_db_txn_count_total | synapse_background_process_db_txn_count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_db_txn_duration_seconds_total | synapse_background_process_db_txn_duration_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_background_process_db_sched_duration_seconds_total | synapse_background_process_db_sched_duration_seconds |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_storage_events_persisted_events_total | synapse_storage_events_persisted_events |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_storage_events_persisted_events_sep_total | synapse_storage_events_persisted_events_sep |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_storage_events_state_delta_total | synapse_storage_events_state_delta |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_storage_events_state_delta_single_event_total | synapse_storage_events_state_delta_single_event |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_storage_events_state_delta_reuse_delta_total | synapse_storage_events_state_delta_reuse_delta |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_server_received_pdus_total | synapse_federation_server_received_pdus |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_server_received_edus_total | synapse_federation_server_received_edus |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_notified_presence_total | synapse_handler_presence_notified_presence |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_federation_presence_out_total | synapse_handler_presence_federation_presence_out |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_presence_updates_total | synapse_handler_presence_presence_updates |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_timers_fired_total | synapse_handler_presence_timers_fired |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_federation_presence_total | synapse_handler_presence_federation_presence |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handler_presence_bump_active_time_total | synapse_handler_presence_bump_active_time |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_client_sent_edus_total | synapse_federation_client_sent_edus |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_client_sent_pdu_destinations_count_total | synapse_federation_client_sent_pdu_destinations:count |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_federation_client_sent_pdu_destinations_total | synapse_federation_client_sent_pdu_destinations:total |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_handlers_appservice_events_processed_total | synapse_handlers_appservice_events_processed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_notifier_notified_events_total | synapse_notifier_notified_events |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_push_bulk_push_rule_evaluator_push_rules_invalidation_counter_total | synapse_push_bulk_push_rule_evaluator_push_rules_invalidation_counter |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_push_bulk_push_rule_evaluator_push_rules_state_size_counter_total | synapse_push_bulk_push_rule_evaluator_push_rules_state_size_counter |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_http_httppusher_http_pushes_processed_total | synapse_http_httppusher_http_pushes_processed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_http_httppusher_http_pushes_failed_total | synapse_http_httppusher_http_pushes_failed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_http_httppusher_badge_updates_processed_total | synapse_http_httppusher_badge_updates_processed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+| synapse_http_httppusher_badge_updates_failed_total | synapse_http_httppusher_badge_updates_failed |
++-----------------------------------------------------------------------------+-----------------------------------------------------------------------+
+
+
Removal of deprecated metrics & time based counters becoming histograms in 0.31.0
---------------------------------------------------------------------------------
diff --git a/docs/postgres.rst b/docs/postgres.rst
index 0ae52ccbd8..e08a5116b9 100644
--- a/docs/postgres.rst
+++ b/docs/postgres.rst
@@ -11,7 +11,9 @@ a postgres database.
* If you are using the `matrix.org debian/ubuntu
packages <../INSTALL.md#matrixorg-packages>`_,
- the necessary libraries will already be installed.
+ the necessary python library will already be installed, but you will need to
+ ensure the low-level postgres library is installed, which you can do with
+ ``apt install libpq5``.
* For other pre-built packages, please consult the documentation from the
relevant package.
@@ -34,7 +36,7 @@ Assuming your PostgreSQL database user is called ``postgres``, create a user
su - postgres
createuser --pwprompt synapse_user
-Before you can authenticate with the ``synapse_user``, you must create a
+Before you can authenticate with the ``synapse_user``, you must create a
database that it can access. To create a database, first connect to the database
with your database user::
@@ -53,7 +55,7 @@ and then run::
This would create an appropriate database named ``synapse`` owned by the
``synapse_user`` user (which must already have been created as above).
-Note that the PostgreSQL database *must* have the correct encoding set (as
+Note that the PostgreSQL database *must* have the correct encoding set (as
shown above), otherwise it will not be able to store UTF8 strings.
You may need to enable password authentication so ``synapse_user`` can connect
diff --git a/docs/sample_config.yaml b/docs/sample_config.yaml
index 663ff31622..5b804d16a4 100644
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@@ -1409,17 +1409,34 @@ password_config:
## Opentracing ##
-# These settings enable opentracing which implements distributed tracing
-# This allows you to observe the causal chain of events across servers
-# including requests, key lookups etc. across any server running
-# synapse or any other other services which supports opentracing.
-# (specifically those implemented with jaeger)
-
-#opentracing:
-# # Enable / disable tracer
-# tracer_enabled: false
-# # The list of homeservers we wish to expose our current traces to.
-# # The list is a list of regexes which are matched against the
-# # servername of the homeserver
-# homeserver_whitelist:
-# - ".*"
+
+# These settings enable opentracing, which implements distributed tracing.
+# This allows you to observe the causal chains of events across servers
+# including requests, key lookups etc., across any server running
+# synapse or any other other services which supports opentracing
+# (specifically those implemented with Jaeger).
+#
+opentracing:
+ # tracing is disabled by default. Uncomment the following line to enable it.
+ #
+ #enabled: true
+
+ # The list of homeservers we wish to send and receive span contexts and span baggage.
+ #
+ # Though it's mostly safe to send and receive span contexts to and from
+ # untrusted users since span contexts are usually opaque ids it can lead to
+ # two problems, namely:
+ # - If the span context is marked as sampled by the sending homeserver the receiver will
+ # sample it. Therefore two homeservers with wildly disparaging sampling policies
+ # could incur higher sampling counts than intended.
+ # - Span baggage can be arbitrary data. For safety this has been disabled in synapse
+ # but that doesn't prevent another server sending you baggage which will be logged
+ # to opentracing logs.
+ #
+ # This a list of regexes which are matched against the server_name of the
+ # homeserver.
+ #
+ # By defult, it is empty, so no servers are matched.
+ #
+ #homeserver_whitelist:
+ # - ".*"
|
