summary refs log tree commit diff
path: root/synapse/app/_base.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Merge remote-tracking branch 'gitlab/clokep/license-license' into new_developErik Johnston2023-12-131-11/+16
|\
| * Update license headersPatrick Cloke2023-11-211-11/+16
| |
* | Sentry Alert configuration based on production and development environment ↵Zeeshan Rafiq2023-12-121-0/+1
|/ | | | (#16738)
* Use StrCollection in additional places. (#16301)Patrick Cloke2023-09-131-7/+5
|
* Re-introduce the outbound federation proxy (#15913)Eric Eastwood2023-07-181-0/+2
| | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
* Revert "Federation outbound proxy" (#15910)Eric Eastwood2023-07-101-2/+0
| | | | | | Revert "Federation outbound proxy (#15773)" This reverts commit b07b14b494ae1dd564b4c44f844c9a9545b3d08a.
* Federation outbound proxy (#15773)Eric Eastwood2023-07-051-0/+2
| | | | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
* `traceback.format_exception(...)` usage that is compatible with Python 3.7 ↵Eric Eastwood2023-05-161-1/+1
| | | | | | | | | | | | | and 3.11 (#15599) * Usage that is compatible with Python 3.8 and 3.11 > Since Python 3.10, instead of passing value and tb, an exception object can be passed as the first argument. If value and tb are provided, the first argument is ignored in order to provide backwards compatibility. > > -- https://docs.python.org/3/library/traceback.html * Add changelog
* Print full startup/initialization error (#15569)Eric Eastwood2023-05-111-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I found the error in the **Before** really vague and obtuse and didn't realize port `5432` corresponded to the Postgres port until searching the codebase. It says to check the logs but that wasn't my first instinct. It's just more obvious if we just print the full thing which gives context of the error type and the traceback to the relevant area of code. #### Before ``` $ poetry run python -m synapse.app.homeserver -c homeserver.yaml ********************************************************************************** Error during initialisation: connection to server at "localhost" (::1), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? There may be more information in the logs. ********************************************************************************** ``` #### After ```sh $ poetry run python -m synapse.app.homeserver -c homeserver.yaml ********************************************************************************** Error during initialisation: Traceback (most recent call last): File "/home/eric/Documents/github/element/synapse/synapse/app/homeserver.py", line 352, in setup hs.setup() File "/home/eric/Documents/github/element/synapse/synapse/server.py", line 337, in setup self.datastores = Databases(self.DATASTORE_CLASS, self) File "/home/eric/Documents/github/element/synapse/synapse/storage/databases/__init__.py", line 65, in __init__ with make_conn(database_config, engine, "startup") as db_conn: File "/home/eric/Documents/github/element/synapse/synapse/storage/database.py", line 161, in make_conn native_db_conn = engine.module.connect(**db_params) File "/home/eric/.cache/pypoetry/virtualenvs/matrix-synapse-xCtC9ulO-py3.10/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: connection to server at "localhost" (::1), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? There may be more information in the logs. ********************************************************************************** ```
* Move ThirdPartyEventRules into module_api/callbacks (#15535)Andrew Morgan2023-05-041-1/+3
|
* Move Spam Checker callbacks to a dedicated file (#15453)Andrew Morgan2023-04-181-1/+1
|
* Experimental Unix socket support (#15353)Jason Little2023-04-031-27/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add IReactorUNIX to ISynapseReactor type hint. * Create listen_unix(). Two options, 'path' to the file and 'mode' of permissions(not umask, recommend 666 as default as nginx/other reverse proxies write to it and it's setup as user www-data) For the moment, leave the option to always create a PID lockfile turned on by default * Create UnixListenerConfig and wire it up. Rename ListenerConfig to TCPListenerConfig, then Union them together into ListenerConfig. This spidered around a bit, but I think I got it all. Metrics and manhole have been placed behind a conditional in case of accidental putting them onto a unix socket. Use new helpers to get if a listener is configured for TLS, and to help create a site tag for logging. There are 2 TODO things in parse_listener_def() to finish up at a later point. * Refactor SynapseRequest to handle logging correctly when using a unix socket. This prevents an exception when an IP address can not be retrieved for a request. * Make the 'Synapse now listening on Unix socket' log line a little prettier. * No silent failures on generic workers when trying to use a unix socket with metrics or manhole. * Inline variables in app/_base.py * Update docstring for listen_unix() to remove reference to a hardcoded permission of 0o666 and add a few comments saying where the default IS declared. * Disallow both a unix socket and a ip/port combo on the same listener resource * Linting * Changelog * review: simplify how listen_unix returns(and get rid of a type: ignore) * review: fix typo from ConfigError in app/homeserver.py * review: roll conditional for http_options.tag into get_site_tag() helper(and add docstring) * review: enhance the conditionals for checking if a port or path is valid, remove a TODO line * review: Try updating comment in get_client_ip_if_available to clarify what is being retrieved and why * Pretty up how 'Synapse now listening on Unix Socket' looks by decoding the byte string. * review: In parse_listener_def(), raise ConfigError if neither socket_path nor port is declared(and fix a typo)
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-221-1/+1
|
* Remove legacy Prometheus metrics names. They were deprecated in Synapse ↵reivilibre2022-11-241-12/+4
| | | | v1.69.0 and disabled by default in Synapse v1.71.0. (#14538)
* Fix typechecking errors introduced in #14128 (#14455)David Robertson2022-11-151-2/+2
| | | | | | | | | * Fix typechecking errors introduced in #14128 * Changelog * Correct annotations so that context_factory works if you don't use TLS
* Support using SSL on worker endpoints. (#14128)Tuomas Ojamies2022-11-151-1/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix missing SSL support in worker endpoints. * Add changelog * SSL for Replication endpoint * Remove unit test change * Refactor listener creation to reduce duplicated code * Fix the logger message * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add config documentation for new TLS option Co-authored-by: Tuomas Ojamies <tojamies@palantir.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Fix logging error on SIGHUP (#14258)Richard van der Hoff2022-10-211-1/+1
|
* Update mypy and mypy-zope, attempt 3 (#13993)David Robertson2022-09-301-3/+1
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Revert "Update mypy and mypy-zope (#13925)"David Robertson2022-09-301-1/+3
| | | | This reverts commit 6d543d6d9f56e39199b7e460d0081b02d61f12be.
* Update mypy and mypy-zope (#13925)David Robertson2022-09-301-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update mypy and mypy-zope * Unignore assigning to LogRecord attributes Presumably https://github.com/python/typeshed/pull/8064 makes this ok Cherry-picked from #13521 * Remove unused ignores due to mypy ParamSpec fixes https://github.com/python/mypy/pull/12668 Cherry-picked from #13521 * Remove additional unused ignores * Fix new mypy complaints related to `assertGreater` Presumably due to https://github.com/python/typeshed/pull/8077 * Changelog * Reword changelog Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Share some metrics between the Prometheus exporter and the phone home stats ↵Brendan Abolivier2022-09-051-2/+3
| | | | (#13671)
* Add experimental configuration option to allow disabling legacy Prometheus ↵reivilibre2022-08-241-3/+36
| | | | | metric names. (#13540) Co-authored-by: David Robertson <davidr@element.io>
* Revert "Update locked versions of mypy and mypy-zope (#13521)"David Robertson2022-08-151-1/+3
| | | | | | | | This reverts commit f383b9b3eceaa082d5ae690550fe41460b711779. Other PRs were seeing mypy failures that looked to be related to mypy-zope. Confusingly, we didn't see this on #13521. Revert this for now and investigate later.
* Update locked versions of mypy and mypy-zope (#13521)David Robertson2022-08-151-3/+1
|
* Improve startup times in Complement test runs against workers, particularly ↵reivilibre2022-06-301-2/+6
| | | | | in CPU-constrained environments. (#13127) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Don't use keyword arguments when initialising modules (#13060)Brendan Abolivier2022-06-151-1/+1
|
* Fix Synapse git info missing in version strings (#12973)David Robertson2022-06-071-2/+2
|
* Reload cache factors from disk on SIGHUP (#12673)David Robertson2022-05-111-0/+44
|
* Use `ParamSpec` in a few places (#12667)David Robertson2022-05-091-4/+10
|
* Remove unused `# type: ignore`s (#12531)David Robertson2022-04-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | Over time we've begun to use newer versions of mypy, typeshed, stub packages---and of course we've improved our own annotations. This makes some type ignore comments no longer necessary. I have removed them. There was one exception: a module that imports `select.epoll`. The ignore is redundant on Linux, but I've kept it ignored for those of us who work on the source tree using not-Linux. (#11771) I'm more interested in the config line which enforces this. I want unused ignores to be reported, because I think it's useful feedback when annotating to know when you've fixed a problem you had to previously ignore. * Installing extras before typechecking Lacking an easy way to install all extras generically, let's bite the bullet and make install the hand-maintained `all` extra before typechecking. Now that https://github.com/matrix-org/backend-meta/pull/6 is merged to the release/v1 branch.
* Add missing type hints to config classes. (#12402)Patrick Cloke2022-04-111-1/+3
|
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Use version string helper from matrix-common (#11979)David Robertson2022-02-141-2/+3
| | | | * Require latest matrix-common * Use the common function
* Check that `gc` method is available before using in `synapse/app/_base` ↵Shay2022-01-251-6/+8
| | | | | | | | | | | | | | | (#11816) * add check that gc.freeze is available before calling * newsfragment * lint * Update comment Co-authored-by: Dan Callahan <danc@element.io> Co-authored-by: Dan Callahan <danc@element.io>
* Log modules at startup (#11813)Brendan Abolivier2022-01-251-1/+2
|
* Drop support for and remove references to EOL Python 3.6 (#11683)Shay2022-01-211-8/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * remove reference in comments to python3.6 * upgrade tox python env in script * bump python version in example for completeness * upgrade python version requirement in setup doc * upgrade necessary python version in __init__.py * upgrade python version in setup.py * newsfragment * drops refs to bionic and replace with focal * bump refs to postgres 9.6 to 10 * fix hanging ci * try installing tzdata first * revert change made in b979f336 * ignore new random mypy error while debugging other error * fix lint error for temporary workaround * revert change to install list * try passing env var * export debian frontend var? * move line and add comment * bump pillow dependency * bump lxml depenency * install libjpeg-dev for pillow * bump automat version to one compatible with py3.8 * add libwebp for pillow * bump twisted trunk python version * change suffix of newsfragment * remove redundant python 3.7 checks * lint
* Simplify GC prometheus metrics (#11723)Richard van der Hoff2022-01-131-1/+2
| | | | | Rather than hooking into the reactor loop, just add a timed task that runs every 100 ms to do the garbage collection. Part 1 of a quest to simplify the reactor monkey-patching.
* Additional type hints for config module. (#11465)Patrick Cloke2021-12-011-1/+2
| | | | This adds some misc. type hints to helper methods used in the `synapse.config` module.
* Add type annotations to `synapse.metrics` (#10847)Sean Quah2021-11-171-1/+1
|
* Add missing type hints to `synapse.app`. (#11287)Patrick Cloke2021-11-101-51/+89
|
* Add metrics to the threadpools (#11178)Erik Johnston2021-11-011-0/+5
|
* Shut down the DNS threadpool (#11190)Sean Quah2021-10-271-0/+1
| | | | | The DNS threadpool must be explicitly stopped, otherwise Synapse will hang indefinitely when asked to shut down.
* Move DNS lookups into separate thread pool (#11177)Erik Johnston2021-10-261-1/+12
| | | | | | This is to stop large bursts of lookups starving out other users of the thread pools. Fixes #11049.
* Add type hints for most `HomeServer` parameters (#11095)Sean Quah2021-10-221-4/+4
|
* Port the Password Auth Providers module interface to the new generic ↵Azrenbeth2021-10-131-0/+2
| | | | | | interface (#10548) Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Require direct references to configuration variables. (#10985)Patrick Cloke2021-10-061-1/+1
| | | | | | This removes the magic allowing accessing configurable variables directly from the config object. It is now required that a specific configuration class is used (e.g. `config.foo` must be replaced with `config.server.foo`).
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-291-4/+4
|
* Use direct references for configuration variables (part 5). (#10897)Patrick Cloke2021-09-241-2/+2
|
* Use direct references for configuration variables (part 4). (#10893)Patrick Cloke2021-09-231-2/+4
|
* Use direct references for some configuration variables (#10798)Patrick Cloke2021-09-131-4/+8
| | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
* Add config option to use non-default manhole password and keys (#10643)Azrenbeth2021-09-061-2/+8
|
* Skip the final GC on shutdown to improve restart times (#10712)Sean2021-09-011-0/+7
| | | | | | | | | | Use `gc.freeze()` on exit to exclude all existing objects from the final GC. In testing, this sped up shutdown by up to a few seconds. `gc.freeze()` runs in constant time, so there is little chance of performance regression. Signed-off-by: Sean Quah <seanq@element.io>
* Port the PresenceRouter module interface to the new generic interface (#10524)Azrenbeth2021-08-171-0/+2
| | | Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
* Port the ThirdPartyEventRules module interface to the new generic interface ↵Brendan Abolivier2021-07-201-0/+2
| | | | | (#10386) Port the third-party event rules interface to the generic module interface introduced in v1.37.0
* Add support for evicting cache entries based on last access time. (#10205)Erik Johnston2021-07-051-2/+9
|
* Ensure that errors during startup are written to the logs and the console. ↵Richard van der Hoff2021-06-211-1/+27
| | | | | | | | (#10191) * Defer stdio redirection until we are about to start the reactor * Catch and handle exceptions during startup
* Standardise the module interface (#10062)Brendan Abolivier2021-06-181-0/+9
| | | This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
* Remove support for ACME v1 (#10194)Brendan Abolivier2021-06-171-2/+1
| | | | | Fixes #9778 ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
* Rip out the DNS lookup limiter (#10190)Richard van der Hoff2021-06-171-104/+0
| | | As I've written in various places in the past (#7113, #9865) I'm pretty sure this is doing nothing useful at all.
* Remove redundant code to reload tls cert (#10054)Richard van der Hoff2021-05-271-4/+1
| | | | | we don't need to reload the tls cert if we don't have any tls listeners. Follow-up to #9280.
* Export jemalloc stats to prometheus when used (#9882)Erik Johnston2021-05-061-0/+2
|
* Improved validation for received requests (#9817)Richard van der Hoff2021-04-231-4/+26
| | | | | | * Simplify `start_listening` callpath * Correctly check the size of uploaded files
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Suppress CryptographyDeprecationWarning (#9698)Richard van der Hoff2021-03-261-0/+21
| | | This warning is somewhat confusing to users, so let's suppress it
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-9/+5
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Fixes for PyPy compatibility (#9270)Ivan Shapovalov2021-01-301-1/+2
| | | | | | | | | | | | * synapse.app.base: only call gc.freeze() on CPython gc.freeze() is an implementation detail of CPython garbage collector, and notably does not exist on PyPy. Rather than playing whack-a-mole and skipping the call when under PyPy, simply restrict it to CPython because the whole gc module is implementation-defined. Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
* Clean up exception handling in the startup code (#9059)Richard van der Hoff2021-01-111-59/+91
| | | | Factor out the exception handling in the startup code to a utility function, and fix the some logging and exit code stuff.
* Fix deadlock on SIGHUP (#8918)Erik Johnston2020-12-101-1/+5
| | | Fixes #8892
* Defer SIGHUP handlers to reactor. (#8817)Erik Johnston2020-11-261-1/+9
| | | | | | | | | We can get a SIGHUP at any point, including times where we are not in a sane state. By deferring calling the handlers until the next reactor tick we ensure that we don't get unexpected conflicts, e.g. trying to flush logs from the signal handler while the code was in the process of writing a log entry. Fixes #8769.
* Fix SIGHUP handler (#8697)Richard van der Hoff2020-11-061-3/+2
| | | | | | | Fixes: ``` builtins.TypeError: _reload_logging_config() takes 1 positional argument but 2 were given ```
* Add logging on startup/shutdown (#8448)Erik Johnston2020-10-021-0/+5
| | | | | This is so we can tell what is going on when things are taking a while to start up. The main change here is to ensure that transactions that are created during startup get correctly logged like normal transactions.
* Allow background tasks to be run on a separate worker. (#8369)Patrick Cloke2020-10-021-0/+6
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-2/+2
|
* Add a comment about _LimitedHostnameResolverRichard van der Hoff2020-08-291-0/+7
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-1/+1
|
* re-implement daemonize (#8011)Richard van der Hoff2020-08-041-13/+3
| | | | | | | | | | | | | This has long been something I've wanted to do. Basically the `Daemonize` code is both too flexible and not flexible enough, in that it offers a bunch of features that we don't use (changing UID, closing FDs in the child, logging to syslog) and doesn't offer a bunch that we could do with (redirecting stdout/err to a file instead of /dev/null; having the parent not exit until the child is running). As a first step, I've lifted the Daemonize code and removed the bits we don't use. This should be a non-functional change. Fixing everything else will come later.
* Create a ListenerConfig object (#7681)Richard van der Hoff2020-06-161-3/+5
| | | | | | | | | | This ended up being a bit more invasive than I'd hoped for (not helped by generic_worker duplicating some of the code from homeserver), but hopefully it's an improvement. The idea is that, rather than storing unstructured `dict`s in the config for the listener configurations, we instead parse it into a structured `ListenerConfig` object.
* Fix new flake8 errors (#7470)Erik Johnston2020-05-121-2/+3
|
* Add instance name to RDATA/POSITION commands (#7364)Erik Johnston2020-04-291-2/+2
| | | | | This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
* Move pusherpool startup into _base.setup (#7104)Richard van der Hoff2020-03-191-0/+1
| | | | | This should be safe to do on all workers/masters because it is guarded by a config option which will ensure it is only actually done on the worker assigned as a pusher.
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-1/+1
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Freeze allocated objects on startup. (#6953)Erik Johnston2020-02-191-0/+9
| | | | | This may make gc go a bit faster as the gc will know things like caches/data stores etc. are frozen without having to check.
* Fix sdnotify with acme enabled (#6571)Richard van der Hoff2019-12-191-3/+7
| | | | | | | If acme was enabled, the sdnotify startup hook would never be run because we would try to add it to a hook which had already fired. There's no need to delay it: we can sdnotify as soon as we've started the listeners.
* Move DB pool and helper functions into dedicated Database classErik Johnston2019-12-051-1/+1
|
* Fix up some typechecking (#6150)Amber Brown2019-10-021-1/+3
| | | | | | * type checking fixes * changelog
* Implement a structured logging output system. (#5680)Amber Brown2019-08-281-5/+7
|
* Drop dependency on sdnotify (#5871)Richard van der Hoff2019-08-171-10/+37
| | | | | | ... to save OSes which don't use it from having to maintain a port. Fixes #5865.
* Add process hooks to tell systemd our state.Erik Johnston2019-07-221-0/+29
| | | | Fixes #5676.
* Support Prometheus_client 0.4.0+ (#5636)Amber Brown2019-07-181-2/+1
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_api_cmdErik Johnston2019-07-151-29/+34
|\
| * Add basic opentracing support (#5544)Jorik Schellekens2019-07-111-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter
| * Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
| |
| * Fix 'utime went backwards' errors on daemonization. (#5609)Richard van der Hoff2019-07-031-27/+30
| | | | | | | | | | | | | | | | * Fix 'utime went backwards' errors on daemonization. Fixes #5608 * remove spurious debug
* | Add basic admin cmd appErik Johnston2019-07-021-3/+14
|/
* Drop support for cpu_affinity (#5525)Richard van der Hoff2019-06-221-25/+1
| | | This has no useful purpose on python3, and is generally a source of confusion.
* Run Black. (#5482)Amber Brown2019-06-201-41/+36
|
* Fix dropped logcontexts during high outbound traffic. (#5277)Richard van der Hoff2019-05-291-7/+13
| | | Fixes #5271.
* Limit in flight DNS requestsErik Johnston2019-04-091-1/+82
| | | | | | | This is to work around a bug in twisted where a large number of concurrent DNS requests cause it to tight loop forever. c.f. https://twistedmatrix.com/trac/ticket/9620#ticket
* Correctly handle all command line optionsErik Johnston2019-03-141-6/+12
|
* Merge pull request #4632 from matrix-org/erikj/basic_sentryErik Johnston2019-02-181-0/+30
|\ | | | | Add basic optional sentry.io integration
| * Fixup comments and add warningErik Johnston2019-02-131-3/+3
| |
| * CommentsErik Johnston2019-02-121-0/+8
| |
| * Basic sentry integrationErik Johnston2019-02-121-0/+22
| |
* | Update synapse/app/_base.pyErik Johnston2019-02-131-1/+1
| | | | | | Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
* | Improve logging around listening servicesRichard van der Hoff2019-02-131-9/+14
| | | | | | | | | | | | | | | | | | I wanted to bring listen_tcp into line with listen_ssl in terms of returning a list of ports, and wanted to check that was a safe thing to do - hence the logging in `refresh_certificate`. Also, pull the 'Synapse now listening' message up to homeserver.py, because it was being duplicated everywhere else.
* | Special-case the default bind_addresses for metrics listenerRichard van der Hoff2019-02-131-3/+2
|/ | | | | turns out it doesn't really support ipv6, so let's hack around that by only listening on ipv4 by default.
* Fix error when loading cert if tls is disabled (#4618)Richard van der Hoff2019-02-121-2/+3
| | | | | | If TLS is disabled, it should not be an error if no cert is given. Fixes #4554.
* Infer no_tls from presence of TLS listenersRichard van der Hoff2019-02-111-1/+1
| | | | | Rather than have to specify `no_tls` explicitly, infer whether we need to load the TLS keys etc from whether we have any TLS-enabled listeners.
* Don't create server contexts when TLS is disabledRichard van der Hoff2019-02-111-0/+5
| | | | we aren't going to use them anyway.
* Logging improvements around TLS certsRichard van der Hoff2019-02-111-4/+2
| | | | | Log which file we're reading keys and certs from, and refactor the code a bit in preparation for other work
* Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)Richard van der Hoff2019-02-111-3/+0
| | | | | It's nothing to do with refreshing the certificates. No idea why it was here.
* ACME Reprovisioning (#4522)Amber Brown2019-02-111-0/+19
|
* Deduplicate some code in synapse.app (#4567)Amber Brown2019-02-081-0/+63
|
* ACME config cleanups (#4525)Richard van der Hoff2019-01-301-21/+1
| | | | | | | | | | | | | | | | * Handle listening for ACME requests on IPv6 addresses the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses without extra quoting. Building a string which you are about to parse again seems like a weird choice. Let's just use listenTCP, which is consistent with what we do elsewhere. * Clean up the default ACME config make it look a bit more consistent with everything else, and tweak the defaults to listen on port 80. * newsfile
* SIGHUP for TLS cert reloading (#4495)Amber Brown2019-01-301-7/+22
|
* Make workers work on Py3 (#4027)Amber Brown2018-10-131-15/+15
|
* Integrate presence from hotfixes (#3694)Amber Brown2018-08-181-2/+4
|
* run isortAmber Brown2018-07-091-4/+7
|
* Run Prometheus on a different port, optionally. (#3274)Amber Brown2018-05-311-0/+13
|
* Remove logger argument and do not catch replication listenerSilke2017-12-181-6/+7
| | | | Signed-off-by: Silke <silke@slxh.eu>
* Add methods for listening on multiple addressesSilke Hofstra2017-12-171-1/+56
| | | | | | | Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP and reactor.listenSSL for multiple addresses. Signed-off-by: Silke Hofstra <silke@slxh.eu>
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Make 'affinity' package optionalErik Johnston2017-10-021-1/+14
|
* Move quit_with_errorErik Johnston2017-10-021-0/+10
|
* Allow configuration of CPU affinityRichard van der Hoff2017-08-151-1/+8
| | | | | Make it possible to set the CPU affinity in the config file, so that we don't need to remember to do it manually every time.
* Factor out common application startRichard van der Hoff2017-08-151-0/+92
We have 10 copies of this code, and I don't really want to update each one separately.