| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
| |
Signed-off-by: Kai A. Hiller <V02460@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
This PR is tantamount to running
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```
Part of #9744
|
| |
|
|
|
|
|
|
| |
(#10191)
* Defer stdio redirection until we are about to start the reactor
* Catch and handle exceptions during startup
|
| |
|
| |
This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
|
| |
|
|
|
| |
Fixes #9778
ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
|
| |
|
|
|
| |
This will double count slightly in the presence of interned strings. It's off by default as it can consume a lot of resources.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Synapse can be quite memory intensive, and unless care is taken to tune
the GC thresholds it can end up thrashing, causing noticable performance
problems for large servers. We fix this by limiting how often we GC a
given generation, regardless of current counts/thresholds.
This does not help with the reverse problem where the thresholds are set
too high, but that should only happen in situations where they've been
manually configured.
Adds a `gc_min_seconds_between` config option to override the defaults.
Fixes #9890.
|
| |
|
|
|
|
| |
* Simplify `start_listening` callpath
* Correctly check the size of uploaded files
|
| | |
|
| |
|
|
| |
Every single time I want to access the config object, I have to remember
whether or not we use `get_config`. Let's just get rid of it.
|
| |
|
|
|
|
|
| |
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
|
| |
|
| |
This warning is somewhat confusing to users, so let's suppress it
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes #8966.
* Factor out build_synapse_client_resource_tree
Start a function which will mount resources common to all workers.
* Move sso init into build_synapse_client_resource_tree
... so that we don't have to do it for each worker
* Fix SSO-login-via-a-worker
Expose the SSO login endpoints on workers, like the documentation says.
* Update workers config for new endpoints
Add documentation for endpoints recently added (#8942, #9017, #9262)
* remove submit_token from workers endpoints list
this *doesn't* work on workers (yet).
* changelog
* Add a comment about the odd path for SAML2Resource
|
| |
|
| |
There are going to be a couple of paths to get to the final step of SSO reg, and I want the URL in the browser to consistent. So, let's move the final step onto a separate path, which we redirect to.
|
| |
|
|
|
|
|
| |
The idea here is that we will have an instance of OidcProvider for each
configured IdP, with OidcHandler just doing the marshalling of them.
For now it's still hardcoded with a single provider.
|
| |
|
|
| |
Factor out the exception handling in the startup code to a utility function,
and fix the some logging and exit code stuff.
|
| | |
|
| |
|
|
|
| |
During login, if there are multiple IdPs enabled, offer the user a choice of
IdPs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow
`UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser
back to the client, we redirect to a username-picker resource, which allows the user to enter a username.
We *then* complete the SSO flow (including doing the client permission checks).
The static resources for the username picker itself (in
https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker)
are essentially lifted wholesale from
https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res.
As the comment says, we might want to think about making them customisable, but that can be a follow-up.
Fixes #8876.
|
| |
|
|
|
|
|
|
|
|
| |
The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError,
and it will be magically turned into a legible error message. There's a few components to it:
* Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it
into an absolute path.
* Generally improving the way ConfigErrors get printed.
* Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.
|
| |
|
|
|
| |
Lots of different module apis is not easy to maintain.
Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
|
| | |
|
| | |
|
| |
|
|
|
| |
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.
This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.
The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
|
| |/ |
|
| | |
|
| |
|
|
|
| |
Based on #7619
async's `get_user_id_by_threepid` and its call stack.
|
| | |
|
| |
|
|
| |
variables (#6391)
|
| | |
|
| |
|
| |
This is configured via the `redis` config options.
|
| |
|
|
|
| |
Log warning when filesystem path is used.
Signed-off-by: Martin Milata <martin@martinmilata.cz>
|
| |
|
|
|
| |
This should be safe to do on all workers/masters because it is guarded by
a config option which will ensure it is only actually done on the worker
assigned as a pusher.
|
| |
|
|
|
| |
* Break down monthly active users by appservice_id and emit via prometheus.
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
| |
|
|
|
|
| |
AdditionalResource really doesn't add any value, and it gets in the way for
resources which want to support child resources or the like. So, if the
resource object already implements the IResource interface, don't bother
wrapping it.
|
| |
|
|
| |
This has caused some confusion for people who didn't notice it going away.
|
| |
|
|
|
|
| |
This looks like it got half-killed back in #888.
Fixes #6567.
|
| |
|
|
| |
`Failed to upgrade database` is not helpful, and it's unlikely that UPGRADE.rst
has anything useful.
|
| |\
| |
| | |
Move database config from apps into HomeServer object
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Fix phone home stats
|
| | |
|
| |
|
| |
* remove psutil and replace with resource
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy.
The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`.
The proxy will then be used for
* push
* url previews
* phone-home stats
* recaptcha validation
* CAS auth validation
It will *not* be used for:
* Application Services
* Identity servers
* Outbound federation
* In worker configurations, connections from workers to masters
Fixes #4198.
|
| |
|
| |
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
|
| | |
|
| |
|
| |
This PR adds the optional `report_stats_endpoint` to configure where stats are reported to, if enabled.
|
| | |
|
| |
|
|
| |
Signed-off-by: Chris Moos <chris@chrismoos.com>
|
| | |
|
| |
|
|
|
| |
This helps ensures that we only consider ourselves "up" once all the
startup functions have completed.
|
| | |
|
| | |
|
| |
|
| |
This has no useful purpose on python3, and is generally a source of confusion.
|
| | |
|
| |\ |
|
| | | |
|
| |/
|
|
| |
* add monthly active users to phonehome stats
|
| |
|
|
|
|
|
|
|
|
|
|
| |
identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.
This PR is a culmination of 3 smaller PRs which have each been separately reviewed:
* #5308
* #5345
* #5368
|
| |
|
|
| |
It doesn't really belong under rest/client/v1 any more.
|
| | |
|
| |
|
|
| |
add context to phonehome stats
|
| |\
| |
| | |
Allow passing --daemonize to workers
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* Better logging for errors on startup
* Fix "TypeError: '>' not supported" when starting without an existing
certificate
* Fix a bug where an existing certificate would be reprovisoned every day
|
| |
|
|
|
| |
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
|
| |
|
|
|
|
|
|
|
| |
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.
Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
|
| |
|
|
|
| |
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
|
| | |
|
| |\
| |
| | |
New listener resource for the federation API "openid/userinfo" endpoint
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
For all the homeserver classes, only the FrontendProxyServer passes
its reactor when doing the http listen. Looking at previous PR's looks
like this was introduced to make it possible to write a test, otherwise
when you try to run a test with the test homeserver it tries to
do a real bind to a port. Passing the reactor that the homeserver
is instantiated with should probably be the right thing to do anyway?
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
| |
optional dependencies to setuptools (#4298)
|
| |
|
| |
* ensure can report mau stats when hs.config.mau_stats_only is set
|
| |\ |
|
| | |\
| | |
| | | |
Stop installing Matrix Console by default
|
| | | |
| | |
| | |
| | | |
This is based on the work done by @krombel in #2601.
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is largely a precursor for the removal of the bundled webclient. The idea
is to present a page at / which reassures people that something is working, and
to give them some links for next steps.
The welcome page lives at `/_matrix/static/`, so is enabled alongside the other
`static` resources (which, in practice, means the client API is enabled). We'll
redirect to it from `/` if we have nothing better to display there.
It would be nice to have a way to disable it (in the same way that you might
disable the nginx welcome page), but I can't really think of a good way to do
that without a load of ickiness.
It's based on the work done by @krombel for #2601.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.
What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
|
| |/
|
|
| |
Sometimes it's useful for synapse to generate its own .well-known file.
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Run MAU queries as background processes
|
| | |
| |
| |
| | |
Fixes #3820
|
| |\ \
| | |
| | | |
Make psutil an explicit dependency
|
| | |/
| |
| |
| |
| |
| |
| |
| | |
As of #4027, we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992.
Fixes #4062.
|
| |/
|
|
| |
on py3) (#4068)
|
| |
|
|
|
| |
I think this got forgotten in #3932. We were getting away with it because it
was the last call in this function.
|
| |
|
|
| |
Signed-off-by: Schnuffle <schnuffle@github.com>
|
| |
|
|
|
|
|
|
| |
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
We should explicitly close any db connections we open, because failing to do so
can block other transactions as per
https://github.com/matrix-org/synapse/issues/3682.
Let's also try to factor out some of the boilerplate by having server classes
define their datastore class rather than duplicating the whole of `setup`.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
send SNI for federation requests
|
| | |\ |
|
| | |\ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
send_sni_for_federation_requests
# Conflicts:
# synapse/crypto/context_factory.py
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| |\ \ \ \ |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | |_|/
| |/| | |
|
| |/ / / |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
|
| | | |
| | |
| | |
| | |
| | | |
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
|
| | |/
|/| |
|
| |/ |
|
| |
|
|
|
| |
I'm fed up with never being able to find the point a server restarted in the
logs.
|
| |
|
| |
Signed-off-by: Travis Ralston <travpc@gmail.com>
|
| |\
| |
| | |
daily user type phone home stats
|
| | | |
|
| | | |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| | |
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
|
| | |\
| | |
| | | |
user visit data
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
The aim is to keep track of when it was last called and only query from that point in time
|
| | | |
| | |
| | |
| | | |
insert incrementally through the day
|
| | | |\
| | | |
| | | |
| | | | |
cohort_analytics
|
| | | | | |
|
| | | | | |
|
| | |\ \ \
| | |_|/
| |/| | |
ConsentResource to gather policy consent from users
|
| | | |/
| | |
| | |
| | |
| | | |
Hopefully there are enough comments and docs in this that it makes sense on its
own.
|
| |/ / |
|
| |/
|
|
|
|
|
|
|
|
|
|
| |
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).
This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.
Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
|
| |\
| |
| | |
R30 stats
|
| | |
| |
| |
| |
| | |
Use iteritems over item to loop over dict
formatting
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
| |
This requires the psutil module, and is still opt-in based on the report_stats
config option.
|
| |
|
|
| |
fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
This function is identical to all subclasses, so we may as well push it up to
the base class to reduce duplication (and make use of it in the tests)
|
| |
|
|
| |
Signed-off-by: Silke <silke@slxh.eu>
|
| |
|
|
| |
Signed-off-by: Silke <silke@slxh.eu>
|
| |
|
|
|
|
|
| |
Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP
and reactor.listenSSL for multiple addresses.
Signed-off-by: Silke Hofstra <silke@slxh.eu>
|
| |
|
|
|
|
|
|
|
|
| |
Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed.
This causes a warning explaining the behaviour.
Configuration changed to match.
See #2232
Signed-off-by: Silke Hofstra <silke@slxh.eu>
|
| |
|
|
| |
... to stop us doing the cache cleanup jobs on the master.
|
| |
|
|
|
|
| |
This avoids the scenario where we have four different PreviewUrlResources
configured on a single app, each of which have their own caches and cache
clearing jobs.
|
| |\
| |
| | |
Add a hook for custom rest endpoints
|
| | |
| |
| |
| |
| | |
Let the user specify custom modules which can be used for implementing extra
endpoints.
|
| |/
|
|
| |
This was a bit of a code vomit, so let's factor it out to preserve some sanity
|
| | |
|
| |
|
|
|
| |
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
|
| |
|
|
|
| |
We have 10 copies of this code, and I don't really want to update each one
separately.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
|
| |
|
|
| |
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This fixes a class of 'Unexpected logcontext' messages, which were happening
because the logcontext was somewhat arbitrarily swapping between the sentinel
and the `run` logcontext.
|
| |
|
|
| |
- to make it easier to add more config options.
|
| | |
|
| |
|
|
|
|
|
| |
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| |
|
|
| |
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| |
| | |
Conflicts:
synapse/app/homeserver.py
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Mark AS users with their AS's ID
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.
I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.
Also, add some basic tests for config generation and loading.
|
| | |
|
| | |
|
| |\
| |
| | |
Add a Homeserver.setup method
|
| | | |
|
| | |\ |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
|
| | |/
|/| |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Change manhole to use ssh
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
| |
and track the number of transactions that request started.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
This is because otherwise it won't get picked up by python packaging.
This also fixes the problem where the "static" folder was found if
synapse wasn't started from that directory.
|
| |
|
|
| |
set in the config
|
| |
|
|
|
|
| |
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
|
| | |
|
| | |
|
| | |
|
