| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
| |
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
|
| |
|
| |
The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.
|
| | |
|
| |
|
| |
This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.
|
| | |
|
| |
|
| |
This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
|
| |
|
|
|
| |
Fixes #9778
ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
|
| |
|
| |
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| |
|
| |
I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
|
| |
|
|
|
|
| |
* Simplify `start_listening` callpath
* Correctly check the size of uploaded files
|
| | |
|
| |
|
|
|
|
| |
When receiving a /send_join request for a room with join rules set to 'restricted',
check if the user is a member of the spaces defined in the 'allow' key of the join rules.
This only applies to an experimental room version, as defined in MSC3083.
|
| | |
|
| |
|
|
| |
Every single time I want to access the config object, I have to remember
whether or not we use `get_config`. Let's just get rid of it.
|
| |
|
|
|
|
|
|
| |
room. (#9763)"
This reverts commit cc51aaaa7adb0ec2235e027b5184ebda9b660ec4.
The PR was prematurely merged and not yet approved.
|
| |
|
|
|
|
|
| |
When receiving a /send_join request for a room with join rules set to 'restricted',
check if the user is a member of the spaces defined in the 'allow' key of the join
rules.
This only applies to an experimental room version, as defined in MSC3083.
|
| |
|
| |
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| |
|
|
|
|
|
| |
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
|
| |
|
|
|
|
|
|
|
|
|
|
| |
At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though.
This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around.
A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync.
The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being:
* Sending state for a specific set or all known users to a defined set of local and remote users.
* The ability to trigger an initial sync for specific users, so they receive all current state.
|
| |
|
|
|
|
|
| |
This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited.
We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits.
Fixes #9663
|
| |
|
|
| |
Includes an abstract base class which both the FederationSender
and the FederationRemoteSendQueue must implement.
|
| | |
|
| |
|
| |
This is very bare-bones for now: federation will come soon, while pagination is descoped for now but will come later.
|
| |
|
|
| |
By splitting this to two separate methods the callers know
what methods they can expect on the handler.
|
| | |
|
| | |
|
| |
|
|
|
| |
interfaces. (#9528)
This helps fix some type hints when running with Twisted 21.2.0.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
### Changes proposed in this PR
- Add support for the `no_proxy` and `NO_PROXY` environment variables
- Internally rely on urllib's [`proxy_bypass_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2519)
- Extract env variables using urllib's `getproxies`/[`getproxies_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2488) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment
This does contain behaviour changes for consumers so making sure these are called out:
- `no_proxy`/`NO_PROXY` is now respected
- lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY`
Related to #9306 which also uses `ProxyAgent`
Signed-off-by: Timothy Leung tim95@hotmail.co.uk
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Split ShardedWorkerHandlingConfig
This is so that we have a type level understanding of when it is safe to
call `get_instance(..)` (as opposed to `should_handle(..)`).
* Remove special cases in ShardedWorkerHandlingConfig.
`ShardedWorkerHandlingConfig` tried to handle the various different ways
it was possible to configure federation senders and pushers. This led to
special cases that weren't hit during testing.
To fix this the handling of the different cases is moved from there and
`generic_worker` into the worker config class. This allows us to have
the logic in one place and allows the rest of the code to ignore the
different cases.
|
| | |
|
| |
|
|
|
| |
* Adds type hints to the groups servlet and stringutils code.
* Assert the maximum length of some input values for spec compliance.
|
| | |
|
| | |
|
| |
|
| |
This only applies if the user's data is to be erased.
|
| |
|
| |
Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
|
| |
|
|
| |
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
|
| |
|
|
| |
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:
* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events
The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
|
| |
|
|
|
| |
(#8565)
Changes `@cache_in_self` to use underscore-prefixed attributes.
|
| | |
|
| |
|
| |
De-duplicates code between the SAML and OIDC implementations.
|
| |
|
|
|
|
| |
This implements a more standard API for instantiating a homeserver and
moves some of the dependency injection into the test suite.
More concretely this stops using `setattr` on all `kwargs` passed to `HomeServer`.
|
| | |
|
| |
|
| |
All handlers now available via get_*_handler() methods on the HomeServer.
|
| |
|
|
|
| |
Lots of different module apis is not easy to maintain.
Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
|
| | |
|
| |
|
| |
c.f. #8295 for rationale
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Duplicating function signatures between server.py and server.pyi is
silly. This commit changes that by changing all `build_*` methods to
`get_*` methods and changing the `_make_dependency_method` to work work
as a descriptor that caches the produced value.
There are some changes in other files that were made to fix the typing
in server.py.
|
| | |
|
| | |
|
| |
|
| |
I found these made pycharm have more of a clue as to what was going on in other places.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Delete Room admin API allows server admins to remove rooms from server
and block these rooms.
`DELETE /_synapse/admin/v1/rooms/<room_id>`
It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API.
Fixes: #6425
It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`.
It should return `None` if the room is unknown. But it returns an `IndexError`.
https://github.com/matrix-org/synapse/blob/901b1fa561e3cc661d78aa96d59802cf2078cb0d/synapse/storage/data_stores/main/room.py#L99-L105
Related to:
- #5575
- https://github.com/Awesome-Technologies/synapse-admin/issues/17
Signed-off-by: Dirk Klimpel dirk@klimpel.org
|
| |
|
| |
... instead of duplicating `config.signing_key[0]` everywhere
|
| |
|
|
|
|
|
|
|
|
| |
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:
* Rather undocumented, and
* causing a *lot* of config checks
This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation.
Best to be reviewed commit-by-commit.
|
| |
|
|
|
|
|
| |
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).
Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.
People probably want to look at this commit by commit.
|
| |
|
| |
This is so that the logic can happen on both master and workers when we move event persistence out.
|
| | |
|
| |
|
|
|
| |
This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
|
| |
|
| |
The aim here is to move the command handling out of the TCP protocol classes and to also merge the client and server command handling (so that we can reuse them for redis protocol). This PR simply moves the client paths to the new `ReplicationCommandHandler`, a future PR will move the server paths too.
|
| |
|
|
|
|
|
|
| |
If there was an exception setting up one of the attributes of the Homeserver
god object, then future attempts to fetch that attribute would raise a
confusing "Cyclic dependency" error. Let's make sure that we clear the
`building` flag so that we just get the original exception.
Ref: #7169
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove `conn_id` usage for UserSyncCommand.
Each tcp replication connection is assigned a "conn_id", which is used
to give an ID to a remotely connected worker. In a redis world, there
will no longer be a one to one mapping between connection and instance,
so instead we need to replace such usages with an ID generated by the
remote instances and included in the replicaiton commands.
This really only effects UserSyncCommand.
* Add CLEAR_USER_SYNCS command that is sent on shutdown.
This should help with the case where a synchrotron gets restarted
gracefully, rather than rely on 5 minute timeout.
|
| |
|
|
| |
(#7136)
|
| | |
|
| |
|
| |
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
|
| |
|
|
|
|
|
| |
Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests.
Fixes #7092.
|
| | |
|
| |
|
|
|
| |
This encapsulates config for a given database and is the way to get new
connections.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Guess I only tested this on python 2 :/
Fixes #6419.
|
| |
|
| |
* remove psutil and replace with resource
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy.
The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`.
The proxy will then be used for
* push
* url previews
* phone-home stats
* recaptcha validation
* CAS auth validation
It will *not* be used for:
* Application Services
* Identity servers
* Outbound federation
* In worker configurations, connections from workers to masters
Fixes #4198.
|
| | |
|
| | |
|
| |
|
|
|
| |
This is useful to allow room admins to quickly deal with a large number
of abusive messages.
|
| | |
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| |/
|
|
| |
Also: share the saml client between redirect and response handlers.
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Split TransactionQueue up
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
| |
This reverts commit d7dbad3526136cfc9fdbd568635be5016fb637db.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
* Move RegistrationHandler init to HomeServer
* Move post registration actions to RegistrationHandler
* Add post regisration replication endpoint
* Newsfile
|
| |
|
|
|
| |
It's nothing to do with refreshing the certificates. No idea why it was here.
|
| | |
|
| |
|
|
|
| |
This is so that everything is done in one place, making it easier to
change the event format based on room version
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| | |
This gives clearer messages when someone gets it wrong
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
We should explicitly close any db connections we open, because failing to do so
can block other transactions as per
https://github.com/matrix-org/synapse/issues/3682.
Let's also try to factor out some of the boilerplate by having server classes
define their datastore class rather than duplicating the whole of `setup`.
|
| |\| |
|
| | | |
|
| | |\
| | |
| | |
| | | |
erikj/split_profiles
|
| | | | |
|
| | |/ |
|
| |/ |
|
| |\
| |
| |
| | |
erikj/client_apis_move
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
This is in preparation for moving GET /context/ to a worker
|
| |/
|
|
|
| |
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
|
| | |
|
| | |
|
| |
|
|
|
| |
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
|
| |
|
|
|
| |
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
|
| |
|
|
|
|
|
| |
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.
(This doesn't actually do much yet becuse we don't call send_notice anywhere)
|
| |
|
|
|
|
|
| |
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
|
| |
|
|
|
| |
Hopefully there are enough comments and docs in this that it makes sense on its
own.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This closes #2602
v1auth was created to account for the differences in status code between
the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid
tokens). However since those protocols were merged, this makes the r0
version/endpoint internally inconsistent, and violates the
specification for the r0 endpoint.
This might break clients that rely on this inconsistency with the
specification. This is said to affect the legacy angular reference
client. However, I feel that restoring parity with the spec is more
important. Either way, it is critical to inform developers about this
change, in case they rely on the illegal behaviour.
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| | |
|
| | |
|
| |\
| |
| | |
Split RoomMemberHandler into base and master class
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
The intention here is to split the class into the bits that can be done
on workers and the bits that have to be done on the master.
In future there will also be a class that can be run on the worker,
which will delegate work to the master when necessary.
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
|
| |
|
|
|
| |
This function is identical to all subclasses, so we may as well push it up to
the base class to reduce duplication (and make use of it in the tests)
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
|
| | |
| |
| |
| |
| |
| | |
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
|
| | |
| |
| |
| |
| |
| | |
This avoids the scenario where we have four different PreviewUrlResources
configured on a single app, each of which have their own caches and cache
clearing jobs.
|
| | |
| |
| |
| |
| | |
remove those that aren't used at all, and replace the ones that don't have
builders with simple getters rather than dynamically-generated methods.
|
| |/
|
|
|
|
|
|
|
|
| |
additional users
Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive.
It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users...
...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too.
Committing it here as a WIP
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |
|
|
|
| |
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
- This also fixes-up the RM endpoint so that it actually Works.
|
| | |
|
| |
|
|
|
|
| |
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
|
| |
|
|
| |
So I can use it from federation bits without pulling in all the handlers.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
|
| | |
|
| | |
|
| |
|
|
| |
receipts, and are on that server
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Mjark is officially no fun.
|
| |
|
|
|
| |
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
|
| | |
|
| |
|
|
| |
arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class
|
| |\
| |
| |
| |
| | |
Conflicts:
synapse/config/homeserver.py
|
| | |\ |
|
| | | |
| | |
| | |
| | | |
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
|
| | |/
|/| |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| |
| |
| | |
Conflicts:
synapse/handlers/__init__.py
synapse/storage/__init__.py
|
| | |\ |
|
| | | |\
| | | |
| | | |
| | | |
| | | |
| | | | |
Conflicts:
synapse/handlers/events.py
synapse/server.py
|
| | | |\ \
| | | | |
| | | | |
| | | | |
| | | | | |
Conflicts:
synapse/rest/__init__.py
|
| | | |\ \ \ |
|
| | | |\ \ \ \
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Conflicts:
synapse/api/errors.py
synapse/server.py
synapse/storage/__init__.py
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
stdout currently!)
|
| | | |_|_|_|/
| |/| | | |
| | | | | |
| | | | | | |
of user filters into there
|
| |/ / / / / |
|
| | | | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
HS and getting a clock from it and calling time_msec on the clock.
Remove the serialize_event method from the HS since it is no longer
needed.
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | | | | | |
|
| | |_|_|/
|/| | | |
|
| | | | | |
|
| | |_|/
|/| | |
|
| | | |
| | |
| | |
| | | |
data.
|
| | | |
| | |
| | |
| | | |
clients.
|
| | |/
|/| |
|
| | | |
|
| |\ \
| | |
| | |
| | |
| | | |
Conflicts:
setup.py
|
| | |/ |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |\
| |
| |
| |
| |
| | |
Conflicts:
synapse/storage/__init__.py
tests/rest/test_presence.py
|
| | | |
|
| | | |
|
| |/ |
|
| |\ |
|
| | | |
|
| |/ |
|
| |
|
|
| |
hasn't been incorporated in time for launch.
|
| | |
|
| | |
|
| |\
| |
| |
| |
| |
| | |
Conflicts:
synapse/handlers/room.py
synapse/storage/stream.py
|
| | |\
| | |
| | |
| | |
| | |
| | |
| | | |
Conflicts:
synapse/handlers/events.py
synapse/rest/events.py
synapse/rest/room.py
|
| | | |
| | |
| | |
| | | |
reflects the change in the underlying storage model.
|
| |\ \ \
| | |/
| |/| |
|
| | | | |
|
| |\| | |
|
| | |/
| |
| |
| | |
addition to a room alias.
|
| |/ |
|
| | |
|
| |
|
|
| |
FileUploadResource. Added stub methods.
|
| |
|
|
| |
resource_for_federation or resource_for_client depending on what is being tested.
|
| |
|
|
| |
resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree.
|
| |
|
|
| |
RoomAlias.from_sring(..., hs=hs) - similar to parse_userid()
|
| |
|
|
| |
Resource not a RestServlet.
|
