| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
| |
(#10611)
|
| | |
|
| |
|
|
|
|
|
| |
* Move /batch_send to /v2_alpha directory
As pointed out by @erikjohnston,
https://github.com/matrix-org/synapse/pull/10552#discussion_r685836624
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
(MSC2716) (#10552)
* Only return state events that the AS passed in via state_events_at_start
As discovered by @Half-Shot in
https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684158448
Part of MSC2716
* Add changelog
* Fix changelog extension
|
| |
|
|
|
| |
This adds support for the /hierarchy endpoint, which is an
update to MSC2946. Currently this only supports rooms known
locally to the homeserver.
|
| | |
|
| |
|
| |
* Mark all MSC2716 events as historical
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
scrollback history (MSC2716) (#10245)
* Make historical messages available to federated servers
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
Follow-up to https://github.com/matrix-org/synapse/pull/9247
* Debug message not available on federation
* Add base starting insertion point when no chunk ID is provided
* Fix messages from multiple senders in historical chunk
Follow-up to https://github.com/matrix-org/synapse/pull/9247
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
---
Previously, Synapse would throw a 403,
`Cannot force another user to join.`,
because we were trying to use `?user_id` from a single virtual user
which did not match with messages from other users in the chunk.
* Remove debug lines
* Messing with selecting insertion event extremeties
* Move db schema change to new version
* Add more better comments
* Make a fake requester with just what we need
See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080
* Store insertion events in table
* Make base insertion event float off on its own
See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889
Conflicts:
synapse/rest/client/v1/room.py
* Validate that the app service can actually control the given user
See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455
Conflicts:
synapse/rest/client/v1/room.py
* Add some better comments on what we're trying to check for
* Continue debugging
* Share validation logic
* Add inserted historical messages to /backfill response
* Remove debug sql queries
* Some marker event implemntation trials
* Clean up PR
* Rename insertion_event_id to just event_id
* Add some better sql comments
* More accurate description
* Add changelog
* Make it clear what MSC the change is part of
* Add more detail on which insertion event came through
* Address review and improve sql queries
* Only use event_id as unique constraint
* Fix test case where insertion event is already in the normal DAG
* Remove debug changes
* Switch to chunk events so we can auth via power_levels
Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.
So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.
* Switch to chunk events for federation
* Add unstable room version to support new historical PL
* Fix federated events being rejected for no state_groups
Add fix from https://github.com/matrix-org/synapse/pull/10439
until it merges.
* Only connect base insertion event to prev_event_ids
Per discussion with @erikjohnston,
https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org
* Make it possible to get the room_version with txn
* Allow but ignore historical events in unsupported room version
See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489
We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either.
Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` -> `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`.
* Move to unique index syntax
See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509
* High-level document how the insertion->chunk lookup works
* Remove create_event fallback for room_versions
See https://github.com/matrix-org/synapse/pull/10245/files#r677641879
* Use updated method name
|
| |
|
|
|
|
|
|
|
| |
Improves type hints for:
* parse_{boolean,integer}
* parse_{boolean,integer}_from_args
* parse_json_{value,object}_from_request
And fixes any incorrect calls that resulted from unknown types.
|
| |
|
|
| |
And set the required attribute in a few places which will error if
a parameter is not provided.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.
So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.
|
| | |
|
| |
|
|
|
|
|
| |
Fix messages from multiple senders in historical chunk. This also means that an app service does not need to define `?user_id` when using this endpoint.
Follow-up to https://github.com/matrix-org/synapse/pull/9247
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
|
| |
|
|
|
|
|
|
|
| |
(#10250)
* Add base starting insertion point when no chunk ID is provided
This is so we can have the marker event point to this initial
insertion event and be able to traverse the events in the first chunk.
|
| |
|
| |
Work on https://github.com/matrix-org/matrix-doc/pull/2716
|
| |
|
|
|
|
| |
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403
Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
|
| | |
|
| |
|
|
|
|
| |
[MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432) added this endpoint originally but it has since been included in the spec for nearly a year.
This is progress towards https://github.com/matrix-org/synapse/issues/8334
|
| | |
|
| |
|
|
| |
In lieu of just always enabling the unstable spaces endpoint and
unstable room version.
|
| |
|
|
|
|
|
|
|
|
|
| |
Per changes in MSC2946, the C-S and S-S APIs for spaces summary
should use GET requests.
Until this is stable, the POST endpoints still exist.
This does not switch federation requests to use the GET version yet
since it is newly added and already deployed servers might not support
it. When switching to the stable endpoint we should switch to GET
requests.
|
| |
|
|
|
|
|
| |
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
|
| | |
|
| | |
|
| | |
|
| |
|
| |
This is very bare-bones for now: federation will come soon, while pagination is descoped for now but will come later.
|
| |
|
|
| |
By splitting this to two separate methods the callers know
what methods they can expect on the handler.
|
| |
|
| |
There's no need to do aggregation bundling for state events. Doing so can cause performance issues.
|
| |
|
|
|
|
|
| |
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md)
- Update `code_style.md` docs around installing black to use the correct version
|
| | |
|
| | |
|
| |
|
|
| |
If a remote server name is provided, ensure it is something reasonable
before making remote connections to it.
|
| |
|
| |
Adds the redacts endpoint to workers that have the client listener.
|
| | |
|
| |
|
|
| |
The handling of OPTIONS requests was consolidated in #7534, but the endpoint
specific handlers were not removed.
|
| |
|
| |
All handlers now available via get_*_handler() methods on the HomeServer.
|
| |
|
| |
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
|
| |
|
|
|
|
|
| |
This converts calls like super(Foo, self) -> super().
Generated with:
sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
|
| | |
|
| |
|
|
|
|
| |
Small cleanup PR.
* Removed the unused `is_guest` argument
* Added a safeguard to a (currently) impossible code path, fixing static checking at the same time.
|
| | |
|
| | |
|
| | |
|
| |
|
| |
Signed-off-by: Michael Albert <michael.albert@awesome-technologies.de>
|
| | |
|
| |
|
|
| |
json. (#7836)
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).
Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.
People probably want to look at this commit by commit.
|
| |
|
|
| |
public rooms list (#6899)
|
| | |
|
| |
|
| |
it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.
|
| |
|
|
|
| |
per matrix-org/matrix-doc#2432
|
| |
|
|
|
| |
as per MSC2260
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Use room_stats and room_state for room directory search
|
| | |
|
| |
|
|
| |
This will make it easier to search for sending event requests.
|
| |
|
|
|
|
|
| |
This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted.
Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140).
Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.
|
| |
|
|
|
|
|
|
|
|
| |
Remove all the "double return" statements which were a result of us removing all the instances of
```
defer.returnValue(...)
return
```
statements when we switched to python3 fully.
|
| |
|
|
|
| |
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
|
| |
|
|
|
| |
Part of fixing matrix-org/sytest#652
Sytest PR: matrix-org/sytest#667
|
| |\
| |
| |
| |
| |
| |
| | |
Bugfixes
--------
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Fix servlet metric names
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove redundant check
* Cover all return paths
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
First of all, let's get rid of `TOKEN_NOT_FOUND_HTTP_STATUS`. It was a hack we
did at one point when it was possible to return either a 403 or a 401 if the
creds were missing. We always return a 401 in these cases now (thankfully), so
it's not needed.
Let's also stop abusing `AuthError` for these cases. Honestly they have nothing
that relates them to the other places that `AuthError` is used, other than the
fact that they are loosely under the 'Auth' banner. It makes no sense for them
to share exception classes.
Instead, let's add a couple of new exception classes: `InvalidClientTokenError`
and `MissingClientTokenError`, for the `M_UNKNOWN_TOKEN` and `M_MISSING_TOKEN`
cases respectively - and an `InvalidClientCredentialsError` base class for the
two of them.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
This fixes a bug which were causing the "event_format" field to be
ignored in the filter of requests to the `/messages` endpoint of the
CS API.
Signed-off-by: Eisha Chen-yen-su <chenyensu0@gmail.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CS API (#5083)
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
|
| | |
|
| |
|
|
|
|
|
|
| |
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.
Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
|
| | |
| |
| | |
make /context lazyload & filter aware.
|
| |\ \
| | |
| | |
| | | |
erikj/client_apis_move
|
| | |/ |
|
| | |
| |
| |
| | |
This is in preparation for moving GET /context/ to a worker
|
| |/
|
|
|
| |
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
|
| | |
|
| |
|
|
|
|
| |
the method "assert_params_in_request" does handle dicts and not
requests. A request body has to be parsed to json before this method
can be used
|
| |
|
|
|
|
|
| |
parse_integer and parse_string can take a request and raise errors
in case we have wrong or missing params.
This PR tries to use them more to deduplicate some code and make it
better readable
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
|
| |
|
|
|
|
| |
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| | |
|
| |\
| |
| | |
Add room_id to the response of `rooms/{roomId}/join`
|
| | |
| |
| |
| | |
Fixes #2349
|
| |\ \
| |/
|/| |
|
| | | |
|
| | |
| |
| |
| |
| | |
Due to the order we register the REST handlers `/forget` was handled by
the correct handler.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| | |
blindly implement ?ts for AS. untested
|
| |/
|
|
|
|
| |
Turns out that there is a valid usecase for retrieving event by id (notably
having received a push), but event ids should be scoped to room, so /event/{id}
is wrong.
|
| |
|
|
| |
what could possibly go wrong
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.
We also try converting the string to ascii to further reduce the size.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We somehow specced APIs with reason strings, preserve the content
in the events and even have the clients display them, but failed
to actually pass the parameter through to the event content.
|
| | |
|
| |\
| |
| | |
Add new API appservice specific public room list
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a race whereby:
- User hits an endpoint.
- No cached transaction so executes main code.
- User hits same endpoint.
- No cache transaction so executes main code.
- Main code finishes executing and caches response and returns.
- Main code finishes executing and caches response and returns.
This race is common in the wild when Synapse is struggling under load.
This commit fixes the race by:
- User hits an endpoint.
- Caches the promise to execute the main code and executes main code.
- User hits same endpoint.
- Yields on the same promise as the first request.
- Main code finishes executing and returns, unblocking both requests.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.
If the user specifies keys like "avatar_url" then they will be clobbered.
|
| |
|
|
|
|
| |
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
So I can use it from federation bits without pulling in all the handlers.
|
| |
|
|
| |
Multiple server_names are supported via ?server_name=foo&server_name=bar
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
This dramatically simplifies the forget API code - in particular it no
longer generates a leave event.
|
| | |
|
| |
|
|
|
| |
to deduplicate all the copy+pasted _parse_json functions. Also document
the parse_.* functions.
|
| | |
|
| |
|
|
| |
This will enable more detailed decisions
|
| | |
|
| | |
|
| |\
| |
| | |
Rewrite presence for performance.
|
| | | |
|
| |\|
| |
| |
| |
| | |
Conflicts:
synapse/rest/client/v1/room.py
|
| | |
| |
| |
| |
| |
| | |
accordance with the spec
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
|
| | | |
|
| | |
| |
| |
| |
| | |
I'm not particularly happy with the "action" switching, but there's no
convenient way to defer the work that needs to happen after it, so... :(
|
| | |
| |
| |
| | |
Unclean, needs tidy-up, but works
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
This code is kind of rough (passing the remote servers down a long
chain), but is a step towards improvement.
|
| |/
|
|
|
|
| |
Force joining by alias to go through the send_membership_event checks,
rather than bypassing them straight into _do_join. This is the first of
many stages of cleanup.
|
| |
|
|
|
|
| |
This reverts commit cf81375b94c4763766440471e632fc4b103450ab.
It subtly violates a guest joining auth check
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
There's at least one more to merge in.
Side-effects:
* Stop reporting None as displayname and avatar_url in some cases
* Joining a room by alias populates guest-ness in join event
* Remove unspec'd PUT version of /join/<room_id_or_alias> which has not
been called on matrix.org according to logs
* Stop recording access_token_id on /join/room_id - currently we don't
record it on /join/room_alias; I can try to thread it through at some
point.
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| | |
... because the context isn't much use without the event.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This tracks data about the entity which made the request. This is
instead of passing around a tuple, which requires call-site
modifications every time a new piece of optional context is passed
around.
I tried to introduce a User object. I gave up.
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| |
| | |
Merge pull request #360 from matrix-org/daniel/guestroominitialsync
Allow guest access to room initialSync
|
| | | |
|
| |/ |
|
| | |
|
| |\
| |
| | |
Get display name from identity server, not client
|
| | | |
|
| |/
|
|
| |
... merely because I was trying to figure out how it worked, and couldn't.
|
| |\
| |
| | |
Implement pagination, order by and groups in search
|
| | |\ |
|
| | | | |
|
| | |/
|/| |
|
| | |
| |
| |
| | |
Squash-merge of PR #345 from daniel/anonymousevents
|
| |/
|
|
|
|
|
| |
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
| |
Client-Server API
|
| |
|
|
| |
they left
|
| |
|
|
|
|
|
|
|
|
| |
Removes device_id and ClientInfo
device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.
This doesn't remove the columns from the database, because that's
fiddly.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
device_id in the internal meta data for the event along with the transaction id when sending events
|
| | |
|
| |
|
|
|
|
| |
HS and getting a clock from it and calling time_msec on the clock.
Remove the serialize_event method from the HS since it is no longer
needed.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
