| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
| |
I've never found this terribly useful. I think it was added in the early days
of Synapse, without much thought as to what would actually be useful to log,
and has just been cargo-culted ever since.
Rather, it tends to clutter up debug logs with useless information.
|
| |\ |
|
| | | |
|
| | |
| |
| |
| | |
filter appservice users. (#11675)
|
| | |
| |
| | |
Fixes #11741
|
| | |
| |
| | |
... and a minor thinko fix in the sample config.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The existing implementation of the `python_twisted_reactor_tick_time` metric is pretty useless, because it *only*
measures the time taken to execute timed calls and callbacks from threads. That neglects everything that
happens off the back of I/O, which is obviously quite a lot for us.
To improve this, I've hooked into a different place in the reactor - in particular, where it calls `epoll`. That call is
the only place it should wait for something to happen - the rest of the loop *should* be quick.
I've also removed `python_twisted_reactor_pending_calls`, because I don't believe anyone ever looks at it, and
it's a nuisance to populate.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Always add state.room_id after the configurable ORDER BY. Otherwise,
for any sort, certain pages can contain results from
other pages. (Especially when sorting by creator, since there may
be many rooms by the same creator)
* Document different order direction of numerical fields
"joined_members", "joined_local_members", "version" and "state_events"
are ordered in descending direction by default (dir=f). Added a note
in tests to explain the differences in ordering.
Signed-off-by: Daniël Sonck <daniel@sonck.nl>
|
| |\| |
|
| | | |
|
| | |
| |
| |
| |
| | |
fail because too many EDUs were produced for device updates. (#11730)
Co-authored-by: David Robertson <davidr@element.io>
|
| | |
| |
| |
| |
| | |
be sent to remote homeservers if there were too many to send at once. (#11729)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
| | |
| |
| |
| | |
dictionary (#11576)
|
| | |
| |
| |
| |
| |
| |
| | |
* Fix sample_config.yaml in regards track_puppeted_user_ips
Closes #11741
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | |
| |
| |
| | |
This should be (slightly) more efficient and it is simpler
to have a single method for inserting multiple values.
|
| | |
| |
| |
| |
| | |
documentation claims that you can use the %(app)s variable in password_reset and email_validation subjects, but if you do you end up with an error 500
Co-authored-by: br4nnigan <10244835+br4nnigan@users.noreply.github.com>
|
| | |
| |
| |
| |
| |
| |
| | |
* change spec url in config files
* Create 11739.txt
* .txt -> .doc
|
| | | |
|
| | |
| |
| |
| |
| | |
Rather than hooking into the reactor loop, just add a timed task that runs every 100 ms to do the garbage collection.
Part 1 of a quest to simplify the reactor monkey-patching.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently when puppeting another user, the user doing the puppeting is
tracked for client IPs and MAU (if configured).
When tracking MAU is important, it becomes necessary to be possible to
also track the client IPs and MAU of puppeted users. As an example a
client that manages user creation and creation of tokens via the Synapse
admin API, passing those tokens for the client to use.
This PR adds optional configuration to enable tracking of puppeted users
into monthly active users. The default behaviour stays the same.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
By returning all of the m.space.child state of the space, not just
the first 50. The number of rooms returned is still capped at 50.
For the federation API this implies that the requesting server will
need to individually query for any other rooms it is not joined to.
|
| |\| |
|
| | |
| |
| |
| | |
`glob_to_regex` in `synapse.util`, where it moved from. (#11696)
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.
Overall this should cause no change in behavior, but is prep work
for other improvements.
|
| | |
| |
| |
| | |
This was removed from MSC2674 before that was approved
and is not used by any known clients.
|
| | |
| |
| |
| | |
On Python 2, indexing a byte-string gives back a byte-string,
while on Python 3 it gives back the ASCII equivalent as an int.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
federation (#11530)
* add some tests to verify we are stripping unauthorized fields out of unsigned
* add function to strip unauthorized fields from the unsigned object of event
* newsfragment
* update newsfragment number
* add check to on_send_membership_event
* refactor tests
* fix lint error
* slightly refactor tests and add some comments
* slight refactor
* refactor tests
* fix import error
* slight refactor
* remove unsigned filtration code from synapse/handlers/federation_event.py
* lint
* move unsigned filtering code to event base
* refactor tests
* update newsfragment
* requested changes
* remove unused retun values
|
| | |
| |
| |
| |
| | |
error to be returned, not `M_UNKNOWN` (#11672)
Co-authored-by: reivilibre <olivier@librepush.net>
|
| | |
| |
| |
| | |
as per
https://github.com/matrix-org/synapse/pull/11532#discussion_r769123269
|
| | |
| |
| |
| |
| |
| |
| | |
Fixes minor discrepancies between the /hierarchy endpoint described
in MSC2946 and the implementation.
Note that the changes impact the stable and unstable /hierarchy and
unstable /spaces endpoints for both client and federation APIs.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
* newsfragment
* fix newsfragment number
* update changelog
* remove extra space
|
| |/
|
| |
If we've never made a request to a remote homeserver, we should cache the response---even if the response is "this user has no devices".
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* `_auth_and_persist_outliers`: mark persisted events as outliers
Mark any events that get persisted via `_auth_and_persist_outliers` as, well,
outliers.
Currently this will be a no-op as everything will already be flagged as an
outlier, but I'm going to change that.
* `process_remote_join`: stop flagging as outlier
The events are now flagged as outliers later on, by `_auth_and_persist_outliers`.
* `send_join`: remove `outlier=True`
The events created here are returned in the result of `send_join` to
`FederationHandler.do_invite_join`. From there they are passed into
`FederationEventHandler.process_remote_join`, which passes them to
`_auth_and_persist_outliers`... which sets the `outlier` flag.
* `get_event_auth`: remove `outlier=True`
stop flagging the events returned by `get_event_auth` as outliers. This method
is only called by `_get_remote_auth_chain_for_event`, which passes the results
into `_auth_and_persist_outliers`, which will flag them as outliers.
* `_get_remote_auth_chain_for_event`: remove `outlier=True`
we pass all the events into `_auth_and_persist_outliers`, which will now flag
the events as outliers.
* `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter
This param is now never set to True, so we can remove it.
* `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param
This is no longer set anywhere, so we can remove it.
* `get_pdu`: remove unused `outlier` parameter
... and chase it down into `get_pdu_from_destination_raw`.
* `event_from_pdu_json`: remove redundant `outlier` param
This is never set to `True`, so can be removed.
* changelog
* update docstring
|
| |
|
|
| |
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| |
|
| |
Co-authored-by: reivilibre <olivier@librepush.net>
|
| |
|
| |
Co-authored-by: reivilibre <olivier@librepush.net>
|
| |
|
|
|
| |
#11505 (#11687)
Co-authored-by: Sean Quah <seanq@element.io>
|
| |
|
|
|
|
|
|
|
|
|
| |
* Fix AssertionErrors after purging events
If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.
This fixes the situation by rewinding the stream processors.
* `check-newsfragment`: ignore deleted newsfiles
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Events returned by `backfill` should not be flagged as outliers.
Fixes:
```
AssertionError: null
File "synapse/handlers/federation.py", line 313, in try_backfill
dom, room_id, limit=100, extremities=extremities
File "synapse/handlers/federation_event.py", line 517, in backfill
await self._process_pulled_events(dest, events, backfilled=True)
File "synapse/handlers/federation_event.py", line 642, in _process_pulled_events
await self._process_pulled_event(origin, ev, backfilled=backfilled)
File "synapse/handlers/federation_event.py", line 669, in _process_pulled_event
assert not event.internal_metadata.is_outlier()
```
See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/231992
Fixes #8894.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Push `get_room_{min,max_stream_ordering}` into StreamStore
Both implementations of this are identical, so we may as well push it down and
get rid of the abstract base class nonsense.
* Remove redundant `StreamStore` class
This is empty now
* Remove redundant `get_current_events_token`
This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid
of it.
* newsfile
|
| |
|
|
| |
"Unknown room" can mean a multitude of things here. To help with debugging, add
some more words to the exception text.
|
| |
|
|
| |
Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.
|
| |
|
| |
To improve type hints throughout the code.
|
| | |
|
| |
|
| |
By using cast and making ignores more specific.
|
| |
|
|
| |
Both of those APIs return state events, which will not have bundled
aggregations added anyway.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* update black version
* run updated version of black on code
* newsfragment
* enumerate python versions
|
| | |
|
| |\ |
|
| | | |
|
| | |
| |
| |
| | |
... to work around breakage on buster
(https://github.com/Marco-Sulla/python-frozendict/issues/41)
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Wrap `auth.get_user_by_req` in an opentracing span
give `get_user_by_req` its own opentracing span, since it can result in a
non-trivial number of sub-spans which it is useful to group together.
This requires a bit of reorganisation because it also sets some tags (and may
force tracing) on the servlet span.
* Emit opentracing span for encoding json responses
This can be a significant time sink.
* Rename all sync spans with a prefix
* Write an opentracing span for encoding sync response
* opentracing span to group generate_room_entries
* opentracing spans within sync.encode_response
* changelog
* Use the `trace` decorator instead of context managers
|
| | |
| |
| |
| |
| | |
And make bundling aggregations opt-in, instead of opt-out to avoid
having APIs to include extraneous data (and being much heavier than
necessary).
|
| | |
| |
| | |
This adds some opentracing annotations to ResponseCache, to make it easier to see what's going on; in particular, it adds a link back to the initial trace which is actually doing the work of generating the response.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* remove `start_active_span_from_request`
Instead, pull out a separate function, `span_context_from_request`, to extract
the parent span, which we can then pass into `start_active_span` as
normal. This seems to be clearer all round.
* Remove redundant tags from `incoming-federation-request`
These are all wrapped up inside a parent span generated in AsyncResource, so
there's no point duplicating all the tags that are set there.
* Leave request spans open until the request completes
It may take some time for the response to be encoded into JSON, and that JSON
to be streamed back to the client, and really we want that inside the top-level
span, so let's hand responsibility for closure to the SynapseRequest.
* opentracing logs for HTTP request events
* changelog
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Disable aggregation bundling on `/sync` responses
A partial revert of #11478. This turns out to have had a significant CPU impact
on initial-sync handling. For now, let's disable it, until we find a more
efficient way of achieving this.
* Fix tests.
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
|
| | |
| |
| |
| | |
Instead of returning 404 errors with HTML bodies when an unknown
prefix was requested (e.g. /matrix/client/v1 before Synapse v1.49.0).
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
As suggested in https://github.com/matrix-org/matrix-react-sdk/pull/7372#discussion_r769523369
|
| | | |
|
| | |
| |
| |
| | |
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
|
| | | |
|
| | |
| |
| |
| | |
masquerade as specific devices. (#11538)
|
| | | |
|
| | | |
|
| | | |
|
| |\| |
|
| | | |
|
| | |
| |
| |
| |
| | |
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
|
| | | |
|
| | |
| |
| |
| |
| | |
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
|
| | |
| |
| |
| |
| |
| |
| | |
* Splits the logic for parsing HTML from the resource handling code.
* Fix a circular import in the oEmbed code (which uses the HTML parsing code).
* Renames some of the HTML parsing methods to:
* Make it clear which methods are "internal" to the module.
* Clarify what the methods do.
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
raising an exception. (#11565)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
| | |
| |
| |
| |
| | |
The event still needs to have `auth_events` defined to be valid.
Split out from https://github.com/matrix-org/synapse/pull/11114
|
| | |
| |
| | |
C.f. https://github.com/matrix-org/synapse/pull/11494#pullrequestreview-827780886
|
| | |
| |
| |
| |
| |
| | |
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
|
| | |
| |
| | |
this field is never read, so we may as well stop populating it.
|
| | |
| |
| |
| | |
This should fix pushers with an email in non-canonical form is used as
the pushkey.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| | |
And expand some type hints in the receipts storage module.
|
| | | |
|
| | |
| |
| |
| | |
Somehow I'd managed to get my database in a pickle with stream ids. These
changes were useful to debug.
|
| |/ |
|
| | |
|
| |
|
|
|
|
| |
`matrix-python-common` (#11505) (#11527)
This reverts commit a77c36989785c0d5565ab9a1169f4f88e512ce8a.
|
| |
|
|
|
|
|
|
| |
* Move sync_token up to the top
* Pull out _get_ignored_users
* Try to signpost the body of `_generate_sync_entry_for_rooms`
* Pull out _calculate_user_changes
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
| |
Fixes #11517
|
| |
|
|
|
|
| |
After #10847, `looping_background_call` would print an error in the logs
every time a non-async function was called. Since the error would be
caught and ignored immediately, there were no other side effects.
|
| |
|
|
|
| |
If, for some reason, we don't have the create event, we should still be able to
purge a room.
|
| | |
|
| |
|
|
| |
into the Matrix specification. (#11435)
|
| |
|
| |
As a step towards allowing back-channel logout for OIDC.
|
| | |
|
| |
|
|
|
|
|
|
| |
Due to updates to MSC2675 this includes a few fixes:
* Include bundled aggregations for /sync.
* Do not include bundled aggregations for /initialSync and /events.
* Do not bundle aggregations for state events.
* Clarifies comments and variable names.
|
| | |
|
| |
|
|
| |
to refreshable access tokens. (#11445)
|
| |
|
|
|
| |
We're going to add a `state_key` column to the `events` table, so we need to
add some disambiguation to queries which use it.
|
| |
|
|
|
| |
This mainly consists of docstrings and inline comments. There are one or two type annotations and variable renames thrown in while I was here.
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
| |
This skips a few methods which are difficult to type.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
closest event to a given timestamp (#9445)
MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030
Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.
```
GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
"event_id": ...
"origin_server_ts": ...
}
```
Federation API endpoint:
```
GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
"event_id": ...
"origin_server_ts": ...
}
```
Co-authored-by: Erik Johnston <erik@matrix.org>
|
| | |
|
| |
|
|
| |
This adds some misc. type hints to helper methods used
in the `synapse.config` module.
|
| |
|
| |
As specified for Matrix v1.1.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add check to catch syanpse master process starting when workers are configured
* add test to verify that starting master process with worker config raises error
* newsfragment
* specify config.worker.worker_app in check
* update test
* report specific config option that triggered the error
Co-authored-by: reivilibre <oliverw@matrix.org>
* clarify error message
Co-authored-by: reivilibre <oliverw@matrix.org>
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| |
|
| |
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| |
|
|
|
| |
Per updates to MSC2675 which now states that bundled
aggregations should be included from the `/relations`
endpoint.
|
| |
|
|
|
|
|
|
|
|
|
| |
When all entries in an `LruCache` have a size of 0 according to the
provided `size_callback`, and `drop_from_cache` is called on a cache
node, the node would be unlinked from the LRU linked list but remain in
the cache dictionary. An assertion would be later be tripped due to the
inconsistency.
Avoid unintentionally calling `__len__` and use a strict `is None`
check instead when unwrapping the weak reference.
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| | |
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
(`_persist_events_and_state_updates`) (#11417)
Part of https://github.com/matrix-org/synapse/issues/11300
Call stack:
- `_persist_events_and_state_updates` (added `use_negative_stream_ordering`)
- `_persist_events_txn`
- `_update_room_depths_txn` (added `update_room_forward_stream_ordering`)
- `_update_metadata_tables_txn`
- `_store_room_members_txn` (added `inhibit_local_membership_updates`)
Using keyword-only arguments (`*`) to reduce the mistakes from `backfilled` being left as a positional argument somewhere and being interpreted wrong by our new arguments.
|
| | |
| |
| |
| |
| |
| | |
This also makes additional updates where the implementation
had drifted from the approved MSC.
Unstable endpoints will be removed at a later data.
|
| | |
| |
| | |
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
| | |
| |
| |
| |
| | |
This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.
|
| | |
| |
| |
| | |
tokens. (#11428)
|
| | |
| |
| |
| |
| | |
If we tried to request multiple keys for the same server, we would end up
dropping some of those requests.
|
| | |
| |
| |
| | |
Signed-off-by: Tulir Asokan <tulir@beeper.com>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since e81fa9264873369653171157514ff68226491fff, Synapse depends on
the use_float flag which has been introduced in ijson 3.1 and
is not available in 3.0. This is known to cause runtime errors
with send_join.
Signed-off-by: Daniel Molkentin <danimo@infra.run>
Co-authored-by: Daniel Molkentin <danimo@infra.run>
|
| | |
| |
| |
| | |
accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. (#11430)
|
| | |
| |
| |
| | |
Also refactor the stream ID trackers/generators a bit and try to
document them better.
|
| | |
| |
| |
| | |
refresh tokens are in use. (#11425)
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The previous fix for the ongoing event fetches counter
(8eec25a1d9d656905db18a2c62a5552e63db2667) was both insufficient and
incorrect.
When the database is unreachable, `_do_fetch` never gets run and so
`_event_fetch_ongoing` is never decremented.
The previous fix also moved the `_event_fetch_ongoing` decrement outside
of the `_event_fetch_lock` which allowed race conditions to corrupt the
counter.
|
| |/
|
|
|
|
|
|
|
|
| |
This change makes mypy complain if the constants are ever reassigned,
and, more usefully, makes mypy type them as `Literal`s instead of `str`s,
allowing code of the following form to pass mypy:
```py
def do_something(membership: Literal["join", "leave"], ...): ...
do_something(Membership.JOIN, ...)
```
|
| |
|
|
|
| |
(#11421)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| |
|
| |
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| |
|
|
| |
`refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. (#11388)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
'trust_identity_servers_for_password_resets' (#11395)
* remove background update code related to deprecated config flag
* changelog entry
* update changelog
* Delete 11394.removal
Duplicate, wrong number
* add no-op background update and change newfragment so it will be consolidated with associated work
* remove unused code
* Remove code associated with deprecated flag from legacy docker dynamic config file
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| |\ |
|
| | | |
|
| | |
| |
| |
| |
| | |
Also tighten validation of server names by forbidding invalid characters
in IPv6 addresses and empty domain labels.
|
| | | |
|
| | |
| |
| |
| |
| | |
Allows specifying a different claim (from the default "sub") to use
when calculating the localpart of the Matrix ID used during the
JWT login.
|
| | |
| |
| |
| |
| | |
Instead of only known relation types. This also reworks the background
update for thread relations to crawl events and search for any relation
type, not just threaded relations.
|
| | |
| |
| |
| |
| |
| | |
Instead of having admins poke into the database directly.
Can currently run jobs to populate stats and to populate
the user directory.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
If `room_list_publication_rules` was configured with a rule with a
non-wildcard alias and a room was created with an alias then an
internal server error would have been thrown.
This fixes the error and properly applies the publication rules
during room creation.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
Fixes a bug introduced in #11129: objects signed by the local server, but with
keys other than the current one, could not be successfully verified.
We need to check the key id in the signature, and track down the right key.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
`/transactions` (MSC2716) (#11265)
Mark historical state from the MSC2716 `/batch_send` endpoint as `historical` which makes it `backfilled` and have a negative `stream_ordering` so it doesn't get queried by `/transactions`.
Fix https://github.com/matrix-org/synapse/issues/11241
Complement tests: https://github.com/matrix-org/complement/pull/221
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`trust_identity_server_for_password_resets` config flag (#11333)
* remove code legacy code related to deprecated config flag "trust_identity_server_for_password_resets" from synapse/config/emailconfig.py
* remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/config/registration.py
* remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/handlers/identity.py
* add tests to ensure config error is thrown and synapse refuses to start when depreciated config flag is found
* add changelog
* slightly change behavior to only check for deprecated flag if set to 'true'
* Update changelog.d/11333.misc
Co-authored-by: reivilibre <oliverw@matrix.org>
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
(#11370)
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Adds validation to the Client-Server API to ensure that
the potential thread head does not relate to another event
already. This results in not allowing a thread to "fork" into
other threads.
If the target event is unknown for some reason (maybe it isn't
visible to your homeserver), but is the target of other events
it is assumed that the thread can be created from it. Otherwise,
it is rejected as an unknown event.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Otherwise I get this beautiful stacktrace:
```
python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/synapse/synapse/app/homeserver.py", line 455, in <module>
main()
File "/root/synapse/synapse/app/homeserver.py", line 445, in main
hs = setup(sys.argv[1:])
File "/root/synapse/synapse/app/homeserver.py", line 345, in setup
config = HomeServerConfig.load_or_generate_config(
File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config
config_dict = read_config_files(config_files)
File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files
yaml_config = yaml.safe_load(file_stream)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
return load(stream, SafeLoader)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
return loader.get_single_data()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
node = self.get_single_node()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
node = self.compose_sequence_node(anchor)
File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node
while not self.check_event(SequenceEndEvent):
File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry
return self.parse_block_sequence_entry()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
if not self.check_token(BlockEntryToken, BlockEndToken):
File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token
self.fetch_more_tokens()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens
return self.fetch_alias()
File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias
self.tokens.append(self.scan_anchor(AliasToken))
File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor
raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
in "/etc/matrix/homeserver.yaml", line 614, column 5
expected alphabetic or numeric character, but found '.'
in "/etc/matrix/homeserver.yaml", line 614, column 6
```
Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Add support for `/_matrix/media/v3` APIs
Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Update `workers.md` to use v3 client and media APIs
Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| |\| |
|
| | | |
|
| | |
| |
| |
| | |
`create_access_token_for_user_id` (#11369)
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.47.0rc3 (2021-11-16)
==============================
Bugfixes
--------
- Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. ([\#11346](https://github.com/matrix-org/synapse/issues/11346))
- Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from `device_inbox` column' background process from running when updating from a recent Synapse version. ([\#11303](https://github.com/matrix-org/synapse/issues/11303), [\#11353](https://github.com/matrix-org/synapse/issues/11353))
|
| | | |
|
| | |
| |
| |
| |
| | |
(#11353)
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| | |
| |
| | |
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | | |
|
| | |
| |
| |
| |
| | |
Different users may be allowed to see different rooms within a space,
so sharing responses between users is inadvisable.
|
| | |
| |
| |
| |
| | |
This is one of the changes required to support Matrix 1.1
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | | |
|
| | |
| |
| |
| |
| | |
providers (#11340)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
* Add support for the stable version of MSC2778
Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Expect m.login.application_service in login and password provider tests
Signed-off-by: Tulir Asokan <tulir@maunium.net>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
(#11280)
* remove unused tables room_stats_historical and user_stats_historical
* update changelog number
* Bump schema compat version comment
* make linter happy
* Update comment to give more info
Co-authored-by: reivilibre <oliverw@matrix.org>
Co-authored-by: reivilibre <oliverw@matrix.org>
|
| | |
| |
| |
| |
| | |
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Prefer `HTTPStatus` over plain `int`
This is an Opinion that no-one has seemed to object to yet.
* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
storing in DB (#11230)
* change display names/avatar URLS to None if they contain null bytes
* add changelog
* add POC test, requested changes
* add a saner test and remove old one
* update test to verify that display name has been changed to None
* make test less fragile
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore
the former implicitly inherits from the latter, so they should be
ordered like this when used.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Annotate HomeserverTestCase.servlets
* Correct annotation of federation_auth_origin
* Use AnyStr custom_headers instead of a Union
This allows (str, str) and (bytes, bytes).
This disallows (str, bytes) and (bytes, str)
* DomainSpecificString.SIGIL is a ClassVar
|
| | | |
|
| | |
| |
| | |
Signed-off-by: Dirk Klimpel dirk@klimpel.org
|
| | |
| |
| |
| | |
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
|
| | | |
|
| | |
| |
| | |
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| | | |
|
| | | |
|
| | |
| |
| | |
Signed-off-by: Jonas Zeunert <jonas@zeunert.org>
|
| | | |
|
| | | |
|
| | |
| |
| | |
as of #11012, these parameters are unused.
|
| |\|
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.47.0rc2 (2021-11-10)
==============================
This fixes an issue with publishing the Debian packages for 1.47.0rc1.
It is otherwise identical to 1.47.0rc1.
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
Context: https://github.com/matrix-org/synapse/pull/11114/files#r741643968
|
| | |
| |
| | |
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| | |
| |
| |
| | |
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| | |
| |
| |
| | |
Adds experimental support for `relation_types` and `relation_senders`
fields for filters.
|
| |/ |
|
| |
|
|
|
|
|
| |
time (#11234)
When fetching remote devices for the first time, we did not correctly include the cross signing keys in the returned results.
c.f. #11159
|
| |
|
| |
Co-authored-by: reivilibre <olivier@librepush.net>
|
| |
|
|
|
| |
`get_insertion_event_by_batch_id` -> `get_insertion_event_id_by_batch_id`
Split out from https://github.com/matrix-org/synapse/pull/11114
|
| |
|
| |
Fixes #11259
|
| | |
|
| |
|
|
|
| |
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Make lock better handle process being killed
If the process gets killed and restarted (so that it didn't have a
chance to drop its locks gracefully) then there may still be locks in
the DB that are for the same instance that haven't yet timed out but are
safe to delete.
We handle this case by a) checking if the current instance already has
taken out the lock, and b) if not then ignoring locks that are for the
same instance.
* Periodically check for old staged events
This is to protect against other instances dying and their locks timing
out.
|
| |
|
| |
Fixes #11252
|
| | |
|
| |
|
|
|
|
| |
When an event fetcher aborts due to an exception, `_event_fetch_ongoing`
must be decremented, otherwise the event fetcher would never be
replaced. If enough event fetchers were to fail, no more events would be
fetched and requests would get stuck waiting for events.
|
| |
|
|
|
| |
(#11207)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| |
|
|
| |
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
|
| |
|
|
|
|
|
|
| |
endpoint (#11188)
As brought up by @tulir, https://matrix.to/#/!SBYNQlpqkwJzFIdzxI:nevarro.space/$Gwnb2ZvXHc3poYXuBhho0cmoYq4KJ11Jh3m5s8kjNOM?via=nevarro.space&via=beeper.com&via=matrix.org
This use case only works if the user is already joined in the current room state
at the given `?prev_event_id`
|
| |
|
|
| |
This should speed up startup times and generally increase performance of
groups.
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
`_notify_app_services_ephemeral` (#11137)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | |
| |
| |
| | |
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* add code to handle missing content-type header and a test to verify that it works
* add handling for missing content-type in the /upload endpoint as well
* slightly refactor test code to put private method in approriate place
* handle possible null value for content-type when pulling from the local db
* add changelog
* refactor test and add code to handle missing content-type in cached remote media
* requested changes
* Update changelog.d/11200.bugfix
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* add use_float=true to ijson calls
* lints
* add changelog
* Update changelog.d/11217.bugfix
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
| | |
| |
| |
| | |
when an exception is raised (#11033)
|
| | |
| |
| |
| |
| | |
Remove deprecated delete room admin API,
`POST /_synapse/admin/v1/rooms/<room_id>/delete`
|
| | |
| |
| |
| | |
Fixes https://github.com/matrix-org/synapse/issues/8308
|
| | |
| |
| |
| |
| | |
Fixes #11064
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | | |
|
| | |
| |
| | |
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| | |
| |
| |
| | |
#10969 was merged after 1.46.0rc1 was cut and will be included
in v1.47.0rc1 instead.
|
| | | |
|
| | | |
|
| | |
| |
| | |
Signed-off-by: Rafael Goncalves <rafaelgoncalves@riseup.net>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
origin and host are the same. (#11129)
* add tests for fetching key locally
* add logic to check if origin server is same as host and fetch verify key locally rather than over federation
* add changelog
* slight refactor, add docstring, change changelog entry
* Make changelog entry one line
* remove verify_json_locally and push locality check to process_request, add function process_request_locally
* remove leftover code reference
* refactor to add common call to 'verify_json and associated handling code
* add type hint to process_json
* add some docstrings + very slight refactor
|
