| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If we send out an event which refers to `prev_events` which other servers in
the federation are missing, then (after a round or two of backfill attempts),
they will end up asking us for `/state_ids` at a particular point in the DAG.
As per https://github.com/matrix-org/synapse/issues/7893, this is quite
expensive, and we tend to see lots of very similar requests around the same
time.
We can therefore handle this much more efficiently by using a cache, which (a)
ensures that if we see the same request from multiple servers (or even the same
server, multiple times), then they share the result, and (b) any other servers
that miss the initial excitement can also benefit from the work.
[It's interesting to note that `/state` has a cache for exactly this
reason. `/state` is now essentially unused and replaced with `/state_ids`, but
evidently when we replaced it we forgot to add a cache to the new endpoint.]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For inbound federation requests, if a given remote server makes too many
requests at once, we start stacking them up rather than processing them
immediatedly.
However, that means that there is a fair chance that the requesting server will
disconnect before we start processing the request. In that case, if it was a
read-only request (ie, a GET request), there is absolutely no point in
building a response (and some requests are quite expensive to handle).
Even in the case of a POST request, one of two things will happen:
* Most likely, the requesting server will retry the request and we'll get the
information anyway.
* Even if it doesn't, the requesting server has to assume that we didn't get
the memo, and act accordingly.
In short, we're better off aborting the request at this point rather than
ploughing on with what might be a quite expensive request.
|
| | |
|
| | |
|
| | |
|
| |
|
| |
This reuses the same scheme as federation sender sharding
|
| |
|
|
|
|
|
|
| |
It was correct at the time of our friend Jorik writing it (checking
git blame), but the world has moved now and it is no longer a
generator.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| |
|
|
| |
json. (#7836)
|
| | |
|
| | |
|
| | |
|
| |
|
| |
... instead of duplicating `config.signing_key[0]` everywhere
|
| | |
|
| |
|
|
| |
Introduced in #7755, not yet released.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
We're pretty close to having mypy working for `synapse.federation`, so let's
finish the job.
|
| |
|
|
| |
looks like we managed to break this during the refactorathon.
|
| | |
|
| |
|
|
| |
which has it set to private (#7368)
|
| |\ |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| | |
This was incorrectly merged to the release branch before it was ready.
This reverts commit 72fe2affb6ac86d433b80b6452da57052365aa26.
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add changelog
Save retrieved keys to the db
lint
Fix and de-brittle remote result dict processing
Use query_user_devices instead, assume only master, self_signing key types
Make changelog more useful
Remove very specific exception handling
Wrap get_verify_key_from_cross_signing_key in a try/except
Note that _get_e2e_cross_signing_verify_key can raise a SynapseError
lint
Add comment explaining why this is useful
Only fetch master and self_signing key types
Fix log statements, docstrings
Remove extraneous items from remote query try/except
lint
Factor key retrieval out into a separate function
Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates
Update method docstring
|
| | | |
|
| | |
| |
| | |
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Pull Sentinel out of LoggingContext
... and drop a few unnecessary references to it
* Factor out LoggingContext.current_context
move `current_context` and `set_context` out to top-level functions.
Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.
* move copy-to-parent into `stop`
this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.
* Replace `LoggingContext.alive` with `finished`
Turn `alive` into `finished` and make it a bit better defined.
|
| | |
| |
| |
| |
| | |
This just helps keep the rows closer to their streams, so that it's easier to
see what the format of each stream is.
|
| |/
|
|
| |
PDU" errors (#7089).
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| | |
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
|
| |/
|
|
| |
Ensure good comprehension hygiene using flake8-comprehensions.
|
| | |
|
| |
|
|
|
|
|
| |
... and use it in places where it's trivial to do so.
This will make it easier to pass room versions into the FrozenEvent
constructors.
|
| | |
|
| |
|
|
| |
It's called from all over the shop, so this one's a bit messy.
|
| |
|
|
| |
... which allows us to sanity-check the create event.
|
| |
|
|
|
|
|
|
| |
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
|
| | |
|
| | |
|
| | |
|
| |
|
| |
Co-Authored-By: Erik Johnston <erik@matrix.org>
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
I messed this up a bit in #6805, but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
|
| |\
| |
| | |
Pass room_version into add_hashes_and_signatures
|
| | | |
|
| | |
| |
| |
| |
| | |
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
|
| |/ |
|
| |
|
|
| |
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
We were sending device updates down both the federation stream and
device streams. This mean there was a race if the federation sender
worker processed the federation stream first, as when the sender checked
if there were new device updates the slaved ID generator hadn't been
updated with the new stream IDs and so returned nothing.
This situation is correctly handled by events/receipts/etc by not
sending updates down the federation stream and instead having the
federation sender worker listen on the other streams and poke the
transaction queues as appropriate.
|
| |
|
| |
This is so that we don't have to rely on pulling it out from `current_state_events` table.
|
| |
|
|
|
| |
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
* Port synapse.replication.tcp to async/await
* Newsfile
* Correctly document type of on_<FOO> functions as async
* Don't be overenthusiastic with the asyncing....
|
| |
|
|
| |
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
|
| |\
| |
| | |
Implement v2 APIs for send_join and send_leave
|
| | |\ |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | | |
This is a non-functional refactor as a precursor to some other work.
|
| | | |
| | |
| | |
| | |
| | | |
There was a bunch of unnecessary conversion back and forth between dict and
list going on here. We can simplify a bunch of the code.
|
| | |/
|/|
| |
| | |
This API was added way back in 0.17.0; the code here is annoying to maintain
and entirely redundant.
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| |\ |
|
| | |
| |
| | |
* update version of black and also fix the mypy config being overridden
|
| | |
| |
| | |
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
|
| | | |
|
| |\| |
|
| | | |
|
| | |\
| | |
| | | |
Port receipt and read markers to async/wait
|
| | | | |
|
| | |/ |
|
| | | |
|
| | |\
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Synapse 1.5.0rc2 (2019-10-28)
=============================
Bugfixes
--------
- Update list of boolean columns in `synapse_port_db`. ([\#6247](https://github.com/matrix-org/synapse/issues/6247))
- Fix /keys/query API on workers. ([\#6256](https://github.com/matrix-org/synapse/issues/6256))
- Improve signature checking on some federation APIs. ([\#6262](https://github.com/matrix-org/synapse/issues/6262))
Internal Changes
----------------
- Move schema delta files to the correct data store. ([\#6248](https://github.com/matrix-org/synapse/issues/6248))
- Small performance improvement by removing repeated config lookups in room stats calculation. ([\#6255](https://github.com/matrix-org/synapse/issues/6255))
|
| | | |
| | |
| | |
| | |
| | | |
Make sure that we check that events sent over /send_join, /send_leave, and
/invite, are correctly signed and come from the expected servers.
|
| | |/
| |
| |
| | |
This avoids calculating __repr__ unless we are going to log.
|
| |\| |
|
| | |\
| | |
| | |
| | | |
erikj/refactor_stores
|
| | | |
| | |
| | |
| | |
| | | |
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
|
| | |/
|/| |
|
| |/
|
|
| |
The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
|
| |
|
|
|
|
|
| |
* Fix /federation/v1/state for recent room versions
Turns out this endpoint was completely broken for v3 rooms. Hopefully this
re-signing code is irrelevant nowadays anyway.
|
| | |
|
| |
|
|
| |
Use room_stats and room_state for room directory search
|
| |
|
|
|
|
|
|
| |
Fixes:
> TypeError: set_destination_retry_timings() missing 1 required positional
argument: 'retry_interval'
Introduced in #6016.
|
| |
|
|
|
|
|
| |
params to docstring (#6010)
Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument.
Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
|
| |\
| |
| |
| |
| | |
matrix-org/joriks/opentracing_link_send_to_edu_contexts
Link the send loop with the edus contexts
|
| | |
| |
| | |
Co-Authored-By: Erik Johnston <erik@matrix.org>
|
| | |
| |
| |
| |
| |
| | |
The contexts were being filtered too early so the send loop wasn't
being linked to them unless the destination
was whitelisted.
|
| |/ |
|
| |
|
|
|
| |
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
|
| |\
| |
| | |
MSC2197 Search Filters over Federation
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| |
| | |
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Propagate opentracing contexts across workers
Also includes some Convenience modifications to opentracing for servlets, notably:
- Add boolean to skip the whitelisting check on inject
extract methods. - useful when injecting into carriers
locally. Otherwise we'd always have to include our
own servername and whitelist our servername
- start_active_span_from_request instead of header
- Add boolean to decide whether to extract context
from a request to a servlet
|
| | |
| |
| | |
Add opentracing tags and logs for e2e keys
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add authenticated_entity and servlet_names tags.
Functionally:
- Add a tag for authenticated_entity
- Add a tag for servlet_names
Stylistically:
Moved to importing methods directly from opentracing.
|
| |/
|
|
|
|
|
| |
is cached and so does not always return a `Deferred`.
`await` does not silently pass-through non-Deferreds like `yield` used to.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
|
| | |
|
| |\
| |
| | |
Log when we receive a /make_* request from a different origin
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| | |
Bugfixes
--------
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Fix servlet metric names
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove redundant check
* Cover all return paths
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Convert BaseFederationServlet._wrap to async
Empirically, this fixes some lost stacktraces. It should be safe because the
wrapped function is called from JsonResource._async_render, which is already
async.
* Convert the rest of synapse.federation.transport.server to async
We may as well do the whole file while we're here.
* changelog
* flake8
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Configure and initialise tracer
Includes config options for the tracer and sets up JaegerClient.
* Scope manager using LogContexts
We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.
* jaeger is a dependency now
* Carrier inject and extraction for Twisted Headers
* Trace federation requests on the way in and out.
The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.
* Create logcontext for new scope.
Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.
* Remove scope from logcontext if logcontext is top level
* Disable tracer if not configured
* typo
* Remove dependence on jaeger internals
* bools
* Set service name
* :Explicitely state that the tracer is disabled
* Black is the new black
* Newsfile
* Code style
* Use the new config setup.
* Generate config.
* Copyright
* Rename config to opentracing
* Remove user whitelisting
* Empty whitelist by default
* User ConfigError instead of RuntimeError
* Use isinstance
* Use tag constants for opentracing.
* Remove debug comment and no need to explicitely record error
* Two errors a "s(c)entry"
* Docstrings!
* Remove debugging brainslip
* Homeserver Whitlisting
* Better opentracing config comment
* linting
* Inclue worker name in service_name
* Make opentracing an optional dependency
* Neater config retreival
* Clean up dummy tags
* Instantiate tracing as object instead of global class
* Inlcude opentracing as a homeserver member.
* Thread opentracing to the request level
* Reference opetnracing through hs
* Instantiate dummy opentracin g for tests.
* About to revert, just keeping the unfinished changes just in case
* Revert back to global state, commit number:
9ce4a3d9067bf9889b86c360c05ac88618b85c4f
* Use class level methods in tracerutils
* Start and stop requests spans in a place where we
have access to the authenticated entity
* Seen it, isort it
* Make sure to close the active span.
* I'm getting black and blue from this.
* Logger formatting
Co-Authored-By: Erik Johnston <erik@matrix.org>
* Outdated comment
* Import opentracing at the top
* Return a contextmanager
* Start tracing client requests from the servlet
* Return noop context manager if not tracing
* Explicitely say that these are federation requests
* Include servlet name in client requests
* Use context manager
* Move opentracing to logging/
* Seen it, isort it again!
* Ignore twisted return exceptions on context exit
* Escape the scope
* Scopes should be entered to make them useful.
* Nicer decorator names
* Just one init, init?
* Don't need to close something that isn't open
* Docs make you smarter
|
| |
|
|
| |
this hasn't done anything for years
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.
THIS IS REALLY EXPERIMENTAL.
|
| |
|
|
| |
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
|
| |
|
| |
fixes #5153
|
| |
|
|
|
|
|
| |
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
|
| | |
|
| |
|
| |
Remove some spurious stuff, clarify some other stuff
|
| |
|
|
|
| |
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
|
| |
|
|
|
| |
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
|
| |
|
|
|
|
|
|
| |
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.
We do not yet enforce the validity period for event signatures.
|
| | |
|
| |
|
|
|
|
|
| |
If we remove support for a particular room version, we should behave more
gracefully. This should make client requests fail with a 400 rather than a 500,
and will ignore individiual PDUs in a federation transaction, rather than the
whole transaction.
|
| | |
|
| |
|
|
| |
... mostly to fix pep8 fails
|
| |
|
| |
Fixes #3951.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CS API (#5083)
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
|
| |
|
| |
Redo of https://github.com/matrix-org/synapse/pull/4840
|
| |\
| |
| |
| |
| | |
(#4840)
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
|
| | | |
|
| | | |
|
| | |\ |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
Rate-limit outgoing read-receipts as per #4730.
|
| |\ \ \
| | | |
| | | | |
Split TransactionQueue up
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | | |
This is easier than having to have a million fields keyed on destination.
|
| |\| | |
| | | |
| | | | |
Move client receipt processing to federation sender worker.
|
| | | |/
| |/|
| | |
| | |
| | | |
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
|
| |/ /
| |
| |
| |
| |
| | |
endpoints (#4793)"
This reverts commit 290552fd836f4ae2dc1d893a7f72f7fff85365d3.
|
| |/
|
|
|
| |
endpoints (#4793)
Server side of a solution towards #3622.
|
| |
|
|
|
| |
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
|
| |
|
|
|
| |
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* make 'event_id' a required parameter in federated state requests
As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* add changelog entry for bugfix
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* Update server.py
|
| | |
|
| |\
| |
| |
| | |
anoa/public_rooms_federate_develop
|
| | | |
|
| | |\
| | |
| | | |
Config option to prevent showing non-fed rooms in fed /publicRooms
|
| | | |\
| | | |
| | | |
| | | | |
anoa/public_rooms_federate
|
| | | | | |
|
| | |\ \ \
| | | | |
| | | | | |
Log tracebacks correctly
|
| | | | |/
| | |/| |
|
| | |/ / |
|
| | | | |
|
| | |\ \
| | | |
| | | | |
New listener resource for the federation API "openid/userinfo" endpoint
|
| | | | |
| | | |
| | | |
| | | | |
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
* Reject large transactions on federation
* Add changelog
* lint
* Simplify large transaction handling
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
In future version events won't have an event ID, so we won't be able to
do this check.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
We only process events sent to us from a server if the event ID matches
the server, to help guard against federation storms. We replace this
with a check against the event origin.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
|
| | |\ \ \
| | | | |
| | | | | |
Refactor event building into EventBuilder
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This is so that everything is done in one place, making it easier to
change the event format based on room version
|
| | |\ \ \ \
| | | | | |
| | | | | | |
Fixup calls to `comput_event_signature`
|
| | | |/ / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.
This fixes PR #4493 commit 855a151
|
| | |/ / /
| | | |
| | | |
| | | |
| | | | |
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
|
| | | | | |
|
| | |\ \ \ |
|
| | | |\ \ \
| | | | | |
| | | | | | |
Add room_version param to get_pdu
|
| | | | | | | |
|
| | | | |/ /
| | | | |
| | | | |
| | | | |
| | | | | |
When we add new event format we'll need to know the event format or room
version when parsing events.
|
| | | | | | |
|
| | | | | | |
|
| | | |/ /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
|
| | | | | |
|
| | |/ /
| | |
| | |
| | |
| | | |
We also implement `make_membership_event` converting the returned
room version to an event format version.
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
|
| | | |
| | |
| | | |
Co-Authored-By: erikjohnston <erikj@jki.re>
|
| | | |
| | |
| | | |
Co-Authored-By: erikjohnston <erikj@jki.re>
|
| | | |
| | |
| | |
| | |
| | |
| | | |
When we receive events over federation we will need to know the room
version to be able to correctly handle them, e.g. once we start changing
event formats. Currently, we attempt to handle events in unknown rooms.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
|
| | | |
| | |
| | |
| | |
| | | |
This is in preparation to refactor FrozenEvent to support different
event formats for different room versions
|
| | |/ |
|
| |/ |
|
| |\
| |
| |
| | |
erikj/alias_disallow_list
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
|
| |/ |
|
| |
|
|
|
|
|
|
|
| |
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
|
| |\
| |
| | |
remove spurious federation checks on localhost
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
There's really no point in checking for destinations called "localhost" because
there is nothing stopping people creating other DNS entries which point to
127.0.0.1. The right fix for this is
https://github.com/matrix-org/synapse/issues/3953.
Blocking localhost, on the other hand, means that you get a surprise when
trying to connect a test server on localhost to an existing server (with a
'normal' server_name).
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
transactions (#3959)
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.
This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
|
| |/
|
|
| |
trivial fixes for docstring
|
| |\
| |
| | |
Comments and interface cleanup for on_receive_pdu
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
|
| |/
|
|
|
|
|
|
| |
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
|
| |
|
|
|
|
|
|
|
|
| |
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
|
| | |
|
| |\
| |
| | |
add some logging for the keyring queue
|
| | | |
|
