| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| | |
* update version of black and also fix the mypy config being overridden
|
| | |
| |
| | |
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
|
| | | |
|
| |\| |
|
| | | |
|
| | |\
| | |
| | | |
Port receipt and read markers to async/wait
|
| | | | |
|
| | |/ |
|
| | | |
|
| | |\
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Synapse 1.5.0rc2 (2019-10-28)
=============================
Bugfixes
--------
- Update list of boolean columns in `synapse_port_db`. ([\#6247](https://github.com/matrix-org/synapse/issues/6247))
- Fix /keys/query API on workers. ([\#6256](https://github.com/matrix-org/synapse/issues/6256))
- Improve signature checking on some federation APIs. ([\#6262](https://github.com/matrix-org/synapse/issues/6262))
Internal Changes
----------------
- Move schema delta files to the correct data store. ([\#6248](https://github.com/matrix-org/synapse/issues/6248))
- Small performance improvement by removing repeated config lookups in room stats calculation. ([\#6255](https://github.com/matrix-org/synapse/issues/6255))
|
| | | |
| | |
| | |
| | |
| | | |
Make sure that we check that events sent over /send_join, /send_leave, and
/invite, are correctly signed and come from the expected servers.
|
| | |/
| |
| |
| | |
This avoids calculating __repr__ unless we are going to log.
|
| |\| |
|
| | |\
| | |
| | |
| | | |
erikj/refactor_stores
|
| | | |
| | |
| | |
| | |
| | | |
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
|
| | |/
|/| |
|
| |/
|
|
| |
The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
|
| |
|
|
|
|
|
| |
* Fix /federation/v1/state for recent room versions
Turns out this endpoint was completely broken for v3 rooms. Hopefully this
re-signing code is irrelevant nowadays anyway.
|
| | |
|
| |
|
|
| |
Use room_stats and room_state for room directory search
|
| |
|
|
|
|
|
|
| |
Fixes:
> TypeError: set_destination_retry_timings() missing 1 required positional
argument: 'retry_interval'
Introduced in #6016.
|
| |
|
|
|
|
|
| |
params to docstring (#6010)
Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument.
Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
|
| |\
| |
| |
| |
| | |
matrix-org/joriks/opentracing_link_send_to_edu_contexts
Link the send loop with the edus contexts
|
| | |
| |
| | |
Co-Authored-By: Erik Johnston <erik@matrix.org>
|
| | |
| |
| |
| |
| |
| | |
The contexts were being filtered too early so the send loop wasn't
being linked to them unless the destination
was whitelisted.
|
| |/ |
|
| |
|
|
|
| |
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
|
| |\
| |
| | |
MSC2197 Search Filters over Federation
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| | |
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
| |
| |
| |
| | |
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Propagate opentracing contexts across workers
Also includes some Convenience modifications to opentracing for servlets, notably:
- Add boolean to skip the whitelisting check on inject
extract methods. - useful when injecting into carriers
locally. Otherwise we'd always have to include our
own servername and whitelist our servername
- start_active_span_from_request instead of header
- Add boolean to decide whether to extract context
from a request to a servlet
|
| | |
| |
| | |
Add opentracing tags and logs for e2e keys
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add authenticated_entity and servlet_names tags.
Functionally:
- Add a tag for authenticated_entity
- Add a tag for servlet_names
Stylistically:
Moved to importing methods directly from opentracing.
|
| |/
|
|
|
|
|
| |
is cached and so does not always return a `Deferred`.
`await` does not silently pass-through non-Deferreds like `yield` used to.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
|
| | |
|
| | |
|
| |\
| |
| | |
Log when we receive a /make_* request from a different origin
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| | |
Bugfixes
--------
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Fix servlet metric names
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove redundant check
* Cover all return paths
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Convert BaseFederationServlet._wrap to async
Empirically, this fixes some lost stacktraces. It should be safe because the
wrapped function is called from JsonResource._async_render, which is already
async.
* Convert the rest of synapse.federation.transport.server to async
We may as well do the whole file while we're here.
* changelog
* flake8
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Configure and initialise tracer
Includes config options for the tracer and sets up JaegerClient.
* Scope manager using LogContexts
We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.
* jaeger is a dependency now
* Carrier inject and extraction for Twisted Headers
* Trace federation requests on the way in and out.
The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.
* Create logcontext for new scope.
Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.
* Remove scope from logcontext if logcontext is top level
* Disable tracer if not configured
* typo
* Remove dependence on jaeger internals
* bools
* Set service name
* :Explicitely state that the tracer is disabled
* Black is the new black
* Newsfile
* Code style
* Use the new config setup.
* Generate config.
* Copyright
* Rename config to opentracing
* Remove user whitelisting
* Empty whitelist by default
* User ConfigError instead of RuntimeError
* Use isinstance
* Use tag constants for opentracing.
* Remove debug comment and no need to explicitely record error
* Two errors a "s(c)entry"
* Docstrings!
* Remove debugging brainslip
* Homeserver Whitlisting
* Better opentracing config comment
* linting
* Inclue worker name in service_name
* Make opentracing an optional dependency
* Neater config retreival
* Clean up dummy tags
* Instantiate tracing as object instead of global class
* Inlcude opentracing as a homeserver member.
* Thread opentracing to the request level
* Reference opetnracing through hs
* Instantiate dummy opentracin g for tests.
* About to revert, just keeping the unfinished changes just in case
* Revert back to global state, commit number:
9ce4a3d9067bf9889b86c360c05ac88618b85c4f
* Use class level methods in tracerutils
* Start and stop requests spans in a place where we
have access to the authenticated entity
* Seen it, isort it
* Make sure to close the active span.
* I'm getting black and blue from this.
* Logger formatting
Co-Authored-By: Erik Johnston <erik@matrix.org>
* Outdated comment
* Import opentracing at the top
* Return a contextmanager
* Start tracing client requests from the servlet
* Return noop context manager if not tracing
* Explicitely say that these are federation requests
* Include servlet name in client requests
* Use context manager
* Move opentracing to logging/
* Seen it, isort it again!
* Ignore twisted return exceptions on context exit
* Escape the scope
* Scopes should be entered to make them useful.
* Nicer decorator names
* Just one init, init?
* Don't need to close something that isn't open
* Docs make you smarter
|
| |
|
|
| |
this hasn't done anything for years
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.
THIS IS REALLY EXPERIMENTAL.
|
| |
|
|
| |
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
|
| |
|
| |
fixes #5153
|
| |
|
|
|
|
|
| |
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
|
| | |
|
| |
|
| |
Remove some spurious stuff, clarify some other stuff
|
| |
|
|
|
| |
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
|
| |
|
|
|
| |
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
|
| |
|
|
|
|
|
|
| |
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.
We do not yet enforce the validity period for event signatures.
|
| | |
|
| |
|
|
|
|
|
| |
If we remove support for a particular room version, we should behave more
gracefully. This should make client requests fail with a 400 rather than a 500,
and will ignore individiual PDUs in a federation transaction, rather than the
whole transaction.
|
| | |
|
| |
|
|
| |
... mostly to fix pep8 fails
|
| |
|
| |
Fixes #3951.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CS API (#5083)
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
|
| |
|
| |
Redo of https://github.com/matrix-org/synapse/pull/4840
|
| |\
| |
| |
| |
| | |
(#4840)
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
|
| | | |
|
| | | |
|
| | |\ |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
Rate-limit outgoing read-receipts as per #4730.
|
| |\ \ \
| | | |
| | | | |
Split TransactionQueue up
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | | |
This is easier than having to have a million fields keyed on destination.
|
| |\| | |
| | | |
| | | | |
Move client receipt processing to federation sender worker.
|
| | | |/
| |/|
| | |
| | |
| | | |
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
|
| |/ /
| |
| |
| |
| |
| | |
endpoints (#4793)"
This reverts commit 290552fd836f4ae2dc1d893a7f72f7fff85365d3.
|
| |/
|
|
|
| |
endpoints (#4793)
Server side of a solution towards #3622.
|
| |
|
|
|
| |
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
|
| |
|
|
|
| |
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* make 'event_id' a required parameter in federated state requests
As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* add changelog entry for bugfix
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* Update server.py
|
| | |
|
| |\
| |
| |
| | |
anoa/public_rooms_federate_develop
|
| | | |
|
| | |\
| | |
| | | |
Config option to prevent showing non-fed rooms in fed /publicRooms
|
| | | |\
| | | |
| | | |
| | | | |
anoa/public_rooms_federate
|
| | | | | |
|
| | |\ \ \
| | | | |
| | | | | |
Log tracebacks correctly
|
| | | | |/
| | |/| |
|
| | |/ / |
|
| | | | |
|
| | |\ \
| | | |
| | | | |
New listener resource for the federation API "openid/userinfo" endpoint
|
| | | | |
| | | |
| | | |
| | | | |
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
* Reject large transactions on federation
* Add changelog
* lint
* Simplify large transaction handling
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
In future version events won't have an event ID, so we won't be able to
do this check.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
We only process events sent to us from a server if the event ID matches
the server, to help guard against federation storms. We replace this
with a check against the event origin.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
|
| | |\ \ \
| | | | |
| | | | | |
Refactor event building into EventBuilder
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This is so that everything is done in one place, making it easier to
change the event format based on room version
|
| | |\ \ \ \
| | | | | |
| | | | | | |
Fixup calls to `comput_event_signature`
|
| | | |/ / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.
This fixes PR #4493 commit 855a151
|
| | |/ / /
| | | |
| | | |
| | | |
| | | | |
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
|
| | | | | |
|
| | |\ \ \ |
|
| | | |\ \ \
| | | | | |
| | | | | | |
Add room_version param to get_pdu
|
| | | | | | | |
|
| | | | |/ /
| | | | |
| | | | |
| | | | |
| | | | | |
When we add new event format we'll need to know the event format or room
version when parsing events.
|
| | | | | | |
|
| | | | | | |
|
| | | |/ /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
|
| | | | | |
|
| | |/ /
| | |
| | |
| | |
| | | |
We also implement `make_membership_event` converting the returned
room version to an event format version.
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
|
| | | |
| | |
| | | |
Co-Authored-By: erikjohnston <erikj@jki.re>
|
| | | |
| | |
| | | |
Co-Authored-By: erikjohnston <erikj@jki.re>
|
| | | |
| | |
| | |
| | |
| | |
| | | |
When we receive events over federation we will need to know the room
version to be able to correctly handle them, e.g. once we start changing
event formats. Currently, we attempt to handle events in unknown rooms.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
|
| | | |
| | |
| | |
| | |
| | | |
This is in preparation to refactor FrozenEvent to support different
event formats for different room versions
|
| | |/ |
|
| |/ |
|
| |\
| |
| |
| | |
erikj/alias_disallow_list
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
|
| |/ |
|
| |
|
|
|
|
|
|
|
| |
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
|
| |\
| |
| | |
remove spurious federation checks on localhost
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
There's really no point in checking for destinations called "localhost" because
there is nothing stopping people creating other DNS entries which point to
127.0.0.1. The right fix for this is
https://github.com/matrix-org/synapse/issues/3953.
Blocking localhost, on the other hand, means that you get a surprise when
trying to connect a test server on localhost to an existing server (with a
'normal' server_name).
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
transactions (#3959)
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.
This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
|
| |/
|
|
| |
trivial fixes for docstring
|
| |\
| |
| | |
Comments and interface cleanup for on_receive_pdu
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
|
| |/
|
|
|
|
|
|
| |
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
|
| |
|
|
|
|
|
|
|
|
| |
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
|
| | |
|
| |\
| |
| | |
add some logging for the keyring queue
|
| | | |
|
| |/ |
|
| |\ |
|
| | |
| |
| |
| |
| | |
Use the actual origin for push transactions, rather than whatever the remote
server claimed.
|
| | |
| |
| |
| |
| |
| | |
We should check that both the sender's server, and the server which created the
event_id (which may be different from whatever the remote server has told us
the origin is), have signed the event.
|
| | | |
|
| | |
| |
| |
| |
| | |
itervalues(d) calls d.itervalues() [PY2] and d.values() [PY3]
but SortedDict only implements d.values()
|
| |\ \
| | |
| | | |
limt -> limit
|
| | | | |
|
| | | | |
|
| | |/
|/|
| |
| |
| | |
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
|
| | | |
|
| |\ \
| | |
| | |
| | | |
erikj/split_federation
|
| | | | |
|
| | | | |
|
| |\| |
| | |
| | |
| | | |
erikj/split_federation
|
| | |\ \
| | | |
| | | | |
more metrics for the federation and appservice senders
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Reject make_join requests from servers which do not support the room version.
Also include the room version in the response.
|
| | |/ /
| | |
| | |
| | | |
... to save me reverse-engineering this stuff again.
|
| |/ / |
|
| |\ \
| | |
| | | |
Clean up handling of errors from outbound requests
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This commit replaces SynapseError.from_http_response_exception with
HttpResponseException.to_synapse_error.
The new method actually returns a ProxiedRequestError, which allows us to pass
through additional metadata from the API call.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We really shouldn't be sending all CodeMessageExceptions back over the C-S API;
it will include things like 401s which we shouldn't proxy.
That means that we need to explicitly turn a few HttpResponseExceptions into
SynapseErrors in the federation layer.
The effect of the latter is that the matrix errcode will get passed through
correctly to calling clients, which might help with some of the random
M_UNKNOWN errors when trying to join rooms.
|
| |\| |
| | |
| | |
| | |
| | | |
matrix-org/rav/refactor_federation_client_exception_handling
Factor out exception handling in federation_client
|
| | | |
| | |
| | |
| | |
| | | |
Factor out the error handling from make_membership_event, send_join, and
send_leave, so that it can be shared.
|
| |\ \ \
| |/ /
|/| | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.
(patch supplied by @turt2live)
|
| | |/
|/| |
|
| | |
| |
| | |
The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
|
| | |
| |
| | |
It's still not used, however the parameter is an event ID not a transaction ID.
|
| | |
| |
| |
| | |
when we get an exception handling a federation PDU, log the whole stacktrace.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This fixes #3518, and ensures that we get useful logs and metrics for lots of
things that happen in the background.
(There are certainly more things that happen in the background; these are just
the common ones I've found running a single-process synapse locally).
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a mechanism for tracking resource usage by background
processes, along with an example of how it will be used.
This will help address #3518, but more importantly will give us better insights
into things which are happening but not being shown up by the request metrics.
We *could* do this with Measure blocks, but:
- I think having them pulled out as a completely separate metric class will
make it easier to distinguish top-level processes from those which are
nested.
- I want to be able to report on in-flight background processes, and I don't
think we want to do this for *all* Measure blocks.
|
| |
|
|
|
|
| |
the method "assert_params_in_request" does handle dicts and not
requests. A request body has to be parsed to json before this method
can be used
|
| | |
|
| |
|
|
|
| |
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
|
| |
|
|
|
|
|
|
| |
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.
Also, use it to verify the server name in the config file.
|
| |
|
|
|
| |
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
|
| |\
| |
| | |
Check the state of prev_events a bit more thoroughly when coming over federation
|
| | | |
|
| |/ |
|
| |\
| |
| | |
Simplify get_persisted_pdu
|
| | |
| |
| |
| |
| | |
it doesn't make much sense to use get_persisted_pdu on the receive path: just
get the event straight from the store.
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |
|
|
|
| |
Fixes a startup crash due to commit df9f72d9e5fe264b86005208e0f096156eb03e4b
"replacing portions".
|
| |
|
|
| |
they're not meant to be lazy (#3307)
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
transaction_id, destination defined twice
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| | |
* When creating a new event, cap its depth to 2^63 - 1
* When receiving events, reject any without a sensible depth
As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
|
| |\ \ |
|
| | |\ \ |
|
| | | |/
| |/|
| | |
| | |
| | |
| | | |
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
|
| | |/
|/|
| |
| |
| |
| | |
plus a bonus next()
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
|
| |\
| |
| | |
Reject events which have lots of prev_events
|
| | | |
|
| |\ \
| | |
| | | |
Use six.itervalues in some places
|
| | |/
| |
| |
| |
| |
| | |
There's more where that came from
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Refactor ResponseCache usage
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.
This will be largely non-functional, but does include the following functional
changes:
* federation_server.on_context_state_request: drops use of _server_linearizer
which looked redundant and could cause incorrect cache misses by yielding
between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
on production.
|
| | |/
|/|
| |
| |
| |
| | |
It turns out that most of the time we were calling have_events, we were only
using half of the result. Replace have_events with have_seen_events and
get_rejection_reasons, so that we can see what's going on a bit more clearly.
|
| | |
| |
| |
| | |
we were checking the wrong server_name on inbound requests
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
| |
This reverts commit 9fbe70a7dc3afabfdac176ba1f4be32dd44602aa.
It turns out that sortedcontainers.SortedDict is not an exact match for
blist.sorteddict; in particular, `popitem()` removes things from the opposite
end of the dict.
This is trivial to fix, but I want to add some unit tests, and potentially some
more thought about it, before we do so.
|
| |\
| |
| | |
Add metrics for ResponseCache
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |\ \
| | |
| | | |
Synapse on PyPy
|
| | |/
| |
| |
| |
| |
| |
| |
| | |
This commit drop-in replaces blist with SortedContainers. They are
written in pure python so work with pypy, but perform as good as
native implementations, at least in a couple benchmarks:
http://www.grantjenks.com/docs/sortedcontainers/performance.html
|
| |\ \
| | |
| | | |
Send federation events concurrently
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | |/ |
|
| | | |
|
