summary refs log tree commit diff
path: root/synapse/federation/federation_server.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Clean up some logging in the federation event handler (#10591)Richard van der Hoff2021-08-161-0/+1
| | | | | | | | | | | | | | | | | | | * Include outlier status in `str(event)` In places where we log event objects, knowing whether or not you're dealing with an outlier is super useful. * Remove duplicated logging in get_missing_events When we process events received from get_missing_events, we log them twice (once in `_get_missing_events_for_pdu`, and once in `on_receive_pdu`). Reduce the duplication by removing the logging in `on_receive_pdu`, and ensuring the call sites do sensible logging. * log in `on_receive_pdu` when we already have the event * Log which prev_events we are missing * changelog
* Fix a harmless exception when the staged events queue is empty. (#10592)Patrick Cloke2021-08-131-5/+10
|
* Convert Transaction and Edu object to attrs (#10542)Patrick Cloke2021-08-061-20/+30
| | | | | Instead of wrapping the JSON into an object, this creates concrete instances for Transaction and Edu. This allows for improved type hints and simplified code.
* Prune inbound federation queues if they get too long (#10390)Erik Johnston2021-08-021-0/+17
|
* Update the MSC3083 support to verify if joins are from an authorized server. ↵Patrick Cloke2021-07-261-6/+35
| | | | (#10254)
* Do not include signatures/hashes in make_{join,leave,knock} responses. (#10404)Patrick Cloke2021-07-161-6/+3
| | | | These signatures would end up invalid since the joining/leaving/knocking server would modify the response before calling send_{join,leave,knock}.
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-19/+15
|
* Ensure we always drop the federation inbound lock (#10336)Erik Johnston2021-07-091-0/+1
|
* Handle old staged inbound events (#10303)Erik Johnston2021-07-061-10/+57
| | | | | | | We might have events in the staging area if the service was restarted while there were unhandled events in the staging area. Fixes #10295
* Move methods involving event authentication to EventAuthHandler. (#10268)Patrick Cloke2021-07-011-3/+3
| | | Instead of mixing them with user authentication methods.
* Fix the inbound PDU metric (#10279)Erik Johnston2021-06-301-17/+20
| | | This broke in #10272
* Merge branch 'release-v1.37' into developRichard van der Hoff2021-06-291-2/+96
|\
| * Handle inbound events from federation asynchronously (#10272)Erik Johnston2021-06-291-2/+96
| | | | | | | | | | | | | | | | | | | | | | Fixes #9490 This will break a couple of SyTest that are expecting failures to be added to the response of a federation /send, which obviously doesn't happen now that things are asynchronous. Two drawbacks: Currently there is no logic to handle any events left in the staging area after restart, and so they'll only be handled on the next incoming event in that room. That can be fixed separately. We now only process one event per room at a time. This can be fixed up further down the line.
* | Improve validation for `send_{join,leave,knock}` (#10225)Richard van der Hoff2021-06-241-49/+72
|/ | | The idea here is to stop people sending things that aren't joins/leaves/knocks through these endpoints: previously you could send anything you liked through them. I wasn't able to find any security holes from doing so, but it doesn't sound like a good thing.
* Implement knock feature (#6739)Sorunome2021-06-091-0/+99
| | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io
* Add type hints to the federation server transport. (#10080)Patrick Cloke2021-06-081-3/+3
|
* Fix `m.room_key_request` to-device messages (#9961)Richard van der Hoff2021-05-111-19/+0
| | | fixes #9960
* remove `HomeServer.get_config` (#9815)Richard van der Hoff2021-04-141-1/+1
| | | | Every single time I want to access the config object, I have to remember whether or not we use `get_config`. Let's just get rid of it.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Add type hints to the federation handler and server. (#9743)Patrick Cloke2021-04-061-13/+13
|
* Make RateLimiter class check for ratelimit overrides (#9711)Erik Johnston2021-03-301-1/+4
| | | | | | | This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited. We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits. Fixes #9663
* Make federation catchup send last event from any server. (#9640)Erik Johnston2021-03-181-23/+2
| | | | | | | | | | | | | | Currently federation catchup will send the last *local* event that we failed to send to the remote. This can cause issues for large rooms where lots of servers have sent events while the remote server was down, as when it comes back up again it'll be flooded with events from various points in the DAG. Instead, let's make it so that all the servers send the most recent events, even if its not theirs. The remote should deduplicate the events, so there shouldn't be much overhead in doing this. Alternatively, the servers could only send local events if they were also extremities and hope that the other server will send the event over, but that is a bit risky.
* Fix additional type hints from Twisted 21.2.0. (#9591)Patrick Cloke2021-03-121-3/+5
|
* Reject concurrent transactions (#9597)Richard van der Hoff2021-03-121-35/+42
| | | | | | If more transactions arrive from an origin while we're still processing the first one, reject them. Hopefully a quick fix to https://github.com/matrix-org/synapse/issues/9489
* Improve logging when processing incoming transactions (#9596)Richard van der Hoff2021-03-121-27/+34
| | | Put the room id in the logcontext, to make it easier to understand what's going on.
* Use the chain cover index in get_auth_chain_ids. (#9576)Patrick Cloke2021-03-101-2/+4
| | | | This uses a simplified version of get_chain_cover_difference to calculate auth chain of events.
* Fix additional type hints. (#9543)Patrick Cloke2021-03-091-1/+1
| | | Type hint fixes due to Twisted 21.2.0 adding type hints.
* Add ResponseCache tests. (#9458)Jonathan de Jong2021-03-081-5/+8
|
* Replace `last_*_pdu_age` metrics with timestamps (#9540)Richard van der Hoff2021-03-041-6/+4
| | | | | | | | Following the advice at https://prometheus.io/docs/practices/instrumentation/#timestamps-not-time-since, it's preferable to export unix timestamps, not ages. There doesn't seem to be any particular naming convention for timestamp metrics.
* Ratelimit cross-user key sharing requests. (#8957)Patrick Cloke2021-02-191-2/+18
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-10/+11
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Support icons for Identity Providers (#9154)Richard van der Hoff2021-01-201-1/+1
|
* Support routing edu's to multiple instances (#9042)Erik Johnston2021-01-071-5/+16
| | | This is in preparation for moving `SendToDeviceServlet` off master
* Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-021-1/+0
| | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* Consistently use room_id from federation request body (#8776)Richard van der Hoff2020-11-191-13/+10
| | | | | | | | | | | | | * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts".
* Add type hints to response cache. (#8507)Patrick Cloke2020-10-091-3/+5
|
* Remove the deprecated Handlers object (#8494)Patrick Cloke2020-10-091-1/+6
| | | All handlers now available via get_*_handler() methods on the HomeServer.
* Add type hints to some handlers (#8505)Patrick Cloke2020-10-091-1/+1
|
* Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-3/+2
|
* Add prometheus metrics to track federation delays (#8430)Richard van der Hoff2020-10-011-1/+23
| | | | | Add a pair of federation metrics to track the delays in sending PDUs to/from particular servers.
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-181-1/+1
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Fix ratelimiting for federation `/send` requests. (#8342)Erik Johnston2020-09-181-12/+40
| | | c.f. #8295 for rationale
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-1/+1
|
* Be stricter about JSON that is accepted by Synapse (#8106)Patrick Cloke2020-08-191-3/+2
|
* Put a cache on `/state_ids` (#7931)Richard van der Hoff2020-07-231-2/+11
| | | | | | | | | | | | | | | | | | If we send out an event which refers to `prev_events` which other servers in the federation are missing, then (after a round or two of backfill attempts), they will end up asking us for `/state_ids` at a particular point in the DAG. As per https://github.com/matrix-org/synapse/issues/7893, this is quite expensive, and we tend to see lots of very similar requests around the same time. We can therefore handle this much more efficiently by using a cache, which (a) ensures that if we see the same request from multiple servers (or even the same server, multiple times), then they share the result, and (b) any other servers that miss the initial excitement can also benefit from the work. [It's interesting to note that `/state` has a cache for exactly this reason. `/state` is now essentially unused and replaced with `/state_ids`, but evidently when we replaced it we forgot to add a cache to the new endpoint.]
* Allow moving typing off master (#7869)Erik Johnston2020-07-161-51/+74
|
* Ensure that calls to `json.dumps` are compatible with the standard library ↵Patrick Cloke2020-07-151-1/+1
| | | | json. (#7836)
* Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802)Patrick Cloke2020-07-101-3/+3
|
* Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-091-3/+3
|
* Add some metrics for inbound and outbound federation processing times (#7755)Erik Johnston2020-06-301-16/+21
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-3/+1
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-4/+3
|
* Add typing information to federation_server. (#7219)Patrick Cloke2020-04-071-66/+107
|
* Fix a bug in the federation API which could cause occasional "Failed to get ↵Patrick Cloke2020-03-191-4/+4
| | | | PDU" errors (#7089).
* Remove unused federation endpoint (`query_auth`) (#7026)Patrick Cloke2020-03-171-51/+0
|
* Add support for putting fed user query API on workers (#6873)Erik Johnston2020-02-071-2/+5
|
* Add typing to synapse.federation.sender (#6871)Erik Johnston2020-02-071-1/+6
|
* Pass room_version into `event_from_pdu_json`Richard van der Hoff2020-02-061-27/+14
| | | | It's called from all over the shop, so this one's a bit messy.
* Reduce amount of logging at INFO level. (#6862)Erik Johnston2020-02-061-3/+3
| | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* Fix `room_version` in `on_invite_request` flow (#6827)Richard van der Hoff2020-02-031-5/+8
| | | | I messed this up a bit in #6805, but fortunately we weren't actually doing anything with the room_version so it didn't matter that it was a str not a RoomVersion.
* s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-8/+8
| | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* pass room version into FederationHandler.on_invite_request (#6805)Richard van der Hoff2020-01-301-1/+1
|
* Process EDUs in parallel with PDUs. (#6697)Erik Johnston2020-01-141-12/+58
| | | | This means that things like to device messages don't get blocked behind processing PDUs, which can potentially take *ages*.
* Merge branch 'develop' into babolivier/msc1802Brendan Abolivier2019-12-051-9/+17
|\
| * Implementation of MSC2314 (#6176)Amber Brown2019-11-281-9/+17
| |
* | LintBrendan Abolivier2019-11-111-3/+1
| |
* | Add server-side support to the v2 APIBrendan Abolivier2019-11-111-10/+7
|/
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-9/+11
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Fix small typo in comment (#6269)Andrew Morgan2019-10-301-1/+1
|
* Don't return coroutinesErik Johnston2019-10-291-6/+6
|
* Port federation_server to async/awaitErik Johnston2019-10-291-118/+87
|
* Improve signature checking on some federation APIs (#6262)Richard van der Hoff2019-10-281-0/+7
| | | | | Make sure that we check that events sent over /send_join, /send_leave, and /invite, are correctly signed and come from the expected servers.
* Fix /federation/v1/state for recent room versions (#6170)Richard van der Hoff2019-10-081-13/+0
| | | | | | | * Fix /federation/v1/state for recent room versions Turns out this endpoint was completely broken for v3 rooms. Hopefully this re-signing code is irrelevant nowadays anyway.
* Remove origin parameter from add_display_name_to_third_party_invite and add ↵Andrew Morgan2019-09-111-2/+2
| | | | | | | params to docstring (#6010) Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument. Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
* Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-11/+11
| | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* Propagate opentracing contexts through EDUs (#5852)Jorik Schellekens2019-08-221-7/+8
| | | | | Propagate opentracing contexts through EDUs Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Opentrace e2e keys (#5855)Jorik Schellekens2019-08-221-0/+3
| | | Add opentracing tags and logs for e2e keys
* Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatchRichard van der Hoff2019-07-261-2/+2
|\ | | | | Log when we receive a /make_* request from a different origin
| * Log when we receive a /make_* request from a different originRichard van der Hoff2019-07-261-2/+2
| |
* | Replace returnValue with return (#5736)Amber Brown2019-07-231-45/+32
|/
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-2/+2
|
* Run Black. (#5482)Amber Brown2019-06-201-141/+93
|
* Fix error handling for rooms whose versions are unknown. (#5219)Richard van der Hoff2019-05-211-1/+13
| | | | | | | If we remove support for a particular room version, we should behave more gracefully. This should make client requests fail with a 400 rather than a 500, and will ignore individiual PDUs in a federation transaction, rather than the whole transaction.
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-011-1/+2
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* When presence is enabled don't send over replicationErik Johnston2019-02-271-0/+3
|
* Merge pull request #4737 from matrix-org/erikj/failure_log_tbErik Johnston2019-02-251-2/+3
|\ | | | | Log tracebacks correctly
| * Log tracebacks correctlyErik Johnston2019-02-251-2/+3
| |
* | MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite APIErik Johnston2019-02-231-1/+9
|/
* Reject large transactions on federation (#4513)Andrew Morgan2019-01-311-0/+16
| | | | | | | | | | * Reject large transactions on federation * Add changelog * lint * Simplify large transaction handling
* Only check event ID domain for signatures for V1 eventsErik Johnston2019-01-291-1/+4
| | | | | In future version events won't have an event ID, so we won't be able to do this check.
* Use event origin for filtering incoming eventsErik Johnston2019-01-291-4/+7
| | | | | | We only process events sent to us from a server if the event ID matches the server, to help guard against federation storms. We replace this with a check against the event origin.
* Fixup calls to `comput_event_signature`Erik Johnston2019-01-291-1/+1
| | | | | | | | We currently pass FrozenEvent instead of `dict` to `compute_event_signature`, which works by accident due to `dict(event)` producing the correct result. This fixes PR #4493 commit 855a151
* Require event format version to parse or create eventsErik Johnston2019-01-251-12/+21
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813Erik Johnston2019-01-251-1/+3
|\
| * Add room_version param to get_pduErik Johnston2019-01-231-1/+3
| | | | | | | | | | When we add new event format we'll need to know the event format or room version when parsing events.
* | Implement MSC 1813 - Add room version to make APIsErik Johnston2019-01-231-1/+7
|/ | | | | We also implement `make_membership_event` converting the returned room version to an event format version.
* Add /v2/invite federation APIErik Johnston2019-01-151-2/+2
|
* Update synapse/federation/federation_server.pyRichard van der Hoff2018-11-091-1/+1
| | | Co-Authored-By: erikjohnston <erikj@jki.re>
* Update synapse/federation/federation_server.pyRichard van der Hoff2018-11-091-1/+1
| | | Co-Authored-By: erikjohnston <erikj@jki.re>
* Drop PDUs of unknown roomsErik Johnston2018-11-081-1/+23
| | | | | | When we receive events over federation we will need to know the room version to be able to correctly handle them, e.g. once we start changing event formats. Currently, we attempt to handle events in unknown rooms.
* Remove the unused /pull federation API (#4118)Amber Brown2018-10-311-5/+0
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-10-251-1/+1
|\ | | | | | | erikj/alias_disallow_list
| * Fix a number of flake8 errorsRichard van der Hoff2018-10-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.
* | Add config option to control alias creationErik Johnston2018-10-191-14/+2
|/
* Fix incorrect truncation in get_missing_eventsRichard van der Hoff2018-10-161-4/+4
| | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id.
* Include eventid in log lines when processing incoming federation ↵Richard van der Hoff2018-09-271-15/+17
| | | | | | | | | | | transactions (#3959) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.
* Comments and interface cleanup for on_receive_pduRichard van der Hoff2018-09-201-1/+1
| | | | | | | | Add some informative comments about what's going on here. Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of `get_missing` and use `sent_to_us_directly` consistently.
* Merge branch 'master' into developRichard van der Hoff2018-09-061-10/+10
|\
| * Fix origin handling for pushed transactionsRichard van der Hoff2018-09-051-10/+10
| | | | | | | | | | Use the actual origin for push transactions, rather than whatever the remote server claimed.
* | Port http/ to Python 3 (#3771)Amber Brown2018-09-061-5/+5
|/
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-151-4/+4
|\ | | | | | | erikj/split_federation
| * Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-4/+4
| |
* | Fixup doc commentsErik Johnston2018-08-091-0/+11
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-091-3/+18
|\| | | | | | | erikj/split_federation
| * Enforce compatibility when processing make_join requestsRichard van der Hoff2018-08-031-3/+18
| | | | | | | | | | | | Reject make_join requests from servers which do not support the room version. Also include the room version in the response.
* | Add EDU/query handling over replicationErik Johnston2018-08-061-0/+43
|/
* Merge branch 'master' into developRichard van der Hoff2018-08-021-0/+1
|\
| * Validation for events/rooms in fed requestsRichard van der Hoff2018-08-021-0/+1
| | | | | | | | | | | | | | When we get a federation request which refers to an event id, make sure that said event is in the room the caller claims it is in. (patch supplied by @turt2live)
* | Remove pdu_failures from transactionsTravis Ralston2018-07-301-4/+0
| | | | | | The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.
* | Improve logging for exceptions handling PDUsRichard van der Hoff2018-07-231-3/+8
|/ | | | when we get an exception handling a federation PDU, log the whole stacktrace.
* run isortAmber Brown2018-07-091-11/+7
|
* Implementation of server_aclsRichard van der Hoff2018-07-041-2/+148
| | | | | ... as described at https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
* Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checkingErik Johnston2018-06-291-1/+3
|\ | | | | Check the state of prev_events a bit more thoroughly when coming over federation
| * handle federation not telling us about prev_eventsAmber Brown2018-06-271-1/+3
| |
* | Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-1/+1
|/
* simplify get_persisted_pduRichard van der Hoff2018-06-121-12/+1
| | | | | it doesn't make much sense to use get_persisted_pdu on the receive path: just get the event straight from the store.
* cleanup pep8 errorsAmber Brown2018-05-221-1/+4
|
* replacing portionsAmber Brown2018-05-211-9/+7
|
* Merge pull request #3118 from matrix-org/rav/reject_prev_eventsRichard van der Hoff2018-04-231-2/+23
|\ | | | | Reject events which have lots of prev_events
| * Add some commentsRichard van der Hoff2018-04-181-2/+23
| |
* | Merge pull request #3106 from NotAFile/py3-six-itervalues-1Richard van der Hoff2018-04-201-3/+5
|\ \ | | | | | | Use six.itervalues in some places
| * | Use six.itervalues in some placesAdrian Tschira2018-04-151-3/+5
| |/ | | | | | | | | | | There's more where that came from Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Reinstate linearizer for federation_server.on_context_state_requestRichard van der Hoff2018-04-201-5/+11
| |
* | Refactor ResponseCache usageRichard van der Hoff2018-04-121-11/+5
|/ | | | | | | | | | | | | | | Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production.
* Add metrics for ResponseCacheRichard van der Hoff2018-04-101-1/+1
|
* Merge pull request #2979 from matrix-org/erikj/no_handlersErik Johnston2018-03-131-9/+1
|\ | | | | Don't build handlers on workers unnecessarily
| * Split replication layer into twoErik Johnston2018-03-131-9/+1
| |
* | Merge pull request #2977 from matrix-org/erikj/replication_move_propsErik Johnston2018-03-131-0/+6
|\| | | | | Move property setting from ReplicationLayer to base classes
| * Move property setting from ReplicationLayer to FederationBaseErik Johnston2018-03-131-0/+6
| |
* | Fix docstring typesErik Johnston2018-03-131-2/+2
|/
* Split out edu/query registration to a separate classErik Johnston2018-03-131-48/+69
|
* Factor out `event_from_pdu_json`Richard van der Hoff2017-12-301-16/+9
| | | | | turns out we have two copies of this, and neither needs to be an instance method
* federation_server: clean up importsRichard van der Hoff2017-12-301-14/+11
|
* Do logcontexts outside ResponseCacheErik Johnston2017-10-251-3/+5
|
* Fix 500 error when we get an error handling a PDURichard van der Hoff2017-10-171-1/+0
| | | | | | | | | | FederationServer doesn't have a send_failure (and nor does its subclass, ReplicationLayer), so this was failing. I'm not really sure what the idea behind send_failure is, given (a) we don't do anything at the other end with it except log it, and (b) we also send back the failure via the transaction response. I suspect there's a whole lot of dead code around it, but for now I'm just removing the broken bit.
* Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handlingRichard van der Hoff2017-10-111-2/+3
|\ | | | | log pdu_failures from incoming transactions
| * log pdu_failures from incoming transactionsRichard van der Hoff2017-10-111-2/+3
| | | | | | | | | | | | | | ... even if we have no EDUs. This appears to have been introduced in 476899295f5fd6cff64799bcbc84cd4bf9005e33.
* | fed server: process PDUs for different rooms in parallelRichard van der Hoff2017-10-091-19/+34
| | | | | | | | | | With luck, this will give a real-time improvement when there are many rooms and the server ends up calling out to fetch missing events.
* | Fed server: use a linearizer for ongoing transactionsRichard van der Hoff2017-10-091-1/+29
|/ | | | | We don't want to process the same transaction multiple times concurrently, so use a linearizer.
* fed server: refactor on_incoming_transactionRichard van der Hoff2017-10-091-24/+29
| | | | | Move as much as possible to after the have_responded check, and reduce the number of times we iterate over the pdu list.
* Fed server: Move origin-check code to _handle_received_pduRichard van der Hoff2017-10-091-24/+24
| | | | | | | The response-building code expects there to be an entry in the `results` list for each entry in the pdu_list, so the early `continue` was messing this up. That doesn't really matter, because all that the federation client does is log any errors, but it's pretty poor form.
* Do some logging when one-time-keys get claimedRichard van der Hoff2017-05-091-0/+10
| | | | | might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.
* Accept join events from all serversRichard van der Hoff2017-04-031-2/+6
| | | | | | | Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).
* Move FederationServer._handle_new_pdu to FederationHandlerRichard van der Hoff2017-03-091-193/+1
| | | | | | | | | | | Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.
* Move sig check out of _handle_new_pduRichard van der Hoff2017-03-091-12/+36
| | | | | When we receive PDUs via `get_missing_events`, we have already checked their sigs, so there is no need to do it again.
* Factor _get_missing_events_for_pdu out of _handle_new_pduRichard van der Hoff2017-03-091-62/+82
| | | | | This should be functionally identical: it just seeks to improve readability by reducing indentation.
* Implement device key caching over federationErik Johnston2017-01-261-0/+3
|
* Name linearizer's for better logsErik Johnston2017-01-091-2/+2
|
* do the discard check in the right place to avoid grabbing dependent eventsMatthew2017-01-071-20/+20
|
* Discard PDUs from invalid origins due to #1753 in 0.18.[56] v0.18.7-rc1Matthew2017-01-071-1/+23
|
* fix commentMatthew Hodgson2017-01-051-1/+17
|
* limit total timeout for get_missing_events to 10sMatthew Hodgson2017-01-051-0/+5
|
* s/aquire/acquire/gMark Haines2016-12-301-2/+2
|
* Add more useful logging when we block fetching eventsMark Haines2016-12-301-0/+9
|
* Send device messages over federationMark Haines2016-09-061-1/+1
|
* Only pull out IDs from DB for /state_ids/ requestErik Johnston2016-09-021-6/+4
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_apiErik Johnston2016-08-041-19/+1
|\
| * Merge branch 'develop' into rav/refactor_device_queryMark Haines2016-08-031-23/+47
| |\
| * | E2E keys: Make federation query share code with client queryRichard van der Hoff2016-08-021-19/+1
| | | | | | | | | | | | | | | Refactor the e2e query handler to separate out the local query, and then make the federation handler use it.
* | | Rename fields to _idsErik Johnston2016-08-031-2/+2
| | |
* | | Fix copy + paste failsErik Johnston2016-08-031-1/+1
| | |
* | | Add /state_ids federation APIErik Johnston2016-08-031-0/+21
| |/ |/| | | | | | | The new API only returns the event_ids for the state, as most requesters will already have the vast majority of the events already.
* | Don't double wrap 200Erik Johnston2016-08-021-2/+2
| |
* | Fix response cacheErik Johnston2016-08-021-1/+1
| |
* | Cache federation state responsesErik Johnston2016-07-211-21/+45
|/
* Linearize some federation endpoints based on (origin, room_id)Erik Johnston2016-06-171-66/+77
|
* Only re-sign our own eventsErik Johnston2016-06-171-6/+9
|
* Linearize fetching of gaps on incoming eventsErik Johnston2016-06-151-37/+51
| | | | | This potentially stops the server from doing multiple requests for the same data.
* Add some logging for when servers ask for missing eventsErik Johnston2016-06-081-0/+19
|
* Add an openidish mechanism for proving to third parties that you own a given ↵Mark Haines2016-05-051-0/+5
| | | | user_id
* Remove unused backfilled parameter from persist_eventMark Haines2016-03-211-1/+0
|
* Catch exceptions from EDU handlingErik Johnston2016-03-181-1/+6
|
* Yield on EDU handlingErik Johnston2016-03-181-3/+4
|
* Allow third_party_signed to be specified on /joinDaniel Wagner-Hall2016-02-231-2/+13
|
* Fix up logcontextsErik Johnston2016-02-081-3/+1
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Exchange 3pid invites for m.room.member invitesDaniel Wagner-Hall2015-11-051-16/+15
|
* Allow rejecting invitesDaniel Wagner-Hall2015-10-201-0/+14
| | | | | This is done by using the same /leave flow as you would use if you had already accepted the invite and wanted to leave.
* Remove unnecessary class-wrappingDaniel Wagner-Hall2015-10-131-3/+3
|
* Implement third party identifier invitesDaniel Wagner-Hall2015-10-011-3/+16
|
* Add a few strategic new lines to break up the on_query_client_keys and ↵Mark Haines2015-08-131-0/+6
| | | | on_claim_client_keys methods in federation_server.py
* Add federation support for end-to-end key requestsMark Haines2015-07-231-0/+37
|
* Change the way we do logging contexts so that they survive divergencesErik Johnston2015-05-081-24/+22
|
* Split a storage function in two so that we don't have to do extra work.Erik Johnston2015-05-011-2/+2
|
* Replace the @metrics.counted annotations in federation with ↵Paul "LeoNerd" Evans2015-03-121-12/+14
| | | | specifically-written counters and distributions
* Initial attempt at sprinkling some @metrics.counted decorations around the ↵Paul "LeoNerd" Evans2015-03-121-0/+14
| | | | federation code
* DocsErik Johnston2015-03-051-0/+2
|
* Handle if get_missing_pdu returns 400 or not all events.Erik Johnston2015-03-051-2/+4
|
* Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pduErik Johnston2015-03-021-28/+13
|\
| * If we're yielding don't add errbackErik Johnston2015-03-021-7/+3
| |
| * Process transactions serially.Erik Johnston2015-03-021-14/+10
| | | | | | | | | | | | Since the events received in a transaction are ordered, later events might depend on earlier events and so we shouldn't blindly process them in parellel.
* | Implement and use new batched get missing pduErik Johnston2015-02-231-99/+51
| |
* | Initial stab at implementing a batched get_missing_pdus requestErik Johnston2015-02-191-0/+72
|/
* Remove debug raiseErik Johnston2015-02-171-7/+0
|
* Merge branch 'develop' of github.com:matrix-org/synapse into failuresErik Johnston2015-02-171-5/+8
|\
| * Mark old events as outliers.Erik Johnston2015-02-161-1/+7
| | | | | | | | | | | | | | | | | | This is to fix the issue where if a remote server sends an event that references a really "old" event, then the local server will pull that in and send to all clients. We decide if an event is old if its depth is less than the minimum depth of the room.
| * Don't return anything from _handle_new_pdu, since we ignore the return value ↵Erik Johnston2015-02-161-4/+1
| | | | | | | | anyway
* | Format the response of transaction request in a nicer wayErik Johnston2015-02-171-4/+15
| |
* | Actually respond with JSON to incoming transactionErik Johnston2015-02-171-0/+2
| |
* | Handle recieving failures in transactionsErik Johnston2015-02-171-1/+12
| |
* | Use consumeErrors=True on all DeferredLists.Erik Johnston2015-02-171-1/+1
|/ | | | | | This is so that the DeferredLists actually consume the error instead of propogating down the non-existent errback chain. This should reduce the number of unhandled errors we are seeing.
* Correctly handle all the places that can throw exceptionsErik Johnston2015-02-121-3/+6
|
* Sign auth_chains when returned by /state/ requestsErik Johnston2015-02-101-0/+11
|
* Remove unused importsErik Johnston2015-02-031-5/+0
|
* Keep around the old (buggy) version of the prune_event function so that we ↵Erik Johnston2015-02-031-41/+11
| | | | can use it to check signatures for events on old servers
* Briefly doc structure of query_auth API.Erik Johnston2015-01-301-0/+18
|
* Fix bug where we superfluously asked for current state. Change API of ↵Erik Johnston2015-01-301-6/+6
| | | | /query_auth/ so that we don't duplicate events in the response.
* Fix regression where we no longer correctly handled the case of gaps in our ↵Erik Johnston2015-01-301-0/+3
| | | | event graph
* Initial implementation of auth conflict resolutionErik Johnston2015-01-291-0/+33
|
* Make it the responsibility of the replication layer to check signature and ↵Erik Johnston2015-01-261-13/+76
| | | | hashes.
* Split up replication_layer module into client, server and transaction queueErik Johnston2015-01-261-0/+345