summary refs log tree commit diff
path: root/synapse/federation/federation_client.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Track when the pulled event signature fails (#13815)Eric Eastwood2022-10-031-10/+40
| | | | | | | | | Because we're doing the recording in `_check_sigs_and_hash_for_pulled_events_and_fetch` (previously named `_check_sigs_and_hash_and_fetch`), this means we will track signature failures for `backfill`, `get_room_state`, `get_event_auth`, and `get_missing_events` (all pulled event scenarios). And we also record signature failures from `get_pdu`. Part of https://github.com/matrix-org/synapse/issues/13700 Part of https://github.com/matrix-org/synapse/issues/13676 and https://github.com/matrix-org/synapse/issues/13356 This PR will be especially important for https://github.com/matrix-org/synapse/pull/13816 so we can avoid the costly `_get_state_ids_after_missing_prev_event` down the line when `/messages` calls backfill.
* Don't include redundant prev_state in new events (#13791)Denis2022-09-201-3/+0
|
* Rename the `EventFormatVersions` enum values so that they line up with room ↵reivilibre2022-09-071-1/+1
| | | | version numbers. (#13706)
* Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child ↵Eric Eastwood2022-08-231-3/+20
| | | | | | | | | concurrent calls (#13588) Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls because I've see `_check_sigs_and_hash_and_fetch` take [10.41s to process 100 events](https://github.com/matrix-org/synapse/issues/13587) Fix https://github.com/matrix-org/synapse/issues/13587 Part of https://github.com/matrix-org/synapse/issues/13356
* Instrument the federation/backfill part of `/messages` (#13489)Eric Eastwood2022-08-161-1/+26
| | | | | | | | | Instrument the federation/backfill part of `/messages` so it's easier to follow what's going on in Jaeger when viewing a trace. Split out from https://github.com/matrix-org/synapse/pull/13440 Follow-up from https://github.com/matrix-org/synapse/pull/13368 Part of https://github.com/matrix-org/synapse/issues/13356
* Instrument `/messages` for understandable traces in Jaeger (#13368)Eric Eastwood2022-08-031-0/+2
| | | | | | In Jaeger: - Before: huge list of uncategorized database calls - After: nice and collapsible into units of work
* Make minor clarifications to the error messages given when we fail to join a ↵reivilibre2022-07-271-1/+7
| | | | room via any server. (#13160)
* Fix `get_pdu` asking every remote destination even after it finds an event ↵Eric Eastwood2022-07-271-3/+3
| | | | (#13346)
* Update `get_pdu` to return the original, pristine `EventBase` (#13320)Eric Eastwood2022-07-201-42/+81
| | | | | | | | | | | | Update `get_pdu` to return the untouched, pristine `EventBase` as it was originally seen over federation (no metadata added). Previously, we returned the same `event` reference that we stored in the cache which downstream code modified in place and added metadata like setting it as an `outlier` and essentially poisoned our cache. Now we always return a copy of the `event` so the original can stay pristine in our cache and re-used for the next cache call. Split out from https://github.com/matrix-org/synapse/pull/13205 As discussed at: - https://github.com/matrix-org/synapse/pull/13205#discussion_r918365746 - https://github.com/matrix-org/synapse/pull/13205#discussion_r918366125 Related to https://github.com/matrix-org/synapse/issues/12584. This PR doesn't fix that issue because it hits [`get_event` which exists from the local database before it tries to `get_pdu`](https://github.com/matrix-org/synapse/blob/7864f33e286dec22368dc0b11c06eebb1462a51e/synapse/federation/federation_client.py#L581-L594).
* Add type annotations to `trace` decorator. (#13328)Patrick Cloke2022-07-191-1/+1
| | | | Functions that are decorated with `trace` are now properly typed and the type hints for them are fixed.
* Stop depending on `room_id` to be returned for children state in the ↵Patrick Cloke2022-06-101-4/+0
| | | | | | | | | | | hierarchy response. (#12991) The `room_id` field was removed from MSC2946 before it was accepted. It was initially kept for backwards compatibility and should be removed now that the stable form of the API is used. This change only stops Synapse from validating that it is returned, a future PR will remove returning it as part of the response.
* Improve logging when signature checks fail (#12925)Richard van der Hoff2022-05-311-13/+32
| | | | | | | | | | | | | * Raise a dedicated `InvalidEventSignatureError` from `_check_sigs_on_pdu` * Downgrade logging about redactions to DEBUG this can be very spammy during a room join, and it's not very useful. * Raise `InvalidEventSignatureError` from `_check_sigs_and_hash` ... and, more importantly, move the logging out to the callers. * changelog
* Faster room joins: Try other destinations when resyncing the state of a ↵Sean Quah2022-05-311-1/+4
| | | | | | | partial-state room (#12812) Signed-off-by: Sean Quah <seanq@matrix.org>
* Fix inconsistent spelling of 'M_UNRECOGNIZED'. (#12665)Val Lorentz2022-05-091-1/+1
|
* Support MSC3266 room summaries over federation (#11507)DeepBlueV7.X2022-05-051-0/+2
| | | | Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* Unify HTTP query parameter type hints (#12415)David Robertson2022-04-081-1/+2
| | | | | | * Pull out query param types to `synapse.http.types` * Use QueryParams everywhere * Simplify `encode_query_args` * Add annotation which would have caught #12410
* Check if instances are lists, not sequences. (#12128)Patrick Cloke2022-03-021-4/+4
| | | | | As a str is a sequence, the checks were not granular enough and would allow lists or strings, when only lists were valid.
* Remove the unstable `/spaces` endpoint. (#12073)Patrick Cloke2022-02-281-194/+32
| | | | | | | | ...and various code supporting it. The /spaces endpoint was from an old version of MSC2946 and included both a Client-Server and Server-Server API. Note that the unstable /hierarchy endpoint (from the final version of MSC2946) is not yet removed.
* Properly failover for unknown endpoints from Conduit/Dendrite. (#12077)Patrick Cloke2022-02-281-9/+13
| | | | | Before this fix, a legitimate 404 from a federation endpoint (e.g. due to an unknown room) would be treated as an unknown endpoint. This could cause unnecessary federation traffic.
* Implement account status endpoints (MSC3720) (#12001)Brendan Abolivier2022-02-221-1/+59
| | | | | See matrix-org/matrix-doc#3720 Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Faster joins: Support for calling `/federation/v1/state` (#12013)Richard van der Hoff2022-02-221-12/+81
| | | | This is an endpoint that we have server-side support for, but no client-side support. It's going to be useful for resyncing partial-stated rooms, so let's introduce it.
* remote join processing: get create event from state, not auth_chain (#12039)Richard van der Hoff2022-02-211-2/+4
| | | A follow-up to #12005, in which I apparently missed that there are a bunch of other places that assume the create event is in the auth chain.
* Faster joins: parse msc3706 fields in send_join response (#12011)Richard van der Hoff2022-02-171-1/+14
| | | Part of my work on #11249: add code to handle the new fields added in MSC3706.
* Fix a bug that corrupted the cache of federated space hierarchies (#11775)Sean Quah2022-01-201-9/+9
| | | | `FederationClient.get_room_hierarchy()` caches its return values, so refactor the code to avoid modifying the returned room summary.
* Remove `log_function` and its uses (#11761)Richard van der Hoff2022-01-181-5/+0
| | | | | | | I've never found this terribly useful. I think it was added in the early days of Synapse, without much thought as to what would actually be useful to log, and has just been cargo-culted ever since. Rather, it tends to clutter up debug logs with useless information.
* Refactor the way we set `outlier` (#11634)Richard van der Hoff2022-01-051-31/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * `_auth_and_persist_outliers`: mark persisted events as outliers Mark any events that get persisted via `_auth_and_persist_outliers` as, well, outliers. Currently this will be a no-op as everything will already be flagged as an outlier, but I'm going to change that. * `process_remote_join`: stop flagging as outlier The events are now flagged as outliers later on, by `_auth_and_persist_outliers`. * `send_join`: remove `outlier=True` The events created here are returned in the result of `send_join` to `FederationHandler.do_invite_join`. From there they are passed into `FederationEventHandler.process_remote_join`, which passes them to `_auth_and_persist_outliers`... which sets the `outlier` flag. * `get_event_auth`: remove `outlier=True` stop flagging the events returned by `get_event_auth` as outliers. This method is only called by `_get_remote_auth_chain_for_event`, which passes the results into `_auth_and_persist_outliers`, which will flag them as outliers. * `_get_remote_auth_chain_for_event`: remove `outlier=True` we pass all the events into `_auth_and_persist_outliers`, which will now flag the events as outliers. * `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter This param is now never set to True, so we can remove it. * `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param This is no longer set anywhere, so we can remove it. * `get_pdu`: remove unused `outlier` parameter ... and chase it down into `get_pdu_from_destination_raw`. * `event_from_pdu_json`: remove redundant `outlier` param This is never set to `True`, so can be removed. * changelog * update docstring
* `FederationClient.backfill`: stop flagging events as outliers (#11632)Richard van der Hoff2022-01-041-1/+1
| | | | | | | | | | | | | | | | | | | | | Events returned by `backfill` should not be flagged as outliers. Fixes: ``` AssertionError: null File "synapse/handlers/federation.py", line 313, in try_backfill dom, room_id, limit=100, extremities=extremities File "synapse/handlers/federation_event.py", line 517, in backfill await self._process_pulled_events(dest, events, backfilled=True) File "synapse/handlers/federation_event.py", line 642, in _process_pulled_events await self._process_pulled_event(origin, ev, backfilled=backfilled) File "synapse/handlers/federation_event.py", line 669, in _process_pulled_event assert not event.internal_metadata.is_outlier() ``` See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/231992 Fixes #8894.
* Add most of the missing type hints to `synapse.federation`. (#11483)Patrick Cloke2021-12-021-2/+2
| | | This skips a few methods which are difficult to type.
* Add MSC3030 experimental client and federation API endpoints to get the ↵Eric Eastwood2021-12-021-0/+77
| | | | | | | | | | | | | | | | | | | | | | | | | closest event to a given timestamp (#9445) MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030 Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about. ``` GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Federation API endpoint: ``` GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction> { "event_id": ... "origin_server_ts": ... } ``` Co-authored-by: Erik Johnston <erik@matrix.org>
* Support the stable /hierarchy endpoint from MSC2946 (#11329)Patrick Cloke2021-11-291-5/+26
| | | | | | This also makes additional updates where the implementation had drifted from the approved MSC. Unstable endpoints will be removed at a later data.
* Split out federated PDU retrieval into a non-cached version (#11242)Eric Eastwood2021-11-091-22/+58
| | | | Context: https://github.com/matrix-org/synapse/pull/11114/files#r741643968
* Annotate `log_function` decorator (#10943)reivilibre2021-10-271-2/+15
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Strip "join_authorised_via_users_server" from join events which do not need ↵Patrick Cloke2021-09-301-3/+3
| | | | | | | it. (#10933) This fixes a "Event not signed by authorising server" error when transition room member from join -> join, e.g. when updating a display name or avatar URL for restricted rooms.
* Factor out common code for persisting fetched auth events (#10896)Richard van der Hoff2021-09-241-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Factor more stuff out of `_get_events_and_persist` It turns out that the event-sorting algorithm in `_get_events_and_persist` is also useful in other circumstances. Here we move the current `_auth_and_persist_fetched_events` to `_auth_and_persist_fetched_events_inner`, and then factor the sorting part out to `_auth_and_persist_fetched_events`. * `_get_remote_auth_chain_for_event`: remove redundant `outlier` assignment `get_event_auth` returns events with the outlier flag already set, so this is redundant (though we need to update a test where `get_event_auth` is mocked). * `_get_remote_auth_chain_for_event`: move existing-event tests earlier Move a couple of tests outside the loop. This is a bit inefficient for now, but a future commit will make it better. It should be functionally identical. * `_get_remote_auth_chain_for_event`: use `_auth_and_persist_fetched_events` We can use the same codepath for persisting the events fetched as part of an auth chain as for those fetched individually by `_get_events_and_persist` for building the state at a backwards extremity. * `_get_remote_auth_chain_for_event`: use a dict for efficiency `_auth_and_persist_fetched_events` sorts the events itself, so we no longer need to care about maintaining the ordering from `get_event_auth` (and no longer need to sort by depth in `get_event_auth`). That means that we can use a map, making it easier to filter out events we already have, etc. * changelog * `_auth_and_persist_fetched_events`: improve docstring
* Cache the result of fetching the room hierarchy over federation. (#10647)Patrick Cloke2021-08-261-40/+66
|
* Do not include stack traces for known exceptions when trying multiple ↵Patrick Cloke2021-08-231-1/+6
| | | | federation destinations. (#10662)
* Attempt to pull from the legacy spaces summary API over federation. (#10583)Patrick Cloke2021-08-171-9/+55
| | | | | | | If the new /hierarchy API does not exist on all destinations, fallback to querying the /spaces API and translating the results. This is a backwards compatibility hack since not all of the federated homeservers will update at the same time.
* Support federation in the new spaces summary API (MSC2946). (#10569)Patrick Cloke2021-08-161-0/+82
|
* Fix exceptions in logs when failing to get remote room list (#10541)Erik Johnston2021-08-061-1/+2
|
* Refactoring before implementing the updated spaces summary. (#10527)Patrick Cloke2021-08-051-9/+14
| | | | | This should have no user-visible changes, but refactors some pieces of the SpaceSummaryHandler before adding support for the updated MSC2946.
* Improve failover logic for MSC3083 restricted rooms. (#10447)Patrick Cloke2021-07-291-4/+39
| | | | | If the federation client receives an M_UNABLE_TO_AUTHORISE_JOIN or M_UNABLE_TO_GRANT_JOIN response it will attempt another server before giving up completely.
* Update the MSC3083 support to verify if joins are from an authorized server. ↵Patrick Cloke2021-07-261-12/+49
| | | | (#10254)
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-5/+5
|
* Remove the experimental flag for knocking and use stable prefixes / ↵Patrick Cloke2021-06-151-7/+2
| | | | | | | endpoints. (#10167) * Room version 7 for knocking. * Stable prefixes and endpoints (both client and federation) for knocking. * Removes the experimental configuration flag.
* Implement knock feature (#6739)Sorunome2021-06-091-1/+71
| | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io
* When joining a remote room limit the number of events we concurrently check ↵Erik Johnston2021-06-081-54/+93
| | | | | signatures/hashes for (#10117) If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
* Use ijson to parse the response to `/send_join`, reducing memory usage. (#9958)Erik Johnston2021-05-201-18/+10
| | | Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
* Pass errors back to the client when trying multiple federation destinations. ↵Patrick Cloke2021-04-271-58/+60
| | | | | | | | (#9868) This ensures that something like an auth error (403) will be returned to the requester instead of attempting to try more servers, which will likely result in the same error, and then passing back a generic 400 error.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Add type hints to expiring cache. (#9730)Patrick Cloke2021-04-061-1/+1
|
* Spaces summary: call out to other servers (#9653)Richard van der Hoff2021-03-241-10/+170
| | | | | When we hit an unknown room in the space tree, see if there are other servers that we might be able to poll to get the data. Fixes: #9447
* Import HomeServer from the proper module. (#9665)Patrick Cloke2021-03-231-1/+1
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-1/+5
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Ratelimit invites by room and target user (#9258)Erik Johnston2021-01-291-1/+1
|
* Various improvements to the federation client. (#9129)Patrick Cloke2021-01-201-58/+67
| | | | * Type hints for `FederationClient`. * Using `async` functions instead of returning `Awaitable` instances.
* Mypy fixes for `synapse.handlers.federation` (#8422)Richard van der Hoff2020-09-291-1/+3
| | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
* Merge tag 'v1.20.0rc5' into developPatrick Cloke2020-09-181-5/+3
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.20.0rc5 (2020-09-18) ============================== In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3. Features -------- - Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343)) Bugfixes -------- - Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342)) - Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349)) Internal Changes ---------------- - Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285))
| * Use _check_sigs_and_hash_and_fetch to validate backfill requests (#8350)Andrew Morgan2020-09-181-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a bit of a hack, as `_check_sigs_and_hash_and_fetch` is intended for attempting to pull an event from the database/(re)pull it from the server that originally sent the event if checking the signature of the event fails. During backfill we *know* that we won't have the event in our database, however it is still useful to be able to query the original sending server as the server we're backfilling from may be acting maliciously. The main benefit and reason for this change however is that `_check_sigs_and_hash_and_fetch` will drop an event during backfill if it cannot be successfully validated, whereas the current code will simply fail the backfill request - resulting in the client's /messages request silently being dropped. This is a quick patch to fix backfilling rooms that contain malformed events. A better implementation in planned in future.
* | Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-181-1/+1
|/ | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Don't assume that an event has an origin fieldErik Johnston2020-09-161-3/+4
| | | | This fixes #8319.
* Convert federation client to async/await. (#7975)Patrick Cloke2020-07-301-4/+4
|
* Convert synapse.app to async/await. (#7868)Patrick Cloke2020-07-171-21/+19
|
* Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-091-2/+2
|
* Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-1/+1
| | | ... instead of duplicating `config.signing_key[0]` everywhere
* async/await is_server_admin (#7363)Andrew Morgan2020-05-011-3/+2
|
* Further improvements to requesting the public rooms list on a homeserver ↵Andrew Morgan2020-05-011-9/+28
| | | | which has it set to private (#7368)
* Fix a bug in the federation API which could cause occasional "Failed to get ↵Patrick Cloke2020-03-191-11/+8
| | | | PDU" errors (#7089).
* Remove unused federation endpoint (`query_auth`) (#7026)Patrick Cloke2020-03-171-1/+79
|
* Add some type annotations to the federation base & client classes (#6995)Patrick Cloke2020-02-281-5/+5
|
* Pass room_version into `event_from_pdu_json`Richard van der Hoff2020-02-061-19/+16
| | | | It's called from all over the shop, so this one's a bit messy.
* pass room version into FederationClient.send_join (#6854)Richard van der Hoff2020-02-061-28/+32
| | | | ... which allows us to sanity-check the create event.
* Simplify `room_version` handling in `FederationClient.send_invite`Richard van der Hoff2020-02-051-10/+6
|
* Pass room version object into `FederationClient.get_pdu`Richard van der Hoff2020-02-051-3/+5
|
* make FederationClient._do_send_invite asyncRichard van der Hoff2020-02-051-10/+7
|
* Apply suggestions from code reviewRichard van der Hoff2020-02-051-2/+4
| | | Co-Authored-By: Erik Johnston <erik@matrix.org>
* make FederationClient.send_invite asyncRichard van der Hoff2020-02-031-6/+5
|
* make FederationClient.get_missing_events asyncRichard van der Hoff2020-02-031-20/+20
|
* make FederationClient._do_send_leave asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.send_leave.send_request asyncRichard van der Hoff2020-02-031-5/+2
|
* make FederationClient._do_send_join asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.send_join.send_request asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.make_membership_event.send_request asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient._try_destination_list asyncRichard van der Hoff2020-02-031-10/+26
|
* make FederationClient.send_leave asyncRichard van der Hoff2020-02-031-10/+9
|
* make FederationClient.send_join asyncRichard van der Hoff2020-02-031-11/+13
|
* make FederationClient.make_membership_event asyncRichard van der Hoff2020-02-031-10/+11
|
* make FederationClient.get_event_auth asyncRichard van der Hoff2020-02-031-6/+4
|
* make FederationClient.get_room_state_ids asyncRichard van der Hoff2020-02-031-4/+5
|
* make FederationClient.get_pdu asyncRichard van der Hoff2020-02-031-15/+17
|
* make FederationClient.backfill asyncRichard van der Hoff2020-02-031-15/+11
|
* Merge pull request #6806 from matrix-org/rav/redact_changes/3Richard van der Hoff2020-01-311-3/+1
|\ | | | | Pass room_version into add_hashes_and_signatures
| * Pass room_version into create_local_event_from_event_dictRichard van der Hoff2020-01-301-3/+1
| |
* | s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-5/+5
|/ | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* Add `rooms.room_version` column (#6729)Erik Johnston2020-01-271-17/+33
| | | This is so that we don't have to rely on pulling it out from `current_state_events` table.
* Merge pull request #6349 from matrix-org/babolivier/msc1802Brendan Abolivier2019-12-111-12/+76
|\ | | | | Implement v2 APIs for send_join and send_leave
| * Merge branch 'develop' into babolivier/msc1802Brendan Abolivier2019-12-051-3/+3
| |\
| * | LintBrendan Abolivier2019-11-111-3/+2
| | |
| * | Implement v2 API for send_leaveBrendan Abolivier2019-11-111-4/+37
| | |
| * | Add missing yieldBrendan Abolivier2019-11-111-1/+1
| | |
| * | Implement v2 API for send_joinBrendan Abolivier2019-11-111-7/+39
| | |
* | | Move get_state methods into FederationHandler (#6503)Richard van der Hoff2019-12-101-80/+11
| | | | | | | | | | | | | | | This is a non-functional refactor as a precursor to some other work.
* | | Refactor get_events_from_store_or_dest to return a dict (#6501)Richard van der Hoff2019-12-101-29/+15
| | | | | | | | | | | | | | | There was a bunch of unnecessary conversion back and forth between dict and list going on here. We can simplify a bunch of the code.
* | | Remove fallback for missing /federation/v1/state_ids API (#6488)Richard van der Hoff2019-12-091-72/+17
| |/ |/| | | | | This API was added way back in 0.17.0; the code here is annoying to maintain and entirely redundant.
* | Replace instance variations of homeserver with correct case/spacingAndrew Morgan2019-11-121-3/+3
|/
* Depublish a room from the public rooms list when it is upgraded (#6232)Andrew Morgan2019-11-011-1/+1
|
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-3/+5
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* use %r to __repr__ objectsMichael Kaye2019-10-241-1/+1
| | | | This avoids calculating __repr__ unless we are going to log.
* rip out some unreachable codeRichard van der Hoff2019-10-171-38/+0
| | | | The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
* Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-4/+4
| | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* Return 502 not 500 when failing to reach any remote server.Erik Johnston2019-08-011-4/+3
|
* Room Complexity Client Implementation (#5783)Amber Brown2019-07-301-0/+36
|
* Replace returnValue with return (#5736)Amber Brown2019-07-231-24/+22
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-4/+4
|
* Run Black. (#5482)Amber Brown2019-06-201-109/+108
|
* Clean up debug logging (#5347)Richard van der Hoff2019-06-051-3/+13
| | | Remove some spurious stuff, clarify some other stuff
* Fix failure to fetch batches of PDUs (#5342)Richard van der Hoff2019-06-041-0/+1
| | | | | FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A failure to fetch one should not result in a failure of the whole batch. Add the missing `continue`.
* Rename get_events->get_events_from_store_or_dest (#5344)Richard van der Hoff2019-06-041-20/+13
| | | | | We have too many things called get_event, and it's hard to figure out what we mean. Also remove some unused params from the signature, and add some logging.
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-011-14/+15
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite APIErik Johnston2019-02-231-1/+17
|
* Only check event ID domain for signatures for V1 eventsErik Johnston2019-01-291-3/+3
| | | | | In future version events won't have an event ID, so we won't be able to do this check.
* Merge pull request #4481 from matrix-org/erikj/event_builderErik Johnston2019-01-291-14/+6
|\ | | | | Refactor event building into EventBuilder
| * Refactor event building into EventBuilderErik Johnston2019-01-291-14/+6
| | | | | | | | | | This is so that everything is done in one place, making it easier to change the event format based on room version
* | Implement fallback for V2 invite APIErik Johnston2019-01-281-12/+52
|/ | | | | If the room version is either 1 or 2 then a server should retry failed `/v2/invite` requests with the v1 API
* Require event format version to parse or create eventsErik Johnston2019-01-251-23/+37
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813Erik Johnston2019-01-251-9/+53
|\
| * Merge pull request #4448 from matrix-org/erikj/get_pdu_versionsErik Johnston2019-01-241-7/+40
| |\ | | | | | | Add room_version param to get_pdu
| | * Review commentsErik Johnston2019-01-241-1/+2
| | |
| | * Add room_version param to get_pduErik Johnston2019-01-231-7/+39
| | | | | | | | | | | | | | | When we add new event format we'll need to know the event format or room version when parsing events.
| * | Update make_membership_event docsErik Johnston2019-01-241-2/+5
| | |
| * | Remove unnecessary '_sign_event'Erik Johnston2019-01-231-1/+8
| | |
| * | Store rejected remote invite events as outliersErik Johnston2019-01-231-2/+8
| |/ | | | | | | | | | | | | | | | | | | | | Currently they're stored as non-outliers even though the server isn't in the room, which can be problematic in places where the code assumes it has the state for all non outlier events. In particular, there is an edge case where persisting the leave event triggers a state resolution, which requires looking up the room version from state. Since the server doesn't have the state, this causes an exception to be thrown.
* | Review commentsErik Johnston2019-01-241-1/+2
| |
* | Implement MSC 1813 - Add room version to make APIsErik Johnston2019-01-231-5/+16
|/ | | | | We also implement `make_membership_event` converting the returned room version to an event format version.
* more commentsRichard van der Hoff2018-09-271-2/+0
|
* fix docstring for FederationClient.get_state_for_roomRichard van der Hoff2018-09-261-4/+4
| | | | trivial fixes for docstring
* Fix some instances of ExpiringCache not expiring cache itemsErik Johnston2018-09-211-16/+12
| | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.
* Port federation/ to py3 (#3847)Amber Brown2018-09-121-4/+4
|
* Check m.room.create for sane room_versionsRichard van der Hoff2018-08-061-3/+23
|
* include known room versions in outgoing make_joinsRichard van der Hoff2018-08-061-3/+5
|
* sanity check response from make_joinRichard van der Hoff2018-08-031-1/+3
|
* Refactor code for turning HttpResponseException into SynapseErrorRichard van der Hoff2018-08-011-2/+2
| | | | | | | | This commit replaces SynapseError.from_http_response_exception with HttpResponseException.to_synapse_error. The new method actually returns a ProxiedRequestError, which allows us to pass through additional metadata from the API call.
* Be more careful which errors we send back over the C-S APIRichard van der Hoff2018-08-011-12/+17
| | | | | | | | | | | | We really shouldn't be sending all CodeMessageExceptions back over the C-S API; it will include things like 401s which we shouldn't proxy. That means that we need to explicitly turn a few HttpResponseExceptions into SynapseErrors in the federation layer. The effect of the latter is that the matrix errcode will get passed through correctly to calling clients, which might help with some of the random M_UNKNOWN errors when trying to join rooms.
* Factor out exception handling in federation_clientRichard van der Hoff2018-08-011-129/+148
| | | | | Factor out the error handling from make_membership_event, send_join, and send_leave, so that it can be shared.
* run isortAmber Brown2018-07-091-7/+7
|
* Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-2/+2
| | | | they're not meant to be lazy (#3307)
* replacing portionsAmber Brown2018-05-211-10/+7
|
* Merge branch 'develop' into py3-xrange-1Richard van der Hoff2018-04-301-2/+3
|\
| * Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-2/+3
| | | | | | | | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* | Move more xrange to sixAdrian Tschira2018-04-281-1/+3
|/ | | | | | plus a bonus next() Signed-off-by: Adrian Tschira <nota@notafile.com>
* Refactor store.have_eventsRichard van der Hoff2018-04-201-1/+1
| | | | | | It turns out that most of the time we were calling have_events, we were only using half of the result. Replace have_events with have_seen_events and get_rejection_reasons, so that we can see what's going on a bit more clearly.
* Move property setting from ReplicationLayer to FederationBaseErik Johnston2018-03-131-0/+1
|
* Add federation_domain_whitelist option (#2820)Matthew Hodgson2018-01-221-1/+4
| | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
* Factor out `event_from_pdu_json`Richard van der Hoff2017-12-301-21/+15
| | | | | turns out we have two copies of this, and neither needs to be an instance method
* federation_client: clean up importsRichard van der Hoff2017-12-301-13/+10
|
* Remove preserve_context_over_{fn, deferred}Richard van der Hoff2017-11-141-2/+2
| | | | | Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off.
* Fix logcontexts in _check_sigs_and_hashesRichard van der Hoff2017-09-201-4/+4
|
* Fix some lies, and other clarifications, in docstringsRichard van der Hoff2017-04-211-2/+48
| | | | | The documentation on get_json has been wrong ever since the very first commit to synapse...
* Ignore backoff history for invites, aliases, and roomdirsRichard van der Hoff2017-03-231-2/+5
| | | | | Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.
* push federation retry limiter down to matrixfederationclientRichard van der Hoff2017-03-231-20/+13
| | | | | rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.
* Fix docstringRichard van der Hoff2017-03-091-2/+1
|
* Implement device key caching over federationErik Johnston2017-01-261-0/+10
|
* Fix couple of federation state bugsErik Johnston2017-01-171-2/+4
|
* limit total timeout for get_missing_events to 10sMatthew Hodgson2016-12-311-1/+3
|
* Remove fallback from get_missing_events.Mark Haines2016-12-301-69/+0
| | | | | | | get_missing_events used to fallback to fetching the missing events individually requesting from every server in the room, one by one.e This could be unacceptably slow, possibly causing #1732
* Add new API appservice specific public room listErik Johnston2016-12-061-2/+5
|
* Use new federation_sender DIErik Johnston2016-11-161-49/+0
|
* Add transaction queue and transport layer to DIErik Johnston2016-11-161-1/+0
|
* Move logic into transaction_queueErik Johnston2016-11-161-14/+2
|
* Rename transaction queue functions to send_*Erik Johnston2016-11-161-7/+7
|
* Time out typing over federationErik Johnston2016-09-231-2/+0
|
* Support filtering remote room listsErik Johnston2016-09-161-2/+5
|
* Pass since/from parameters over federationErik Johnston2016-09-151-18/+4
|
* Merge pull request #1110 from matrix-org/markjh/e2e_timeoutMark Haines2016-09-131-4/+8
|\ | | | | Add a timeout parameter for end2end key queries.
| * Add a timeout parameter for end2end key queries.Mark Haines2016-09-121-4/+8
| | | | | | | | | | | | | | | | | | | | Add a timeout parameter for controlling how long synapse will wait for responses from remote servers. For servers that fail include how they failed to make it easier to debug. Fetch keys from different servers in parallel rather than in series. Set the default timeout to 10s.
* | Fix backfill when cannot find an event.Erik Johnston2016-09-121-1/+1
|/ | | | `get_pdu` can succeed but return None.
* Clobber EDUs in send queueErik Johnston2016-09-091-2/+6
|
* Add a new method to enqueue the device messages rather than sending a dummy EDUMark Haines2016-09-071-0/+6
|
* Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into developErik Johnston2016-09-011-5/+5
|\
| * Ensure we only return a validated pdu in get_pduErik Johnston2016-09-011-5/+5
| |
* | Use state handler instead of get_users_in_room/get_joined_hostsErik Johnston2016-08-261-1/+4
|/
* Preserve some logcontextsErik Johnston2016-08-241-6/+11
|
* Don't stop on 4xx series errorsErik Johnston2016-08-101-11/+1
|
* Also pull out rejected eventsErik Johnston2016-08-101-1/+1
|
* Store if we fail to fetch an event from a destinationErik Johnston2016-08-101-1/+36
|
* Fix a couple of python bugsErik Johnston2016-08-051-6/+8
|
* Retry joining via other servers if first one failedErik Johnston2016-08-051-5/+16
|
* Fix typoErik Johnston2016-08-051-1/+1
|
* Check if we already have the events returned by /state/Erik Johnston2016-08-051-2/+16
|
* Fallback to /state/ on both 400 and 404Erik Johnston2016-08-051-1/+1
|
* Tweak integrity error recovery to work as intendedBenjamin Saunders2016-08-041-1/+1
|
* Fix typos.Erik Johnston2016-08-041-3/+3
|
* Tidy up get_eventsErik Johnston2016-08-041-12/+10
|
* Rename fields to _idsErik Johnston2016-08-031-2/+2
|
* Fix copy + paste failsErik Johnston2016-08-031-5/+10
|
* Actually call get_room_stateErik Johnston2016-08-031-0/+4
|
* Fix syntax errorErik Johnston2016-08-031-1/+8
|
* CommentErik Johnston2016-08-031-0/+14
|
* Add /state_ids federation APIErik Johnston2016-08-031-3/+70
| | | | | The new API only returns the event_ids for the state, as most requesters will already have the vast majority of the events already.
* Linearize fetching of gaps on incoming eventsErik Johnston2016-06-151-0/+2
| | | | | This potentially stops the server from doing multiple requests for the same data.
* Apparently I am not permitted to have two blank lines hereDavid Baker2016-05-311-1/+0
|
* Basic, un-cached support for secondary_directory_serversDavid Baker2016-05-311-0/+21
|
* Intern all the thingsErik Johnston2016-03-231-0/+1
|
* Add profile information to invitesErik Johnston2016-03-041-1/+1
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-1/+1
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Don't complain if /make_join response lacks 'prev_state' list (SYN-517)Paul "LeoNerd" Evans2015-11-131-0/+6
|
* Fix missing profile data in federation joinsErik Johnston2015-11-121-1/+4
| | | | | There was a regression where we stopped including profile data in initial joins for rooms joined over federation.
* Exchange 3pid invites for m.room.member invitesDaniel Wagner-Hall2015-11-051-8/+25
|
* Allow rejecting invitesDaniel Wagner-Hall2015-10-201-6/+61
| | | | | This is done by using the same /leave flow as you would use if you had already accepted the invite and wanted to leave.
* Look for keys on the right objectsDaniel Wagner-Hall2015-10-151-1/+3
|
* Remove unnecessary class-wrappingDaniel Wagner-Hall2015-10-131-3/+3
|
* Move event contents into third_party_layout fieldDaniel Wagner-Hall2015-10-131-2/+2
|
* Fix lint errorsDaniel Wagner-Hall2015-10-061-1/+0
|
* Implement third party identifier invitesDaniel Wagner-Hall2015-10-011-2/+7
|
* Merge pull request #208 from matrix-org/markjh/end-to-end-key-federationMark Haines2015-08-181-0/+30
|\ | | | | Federation for end-to-end key requests.
| * Get the end-to-end key federation workingMark Haines2015-07-241-8/+4
| |
| * Add federation support for end-to-end key requestsMark Haines2015-07-231-0/+34
| |
* | Move all the caches into their own package, synapse.util.cachesErik Johnston2015-08-111-1/+1
|/
* Implement bulk verify_signed_json APIErik Johnston2015-06-261-14/+43
|
* Remove redundant for loopErik Johnston2015-05-221-3/+0
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2015-05-221-0/+2
|\ | | | | | | erikj/backfill_fixes
| * Add doc stringsErik Johnston2015-05-221-0/+2
| |
* | Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into ↵Erik Johnston2015-05-211-3/+3
|\| | | | | | | erikj/backfill_fixes
| * Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perfErik Johnston2015-05-211-3/+3
| |\
| | * Remove pointless newlineErik Johnston2015-05-181-1/+0
| | |
| | * Remove debug loggingErik Johnston2015-05-181-8/+0
| | |
| | * Fix daedlockErik Johnston2015-05-151-3/+12
| | |
* | | Actually fetch state for new backwards extremeties when backfilling.Erik Johnston2015-05-201-1/+5
|/ /
* | Add None checkErik Johnston2015-05-191-1/+1
| |
* | Add a timeout param to get_eventErik Johnston2015-05-191-9/+14
|/
* Missed events are not outliersErik Johnston2015-05-051-1/+1
|
* Replace the @metrics.counted annotations in federation with ↵Paul "LeoNerd" Evans2015-03-121-12/+15
| | | | specifically-written counters and distributions
* Initial attempt at sprinkling some @metrics.counted decorations around the ↵Paul "LeoNerd" Evans2015-03-121-0/+14
| | | | federation code
* Delete pointless (and unreachable) __init__ method from FederationClientPaul "LeoNerd" Evans2015-03-121-2/+0
|
* Remove unecessary checkErik Johnston2015-03-051-6/+0
|
* Respect min_depth argumentErik Johnston2015-03-051-4/+6
|
* Docs.Erik Johnston2015-03-051-0/+16
|
* Handle if get_missing_pdu returns 400 or not all events.Erik Johnston2015-03-051-13/+96
|
* Implement and use new batched get missing pduErik Johnston2015-02-231-0/+19
|
* Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into ↵Erik Johnston2015-02-181-12/+25
|\ | | | | | | develop
| * Add per server retry limiting.Erik Johnston2015-02-171-12/+24
| | | | | | | | | | Factor out the pre destination retry logic from TransactionQueue so it can be reused in both get_pdu and crypto.keyring
* | Rename _fail_fetch_pdu_cache to _get_pdu_cacheErik Johnston2015-02-181-8/+8
| |
* | Cache results of get_pdu.Erik Johnston2015-02-161-3/+39
|/
* Connection errors in twisted aren't RuntimeErrorsErik Johnston2015-02-051-2/+2
|
* Pass through list of room hosts from room alias query to federation so that ↵Erik Johnston2015-02-051-1/+4
| | | | it can retry against different room hosts
* Apply sanity to the transport client interface. Convert 'make_join' and ↵Erik Johnston2015-02-041-31/+51
| | | | 'send_join' to accept iterables of destinations
* Retry make_joinErik Johnston2015-02-041-7/+12
|
* Keep around the old (buggy) version of the prune_event function so that we ↵Erik Johnston2015-02-031-94/+2
| | | | can use it to check signatures for events on old servers
* PEP8Erik Johnston2015-02-031-1/+0
|
* Don't bother requesting PDUs with bad signatures from the same serverErik Johnston2015-02-021-16/+17
|