summary refs log tree commit diff
path: root/synapse/federation (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge commit '1c9a85056' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-211-1/+1
|\
| * Add type hints to the crypto module. (#8999)Patrick Cloke2021-01-041-1/+1
| |
* | Merge commit 'f14428b25' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-161-1/+6
|\|
| * Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-1/+6
| | | | | | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* | Merge commit 'cf7d3c90d' into dinsicAndrew Morgan2021-04-161-1/+1
|\|
| * Merge pull request #8858 from matrix-org/rav/sso_uiaRichard van der Hoff2020-12-021-1/+1
| |\ | | | | | | UIA: offer only available auth flows
| | * fix up various test casesRichard van der Hoff2020-12-021-1/+1
| | | | | | | | | | | | | | | A few test cases were relying on being able to mount non-client servlets on the test resource. it's better to give them their own Resources.
| * | Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-022-2/+1
| |/ | | | | | | | | | | | | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* | Port "Add support for no_proxy and case insensitive env variables" from ↵Andrew Morgan2021-03-222-2/+1
| | | | | | | | | | | | | | mainline to dinsic (#93) This PR is simply porting https://github.com/matrix-org/synapse/pull/9372 to dinsic. I also had to bring in https://github.com/matrix-org/synapse/pull/8821 and https://github.com/matrix-org/synapse/pull/9084 for this code to work properly - a sign that we should merge mainline into dinsic again soon.
* | Send a ver query parameter for make_knock (#83)Andrew Morgan2021-02-172-4/+18
| | | | | | | | | | | | | | | | | | This informs the remote server of the room versions we support. If the room we're trying to knock on has a version that is not one of our supported room versions, the remote server will return an unsupported room version error. Noticed in https://github.com/matrix-org/matrix-doc/pull/2403#discussion_r577042144 Ported from https://github.com/matrix-org/synapse/pull/6739
* | Add knocking support (#81)Andrew Morgan2021-02-094-4/+199
| | | | | | | | | | | | | | | | | | Implement knocking as defined by https://github.com/matrix-org/matrix-doc/pull/2403 This is the base knocking stuff, taken from https://github.com/matrix-org/synapse/pull/6739 and does not include any public room directory changes. While knocking hasn't merged yet on mainline due to waiting on getting Complement into Synapse's CI, the code has been well-tested.
* | Merge commit '8ca120df7' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-312-48/+43
|\|
| * Consistently use room_id from federation request body (#8776)Richard van der Hoff2020-11-192-48/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts".
* | Merge commit 'ef2d62701' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-1/+1
|\|
| * Add ability for access tokens to belong to one user but grant access to ↵Erik Johnston2020-10-291-1/+1
| | | | | | | | | | | | | | | | | | | | another user. (#8616) We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't). A future PR will add an API for creating such a token. When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
* | Merge commit '74976a8e4' into dinsicAndrew Morgan2020-12-314-11/+26
|\|
| * Fix not sending events over federation when using sharded event persisters ↵Erik Johnston2020-10-142-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#8536) * Fix outbound federaion with multiple event persisters. We incorrectly notified federation senders that the minimum persisted stream position had advanced when we got an `RDATA` from an event persister. Notifying of federation senders already correctly happens in the notifier, so we just delete the offending line. * Change some interfaces to use RoomStreamToken. By enforcing use of `RoomStreamTokens` we make it less likely that people pass in random ints that they got from somewhere random.
| * Add type hints to response cache. (#8507)Patrick Cloke2020-10-091-3/+5
| |
| * Remove the deprecated Handlers object (#8494)Patrick Cloke2020-10-091-1/+6
| | | | | | All handlers now available via get_*_handler() methods on the HomeServer.
| * Add type hints to some handlers (#8505)Patrick Cloke2020-10-091-1/+1
| |
| * Remove stream ordering from Metadata dict (#8452)Richard van der Hoff2020-10-052-0/+4
| | | | | | | | | | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database.
| * Merge tag 'v1.21.0rc2' into developRichard van der Hoff2020-10-021-1/+1
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.21.0rc2 (2020-10-02) ============================== Features -------- - Convert additional templates from inline HTML to Jinja2 templates. ([\#8444](https://github.com/matrix-org/synapse/issues/8444)) Bugfixes -------- - Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. ([\#8438](https://github.com/matrix-org/synapse/issues/8438)) - Do not expose the experimental `uk.half-shot.msc2778.login.application_service` flow in the login API, which caused a compatibility problem with Element iOS. ([\#8440](https://github.com/matrix-org/synapse/issues/8440)) - Fix malformed log line in new federation "catch up" logic. ([\#8442](https://github.com/matrix-org/synapse/issues/8442)) - Fix DB query on startup for negative streams which caused long start up times. Introduced in [\#8374](https://github.com/matrix-org/synapse/issues/8374). ([\#8447](https://github.com/matrix-org/synapse/issues/8447))
| * | Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-3/+2
| | |
* | | Merge commit '6a8fd03ac' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-1/+1
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '6a8fd03ac': 1.21.0rc2 1.21.0rc2 Fix bug in remote thumbnail search (#8438) Fix DB query on startup for negative streams. (#8447) Convert additional templates to Jinja (#8444) Fix malformed log line in new federation "catch up" logic (#8442) Do not expose the experimental appservice login flow to clients. (#8440) update changelog fix a logging error in thumbnailer (#8435) changelog fixes fix version number
| * | Fix malformed log line in new federation "catch up" logic (#8442)Richard van der Hoff2020-10-021-1/+1
| |/
* | Merge commit 'cc40a59b4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-212-1/+45
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'cc40a59b4': 1.21.0 Add prometheus metrics to track federation delays (#8430) Make token serializing/deserializing async (#8427) Allow additional SSO properties to be passed to the client (#8413) changelog Add an improved "forward extremities" metric Rewrite BucketCollector Fix _exposition.py to stop stripping samples Drop support for ancient prometheus_client (#8426) Various clean ups to room stream tokens. (#8423) changelog Report state res metrics to Prometheus and log Move Measure calls into `resolve_events_with_store` Expose a `get_resource_usage` method in `Measure` Move `resolve_events_with_store` into StateResolutionHandler
| * Add prometheus metrics to track federation delays (#8430)Richard van der Hoff2020-10-012-1/+45
| | | | | | | | | | Add a pair of federation metrics to track the delays in sending PDUs to/from particular servers.
* | Merge commit '8238b55e0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-1/+3
|\| | | | | | | | | | | | | | | * commit '8238b55e0': Update description of server_name config option (#8415) Discard an empty upload_name before persisting an uploaded file (#7905) Don't table scan events on worker startup (#8419) Mypy fixes for `synapse.handlers.federation` (#8422)
| * Mypy fixes for `synapse.handlers.federation` (#8422)Richard van der Hoff2020-09-291-1/+3
| | | | | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
* | Merge commit '4325be1a5' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-214-9/+56
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '4325be1a5': Fix missing null character check on guest_access room state Fixed a bug with reactivating users with the admin API (#8362) Admin API for reported events (#8217) Fix wording of deprecation notice in changelog Deprecation warning for synapse admin api being accessible under /_matrix Create function to check for long names in devices (#8364) Add a comment re #1691 Fix a bad merge from release-v1.20.0. (#8354) Admin API for querying rooms where a user is a member (#8306) Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322) Simplify super() calls to Python 3 syntax. (#8344) Allow appservice users to /login (#8320) Update test logging to be able to accept braces (#8335) Move lint dependencies to extras_require (#8330)
| * Merge tag 'v1.20.0rc5' into developPatrick Cloke2020-09-181-5/+3
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.20.0rc5 (2020-09-18) ============================== In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3. Features -------- - Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343)) Bugfixes -------- - Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342)) - Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349)) Internal Changes ---------------- - Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285))
| * | Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322)reivilibre2020-09-181-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Fix _set_destination_retry_timings This came about because the code assumed that retry_interval could not be NULL — which has been challenged by catch-up.
| * | Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-183-9/+5
| | | | | | | | | | | | | | | | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
| * | Merge remote-tracking branch 'origin/release-v1.20.0' into developErik Johnston2020-09-182-17/+48
| |\ \
* | \ \ Merge commit 'd5f7182ba' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-5/+3
|\ \ \ \ | | |_|/ | |/| | | | | | | | | | | | | | | | | | * commit 'd5f7182ba': 1.20.0rc5 1.19.3 Use _check_sigs_and_hash_and_fetch to validate backfill requests (#8350)
| * | | Merge tag 'v1.19.3' into release-v1.20.0Patrick Cloke2020-09-181-5/+3
| |\ \ \ | | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1.19.3 Synapse 1.19.3 (2020-09-18) =========================== Bugfixes -------- - Partially mitigate bug where newly joined servers couldn't get past events in a room when there is a malformed event. ([\#8350](https://github.com/matrix-org/synapse/issues/8350))
| | * | Use _check_sigs_and_hash_and_fetch to validate backfill requests (#8350)Andrew Morgan2020-09-181-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a bit of a hack, as `_check_sigs_and_hash_and_fetch` is intended for attempting to pull an event from the database/(re)pull it from the server that originally sent the event if checking the signature of the event fails. During backfill we *know* that we won't have the event in our database, however it is still useful to be able to query the original sending server as the server we're backfilling from may be acting maliciously. The main benefit and reason for this change however is that `_check_sigs_and_hash_and_fetch` will drop an event during backfill if it cannot be successfully validated, whereas the current code will simply fail the backfill request - resulting in the client's /messages request silently being dropped. This is a quick patch to fix backfilling rooms that contain malformed events. A better implementation in planned in future.
* | | | Merge commit '43f2b67e4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-17/+48
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '43f2b67e4': Intelligently select extremities used in backfill. (#8349) Add flags to /versions about whether new rooms are encrypted by default. (#8343) Fix ratelimiting for federation `/send` requests. (#8342) blacklist MSC2753 sytests until it's implemented in synapse (#8285)
| * | | Fix ratelimiting for federation `/send` requests. (#8342)Erik Johnston2020-09-182-17/+48
| | | | | | | | | | | | c.f. #8295 for rationale
| * | | Merge branch 'erikj/fix_origin_check' into release-v1.20.0Erik Johnston2020-09-161-3/+4
| |\| |
* | | | Merge commit '837293c31' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-4/+125
|\ \ \ \ | | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '837293c31': Remove obsolete __future__ imports (#8337) Use admin_patterns for all admin APIs. (#8331) Fix a potential bug of UnboundLocalError (#8329) Switch metaclass initialization to python 3-compatible syntax (#8326) Catch-up after Federation Outage (split, 4): catch-up loop (#8272) Use slots in attrs classes where possible (#8296) Fix typos in comments. Add the topic and avatar to the room details admin API (#8305) Improve SAML error messages (#8248) Add experimental support for sharding event persister. Again. (#8294) Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) Use TLSv1.2 for fake servers in tests (#8208) Add /_synapse/client to the reverse proxy docs (#8227) Clean up `Notifier.on_new_room_event` code path (#8288)
| * | | Merge branch 'erikj/fix_origin_check' into developErik Johnston2020-09-161-3/+4
| |\ \ \ | | | |/ | | |/|
| * | | Catch-up after Federation Outage (split, 4): catch-up loop (#8272)reivilibre2020-09-151-4/+125
| | | |
* | | | Merge commit '5ffd68dca' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-3/+4
|\ \ \ \ | | |/ / | |/| | | | | | | | | | | | | | | | | | * commit '5ffd68dca': 1.19.2 Newsfile Don't assume that an event has an origin field
| * | | Don't assume that an event has an origin fieldErik Johnston2020-09-161-3/+4
| | | | | | | | | | | | | | | | This fixes #8319.
* | | | Merge commit '17fa4c7ca' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-209-13/+31
|\ \ \ \ | | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | * commit '17fa4c7ca': Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247) Catch-up after Federation Outage (split, 1) (#8230) Fix type signature in simple_select_one_onecol and friends (#8241) Stop sub-classing object (#8249)
| * | | Catch up after Federation Outage (split, 2): Track last successful stream ↵reivilibre2020-09-041-0/+11
| | | | | | | | | | | | | | | | | | | | ordering after transmission (#8247) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * | | Catch-up after Federation Outage (split, 1) (#8230)reivilibre2020-09-041-2/+9
| | |/ | |/| | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * | Stop sub-classing object (#8249)Patrick Cloke2020-09-049-11/+11
| | |
* | | Merge commit 'be16ee59a' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-203-28/+22
|\| | | | | | | | | | | | | | | | | | | | * commit 'be16ee59a': Add type hints to more handlers (#8244) Remove obsolete order field in `send_new_transaction` (#8245) Split fetching device keys and signatures into two transactions (#8233)
| * | Remove obsolete order field in `send_new_transaction` (#8245)reivilibre2020-09-033-28/+22
| | | | | | | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | | Merge commit '5c03134d0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-8/+11
|\| | | | | | | | | | | | | | | | | | | | * commit '5c03134d0': Convert additional database code to async/await. (#8195) Define StateMap as immutable and add a MutableStateMap type. (#8183) Move and refactor LoginRestServlet helper methods (#8182)
| * | Convert additional database code to async/await. (#8195)Patrick Cloke2020-08-281-8/+11
| | |
* | | Merge commit '9b7ac03af' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-10/+10
|\| | | | | | | | | | | | | | | | | | | | * commit '9b7ac03af': Convert calls of async database methods to async (#8166) simple_search_list_txn should return None, not 0. (#8187) Fix missing _add_persisted_position (#8179)
| * | Convert calls of async database methods to async (#8166)Patrick Cloke2020-08-272-10/+10
| | |
* | | Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-2/+2
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '56efa9ec7': (22 commits) Fix rate limiting unit tests. (#8167) Add functions to `MultiWriterIdGen` used by events stream (#8164) Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) Changelog fixes Make StreamIdGen `get_next` and `get_next_mult` async (#8161) Wording fixes to 'name' user admin api filter (#8163) Fix missing double-backtick in RST document Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) Add type hints for state. (#8140) Stop shadow-banned users from sending non-member events. (#8142) Allow capping a room's retention policy (#8104) Add healthcheck for default localhost 8008 port on /health endpoint. (#8147) Fix flaky shadow-ban tests. (#8152) Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991) Do not apply ratelimiting on joins to appservices (#8139) Micro-optimisations to get_auth_chain_ids (#8132) Allow denying or shadow banning registrations via the spam checker (#8034) Stop shadow-banned users from sending invites. (#8095) Be more tolerant of membership events in unknown rooms (#8110) Improve the error code when trying to register using a name reserved for guests. (#8135) ...
| * | Add type hints for state. (#8140)Patrick Cloke2020-08-241-2/+2
| | |
* | | Merge commit 'c9c544cda' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-192-6/+4
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'c9c544cda': Remove `ChainedIdGenerator`. (#8123) Switch the JSON byte producer from a pull to a push producer. (#8116) Updated docs: Added note about missing 308 redirect support. (#8120) Be stricter about JSON that is accepted by Synapse (#8106) Convert runWithConnection to async. (#8121) Remove the unused inlineCallbacks code-paths in the caching code (#8119) Separate `get_current_token` into two. (#8113) Convert events worker database to async/await. (#8071) Add a link to the matrix-synapse-rest-password-provider. (#8111)
| * | Be stricter about JSON that is accepted by Synapse (#8106)Patrick Cloke2020-08-192-6/+4
| | |
* | | Merge commit 'e04e465b4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-193-3/+3
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'e04e465b4': Use the default templates when a custom template file cannot be found (#8037) Changelog changes Convert stream database to async/await. (#8074) Add a shadow-banned flag to users. (#8092) Convert pusher databases to async/await. (#8075) Convert receipts and events databases to async/await. (#8076)
| * | Convert stream database to async/await. (#8074)Patrick Cloke2020-08-173-3/+3
| | |
* | | Merge commit 'ff0e89465' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-0/+22
|\| | | | | | | | | | | | | | * commit 'ff0e89465': Drop federation transmission queues during a significant remote outage. (#7864)
| * | Drop federation transmission queues during a significant remote outage. (#7864)reivilibre2020-08-131-0/+22
| |/ | | | | | | | | | | | | | | | | | | | | | | | | * Empty federation transmission queues when we are backing off. Fixes #7828. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Address feedback Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Reword newsfile
* | Merge commit '9d1e4942a' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-2/+5
|\| | | | | | | | | | | | | | | | | * commit '9d1e4942a': Fix typing for notifier (#8064) Add comment explaining cast Handle optional dependencies for Oidc and Saml Newsfile Change HomeServer definition to work with typing.
| * Fix typing for notifier (#8064)Erik Johnston2020-08-121-2/+5
| |
| * Merge branch 'master' into developOlivier Wilkinson (reivilibre)2020-07-303-3/+3
| |\
* | | Merge commit 'c978f6c45' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-163-73/+50
|\| | | | | | | | | | | | | | * commit 'c978f6c45': Convert federation client to async/await. (#7975)
| * | Convert federation client to async/await. (#7975)Patrick Cloke2020-07-303-73/+50
| | |
* | | Merge commit 'a9631b7b4' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-033-3/+3
|\ \ \ | | |/ | |/| | | | | | | | | | * commit 'a9631b7b4': 1.18.0 Update worker docs with recent enhancements (#7969)
| * | Update worker docs with recent enhancements (#7969)Erik Johnston2020-07-293-3/+3
| |/
* | Merge commit 'f88c48f3b' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-032-4/+17
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'f88c48f3b': 1.18.0rc1 Fix error reporting when using `opentracing.trace` (#7961) Fix typing replication not being handled on master (#7959) Remove hacky error handling for inlineDeferreds. (#7950) Convert tests/rest/admin/test_room.py to unix file endings (#7953) Support oEmbed for media previews. (#7920) Convert state resolution to async/await (#7942) Fix up types and comments that refer to Deferreds. (#7945) Do not convert async functions to Deferreds in the interactive_auth_handler (#7944) Convert more of the media code to async/await (#7873) Return an empty body for OPTIONS requests. (#7886) Downgrade warning on client disconnect to INFO (#7928) Convert presence handler helpers to async/await. (#7939) Update the auth providers to be async. (#7935) Put a cache on `/state_ids` (#7931)
| * Convert state resolution to async/await (#7942)Patrick Cloke2020-07-241-1/+3
| |
| * Convert presence handler helpers to async/await. (#7939)Patrick Cloke2020-07-231-1/+3
| |
| * Put a cache on `/state_ids` (#7931)Richard van der Hoff2020-07-231-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we send out an event which refers to `prev_events` which other servers in the federation are missing, then (after a round or two of backfill attempts), they will end up asking us for `/state_ids` at a particular point in the DAG. As per https://github.com/matrix-org/synapse/issues/7893, this is quite expensive, and we tend to see lots of very similar requests around the same time. We can therefore handle this much more efficiently by using a cache, which (a) ensures that if we see the same request from multiple servers (or even the same server, multiple times), then they share the result, and (b) any other servers that miss the initial excitement can also benefit from the work. [It's interesting to note that `/state` has a cache for exactly this reason. `/state` is now essentially unused and replaced with `/state_ids`, but evidently when we replaced it we forgot to add a cache to the new endpoint.]
* | Merge commit '4876af06d' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-0/+6
|\| | | | | | | | | * commit '4876af06d': Abort federation requests if the client disconnects early (#7930)
| * Abort federation requests if the client disconnects early (#7930)Richard van der Hoff2020-07-231-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For inbound federation requests, if a given remote server makes too many requests at once, we start stacking them up rather than processing them immediatedly. However, that means that there is a fair chance that the requesting server will disconnect before we start processing the request. In that case, if it was a read-only request (ie, a GET request), there is absolutely no point in building a response (and some requests are quite expensive to handle). Even in the case of a POST request, one of two things will happen: * Most likely, the requesting server will retry the request and we'll get the information anyway. * Even if it doesn't, the requesting server has to assume that we didn't get the memo, and act accordingly. In short, we're better off aborting the request at this point rather than ploughing on with what might be a quite expensive request.
* | Merge commit 'de119063f' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-035-89/+104
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'de119063f': (31 commits) Convert room list handler to async/await. (#7912) Element CSS and logo in email templates (#7919) Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914) Remove unused code from synapse.logging.utils. (#7897) Fix a typo in the sample config. (#7890) Fix deprecation warning: import ABC from collections.abc (#7892) Change sample config's postgres user to synapse_user (#7889) Fix deprecation warning due to invalid escape sequences (#7895) Remove Ubuntu Eoan that is now EOL (#7888) Fix the trace function for async functions. (#7872) Add help for creating a user via docker (#7885) Switch to Debian:Slim from Alpine for the docker image (#7839) Stop using 'device_max_stream_id' (#7882) Fix TypeError in synapse.notifier (#7880) Add a default limit (of 100) to get/sync operations. (#7858) Change "unknown room ver" logging to warning. (#7881) Convert device handler to async/await (#7871) Convert synapse.app to async/await. (#7868) Convert _base, profile, and _receipts handlers to async/await (#7860) Add admin endpoint to get members in a room. (#7842) ...
| * Convert room list handler to async/await. (#7912)Patrick Cloke2020-07-211-8/+2
| |
| * Convert synapse.app to async/await. (#7868)Patrick Cloke2020-07-171-21/+19
| |
| * Allow moving typing off master (#7869)Erik Johnston2020-07-161-51/+74
| |
| * Add ability to run multiple pusher instances (#7855)Erik Johnston2020-07-162-9/+9
| | | | | | This reuses the same scheme as federation sender sharding
* | Merge commit 'a973bcb8a' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-032-3/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'a973bcb8a': Add some tiny type annotations (#7870) Remove obsolete comment. Ensure that calls to `json.dumps` are compatible with the standard library json. (#7836) Avoid brand new rooms in `delete_old_current_state_events` (#7854) Allow accounts to be re-activated from the admin APIs. (#7847) Fix tests Fix typo Newsfile Use get_users_in_room rather than state handler in typing for speed Fix client reader sharding tests (#7853) Convert E2E key and room key handlers to async/await. (#7851) Return the proper 403 Forbidden error during errors with JWT logins. (#7844) remove `retry_on_integrity_error` wrapper for persist_events (#7848)
| * Remove obsolete comment.Olivier Wilkinson (reivilibre)2020-07-161-2/+0
| | | | | | | | | | | | | | | | It was correct at the time of our friend Jorik writing it (checking git blame), but the world has moved now and it is no longer a generator. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Ensure that calls to `json.dumps` are compatible with the standard library ↵Patrick Cloke2020-07-151-1/+1
| | | | | | | | json. (#7836)
* | Merge commit '66a4af8d9' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-3/+3
|\| | | | | | | | | * commit '66a4af8d9': Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802)
| * Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802)Patrick Cloke2020-07-101-3/+3
| |
* | Merge commit 'f299441cc' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-033-3/+81
|\| | | | | | | | | * commit 'f299441cc': Add ability to shard the federation sender (#7798)
| * Add ability to shard the federation sender (#7798)Erik Johnston2020-07-103-3/+81
| |
* | Merge commit '43726783e' into dinsicAndrew Morgan2020-08-036-12/+12
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '43726783e': (22 commits) 1.17.0rc1 Fix some spelling mistakes / typos. (#7811) `update_membership` declaration: now always returns an event id. (#7809) Improve stacktraces from exceptions in background processes (#7808) Fix `can only concatenate list (not "tuple") to list` exception (#7810) Pass original request headers from workers to the main process. (#7797) Generate real events when we reject invites (#7804) Add `HomeServer.signing_key` property (#7805) Revert "Update the installation docs on apt-transport-https (#7801)" Do not use simplejson in Synapse. (#7800) Stop passing bytes when dumping JSON (#7799) Update the installation docs on apt-transport-https (#7801) shuffle changelog slightly Change Caddy links (old is deprecated) (#7789) Stop populating unused table `local_invites`. (#7793) Refactor getting replication updates from database v2. (#7740) Add libwebp dependency to Dockerfile (#7791) Add documentation for JWT login type and improve sample config. (#7776) Convert the appservice handler to async/await. (#7775) Don't ignore `set_tweak` actions with no explicit `value`. (#7766) ...
| * Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-096-11/+11
| |
| * Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-1/+1
| | | | | | ... instead of duplicating `config.signing_key[0]` everywhere
* | Merge commit '5cdca53aa' into dinsicAndrew Morgan2020-08-032-6/+2
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '5cdca53aa': Merge different Resource implementation classes (#7732) Fix inconsistent handling of upper and lower cases of email addresses. (#7021) Allow YAML config file to contain None (#7779) Fix a typo. Move 1.15.2 after 1.16.0rc2. 1.16.0rc2 Remove an extraneous space. Add links to the fixes. Fix tense in the release notes. Hack to add push priority to push notifications (#7765) Add early returns to `_check_for_soft_fail` (#7769) Use symbolic names for replication stream names (#7768) Type checking for `FederationHandler` (#7770) Fix new metric where we used ms instead of seconds (#7771) Fix incorrect error message when database CTYPE was set incorrectly. (#7760) Pin link in CHANGES.md Fixes to CHANGES.md
| * Merge different Resource implementation classes (#7732)Erik Johnston2020-07-031-5/+1
| |
| * Fix new metric where we used ms instead of seconds (#7771)Erik Johnston2020-07-011-1/+1
| | | | | | | | Introduced in #7755, not yet released.
* | Merge commit 'dc80a0762' into dinsicAndrew Morgan2020-08-032-17/+30
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'dc80a0762': 1.16.0rc1 Back out MSC2625 implementation (#7761) Additional configuration options for auto-join rooms (#7763) Add some metrics for inbound and outbound federation processing times (#7755) Explain the purpose of the "tests" conditional dependency requirement (#7751) Add another yield point to state res v2 (#7746) Move flake8 to end. Don't exit script on failure (#7738) Make tox actions work on Debian 10 (#7703) Yield during large v2 state res. (#7735) add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675) Set Content-Length for Metrics requests (#7730) Sync ignored table names in synapse_port_db to current database schema (#7717) Allow local media to be marked as safe from being quarantined. (#7718) Convert directory handler to async/await (#7727) Speed up state res v2 across large state differences. (#7725)
| * Add some metrics for inbound and outbound federation processing times (#7755)Erik Johnston2020-06-302-17/+30
| |
* | Merge commit 'a3f11567d' into dinsicAndrew Morgan2020-08-033-8/+3
|\| | | | | | | | | * commit 'a3f11567d': Replace all remaining six usage with native Python 3 equivalents (#7704)
| * Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-163-8/+3
| |
* | Merge commit '03619324f' into dinsicAndrew Morgan2020-08-034-14/+9
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '03619324f': Create a ListenerConfig object (#7681) Fix changelog wording 1.15.1 Wrap register_device coroutine in an ensureDeferred (#7684) Ensure the body is a string before comparing push rules. (#7701) Ensure etag is a string for GET room_keys/version response (#7691) Update m.id.phone to use 'phone' instead of 'number' (#7687) Fix "There was no active span when trying to log." error (#7698) Enable 3PID add/bind/unbind endpoints on r0 routes Discard RDATA from already seen positions. (#7648) Replace iteritems/itervalues/iterkeys with native versions. (#7692) Fix warnings about losing log context during UI auth. (#7688) Fix a typo when comparing the URI & method during UI Auth. (#7689) Remove "user_id" from GET /presence. (#7606) Increase the default SAML session expirary time to 15 minutes. (#7664) fix typo in sample_config.yaml (#7652) Take out a lock before modifying _CACHES (#7663) Add option to enable encryption by default for new rooms (#7639) Clean-up the fallback login code. (#7657)
| * Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-154-14/+9
| |
* | Add a bulk user info endpoint and deprecate the old one (#46)Andrew Morgan2020-06-192-1/+68
|/ | | | | | | | | The current `/user/<user_id>/info` API was useful in that it could be used by any user to lookup whether another user was deactivate or expired. However, it was impractical as it only allowed for a single lookup at once. Clients trying to use this API were met with speed issues as they tried to query this information for all users in a room. This PR adds an equivalent CS and Federation API that takes a list of user IDs, and returning a mapping from user ID to info dictionary. Note that the federation in this PR was a bit trickier than in the original #12 as we can no longer use a federation query, as those don't allow for JSON bodies - which we require to pass a list of user IDs. Instead we do the whole thing of adding a method to transport/client and transport/server. This PR also adds unittests. The earlier PR used Sytest, presumably for testing across federation, but as this is Synapse-specific that felt a little gross. Unit tests for the deprecated endpoint have not been added.
* add a commentRichard van der Hoff2020-05-211-0/+3
|
* Strictly enforce canonicaljson requirements in a new room version (#7381)Patrick Cloke2020-05-141-1/+5
|
* Fix typing annotations in synapse/federation (#7382)Richard van der Hoff2020-05-051-15/+31
| | | | We're pretty close to having mypy working for `synapse.federation`, so let's finish the job.
* Fix catchup-on-reconnect for the Federation Stream (#7374)Richard van der Hoff2020-05-054-25/+39
| | | | looks like we managed to break this during the refactorathon.
* async/await is_server_admin (#7363)Andrew Morgan2020-05-011-3/+2
|
* Further improvements to requesting the public rooms list on a homeserver ↵Andrew Morgan2020-05-012-23/+70
| | | | which has it set to private (#7368)
* Merge branch 'release-v1.12.4' into developRichard van der Hoff2020-04-221-9/+40
|\
| * Query missing cross-signing keys on local sig upload (#7289)Andrew Morgan2020-04-221-9/+40
| |
* | Merge branch 'release-v1.12.4' into developRichard van der Hoff2020-04-201-11/+3
|\|
| * Revert "Query missing cross-signing keys on local sig upload"Richard van der Hoff2020-04-201-11/+3
| | | | | | | | | | | | This was incorrectly merged to the release branch before it was ready. This reverts commit 72fe2affb6ac86d433b80b6452da57052365aa26.
* | Merge branch 'release-v1.12.4' into developRichard van der Hoff2020-04-201-3/+11
|\|
| * Query missing cross-signing keys on local sig uploadAndrew Morgan2020-04-171-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add changelog Save retrieved keys to the db lint Fix and de-brittle remote result dict processing Use query_user_devices instead, assume only master, self_signing key types Make changelog more useful Remove very specific exception handling Wrap get_verify_key_from_cross_signing_key in a try/except Note that _get_e2e_cross_signing_verify_key can raise a SynapseError lint Add comment explaining why this is useful Only fetch master and self_signing key types Fix log statements, docstrings Remove extraneous items from remote query try/except lint Factor key retrieval out into a separate function Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates Update method docstring
* | Add typing information to federation_server. (#7219)Patrick Cloke2020-04-071-66/+107
| |
* | Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-0/+9
| | | | | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* | Clean up some LoggingContext stuff (#7120)Richard van der Hoff2020-03-241-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.
* | Convert `*StreamRow` classes to inner classes (#7116)Richard van der Hoff2020-03-231-1/+1
| | | | | | | | | | This just helps keep the rows closer to their streams, so that it's easier to see what the format of each stream is.
* | Fix a bug in the federation API which could cause occasional "Failed to get ↵Patrick Cloke2020-03-193-30/+21
|/ | | | PDU" errors (#7089).
* Remove unused federation endpoint (`query_auth`) (#7026)Patrick Cloke2020-03-174-146/+79
|
* Add some type annotations to the federation base & client classes (#6995)Patrick Cloke2020-02-282-28/+42
|
* Merge remote-tracking branch 'origin/release-v1.11.1' into developPatrick Cloke2020-02-271-6/+8
|\
| * Cast a coroutine into a Deferred in the federation base (#6996)Patrick Cloke2020-02-261-6/+8
| | | | | | | | Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
* | Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-2/+2
|/ | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Add support for putting fed user query API on workers (#6873)Erik Johnston2020-02-071-2/+5
|
* Add a `make_event_from_dict` method (#6858)Richard van der Hoff2020-02-071-3/+2
| | | | | | | ... and use it in places where it's trivial to do so. This will make it easier to pass room versions into the FrozenEvent constructors.
* Add typing to synapse.federation.sender (#6871)Erik Johnston2020-02-075-105/+128
|
* Pass room_version into `event_from_pdu_json`Richard van der Hoff2020-02-063-58/+46
| | | | It's called from all over the shop, so this one's a bit messy.
* pass room version into FederationClient.send_join (#6854)Richard van der Hoff2020-02-061-28/+32
| | | | ... which allows us to sanity-check the create event.
* Reduce amount of logging at INFO level. (#6862)Erik Johnston2020-02-062-4/+4
| | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* Simplify `room_version` handling in `FederationClient.send_invite`Richard van der Hoff2020-02-051-10/+6
|
* Pass room version object into `FederationClient.get_pdu`Richard van der Hoff2020-02-051-3/+5
|
* make FederationClient._do_send_invite asyncRichard van der Hoff2020-02-051-10/+7
|
* Apply suggestions from code reviewRichard van der Hoff2020-02-051-2/+4
| | | Co-Authored-By: Erik Johnston <erik@matrix.org>
* make FederationClient.send_invite asyncRichard van der Hoff2020-02-031-6/+5
|
* make FederationClient.get_missing_events asyncRichard van der Hoff2020-02-031-20/+20
|
* make FederationClient._do_send_leave asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.send_leave.send_request asyncRichard van der Hoff2020-02-031-5/+2
|
* make FederationClient._do_send_join asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.send_join.send_request asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient.make_membership_event.send_request asyncRichard van der Hoff2020-02-031-4/+3
|
* make FederationClient._try_destination_list asyncRichard van der Hoff2020-02-031-10/+26
|
* make FederationClient.send_leave asyncRichard van der Hoff2020-02-031-10/+9
|
* make FederationClient.send_join asyncRichard van der Hoff2020-02-031-11/+13
|
* make FederationClient.make_membership_event asyncRichard van der Hoff2020-02-031-10/+11
|
* make FederationClient.get_event_auth asyncRichard van der Hoff2020-02-031-6/+4
|
* make FederationClient.get_room_state_ids asyncRichard van der Hoff2020-02-031-4/+5
|
* make FederationClient.get_pdu asyncRichard van der Hoff2020-02-031-15/+17
|
* make FederationClient.backfill asyncRichard van der Hoff2020-02-031-15/+11
|
* Fix `room_version` in `on_invite_request` flow (#6827)Richard van der Hoff2020-02-032-7/+10
| | | | I messed this up a bit in #6805, but fortunately we weren't actually doing anything with the room_version so it didn't matter that it was a str not a RoomVersion.
* Merge pull request #6806 from matrix-org/rav/redact_changes/3Richard van der Hoff2020-01-311-3/+1
|\ | | | | Pass room_version into add_hashes_and_signatures
| * Pass room_version into create_local_event_from_event_dictRichard van der Hoff2020-01-301-3/+1
| |
* | s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-312-13/+13
| | | | | | | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* | pass room version into FederationHandler.on_invite_request (#6805)Richard van der Hoff2020-01-301-1/+1
|/
* Fix sending server up commands from workers (#6811)Erik Johnston2020-01-302-12/+19
| | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Remove unused DeviceRow class (#6800)Erik Johnston2020-01-291-20/+1
|
* Fix race in federation sender that delayed device updates. (#6799)Erik Johnston2020-01-291-29/+3
| | | | | | | | | | | | We were sending device updates down both the federation stream and device streams. This mean there was a race if the federation sender worker processed the federation stream first, as when the sender checked if there were new device updates the slaved ID generator hadn't been updated with the new stream IDs and so returned nothing. This situation is correctly handled by events/receipts/etc by not sending updates down the federation stream and instead having the federation sender worker listen on the other streams and poke the transaction queues as appropriate.
* Add `rooms.room_version` column (#6729)Erik Johnston2020-01-271-17/+33
| | | This is so that we don't have to rely on pulling it out from `current_state_events` table.
* Wake up transaction queue when remote server comes back online (#6706)Erik Johnston2020-01-172-3/+34
| | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up.
* Add StateMap type alias (#6715)Erik Johnston2020-01-161-1/+2
|
* Port synapse.replication.tcp to async/await (#6666)Erik Johnston2020-01-161-1/+3
| | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing....
* Process EDUs in parallel with PDUs. (#6697)Erik Johnston2020-01-141-12/+58
| | | | This means that things like to device messages don't get blocked behind processing PDUs, which can potentially take *ages*.
* Merge pull request #6349 from matrix-org/babolivier/msc1802Brendan Abolivier2019-12-114-28/+140
|\ | | | | Implement v2 APIs for send_join and send_leave
| * Merge branch 'develop' into babolivier/msc1802Brendan Abolivier2019-12-058-26/+34
| |\
| * | LintBrendan Abolivier2019-11-111-3/+2
| | |
| * | LintBrendan Abolivier2019-11-111-3/+1
| | |
| * | Fix prefix for v2/send_leaveBrendan Abolivier2019-11-111-0/+2
| | |
| * | Add server-side support to the v2 APIBrendan Abolivier2019-11-112-14/+33
| | |
| * | Implement v2 API for send_leaveBrendan Abolivier2019-11-112-5/+56
| | |
| * | Add missing yieldBrendan Abolivier2019-11-111-1/+1
| | |
| * | Implement v2 API for send_joinBrendan Abolivier2019-11-112-8/+51
| | |
* | | Move get_state methods into FederationHandler (#6503)Richard van der Hoff2019-12-101-80/+11
| | | | | | | | | | | | | | | This is a non-functional refactor as a precursor to some other work.
* | | Refactor get_events_from_store_or_dest to return a dict (#6501)Richard van der Hoff2019-12-101-29/+15
| | | | | | | | | | | | | | | There was a bunch of unnecessary conversion back and forth between dict and list going on here. We can simplify a bunch of the code.
* | | Remove fallback for missing /federation/v1/state_ids API (#6488)Richard van der Hoff2019-12-092-96/+17
| |/ |/| | | | | This API was added way back in 0.17.0; the code here is annoying to maintain and entirely redundant.
* | Implementation of MSC2314 (#6176)Amber Brown2019-11-282-12/+20
| |
* | Clean up newline quote marks around the codebase (#6362)Andrew Morgan2019-11-213-5/+5
| |
* | Replace instance variations of homeserver with correct case/spacingAndrew Morgan2019-11-124-9/+9
|/
* Depublish a room from the public rooms list when it is upgraded (#6232)Andrew Morgan2019-11-011-1/+1
|
* Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-316-25/+32
|\
| * Update black to 19.10b0 (#6304)Amber Brown2019-11-011-5/+6
| | | | | | * update version of black and also fix the mypy config being overridden
| * Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-315-20/+26
| | | | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* | rename get_devices_by_remote to get_device_updates_by_remoteHubert Chathi2019-10-301-2/+2
| |
* | Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-305-131/+106
|\|
| * Fix small typo in comment (#6269)Andrew Morgan2019-10-301-1/+1
| |
| * Merge pull request #6280 from matrix-org/erikj/receipts_async_awaitErik Johnston2019-10-301-1/+3
| |\ | | | | | | Port receipt and read markers to async/wait
| | * Port receipt and read markers to async/waitErik Johnston2019-10-291-1/+3
| | |
| * | Don't return coroutinesErik Johnston2019-10-291-6/+6
| |/
| * Port federation_server to async/awaitErik Johnston2019-10-291-118/+87
| |
| * Merge tag 'v1.5.0rc2' into developRichard van der Hoff2019-10-282-5/+9
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.5.0rc2 (2019-10-28) ============================= Bugfixes -------- - Update list of boolean columns in `synapse_port_db`. ([\#6247](https://github.com/matrix-org/synapse/issues/6247)) - Fix /keys/query API on workers. ([\#6256](https://github.com/matrix-org/synapse/issues/6256)) - Improve signature checking on some federation APIs. ([\#6262](https://github.com/matrix-org/synapse/issues/6262)) Internal Changes ---------------- - Move schema delta files to the correct data store. ([\#6248](https://github.com/matrix-org/synapse/issues/6248)) - Small performance improvement by removing repeated config lookups in room stats calculation. ([\#6255](https://github.com/matrix-org/synapse/issues/6255))
| | * Improve signature checking on some federation APIs (#6262)Richard van der Hoff2019-10-282-5/+9
| | | | | | | | | | | | | | | Make sure that we check that events sent over /send_join, /send_leave, and /invite, are correctly signed and come from the expected servers.
| * | use %r to __repr__ objectsMichael Kaye2019-10-242-3/+3
| |/ | | | | | | This avoids calculating __repr__ unless we are going to log.
* | Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-241-1/+1
|\|
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-222-49/+0
| |\ | | | | | | | | | erikj/refactor_stores
| * | Move storage classes into a main "data store".Erik Johnston2019-10-211-1/+1
| | | | | | | | | | | | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* | | implement federation parts of cross-signingHubert Chathi2019-10-221-2/+2
| |/ |/|
* | rip out some unreachable codeRichard van der Hoff2019-10-172-49/+0
|/ | | | The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
* Fix /federation/v1/state for recent room versions (#6170)Richard van der Hoff2019-10-081-13/+0
| | | | | | | * Fix /federation/v1/state for recent room versions Turns out this endpoint was completely broken for v3 rooms. Hopefully this re-signing code is irrelevant nowadays anyway.
* add some metrics on the federation sender (#6160)Richard van der Hoff2019-10-031-5/+6
|
* Land improved room list based on room stats (#6019)Erik Johnston2019-10-021-0/+8
| | | | Use room_stats and room_state for room directory search
* Fix exception when resetting retry timingsRichard van der Hoff2019-09-201-1/+1
| | | | | | | | Fixes: > TypeError: set_destination_retry_timings() missing 1 required positional argument: 'retry_interval' Introduced in #6016.
* Remove origin parameter from add_display_name_to_third_party_invite and add ↵Andrew Morgan2019-09-112-3/+3
| | | | | | | params to docstring (#6010) Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument. Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
* Merge pull request #5984 from ↵Erik Johnston2019-09-052-3/+13
|\ | | | | | | | | matrix-org/joriks/opentracing_link_send_to_edu_contexts Link the send loop with the edus contexts
| * use access methods (duh..)Jorik Schellekens2019-09-051-1/+3
| | | | | | Co-Authored-By: Erik Johnston <erik@matrix.org>
| * Link the send loop with the edus contextsJorik Schellekens2019-09-052-3/+11
| | | | | | | | | | | | The contexts were being filtered too early so the send loop wasn't being linked to them unless the destination was whitelisted.
* | Add opentracing to all client servlets (#5983)Jorik Schellekens2019-09-051-1/+5
|/
* Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-302-15/+15
| | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* Merge pull request #5859 from matrix-org/rei/msc2197reivilibre2019-08-282-15/+67
|\ | | | | MSC2197 Search Filters over Federation
| * Use MSC2197 on stable prefix as it has almost finished FCPOlivier Wilkinson (reivilibre)2019-08-202-26/+4
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Add support for inbound MSC2197 requests on unstable Federation APIOlivier Wilkinson (reivilibre)2019-08-151-1/+59
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Support MSC2197 outbound with unstable prefixOlivier Wilkinson (reivilibre)2019-08-151-15/+31
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Propagate opentracing contexts through EDUs (#5852)Jorik Schellekens2019-08-223-80/+108
| | | | | | | | | | Propagate opentracing contexts through EDUs Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Opentracing across workers (#5771)Jorik Schellekens2019-08-221-15/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Propagate opentracing contexts across workers Also includes some Convenience modifications to opentracing for servlets, notably: - Add boolean to skip the whitelisting check on inject extract methods. - useful when injecting into carriers locally. Otherwise we'd always have to include our own servername and whitelist our servername - start_active_span_from_request instead of header - Add boolean to decide whether to extract context from a request to a servlet
* | Opentrace e2e keys (#5855)Jorik Schellekens2019-08-221-0/+3
| | | | | | Add opentracing tags and logs for e2e keys
* | Opentracing misc (#5856)Jorik Schellekens2019-08-161-6/+7
| | | | | | | | | | | | | | | | | | | | | | Add authenticated_entity and servlet_names tags. Functionally: - Add a tag for authenticated_entity - Add a tag for servlet_names Stylistically: Moved to importing methods directly from opentracing.
* | Wrap `get_local_public_room_list` call in `maybeDeferred` because itOlivier Wilkinson (reivilibre)2019-08-141-2/+8
|/ | | | | | | is cached and so does not always return a `Deferred`. `await` does not silently pass-through non-Deferreds like `yield` used to. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Return 502 not 500 when failing to reach any remote server.Erik Johnston2019-08-011-4/+3
|
* Room Complexity Client Implementation (#5783)Amber Brown2019-07-302-7/+60
|
* Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatchRichard van der Hoff2019-07-261-2/+2
|\ | | | | Log when we receive a /make_* request from a different origin
| * Log when we receive a /make_* request from a different originRichard van der Hoff2019-07-261-2/+2
| |
* | Merge tag 'v1.2.0rc2' into developAndrew Morgan2019-07-241-1/+3
|\| | | | | | | | | | | | | Bugfixes -------- - Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
| * Fix servlet metric names (#5734)Jorik Schellekens2019-07-241-1/+3
| | | | | | | | | | | | | | | | | | | | * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths
* | Replace returnValue with return (#5736)Amber Brown2019-07-236-90/+75
|/
* Convert synapse.federation.transport.server to async (#5689)Richard van der Hoff2019-07-181-242/+188
| | | | | | | | | | | | | | | | * Convert BaseFederationServlet._wrap to async Empirically, this fixes some lost stacktraces. It should be safe because the wrapped function is called from JsonResource._async_render, which is already async. * Convert the rest of synapse.federation.transport.server to async We may as well do the whole file while we're here. * changelog * flake8
* Add basic opentracing support (#5544)Jorik Schellekens2019-07-111-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter
* remove dead transaction persist code (#5622)Richard van der Hoff2019-07-052-41/+0
| | | | this hasn't done anything for years
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-047-22/+32
|
* Split public rooms directory auth config in twoBrendan Abolivier2019-06-241-4/+4
|
* Fix logging error when a tampered event is detected. (#5500)Richard van der Hoff2019-06-211-1/+0
|
* Run Black. (#5482)Amber Brown2019-06-2011-752/+613
|
* Add experimental option to reduce extremities.Erik Johnston2019-06-181-0/+3
| | | | | | | Adds new config option `cleanup_extremities_with_dummy_events` which periodically sends dummy events to rooms with more than 10 extremities. THIS IS REALLY EXPERIMENTAL.
* Clean up code for sending federation EDUs. (#5381)Richard van der Hoff2019-06-131-14/+26
| | | | This code confused the hell out of me today. Split _get_new_device_messages into its two (unrelated) parts.
* Prevent multiple device list updates from breaking a batch send (#5156)Andrew Morgan2019-06-061-2/+3
| | | fixes #5153
* Associate a request_name with each verify request, for loggingRichard van der Hoff2019-06-052-1/+5
| | | | | | | Also: * rename VerifyKeyRequest->VerifyJsonRequest * calculate key_ids on VerifyJsonRequest construction * refactor things to pass around VerifyJsonRequests instead of 4-tuples
* Add a test room version where we enforce key validity (#5348)Richard van der Hoff2019-06-051-15/+24
|
* Clean up debug logging (#5347)Richard van der Hoff2019-06-051-3/+13
| | | Remove some spurious stuff, clarify some other stuff
* Fix failure to fetch batches of PDUs (#5342)Richard van der Hoff2019-06-041-0/+1
| | | | | FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A failure to fetch one should not result in a failure of the whole batch. Add the missing `continue`.
* Rename get_events->get_events_from_store_or_dest (#5344)Richard van der Hoff2019-06-041-20/+13
| | | | | We have too many things called get_event, and it's hard to figure out what we mean. Also remove some unused params from the signature, and add some logging.
* Enforce validity period on server_keys for fed requests. (#5321)Richard van der Hoff2019-06-032-3/+5
| | | | | | | | When handling incoming federation requests, make sure that we have an up-to-date copy of the signing key. We do not yet enforce the validity period for event signatures.
* Implement the SHHS complexity API (#5216)Amber Brown2019-05-301-1/+30
|
* Fix error handling for rooms whose versions are unknown. (#5219)Richard van der Hoff2019-05-211-1/+13
| | | | | | | If we remove support for a particular room version, we should behave more gracefully. This should make client requests fail with a 400 rather than a 500, and will ignore individiual PDUs in a federation transaction, rather than the whole transaction.
* Make all the rate limiting options more consistent (#5181)Amber Brown2019-05-151-5/+1
|
* Run `black` on per_destination_queueRichard van der Hoff2019-05-091-35/+39
| | | | ... mostly to fix pep8 fails
* Limit the number of EDUs in transactions to 100 as expected by receiver (#5138)Quentin Dufour2019-05-091-26/+30
| | | Fixes #3951.
* add options to require an access_token to GET /profile and /publicRooms on ↵Matthew Hodgson2019-05-081-0/+10
| | | | | | | | | | | | | | | | | CS API (#5083) This commit adds two config options: * `restrict_public_rooms_to_local_users` Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API. * `require_auth_for_profile_requests` When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301. MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though. Groups have been intentionally omitted from this commit.
* more logging improvementsRichard van der Hoff2019-04-251-8/+11
|
* remove extraneous exception loggingRichard van der Hoff2019-04-251-2/+2
|
* Clarify logging when PDU signature checking failsRichard van der Hoff2019-04-251-0/+19
|
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-014-25/+27
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* Use event streams to calculate presenceErik Johnston2019-03-272-3/+89
| | | | | | | | | | | | | | | | | Primarily this fixes a bug in the handling of remote users joining a room where the server sent out the presence for all local users in the room to all servers in the room. We also change to using the state delta stream, rather than the distributor, as it will make it easier to split processing out of the master process (as well as being more flexible). Finally, when sending presence states to newly joined servers we filter out old presence states to reduce the number sent. Initially we filter out states that are offline and have a last active more than a week ago, though this can be changed down the line. Fixes #3962
* Make federation endpoints more tolerant of trailing slashes v2 (#4935)Andrew Morgan2019-03-262-8/+8
| | | Redo of https://github.com/matrix-org/synapse/pull/4840
* Remove trailing slashes from outbound federation requests and retry on 400 ↵Andrew Morgan2019-03-211-7/+14
|\ | | | | | | | | (#4840) As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-08-12 01:23:49 +0000