summary refs log tree commit diff
path: root/synapse/handlers/federation.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-05-11Uniformize spam-checker API: github/ts/spam-errors ts/spam-errorsDavid Teller1-3/+7
- Some callbacks should return `True` to allow, `False` to deny, while others should return `True` to deny and `False` to allow. With this PR, all callbacks return `ALLOW` to allow or a `Codes` (typically `Codes.FORBIDDEN`) to deny. - Similarly, some methods returned `True` to allow, `False` to deny, while others returned `True` to deny and `False` to allow. They now all return `ALLOW` to allow or a `Codes` to deny. - Spam-checker implementations may now return an explicit code, e.g. to differentiate between "User account has been suspended" (which is in practice required by law in some countries, including UK) and "This message looks like spam".
2022-05-10Fix `/messages` throwing a 500 when querying for non-existent room (#12683)Eric Eastwood3-16/+13
Fix https://github.com/matrix-org/synapse/issues/12678 Complement test added: https://github.com/matrix-org/complement/pull/369 **Before:** 500 internal server error **After:** According to the [spec](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3roomsroomidmessages), calling `/messages` against a non-existent `room_id` should throw a 403 forbidden (since you're not part of the room). This also matches the behavior before https://github.com/matrix-org/synapse/pull/12370 which regressed Synapse to the 500 behavior. ```json { "errcode": "M_FORBIDDEN", "error": "User @test:my.synapse.server not in room !dne:my.synapse.server, and room previews are disabled" } ```
2022-05-10Refactor `EventContext` (#12689)Erik Johnston13-200/+70
Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing. The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching). One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used. Part of #12684
2022-05-10Capture the `Deferred` for request cancellation in `_AsyncResource` (#12694)Sean Quah3-5/+9
All async request processing goes through `_AsyncResource`, so this is the only place where a `Deferred` needs to be captured for cancellation. Unfortunately, the same isn't true for determining whether a request can be cancelled. Each of `RestServlet`, `BaseFederationServlet`, `DirectServe{Html,Json}Resource` and `ReplicationEndpoint` have different wrappers around the method doing the request handling and they all need to be handled separately. Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10Fix incorrect type hint in filtering code. (#12695)Patrick Cloke2-2/+4
2022-05-10Add some type hints to datastore. (#12477)Dirk Klimpel4-71/+122
2022-05-10Stop writing to `event_reference_hashes` (#12679)Richard van der Hoff5-38/+5
This table is never read, since #11794. We stop writing to it; in future we can drop it altogether.
2022-05-10Add helper class for testing request cancellation (#12630)Sean Quah4-0/+127
Also expose the `SynapseRequest` from `FakeChannel` in tests, so that we can call `Request.connectionLost` to simulate a client disconnecting. Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10Add ability to cancel disconnected requests to `SynapseRequest` (#12588)Sean Quah2-1/+24
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10Add `@cancellable` decorator, for use on request handlers (#12586)Sean Quah2-0/+62
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10Move free functions into PushRuleEvaluatorForEvent. (#12677)Patrick Cloke3-34/+69
* Move `_condition_checker` into `PushRuleEvaluatorForEvent`. * Move the condition cache into `PushRuleEvaluatorForEvent`. * Improve docstrings. * Inline a method which is only called once.
2022-05-10Add class-diagrams and notes for push. (#12676)Patrick Cloke2-0/+80
2022-05-10other fixes v1.59.0rc1David Robertson1-2/+2
2022-05-10backquote `m.room.server_acl`David Robertson1-1/+1
2022-05-10v1 -> 1David Robertson1-4/+4
2022-05-10Fix deprecation noticeDavid Robertson1-1/+1
2022-05-10I manually added O's change, remove newsfileDavid Robertson1-1/+0
2022-05-10Add Olivier's last-minute mergeDavid Robertson1-0/+1
2022-05-10Group release script changesDavid Robertson1-1/+1
2022-05-10Fix changelog linkDavid Robertson1-1/+1
2022-05-10Add the `update_user_directory_from_worker` configuration option ↵reivilibre9-34/+76
(superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. (#12654) Co-authored-by: Shay <hillerys@element.io>
2022-05-10Changelog typoDavid Robertson1-1/+1
2022-05-10Adjust changelogDavid Robertson1-25/+29
2022-05-101.59.0rc1David Robertson65-66/+96
2022-05-10Immediately retry any requests that have backed off when a server comes back ↵Erik Johnston6-6/+179
online. (#12500) Otherwise it can take up to a minute for any in-flight `/send` requests to be retried.
2022-05-10Implement MSC3786: Add a default push rule to ignore m.room.server_acl ↵Šimon Brandner4-11/+56
events (#12601) Fixes vector-im/element-web#20788 Implements matrix-org/matrix-spec-proposals#3786
2022-05-09Update `replication.md` with info on TCP module structure (#12621)Shay3-1/+7
2022-05-09Update SQL statements in docs for Synapse Admins (#12536)Dirk Klimpel2-70/+122
2022-05-09Tweaks to workers-under-complement (#12637)Richard van der Hoff3-5/+15
* Bump the HS startup timeout * Log prefixes for more processes * Bump the overall timeout
2022-05-09Fix inconsistent spelling of 'M_UNRECOGNIZED'. (#12665)Val Lorentz2-1/+2
2022-05-09Implement cancellation support/protection for module callbacks (#12568)Sean Quah6-27/+86
There's no guarantee that module callbacks will handle cancellation appropriately. Protect module callbacks with read semantics from cancellation and avoid swallowing `CancelledError`s that arise. Other module callbacks, such as the `on_*` callbacks, are presumed to live on code paths that involve writes and aren't cancellation-friendly. These module callbacks have been left alone. Signed-off-by: Sean Quah <seanq@element.io>
2022-05-09Fix mypy against latest pillow stubs (#12671)David Robertson3-6/+7
2022-05-09Update changelog for #12587 to be more accurate (#12663)Sean Quah2-1/+2
#12587 has fallen on the wrong side of the release cutoff to the rest of the related PRs. Signed-off-by: Sean Quah <seanq@element.io>
2022-05-09Use `ParamSpec` in a few places (#12667)David Robertson14-68/+148
2022-05-09Don't error on unknown receipt types (#12670)Erik Johnston2-12/+16
Fixes #12669
2022-05-09docs(contrib): Add link to documentation in dashboard (#12602)Sheogorath2-1/+14
2022-05-09Fix docs on how to run specific Complement tests after recent ↵Eric Eastwood2-2/+3
`complement.sh` change (#12664)
2022-05-09Use `Concatenate` to annotate `do_execute` (#12666)David Robertson3-6/+16
2022-05-07Move `pympler` back into the `all` extras (#12652)David Robertson5-6/+15
* Move `pympler` back into the `all` extras Undoes a change I made in #12381. I can't fully remember my reasoning, but this changed the contents of the debian packages in a backwards incompatible way. We're not aware of anyone who's been bitten by this, but we still want to fix it. To the reviewer: please be convinced that the debian packages will still contain pympler after this change. * Debian changelog entry to keep the linter happy
2022-05-07Convert stringy power levels to integers on room upgrade (#12657)David Robertson5-24/+137
2022-05-06Prevent memory leak from reoccurring when presence is disabled. (#12656)Erik Johnston3-28/+54
2022-05-06Update mypy to 0.950 and fix complaints (#12650)David Robertson10-57/+98
2022-05-06Add the `notify_appservices_from_worker` configuration option (superseding ↵reivilibre9-21/+447
`notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452)
2022-05-05Support MSC3266 room summaries over federation (#11507)DeepBlueV7.X4-5/+78
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
2022-05-05Update v1.58.1 changelog entry with more familiar language v1.58.1Andrew Morgan1-2/+2
2022-05-05link to relevant bug report in v1.58.1 changelogAndrew Morgan1-1/+2
2022-05-05Minor wording change to v1.58.1 release notesAndrew Morgan1-1/+1
2022-05-051.58.1Andrew Morgan3-3/+15
2022-05-05Include extra dependency groups 'systemd' and 'cache_memory' in debian ↵Andrew Morgan2-1/+13
packages (#12640) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-05-05Update `_on_new_receipts()` to work with MSC2285 changes. (#12636)Šimon Brandner2-5/+4
2022-05-05Improve comments and error messages around access tokens. (#12577)reivilibre2-8/+12
2022-05-05Use `docker/metadata-action` to generate docker image tags (#12573)Henry2-19/+12
Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Signed-off-by: Henry <97804910+henryclw@users.noreply.github.com>
2022-05-05Use `private` instead of `hidden` in MSC2285 related code. (#12635)Šimon Brandner7-25/+26
2022-05-05Edits/annotations should not have any bundled aggregations calculated. (#12633)Patrick Cloke3-20/+50
Fixes a regression from 8b309adb436c162510ed1402f33b8741d71fc058 (#11660) and b65acead428653b988351ae8d7b22127a22039cd (#11752) where events which themselves were an edit or an annotation could have bundled aggregations calculated, which is not allowed.
2022-05-05Remove unused receipt datastore methods. (#12632)Patrick Cloke2-54/+1
The last usage was removed in 5a1dd297c3ce105a7f516d9d9fe87b94b9d356c8 (#8059).
2022-05-05Fix typo in some instances of enable_registration_token_3pid_bypass. (#12639)Will Hunt4-4/+5
2022-05-05Reduce log spam when running multiple event persisters (#12610)Erik Johnston3-2/+17
2022-05-04Add `mau_appservice_trial_days` config (#12619)Will Hunt6-2/+110
* Add mau_appservice_trial_days * Add a test * Tweaks * changelog * Ensure we sync after the delay * Fix types * Add config statement * Fix test * Reinstate logging that got removed * Fix feature name
2022-05-04Use `getClientAddress` instead of `getClientIP`. (#12599)Patrick Cloke16-46/+62
getClientIP was deprecated in Twisted 18.4.0, which also added getClientAddress. The Synapse minimum version for Twisted is currently 18.9.0, so all supported versions have the new API.
2022-05-04Implement changes to MSC2285 (hidden read receipts) (#12168)Šimon Brandner12-186/+647
* Changes hidden read receipts to be a separate receipt type (instead of a field on `m.read`). * Updates the `/receipts` endpoint to accept `m.fully_read`.
2022-05-04Disable device name lookup over federation by default (#12616)Andrew Morgan5-12/+24
2022-05-04Remove unstable/unspecced login types. (#12597)Patrick Cloke6-20/+14
* `m.login.jwt`, which was never specced and has been deprecated since Synapse 1.16.0. (`org.matrix.login.jwt` can be used instead.) * `uk.half-shot.msc2778.login.application_service`, which was stabilized as part of the Matrix spec v1.2 release.
2022-05-04Improve logging for cancelled requests (#12587)Sean Quah3-1/+32
Don't log stack traces for cancelled requests and use a custom HTTP status code of 499. Signed-off-by: Sean Quah <seanq@element.io>
2022-05-04Include bundled aggregations for the latest event in a thread. (#12273)Patrick Cloke5-51/+198
The `latest_event` field of the bundled aggregations for `m.thread` relations did not include bundled aggregations itself. This resulted in clients needing to immediately request the event from the server (and thus making it useless that the latest event itself was serialized instead of just including an event ID).
2022-05-04remove constantly lib use and switch to enums. (#12624)andrew do11-38/+36
2022-05-04Fixes to the formatting of README.rst (#12627)Richard van der Hoff2-7/+8
Fixes a couple of formatting errors which were introduced in #12475.
2022-05-03Add a consistency check on events read from the database (#12620)Richard van der Hoff3-22/+50
I've seen a few errors which can only plausibly be explained by the calculated event id for an event being different from the ID of the event in the database. It should be cheap to check this, so let's do so and raise an exception.
2022-05-03Bump Synapse minimum Python version to 3.7.1 (#12613)David Robertson3-3/+4
2022-05-03Remove unstable identifiers for MSC3069. (#12596)Patrick Cloke3-8/+1
2022-05-03Add extra debug logging to federation sender (#12614)Richard van der Hoff2-2/+19
... in order to debug some problems we've been having with certain events not being sent when expected.
2022-05-03Add missing space before 'docker' link in release announcement script (#12612)Andrew Morgan2-1/+2
2022-05-03fix importsRichard van der Hoff1-1/+2
broken in 5938928 :-S
2022-05-03minor wording fix in docstringRichard van der Hoff1-3/+2
2022-05-03Exclude OOB memberships from the federation sender (#12570)Richard van der Hoff3-4/+51
As the comment says, there is no need to process such events, and indeed we need to avoid doing so. Fixes #12509.
2022-05-03Add sanity checks to the release script (#12556)David Robertson2-26/+38
Check we're on the right branch before tagging, and on the right tag before uploading * Abort if we're on the wrong branch * Check we have the right tag checked out * Clarify that `publish` only releases to GitHub
2022-05-03Fix race when persisting an event and deleting a room (#12594)Erik Johnston3-2/+22
This works by taking a row level lock on the `rooms` table at the start of both transactions, ensuring that they don't run at the same time. In the event persistence transaction we also check that there is an entry still in the `rooms` table. I can't figure out how to do this in SQLite. I was just going to lock the table, but it seems that we don't support that in SQLite either, so I'm *really* confused as to how we maintain integrity in SQLite when using `lock_table`....
2022-05-03Prune mypy ignore_missing_imports list (#12608)David Robertson2-51/+3
2022-05-03Move groups/communities deprecation notice to 1.58.0 heading v1.58.0Andrew Morgan1-2/+2
2022-05-031.58.0Andrew Morgan3-1/+13
2022-04-29Remove special-case for `twisted` logger (#12589)Richard van der Hoff3-14/+1
This was originally added when we first added a `MemoryHandler` to the default log config back in https://github.com/matrix-org/synapse/pull/8040, to ensure that we didn't explode with an infinite loop if there was an error formatting the logs. Since then, we made additional improvements to logging which make this workaround redundant. In particular: * we no longer attempt to log un-UTF8-decodable byte sequences, which were the most likely cause of an error in the first place. * https://github.com/matrix-org/synapse/pull/8268 ensures that in the unlikely case that there *is* an error, it won't cause an infinite loop.
2022-04-29Allow unused ignores in "bleeding edge" CI (#12576)David Robertson3-0/+6
* Allow unused ignores in "bleeding edge" CI Where "bleeding edge" means the Twisted Trunk and Latest Deps jobs. Follow up from #12531. Resolves #12574. * Use `--extras all` in latest deps mypy CI Twisted trunk job already does this. Missed in #12531. * changelog
2022-04-28Improve the docstrings for the receipts store. (#12581)Patrick Cloke2-5/+52
2022-04-28Use constants for receipt types in tests. (#12582)Šimon Brandner4-23/+30
2022-04-28Fix logging of incorrect status codes for disconnected requests (#12580)Sean Quah4-5/+36
The status code of requests must always be set, regardless of client disconnection, otherwise they will always be logged as 200!. Broken for `respond_with_json` in f48792eec43f893f4f893ffdcbf00f8958b6f6b5. Broken for `respond_with_json_bytes` in 3e58ce72b42f2ae473c1e76a967548cd6fa7e2e6. Broken for `respond_with_html_bytes` in ea26e9a98b0541fc886a1cb826a38352b7599dbe. Signed-off-by: Sean Quah <seanq@element.io>
2022-04-28