summary refs log tree commit diff
path: root/synapse/events (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Reorganize Pydantic models and types used in handlers (#17279)Eric Eastwood2024-06-101-1/+1
| | | | | | Spawning from https://github.com/element-hq/synapse/pull/17187#discussion_r1619492779 around wanting to put `SlidingSyncBody` (parse the request in the rest layer), `SlidingSyncConfig` (from the rest layer, pass to the handler), `SlidingSyncResponse` (pass the response from the handler back to the rest layer to respond) somewhere that doesn't contaminate the imports and cause circular import issues. - Moved Pydantic parsing models to `synapse/types/rest` - Moved handler types to `synapse/types/handlers`
* Bring auto-accept invite logic into Synapse (#17147)devonh2024-05-211-0/+196
| | | | | | | | | | | | | | This PR ports the logic from the [synapse_auto_accept_invite](https://github.com/matrix-org/synapse-auto-accept-invite) module into synapse. I went with the naive approach of injecting the "module" next to where third party modules are currently loaded. If there is a better/preferred way to handle this, I'm all ears. It wasn't obvious to me if there was a better location to add this logic that would cleanly apply to all incoming invite events. Relies on https://github.com/element-hq/synapse/pull/17166 to fix linter errors.
* Add support for MSC4115 (#17104)Richard van der Hoff2024-04-291-5/+25
| | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Bump black from 23.10.1 to 24.2.0 (#16936)dependabot[bot]2024-03-132-11/+7
|
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-236-0/+8
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Port `EventInternalMetadata` class to Rust (#16782)Erik Johnston2024-01-082-117/+11
| | | | | | | | | | | | | There are a couple of things we need to be careful of here: 1. The current python code does no validation when loading from the DB, so we need to be careful to ignore such errors (at least on jki.re there are some old events with internal metadata fields of the wrong type). 2. We want to be memory efficient, as we often have many hundreds of thousands of events in the cache at a time. --------- Co-authored-by: Quentin Gliech <quenting@element.io>
* Simplify internal metadata class. (#16762)Erik Johnston2024-01-051-8/+1
| | | | | | | | | We remove these fields as they're just duplicating data the event already stores, and (for reasons :shushing_face:) I'd like to simplify the class to only store simple types. I'm not entirely convinced that we shouldn't instead add helper methods to the event class to generate stream tokens, but I don't really think that's where they belong either
* Update book locationErik Johnston2023-12-131-1/+1
|
* Update license headersPatrick Cloke2023-11-216-63/+96
|
* Add new module API for adding custom fields to events `unsigned` section ↵Erik Johnston2023-10-271-7/+41
| | | | (#16549)
* Add a cache around server ACL checking (#16360)Patrick Cloke2023-09-261-2/+5
| | | | | * Pre-compiles the server ACLs onto an object per room and invalidates them when new events come in. * Converts the server ACL checking into Rust.
* Add support for pydantic v2 via pydantic.v1 compat module (#16332)Maxwell G2023-09-251-2/+8
| | | While maintaining support with pydantic v1.
* Return an immutable value from get_latest_event_ids_in_room. (#16326)Patrick Cloke2023-09-181-1/+1
|
* Use StrCollection in additional places. (#16301)Patrick Cloke2023-09-133-10/+9
|
* Update ruff config (#16283)Patrick Cloke2023-09-081-2/+0
| | | Enable additional checks & clean-up unneeded configuration.
* Bump ruff from 0.0.277 to 0.0.286 (#16198)dependabot[bot]2023-08-292-4/+4
| | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Drop backwards compat hack for event serialization (#16069)Shay2023-08-071-11/+1
|
* Stabilize support for MSC3970: updated transaction semantics (scope to ↵Patrick Cloke2023-08-041-21/+21
| | | | | | | `device_id`) (#15629) For now this maintains compatible with old Synapses by falling back to using transaction semantics on a per-access token. A future version of Synapse will drop support for this.
* Add forward-compatibility for the redacts property (MSC2174). (#16013)Patrick Cloke2023-08-021-8/+10
| | | | | | | | The location of the redacts field changes in room version 11. Ensure it is copied to the *new* location for *old* room versions for forwards-compatibility with clients. Note that copying it to the *old* location for the *new* room version was previously handled.
* Properly handle redactions of creation events (#15973)Shay2023-07-231-3/+5
|
* Support room version 11 (#15912)Patrick Cloke2023-07-183-15/+20
| | | | | And fix a bug in the implementation of the updated redaction format (MSC2174) where the top-level redacts field was not properly added for backwards-compatibility.
* Remove experimental MSC2716 implementation to incrementally import history ↵Eric Eastwood2023-06-162-18/+0
| | | | | | | | | | | | into existing rooms (#15748) Context for why we're removing the implementation: - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734 Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737 Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
* Replace `EventContext` fields `prev_group` and `delta_ids` with field ↵Shay2023-06-131-49/+110
| | | | `state_group_deltas` (#15233)
* Stabilize support for MSC3952: Intentional mentions. (#15520)Patrick Cloke2023-06-061-7/+2
|
* Properly parse event_fields in filters (#15607)Patrick Cloke2023-05-221-14/+58
| | | | | | | | The event_fields property in filters should use the proper escape rules, namely backslashes can be escaped with an additional backslash. This adds tests (adapted from matrix-js-sdk) and implements the logic to properly split the event_fields strings.
* Instrument `state` and `state_group` storage related things (tracing) (#15610)Eric Eastwood2023-05-191-0/+5
| | | | | Instrument `state` and `state_group` storage related things (tracing) so it's a little more clear where these database transactions are coming from as there is a lot of wires crossing in these functions. Part of `/messages` performance investigation: https://github.com/matrix-org/synapse/issues/13356
* Implement MSC3821 to update redaction rules (`third_party_invite.signed`) ↵Patrick Cloke2023-05-151-0/+10
| | | | | | (#15563) Updates the redaction rules to protect enough information that the event can still be properly verified.
* Implement MSC3389 to protect relations from redaction. (#15565)Patrick Cloke2023-05-151-0/+12
| | | | | | MSC3389 proposes protecting the relation type & parent event ID from redaction. This keeps the relation information intact after redaction which helps with some UX flaws (e.g. deleting an event causes it to no longer be in a thread, which is confusing).
* Move ThirdPartyEventRules into module_api/callbacks (#15535)Andrew Morgan2023-05-041-593/+0
|
* Experimental support for MSC3970: per-device transaction IDs (#15318)Quentin Gliech2023-04-252-17/+50
|
* Move Spam Checker callbacks to a dedicated file (#15453)Andrew Morgan2023-04-181-822/+0
|
* Implement MSC2174: move redacts to a content property. (#15395)Patrick Cloke2023-04-132-2/+10
| | | | | | | This moves `redacts` from being a top-level property to a `content` property in a new room version. MSC2176 (which was previously implemented) states to not `redact` this property.
* Implement MSC3989 to redact the origin field. (#15393)Patrick Cloke2023-04-051-1/+4
| | | | This will be done in a future room version, for now an unstable room version is added which redacts the origin field.
* Bump ruff from 0.0.252 to 0.0.259 (#15328)dependabot[bot]2023-03-282-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Bump ruff from 0.0.252 to 0.0.259 Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.252 to 0.0.259. - [Release notes](https://github.com/charliermarsh/ruff/releases) - [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md) - [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.252...v0.0.259) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Fix new warnings * Mypy * Newsfile --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
* Reject mentions on the C-S API which are invalid. (#15311)Patrick Cloke2023-03-241-10/+32
| | | | | | Invalid mentions data received over the Client-Server API should be rejected with a 400 error. This will hopefully stop clients from sending invalid data, although does not help with data received over federation.
* Use immutabledict instead of frozendict (#15113)David Robertson2023-03-223-4/+4
| | | | | | | | | | Additionally: * Consistently use `freeze()` in test --------- Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: 6543 <6543@obermui.de>
* Merge branch 'release-v1.79' into developDavid Robertson2023-03-131-0/+5
|\
| * Fix missing conditional for registering ↵Andrew Morgan2023-03-101-0/+5
| | | | | | | | `on_remove_user_third_party_identifier` module api callbacks (#15227
* | More speedups/fixes to creating batched events (#15195)Shay2023-03-071-0/+1
|/
* Pass the requester during event serialization. (#15174)Quentin Gliech2023-03-061-8/+22
| | | | This allows Synapse to properly include the transaction ID in the unsigned data of events.
* Stop applying edits to event contents (MSC3925). (#15193)Patrick Cloke2023-03-061-55/+2
| | | | | | | | | | | Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.
* Remove support for aggregating reactions (#15172)Richard van der Hoff2023-02-281-5/+0
| | | | | | | | | | It turns out that no clients rely on server-side aggregation of `m.annotation` relationships: it's just not very useful as currently implemented. It's also non-trivial to calculate. I want to remove it from MSC2677, so to keep the implementation in line, let's remove it here.
* Add module API callbacks for adding and deleting local 3PID associations (#15044Andrew Morgan2023-02-271-0/+63
|
* Refactor media modules. (#15146)Patrick Cloke2023-02-271-2/+2
| | | | | | | * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
* Batch up storing state groups when creating new room (#14918)Shay2023-02-241-0/+49
|
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-221-2/+0
|
* Change collection[str] to StrCollection in event_auth code (#14929)Harishankar Kumar2023-02-141-3/+3
| | | Signed-off-by: Harishankar Kumar <hari01584@gmail.com>
* Return read-only collections from `@cached` methods (#13755)Sean Quah2023-02-101-3/+3
| | | | | | | | | | | | | It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>
* Add a class UnpersistedEventContext to allow for the batching up of storing ↵Shay2023-02-092-6/+174
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | state groups (#14675) * add class UnpersistedEventContext * modify create new client event to create unpersistedeventcontexts * persist event contexts after creation * fix tests to persist unpersisted event contexts * cleanup * misc lints + cleanup * changelog + fix comments * lints * fix batch insertion? * reduce redundant calculation * add unpersisted event classes * rework compute_event_context, split into function that returns unpersisted event context and then persists it * use calculate_context_info to create unpersisted event contexts * update typing * $%#^&* * black * fix comments and consolidate classes, use attr.s for class * requested changes * lint * requested changes * requested changes * refactor to be stupidly explicit * clearer renaming and flow * make partial state non-optional * update docstrings --------- Co-authored-by: Erik Johnston <erik@matrix.org>
* Reject boolean power levels (#14944)David Robertson2023-01-311-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Better test for bad values in power levels events The previous test only checked that Synapse didn't raise an exception, but didn't check that we had correctly interpreted the value of the dodgy power level. It also conflated two things: bad room notification levels, and bad user levels. There _is_ logic for converting the latter to integers, but we should test it separately. * Check we ignore types that don't convert to int * Handle `None` values in `notifications.room` * Changelog * Also test that bad values are rejected by event auth * Docstring * linter scripttttttttt * Test boolean values in PL content * Reject boolean power levels * Changelog
* Prefer `type(x) is int` to `isinstance(x, int)` (#14945)David Robertson2023-01-311-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Perfer `type(x) is int` to `isinstance(x, int)` This covered all additional instances I could see where `x` was user-controlled. The remaining cases are ``` $ rg -s 'isinstance.*[^_]int' tests/replication/_base.py 576: if isinstance(obj, int): synapse/util/caches/stream_change_cache.py 136: assert isinstance(stream_pos, int) 214: assert isinstance(stream_pos, int) 246: assert isinstance(stream_pos, int) 267: assert isinstance(stream_pos, int) synapse/replication/tcp/external_cache.py 133: if isinstance(result, int): synapse/metrics/__init__.py 100: if isinstance(calls, (int, float)): synapse/handlers/appservice.py 262: assert isinstance(new_token, int) synapse/config/_util.py 62: if isinstance(p, int): ``` which cover metrics, logic related to `jsonschema`, and replication and data streams. AFAICS these are all internal to Synapse * Changelog
* Add missing type hints for tests.events. (#14904)Patrick Cloke2023-01-251-1/+2
|
* Implement MSC3925: changes to bundling of edits (#14811)Richard van der Hoff2023-01-101-7/+24
| | | | | | | | Two parts to this: * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag. * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.
* Allow selecting "prejoin" events by state keys (#14642)David Robertson2022-12-131-1/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Declare new config * Parse new config * Read new config * Don't use trial/our TestCase where it's not needed Before: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m2.277s user 0m2.186s sys 0m0.083s ``` After: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m0.566s user 0m0.508s sys 0m0.056s ``` * Helper to upsert to event fields without exceeding size limits. * Use helper when adding invite/knock state Now that we allow admins to include events in prejoin room state with arbitrary state keys, be a good Matrix citizen and ensure they don't accidentally create an oversized event. * Changelog * Move StateFilter tests should have done this in #14668 * Add extra methods to StateFilter * Use StateFilter * Ensure test file enforces typed defs; alphabetise * Workaround surprising get_current_state_ids * Whoops, fix mypy
* Move `StateFilter` to `synapse.types` (#14668)David Robertson2022-12-122-2/+2
| | | | | * Move `StateFilter` to `synapse.types` * Changelog
* Faster joins: do not wait for full state when creating events to send (#14403)Mathieu Velten2022-11-171-0/+1
| | | Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-2/+1
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Fix docstring in EventContext (#14145)Shay2022-10-181-1/+2
|
* fix: Push notifications for invite over federation (#13719)Kateřina Churanová2022-09-281-0/+4
|
* Don't include redundant prev_state in new events (#13791)Denis2022-09-201-1/+0
|
* Rename the `EventFormatVersions` enum values so that they line up with room ↵reivilibre2022-09-073-9/+9
| | | | version numbers. (#13706)
* Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child ↵Eric Eastwood2022-08-231-0/+2
| | | | | | | | | concurrent calls (#13588) Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls because I've see `_check_sigs_and_hash_and_fetch` take [10.41s to process 100 events](https://github.com/matrix-org/synapse/issues/13587) Fix https://github.com/matrix-org/synapse/issues/13587 Part of https://github.com/matrix-org/synapse/issues/13356
* MSC2716v4 room version - remove namespace from MSC2716 event content fields ↵Eric Eastwood2022-08-191-1/+1
| | | | | | | | (#13551) Complement PR: https://github.com/matrix-org/complement/pull/450 As suggested in https://github.com/matrix-org/matrix-spec-proposals/pull/2716#discussion_r941444525
* Update type of `EventContext.rejected` (#13460)Richard van der Hoff2022-08-041-4/+3
|
* Don't pull out the full state when creating an event (#13281)Erik Johnston2022-07-181-1/+7
|
* Use state before join to determine if we `_should_perform_remote_join` (#13270)David Robertson2022-07-151-1/+1
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Uniformize spam-checker API, part 5: expand other spam-checker callbacks to ↵David Teller2022-07-111-35/+128
| | | | | | return `Tuple[Codes, dict]` (#13044) Signed-off-by: David Teller <davidt@element.io> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Make `_get_state_map_for_room` not break when room state events don't ↵David Teller2022-07-071-8/+1
| | | | | contain an event id. (#13174) Method `_get_state_map_for_room` seems to break in presence of some ill-formed events in the database. Reimplementing this method to use `get_current_state`, which is more robust to such events.
* Uniformize spam-checker API, part 4: port other spam-checker callbacks to ↵David Teller2022-06-131-80/+197
| | | | | return `Union[Allow, Codes]`. (#12857) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Remove `room_version` param from `validate_event_for_room_version`Richard van der Hoff2022-06-121-0/+4
| | | | | | | Instead, use the `room_version` property of the event we're validating. The `room_version` was originally added as a parameter somewhere around #4482, but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
* Wait for lazy join to complete when getting current state (#12872)Erik Johnston2022-06-011-1/+2
|
* Merge branch 'master' into developErik Johnston2022-05-311-20/+27
|\
| * Fix import in module_api module and docs on the new check_event_for_spam ↵Brendan Abolivier2022-05-311-22/+27
| | | | | | | | | | signature (#12918) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Rename storage classes (#12913)Erik Johnston2022-05-311-5/+5
| |
* | Uniformize spam-checker API, part 3: Expand check_event_for_spam with the ↵David Teller2022-05-301-7/+13
| | | | | | | | | | ability to return additional fields (#12846) Signed-off-by: David Teller <davidt@element.io>
* | Bump jsonschema stubs (#12912)David Robertson2022-05-301-2/+7
|/
* Uniformize spam-checker API, part 2: check_event_for_spam (#12808)David Teller2022-05-231-10/+39
| | | Signed-off-by: David Teller <davidt@element.io>
* add SpamChecker callback for silently dropping inbound federated events (#12744)Jess Porter2022-05-231-0/+40
| | | Signed-off-by: jesopo <github@lolnerd.net>
* Update EventContext `get_current_event_ids` and `get_prev_event_ids` to ↵Shay2022-05-201-4/+15
| | | | accept state filters and update calls where possible (#12791)
* Consolidate logic for parsing relations. (#12693)Patrick Cloke2022-05-161-0/+45
| | | | | | | | | | | | | Parse the `m.relates_to` event content field (which describes relations) in a single place, this is used during: * Event persistence. * Validation of the Client-Server API. * Fetching bundled aggregations. * Processing of push rules. Each of these separately implement the logic and each made slightly different assumptions about what was valid. Some had minor / potential bugs.
* SpamChecker metrics (#12513)Jess Porter2022-05-131-23/+58
| | | | | | | | | * add Measure blocks all over SpamChecker Signed-off-by: jesopo <github@lolnerd.net> * fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck * better changelog entry
* Refactor `EventContext` (#12689)Erik Johnston2022-05-101-145/+32
| | | | | | | | | | Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing. The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching). One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used. Part of #12684
* Add some type hints to datastore. (#12477)Dirk Klimpel2022-05-101-1/+2
|
* Implement cancellation support/protection for module callbacks (#12568)Sean Quah2022-05-093-20/+64
| | | | | | | | | | | | There's no guarantee that module callbacks will handle cancellation appropriately. Protect module callbacks with read semantics from cancellation and avoid swallowing `CancelledError`s that arise. Other module callbacks, such as the `on_*` callbacks, are presumed to live on code paths that involve writes and aren't cancellation-friendly. These module callbacks have been left alone. Signed-off-by: Sean Quah <seanq@element.io>
* Use `ParamSpec` in a few places (#12667)David Robertson2022-05-091-3/+12
|
* Convert stringy power levels to integers on room upgrade (#12657)David Robertson2022-05-071-16/+45
|
* Update mypy to 0.950 and fix complaints (#12650)David Robertson2022-05-061-1/+3
|
* Include bundled aggregations for the latest event in a thread. (#12273)Patrick Cloke2022-05-041-22/+33
| | | | | | The `latest_event` field of the bundled aggregations for `m.thread` relations did not include bundled aggregations itself. This resulted in clients needing to immediately request the event from the server (and thus making it useless that the latest event itself was serialized instead of just including an event ID).
* fix importsRichard van der Hoff2022-05-031-1/+2
| | | broken in 5938928 :-S
* minor wording fix in docstringRichard van der Hoff2022-05-031-3/+2
|
* Exclude OOB memberships from the federation sender (#12570)Richard van der Hoff2022-05-031-4/+11
| | | | | | | As the comment says, there is no need to process such events, and indeed we need to avoid doing so. Fixes #12509.
* Consistently use collections.abc.Mapping to check frozendict. (#12564)Patrick Cloke2022-04-271-3/+4
|
* Misc. clean-ups to the relations code (#12519)Patrick Cloke2022-04-251-1/+1
| | | | | * Corrects some typos / copy & paste errors in tests. * Clarifies docstrings. * Removes an unnecessary method.
* Fix `/room/.../event/...` to return the *original* event after any edits ↵Richard van der Hoff2022-04-191-8/+13
| | | | | | (#12476) This is what the MSC (now) requires. Fixes https://github.com/matrix-org/synapse/issues/10310.
* Remove references to unstable identifiers from MSC3440. (#12382)Patrick Cloke2022-04-121-6/+0
| | | | | Removes references to unstable thread relation, unstable identifiers for filtering parameters, and the experimental config flag.
* Add a callback to react to 3PID associations (#12302)Brendan Abolivier2022-03-311-0/+26
|
* Fix typechecker problems exposed by signedjson 1.1.2 (#12326)David Robertson2022-03-291-1/+1
|
* Bump `black` and `click` versions (#12320)David Robertson2022-03-291-1/+1
|
* Move get_bundled_aggregations to relations handler. (#12237)Patrick Cloke2022-03-181-1/+1
| | | | | The get_bundled_aggregations code is fairly high-level and uses a lot of store methods, we move it into the handler as that seems like a better fit.
* Correct `check_username_for_spam` annotations and docs (#12246)David Robertson2022-03-181-4/+3
| | | | | | | * Formally type the UserProfile in user searches * export UserProfile in synapse.module_api * Update docs Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Fix bundling aggregations if unsigned is not a returned event field. (#12234)Patrick Cloke2022-03-161-3/+6
| | | | | | | An error occured if a filter was supplied with `event_fields` which did not include `unsigned`. In that case, bundled aggregations are still added as the spec states it is allowed for servers to add additional fields.
* fix misleading comment in `check_events_for_spam` (#12203)Shay2022-03-101-2/+2
|
* Support stable identifiers for MSC3440: Threading (#12151)Patrick Cloke2022-03-101-1/+8
| | | | The unstable identifiers are still supported if the experimental configuration flag is enabled. The unstable identifiers will be removed in a future release.
* Add third_party module callbacks to check if a user can delete a room and ↵Will Hunt2022-03-091-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deactivate a user (#12028) * Add check_can_deactivate_user * Add check_can_shutdown_rooms * Documentation * callbacks, not functions * Various suggested tweaks * Add tests for test_check_can_shutdown_room and test_check_can_deactivate_user * Update check_can_deactivate_user to not take a Requester * Fix check_can_shutdown_room docs * Renegade and use `by_admin` instead of `admin_user_id` * fix lint * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Update docs/modules/third_party_rules_callbacks.md Co-authored-by: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Merge remote-tracking branch 'origin/release-v1.54' into developRichard van der Hoff2022-03-041-3/+7
|\
| * Correctly register deactivation and profile update module callbacks (#12141)Brendan Abolivier2022-03-031-3/+7
| |
* | Use the proper serialization format when bundling aggregations. (#12090)Patrick Cloke2022-03-031-27/+54
|/ | | | This ensures that the `latest_event` field of the bundled aggregation for threads uses the same format as the other events in the response.
* Add module callbacks called for reacting to deactivation status change and ↵Brendan Abolivier2022-03-011-3/+53
| | | | profile update (#12062)
* Faster joins: persist to database (#12012)Richard van der Hoff2022-03-011-0/+9
| | | | | | | | | | | | When we get a partial_state response from send_join, store information in the database about it: * store a record about the room as a whole having partial state, and stash the list of member servers too. * flag the join event itself as having partial state * also, for any new events whose prev-events are partial-stated, note that they will *also* be partial-stated. We don't yet make any attempt to interpret this data, so API calls (and a bunch of other things) are just going to get incorrect data.
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-232-2/+2
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Fix incorrect thread summaries when the latest event is edited. (#11992)Patrick Cloke2022-02-151-24/+45
| | | | | If the latest event in a thread was edited than the original event content was included in bundled aggregation for threads instead of the edited event content.
* Remove deprecated user_may_create_room_with_invites callback (#11950)Brendan Abolivier2022-02-111-42/+0
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Improvements to bundling aggregations. (#11815)Patrick Cloke2022-01-261-17/+40
| | | | | | | | | | | This is some odds and ends found during the review of #11791 and while continuing to work in this code: * Return attrs classes instead of dictionaries from some methods to improve type safety. * Call `get_bundled_aggregations` fewer times. * Adds a missing assertion in the tests. * Do not return empty bundled aggregations for an event (preferring to not include the bundle at all, as the docstring states).
* Fix another jsonschema typecheck error (#11830)David Robertson2022-01-251-4/+2
| | | | | | | | | | | | | | | | | | | | | Similar to #11817. In `_create_power_level_validator` we - retrieve `validator`. This is a class implementing the `jsonschema.protocols.Validator` interface. In other words, `validator: Type[jsonschema.protocols.Validator]`. - we then create an second validator class by modifying the original `validator`. We return that class, which is also of type `Type[jsonschema.protocols.Validator]`. So the original annotation was incorrect: it claimed we were returning an instance of jsonSchema.Draft7Validator, not the class (or a subclass) itself. (Strictly speaking this is incorrect, because `POWER_LEVELS_SCHEMA` isn't pinned to a particular version of JSON Schema. But there are other complications with the type stubs if you try to fix this; I felt like the change herein was a decent compromise that better expresses intent). (I suspect/hope the typeshed project would welcome an effort to improve the jsonschema stubs. Let's see if I get some spare time.)
* Ignore the jsonschema type. (#11817)Patrick Cloke2022-01-251-1/+3
|
* Do not try to serialize raw aggregations dict. (#11791)Patrick Cloke2022-01-211-2/+2
|
* Add `FrozenEvent.get_state_key` and use it in a couple of places (#11793)Richard van der Hoff2022-01-212-5/+10
| | | | This is more efficient, since we only have to look up `state_key` in the event dict once, rather than three (!) times.
* Use auto_attribs/native type hints for attrs classes. (#11692)Patrick Cloke2022-01-131-9/+9
|
* Bundle aggregations outside of the serialization method. (#11612)Patrick Cloke2022-01-071-89/+37
| | | | | | | | This makes the serialization of events synchronous (and it no longer access the database), but we must manually calculate and provide the bundled aggregations. Overall this should cause no change in behavior, but is prep work for other improvements.
* Do not bundle aggregations for APIs which shouldn't include them. (#11592)Patrick Cloke2021-12-201-1/+1
| | | | | And make bundling aggregations opt-in, instead of opt-out to avoid having APIs to include extraneous data (and being much heavier than necessary).
* Do not allow cross-room relations, per MSC2674. (#11516)Patrick Cloke2021-12-091-4/+7
|
* Include bundled aggregations in /sync and related fixes (#11478)Patrick Cloke2021-12-061-23/+35
| | | | | | | | Due to updates to MSC2675 this includes a few fixes: * Include bundled aggregations for /sync. * Do not include bundled aggregations for /initialSync and /events. * Do not bundle aggregations for state events. * Clarifies comments and variable names.
* Bundle relations of relations into the `/relations` result. (#11284)Patrick Cloke2021-11-301-0/+8
| | | | | Per updates to MSC2675 which now states that bundled aggregations should be included from the `/relations` endpoint.
* Fix using MSC2716 batch sending with event persistence workers (#11220)Tulir Asokan2021-11-291-0/+5
| | | | Signed-off-by: Tulir Asokan <tulir@beeper.com>
* Refactor the code to inject bundled relations during serialization. (#11408)Patrick Cloke2021-11-231-64/+82
|
* Get db signatures file to pass mypy (#11312)David Robertson2021-11-111-7/+5
|
* Add remaining type hints to `synapse.events`. (#11098)Patrick Cloke2021-11-022-81/+148
|
* Make `check_event_allowed` module API callback not fail open (accept events) ↵reivilibre2021-11-011-4/+5
| | | | when an exception is raised (#11033)
* Implement an `on_new_event` callback (#11126)Brendan Abolivier2021-10-261-0/+31
| | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Add a thread relation type per MSC3440. (#11088)Patrick Cloke2021-10-211-0/+17
| | | | Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
* Include rejected status when we log events. (#11008)Richard van der Hoff2021-10-191-6/+10
| | | | | | If we find ourselves dealing with rejected events, we proably want to know about it. Let's include it in the stringification of the event so that it gets logged.
* Move experimental & retention config out of the server module. (#11070)Patrick Cloke2021-10-151-4/+2
|
* Add type hints to synapse.events.*. (#11066)Patrick Cloke2021-10-137-139/+177
| | | Except `synapse/events/__init__.py`, which will be done in a follow-up.
* Pass through `SynapseError`s that are raised from experimental ↵reivilibre2021-10-111-0/+9
| | | | | `check_event_allowed` callback of the module API (#11042) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Add a spamchecker method to allow or deny 3pid invites (#10894)Brendan Abolivier2021-10-061-0/+35
| | | | | This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
* Add a spamchecker callback to allow or deny room joins (#10910)Brendan Abolivier2021-10-061-0/+24
| | | Co-authored-by: Erik Johnston <erik@matrix.org>
* Strip "join_authorised_via_users_server" from join events which do not need ↵Patrick Cloke2021-09-301-1/+1
| | | | | | | it. (#10933) This fixes a "Event not signed by authorising server" error when transition room member from join -> join, e.g. when updating a display name or avatar URL for restricted rooms.
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-292-4/+4
|
* Use `RoomVersion` objects (#10934)Richard van der Hoff2021-09-291-20/+0
| | | Various refactors to use `RoomVersion` objects instead of room version identifiers.
* Add a spamchecker callback to allow or deny room creation based on invites ↵Brendan Abolivier2021-09-241-0/+42
| | | | | | | (#10898) This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
* Use direct references for configuration variables (part 5). (#10897)Patrick Cloke2021-09-242-3/+3
|
* Factor out a separate `EventContext.for_outlier` (#10883)Richard van der Hoff2021-09-221-4/+10
| | | | | | Constructing an EventContext for an outlier is actually really simple, and there's no sense in going via an `async` method in the `StateHandler`. This also means that we can resolve a bunch of FIXMEs.
* Include outlier status in `str(event)` for V2/V3 events (#10879)Richard van der Hoff2021-09-221-22/+12
| | | | | | | | | | | | | | | | | | | | | I meant to do this before, in #10591, but because I'm stupid I forgot to do it for V2 and V3 events. I've factored the common code out to `EventBase` to save us having two copies of it. This means that for `FrozenEvent` we replace `self.get("event_id", None)` with `self.event_id`, which I think is safe. `get()` is an alias for `self._dict.get()`, whereas `event_id()` is an `@property` method which looks up `self._event_id`, which is populated during construction from the same dict. We don't seem to rely on the fallback, because if the `event_id` key is absent from the dict then construction of the `EventBase` object will fail. Long story short, the only way this could change behaviour is if `event_dict["event_id"]` is changed *after* the `EventBase` object is constructed without updating the `_event_id` field, or vice versa - either of which would be very problematic anyway and the behavior of `str(event)` is the least of our worries.
* Rename MSC2716 things from `chunk` to `batch` to match `/batch_send` ↵Eric Eastwood2021-09-211-3/+3
| | | | | | | | endpoint (#10838) See https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684574497 Dropping support for older MSC2716 room versions so we don't have to worry about supporting both chunk and batch events.
* Use direct references for some configuration variables (#10798)Patrick Cloke2021-09-131-1/+1
| | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
* Support MSC3375: room version 9. (#10747)Patrick Cloke2021-09-031-0/+2
|
* Validate new m.room.power_levels events (#10232)Aaron Raimist2021-08-262-3/+79
| | | | Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Port the PresenceRouter module interface to the new generic interface (#10524)Azrenbeth2021-08-171-36/+156
| | | Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
* Clean up some logging in the federation event handler (#10591)Richard van der Hoff2021-08-161-1/+2
| | | | | | | | | | | | | | | | | | | * Include outlier status in `str(event)` In places where we log event objects, knowing whether or not you're dealing with an outlier is super useful. * Remove duplicated logging in get_missing_events When we process events received from get_missing_events, we log them twice (once in `_get_missing_events_for_pdu`, and once in `on_receive_pdu`). Reduce the duplication by removing the logging in `on_receive_pdu`, and ensuring the call sites do sensible logging. * log in `on_receive_pdu` when we already have the event * Log which prev_events we are missing * changelog
* When redacting, keep event fields around that maintain the historical event ↵Eric Eastwood2021-08-091-1/+7
| | | | | | | | | | | | | | | structure intact (MSC2716) (#10538) * Keep event fields that maintain the historical event structure intact Fix https://github.com/matrix-org/synapse/issues/10521 * Add changelog * Bump room version * Better changelog text * Fix up room version after develop merge
* Make historical events discoverable from backfill for servers without any ↵Eric Eastwood2021-07-281-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | scrollback history (MSC2716) (#10245) * Make historical messages available to federated servers Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 Follow-up to https://github.com/matrix-org/synapse/pull/9247 * Debug message not available on federation * Add base starting insertion point when no chunk ID is provided * Fix messages from multiple senders in historical chunk Follow-up to https://github.com/matrix-org/synapse/pull/9247 Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 --- Previously, Synapse would throw a 403, `Cannot force another user to join.`, because we were trying to use `?user_id` from a single virtual user which did not match with messages from other users in the chunk. * Remove debug lines * Messing with selecting insertion event extremeties * Move db schema change to new version * Add more better comments * Make a fake requester with just what we need See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080 * Store insertion events in table * Make base insertion event float off on its own See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889 Conflicts: synapse/rest/client/v1/room.py * Validate that the app service can actually control the given user See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455 Conflicts: synapse/rest/client/v1/room.py * Add some better comments on what we're trying to check for * Continue debugging * Share validation logic * Add inserted historical messages to /backfill response * Remove debug sql queries * Some marker event implemntation trials * Clean up PR * Rename insertion_event_id to just event_id * Add some better sql comments * More accurate description * Add changelog * Make it clear what MSC the change is part of * Add more detail on which insertion event came through * Address review and improve sql queries * Only use event_id as unique constraint * Fix test case where insertion event is already in the normal DAG * Remove debug changes * Switch to chunk events so we can auth via power_levels Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room. * Switch to chunk events for federation * Add unstable room version to support new historical PL * Fix federated events being rejected for no state_groups Add fix from https://github.com/matrix-org/synapse/pull/10439 until it merges. * Only connect base insertion event to prev_event_ids Per discussion with @erikjohnston, https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org * Make it possible to get the room_version with txn * Allow but ignore historical events in unsupported room version See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489 We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either. Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` -> `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`. * Move to unique index syntax See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509 * High-level document how the insertion->chunk lookup works * Remove create_event fallback for room_versions See https://github.com/matrix-org/synapse/pull/10245/files#r677641879 * Use updated method name
* Restricted rooms (MSC3083) should not have their allow key redacted. (#10489)Patrick Cloke2021-07-281-0/+2
|
* Port the ThirdPartyEventRules module interface to the new generic interface ↵Brendan Abolivier2021-07-201-42/+203
| | | | | (#10386) Port the third-party event rules interface to the generic module interface introduced in v1.37.0
* Do not include signatures/hashes in make_{join,leave,knock} responses. (#10404)Patrick Cloke2021-07-161-0/+14
| | | | These signatures would end up invalid since the joining/leaving/knocking server would modify the response before calling send_{join,leave,knock}.
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-153-23/+23
|
* Move methods involving event authentication to EventAuthHandler. (#10268)Patrick Cloke2021-07-011-5/+7
| | | Instead of mixing them with user authentication methods.
* Correct type hints for synapse.event_auth. (#10253)Patrick Cloke2021-06-302-34/+37
|
* Fix wrapping of legacy check_registration_for_spam (#10238)Brendan Abolivier2021-06-231-6/+7
| | | Fixes #10234
* Add endpoints for backfilling history (MSC2716) (#9247)Eric Eastwood2021-06-222-2/+24
| | | Work on https://github.com/matrix-org/matrix-doc/pull/2716
* Standardise the module interface (#10062)Brendan Abolivier2021-06-181-88/+218
| | | This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
* Implement knock feature (#6739)Sorunome2021-06-091-7/+12
| | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io
* Add missing type hints to handlers and fix a Spam Checker type hint. (#9896)Patrick Cloke2021-04-291-1/+4
| | | | | The user_may_create_room_alias method on spam checkers declared the room_alias parameter as a str when in reality it is passed a RoomAlias object.
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-221-2/+1
| | | This is no longer required, since we have dropped support for Python 3.5.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-148-8/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-4/+10
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Add a Synapse Module for configuring presence update routing (#9491)Andrew Morgan2021-04-061-0/+104
| | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.
* Add type hints to misc. files. (#9676)Patrick Cloke2021-03-241-7/+8
|
* Ensure we use a copy of the event content dict before modifying it in ↵Andrew Morgan2021-03-171-2/+12
| | | | | | | | | serialize_event (#9585) This bug was discovered by DINUM. We were modifying `serialized_event["content"]`, which - if you've got `USE_FROZEN_DICTS` turned on or are [using a third party rules module](https://github.com/matrix-org/synapse/blob/17cd48fe5171d50da4cb59db647b993168e7dfab/synapse/events/third_party_rules.py#L73-L76) - will raise a 500 if you try to a edit a reply to a message. `serialized_event["content"]` could be set to the edit event's content, instead of a copy of it, which is bad as we attempt to modify it. Instead, we also end up modifying the original event's content. DINUM uses a third party rules module, which meant the event's content got frozen and thus an exception was raised. To be clear, the problem is not that the event's content was frozen. In fact doing so helped us uncover the fact we weren't copying event content correctly.
* Prep work for removing `outlier` from `internal_metadata` (#9411)Richard van der Hoff2021-03-172-3/+8
| | | | | | | | | | | | * Populate `internal_metadata.outlier` based on `events` table Rather than relying on `outlier` being in the `internal_metadata` column, populate it based on the `events.outlier` column. * Move `outlier` out of InternalMetadata._dict Ultimately, this will allow us to stop writing it to the database. For now, we have to grandfather it back in so as to maintain compatibility with older versions of Synapse.
* Pass SSO IdP information to spam checker's registration function (#9626)Andrew Morgan2021-03-161-3/+26
| | | | | | | Fixes https://github.com/matrix-org/synapse/issues/9572 When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time. This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-164-5/+7
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Add check_media_file_for_spam spam checker hookErik Johnston2021-02-041-0/+47
|
* Remote dependency on distutils (#9125)Richard van der Hoff2021-01-151-1/+2
| | | | | | | | | `distutils` is pretty much deprecated these days, and replaced with `setuptools`. It's also annoying because it's you can't `pip install` it, and it's hard to figure out which debian package we should depend on to make sure it's there. Since we only use it for a tiny function anyway, let's just vendor said function into our codebase.
* Implement MSC2176: Updated redaction rules (#8984)Patrick Cloke2021-01-051-2/+14
| | | | An experimental room version ("org.matrix.msc2176") contains the new redaction rules for testing.
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-16/+39
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Use TYPE_CHECKING instead of magic MYPY variable. (#8770)Patrick Cloke2020-11-171-3/+2
|
* Block clients from sending server ACLs that lock the local server out. (#8708)Erik Johnston2020-11-031-9/+18
| | | Fixes #4042
* Use `%r` rather than `%s` for stringifying events (#8679)Richard van der Hoff2020-10-291-2/+2
| | | | otherwise non-state events get written as `<FrozenEvent ... state_key='None'>` which is indistinguishable from state events with the actual state_key `None`.
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-232-2/+2
|
* Fix modifying events in `ThirdPartyRules` modules (#8564)Richard van der Hoff2020-10-161-1/+1
| | | EventBuilder.build wants auth events these days
* Merge pull request #8535 from matrix-org/rav/third_party_events_updatesRichard van der Hoff2020-10-152-6/+19
|\ | | | | Support modifying event content from ThirdPartyRules modules
| * guard against accidental modificationRichard van der Hoff2020-10-132-3/+10
| |
| * Allow ThirdPartyRules modules to replace event contentRichard van der Hoff2020-10-131-3/+9
| | | | | | | | Support returning a new event dict from `check_event_allowed`.
* | Merge pull request #8537 from matrix-org/rav/simplify_locally_reject_inviteRichard van der Hoff2020-10-151-8/+13
|\ \ | | | | | | Simplify `_locally_reject_invite`
| * | Simplify `_locally_reject_invite`Richard van der Hoff2020-10-131-8/+13
| |/ | | | | | | | | Update `EventCreationHandler.create_event` to accept an auth_events param, and use it in `_locally_reject_invite` instead of reinventing the wheel.
* / Make sure a retention policy is a state event (#8527)Brendan Abolivier2020-10-141-0/+3
|/ | | | | * Make sure a retention policy is a state event * Changelog
* Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464)Richard van der Hoff2020-10-072-4/+4
| | | | | Lots of different module apis is not easy to maintain. Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
* Additional tests for third-party event rules (#8468)Richard van der Hoff2020-10-061-5/+7
| | | | | | | | | * Optimise and test state fetching for 3p event rules Getting all the events at once is much more efficient than getting them individually * Test that 3p event rules can modify events
* Fix third-party event modules for `check_visibility_can_be_modified` checkRichard van der Hoff2020-10-051-1/+3
| | | | | | PR #8292 tried to maintain backwards compat with modules which don't provide a `check_visibility_can_be_modified` method, but the tests weren't being run, and the check didn't work.
* Allow ThirdPartyEventRules modules to manipulate public room state (#8292)Andrew Morgan2020-10-051-6/+45
| | | | | This PR allows `ThirdPartyEventRules` modules to view, manipulate and block changes to the state of whether a room is published in the public rooms directory. While the idea of whether a room is in the public rooms list is not kept within an event in the room, `ThirdPartyEventRules` generally deal with controlling which modifications can happen to a room. Public rooms fits within that idea, even if its toggle state isn't controlled through a state event.
* Remove stream ordering from Metadata dict (#8452)Richard van der Hoff2020-10-052-2/+9
| | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database.
* Various clean ups to room stream tokens. (#8423)Erik Johnston2020-09-291-3/+3
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-046-7/+7
|
* Add StreamStore to mypy (#8232)Erik Johnston2020-09-021-2/+2
|
* Convert additional databases to async/await part 2 (#8200)Patrick Cloke2020-09-011-7/+12
|
* Allow capping a room's retention policy (#8104)Brendan Abolivier2020-08-241-56/+3
|
* Allow denying or shadow banning registrations via the spam checker (#8034)Patrick Cloke2020-08-201-2/+33
|
* Be more tolerant of membership events in unknown rooms (#8110)Richard van der Hoff2020-08-201-0/+2
| | | | It turns out that not all out-of-band membership events are labelled as such, so we need to be more accepting here.
* Add type hints to handlers.message and events.builder (#8067)Erik Johnston2020-08-121-27/+31
|
* Convert synapse.api to async/await (#8031)Patrick Cloke2020-08-061-1/+1
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-1/+1
|
* Convert a synapse.events to async/await. (#7949)Patrick Cloke2020-07-274-68/+67
|
* Convert state resolution to async/await (#7942)Patrick Cloke2020-07-241-2/+2
|
* Fix deprecation warning: import ABC from collections.abc (#7892)Karthikeyan Singaravelan2020-07-201-3/+3
|
* Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-1/+1
| | | ... instead of duplicating `config.signing_key[0]` everywhere
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-162-10/+6
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-152-6/+2
|
* Strictly enforce canonicaljson requirements in a new room version (#7381)Patrick Cloke2020-05-142-1/+41
|
* Extend spam checker to allow for multiple modules (#7435)Andrew Morgan2020-05-081-39/+39
|
* Fix bug in EventContext.deserialize. (#7393)Erik Johnston2020-05-051-2/+5
| | | | This caused `prev_state_ids` to be incorrect if the state event was not replacing an existing state entry.
* Remove special auth and redaction rules for aliases events in experimental ↵Patrick Cloke2020-03-091-7/+5
| | | | room ver. (#7037)
* Store room_versions in EventBase objects (#6875)Richard van der Hoff2020-03-052-24/+43
| | | | | | | This is a bit fiddly because it all has to be done on one fell swoop: * Wherever we create a new event, pass in the room version (and check it matches the format version) * When we prune an event, use the room version of the unpruned event to create the pruned version. * When we pass an event over the replication protocol, pass the room version over alongside it, and use it when deserialising the event again.
* Merge pull request #6872 from matrix-org/rav/dictpropertyRichard van der Hoff2020-02-181-81/+118
|\ | | | | Rewrite _EventInternalMetadata to back it with a dict
| * Rewrite _EventInternalMetadata to back it with a _dictRichard van der Hoff2020-02-141-16/+39
| | | | | | | | Mostly, this gives mypy an easier time.
| * Replace _event_dict_property with DictPropertyRichard van der Hoff2020-02-141-65/+79
| | | | | | | | | | this amounts to the same thing, but replaces `_event_dict` with `_dict`, and removes some of the function layers generated by `property`.
* | Add type hints to the spam check module (#6915)Patrick Cloke2020-02-141-18/+26
|/ | | | Add typing information to the spam checker modules.
* Filter the results of user directory searching via the spam checker (#6888)Patrick Cloke2020-02-141-0/+27
| | | Add a method to the spam checker to filter the user directory results.
* Add a `make_event_from_dict` method (#6858)Richard van der Hoff2020-02-072-9/+17
| | | | | | | ... and use it in places where it's trivial to do so. This will make it easier to pass room versions into the FrozenEvent constructors.
* remove unused room_version_to_event_format (#6857)Richard van der Hoff2020-02-071-23/+1
|
* Third party event rules Update (#6781)PeerD2020-02-061-3/+4
|
* Add typing to SyncHandler (#6821)Erik Johnston2020-02-031-7/+11
| | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Pass room_version into add_hashes_and_signaturesRichard van der Hoff2020-01-301-1/+1
|
* Pass room_version into create_local_event_from_event_dictRichard van der Hoff2020-01-301-22/+18
|
* Store the room version in EventBuilderRichard van der Hoff2020-01-301-5/+7
|
* Type defintions for use in refactoring for redaction changes (#6803)Richard van der Hoff2020-01-301-2/+3
| | | | | | | | | | * Bump signedjson to 1.1 ... so that we can use the type definitions * Fix breakage caused by upgrade to signedjson 1.1 Thanks, @illicitonion...
* Set the PL for aliases events to 0.Richard van der Hoff2020-01-291-1/+1
|
* Factor out a `copy_power_levels_contents` methodRichard van der Hoff2020-01-291-1/+36
| | | | I'm going to need another copy (hah!) of this.
* Make 'event.redacts' never raise. (#6771)Erik Johnston2020-01-231-4/+24
| | | | | | There are quite a few places that we assume that a redaction event has a corresponding `redacts` key, which is not always the case. So lets cheekily make it so that event.redacts just returns None instead.
* Add StateMap type alias (#6715)Erik Johnston2020-01-161-6/+5
|
* Change EventContext to use the Storage class (#6564)Erik Johnston2019-12-202-16/+22
|
* Merge branch 'develop' into babolivier/message_retentionBrendan Abolivier2019-11-261-30/+129
|\
| * Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
| * Add some checks that we aren't using state from rejected events (#6330)Richard van der Hoff2019-11-051-5/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if accessing state for rejected events Add some sanity checks on accessing state_group etc for rejected events. * Skip calculating push actions for rejected events It didn't actually cause any bugs, because rejected events get filtered out at various later points, but there's not point in trying to calculate the push actions for a rejected event.
| * Improve documentation for EventContext fields (#6319)Richard van der Hoff2019-11-051-26/+65
| |
* | Implement per-room message retention policiesBrendan Abolivier2019-11-041-3/+97
|/
* Factor out an _AsyncEventContextImpl (#6298)Richard van der Hoff2019-11-011-65/+42
| | | | | | The intention here is to make it clearer which fields we can expect to be populated when: notably, that the _event_type etc aren't used for the synchronous impl of EventContext.
* Expose some homeserver functionality to spam checkers (#6259)Travis Ralston2019-10-311-1/+13
| | | | | | | | | | | | | | * Offer the homeserver instance to the spam checker * Newsfile * Linting * Expose a Spam Checker API instead of passing the homeserver object * Alter changelog * s/hs/api
* Convert EventContext to attrs (#6218)Richard van der Hoff2019-10-281-61/+39
| | | | * make EventContext use an attr
* Merge pull request #5805 from matrix-org/erikj/validate_stateErik Johnston2019-08-011-2/+9
|\ | | | | Validate well known state events are state events.
| * Validate well-known state events are state events.Erik Johnston2019-07-311-2/+9
| | | | | | | | | | Lets disallow sending things like memberships, topics etc as non-state events.
* | Fix spelling.Erik Johnston2019-08-011-1/+3
|/ | | | Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Don't allow clients to send tombstones that reference the same roomErik Johnston2019-07-311-0/+7
|
* Replace returnValue with return (#5736)Amber Brown2019-07-234-30/+26
|
* Remove the ability to query relations when the original event was redacted. ↵Andrew Morgan2019-07-182-4/+23
| | | | | | | (#5629) Fixes #5594 Forbid viewing relations on an event once it has been redacted.
* Add origin_server_ts and sender fields to m.replace (#5613)Andrew Morgan2019-07-051-1/+5
| | | | | Riot team would like some extra fields as part of m.replace, so here you go. Fixes: #5598
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-207-106/+98
|
* Merge pull request #5480 from matrix-org/erikj/extremities_dummy_eventsErik Johnston2019-06-191-0/+12
|\ | | | | Add experimental option to reduce extremities.
| * Run as background process and fix commentsErik Johnston2019-06-191-1/+1
| |
| * Add experimental option to reduce extremities.Erik Johnston2019-06-181-0/+12
| | | | | | | | | | | | | | Adds new config option `cleanup_extremities_with_dummy_events` which periodically sends dummy events to rooms with more than 10 extremities. THIS IS REALLY EXPERIMENTAL.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-23 15:41:47 +0000