summary refs log tree commit diff
path: root/synapse/events (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Do not bundle aggregations for APIs which shouldn't include them. (#11592)Patrick Cloke2021-12-201-1/+1
| | | | | And make bundling aggregations opt-in, instead of opt-out to avoid having APIs to include extraneous data (and being much heavier than necessary).
* Do not allow cross-room relations, per MSC2674. (#11516)Patrick Cloke2021-12-091-4/+7
|
* Include bundled aggregations in /sync and related fixes (#11478)Patrick Cloke2021-12-061-23/+35
| | | | | | | | Due to updates to MSC2675 this includes a few fixes: * Include bundled aggregations for /sync. * Do not include bundled aggregations for /initialSync and /events. * Do not bundle aggregations for state events. * Clarifies comments and variable names.
* Bundle relations of relations into the `/relations` result. (#11284)Patrick Cloke2021-11-301-0/+8
| | | | | Per updates to MSC2675 which now states that bundled aggregations should be included from the `/relations` endpoint.
* Fix using MSC2716 batch sending with event persistence workers (#11220)Tulir Asokan2021-11-291-0/+5
| | | | Signed-off-by: Tulir Asokan <tulir@beeper.com>
* Refactor the code to inject bundled relations during serialization. (#11408)Patrick Cloke2021-11-231-64/+82
|
* Get db signatures file to pass mypy (#11312)David Robertson2021-11-111-7/+5
|
* Add remaining type hints to `synapse.events`. (#11098)Patrick Cloke2021-11-022-81/+148
|
* Make `check_event_allowed` module API callback not fail open (accept events) ↵reivilibre2021-11-011-4/+5
| | | | when an exception is raised (#11033)
* Implement an `on_new_event` callback (#11126)Brendan Abolivier2021-10-261-0/+31
| | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Add a thread relation type per MSC3440. (#11088)Patrick Cloke2021-10-211-0/+17
| | | | Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
* Include rejected status when we log events. (#11008)Richard van der Hoff2021-10-191-6/+10
| | | | | | If we find ourselves dealing with rejected events, we proably want to know about it. Let's include it in the stringification of the event so that it gets logged.
* Move experimental & retention config out of the server module. (#11070)Patrick Cloke2021-10-151-4/+2
|
* Add type hints to synapse.events.*. (#11066)Patrick Cloke2021-10-137-139/+177
| | | Except `synapse/events/__init__.py`, which will be done in a follow-up.
* Pass through `SynapseError`s that are raised from experimental ↵reivilibre2021-10-111-0/+9
| | | | | `check_event_allowed` callback of the module API (#11042) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* Add a spamchecker method to allow or deny 3pid invites (#10894)Brendan Abolivier2021-10-061-0/+35
| | | | | This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
* Add a spamchecker callback to allow or deny room joins (#10910)Brendan Abolivier2021-10-061-0/+24
| | | Co-authored-by: Erik Johnston <erik@matrix.org>
* Strip "join_authorised_via_users_server" from join events which do not need ↵Patrick Cloke2021-09-301-1/+1
| | | | | | | it. (#10933) This fixes a "Event not signed by authorising server" error when transition room member from join -> join, e.g. when updating a display name or avatar URL for restricted rooms.
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-292-4/+4
|
* Use `RoomVersion` objects (#10934)Richard van der Hoff2021-09-291-20/+0
| | | Various refactors to use `RoomVersion` objects instead of room version identifiers.
* Add a spamchecker callback to allow or deny room creation based on invites ↵Brendan Abolivier2021-09-241-0/+42
| | | | | | | (#10898) This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
* Use direct references for configuration variables (part 5). (#10897)Patrick Cloke2021-09-242-3/+3
|
* Factor out a separate `EventContext.for_outlier` (#10883)Richard van der Hoff2021-09-221-4/+10
| | | | | | Constructing an EventContext for an outlier is actually really simple, and there's no sense in going via an `async` method in the `StateHandler`. This also means that we can resolve a bunch of FIXMEs.
* Include outlier status in `str(event)` for V2/V3 events (#10879)Richard van der Hoff2021-09-221-22/+12
| | | | | | | | | | | | | | | | | | | | | I meant to do this before, in #10591, but because I'm stupid I forgot to do it for V2 and V3 events. I've factored the common code out to `EventBase` to save us having two copies of it. This means that for `FrozenEvent` we replace `self.get("event_id", None)` with `self.event_id`, which I think is safe. `get()` is an alias for `self._dict.get()`, whereas `event_id()` is an `@property` method which looks up `self._event_id`, which is populated during construction from the same dict. We don't seem to rely on the fallback, because if the `event_id` key is absent from the dict then construction of the `EventBase` object will fail. Long story short, the only way this could change behaviour is if `event_dict["event_id"]` is changed *after* the `EventBase` object is constructed without updating the `_event_id` field, or vice versa - either of which would be very problematic anyway and the behavior of `str(event)` is the least of our worries.
* Rename MSC2716 things from `chunk` to `batch` to match `/batch_send` ↵Eric Eastwood2021-09-211-3/+3
| | | | | | | | endpoint (#10838) See https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684574497 Dropping support for older MSC2716 room versions so we don't have to worry about supporting both chunk and batch events.
* Use direct references for some configuration variables (#10798)Patrick Cloke2021-09-131-1/+1
| | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
* Support MSC3375: room version 9. (#10747)Patrick Cloke2021-09-031-0/+2
|
* Validate new m.room.power_levels events (#10232)Aaron Raimist2021-08-262-3/+79
| | | | Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Port the PresenceRouter module interface to the new generic interface (#10524)Azrenbeth2021-08-171-36/+156
| | | Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
* Clean up some logging in the federation event handler (#10591)Richard van der Hoff2021-08-161-1/+2
| | | | | | | | | | | | | | | | | | | * Include outlier status in `str(event)` In places where we log event objects, knowing whether or not you're dealing with an outlier is super useful. * Remove duplicated logging in get_missing_events When we process events received from get_missing_events, we log them twice (once in `_get_missing_events_for_pdu`, and once in `on_receive_pdu`). Reduce the duplication by removing the logging in `on_receive_pdu`, and ensuring the call sites do sensible logging. * log in `on_receive_pdu` when we already have the event * Log which prev_events we are missing * changelog
* When redacting, keep event fields around that maintain the historical event ↵Eric Eastwood2021-08-091-1/+7
| | | | | | | | | | | | | | | structure intact (MSC2716) (#10538) * Keep event fields that maintain the historical event structure intact Fix https://github.com/matrix-org/synapse/issues/10521 * Add changelog * Bump room version * Better changelog text * Fix up room version after develop merge
* Make historical events discoverable from backfill for servers without any ↵Eric Eastwood2021-07-281-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | scrollback history (MSC2716) (#10245) * Make historical messages available to federated servers Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 Follow-up to https://github.com/matrix-org/synapse/pull/9247 * Debug message not available on federation * Add base starting insertion point when no chunk ID is provided * Fix messages from multiple senders in historical chunk Follow-up to https://github.com/matrix-org/synapse/pull/9247 Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 --- Previously, Synapse would throw a 403, `Cannot force another user to join.`, because we were trying to use `?user_id` from a single virtual user which did not match with messages from other users in the chunk. * Remove debug lines * Messing with selecting insertion event extremeties * Move db schema change to new version * Add more better comments * Make a fake requester with just what we need See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080 * Store insertion events in table * Make base insertion event float off on its own See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889 Conflicts: synapse/rest/client/v1/room.py * Validate that the app service can actually control the given user See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455 Conflicts: synapse/rest/client/v1/room.py * Add some better comments on what we're trying to check for * Continue debugging * Share validation logic * Add inserted historical messages to /backfill response * Remove debug sql queries * Some marker event implemntation trials * Clean up PR * Rename insertion_event_id to just event_id * Add some better sql comments * More accurate description * Add changelog * Make it clear what MSC the change is part of * Add more detail on which insertion event came through * Address review and improve sql queries * Only use event_id as unique constraint * Fix test case where insertion event is already in the normal DAG * Remove debug changes * Switch to chunk events so we can auth via power_levels Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room. * Switch to chunk events for federation * Add unstable room version to support new historical PL * Fix federated events being rejected for no state_groups Add fix from https://github.com/matrix-org/synapse/pull/10439 until it merges. * Only connect base insertion event to prev_event_ids Per discussion with @erikjohnston, https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org * Make it possible to get the room_version with txn * Allow but ignore historical events in unsupported room version See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489 We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either. Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` -> `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`. * Move to unique index syntax See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509 * High-level document how the insertion->chunk lookup works * Remove create_event fallback for room_versions See https://github.com/matrix-org/synapse/pull/10245/files#r677641879 * Use updated method name
* Restricted rooms (MSC3083) should not have their allow key redacted. (#10489)Patrick Cloke2021-07-281-0/+2
|
* Port the ThirdPartyEventRules module interface to the new generic interface ↵Brendan Abolivier2021-07-201-42/+203
| | | | | (#10386) Port the third-party event rules interface to the generic module interface introduced in v1.37.0
* Do not include signatures/hashes in make_{join,leave,knock} responses. (#10404)Patrick Cloke2021-07-161-0/+14
| | | | These signatures would end up invalid since the joining/leaving/knocking server would modify the response before calling send_{join,leave,knock}.
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-153-23/+23
|
* Move methods involving event authentication to EventAuthHandler. (#10268)Patrick Cloke2021-07-011-5/+7
| | | Instead of mixing them with user authentication methods.
* Correct type hints for synapse.event_auth. (#10253)Patrick Cloke2021-06-302-34/+37
|
* Fix wrapping of legacy check_registration_for_spam (#10238)Brendan Abolivier2021-06-231-6/+7
| | | Fixes #10234
* Add endpoints for backfilling history (MSC2716) (#9247)Eric Eastwood2021-06-222-2/+24
| | | Work on https://github.com/matrix-org/matrix-doc/pull/2716
* Standardise the module interface (#10062)Brendan Abolivier2021-06-181-88/+218
| | | This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
* Implement knock feature (#6739)Sorunome2021-06-091-7/+12
| | | | | | This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403 Signed-off-by: Sorunome mail@sorunome.de Signed-off-by: Andrew Morgan andrewm@element.io
* Add missing type hints to handlers and fix a Spam Checker type hint. (#9896)Patrick Cloke2021-04-291-1/+4
| | | | | The user_may_create_room_alias method on spam checkers declared the room_alias parameter as a str when in reality it is passed a RoomAlias object.
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-221-2/+1
| | | This is no longer required, since we have dropped support for Python 3.5.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-148-8/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-4/+10
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Add a Synapse Module for configuring presence update routing (#9491)Andrew Morgan2021-04-061-0/+104
| | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.
* Add type hints to misc. files. (#9676)Patrick Cloke2021-03-241-7/+8
|
* Ensure we use a copy of the event content dict before modifying it in ↵Andrew Morgan2021-03-171-2/+12
| | | | | | | | | serialize_event (#9585) This bug was discovered by DINUM. We were modifying `serialized_event["content"]`, which - if you've got `USE_FROZEN_DICTS` turned on or are [using a third party rules module](https://github.com/matrix-org/synapse/blob/17cd48fe5171d50da4cb59db647b993168e7dfab/synapse/events/third_party_rules.py#L73-L76) - will raise a 500 if you try to a edit a reply to a message. `serialized_event["content"]` could be set to the edit event's content, instead of a copy of it, which is bad as we attempt to modify it. Instead, we also end up modifying the original event's content. DINUM uses a third party rules module, which meant the event's content got frozen and thus an exception was raised. To be clear, the problem is not that the event's content was frozen. In fact doing so helped us uncover the fact we weren't copying event content correctly.
* Prep work for removing `outlier` from `internal_metadata` (#9411)Richard van der Hoff2021-03-172-3/+8
| | | | | | | | | | | | * Populate `internal_metadata.outlier` based on `events` table Rather than relying on `outlier` being in the `internal_metadata` column, populate it based on the `events.outlier` column. * Move `outlier` out of InternalMetadata._dict Ultimately, this will allow us to stop writing it to the database. For now, we have to grandfather it back in so as to maintain compatibility with older versions of Synapse.
* Pass SSO IdP information to spam checker's registration function (#9626)Andrew Morgan2021-03-161-3/+26
| | | | | | | Fixes https://github.com/matrix-org/synapse/issues/9572 When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time. This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-164-5/+7
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Add check_media_file_for_spam spam checker hookErik Johnston2021-02-041-0/+47
|
* Remote dependency on distutils (#9125)Richard van der Hoff2021-01-151-1/+2
| | | | | | | | | `distutils` is pretty much deprecated these days, and replaced with `setuptools`. It's also annoying because it's you can't `pip install` it, and it's hard to figure out which debian package we should depend on to make sure it's there. Since we only use it for a tiny function anyway, let's just vendor said function into our codebase.
* Implement MSC2176: Updated redaction rules (#8984)Patrick Cloke2021-01-051-2/+14
| | | | An experimental room version ("org.matrix.msc2176") contains the new redaction rules for testing.
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-16/+39
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Use TYPE_CHECKING instead of magic MYPY variable. (#8770)Patrick Cloke2020-11-171-3/+2
|
* Block clients from sending server ACLs that lock the local server out. (#8708)Erik Johnston2020-11-031-9/+18
| | | Fixes #4042
* Use `%r` rather than `%s` for stringifying events (#8679)Richard van der Hoff2020-10-291-2/+2
| | | | otherwise non-state events get written as `<FrozenEvent ... state_key='None'>` which is indistinguishable from state events with the actual state_key `None`.
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-232-2/+2
|
* Fix modifying events in `ThirdPartyRules` modules (#8564)Richard van der Hoff2020-10-161-1/+1
| | | EventBuilder.build wants auth events these days
* Merge pull request #8535 from matrix-org/rav/third_party_events_updatesRichard van der Hoff2020-10-152-6/+19
|\ | | | | Support modifying event content from ThirdPartyRules modules
| * guard against accidental modificationRichard van der Hoff2020-10-132-3/+10
| |
| * Allow ThirdPartyRules modules to replace event contentRichard van der Hoff2020-10-131-3/+9
| | | | | | | | Support returning a new event dict from `check_event_allowed`.
* | Merge pull request #8537 from matrix-org/rav/simplify_locally_reject_inviteRichard van der Hoff2020-10-151-8/+13
|\ \ | | | | | | Simplify `_locally_reject_invite`
| * | Simplify `_locally_reject_invite`Richard van der Hoff2020-10-131-8/+13
| |/ | | | | | | | | Update `EventCreationHandler.create_event` to accept an auth_events param, and use it in `_locally_reject_invite` instead of reinventing the wheel.
* / Make sure a retention policy is a state event (#8527)Brendan Abolivier2020-10-141-0/+3
|/ | | | | * Make sure a retention policy is a state event * Changelog
* Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464)Richard van der Hoff2020-10-072-4/+4
| | | | | Lots of different module apis is not easy to maintain. Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.
* Additional tests for third-party event rules (#8468)Richard van der Hoff2020-10-061-5/+7
| | | | | | | | | * Optimise and test state fetching for 3p event rules Getting all the events at once is much more efficient than getting them individually * Test that 3p event rules can modify events
* Fix third-party event modules for `check_visibility_can_be_modified` checkRichard van der Hoff2020-10-051-1/+3
| | | | | | PR #8292 tried to maintain backwards compat with modules which don't provide a `check_visibility_can_be_modified` method, but the tests weren't being run, and the check didn't work.
* Allow ThirdPartyEventRules modules to manipulate public room state (#8292)Andrew Morgan2020-10-051-6/+45
| | | | | This PR allows `ThirdPartyEventRules` modules to view, manipulate and block changes to the state of whether a room is published in the public rooms directory. While the idea of whether a room is in the public rooms list is not kept within an event in the room, `ThirdPartyEventRules` generally deal with controlling which modifications can happen to a room. Public rooms fits within that idea, even if its toggle state isn't controlled through a state event.
* Remove stream ordering from Metadata dict (#8452)Richard van der Hoff2020-10-052-2/+9
| | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database.
* Various clean ups to room stream tokens. (#8423)Erik Johnston2020-09-291-3/+3
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-046-7/+7
|
* Add StreamStore to mypy (#8232)Erik Johnston2020-09-021-2/+2
|
* Convert additional databases to async/await part 2 (#8200)Patrick Cloke2020-09-011-7/+12
|
* Allow capping a room's retention policy (#8104)Brendan Abolivier2020-08-241-56/+3
|
* Allow denying or shadow banning registrations via the spam checker (#8034)Patrick Cloke2020-08-201-2/+33
|
* Be more tolerant of membership events in unknown rooms (#8110)Richard van der Hoff2020-08-201-0/+2
| | | | It turns out that not all out-of-band membership events are labelled as such, so we need to be more accepting here.
* Add type hints to handlers.message and events.builder (#8067)Erik Johnston2020-08-121-27/+31
|
* Convert synapse.api to async/await (#8031)Patrick Cloke2020-08-061-1/+1
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-1/+1
|
* Convert a synapse.events to async/await. (#7949)Patrick Cloke2020-07-274-68/+67
|
* Convert state resolution to async/await (#7942)Patrick Cloke2020-07-241-2/+2
|
* Fix deprecation warning: import ABC from collections.abc (#7892)Karthikeyan Singaravelan2020-07-201-3/+3
|
* Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-1/+1
| | | ... instead of duplicating `config.signing_key[0]` everywhere
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-162-10/+6
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-152-6/+2
|
* Strictly enforce canonicaljson requirements in a new room version (#7381)Patrick Cloke2020-05-142-1/+41
|
* Extend spam checker to allow for multiple modules (#7435)Andrew Morgan2020-05-081-39/+39
|
* Fix bug in EventContext.deserialize. (#7393)Erik Johnston2020-05-051-2/+5
| | | | This caused `prev_state_ids` to be incorrect if the state event was not replacing an existing state entry.
* Remove special auth and redaction rules for aliases events in experimental ↵Patrick Cloke2020-03-091-7/+5
| | | | room ver. (#7037)
* Store room_versions in EventBase objects (#6875)Richard van der Hoff2020-03-052-24/+43
| | | | | | | This is a bit fiddly because it all has to be done on one fell swoop: * Wherever we create a new event, pass in the room version (and check it matches the format version) * When we prune an event, use the room version of the unpruned event to create the pruned version. * When we pass an event over the replication protocol, pass the room version over alongside it, and use it when deserialising the event again.
* Merge pull request #6872 from matrix-org/rav/dictpropertyRichard van der Hoff2020-02-181-81/+118
|\ | | | | Rewrite _EventInternalMetadata to back it with a dict
| * Rewrite _EventInternalMetadata to back it with a _dictRichard van der Hoff2020-02-141-16/+39
| | | | | | | | Mostly, this gives mypy an easier time.
| * Replace _event_dict_property with DictPropertyRichard van der Hoff2020-02-141-65/+79
| | | | | | | | | | this amounts to the same thing, but replaces `_event_dict` with `_dict`, and removes some of the function layers generated by `property`.
* | Add type hints to the spam check module (#6915)Patrick Cloke2020-02-141-18/+26
|/ | | | Add typing information to the spam checker modules.
* Filter the results of user directory searching via the spam checker (#6888)Patrick Cloke2020-02-141-0/+27
| | | Add a method to the spam checker to filter the user directory results.
* Add a `make_event_from_dict` method (#6858)Richard van der Hoff2020-02-072-9/+17
| | | | | | | ... and use it in places where it's trivial to do so. This will make it easier to pass room versions into the FrozenEvent constructors.
* remove unused room_version_to_event_format (#6857)Richard van der Hoff2020-02-071-23/+1
|
* Third party event rules Update (#6781)PeerD2020-02-061-3/+4
|
* Add typing to SyncHandler (#6821)Erik Johnston2020-02-031-7/+11
| | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Pass room_version into add_hashes_and_signaturesRichard van der Hoff2020-01-301-1/+1
|
* Pass room_version into create_local_event_from_event_dictRichard van der Hoff2020-01-301-22/+18
|
* Store the room version in EventBuilderRichard van der Hoff2020-01-301-5/+7
|
* Type defintions for use in refactoring for redaction changes (#6803)Richard van der Hoff2020-01-301-2/+3
| | | | | | | | | | * Bump signedjson to 1.1 ... so that we can use the type definitions * Fix breakage caused by upgrade to signedjson 1.1 Thanks, @illicitonion...
* Set the PL for aliases events to 0.Richard van der Hoff2020-01-291-1/+1
|
* Factor out a `copy_power_levels_contents` methodRichard van der Hoff2020-01-291-1/+36
| | | | I'm going to need another copy (hah!) of this.
* Make 'event.redacts' never raise. (#6771)Erik Johnston2020-01-231-4/+24
| | | | | | There are quite a few places that we assume that a redaction event has a corresponding `redacts` key, which is not always the case. So lets cheekily make it so that event.redacts just returns None instead.
* Add StateMap type alias (#6715)Erik Johnston2020-01-161-6/+5
|
* Change EventContext to use the Storage class (#6564)Erik Johnston2019-12-202-16/+22
|
* Merge branch 'develop' into babolivier/message_retentionBrendan Abolivier2019-11-261-30/+129
|\
| * Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
| * Add some checks that we aren't using state from rejected events (#6330)Richard van der Hoff2019-11-051-5/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if accessing state for rejected events Add some sanity checks on accessing state_group etc for rejected events. * Skip calculating push actions for rejected events It didn't actually cause any bugs, because rejected events get filtered out at various later points, but there's not point in trying to calculate the push actions for a rejected event.
| * Improve documentation for EventContext fields (#6319)Richard van der Hoff2019-11-051-26/+65
| |
* | Implement per-room message retention policiesBrendan Abolivier2019-11-041-3/+97
|/
* Factor out an _AsyncEventContextImpl (#6298)Richard van der Hoff2019-11-011-65/+42
| | | | | | The intention here is to make it clearer which fields we can expect to be populated when: notably, that the _event_type etc aren't used for the synchronous impl of EventContext.
* Expose some homeserver functionality to spam checkers (#6259)Travis Ralston2019-10-311-1/+13
| | | | | | | | | | | | | | * Offer the homeserver instance to the spam checker * Newsfile * Linting * Expose a Spam Checker API instead of passing the homeserver object * Alter changelog * s/hs/api
* Convert EventContext to attrs (#6218)Richard van der Hoff2019-10-281-61/+39
| | | | * make EventContext use an attr
* Merge pull request #5805 from matrix-org/erikj/validate_stateErik Johnston2019-08-011-2/+9
|\ | | | | Validate well known state events are state events.
| * Validate well-known state events are state events.Erik Johnston2019-07-311-2/+9
| | | | | | | | | | Lets disallow sending things like memberships, topics etc as non-state events.
* | Fix spelling.Erik Johnston2019-08-011-1/+3
|/ | | | Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Don't allow clients to send tombstones that reference the same roomErik Johnston2019-07-311-0/+7
|
* Replace returnValue with return (#5736)Amber Brown2019-07-234-30/+26
|
* Remove the ability to query relations when the original event was redacted. ↵Andrew Morgan2019-07-182-4/+23
| | | | | | | (#5629) Fixes #5594 Forbid viewing relations on an event once it has been redacted.
* Add origin_server_ts and sender fields to m.replace (#5613)Andrew Morgan2019-07-051-1/+5
| | | | | Riot team would like some extra fields as part of m.replace, so here you go. Fixes: #5598
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-207-106/+98
|
* Merge pull request #5480 from matrix-org/erikj/extremities_dummy_eventsErik Johnston2019-06-191-0/+12
|\ | | | | Add experimental option to reduce extremities.
| * Run as background process and fix commentsErik Johnston2019-06-191-1/+1
| |
| * Add experimental option to reduce extremities.Erik Johnston2019-06-181-0/+12
| | | | | | | | | | | | | | Adds new config option `cleanup_extremities_with_dummy_events` which periodically sends dummy events to rooms with more than 10 extremities. THIS IS REALLY EXPERIMENTAL.
* | Make check_threepid_can_be_invited asyncBrendan Abolivier2019-06-171-0/+1
| |
* | Add third party rules hook for 3PID invitesBrendan Abolivier2019-06-171-1/+31
| |
* | Add third party rules hook into create roomErik Johnston2019-06-171-3/+24
|/
* Add plugin APIs for implementations of custom event rules.Brendan Abolivier2019-06-141-0/+62
|
* Fix appservice timestamp massaging (#5233)Tulir Asokan2019-05-241-1/+6
| | | Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Don't bundle events in /sync or /eventsErik Johnston2019-05-211-2/+3
| | | | | As we'll send down the annotations too anyway, so this just ends up confusing clients.
* Fix error handling for rooms whose versions are unknown. (#5219)Richard van der Hoff2019-05-212-5/+9
| | | | | | | If we remove support for a particular room version, we should behave more gracefully. This should make client requests fail with a 400 rather than a 500, and will ignore individiual PDUs in a federation transaction, rather than the whole transaction.
* Add a test room version which updates event ID format (#5210)Richard van der Hoff2019-05-201-1/+22
| | | Implements MSC1884
* Rename relation types to match MSCErik Johnston2019-05-201-3/+3
|
* Fixup bsaed on review commentsErik Johnston2019-05-171-3/+1
|
* Add basic editing supportErik Johnston2019-05-161-4/+26
|
* Add aggregations APIErik Johnston2019-05-161-3/+31
|
* Allow client event serialization to be asyncErik Johnston2019-05-141-0/+44
|
* Do checks on aliases for incoming m.room.aliases events (#5128)Brendan Abolivier2019-05-082-4/+19
| | | | | Follow-up to #5124 Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
* Remove periods from copyright headers (#5046)Andrew Morgan2019-04-111-1/+1
|
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-013-25/+29
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* Make `prev_state` field optionalErik Johnston2019-03-081-1/+0
| | | | | | | The `prev_state` field on events is not specced and so synapse shouldn't explode if an event is missing the field. Fixes #4787
* Implement soft failErik Johnston2019-03-061-0/+14
|
* Enable room version v3Erik Johnston2019-01-291-0/+2
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-01-291-2/+3
|\ | | | | | | erikj/redactions_eiah
| * Merge pull request #4514 from matrix-org/erikj/remove_event_idErik Johnston2019-01-291-2/+3
| |\ | | | | | | Remove usages of event ID's domain
| | * Don't assert an event must have an event IDErik Johnston2019-01-291-2/+3
| | |
* | | Fixup commentErik Johnston2019-01-291-2/+2
| | |
* | | kill vdh test some moreErik Johnston2019-01-291-2/+1
| | |
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-01-293-79/+320
|\| | | | | | | | | | | erikj/redactions_eiah
| * | Implement event format v2Erik Johnston2019-01-293-19/+106
| |/
| * Don't set event_id twiceErik Johnston2019-01-291-2/+0
| |
| * Refactor event building into EventBuilderErik Johnston2019-01-291-65/+219
| | | | | | | | | | This is so that everything is done in one place, making it easier to change the event format based on room version
* | Drop vdh supportErik Johnston2019-01-291-2/+0
| |
* | Add docstringErik Johnston2019-01-291-0/+12
| |
* | Implement rechecking of redactionsErik Johnston2019-01-291-0/+3
|/
* Merge pull request #4494 from matrix-org/erikj/fixup_event_validatorErik Johnston2019-01-291-23/+37
|\ | | | | Split up event validation between event and builder
| * Remove duplicate checksErik Johnston2019-01-291-14/+0
| |
| * Fix up error messagesErik Johnston2019-01-291-2/+2
| |
| * Fold validate into validate_newErik Johnston2019-01-291-16/+11
| |
| * Split up event validation between event and builderErik Johnston2019-01-281-20/+53
| | | | | | | | | | | | | | | | | | The validator was being run on the EventBuilder objects, and so the validator only checked a subset of fields. With the upcoming EventBuilder refactor even fewer fields will be there to validate. To get around this we split the validation into those that can be run against an EventBuilder and those run against a fully fledged event.
* | Refactor event signing to work on dictsErik Johnston2019-01-281-12/+35
|/ | | | | | This is in preparation for making EventBuilder format agnostic, which means event signing should be done against the event dict rather than the EventBuilder object.
* Require event format version to parse or create eventsErik Johnston2019-01-252-2/+73
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813Erik Johnston2019-01-251-2/+7
|\
| * Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invitesErik Johnston2019-01-241-2/+7
| |\ | | | | | | Store rejected remote invite events as outliers
| | * Use term 'out of band membership' insteadErik Johnston2019-01-241-4/+5
| | |
| | * Clarify the invite flowsErik Johnston2019-01-231-2/+6
| | |
* | | Review commentsErik Johnston2019-01-241-1/+2
| | |
* | | Implement MSC 1813 - Add room version to make APIsErik Johnston2019-01-231-1/+16
|/ / | | | | | | | | We also implement `make_membership_event` converting the returned room version to an event format version.
* | isortErik Johnston2019-01-231-3/+1
| |
* | Add support for persisting event format versionsErik Johnston2019-01-231-0/+5
|/ | | | | | | | | | | | | | Currently we only have the one event format version defined, but this adds the necessary infrastructure to persist and fetch the format versions alongside the events. We specify the format version rather than the room version as: 1. We don't necessarily know the room version, existing events may be either v1 or v2. 2. We'd need to be careful to prevent/handle correctly if different events in the same room reported to be of different versions, which sounds annoying.
* Add helpers for getting prev and auth events (#4139)Erik Johnston2018-11-061-0/+18
| | | | | | | * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions.
* Disable frozen dicts by default (#3987)Amber Brown2018-10-021-3/+10
|
* Fix handling of redacted events from federationErik Johnston2018-09-131-0/+5
| | | | | | | | | | If we receive an event that doesn't pass their content hash check (e.g. due to already being redacted) then we hit a bug which causes an exception to be raised, which then promplty stops the event (and request) from being processed. This effects all sorts of federation APIs, including joining rooms with a redacted state event.
* Handle delta_ids being None in _update_context_for_auth_eventsRichard van der Hoff2018-07-231-1/+2
| | | | | | | it's easier to create the new state group as a delta from the existing one. (There's an outside chance this will help with https://github.com/matrix-org/synapse/issues/3364)
* CommentsErik Johnston2018-07-231-2/+3
|
* Only get cached state from context in persist_eventErik Johnston2018-07-231-0/+13
| | | | | | We don't want to bother pulling out the current state from the DB since until we know we have to. Checking the context for state is just an optimisation.
* Fix missing attributes on workers.Erik Johnston2018-07-231-4/+6
| | | | | This was missed during the transition from attribute to getter for getting state from context.
* Fix EventContext when using workersErik Johnston2018-07-231-7/+7
| | | | | | We were: 1. Not correctly setting all attributes 2. Using defer.inlineCallbacks in a non-generator
* Update docstrings to make senseErik Johnston2018-07-231-5/+8
|
* Add support for updating stateErik Johnston2018-07-231-0/+19
|
* Make EventContext lazy load stateErik Johnston2018-07-231-41/+112
|
* pep8Erik Johnston2018-07-231-1/+1
|
* Refcator EventContext to accept state during initErik Johnston2018-07-231-21/+27
|
* Make the rest of the .iterwhatever go away (#3562)Amber Brown2018-07-211-1/+3
|
* run isortAmber Brown2018-07-095-15/+14
|
* Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-1/+1
| | | | they're not meant to be lazy (#3307)
* Replace some more comparisons with sixAdrian Tschira2018-05-192-3/+7
| | | | | | plus a bonus b"" string I missed last time Signed-off-by: Adrian Tschira <nota@notafile.com>
* add comment explaining attributeerrorAdrian Tschira2018-04-301-0/+3
|
* Make event properties raise AttributeError insteadAdrian Tschira2018-04-151-3/+12
| | | | | | | | | | | | | They raised KeyError before. I'm changing this because the code uses hasattr() to check for the presence of a key. This worked accidentally before, because hasattr() silences all exceptions in python 2. However, in python3, this isn't the case anymore. I had a look around to see if anything depended on this raising a KeyError and I couldn't find anything. Of course, I could have simply missed it. Signed-off-by: Adrian Tschira <nota@notafile.com>
* Remove context.push_actionsErik Johnston2018-02-151-4/+0
|
* Don't serialize current state over replicationErik Johnston2018-02-151-6/+35
|
* Add replication http endpoint for event sendingErik Johnston2018-02-071-0/+72
|
* Store state groups separately from events (#2784)Erik Johnston2018-02-061-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Remove pointless create() methodRichard van der Hoff2017-10-201-1/+1
| | | | | It just calls the constructor, so we may as well kill it rather than having random codepaths.
* Spam checking: add the invitee to user_may_inviteDavid Baker2017-10-051-2/+2
|
* spam check room publishingDavid Baker2017-10-041-0/+18
|
* Add room creation checks to spam checkerDavid Baker2017-10-041-0/+32
| | | | | Lets the spam checker deny attempts to create rooms and add aliases to them.
* s/roomid/room_id/David Baker2017-10-031-2/+2
|
* pass room id tooDavid Baker2017-10-031-2/+2
|
* Actually write warpper functionDavid Baker2017-10-031-0/+16
|
* Don't test is spam_checker not NoneDavid Baker2017-09-271-1/+7
| | | | | Sometimes it's a Mock object which is not none but is still not what we're after
* Remove unintentional debuggingDavid Baker2017-09-271-1/+0
|
* pep8David Baker2017-09-271-0/+1
|
* Make the spam checker a moduleDavid Baker2017-09-261-17/+20
|
* fix thinko'd docstringMatthew Hodgson2017-09-221-1/+1
|
* PoC for filtering spammy events (#2456)Richard van der Hoff2017-09-191-0/+38
| | | Demonstration of how you might add some hooks to filter out spammy events.
* Add more granular event send metricsErik Johnston2017-05-021-0/+3
|
* Comment and remove spurious loggingErik Johnston2017-04-271-0/+3
|
* Fix testsErik Johnston2017-04-261-3/+3
|
* Fix invite state to always include all eventsErik Johnston2017-04-261-1/+19
|
* Refactoring and cleanupsRichard van der Hoff2017-03-171-0/+26
| | | | | | | | | | | | A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
* Optimise state resolutionErik Johnston2017-01-172-4/+10
|
* gah, fix mangled merge of 0.18.7 into developMatthew Hodgson2017-01-071-1/+1
|
* Only send events that originate on this server.Mark Haines2017-01-051-0/+9
| | | | | | Or events that are sent via the federation "send_join" API. This should match the behaviour from before v0.18.5 and #1635 landed.
* Review commentsKegan Dougal2016-11-221-11/+9
|
* Also check for dict since sometimes they aren't frozenKegan Dougal2016-11-221-1/+1
|
* Glue only_event_fields into the sync rest servletKegan Dougal2016-11-221-1/+1
|
* More testsKegan Dougal2016-11-221-3/+4
|
* Start adding some testsKegan Dougal2016-11-211-1/+3
|
* Move event_fields filtering to serialize_eventKegan Dougal2016-11-211-4/+97
| | | | Also make it an inclusive not exclusive filter, as the spec demands.
* Add commentsErik Johnston2016-09-051-0/+5
|
* Move to storing state_groups_state as deltasErik Johnston2016-09-011-0/+16
|
* Correctly handle the difference between prev and current stateErik Johnston2016-08-311-2/+3
|
* Make None optionalErik Johnston2016-08-261-1/+1
|
* Replace context.current_state with context.current_state_idsErik Johnston2016-08-251-11/+2
|
* Pull out event ids rather than full events for stateErik Johnston2016-08-251-0/+8
|
* Include prev_content in redacted state eventsErik Johnston2016-08-111-0/+2
|
* Remove the is_new_state argument to persist event.Mark Haines2016-03-311-0/+3
| | | | | | | | | Move the checks for whether an event is new state inside persist event itself. This was harder than expected because there wasn't enough information passed to persist event to correctly handle invites from remote servers for new rooms.
* Use a stream id generator to assign state group idsMark Haines2016-03-301-1/+1
|
* Intern all the thingsErik Johnston2016-03-231-9/+2
|
* Intern sender, event_id and room_id in eventsErik Johnston2016-03-231-0/+5
|
* Intern type and state_key on eventsErik Johnston2016-03-231-0/+7
|
* Don't assume existence of event_id in __str__Erik Johnston2016-03-181-1/+3
|
* Atomically persit push actions when we persist the eventErik Johnston2016-02-091-0/+1
|
* Make Event objects behave more like dictsErik Johnston2016-01-181-0/+9
|
* copyrightsMatthew Hodgson2016-01-075-5/+5
|
* Only add the user_id if the sender is presentMark Haines2015-12-011-1/+3
|
* Copy rather than move the fields to shuffle between a v1 and a v2 event.Mark Haines2015-11-301-9/+5
| | | | | | | | | | This should make all v1 APIs compatible with v2 clients. While still allowing v1 clients to access the fields. This makes the documentation easier since we can just document the v2 format and explain that some of the fields, in some of the APIs are duplicated for backwards compatibility, rather than having to document two separate event formats.
* Flatten the /sync response to remove the event_mapRichard van der Hoff2015-11-191-2/+1
|
* Update the other place check_joined_room is calledMark Haines2015-10-301-1/+2
|
* Surely we don't need to preserve 'events_default' twicePaul "LeoNerd" Evans2015-10-161-1/+0
|
* Bundle in some room state in the unsigned bit of the invite when sending to ↵Erik Johnston2015-09-101-1/+4
| | | | invited servers