summary refs log tree commit diff
path: root/synapse/storage/persist_events.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-4/+8
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Try and drop stale extremities. (#8929)Erik Johnston2020-12-181-16/+184
| | | | If we see stale extremities while persisting events, and notice that they don't change the result of state resolution, we drop them.
* Fix message duplication if something goes wrong after persisting the event ↵Erik Johnston2020-10-131-13/+83
| | | | | (#8476) Should fix #3365.
* Remove stream ordering from Metadata dict (#8452)Richard van der Hoff2020-10-051-0/+2
| | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database.
* Various clean ups to room stream tokens. (#8423)Erik Johnston2020-09-291-3/+2
|
* Mypy fixes for `synapse.handlers.federation` (#8422)Richard van der Hoff2020-09-291-1/+1
| | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
* Add EventStreamPosition type (#8388)Erik Johnston2020-09-241-5/+9
| | | | | | | | | | | | | | The idea is to remove some of the places we pass around `int`, where it can represent one of two things: 1. the position of an event in the stream; or 2. a token that partitions the stream, used as part of the stream tokens. The valid operations are then: 1. did a position happen before or after a token; 2. get all events that happened before or after a token; and 3. get all events between two tokens. (Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
* Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)Erik Johnston2020-09-111-6/+10
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-2/+2
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-3/+3
|
* Convert storage layer to async/await. (#7963)Patrick Cloke2020-07-281-22/+18
|
* Convert state resolution to async/await (#7942)Patrick Cloke2020-07-241-1/+4
|
* Generate real events when we reject invites (#7804)Richard van der Hoff2020-07-091-6/+0
| | | | | | | | Fixes #2181. The basic premise is that, when we fail to reject an invite via the remote server, we can generate our own out-of-band leave event and persist it as an outlier, so that we have something to send to the client.
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+0
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-4/+3
|
* Add option to move event persistence off master (#7517)Erik Johnston2020-05-221-0/+6
|
* Fix bug in persist events when dealing with non member types. (#7548)Erik Johnston2020-05-211-2/+2
| | | | | `_is_server_still_joined` will throw if it is given state updates with non-user ID state keys with local user leaves. This is actually rarely a problem since local leaves almost always get persisted by themselves. (I discovered this on a branch that was otherwise broken, so I haven't seen this in the wild)
* Shuffle persist event data store functions. (#7440)Erik Johnston2020-05-131-20/+7
| | | | | | | | | | The aim here is to get to a stage where we have a `PersistEventStore` that holds all the write methods used during event persistence, so that we can take that class out of the `DataStore` mixin and instansiate it separately. This will allow us to instansiate it on processes other than master, while also ensuring it is only available on processes that are configured to write to events stream. This is a bit of an architectural change, where we end up with multiple classes per data store (rather than one per data store we have now). We end up having: 1. Storage classes that provide high level APIs that can talk to multiple data stores. 2. Data store modules that consist of classes that must point at the same database instance. 3. Classes in a data store that can be instantiated on processes depending on config.
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-4/+4
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Increase DB/CPU perf of `_is_server_still_joined` check. (#6936)Erik Johnston2020-02-191-15/+28
| | | | | | | | | | | | | | | | | | | | | | | | * Increase DB/CPU perf of `_is_server_still_joined` check. For rooms with large amount of state a single user leaving could cause us to go and load a lot of membership events and then pull out membership state in a large number of batches. * Newsfile * Update synapse/storage/persist_events.py Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Fix adding if too soon * Update docstring * Review comments * Woops typo Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Reduce amount of logging at INFO level. (#6862)Erik Johnston2020-02-061-1/+1
| | | | | | | | A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-1/+1
| | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* When server leaves room check for stale device lists. (#6801)Erik Johnston2020-01-301-4/+47
| | | | | | | | | When a server leaves a room it may stop sharing a room with remote users, and thus not get any updates to their device lists. So we need to check for this case and delete those device lists from the cache. We don't need to do this if we stop sharing a room because the remote user leaves the room, because we track that case via looking at membership changes.
* Delete current state when server leaves a room (#6792)Erik Johnston2020-01-291-3/+86
| | | | | | Otherwise its just stale data, which may get deleted later anyway so can't be relied on. It's also a bit of a shotgun if we're trying to get the current state of a room we're not in.
* Add a DeltaState to track changes to be made to current state (#6716)Erik Johnston2020-01-201-55/+68
|
* Split state groups into a separate data store (#6296)Erik Johnston2019-12-201-1/+1
|
* Merge pull request #6294 from matrix-org/erikj/add_state_storageErik Johnston2019-10-311-1/+1
|\ | | | | Add StateGroupStorage interface
| * Add StateGroupStorage interfaceErik Johnston2019-10-301-1/+1
| |
* | fix delete_existing for _persist_events (#6300)Richard van der Hoff2019-10-301-4/+1
|/ | | this is part of _retry_on_integrity_error, so should only be on _persist_events_and_state_updates
* Review commentsErik Johnston2019-10-301-3/+10
|
* Add DataStores and Storage classes.Erik Johnston2019-10-231-3/+4
|
* Move persist_events out from main data store.Erik Johnston2019-10-231-0/+644
This is in preparation for splitting out of state_groups_state from the main store into it own one, as persisting events depends on calculating state.