summary refs log tree commit diff
path: root/synapse/storage/events_worker.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move storage classes into a main "data store".Erik Johnston2019-10-211-882/+0
| | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* Replace IN usage with helper funcsErik Johnston2019-10-101-12/+16
|
* Fix fetching censored redactions from DBErik Johnston2019-10-021-0/+14
| | | | | | Fetching a censored redactions caused an exception due to the code expecting redactions to have a `redact` key, which redacted redactions don't have.
* Handle loops in redaction eventsRichard van der Hoff2019-07-301-60/+36
|
* recursively fetch redactionsRichard van der Hoff2019-07-301-32/+36
|
* split _get_events_from_db out of _enqueue_eventsRichard van der Hoff2019-07-291-32/+51
|
* Return dicts from _fetch_event_listRichard van der Hoff2019-07-291-15/+27
|
* Merge branch 'master' into developRichard van der Hoff2019-07-261-0/+4
|\
| * Fix DoS when there is a cycle in redaction eventsRichard van der Hoff2019-07-261-0/+4
| | | | | | | | | | Make sure that synapse doesn't explode when a redaction redacts itself, or there is a larger cycle.
* | Merge pull request #5767 from matrix-org/rav/redactions/cross_room_idRichard van der Hoff2019-07-251-0/+27
|\| | | | | log when a redaction attempts to redact an event in a different room
| * log when a redaction attempts to redact an event in a different roomRichard van der Hoff2019-07-251-0/+27
| |
* | Stop trying to fetch events with event_id=None. (#5753)Richard van der Hoff2019-07-241-1/+4
| | | | | | | | | | | | | | `None` is not a valid event id, so queuing up a database fetch for it seems like a silly thing to do. I considered making `get_event` return `None` if `event_id is None`, but then its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
* | Replace returnValue with return (#5736)Amber Brown2019-07-231-10/+10
|/
* Ignore redactions of m.room.create events (#5701)Richard van der Hoff2019-07-171-0/+12
|
* More refactoring in `get_events_as_list` (#5707)Richard van der Hoff2019-07-171-27/+37
| | | | | | | | We can now use `_get_events_from_cache_or_db` rather than going right back to the database, which means that (a) we can benefit from caching, and (b) it opens the way forward to more extensive checks on the original event. We now always require the original event to exist before we will serve up a redaction.
* Fix redaction authentication (#5700)Richard van der Hoff2019-07-171-85/+129
| | | | | | | | | | | | | | | | | Ensures that redactions are correctly authenticated for recent room versions. There are a few things going on here: * `_fetch_event_rows` is updated to return a dict rather than a list of rows. * Rather than returning multiple copies of an event which was redacted multiple times, it returns the redactions as a list within the dict. * It also returns the actual rejection reason, rather than merely the fact that it was rejected, so that we don't have to query the table again in `_get_event_from_row`. * The redaction handling is factored out of `_get_event_from_row`, and now checks if any of the redactions are valid.
* Refactor `get_events_as_list` (#5699)Richard van der Hoff2019-07-171-48/+74
| | | | | | | | | | | A couple of changes here: * get rid of a redundant `allow_rejected` condition - we should already have filtered out any rejected events before we get to that point in the code, and the redundancy is confusing. Instead, let's stick in an assertion just to make double-sure we aren't leaking rejected events by mistake. * factor out a `_get_events_from_cache_or_db` method, which is going to be important for a forthcoming fix to redactions.
* small typo fix (#5655)Andrew Morgan2019-07-111-1/+1
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-3/+3
|
* Run Black. (#5482)Amber Brown2019-06-201-5/+5
|
* Fix background updates to handle redactions/rejections (#5352)Erik Johnston2019-06-061-0/+37
| | | | | | | | * Fix background updates to handle redactions/rejections In background updates based on current state delta stream we need to handle that we may not have all the events (or at least that `get_events` may raise an exception).
* Join against events to use its room_id indexErik Johnston2019-05-301-1/+6
|
* Implement the SHHS complexity API (#5216)Amber Brown2019-05-301-1/+49
|
* Simplifications and comments in do_auth (#5227)Richard van der Hoff2019-05-231-1/+1
| | | | | I was staring at this function trying to figure out wtf it was actually doing. This is (hopefully) a non-functional refactor which makes it a bit clearer.
* Room Statistics (#4338)Amber Brown2019-05-211-0/+24
|
* Update docstring with correct typeErik Johnston2019-05-151-1/+1
| | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Expose DataStore._get_events as get_events_as_listErik Johnston2019-05-141-15/+35
| | | | This is in preparation for reaction work which requires it.
* Run black on the rest of the storage module (#4996)Amber Brown2019-04-031-72/+66
|
* Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-011-1/+2
| | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
* Fix infinite loop when an event is redacted in a v3 room (#4535)Richard van der Hoff2019-01-311-5/+32
|
* Fix flake8 (#4519)Amber Brown2019-01-301-1/+1
|
* Update synapse/storage/events_worker.pyErik Johnston2019-01-291-1/+1
|
* Check redaction state when event is pulled out of the databaseErik Johnston2019-01-291-0/+13
|
* Fix typoErik Johnston2019-01-291-1/+1
|
* Remove unused argErik Johnston2019-01-291-2/+1
|
* Update synapse/storage/events_worker.pyRichard van der Hoff2019-01-291-1/+1
| | | Co-Authored-By: erikjohnston <erikj@jki.re>
* Implement rechecking of redactionsErik Johnston2019-01-291-1/+25
|
* Replace missed usages of FrozenEventErik Johnston2019-01-251-6/+2
|
* Revert "Require event format version to parse or create events"Erik Johnston2019-01-251-2/+6
|
* Replace missed usages of FrozenEventErik Johnston2019-01-241-6/+2
|
* isortErik Johnston2019-01-231-1/+1
|
* Add support for persisting event format versionsErik Johnston2019-01-231-4/+15
| | | | | | | | | | | | | | Currently we only have the one event format version defined, but this adds the necessary infrastructure to persist and fetch the format versions alongside the events. We specify the format version rather than the room version as: 1. We don't necessarily know the room version, existing events may be either v1 or v2. 2. We'd need to be careful to prevent/handle correctly if different events in the same room reported to be of different versions, which sounds annoying.
* Port storage/ to Python 3 (#3725)Amber Brown2018-08-311-4/+5
|
* Pull in necessary stores in federation_readerErik Johnston2018-08-061-1/+0
|
* Move necessary storage functions to worker classesErik Johnston2018-08-061-0/+84
|
* Merge branch 'master' into developRichard van der Hoff2018-08-021-6/+14
|\
| * Avoid extra db lookupsRichard van der Hoff2018-08-021-6/+14
| | | | | | | | | | Since we're about to look up the events themselves anyway, we can skip the extra db queries here.
* | Run things as background processesRichard van der Hoff2018-07-181-4/+6
|/ | | | | | | | This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).
* Attempt to include db threads in cpu usage stats (#3496)Richard van der Hoff2018-07-101-1/+2
| | | | | Let's try to include time spent in the DB threads in the per-request/block cpu usage metrics.
* Add CPU metrics for _fetch_event_listRichard van der Hoff2018-07-091-19/+32
| | | | | add a Measure block on _fetch_event_list, in the hope that we can better measure CPU usage here.
* run isortAmber Brown2018-07-091-13/+12
|
* Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-1/+2
|
* Merge pull request #3419 from matrix-org/rav/events_per_requestRichard van der Hoff2018-06-221-0/+4
|\ | | | | Log number of events fetched from DB
| * Indirect evt_count updates via method callRichard van der Hoff2018-06-221-1/+1
| | | | | | | | so that we can stub it for the sentinel and not have a billion failing UTs
| * Log number of events fetched from DBRichard van der Hoff2018-06-211-0/+4
| | | | | | | | | | | | | | | | | | | | When we finish processing a request, log the number of events we fetched from the database to handle it. [I'm trying to figure out which requests are responsible for large amounts of event cache churn. It may turn out to be more helpful to add counts to the prometheus per-request/block metrics, but that is an extension to this code anyway.]
* | Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-3/+3
|/
* Misc. py3 fixesAdrian Tschira2018-05-241-1/+1
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-2/+3
| | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* Format docstringErik Johnston2018-04-121-2/+3
|
* Doc we raise on unknown eventErik Johnston2018-04-121-2/+3
|
* Track last processed event received_tsErik Johnston2018-04-111-0/+18
|
* Replace some ujson with simplejson to make it workErik Johnston2018-03-161-1/+1
|
* Split EventsWorkerStore into separate fileErik Johnston2018-02-231-0/+395