summary refs log tree commit diff
path: root/synapse/storage/event_federation.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move storage classes into a main "data store".Erik Johnston2019-10-211-672/+0
| | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* Replace IN usage with helper funcsErik Johnston2019-10-101-3/+6
|
* Fix dummy event insertion consent bug (#6053)Neil Johnson2019-09-261-3/+15
| | | Fixes #5905
* Replace returnValue with return (#5736)Amber Brown2019-07-231-6/+6
|
* Run Black. (#5482)Amber Brown2019-06-201-2/+1
|
* Add experimental option to reduce extremities.Erik Johnston2019-06-181-0/+29
| | | | | | | Adds new config option `cleanup_extremities_with_dummy_events` which periodically sends dummy events to rooms with more than 10 extremities. THIS IS REALLY EXPERIMENTAL.
* Expose DataStore._get_events as get_events_as_listErik Johnston2019-05-141-3/+3
| | | | This is in preparation for reaction work which requires it.
* Run black on the rest of the storage module (#4996)Amber Brown2019-04-031-89/+60
|
* s/get_forward_events/get_successor_events/Erik Johnston2019-03-041-2/+2
|
* Stop backpaginating when events not visibleErik Johnston2019-02-201-0/+22
|
* Refactor event building into EventBuilderErik Johnston2019-01-291-0/+23
| | | | | This is so that everything is done in one place, making it easier to change the event format based on room version
* Add helpers for getting prev and auth events (#4139)Erik Johnston2018-11-061-2/+2
| | | | | | | * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions.
* Fix incorrect truncation in get_missing_eventsRichard van der Hoff2018-10-161-22/+16
| | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id.
* Merge branch 'master' into developRichard van der Hoff2018-08-021-0/+1
|\
| * Avoid extra db lookupsRichard van der Hoff2018-08-021-29/+1
| | | | | | | | | | Since we're about to look up the events themselves anyway, we can skip the extra db queries here.
| * Validation for events/rooms in fed requestsRichard van der Hoff2018-08-021-0/+29
| | | | | | | | | | | | | | When we get a federation request which refers to an event id, make sure that said event is in the room the caller claims it is in. (patch supplied by @turt2live)
* | Merge pull request #3612 from matrix-org/rav/store_heirarchyRichard van der Hoff2018-07-311-1/+1
|\ \ | | | | | | Make EventStore inherit from EventFederationStore
| * | Make EventStore inherit from EventFederationStoreRichard van der Hoff2018-07-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | (since it uses methods therein) Turns out that we had a bunch of things which were incorrectly importing EventWorkerStore from events.py rather than events_worker.py, which broke once I removed the import into events.py.
* | | Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_idRichard van der Hoff2018-07-261-7/+6
|\ \ \ | | | | | | | | Remove some redundant joins on event_edges.room_id
| * | | Remove some redundant joins on event_edges.room_idRichard van der Hoff2018-07-261-7/+6
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | We've long passed the point where it's possible to have the same event_id in different tables, so these join conditions are redundant: we can just join on event_id. event_edges is of non-trivial size, and the room_id column is wasteful, so let's stop reading from it. In future, we can stop writing to it, and then drop it.
* / / Fix some looping_call calls which were broken in #3604Richard van der Hoff2018-07-261-1/+1
|/ / | | | | | | | | | | | | | | | | It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.
* / Wrap a number of things that run in the backgroundRichard van der Hoff2018-07-251-3/+6
|/ | | | | This will reduce the number of "Starting db connection from sentinel context" warnings, and will help with our metrics.
* run isortAmber Brown2018-07-091-9/+7
|
* Merge pull request #3110 from NotAFile/py3-six-queueRichard van der Hoff2018-04-201-2/+4
|\ | | | | Replace Queue with six.moves.queue
| * Replace Queue with six.moves.queueAdrian Tschira2018-04-161-2/+4
| | | | | | | | | | | | and a six.range change which I missed the last time Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Avoid creating events with huge numbers of prev_eventsRichard van der Hoff2018-04-161-16/+41
|/ | | | | | In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events.
* Split out SignatureStore and EventFederationStoreErik Johnston2018-03-011-129/+135
|
* Make __init__ consitstent across Store heirarchyRichard van der Hoff2017-11-131-2/+2
| | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)
* Add background task to clear out old event_authErik Johnston2017-05-241-0/+56
|
* Only store event_auth for state eventsErik Johnston2017-05-241-6/+29
|
* Use txn.fetchall() so we can reuse txnErik Johnston2017-03-301-1/+1
|
* User Cursor.__iter__ instead of fetchallErik Johnston2017-03-231-7/+6
| | | | This prevents unnecessary construction of lists
* Refactoring and cleanupsRichard van der Hoff2017-03-171-12/+12
| | | | | | | | | | | | A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
* Fix bugs in the /keys/changes apiRichard van der Hoff2017-02-141-1/+16
| | | | | | | | | | | | | | | * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.
* Up get_latest_event_ids_in_room cacheErik Johnston2017-02-011-1/+1
|
* Calculate the forward extremeties onceErik Johnston2017-01-201-70/+6
|
* Remove spurious AS clauseErik Johnston2016-09-211-1/+1
|
* Fix _delete_old_forward_extrem_cache queryErik Johnston2016-09-211-5/+4
|
* CommentErik Johnston2016-09-161-0/+2
|
* Filter remote rooms lists locallyErik Johnston2016-09-161-1/+1
|
* Handling expiring stream extrems correctly.Erik Johnston2016-09-151-3/+20
|
* Change get_pos_of_last_change to return upper boundErik Johnston2016-09-151-3/+2
|
* Use stream_change cache to make get_forward_extremeties_for_room cache more ↵Erik Johnston2016-09-151-1/+10
| | | | effective
* Add cache to get_forward_extremeties_for_roomErik Johnston2016-09-151-0/+1
|
* Add a room visibility streamErik Johnston2016-09-141-1/+1
|
* Add a DB index to figure out past state at a stream ordering in a roomErik Johnston2016-09-141-0/+81
|
* Use computed prev event idsErik Johnston2016-04-011-0/+16
|
* Respond to federated invite with non-empty contextDaniel Wagner-Hall2016-02-171-4/+4
| | | | | | | Currently, we magically perform an extra database hit to find the inviter, and use this to guess where we should send the event. Instead, fill in a valid context, so that other callers relying on the context actually have one.
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-1/+1
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Remove unused functions from synapse/storage/event_federation.pyMark Haines2015-09-231-71/+0
|
* synapse/storage/event_federation.py:_get_auth_events is unusedMark Haines2015-09-221-21/+0
|
* Merge pull request #281 from matrix-org/erikj/update_extremetiesErik Johnston2015-09-151-0/+9
|\ | | | | When updating a stored event from outlier to non-outlier, remember to update the extremeties
| * "Comments"Erik Johnston2015-09-151-0/+6
| |
| * When updating a stored event from outlier to non-outlier, remember to update ↵Erik Johnston2015-09-151-0/+3
| | | | | | | | the extremeties
* | Merge branch 'release-v0.10.0' into developMark Haines2015-08-281-2/+8
|\| | | | | | | | | | | | | Conflicts: synapse/handlers/auth.py synapse/python_dependencies.py synapse/rest/client/v1/login.py
| * Don't insert events into 'event_*_extremeties' tables if they're outliersErik Johnston2015-08-231-2/+8
| |
* | Remove syutil dependency in favour of smaller single-purpose librariesMark Haines2015-08-241-1/+1
|/
* Move all the caches into their own package, synapse.util.cachesErik Johnston2015-08-111-1/+2
|
* Change Cache to not use *args in its interfaceErik Johnston2015-08-071-2/+2
|
* Merge pull request #193 from matrix-org/erikj/bulk_persist_eventErik Johnston2015-07-141-51/+57
|\ | | | | Add bulk insert events API
| * Add bulk insert events APIErik Johnston2015-06-251-51/+57
| |
* | Don't explode if we don't recognize one of the event_ids in the backfill requestErik Johnston2015-07-061-2/+4
|/
* Batch SELECTs in _get_auth_chain_ids_txnErik Johnston2015-06-231-3/+11
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2015-05-221-0/+1
|\ | | | | | | erikj/backfill_fixes
| * Merge pull request #156 from matrix-org/erikj/join_perfMark Haines2015-05-221-2/+3
| |\ | | | | | | Make joining #matrix:matrix.org over federation quicker
| * | Invalidate the get_latest_event_ids_in_room cache when deleting from ↵Mark Haines2015-05-221-0/+1
| | | | | | | | | | | | event_forward_extremities
* | | Actually return something from lambdaErik Johnston2015-05-211-1/+1
| | |
* | | Sort backfill eventsErik Johnston2015-05-211-1/+5
| | |
* | | Remove dead codeErik Johnston2015-05-211-5/+0
| | |
* | | Only get non-stateErik Johnston2015-05-211-1/+2
| | |
* | | Skip events we've already seenErik Johnston2015-05-211-0/+3
| | |
* | | Correctly prepopulate queueErik Johnston2015-05-211-2/+10
| | |
* | | Don't prepopulate event_resultsErik Johnston2015-05-211-1/+1
| | |
* | | Don't readd things that are already in event_resultsErik Johnston2015-05-211-1/+2
| | |
* | | PriorityQueue gives lowest firstErik Johnston2015-05-211-2/+2
| | |
* | | Correctly capture Queue.Empty exceptionErik Johnston2015-05-211-2/+5
| | |
* | | Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into ↵Erik Johnston2015-05-211-21/+17
|\ \ \ | | |/ | |/| | | | erikj/backfill_fixes
| * | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perfErik Johnston2015-05-211-21/+17
| |\|
| | * Move fetching of events into their own transactionsErik Johnston2015-05-141-21/+17
| | |
* | | Fix _get_backfill_events to return events in the correct orderErik Johnston2015-05-201-27/+28
| | |
* | | Ensure event_results is a setErik Johnston2015-05-191-1/+1
|/ /
* / Fix event_backwards_extrem insertion to ignore outliersErik Johnston2015-05-191-2/+3
|/
* Err, delete the right stuffErik Johnston2015-05-121-4/+5
|
* NewlinesErik Johnston2015-05-121-2/+0
|
* We do actually want to delete rows out of event_backward_extremitiesErik Johnston2015-05-121-4/+11
|
* Fix up _handle_prev_events to not try to insert duplicate rowsErik Johnston2015-05-121-23/+13
|
* Initial hack at wiring together pagination and backfillErik Johnston2015-05-111-2/+26
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemanyErik Johnston2015-05-051-1/+3
|\
| * Collect the invalidate callbacks on the transaction object rather than ↵Mark Haines2015-05-051-5/+5
| | | | | | | | passing around a separate list
| * Invalidate the caches from the correct threadMark Haines2015-05-051-4/+6
| |
* | Fix indentationErik Johnston2015-05-051-10/+10
| |
* | Add support for using executemanyErik Johnston2015-05-051-19/+21
|/
* Correctly name transactionErik Johnston2015-05-051-1/+1
|
* Cache latest_event_ids_in_roomErik Johnston2015-05-011-1/+4
|
* Split a storage function in two so that we don't have to do extra work.Erik Johnston2015-05-011-0/+11
|
* Add support for postgres instead of mysql. Change sql accourdingly. blob + ↵Erik Johnston2015-04-141-5/+5
| | | | varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
* Merge branch 'develop' of github.com:matrix-org/synapse into mysqlErik Johnston2015-03-201-0/+12
|\
| * Clean out event_forward_extremities table when the server rejoins the roomErik Johnston2015-03-181-0/+12
| |
* | Convert storage layer to be mysql compatibleErik Johnston2015-03-191-14/+11
|/
* Revert get_auth_chain changesErik Johnston2015-02-231-7/+5
|
* Initial stab at implementing a batched get_missing_pdus requestErik Johnston2015-02-191-5/+58
|
* Get an auth query one at a timeErik Johnston2015-02-121-7/+6
|
* Update copyright noticesMark Haines2015-01-061-1/+1
|
* Use _get_events_txn instead of _parse_events_txnErik Johnston2014-12-161-7/+1
|
* Merge branch 'hotfixes-v0.5.4a' of github.com:matrix-org/synapse into ↵Erik Johnston2014-12-161-8/+8
|\ | | | | | | release-v0.6.0
| * Fix bug where we did not send the full auth chain to people that joined over ↵Erik Johnston2014-12-161-8/+8
| | | | | | | | federation
* | Fix bug where we ignored event_edge_hashes tableErik Johnston2014-12-151-5/+6
|/
* Update some of the docs in event_federationErik Johnston2014-11-121-11/+20
|
* PEP8Erik Johnston2014-11-101-6/+3
|
* Fix backfill to work. Add auth to backfill requestErik Johnston2014-11-101-2/+2
|
* Add '/event_auth/' federation apiErik Johnston2014-11-071-7/+19
|
* Fix bug in _get_auth_chain_txnErik Johnston2014-11-071-14/+11
|
* Implement method to get auth_chain from a given event_idErik Johnston2014-11-071-0/+35
|
* Start implementing auth chainsErik Johnston2014-11-071-0/+21
|
* Implement new replace_state and changed prev_stateErik Johnston2014-11-061-7/+57
| | | | | `prev_state` is now a list of previous state ids, similiar to prev_events. `replace_state` now points to what we think was replaced.
* Get correct prev_eventsErik Johnston2014-11-051-9/+21
|
* Make prev_event signing work again.Erik Johnston2014-10-311-1/+1
|
* Actually don't store any PDUsErik Johnston2014-10-311-0/+7
|
* Move the impl of backfill to use events.Erik Johnston2014-10-311-1/+85
|
* Fix SQL so that accepts we may want to persist events twice.Erik Johnston2014-10-301-3/+5
|
* Start filling out and using new events tablesErik Johnston2014-10-291-16/+33
|
* Don't reference PDU when persisting eventErik Johnston2014-10-281-2/+2
|
* Begin implementing all the PDU storage stuff in Events landErik Johnston2014-10-281-0/+143
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-06 11:16:36 +0000