summary refs log tree commit diff
path: root/synapse/storage/events.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Correct logic in is_event_afterLuke Barnard2017-04-131-1/+1
|
* Simplify is_event_after logicLuke Barnard2017-04-121-10/+3
|
* travis flake8..Luke Barnard2017-04-111-0/+1
|
* Refactor event ordering check to events storeLuke Barnard2017-04-111-0/+28
|
* Add new storage functions for new replicationErik Johnston2017-03-301-0/+88
| | | | | The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.
* TypoErik Johnston2017-03-271-1/+1
|
* Short circuit if all new events have same state groupErik Johnston2017-03-271-36/+51
|
* Use iter(items|values)Erik Johnston2017-03-241-18/+14
|
* User Cursor.__iter__ instead of fetchallErik Johnston2017-03-231-1/+1
| | | | This prevents unnecessary construction of lists
* Merge pull request #2033 from matrix-org/erikj/repl_speedErik Johnston2017-03-211-17/+14
|\ | | | | Don't send the full event json over replication
| * Don't send the full event json over replicationErik Johnston2017-03-171-17/+14
| |
* | Avoid resetting state on rejected eventsRichard van der Hoff2017-03-171-4/+4
| | | | | | | | | | | | | | When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.
* | Refactoring and cleanupsRichard van der Hoff2017-03-171-60/+213
|/ | | | | | | | | | | | A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-03-141-3/+28
|\ | | | | | | erikj/public_list_speed
| * Fix current_state_events table to not lieErik Johnston2017-03-141-3/+28
| | | | | | | | | | If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.
* | Get current state by using current_state_events tableErik Johnston2017-03-101-10/+8
|/
* Remove needless checkErik Johnston2017-02-271-2/+0
|
* It should be allErik Johnston2017-02-271-1/+1
|
* Don't fetch current state in common caseErik Johnston2017-02-271-0/+17
| | | | | | Currently we fetch the list of current state events whenever we send something in a room. This is overkill for the common case of persisting a simple chain of non-state events, so lets handle that case specially.
* Ignore new rejected events when working out forward extremeties.Erik Johnston2017-02-081-9/+6
|
* Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_eventsErik Johnston2017-02-011-0/+2
|\ | | | | Add a small cache get_all_new_events
| * Add a small cache get_all_new_eventsErik Johnston2017-02-011-0/+2
| |
* | Only invalidate membership caches based on the cache streamErik Johnston2017-01-311-20/+0
|/ | | | | Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.
* Stream cache invalidations for room membership storage functionsErik Johnston2017-01-301-2/+6
|
* TypoErik Johnston2017-01-231-1/+1
|
* Refactor to calculate state delta outside transactionErik Johnston2017-01-231-87/+118
|
* CommentsErik Johnston2017-01-201-8/+19
|
* Insert delta of current_state_events to be more efficientErik Johnston2017-01-201-26/+52
|
* SpellingErik Johnston2017-01-201-1/+1
|
* Update all call sites after renameErik Johnston2017-01-201-1/+1
|
* CommentsErik Johnston2017-01-201-0/+11
|
* Calculate the forward extremeties onceErik Johnston2017-01-201-56/+86
|
* Derive current_state_events from state groupsErik Johnston2017-01-201-70/+118
|
* Lower loading events log to DEBUGErik Johnston2017-01-171-2/+2
|
* More logging for the linearizer and for get_eventsMark Haines2017-01-051-0/+2
|
* New Flake8 fixesErik Johnston2016-11-151-0/+1
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_authErik Johnston2016-10-171-15/+29
|\
| * Fix background reindex of origin_server_tsErik Johnston2016-09-271-15/+29
| | | | | | | | | | | | | | | | The storage function `_get_events_txn` was removed everywhere except from this background reindex. The function was removed due to it being (almost) completely unused while also being large and complex. Therefore, instead of resurrecting `_get_events_txn` we manually reimplement the bits that are needed directly.
* | Drop some unused indicesErik Johnston2016-10-171-1/+0
|/
* Index contains_url for file search queriesErik Johnston2016-09-121-0/+8
|
* Take value in a better wayErik Johnston2016-09-051-1/+1
|
* Correctly delete old state groups in purge history APIErik Johnston2016-09-051-20/+79
|
* Limit the length of state chainsErik Johnston2016-09-021-21/+28
|
* Generate state group ids in state layerErik Johnston2016-08-311-9/+1
|
* Make _state_groups_id_gen a normal IdGeneratorErik Johnston2016-08-301-44/+39
|
* Use state handler instead of get_users_in_room/get_joined_hostsErik Johnston2016-08-261-1/+0
|
* Preserve some logcontextsErik Johnston2016-08-241-6/+10
|
* Move defer.returnValue out of MeasureErik Johnston2016-08-191-1/+1
|
* Measure _get_event_from_rowErik Johnston2016-08-191-41/+43
|
* Add missing database corruption recovery caseBenjamin Saunders2016-08-141-1/+2
| | | | Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
* Raise 404 when couldn't find eventErik Johnston2016-08-051-1/+1
|
* Delete more tablesErik Johnston2016-08-051-0/+5
|
* Tweak integrity error recovery to work as intendedBenjamin Saunders2016-08-041-0/+3
|
* Fix typoErik Johnston2016-08-041-3/+4
|
* Retry event persistence on IntegrityErrorErik Johnston2016-08-041-4/+65
| | | | | | | | | | Due to a bug in the porting script some backfilled events were not correctly persisted, causing irrecoverable IntegrityErrors on future attempts to persist those events. This commit adds a retry mechanism invoked upon IntegrityError, where when retried the tables are purged for all references to the events being persisted.
* Remove other bit of deduplicationErik Johnston2016-08-031-19/+0
|
* Ensure we only persist an event once at a timeErik Johnston2016-08-031-1/+18
|
* Fix how push_actions are redacted.Mark Haines2016-07-261-8/+10
|
* Don't add rejected events if we've seen them befrore. Add some comments to ↵Mark Haines2016-07-261-5/+48
| | | | explain what the code is doing mechanically
* Only update the events and event_json tables for rejected eventsMark Haines2016-07-251-51/+62
|
* Don't add the events to forward extremities if the event is rejectedMark Haines2016-07-251-3/+6
|
* Don't add rejections to the state_group, persist all rejectionsMark Haines2016-07-251-4/+5
|
* Merge pull request #922 from matrix-org/erikj/file_api2Erik Johnston2016-07-201-0/+82
|\ | | | | Feature: Add filter to /messages. Add 'contains_url' to filter.
| * Add sender and contains_url field to events tableErik Johnston2016-07-141-0/+82
| |
* | Fix /purge_history bugErik Johnston2016-07-151-19/+15
| | | | | | | | This was caused by trying to insert duplicate backward extremeties
* | event_backwards_extremeties may not be emptyErik Johnston2016-07-151-1/+11
|/
* CommentErik Johnston2016-07-071-0/+3
|
* Return 400 rather than 500Erik Johnston2016-07-071-1/+4
|
* Add rest servlet. Fix SQL.Erik Johnston2016-07-061-5/+4
|
* Add purge_history APIErik Johnston2016-07-051-0/+6
|
* Add storage function to purge history for a roomErik Johnston2016-07-041-0/+140
|
* Use similar naming we use in email notifs for pushDavid Baker2016-06-241-7/+0
| | | | Fixes https://github.com/vector-im/vector-web/issues/1654
* Merge pull request #841 from matrix-org/erikj/event_counterErik Johnston2016-06-061-0/+9
|\ | | | | Add metric counter for number of persisted events
| * Add metric counter for number of persisted eventsErik Johnston2016-06-061-0/+9
| |
* | Add events to cache when we persist themErik Johnston2016-06-061-0/+41
|/
* Don't rely on options when inserting event into cacheErik Johnston2016-06-031-40/+43
|
* Change the way we cache eventsErik Johnston2016-06-031-39/+41
|
* Remove event fetching from DB threadsErik Johnston2016-06-031-138/+0
|
* Use state to calculate get_users_in_roomErik Johnston2016-06-011-3/+0
|
* Merge remote-tracking branch 'origin/develop' into dbkr/email_notifsDavid Baker2016-05-101-11/+149
|\
| * Remove unused importErik Johnston2016-05-061-1/+1
| |
| * Pull loop one level upErik Johnston2016-05-061-36/+41
| |
| * Queue events for persistenceErik Johnston2016-05-061-11/+144
| |
* | Merge remote-tracking branch 'origin/develop' into dbkr/email_notifsDavid Baker2016-04-291-2/+8
|\|
| * Check that somethign has happend before running the selectsMark Haines2016-04-271-2/+8
| |
* | First bits of emailpusherDavid Baker2016-04-191-0/+2
|/ | | | Mostly logic of when to send an email
* Add back backfilled parameter that was removedErik Johnston2016-04-121-3/+4
|
* Check if we've already backfilled eventsErik Johnston2016-04-121-0/+16
|
* Merge remote-tracking branch 'origin/develop' into ↵David Baker2016-04-071-2/+2
|\ | | | | | | dbkr/pushers_use_event_actions
| * Add a slaved events store classMark Haines2016-04-061-2/+2
| | | | | | | | | | Add a test to check that get_room_names_and_aliases does the same thing on both the master and on the slave data store.
* | pep8David Baker2016-04-061-1/+3
| |
* | Make pushers use the event_push_actions table instead of listening on an ↵David Baker2016-04-061-0/+12
|/ | | | | | | event stream & running the rules again. Sytest passes, but remaining to do: * Make badges work again * Remove old, unused code
* Store invites in a separate table.Erik Johnston2016-04-041-9/+4
|
* Merge pull request #684 from matrix-org/markjh/backfill_id_genMark Haines2016-04-011-14/+5
|\ | | | | Use a stream id generator for backfilled ids
| * Use a stream id generator for backfilled idsMark Haines2016-04-011-14/+5
| |
* | Fix the invalidation of the names and aliases cacheMark Haines2016-04-011-1/+1
|/
* Move the check for backfilled outside the for loopMark Haines2016-03-311-5/+5
|
* Remove the is_new_state argument to persist event.Mark Haines2016-03-311-40/+50
| | | | | | | | | Move the checks for whether an event is new state inside persist event itself. This was harder than expected because there wasn't enough information passed to persist event to correctly handle invites from remote servers for new rooms.
* Merge pull request #676 from matrix-org/markjh/replicate_stateIIIMark Haines2016-03-311-2/+76
|\ | | | | Add replication streams for ex outliers and current state resets
| * Use a namedtuple rather than tuple unpackingMark Haines2016-03-311-2/+9
| |
| * Add replication streams for ex outliers and current state resetsMark Haines2016-03-301-1/+59
| |
| * Add a entry to current_state_resets table when the current state is resetMark Haines2016-03-301-0/+9
| |
* | return the state_group for backfillMark Haines2016-03-301-1/+2
|/
* Add a replication stream for state groupsMark Haines2016-03-301-1/+5
|
* Use a stream id generator to assign state group idsMark Haines2016-03-301-40/+50
|
* Merge pull request #665 from matrix-org/erikj/dont_cache_eventsErik Johnston2016-03-231-2/+2
|\ | | | | Only cache events in the event cache
| * Don't cache events in get_current_state_for_keyErik Johnston2016-03-231-2/+2
| |
* | Make StateHandler._state_cache only store event_ids.Erik Johnston2016-03-221-0/+25
|/
* Remove unused backfilled parameter from persist_eventMark Haines2016-03-211-18/+4
|
* Dedupe requested event list in _get_eventsErik Johnston2016-03-181-14/+12
|
* Fix relative imports so they work in both py3 and py27Mark Haines2016-03-081-1/+1
|
* Merge pull request #489 from matrix-org/markjh/replicationMark Haines2016-03-011-0/+45
|\ | | | | Add a /replication API for extracting the updates that happened on synapse.
| * Add a /replication API for extracting the updates that happened onMark Haines2016-03-011-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.
* | Load the current id in the IdGenerator constructorMark Haines2016-03-011-3/+3
|/ | | | | | | | | Rather than loading them lazily. This allows us to remove all the yield statements and spurious arguments for the get_next methods. It also allows us to replace all instances of get_next_txn with get_next since get_next no longer needs to access the db.
* Remove unused param from get_max_tokenErik Johnston2016-02-181-1/+1
|
* Rename functionsErik Johnston2016-02-101-2/+2
|
* Atomically persit push actions when we persist the eventErik Johnston2016-02-091-10/+16
|
* Fix up logcontextsErik Johnston2016-02-081-14/+20
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-3/+3
|
* Cache tags and account dataErik Johnston2016-01-281-1/+1
|
* Invalidate caches properly. Remove unused argErik Johnston2016-01-281-3/+6
|
* PEP8Erik Johnston2016-01-271-1/+3
|
* Use the same path for incremental with gap or without gapErik Johnston2016-01-271-1/+0
|
* Add cache to room streamErik Johnston2016-01-271-0/+2
|
* Add a Homeserver.setup method.Erik Johnston2016-01-261-9/+5
| | | | | | This is for setting up dependencies that require work on startup. This is useful for the DataStore that wants to read a bunch from the database before initiliazing.
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Log when we skip daily messagesDaniel Wagner-Hall2016-01-061-1/+10
|
* Track the time spent in the database per request.Mark Haines2015-12-071-1/+1
| | | | and track the number of transactions that request started.
* Allow paginating search ordered by recentsErik Johnston2015-11-301-0/+77
|
* Implementation of state rollback in /syncRichard van der Hoff2015-11-131-2/+4
| | | | | | | Implementation of SPEC-254: roll back the state dictionary to how it looked at the start of the timeline. Merged PR https://github.com/matrix-org/synapse/pull/373
* Return world_readable and guest_can_join in /publicRoomsDaniel Wagner-Hall2015-11-101-0/+2
|
* Open up /events to anonymous users for room events onlyDaniel Wagner-Hall2015-11-051-0/+2
| | | | Squash-merge of PR #345 from daniel/anonymousevents
* Keep FTS indexes up to date. Only search through rooms currently joinedErik Johnston2015-10-121-0/+2
|
* Remove unused functions from synapse/storage/events.pyMark Haines2015-09-231-11/+0
|
* Add docstringDaniel Wagner-Hall2015-09-221-0/+6
|
* Implement configurable stats reportingDaniel Wagner-Hall2015-09-221-1/+57
| | | | | | | | | | SYN-287 This requires that HS owners either opt in or out of stats reporting. When --generate-config is passed, --report-stats must be specified If an already-generated config is used, and doesn't have the report_stats key, it is requested to be set.
* Merge pull request #281 from matrix-org/erikj/update_extremetiesErik Johnston2015-09-151-0/+2
|\ | | | | When updating a stored event from outlier to non-outlier, remember to update the extremeties
| * When updating a stored event from outlier to non-outlier, remember to update ↵Erik Johnston2015-09-151-0/+2
| | | | | | | | the extremeties
* | Fix indentMark Haines2015-08-241-1/+1
| |
* | Remove syutil dependency in favour of smaller single-purpose librariesMark Haines2015-08-241-11/+12
|/
* Fix bug where we didn't correctly serialize the redacted_because key over ↵Erik Johnston2015-08-211-0/+2
| | | | federation
* Change Cache to not use *args in its interfaceErik Johnston2015-08-071-10/+11
|
* Remove commented out codeErik Johnston2015-07-101-14/+0
|
* Add bulk insert events APIErik Johnston2015-06-251-166/+249
|
* Don't explode if we don't have the eventErik Johnston2015-06-011-1/+3
|
* Add config option to turn off freezing events. Use new encode_json api and ↵Erik Johnston2015-05-291-10/+13
| | | | ujson.loads
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2015-05-221-7/+18
|\
| * Add caches for things requested by the pushersMark Haines2015-05-211-7/+12
| |
| * Add a cache for get_current_state with state_keyMark Haines2015-05-211-0/+6
| |
* | Merge pull request #155 from matrix-org/erikj/perfErik Johnston2015-05-211-51/+337
|\ \ | | | | | | Bulk and batch retrieval of events.
| * | s/for events/for requests for events/Erik Johnston2015-05-211-1/+1
| | |
| * | s/for events/for requests for events/Erik Johnston2015-05-211-1/+1
| | |
| * | Add commentsErik Johnston2015-05-211-3/+14
| | |
| * | Split up _get_events into defer and txn versionsErik Johnston2015-05-201-24/+35
| | |
| * | Make not return a deferred _get_event_from_row_txnErik Johnston2015-05-201-20/+5
| | |
| * | PEP8Erik Johnston2015-05-201-2/+2
| | |
| * | Split out _get_event_from_row back into defer and _txn versionErik Johnston2015-05-201-11/+57
| | |
| * | Comments and shuffle of functionsErik Johnston2015-05-181-38/+42
| | |
| * | Remove rejected events if we don't want rejected eventsErik Johnston2015-05-181-0/+3
| | |
| * | PEP8Erik Johnston2015-05-151-1/+1
| | |
| * | Revert limiting of fetching, it didn't help perf.Erik Johnston2015-05-151-9/+2
| | |
| * | init jErik Johnston2015-05-151-0/+1
| | |
| * | Don't completely drain the listErik Johnston2015-05-151-2/+9
| | |
| * | Make store.get_current_state fetch events asynclyErik Johnston2015-05-151-1/+0
| | |
| * | Remove more debug loggingErik Johnston2015-05-151-2/+0
| | |
| * | Remove debug loggingErik Johnston2015-05-151-11/+3
| | |
| * | preserve log contextErik Johnston2015-05-151-1/+2
| | |
| * | Add a waitErik Johnston2015-05-151-2/+11
| | |
| * | Srsly. Don't use closures. BaaaaaadErik Johnston2015-05-151-10/+8
| | |
| * | Correctly pass through paramsErik Johnston2015-05-151-3/+3
| | |
| * | Sort out error handlingErik Johnston2015-05-151-25/+22
| | |
| * | Fix daedlockErik Johnston2015-05-151-52/+73
| | |
| * | Remove race conditionErik Johnston2015-05-141-35/+46
| | |
| * | Call from right threadErik Johnston2015-05-141-1/+3
| | |
| * | Count and loopErik Johnston2015-05-141-36/+34
| | |
| * | Don't start up more fetch_eventsErik Johnston2015-05-141-0/+1
| | |
| * | Awful idea for speeding up fetching of eventsErik Johnston2015-05-141-36/+131
| | |
| * | Turn off preemptive transactionsErik Johnston2015-05-141-17/+7
| | |
| * | PEP8Erik Johnston2015-05-141-2/+1
| | |
| * | Move from _base to eventsErik Johnston2015-05-141-0/+246
| | |
* | | Merge pull request #152 from matrix-org/notifier_performanceMark Haines2015-05-181-0/+3
|\ \ \ | | | | | | | | Notifier performance
| * | | Merge branch 'notifier_unify' into notifier_performanceMark Haines2015-05-141-0/+1
| |\| |
| * | | Don't bother checking for updates if the stream token hasn't advanced for a userMark Haines2015-05-131-0/+3
| | | |
* | | | Move get_events functions to storage.eventsErik Johnston2015-05-181-0/+132
| |/ / |/| |
* | | You need to call contextmanagerErik Johnston2015-05-121-0/+1
|/ /
* | Merge pull request #149 from matrix-org/erikj/backfillMark Haines2015-05-121-6/+9
|\ \ | | | | | | Backfill support
| * | Do state groups persistence /after/ checking if we have already persisted ↵Erik Johnston2015-05-121-6/+9
| |/ | | | | | | the event
* / SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called ↵Mark Haines2015-05-121-19/+19
|/ | | | from the main thread after the transaction completes, not from database thread before the transaction completes.
* Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into ↵Erik Johnston2015-05-051-11/+21
|\ | | | | | | erikj/SYN-371
| * Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemanyErik Johnston2015-05-051-11/+21
| |\
| | * Collect the invalidate callbacks on the transaction object rather than ↵Mark Haines2015-05-051-27/+21
| | | | | | | | | | | | passing around a separate list
| | * Invalidate the room_member cache if the current state events updatesMark Haines2015-05-051-0/+5
| | |
| | * Missing return for when the event was already persistedMark Haines2015-05-051-1/+1
| | |
| | * Invalidate the caches from the correct threadMark Haines2015-05-051-13/+26
| | |
* | | Don't insert without deduplication. In this case we never actually use this ↵Erik Johnston2015-05-051-22/+0
|/ / | | | | | | table, so simply remove the insert entirely
* / Add support for using executemanyErik Johnston2015-05-051-20/+26
|/
* Shuffle operations so that locking upsert happens last in the txn. This ↵Erik Johnston2015-04-271-41/+41
| | | | ensures the lock is held for the least amount of time possible.
* Go back to storing JSON in TEXTErik Johnston2015-04-161-8/+8
|
* Use True for True rather than 1Erik Johnston2015-04-151-5/+7
|
* Add support for postgres instead of mysql. Change sql accourdingly. blob + ↵Erik Johnston2015-04-141-2/+2
| | | | varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
* Merge branch 'develop' of github.com:matrix-org/synapse into mysqlErik Johnston2015-04-091-2/+2
|\
| * Indirect invalidations of _get_event_cache via a helper method to keep all ↵Paul "LeoNerd" Evans2015-03-251-2/+2
| | | | | | | | uses of the cache lexically within one .py file
* | Stream ordering and out of order insertions.Erik Johnston2015-04-091-2/+7
| | | | | | | | | | | | | | Handle the fact that events can be persisted out of order, and so to get the "current max" stream token becomes non trivial - as we need to make sure that *all* stream tokens less than the current max have also successfully been persisted.
* | Don't use multiple UNIQUE constraints; it will cause deadlocksErik Johnston2015-04-071-7/+7
| |
* | Don't use AUTOINCREMENT, use an in memory versionErik Johnston2015-04-071-6/+7
| |
* | Fix unicode database supportErik Johnston2015-03-251-3/+5
| |
* | Don't reinsert into event_edgesErik Johnston2015-03-241-8/+8
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into mysqlErik Johnston2015-03-201-1/+1
|\|
| * Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangementErik Johnston2015-03-201-0/+1
| |
| * Rearrange storage modulesErik Johnston2015-03-201-0/+394
|
* Rearrange storage modulesErik Johnston2015-03-201-0/+387
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-21 05:13:08 +0000