summary refs log tree commit diff
path: root/synapse/storage/events.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Python 3: Convert some unicode/bytes uses (#3569)Amber Brown2018-08-021-4/+10
|
* Merge pull request #3612 from matrix-org/rav/store_heirarchyRichard van der Hoff2018-07-311-1/+5
|\ | | | | Make EventStore inherit from EventFederationStore
| * reinstate explicit include of EventsWorkerStoreRichard van der Hoff2018-07-311-1/+3
| |
| * Make EventStore inherit from EventFederationStoreRichard van der Hoff2018-07-261-2/+4
| | | | | | | | | | | | | | | | (since it uses methods therein) Turns out that we had a bunch of things which were incorrectly importing EventWorkerStore from events.py rather than events_worker.py, which broke once I removed the import into events.py.
* | Merge pull request #3614 from matrix-org/rav/stop_populating_event_contentRichard van der Hoff2018-07-261-1/+0
|\ \ | | | | | | Stop populating events.content
| * | Stop populating events.contentRichard van der Hoff2018-07-261-1/+0
| |/ | | | | | | | | | | This field is no longer read from, so we should stop populating it. Once we're happy that this doesn't break everything, and a rollback is unlikely, we can think about dropping the column.
* / Remove some redundant joins on event_edges.room_idRichard van der Hoff2018-07-261-1/+0
|/ | | | | | | | | | We've long passed the point where it's possible to have the same event_id in different tables, so these join conditions are redundant: we can just join on event_id. event_edges is of non-trivial size, and the room_id column is wasteful, so let's stop reading from it. In future, we can stop writing to it, and then drop it.
* Merge pull request #3603 from matrix-org/erikj/handle_outliersErik Johnston2018-07-251-6/+8
|\ | | | | Correctly handle outliers during persist events
| * Actually fix it by adding continueErik Johnston2018-07-251-0/+1
| |
| * Fix typo in conditionalErik Johnston2018-07-251-1/+1
| |
| * Correctly handle outliers during persist eventsErik Johnston2018-07-251-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | We incorrectly asserted that all contexts must have a non None state group without consider outliers. This would usually be fine as the assertion would never be hit, as there is a shortcut during persistence if the forward extremities don't change. However, if the outlier is being persisted with non-outlier events, the function would be called and the assertion would be hit. Fixes #3601
* | Merge pull request #3606 from matrix-org/rav/logcontext_fixes_once_moreRichard van der Hoff2018-07-251-5/+4
|\ \ | | | | | | Fix another logcontext leak in _persist_events
| * | Fix another logcontext leak in _persist_eventsRichard van der Hoff2018-07-251-5/+4
| |/ | | | | | | | | | | | | | | We need to run the errback in the sentinel context to avoid losing our own context. Also: add logging to runInteraction to help identify where "Starting db connection from sentinel context" warnings are coming from
* / Fix occasional 'tuple index out of range' errorRichard van der Hoff2018-07-251-1/+1
|/ | | | | | | This fixes a bug in _delete_existing_rows_txn which was introduced in #3435 (though it's been on matrix-org-hotfixes for *years*). This code is only called when there is some sort of conflict the first time we try to persist an event, so it only happens rarely. Still, the exceptions are annoying.
* Expand on docstring comment about return valueErik Johnston2018-07-241-0/+5
|
* Remove unnecessary iteritemsErik Johnston2018-07-241-1/+1
|
* Fixup comment (and indent)Erik Johnston2018-07-241-16/+20
|
* Don't fetch state from the database unless neededErik Johnston2018-07-241-12/+18
|
* Have _get_new_state_after_events return deltaErik Johnston2018-07-241-17/+48
| | | | | | If we have a delta from the existing to new current state, then we can reuse that rather than manually working it out by fetching both lots of state.
* Don't require to_delete to have event_idsErik Johnston2018-07-241-36/+55
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-241-12/+23
|\ | | | | | | erikj/speed_up_calculate_state_delta
| * Merge branch 'develop' into rav/logcontext_fixesRichard van der Hoff2018-07-241-11/+21
| |\
| | * fix idiocyRichard van der Hoff2018-07-241-2/+8
| | |
| | * Add some measure blocks to persist_eventsRichard van der Hoff2018-07-231-11/+15
| | | | | | | | | | | | ... to help us figure out where 40% of CPU is going
| * | Logcontext fixesRichard van der Hoff2018-07-241-2/+3
| |/ | | | | | | Fix some random logcontext leaks.
* | Remove unnecessary ifErik Johnston2018-07-241-2/+1
| |
* | Speed up _calculate_state_deltaErik Johnston2018-07-241-10/+3
|/
* CommentsErik Johnston2018-07-231-0/+3
|
* Only get cached state from context in persist_eventErik Johnston2018-07-231-1/+3
| | | | | | We don't want to bother pulling out the current state from the DB since until we know we have to. Checking the context for state is just an optimisation.
* Use new gettersErik Johnston2018-07-231-1/+1
|
* Run things as background processesRichard van der Hoff2018-07-181-6/+4
| | | | | | | | This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).
* run isortAmber Brown2018-07-091-19/+15
|
* Invalidate cache on correct threadErik Johnston2018-07-021-1/+2
|
* Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-1/+2
|
* Deleting from event_push_actions needs to use an indexMark Haines2018-06-221-1/+8
|
* Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-22/+25
| | | | they're not meant to be lazy (#3307)
* cleanupAmber Brown2018-05-221-3/+6
|
* Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-221-24/+53
|\
| * Clarify commentErik Johnston2018-05-161-1/+2
| |
| * CommentsErik Johnston2018-05-161-0/+8
| |
| * Move and rename variableErik Johnston2018-05-161-3/+3
| |
| * Make purge_history operate on tokensErik Johnston2018-05-151-8/+9
| | | | | | | | As we're soon going to change how topological_ordering works
| * Use events_to_purge table rather than tokenErik Johnston2018-05-151-15/+34
| |
* | rest of the changesAmber Brown2018-05-211-17/+11
|/
* Fix up grammarErik Johnston2018-05-031-3/+3
|
* Refactor event storage to not require stateErik Johnston2018-04-271-37/+67
| | | | | | | | | This is in preparation for using contexts that may or may not have the current_state_ids set. This will allow us to avoid unnecessarily pulling out state for an event on the master process when using workers. We also add a check to see if the state groups of the old extremities are the same as the new ones.
* Refactor store.have_eventsRichard van der Hoff2018-04-201-7/+42
| | | | | | It turns out that most of the time we were calling have_events, we were only using half of the result. Replace have_events with have_seen_events and get_rejection_reasons, so that we can see what's going on a bit more clearly.
* Track where event stream processing have gotten up toErik Johnston2018-04-111-0/+3
|
* Use static JSONEncodersRichard van der Hoff2018-03-291-15/+8
| | | | | using json.dumps with custom options requires us to create a new JSONEncoder on each call. It's more efficient to create one upfront and reuse it.
* CommentErik Johnston2018-03-271-4/+5
|
* Fix indentErik Johnston2018-03-271-1/+1
|
* CommentErik Johnston2018-03-271-0/+7
|
* Add counter metrics for calculating state deltaErik Johnston2018-03-271-1/+30
| | | | | | This will allow us to measure how often we calculate state deltas in event persistence that we would have been able to calculate at the same time we calculated the state for the event.
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-191-1/+0
|\
| * Remove wrong commentErik Johnston2018-03-161-1/+0
| |
| * Replace ujson with simplejsonErik Johnston2018-03-151-2/+2
| |
* | Replace some ujson with simplejson to make it workErik Johnston2018-03-161-1/+1
| |
* | Add Measure block for persist_eventsRichard van der Hoff2018-03-131-4/+5
| | | | | | | | This seems like a useful thing to measure.
* | Fix race in sync when joining roomErik Johnston2018-03-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | The race happens when the user joins a room at the same time as doing a sync. We fetch the current token and then get the rooms the user is in. If the join happens after the current token, but before we get the rooms we end up sending down a partial room entry in the sync. This is fixed by looking at the stream ordering of the membership returned by get_rooms_for_user, and handling the case when that stream ordering is after the current token.
* | Add some caches to help read marker APIErik Johnston2018-03-011-2/+2
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-261-359/+7
|\ \ | | | | | | | | | erikj/handle_unpersisted_events_push
| * | Split EventsWorkerStore into separate fileErik Johnston2018-02-231-360/+5
| | |
| * | Update copyrightErik Johnston2018-02-231-0/+1
| | |
| * | Remove redundant clockErik Johnston2018-02-231-3/+0
| | |
| * | _event_persist_queue shouldn't be in worker storeErik Johnston2018-02-231-4/+4
| | |
| * | Split out get_events and co into a worker storeErik Johnston2018-02-211-345/+350
| | |
* | | Ensure all push actions are deleted from stagingErik Johnston2018-02-201-1/+10
| | |
* | | Refactor _set_push_actions_for_event_and_users_txn to use events_and_contextsErik Johnston2018-02-201-5/+6
|/ /
* | (Really) fix tablescan of event_push_actions on purgeRichard van der Hoff2018-02-161-1/+0
| | | | | | | | | | commit 278d21b5 added new code to avoid the tablescan, but didn't remove the old :/
* | Remove context.push_actionsErik Johnston2018-02-151-4/+3
| |
* | Update event_push_actions table from staging tableErik Johnston2018-02-151-1/+1
| |
* | purge_history: fix sqlite syntax errorRichard van der Hoff2018-02-141-1/+4
| | | | | | | | apparently sqlite insists on indexes being named
* | purge_history: handle sqlite asshatteryRichard van der Hoff2018-02-141-19/+27
| | | | | | | | | | apparently creating a temporary table commits the transaction. because that's a useful thing.
* | purge_history: fix index useRichard van der Hoff2018-02-141-0/+14
| | | | | | | | | | event_push_actions doesn't have an index on event_id, so we need to specify room_id.
* | Rework event purge to use a temporary tableRichard van der Hoff2018-02-141-35/+58
| | | | | | | | | | ... which should speed things up by reducing the amount of data being shuffled across the connection
* | Fix log message in purge_historyRichard van der Hoff2018-02-131-2/+1
| | | | | | | | (we don't just remove remote events)
* | Merge pull request #2864 from matrix-org/rav/persist_event_cachingRichard van der Hoff2018-02-131-56/+40
|\ \ | | | | | | Use StateResolutionHandler to resolve state in persist_events
| * | style nitRichard van der Hoff2018-02-131-1/+1
| | |
| * | Use StateResolutionHandler to resolve state in persist eventsRichard van der Hoff2018-02-051-48/+24
| | | | | | | | | | | | ... and thus benefit (hopefully) from its cache.
| * | Flatten _get_new_state_after_eventsRichard van der Hoff2018-02-051-44/+46
| | | | | | | | | | | | rejig the if statements to simplify the logic and reduce indentation
| * | Check that events being persisted have state_groupRichard van der Hoff2018-02-051-4/+9
| | |
| * | Add event_map param to resolve_state_groupsRichard van der Hoff2018-02-051-0/+1
| | |
* | | Merge pull request #2858 from matrix-org/rav/purge_updatesRichard van der Hoff2018-02-091-33/+61
|\ \ \ | | | | | | | | delete_local_events for purge_room_history
| * | | purge: move room_depth update to endRichard van der Hoff2018-02-091-6/+12
| | | | | | | | | | | | | | | | ... to avoid locking the table for too long
| * | | delete_local_events for purge_historyRichard van der Hoff2018-02-091-7/+28
| | | | | | | | | | | | | | | | Add a flag which makes the purger delete local events
| * | | purge: Move cache invalidation to more appropriate placeRichard van der Hoff2018-02-091-4/+4
| | | | | | | | | | | | | | | | it was a bit of a non-sequitur there
| * | | bump purge logging to infoRichard van der Hoff2018-02-091-13/+14
| | | | | | | | | | | | | | | | | | | | this thing takes ages and the only sign of any progress is the logs, so having some logs is useful.
| * | | rename delete_old_state -> purge_historyRichard van der Hoff2018-02-091-7/+7
| | | | | | | | | | | | | | | | (beacause it deletes more than state)
* | | | Store state groups separately from events (#2784)Erik Johnston2018-02-061-6/+4
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* | | Remove redundant return value from _calculate_state_deltaRichard van der Hoff2018-02-051-10/+23
| | | | | | | | | | | | | | | we already have the state from _get_new_state_after_events, so returning it from _calculate_state_delta is just confusing.
* | | factor _get_new_state_after_events out of _calculate_state_deltaRichard van der Hoff2018-01-311-18/+39
| | | | | | | | | | | | This reduces the scope of a bunch of variables
* | | Improve exception handling in persist_eventRichard van der Hoff2018-01-291-3/+3
|/ / | | | | | | | | | | | | | | | | | | | | 1. use `deferred.errback()` instead of `deferred.errback(e)`, which means that a Failure object will be constructed using the current exception state, *including* its stack trace - so the stack trace is saved in the Failure, leading to better exception reports. 2. Set `consumeErrors=True` on the ObservableDeferred, because we know that there will always be at least one observer - which avoids a spurious "CRITICAL: unhandled exception in Deferred" error in the logs
* | Merge pull request #2805 from matrix-org/rav/log_state_resRichard van der Hoff2018-01-171-0/+6
|\ \ | | | | | | Log room when doing state resolution
| * | Log room when doing state resolutionRichard van der Hoff2018-01-171-0/+6
| |/ | | | | | | Mostly because it helps figure out what is prompting the resolution
* | Split resolve_events into two functionsRichard van der Hoff2018-01-171-2/+2
| | | | | | | | ... so that the return type doesn't depend on the arg types
* | Fix a logcontext leak in persist_eventsRichard van der Hoff2018-01-161-1/+8
|/ | | | | | | | | | | | | ObserveableDeferred expects its callbacks to be called without any logcontexts, whereas it turns out we were calling them with the logcontext of the request which initiated the persistence loop. It seems wrong that we are attributing work done in the persistence loop to the request that happened to initiate it, so let's solve this by dropping the logcontext for it. (I'm not sure this actually causes any real problems other than messages in the debug log, but let's clean it up anyway)
* Make __init__ consitstent across Store heirarchyRichard van der Hoff2017-11-131-2/+2
| | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Fix logcontext handling for persist_eventsRichard van der Hoff2017-10-171-7/+17
| | | | | | | | * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments
* Invalidate cacheErik Johnston2017-10-031-0/+3
|
* Make into listErik Johnston2017-06-291-1/+1
|
* Prefill forward extrems and event to state groupsErik Johnston2017-06-291-0/+5
|
* Merge pull request #2286 from matrix-org/erikj/split_out_user_dirErik Johnston2017-06-161-0/+18
|\ | | | | Split out user directory to a separate process
| * Initial worker implErik Johnston2017-06-161-0/+18
| |
* | Merge pull request #2281 from matrix-org/erikj/phone_home_statsErik Johnston2017-06-151-58/+43
|\ \ | |/ |/| Fix phone home stats
| * Add some more statsErik Johnston2017-06-151-0/+15
| |
| * Fix phone home statsErik Johnston2017-06-141-59/+29
| |
* | Add cache for is_host_joinedErik Johnston2017-06-131-0/+5
| |
* | Merge pull request #2259 from matrix-org/erikj/fix_state_woesErik Johnston2017-06-071-1/+1
|\ \ | | | | | | Fix bug where state_group tables got corrupted
| * | Fix bug where state_group tables got corruptedErik Johnston2017-06-071-1/+1
| |/ | | | | | | | | | | | | | | This is due to the fact that we prefilled caches using txn.call_after, which always gets called including on error. We fix this by making txn.call_after only fire when a transaction completes successfully, which is what we want most of the time anyway.
* | Fix typoErik Johnston2017-05-311-1/+1
| |
* | Add stream change cacheErik Johnston2017-05-311-0/+4
| |
* | Add clobbered event_idErik Johnston2017-05-301-0/+1
| |
* | Add current_state_delta_stream tableErik Johnston2017-05-301-8/+23
| |
* | Only store event_auth for state eventsErik Johnston2017-05-241-0/+1
|/
* Merge pull request #2224 from matrix-org/erikj/prefill_stateErik Johnston2017-05-161-5/+10
|\ | | | | Prefill state caches
| * Remove spurious merge artifactsErik Johnston2017-05-161-13/+0
| |
| * CommentsErik Johnston2017-05-151-3/+3
| |
| * Add more granular event send metricsErik Johnston2017-05-151-0/+12
| |
| * Prefill state cachesErik Johnston2017-05-151-2/+8
| |
* | Don't create event_search index on sqliteRichard van der Hoff2017-05-111-0/+1
| | | | | | | | ... because the table is virtual
* | Add more logging for purgingRichard van der Hoff2017-05-111-7/+14
| | | | | | | | Log the number of events we will be deleting at info.
* | Add an index to event_searchRichard van der Hoff2017-05-111-0/+11
|/ | | | - to make the purge API quicker
* Tidy purge code and add some commentsRichard van der Hoff2017-05-111-12/+19
| | | | Try to make this clearer with more comments and some variable renames
* Don't de-delta state groups we're about to deleteRichard van der Hoff2017-05-101-11/+9
|
* add some logging to purge_historyRichard van der Hoff2017-05-101-4/+21
|
* Expand docstring a bitErik Johnston2017-05-081-1/+7
|
* Don't update event cache hit ratio from get_joined_usersErik Johnston2017-05-081-2/+11
| | | | | Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.
* Revert "Prefill state caches"Erik Johnston2017-05-041-11/+5
|
* CommentsErik Johnston2017-05-031-3/+3
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_stateErik Johnston2017-05-031-0/+18
|\
| * Add more granular event send metricsErik Johnston2017-05-021-0/+16
| |
* | Prefill state cachesErik Johnston2017-05-021-2/+6
|/
* Correct logic in is_event_afterLuke Barnard2017-04-131-1/+1
|
* Simplify is_event_after logicLuke Barnard2017-04-121-10/+3
|
* travis flake8..Luke Barnard2017-04-111-0/+1
|
* Refactor event ordering check to events storeLuke Barnard2017-04-111-0/+28
|
* Add new storage functions for new replicationErik Johnston2017-03-301-0/+88
| | | | | The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.
* TypoErik Johnston2017-03-271-1/+1
|
* Short circuit if all new events have same state groupErik Johnston2017-03-271-36/+51
|
* Use iter(items|values)Erik Johnston2017-03-241-18/+14
|
* User Cursor.__iter__ instead of fetchallErik Johnston2017-03-231-1/+1
| | | | This prevents unnecessary construction of lists
* Merge pull request #2033 from matrix-org/erikj/repl_speedErik Johnston2017-03-211-17/+14
|\ | | | | Don't send the full event json over replication
| * Don't send the full event json over replicationErik Johnston2017-03-171-17/+14
| |
* | Avoid resetting state on rejected eventsRichard van der Hoff2017-03-171-4/+4
| | | | | | | | | | | | | | When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.
* | Refactoring and cleanupsRichard van der Hoff2017-03-171-60/+213
|/ | | | | | | | | | | | A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-03-141-3/+28
|\ | | | | | | erikj/public_list_speed
| * Fix current_state_events table to not lieErik Johnston2017-03-141-3/+28
| | | | | | | | | | If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.
* | Get current state by using current_state_events tableErik Johnston2017-03-101-10/+8
|/
* Remove needless checkErik Johnston2017-02-271-2/+0
|
* It should be allErik Johnston2017-02-271-1/+1
|
* Don't fetch current state in common caseErik Johnston2017-02-271-0/+17
| | | | | | Currently we fetch the list of current state events whenever we send something in a room. This is overkill for the common case of persisting a simple chain of non-state events, so lets handle that case specially.
* Ignore new rejected events when working out forward extremeties.Erik Johnston2017-02-081-9/+6
|
* Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_eventsErik Johnston2017-02-011-0/+2
|\ | | | | Add a small cache get_all_new_events
| * Add a small cache get_all_new_eventsErik Johnston2017-02-011-0/+2
| |
* | Only invalidate membership caches based on the cache streamErik Johnston2017-01-311-20/+0
|/ | | | | Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.
* Stream cache invalidations for room membership storage functionsErik Johnston2017-01-301-2/+6
|
* TypoErik Johnston2017-01-231-1/+1
|
* Refactor to calculate state delta outside transactionErik Johnston2017-01-231-87/+118
|
* CommentsErik Johnston2017-01-201-8/+19
|
* Insert delta of current_state_events to be more efficientErik Johnston2017-01-201-26/+52
|
* SpellingErik Johnston2017-01-201-1/+1
|
* Update all call sites after renameErik Johnston2017-01-201-1/+1
|
* CommentsErik Johnston2017-01-201-0/+11
|
* Calculate the forward extremeties onceErik Johnston2017-01-201-56/+86
|
* Derive current_state_events from state groupsErik Johnston2017-01-201-70/+118
|
* Lower loading events log to DEBUGErik Johnston2017-01-171-2/+2
|
* More logging for the linearizer and for get_eventsMark Haines2017-01-051-0/+2
|
* New Flake8 fixesErik Johnston2016-11-151-0/+1
|
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_authErik Johnston2016-10-171-15/+29
|\
| * Fix background reindex of origin_server_tsErik Johnston2016-09-271-15/+29
| | | | | | | | | | | | | | | | The storage function `_get_events_txn` was removed everywhere except from this background reindex. The function was removed due to it being (almost) completely unused while also being large and complex. Therefore, instead of resurrecting `_get_events_txn` we manually reimplement the bits that are needed directly.
* | Drop some unused indicesErik Johnston2016-10-171-1/+0
|/
* Index contains_url for file search queriesErik Johnston2016-09-121-0/+8
|
* Take value in a better wayErik Johnston2016-09-051-1/+1
|
* Correctly delete old state groups in purge history APIErik Johnston2016-09-051-20/+79
|
* Limit the length of state chainsErik Johnston2016-09-021-21/+28
|
* Generate state group ids in state layerErik Johnston2016-08-311-9/+1
|
* Make _state_groups_id_gen a normal IdGeneratorErik Johnston2016-08-301-44/+39
|
* Use state handler instead of get_users_in_room/get_joined_hostsErik Johnston2016-08-261-1/+0
|
* Preserve some logcontextsErik Johnston2016-08-241-6/+10
|
* Move defer.returnValue out of MeasureErik Johnston2016-08-191-1/+1
|
* Measure _get_event_from_rowErik Johnston2016-08-191-41/+43
|
* Add missing database corruption recovery caseBenjamin Saunders2016-08-141-1/+2
| | | | Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
* Raise 404 when couldn't find eventErik Johnston2016-08-051-1/+1
|
* Delete more tablesErik Johnston2016-08-051-0/+5
|
* Tweak integrity error recovery to work as intendedBenjamin Saunders2016-08-041-0/+3
|
* Fix typoErik Johnston2016-08-041-3/+4
|
* Retry event persistence on IntegrityErrorErik Johnston2016-08-041-4/+65
| | | | | | | | | | Due to a bug in the porting script some backfilled events were not correctly persisted, causing irrecoverable IntegrityErrors on future attempts to persist those events. This commit adds a retry mechanism invoked upon IntegrityError, where when retried the tables are purged for all references to the events being persisted.
* Remove other bit of deduplicationErik Johnston2016-08-031-19/+0
|
* Ensure we only persist an event once at a timeErik Johnston2016-08-031-1/+18
|
* Fix how push_actions are redacted.Mark Haines2016-07-261-8/+10
|
* Don't add rejected events if we've seen them befrore. Add some comments to ↵Mark Haines2016-07-261-5/+48
| | | | explain what the code is doing mechanically
* Only update the events and event_json tables for rejected eventsMark Haines2016-07-251-51/+62
|
* Don't add the events to forward extremities if the event is rejectedMark Haines2016-07-251-3/+6
|
* Don't add rejections to the state_group, persist all rejectionsMark Haines2016-07-251-4/+5
|
* Merge pull request #922 from matrix-org/erikj/file_api2Erik Johnston2016-07-201-0/+82
|\ | | | | Feature: Add filter to /messages. Add 'contains_url' to filter.
| * Add sender and contains_url field to events tableErik Johnston2016-07-141-0/+82
| |
* | Fix /purge_history bugErik Johnston2016-07-151-19/+15
| | | | | | | | This was caused by trying to insert duplicate backward extremeties
* | event_backwards_extremeties may not be emptyErik Johnston2016-07-151-1/+11
|/
* CommentErik Johnston2016-07-071-0/+3
|
* Return 400 rather than 500Erik Johnston2016-07-071-1/+4
|
* Add rest servlet. Fix SQL.Erik Johnston2016-07-061-5/+4
|
* Add purge_history APIErik Johnston2016-07-051-0/+6
|
* Add storage function to purge history for a roomErik Johnston2016-07-041-0/+140
|
* Use similar naming we use in email notifs for pushDavid Baker2016-06-241-7/+0
| | | | Fixes https://github.com/vector-im/vector-web/issues/1654
* Merge pull request #841 from matrix-org/erikj/event_counterErik Johnston2016-06-061-0/+9
|\ | | | | Add metric counter for number of persisted events
| * Add metric counter for number of persisted eventsErik Johnston2016-06-061-0/+9
| |
* | Add events to cache when we persist themErik Johnston2016-06-061-0/+41
|/
* Don't rely on options when inserting event into cacheErik Johnston2016-06-031-40/+43
|
* Change the way we cache eventsErik Johnston2016-06-031-39/+41
|
* Remove event fetching from DB threadsErik Johnston2016-06-031-138/+0
|
* Use state to calculate get_users_in_roomErik Johnston2016-06-011-3/+0
|
* Merge remote-tracking branch 'origin/develop' into dbkr/email_notifsDavid Baker2016-05-101-11/+149
|\
| * Remove unused importErik Johnston2016-05-061-1/+1
| |
| * Pull loop one level upErik Johnston2016-05-061-36/+41
| |
| * Queue events for persistenceErik Johnston2016-05-061-11/+144
| |
* | Merge remote-tracking branch 'origin/develop' into dbkr/email_notifsDavid Baker2016-04-291-2/+8
|\|
| * Check that somethign has happend before running the selectsMark Haines2016-04-271-2/+8
| |
* | First bits of emailpusherDavid Baker2016-04-191-0/+2
|/ | | | Mostly logic of when to send an email
* Add back backfilled parameter that was removedErik Johnston2016-04-121-3/+4
|
* Check if we've already backfilled eventsErik Johnston2016-04-121-0/+16
|
* Merge remote-tracking branch 'origin/develop' into ↵David Baker2016-04-071-2/+2
|\ | | | | | | dbkr/pushers_use_event_actions
| * Add a slaved events store classMark Haines2016-04-061-2/+2
| | | | | | | | | | Add a test to check that get_room_names_and_aliases does the same thing on both the master and on the slave data store.
* | pep8David Baker2016-04-061-1/+3
| |
* | Make pushers use the event_push_actions table instead of listening on an ↵David Baker2016-04-061-0/+12
|/ | | | | | | event stream & running the rules again. Sytest passes, but remaining to do: * Make badges work again * Remove old, unused code
* Store invites in a separate table.Erik Johnston2016-04-041-9/+4
|
* Merge pull request #684 from matrix-org/markjh/backfill_id_genMark Haines2016-04-011-14/+5
|\ | | | | Use a stream id generator for backfilled ids
| * Use a stream id generator for backfilled idsMark Haines2016-04-011-14/+5
| |
* | Fix the invalidation of the names and aliases cacheMark Haines2016-04-011-1/+1
|/
* Move the check for backfilled outside the for loopMark Haines2016-03-311-5/+5
|
* Remove the is_new_state argument to persist event.Mark Haines2016-03-311-40/+50
| | | | | | | | | Move the checks for whether an event is new state inside persist event itself. This was harder than expected because there wasn't enough information passed to persist event to correctly handle invites from remote servers for new rooms.
* Merge pull request #676 from matrix-org/markjh/replicate_stateIIIMark Haines2016-03-311-2/+76
|\ | | | | Add replication streams for ex outliers and current state resets
| * Use a namedtuple rather than tuple unpackingMark Haines2016-03-311-2/+9
| |
| * Add replication streams for ex outliers and current state resetsMark Haines2016-03-301-1/+59
| |
| * Add a entry to current_state_resets table when the current state is resetMark Haines2016-03-301-0/+9
| |
* | return the state_group for backfillMark Haines2016-03-301-1/+2
|/
* Add a replication stream for state groupsMark Haines2016-03-301-1/+5
|
* Use a stream id generator to assign state group idsMark Haines2016-03-301-40/+50
|
* Merge pull request #665 from matrix-org/erikj/dont_cache_eventsErik Johnston2016-03-231-2/+2
|\ | | | | Only cache events in the event cache
| * Don't cache events in get_current_state_for_keyErik Johnston2016-03-231-2/+2
| |
* | Make StateHandler._state_cache only store event_ids.Erik Johnston2016-03-221-0/+25
|/
* Remove unused backfilled parameter from persist_eventMark Haines2016-03-211-18/+4
|
* Dedupe requested event list in _get_eventsErik Johnston2016-03-181-14/+12
|
* Fix relative imports so they work in both py3 and py27Mark Haines2016-03-081-1/+1
|
* Merge pull request #489 from matrix-org/markjh/replicationMark Haines2016-03-011-0/+45
|\ | | | | Add a /replication API for extracting the updates that happened on synapse.
| * Add a /replication API for extracting the updates that happened onMark Haines2016-03-011-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.
* | Load the current id in the IdGenerator constructorMark Haines2016-03-011-3/+3
|/ | | | | | | | | Rather than loading them lazily. This allows us to remove all the yield statements and spurious arguments for the get_next methods. It also allows us to replace all instances of get_next_txn with get_next since get_next no longer needs to access the db.
* Remove unused param from get_max_tokenErik Johnston2016-02-181-1/+1
|
* Rename functionsErik Johnston2016-02-101-2/+2
|
* Atomically persit push actions when we persist the eventErik Johnston2016-02-091-10/+16
|
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-25 00:13:10 +0000