summary refs log tree commit diff
path: root/synapse/storage/stats.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Renamve get_room_stateErik Johnston2019-09-021-2/+2
|
* Merge branch 'rei/rss_target' of github.com:matrix-org/synapse into rei/rss_inc8Erik Johnston2019-09-021-1/+1
|\
| * Merge pull request #5941 from matrix-org/rei/rss_inc7reivilibre2019-08-301-3/+504
| |\ | | | | | | Separated Statistics [7/7ish]
| | * Fix to use milliseconds github/rei/rss_inc7 rei/rss_inc7Olivier Wilkinson (reivilibre)2019-08-301-1/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | LintingOlivier Wilkinson (reivilibre)2019-08-301-1/+0
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Add stats testsOlivier Wilkinson (reivilibre)2019-08-301-0/+74
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Various fixesOlivier Wilkinson (reivilibre)2019-08-301-2/+3
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Fixes to counting and stats deltasOlivier Wilkinson (reivilibre)2019-08-301-3/+3
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Whoopsies; these things come in order…Olivier Wilkinson (reivilibre)2019-08-301-1/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-1/+1
|\ \ \ | | |/ | |/|
| * | Adapt to use renamed `room_state`Olivier Wilkinson (reivilibre)2019-08-301-1/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-1/+1
|\| |
| * | Ambiguous `room_id`Olivier Wilkinson (reivilibre)2019-08-301-1/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-0/+1
|\| |
| * | Whoops, took out a line there...Olivier Wilkinson (reivilibre)2019-08-301-0/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-5/+6
|\| |
| * | `users` table's ID field is actually called `name`.Olivier Wilkinson (reivilibre)2019-08-301-5/+6
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-1/+1
|\| |
| * | Fix that became apparent after unit testingOlivier Wilkinson (reivilibre)2019-08-301-1/+1
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-16/+25
|\| |
| * | Code formatting and typo pointed out by Erik.Olivier Wilkinson (reivilibre)2019-08-301-10/+15
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * | Some fixes that have become necessary due to changes in other PRsOlivier Wilkinson (reivilibre)2019-08-301-8/+12
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | | Merge branch 'rei/rss_inc7' into rei/rss_inc8Olivier Wilkinson (reivilibre)2019-08-301-3/+493
|\| | | |/ |/|
| * Code formattingOlivier Wilkinson (reivilibre)2019-08-301-18/+20
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Add stats regeneratorOlivier Wilkinson (reivilibre)2019-08-301-3/+491
| |
* | Convert `chain` to `list` as `chain` is only once iterable. github/rei/rss_inc6 rei/rss_inc6Olivier Wilkinson (reivilibre)2019-08-301-2/+2
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Merge branch 'rei/rss_target' into rei/rss_inc6Olivier Wilkinson (reivilibre)2019-08-301-0/+112
|\|
| * Count total_events and total_event_bytes within the loop.Olivier Wilkinson (reivilibre)2019-08-291-4/+9
| | | | | | | | | | | | | | In this case, we still update these counts if we get stuck in the loop because the server is busy. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Add incremental counting for rooms' total events and total event bytes.Olivier Wilkinson (reivilibre)2019-08-281-0/+106
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Add `total_event_bytes` to room statistics schema.Olivier Wilkinson (reivilibre)2019-08-281-0/+1
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Code formatting (Black)Olivier Wilkinson (reivilibre)2019-08-291-12/+30
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Upsert fixesOlivier Wilkinson (reivilibre)2019-08-281-8/+15
| |
* | Hoist up None check to prevent trying to iterate over NoneType.keys()Olivier Wilkinson (reivilibre)2019-08-281-3/+4
|/
* Merge branch 'rei/rss_target' into rei/rss_inc3Olivier Wilkinson (reivilibre)2019-08-281-87/+94
|\
| * Code formatting (Black) github/rei/rss_inc2 rei/rss_inc2Olivier Wilkinson (reivilibre)2019-08-281-1/+3
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Update synapse/storage/stats.pyreivilibre2019-08-281-1/+1
| | | | | | Co-Authored-By: Erik Johnston <erik@matrix.org>
| * Clean up code with improved naming and hoist around functions.Olivier Wilkinson (reivilibre)2019-08-271-79/+91
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Fix logic error.Olivier Wilkinson (reivilibre)2019-08-271-1/+2
| | | | | | | | | | | | | | `absolute_fields` being None shouldn't preclude completion of a current stats row. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Rename `room_state` table to `room_stats_state`Olivier Wilkinson (reivilibre)2019-08-271-1/+1
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Remove obsolete `OldCollectionRequired` as old collection is obsolete.Olivier Wilkinson (reivilibre)2019-08-271-6/+0
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Use `DeferredLock` instead of `threading.Lock`Olivier Wilkinson (reivilibre)2019-08-271-2/+3
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Fix logic error.Olivier Wilkinson (reivilibre)2019-08-271-1/+2
| | | | | | | | | | | | | | `absolute_fields` being None shouldn't preclude completion of a current stats row. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Invalidate `get_earliest_token_for_stats` cache as required.Olivier Wilkinson (reivilibre)2019-08-271-0/+1
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | For user stats, handle other membership transitions properly.Olivier Wilkinson (reivilibre)2019-08-271-4/+1
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Merge branch 'rei/rss_inc2' into rei/rss_inc3Olivier Wilkinson (reivilibre)2019-08-271-99/+214
|\|
| * Switch to milliseconds in room/user stats for consistency.Olivier Wilkinson (reivilibre)2019-08-271-1/+2
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Code formatting (Black)Olivier Wilkinson (reivilibre)2019-08-271-5/+5
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Lock tables in upsert fall-backs.Olivier Wilkinson (reivilibre)2019-08-271-0/+2
| | | | | | | | | | | | | | Should not be too much of a performance concern as this code won't be hit on Postgres, which large deployments should be using. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Apply minor suggestions from reviewOlivier Wilkinson (reivilibre)2019-08-271-4/+3
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Clarify `_update_stats_delta_txn` by adding code comments and kwargs.Olivier Wilkinson (reivilibre)2019-08-271-16/+18
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Apply suggestions from code reviewreivilibre2019-08-271-2/+2
| | | | | | Co-Authored-By: Erik Johnston <erik@matrix.org>
| * Fix up historical stats support.Olivier Wilkinson (reivilibre)2019-08-221-11/+32
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Simplify table structureOlivier Wilkinson (reivilibre)2019-08-221-109/+171
| | | | | | | | | | | | | | | | This obviates the need for old collection, but comes at the minor cost of not being able to track historical stats or per-slice fields until after the statistics regenerator is finished. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Clarify `_update_stats_delta_txn`Olivier Wilkinson (reivilibre)2019-08-201-2/+15
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Sanitise accepted fields in `_update_stats_delta_txn`Olivier Wilkinson (reivilibre)2019-08-201-0/+12
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * Update synapse/storage/stats.pyreivilibre2019-08-201-1/+1
| | | | | | Co-Authored-By: Erik Johnston <erik@matrix.org>
| * Update synapse/storage/stats.pyreivilibre2019-08-201-1/+1
| | | | | | Co-Authored-By: Erik Johnston <erik@matrix.org>
| * Ack, isort!Olivier Wilkinson (reivilibre)2019-08-201-1/+2
| | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Handle state deltas and turn them into stats deltasOlivier Wilkinson (reivilibre)2019-08-201-1/+111
|/ | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Add storage function for storing stats deltasOlivier Wilkinson (reivilibre)2019-08-201-2/+169
| | | | | | Old collection is not included in this commit Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Tear out current room & user statistics (#5880)reivilibre2019-08-201-409/+3
| | | | | | | | | | * Tear out current room & user statistics. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Black is back with more linting complaints Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Stop trying to fetch events with event_id=None. (#5753)Richard van der Hoff2019-07-241-9/+11
| | | | | | | `None` is not a valid event id, so queuing up a database fetch for it seems like a silly thing to do. I considered making `get_event` return `None` if `event_id is None`, but then its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-8/+8
|
* Run Black. (#5482)Amber Brown2019-06-201-10/+24
|
* FixErik Johnston2019-06-031-1/+2
|
* Ignore room state with null bytes in for room statsErik Johnston2019-06-031-0/+15
|
* Add indices. Remove room_ids accidentally addedErik Johnston2019-05-311-13/+28
| | | | | | | | | | We have to do this by re-inserting a background update and recreating tables, as the tables only get created during a background update and will later be deleted. We also make sure that we remove any entries that should have been removed but weren't due to a race that has been fixed in a previous commit.
* Move deletion from table inside txnErik Johnston2019-05-301-2/+5
|
* Fetch membership counts all at onceErik Johnston2019-05-301-16/+7
|
* Get events all at onceErik Johnston2019-05-301-33/+26
|
* Fix logging for room stats background updateTravis Ralston2019-05-251-1/+1
|
* Room Statistics (#4338)Amber Brown2019-05-211-0/+450