summary refs log tree commit diff
path: root/synapse/storage/client_ips.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move storage classes into a main "data store".Erik Johnston2019-10-211-581/+0
| | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* Cleanup client_ipsBrendan Abolivier2019-10-081-7/+0
|
* Move client_ips's bg updates to a dedicated storeBrendan Abolivier2019-10-071-94/+106
|
* Fix `devices_last_seen` background update.Erik Johnston2019-09-301-7/+39
| | | | Fixes #6134.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-09-251-1/+1
|\ | | | | | | erikj/cleanup_user_ips_2
| * Use if `is not None`Erik Johnston2019-09-251-1/+1
| | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Review commentsErik Johnston2019-09-251-1/+1
| |
* | Prune rows in user_ips older than configured periodErik Johnston2019-09-241-8/+54
|/ | | | Defaults to pruning everything older than 28d.
* Query devices table for last seen info.Erik Johnston2019-09-231-49/+8
| | | | | | | | This is a) simpler than querying user_ips directly and b) means we can purge older entries from user_ips without losing the required info. The storage functions now no longer return the access_token, since it was unused.
* Add BG update to populate devices last seen infoErik Johnston2019-09-231-0/+52
|
* Add last seen info to devices table.Erik Johnston2019-09-231-0/+15
| | | | | This allows us to purge old user_ips entries without having to preserve the latest last seen info for active devices.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-14/+12
|
* Run black on the rest of the storage module (#4996)Amber Brown2019-04-031-47/+38
|
* Merge pull request #4627 from matrix-org/erikj/user_ips_analyzeErik Johnston2019-02-121-0/+24
|\ | | | | Analyze user_ips before running deduplication
| * Fix pep8Erik Johnston2019-02-121-1/+1
| |
| * Analyze user_ips before running deduplicationErik Johnston2019-02-121-0/+24
| | | | | | | | | | | | | | Due to the table locks taken out by the naive upsert, the table statistics may be out of date. During deduplication it is important that the correct index is used as otherwise a full table scan may be incorrectly used, which can end up thrashing the database badly.
* | Reduce user_ips bloat during dedupe background updateErik Johnston2019-02-121-3/+60
|/ | | | | | | | | | | | The background update to remove duplicate rows naively deleted and reinserted the duplicates. For large tables with a large number of duplicates this causes a lot of bloat (with postgres), as the inserted rows are appended to the table, since deleted rows will not be overwritten until a VACUUM has happened. This should hopefully also help ensure that the query in the last batch uses the correct index, as inserting a large number of new rows without analyzing will upset the query planner.
* Use native UPSERTs where possible (#4306)Amber Brown2019-01-241-1/+4
|
* Fixup removal of duplicate `user_ips` rows (#4432)Erik Johnston2019-01-231-1/+5
| | | | | | | | * Remove unnecessary ORDER BY clause * Add logging * Newsfile
* Refactor to rewrite the SQL insteadErik Johnston2019-01-221-13/+16
|
* Don't shadow paramsErik Johnston2019-01-221-4/+3
|
* Fix bug when removing duplicate rows from user_ipsErik Johnston2019-01-221-13/+20
| | | | | | | | This was caused by accidentally overwritting a `last_seen` variable in a for loop, causing the wrong value to be written to the progress table. The result of which was that we didn't scan sections of the table when searching for duplicates, and so some duplicates did not get deleted.
* Remove duplicates in the user_ips table and add an index (#4370)Amber Brown2019-01-121-4/+134
|
* Fix client IPs being broken on Python 3 (#3908)Amber Brown2018-09-201-15/+19
|
* Run tests under PostgreSQL (#3423)Amber Brown2018-08-131-0/+5
|
* wip commit - tests failingNeil Johnson2018-08-031-20/+1
|
* do mau checks based on monthly_active_users tableNeil Johnson2018-08-021-7/+8
|
* insertion into monthly_active_usersNeil Johnson2018-08-021-1/+21
|
* Fix some looping_call calls which were broken in #3604Richard van der Hoff2018-07-261-1/+1
| | | | | | | | | It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.
* Run things as background processesRichard van der Hoff2018-07-181-4/+11
| | | | | | | | This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).
* run isortAmber Brown2018-07-091-5/+4
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-2/+4
|
* replace some iteritems with sixAdrian Tschira2018-05-191-2/+4
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Generate user daily statsNeil Johnson2018-04-251-0/+7
|
* Add user_ips last seen indexNeil Johnson2018-03-281-0/+7
|
* Make __init__ consitstent across Store heirarchyRichard van der Hoff2017-11-131-2/+2
| | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)
* Merge pull request #2309 from matrix-org/erikj/user_ip_replErik Johnston2017-07-061-3/+5
|\ | | | | Fix up user_ip replication commands
| * Fix upErik Johnston2017-06-271-3/+5
| |
* | Define CACHE_SIZE_FACTOR onceErik Johnston2017-07-041-4/+2
|/
* Fix up client ips to read from pending dataErik Johnston2017-06-271-11/+59
|
* Batch upsert user ipsErik Johnston2017-06-271-19/+38
|
* Increase size of IP cacheErik Johnston2017-06-071-1/+6
|
* Increase client_ip cache sizeErik Johnston2017-05-081-0/+1
|
* Bail on where clause insteadErik Johnston2017-04-111-3/+4
|
* Don't bother with outer check for nowErik Johnston2017-04-111-3/+0
|
* Fix getting latest device IP for user with no devicesErik Johnston2017-04-111-0/+5
|
* PEP8Richard van der Hoff2016-07-221-1/+0
|
* Create index on user_ips in the backgroundRichard van der Hoff2016-07-221-4/+12
| | | | | | | | user_ips is kinda big, so really we want to add the index in the background once we're running. Replace the schema delta with one which will do that. I've done this in a way that's reasonably easy to reuse as there a few other indexes I need, and I don't suppose they will be the last.
* storage/client_ips: remove some dead codeRichard van der Hoff2016-07-211-3/+0
|
* Merge branch 'develop' into rav/get_devices_apiRichard van der Hoff2016-07-201-1/+2
|\ | | | | | | (pick up PR #938 in the hope of fixing the UTs)
| * Record device_id in client_ipsRichard van der Hoff2016-07-201-1/+2
| | | | | | | | | | Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.
* | GET /devices endpointRichard van der Hoff2016-07-201-0/+72
|/ | | | | | | implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.
* Move insert_client_ip to a separate classMark Haines2016-06-031-0/+68
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-11 06:13:51 +0000