Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Clear event caches when we purge history (#15609) | Erik Johnston | 2023-06-08 | 1 | -0/+31 |
| | | | | | | | This should help a little with #13476 --------- Co-authored-by: Patrick Cloke <patrickc@matrix.org> | ||||
* | Add final type hint to synapse.server. (#15035) | Patrick Cloke | 2023-02-09 | 1 | -0/+2 |
| | |||||
* | Update all stream IDs after processing replication rows (#14723) | Nick Mills-Barrett | 2023-01-04 | 1 | -1/+16 |
| | | | | | | | | | | | | | | This creates a new store method, `process_replication_position` that is called after `process_replication_rows`. By moving stream ID advances here this guarantees any relevant cache invalidations will have been applied before the stream is advanced. This avoids race conditions where Python switches between threads mid way through processing the `process_replication_rows` method where stream IDs may be advanced before caches are invalidated due to class resolution ordering. See this comment/issue for further discussion: https://github.com/matrix-org/synapse/issues/14158#issuecomment-1344048703 | ||||
* | Bump flake8-bugbear from 22.9.23 to 22.10.27 (#14329) | dependabot[bot] | 2022-11-04 | 1 | -1/+1 |
| | | | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> | ||||
* | Optimise get_rooms_for_user (drop with_stream_ordering) (#13787) | Nick Mills-Barrett | 2022-09-29 | 1 | -0/+1 |
| | |||||
* | Simplify cache invalidation after event persist txn (#13796) | Nick Mills-Barrett | 2022-09-26 | 1 | -0/+3 |
| | | | | | This moves all the invalidations into a single place and de-duplicates the code involved in invalidating caches for a given event by using the base class method. | ||||
* | Add cache invalidation across workers to module API (#13667) | Mathieu Velten | 2022-09-21 | 1 | -5/+18 |
| | | | Signed-off-by: Mathieu Velten <mathieuv@matrix.org> | ||||
* | Refactor presence so we can prune user in room caches (#13313) | Erik Johnston | 2022-07-25 | 1 | -0/+4 |
| | | | | | | | | See #10826 and #10786 for context as to why we had to disable pruning on those caches. Now that `get_users_who_share_room_with_user` is called frequently only for presence, we just need to make calls to it less frequent and then we can remove the various levels of caching that is going on. | ||||
* | Safe async event cache (#13308) | Nick Mills-Barrett | 2022-07-19 | 1 | -1/+8 |
| | | | | | | | | Fix race conditions in the async cache invalidation logic, by separating the async & local invalidation calls and ensuring any async call i executed first. Signed off by Nick @ Beeper (@Fizzadar). | ||||
* | Revert "Make all `process_replication_rows` methods async (#13304)" (#13312) | Erik Johnston | 2022-07-18 | 1 | -1/+1 |
| | | | This reverts commit 5d4028f217f178fcd384d5bfddd92225b4e78c51. | ||||
* | Make all `process_replication_rows` methods async (#13304) | Nick Mills-Barrett | 2022-07-17 | 1 | -1/+1 |
| | | | | | More prep work for asyncronous caching, also makes all process_replication_rows methods consistent (presence handler already is so). Signed off by Nick @ Beeper (@Fizzadar) | ||||
* | Don't pull out the full state when calculating push actions (#13078) | Erik Johnston | 2022-07-11 | 1 | -0/+9 |
| | |||||
* | Reduce state pulled from DB due to sending typing and receipts over ↵ | Erik Johnston | 2022-06-06 | 1 | -0/+1 |
| | | | | | federation (#12964) Reducing the amount of state we pull from the DB is useful as fetching state is expensive in terms of DB, CPU and memory. | ||||
* | Wait for lazy join to complete when getting current state (#12872) | Erik Johnston | 2022-06-01 | 1 | -1/+1 |
| | |||||
* | Invalidate the get_users_in_room{_with_profile} caches only when necessary. ↵ | Patrick Cloke | 2022-02-02 | 1 | -3/+8 |
| | | | | | | | (#11878) The get_users_in_room and get_users_in_room_with_profiles are now only invalidated when the membership of a room changes, instead of during any state change in the room. | ||||
* | Type hint the constructors of the data store classes (#11555) | Sean Quah | 2021-12-13 | 1 | -5/+8 |
| | |||||
* | Add type hints to `synapse/storage/databases/main/events_worker.py` (#11411) | Sean Quah | 2021-11-26 | 1 | -2/+2 |
| | | | | Also refactor the stream ID trackers/generators a bit and try to document them better. | ||||
* | Remove unnecessary SystemRandom from SQLBaseStore (#9987) | Dan Callahan | 2021-05-14 | 1 | -2/+0 |
| | | | | | | | | It's not obvious that instances of SQLBaseStore each need their own instances of random.SystemRandom(); let's just use random directly. Introduced by 52839886d664576831462e033b88e5aba4c019e3 Signed-off-by: Dan Callahan <danc@element.io> | ||||
* | Use get_current_users_in_room from store and not StateHandler (#9910) | Erik Johnston | 2021-05-05 | 1 | -0/+1 |
| | |||||
* | Remove various bits of compatibility code for Python <3.6 (#9879) | Andrew Morgan | 2021-04-27 | 1 | -1/+1 |
| | | | I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+. | ||||
* | Remove `synapse.types.Collection` (#9856) | Richard van der Hoff | 2021-04-22 | 1 | -2/+2 |
| | | | This is no longer required, since we have dropped support for Python 3.5. | ||||
* | Remove redundant "coding: utf-8" lines (#9786) | Jonathan de Jong | 2021-04-14 | 1 | -1/+0 |
| | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>` | ||||
* | Import HomeServer from the proper module. (#9665) | Patrick Cloke | 2021-03-23 | 1 | -1/+1 |
| | |||||
* | Add additional type hints to the storage module. (#8980) | Patrick Cloke | 2020-12-30 | 1 | -11/+25 |
| | |||||
* | Replace DeferredCache with LruCache where possible (#8563) | Richard van der Hoff | 2020-10-19 | 1 | -5/+7 |
| | | | Most of these uses don't need a full-blown DeferredCache; LruCache is lighter and more appropriate. | ||||
* | Be stricter about JSON that is accepted by Synapse (#8106) | Patrick Cloke | 2020-08-19 | 1 | -4/+3 |
| | |||||
* | Convert the roommember database to async/await. (#8070) | Patrick Cloke | 2020-08-12 | 1 | -1/+0 |
| | |||||
* | Rename database classes to make some sense (#8033) | Erik Johnston | 2020-08-05 | 1 | -3/+3 |
| | |||||
* | Consistently use `db_to_json` to convert from database values to JSON ↵ | Patrick Cloke | 2020-07-16 | 1 | -2/+2 |
| | | | | objects. (#7849) | ||||
* | remove `builtins.buffer` code from storage code | Richard van der Hoff | 2020-05-15 | 1 | -8/+0 |
| | | | | this is no longer needed on python 3 | ||||
* | Support any process writing to cache invalidation stream. (#7436) | Erik Johnston | 2020-05-07 | 1 | -0/+3 |
| | |||||
* | Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) | Patrick Cloke | 2020-02-21 | 1 | -1/+1 |
| | | | | Ensure good comprehension hygiene using flake8-comprehensions. | ||||
* | Allow streaming cache invalidate all to workers. (#6749) | Erik Johnston | 2020-01-22 | 1 | -5/+13 |
| | |||||
* | Add a background update to clear tombstoned rooms from the directory (#6648) | Richard van der Hoff | 2020-01-07 | 1 | -1/+3 |
| | | | | | | * Add a background update to clear tombstoned rooms from the directory * use the ABC metaclass | ||||
* | Add database config class (#6513) | Erik Johnston | 2019-12-18 | 1 | -1/+1 |
| | | | | | This encapsulates config for a given database and is the way to get new connections. | ||||
* | Pass Database into the data store | Erik Johnston | 2019-12-06 | 1 | -1/+1 |
| | |||||
* | Change DataStores to accept 'database' param. | Erik Johnston | 2019-12-06 | 1 | -1/+1 |
| | |||||
* | Comments | Erik Johnston | 2019-12-05 | 1 | -1/+7 |
| | |||||
* | Move DB pool and helper functions into dedicated Database class | Erik Johnston | 2019-12-05 | 1 | -1462/+6 |
| | |||||
* | Remove underscore from SQLBaseStore functions | Erik Johnston | 2019-12-04 | 1 | -73/+69 |
| | |||||
* | Don't call SQLBaseStore methods from outside stores | Erik Johnston | 2019-12-04 | 1 | -8/+0 |
| | |||||
* | Revert "Move get_user_count_txn out of base store" | Erik Johnston | 2019-12-04 | 1 | -0/+12 |
| | | | | | | This reverts commit 00f0d67566cdfe8eae44aeae1c982c42a255cfcd. Its going to get removed soon, so lets not make merge conflicts. | ||||
* | Move get_user_count_txn out of base store | Erik Johnston | 2019-12-04 | 1 | -12/+0 |
| | |||||
* | Move cache invalidation to main data store | Erik Johnston | 2019-12-04 | 1 | -104/+0 |
| | |||||
* | Move event fetch vars to EventWorkStore | Erik Johnston | 2019-12-04 | 1 | -12/+0 |
| | |||||
* | Move account validity bg updates to registration store | Erik Johnston | 2019-12-04 | 1 | -66/+0 |
| | |||||
* | Make sure that we close cursors before returning from a query (#6408) | Richard van der Hoff | 2019-11-25 | 1 | -9/+42 |
| | | | | | | There are lots of words in the comment as to why this is a good idea. Fixes #6403. | ||||
* | Clean up newline quote marks around the codebase (#6362) | Andrew Morgan | 2019-11-21 | 1 | -1/+1 |
| | |||||
* | Fix guest -> real account upgrade with account validity enabled (#6359) | Andrew Morgan | 2019-11-14 | 1 | -6/+3 |
| | |||||
* | Remove usage of deprecated logger.warn method from codebase (#6271) | Andrew Morgan | 2019-10-31 | 1 | -3/+3 |
| | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated. | ||||
* | Fix SQLite take 2 | Erik Johnston | 2019-10-10 | 1 | -1/+1 |
| | |||||
* | Remove add_in_list_sql_clause | Erik Johnston | 2019-10-10 | 1 | -29/+6 |
| | |||||
* | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any | Erik Johnston | 2019-10-10 | 1 | -3/+4 |
|\ | |||||
| * | Patch inlinecallbacks for log contexts | Erik Johnston | 2019-09-27 | 1 | -3/+4 |
| | | |||||
* | | Add helper funcs to use postgres ANY | Erik Johnston | 2019-10-10 | 1 | -8/+56 |
|/ | | | | | This means that we can write queries with `col = ANY(?)`, which helps postgres. | ||||
* | Fix a cache-invalidation bug for worker-based deployments (#5920) | Richard van der Hoff | 2019-08-28 | 1 | -8/+16 |
| | | | | | | | | Some of the caches on worker processes were not being correctly invalidated when a room's state was changed in a way that did not affect the membership list of the room. We need to make sure we send out cache invalidations even when no memberships are changing. | ||||
* | Replace returnValue with return (#5736) | Amber Brown | 2019-07-23 | 1 | -7/+7 |
| | |||||
* | LoggingTransaction accepts None for callback lists. | Erik Johnston | 2019-07-19 | 1 | -2/+16 |
| | | | | | Its a bit disingenuousto give LoggingTransaction lists to append callbacks to if we're not going to run the callbacks. | ||||
* | Move logging utilities out of the side drawer of util/ and into logging/ (#5606) | Amber Brown | 2019-07-04 | 1 | -1/+1 |
| | |||||
* | Merge pull request #5499 from matrix-org/rav/cleanup_metrics | Richard van der Hoff | 2019-06-24 | 1 | -22/+13 |
|\ | | | | | Cleanups and sanity-checking in cpu and db metrics | ||||
| * | black | Richard van der Hoff | 2019-06-24 | 1 | -6/+2 |
| | | |||||
| * | Merge branch 'develop' into rav/cleanup_metrics | Richard van der Hoff | 2019-06-24 | 1 | -4/+2 |
| |\ | |||||
| * | | Simplify PerformanceCounters.update interface | Richard van der Hoff | 2019-06-19 | 1 | -11/+12 |
| | | | | | | | | | | | | | | | we already have the duration for the update, so may as well use it rather than passing extra params around and recalculating it. | ||||
| * | | Remove unused _get_event_counters | Richard van der Hoff | 2019-06-19 | 1 | -7/+1 |
| | | | | | | | | | | | | This has been redundant since cdb3757942fefdcdc3d33b9c6d7c9e44decefd6f. | ||||
* | | | Merge pull request #5498 from matrix-org/rav/fix_clock_reversal | Richard van der Hoff | 2019-06-24 | 1 | -6/+14 |
|\ \ \ | |_|/ |/| | | Use monotonic clock where possible for metrics | ||||
| * | | Use monotonic clock where possible for metrics | Richard van der Hoff | 2019-06-19 | 1 | -6/+14 |
| | | | | | | | | | | | | | | | | | | | | | | | | Fixes intermittent errors observed on Apple hardware which were caused by time.clock() appearing to go backwards when called from different threads. Also fixes a bug where database activity times were logged as 1/1000 of their correct ratio due to confusion between milliseconds and seconds. | ||||
* | | | Run Black. (#5482) | Amber Brown | 2019-06-20 | 1 | -4/+2 |
| |/ |/| | |||||
* | | Don't send renewal emails to deactivated users | Brendan Abolivier | 2019-06-14 | 1 | -2/+2 |
|/ | |||||
* | Add ability to perform password reset via email without trusting the ↵ | Andrew Morgan | 2019-06-06 | 1 | -1/+5 |
| | | | | | | | | | | | | identity server (#5377) Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option. This PR is a culmination of 3 smaller PRs which have each been separately reviewed: * #5308 * #5345 * #5368 | ||||
* | Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta | Erik Johnston | 2019-05-31 | 1 | -2/+20 |
|\ | | | | | Allow configuring a range for the account validity startup job | ||||
| * | Move delta from +10% to -10% | Brendan Abolivier | 2019-05-31 | 1 | -2/+2 |
| | | |||||
| * | Make max_delta equal to period * 10% | Brendan Abolivier | 2019-05-31 | 1 | -4/+3 |
| | | |||||
| * | Allow configuring a range for the account validity startup job | Brendan Abolivier | 2019-05-28 | 1 | -2/+21 |
| | | | | | | | | | | When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal. In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch. | ||||
* | | Log actual number of entries deleted | Erik Johnston | 2019-05-29 | 1 | -3/+9 |
|/ | |||||
* | Do the select and insert in a single transaction | Brendan Abolivier | 2019-05-21 | 1 | -10/+6 |
| | |||||
* | Add startup background job for account validity | Brendan Abolivier | 2019-05-17 | 1 | -0/+62 |
| | | | | If account validity is enabled in the server's configuration, this job will run at startup as a background job and will stick an expiration date to any registered account missing one. | ||||
* | Clean up the database pagination code (#5007) | Amber Brown | 2019-04-05 | 1 | -60/+50 |
| | | | | | | | | * rewrite & simplify * changelog * cleanup potential sql injection | ||||
* | Run black on the rest of the storage module (#4996) | Amber Brown | 2019-04-03 | 1 | -223/+173 |
| | |||||
* | Fix sync bug when accepting invites (#4956) | Richard van der Hoff | 2019-04-02 | 1 | -5/+0 |
| | | | | | | | | | | Hopefully this time we really will fix #4422. We need to make sure that the cache on `get_rooms_for_user_with_stream_ordering` is invalidated *before* the SyncHandler is notified for the new events, and we can now do so reliably via the `events` stream. | ||||
* | fixes | Amber Brown | 2019-03-12 | 1 | -3/+10 |
| | |||||
* | Limit cache invalidation replication line length (#4748) | Erik Johnston | 2019-02-27 | 1 | -4/+11 |
| | |||||
* | Add database version to phonehome stats. (#4753) | Richard van der Hoff | 2019-02-27 | 1 | -0/+8 |
| | |||||
* | Fix state cache invalidation on workers | Erik Johnston | 2019-02-22 | 1 | -6/+34 |
| | |||||
* | UPSERT many functionality (#4644) | Amber Brown | 2019-02-20 | 1 | -12/+134 |
| | |||||
* | Docs | Erik Johnston | 2019-02-19 | 1 | -2/+3 |
| | |||||
* | Use itertools | Erik Johnston | 2019-02-19 | 1 | -2/+2 |
| | |||||
* | Documentation | Erik Johnston | 2019-02-19 | 1 | -4/+4 |
| | |||||
* | Batch cache invalidation over replication | Erik Johnston | 2019-02-18 | 1 | -1/+56 |
| | | | | | | | | | | Currently whenever the current state changes in a room invalidate a lot of caches, which cause *a lot* of traffic over replication. Instead, lets batch up all those invalidations and send a single poke down the replication streams. Hopefully this will reduce load on the master process by substantially reducing traffic. | ||||
* | Fix 'no unique or exclusion constraint' error (#4591) | Richard van der Hoff | 2019-02-08 | 1 | -6/+21 |
| | | | | | Add more tables to the list of tables which need a background update to complete before we can upsert into them, which fixes a race against the background updates. | ||||
* | Fix typo in upserts code (#4505) | Amber Brown | 2019-01-29 | 1 | -1/+1 |
| | | | | | | * fix obvious problem :| * changelog | ||||
* | Fix UPSERTs on SQLite 3.24+ (#4477) | Amber Brown | 2019-01-28 | 1 | -2/+8 |
| | |||||
* | Fix UPSERT check (#4459) | Amber Brown | 2019-01-24 | 1 | -5/+28 |
| | |||||
* | Use native UPSERTs where possible (#4306) | Amber Brown | 2019-01-24 | 1 | -11/+137 |
| | |||||
* | Fix adding new rows instead of updating them if one of the key values is a ↵ | Amber Brown | 2019-01-09 | 1 | -1/+9 |
| | | | | NULL in upserts. (#4369) | ||||
* | add more detail to logging regarding "More than one row matched" error (#4234) | Ben Parsons | 2018-12-04 | 1 | -6/+6 |
| | |||||
* | Fix UnicodeDecodeError when postgres is not configured in english (#4253) | Richard van der Hoff | 2018-12-04 | 1 | -7/+8 |
| | | | | This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2. | ||||
* | Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase ↵ | Amber Brown | 2018-10-20 | 1 | -2/+2 |
| | | | | on py3) (#4068) | ||||
* | Port storage/ to Python 3 (#3725) | Amber Brown | 2018-08-31 | 1 | -1/+31 |
| | |||||
* | initial cut at a room summary API (#3574) | Matthew Hodgson | 2018-08-16 | 1 | -4/+3 |
| | |||||
* | Fix another logcontext leak in _persist_events | Richard van der Hoff | 2018-07-25 | 1 | -0/+6 |
| | | | | | | | | We need to run the errback in the sentinel context to avoid losing our own context. Also: add logging to runInteraction to help identify where "Starting db connection from sentinel context" warnings are coming from | ||||
* | revert 00bc979 | Richard van der Hoff | 2018-07-19 | 1 | -4/+3 |
| | | | | | ... we've fixed the things that caused the warnings, so we should reinstate the warning. | ||||
* | Disable logcontext warning | Richard van der Hoff | 2018-07-19 | 1 | -3/+4 |
| | | | | Temporary workaround to #3518 while we release 0.33.0. | ||||
* | Attempt to include db threads in cpu usage stats (#3496) | Richard van der Hoff | 2018-07-10 | 1 | -17/+15 |
| | | | | | Let's try to include time spent in the DB threads in the per-request/block cpu usage metrics. | ||||
* | run isort | Amber Brown | 2018-07-09 | 1 | -10/+9 |
| | |||||
* | update metrics to be in seconds | Amber Brown | 2018-05-28 | 1 | -15/+15 |
| | |||||
* | Merge remote-tracking branch 'origin/develop' into 3218-official-prom | Amber Brown | 2018-05-28 | 1 | -18/+29 |
|\ | |||||
| * | Merge pull request #3278 from NotAFile/py3-storage-base | Amber Brown | 2018-05-24 | 1 | -17/+22 |
| |\ | | | | | | | Py3 storage/_base.py | ||||
| | * | Py3 storage/_base.py | Adrian Tschira | 2018-05-24 | 1 | -17/+22 |
| | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
| * | | use stand-in value if maxint is not available | Adrian Tschira | 2018-05-19 | 1 | -1/+7 |
| |/ | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* / | rest of the changes | Amber Brown | 2018-05-21 | 1 | -10/+7 |
|/ | |||||
* | Use psycopg2cffi module instead of psycopg2 if running on pypy | Vincent Breitmoser | 2018-04-10 | 1 | -1/+1 |
| | | | | | | | The psycopg2 package isn't available for PyPy. This commit adds a check if the runtime is PyPy, and if it is uses psycopg2cffi module in favor of psycopg2. This is almost a drop-in replacement, except for one place where an additional cast to string is required. | ||||
* | Fix bug with delayed cache invalidation stream | Erik Johnston | 2018-03-02 | 1 | -12/+14 |
| | | | | | | We poked the notifier before updated the current token for the cache invalidation stream. This mean that sometimes the update wouldn't be sent until the next time a cache was invalidated. | ||||
* | Track DB scheduling delay per-request | Richard van der Hoff | 2018-01-16 | 1 | -1/+3 |
| | | | | | | For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing. | ||||
* | rework runInteraction in terms of runConnection | Richard van der Hoff | 2018-01-16 | 1 | -20/+31 |
| | | | | ... so that we can share the code | ||||
* | Merge branch 'develop' into matthew/search-all-local-users | Matthew Hodgson | 2017-11-30 | 1 | -9/+14 |
|\ | |||||
| * | Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_error | Richard van der Hoff | 2017-11-27 | 1 | -9/+7 |
| |\ | | | | | | | Fix error on sqlite 3.7 | ||||
| | * | fix sql fails | Richard van der Hoff | 2017-11-22 | 1 | -1/+1 |
| | | | |||||
| | * | Check database in has_completed_background_updates | Richard van der Hoff | 2017-11-22 | 1 | -9/+7 |
| | | | | | | | | | | | | so that the right thing happens on workers. | ||||
| * | | Avoid retrying forever on IntegrityError | Richard van der Hoff | 2017-11-27 | 1 | -0/+7 |
| |/ | |||||
* / | untested WIP but might actually work | Matthew Hodgson | 2017-11-29 | 1 | -1/+1 |
|/ | |||||
* | Fix broken ref to IntegrityError | Richard van der Hoff | 2017-11-16 | 1 | -1/+1 |
| | |||||
* | _simple_upsert: retry on IntegrityError | Richard van der Hoff | 2017-11-16 | 1 | -6/+29 |
| | | | | | | wrap the call to _simple_upsert_txn in a loop so that we retry on an integrityerror: this means we can avoid locking the table provided there is an unique index. | ||||
* | Cleanup in _simple_upsert_txn | Richard van der Hoff | 2017-11-16 | 1 | -16/+17 |
| | | | | Bail out early to reduce indentation | ||||
* | Revert "Revert "move _state_group_cache to statestore"" | Richard van der Hoff | 2017-11-14 | 1 | -6/+0 |
| | | | | | | | We're going to fix this properly on this branch, so that the _state_group_cache can end up in StateGroupReadStore. This reverts commit ab335edb023d66cd0be439e045b10ca104b73cb5. | ||||
* | Make __init__ consitstent across Store heirarchy | Richard van der Hoff | 2017-11-13 | 1 | -1/+1 |
| | | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores) | ||||
* | Revert "move _state_group_cache to statestore" | Erik Johnston | 2017-11-13 | 1 | -0/+6 |
| | | | | This reverts commit f5cf3638e9c6086e1c33ddad8eda9298cf53a58e. | ||||
* | move _state_group_cache to statestore | Richard van der Hoff | 2017-11-07 | 1 | -6/+0 |
| | | | | this is internal to statestore, so let's keep it there. | ||||
* | replace 'except:' with 'except Exception:' | Richard van der Hoff | 2017-10-23 | 1 | -1/+1 |
| | | | | what could possibly go wrong | ||||
* | Add _simple_update | Erik Johnston | 2017-08-25 | 1 | -19/+32 |
| | |||||
* | Define CACHE_SIZE_FACTOR once | Erik Johnston | 2017-07-04 | 1 | -4/+1 |
| | |||||
* | Merge pull request #2259 from matrix-org/erikj/fix_state_woes | Erik Johnston | 2017-06-07 | 1 | -8/+21 |
|\ | | | | | Fix bug where state_group tables got corrupted | ||||
| * | Fix bug where state_group tables got corrupted | Erik Johnston | 2017-06-07 | 1 | -8/+21 |
| | | | | | | | | | | | | | | | | This is due to the fact that we prefilled caches using txn.call_after, which always gets called including on error. We fix this by making txn.call_after only fire when a transaction completes successfully, which is what we want most of the time anyway. | ||||
* | | Split the table in two | Erik Johnston | 2017-06-01 | 1 | -0/+5 |
|/ | |||||
* | Prefill state caches | Erik Johnston | 2017-05-15 | 1 | -4/+4 |
| | |||||
* | Revert "Prefill state caches" | Erik Johnston | 2017-05-04 | 1 | -4/+4 |
| | |||||
* | Prefill state caches | Erik Johnston | 2017-05-02 | 1 | -4/+4 |
| | |||||
* | Use iter(items|values) | Erik Johnston | 2017-03-24 | 1 | -6/+6 |
| | |||||
* | User Cursor.__iter__ instead of fetchall | Erik Johnston | 2017-03-23 | 1 | -5/+8 |
| | | | | This prevents unnecessary construction of lists | ||||
* | Implement _simple_delete_many_txn, use it to delete devices | Luke Barnard | 2017-03-13 | 1 | -0/+41 |
| | | | | | | (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review. | ||||
* | Remove spurious SQL logging (#1972) | Richard van der Hoff | 2017-03-08 | 1 | -8/+0 |
| | | | | | looks like the upsert function was accidentally sending sql logging to the general logger. We already log the sql in `txn.execute`. | ||||
* | Intern table column names once | Erik Johnston | 2017-02-28 | 1 | -3/+2 |
| | |||||
* | Strip newlines from SQL queries | Erik Johnston | 2017-02-23 | 1 | -0/+6 |
| | |||||
* | admin,storage: added more administrator functionalities | Morteza Araby | 2017-02-02 | 1 | -0/+159 |
| | | | | | | | | | | | | | | administrators can now: - Set displayname of users - Update user avatars - Search for users by user_id - Browse all users in a paginated API - Reset user passwords - Deactivate users Helpers for doing paginated queries has also been added to storage Signed-off-by: Morteza Araby <morteza.araby@ericsson.com> | ||||
* | Add basic implementation of local device list changes | Erik Johnston | 2017-01-25 | 1 | -0/+6 |
| | |||||
* | Increase state_group_cache_size | Erik Johnston | 2017-01-17 | 1 | -1/+1 |
| | |||||
* | Limit number of entries to prefill from cache | Erik Johnston | 2017-01-10 | 1 | -2/+3 |
| | | | | | | | | | Some tables, like device_inbox, take a long time to query at startup for the stream change cache prefills. This is likely because they are slower growing streams and so are more fragmented on disk. For now, lets pull fewer entries out to make startup quicker. In future, we should add a better index to make it even faster. | ||||
* | Fix tests | Erik Johnston | 2016-11-21 | 1 | -2/+2 |
| | |||||
* | Store federation stream positions in the database | Erik Johnston | 2016-11-21 | 1 | -4/+14 |
| | |||||
* | Refactor test_filter to use real DataStore | pik | 2016-10-18 | 1 | -1/+0 |
| | | | | * add tests for filter api errors | ||||
* | Remove lru option | Erik Johnston | 2016-08-19 | 1 | -1/+1 |
| | |||||
* | Do it in storage function | Erik Johnston | 2016-08-16 | 1 | -0/+3 |
| | |||||
* | Use cached get_user_by_access_token in slaves | Erik Johnston | 2016-08-16 | 1 | -0/+1 |
| | |||||
* | Doc get_next() context manager usage | Erik Johnston | 2016-08-15 | 1 | -0/+4 |
| | |||||
* | Rename table. Add docs. | Erik Johnston | 2016-08-15 | 1 | -2/+10 |
| | |||||
* | Implement cache replication stream | Erik Johnston | 2016-08-15 | 1 | -12/+35 |
| | |||||
* | Add some invalidations to a cache_stream | Erik Johnston | 2016-08-15 | 1 | -0/+18 |
| | |||||
* | Always run txn.after_callbacks | Erik Johnston | 2016-08-15 | 1 | -7/+8 |
| | |||||
* | More doc-comments | Richard van der Hoff | 2016-07-20 | 1 | -7/+12 |
| | | | | Fix some more comments on some things | ||||
* | Feature: Add deactivate account admin API | Erik Johnston | 2016-06-30 | 1 | -0/+5 |
| | | | | | | | | | | Allows server admins to "deactivate" accounts, which: - Revokes all access tokens - Removes all threepids - Removes password The API is a POST to `/admin/deactivate/<user_id>` | ||||
* | Fix setting the _clock in SQLBaseStore | Mark Haines | 2016-06-02 | 1 | -0/+1 |
| | |||||
* | Make deleting push actions more efficient | David Baker | 2016-05-20 | 1 | -1/+0 |
| | | | | There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour. | ||||
* | Spelling | Erik Johnston | 2016-05-13 | 1 | -1/+1 |
| | |||||
* | Fixup add_pusher | Erik Johnston | 2016-05-13 | 1 | -1/+7 |
| | |||||
* | Remove some unused functions (#711) | Mark Haines | 2016-04-08 | 1 | -6/+0 |
| | | | | | | | | * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want | ||||
* | Move _get_cache_dict into the SQLBaseStore | Mark Haines | 2016-04-06 | 1 | -0/+34 |
| | |||||
* | Intern all the things | Erik Johnston | 2016-03-23 | 1 | -1/+2 |
| | |||||
* | Make stateGroupCache honour CACHE_SIZE_FACTOR | Erik Johnston | 2016-03-22 | 1 | -1/+7 |
| | |||||
* | Add a stream for push rule updates | Mark Haines | 2016-03-01 | 1 | -7/+18 |
| | |||||
* | Fix up logcontexts | Erik Johnston | 2016-02-08 | 1 | -9/+9 |
| | |||||
* | Fix flake8 warnings for new flake8 | Daniel Wagner-Hall | 2016-02-02 | 1 | -2/+5 |
| | |||||
* | Add a Homeserver.setup method. | Erik Johnston | 2016-01-26 | 1 | -26/+23 |
| | | | | | | This is for setting up dependencies that require work on startup. This is useful for the DataStore that wants to read a bunch from the database before initiliazing. | ||||
* | Guard against empty iterables | Erik Johnston | 2016-01-25 | 1 | -0/+6 |
| | |||||
* | Correct docstring | Erik Johnston | 2016-01-25 | 1 | -1/+0 |
| | |||||
* | Implement a _simple_select_many_batch | Erik Johnston | 2016-01-25 | 1 | -0/+67 |
| | |||||
* | copyrights | Matthew Hodgson | 2016-01-07 | 1 | -1/+1 |
| | |||||
* | Track the time spent in the database per request. | Mark Haines | 2015-12-07 | 1 | -2/+7 |
| | | | | and track the number of transactions that request started. | ||||
* | Add basic full text search impl. | Erik Johnston | 2015-10-09 | 1 | -1/+1 |
| | |||||
* | synapse/storage/_base.py:_simple_max_id was unused | Mark Haines | 2015-09-23 | 1 | -18/+0 |
| | |||||
* | synapse/storage/_base.py:_simple_delete was unused | Mark Haines | 2015-09-23 | 1 | -10/+0 |
| | |||||
* | synapse/storage/_base.py:_simple_selectupdate_one was unused | Mark Haines | 2015-09-23 | 1 | -31/+0 |
| | |||||
* | synapse/storage/_base.py:_execute_and_decode was unused | Mark Haines | 2015-09-22 | 1 | -3/+0 |
| | |||||
* | synapse/storage/_base.py:Table was unused | Mark Haines | 2015-09-22 | 1 | -128/+0 |
| | |||||
* | Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop | Erik Johnston | 2015-09-01 | 1 | -1/+1 |
|\ | |||||
| * | Lower size of 'stateGroupCache' now that we have data from matrix.org to ↵ | Erik Johnston | 2015-09-01 | 1 | -1/+1 |
| | | | | | | | | support doing so | ||||
* | | /tokenrefresh POST endpoint | Daniel Wagner-Hall | 2015-08-20 | 1 | -0/+1 |
|/ | | | | | | | | This allows refresh tokens to be exchanged for (access_token, refresh_token). It also starts issuing them on login, though no clients currently interpret them. | ||||
* | Move all the caches into their own package, synapse.util.caches | Erik Johnston | 2015-08-11 | 1 | -332/+3 |
| | |||||
* | Comments | Erik Johnston | 2015-08-11 | 1 | -0/+2 |
| | |||||
* | Comment | Erik Johnston | 2015-08-11 | 1 | -0/+2 |
| | |||||
* | Docs | Erik Johnston | 2015-08-11 | 1 | -0/+9 |
| | |||||
* | Comments | Erik Johnston | 2015-08-10 | 1 | -0/+6 |
| | |||||
* | Merge branch 'erikj/cache_varargs_interface' of ↵ | Erik Johnston | 2015-08-10 | 1 | -5/+5 |
|\ | | | | | | | github.com:matrix-org/synapse into erikj/dictionary_cache | ||||
| * | Rename keyargs to args in CacheDescriptor | Erik Johnston | 2015-08-10 | 1 | -5/+5 |
| | | |||||
* | | Merge branch 'erikj/cache_varargs_interface' of ↵ | Erik Johnston | 2015-08-10 | 1 | -9/+9 |
|\| | | | | | | | github.com:matrix-org/synapse into erikj/dictionary_cache | ||||
| * | Rename keyargs to args in Cache | Erik Johnston | 2015-08-10 | 1 | -9/+9 |
| | | |||||
| * | Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-08-07 | 1 | -3/+3 |
| |\ | | | | | | | | | | erikj/cache_varargs_interface | ||||
| * \ | Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-08-07 | 1 | -3/+4 |
| |\ \ | | | | | | | | | | | | | erikj/cache_varargs_interface | ||||
| * | | | Change Cache to not use *args in its interface | Erik Johnston | 2015-08-07 | 1 | -30/+31 |
| | | | | |||||
* | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-08-07 | 1 | -3/+4 |
|\ \ \ \ | | |_|/ | |/| | | | | | | erikj/dictionary_cache | ||||
| * | | | s/observed/observer/ | Erik Johnston | 2015-08-07 | 1 | -3/+3 |
| | |/ | |/| | |||||
| * | | Propogate stale cache errors to calling functions | Erik Johnston | 2015-08-07 | 1 | -3/+4 |
| |/ | |||||
* | | Implement a CacheListDescriptor | Erik Johnston | 2015-08-07 | 1 | -0/+106 |
| | | |||||
* | | Change Cache to not use *args in its interface | Erik Johnston | 2015-08-07 | 1 | -50/+35 |
| | | |||||
* | | Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache | Erik Johnston | 2015-08-06 | 1 | -3/+27 |
|\| | |||||
| * | Docs | Erik Johnston | 2015-08-06 | 1 | -0/+3 |
| | | |||||
| * | Remove failed deferreds from cache | Erik Johnston | 2015-08-06 | 1 | -1/+7 |
| | | |||||
| * | Re-implement DEBUG_CACHES flag | Erik Johnston | 2015-08-06 | 1 | -2/+17 |
| | | |||||
* | | Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache | Erik Johnston | 2015-08-06 | 1 | -13/+8 |
|\| | |||||
| * | Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds | Erik Johnston | 2015-08-06 | 1 | -3/+3 |
| |\ | |||||
| | * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-08-06 | 1 | -3/+3 |
| | |\ | | | | | | | | | | | | | erikj/cached_keyword_args | ||||
| * | | | Make @cached cache deferreds rather than the deferreds' values | Erik Johnston | 2015-08-06 | 1 | -13/+8 |
| |/ / | |||||
* | | | Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse ↵ | Erik Johnston | 2015-08-05 | 1 | -6/+34 |
|\| | | | | | | | | | | | into erikj/dictionary_cache | ||||
| * | | Add support for using keyword arguments with cached functions | Erik Johnston | 2015-07-27 | 1 | -6/+34 |
| | | | |||||
* | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-08-05 | 1 | -3/+3 |
|\ \ \ | | |/ | |/| | | | | erikj/dictionary_cache | ||||
| * | | Use LRU cache by default | Erik Johnston | 2015-08-05 | 1 | -3/+3 |
| |/ | |||||
* | | Use dictionary cache to do group -> state fetching | Erik Johnston | 2015-08-05 | 1 | -13/+26 |
| | | |||||
* | | Speed up event filtering (for ACL) logic | Erik Johnston | 2015-08-04 | 1 | -2/+8 |
|/ | |||||
* | Add basic storage functions for handling of receipts | Erik Johnston | 2015-07-01 | 1 | -1/+2 |
| | |||||
* | Caches should be bound to instances. | Erik Johnston | 2015-06-03 | 1 | -14/+31 |
| | | | | | Before, caches were global and so different instances of the stores would share caches. This caused problems in the unit tests. | ||||
* | Merge branch 'master' of github.com:matrix-org/synapse into develop | Erik Johnston | 2015-05-22 | 1 | -0/+6 |
|\ | |||||
| * | Add caches for things requested by the pushers | Mark Haines | 2015-05-21 | 1 | -0/+1 |
| | | |||||
| * | Add a cache for get_current_state with state_key | Mark Haines | 2015-05-21 | 1 | -0/+5 |
| | | |||||
* | | Remove unused metric | Erik Johnston | 2015-05-18 | 1 | -1/+0 |
| | | |||||
* | | Newline, remove debug logging | Erik Johnston | 2015-05-18 | 1 | -1/+0 |
| | | |||||
* | | PEP8 | Erik Johnston | 2015-05-15 | 1 | -1/+0 |
| | | |||||
* | | Add a wait | Erik Johnston | 2015-05-15 | 1 | -1/+1 |
| | | |||||
* | | Fix daedlock | Erik Johnston | 2015-05-15 | 1 | -16/+10 |
| | | |||||
* | | Remove race condition | Erik Johnston | 2015-05-14 | 1 | -61/+107 |
| | | |||||
* | | Count and loop | Erik Johnston | 2015-05-14 | 1 | -1/+1 |
| | | |||||
* | | Awful idea for speeding up fetching of events | Erik Johnston | 2015-05-14 | 1 | -0/+4 |
| | | |||||
* | | Move from _base to events | Erik Johnston | 2015-05-14 | 1 | -232/+1 |
| | | |||||
* | | Jump out early | Erik Johnston | 2015-05-14 | 1 | -0/+6 |
| | | |||||
* | | Actually, we probably want to run this in a transaction | Erik Johnston | 2015-05-14 | 1 | -15/+12 |
| | | |||||
* | | Preemptively jump into a transaction if we ask for get_prev_content | Erik Johnston | 2015-05-14 | 1 | -12/+22 |
| | | |||||
* | | loop -> gatherResults | Erik Johnston | 2015-05-14 | 1 | -10/+14 |
| | | |||||
* | | Err, we probably want a bigger limit | Erik Johnston | 2015-05-14 | 1 | -1/+1 |
| | | |||||
* | | Refactor _get_events | Erik Johnston | 2015-05-14 | 1 | -252/+94 |
| | | |||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf | Erik Johnston | 2015-05-14 | 1 | -0/+1 |
|\ \ | |||||
| * | | Add ID generator for push_rules_enable to #resolve SYN-378 | David Baker | 2015-05-14 | 1 | -0/+1 |
| | | | |||||
* | | | Actually use async method | Erik Johnston | 2015-05-13 | 1 | -3/+1 |
| | | | |||||
* | | | Fetch events from events_id in their own transactions | Erik Johnston | 2015-05-13 | 1 | -3/+151 |
| | | | |||||
* | | | Typo | Erik Johnston | 2015-05-13 | 1 | -1/+1 |
| | | | |||||
* | | | Don't insert None | Erik Johnston | 2015-05-13 | 1 | -1/+4 |
| | | | |||||
* | | | Don't return None | Erik Johnston | 2015-05-13 | 1 | -1/+1 |
| | | | |||||
* | | | Limit batch size | Erik Johnston | 2015-05-13 | 1 | -5/+10 |
| | | | |||||
* | | | Fetch events in bulk | Erik Johnston | 2015-05-13 | 1 | -13/+62 |
| | | | |||||
* | | | Don't fetch redaction and rejection stuff for each event, so we can use ↵ | Erik Johnston | 2015-05-13 | 1 | -4/+19 |
| | | | | | | | | | | | | index only scan | ||||
* | | | Load events for state group seperately | Erik Johnston | 2015-05-13 | 1 | -2/+2 |
|/ / | |||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2015-05-12 | 1 | -0/+1 |
|\| | | | | | | | erikj/logging_context | ||||
| * | push_rules table expects an 'id' field | Erik Johnston | 2015-05-11 | 1 | -0/+1 |
| | | |||||
* | | Change the way we do logging contexts so that they survive divergences | Erik Johnston | 2015-05-08 | 1 | -5/+6 |
|/ |