summary refs log tree commit diff
path: root/synapse/util/caches/descriptors.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Remove `keylen` from `LruCache`. (#9993)Richard van der Hoff2021-05-241-1/+0
| | | | | | | `keylen` seems to be a thing that is frequently incorrectly set, and we don't really need it. The only time it was used was to figure out if we had removed a subtree in `del_multi`, which we can do better by changing `TreeCache.pop` to return a different type (`TreeCacheNode`). Commits should be independently reviewable.
* Minor `@cachedList` enhancements (#9975)Richard van der Hoff2021-05-141-6/+8
| | | | | | - use a tuple rather than a list for the iterable that is passed into the wrapped function, for performance - test that we can pass an iterable and that keys are correctly deduped.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-5/+12
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Implement and use an @lru_cache decorator (#8595)Richard van der Hoff2020-10-301-47/+188
| | | We don't always need the full power of a DeferredCache.
* Optimise CacheDescriptor (#8594) github/release-v1.21.3 release-v1.21.3Richard van der Hoff2020-10-211-5/+7
| | | don't bother constricting a CacheContext unless we need one.
* Push some deferred wrangling down into DeferredCacheRichard van der Hoff2020-10-211-25/+7
|
* move DeferredCache into its own moduleRichard van der Hoff2020-10-141-282/+2
|
* Rename Cache->DeferredCacheRichard van der Hoff2020-10-141-7/+12
|
* Add some more type annotations to CacheRichard van der Hoff2020-10-141-22/+59
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-4/+4
|
* Fix typing for `@cached` wrapped functions (#8240)Erik Johnston2020-09-031-14/+28
| | | This requires adding a mypy plugin to fiddle with the type signatures a bit.
* Remove the unused inlineCallbacks code-paths in the caching code (#8119)Patrick Cloke2020-08-191-46/+8
|
* Remove unnecessary maybeDeferred calls (#8044)Patrick Cloke2020-08-071-1/+1
|
* Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-091-1/+1
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-3/+1
|
* Allow configuration of Synapse's cache without using synctl or environment ↵Amber Brown2020-05-111-6/+30
| | | | variables (#6391)
* look up cross-signing keys from the DB in bulk (#6486)Hubert Chathi2019-12-121-1/+1
|
* Fix LruCache callback deduplication (#6213)V024602019-11-071-11/+37
|
* Update docstringErik Johnston2019-10-291-3/+2
|
* Quick fix to ensure cache descriptors always return deferredsErik Johnston2019-10-281-2/+2
|
* Fix up some typechecking (#6150)Amber Brown2019-10-021-2/+20
| | | | | | * type checking fixes * changelog
* Fix some error cases in the caching layer. (#5749)Richard van der Hoff2019-07-251-32/+42
| | | | | | | There was some inconsistent behaviour in the caching layer around how exceptions were handled - particularly synchronously-thrown ones. This seems to be most easily handled by pushing the creation of ObservableDeferreds down from CacheDescriptor to the Cache.
* Add a prometheus metric for active cache lookups. (#5750)Richard van der Hoff2019-07-241-1/+17
| | | | | | * Add a prometheus metric for active cache lookups. * changelog
* Replace returnValue with return (#5736)Amber Brown2019-07-231-1/+1
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-5/+6
|
* Run Black. (#5482)Amber Brown2019-06-201-47/+53
|
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-1/+1
|
* fix invalidationRichard van der Hoff2018-07-271-1/+1
|
* Rewrite cache list decoratorRichard van der Hoff2018-07-271-67/+64
| | | | | Because it was complicated and annoyed me. I suspect this will be more efficient too.
* run isortAmber Brown2018-07-091-12/+10
|
* Add hacky cache factor override systemErik Johnston2018-06-041-2/+2
|
* Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-1/+1
| | | | they're not meant to be lazy (#3307)
* Merge pull request #3281 from NotAFile/py3-six-isinstanceAmber Brown2018-05-301-4/+8
|\ | | | | remaining isintance fixes
| * remaining isintance fixesAdrian Tschira2018-05-241-4/+8
| | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* | fixesAmber Brown2018-05-221-1/+1
| |
* | replacing portionsAmber Brown2018-05-211-1/+1
|/
* Fix overzealous cache invalidationRichard van der Hoff2018-04-051-26/+38
| | | | | Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate.
* report metrics on number of cache evictionsRichard van der Hoff2018-02-051-0/+4
|
* Define CACHE_SIZE_FACTOR onceErik Johnston2017-07-041-4/+1
|
* Pull out if statement from for loopErik Johnston2017-05-221-6/+14
|
* Update list cache to handle one arg caseErik Johnston2017-05-221-17/+33
| | | | | | We update the normal cache descriptors to handle caches with a single argument specially so that the key wasn't a 1-tuple. We need to update the cache list to be aware of this.
* Don't update event cache hit ratio from get_joined_usersErik Johnston2017-05-081-3/+6
| | | | | Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.
* Optimise caches with single keyErik Johnston2017-05-041-9/+33
|
* Reduce cache size by not storing deferredsErik Johnston2017-04-251-18/+21
| | | | | | | | | | | | | | | | | | | | Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.
* Remove unused instance variableErik Johnston2017-03-311-4/+0
|
* Doc new instance variablesErik Johnston2017-03-301-1/+8
|
* Manually calculate cache key as getcallargs is expensiveErik Johnston2017-03-301-6/+28
| | | | | This is because getcallargs recomputes the getargspec, amongst other things, which we don't need to do as its already been done
* Don't convert to deferreds when not necessaryErik Johnston2017-03-301-1/+4
|
* Fix the logcontext handling in the cache wrappers (#2077)Richard van der Hoff2017-03-301-16/+14
| | | | | | | The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.
* Fix caching of remote servers' signature keysRichard van der Hoff2017-03-221-63/+72
| | | | | | | | | The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
* CommentErik Johnston2017-02-021-1/+4
|
* CommentErik Johnston2017-02-021-0/+2
|
* Fix typo in return typeErik Johnston2017-01-171-3/+2
|
* Rename and comment tree_to_leaves_iteratorErik Johnston2017-01-171-2/+2
|
* Speed up cache size calculationErik Johnston2017-01-171-23/+74
| | | | | | | | | | | Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.
* Optionally measure size of cache by sum of length of valuesErik Johnston2017-01-131-5/+20
|
* Ensure invalidation list does not grow unboundedlyErik Johnston2016-08-191-12/+8
|
* Rename to on_invalidateErik Johnston2016-08-191-16/+10
|
* Make cache_context an explicit optionErik Johnston2016-08-191-8/+27
|
* Remove lru optionErik Johnston2016-08-191-23/+8
|
* Add concept of cache contextsErik Johnston2016-08-191-13/+64
|
* Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into developErik Johnston2016-06-031-10/+25
|\
| * Pull out update_results_dictErik Johnston2016-06-031-8/+7
| |
| * Small optimisation to CacheListDescriptorErik Johnston2016-06-031-2/+2
| |
| * Make cachedList go a bit fasterErik Johnston2016-06-031-10/+34
| |
* | Change CacheMetrics to be quickerErik Johnston2016-06-031-4/+13
|/ | | | | | We change it so that each cache has an individual CacheMetric, instead of having one global CacheMetric. This means that when a cache tries to increment a counter it does not need to go through so many indirections.
* Make the cache objects be per instance rather than being globalMark Haines2016-04-061-21/+24
|
* Move cache size fiddling to descriptors only. Fix testsErik Johnston2016-03-011-2/+2
|
* Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1Erik Johnston2016-03-011-0/+6
|
* Fix up logcontextsErik Johnston2016-02-081-5/+11
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-2/+2
|
* Make LRU cache not default to treecache & add options to use itDavid Baker2016-01-221-6/+14
|
* Add invalidate_many here tooDavid Baker2016-01-221-0/+1
|
* Change LRUCache to be tree-based so we can delete subtrees.David Baker2016-01-211-1/+10
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Don't assume @cachedList function returns keys for everythingErik Johnston2015-08-181-1/+1
|
* DocstringErik Johnston2015-08-121-0/+27
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2015-08-121-1/+3
| | | | erikj/dictionary_cache
* Wire up the dictionarycache to the metricsErik Johnston2015-08-121-14/+3
|
* Move all the caches into their own package, synapse.util.cachesErik Johnston2015-08-111-0/+359
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-20 09:09:41 +0000