summary refs log tree commit diff
path: root/tests/util (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #6505 from matrix-org/erikj/make_deferred_yiedableErik Johnston2019-12-101-0/+24
|\ | | | | Fix `make_deferred_yieldable` to work with coroutines
| * Fix make_deferred_yieldable to work with coroutinesErik Johnston2019-12-101-0/+24
| |
* | Remove SnapshotCache in favour of ResponseCacheErik Johnston2019-12-091-63/+0
|/
* Make ObservableDeferred.observe() always return deferred.Erik Johnston2019-10-301-2/+2
| | | | | | | This makes it easier to use in an async/await world. Also fixes a bug where cache descriptors would occaisonally return a raw value rather than a deferred.
* Quick fix to ensure cache descriptors always return deferredsErik Johnston2019-10-281-2/+2
|
* Add 'failure_ts' column to 'destinations' table (#6016)Richard van der Hoff2019-09-171-0/+127
| | | | Track the time that a server started failing at, for general analysis purposes.
* Retry well-known lookup before expiry.Erik Johnston2019-08-131-2/+2
| | | | | | | | | This gives a bit of a grace period where we can attempt to refetch a remote `well-known`, while still using the cached result if that fails. Hopefully this will make the well-known resolution a bit more torelant of failures, rather than it immediately treating failures as "no result" and caching that for an hour.
* Fix some error cases in the caching layer. (#5749)Richard van der Hoff2019-07-251-3/+87
| | | | | | | There was some inconsistent behaviour in the caching layer around how exceptions were handled - particularly synchronously-thrown ones. This seems to be most easily handled by pushing the creation of ObservableDeferreds down from CacheDescriptor to the Cache.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-4/+4
|
* Fixes to the federation rate limiter (#5621)Richard van der Hoff2019-07-051-0/+97
| | | | | | | - Put the default window_size back to 1000ms (broken by #5181) - Make the `rc_federation` config actually do something - fix an off-by-one error in the 'concurrent' limit - Avoid creating an unused `_PerHostRatelimiter` object for every single incoming request
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-045-47/+44
|
* Fix media repo breaking (#5593)Amber Brown2019-07-021-12/+21
|
* Run Black. (#5482)Amber Brown2019-06-202-53/+53
|
* Run Black on the tests again (#5170)Amber Brown2019-05-101-12/+11
|
* Remove periods from copyright headers (#5046)Andrew Morgan2019-04-111-1/+1
|
* Add a caching layer to .well-known responses (#4516)Richard van der Hoff2019-01-301-0/+83
|
* Fix incorrect logcontexts after a Deferred was cancelled (#4407)Richard van der Hoff2019-01-171-0/+104
|
* Include eventid in log lines when processing incoming federation ↵Richard van der Hoff2018-09-271-0/+5
| | | | | | | | | | | transactions (#3959) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.
* Fix some instances of ExpiringCache not expiring cache itemsErik Johnston2018-09-211-1/+0
| | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.
* Run black.black2018-08-1010-76/+41
|
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-102-2/+2
|
* fix invalidationRichard van der Hoff2018-07-271-1/+41
|
* Rewrite cache list decoratorRichard van der Hoff2018-07-271-0/+61
| | | | | Because it was complicated and annoyed me. I suspect this will be more efficient too.
* Test and fix support for cancellation in LinearizerRichard van der Hoff2018-07-201-0/+31
|
* Combine Limiter and LinearizerRichard van der Hoff2018-07-202-70/+47
| | | | | Linearizer was effectively a Limiter with max_count=1, so rather than maintaining two sets of code, let's combine them.
* Add a sleep to the Limiter to fix stack overflows.Richard van der Hoff2018-07-201-4/+4
| | | | Fixes #3570
* Add unit testErik Johnston2018-07-171-0/+16
|
* Fixup unit testErik Johnston2018-07-131-3/+3
|
* run isortAmber Brown2018-07-0914-30/+38
|
* Reinstate lost run_on_reactor in unit testRichard van der Hoff2018-07-041-2/+15
| | | | | | | | a61738b removed a call to run_on_reactor from a unit test, but that call was doing something useful, in making the function in question asynchronous. Reinstate the call and add a check that we are testing what we wanted to be testing.
* Disable partial state group caching for wildcard lookupsRichard van der Hoff2018-06-221-6/+6
| | | | | | | When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase.
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-223-11/+13
|
* Remove run_on_reactor (#3395)Amber Brown2018-06-141-2/+0
|
* Port to sortedcontainers (with tests!) (#3332)Amber Brown2018-06-061-0/+198
|
* fix another dumb typoMatthew Hodgson2018-05-291-2/+2
|
* Merge branch 'develop' into rav/more_logcontext_leaksRichard van der Hoff2018-05-021-0/+38
|\
| * Fix incorrect reference to StringIORichard van der Hoff2018-05-021-0/+38
| | | | | | | | This was introduced in 4f2f5171
* | Remove spurious unittest.DEBUGRichard van der Hoff2018-05-021-1/+0
| |
* | Fix a class of logcontext leaksRichard van der Hoff2018-05-021-11/+56
|/ | | | | | | | | | | | | | | | So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the *result* of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
* Merge branch 'develop' into rav/deferred_timeoutRichard van der Hoff2018-04-272-2/+3
|\
| * Make tests py3 compatibleAdrian Tschira2018-04-162-2/+3
| | | | | | | | | | | | | | | | | | | | | | This is a mixed commit that fixes various small issues * print parentheses * 01 is invalid syntax (it was octal in py2) * [x for i in 1, 2] is invalid syntax * six moves Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Use deferred.addTimeout instead of time_bound_deferredRichard van der Hoff2018-04-231-33/+0
|/ | | | This doesn't feel like a wheel we need to reinvent.
* Fix overzealous cache invalidationRichard van der Hoff2018-04-051-0/+46
| | | | | Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate.
* Merge pull request #2804 from matrix-org/erikj/file_consumerErik Johnston2018-01-181-0/+176
|\ | | | | Add decent impl of a FileConsumer
| * Move test stuff to testsErik Johnston2018-01-181-8/+46
| |
| * Add decent impl of a FileConsumerErik Johnston2018-01-171-0/+138
| | | | | | | | | | | | | | | | | | | | Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind
* | Optimise LoggingContext creation and copyingRichard van der Hoff2018-01-161-8/+8
|/ | | | | | | | It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.
* Fix name of test_logcontextRichard van der Hoff2017-10-171-0/+0
| | | | The file under test is logcontext.py, not log_context.py
* Add some tests for make_deferred_yieldableRichard van der Hoff2017-10-171-0/+38
|
* Fix stackoverflow and logcontexts from linearizerRichard van der Hoff2017-10-111-4/+24
| | | | | | | 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.
* Make get_state_groups_from_groups faster.Erik Johnston2017-05-171-1/+1
| | | | | | | | | Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.
* Fix up testsErik Johnston2017-03-302-1/+41
|
* Fix the logcontext handling in the cache wrappers (#2077)Richard van der Hoff2017-03-301-0/+91
| | | | | | | The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.
* Merge pull request #2052 from matrix-org/rav/time_bound_deferredRichard van der Hoff2017-03-231-0/+33
|\ | | | | Fix time_bound_deferred to throw the right exception
| * Fix time_bound_deferred to throw the right exceptionRichard van der Hoff2017-03-231-0/+33
| | | | | | | | | | | | Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.
* | Fix caching of remote servers' signature keysRichard van der Hoff2017-03-222-0/+100
|/ | | | | | | | | The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
* Stop preserve_fn leaking context into the reactorRichard van der Hoff2017-03-181-0/+61
| | | | | | | | Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.
* Tidy up testErik Johnston2017-01-171-12/+12
|
* Speed up cache size calculationErik Johnston2017-01-171-15/+15
| | | | | | | | | | | Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.
* Add ExpiringCache testsErik Johnston2017-01-161-0/+84
|
* Optionally measure size of cache by sum of length of valuesErik Johnston2017-01-131-0/+25
|
* Add Limiter: limit concurrent access to resourceErik Johnston2016-11-101-0/+70
|
* Ensure invalidation list does not grow unboundedlyErik Johnston2016-08-191-0/+40
|
* Add concept of cache contextsErik Johnston2016-08-191-0/+113
|
* Add ReadWriteLockErik Johnston2016-07-051-0/+85
|
* Deduplicate joinsErik Johnston2016-04-071-0/+44
|
* Add a test for TreeCache.__contains__Mark Haines2016-02-221-0/+6
|
* Fix flake8 warnings for testsMark Haines2016-02-194-2/+2
|
* Add wheeltimer implErik Johnston2016-02-171-0/+74
|
* Fix testErik Johnston2016-02-091-9/+1
|
* Add testsErik Johnston2016-01-292-0/+19
|
* Revert all the bits changing keys of eeverything that used LRUCaches to tuplesDavid Baker2016-01-221-22/+22
|
* Make LRU cache not default to treecache & add options to use itDavid Baker2016-01-221-1/+2
|
* Test treecache directlyDavid Baker2016-01-221-0/+19
|
* Add tests for treecache directly and test del_multi at the LruCache level too.David Baker2016-01-221-0/+66
|
* Change LRUCache to be tree-based so we can delete subtrees.David Baker2016-01-211-22/+22
|
* copyrightsMatthew Hodgson2016-01-074-4/+4
|
* Add a unit test for the snapshot cacheMark Haines2015-12-231-0/+60
|
* Remove the LockManager class because it wasn't being usedMark Haines2015-11-041-108/+0
|
* Move all the caches into their own package, synapse.util.cachesErik Johnston2015-08-112-4/+2
|
* Move DictionaryCacheErik Johnston2015-08-041-1/+1
|
* Add basic dictionary cacheErik Johnston2015-08-041-0/+101
|
* Add a lru cache classMark Haines2015-02-111-0/+56
|
* Add a request-id to each log lineMark Haines2014-10-301-0/+43
|
* Have all unit tests import from our own subclass of trial's unittest ↵Paul "LeoNerd" Evans2014-09-121-2/+2
| | | | TestCase; set up logging in ONE PLACE ONLY
* fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-032-2/+2
| | | | hasn't been incorporated in time for launch.
* add in copyrights to everything, not just the synapse subdir, and add a ↵Matthew Hodgson2014-08-132-0/+28
| | | | copyrighter.pl whilst we're at it
* Reference Matrix Home Servermatrix.org2014-08-122-0/+95
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-05 23:14:34 +0000