summary refs log tree commit diff
path: root/synapse/util/async_helpers.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Collect information for PushRuleEvaluator in parallel. (#16590)Patrick Cloke2023-11-061-0/+14
| | | | | | | | Fetch information needed for push rule evaluation in parallel. Ideally this would use query pipelining, but this is not available in psycopg2. Due to the database thread pool this may result in little to no parallelization.
* Improve type hints for attrs classes (#16276)David Robertson2023-09-081-14/+11
|
* Run pyupgrade for python 3.7 & 3.8. (#16110)Patrick Cloke2023-08-151-1/+2
|
* Add stricter mypy options (#15694)Patrick Cloke2023-05-311-1/+1
| | | | Enable warn_unused_configs, strict_concatenate, disallow_subclassing_any, and disallow_incomplete_defs.
* Batch up replication requests to request the resyncing of remote users's ↵reivilibre2023-01-101-4/+51
| | | | devices. (#14716)
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-1/+2
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Immediately retry any requests that have backed off when a server comes back ↵Erik Johnston2022-05-101-0/+57
| | | | | online. (#12500) Otherwise it can take up to a minute for any in-flight `/send` requests to be retried.
* Use `ParamSpec` in a few places (#12667)David Robertson2022-05-091-7/+19
|
* Update `delay_cancellation` to accept any awaitable (#12468)Sean Quah2022-04-221-10/+42
| | | | | | | | This will mainly be useful when dealing with module callbacks, which are all typed as returning `Awaitable`s instead of coroutines or `Deferred`s. Signed-off-by: Sean Quah <seanq@element.io>
* Fix edge case where a `Linearizer` could get stuck (#12358)Sean Quah2022-04-051-1/+5
| | | | | | | Just after a task acquires a contended `Linearizer` lock, it sleeps. If the task is cancelled during this sleep, we need to release the lock. Signed-off-by: Sean Quah <seanq@element.io>
* Refactor and convert `Linearizer` to async (#12357)Sean Quah2022-04-051-77/+67
| | | | | | | | | | | Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah <seanq@element.io>
* Add cancellation support to `ReadWriteLock` (#12120)Sean Quah2022-03-141-31/+40
| | | | | | Also convert `ReadWriteLock` to use async context managers. Signed-off-by: Sean Quah <seanq@element.io>
* Add `delay_cancellation` utility function (#12180)Sean Quah2022-03-141-6/+42
| | | | | | | | | `delay_cancellation` behaves like `stop_cancellation`, except it delays `CancelledError`s until the original `Deferred` resolves. This is handy for unifying cleanup paths and ensuring that uncancelled coroutines don't use finished logcontexts. Signed-off-by: Sean Quah <seanq@element.io>
* Add type hints for `ObservableDeferred` attributes (#12159)Sean Quah2022-03-041-3/+11
| | | Signed-off-by: Sean Quah <seanq@element.io>
* Fix rare error in `ReadWriteLock` when writers complete immediately (#12105)Sean Quah2022-03-011-1/+4
| | | | Signed-off-by: Sean Quah <seanq@element.io>
* Add `stop_cancellation` utility function (#12106)Sean Quah2022-03-011-0/+19
|
* Improve exception handling for concurrent execution (#12109)Richard van der Hoff2022-03-011-22/+32
| | | | | | | | | | | | | | | * fix incorrect unwrapFirstError import this was being imported from the wrong place * Refactor `concurrently_execute` to use `yieldable_gather_results` * Improve exception handling in `yieldable_gather_results` Try to avoid swallowing so many stack traces. * mark unwrapFirstError deprecated * changelog
* Use auto_attribs/native type hints for attrs classes. (#11692)Patrick Cloke2022-01-131-3/+3
|
* Improve opentracing support for `ResponseCache` (#11607)Richard van der Hoff2021-12-201-1/+21
| | | This adds some opentracing annotations to ResponseCache, to make it easier to see what's going on; in particular, it adds a link back to the initial trace which is actually doing the work of generating the response.
* Add missing type hints to `synapse.logging.context` (#11556)Sean Quah2021-12-141-1/+56
|
* Add most missing type hints to synapse.util (#11328)Patrick Cloke2021-11-161-16/+16
|
* Add missing type hints to `synapse.app`. (#11287)Patrick Cloke2021-11-101-3/+2
|
* ObservableDeferred: run observers in order (#11229)Richard van der Hoff2021-11-021-16/+18
|
* Fix long-standing bug where `ReadWriteLock` could drop logging contexts (#10993)Sean Quah2021-10-081-2/+4
| | | | | | | | | | | Use `PreserveLoggingContext()` to ensure that logging contexts are not lost when exiting a read/write lock. When exiting a read/write lock, callbacks on a `Deferred` are triggered as a signal to any waiting coroutines. Any waiting coroutine that becomes runnable is likely to follow the Synapse logging context rules and will restore its own logging context, then either run to completion or await another `Deferred`, resetting the logging context in the process.
* Add types to synapse.util. (#10601)reivilibre2021-09-101-6/+10
|
* Generics for `ObservableDeferred` (#10491)Richard van der Hoff2021-07-281-6/+8
| | | | | Now that `Deferred` is a generic class, let's update `ObeservableDeferred` to follow suit.
* Fix up type hints for Twisted 21.7 (#10490)Richard van der Hoff2021-07-281-8/+8
| | | Mostly this involves decorating a few Deferred declarations with extra type hints. We wrap the types in quotes to avoid runtime errors when running against older versions of Twisted that don't have generics on Deferred.
* Use inline type hints in `http/federation/`, `storage/` and `util/` (#10381)Jonathan de Jong2021-07-151-4/+4
|
* When joining a remote room limit the number of events we concurrently check ↵Erik Johnston2021-06-081-5/+16
| | | | | signatures/hashes for (#10117) If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Enable addtional flake8-bugbear linting checks. (#9659)Jonathan de Jong2021-03-241-1/+1
|
* Add logging to ObservableDeferred callbacks (#9523)Jonathan de Jong2021-03-091-8/+18
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-7/+6
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Fix some typos.Patrick Cloke2021-02-121-1/+1
|
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-3/+5
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Fix handling of connection timeouts in outgoing http requests (#8400)Richard van der Hoff2020-09-291-29/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove `on_timeout_cancel` from `timeout_deferred` The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a timeout (in particular if the canceller raised an exception), so it was unreliable. It was also only used in one place, and to be honest it's easier to do what it does a different way. * Fix handling of connection timeouts in outgoing http requests Turns out that if we get a timeout during connection, then a different exception is raised, which wasn't always handled correctly. To fix it, catch the exception in SimpleHttpClient and turn it into a RequestTimedOutError (which is already a documented exception). Also add a description to RequestTimedOutError so that we can see which stage it failed at. * Fix incorrect handling of timeouts reading federation responses This was trapping the wrong sort of TimeoutError, so was never being hit. The effect was relatively minor, but we should fix this so that it does the expected thing. * Fix inconsistent handling of `timeout` param between methods `get_json`, `put_json` and `delete_json` were applying a different timeout to the response body to `post_json`; bring them in line and test. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
* Add types to async_helpers (#8260)Patrick Cloke2020-09-081-50/+85
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-4/+4
|
* Convert ReadWriteLock to async/await. (#8202)Patrick Cloke2020-08-281-8/+8
|
* Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-091-1/+1
|
* Fix "argument of type 'ObservableDeferred' is not iterable" error (#7708)Patrick Cloke2020-06-161-1/+1
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+0
|
* Speed up processing of federation stream RDATA rows.Erik Johnston2020-05-271-0/+12
| | | | | | Instead of storing and sending an ACK for every single row we send synchronously, we instead do it asynchronously while batching up updates.
* Fix stacktraces when using ObservableDeferred and async/await (#6836)Erik Johnston2020-02-031-0/+4
|
* Persist auth/state events at backwards extremities when we fetch them (#6526)Richard van der Hoff2019-12-161-2/+2
| | | The main point here is to make sure that the state returned by _get_state_in_room has been authed before we try to use it as state in the room.
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-1/+1
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Clarify docstringErik Johnston2019-10-301-0/+4
|
* Make ObservableDeferred.observe() always return deferred.Erik Johnston2019-10-301-5/+2
| | | | | | | This makes it easier to use in an async/await world. Also fixes a bug where cache descriptors would occaisonally return a raw value rather than a deferred.
* Make concurrently_execute work with async/awaitErik Johnston2019-10-291-4/+3
|
* Add maybe_awaitable and fix __init__ bugsErik Johnston2019-10-111-0/+29
|
* Fix up some typechecking (#6150)Amber Brown2019-10-021-3/+7
| | | | | | * type checking fixes * changelog
* Replace returnValue with return (#5736)Amber Brown2019-07-231-2/+2
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-5/+4
|
* Run Black. (#5482)Amber Brown2019-06-201-17/+24
|
* Update docstring with correct return typeErik Johnston2019-05-151-1/+1
| | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Allow client event serialization to be asyncErik Johnston2019-05-141-0/+19
|
* Remove periods from copyright headers (#5046)Andrew Morgan2019-04-111-1/+1
|
* Make linearizer more quiet (#4507)Amber Brown2019-01-291-5/+5
|
* Fix incorrect logcontexts after a Deferred was cancelled (#4407)Richard van der Hoff2019-01-171-1/+3
|
* Replace custom DeferredTimeoutError with defer.TimeoutErrorErik Johnston2018-09-191-9/+3
|
* Run canceller first to allow it to generate correct errorErik Johnston2018-09-191-2/+5
|
* Update to use new timeout function everywhere.Erik Johnston2018-09-191-54/+19
| | | | | | | The existing deferred timeout helper function (and the one into twisted) suffer from a bug when a deferred's canceller throws an exception, #3842. The new helper function doesn't suffer from this problem.
* Fix timeout functionErik Johnston2018-09-151-1/+2
| | | | | Turns out deferred.cancel sometimes throws, so we do that last to ensure that we always do resolve the new deferred.
* Add an awful secondary timeout to fix wedged requestsErik Johnston2018-09-141-0/+51
| | | | This is an attempt to mitigate #3842 by adding yet-another-timeout
* Merge branch 'rav/fix_linearizer_cancellation' into developRichard van der Hoff2018-08-101-43/+68
|
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-0/+415