summary refs log tree commit diff
path: root/synapse/util/ratelimitutils.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix `HomeServer`s leaking during `trial` test runs (#15630)Sean Quah2023-05-191-1/+5
| | | | | | | | | | | | | | | | | | | This change fixes two memory leaks during `trial` test runs. Garbage collection is disabled during each test case and a gen-0 GC is run at the end of each test. However, when the gen-0 GC is run, the `TestCase` object usually still holds references to the `HomeServer` used during the test. As a result, the `HomeServer` gets promoted to gen-1 and then never garbage collected. Fix this by periodically running full GCs. Additionally, fix `HomeServer`s leaking after tests that touch inbound federation due to `FederationRateLimiter`s adding themselves to a global set, by turning the set into a `WeakSet`. Resolves #15622. Signed-off-by: Sean Quah <seanq@matrix.org>
* Remove unnecessary reactor reference from `_PerHostRatelimiter` (#14842)Sean Quah2023-01-161-8/+2
| | | | | Fix up #14812 to avoid introducing a reference to the reactor. Signed-off-by: Sean Quah <seanq@matrix.org>
* Fix stack overflow in `_PerHostRatelimiter` due to synchronous requests (#14812)Sean Quah2023-01-131-9/+25
| | | | | | | | | | | | | | | | | | When there are many synchronous requests waiting on a `_PerHostRatelimiter`, each request will be started recursively just after the previous request has completed. Under the right conditions, this leads to stack exhaustion. A common way for requests to become synchronous is when the remote client disconnects early, because the homeserver is overloaded and slow to respond. Avoid stack exhaustion under these conditions by deferring subsequent requests until the next reactor tick. Fixes #14480. Signed-off-by: Sean Quah <seanq@matrix.org>
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-1/+1
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Fix rate limit metrics registering twice and misreporting (#13649)Eric Eastwood2022-08-301-31/+124
| | | | | | | | | | | * Fix rate limit metrics registering twice and misreporting Fix https://github.com/matrix-org/synapse/issues/13641 * Fix lints * Add changelog * Document `metrics_name=None`.
* Fix incorrect juggling of logging contexts in `_PerHostRatelimiter` (#13554)Sean Quah2022-08-181-10/+7
| | | | | | Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Track number of hosts affected by the rate limiter (#13541)Eric Eastwood2022-08-181-4/+39
| | | | | | | Track number of hosts affected by the rate limiter so we can differentiate one really noisy homeserver from a general ratelimit tuning problem across the federation. Follow-up to https://github.com/matrix-org/synapse/pull/13534 Part of https://github.com/matrix-org/synapse/issues/13356
* Add metrics to track how the rate limiter is affecting requests ↵Eric Eastwood2022-08-171-8/+29
| | | | | | | (sleep/reject) (#13534) Related to https://github.com/matrix-org/synapse/pull/13499 Part of https://github.com/matrix-org/synapse/issues/13356
* Add metrics to track rate limiter queue timing (#13544)Eric Eastwood2022-08-171-0/+30
|
* Instrument `FederationStateIdsServlet` - `/state_ids` (#13499)Eric Eastwood2022-08-151-0/+4
| | | Instrument FederationStateIdsServlet - `/state_ids` so it's easier to follow what's going on in Jaeger when viewing a trace.
* Rename `RateLimitConfig` to `RatelimitSettings` (#13442)Dirk Klimpel2022-08-031-3/+3
|
* Add types to synapse.util. (#10601)reivilibre2021-09-101-26/+31
|
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-2/+2
|
* Performance improvements and refactor of Ratelimiter (#7595)Andrew Morgan2020-06-051-1/+1
| | | | | | | | | | While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both: * Rather undocumented, and * causing a *lot* of config checks This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. Best to be reviewed commit-by-commit.
* Fixes to the federation rate limiter (#5621)Richard van der Hoff2019-07-051-8/+8
| | | | | | | - Put the default window_size back to 1000ms (broken by #5181) - Make the `rc_federation` config actually do something - fix an off-by-one error in the 'concurrent' limit - Avoid creating an unused `_PerHostRatelimiter` object for every single incoming request
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-201-26/+9
|
* Make all the rate limiting options more consistent (#5181)Amber Brown2019-05-151-32/+15
|
* Make FederationRateLimiter queue requests properlyRichard van der Hoff2018-07-131-10/+23
| | | | | | | | popitem removes the *most recent* item by default [1]. We want the oldest. Fixes #3524 [1]: https://docs.python.org/2/library/collections.html#collections.OrderedDict.popitem
* run isortAmber Brown2018-07-091-7/+6
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-2/+1
|
* Fix logcontext leaks in rate limiterRichard van der Hoff2018-05-031-3/+14
|
* Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-2/+2
| | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* Remove some unused functions (#711)Mark Haines2016-04-081-14/+0
| | | | | | | | * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want
* Fix up logcontextsErik Johnston2016-02-081-1/+2
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-1/+1
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Use contextlib.contextmanager instead of a custom classErik Johnston2015-03-021-22/+12
|
* Move federation rate limiting out of transport layerErik Johnston2015-02-271-0/+226