summary refs log tree commit diff
path: root/synapse/metrics/background_process_metrics.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Only use literal strings for process names (#16315)Erik Johnston2023-09-151-2/+5
|
* Remove a reference cycle in background process (#16314)Erik Johnston2023-09-131-1/+20
|
* Avoid incrementing bg process utime/stime counters by negative durations ↵David Robertson2022-10-311-2/+4
| | | | (#14323)
* Add missing types to opentracing. (#13345)Patrick Cloke2022-07-211-1/+1
| | | After this change `synapse.logging` is fully typed.
* Easy type hints in synapse.logging.opentracing (#12894)David Robertson2022-05-271-6/+3
|
* Update mypy to 0.950 and fix complaints (#12650)David Robertson2022-05-061-10/+34
|
* Update type annotations for compatiblity with prometheus_client 0.14 (#12389)Richard van der Hoff2022-04-061-1/+2
| | | | | | | Principally, `prometheus_client.REGISTRY.register` now requires its argument to extend `prometheus_client.Collector`. Additionally, `Gauge.set` is now annotated so that passing `Optional[int]` causes an error.
* Add type annotations to `synapse.metrics` (#10847)Sean Quah2021-11-171-19/+59
|
* Relax `ignore-missing-imports` for modules that have stubs now and update ↵David Robertson2021-10-081-1/+1
| | | | | | | | | | | | mypy (#11006) Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details. We therefore pull in stub packages in setup.py Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them. The rest of this change consists of fixups to make the newer mypy + stubs pass CI. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* [pyupgrade] `synapse/` (#10348)Jonathan de Jong2021-07-191-2/+1
| | | | | | | | | This PR is tantamount to running ``` pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"` ``` Part of #9744
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-2/+2
|
* opentracing: use a consistent name for background processes (#10135)Richard van der Hoff2021-06-071-2/+3
| | | | ... otherwise we tend to get a namespace clash between the bg process and the functions that it calls.
* Set opentracing priority before setting other tags (#10092)Richard van der Hoff2021-05-281-2/+8
| | | ... because tags on spans which aren't being sampled get thrown away.
* Merge branch 'master' into developAndrew Morgan2021-04-211-4/+16
|\
| * Stop BackgroundProcessLoggingContext making new prometheus timeseries (#9854)Richard van der Hoff2021-04-211-4/+16
| | | | | | | | This undoes part of b076bc276e881b262048307b6a226061d96c4a8d.
* | Merge branch 'master' into developAndrew Morgan2021-04-201-11/+4
|\|
| * Always use the name as the log ID. (#9829)Patrick Cloke2021-04-201-11/+4
| | | | | | | | | | As far as I can tell our logging contexts are meant to log the request ID, or sometimes the request ID followed by a suffix (this is generally stored in the name field of LoggingContext). There's also code to log the name@memory location, but I'm not sure this is ever used. This simplifies the code paths to require every logging context to have a name and use that in logging. For sub-contexts (created via nested_logging_contexts, defer_to_threadpool, Measure) we use the current context's str (which becomes their name or the string "sentinel") and then potentially modify that (e.g. add a suffix).
* | Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
|/ | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Record more information into structured logs. (#9654)Patrick Cloke2021-04-081-6/+12
| | | | Records additional request information into the structured logs, e.g. the requester, IP address, etc.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-5/+4
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Various clean-ups to the logging context code (#8935)Patrick Cloke2020-12-141-4/+3
|
* Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-111-7/+2
| | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
* Start fewer opentracing spans (#8640)Erik Johnston2020-10-261-3/+9
| | | | | | | #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.
* Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-231-1/+1
|
* Start an opentracing span for background processes. (#8567)Erik Johnston2020-10-191-5/+6
| | | | | | This should reduce the number of `There was no active span` errors we see. Fixes #8510.
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-2/+2
|
* Convert runWithConnection to async. (#8121)Patrick Cloke2020-08-191-1/+1
|
* Convert run_as_background_process inner function to async. (#8032)Patrick Cloke2020-08-061-22/+12
|
* Improve stacktraces from exceptions in background processes (#7808)Richard van der Hoff2020-07-091-1/+9
| | | use `Failure()` to fish out the real exception.
* Make inflight background metrics more efficient. (#7597)Erik Johnston2020-05-291-34/+70
| | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Add prometheus metrics for the number of active pushers (#7103)Richard van der Hoff2020-03-191-2/+3
|
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-2/+2
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Update comments and docstringRichard van der Hoff2019-09-251-4/+6
|
* Add wrap_as_background_process decorator.Erik Johnston2019-09-241-1/+28
| | | | | This does the same thing as `run_as_background_process` but means we don't need to create superfluous functions.
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-201-17/+16
|
* Fix exception in background metrics collectionErik Johnston2018-10-031-2/+6
| | | | | We attempted to iterate through a list on a separate thread without doing the necessary copying.
* Add missing loggerErik Johnston2018-09-201-0/+4
|
* Handle exceptions thrown by background tasksErik Johnston2018-09-201-0/+2
| | | | Fixes #3921
* isortErik Johnston2018-08-211-1/+2
|
* Make the in flight background process metrics thread safeErik Johnston2018-08-201-5/+20
|
* Fix some looping_call calls which were broken in #3604Richard van der Hoff2018-07-261-2/+8
| | | | | | | | | It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.
* Resource tracking for background processesRichard van der Hoff2018-07-181-0/+179
This introduces a mechanism for tracking resource usage by background processes, along with an example of how it will be used. This will help address #3518, but more importantly will give us better insights into things which are happening but not being shown up by the request metrics. We *could* do this with Measure blocks, but: - I think having them pulled out as a completely separate metric class will make it easier to distinguish top-level processes from those which are nested. - I want to be able to report on in-flight background processes, and I don't think we want to do this for *all* Measure blocks.