summary refs log tree commit diff
path: root/synapse/logging (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Various clean-ups to the logging context code (#8935)Patrick Cloke2020-12-141-19/+5
|
* Gracefully handle a pending logging connection during shutdown. (#8685)Patrick Cloke2020-10-291-9/+18
|
* Support generating structured logs in addition to standard logs. (#8607)Patrick Cloke2020-10-295-453/+218
| | | | | | | This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.
* Start fewer opentracing spans (#8640)Erik Johnston2020-10-261-5/+5
| | | | | | | #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.
* Separate the TCP and terse JSON formatting code. (#8587)Patrick Cloke2020-10-212-186/+238
| | | | This should (theoretically) allow for using the TCP code with a different output type and make it easier to use the JSON code with files / console.
* Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-9/+1
|
* Create a mechanism for marking tests "logcontext clean" (#8399)Richard van der Hoff2020-09-281-20/+23
|
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-182-4/+4
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Use slots in attrs classes where possible (#8296)Patrick Cloke2020-09-141-3/+1
| | | | | slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.
* Fix typos in comments.Patrick Cloke2020-09-141-1/+1
|
* Use the right constructor for log records (#8278)Richard van der Hoff2020-09-081-3/+3
| | | | | | Update `log_function` to use the right factory to create log records, to make sure that they have `request` attributes. Fixes: #8267.
* Stop sub-classing object (#8249)Patrick Cloke2020-09-044-9/+9
|
* Use the JSON encoder without whitespace in more places. (#8124)Patrick Cloke2020-08-201-3/+2
|
* Be stricter about JSON that is accepted by Synapse (#8106)Patrick Cloke2020-08-191-2/+5
|
* Fix error reporting when using `opentracing.trace` (#7961)Erik Johnston2020-07-272-12/+3
|
* Clean up PreserveLoggingContext (#7877)Richard van der Hoff2020-07-221-16/+13
| | | This had some dead code and some just plain wrong docstrings.
* Remove unused code from synapse.logging.utils. (#7897)Karthikeyan Singaravelan2020-07-201-126/+0
|
* Fix the trace function for async functions. (#7872)Patrick Cloke2020-07-171-23/+40
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* isort 5 compatibility (#7786)Will Hunt2020-07-051-2/+2
| | | The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
* Merge different Resource implementation classes (#7732)Erik Johnston2020-07-031-37/+31
|
* Add support for using rust-python-jaeger-reporter (#7697)Erik Johnston2020-06-171-3/+36
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+1
|
* remove miscellaneous PY2 codeRichard van der Hoff2020-05-151-8/+2
|
* Improve per-block CPU and DB usage metrics (#7426)Patrick Cloke2020-05-061-12/+26
|
* Add backwards compatibility codepath to LoggingContext. (#7408)Patrick Cloke2020-05-051-0/+41
|
* Add instance name to RDATA/POSITION commands (#7364)Erik Johnston2020-04-291-13/+10
| | | | | This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
* Reduce the number of calls to `resource.getrusage` (#7183)Richard van der Hoff2020-04-031-39/+63
| | | Let's just call `getrusage` once on each logcontext change, rather than twice.
* Fix "'NoneType' has no attribute start|stop" logcontext errors (#7181)Richard van der Hoff2020-03-311-0/+5
| | | | Fixes #7179.
* Clean up some LoggingContext stuff (#7120)Richard van der Hoff2020-03-243-126/+125
| | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.
* Hopefully mypy is happy nowBrendan Abolivier2020-03-101-1/+2
|
* Attempt at appeasing the gods of mypyBrendan Abolivier2020-03-101-1/+1
|
* Add type hints to logging/context.py (#6309)Neil Pilgrim2020-03-071-47/+74
| | | | | | * Add type hints to logging/context.py Signed-off-by: neiljp (Neil Pilgrim) <github@kepier.clara.net>
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-1/+1
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* comment for run_in_backgroundRichard van der Hoff2020-01-121-0/+3
|
* Bump version of mypyErik Johnston2019-12-122-1/+4
|
* Update commentErik Johnston2019-12-101-2/+4
|
* Fix make_deferred_yieldable to work with coroutinesErik Johnston2019-12-101-1/+8
|
* Add benchmarks for structured logging performance (#6266)Amber Brown2019-12-031-0/+1
|
* Improve the performance of structured logging (#6322)Amber Brown2019-11-262-30/+90
|
* A couple more instancesAndrew Morgan2019-11-121-1/+1
|
* Update black to 19.10b0 (#6304)Amber Brown2019-11-011-1/+1
| | | * update version of black and also fix the mypy config being overridden
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-1/+1
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Trace non-JSON APIs, /media, /key etcErik Johnston2019-10-111-4/+10
|
* Fix up some typechecking (#6150)Amber Brown2019-10-022-8/+21
| | | | | | * type checking fixes * changelog
* Fix logcontext spam on non-Linux platforms (#6059)J. Ryan Stinnett2019-09-181-2/+11
| | | | | | | This checks whether the current platform supports thread resource usage tracking before logging a warning to avoid log spam. Fixes https://github.com/matrix-org/synapse/issues/6055
* Fix for structured logging tests stomping on logs (#6023)Amber Brown2019-09-133-9/+11
|
* Don't assume there is a 'self' arg in @trace decoratorErik Johnston2019-09-061-5/+5
|
* Gracefully handle log context slips and missing opentracing import errors. ↵Jorik Schellekens2019-09-051-24/+58
| | | | (#5988)
* Add opentracing to all client servlets (#5983)Jorik Schellekens2019-09-051-1/+1
|
* Fix opentracing contexts missing from outbound replication requests (#5982)Jorik Schellekens2019-09-051-4/+16
|
* Opentrace device lists (#5853)Jorik Schellekens2019-09-031-53/+17
| | | Trace device list changes.
* Implement a structured logging output system. (#5680)Amber Brown2019-08-283-1/+665
|
* Propagate opentracing contexts through EDUs (#5852)Jorik Schellekens2019-08-221-0/+26
| | | | | Propagate opentracing contexts through EDUs Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Opentracing across workers (#5771)Jorik Schellekens2019-08-221-65/+79
| | | | | | | | | | | | | | Propagate opentracing contexts across workers Also includes some Convenience modifications to opentracing for servlets, notably: - Add boolean to skip the whitelisting check on inject extract methods. - useful when injecting into carriers locally. Otherwise we'd always have to include our own servername and whitelist our servername - start_active_span_from_request instead of header - Add boolean to decide whether to extract context from a request to a servlet
* Opentracing doc update (#5776)Jorik Schellekens2019-08-201-27/+40
| | | Update opentracing docs to use the unified 'trace' method
* Make Jaeger fully configurable (#5694)Jorik Schellekens2019-07-231-4/+7
| | | | | | * Allow Jaeger to be configured * Update sample config
* Replace returnValue with return (#5736)Amber Brown2019-07-231-3/+3
|
* Opentracing Utils (#5722)Jorik Schellekens2019-07-232-101/+356
| | | | | | | | | | | | | | | | | | | | | | | | | | * Add decerators for tracing functions * Use the new clean contexts * Context and edu utils * Move opentracing setters * Move whitelisting * Sectioning comments * Better args wrapper * Docstrings Co-Authored-By: Erik Johnston <erik@matrix.org> * Remove unused methods. * Don't use global * One tracing decorator to rule them all.
* Merge branch 'release-v1.2.0' into developJorik Schellekens2019-07-221-0/+125
|\
| * Opentracing Documentation (#5703)Jorik Schellekens2019-07-221-0/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Opentracing survival guide * Update decorator names in doc * Doc cleanup These are all alterations as a result of comments in #5703, it includes mostly typos and clarifications. The most interesting changes are: - Split developer and user docs into two sections - Add a high level description of OpenTracing * newsfile * Move contributer specific info to docstring. * Sample config. * Trailing whitespace. * Update 5703.misc * Apply suggestions from code review Mostly just rewording parts of the docs for clarity. Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Don't accept opentracing data from clients. (#5715)Jorik Schellekens2019-07-191-2/+1
|/ | | | | | * Don't accept opentracing data from clients. * newsfile
* Clean up opentracing configuration options (#5712)Richard van der Hoff2019-07-182-26/+20
| | | | | | | | | | | | | | | | | Clean up config settings and dead code. This is mostly about cleaning up the config format, to bring it into line with our conventions. In particular: * There should be a blank line after `## Section ##' headings * There should be a blank line between each config setting * There should be a `#`-only line between a comment and the setting it describes * We don't really do the `# #` style commenting-out of whole sections if we can help it * rename `tracer_enabled` to `enabled` While we're here, do more config parsing upfront, which makes it easier to use later on. Also removes redundant code from LogContextScopeManager. Also changes the changelog fragment to a `feature` - it's exciting!
* Add basic opentracing support (#5544)Jorik Schellekens2019-07-113-2/+508
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-044-0/+940