summary refs log tree commit diff
path: root/synapse/http/server.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Fix email verification redirection (#16761)FadhlanR2024-01-021-1/+1
| | | | | | Previously, the response status of `HTMLResource` was hardcoded as `200`. However, for proper redirection after the user verifies their email, we require the status to be `302`. This PR addresses that issue by using `code` as response status.
* Update license headersPatrick Cloke2023-11-211-11/+16
|
* Register media servlets via regex. (#16419)Patrick Cloke2023-10-061-1/+1
| | | | | This converts the media servlet URLs in the same way as (most) of the rest of Synapse. This will give more flexibility in the versions each endpoint exists under.
* Describe which rate limiter was hit in logs (#16135)David Robertson2023-08-301-1/+7
|
* Don't log exceptions for every non-200 response (#15969)Erik Johnston2023-07-201-4/+0
| | | Introduced in #15913
* Re-introduce the outbound federation proxy (#15913)Eric Eastwood2023-07-181-24/+31
| | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
* Add `Server` to Access-Control-Expose-Headers header (#15908)Michael Telatynski2023-07-111-1/+1
|
* Revert "Federation outbound proxy" (#15910)Eric Eastwood2023-07-101-31/+24
| | | | | | Revert "Federation outbound proxy (#15773)" This reverts commit b07b14b494ae1dd564b4c44f844c9a9545b3d08a.
* Federation outbound proxy (#15773)Eric Eastwood2023-07-051-24/+31
| | | | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
* Add stricter mypy options (#15694)Patrick Cloke2023-05-311-7/+7
| | | | Enable warn_unused_configs, strict_concatenate, disallow_subclassing_any, and disallow_incomplete_defs.
* Actually enforce guest + return www-authenticate headerHugh Nimmo-Smith2023-05-301-0/+6
|
* Disable directory listing for `StaticResource` (#15438)Dirk Klimpel2023-04-141-0/+10
|
* Add Synapse-Trace-Id to access-control-expose-headers header (#14974)Shay2023-03-201-0/+4
|
* Respond correctly to unknown methods on known endpoints (#14605)Patrick Cloke2023-02-091-25/+15
| | | | Respond with a 405 error if a request is received on a known endpoint, but to an unknown method, per MSC3743.
* Respond with proper error responses on unknown paths. (#14621)Patrick Cloke2022-12-081-1/+18
| | | | Returns a proper 404 with an errcode of M_RECOGNIZED for unknown endpoints per MSC3743.
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-1/+1
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Implementation of HTTP 307 response for MSC3886 POST endpoint (#14018)Hugh Nimmo-Smith2022-10-181-11/+37
| | | | Co-authored-by: reivilibre <olivier@librepush.net> Co-authored-by: Andrew Morgan <andrewm@element.io>
* Fix mypy errors with latest canonicaljson (#13905)David Robertson2022-09-261-2/+2
| | | | | | | | | | | | | | | | * Lockfile: update canonicaljson 1.6.0 -> 1.6.3 * Fix mypy errors with latest canonicaljson The change to `_encode_json_bytes` definition wasn't sufficient: ``` synapse/http/server.py:751: error: Incompatible types in assignment (expression has type "Callable[[Arg(object, 'json_object')], bytes]", variable has type "Callable[[Arg(object, 'data')], bytes]") [assignment] ``` Which I think is mypy warning us that the two functions accept different sets of kwargs. Fair enough! * Changelog
* Generalise the `@cancellable` annotation so it can be used on functions ↵reivilibre2022-08-311-65/+3
| | | | other than just servlet methods. (#13662)
* Implement MSC3848: Introduce errcodes for specific event sending failures ↵Will Hunt2022-07-271-4/+14
| | | | | (#13343) Implements MSC3848
* Add Cross-Origin-Resource-Policy header to thumbnail and download media ↵Robert Long2022-06-271-0/+11
| | | | endpoints (#12944)
* Enable cancellation of `GET /members` and `GET /state` requests (#12708)Sean Quah2022-05-111-1/+3
| | | | | | | | Enable cancellation of `GET /rooms/$room_id/members`, `GET /rooms/$room_id/state` and `GET /rooms/$room_id/state/$state_key/*` requests. Signed-off-by: Sean Quah <seanq@element.io>
* Respect the `@cancellable` flag for `RestServlet`s and ↵Sean Quah2022-05-111-0/+5
| | | | | | | | | | | | | | | `BaseFederationServlet`s (#12699) Both `RestServlet`s and `BaseFederationServlet`s register their handlers with `HttpServer.register_paths` / `JsonResource.register_paths`. Update `JsonResource` to respect the `@cancellable` flag on handlers registered in this way. Although `ReplicationEndpoint` also registers itself using `register_paths`, it does not pass the handler method that would have the `@cancellable` flag directly, and so needs separate handling. Signed-off-by: Sean Quah <seanq@element.io>
* Respect the `@cancellable` flag for `DirectServe{Html,Json}Resource`s (#12698)Sean Quah2022-05-111-0/+2
| | | | | | | | | | | | `DirectServeHtmlResource` and `DirectServeJsonResource` both inherit from `_AsyncResource`. These classes expect to be subclassed with `_async_render_*` methods. This commit has no effect on `JsonResource`, despite inheriting from `_AsyncResource`. `JsonResource` has its own `_async_render` override which will need to be updated separately. Signed-off-by: Sean Quah <seanq@element.io>
* Capture the `Deferred` for request cancellation in `_AsyncResource` (#12694)Sean Quah2022-05-101-1/+3
| | | | | | | | | | | | | All async request processing goes through `_AsyncResource`, so this is the only place where a `Deferred` needs to be captured for cancellation. Unfortunately, the same isn't true for determining whether a request can be cancelled. Each of `RestServlet`, `BaseFederationServlet`, `DirectServe{Html,Json}Resource` and `ReplicationEndpoint` have different wrappers around the method doing the request handling and they all need to be handled separately. Signed-off-by: Sean Quah <seanq@element.io>
* Add `@cancellable` decorator, for use on request handlers (#12586)Sean Quah2022-05-101-0/+61
| | | | Signed-off-by: Sean Quah <seanq@element.io>
* Improve logging for cancelled requests (#12587)Sean Quah2022-05-041-0/+30
| | | | | | | Don't log stack traces for cancelled requests and use a custom HTTP status code of 499. Signed-off-by: Sean Quah <seanq@element.io>
* Fix logging of incorrect status codes for disconnected requests (#12580)Sean Quah2022-04-281-3/+9
| | | | | | | | | | | | | | The status code of requests must always be set, regardless of client disconnection, otherwise they will always be logged as 200!. Broken for `respond_with_json` in f48792eec43f893f4f893ffdcbf00f8958b6f6b5. Broken for `respond_with_json_bytes` in 3e58ce72b42f2ae473c1e76a967548cd6fa7e2e6. Broken for `respond_with_html_bytes` in ea26e9a98b0541fc886a1cb826a38352b7599dbe. Signed-off-by: Sean Quah <seanq@element.io>
* Remove unused `# type: ignore`s (#12531)David Robertson2022-04-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | Over time we've begun to use newer versions of mypy, typeshed, stub packages---and of course we've improved our own annotations. This makes some type ignore comments no longer necessary. I have removed them. There was one exception: a module that imports `select.epoll`. The ignore is redundant on Linux, but I've kept it ignored for those of us who work on the source tree using not-Linux. (#11771) I'm more interested in the config line which enforces this. I want unused ignores to be reported, because I think it's useful feedback when annotating to know when you've fixed a problem you had to previously ignore. * Installing extras before typechecking Lacking an easy way to install all extras generically, let's bite the bullet and make install the hand-maintained `all` extra before typechecking. Now that https://github.com/matrix-org/backend-meta/pull/6 is merged to the release/v1 branch.
* Remove unnecessary pass statements. (#12206)Patrick Cloke2022-03-111-1/+0
|
* Convert all namedtuples to attrs. (#11665)Patrick Cloke2021-12-301-4/+6
| | | To improve type hints throughout the code.
* Various opentracing enhancements (#11619)Richard van der Hoff2021-12-211-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | * Wrap `auth.get_user_by_req` in an opentracing span give `get_user_by_req` its own opentracing span, since it can result in a non-trivial number of sub-spans which it is useful to group together. This requires a bit of reorganisation because it also sets some tags (and may force tracing) on the servlet span. * Emit opentracing span for encoding json responses This can be a significant time sink. * Rename all sync spans with a prefix * Write an opentracing span for encoding sync response * opentracing span to group generate_room_entries * opentracing spans within sync.encode_response * changelog * Use the `trace` decorator instead of context managers
* Return JSON errors for unknown resources under /matrix/client. (#11602)Patrick Cloke2021-12-201-3/+3
| | | | Instead of returning 404 errors with HTML bodies when an unknown prefix was requested (e.g. /matrix/client/v1 before Synapse v1.49.0).
* Add missing type hints to synapse.http. (#11571)Patrick Cloke2021-12-141-37/+53
|
* Add more type hints to synapse.util. (#11321)Patrick Cloke2021-11-121-3/+3
|
* Add type hints for most `HomeServer` parameters (#11095)Sean Quah2021-10-221-7/+12
|
* Catch AttributeErrors when calling registerProducer (#10995)Brendan Abolivier2021-10-071-1/+4
| | | Looks like the wrong exception type was caught in #10932.
* Fix exception responding to request that has been closed (#10932)Erik Johnston2021-09-281-3/+11
| | | Introduced in #10905
* Encode JSON responses on a thread in C, mk2 (#10905)Erik Johnston2021-09-281-15/+57
| | | | | | | | | | Currently we use `JsonEncoder.iterencode` to write JSON responses, which ensures that we don't block the main reactor thread when encoding huge objects. The downside to this is that `iterencode` falls back to using a pure Python encoder that is *much* less efficient and can easily burn a lot of CPU for huge responses. To fix this, while still ensuring we don't block the reactor loop, we encode the JSON on a threadpool using the standard `JsonEncoder.encode` functions, which is backed by a C library. Doing so, however, requires `respond_with_json` to have access to the reactor, which it previously didn't. There are two ways of doing this: 1. threading through the reactor object, which is a bit fiddly as e.g. `DirectServeJsonResource` doesn't currently take a reactor, but is exposed to modules and so is a PITA to change; or 2. expose the reactor in `SynapseRequest`, which requires updating a bunch of servlet types. I went with the latter as that is just a mechanical change, and I think makes sense as a request already has a reactor associated with it (via its http channel).
* Add reactor to `SynapseRequest` and fix up types. (#10868)Erik Johnston2021-09-241-2/+2
|
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-4/+4
|
* Drop Origin & Accept from Access-Control-Allow-Headers value (#10114)Michael[tm] Smith2021-06-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Drop Origin & Accept from Access-Control-Allow-Headers value This change drops the Origin and Accept header names from the value of the Access-Control-Allow-Headers response header sent by Synapse. Per the CORS protocol, it’s not necessary or useful to include those header names. Details: Per-spec at https://fetch.spec.whatwg.org/#forbidden-header-name, Origin is a “forbidden header name” set by the browser and that frontend JavaScript code is never allowed to set. So the value of Access-Control-Allow-Headers isn’t relevant to Origin or in general to other headers set by the browser itself — the browser never ever consults the Access-Control-Allow-Headers value to confirm that it’s OK for the request to include an Origin header. And per-spec at https://fetch.spec.whatwg.org/#cors-safelisted-request-header, Accept is a “CORS-safelisted request-header”, which means that browsers allow requests to contain the Accept header regardless of whether the Access-Control-Allow-Headers value contains "Accept". So it’s unnecessary for the Access-Control-Allow-Headers to explicitly include Accept. Browsers will not perform a CORS preflight for requests containing an Accept request header. Related: https://github.com/matrix-org/matrix-doc/pull/3225 Signed-off-by: Michael[tm] Smith <mike@w3.org>
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Fix additional type hints from Twisted upgrade. (#9518)Patrick Cloke2021-03-031-11/+18
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-27/+45
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Split out a separate endpoint to complete SSO registration (#9262)Richard van der Hoff2021-02-011-0/+7
| | | There are going to be a couple of paths to get to the final step of SSO reg, and I want the URL in the browser to consistent. So, let's move the final step onto a separate path, which we redirect to.
* Implement MSC2858 support (#9183)Richard van der Hoff2021-01-271-8/+36
| | | Fixes #8928.
* Fix buglet in DirectRenderJsonResource (#8897)Richard van der Hoff2020-12-101-3/+5
| | | | this was using `canonical_json` without setting it, so when you used it as a standalone class, you would get exceptions.
* Allow Date header through CORS (#8804)Nicolas Chamo2020-12-011-1/+1
|
* Remove special case of pretty printing JSON responses for curl (#8833)Tulir Asokan2020-11-271-24/+5
| | | | | * Remove special case of pretty printing JSON responses for curl Signed-off-by: Tulir Asokan <tulir@maunium.net>
* Merge pull request #8678 from matrix-org/rav/fix_frozen_eventsRichard van der Hoff2020-10-281-3/+1
|\ | | | | Fix serialisation errors when using third-party event rules.
| * remove unused importsRichard van der Hoff2020-10-281-2/+0
| |
| * Remove frozendict_json_encoder and support frozendicts everywhereRichard van der Hoff2020-10-281-1/+1
| | | | | | | | | | | | Not being able to serialise `frozendicts` is fragile, and it's annoying to have to think about which serialiser you want. There's no real downside to supporting frozendicts, so let's just have one json encoder.
* | Fix typos and spelling errors. (#8639)Patrick Cloke2020-10-231-3/+3
|/
* Don't bother responding to client requests that have already disconnected ↵Andrew Morgan2020-10-061-0/+5
| | | | | | | | | | (#8465) This PR ports the quick fix from https://github.com/matrix-org/synapse/pull/2796 to further methods which handle media, URL preview and `/key/v2/server` requests. This prevents a harmless `ERROR` that comes up in the logs when we were unable to respond to a client request when the client had already disconnected. In this case we simply bail out if the client has already done so. This is the 'simple fix' as suggested by https://github.com/matrix-org/synapse/issues/5304#issuecomment-574740003. Fixes https://github.com/matrix-org/synapse/issues/6700 Fixes https://github.com/matrix-org/synapse/issues/5304
* Enable mypy checking for unreachable code and fix instances. (#8432)Patrick Cloke2020-10-011-2/+2
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-1/+1
|
* Switch the JSON byte producer from a pull to a push producer. (#8116)Patrick Cloke2020-08-191-32/+43
|
* Iteratively encode JSON responses to avoid blocking the reactor. (#8013)Patrick Cloke2020-08-181-8/+89
|
* Reduce unnecessary whitespace in JSON. (#7372)David Vo2020-08-071-2/+3
|
* Implement handling of HTTP HEAD requests. (#7999)Patrick Cloke2020-08-031-5/+11
|
* Return an empty body for OPTIONS requests. (#7886)Patrick Cloke2020-07-241-19/+5
|
* fix an incorrect commentRichard van der Hoff2020-07-221-2/+2
|
* Add types to the server code and remove unused parameter (#7813)Patrick Cloke2020-07-101-30/+41
|
* Merge different Resource implementation classes (#7732)Erik Johnston2020-07-031-175/+190
|
* Merge branch 'master' into developPatrick Cloke2020-07-021-8/+68
|\
| * Ensure that HTML pages served from Synapse include headers to avoid embedding.Patrick Cloke2020-07-021-8/+68
| |
* | Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+2
|/
* Clean up exception handling in SAML2ResponseResource (#7614)Richard van der Hoff2020-06-031-12/+31
| | | | | | | | | | | | | * Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string * Clean up exception handling in SAML2ResponseResource * use the existing code in `return_html_error` instead of re-implementing it (giving it a jinja2 template rather than inventing a new form of template) * do the exception-catching in the REST layer rather than in the handler layer, to make sure we catch all exceptions.
* Fix missing CORS headers on OPTION responses (#7560)Erik Johnston2020-05-221-1/+1
| | | Broke in #7534.
* Return 200 OK for all OPTIONS requests (#7534)Patrick Cloke2020-05-221-3/+20
|
* Tiny optimisation for _get_handler_for_request (#6950)Richard van der Hoff2020-02-191-1/+3
| | | | we have hundreds of path_regexes (see #5118), so let's not convert the same bytes to str for each of them.
* Implement RedirectException (#6687)Richard van der Hoff2020-01-151-4/+9
| | | | | Allow REST endpoint implemnentations to raise a RedirectException, which will redirect the user's browser to a given location.
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-1/+1
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Trace non-JSON APIs, /media, /key etcErik Johnston2019-10-111-1/+1
|
* Add opentracing to all client servlets (#5983)Jorik Schellekens2019-09-051-1/+12
|
* Fix servlet metric names (#5734)Jorik Schellekens2019-07-241-13/+28
| | | | | | | | | | * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Fix media repo breaking (#5593)Amber Brown2019-07-021-10/+16
|
* Make the http server handle coroutine-making REST servlets (#5475)Amber Brown2019-06-291-36/+41
|
* Run Black. (#5482)Amber Brown2019-06-201-36/+44
|
* Log tracebacks correctlyErik Johnston2019-02-251-4/+4
|
* Cleanup top level request exception loggingErik Johnston2019-02-181-3/+3
| | | | | | | | | | | Firstly, we always logged that the request was being handled via `JsonResource._async_render`, so we change that to use the servlet name we add to the request. Secondly, we pass the exception information to the logger rather than formatting it manually. This makes it consistent with other exception logging, allwoing logging hooks and formatters to access the exception information.
* Fix URL preview bugs (type error when loading cache from db, content-type ↵Amber Brown2018-11-081-4/+4
| | | | including quotes) (#4157)
* Fix spurious exceptions when client closes conncetionErik Johnston2018-09-201-14/+35
| | | | | | | | | If a HTTP handler throws an exception while processing a request we automatically write a JSON error response. If the handler had already started writing a response twisted throws an exception. We should check for this case and simple abort the connection if there was an error after the response had started being written.
* Merge pull request #3701 from matrix-org/rav/use_producer_for_responsesRichard van der Hoff2018-08-171-4/+13
|\ | | | | Use a producer to stream back responses
| * Use a producer to stream back responsesRichard van der Hoff2018-08-151-4/+13
| | | | | | | | | | | | | | | | | | | | The problem with dumping all of the json response into the Request object at once is that doing so starts the timeout for the next request to be received: so if it takes longer than 60s to stream back the response to the client, the client never gets it. The correct solution is to use a Producer; then the timeout is only started once all of the content is sent over the TCP connection.
* | Refactor request logging codeRichard van der Hoff2018-08-151-47/+23
|/ | | | | | | | | | | | | | | This commit moves a bunch of the logic for deciding when to log the receipt and completion of HTTP requests into SynapseRequest, rather than in the request handling wrappers. Advantages of this are: * we get logs for *all* requests (including OPTIONS and HEADs), rather than just those that end up hitting handlers we've remembered to decorate correctly. * when a request handler wires up a Producer (as the media stuff does currently, and as other things will do soon), we log at the point that all of the traffic has been sent to the client.
* Be more careful which errors we send back over the C-S APIRichard van der Hoff2018-08-011-9/+5
| | | | | | | | | | | | We really shouldn't be sending all CodeMessageExceptions back over the C-S API; it will include things like 401s which we shouldn't proxy. That means that we need to explicitly turn a few HttpResponseExceptions into SynapseErrors in the federation layer. The effect of the latter is that the matrix errcode will get passed through correctly to calling clients, which might help with some of the random M_UNKNOWN errors when trying to join rooms.
* Python 3: Convert some unicode/bytes uses (#3569)Amber Brown2018-08-021-10/+25
|
* run isortAmber Brown2018-07-091-19/+19
|
* Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-3/+2
|
* Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-221-2/+74
|\
| * Merge pull request #3213 from matrix-org/rav/consent_handlerRichard van der Hoff2018-05-161-2/+74
| |\ | | | | | | ConsentResource to gather policy consent from users
| | * ConsentResource to gather policy consent from usersRichard van der Hoff2018-05-151-2/+74
| | | | | | | | | | | | | | | Hopefully there are enough comments and docs in this that it makes sense on its own.
* | | rest of the changesAmber Brown2018-05-211-2/+2
|/ /
* / Set Server header in SynapseRequestRichard van der Hoff2018-05-101-10/+4
|/ | | | | | | | | | | | (instead of everywhere that writes a response. Or rather, the subset of places which write responses where we haven't forgotten it). This also means that we don't have to have the mysterious version_string attribute in anything with a request handler. Unfortunately it does mean that we have to pass the version string wherever we instantiate a SynapseSite, which has been c&ped 150 times, but that is code that ought to be cleaned up anyway really.
* Remove redundant request_handler decoratorRichard van der Hoff2018-05-101-7/+2
| | | | | | This is needless complexity; we might as well use the wrapper directly. Also rename wrap_request_handler->wrap_json_request_handler.
* Factor wrap_request_handler_with_logging out of wrap_request_handlerRichard van der Hoff2018-05-101-54/+66
| | | | ... so that it can be used on non-JSON endpoints
* Remove include_metrics paramRichard van der Hoff2018-05-101-17/+7
| | | | | The metrics are now available via the request, so this is redundant and can go away at last.
* Move outgoing_responses_counter handling to RequestMetricsRichard van der Hoff2018-05-101-4/+0
| | | | it's much neater there.
* Bump requests_counter in wrapped_request_handlerRichard van der Hoff2018-05-101-4/+11
| | | | less magic
* Move RequestMetrics handling into SynapseRequest.processing()Richard van der Hoff2018-05-101-13/+6
| | | | | It fits quite nicely here, and opens the path to getting rid of the "include_metrics" mess.
* Make RequestMetrics take a raw time rather than a clockRichard van der Hoff2018-05-101-2/+2
| | | | ... which is going to make it easier to move around.
* Move request_id management into SynapseRequestRichard van der Hoff2018-05-101-15/+16
|
* Move RequestsMetrics to its own fileRichard van der Hoff2018-05-091-124/+4
| | | | | | This is useful in its own right, because server.py is full of stuff; but more importantly, I want to do some refactoring that will cause a circular reference as it is.
* more bytes stringsAdrian Tschira2018-04-291-1/+1
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Merge pull request #3061 from NotAFile/add-some-byte-stringsRichard van der Hoff2018-04-091-2/+2
|\ | | | | Add b prefixes to some strings that are bytes in py3
| * Add b prefixes to some strings that are bytes in py3Adrian Tschira2018-04-041-2/+2
| | | | | | | | | | | | This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics"Richard van der Hoff2018-04-091-0/+26
| | | | | | | | | | | | | | We aren't ready to release this yet, so I'm reverting it for now. This reverts commit d1679a4ed7947b0814e0f2af9b888a16c588f1a1, reversing changes made to e089100c6231541c446e37e157dec8feed02d283.
* | Add response size metricsErik Johnston2018-04-061-0/+7
| |
* | Remove redundant metrics which were deprecated in 0.27.0.Richard van der Hoff2018-04-041-26/+0
|/
* Add Cache-Control headers to all JSON APIsErik Johnston2018-03-211-0/+1
| | | | | | | | | | | | It is especially important that sync requests don't get cached, as if a sync returns the same token given then the client will call sync with the same parameters again. If the previous response was cached it will get reused, resulting in the client tight looping making the same request and never making any progress. In general, clients will expect to get up to date data when requesting APIs, and so its safer to do a blanket no cache policy than only whitelisting APIs that we know will break things if they get cached.
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-191-2/+0
|\
| * Replace ujson with simplejsonErik Johnston2018-03-151-3/+2
| |
* | Replace some ujson with simplejson to make it workErik Johnston2018-03-161-1/+2
| |
* | Add some docstrings to help figure this outRichard van der Hoff2018-03-091-2/+26
| |
* | Add a metric which increments when a request is receivedRichard van der Hoff2018-03-091-2/+10
| | | | | | | | | | | | It's useful to know when there are peaks in incoming requests - which isn't quite the same as there being peaks in outgoing responses, due to the time taken to handle requests.
* | refactor JsonResourceRichard van der Hoff2018-03-091-32/+46
| | | | | | | | | | rephrase the OPTIONS and unrecognised request handling so that they look similar to the common flow.
* | Merge remote-tracking branch 'origin/develop' into rav/track_db_schedulingRichard van der Hoff2018-01-171-9/+9
|\ \
| * \ Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millisRichard van der Hoff2018-01-171-1/+3
| |\ \ | | | | | | | | Track db txn time in millisecs
| * | | Fix 'NoneType' object has no attribute 'writeHeaders'Richard van der Hoff2018-01-161-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid throwing a (harmless) exception when we try to write an error response to an http request where the client has disconnected. This comes up as a CRITICAL error in the logs which tends to mislead people into thinking there's an actual problem
* | | | Track DB scheduling delay per-requestRichard van der Hoff2018-01-161-0/+7
| |/ / |/| | | | | | | | | | | | | | For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing.
* | | Track db txn time in millisecsRichard van der Hoff2018-01-161-1/+3
|/ / | | | | | | ... to reduce the amount of floating-point foo we do.
* | Reorganise request and block metricsRichard van der Hoff2018-01-151-14/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to circumvent the number of duplicate foo:count metrics increasing without bounds, it's time for a rearrangement. The following are all deprecated, and replaced with synapse_util_metrics_block_count: synapse_util_metrics_block_timer:count synapse_util_metrics_block_ru_utime:count synapse_util_metrics_block_ru_stime:count synapse_util_metrics_block_db_txn_count:count synapse_util_metrics_block_db_txn_duration:count The following are all deprecated, and replaced with synapse_http_server_response_count: synapse_http_server_requests synapse_http_server_response_time:count synapse_http_server_response_ru_utime:count synapse_http_server_response_ru_stime:count synapse_http_server_response_db_txn_count:count synapse_http_server_response_db_txn_duration:count The following are renamed (the old metrics are kept for now, but deprecated): synapse_util_metrics_block_timer:total -> synapse_util_metrics_block_time_seconds synapse_util_metrics_block_ru_utime:total -> synapse_util_metrics_block_ru_utime_seconds synapse_util_metrics_block_ru_stime:total -> synapse_util_metrics_block_ru_stime_seconds synapse_util_metrics_block_db_txn_count:total -> synapse_util_metrics_block_db_txn_count synapse_util_metrics_block_db_txn_duration:total -> synapse_util_metrics_block_db_txn_duration_seconds synapse_http_server_response_time:total -> synapse_http_server_response_time_seconds synapse_http_server_response_ru_utime:total -> synapse_http_server_response_ru_utime_seconds synapse_http_server_response_ru_stime:total -> synapse_http_server_response_ru_stime_seconds synapse_http_server_response_db_txn_count:total -> synapse_http_server_response_db_txn_count synapse_http_server_response_db_txn_duration:total synapse_http_server_response_db_txn_duration_seconds
* | Update http request metrics before calling servletRichard van der Hoff2018-01-091-11/+19
|/ | | | | Make sure that we set the servlet name in the metrics object *before* calling the servlet, in case the servlet throws an exception.
* Improve tracebacks on exceptionsRichard van der Hoff2017-11-271-3/+9
| | | | | Use failure.Failure to recover our failure, which will give us a useful stacktrace, unlike the rethrown exception.
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Initial group server implementationErik Johnston2017-07-101-1/+3
|
* allow Authorization header which handling got implemented in #1098Krombel2017-06-161-1/+1
| | | | Signed-off-by: Matthias Kesler <krombel@krombel.de>
* Set CORs headers on responses from the media repoMark Haines2016-11-021-5/+18
|
* Add exception logging. Fix typoErik Johnston2016-08-221-3/+3
|
* Allow request handlers to override metric nameErik Johnston2016-08-221-21/+31
|
* Use top level measureErik Johnston2016-08-221-15/+1
|
* Add a top level measureErik Johnston2016-08-191-45/+46
|
* Remove redundant measureErik Johnston2016-08-191-35/+34
|
* Measure http.server renderErik Johnston2016-08-191-28/+30
|
* Log when adding listenersRichard van der Hoff2016-07-251-0/+1
|
* Fix more typos in per-request metricsMark Haines2016-04-281-4/+5
|
* Fix typo in request metricsMark Haines2016-04-281-1/+1
|
* Add a comment explaining why automatic metric reporting is disabled for ↵Mark Haines2016-04-281-0/+4
| | | | JsonResource
* Check if report_metrics is TrueMark Haines2016-04-281-3/+4
|
* Report per request metrics for all of the things using request_handlerMark Haines2016-04-281-38/+63
|
* Intern all the thingsErik Johnston2016-03-231-4/+6
|
* Catch the exceptions thrown by twisted when you write to a closed connectionMark Haines2016-02-121-1/+20
|
* Fix up logcontextsErik Johnston2016-02-081-3/+2
|
* Add metrics to pushersErik Johnston2016-02-081-0/+10
|
* Allowing tagging log contextsErik Johnston2016-02-031-15/+26
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Fix typoDaniel Wagner-Hall2015-12-161-1/+1
|
* Include errcode on Internal Server ErrorDaniel Wagner-Hall2015-12-141-2/+5
|
* Track the time spent in the database per request.Mark Haines2015-12-071-0/+15
| | | | and track the number of transactions that request started.
* Add metrics to track the cpu on the main thread consumed by each type of requestMark Haines2015-12-041-0/+17
|
* Host /unstable and /r0 versions of r0 APIsDaniel Wagner-Hall2015-12-011-6/+7
|
* Remove syutil dependency in favour of smaller single-purpose librariesMark Haines2015-08-241-6/+6
|
* Don't 500 if a group is missing from the regexMark Haines2015-07-061-1/+1
|
* Disable twisted access logging. Move access logging to SynapseRequest objectErik Johnston2015-06-151-58/+32
|
* Log more when we have processed the requestErik Johnston2015-06-151-2/+12
|
* Create SynapseRequest that overrides __repr__ to not print access_tokenErik Johnston2015-06-151-11/+3
|
* Make http.server request logging more verbose, but redact access_tokensErik Johnston2015-06-151-2/+12
|
* Remove log lineErik Johnston2015-05-291-1/+0
|
* Add config option to turn off freezing events. Use new encode_json api and ↵Erik Johnston2015-05-291-4/+13
| | | | ujson.loads
* Fix up leak. Add warnings.Erik Johnston2015-05-081-2/+4
|
* Unused importDavid Baker2015-04-301-1/+1
|
* Doesn't look like this is used anymoreDavid Baker2015-04-301-13/+0
|
* Merge branch 'develop' into csauthDavid Baker2015-04-241-97/+124
|\ | | | | | | | | Conflicts: synapse/http/server.py
| * Combine the request wrappers in rest/media/v1 and http/server into a single ↵Mark Haines2015-04-211-3/+12
| | | | | | | | wrapper decorator
| * Split out the JsonResource request logging and error handling into a ↵Mark Haines2015-04-211-94/+113
| | | | | | | | separate wrapper function
* | Explain how I justified to myself making JsonResource not always send JSON.David Baker2015-04-011-0/+7
| |
* | Completely replace fallback auth for C/S V2:David Baker2015-04-011-2/+5
| | | | | | | | | | | | | | * Now only the auth part goes to fallback, not the whole operation * Auth fallback is a normal API endpoint, not a static page * Params like the recaptcha pubkey can just live in the config Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
* | Make docs a bit more trueDavid Baker2015-03-311-1/+1
| |
* | Grammar and deduplicationDavid Baker2015-03-311-5/+4
|/
* Add a DistributionMetric to HTTP request/response processing time in the serverPaul "LeoNerd" Evans2015-03-161-0/+9
|
* actually uphold the bind_host parameter. in theory should make ipv6 binds ↵Matthew Hodgson2015-03-141-1/+6
| | | | work like bind_host: 'fe80::1%lo0'
* Appease pep8Paul "LeoNerd" Evans2015-03-121-3/+5
|
* Count incoming HTTP requests per servlet that respondsPaul "LeoNerd" Evans2015-03-121-7/+11
|
* Use _ instead of . as a metric namespacing separator, for PrometheusPaul "LeoNerd" Evans2015-03-121-2/+2
|
* Rename Metrics' "keys" to "labels"Paul "LeoNerd" Evans2015-03-121-2/+2
|
* Add vector counters to HTTP clients and servers; count the requests by ↵Paul "LeoNerd" Evans2015-03-121-0/+14
| | | | method and responses by method and response code
* Slightly reduce the insane amounts of indentation in main http server ↵Paul "LeoNerd" Evans2015-03-051-21/+23
| | | | response path, by 'continue'ing around a non-match or falling through
* Use git aware version string in User-Agent and Server headersErik Johnston2015-02-181-9/+18
|
* Fix bug where variable was not always definedErik Johnston2015-02-111-2/+1
|
* Log when we receive a request, when we send a response and how long it took ↵Erik Johnston2015-02-091-2/+21
| | | | to process it.
* Oops: second part of commit dc938606David Baker2015-01-221-6/+2
|
* Update copyright noticesMark Haines2015-01-061-2/+2
|
* Set a content-length for JSON responsesMark Haines2015-01-061-0/+1
|
* Include version in User-Agent and Server headersMark Haines2014-12-221-3/+6
|
* Fix tests and remove debug loggingErik Johnston2014-12-121-4/+0
|
* Consistently url decode and decode as utf 8 the URL partsErik Johnston2014-12-121-1/+11
|
* Get uploads working with new media repoMark Haines2014-12-021-2/+2
|
* Write the upload portion of version 1 of the media repositoryMark Haines2014-12-021-7/+14
|
* Fix pep8 codestyle warningsMark Haines2014-11-201-2/+1
|
* Use module loggers rather than the root logger. Exceptions caused by bad ↵Mark Haines2014-11-201-1/+1
| | | | clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
* Add a request-id to each log lineMark Haines2014-10-301-1/+12
|
* fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-031-1/+1
| | | | hasn't been incorporated in time for launch.
* move contentrepo class to it's own fileMark Haines2014-09-031-162/+1
|
* Set status message for ratelimit error responsesMark Haines2014-09-031-5/+9
|
* given everything's apparently gone https:// by default for c-s, change this ↵Matthew Hodgson2014-09-011-1/+5
| | | | hardcoding...
* missed a s#/matrix#/_matrix/gMatthew Hodgson2014-08-311-1/+1
|
* For the content repo, don't just use homeserver.hostname as that might not ↵Erik Johnston2014-08-241-1/+3
| | | | include the port due to SRV.
* Make the content repo work with in daemon mode. Return the full url on ↵Erik Johnston2014-08-221-6/+20
| | | | upload. Update the webclient to use new content repo api.
* Reduce the amount of incredibly spammy stack traces. Expected errors (e.g. ↵Kegan Dougal2014-08-191-1/+13
| | | | SynapseErrors) shouldn't have their full trace logged every time. Don't send responses to disconnected requests.
* Small amounts of cleanup and bonus round comments.Kegan Dougal2014-08-181-4/+8
|
* Implemented GETs for the ContentRepoResource. It all actually appears to be ↵Kegan Dougal2014-08-181-11/+60
| | | | working.
* hs: Make the uploads directory if it doesn't exist. Namespace uploads by the ↵Kegan Dougal2014-08-181-5/+46
| | | | base64 encoded user id of the uploader. Make a reasonable attempt to retry clashing upload paths. Try to guess a sensible file extension depending on the content type.
* Auth content uploads. Added a mapping function from request > filename. ↵Kegan Dougal2014-08-181-12/+32
| | | | Added exception handling for content uploads. webclient: Only prefix the client API path on doRequest, not doBaseRequest (this would've broken the identity server auth too). Added matrixService.uploadContent. May not require mFileUpload anymore.
* Added /matrix/content path, HS resource_for_content_repo attribute and ↵Kegan Dougal2014-08-181-0/+28
| | | | FileUploadResource. Added stub methods.
* If the web client is enabled, automatically redirect root '/' to the web ↵Kegan Dougal2014-08-141-0/+17
| | | | client path.
* Removed http_server from HomeServer. Updated unit tests to use either ↵Kegan Dougal2014-08-141-9/+3
| | | | resource_for_federation or resource_for_client depending on what is being tested.
* Start phasing out HttpServer: we should be using Resources instead. Added ↵Kegan Dougal2014-08-141-0/+5
| | | | resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree.
* add in whitespace after copyright statements to improve legibilityMatthew Hodgson2014-08-131-0/+1
|
* Just say yes to OPTIONS requests, since we want to turn on CORS headers for ↵Erik Johnston2014-08-121-0/+5
| | | | all paths
* Reference Matrix Home Servermatrix.org2014-08-121-0/+181