summary refs log tree commit diff
path: root/tests/http (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix tests on Twisted trunk. (#16528)Patrick Cloke2023-10-253-72/+69
| | | | | | | | | | | | | Twisted trunk makes a change to the `TLSMemoryBIOFactory` where the underlying protocol is changed from `TLSMemoryBIOProtocol` to `BufferingTLSTransport` to improve performance of TLS code (see https://github.com/twisted/twisted/issues/11989). In order to properly hook this code up in tests we need to pass the test reactor's clock into `TLSMemoryBIOFactory` to avoid the global (trial) reactor being used by default. Twisted does something similar internally for tests: https://github.com/twisted/twisted/blob/157cd8e659705940e895d321339d467e76ae9d0a/src/twisted/web/test/test_agent.py#L871-L874
* Fix http/s proxy authentication with long username/passwords (#16504)Richard Brežák2023-10-241-0/+21
|
* Revert "Add test case to detect dodgy b64 encoding"David Robertson2023-10-241-14/+0
| | | | | | | | | This reverts commit 5fe76b9434e22bb752c252dd9c66c3c2bfb90dfc. I think I had this accidentally commited on my local develop branch, and so it accidentally got merged into upstream develop. This should re-land with corrections in #16504.
* Merge branch 'develop' of github.com:matrix-org/synapse into developDavid Robertson2023-10-242-2/+2
|\
| * Fix type hint errors from Twisted trunk (#16526)Patrick Cloke2023-10-232-2/+2
| |
* | Add test case to detect dodgy b64 encodingDavid Robertson2023-10-231-0/+14
|/
* Add MSC4040 `matrix-fed` service lookups (#16137)Travis Ralston2023-09-051-29/+294
|
* Replace make_awaitable with AsyncMock (#16179)Patrick Cloke2023-08-241-36/+24
| | | | Python 3.8 provides a native AsyncMock, we can replace the homegrown version we have.
* Run pyupgrade for python 3.7 & 3.8. (#16110)Patrick Cloke2023-08-151-1/+1
|
* Re-introduce the outbound federation proxy (#15913)Eric Eastwood2023-07-183-6/+335
| | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
* Revert "Federation outbound proxy" (#15910)Eric Eastwood2023-07-102-238/+4
| | | | | | Revert "Federation outbound proxy (#15773)" This reverts commit b07b14b494ae1dd564b4c44f844c9a9545b3d08a.
* Federation outbound proxy (#15773)Eric Eastwood2023-07-052-4/+238
| | | | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
* Allow for the configuration of max request retries and min/max retry delays ↵Mathieu Velten2023-06-211-1/+19
| | | | in the matrix federation client (#15783)
* Switch from `matrix://` to `matrix-federation://` scheme for internal ↵Eric Eastwood2023-06-201-17/+21
| | | | | | | Synapse routing of outbound federation traffic (#15806) `matrix://` is a registered specced scheme nowadays and doesn't make sense for our internal to Synapse use case anymore. ([discussion] (https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
* Revert "Allow for the configuration of max request retries and min/max retry ↵Mathieu Velten2023-06-141-19/+1
| | | | | | delays in the matrix federation client (#12504)" This reverts commit d84e66144dc12dacf71c987a2ba802dd59c0b68e.
* Allow for the configuration of max request retries and min/max retry delays ↵Shay2023-06-091-1/+19
| | | | | | | in the matrix federation client (#12504) Co-authored-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org>
* Rename blacklist/whitelist internally. (#15620)Patrick Cloke2023-05-195-43/+43
| | | | Avoid renaming configuration settings for now and rename internal code to use blocklist and allowlist instead.
* Finish type hints for federation client HTTP code. (#15465)Patrick Cloke2023-04-241-3/+3
|
* Pull in netaddr type hints. (#15231)Patrick Cloke2023-03-091-1/+1
| | | And fix any issues from having those type hints.
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-222-2/+1
|
* Fix-up type hints in tests/server.py. (#15084)Patrick Cloke2023-02-172-4/+6
| | | | | This file was being ignored by mypy, we remove that and add the missing type hints & deal with any fallout.
* Use mypy 1.0 (#15052)David Robertson2023-02-162-26/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update mypy and mypy-zope * Remove unused ignores These used to suppress ``` synapse/storage/engines/__init__.py:28: error: "__new__" must return a class instance (got "NoReturn") [misc] ``` and ``` synapse/http/matrixfederationclient.py:1270: error: "BaseException" has no attribute "reasons" [attr-defined] ``` (note that we check `hasattr(e, "reasons")` above) * Avoid empty body warnings, sometimes by marking methods as abstract E.g. ``` tests/handlers/test_register.py:58: error: Missing return statement [empty-body] tests/handlers/test_register.py:108: error: Missing return statement [empty-body] ``` * Suppress false positive about `JaegerConfig` Complaint was ``` synapse/logging/opentracing.py:450: error: Function "Type[Config]" could always be true in boolean context [truthy-function] ``` * Fix not calling `is_state()` Oops! ``` tests/rest/client/test_third_party_rules.py:428: error: Function "Callable[[], bool]" could always be true in boolean context [truthy-function] ``` * Suppress false positives from ParamSpecs ```` synapse/logging/opentracing.py:971: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] synapse/logging/opentracing.py:1017: error: Argument 2 to "_custom_sync_async_decorator" has incompatible type "Callable[[Arg(Callable[P, R], 'func'), **P], _GeneratorContextManager[None]]"; expected "Callable[[Callable[P, R], **P], _GeneratorContextManager[None]]" [arg-type] ```` * Drive-by improvement to `wrapping_logic` annotation * Workaround false "unreachable" positives See https://github.com/Shoobx/mypy-zope/issues/91 ``` tests/http/test_proxyagent.py:626: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:762: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:826: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:838: error: Statement is unreachable [unreachable] tests/http/test_proxyagent.py:845: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:151: error: Statement is unreachable [unreachable] tests/http/federation/test_matrix_federation_agent.py:452: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:60: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:93: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:127: error: Statement is unreachable [unreachable] tests/logging/test_remote_handler.py:152: error: Statement is unreachable [unreachable] ``` * Changelog * Tweak DBAPI2 Protocol to be accepted by mypy 1.0 Some extra context in: - https://github.com/matrix-org/python-canonicaljson/pull/57 - https://github.com/python/mypy/issues/6002 - https://mypy.readthedocs.io/en/latest/common_issues.html#covariant-subtyping-of-mutable-protocol-members-is-rejected * Pull in updated canonicaljson lib so the protocol check just works * Improve comments in opentracing I tried to workaround the ignores but found it too much trouble. I think the corresponding issue is https://github.com/python/mypy/issues/12909. The mypy repo has a PR claiming to fix this (https://github.com/python/mypy/pull/14677) which might mean this gets resolved soon? * Better annotation for INTERACTIVE_AUTH_CHECKERS * Drive-by AUTH_TYPE annotation, to remove an ignore
* Properly typecheck types.http (#14988)David Robertson2023-02-0712-183/+285
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Tweak http types in Synapse AFACIS these are correct, and they make mypy happier on tests.http. * Type hints for test_proxyagent * type hints for test_srv_resolver * test_matrix_federation_agent * tests.http.server._base * tests.http.__init__ * tests.http.test_additional_resource * tests.http.test_client * tests.http.test_endpoint * tests.http.test_matrixfederationclient * tests.http.test_servlet * tests.http.test_simple_client * tests.http.test_site * One fixup in tests.server * Untyped defs * Changelog * Fixup syntax for Python 3.7 * Fix olddeps syntax * Use a twisted IPv4 addr for dummy_address * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Remove redundant `Optional` --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Remove redundant types from comments. (#14412)Patrick Cloke2022-11-161-3/+4
| | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
* Making parse_server_name more consistent (#14007)Abdullah Osama2022-10-111-0/+3
| | | Fixes #12122
* Indicate what endpoint came back with a JSON response we were unable to ↵Eric Eastwood2022-10-071-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | parse (#14097) **Before:** ``` WARNING - POST-11 - Unable to parse JSON: Expecting value: line 1 column 1 (char 0) (b'') ``` **After:** ``` WARNING - POST-11 - Unable to parse JSON from POST /_matrix/client/v3/join/%21ZlmJtelqFroDRJYZaq:hs1?server_name=hs1 response: Expecting value: line 1 column 1 (char 0) (b'') ``` --- It's possible to figure out which endpoint these warnings were coming from before but you had to follow the request ID `POST-11` to the log line that says `Completed request [...]`. Including this key information next to the JSON parsing error makes it much easier to reason whether it matters or not. ``` 2022-09-29T08:23:25.7875506Z synapse_main | 2022-09-29 08:21:10,336 - synapse.http.matrixfederationclient - 299 - INFO - POST-11 - {GET-O-13} [hs1] Completed request: 200 OK in 0.53 secs, got 450 bytes - GET matrix://hs1/_matrix/federation/v1/make_join/%21ohtKoQiXlPePSycXwp%3Ahs1/%40charlie%3Ahs2?ver=1&ver=2&ver=3&ver=4&ver=5&ver=6&ver=org.matrix.msc2176&ver=7&ver=8&ver=9&ver=org.matrix.msc3787&ver=10&ver=org.matrix.msc2716v4 ``` --- As a note, having no `body` is normal for the `/join` endpoint and it can handle it. https://github.com/matrix-org/synapse/blob/0c853e09709d52783efd37060ed9e8f55a4fc704/synapse/rest/client/room.py#L398-L403 Alternatively we could remove these extra logs but they are probably more usually helpful to figure out what went wrong.
* Cancel the processing of key query requests when they time out. (#13680)reivilibre2022-09-071-1/+9
|
* Generalise the `@cancellable` annotation so it can be used on functions ↵reivilibre2022-08-311-1/+1
| | | | other than just servlet methods. (#13662)
* Use literals in place of `HTTPStatus` constants in tests (#13463)Dirk Klimpel2022-08-051-2/+1
|
* Fix `destination_is` errors seen in sentry. (#13041)David Robertson2022-06-141-0/+14
| | | | | | * Rename test_fedclient to match its source file * Require at least one destination to be truthy * Explicitly validate user ID in profile endpoint GETs Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Clean up the test code for client disconnections (#12929)Sean Quah2022-06-072-69/+73
| | | | | | | * Reword failure message about `await_result=False` * Use `reactor.advance()` instead of `reactor.pump()` * Raise `AssertionError`s ourselves * Un-instance method `_test_disconnect` * Replace `ThreadedMemoryReactorClock` with `MemoryReactorClock`
* Test cancellation at every `await` during request handling (#12674)Sean Quah2022-06-071-2/+450
| | | | | | * Add tests for `/rooms/<room_id>/members` cancellation. * Add tests for `/rooms/<room_id>/state` cancellation. Signed-off-by: Sean Quah <seanq@element.io>
* Allow bigger responses to `/federation/v1/state` (#12877)Richard van der Hoff2022-05-251-3/+3
| | | | | | | | | | | * Refactor HTTP response size limits Rather than passing a separate `max_response_size` down the stack, make it an attribute of the `parser`. * Allow bigger responses on `federation/v1/state` `/state` can return huge responses, so we need to handle that.
* Add some type hints to tests files (#12833)Dirk Klimpel2022-05-232-7/+9
| | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Respect the `@cancellable` flag for `RestServlet`s and ↵Sean Quah2022-05-111-1/+59
| | | | | | | | | | | | | | | `BaseFederationServlet`s (#12699) Both `RestServlet`s and `BaseFederationServlet`s register their handlers with `HttpServer.register_paths` / `JsonResource.register_paths`. Update `JsonResource` to respect the `@cancellable` flag on handlers registered in this way. Although `ReplicationEndpoint` also registers itself using `register_paths`, it does not pass the handler method that would have the `@cancellable` flag directly, and so needs separate handling. Signed-off-by: Sean Quah <seanq@element.io>
* Add helper class for testing request cancellation (#12630)Sean Quah2022-05-102-0/+113
| | | | | | | Also expose the `SynapseRequest` from `FakeChannel` in tests, so that we can call `Request.connectionLost` to simulate a client disconnecting. Signed-off-by: Sean Quah <seanq@element.io>
* Replace assertEquals and friends with non-deprecated versions. (#12092)Patrick Cloke2022-02-281-12/+12
|
* Remove support for the webclient listener. (#11895)Patrick Cloke2022-02-031-108/+0
| | | Also remove support for non-HTTP(S) web_client_location.
* Add a regression test for using both webclient and client resources ↵Andrew Morgan2022-01-201-0/+108
| | | | simultaneously (#11765)
* Prevent the media store from writing outside of the configured directorySean Quah2021-11-191-0/+3
| | | | | Also tighten validation of server names by forbidding invalid characters in IPv6 addresses and empty domain labels.
* Use direct references for configuration variables (part 6). (#10916)Patrick Cloke2021-09-291-1/+1
|
* Add reactor to `SynapseRequest` and fix up types. (#10868)Erik Johnston2021-09-241-2/+6
|
* support federation queries through http connect proxy (#10475)Dirk Klimpel2021-08-112-109/+372
| | | | | Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu> Signed-off-by: Dirk Klimpel dirk@klimpel.org
* allow specifying https:// proxy (#10411)Dirk Klimpel2021-07-271-58/+340
|
* Allow providing credentials to `http_proxy` (#10360)Dirk Klimpel2021-07-151-13/+52
|
* [pyupgrade] `tests/` (#10347)Jonathan de Jong2021-07-132-6/+4
|
* Improved validation for received requests (#9817)Richard van der Hoff2021-04-231-0/+83
| | | | | | * Simplify `start_listening` callpath * Correctly check the size of uploaded files
* Limit the size of HTTP responses read over federation. (#9833)Richard van der Hoff2021-04-231-0/+59
|
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-1410-10/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Use mock from the stdlib. (#9772)Patrick Cloke2021-04-096-9/+6
|
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-4/+11
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Allow providing credentials to HTTPS_PROXY (#9657)Andrew Morgan2021-03-221-0/+40
| | | | | | | Addresses https://github.com/matrix-org/synapse-dinsic/issues/70 This PR causes `ProxyAgent` to attempt to extract credentials from an `HTTPS_PROXY` env var. If credentials are found, a `Proxy-Authorization` header ([details](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization)) is sent to the proxy server to authenticate against it. The headers are *not* passed to the remote server. Also added some type hints.
* Add tests for blacklisting reactor/agent. (#9563)Patrick Cloke2021-03-111-2/+124
|
* (Hopefully) stop leaking file descriptors in media repo. (#9497)Patrick Cloke2021-03-011-36/+55
| | | | By consuming the response if the headers imply that the content is too large.
* Add support for no_proxy and case insensitive env variables (#9372)Tim Leung2021-02-261-44/+73
| | | | | | | | | | | | | | | ### Changes proposed in this PR - Add support for the `no_proxy` and `NO_PROXY` environment variables - Internally rely on urllib's [`proxy_bypass_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2519) - Extract env variables using urllib's `getproxies`/[`getproxies_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2488) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment This does contain behaviour changes for consumers so making sure these are called out: - `no_proxy`/`NO_PROXY` is now respected - lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY` Related to #9306 which also uses `ProxyAgent` Signed-off-by: Timothy Leung tim95@hotmail.co.uk
* Reduce the memory usage of previewing media files. (#9421)Patrick Cloke2021-02-181-4/+5
| | | | | | | | This reduces the memory usage of previewing media files which end up larger than the `max_spider_size` by avoiding buffering content internally in treq. It also checks the `Content-Length` header in additional places instead of streaming the content to check the body length.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-4/+2
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Support icons for Identity Providers (#9154)Richard van der Hoff2021-01-201-1/+1
|
* Avoid raising the body exceeded error multiple times. (#9108)Patrick Cloke2021-01-152-2/+103
| | | | | Previously this code generated unreferenced `Deferred` instances which caused "Unhandled Deferreds" errors to appear in error situations.
* Merge branch 'master' into developErik Johnston2021-01-131-0/+130
|\
| * Don't apply the IP range blacklist to proxy connections (#9084)Marcus2021-01-121-0/+130
| | | | | | | | | | It is expected that the proxy would be on a private IP address so the configured proxy should be connected to regardless of the IP range blacklist.
* | Handle bad JSON data being returned from the federation API. (#9070)Patrick Cloke2021-01-121-1/+1
|/
* Add a maximum size for well-known lookups. (#8950)Patrick Cloke2020-12-161-0/+27
|
* Remove spurious "SynapseRequest" result from `make_request"Richard van der Hoff2020-12-151-2/+2
| | | | This was never used, so let's get rid of it.
* Replace `request.code` with `channel.code`Richard van der Hoff2020-12-151-2/+2
| | | | | | The two are equivalent, but really we want to check the HTTP result that got returned to the channel, not the code that the Request object *intended* to return to the channel.
* Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-021-0/+3
| | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* Remove redundant calls to `render()`Richard van der Hoff2020-11-161-3/+1
|
* use global make_request() directly where we have a custom ResourceRichard van der Hoff2020-11-151-6/+7
| | | | | | Where we want to render a request against a specific Resource, call the global make_request() function rather than the one in HomeserverTestCase, allowing us to pass in an appropriate `Site`.
* Fix handling of connection timeouts in outgoing http requests (#8400)Richard van der Hoff2020-09-292-6/+188
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove `on_timeout_cancel` from `timeout_deferred` The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a timeout (in particular if the canceller raised an exception), so it was unreliable. It was also only used in one place, and to be honest it's easier to do what it does a different way. * Fix handling of connection timeouts in outgoing http requests Turns out that if we get a timeout during connection, then a different exception is raised, which wasn't always handled correctly. To fix it, catch the exception in SimpleHttpClient and turn it into a RequestTimedOutError (which is already a documented exception). Also add a description to RequestTimedOutError so that we can see which stage it failed at. * Fix incorrect handling of timeouts reading federation responses This was trapping the wrong sort of TimeoutError, so was never being hit. The effect was relatively minor, but we should fix this so that it does the expected thing. * Fix inconsistent handling of `timeout` param between methods `get_json`, `put_json` and `delete_json` were applying a different timeout to the response body to `post_json`; bring them in line and test. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
* Merge tag 'v1.20.0rc3' into developPatrick Cloke2020-09-112-0/+128
|\ | | | | | | | | | | | | | | | | | | Synapse 1.20.0rc3 (2020-09-11) ============================== Bugfixes -------- - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
| * Fix the exception that is raised when invalid JSON is encountered. (#8291)Patrick Cloke2020-09-102-0/+128
| |
* | Use TLSv1.2 for fake servers in tests (#8208)Dan Callaghan2020-09-101-1/+1
|/ | | | | | | | | | | | | | Some Linux distros have begun disabling TLSv1.0 and TLSv1.1 by default for security reasons, for example in Fedora 33 onwards: https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2 Use TLSv1.2 for the fake TLS servers created in the test suite, to avoid failures due to OpenSSL disallowing TLSv1.0: <twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl_choose_client_version', 'unsupported protocol')]> Signed-off-by: Dan Callaghan <djc@djc.id.au>
* Stop sub-classing object (#8249)Patrick Cloke2020-09-042-2/+2
|
* Convert the well known resolver to async (#8214)Patrick Cloke2020-09-011-6/+18
|
* Reduce run-times of tests by advancing the reactor less (#7757)Andrew Morgan2020-08-271-1/+1
|
* Convert federation client to async/await. (#7975)Patrick Cloke2020-07-301-14/+36
|
* Convert the federation agent and related code to async/await. (#7874)Patrick Cloke2020-07-232-37/+40
|
* Merge different Resource implementation classes (#7732)Erik Johnston2020-07-031-0/+62
|
* Include a user agent in federation requests. (#7677)Patrick Cloke2020-06-161-0/+10
|
* Clean up some LoggingContext stuff (#7120)Richard van der Hoff2020-03-243-9/+9
| | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.
* Share SSL contexts for non-federation requests (#7094)Richard van der Hoff2020-03-171-3/+3
| | | | | | | Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests. Fixes #7092.
* Support for routing outbound HTTP requests via a proxy (#6239)Richard van der Hoff2019-11-013-3/+359
| | | | | | | | | | | | | | | | | | | | The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy. The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`. The proxy will then be used for * push * url previews * phone-home stats * recaptcha validation * CAS auth validation It will *not* be used for: * Application Services * Identity servers * Outbound federation * In worker configurations, connections from workers to masters Fixes #4198.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-08-221-35/+48
|\ | | | | | | erikj/reliable_lookups
| * Change jitter to be a factor rather than absolute valueErik Johnston2019-08-201-2/+2
| |
| * Retry well known on fail.Erik Johnston2019-08-151-33/+46
| | | | | | | | | | | | If we have recently seen a valid well-known for a domain we want to retry on (non-final) errors a few times, to handle temporary blips in networking/etc.
* | Remove test debugsErik Johnston2019-08-201-1/+0
| |
* | Refactor MatrixFederationAgent to retry SRV.Erik Johnston2019-08-152-5/+66
|/ | | | | | | | This refactors MatrixFederationAgent to move the SRV lookup into the endpoint code, this has two benefits: 1. Its easier to retry different host/ports in the same way as HostnameEndpoint. 2. We avoid SRV lookups if we have a free connection in the pool
* Retry well-known lookup before expiry.Erik Johnston2019-08-131-0/+69
| | | | | | | | | This gives a bit of a grace period where we can attempt to refetch a remote `well-known`, while still using the cached result if that fails. Hopefully this will make the well-known resolution a bit more torelant of failures, rather than it immediately treating failures as "no result" and caching that for an hour.
* Move well known lookup into a separate clasErik Johnston2019-08-071-20/+19
|
* Add a lower bound for TTL on well known results.Erik Johnston2019-08-061-2/+2
| | | | | | | It costs both us and the remote server for us to fetch the well known for every single request we send, so we add a minimum cache period. This is set to 5m so that we still honour the basic premise of "refetch frequently".
* Share SSL options for well-known requestsErik Johnston2019-07-311-6/+6
|
* Replace returnValue with return (#5736)Amber Brown2019-07-233-4/+4
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-043-3/+3
|
* Pass config_dir_path and data_dir_path into Config.read_config. (#5522)Richard van der Hoff2019-06-241-1/+1
| | | | | | * Pull config_dir_path and data_dir_path calculation out of read_config_files * Pass config_dir_path and data_dir_path into read_config
* Run Black. (#5482)Amber Brown2019-06-204-125/+124
|
* fix ci on py2, againRichard van der Hoff2019-06-101-1/+1
|
* fix CI on python 2.7Richard van der Hoff2019-06-101-6/+4
|
* Tests for SSL certs for federation connectionsRichard van der Hoff2019-06-106-106/+343
| | | | Add some tests for bad certificates for federation and .well-known connections
* Stop hardcoding trust of old matrix.org key (#5374)Richard van der Hoff2019-06-061-0/+1
| | | | | | | | | | | | | | | | | There are a few changes going on here: * We make checking the signature on a key server response optional: if no verify_keys are specified, we trust to TLS to validate the connection. * We change the default config so that it does not require responses to be signed by the old key. * We replace the old 'perspectives' config with 'trusted_key_servers', which is also formatted slightly differently. * We emit a warning to the logs every time we trust a key server response signed by the old key.
* Validate federation server TLS certificates by default.Richard van der Hoff2019-06-051-3/+9
|
* Migrate all tests to use the dict-based config format instead of hanging ↵Amber Brown2019-05-131-1/+3
| | | | items off HomeserverConfig (#5171)
* Add ability to blacklist ip ranges for federation traffic (#5043)Andrew Morgan2019-05-131-0/+71
|
* Run Black on the tests again (#5170)Amber Brown2019-05-104-172/+113
|
* Config option for verifying federation certificates (MSC 1711) (#4967)Andrew Morgan2019-04-251-1/+2
|
* lintAndrew Morgan2019-03-201-1/+1
|
* New test, fix issuesAndrew Morgan2019-03-201-0/+45
|
* Fix comments. v0.99.2 -> v0.99.3Andrew Morgan2019-03-201-1/+1
|
* Federation test fixed!Andrew Morgan2019-03-201-2/+5
|
* Better exception handlingAndrew Morgan2019-03-181-4/+1
|
* Correct var nameAndrew Morgan2019-03-131-0/+54
|
* i should have given up x3Andrew Morgan2019-03-131-39/+0
|
* i should have given up x2Andrew Morgan2019-03-131-0/+39
|
* Move ClientTLSOptionsFactory init out of refresh_certificates (#4611)Richard van der Hoff2019-02-111-3/+1
| | | | | It's nothing to do with refreshing the certificates. No idea why it was here.
* Treat an invalid .well-known the same as an absent oneRichard van der Hoff2019-02-011-11/+70
| | | | ... basically, carry on and fall back to SRV etc.
* Update federation routing logic to check .well-known before SRVRichard van der Hoff2019-01-311-30/+21
|
* Follow redirects on .well-known (#4520)Richard van der Hoff2019-01-301-0/+97
|
* Add a caching layer to .well-known responses (#4516)Richard van der Hoff2019-01-301-4/+146
|
* Relax requirement for a content-type on .well-known (#4511)Richard van der Hoff2019-01-291-1/+0
|
* Implement MSC1708 (.well-known lookups for server routing) (#4489)Richard van der Hoff2019-01-293-7/+339
|
* Use SimpleResolverComplexifier in tests (#4497)Richard van der Hoff2019-01-291-4/+3
| | | | | two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we rely on to give consistent behaviour between python 2 and 3.
* Handle IP literals explicitlyRichard van der Hoff2019-01-281-17/+2
| | | | We don't want to be doing .well-known lookups on these guys.
* Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487)Richard van der Hoff2019-01-281-1/+180
| | | | | | | | | | | | | | | | Turns out that the library does a better job of parsing URIs than our reinvented wheel. Who knew. There are two things going on here. The first is that, unlike parse_server_name, URI.fromBytes will strip off square brackets from IPv6 literals, which means that it is valid input to ClientTLSOptionsFactory and HostnameEndpoint. The second is that we stay in `bytes` throughout (except for the argument to ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up with idna-encoded values being held in `unicode` variables. TBH it probably would have been ok but it made the tests fragile.
* Fix Host header sent by MatrixFederationAgent (#4468)Richard van der Hoff2019-01-252-1/+17
| | | | | | Move the Host header logic down here so that (a) it is used if we reuse the agent elsewhere, and (b) we can mess about with it with .well-known.
* Look up the right SRV recordRichard van der Hoff2019-01-241-3/+9
|
* lots more tests for MatrixFederationAgentRichard van der Hoff2019-01-241-10/+79
|
* Don't send IP addresses as SNI (#4452)Richard van der Hoff2019-01-241-3/+60
| | | | | | The problem here is that we have cut-and-pasted an impl from Twisted, and then failed to maintain it. It was fixed in Twisted in https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
* Add a test for MatrixFederationAgentRichard van der Hoff2019-01-221-0/+183
|
* put resolve_service in an objectRichard van der Hoff2019-01-221-20/+18
| | | | this makes it easier to stub things out for tests.
* Require that service_name be a byte stringRichard van der Hoff2019-01-221-4/+4
| | | | it is only ever a bytes now, so let's enforce that.
* Make MatrixFederationClient use MatrixFederationAgentRichard van der Hoff2019-01-221-0/+96
| | | | ... instead of the matrix_federation_endpoint
* Refactor and bugfix for resove_service (#4427)Richard van der Hoff2019-01-222-0/+223
|
* Remove redundant WrappedConnection (#4409)Richard van der Hoff2019-01-181-7/+47
| | | | | | | | * Remove redundant WrappedConnection The matrix federation client uses an HTTP connection pool, which times out its idle HTTP connections, so there is no need for any of this business.
* Refactor request sending to have better excpetions (#4358)Erik Johnston2019-01-081-3/+10
| | | | | | | | | | | | | | * Correctly retry and back off if we get a HTTPerror response * Refactor request sending to have better excpetions MatrixFederationHttpClient blindly reraised exceptions to the caller without differentiating "expected" failures (e.g. connection timeouts etc) versus more severe problems (e.g. programming errors). This commit adds a RequestSendFailed exception that is raised when "expected" failures happen, allowing the TransactionQueue to log them as warnings while allowing us to log other exceptions as actual exceptions.
* Merge branch 'master' into developAmber Brown2018-09-251-2/+2
|\
| * Fix compatibility issue with older Twisted in tests.Oleg Girko2018-09-251-2/+2
| | | | | | | | | | | | | | | | Older Twisted (18.4.0) returns TimeoutError instead of ConnectingCancelledError when connection times out. This change allows tests to be compatible with this behaviour. Signed-off-by: Oleg Girko <ol@infoserver.lv>
* | Refactor matrixfederationclient to fix logging (#3906)Richard van der Hoff2018-09-181-5/+38
|/ | | | | | | | We want to wait until we have read the response body before we log the request as complete, otherwise a confusing thing happens where the request appears to have completed, but we later fail it. To do this, we factor the salient details of a request out to a separate object, which can then keep track of the txn_id, so that it can be logged.
* Attempt to figure out what's going on with timeouts (#3857)Amber Brown2018-09-141-0/+157
|
* Run black.black2018-08-101-4/+2
|
* run isortAmber Brown2018-07-091-4/+2
|
* More server_name validationRichard van der Hoff2018-07-041-4/+13
| | | | | | | | We need to do a bit more validation when we get a server name, but don't want to be re-doing it all over the shop, so factor out a separate parse_and_validate_server_name, and do the extra validation. Also, use it to verify the server name in the config file.
* Reject invalid server names (#3480)Richard van der Hoff2018-07-032-0/+46
Make sure that server_names used in auth headers are sane, and reject them with a sensible error code, before they disappear off into the depths of the system.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-25 19:57:24 +0000