summary refs log tree commit diff
path: root/synapse/http/client.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Re-introduce the outbound federation proxy (#15913)Eric Eastwood2023-07-181-1/+6
| | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
* Revert "Federation outbound proxy" (#15910)Eric Eastwood2023-07-101-6/+1
| | | | | | Revert "Federation outbound proxy (#15773)" This reverts commit b07b14b494ae1dd564b4c44f844c9a9545b3d08a.
* Federation outbound proxy (#15773)Eric Eastwood2023-07-051-1/+6
| | | | | | | Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
* Use a custom scheme & the worker name for replication requests. (#15578)Jason Little2023-05-231-0/+1
| | | | | | | | All the information needed is already in the `instance_map`, so use that instead of passing the hostname / IP & port manually for each replication request. This consolidates logic for future improvements of using e.g. UNIX sockets for workers.
* Rename blacklist/whitelist internally. (#15620)Patrick Cloke2023-05-191-64/+55
| | | | Avoid renaming configuration settings for now and rename internal code to use blocklist and allowlist instead.
* HTTP Replication Client (#15470)Jason Little2023-05-091-1/+132
| | | | | | Separate out a HTTP client for replication in preparation for also supporting using UNIX sockets. The major difference from the base class is that this does not use treq to handle HTTP requests.
* Revert "Reduce the size of the HTTP connection pool for non-pushers" (#15530) v1.83.0rc1Sean Quah2023-05-031-3/+11
| | | | | | | | | #15514 introduced a regression where Synapse would encounter `PartialDownloadError`s when fetching OpenID metadata for certain providers on startup. Due to #8088, this prevents Synapse from starting entirely. Revert the change while we decide what to do about the regression.
* Reduce the size of the HTTP connection pool for non-pushers. (#15514)Patrick Cloke2023-05-021-11/+3
| | | | | | | | | | | | | | Pushers tend to make many connections to the same HTTP host (e.g. a new event comes in, causes events to be pushed, and then the homeserver connects to the same host many times). Due to this the per-host HTTP connection pool size was increased, but this does not make sense for other SimpleHttpClients. Add a parameter for the connection pool and override it for pushers (making a separate SimpleHttpClient for pushers with the increased configuration). This returns the HTTP connection pool settings to the default Twisted ones for non-pusher HTTP clients.
* Refactor `SimpleHttpClient` to pull out reusable methods (#15427)Jason Little2023-04-141-56/+76
| | | | Pulls out some methods to `BaseHttpClient` to eventually be reused in other contexts.
* Call appservices on modern paths, falling back to legacy paths. (#15317)Patrick Cloke2023-04-031-6/+7
| | | | | This uses the specced /_matrix/app/v1/... paths instead of the "legacy" paths. If the homeserver receives an error it will retry using the legacy path.
* Implement MSC3984 to proxy /keys/query requests to appservices. (#15321)Patrick Cloke2023-03-301-0/+38
| | | | | If enabled, for users which are exclusively owned by an application service then the appservice will be queried for devices in addition to any information stored in the Synapse database.
* Pull in netaddr type hints. (#15231)Patrick Cloke2023-03-091-3/+5
| | | And fix any issues from having those type hints.
* Add missing type hints to InsecureInterceptableContextFactory. (#15164)Patrick Cloke2023-02-281-2/+3
|
* Properly typecheck types.http (#14988)David Robertson2023-02-071-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Tweak http types in Synapse AFACIS these are correct, and they make mypy happier on tests.http. * Type hints for test_proxyagent * type hints for test_srv_resolver * test_matrix_federation_agent * tests.http.server._base * tests.http.__init__ * tests.http.test_additional_resource * tests.http.test_client * tests.http.test_endpoint * tests.http.test_matrixfederationclient * tests.http.test_servlet * tests.http.test_simple_client * tests.http.test_site * One fixup in tests.server * Untyped defs * Changelog * Fixup syntax for Python 3.7 * Fix olddeps syntax * Use a twisted IPv4 addr for dummy_address * Fix typo, thanks Sean Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Remove redundant `Optional` --------- Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Improve `RawHeaders` type hints (#14303)David Robertson2022-10-281-5/+19
|
* Reduce the number of "untyped defs" (#12716)David Robertson2022-05-121-6/+10
|
* Reload cache factors from disk on SIGHUP (#12673)David Robertson2022-05-111-1/+1
|
* Unify HTTP query parameter type hints (#12415)David Robertson2022-04-081-13/+3
| | | | | | * Pull out query param types to `synapse.http.types` * Use QueryParams everywhere * Simplify `encode_query_args` * Add annotation which would have caught #12410
* Adds misc missing type hints (#11953)Patrick Cloke2022-02-111-6/+5
|
* Implement a content type allow list for URL previews (#11936)Denis Kasak2022-02-101-0/+18
| | | | | | | This implements an allow list for content types for which Synapse will attempt URL preview. If a URL resolves to a resource with a content type which isn't in the list, the download will terminate immediately. This makes sense given that Synapse would never successfully generate a URL preview for such files in the first place, and helps prevent issues with streaming media servers, such as #8302. Signed-off-by: Denis Kasak dkasak@termina.org.uk
* Apply a timeout to reading the body when fetching a file. (#11784)Patrick Cloke2022-01-241-3/+12
| | | | This prevents the URL preview code from reading a stream forever.
* Fix SimpleHttpClient not sending Accept header in `get_json` (#11677)Fr3shTea2022-01-051-1/+1
| | | Co-authored-by: reivilibre <olivier@librepush.net>
* Use HTTPStatus constants in place of literals in `synapse.http` (#11543)Dirk Klimpel2021-12-091-4/+11
|
* Relax `ignore-missing-imports` for modules that have stubs now and update ↵David Robertson2021-10-081-1/+1
| | | | | | | | | | | | mypy (#11006) Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details. We therefore pull in stub packages in setup.py Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them. The rest of this change consists of fixups to make the newer mypy + stubs pass CI. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Use direct references for some configuration variables (part 2) (#10812)Patrick Cloke2021-09-151-2/+5
|
* Fix up type hints for Twisted 21.7 (#10490)Richard van der Hoff2021-07-281-2/+2
| | | Mostly this involves decorating a few Deferred declarations with extra type hints. We wrap the types in quotes to avoid runtime errors when running against older versions of Twisted that don't have generics on Deferred.
* Use inline type hints in various other places (in `synapse/`) (#10380)Jonathan de Jong2021-07-151-9/+9
|
* Use ijson to parse the response to `/send_join`, reducing memory usage. (#9958)Erik Johnston2021-05-201-1/+6
| | | Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
* Limit the size of HTTP responses read over federation. (#9833)Richard van der Hoff2021-04-231-2/+13
|
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-2/+2
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Update mypy configuration: `no_implicit_optional = True` (#9742)Jonathan de Jong2021-04-051-1/+1
|
* Import HomeServer from the proper module. (#9665)Patrick Cloke2021-03-231-1/+1
|
* Fix remaining mypy issues due to Twisted upgrade. (#9608)Patrick Cloke2021-03-151-2/+10
|
* Fix additional type hints from Twisted 21.2.0. (#9591)Patrick Cloke2021-03-121-1/+8
|
* Add tests for blacklisting reactor/agent. (#9563)Patrick Cloke2021-03-111-12/+14
|
* Create a SynapseReactor type which incorporates the necessary reactor ↵Patrick Cloke2021-03-081-2/+3
| | | | | interfaces. (#9528) This helps fix some type hints when running with Twisted 21.2.0.
* (Hopefully) stop leaking file descriptors in media repo. (#9497)Patrick Cloke2021-03-011-2/+29
| | | | By consuming the response if the headers imply that the content is too large.
* Add support for no_proxy and case insensitive env variables (#9372)Tim Leung2021-02-261-6/+4
| | | | | | | | | | | | | | | ### Changes proposed in this PR - Add support for the `no_proxy` and `NO_PROXY` environment variables - Internally rely on urllib's [`proxy_bypass_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2519) - Extract env variables using urllib's `getproxies`/[`getproxies_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2488) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment This does contain behaviour changes for consumers so making sure these are called out: - `no_proxy`/`NO_PROXY` is now respected - lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY` Related to #9306 which also uses `ProxyAgent` Signed-off-by: Timothy Leung tim95@hotmail.co.uk
* Reduce the memory usage of previewing media files. (#9421)Patrick Cloke2021-02-181-14/+12
| | | | | | | | This reduces the memory usage of previewing media files which end up larger than the `max_spider_size` by avoiding buffering content internally in treq. It also checks the `Content-Length` header in additional places instead of streaming the content to check the body length.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-2/+5
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Properly raise an exception when the body exceeds the max size. (#9145)Patrick Cloke2021-01-181-1/+1
| | | ...instead of just creating the exception object and doing nothing with it.
* Avoid raising the body exceeded error multiple times. (#9108)Patrick Cloke2021-01-151-1/+11
| | | | | Previously this code generated unreferenced `Deferred` instances which caused "Unhandled Deferreds" errors to appear in error situations.
* Reduce scope of exception handler. (#9106)Patrick Cloke2021-01-131-5/+5
| | | | Removes a bare `except Exception` clause and replaces it with catching a specific exception around the portion that might throw.
* Don't apply the IP range blacklist to proxy connections (#9084)Marcus2021-01-121-0/+1
| | | | | It is expected that the proxy would be on a private IP address so the configured proxy should be connected to regardless of the IP range blacklist.
* Add a maximum size for well-known lookups. (#8950)Patrick Cloke2020-12-161-14/+18
|
* Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-021-14/+32
| | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* Add additional type hints to HTTP client. (#8812)Patrick Cloke2020-11-251-83/+128
| | | | This also removes some duplicated code between the simple HTTP client and matrix federation client.
* Tell Black to format code for Python 3.5 (#8664)Dan Callahan2020-10-271-1/+1
| | | | | | | | This allows trailing commas in multi-line arg lists. Minor, but we might as well keep our formatting current with regard to our minimum supported Python version. Signed-off-by: Dan Callahan <danc@element.io>
* Fix handling of connection timeouts in outgoing http requests (#8400)Richard van der Hoff2020-09-291-21/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove `on_timeout_cancel` from `timeout_deferred` The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a timeout (in particular if the canceller raised an exception), so it was unreliable. It was also only used in one place, and to be honest it's easier to do what it does a different way. * Fix handling of connection timeouts in outgoing http requests Turns out that if we get a timeout during connection, then a different exception is raised, which wasn't always handled correctly. To fix it, catch the exception in SimpleHttpClient and turn it into a RequestTimedOutError (which is already a documented exception). Also add a description to RequestTimedOutError so that we can see which stage it failed at. * Fix incorrect handling of timeouts reading federation responses This was trapping the wrong sort of TimeoutError, so was never being hit. The effect was relatively minor, but we should fix this so that it does the expected thing. * Fix inconsistent handling of `timeout` param between methods `get_json`, `put_json` and `delete_json` were applying a different timeout to the response body to `post_json`; bring them in line and test. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>
* Add type annotations to SimpleHttpClient (#8372)Richard van der Hoff2020-09-241-56/+131
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-4/+4
|
* Be stricter about JSON that is accepted by Synapse (#8106)Patrick Cloke2020-08-191-5/+6
|
* Reduce INFO logging (#8050)Erik Johnston2020-08-111-1/+1
| | | | | | | | | | c.f. #8021 A lot of the code here is to change the `Completed 200 OK` logging to include the request URI so that we can drop the `Sending request...` log line. Some notes: 1. We won't log retries, which may be confusing considering the time taken log line includes retries and sleeps. 2. The `_send_request_with_optional_trailing_slash` will always be logged *without* the forward slash, even if it succeeded only with the forward slash.
* Convert the SimpleHttpClient to async. (#8016)Patrick Cloke2020-08-041-31/+24
|
* Ensure the msg property of HttpResponseException is a string. (#7979)Patrick Cloke2020-07-291-4/+12
|
* Ensure that calls to `json.dumps` are compatible with the standard library ↵Patrick Cloke2020-07-151-2/+2
| | | | json. (#7836)
* Fix client reader sharding tests (#7853)Erik Johnston2020-07-151-1/+23
| | | | | | | | | | | | | | | * Fix client reader sharding tests * Newsfile * Fix typing * Update changelog.d/7853.misc Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Move mocking of http_client to tests Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802)Patrick Cloke2020-07-101-7/+7
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-5/+3
|
* Allow configuration of Synapse's cache without using synctl or environment ↵Amber Brown2020-05-111-2/+4
| | | | variables (#6391)
* Implement OpenID Connect-based login (#7256)Quentin Gliech2020-05-081-0/+7
|
* Share SSL contexts for non-federation requests (#7094)Richard van der Hoff2020-03-171-3/+0
| | | | | | | Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests. Fixes #7092.
* Support for routing outbound HTTP requests via a proxy (#6239)Richard van der Hoff2019-11-011-2/+15
| | | | | | | | | | | | | | | | | | | | The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy. The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`. The proxy will then be used for * push * url previews * phone-home stats * recaptcha validation * CAS auth validation It will *not* be used for: * Application Services * Identity servers * Outbound federation * In worker configurations, connections from workers to masters Fixes #4198.
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-2/+2
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Edit SimpleHttpClient to reference that header keys can be passed as str or ↵Andrew Morgan2019-09-271-6/+6
| | | | bytes (#6077)
* Trace how long it takes for the send trasaction to complete, including ↵Jorik Schellekens2019-09-051-36/+51
| | | | retrys (#5986)
* Remove unused methods from c/s api v1 in register.py (#5963)Andrew Morgan2019-09-021-33/+1
| | | These methods were part of the v1 C/S API. Remove them as they are no longer used by any code paths.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-15/+13
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-201-18/+10
|
* Don't log exception when failing to fetch remote content.Erik Johnston2019-06-071-4/+9
| | | | | In particular, let's not log stack traces when we stop processing becuase the response body was too large.
* Add ability to blacklist ip ranges for federation traffic (#5043)Andrew Morgan2019-05-131-4/+2
|
* URL preview blacklisting fixes (#5155)Andrew Morgan2019-05-101-20/+25
| | | Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.
* Fix TaskStopped exceptions when outbound requests time out (#4690)Richard van der Hoff2019-02-201-3/+12
|
* Fix UnboundLocalError in post_urlencoded_get_json (#4460)Richard van der Hoff2019-01-241-2/+3
| | | This could cause exceptions if the id server returned 4xx responses.
* Fix IP URL previews on Python 3 (#4215)Amber Brown2018-12-221-134/+243
|
* Update to use new timeout function everywhere.Erik Johnston2018-09-191-2/+2
| | | | | | | The existing deferred timeout helper function (and the one into twisted) suffer from a bug when a deferred's canceller throws an exception, #3842. The new helper function doesn't suffer from this problem.
* fixAmber Brown2018-09-151-3/+3
|
* merge (#3576)Amber Brown2018-09-141-4/+10
|
* Port http/ to Python 3 (#3771)Amber Brown2018-09-061-51/+31
|
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-1/+1
|
* Kill off MatrixCodeMessageExceptionRichard van der Hoff2018-08-011-34/+27
| | | | | | | | | | | This code brings the SimpleHttpClient into line with the MatrixFederationHttpClient by having it raise HttpResponseExceptions when a request fails (rather than trying to parse for matrix errors and maybe raising MatrixCodeMessageException). Then, whenever we were checking for MatrixCodeMessageException and turning them into SynapseErrors, we now need to check for HttpResponseExceptions and call to_synapse_error.
* Move v1-only APIs into their own module & isolate deprecated ones (#3460)Amber Brown2018-07-191-2/+4
|
* run isortAmber Brown2018-07-091-23/+25
|
* Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-1/+1
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-3/+3
|
* redact_uri in two missed log pathsMichael Telatynski2018-06-061-2/+2
|
* factor out uri redaction into a method on httpMichael Telatynski2018-06-051-6/+2
|
* Strip `access_token` from outgoing requests using existing regexMichael Telatynski2018-06-021-1/+6
|
* cleanupAmber Brown2018-05-221-1/+2
|
* replacing portionsAmber Brown2018-05-211-14/+6
|
* replace stringIO importsAdrian Tschira2018-04-281-2/+2
|
* Backport deferred.addTimeoutRichard van der Hoff2018-04-271-2/+4
| | | | Twisted 16.0 doesn't have addTimeout, so let's backport it.
* Use deferred.addTimeout instead of time_bound_deferredRichard van der Hoff2018-04-231-12/+8
| | | | This doesn't feel like a wheel we need to reinvent.
* Increase http conn pool sizeRichard van der Hoff2018-01-291-1/+6
|
* Use a connection pool for the SimpleHttpClientRichard van der Hoff2018-01-201-1/+8
| | | | | In particular I hope this will help the pusher, which makes many requests to sygnal, and is currently negotiating SSL for each one.
* Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_headerRichard van der Hoff2017-10-271-28/+80
|\ | | | | Front-end proxy: pass through auth header
| * Docstring for post_urlencoded_get_jsonRichard van der Hoff2017-10-271-0/+11
| |
| * SimpleHTTPClient: add support for headersRichard van der Hoff2017-10-261-28/+69
| | | | | | | | Sometimes we need to pass headers into these methods
* | Fix logcontext leaks in httpclientRichard van der Hoff2017-10-261-10/+9
|/ | | | `preserve_context_over_fn` is borked
* Remember how twisted worksKegan Dougal2017-05-051-2/+2
|
* Include the clockKegan Dougal2017-05-051-0/+1
|
* Rewrite SimpleHttpClient.request to include timeoutsKegan Dougal2017-05-051-13/+19
| | | | Fixes #2191
* No need for the exception variableDavid Baker2017-05-031-1/+1
|
* List caught expection typesDavid Baker2017-05-031-1/+1
|
* Remove debuggingDavid Baker2017-04-261-2/+1
|
* Fix get_jsonDavid Baker2017-04-261-5/+4
|
* Use CodeMessageException subclass insteadDavid Baker2017-04-251-44/+20
| | | | | Parse json errors from get_json client methods and throw special errors.
* Need the HTTP status codeDavid Baker2017-04-211-5/+5
|
* Do the same for get_jsonDavid Baker2017-04-211-8/+21
|
* Don't error for 3xx responsesDavid Baker2017-04-211-1/+1
|
* Propagate errors sensibly from proxied IS requestsDavid Baker2017-04-211-0/+30
| | | | | | When we're proxying Matrix endpoints, parse out Matrix error responses and turn them into SynapseErrors so they can be propagated sensibly upstream.
* Remove spurious newlineJohannes Löthberg2016-12-121-1/+0
| | | | | | Apparently I just removed the spaces instead... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* Fixup for #1689 and #1690Johannes Löthberg2016-12-121-4/+7
| | | | Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* IPv6 support for client.pyGlyph2016-12-111-16/+11
| | | This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.
* Work around TLS bug in twistedRichard van der Hoff2016-06-151-2/+26
| | | | | | Wrap up twisted's FileBodyProducer to work around https://twistedmatrix.com/trac/ticket/8473. Hopefully this fixes https://matrix.org/jira/browse/SYN-700.
* Clean up the blacklist/whitelist handling.Mark Haines2016-05-161-2/+1
| | | | | | | Always set the config key with an empty list, even if a list isn't specified. This means that the codepaths are the same for both the empty list and for a missing key. Since the behaviour is the same for both cases this makes the code somewhat easier to reason about.
* add a url_preview_ip_range_whitelist config param so we can whitelist the ↵Matthew Hodgson2016-05-011-2/+4
| | | | matrix.org IP space
* Make InsecureInterceptableContextFactory work with SpiderEndpointErik Johnston2016-04-191-1/+4
|
* actually throw meaningful errorsMatthew Hodgson2016-04-081-5/+16
|
* Add url_preview_enabled config option to turn on/off preview_url endpoint. ↵Matthew Hodgson2016-04-081-5/+39
| | | | | | | | | defaults to off. Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered. Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered. Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist Add commentary and generally address PR feedback
* pep8Matthew Hodgson2016-04-031-4/+10
|
* add a persistent cache of URL lookups, and fix up the in-memory one to workMatthew Hodgson2016-04-031-3/+3
|
* support gzip compression, and don't pass through error msgsMatthew Hodgson2016-04-021-3/+8
|
* handle spidered relative images correctlyMatthew Hodgson2016-03-311-1/+1
|
* sync in changes from matrixfederationclientMatthew Hodgson2016-03-311-2/+6
|
* handle requests with missing content-length headers (e.g. YouTube)Matthew Hodgson2016-03-311-7/+26
|
* implement redirectsMatthew Hodgson2016-03-291-3/+3
|
* make it workMatthew Hodgson2016-03-291-1/+2
|
* make it buildMatthew Hodgson2016-03-291-1/+1
|
* Merge branch 'develop' into matthew/preview_urlsMatthew Hodgson2016-03-271-2/+15
|\
| * pep8David Baker2016-03-081-2/+5
| |
| * Encode unicode objects given to post_urlencode* otherwise urllib.urlencode ↵David Baker2016-03-081-2/+12
| | | | | | | | chokes.
* | initial WIP of a tentative preview_url endpoint - incomplete, untested, ↵Matthew Hodgson2016-01-241-0/+81
|/ | | | experimental, etc. just putting it here for safekeeping for now
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Verify third party ID server certificatesDaniel Wagner-Hall2015-10-161-4/+0
|
* Fix previous merge to s/version_string/user_agent/Erik Johnston2015-10-101-2/+2
|
* Add get_raw method to SimpleHttpClient, use this in CAS auth rather than ↵Steven Hammerton2015-10-101-19/+36
| | | | requests
* Use space not dash as delimiterDaniel Wagner-Hall2015-10-061-1/+1
|
* Use space not dash as delimiterDaniel Wagner-Hall2015-10-061-1/+1
|
* Preserve version string in user agentDaniel Wagner-Hall2015-10-051-3/+3
|
* Allow synapse's useragent to be customizedDaniel Wagner-Hall2015-10-021-6/+9
| | | | | This will allow me to write tests which verify which server made HTTP requests in a federation context.
* Rename context factoryDaniel Wagner-Hall2015-09-151-3/+3
| | | | Mjark is officially no fun.
* Fix random formattingDaniel Wagner-Hall2015-09-091-1/+2
|
* Allow configuration to ignore invalid SSL certsDaniel Wagner-Hall2015-09-091-2/+23
| | | | | This will be useful for sytest, and sytest only, hence the aggressive config key name.
* Remove syutil dependency in favour of smaller single-purpose librariesMark Haines2015-08-241-1/+2
|
* Fix log context when sending requestsErik Johnston2015-06-191-5/+5
|
* Log requests and responses sent via http.clientErik Johnston2015-06-151-3/+15
|
* Correctly pass connection pool parameterErik Johnston2015-05-281-1/+1
|
* Up maxPersistentPerHost countErik Johnston2015-05-281-0/+1
|
* Add connection pooling to SimpleHttpClientErik Johnston2015-05-281-2/+4
|
* Change the way we do logging contexts so that they survive divergencesErik Johnston2015-05-081-1/+5
|
* New registration for C/S API v2. Only ReCAPTCHA working currently.David Baker2015-03-301-0/+2
|
* Appease pep8Paul "LeoNerd" Evans2015-03-121-3/+7
|
* Use _ instead of . as a metric namespacing separator, for PrometheusPaul "LeoNerd" Evans2015-03-121-2/+2
|
* Rename Metrics' "keys" to "labels"Paul "LeoNerd" Evans2015-03-121-2/+2
|
* Add vector counters to HTTP clients and servers; count the requests by ↵Paul "LeoNerd" Evans2015-03-121-1/+22
| | | | method and responses by method and response code
* Pull out all uses of the underlying HTTP user agent .request() method into a ↵Paul "LeoNerd" Evans2015-03-121-5/+10
| | | | single wrapper function, to make adding metrics easier
* Fix bug which prevented the HS pushing events to the AS due to FrozenEventsKegan Dougal2015-03-021-1/+1
|
* Use git aware version string in User-Agent and Server headersErik Johnston2015-02-181-5/+5
|
* Merge pull request #50 from matrix-org/application-servicesMark Haines2015-02-131-10/+57
|\ | | | | Application Services
| * Minor tweaks based on PR feedback.Kegan Dougal2015-02-111-1/+2
| |
| * Add SimpleHttpClient.put_json with the same semantics as get_json.Kegan Dougal2015-02-041-2/+46
| |
| * Update user/alias query APIs to use new format of SimpleHttpClient.get_jsonKegan Dougal2015-02-041-0/+3
| |
| * Modify API for SimpleHttpClient.get_json and update usages.Kegan Dougal2015-02-041-10/+9
| | | | | | | | | | | | | | | | | | | | Previously, this would only return the HTTP body as JSON, and discard other response information (e.g. the HTTP response code). This has now been changed to throw a CodeMessageException on a non-2xx response, with the response code and body, which can then be parsed as JSON. Affected modules include: - Registration/Login (when using an email for IS auth)
* | Use encode_canonical_json for http clientErik Johnston2015-02-111-1/+3
| |
* | Blunty replace json with simplejsonErik Johnston2015-02-111-1/+1
|/
* Merge branch 'develop' into pushersDavid Baker2015-01-131-3/+9
|\
| * SYN-231: User agent header brokenMark Haines2015-01-081-3/+3
| |
| * Update copyright noticesMark Haines2015-01-061-1/+1
| |
| * Include version in User-Agent and Server headersMark Haines2014-12-221-2/+8
| |
* | More work on pushers. Attempt to do HTTP pokes. Not sure if the actual HTTP ↵David Baker2014-11-211-0/+19
|/ | | | pokes work or not yet but the retry semantics are pretty good.
* Fix pep8 and pyflakes warningsMark Haines2014-11-201-3/+4
|
* Separate out the matrix http client completely because just about all of its ↵David Baker2014-11-201-322/+29
| | | | code it now separate from the simple case we need for standard HTTP(S)
* Refactor the HTTP clients a little.David Baker2014-11-201-79/+75
|
* CaptchaServerHttpClient should extend the base, not matrix http client.David Baker2014-11-201-1/+1
|
* SYN-141: Encode query params as UTF-8.Mark Haines2014-11-181-1/+7
|
* Merge branch 'develop' into request_loggingMark Haines2014-10-301-3/+4
|\ | | | | | | | | Conflicts: synapse/config/logger.py
| * Fix pep8 warningsMark Haines2014-10-301-4/+7
| |
* | Add a request-id to each log lineMark Haines2014-10-301-11/+15
|/
* Add get_json method to 3pid http client. Better logging for errors in 3pid ↵Mark Haines2014-10-201-1/+39
| | | | requests
* remove debugging logging for signing requestsMark Haines2014-10-141-4/+0
|
* Verify signatures for server2server requestsMark Haines2014-10-131-3/+7
|
* SYN-75 sign at the request level rather than the transaction levelMark Haines2014-10-131-7/+45
|
* Replace on_send_callback with something a bit clearer so that we can sign ↵Mark Haines2014-10-131-11/+35
| | | | messages
* Fix pyflakes errorsMark Haines2014-10-021-8/+2
|
* Add a keyword argument to get_json to avoid retrying on DNS failures. Rather ↵Mark Haines2014-10-021-8/+1
| | | | than passing MatrixHttpClient.RETRY_DNS_LOOKUP_FAILURES as a fake query string parameter
* Split PlainHttpClient into separate clients for talking to Identity servers ↵Mark Haines2014-10-021-144/+146
| | | | and talking to Capatcha servers
* remove "red", "blue" and "green" server_name mappingsMark Haines2014-10-011-7/+0
|
* Fix 'age' key to update on retriesErik Johnston2014-09-151-3/+10
|
* Captchas now work on registration. Missing x-forwarded-for config arg ↵Kegan Dougal2014-09-051-1/+27
| | | | support. Missing reloading a new captcha on the web client / displaying a sensible error message.
* Merge branch 'develop' of github.com:matrix-org/synapse into developDavid Baker2014-09-031-4/+22
|\ | | | | | | | | Conflicts: synapse/http/client.py
| * fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-031-1/+1
| | | | | | | | hasn't been incorporated in time for launch.
| * Make retrying requests on DNS failures configurable, and turn off retrying ↵Kegan Dougal2014-09-031-6/+19
| | | | | | | | only in directory.get_association
| * hs: Updated synapse.http.client to handle DNSLookupErrors and bail immediately.Kegan Dougal2014-09-031-1/+6
| |
* | Add support for registering with a threepid to the HS (get credentials from ↵David Baker2014-09-031-5/+35
|/ | | | the client and check them against an ID server).
* Fix SSL for federation http clientMark Haines2014-09-011-1/+1
|
* Enable SSL for s2s http clientMark Haines2014-09-011-2/+6
|
* fix http client GET parameters; somehow missing named param. how could this ↵Matthew Hodgson2014-08-141-2/+2
| | | | have ever worked!?
* Update get_json()'s documentation to match the actual observed behaviourPaul "LeoNerd" Evans2014-08-131-4/+3
|
* add in whitespace after copyright statements to improve legibilityMatthew Hodgson2014-08-131-0/+1
|
* Reference Matrix Home Servermatrix.org2014-08-121-0/+246
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-22 11:28:58 +0000