summary refs log tree commit diff
path: root/synapse/crypto/keyring.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Clean up some LoggingContext stuff (#7120)Richard van der Hoff2020-03-241-2/+2
| | | | | | | | | | | | | | | | | | | | | | | * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-4/+2
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Fix exception when fetching notary server's old keys (#6625)Richard van der Hoff2020-01-061-6/+7
| | | | | | Lift the restriction that *all* the keys used for signing v2 key responses be present in verify_keys. Fixes #6596.
* Merge pull request #5895 from matrix-org/erikj/notary_keyErik Johnston2019-08-271-9/+2
|\ | | | | Add config option to sign remote key query responses with a separate key.
| * Fixup review commentsErik Johnston2019-08-231-2/+2
| |
| * Only sign when we respond to remote key requestsErik Johnston2019-08-211-10/+1
| |
| * Add config option for keys to use to sign keysErik Johnston2019-08-211-5/+7
| | | | | | | | | | This allows servers to separate keys that are used to sign remote keys when acting as a notary server.
* | Improve error msg when key-fetch fails (#5896)Richard van der Hoff2019-08-221-5/+7
|/ | | | | | | | | There's no point doing a raise_from here, because the exception is always logged at warn with no stacktrace in the caller. Instead, let's try to give better messages to reduce confusion. In particular, this means that we won't log 'Failed to connect to remote server' when we don't even attempt to connect to the remote server due to blacklisting.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-7/+7
|
* Add a delay to key lookup lock release to fix stack overflowRichard van der Hoff2019-07-191-2/+8
| | | | A tactical call_later here should fix #5723
* Refactor Keyring._start_key_lookupsRichard van der Hoff2019-07-191-51/+35
| | | | | There's an awful lot of deferreds and dictionaries flying around here. The whole thing can be made much simpler and achieve the same effect.
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-7/+8
|
* Run Black. (#5482)Amber Brown2019-06-201-22/+11
|
* Improve startup checks for insecure notary configs (#5392)Richard van der Hoff2019-06-101-7/+0
| | | | | | | | | | | | | It's not really a problem to trust notary responses signed by the old key so long as we are also doing TLS validation. This commit adds a check to the config parsing code at startup to check that we do not have the insecure matrix.org key without tls validation, and refuses to start without it. This allows us to remove the rather alarming-looking warning which happens at runtime.
* Stop hardcoding trust of old matrix.org key (#5374)Richard van der Hoff2019-06-061-35/+37
| | | | | | | | | | | | | | | | | There are a few changes going on here: * We make checking the signature on a key server response optional: if no verify_keys are specified, we trust to TLS to validate the connection. * We change the default config so that it does not require responses to be signed by the old key. * We replace the old 'perspectives' config with 'trusted_key_servers', which is also formatted slightly differently. * We emit a warning to the logs every time we trust a key server response signed by the old key.
* Associate a request_name with each verify request, for loggingRichard van der Hoff2019-06-051-44/+68
| | | | | | | Also: * rename VerifyKeyRequest->VerifyJsonRequest * calculate key_ids on VerifyJsonRequest construction * refactor things to pass around VerifyJsonRequests instead of 4-tuples
* Clean up debug logging (#5347)Richard van der Hoff2019-06-051-4/+0
| | | Remove some spurious stuff, clarify some other stuff
* Rename VerifyKeyRequest.deferred field (#5343)Richard van der Hoff2019-06-051-12/+12
| | | it's a bit confusing
* Don't do long retries when calling the key notary server. (#5334)Richard van der Hoff2019-06-041-1/+0
| | | | | | | | It takes at least 20 minutes to work through the long_retries schedule (11 attempts, each with a 60 second timeout, and 60 seconds between each request), so if the notary server isn't returning within the timeout, we'll just end up blocking whatever request is happening for 20 minutes. Ain't nobody got time for that.
* Notary server: make requests to origins in parallelRichard van der Hoff2019-06-041-1/+9
| | | | ... else we're guaranteed to time out.
* Don't bomb out on direct key fetches as soon as one failsRichard van der Hoff2019-06-041-22/+36
|
* Reduce timeout for outbound /key/v2/server requests.Richard van der Hoff2019-06-031-0/+13
|
* Enforce validity period on server_keys for fed requests. (#5321)Richard van der Hoff2019-06-031-56/+111
| | | | | | | | When handling incoming federation requests, make sure that we have an up-to-date copy of the signing key. We do not yet enforce the validity period for event signatures.
* Merge pull request #5307 from ↵Richard van der Hoff2019-06-031-12/+2
|\ | | | | | | | | matrix-org/rav/server_keys/07-fix-notary-cache-poison Stop overwriting server keys with other keys
| * Stop overwriting server keys with other keysRichard van der Hoff2019-05-311-12/+2
| | | | | | | | | | Fix a bug where we would discard a key result which the origin server is no longer returning. Fixes #5305.
* | Merge remote-tracking branch 'origin/develop' into ↵Richard van der Hoff2019-05-311-41/+28
|\| | | | | | | rav/server_keys/05-rewrite-gsvk-again
| * Remove some pointless exception handlingRichard van der Hoff2019-05-301-25/+8
| | | | | | | | | | | | | | | | The verify_request deferred already returns a suitable SynapseError, so I don't really know what we expect to achieve by doing more wrapping, other than log spam. Fixes #4278.
| * use attr.s for VerifyKeyRequestRichard van der Hoff2019-05-301-17/+21
| | | | | | | | because namedtuple is awful
* | Rewrite get_server_verify_keys, again.Richard van der Hoff2019-05-301-48/+53
|/ | | | | Attempt to simplify the logic in get_server_verify_keys by splitting it into two methods.
* Merge pull request #5251 from matrix-org/rav/server_keys/01-check_sigRichard van der Hoff2019-05-281-41/+92
|\ | | | | Ensure that server_keys fetched via a notary server are correctly signed.
| * Improve error handling/logging for perspectives-key fetching.Richard van der Hoff2019-05-241-28/+77
| | | | | | | | In particular, don't give up on the first failure.
| * Require sig from origin server on perspectives responsesRichard van der Hoff2019-05-231-13/+15
| |
* | Simplification to Keyring.wait_for_previous_lookups. (#5250)Richard van der Hoff2019-05-241-7/+4
|/ | | | | | The list of server names was redundant, since it was equivalent to the keys on the server_to_deferred map. This reduces the number of large lists being passed around, and has the benefit of deduplicating the entries in `wait_on`.
* Factor out KeyFetchers from KeyRingRichard van der Hoff2019-05-231-140/+175
| | | | | | | Rather than have three methods which have to have the same interface, factor out a separate interface which is provided by three implementations. I find it easier to grok the code this way.
* Store key validity time in the storage layerRichard van der Hoff2019-05-231-14/+33
| | | | | | | | This is a first step to checking that the key is valid at the required moment. The idea here is that, rather than passing VerifyKey objects in and out of the storage layer, we instead pass FetchKeyResult objects, which simply wrap the VerifyKey and add a valid_until_ts field.
* Simplify process_v2_response (#5236)Richard van der Hoff2019-05-231-21/+29
| | | | | | | | | * Pass time_added_ms into process_v2_response * Simplify process_v2_response We can merge old_verify_keys into verify_keys, and reduce the number of dicts flying around.
* Remove unused VerifyKey.expired and .time_added fields (#5235)Richard van der Hoff2019-05-231-3/+0
| | | | | These were never used, and poking arbitary data into objects from other packages seems confusing at best.
* Rewrite store_server_verify_key to store several keys at once (#5234)Richard van der Hoff2019-05-231-45/+14
| | | | | Storing server keys hammered the database a bit. This replaces the implementation which stored a single key, with one which can do many updates at once.
* Run black on synapse.crypto.keyring (#5232)Richard van der Hoff2019-05-221-149/+137
|
* remove extraneous exception loggingRichard van der Hoff2019-04-251-16/+32
|
* Clarify logging when PDU signature checking failsRichard van der Hoff2019-04-251-3/+1
|
* Remove periods from copyright headers (#5046)Andrew Morgan2019-04-111-1/+1
|
* Rewrite Datastore.get_server_verify_keysRichard van der Hoff2019-04-091-21/+17
| | | | Rewrite this so that it doesn't hammer the database.
* Remove redundant merged_keys dictRichard van der Hoff2019-04-081-8/+5
| | | | | | There's no point in collecting a merged dict of keys: it is sufficient to consider just the new keys which have been fetched by the most recent key_fetch_fns.
* Fix from_server buglet in get_keys_from_perspectivesRichard van der Hoff2019-04-081-1/+1
| | | | | make sure we store the name of the server the keys came from, rather than the origin server, after doing a fetch-from-perspectives.
* Hoist server_name check out of process_v2_responseRichard van der Hoff2019-04-041-13/+7
| | | | | It's easier to check it in the caller than to complicate the interface with an extra param.
* Clean up Keyring.process_v2_responseRichard van der Hoff2019-04-041-23/+46
| | | | | | Make this just return the key dict, rather than a single-entry dict mapping the server name to the key dict. It's easy for the caller to get the server name from from the response object anyway.
* Correctly log expected errors when fetching server keysErik Johnston2019-03-111-2/+2
|
* raise_from already raisesErik Johnston2019-02-251-4/+4
|
* Handle errors when fetching remote server keysErik Johnston2019-02-231-21/+51
|
* Correctly handle RequestSendFailed exceptionsErik Johnston2019-02-141-2/+2
| | | | This mainly reduces the number of exceptions we log.
* Make key fetches use regular federation client (#4426)Richard van der Hoff2019-01-221-23/+7
| | | | | All this magic is redundant.
* Remove fetching keys via the deprecated v1 kex method (#4120)Amber Brown2018-10-311-103/+7
|
* Merge pull request #3826 from matrix-org/rav/logging_for_keyringAmber Brown2018-09-121-6/+18
|\ | | | | add some logging for the keyring queue
| * add some logging for the keyring queueRichard van der Hoff2018-09-061-6/+18
| | | | | | | | why is it so damn slow?
* | Port crypto/ to Python 3 (#3822)Amber Brown2018-09-121-4/+5
|/
* fix accidental removal of hsJeroen2018-07-271-1/+1
|
* Merge remote-tracking branch 'upstream/develop' into ↵Jeroen2018-07-141-21/+23
|\ | | | | | | | | | | | | send_sni_for_federation_requests # Conflicts: # synapse/crypto/context_factory.py
| * run isortAmber Brown2018-07-091-21/+23
| |
* | send SNI for federation requestsJeroen2018-06-241-2/+2
|/
* Try to log more helpful info when a sig verification failsRichard van der Hoff2018-06-081-6/+25
| | | | | | | | Firstly, don't swallow the reason for the failure Secondly, don't assume all exceptions are verification failures Thirdly, log a bit of info about the key being used if debug is enabled
* Merge remote-tracking branch 'origin/develop' into rav/use_run_in_backgroundRichard van der Hoff2018-04-271-45/+48
|\
| * Improve exception handling for background processesRichard van der Hoff2018-04-271-45/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.
* | Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-11/+17
|/ | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* Use str(e) instead of e.messageAdrian Tschira2018-04-151-4/+4
| | | | | | | Doing this I learned e.message was pretty shortlived, added in 2.6, they realized it was a bad idea and deprecated it in 2.7 Signed-off-by: Adrian Tschira <nota@notafile.com>
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Merge branch 'develop' into developRichard van der Hoff2017-09-251-124/+161
|\
| * Fix logcontext handling in verify_json_objects_for_serverRichard van der Hoff2017-09-201-8/+12
| | | | | | | | | | | | | | | | | | | | | | | | preserve_context_over_fn is essentially broken, because (a) it pointlessly drops the current logcontext before calling its wrapped function, which means we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the resulting deferred in a _PreservingContextDeferred, which is very dangerous because you then can't yield on it without leaking context back into the reactor. Instead, let's specify that the resultant deferreds call their callbacks with no logcontext.
| * Turn _start_key_lookups into an inlineCallbacks functionRichard van der Hoff2017-09-201-40/+37
| | | | | | | | | | | | | | | | | | | | | | ... which means that logcontexts can be correctly preserved for the stuff it does. get_server_verify_keys is now called with the logcontext, so needs to preserve_fn when it fires off its nested inlineCallbacks function. Also renames get_server_verify_keys to reflect the fact it's meant to be private.
| * Fix potential race in _start_key_lookupsRichard van der Hoff2017-09-201-5/+8
| | | | | | | | | | | | | | | | | | If the verify_request.deferred has already completed, then `remove_deferreds` will be called immediately. It therefore might resolve the server_to_deferred deferred while there are still other requests for that server in flight. To avoid that, we should build the complete list of requests, and *then* add the callbacks.
| * Add some comments to _start_key_lookupsRichard van der Hoff2017-09-201-0/+8
| |
| * Consistency for how verify_request.deferred is calledRichard van der Hoff2017-09-201-13/+17
| | | | | | | | | | | | | | | | | | | | Define that it is run with no log context, and make sure that happens. If we aren't careful to reset the logcontext, we can't bung the deferreds into defer.gatherResults etc. We don't actually do that directly, but we *do* resolve other deferreds from affected callbacks (notably the server_to_deferred map in _start_key_lookups), and those *do* get passed into defer.gatherResults. It turns out that this way ends up being least confusing.
| * Factor out _start_key_lookupsRichard van der Hoff2017-09-201-7/+17
| | | | | | | | ... to make it easier to see what's going on.
| * Replace `server_and_json` with `verify_requests`Richard van der Hoff2017-09-201-3/+3
| | | | | | | | This is a precursor to factoring some of this code out.
| * pull out handle_key_deferred to top levelRichard van der Hoff2017-09-201-43/+44
| | | | | | | | | | | | There's no need for this to be a nested definition; pulling it out not only makes it more efficient, but makes it easier to check that it's not accessing any local variables it shouldn't be.
| * Fix incorrect key_ids in error messageRichard van der Hoff2017-09-201-1/+1
| |
| * Fix concurrent server_key requests (#2458)Richard van der Hoff2017-09-191-1/+3
| | | | | | | | Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.
| * Clean up and document handling of logcontexts in Keyring (#2452)Richard van der Hoff2017-09-181-28/+36
| | | | | | | | | | | | I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls...
* | Fix iteration of requests_missing_keys; list doesn't have .values()Kenny Keslar2017-07-261-1/+1
|/ | | | Signed-off-by: Kenny Keslar <r3dey3@r3dey3.com>
* Merge pull request #2050 from matrix-org/rav/federation_backoffRichard van der Hoff2017-03-231-23/+16
|\ | | | | push federation retry limiter down to matrixfederationclient
| * push federation retry limiter down to matrixfederationclientRichard van der Hoff2017-03-231-23/+16
| | | | | | | | | | rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.
* | fix up some key verif docstringsRichard van der Hoff2017-03-211-2/+21
|/
* Add some debug to help diagnose weird federation issueRichard van der Hoff2017-03-201-1/+8
|
* Preserve some logcontextsErik Johnston2016-08-241-18/+18
|
* Update keyring MeasureErik Johnston2016-08-191-85/+85
|
* Measure keyringsErik Johnston2016-08-191-34/+36
|
* Don't print stack traces when failing to get remote keysErik Johnston2016-08-101-12/+16
|
* Merge branch 'develop' into markjh/verifyMark Haines2016-07-271-2/+9
|\ | | | | | | | | Conflicts: synapse/crypto/keyring.py
| * Merge pull request #955 from matrix-org/markjh/only_from2Mark Haines2016-07-271-2/+9
| |\ | | | | | | Add a couple more checks to the keyring
| | * Add a couple more checks to the keyringMark Haines2016-07-261-2/+9
| | |
| * | Fix a couple of bugs in the transaction and keyring codeMark Haines2016-07-261-8/+9
| |/
* / Clean up verify_json_objects_for_serverMark Haines2016-07-271-68/+75
|/
* Fix up logcontextsErik Johnston2016-02-081-37/+46
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Actually look up required remote server key IDsPaul "LeoNerd" Evans2015-12-181-1/+3
| | | | | | set.union() is a side-effect-free function that returns the union of two sets. This clearly wanted .update(), which is the side-effecting mutator version.
* Fix typoErik Johnston2015-11-201-1/+1
|
* Don't limit connections to perspective serversErik Johnston2015-11-201-21/+17
|
* Fix bug where we sometimes didn't fetch all the keys requested for aErik Johnston2015-09-171-4/+3
| | | | server.
* Merge pull request #272 from matrix-org/daniel/insecureclientDaniel Wagner-Hall2015-09-151-2/+2
|\ | | | | Allow configuration to ignore invalid SSL certs
| * Allow configuration to ignore invalid SSL certsDaniel Wagner-Hall2015-09-091-2/+2
| | | | | | | | | | This will be useful for sytest, and sytest only, hence the aggressive config key name.
* | Various bug fixes to crypto.keyringErik Johnston2015-09-091-10/+17
|/
* Remove syutil dependency in favour of smaller single-purpose librariesMark Haines2015-08-241-9/+9
|
* Wait for previous attempts at fetching keys for a given server before trying ↵Erik Johnston2015-06-261-15/+68
| | | | to fetch more
* Implement bulk verify_signed_json APIErik Johnston2015-06-261-134/+292
|
* Handle the case when things return empty but non none thingsErik Johnston2015-05-191-2/+2
|
* Don't always hit get_server_verify_key_v1_directErik Johnston2015-05-191-5/+10
|
* SYN-383: Actually, we expect this value to be a dictErik Johnston2015-05-191-1/+2
|
* SYN-383: Handle the fact the server might not have signed thingsErik Johnston2015-05-191-1/+1
|
* Don't reuse var namesErik Johnston2015-05-191-2/+2
|
* SYN-383: Fix parsing of verify_keys and catching of _DefGen_ReturnErik Johnston2015-05-191-18/+18
|
* SYN-383: Extract the response list from 'server_keys' in the response JSON ↵Mark Haines2015-05-191-1/+3
| | | | as it might work better than iterating over the top level dict
* Change the way we create observers to deferreds so that we don't get spammed ↵Erik Johnston2015-05-081-2/+6
| | | | by 'unhandled errors'
* Use a defer.gatherResults to collect results from the perspective serversMark Haines2015-04-291-11/+21
|
* Update the query format used by keyring to match current key v2 specMark Haines2015-04-291-1/+12
|
* Implement minimum_valid_until_ts in the remote key resourceMark Haines2015-04-291-0/+1
|
* Merge branch 'develop' into key_distributionMark Haines2015-04-271-0/+20
|\ | | | | | | | | Conflicts: synapse/crypto/keyring.py
| * Fix newlinesErik Johnston2015-04-271-2/+1
| |
| * Pull inner function out.Erik Johnston2015-04-271-76/+77
| |
| * Implement locks using create_observer for fetching media and server keysErik Johnston2015-04-271-59/+79
| |
* | Add config for setting the perspective serversMark Haines2015-04-241-1/+5
| |
* | Update to match the specification for key/v2Mark Haines2015-04-231-2/+2
| |
* | Implement remote key lookup apiMark Haines2015-04-221-35/+40
| |
* | Implement v2 key lookupMark Haines2015-04-201-17/+251
|/
* Log error message when we fail to fetch remote server keysErik Johnston2015-03-051-2/+11
|
* Try to only back off if we think we failed to connect to the remoteErik Johnston2015-02-171-54/+54
|
* Add per server retry limiting.Erik Johnston2015-02-171-15/+7
| | | | | Factor out the pre destination retry logic from TransactionQueue so it can be reused in both get_pdu and crypto.keyring
* Rate limit retries when fetching server keys.Erik Johnston2015-02-171-55/+71
|
* Update copyright noticesMark Haines2015-01-061-1/+1
|
* Add a few missing yields, Move deferred lists inside PreserveLoggingContext ↵Mark Haines2014-11-201-2/+2
| | | | because they don't interact well with the logging contexts
* Fix pep8 warningsMark Haines2014-10-301-2/+2
|
* add log line for checking verifying signaturesMark Haines2014-10-171-0/+1
|
* Better response message when signature is missing or unsupportedMark Haines2014-10-131-1/+1
|
* Respond with more helpful error messages for unsigned requestsMark Haines2014-10-131-2/+31
|
* SYN-75 Verify signatures on server to server transactionsMark Haines2014-09-301-0/+125
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-25 09:59:45 +0000