summary refs log tree commit diff
path: root/synapse/util/async.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2017-04-07When we do an invite rejection, save the signed leave event to the dbRichard van der Hoff1-4/+4
During a rejection of an invite received over federation, we ask a remote server to make us a `leave` event, then sign it, then send that with `send_leave`. We were saving the *unsigned* version of the event (which has a different event id to the signed version) to our db (and sending it to the clients), whereas other servers in the room will have seen the *signed* version. We're not aware of any actual problems that caused, except that it makes the database confusing to look at and generally leaves the room in a weird state.
2017-04-07Bump version and changelogErik Johnston2-1/+10
2017-04-03Accept join events from all serversRichard van der Hoff2-5/+19
Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).
2017-03-30Bump version and changelog v0.20.0-rc1Erik Johnston2-1/+45
2017-03-30Fix the logcontext handling in the cache wrappers (#2077)Richard van der Hoff4-19/+136
The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.
2017-03-30Use txn.fetchall() so we can reuse txnErik Johnston1-1/+1
2017-03-29synctl.py: wait for synapse to stop before restarting (#2020)Anant Prakash1-4/+43
2017-03-29Cache glob to regex at a higher level for pushErik Johnston1-47/+57
2017-03-29Increase cache size for _get_state_group_for_eventErik Johnston1-1/+1
2017-03-29Bail early if remote wouldn't be retried (#2064)Erik Johnston1-2/+8
* Bail early if remote wouldn't be retried * Don't always return true * Just use get_retry_limiter * Spelling
2017-03-29Decrank last commitErik Johnston1-4/+5
2017-03-29Correctly look up keyErik Johnston1-6/+5
2017-03-28The algorithm is part of the key idErik Johnston1-4/+4
2017-03-27TypoErik Johnston1-1/+1
2017-03-27Short circuit if all new events have same state groupErik Johnston1-36/+51
2017-03-27Notify on new federation trafficErik Johnston1-0/+7
2017-03-24Batch sending of device list pokesErik Johnston2-0/+5
2017-03-24Actually call invalidateErik Johnston1-1/+1
2017-03-24Use presence replication stream to invalidate cacheErik Johnston2-2/+3
Instead of using the cache invalidation replication stream to invalidate the _get_presence_cache, we can instead rely on the presence replication stream. This reduces the amount of replication traffic considerably.
2017-03-24Add slave transaction storeErik Johnston2-0/+4
2017-03-24Cache hosts in roomErik Johnston2-2/+11
2017-03-24Replace some calls to cursor_to_dictErik Johnston3-39/+13
cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.
2017-03-24Use iter(items|values)Erik Johnston3-51/+48
2017-03-23try not to drop context after federation requestsRichard van der Hoff1-12/+16
preserve_context_over_fn uses a ContextPreservingDeferred, which only restores context for the duration of its callbacks, which isn't really correct, and means that subsequent operations in the same request can end up without their logcontexts.
2017-03-23Add another missing yield on check_device_registeredRichard van der Hoff1-1/+1
2017-03-23Fix rejection of invites not reaching syncDavid Baker1-0/+7
Always allow the user to see their own leave events, otherwise they won't see the event if they reject an invite for a room whose history visibility is set such that they cannot see events before joining.
2017-03-23Oops, remove unintentional changeDavid Baker1-7/+0
2017-03-23Fix token request for addition of phone numbersDavid Baker2-1/+8
2017-03-23User Cursor.__iter__ instead of fetchallErik Johnston16-41/+40
This prevents unnecessary construction of lists
2017-03-23Add jsonschema to python_dependencies.pypik1-0/+1
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23use jsonschema.FormatChecker for RoomID and UserID stringspik3-24/+40
* use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23Add valid filter tests, flake8, fix typopik2-10/+55
Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23check_valid_filter using JSONSchemapik2-94/+175
* add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>
2017-03-23Raise a more helpful exceptionErik Johnston1-2/+4
2017-03-23fix testsRichard van der Hoff2-1/+3
2017-03-23Don't user upsert to persist new one time keysErik Johnston1-9/+48
Instead we no-op duplicate one time key uploads, an error if the key_id already exists but encodes a different key.
2017-03-23Ignore backoff history for invites, aliases, and roomdirsRichard van der Hoff6-14/+52
Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.
2017-03-23Fix time_bound_deferred to throw the right exceptionRichard van der Hoff2-4/+39
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.
2017-03-23push federation retry limiter down to matrixfederationclientRichard van der Hoff8-287/+280
rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23MatrixFederationHttpClient: clean upRichard van der Hoff1-17/+24
rename _create_request to _request, and push ascii-encoding of `destination` and `path` down into it
2017-03-23Fix a couple of logcontext leaksRichard van der Hoff2-5/+5
Use preserve_fn to correctly manage the logcontexts around things we don't want to yield on.
2017-03-23Add a missing yield in device key uploadRichard van der Hoff1-1/+1
(this would only very very rarely actually be a useful thing, so the main problem was the logcontext leak...)
2017-03-22Removed requirement that is not neededRichard Kellner1-1/+1
I have removed libsodium from CentOS system requirements, as it is part PyNaCl. Signed-off-by: Richard Kellner <richard.kellner@gmail.com>
2017-03-22Fix caching of remote servers' signature keysRichard van der Hoff4-63/+225
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
2017-03-21fix up some key verif docstringsRichard van der Hoff2-4/+24
2017-03-20Updated user creation sectionRichard Kellner1-0/+1
register_new_matrix_user command has one more question, I have updated the documentation to match the reality.
2017-03-20Added missing system requiremnt and pip upgrade before installRichard Kellner1-1/+2
When installing on CentOS7 I wans't able to follow README instructions to install due to errors. I was missing libsodium in order to compile python dependencies. Default version of Python pip is really old and therefore setuptools upgrade ended with error as well. In order to be able to continue I needed to upgrade pip as well.
2017-03-20Bump changelog and version release-v0.19.3Erik Johnston2-1/+7
2017-03-20Add some debug to help diagnose weird federation issueRichard van der Hoff1-1/+8
2017-03-20Fix unit testErik Johnston1-1/+1
2017-03-20PEP8Erik Johnston2-3/+0
2017-03-20log_contexts.rst: fix formatting of Note blockRichard van der Hoff1-6/+6
Apparently the github RST renderer doesn't like Note blocks.
2017-03-18log_contexts.rst: fix typosRichard van der Hoff1-2/+2
2017-03-18README.md: fix link to client list on matrix.org/docsStefan Majewsky1-3/+3
2017-03-18Stop preserve_fn leaking context into the reactorRichard van der Hoff3-34/+93
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.
2017-03-18Run the reactor with the sentinel logcontextRichard van der Hoff8-16/+51
This fixes a class of 'Unexpected logcontext' messages, which were happening because the logcontext was somewhat arbitrarily swapping between the sentinel and the `run` logcontext.
2017-03-18Remove broken use of clock.call_laterRichard van der Hoff1-12/+3
background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.
2017-03-17Logcontext docsRichard van der Hoff2-8/+449
2017-03-17Don't send the full event json over replicationErik Johnston4-50/+38
2017-03-17Avoid resetting state on rejected eventsRichard van der Hoff2-4/+14
When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.
2017-03-17Refactoring and cleanupsRichard van der Hoff5-80/+264
A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
2017-03-16Update README: specify python2.7 in virtualenvKeyvan Fatehi1-1/+1
Signed-off-by: Keyvan Fatehi <keyvanfatehi@gmail.com>
2017-03-16CommentsErik Johnston2-2/+4
2017-03-16CommentErik Johnston1-3/+8
2017-03-16Don't recreate so many setsErik Johnston10-51/+42
2017-03-16Implement no op for room stream in syncErik Johnston1-4/+47
2017-03-15Queue up federation PDUs while a room join is in progressRichard van der Hoff1-24/+44
This just takes the existing `room_queues` logic and moves it out to `on_receive_pdu` instead of `_process_received_pdu`, which ensures that we don't start trying to fetch prev_events and whathaveyou until the join has completed.
2017-03-15CommentErik Johnston1-1/+3
2017-03-15Modify test_user_sync so it doesn't look at last_active_ts over ↵Will Hunt1-1/+1
last_user_sync_ts
2017-03-15Cache set of users whose presence the other user should seeErik Johnston1-13/+27
2017-03-15Add a great comment to handle_timeout for active vs sync times.Will Hunt1-0/+2
2017-03-15Add fallback to last_active_ts if it beats the last sync time.Will Hunt1-1/+2
2017-03-15CommentErik Johnston2-0/+6
2017-03-15Remove unused importErik Johnston1-1/+0
2017-03-15Format presence events on the edges instead of reformatting them multiple timesErik Johnston7-39/+80
2017-03-15Fix assertion to stop transaction queue getting wedgedRichard van der Hoff6-5/+29
... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop.
2017-03-15Add some metrics on notifierErik Johnston1-0/+6
2017-03-15Implement reset_context_after_deferredRichard van der Hoff1-0/+25
to correctly reset the context when we fire off a deferred we aren't going to wait for.
2017-03-14Reduce spurious calls to generate syncErik Johnston1-13/+31
2017-03-14Address review commentsRichard van der Hoff2-20/+31
- don't blindly proxy all HTTPRequestExceptions - log unexpected exceptions at error - avoid `isinstance` - improve docs on `from_http_response_exception`
2017-03-14re-refactor exception heirarchyRichard van der Hoff1-16/+14
Give CodeMessageException back its `msg` attribute, and use that to hold the HTTP status message for HttpResponseException.
2017-03-14Handle PartialDownloadError in CAS loginDavid Baker1-1/+8
2017-03-14Fix current_state_events table to not lieErik Johnston1-3/+28
If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.
2017-03-14Use extend instead of +=David Baker1-4/+4
2017-03-14Oops, remove printDavid Baker1-1/+0
2017-03-13Implement _simple_delete_many_txn, use it to delete devicesLuke Barnard4-11/+101
(But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.
2017-03-13Fix registration for broken clientsDavid Baker1-4/+22
Only offer msisdn flows if the x_show_msisdn option is given.
2017-03-13Support registration / login with phone numberDavid Baker9-50/+395
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13FlakeLuke Barnard1-0/+2
2017-03-13Implement delete_devices APILuke Barnard1-0/+47
This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.
2017-03-13Fix routing loop when fetching remote mediaRichard van der Hoff4-14/+102
When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502.
2017-03-13Add new storage function to slave storeErik Johnston1-0/+4
2017-03-13Bring example log config into line with defaultRichard van der Hoff1-2/+4
2017-03-13Add helpful texts to logger config optionsRichard van der Hoff1-5/+7
2017-03-13CommentsErik Johnston1-0/+12
2017-03-13Bump changelog and versions v0.19.3-rc2Erik Johnston2-1/+10
2017-03-13Revert "Support registration & login with phone number"Erik Johnston9-395/+50
2017-03-13Assume rooms likely haven't changedErik Johnston2-8/+14
2017-03-10Get current state by using current_state_events tableErik Johnston5-31/+52
2017-03-10Add an option to disable stdio redirectRichard van der Hoff1-1/+12
This makes it tractable to run synapse under pdb.
2017-03-10Refactor logger config for workersRichard van der Hoff9-13/+26
- to make it easier to add more config options.
2017-03-10Reread log config on SIGHUPRichard van der Hoff1-10/+19
When we are using a log_config file, reread it on SIGHUP.
2017-03-10Fix it for the workersErik Johnston1-0/+8
2017-03-10Noop repated delete device inbox calls from /syncErik Johnston2-4/+38
2017-03-10Add setdefault key to ExpiringCacheErik Johnston1-0/+7
2017-03-09Move FederationServer._handle_new_pdu to FederationHandlerRichard van der Hoff2-198/+198
Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09Move sig check out of _handle_new_pduRichard van der Hoff1-12/+36
When we receive PDUs via `get_missing_events`, we have already checked their sigs, so there is no need to do it again.
2017-03-09Factor _get_missing_events_for_pdu out of _handle_new_pduRichard van der Hoff1-62/+82
This should be functionally identical: it just seeks to improve readability by reducing indentation.
2017-03-09Fix docstringRichard van der Hoff1-2/+1
2017-03-08Comment when our 3pids would be incompleteDavid Baker1-0/+2
2017-03-08DocsDavid Baker1-0/+13
2017-03-08Incorrectly copied copyrightDavid Baker1-1/+0
This file post-dates OM
2017-03-08Not any more, it doesn'tDavid Baker1-1/+0
2017-03-08Remove spurious SQL logging (#1972)Richard van der Hoff1-8/+0