summary refs log tree commit diff
path: root/synapse/notifier.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Wire up tokenErik Johnston2020-09-291-2/+2
|
* Add EventStreamPosition type (#8388)Erik Johnston2020-09-241-26/+29
| | | | | | | | | | | | | | The idea is to remove some of the places we pass around `int`, where it can represent one of two things: 1. the position of an event in the stream; or 2. a token that partitions the stream, used as part of the stream tokens. The valid operations are then: 1. did a position happen before or after a token; 2. get all events that happened before or after a token; and 3. get all events between two tokens. (Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
* Use slots in attrs classes where possible (#8296)Patrick Cloke2020-09-141-3/+1
| | | | | slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.
* Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281)Erik Johnston2020-09-111-4/+12
|
* Clean up `Notifier.on_new_room_event` code path (#8288)Erik Johnston2020-09-101-23/+39
| | | | | | | | | | | | | The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it. This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens. The valid operations here are: 1. Is a position before or after a token 2. Fetching all events between two tokens 3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C. Future PR will change the token type to a dedicated type.
* Clean up types for PaginationConfig (#8250)Erik Johnston2020-09-081-2/+3
| | | This removes `SourcePaginationConfig` and `get_pagination_rows`. The reasoning behind this is that these generic classes/functions erased the types of the IDs it used (i.e. instead of passing around `StreamToken` it'd pass in e.g. `token.room_key`, which don't have uniform types).
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-3/+3
|
* Fix typing for notifier (#8064)Erik Johnston2020-08-121-4/+8
|
* Add typing info to Notifier (#8058)Erik Johnston2020-08-111-48/+83
|
* Convert streams to async. (#8014)Patrick Cloke2020-08-041-2/+2
|
* Fix some spelling mistakes / typos. (#7811)Patrick Cloke2020-07-091-1/+1
|
* Fix new flake8 errors (#7470)Erik Johnston2020-05-121-4/+6
|
* Convert the synapse.notifier module to async/await. (#7395)Patrick Cloke2020-05-011-10/+7
|
* Don't relay REMOTE_SERVER_UP cmds to same conn. (#7352)Erik Johnston2020-04-291-9/+0
| | | | | | | | | | | | | | For direct TCP connections we need the master to relay REMOTE_SERVER_UP commands to the other connections so that all instances get notified about it. The old implementation just relayed to all connections, assuming that sending back to the original sender of the command was safe. This is not true for redis, where commands sent get echoed back to the sender, which was causing master to effectively infinite loop sending and then re-receiving REMOTE_SERVER_UP commands that it sent. The fix is to ensure that we only relay to *other* connections and not to the connection we received the notification from. Fixes #7334.
* Wake up transaction queue when remote server comes back online (#6706)Erik Johnston2020-01-171-3/+28
| | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up.
* Port SyncHandler to async/awaitErik Johnston2019-12-051-16/+13
|
* Port to use state storageErik Johnston2019-10-301-1/+5
|
* Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-3/+3
| | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* Replace returnValue with return (#5736)Amber Brown2019-07-231-9/+9
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-2/+2
|
* Run Black. (#5482)Amber Brown2019-06-201-34/+36
|
* Remove redundant PreserveLoggingContextRichard van der Hoff2019-03-041-10/+5
| | | | | Both (!) things that register as replication listeners do the right thing wrt logcontexts, so this is redundant.
* Remove unused `wait_for_replication` methodRichard van der Hoff2019-03-041-50/+0
| | | | I guess this was used once? It's not now, anyway.
* Make the metrics less racy (#4061)Amber Brown2018-10-191-3/+3
|
* Run notify_app_services as a bg processRichard van der Hoff2018-09-261-2/+6
| | | | | | | | This ensures that its resource usage metrics get recorded somewhere rather than getting lost. (It also fixes an error when called from a nested logging context which completes before the bg process)
* FixupErik Johnston2018-09-191-4/+1
|
* Replace custom DeferredTimeoutError with defer.TimeoutErrorErik Johnston2018-09-191-3/+2
|
* Update to use new timeout function everywhere.Erik Johnston2018-09-191-6/+7
| | | | | | | The existing deferred timeout helper function (and the one into twisted) suffer from a bug when a deferred's canceller throws an exception, #3842. The new helper function doesn't suffer from this problem.
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-1/+1
|
* WIP to announce deleted devices over federationMatthew Hodgson2018-07-121-1/+1
| | | | | | Previously we queued up the poke correctly when the device was deleted, but then the actual EDU wouldn't get sent, as the device was no longer known. Instead, we now send EDUs for deleted devices too if there's a poke for them.
* run isortAmber Brown2018-07-091-11/+10
|
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-0/+3
|
* cleanup pep8 errorsAmber Brown2018-05-221-1/+2
|
* fixesAmber Brown2018-05-221-3/+3
|
* replacing portionsAmber Brown2018-05-211-13/+10
|
* Merge remote-tracking branch 'origin/develop' into rav/deferred_timeoutRichard van der Hoff2018-04-271-4/+9
|\
| * Improve exception handling for background processesRichard van der Hoff2018-04-271-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.
* | Backport deferred.addTimeoutRichard van der Hoff2018-04-271-8/+14
| | | | | | | | Twisted 16.0 doesn't have addTimeout, so let's backport it.
* | Use deferred.addTimeout instead of time_bound_deferredRichard van der Hoff2018-04-231-12/+11
|/ | | | This doesn't feel like a wheel we need to reinvent.
* add __bool__ alias to __nonzero__ methodsAdrian Tschira2018-04-151-0/+1
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Clear logcontext before starting fed txn queue runnerRichard van der Hoff2017-11-281-7/+10
| | | | | These processes take a long time compared to the request, so there is lots of "Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Don't start user_directory handling on workersErik Johnston2017-06-071-3/+0
|
* Add user_directory to databaseErik Johnston2017-05-311-1/+5
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-04-041-13/+10
|\ | | | | | | erikj/repl_tcp_server
| * Remove spurious PreserveLoggingContextRichard van der Hoff2017-04-031-6/+5
| | | | | | | | | | In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its subroutines with the logcontext set.
| * preserve_fn some deferred-returning thingsRichard van der Hoff2017-04-031-2/+5
| | | | | | | | | | In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which return deferreds, so that it is safe to call it with an active logcontext.
| * Remove spurious @preserve_fn decoratorsRichard van der Hoff2017-04-031-5/+0
| | | | | | | | | | | | | | | | Remove `@preserve_fn` decorators on `on_new_room_event`, `_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and `on_new_replication_data` - none of these functions return a deferred, and the decorator does nothing unless the wrapped function returns a deferred, so the decorator was a no-op.
* | Use callbacks to notify tcp replication rather than deferredsErik Johnston2017-03-311-6/+11
| |
* | Add a simple hook to wait for replication trafficErik Johnston2017-03-301-0/+6
|/
* Don't recreate so many setsErik Johnston2017-03-161-4/+2
|
* Format presence events on the edges instead of reformatting them multiple timesErik Johnston2017-03-151-0/+10
|
* Add some metrics on notifierErik Johnston2017-03-151-0/+6
|
* Reduce spurious calls to generate syncErik Johnston2017-03-141-13/+31
|
* Make presence.get_new_events a bit fasterErik Johnston2017-02-021-0/+1
| | | | We do this by caching the set of users a user shares rooms with.
* Correctly handle timeout errorsErik Johnston2016-12-091-6/+14
|
* Fix rare notifier bug where listeners dont timeoutErik Johnston2016-12-091-11/+9
| | | | | | There was a race condition that caused the notifier to 'miss' the timeout notification, since there were no other checks for the timeout this caused listeners to get stuck in a loop until something happened.
* Ensure only main or federation_sender process can send federation trafficErik Johnston2016-11-231-2/+9
|
* Handle sending events and device messages over federationErik Johnston2016-11-171-0/+2
|
* Explicitly specify state_key for history_visibility fetchingErik Johnston2016-09-021-1/+2
|
* Preserve some logcontextsErik Johnston2016-08-241-1/+6
|
* defer.returnValue must not be called within MeasureErik Johnston2016-08-191-2/+1
|
* Add measure blocks to notifierErik Johnston2016-08-191-15/+18
|
* Make AppserviceHandler stream events from databaseErik Johnston2016-08-181-1/+1
| | | | | | | | | This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.
* Remove dead appservice codeErik Johnston2016-08-171-39/+2
|
* Notify users for events in rooms they join.Mark Haines2016-06-071-8/+6
| | | | | | Change how the notifier updates the map from room_id to user streams on receiving a join event. Make it update the map when it notifies for the join event, rather than using the "user_joined_room" distributor signal
* Move the AS handler out of the Handlers object.Mark Haines2016-05-311-6/+4
| | | | | | Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.
* move filter_events_for_client out of base handlerMark Haines2016-05-111-2/+3
|
* Run filter_events_for_clientDavid Baker2016-04-271-1/+1
| | | | so we don't accidentally mail out events people shouldn't see
* Use google style doc strings.Mark Haines2016-04-011-7/+8
| | | | | | | pycharm supports them so there is no need to use the other format. Might as well convert the existing strings to reduce the risk of people accidentally cargo culting the wrong doc string format.
* Hook up adding a pusher to the notifier for replication.Mark Haines2016-03-151-0/+6
|
* Hook up the push rules to the notifierMark Haines2016-03-031-1/+1
|
* Add a /replication API for extracting the updates that happened onMark Haines2016-03-011-0/+48
| | | | | | | | | | | | | | | | | | | | | | | | synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.
* Fix up logcontextsErik Johnston2016-02-081-29/+29
|
* Fix flake8 warnings for new flake8Daniel Wagner-Hall2016-02-021-1/+1
|
* Don't use before_token. Its wrong. Use actual limit.Erik Johnston2016-01-311-6/+3
|
* Deal with None limitErik Johnston2016-01-291-1/+5
|
* Make /events always return a newer token, if one existsErik Johnston2016-01-291-9/+10
|
* Only fetch events for rooms and receiptsErik Johnston2016-01-211-3/+3
|
* SYN-606: Peeking does not wake up /eventsErik Johnston2016-01-211-1/+3
| | | | | | If a real user attempted to first peek into one room, and then another, their room event stream would not be woken up for events in the later room.
* Allow non-guests to peek on rooms using /eventsDaniel Wagner-Hall2016-01-201-23/+48
|
* Skip, rather than erroring, invalid guest requestsDaniel Wagner-Hall2016-01-051-2/+1
| | | | | | | | | | Erroring causes problems when people make illegal requests, because they don't know what limit parameter they should pass. This is definitely buggy. It leaks message counts for rooms people don't have permission to see, via tokens. But apparently we already consciously decided to allow that as a team, so this preserves that behaviour.
* Missing yield on guest access auth checkMark Haines2015-12-231-1/+1
| | | | Needs matrix-org/sytest#125 to land first
* Return non-room events from guest /events callsDaniel Wagner-Hall2015-11-121-3/+17
|
* Open up /events to anonymous users for room events onlyDaniel Wagner-Hall2015-11-051-8/+34
| | | | Squash-merge of PR #345 from daniel/anonymousevents
* Merge branch 'develop' into daniel/removesomeliesMark Haines2015-11-041-1/+1
|\ | | | | | | | | Conflicts: synapse/notifier.py
| * Inform the client of new room tags using v1 /eventsMark Haines2015-10-291-1/+1
| |
* | Remove unused arguments and codeDaniel Wagner-Hall2015-11-021-3/+3
|/
* Don't make pushers handle presence/typing eventsErik Johnston2015-08-241-1/+6
|
* Improve namingDaniel Wagner-Hall2015-07-201-2/+2
|
* Consolidate duplicate code in notifierErik Johnston2015-07-021-24/+11
|
* Add receipts_key to StreamTokenErik Johnston2015-07-021-1/+1
|
* Fix typoErik Johnston2015-06-191-1/+1
|
* PEP8Erik Johnston2015-06-181-2/+3
|
* DocumentationErik Johnston2015-06-181-4/+11
|
* Don't bother proxying lookups on _NotificationListener to underlying deferredErik Johnston2015-06-181-10/+2
|
* Refactor the notifier.wait_for_events code to be clearer. Add ↵Erik Johnston2015-06-181-66/+56
| | | | _NotifierUserStream.new_listener that accpets a token to avoid races.
* Fix notifier leakErik Johnston2015-06-181-21/+20
|
* Iterate over the user_streams not the user_idsMark Haines2015-05-261-1/+1
|
* Oops, get_rooms_for_user returns a namedtuple, not a room_idMark Haines2015-05-211-0/+1
|
* Don't bother sorting by the room_stream_ids, it shouldn't matter which order ↵Mark Haines2015-05-181-1/+1
| | | | they are notified in
* Make sure the notifier stream token goes forward when it is updated. Sort ↵Mark Haines2015-05-181-4/+4
| | | | the pending events by the correct room_stream_id
* Add some doc-strings to notifierMark Haines2015-05-141-19/+33
|
* Use the current token when timing out a notifier, make sure the user_id is a ↵Mark Haines2015-05-141-2/+2
| | | | string in on_new_user_event
* Fix v2 sync, update the last_notified_ms only if there was an active listenerMark Haines2015-05-141-5/+6
|
* Fix metric counterMark Haines2015-05-131-2/+2
|
* Discard unused NotifierUserStreamsMark Haines2015-05-131-16/+34
|
* Don't set a timer if there's already a result to returnMark Haines2015-05-131-3/+10
|
* Don't bother checking for new events from a source if the stream token ↵Mark Haines2015-05-131-2/+6
| | | | hasn't advanced for that source
* Don't bother checking for updates if the stream token hasn't advanced for a userMark Haines2015-05-131-24/+51
|
* Merge branch 'notifier_unify' into notifier_performanceMark Haines2015-05-121-13/+10
|\ | | | | | | | | Conflicts: synapse/notifier.py
| * Merge branch 'develop' into notifier_unifyMark Haines2015-05-121-13/+10
| |\ | | | | | | | | | | | | Conflicts: synapse/notifier.py
| | * PEP8Erik Johnston2015-05-081-3/+2
| | |
| | * Change the way we do logging contexts so that they survive divergencesErik Johnston2015-05-081-9/+7
| | |
* | | Merge branch 'notifier_unify' into notifier_performanceMark Haines2015-05-121-1/+1
|\| |
| * | Update the end_token correctly, otherwise the token doesn't advance and the ↵Mark Haines2015-05-121-1/+1
| | | | | | | | | | | | client gets duplicate events
* | | Add a NotifierUserStream to hold all the notification listeners for a userMark Haines2015-05-121-114/+116
|/ /
* | Don't bother passing the events to the notifier since it isn't using themMark Haines2015-05-111-95/+18
| |
* | Use wait_for_events to implement 'get_events'Mark Haines2015-05-111-79/+30
|/
* Remove some run_on_reactorsErik Johnston2015-05-011-5/+0
|
* Change from exception to warnErik Johnston2015-04-151-1/+1
|
* Add commentMark Haines2015-04-091-0/+1
|
* Unset the timer in the timeout callback so that we don't try to cancel it if ↵Mark Haines2015-04-091-0/+1
| | | | it has been called
* SYN-339: Cancel the notifier timeout when the notifier firesMark Haines2015-04-091-4/+26
|
* Move comment into docstringErik Johnston2015-04-081-5/+2
|
* Factor out loops into '_discard_if_notified'Erik Johnston2015-04-081-15/+14
|
* Also perform paranoia checks in 'on_new_user_event'Erik Johnston2015-04-081-2/+16
|
* Add paranoia checks to make sure that we evict stale NotificationListeners ↵Erik Johnston2015-04-081-5/+31
| | | | when we are about to process them
* Fix bug where we didn't inform the NotificataionListeners about new rooms ↵Erik Johnston2015-04-081-0/+3
| | | | they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts
* Appease pep8Paul "LeoNerd" Evans2015-03-121-6/+9
|
* Add a counter to track total number of events served by the notifierPaul "LeoNerd" Evans2015-03-121-0/+3
|
* Use _ instead of . as a metric namespacing separator, for PrometheusPaul "LeoNerd" Evans2015-03-121-1/+1
|
* Put some gauge metrics on the number of notifier listeners, and notified-on ↵Paul "LeoNerd" Evans2015-03-121-0/+39
| | | | objects (users, rooms, appservices)
* Rename rooms_to_listeners to room_to_listeners, for consistency with user_ ↵Paul "LeoNerd" Evans2015-03-051-8/+8
| | | | and appservice_*
* Notify appservices of invites mid-poll.Kegan Dougal2015-02-271-1/+29
| | | | | | | | This requires the notifier to have knowledge of appservice listeners so it can do the regex checks on incoming invites to see if the state_key matches. It isn't enough to just rely on the room listeners and store.get_app_service_rooms as the room will initially not exist or won't be on the ASes radar due to having none of its users in the room.
* Merge branch 'consumeErrors' of github.com:matrix-org/synapse into developErik Johnston2015-02-171-2/+4
|\
| * Use consumeErrors=True on all DeferredLists.Erik Johnston2015-02-171-2/+4
| | | | | | | | | | | | This is so that the DeferredLists actually consume the error instead of propogating down the non-existent errback chain. This should reduce the number of unhandled errors we are seeing.
* | Glue AS work to general event notifications. Add more exception handling ↵Kegan Dougal2015-02-051-0/+6
|/ | | | when poking ASes.
* Fix FormattingMark Haines2015-01-271-2/+2
|
* Wait for events if the incremental sync is empty and a timeout is givenMark Haines2015-01-271-0/+48
|
* Only start the notifier timeout once we've had a chance to check for ↵Mark Haines2015-01-191-2/+2
| | | | updates. Otherwise the timeout could fire while we are waiting for the database to return any updates it might have
* Update copyright noticesMark Haines2015-01-061-1/+1
|
* Remember to hook up the typing event stream to the notifier as wellPaul "LeoNerd" Evans2014-12-151-7/+25
|
* Fix pep8 codestyle warningsMark Haines2014-11-201-0/+1
|
* Add a few missing yields, Move deferred lists inside PreserveLoggingContext ↵Mark Haines2014-11-201-0/+3
| | | | because they don't interact well with the logging contexts
* remove unused importMark Haines2014-11-191-1/+1
|
* Preserve logging context in a few more places, drop the logging context ↵Mark Haines2014-11-191-18/+22
| | | | after it has been stashed to reduce potential for confusion
* Fix pep8 warningsMark Haines2014-10-301-1/+2
|
* Use floating-point rather than integer division to handle timeouts so that ↵Paul "LeoNerd" Evans2014-10-291-1/+1
| | | | non-zero but sub-second waits don't collapse to zero
* fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-031-1/+1
| | | | hasn't been incorporated in time for launch.
* Make sure to print exceptions properly from notifier failuresPaul "LeoNerd" Evans2014-09-031-1/+6
|
* Merge branch 'master' of github.com:matrix-org/synapse into release-v0.2.0Erik Johnston2014-09-021-2/+19
|\ | | | | | | | | | | | | Conflicts: synapse/notifier.py webclient/room/room-controller.js webclient/room/room.html
| * When notifying listeners, don't do so in a serial fashionErik Johnston2014-08-291-2/+19
| |
* | Have EventSource's get_new_events_for_user() API work only on keys within ↵Paul "LeoNerd" Evans2014-08-291-10/+20
|/ | | | that source, not overall eventstream tokens
* Turn of trace_function loggingErik Johnston2014-08-281-2/+2
|
* And more logging.Erik Johnston2014-08-281-1/+2
|
* If timeout=0, return immediatelyErik Johnston2014-08-271-1/+5
|
* Comments!Erik Johnston2014-08-271-0/+35
|
* Index sources in a nicer fashion.Erik Johnston2014-08-271-3/+3
|
* Implement presence event source. Change the way the notifier indexes listenersErik Johnston2014-08-271-43/+42
|
* Enable presence again. Fix up api to match old api.Erik Johnston2014-08-261-1/+19
|
* WIP: Completely change how event streaming and pagination work. This ↵Erik Johnston2014-08-261-0/+184
reflects the change in the underlying storage model.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-21 20:01:40 +0000