summary refs log tree commit diff
path: root/synapse/app/pusher.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge worker apps into one. (#6964)Erik Johnston2020-02-251-205/+4
|
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-1/+1
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Port synapse.replication.tcp to async/await (#6666)Erik Johnston2020-01-161-3/+2
| | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing....
* Merge pull request #6511 from matrix-org/erikj/remove_db_config_from_appsErik Johnston2019-12-121-5/+0
|\ | | | | Move database config from apps into HomeServer object
| * Remove database config parsing from apps.Erik Johnston2019-12-101-5/+0
| |
* | Make the PusherSlaveStore inherit from the slave RoomStoreBrendan Abolivier2019-12-101-1/+6
|/ | | | So that it has access to the get_retention_policy_for_room function which is required by filter_events_for_client.
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-2/+2
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Implement a structured logging output system. (#5680)Amber Brown2019-08-281-2/+2
|
* Call startup commands as system triggers.Erik Johnston2019-07-221-1/+1
| | | | | This helps ensures that we only consider ourselves "up" once all the startup functions have completed.
* Support Prometheus_client 0.4.0+ (#5636)Amber Brown2019-07-181-2/+1
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-201-36/+24
|
* Deduplicate some code in synapse.app (#4567)Amber Brown2019-02-081-2/+1
|
* Remove redundant run_as_background_process() from pusherpoolRichard van der Hoff2018-10-221-2/+2
| | | | | | | `on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher` now always return synchronously, so we can remove the `defer.gatherResults` on their results, and the `run_as_background_process` wrappers can be removed too because the PusherPool methods will now complete quickly enough.
* Rename _refresh_pusherRichard van der Hoff2018-10-221-1/+1
| | | | | This is public (or at least, called from outside the class), so ought to have a better name.
* Make workers work on Py3 (#4027)Amber Brown2018-10-131-7/+8
|
* Replaced all occurences of e.message with str(e)Schnuffle2018-09-271-1/+1
| | | | Signed-off-by: Schnuffle <schnuffle@github.com>
* Fix some instances of ExpiringCache not expiring cache itemsErik Johnston2018-09-211-1/+0
| | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.
* Make sure that we close db connections opened during initRichard van der Hoff2018-08-281-4/+1
| | | | | | | | | We should explicitly close any db connections we open, because failing to do so can block other transactions as per https://github.com/matrix-org/synapse/issues/3682. Let's also try to factor out some of the boilerplate by having server classes define their datastore class rather than duplicating the whole of `setup`.
* Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commandsRichard van der Hoff2018-08-171-1/+2
|\ | | | | Logcontexts for replication command handlers
| * Logcontexts for replication command handlersRichard van der Hoff2018-08-171-1/+2
| | | | | | | | | | | | | | | | | | | | Run the handlers for replication commands as background processes. This should improve the visibility in our metrics, and reduce the number of "running db transaction from sentinel context" warnings. Ideally it means converting the things that fire off deferreds into the night into things that actually return a Deferred when they are done. I've made a bit of a stab at this, but it will probably be leaky.
* | Fix logcontexts for running pushersRichard van der Hoff2018-08-171-2/+2
|/ | | | | | | First of all, avoid resetting the logcontext before running the pushers, to fix the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning. Instead, give them their own "background process" logcontexts.
* run isortAmber Brown2018-07-091-2/+3
|
* The flag is named enable_metrics, not collect_metricsTravis Ralston2018-06-121-1/+1
| | | Signed-off-by: Travis Ralston <travpc@gmail.com>
* Run Prometheus on a different port, optionally. (#3274)Amber Brown2018-05-311-1/+9
|
* Set Server header in SynapseRequestRichard van der Hoff2018-05-101-0/+1
| | | | | | | | | | | | (instead of everywhere that writes a response. Or rather, the subset of places which write responses where we haven't forgotten it). This also means that we don't have to have the mysterious version_string attribute in anything with a request handler. Unfortunately it does mean that we have to pass the version string wherever we instantiate a SynapseSite, which has been c&ped 150 times, but that is code that ought to be cleaned up anyway really.
* Merge remote-tracking branch 'origin/develop' into rav/use_run_in_backgroundRichard van der Hoff2018-04-271-14/+17
|\
| * Improve exception handling for background processesRichard van der Hoff2018-04-271-14/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.
* | Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-2/+2
|/ | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* 404 correctly on missing paths via NoResourceMatthew Hodgson2018-03-231-2/+2
| | | | fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
* Move storage functions for push calculationsErik Johnston2018-02-271-5/+0
| | | | This will allow push actions for an event to be calculated on workers.
* Factor out get_db_conn to HomeServer base classRichard van der Hoff2018-01-261-13/+0
| | | | | This function is identical to all subclasses, so we may as well push it up to the base class to reduce duplication (and make use of it in the tests)
* Implement listen_tcp method in remaining workersSilke2017-12-181-21/+17
| | | | Signed-off-by: Silke <silke@slxh.eu>
* Factor out common application startRichard van der Hoff2017-08-151-44/+13
| | | | | We have 10 copies of this code, and I don't really want to update each one separately.
* Add the appropriate amount of preserve_fnErik Johnston2017-04-041-6/+7
|
* Update all the workers and master to use TCP replicationErik Johnston2017-04-031-79/+46
|
* Run the reactor with the sentinel logcontextRichard van der Hoff2017-03-181-2/+7
| | | | | | This fixes a class of 'Unexpected logcontext' messages, which were happening because the logcontext was somewhat arbitrarily swapping between the sentinel and the `run` logcontext.
* Refactor logger config for workersRichard van der Hoff2017-03-101-1/+1
| | | | - to make it easier to add more config options.
* Restore default bind addressErik Johnston2017-01-101-10/+2
|
* Remove spurious for..else..Erik Johnston2017-01-101-11/+0
|
* Fix check for bind_addressJohannes Löthberg2016-12-201-2/+2
| | | | | | | The empty string is a valid setting for the bind_address option, so explicitly check for None here instead. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* Add support for specifying multiple bind addressesJohannes Löthberg2016-12-181-20/+45
| | | | Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* Respect use_frozen_dicts option in workersErik Johnston2016-11-081-0/+4
|
* Reduce DB hits for replicationErik Johnston2016-09-231-2/+2
| | | | | | | | | | | | | | Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.
* Enable state caches on workersErik Johnston2016-09-151-0/+1
|
* Remove broken cache stuffErik Johnston2016-08-151-16/+0
|
* Add federation /version APIErik Johnston2016-08-051-1/+1
|
* Fix setting gc thresholds in the workersMark Haines2016-06-171-1/+2
|
* Use worker_ prefixes for worker config, use existing support for multiple ↵Mark Haines2016-06-161-17/+12
| | | | config files
* Access the event_cache_size directly from the server object.Mark Haines2016-06-161-3/+3
| | | | | This means that the workers can override the event_cache_size directly without clobbering the value in the main synapse config.
* Access replication_url from the worker config directlyMark Haines2016-06-161-2/+3
|
* Inline the synchrotron and pusher configs into the main configMark Haines2016-06-161-125/+46
|
* Add gc_threshold to pusher and synchrotronErik Johnston2016-06-071-0/+14
|
* Allow setting of gc.set_thresholdsErik Johnston2016-06-071-0/+5
|
* Yield on the sleeps intended to backoff replicationMark Haines2016-06-061-1/+1
|
* Working unsubscribe links going straight to the HSDavid Baker2016-06-021-1/+22
| | | | and authed by macaroons that let you delete pushers and nothing else
* Manually expire broken caches like the who_forgot_in_roomMark Haines2016-05-131-0/+16
|
* Use the SlavedAccountDataStoreMark Haines2016-05-131-10/+3
|
* Shift some of the state_group methods into the SlavedEventStoreMark Haines2016-05-131-45/+0
|
* Make email notifs work on the pusher synapseDavid Baker2016-05-131-0/+47
| | | | Plus general bugfix to email notif code
* Pass through _get_state_group_for_eventsDavid Baker2016-05-101-0/+4
|
* StateStore, not EventsStoreDavid Baker2016-05-101-2/+2
|
* Cached functions must be accessed through the dictDavid Baker2016-05-101-1/+2
|
* Pass though _get_state_group_for_eventsDavid Baker2016-05-101-0/+4
|
* Pass through get_state_groupsDavid Baker2016-05-101-0/+4
|
* Also pass through get_profile_displaynameDavid Baker2016-05-101-0/+4
|
* Add date header & message idDavid Baker2016-05-041-1/+2
|
* Add db functions used for email to the pusher appDavid Baker2016-05-041-0/+12
|
* Actually start the pusher daemonMark Haines2016-04-261-0/+2
|
* Fix typo in default pusher configMark Haines2016-04-261-1/+1
|
* Add a log context to the daemonized pusherMark Haines2016-04-261-1/+6
|
* Optionally daemonize the pusherMark Haines2016-04-261-4/+31
|
* Add a couple of update methods to the PusherSlaveStoreMark Haines2016-04-261-0/+8
|
* Remove the uncomments from the commentsMark Haines2016-04-251-2/+2
|
* Add a metrics listener and a ssh listener to the pusherMark Haines2016-04-251-2/+67
|
* Optionally split out the pushers into a separate processMark Haines2016-04-211-0/+208