summary refs log tree commit diff
path: root/synapse/app/generic_worker.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Run replication streamers on workers (#7146)Erik Johnston2020-04-281-4/+9
| | | Currently we never write to streams from workers, but that will change soon
* Stop the master relaying USER_SYNC for other workers (#7318)Richard van der Hoff2020-04-221-36/+49
| | | | | | | Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.
* Merge branch 'release-v1.12.4' into developRichard van der Hoff2020-04-221-0/+6
|\
| * Support GET account_data requests on a worker (#7311)Richard van der Hoff2020-04-211-0/+6
| |
| * Fix starting workers when federation sending not split out.Erik Johnston2020-03-311-0/+12
| |
* | Move client command handling out of TCP protocol (#7185)Erik Johnston2020-04-061-6/+3
| | | | | | The aim here is to move the command handling out of the TCP protocol classes and to also merge the client and server command handling (so that we can reuse them for redis protocol). This PR simply moves the client paths to the new `ReplicationCommandHandler`, a future PR will move the server paths too.
* | Remove some `run_in_background` calls in replication code (#7203)Richard van der Hoff2020-04-031-8/+8
| | | | | | | | | | | | By running this stuff with `run_in_background`, it won't be correctly reported against the relevant CPU usage stats. Fixes #7202
* | Remove usage of "conn_id" for presence. (#7128)Erik Johnston2020-03-301-4/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.
* | Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-0/+3
| | | | | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* | Merge pull request #7133 from matrix-org/erikj/fix_worker_startupErik Johnston2020-03-251-0/+12
|\ \ | | | | | | Fix starting workers when federation sending not split out.
| * | Fix starting workers when federation sending not split out.Erik Johnston2020-03-241-0/+12
| |/
* | Convert `*StreamRow` classes to inner classes (#7116)Richard van der Hoff2020-03-231-1/+1
| | | | | | | | | | This just helps keep the rows closer to their streams, so that it's easier to see what the format of each stream is.
* | Fix processing of `groups` stream, and use symbolic names for streams (#7117)Richard van der Hoff2020-03-231-12/+23
| | | | | | | | | | | | `groups` != `receipts` Introduced in #6964
* | Comments from reviewErik Johnston2020-03-181-0/+3
| |
* | Fix worker handlingErik Johnston2020-03-021-2/+3
| |
* | Change device list replication to match new semantics.Erik Johnston2020-02-281-1/+1
|/ | | | | Instead of sending down batches of user ID/host tuples, send down a row per entity (user ID or host).
* Don't refuse to start worker if media listener configured. (#7002)Erik Johnston2020-02-271-14/+20
| | | | | | | | Instead lets just warn if the worker has a media listener configured but has the media repository disabled. Previously non media repository workers would just ignore the media listener.
* Merge worker apps into one. (#6964)Erik Johnston2020-02-251-0/+917