summary refs log tree commit diff
path: root/synapse/replication/tcp/resource.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move server command handling out of TCP protocol (#7187)Erik Johnston2020-04-071-137/+26
| | | This completes the merging of server and client command processing.
* Remove connections per replication stream metric. (#7195)Erik Johnston2020-04-011-16/+0
| | | | | This broke in a recent PR (#7024) and is no longer useful due to all replication clients implicitly subscribing to all streams, so let's just remove it.
* Remove usage of "conn_id" for presence. (#7128)Erik Johnston2020-03-301-10/+7
| | | | | | | | | | | | | | | | * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.
* Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-7/+12
| | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* Remove concept of a non-limited stream. (#7011)Erik Johnston2020-03-201-7/+2
|
* Port PresenceHandler to async/await (#6991)Erik Johnston2020-02-261-1/+5
|
* Propagate cache invalidates from workers to other workers. (#6748)Erik Johnston2020-01-271-3/+6
| | | Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
* Wake up transaction queue when remote server comes back online (#6706)Erik Johnston2020-01-171-0/+9
| | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up.
* Port synapse.replication.tcp to async/await (#6666)Erik Johnston2020-01-161-17/+14
| | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing....
* Fixup synapse.replication to pass mypy checks (#6667)Erik Johnston2020-01-141-1/+2
|
* Run Black. (#5482)Amber Brown2019-06-201-20/+34
|
* move FederationStream out to its own fileRichard van der Hoff2019-03-271-1/+2
|
* Fix ClientReplicationStreamProtocol.__str__ (#4929)Richard van der Hoff2019-03-251-1/+0
| | | | | | | | `__str__` depended on `self.addr`, which was absent from ClientReplicationStreamProtocol, so attempting to call str on such an object would raise an exception. We can calculate the peer addr from the transport, so there is no need for addr anyway.
* Add a config option for torture-testing worker replication. (#4902)Richard van der Hoff2019-03-201-1/+17
| | | Setting this to 50 or so makes a bunch of sytests fail in worker mode.
* Wrap a number of things that run in the backgroundRichard van der Hoff2018-07-251-6/+8
| | | | | This will reduce the number of "Starting db connection from sentinel context" warnings, and will help with our metrics.
* run isortAmber Brown2018-07-091-8/+9
|
* Remove all global reactor imports & pass it around explicitly (#3424)Amber Brown2018-06-251-2/+2
|
* Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-281-1/+2
|\
| * Merge pull request #3244 from NotAFile/py3-six-4Amber Brown2018-05-241-1/+2
| |\ | | | | | | replace some iteritems with six
| | * replace some iteritems with sixAdrian Tschira2018-05-191-1/+2
| | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* | | more cleanupAmber Brown2018-05-221-3/+4
| | |
* | | cleanups, self-registrationAmber Brown2018-05-221-4/+5
| | |
* | | Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-221-0/+2
|\| |
| * | Send users a server notice about consentRichard van der Hoff2018-05-221-0/+2
| |/ | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.
* / rest of the changesAmber Brown2018-05-211-16/+14
|/
* make imports localAdrian Tschira2018-04-281-2/+2
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Fix some logcontext leaks in replication resourceRichard van der Hoff2017-11-231-2/+4
| | | | | The @measure_func annotations rely on the wrapped function respecting the logcontext rules. Add the necessary yields to make this work.
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* log when we get an exception handling replication updateshera2017-10-121-1/+5
|
* Make workers report to master for user ip updatesErik Johnston2017-06-271-0/+10
|
* Add missing notifierErik Johnston2017-06-091-1/+2
|
* Advance replication streams even if nothing is listeningErik Johnston2017-04-041-1/+1
| | | | | | Otherwise the streams don't advance and steadily fall behind, so when a worker does connect either a) they'll be streamed lots of old updates or b) the connection will fail as the streams are too far behind.
* Always advance stream tokensErik Johnston2017-04-031-1/+4
|
* Use callbacks to notify tcp replication rather than deferredsErik Johnston2017-03-311-14/+1
|
* Add a timestamp to USER_SYNC commandErik Johnston2017-03-311-2/+2
| | | | This timestamp is used to indicate when the user last sync'd
* Add server side resource for tcp replicationErik Johnston2017-03-301-0/+300