summary refs log tree commit diff
path: root/synapse/replication/tcp/streams/_base.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix limit logic for AccountDataStream (#7384)Richard van der Hoff2020-05-151-12/+56
| | | | | | Make sure that the AccountDataStream presents complete updates, in the right order. This is much the same fix as #7337 and #7358, but applied to a different stream.
* Support any process writing to cache invalidation stream. (#7436)Erik Johnston2020-05-071-27/+60
|
* Fix catchup-on-reconnect for the Federation Stream (#7374)Richard van der Hoff2020-05-051-1/+2
| | | | looks like we managed to break this during the refactorathon.
* Thread through instance name to replication client. (#7369)Erik Johnston2020-05-011-13/+37
| | | For in memory streams when fetching updates on workers we need to query the source of the stream, which currently is hard coded to be master. This PR threads through the source instance we received via `POSITION` through to the update function in each stream, which can then be passed to the replication client for in memory streams.
* Workaround for assertion errors from db_query_to_update_function (#7378)Richard van der Hoff2020-05-011-2/+1
| | | Hopefully this is no worse than what we have on master...
* Make it clear that the limit for an update_function is a targetRichard van der Hoff2020-04-231-5/+9
|
* Remove 'limit' param from `get_repl_stream_updates` APIRichard van der Hoff2020-04-231-4/+1
| | | | | there doesn't seem to be much point in passing this limit all around, since both sides agree it's meant to be 100.
* Improve type checking in `replication.tcp.Stream` (#7291)Richard van der Hoff2020-04-171-103/+121
| | | | | | | The general idea here is to get rid of the type: ignore annotations on all of the current_token and update_function assignments, which would have caught #7290. After a bit of experimentation, it seems like the least-awful way to do this is to pass the offending functions in as parameters to the Stream constructor. Unfortunately that means that the concrete implementations no longer have the same constructor signature as Stream itself, which means that it gets hard to correctly annotate STREAMS_MAP. I've also introduced a couple of new types, to take out some duplication.
* Fix 'generator object is not subscriptable' error (#7290)Richard van der Hoff2020-04-161-1/+2
| | | | | | Some of the query functions return generators rather than lists, so we can't index into the result. Happily we already have a copy of the results. (think this was introduced in #7024)
* Fix race in replication (#7226)Erik Johnston2020-04-071-1/+2
| | | | Fixes a race between handling `POSITION` and `RDATA` commands. We do this by simply linearizing handling of them.
* Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-52/+108
| | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* Convert `*StreamRow` classes to inner classes (#7116)Richard van der Hoff2020-03-231-88/+93
| | | | | This just helps keep the rows closer to their streams, so that it's easier to see what the format of each stream is.
* Remove concept of a non-limited stream. (#7011)Erik Johnston2020-03-201-40/+26
|
* Change device list replication to match new semantics.Erik Johnston2020-02-281-4/+9
| | | | | Instead of sending down batches of user ID/host tuples, send down a row per entity (user ID or host).
* Increase MAX_EVENTS_BEHIND for replication clientsErik Johnston2020-02-211-1/+1
|
* Allow streaming cache invalidate all to workers. (#6749)Erik Johnston2020-01-221-5/+21
|
* Port synapse.replication.tcp to async/await (#6666)Erik Johnston2020-01-161-15/+10
| | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing....
* Fixup synapse.replication to pass mypy checks (#6667)Erik Johnston2020-01-141-28/+29
|
* lintAndrew Morgan2019-11-081-2/+1
|
* Remove content from being sent for account data rdata streamAndrew Morgan2019-11-081-3/+3
|
* make user signatures a separate streamHubert Chathi2019-10-301-0/+18
|
* Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-301-4/+4
| | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* Replace returnValue with return (#5736)Amber Brown2019-07-231-6/+6
|
* Run Black. (#5482)Amber Brown2019-06-201-78/+82
|
* Fix relations in worker modeErik Johnston2019-05-161-0/+1
|
* Combine the CurrentStateDeltaStream into the EventStreamRichard van der Hoff2019-03-271-21/+0
|
* Skip building a ROW_TYPE when building updatesRichard van der Hoff2019-03-271-2/+2
| | | | | We're about to turn it straight into a JSON object anyway so building a ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
* Add parse_row method to replication stream classRichard van der Hoff2019-03-271-0/+15
| | | | This will allow individual stream classes to override how a row is parsed.
* move FederationStream out to its own fileRichard van der Hoff2019-03-271-20/+0
|
* move EventsStream out to its own fileRichard van der Hoff2019-03-271-21/+0
|
* Move replication.tcp.streams into a packageRichard van der Hoff2019-03-271-0/+482