summary refs log tree commit diff
path: root/docs/tcp_replication.md (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Make event persisters periodically announce position over replication. (#8499)Erik Johnston2020-10-121-4/+9
| | | | | Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress. This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.
* Support any process writing to cache invalidation stream. (#7436)Erik Johnston2020-05-071-4/+0
|
* Add instance name to RDATA/POSITION commands (#7364)Erik Johnston2020-04-291-17/+24
| | | | | This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
* Stop the master relaying USER_SYNC for other workers (#7318)Richard van der Hoff2020-04-221-5/+1
| | | | | | | Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.
* Remove usage of "conn_id" for presence. (#7128)Erik Johnston2020-03-301-0/+6
| | | | | | | | | | | | | | | | * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.
* Move catchup of replication streams to worker. (#7024)Erik Johnston2020-03-251-33/+13
| | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
* Allow streaming cache invalidate all to workers. (#6749)Erik Johnston2020-01-221-0/+5
|
* Wake up transaction queue when remote server comes back online (#6706)Erik Johnston2020-01-171-1/+5
| | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up.
* document the REPLICATE command a bit better (#6305)Richard van der Hoff2019-11-041-1/+14
| | | | since I found myself wonder how it works
* (#5849) Convert rst to markdown (#6040)dstipp2019-09-171-0/+249
Converting some of the rst documentation to markdown. Attempted to preserve whitespace and line breaks to minimize cosmetic change.