Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Handle one-word replication commands correctly | Richard van der Hoff | 2020-04-07 | 1 | -3/+11 |
| | | | | | `REPLICATE` is now a valid command, and it's nice if you can issue it from the console without remembering to call it `REPLICATE ` with a trailing space. | ||||
* | Move server command handling out of TCP protocol (#7187) | Erik Johnston | 2020-04-07 | 1 | -114/+51 |
| | | | This completes the merging of server and client command processing. | ||||
* | Move client command handling out of TCP protocol (#7185) | Erik Johnston | 2020-04-06 | 1 | -166/+31 |
| | | | The aim here is to move the command handling out of the TCP protocol classes and to also merge the client and server command handling (so that we can reuse them for redis protocol). This PR simply moves the client paths to the new `ReplicationCommandHandler`, a future PR will move the server paths too. | ||||
* | Remove usage of "conn_id" for presence. (#7128) | Erik Johnston | 2020-03-30 | 1 | -2/+7 |
| | | | | | | | | | | | | | | | | * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout. | ||||
* | Move catchup of replication streams to worker. (#7024) | Erik Johnston | 2020-03-25 | 1 | -135/+71 |
| | | | This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date. | ||||
* | Propagate cache invalidates from workers to other workers. (#6748) | Erik Johnston | 2020-01-27 | 1 | -1/+1 |
| | | | Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers. | ||||
* | Wake up transaction queue when remote server comes back online (#6706) | Erik Johnston | 2020-01-17 | 1 | -0/+15 |
| | | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up. | ||||
* | Port synapse.replication.tcp to async/await (#6666) | Erik Johnston | 2020-01-16 | 1 | -40/+32 |
| | | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing.... | ||||
* | Fixup synapse.replication to pass mypy checks (#6667) | Erik Johnston | 2020-01-14 | 1 | -15/+21 |
| | |||||
* | document the REPLICATE command a bit better (#6305) | Richard van der Hoff | 2019-11-04 | 1 | -2/+72 |
| | | | | since I found myself wonder how it works | ||||
* | Remove usage of deprecated logger.warn method from codebase (#6271) | Andrew Morgan | 2019-10-31 | 1 | -1/+1 |
| | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated. | ||||
* | Move logging utilities out of the side drawer of util/ and into logging/ (#5606) | Amber Brown | 2019-07-04 | 1 | -1/+1 |
| | |||||
* | Run Black. (#5482) | Amber Brown | 2019-06-20 | 1 | -37/+39 |
| | |||||
* | Make EventStream rows have a type | Richard van der Hoff | 2019-03-27 | 1 | -2/+2 |
| | | | | ... as a precursor to combining it with the CurrentStateDelta stream. | ||||
* | Add parse_row method to replication stream class | Richard van der Hoff | 2019-03-27 | 1 | -1/+1 |
| | | | | This will allow individual stream classes to override how a row is parsed. | ||||
* | Fix ClientReplicationStreamProtocol.__str__ (#4929) | Richard van der Hoff | 2019-03-25 | 1 | -3/+5 |
| | | | | | | | | `__str__` depended on `self.addr`, which was absent from ClientReplicationStreamProtocol, so attempting to call str on such an object would raise an exception. We can calculate the peer addr from the transport, so there is no need for addr anyway. | ||||
* | Fix bug where read-receipts lost their timestamps (#4927) | Richard van der Hoff | 2019-03-25 | 1 | -8/+19 |
| | | | | | Make sure that they are sent correctly over the replication stream. Fixes: #4898 | ||||
* | Simplify token replication logic | Andrew Morgan | 2019-03-05 | 1 | -23/+14 |
| | |||||
* | Clean up logic and add comments | Andrew Morgan | 2019-03-04 | 1 | -11/+18 |
| | |||||
* | Clearer branching, fix missing list clear | Andrew Morgan | 2019-03-04 | 1 | -4/+11 |
| | |||||
* | Prevent replication wedging | Andrew Morgan | 2019-03-04 | 1 | -4/+24 |
| | |||||
* | Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff | Erik Johnston | 2019-02-27 | 1 | -0/+17 |
|\ | | | | | Fix tightloop over connecting to replication server | ||||
| * | Move connecting logic into ClientReplicationStreamProtocol | Erik Johnston | 2019-02-27 | 1 | -0/+17 |
| | | |||||
* | | Limit cache invalidation replication line length (#4748) | Erik Johnston | 2019-02-27 | 1 | -1/+16 |
|/ | |||||
* | Don't truncate command name in metrics | Erik Johnston | 2018-10-29 | 1 | -2/+2 |
| | |||||
* | Remove conn_id | Erik Johnston | 2018-09-04 | 1 | -2/+2 |
| | |||||
* | Remove conn_id from repl prometheus metrics | Erik Johnston | 2018-09-03 | 1 | -10/+10 |
| | | | | | `conn_id` gets set to a random string, and so we end up filling up prometheus with tonnes of data series, which is bad. | ||||
* | Logcontexts for replication command handlers | Richard van der Hoff | 2018-08-17 | 1 | -13/+29 |
| | | | | | | | | | | Run the handlers for replication commands as background processes. This should improve the visibility in our metrics, and reduce the number of "running db transaction from sentinel context" warnings. Ideally it means converting the things that fire off deferreds into the night into things that actually return a Deferred when they are done. I've made a bit of a stab at this, but it will probably be leaky. | ||||
* | run isort | Amber Brown | 2018-07-09 | 1 | -16/+24 |
| | |||||
* | Fix tcp protocol metrics naming (#3410) | Amber Brown | 2018-06-21 | 1 | -18/+35 |
| | |||||
* | Fix replication metrics | Richard van der Hoff | 2018-06-04 | 1 | -2/+2 |
| | | | | fix bug introduced in #3256 | ||||
* | Merge remote-tracking branch 'origin/develop' into 3218-official-prom | Amber Brown | 2018-05-28 | 1 | -7/+7 |
|\ | |||||
| * | replace some iteritems with six | Adrian Tschira | 2018-05-19 | 1 | -4/+5 |
| | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | more cleanup | Amber Brown | 2018-05-22 | 1 | -3/+6 |
| | | |||||
* | | fix the test failures | Amber Brown | 2018-05-22 | 1 | -1/+1 |
| | | |||||
* | | replacing portions | Amber Brown | 2018-05-21 | 1 | -54/+34 |
|/ | |||||
* | make imports local | Adrian Tschira | 2018-04-28 | 1 | -2/+2 |
| | | | | Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | Metrics for number of RDATA commands received | Richard van der Hoff | 2018-01-15 | 1 | -5/+14 |
| | | | | I found myself wishing we had this. | ||||
* | Reduce log levels in tcp replication | Erik Johnston | 2017-07-11 | 1 | -2/+2 |
| | |||||
* | Make workers report to master for user ip updates | Erik Johnston | 2017-06-27 | 1 | -0/+6 |
| | |||||
* | Typo | Erik Johnston | 2017-04-10 | 1 | -1/+1 |
| | |||||
* | Up replication ping timeout | Erik Johnston | 2017-04-10 | 1 | -2/+4 |
| | |||||
* | Add log lines | Erik Johnston | 2017-04-05 | 1 | -1/+2 |
| | |||||
* | Rearrange metrics | Erik Johnston | 2017-04-05 | 1 | -16/+31 |
| | |||||
* | Fix typo | Erik Johnston | 2017-04-05 | 1 | -2/+2 |
| | |||||
* | Fixup some metrics for tcp repl | Erik Johnston | 2017-04-05 | 1 | -0/+16 |
| | |||||
* | Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix | Erik Johnston | 2017-04-04 | 1 | -5/+7 |
|\ | | | | | Advance replication streams even if nothing is listening | ||||
| * | Advance replication streams even if nothing is listening | Erik Johnston | 2017-04-04 | 1 | -5/+7 |
| | | | | | | | | | | | | Otherwise the streams don't advance and steadily fall behind, so when a worker does connect either a) they'll be streamed lots of old updates or b) the connection will fail as the streams are too far behind. | ||||
* | | Fiddle tcp replication logging | Erik Johnston | 2017-04-04 | 1 | -2/+2 |
|/ | |||||
* | Add a timestamp to USER_SYNC command | Erik Johnston | 2017-03-31 | 1 | -2/+5 |
| | | | | This timestamp is used to indicate when the user last sync'd | ||||
* | Initial TCP protocol implementation | Erik Johnston | 2017-03-30 | 1 | -0/+601 |
This defines the low level TCP replication protocol |