Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Fix missing _add_persisted_position (#8179) | Erik Johnston | 2020-08-27 | 1 | -0/+2 |
| | | | This was forgotten in #8164. | ||||
* | Add functions to `MultiWriterIdGen` used by events stream (#8164) | Erik Johnston | 2020-08-25 | 2 | -3/+108 |
| | |||||
* | Make StreamIdGen `get_next` and `get_next_mult` async (#8161) | Erik Johnston | 2020-08-25 | 1 | -5/+5 |
| | | | | This is mainly so that `StreamIdGenerator` and `MultiWriterIdGenerator` will have the same interface, allowing them to be used interchangeably. | ||||
* | Remove `ChainedIdGenerator`. (#8123) | Erik Johnston | 2020-08-19 | 1 | -67/+1 |
| | | | | | It's just a thin wrapper around two ID gens to make `get_current_token` and `get_next` return tuples. This can easily be replaced by calling the appropriate methods on the underlying ID gens directly. | ||||
* | Separate `get_current_token` into two. (#8113) | Erik Johnston | 2020-08-19 | 1 | -9/+27 |
| | | | | | | | | | | | | The function is used for two purposes: 1) for subscribers of streams to get a token they can use to get further updates with, and 2) for replication to track position of the writers of the stream. For streams with a single writer the two scenarios produce the same result, however the situation becomes complicated for streams with multiple writers. The current `MultiWriterIdGenerator` does not correctly handle the first case (which is not an issue as its only used for the `caches` stream which nothing subscribes to outside of replication). | ||||
* | Rename database classes to make some sense (#8033) | Erik Johnston | 2020-08-05 | 1 | -2/+2 |
| | |||||
* | Use `PostgresSequenceGenerator` from `MultiWriterIdGenerator` | Richard van der Hoff | 2020-07-16 | 1 | -4/+4 |
| | | | | partly just to show it works, but alwo to remove a bit of code duplication. | ||||
* | Add some helper classes for generating ID sequences | Richard van der Hoff | 2020-07-16 | 1 | -0/+98 |
| | |||||
* | Move event stream handling out of slave store. (#7491) | Erik Johnston | 2020-05-15 | 1 | -0/+11 |
| | | | | | This allows us to have the logic on both master and workers, which is necessary to move event persistence off master. We also combine the instantiation of ID generators from DataStore and slave stores to the base worker stores. This allows us to select which process writes events independently of the master/worker splits. | ||||
* | Add MultiWriterIdGenerator. (#7281) | Erik Johnston | 2020-05-04 | 1 | -2/+167 |
| | | | | | | This will be used to coordinate stream IDs across multiple writers. Functions as the equivalent of both `StreamIdGenerator` and `SlavedIdTracker`. | ||||
* | Update black to 19.10b0 (#6304) | Amber Brown | 2019-11-01 | 1 | -1/+1 |
| | | | * update version of black and also fix the mypy config being overridden | ||||
* | Remove unnecessary parentheses around return statements (#5931) | Andrew Morgan | 2019-08-30 | 1 | -2/+2 |
| | | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :) | ||||
* | Run black on the rest of the storage module (#4996) | Amber Brown | 2019-04-03 | 1 | -5/+5 |
| | |||||
* | run isort | Amber Brown | 2018-07-09 | 1 | -1/+1 |
| | |||||
* | Fix assertion to stop transaction queue getting wedged | Richard van der Hoff | 2017-03-15 | 1 | -0/+14 |
| | | | | | | | | ... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop. | ||||
* | Add tests for redactions | Mark Haines | 2016-04-07 | 1 | -1/+1 |
| | |||||
* | Assert that the step != 0 | Mark Haines | 2016-04-01 | 1 | -0/+1 |
| | |||||
* | use google style doc strings | Mark Haines | 2016-04-01 | 1 | -11/+12 |
| | |||||
* | Rename direction to step, apply checks consistently | Mark Haines | 2016-04-01 | 1 | -15/+15 |
| | |||||
* | Use a stream id generator for backfilled ids | Mark Haines | 2016-04-01 | 1 | -20/+41 |
| | |||||
* | Add replication stream for pushers | Mark Haines | 2016-03-15 | 1 | -1/+6 |
| | |||||
* | Ensure integer is an integer | Erik Johnston | 2016-03-09 | 1 | -1/+1 |
| | |||||
* | Add a stream for push rule updates | Mark Haines | 2016-03-01 | 1 | -26/+58 |
| | |||||
* | Load the current id in the IdGenerator constructor | Mark Haines | 2016-03-01 | 1 | -47/+22 |
| | | | | | | | | | Rather than loading them lazily. This allows us to remove all the yield statements and spurious arguments for the get_next methods. It also allows us to replace all instances of get_next_txn with get_next since get_next no longer needs to access the db. | ||||
* | Remove unused param from get_max_token | Erik Johnston | 2016-02-18 | 1 | -3/+1 |
| | |||||
* | Initial cut | Erik Johnston | 2016-02-17 | 1 | -1/+3 |
| | |||||
* | Add a Homeserver.setup method. | Erik Johnston | 2016-01-26 | 1 | -27/+9 |
| | | | | | | This is for setting up dependencies that require work on startup. This is useful for the DataStore that wants to read a bunch from the database before initiliazing. | ||||
* | copyrights | Matthew Hodgson | 2016-01-07 | 2 | -2/+2 |
| | |||||
* | Merge pull request #199 from matrix-org/erikj/receipts | Erik Johnston | 2015-07-16 | 1 | -2/+5 |
|\ | | | | | Implement read receipts. | ||||
| * | Add basic storage functions for handling of receipts | Erik Johnston | 2015-07-01 | 1 | -2/+5 |
| | | |||||
* | | Add bulk insert events API | Erik Johnston | 2015-06-25 | 1 | -0/+31 |
|/ | |||||
* | SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called ↵ | Mark Haines | 2015-05-12 | 1 | -4/+8 |
| | | | | from the main thread after the transaction completes, not from database thread before the transaction completes. | ||||
* | Typo | Erik Johnston | 2015-04-29 | 1 | -1/+1 |
| | |||||
* | Also remove yield from within lock in the other generator | Erik Johnston | 2015-04-29 | 1 | -8/+6 |
| | |||||
* | Fix deadlock in id_generators. No idea why this was an actual deadlock. | Erik Johnston | 2015-04-29 | 1 | -14/+16 |
| | |||||
* | Make get_max_token into inlineCallbacks so that the lock works. | Erik Johnston | 2015-04-27 | 1 | -3/+4 |
| | |||||
* | Use try..finally in contextlib.contextmanager | Erik Johnston | 2015-04-15 | 1 | -3/+5 |
| | |||||
* | Correctly increment the _next_id initially | Erik Johnston | 2015-04-14 | 1 | -2/+4 |
| | |||||
* | Stream ordering and out of order insertions. | Erik Johnston | 2015-04-09 | 2 | -0/+140 |
Handle the fact that events can be persisted out of order, and so to get the "current max" stream token becomes non trivial - as we need to make sure that *all* stream tokens less than the current max have also successfully been persisted. |