Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Rename storage classes (#12913) | Erik Johnston | 2022-05-31 | 1 | -320/+0 |
| | |||||
* | Update EventContext `get_current_event_ids` and `get_prev_event_ids` to ↵ | Shay | 2022-05-20 | 1 | -2/+5 |
| | | | | accept state filters and update calls where possible (#12791) | ||||
* | Skip waiting for full state if a StateFilter does not require it (#12498) | Richard van der Hoff | 2022-05-18 | 1 | -4/+59 |
| | | | | | If `StateFilter` specifies a state set which we will have regardless of state-syncing, then we may as well return it immediately. | ||||
* | Refactor `resolve_state_groups_for_events` to not pull out full state when ↵ | Shay | 2022-05-18 | 1 | -6/+6 |
| | | | | no state resolution happens. (#12775) | ||||
* | Reduce the number of "untyped defs" (#12716) | David Robertson | 2022-05-12 | 1 | -2/+4 |
| | |||||
* | Await un-partial-stating after a partial-state join (#12399) | Richard van der Hoff | 2022-04-21 | 1 | -3/+25 |
| | | | | | | When we join a room via the faster-joins mechanism, we end up with "partial state" at some points on the event DAG. Many parts of the codebase need to wait for the full state to load. So, we implement a mechanism to keep track of which events have partial state, and wait for them to be fully-populated. | ||||
* | Raise an exception when getting state at an outlier (#12191) | Richard van der Hoff | 2022-04-01 | 1 | -0/+20 |
| | | | | | | It seems like calling `_get_state_group_for_events` for an event where the state is unknown is an error. Accordingly, let's raise an exception rather than silently returning an empty result. | ||||
* | Fix type of `events` in `StateGroupStorage` and `StateHandler` (#12156) | Richard van der Hoff | 2022-03-04 | 1 | -4/+4 |
| | | | We make multiple passes over this, so a regular iterable won't do. | ||||
* | Fix bug in `StateFilter.return_expanded()` and add some tests. (#12016) | reivilibre | 2022-02-18 | 1 | -1/+7 |
| | |||||
* | Create singletons for `StateFilter.{all,none}()` (#11836) | Richard van der Hoff | 2022-01-27 | 1 | -5/+9 |
| | | | No point recreating these for each call, since they are frozen | ||||
* | Use auto_attribs/native type hints for attrs classes. (#11692) | Patrick Cloke | 2022-01-13 | 1 | -3/+3 |
| | |||||
* | Add an approximate difference method to StateFilters (#10825) | reivilibre | 2021-10-12 | 1 | -1/+171 |
| | |||||
* | Add type hints to state database module. (#10823) | Patrick Cloke | 2021-09-15 | 1 | -1/+2 |
| | |||||
* | Make StateFilter frozen so we can hash it (#10816) | reivilibre | 2021-09-14 | 1 | -13/+32 |
| | | | Also enables Mypy for related tests. | ||||
* | Add type hints to state handler. (#10482) | Patrick Cloke | 2021-07-26 | 1 | -2/+2 |
| | |||||
* | Use inline type hints in `http/federation/`, `storage/` and `util/` (#10381) | Jonathan de Jong | 2021-07-15 | 1 | -2/+2 |
| | |||||
* | Fix `get_state_ids_for_event` return type typo to match what the function ↵ | Eric Eastwood | 2021-05-24 | 1 | -1/+1 |
| | | | | | actually does (#10050) It looks like a typo copy/paste from `get_state_for_event` above. | ||||
* | Remove redundant "coding: utf-8" lines (#9786) | Jonathan de Jong | 2021-04-14 | 1 | -1/+0 |
| | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>` | ||||
* | Bugbear: Add Mutable Parameter fixes (#9682) | Jonathan de Jong | 2021-04-08 | 1 | -10/+16 |
| | | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl> | ||||
* | Add type hints to misc. files. (#9676) | Patrick Cloke | 2021-03-24 | 1 | -2/+2 |
| | |||||
* | Import HomeServer from the proper module. (#9665) | Patrick Cloke | 2021-03-23 | 1 | -1/+1 |
| | |||||
* | Update black, and run auto formatting over the codebase (#9381) | Eric Eastwood | 2021-02-16 | 1 | -3/+2 |
| | | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version | ||||
* | Add additional type hints to the storage module. (#8980) | Patrick Cloke | 2020-12-30 | 1 | -10/+25 |
| | |||||
* | Mypy fixes for `synapse.handlers.federation` (#8422) | Richard van der Hoff | 2020-09-29 | 1 | -3/+3 |
| | | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes. | ||||
* | Stop sub-classing object (#8249) | Patrick Cloke | 2020-09-04 | 1 | -2/+2 |
| | |||||
* | Convert state and stream stores and related code to async (#8194) | Patrick Cloke | 2020-08-28 | 1 | -8/+8 |
| | |||||
* | Convert some of the data store to async. (#7976) | Patrick Cloke | 2020-07-30 | 1 | -5/+6 |
| | |||||
* | Convert storage layer to async/await. (#7963) | Patrick Cloke | 2020-07-28 | 1 | -98/+109 |
| | |||||
* | Replace iteritems/itervalues/iterkeys with native versions. (#7692) | Patrick Cloke | 2020-06-15 | 1 | -20/+18 |
| | |||||
* | Add StateMap type alias (#6715) | Erik Johnston | 2020-01-16 | 1 | -14/+21 |
| | |||||
* | Split state groups into a separate data store (#6296) | Erik Johnston | 2019-12-20 | 1 | -7/+7 |
| | |||||
* | Apply suggestions from code review | Erik Johnston | 2019-10-31 | 1 | -1/+2 |
| | | | | | Fix docstring Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> | ||||
* | Add StateGroupStorage interface | Erik Johnston | 2019-10-30 | 1 | -0/+232 |
| | |||||
* | Move storage classes into a main "data store". | Erik Johnston | 2019-10-21 | 1 | -1221/+0 |
| | | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage. | ||||
* | Lint | Brendan Abolivier | 2019-10-07 | 1 | -1/+3 |
| | |||||
* | Move state's bg updates to a dedicated store | Brendan Abolivier | 2019-10-07 | 1 | -190/+204 |
| | |||||
* | Replace returnValue with return (#5736) | Amber Brown | 2019-07-23 | 1 | -28/+26 |
| | |||||
* | Delegate to cached version when using get_filtered_current_state_ids | Erik Johnston | 2019-07-18 | 1 | -2/+6 |
| | | | | In the case where it gets called with `StateFilter.all()` | ||||
* | Run black on the rest of the storage module (#4996) | Amber Brown | 2019-04-03 | 1 | -135/+102 |
| | |||||
* | Add configurable room list publishing rules | Erik Johnston | 2019-02-14 | 1 | -0/+25 |
| | | | | | This allows specifying who and what is allowed to be published onto the public room list | ||||
* | Raise an exception instead of returning None | Andrew Morgan | 2019-01-31 | 1 | -2/+9 |
| | |||||
* | New function for getting room's create event | Andrew Morgan | 2019-01-31 | 1 | -11/+20 |
| | |||||
* | Merge branch 'anoa/full_search_upgraded_rooms' of ↵ | Andrew Morgan | 2019-01-24 | 1 | -1/+1 |
|\ | | | | | | | github.com:matrix-org/synapse into anoa/full_search_upgraded_rooms | ||||
| * | Apply suggestions from code review | Richard van der Hoff | 2019-01-24 | 1 | -1/+1 |
| | | | | | | Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com> | ||||
* | | Remove redundant create event None check | Andrew Morgan | 2019-01-24 | 1 | -3/+0 |
|/ | |||||
* | Do not return in a deferred function | Andrew Morgan | 2019-01-22 | 1 | -3/+3 |
| | |||||
* | Ensure new filter is actually created | Andrew Morgan | 2019-01-22 | 1 | -1/+0 |
| | |||||
* | Fix a bug with single-room search searching all rooms | Andrew Morgan | 2019-01-22 | 1 | -1/+28 |
| | | | | | * Create a new method for getting predecessor rooms * Remove formatting change | ||||
* | Search for messages across predecessor rooms | Andrew Morgan | 2019-01-18 | 1 | -0/+1 |
| | | | | Signed-off-by: Andrew Morgan <andrew@amorgan.xyz> | ||||
* | Log roomid along with Unknown room (#4297) | Will Hunt | 2018-12-24 | 1 | -1/+1 |
| | |||||
* | Move _find_unreferenced_groups | Erik Johnston | 2018-10-29 | 1 | -79/+0 |
| | |||||
* | Fix set operations thinko | Erik Johnston | 2018-10-29 | 1 | -3/+4 |
| | |||||
* | Don't make temporary list | Erik Johnston | 2018-10-29 | 1 | -3/+2 |
| | |||||
* | Make SQL a bit cleaner | Erik Johnston | 2018-10-29 | 1 | -3/+2 |
| | |||||
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2018-10-26 | 1 | -326/+519 |
|\ | | | | | | | erikj/purge_state_groups | ||||
| * | Refactor state group lookup to reduce DB hits (#4011) | Erik Johnston | 2018-10-25 | 1 | -326/+519 |
| | | | | | | | | | | | | | | | | Currently when fetching state groups from the data store we make two hits two the database: once for members and once for non-members (unless request is filtered to one or the other). This adds needless load to the datbase, so this PR refactors the lookup to make only a single database hit. | ||||
* | | Batch process handling state groups | Erik Johnston | 2018-10-19 | 1 | -41/+71 |
| | | |||||
* | | Fix up comments | Erik Johnston | 2018-10-12 | 1 | -3/+3 |
| | | |||||
* | | Add state_group index to event_to_state_groups | Erik Johnston | 2018-10-04 | 1 | -0/+7 |
| | | | | | | | | | | This is needed to efficiently check for unreferenced state groups during purge. | ||||
* | | Delete unreferened state groups during purge | Erik Johnston | 2018-10-04 | 1 | -0/+50 |
|/ | |||||
* | docstrings and unittests for storage.state (#3958) | Richard van der Hoff | 2018-09-27 | 1 | -8/+22 |
| | | | | I spent ages trying to figure out how I was going mad... | ||||
* | Split the state_group_cache in two (#3726) | Matthew Hodgson | 2018-08-22 | 1 | -17/+141 |
| | | | | | | | | | Splits the state_group_cache in two. One half contains normal state events; the other contains member events. The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching. Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720. | ||||
* | initial cut at a room summary API (#3574) | Matthew Hodgson | 2018-08-16 | 1 | -2/+3 |
| | |||||
* | speed up /members and add at= and membership params (#3568) | Matthew Hodgson | 2018-08-15 | 1 | -2/+64 |
| | |||||
* | Basic support for room versioning | Richard van der Hoff | 2018-08-03 | 1 | -3/+30 |
| | | | | | | | | This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event | ||||
* | switch missing_types to be a bool | Matthew Hodgson | 2018-07-25 | 1 | -4/+4 |
| | |||||
* | incorporate more review. | Matthew Hodgson | 2018-07-25 | 1 | -16/+14 |
| | |||||
* | handle the edge case for _get_some_state_from_cache where types is [] | Matthew Hodgson | 2018-07-24 | 1 | -1/+7 |
| | |||||
* | consider non-filter_type types as wildcards, thus missing from the ↵ | Matthew Hodgson | 2018-07-24 | 1 | -1/+4 |
| | | | | state-group-cache | ||||
* | incorporate more review | Matthew Hodgson | 2018-07-24 | 1 | -27/+9 |
| | |||||
* | handle case where types is [] on postgres correctly | Matthew Hodgson | 2018-07-23 | 1 | -1/+1 |
| | |||||
* | incorporate review | Matthew Hodgson | 2018-07-23 | 1 | -12/+8 |
| | |||||
* | fix thinkos; unbreak tests | Matthew Hodgson | 2018-07-19 | 1 | -2/+2 |
| | |||||
* | incorporate review | Matthew Hodgson | 2018-07-19 | 1 | -3/+4 |
| | |||||
* | add a filtered_types param to limit filtering to specific types | Matthew Hodgson | 2018-07-19 | 1 | -53/+60 |
| | |||||
* | merge develop pydoc for _get_state_for_groups | Matthew Hodgson | 2018-07-19 | 1 | -24/+43 |
|\ | |||||
| * | run isort | Amber Brown | 2018-07-09 | 1 | -2/+3 |
| | | |||||
| * | Disable partial state group caching for wildcard lookups | Richard van der Hoff | 2018-06-22 | 1 | -13/+43 |
| | | | | | | | | | | | | | | When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase. | ||||
| * | Merge pull request #3382 from matrix-org/rav/optimise_state_groups | Richard van der Hoff | 2018-06-22 | 1 | -5/+1 |
| |\ | | | | | | | Optimise state_group_cache update | ||||
| | * | Optimise state_group_cache update | Richard van der Hoff | 2018-06-11 | 1 | -5/+1 |
| | | | | | | | | | | | | | | | (1) matrix-org-hotfixes has removed the intern calls; let's do the same here. (2) remove redundant iteritems() so we can used an optimised db update. | ||||
* | | | Merge branch 'develop' into matthew/filter_members | Matthew Hodgson | 2018-06-10 | 1 | -3/+3 |
|\| | | |||||
| * | | Fix sql error in _get_state_groups_from_groups | Richard van der Hoff | 2018-06-06 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | If this was called with a `(type, None)` entry in types (which is supposed to return all state of type `type`), it would explode with a sql error. | ||||
| * | | Add hacky cache factor override system | Erik Johnston | 2018-06-04 | 1 | -2/+2 |
| |/ | |||||
* | | Merge branch 'develop' into matthew/filter_members | Matthew Hodgson | 2018-06-04 | 1 | -22/+25 |
|\| | |||||
| * | Consistently use six's iteritems and wrap lazy keys/values in list() if ↵ | Amber Brown | 2018-05-31 | 1 | -22/+25 |
| | | | | | | | | they're not meant to be lazy (#3307) | ||||
| * | fix bug #2926 (loading all state for a given type from the DB if the ↵ | Matthew Hodgson | 2018-03-13 | 1 | -7/+27 |
| | | | | | | | | | | state_key is None) (#2990) Fixes a regression that had crept in where the caching layer upholds requests for loading state which is filtered by type (but not by state_key), but the DB layer itself would interpret a missing state_key as a request to filter by null state_key rather than returning all state_keys. | ||||
* | | add pydoc | Matthew Hodgson | 2018-05-29 | 1 | -22/+54 |
| | | |||||
* | | pep8 | Matthew Hodgson | 2018-03-14 | 1 | -2/+2 |
| | | |||||
* | | make it work | Matthew Hodgson | 2018-03-13 | 1 | -4/+6 |
| | | |||||
* | | remove comment now #2969 is fixed | Matthew Hodgson | 2018-03-13 | 1 | -6/+0 |
| | | |||||
* | | merge proper fix to bug 2969 | Matthew Hodgson | 2018-03-13 | 1 | -11/+25 |
|\ \ | |||||
| * | | typoe | Matthew Hodgson | 2018-03-13 | 1 | -1/+1 |
| | | | |||||
| * | | disable optimisation for searching for state groups | Matthew Hodgson | 2018-03-13 | 1 | -3/+13 |
| | | | | | | | | | | | | when type filter includes wildcards on state_key | ||||
| * | | build where_clause sanely | Matthew Hodgson | 2018-03-13 | 1 | -6/+4 |
| | | | |||||
| * | | PR feedbackz | Matthew Hodgson | 2018-03-13 | 1 | -3/+2 |
| | | | |||||
| * | | fix bug #2926 | Matthew Hodgson | 2018-03-13 | 1 | -4/+17 |
| |/ | |||||
* | | correctly handle None state_keys | Matthew Hodgson | 2018-03-12 | 1 | -10/+16 |
| | | | | | | | | and fix include_other_types thinko | ||||
* | | fix sqlite where clause | Matthew Hodgson | 2018-03-12 | 1 | -2/+3 |
| | | |||||
* | | typos | Matthew Hodgson | 2018-03-11 | 1 | -0/+2 |
| | | |||||
* | | WIP experiment in lazyloading room members | Matthew Hodgson | 2018-03-11 | 1 | -5/+42 |
|/ | |||||
* | Don't serialize current state over replication | Erik Johnston | 2018-02-15 | 1 | -0/+14 |
| | |||||
* | Fix state group storage bug in workers | Erik Johnston | 2018-02-15 | 1 | -41/+41 |
| | | | | | We needed to move `_count_state_group_hops_txn` to the StateGroupWorkerStore. | ||||
* | Store state groups separately from events (#2784) | Erik Johnston | 2018-02-06 | 1 | -96/+100 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints | ||||
* | Pull out bits of StateStore to a mixin | Richard van der Hoff | 2017-11-14 | 1 | -204/+220 |
| | | | | | | | | | ... so that we don't need to secretly gut-wrench it for use in the slaved stores. I haven't done the other stores yet, but we should. I'm tired of the workers breaking every time we tweak the stores because I forgot to gut-wrench the right method. fixes https://github.com/matrix-org/synapse/issues/2655. | ||||
* | Revert "Revert "move _state_group_cache to statestore"" | Richard van der Hoff | 2017-11-14 | 1 | -7/+12 |
| | | | | | | | We're going to fix this properly on this branch, so that the _state_group_cache can end up in StateGroupReadStore. This reverts commit ab335edb023d66cd0be439e045b10ca104b73cb5. | ||||
* | Make __init__ consitstent across Store heirarchy | Richard van der Hoff | 2017-11-13 | 1 | -2/+2 |
| | | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores) | ||||
* | Revert "move _state_group_cache to statestore" | Erik Johnston | 2017-11-13 | 1 | -12/+7 |
| | | | | This reverts commit f5cf3638e9c6086e1c33ddad8eda9298cf53a58e. | ||||
* | move _state_group_cache to statestore | Richard van der Hoff | 2017-11-07 | 1 | -7/+12 |
| | | | | this is internal to statestore, so let's keep it there. | ||||
* | Prefill forward extrems and event to state groups | Erik Johnston | 2017-06-29 | 1 | -2/+8 |
| | |||||
* | Fix replication | Erik Johnston | 2017-06-09 | 1 | -1/+1 |
| | |||||
* | Comments | Erik Johnston | 2017-06-09 | 1 | -0/+3 |
| | |||||
* | Cache state deltas | Erik Johnston | 2017-06-09 | 1 | -3/+12 |
| | |||||
* | Ensure we don't use unpersisted state group as prev group | Erik Johnston | 2017-06-08 | 1 | -0/+13 |
| | |||||
* | Comments | Erik Johnston | 2017-06-07 | 1 | -0/+6 |
| | |||||
* | Faster cache for get_joined_hosts | Erik Johnston | 2017-05-25 | 1 | -0/+33 |
| | |||||
* | Make get_state_groups_from_groups faster. | Erik Johnston | 2017-05-17 | 1 | -29/+11 |
| | | | | | | | | | Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy. | ||||
* | Take a copy before prefilling, as it may be a frozendict | Erik Johnston | 2017-05-16 | 1 | -1/+1 |
| | |||||
* | Update comment | Erik Johnston | 2017-05-15 | 1 | -1/+2 |
| | |||||
* | Comments | Erik Johnston | 2017-05-15 | 1 | -0/+3 |
| | |||||
* | Prefill state caches | Erik Johnston | 2017-05-15 | 1 | -0/+8 |
| | |||||
* | Revert "Prefill state caches" | Erik Johnston | 2017-05-04 | 1 | -12/+0 |
| | |||||
* | Update comment | Erik Johnston | 2017-05-03 | 1 | -1/+2 |
| | |||||
* | Comments | Erik Johnston | 2017-05-03 | 1 | -0/+3 |
| | |||||
* | Prefill state caches | Erik Johnston | 2017-05-02 | 1 | -0/+8 |
| | |||||
* | Make state caches cache in ascii | Erik Johnston | 2017-04-25 | 1 | -2/+3 |
| | |||||
* | Reduce _get_state_group_for_event cache size | Erik Johnston | 2017-04-25 | 1 | -1/+1 |
| | |||||
* | Remove unused cache | Erik Johnston | 2017-04-24 | 1 | -6/+1 |
| | |||||
* | Comment | Erik Johnston | 2017-04-07 | 1 | -0/+9 |
| | |||||
* | Speed up get_current_state_ids | Erik Johnston | 2017-04-07 | 1 | -10/+17 |
| | | | | | | | | | | | | Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation. | ||||
* | Increase cache size for _get_state_group_for_event | Erik Johnston | 2017-03-29 | 1 | -1/+1 |
| | |||||
* | Replace some calls to cursor_to_dict | Erik Johnston | 2017-03-24 | 1 | -4/+4 |
| | | | | | cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to. | ||||
* | Use iter(items|values) | Erik Johnston | 2017-03-24 | 1 | -27/+28 |
| | |||||
* | User Cursor.__iter__ instead of fetchall | Erik Johnston | 2017-03-23 | 1 | -2/+1 |
| | | | | This prevents unnecessary construction of lists | ||||
* | Avoid resetting state on rejected events | Richard van der Hoff | 2017-03-17 | 1 | -0/+10 |
| | | | | | | | When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935. | ||||
* | Get current state by using current_state_events table | Erik Johnston | 2017-03-10 | 1 | -1/+13 |
| | |||||
* | Fix bugs in the /keys/changes api | Richard van der Hoff | 2017-02-14 | 1 | -1/+13 |
| | | | | | | | | | | | | | | | * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments. | ||||
* | Add an index to make membership queries faster | Erik Johnston | 2017-01-31 | 1 | -0/+8 |
| | |||||
* | Insert delta of current_state_events to be more efficient | Erik Johnston | 2017-01-20 | 1 | -52/+0 |
| | |||||
* | Up cache max entries for state | Erik Johnston | 2017-01-16 | 1 | -1/+1 |
| | |||||
* | Increase cache size limit | Erik Johnston | 2017-01-16 | 1 | -1/+1 |
| | |||||
* | Optionally measure size of cache by sum of length of values | Erik Johnston | 2017-01-13 | 1 | -1/+1 |
| | |||||
* | fix annoying typos | Matthew Hodgson | 2017-01-05 | 1 | -1/+1 |
| | |||||
* | Correctly intern keys in state cache | Erik Johnston | 2016-11-08 | 1 | -1/+4 |
| | |||||
* | Remove duplication | Erik Johnston | 2016-09-27 | 1 | -31/+19 |
| | |||||
* | Remove unused variable | Erik Johnston | 2016-09-27 | 1 | -3/+0 |
| | |||||
* | Fix perf of fetching state in SQLite | Erik Johnston | 2016-09-26 | 1 | -20/+41 |
| | |||||
* | Fix reindex | Erik Johnston | 2016-09-14 | 1 | -15/+16 |
| | |||||
* | Ensure we don't mutate state cache entries | Erik Johnston | 2016-09-14 | 1 | -20/+25 |
| | |||||
* | Create new index concurrently | Erik Johnston | 2016-09-12 | 1 | -1/+1 |
| | |||||
* | Change state fetch query for postgres to be faster | Erik Johnston | 2016-09-12 | 1 | -19/+35 |
| | | | | | | It turns out that postgres doesn't like doing a list of OR's and is about 1000x slower, so we just issue a query for each specific type seperately. | ||||
* | Reindex state_groups_state after pruning | Erik Johnston | 2016-09-08 | 1 | -0/+33 |
| | |||||
* | Temporarily disable sequential scans for state fetching | Erik Johnston | 2016-09-08 | 1 | -0/+4 |
| | |||||
* | Comment | Erik Johnston | 2016-09-07 | 1 | -0/+4 |
| | |||||
* | Add appopriate framing clause | Erik Johnston | 2016-09-07 | 1 | -0/+1 |
| | |||||
* | Use windowing function to make use of index | Erik Johnston | 2016-09-07 | 1 | -5/+5 |
| | |||||
* | Scale the batch size so that we're not bitten by the minimum | Erik Johnston | 2016-09-05 | 1 | -1/+5 |
| | |||||
* | Correctly handle reindexing state groups that already have an edge | Erik Johnston | 2016-09-05 | 1 | -0/+19 |
| | |||||
* | Take value in a better way | Erik Johnston | 2016-09-05 | 1 | -2/+2 |
| | |||||
* | Comment about sqlite and WITH RECURSIVE | Erik Johnston | 2016-09-05 | 1 | -0/+4 |
| | |||||
* | Add comments | Erik Johnston | 2016-09-05 | 1 | -41/+38 |
| | |||||
* | Add upgrade script that will slowly prune state_groups_state entries | Erik Johnston | 2016-09-05 | 1 | -75/+203 |
| | |||||
* | Limit the length of state chains | Erik Johnston | 2016-09-02 | 1 | -22/+78 |
| | |||||
* | Move to storing state_groups_state as deltas | Erik Johnston | 2016-09-01 | 1 | -59/+102 |
| | |||||
* | Use state_groups table to test existence | Erik Johnston | 2016-08-31 | 1 | -1/+1 |
| | |||||
* | Correctly handle the difference between prev and current state | Erik Johnston | 2016-08-31 | 1 | -3/+0 |
| | |||||
* | Generate state group ids in state layer | Erik Johnston | 2016-08-31 | 1 | -7/+17 |
| | |||||
* | Make _state_groups_id_gen a normal IdGenerator | Erik Johnston | 2016-08-30 | 1 | -3/+0 |
| | |||||
* | Make sync not pull out full state | Erik Johnston | 2016-08-25 | 1 | -0/+33 |
| | |||||
* | Replace context.current_state with context.current_state_ids | Erik Johnston | 2016-08-25 | 1 | -8/+8 |
| | |||||
* | Pull out event ids rather than full events for state | Erik Johnston | 2016-08-25 | 1 | -17/+38 |
| | |||||
* | Remove lru option | Erik Johnston | 2016-08-19 | 1 | -2/+2 |
| | |||||
* | Add cache to _get_state_groups_from_groups | Erik Johnston | 2016-04-19 | 1 | -3/+16 |
| | |||||
* | Make the cache objects be per instance rather than being global | Mark Haines | 2016-04-06 | 1 | -2/+2 |
| | |||||
* | Merge pull request #686 from matrix-org/markjh/doc_strings | Mark Haines | 2016-04-01 | 1 | -5/+8 |
|\ | | | | | Use google style doc strings. | ||||
| * | Use google style doc strings. | Mark Haines | 2016-04-01 | 1 | -5/+8 |
| | | | | | | | | | | | | | | pycharm supports them so there is no need to use the other format. Might as well convert the existing strings to reduce the risk of people accidentally cargo culting the wrong doc string format. | ||||
* | | Use a stream id generator for backfilled ids | Mark Haines | 2016-04-01 | 1 | -1/+1 |
|/ | |||||
* | Add a replication stream for state groups | Mark Haines | 2016-03-30 | 1 | -0/+30 |
| | |||||
* | Use a stream id generator to assign state group ids | Mark Haines | 2016-03-30 | 1 | -8/+8 |
| | |||||
* | Don't bother interning keys that are already interned | Erik Johnston | 2016-03-23 | 1 | -3/+1 |
| | |||||
* | Intern all the things | Erik Johnston | 2016-03-23 | 1 | -5/+5 |
| | |||||
* | String intern | Erik Johnston | 2016-03-23 | 1 | -3/+9 |
| | |||||
* | Don't cache events in get_current_state_for_key | Erik Johnston | 2016-03-23 | 1 | -7/+9 |
| | |||||
* | Don't cache events in _state_group_cache | Erik Johnston | 2016-03-22 | 1 | -60/+48 |
| | | | | | | | | | Instead, simply cache the event ids, relying on the event cache to cache the actual events. The problem was that while the state groups cache was limited in the number of groups it could hold, each individual group could consist of thousands of events. | ||||
* | Load the current id in the IdGenerator constructor | Mark Haines | 2016-03-01 | 1 | -1/+1 |
| | | | | | | | | | Rather than loading them lazily. This allows us to remove all the yield statements and spurious arguments for the get_next methods. It also allows us to replace all instances of get_next_txn with get_next since get_next no longer needs to access the db. | ||||
* | Batch fetch _get_state_groups_from_groups | Erik Johnston | 2016-02-10 | 1 | -32/+34 |
| | |||||
* | Use _simple_select_many for _get_state_group_for_events | Erik Johnston | 2016-02-10 | 1 | -16/+10 |
| | |||||
* | copyrights | Matthew Hodgson | 2016-01-07 | 1 | -1/+1 |
| | |||||
* | Fix a few race conditions in the state calculation | Richard van der Hoff | 2015-11-13 | 1 | -0/+14 |
| | | | | | | | | | | | | | | | | | | | Be a bit more careful about how we calculate the state to be returned by /sync. In a few places, it was possible for /sync to return slightly later state than that represented by the next_batch token and the timeline. In particular, the following cases were susceptible: * On a full state sync, for an active room * During a per-room incremental sync with a timeline gap * When the user has just joined a room. (Refactor check_joined_room to make it less magical) Also, use store.get_state_for_events() (and thus the existing stategroups) to calculate the state corresponding to a particular sync position, rather than state_handler.compute_event_context(), which recalculates from first principles (and tends to miss some state). Merged from PR https://github.com/matrix-org/synapse/pull/372 | ||||
* | Remove unused room_id parameter | Erik Johnston | 2015-10-12 | 1 | -6/+5 |
| | |||||
* | synapse/storage/state.py: _make_group_id was unused | Mark Haines | 2015-09-23 | 1 | -6/+0 |
| | |||||
* | Actually check if event_id isn't returned by _get_state_groups | Erik Johnston | 2015-08-21 | 1 | -2/+9 |
| | |||||
* | Remove newline because vertical whitespace makes mjark sad | Erik Johnston | 2015-08-18 | 1 | -2/+1 |
| | |||||
* | Rejig the code to make it nicer | Erik Johnston | 2015-08-18 | 1 | -3/+2 |
| | |||||
* | Ensure we never return a None event from _get_state_for_groups | Erik Johnston | 2015-08-18 | 1 | -3/+8 |
| | |||||
* | Reduce cache size from obscenely large to quite large | Erik Johnston | 2015-08-18 | 1 | -1/+1 |
| | |||||
* | Fix bug where we were leaking None into state event lists | Erik Johnston | 2015-08-17 | 1 | -2/+3 |
| | |||||
* | Remove some vertical space | Erik Johnston | 2015-08-13 | 1 | -36/+9 |
| | |||||
* | Replace list comprehension | Erik Johnston | 2015-08-13 | 1 | -8/+3 |
| | |||||
* | Comment | Erik Johnston | 2015-08-12 | 1 | -1/+2 |
| | |||||
* | Explain why we're prefilling dict with Nones | Erik Johnston | 2015-08-12 | 1 | -0/+4 |
| | |||||
* | Better variable name | Erik Johnston | 2015-08-12 | 1 | -5/+5 |
| | |||||
* | Split _get_state_for_group_from_cache into two | Erik Johnston | 2015-08-12 | 1 | -32/+53 |
| | |||||
* | Move all the caches into their own package, synapse.util.caches | Erik Johnston | 2015-08-11 | 1 | -1/+4 |
| | |||||
* | Comments | Erik Johnston | 2015-08-11 | 1 | -3/+11 |
| | |||||
* | Fix application of ACLs | Erik Johnston | 2015-08-11 | 1 | -3/+3 |
| | |||||
* | Fix state cache | Erik Johnston | 2015-08-11 | 1 | -36/+49 |
| | |||||
* | Clean up StateStore | Erik Johnston | 2015-08-10 | 1 | -62/+11 |
| | |||||
* | Remove unused function | Erik Johnston | 2015-08-10 | 1 | -7/+0 |
| | |||||
* | Do bounds check | Erik Johnston | 2015-08-10 | 1 | -1/+4 |
| | |||||
* | Line length | Erik Johnston | 2015-08-10 | 1 | -1/+2 |
| | |||||
* | Merge branch 'erikj/cache_varargs_interface' of ↵ | Erik Johnston | 2015-08-07 | 1 | -1/+0 |
|\ | | | | | | | github.com:matrix-org/synapse into erikj/dictionary_cache | ||||
| * | Remove unnecessary cache | Erik Johnston | 2015-08-07 | 1 | -2/+1 |
| | | |||||
* | | Batch up various DB requests for event -> state | Erik Johnston | 2015-08-07 | 1 | -77/+142 |
| | | |||||
* | | Remove unncessary cache | Erik Johnston | 2015-08-07 | 1 | -1/+0 |
| | | |||||
* | | Store absence of state in cache | Erik Johnston | 2015-08-07 | 1 | -6/+16 |
| | | |||||
* | | Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse ↵ | Erik Johnston | 2015-08-05 | 1 | -3/+2 |
|\| | | | | | | | into erikj/dictionary_cache | ||||
| * | Add support for using keyword arguments with cached functions | Erik Johnston | 2015-07-27 | 1 | -3/+2 |
| | | |||||
* | | Use dictionary cache to do group -> state fetching | Erik Johnston | 2015-08-05 | 1 | -65/+126 |
| | | |||||
* | | Add comment | Erik Johnston | 2015-08-04 | 1 | -0/+15 |
| | | |||||
* | | Speed up event filtering (for ACL) logic | Erik Johnston | 2015-08-04 | 1 | -38/+79 |
|/ | |||||
* | Merge pull request #193 from matrix-org/erikj/bulk_persist_event | Erik Johnston | 2015-07-14 | 1 | -12/+24 |
|\ | | | | | Add bulk insert events API | ||||
| * | Add bulk insert events API | Erik Johnston | 2015-06-25 | 1 | -12/+24 |
| | | |||||
* | | Add basic impl for room history ACL on GET /messages client API | Erik Johnston | 2015-07-02 | 1 | -2/+61 |
|/ | |||||
* | Add cache to get_state_groups | Erik Johnston | 2015-06-03 | 1 | -8/+12 |
| | |||||
* | Merge branch 'master' of github.com:matrix-org/synapse into develop | Erik Johnston | 2015-05-22 | 1 | -1/+24 |
|\ | |||||
| * | Add a cache for get_current_state with state_key | Mark Haines | 2015-05-21 | 1 | -1/+24 |
| | | |||||
* | | Don't fetch prev_content for current_state | Erik Johnston | 2015-05-15 | 1 | -1/+1 |
| | | |||||
* | | Make store.get_current_state fetch events asyncly | Erik Johnston | 2015-05-15 | 1 | -3/+4 |
| | | |||||
* | | Move fetching of events into their own transactions | Erik Johnston | 2015-05-14 | 1 | -2/+0 |
| | | |||||
* | | Refactor _get_events | Erik Johnston | 2015-05-14 | 1 | -1/+1 |
| | | |||||
* | | Fetch events from events_id in their own transactions | Erik Johnston | 2015-05-13 | 1 | -5/+5 |
| | | |||||
* | | Fetch events in bulk | Erik Johnston | 2015-05-13 | 1 | -21/+1 |
| | | |||||
* | | Don't fetch redaction and rejection stuff for each event, so we can use ↵ | Erik Johnston | 2015-05-13 | 1 | -2/+5 |
| | | | | | | | | index only scan | ||||
* | | Temp turn off checking for rejections and redactions | Erik Johnston | 2015-05-13 | 1 | -5/+2 |
| | | |||||
* | | defer.gatherResults loop | Erik Johnston | 2015-05-13 | 1 | -1/+7 |
| | | |||||
* | | Batch fetching of events for state groups | Erik Johnston | 2015-05-13 | 1 | -1/+24 |
| | | |||||
* | | Load events for state group seperately | Erik Johnston | 2015-05-13 | 1 | -3/+9 |
|/ | |||||
* | Add support for using executemany | Erik Johnston | 2015-05-05 | 1 | -7/+9 |
| | |||||
* | Remove pointless join | Erik Johnston | 2015-04-30 | 1 | -5/+4 |
| | |||||
* | Need more yield | Erik Johnston | 2015-04-30 | 1 | -1/+1 |
| | |||||
* | Add missing param | Erik Johnston | 2015-04-30 | 1 | -1/+1 |
| | |||||
* | Fix up get_current_state and get_room_name_and_aliases queries to parse ↵ | Erik Johnston | 2015-04-30 | 1 | -16/+19 |
| | | | | events in transaction | ||||
* | Don't use sub queries, it makes postgres sad | Erik Johnston | 2015-04-30 | 1 | -11/+5 |
| | |||||
* | Don't needlessly join on state_events | Erik Johnston | 2015-04-30 | 1 | -3/+2 |
| | |||||
* | For backwards compat, make state_groups.id have a type of int, not varchar | Erik Johnston | 2015-04-13 | 1 | -1/+1 |
| | |||||
* | PEP8 | Erik Johnston | 2015-04-07 | 1 | -1/+0 |
| | |||||
* | Don't use AUTOINCREMENT, use an in memory version | Erik Johnston | 2015-04-07 | 1 | -3/+3 |
| | |||||
* | Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. | Erik Johnston | 2015-03-23 | 1 | -1/+0 |
| | |||||
* | Merge branch 'develop' of github.com:matrix-org/synapse into mysql | Erik Johnston | 2015-03-20 | 1 | -0/+1 |
|\ | |||||
| * | Rearrange storage modules | Erik Johnston | 2015-03-20 | 1 | -0/+32 |
| | |