summary refs log tree commit diff
path: root/synapse/storage/schema/state/delta (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Add DB delta to remove the old state group deletion job (#18284)Devon Hudson2025-03-271-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This background DB delta removes the old state group deletion background update from the `background_updates` table if it exists. The `delete_unreferenced_state_groups_bg_update` update should only exist in that table if a homeserver ran v1.126.0rc1/v1.126.0rc2, and rolled back or forward to any other version of Synapse before letting the update finish. ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
* Delete unreferenced state groups in background (#18254)Devon Hudson2025-03-211-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR fixes #18154 to avoid de-deltaing state groups which resulted in DB size temporarily increasing until the DB was `VACUUM`'ed. As a result, less state groups will get deleted now. It also attempts to improve performance by not duplicating work when processing state groups it has already processed in previous iterations. ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Erik Johnston <erikj@element.io>
* Revert the background job to clear unreferenced state groups (that was ↵reivilibre2025-03-071-16/+0
| | | | | | | | | | | | | | | | | introduced in v1.126.0rc1), due to a suspected issue that causes increased disk usage. (#18222) Revert "Add background job to clear unreferenced state groups (#18154)" This mechanism is suspected of inserting large numbers of rows into `state_groups_state`, thus unreasonably increasing disk usage. See: https://github.com/element-hq/synapse/issues/18217 This reverts commit 5121f9210c989fcc909e78195133876dff3bc9b9 (#18154). --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
* Add background job to clear unreferenced state groups (#18154)Devon Hudson2025-02-251-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #18150 ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Erik Johnston <erikj@element.io>
* Add locking to more safely delete state groups: Part 1 (#18107)Erik Johnston2025-02-031-0/+39
| | | | | | | | | | | | | | | | Currently we don't really have anything that stops us from deleting state groups when an in-flight event references it. This is a fairly rare race currently, but we want to be able to more aggressively delete state groups so it is important to address this to ensure that the database remains valid. This implements the locking, but doesn't actually use it. See the class docstring of the new data store for an explanation for how this works. --------- Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>
* Update license headersPatrick Cloke2023-11-218-10/+149
|
* Revert "Add a Postgres `REPLICA IDENTITY` to tables that do not have an ↵Erik Johnston2023-11-161-30/+0
| | | | | implicit one. This should allow use of Postgres logical replication. (#16456)" (#16651) This reverts commit 69afe3f7a0d89f3422ddbd3aa16bc9bbc01056eb.
* Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit ↵reivilibre2023-11-131-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | one. This should allow use of Postgres logical replication. (#16456) * Add Postgres replica identities to tables that don't have an implicit one Fixes #16224 * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Move the delta to version 83 as we missed the boat for 82 * Add a test that all tables have a REPLICA IDENTITY * Extend the test to include when indices are deleted * isort * black * Fully qualify `oid` as it is a 'hidden attribute' in Postgres 11 * Update tests/storage/test_database.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add missed tables --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add type hints to schema deltas (#15497)Patrick Cloke2023-04-271-6/+4
| | | | | | | | | | Cleans-up the schema delta files: * Removes no-op functions. * Adds missing type hints to function parameters. * Fixes any issues with type hints. This also renames one (very old) schema delta to avoid a conflict that mypy complains about.
* Remove incorrect migration file from `state` logical DB (#13788)David Robertson2022-09-141-37/+0
| | | | | | | | | | | | | * Remove incorrect migration file from `state` logical DB The table `ex_outlier_stream` is part of the `main` logical DB; it should not have been created in the `state` logical DB. We remove this migration now as a tidy-up. Note: we cannot `DROP TABLE IF EXISTS ex_outlier_stream` in a new migration, because some (most) instances of Synapse host both of these logical DBs on the same DB cluster. * Changelog
* Updates to the schema dump script (#13770)David Robertson2022-09-131-0/+4
|
* Add a unique index to `state_group_edges` to prevent duplicates being ↵reivilibre2022-05-191-0/+17
| | | | accidentally introduced and the consequential impact to performance. (#12687)
* Reduce likelihood of Postgres table scanning `state_groups_state`. (#10359)Erik Johnston2021-07-151-0/+34
| | | | | | | | The postgres statistics collector sometimes massively underestimates the number of distinct state groups are in the `state_groups_state`, which can cause postgres to use table scans for queries for multiple state groups. We fix this by manually setting `n_distinct` on the column.
* Reorganise the database schema directories (#9932)Richard van der Hoff2021-05-078-0/+175
The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate. It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-08-17 21:56:18 +0000