summary refs log tree commit diff
path: root/synapse/storage/schema/state (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Add DB delta to remove the old state group deletion job (#18284)Devon Hudson2025-03-271-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This background DB delta removes the old state group deletion background update from the `background_updates` table if it exists. The `delete_unreferenced_state_groups_bg_update` update should only exist in that table if a homeserver ran v1.126.0rc1/v1.126.0rc2, and rolled back or forward to any other version of Synapse before letting the update finish. ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
* Delete unreferenced state groups in background (#18254)Devon Hudson2025-03-211-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR fixes #18154 to avoid de-deltaing state groups which resulted in DB size temporarily increasing until the DB was `VACUUM`'ed. As a result, less state groups will get deleted now. It also attempts to improve performance by not duplicating work when processing state groups it has already processed in previous iterations. ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Erik Johnston <erikj@element.io>
* Revert the background job to clear unreferenced state groups (that was ↵reivilibre2025-03-071-16/+0
| | | | | | | | | | | | | | | | | introduced in v1.126.0rc1), due to a suspected issue that causes increased disk usage. (#18222) Revert "Add background job to clear unreferenced state groups (#18154)" This mechanism is suspected of inserting large numbers of rows into `state_groups_state`, thus unreasonably increasing disk usage. See: https://github.com/element-hq/synapse/issues/18217 This reverts commit 5121f9210c989fcc909e78195133876dff3bc9b9 (#18154). --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
* Add background job to clear unreferenced state groups (#18154)Devon Hudson2025-02-251-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #18150 ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [x] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Erik Johnston <erikj@element.io>
* Add locking to more safely delete state groups: Part 1 (#18107)Erik Johnston2025-02-031-0/+39
| | | | | | | | | | | | | | | | Currently we don't really have anything that stops us from deleting state groups when an in-flight event references it. This is a fairly rare race currently, but we want to be able to more aggressively delete state groups so it is important to address this to ensure that the database remains valid. This implements the locking, but doesn't actually use it. See the class docstring of the new data store for an explanation for how this works. --------- Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>
* Merge remote-tracking branch 'gitlab/clokep/license-license' into new_developErik Johnston2023-12-138-10/+149
|\
| * Update license headersPatrick Cloke2023-11-219-10/+168
| |
* | Revert postgres logical replication deltaas v1.98.0rc1David Robertson2023-12-052-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts two commits: 0bb8e418a41c6f583ca9d705b400e37e2308a534 "Fix postgres schema after dropping old tables (#16730)" and 51e4e35653f98c3f61222fbdbdb1dcb8864f7fca "Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (take 2, now with no added deadlocks!) (#16658)" and also amends the changelog.
* | Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit ↵reivilibre2023-12-042-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | one. This should allow use of Postgres logical replication. (take 2, now with no added deadlocks!) (#16658) * Add `ALTER TABLE ... REPLICA IDENTITY ...` for individual tables We can't combine them into one file as it makes it likely to hit a deadlock if Synapse is running, as it only takes one other transaction to access two tables in a different order to the schema delta. * Add notes * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Re-introduce REPLICA IDENTITY test --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* | Remove old full schema dumps. (#16697)Patrick Cloke2023-11-282-58/+0
|/ | | | These are not useful and make it difficult to search for table definitions, etc.
* Revert "Add a Postgres `REPLICA IDENTITY` to tables that do not have an ↵Erik Johnston2023-11-161-30/+0
| | | | | implicit one. This should allow use of Postgres logical replication. (#16456)" (#16651) This reverts commit 69afe3f7a0d89f3422ddbd3aa16bc9bbc01056eb.
* Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit ↵reivilibre2023-11-131-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | one. This should allow use of Postgres logical replication. (#16456) * Add Postgres replica identities to tables that don't have an implicit one Fixes #16224 * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Move the delta to version 83 as we missed the boat for 82 * Add a test that all tables have a REPLICA IDENTITY * Extend the test to include when indices are deleted * isort * black * Fully qualify `oid` as it is a 'hidden attribute' in Postgres 11 * Update tests/storage/test_database.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add missed tables --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add type hints to schema deltas (#15497)Patrick Cloke2023-04-271-6/+4
| | | | | | | | | | Cleans-up the schema delta files: * Removes no-op functions. * Adds missing type hints to function parameters. * Fixes any issues with type hints. This also renames one (very old) schema delta to avoid a conflict that mypy complains about.
* Snapshot schema 72 (#13873)David Robertson2022-09-262-0/+50
| | | Including another batch of fixes to the schema dump script
* Remove incorrect migration file from `state` logical DB (#13788)David Robertson2022-09-141-37/+0
| | | | | | | | | | | | | * Remove incorrect migration file from `state` logical DB The table `ex_outlier_stream` is part of the `main` logical DB; it should not have been created in the `state` logical DB. We remove this migration now as a tidy-up. Note: we cannot `DROP TABLE IF EXISTS ex_outlier_stream` in a new migration, because some (most) instances of Synapse host both of these logical DBs on the same DB cluster. * Changelog
* Updates to the schema dump script (#13770)David Robertson2022-09-131-0/+4
|
* Add a unique index to `state_group_edges` to prevent duplicates being ↵reivilibre2022-05-191-0/+17
| | | | accidentally introduced and the consequential impact to performance. (#12687)
* Reduce likelihood of Postgres table scanning `state_groups_state`. (#10359)Erik Johnston2021-07-151-0/+34
| | | | | | | | The postgres statistics collector sometimes massively underestimates the number of distinct state groups are in the `state_groups_state`, which can cause postgres to use table scans for queries for multiple state groups. We fix this by manually setting `n_distinct` on the column.
* Reorganise the database schema directories (#9932)Richard van der Hoff2021-05-0710-0/+233
The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate. It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-08-18 07:41:31 +0000