summary refs log tree commit diff
path: root/synapse/storage/databases/main/transactions.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fix filtering of rooms when supplying the `destination` query parameter to ↵Andrew Morgan2024-04-261-0/+1
| | | | `/_synapse/admin/v1/federation/destinations/<destination>/rooms` (#17077)
* Ensure that pending to-device events are sent over federation at startup ↵Richard van der Hoff2024-03-221-20/+79
| | | | | | | | | | | | | | (#16925) Fixes https://github.com/element-hq/synapse/issues/16680, as well as a related bug, where servers which we had *never* successfully sent an event to would not be retried. In order to fix the case of pending to-device messages, we hook into the existing `wake_destinations_needing_catchup` process, by extending it to look for destinations that have pending to-device messages. The federation transmission loop then attempts to send the pending to-device messages as normal.
* Correctly mention previous copyright (#16820)Erik Johnston2024-01-231-0/+1
| | | | | During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.
* Update license headersPatrick Cloke2023-11-211-10/+16
|
* Convert simple_select_one_txn and simple_select_one to return tuples. (#16612)Patrick Cloke2023-11-091-12/+8
|
* Remove more usages of cursor_to_dict. (#16551)Patrick Cloke2023-10-261-4/+24
| | | Mostly to improve type safety.
* Convert simple_select_many_batch, simple_select_many_txn to tuples. (#16444)Patrick Cloke2023-10-111-9/+19
|
* Convert simple_select_list_paginate_txn to return tuples. (#16433)Patrick Cloke2023-10-061-11/+16
|
* Return immutable objects for cachedList decorators (#16350)Patrick Cloke2023-09-191-2/+2
|
* Don't wake up destination transaction queue if they're not due for retry. ↵Erik Johnston2023-09-041-3/+23
| | | | (#16223)
* Always update `retry_last_ts` (#16164)Erik Johnston2023-08-231-3/+3
|
* Properly update retry_last_ts when hitting the maximum retry interval (#16156)DeepBlueV7.X2023-08-231-1/+3
| | | | | | | | | | | | | | | | | | | | | * Properly update retry_last_ts when hitting the maximum retry interval This was broken in 1.87 when the maximum retry interval got changed from almost infinite to a week (and made configurable). fixes #16101 Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de> * Add changelog * Change fix + add test * Add comment --------- Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de> Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
* Remove unused store method `_set_destination_retry_timings_emulated`. (#15266)reivilibre2023-03-141-54/+2
|
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-221-1/+0
|
* Add helper to parse an enum from query args & use it. (#14956)Patrick Cloke2023-02-011-4/+9
| | | | | | | | The `parse_enum` helper pulls an enum value from the query string (by delegating down to the parse_string helper with values generated from the enum). This is used to pull out "f" and "b" in most places and then we thread the resulting Direction enum throughout more code.
* Require SQLite >= 3.27.0 (#13760)David Robertson2022-09-091-21/+9
|
* Add admin API to get a list of federated rooms (#11658)Dirk Klimpel2022-01-251-0/+48
|
* Fix get federation status of destination if no error occured (#11593)Dirk Klimpel2022-01-051-0/+11
|
* Add type hints to `synapse/storage/databases/main/transactions.py` (#11589)Dirk Klimpel2021-12-161-24/+25
|
* Type hint the constructors of the data store classes (#11555)Sean Quah2021-12-131-2/+11
|
* Add admin API to get some information about federation status (#11407)Dirk Klimpel2021-12-061-0/+70
|
* Add type hints for most `HomeServer` parameters (#11095)Sean Quah2021-10-221-2/+5
|
* Use direct references for some configuration variables (#10798)Patrick Cloke2021-09-131-1/+1
| | | | Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
* Replace `or_ignore` in `simple_insert` with `simple_upsert` (#10442)Erik Johnston2021-07-221-3/+5
| | | | | | | | Now that we have `simple_upsert` that should be used in preference to trying to insert and looking for an exception. The main benefit is that we ERROR message don't get written to postgres logs. We also have tidy up the return value on `simple_upsert`, rather than having a tri-state of inserted/not-inserted/unknown.
* Don't hammer the database for destination retry timings every ~5mins (#10036)Erik Johnston2021-05-211-29/+37
|
* Revert "Experimental Federation Speedup (#9702)"Andrew Morgan2021-04-281-12/+16
| | | | This reverts commit 05e8c70c059f8ebb066e029bc3aa3e0cefef1019.
* Experimental Federation Speedup (#9702)Jonathan de Jong2021-04-141-16/+12
| | | | | This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max). Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Fix federation stall on concurrent access errors (#9639)Jonathan de Jong2021-03-231-36/+9
|
* Don't go into federation catch up mode so easily (#9561)Erik Johnston2021-03-151-5/+5
| | | | | | | | | | Federation catch up mode is very inefficient if the number of events that the remote server has missed is small, since handling gaps can be very expensive, c.f. #9492. Instead of going into catch up mode whenever we see an error, we instead do so only if we've backed off from trying the remote for more than an hour (the assumption being that in such a case it is more than a transient failure).
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-6/+13
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Fix some typos.Patrick Cloke2021-02-121-1/+1
|
* Fix get destinations to catch up query. (#9114)Erik Johnston2021-01-141-13/+11
| | | | t was doing a sequential scan on `destination_rooms`, which took minutes.
* Use autocommit mode for single statement DB functions. (#8542)Erik Johnston2020-10-141-31/+45
| | | | | | | | | | | | | Autocommit means that we don't wrap the functions in transactions, and instead get executed directly. Introduced in #8456. This will help: 1. reduce the number of `could not serialize access due to concurrent delete` errors that we see (though there are a few functions that often cause serialization errors that we don't fix here); 2. improve the DB performance, as it no longer needs to deal with the overhead of `REPEATABLE READ` isolation levels; and 3. improve wall clock speed of these functions, as we no longer need to send `BEGIN` and `COMMIT` to the DB. Some notes about the differences between autocommit mode and our default `REPEATABLE READ` transactions: 1. Currently `autocommit` only applies when using PostgreSQL, and is ignored when using SQLite (due to silliness with [Twisted DB classes](https://twistedmatrix.com/trac/ticket/9998)). 2. Autocommit functions may get retried on error, which means they can get applied *twice* (or more) to the DB (since they are not in a transaction the previous call would not get rolled back). This means that the functions need to be idempotent (or otherwise not care about being called multiple times). Read queries, simple deletes, and updates/upserts that replace rows (rather than generating new values from existing rows) are all idempotent. 3. Autocommit functions no longer get executed in [`REPEATABLE READ`](https://www.postgresql.org/docs/current/transaction-iso.html) isolation level, and so data can change queries, which is fine for single statement queries.
* Move additional tasks to the background worker (#8458)Patrick Cloke2020-10-071-20/+22
|
* Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322)reivilibre2020-09-181-2/+64
| | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Fix _set_destination_retry_timings This came about because the code assumed that retry_interval could not be NULL — which has been challenged by catch-up.
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-181-1/+1
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Catch-up after Federation Outage (split, 4): catch-up loop (#8272)reivilibre2020-09-151-1/+42
|
* Catch up after Federation Outage (split, 2): Track last successful stream ↵reivilibre2020-09-041-0/+38
| | | | | ordering after transmission (#8247) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Catch-up after Federation Outage (split, 1) (#8230)reivilibre2020-09-041-3/+63
| | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Convert additional database code to async/await. (#8195)Patrick Cloke2020-08-281-16/+23
|
* Convert calls of async database methods to async (#8166)Patrick Cloke2020-08-271-8/+10
|
* Convert appservice, group server, profile and more databases to async (#8066)Patrick Cloke2020-08-121-5/+2
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-0/+269