summary refs log tree commit diff
path: root/synapse/storage/databases/main/schema (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge tag 'v1.33.0' into babolivier/dinsic_1.41.0Brendan Abolivier2021-08-315-2/+56
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.33.0 (2021-05-05) =========================== Features -------- - Build Debian packages for Ubuntu 21.04 (Hirsute Hippo). ([\#9909](https://github.com/matrix-org/synapse/issues/9909)) Synapse 1.33.0rc2 (2021-04-29) ============================== Bugfixes -------- - Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](https://github.com/matrix-org/synapse/issues/9900)) Synapse 1.33.0rc1 (2021-04-28) ============================== Features -------- - Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814)) - Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850)) - Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832)) Bugfixes -------- - Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726)) - Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788)) - Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802)) - Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833)) - Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867)) - Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868)) Improved Documentation ---------------------- - Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801)) Internal Changes ---------------- - Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162)) - Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786)) - Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796)) - Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815)) - Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816)) - Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817)) - Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821)) - Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838)) - Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845)) - Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855)) - Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856)) - Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858)) - Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871)) - Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874)) - Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875)) - Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876)) - Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878)) - Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879)) - Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
| * Split presence out of master (#9820)Erik Johnston2021-04-232-0/+38
| |
| * Port "Allow users to click account renewal links multiple times without ↵Andrew Morgan2021-04-191-0/+18
| | | | | | | | | | hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
| * Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-142-2/+0
| | | | | | | | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* | Merge tag 'v1.32.0' into babolivier/dinsic_1.41.0Brendan Abolivier2021-08-311-0/+22
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.32.0 (2021-04-20) =========================== **Note:** This release requires Python 3.6+ and Postgres 9.6+ or SQLite 3.22+. This release removes the deprecated `GET /_synapse/admin/v1/users/<user_id>` admin API. Please use the [v2 API](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/user_admin_api.rst#query-user-account) instead, which has improved capabilities. This release requires Application Services to use type `m.login.application_service` when registering users via the `/_matrix/client/r0/register` endpoint to comply with the spec. Please ensure your Application Services are up to date. Bugfixes -------- - Fix the log lines of nested logging contexts. Broke in 1.32.0rc1. ([\#9829](https://github.com/matrix-org/synapse/issues/9829)) Synapse 1.32.0rc1 (2021-04-13) ============================== Features -------- - Add a Synapse module for routing presence updates between users. ([\#9491](https://github.com/matrix-org/synapse/issues/9491)) - Add an admin API to manage ratelimit for a specific user. ([\#9648](https://github.com/matrix-org/synapse/issues/9648)) - Include request information in structured logging output. ([\#9654](https://github.com/matrix-org/synapse/issues/9654)) - Add `order_by` to the admin API `GET /_synapse/admin/v2/users`. Contributed by @dklimpel. ([\#9691](https://github.com/matrix-org/synapse/issues/9691)) - Replace the `room_invite_state_types` configuration setting with `room_prejoin_state`. ([\#9700](https://github.com/matrix-org/synapse/issues/9700)) - Add experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9717](https://github.com/matrix-org/synapse/issues/9717), [\#9735](https://github.com/matrix-org/synapse/issues/9735)) - Update experimental support for Spaces: include `m.room.create` in the room state sent with room-invites. ([\#9710](https://github.com/matrix-org/synapse/issues/9710)) - Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. ([\#9766](https://github.com/matrix-org/synapse/issues/9766)) Bugfixes -------- - Prevent `synapse_forward_extremities` and `synapse_excess_extremity_events` Prometheus metrics from initially reporting zero-values after startup. ([\#8926](https://github.com/matrix-org/synapse/issues/8926)) - Fix recently added ratelimits to correctly honour the application service `rate_limited` flag. ([\#9711](https://github.com/matrix-org/synapse/issues/9711)) - Fix longstanding bug which caused `duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"` errors. ([\#9725](https://github.com/matrix-org/synapse/issues/9725)) - Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. ([\#9770](https://github.com/matrix-org/synapse/issues/9770)) - Fix duplicate logging of exceptions thrown during federation transaction processing. ([\#9780](https://github.com/matrix-org/synapse/issues/9780)) Updates to the Docker image --------------------------- - Move opencontainers labels to the final Docker image such that users can inspect them. ([\#9765](https://github.com/matrix-org/synapse/issues/9765)) Improved Documentation ---------------------- - Make the `allowed_local_3pids` regex example in the sample config stricter. ([\#9719](https://github.com/matrix-org/synapse/issues/9719)) Deprecations and Removals ------------------------- - Remove old admin API `GET /_synapse/admin/v1/users/<user_id>`. ([\#9401](https://github.com/matrix-org/synapse/issues/9401)) - Make `/_matrix/client/r0/register` expect a type of `m.login.application_service` when an Application Service registers a user, to align with [the relevant spec](https://spec.matrix.org/unstable/application-service-api/#server-admin-style-permissions). ([\#9548](https://github.com/matrix-org/synapse/issues/9548)) Internal Changes ---------------- - Replace deprecated `imp` module with successor `importlib`. Contributed by Cristina Muñoz. ([\#9718](https://github.com/matrix-org/synapse/issues/9718)) - Experiment with GitHub Actions for CI. ([\#9661](https://github.com/matrix-org/synapse/issues/9661)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9682](https://github.com/matrix-org/synapse/issues/9682)) - Update `scripts-dev/complement.sh` to use a local checkout of Complement, allow running a subset of tests and have it use Synapse's Complement test blacklist. ([\#9685](https://github.com/matrix-org/synapse/issues/9685)) - Improve Jaeger tracing for `to_device` messages. ([\#9686](https://github.com/matrix-org/synapse/issues/9686)) - Add release helper script for automating part of the Synapse release process. ([\#9713](https://github.com/matrix-org/synapse/issues/9713)) - Add type hints to expiring cache. ([\#9730](https://github.com/matrix-org/synapse/issues/9730)) - Convert various testcases to `HomeserverTestCase`. ([\#9736](https://github.com/matrix-org/synapse/issues/9736)) - Start linting mypy with `no_implicit_optional`. ([\#9742](https://github.com/matrix-org/synapse/issues/9742)) - Add missing type hints to federation handler and server. ([\#9743](https://github.com/matrix-org/synapse/issues/9743)) - Check that a `ConfigError` is raised, rather than simply `Exception`, when appropriate in homeserver config file generation tests. ([\#9753](https://github.com/matrix-org/synapse/issues/9753)) - Fix incompatibility with `tox` 2.5. ([\#9769](https://github.com/matrix-org/synapse/issues/9769)) - Enable Complement tests for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): Spaces Summary API. ([\#9771](https://github.com/matrix-org/synapse/issues/9771)) - Use mock from the standard library instead of a separate package. ([\#9772](https://github.com/matrix-org/synapse/issues/9772)) - Update Black configuration to target Python 3.6. ([\#9781](https://github.com/matrix-org/synapse/issues/9781)) - Add option to skip unit tests when building Debian packages. ([\#9793](https://github.com/matrix-org/synapse/issues/9793))
| * Remove outdated constraint on remote_media_cache_thumbnails (#9725)Richard van der Hoff2021-04-061-0/+22
| | | | | | | | | | | | | | | | | | | | | | The `remote_media_cache_thumbnails_media_origin_media_id_thumbna_key` constraint is superceded by `remote_media_repository_thumbn_media_origin_id_width_height_met` (which adds `thumbnail_method` to the unique key). PR #7124 made an attempt to remove the old constraint, but got the name wrong, so it didn't work. Here we update the bg update and rerun it. Fixes #8649.
* | Merge commit '918f6ed82' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-0/+17
|\|
| * Add a background task to purge unused chain IDs. (#9542)Patrick Cloke2021-03-091-0/+17
| | | | | | | | | | This is a companion change to apply the fix in #9498 / 922788c6043138165c025c78effeda87de842bab to previously purged rooms.
* | Merge commit '33a02f0f5' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-1/+2
|\|
| * Make deleting stale pushers a background update (#9536)Erik Johnston2021-03-031-1/+2
| |
* | Merge commit 'a5daae2a5' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-5/+13
|\|
| * Re-run rejected metadata background update. (#9503)Erik Johnston2021-03-021-0/+9
| | | | | | | | | | | | | | It landed in schema version 58 after 59 had been created, causing some servers to not run it. The main effect of was that not all rooms had their chain cover calculated correctly. After the BG updates complete the chain covers will get fixed when a new state event in the affected rooms is received.
| * Fix SQL delta file taking a long time to run (#9516)Erik Johnston2021-03-021-5/+4
| | | | | | | | Fixes #9504
* | Merge commit 'e12077a78' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-0/+40
|\|
| * Ensure pushers are deleted for deactivated accounts (#9285)Erik Johnston2021-02-251-0/+21
| |
| * Add SQL delta for deleting stale pushers (#9479)Erik Johnston2021-02-241-0/+19
| |
* | Merge commit 'd2f0ec12d' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-11/+2
|\|
| * Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-1/+2
| | | | | | | | | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
| * Remove conflicting sqlite tables that are "reserved" (shadow fts4 tables) ↵Eric Eastwood2021-02-101-10/+0
| | | | | | | | | | | | | | (#9003) Remove conflicting sqlite tables that throw sqlite3.OperationalError: object name reserved for internal use: event_search_content when running the twisted unit tests. Fix #8996
* | Merge commit '939ef657c' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-222-1/+19
|\|
| * Merge remote-tracking branch 'origin/release-v1.26.0' into developErik Johnston2021-01-211-0/+18
| |\
| | * Add schema update to fix existing DBs affected by #9193 (#9195)Erik Johnston2021-01-211-0/+18
| | |
| * | Use execute_batch in more places (#9188)Erik Johnston2021-01-211-1/+1
| |/ | | | | | | | | * Use execute_batch in more places * Newsfile
* | Merge commit '73b03722f' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-222-0/+52
|\|
| * Allow moving account data and receipts streams off master (#9104)Erik Johnston2021-01-182-0/+52
| |
* | Merge commit '631dd06f2' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-0/+17
|\|
| * Add background update for add chain cover index (#9029)Erik Johnston2021-01-141-0/+17
| |
* | Merge commit '98a64b7f7' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-225-0/+119
|\|
| * Use a chain cover index to efficiently calculate auth chain difference (#8868)Erik Johnston2021-01-112-0/+68
| |
| * Remove old tables after schema version bump (#9055)Erik Johnston2021-01-112-0/+34
| | | | | | | | | | These tables are unused, and can be dropped now the schema version has been bumped.
| * Ensure rejected events get added to some metadata tables (#9016)Erik Johnston2021-01-111-0/+17
| | | | | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* | Merge commit '2fb1c2b6e' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-212-0/+78
|\|
| * Drop last_used column from access_tokens (#9025)Jerin J Titus2021-01-112-0/+78
| | | | | | | | | | * Dropped last_used column from access_tokens Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
* | Merge commit 'a03d71dc9' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-212-0/+43
|\|
| * Allow running sendToDevice on workers (#9044)Erik Johnston2021-01-072-0/+43
| |
* | Merge commit '23d701864' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-211-0/+82
|\|
| * Improve the performance of calculating ignored users in large rooms (#9024)Patrick Cloke2021-01-071-0/+82
| | | | | | | | | | | | This allows for efficiently finding which users ignore a particular user. Co-authored-by: Erik Johnston <erik@matrix.org>
* | Merge commit '9999eb2d0' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-211-0/+18
|\|
| * Drop the unused local_invites table. (#8979)Patrick Cloke2020-12-291-0/+18
| | | | | | This table has been unused since Synapse v1.17.0.
* | Merge commit '28877fade' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-211-0/+18
|\|
| * Allow re-using a UI auth validation for a period of time (#8970)Patrick Cloke2020-12-181-0/+18
| |
* | Merge commit 'cf7d3c90d' into dinsicAndrew Morgan2021-04-161-0/+17
|\|
| * UIA: offer only available auth flowsRichard van der Hoff2020-12-021-0/+17
| | | | | | | | | | | | | | During user-interactive auth, do not offer password auth to users with no password, nor SSO auth to users with no SSO. Fixes #7559.
* | Add knocking support (#81)Andrew Morgan2021-02-091-0/+17
| | | | | | | | | | | | | | | | | | Implement knocking as defined by https://github.com/matrix-org/matrix-doc/pull/2403 This is the base knocking stuff, taken from https://github.com/matrix-org/synapse/pull/6739 and does not include any public room directory changes. While knocking hasn't merged yet on mainline due to waiting on getting Complement into Synapse's CI, the code has been well-tested.
* | Merge commit 'd3ed93504' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-314-12/+12
|\|
| * Add missing `ordering` to background updates (#8850)Richard van der Hoff2020-12-014-12/+12
| | | | | | | | | | | | | | It's important that we make sure our background updates happen in a defined order, to avoid disasters like #6923. Add an ordering to all of the background updates that have landed since #7190.
* | Merge commit '09ac0569f' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-0/+19
|\|
| * Drop (almost) unused index on event_json (#8845)Richard van der Hoff2020-11-301-0/+19
| |
* | Merge commit 'c3119d153' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-0/+17
|\|
| * Improve start time by adding index to e2e_cross_signing_keys (#8694)Erik Johnston2020-11-021-0/+17
| | | | | | We do a `SELECT MAX(stream_id) FROM e2e_cross_signing_keys` on startup.
* | Merge commit 'ef2d62701' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-0/+17
|\|
| * Add ability for access tokens to belong to one user but grant access to ↵Erik Johnston2020-10-291-0/+17
| | | | | | | | | | | | | | | | | | | | another user. (#8616) We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't). A future PR will add an API for creating such a token. When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
* | Merge commit '88e1d0c52' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-0/+2
|\|
| * Add admin API to list users' local media (#8647)Dirk Klimpel2020-10-271-0/+2
| | | | | | Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
* | Merge commit 'de5cafe98' into dinsicAndrew Morgan2020-12-312-3/+3
|\|
| * Move schema file for as_device_stream (#8590)Will Hunt2020-10-201-3/+2
| | | | | | | | | | | | | | | | | | * Move schema file * Add a . * Add matching changelog entry * Fix sqlite
| * Drop unused `device_max_stream_id` table (#8589)Vasilis Gerakaris2020-10-191-0/+1
| | | | | | Signed-off-by: Vasilis Gerakaris <vasilis.gerakaris@navarino.gr>
* | Merge commit '3ee17585c' into dinsicAndrew Morgan2020-12-311-0/+18
|\|
| * Send some ephemeral events to appservices (#8437)Will Hunt2020-10-151-0/+18
| | | | | | Optionally sends typing, presence, and read receipt information to appservices.
* | Merge commit '74976a8e4' into dinsicAndrew Morgan2020-12-3116-41/+168
|\|
| * Include user agent in user daily visits table (#8503)Neil Johnson2020-10-151-0/+18
| | | | | | Include user agent in user daily visits table.
| * Fix message duplication if something goes wrong after persisting the event ↵Erik Johnston2020-10-131-0/+40
| | | | | | | | | | (#8476) Should fix #3365.
| * Only send RDATA for instance local events. (#8496)Erik Johnston2020-10-091-0/+17
| | | | | | | | | | When pulling events out of the DB to send over replication we were not filtering by instance name, and so we were sending events for other instances.
| * Use vector clocks for room stream tokens. (#8439)Erik Johnston2020-10-071-0/+25
| | | | | | | | | | | | | | | | | | | | Currently when using multiple event persisters we (in the worst case) don't tell clients about events until all event persisters have persisted new events after the original event. This is a suboptimal, especially if one of the event persisters goes down. To handle this, we encode the position of each event persister in the room tokens so that we can send events to clients immediately. To reduce the size of the token we do two things: 1. We create a unique immutable persistent mapping between instance names and a generated small integer ID, which we can encode in the tokens instead of the instance name; and 2. We encode the "persisted upto position" of the room token and then only explicitly include instances that have positions strictly greater than that. The new tokens look something like: `m3478~1.3488~2.3489`, where the first number is the min position, and the subsequent `-` separated pairs are the instance ID to positions map. (We use `.` and `~` as separators as they're URL safe and not already used by `StreamToken`).
| * Add support for MSC2697: Dehydrated devices (#8380)Hubert Chathi2020-10-071-0/+20
| | | | | | | | This allows a user to store an offline device on the server and then restore it at a subsequent login.
| * Add support for MSC2732: olm fallback keys (#8312)Hubert Chathi2020-10-061-0/+24
| |
| * Add logging on startup/shutdown (#8448)Erik Johnston2020-10-0210-41/+24
| | | | | | | | | | This is so we can tell what is going on when things are taking a while to start up. The main change here is to ensure that transactions that are created during startup get correctly logged like normal transactions.
* | Allow users to click account renewal links multiple times without hitting an ↵Andrew Morgan2020-12-301-0/+18
| | | | | | | | 'Invalid Token' page (#74)
* | Merge commit 'f43c66d23' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'f43c66d23': Add support for running Complement against the local checkout (#8317) Filter out appservices from mau count (#8404) Only assert valid next_link params when provided (#8417) Add metrics to track success/otherwise of replication requests (#8406) Fix handling of connection timeouts in outgoing http requests (#8400) Changelog Don't check whether a 3pid is allowed to register during password reset Add checks for postgres sequence consistency (#8402) Create a mechanism for marking tests "logcontext clean" (#8399) Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410) A pair of tiny cleanups in the federation request code. (#8401) typo
| * typoMatthew Hodgson2020-09-281-1/+1
| |
* | Merge commit '31acc5c30' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-212-1/+25
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '31acc5c30': Escape the error description on the sso_error template. (#8405) Fix occasional "Re-starting finished log context" from keyring (#8398) Allow existing users to login via OpenID Connect. (#8345) Fix schema delta for servers that have not backfilled (#8396) Fix MultiWriteIdGenerator's handling of restarts. (#8374) s/URLs/variables in changelog s/accidentally/incorrectly in changelog Update changelog wording Add type annotations to SimpleHttpClient (#8372) Add new sequences to port DB script (#8387) Add EventStreamPosition type (#8388) Mark the shadow_banned column as boolean in synapse_port_db. (#8386)
| * Fix schema delta for servers that have not backfilled (#8396)Erik Johnston2020-09-251-1/+3
| | | | | | | | | | Fixes #8395.
| * Fix MultiWriteIdGenerator's handling of restarts. (#8374)Erik Johnston2020-09-241-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On startup `MultiWriteIdGenerator` fetches the maximum stream ID for each instance from the table and uses that as its initial "current position" for each writer. This is problematic as a) it involves either a scan of events table or an index (neither of which is ideal), and b) if rows are being persisted out of order elsewhere while the process restarts then using the maximum stream ID is not correct. This could theoretically lead to race conditions where e.g. events that are persisted out of order are not sent down sync streams. We fix this by creating a new table that tracks the current positions of each writer to the stream, and update it each time we finish persisting a new entry. This is a relatively small overhead when persisting events. However for the cache invalidation stream this is a much bigger relative overhead, so instead we note that for invalidation we don't actually care about reliability over restarts (as there's no caches to invalidate) and simply don't bother reading and writing to the new table in that particular case.
* | Merge commit '837293c31' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-0/+42
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '837293c31': Remove obsolete __future__ imports (#8337) Use admin_patterns for all admin APIs. (#8331) Fix a potential bug of UnboundLocalError (#8329) Switch metaclass initialization to python 3-compatible syntax (#8326) Catch-up after Federation Outage (split, 4): catch-up loop (#8272) Use slots in attrs classes where possible (#8296) Fix typos in comments. Add the topic and avatar to the room details admin API (#8305) Improve SAML error messages (#8248) Add experimental support for sharding event persister. Again. (#8294) Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) Use TLSv1.2 for fake servers in tests (#8208) Add /_synapse/client to the reverse proxy docs (#8227) Clean up `Notifier.on_new_room_event` code path (#8288)
| * Add experimental support for sharding event persister. Again. (#8294)Erik Johnston2020-09-142-0/+42
| | | | | | | | | | | | This is *not* ready for production yet. Caveats: 1. We should write some tests... 2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
* | Merge commit '453dfe210' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-0/+28
|\| | | | | | | | | | | * commit '453dfe210': blacklist MSC2753 sytests until it's implemented in synapse (#8285) Don't remember `enabled` of deleted push rules and properly return 404 for missing push rules in `.../actions` and `.../enabled` (#7796)
| * Don't remember `enabled` of deleted push rules and properly return 404 for ↵reivilibre2020-09-091-0/+28
| | | | | | | | | | | | | | missing push rules in `.../actions` and `.../enabled` (#7796) Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Merge commit 'e45b83411' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-0/+77
|\| | | | | | | | | | | | | | | | | | | | | | | * commit 'e45b83411': Add types to async_helpers (#8260) Fix mypy error on develop (#8282) Include method in thumbnail media name (#7124) Add types to StreamToken and RoomStreamToken (#8279) Add a config option for validating 'next_link' parameters against a domain whitelist (#8275) Clean up types for PaginationConfig (#8250) Use the right constructor for log records (#8278) Fix `MultiWriterIdGenerator.current_position`. (#8257)
| * Include method in thumbnail media name (#7124)DeepBlueV7.X2020-09-082-0/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes an issue where different methods (crop/scale) overwrite each other. This first tries the new path. If that fails and we are looking for a remote thumbnail, it tries the old path. If that still isn't found, it continues as normal. This should probably be removed in the future, after some of the newer thumbnails were generated with the new path on most deployments. Then the overhead should be minimal if the other thumbnails need to be regenerated. Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* | Merge commit 'cca03dbec' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-0/+22
|\| | | | | | | | | | | | | | | | | * commit 'cca03dbec': fix typo s/fixes/fix/ Directly import json from the standard library. (#8259) Allow for make_awaitable's return value to be re-used. (#8261) Rename 'populate_stats_process_rooms_2' background job back to 'populate_stats_process_rooms' again (#8243)
| * Rename 'populate_stats_process_rooms_2' background job back to ↵Andrew Morgan2020-09-081-0/+22
| | | | | | | | | | | | | | 'populate_stats_process_rooms' again (#8243) Fixes https://github.com/matrix-org/synapse/issues/8238 Alongside the delta file, some changes were also necessary to the codebase to remove references to the now defunct `populate_stats_process_rooms_2` background job. Thankfully the latter doesn't seem to have made it into any documentation yet :)
* | Merge commit '77b4711bc' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-3/+3
|\| | | | | | | | | | | | | * commit '77b4711bc': Add cross-signing sigs to the `keys` object (#8234) Unread counts fixes (#8254) Fix a regression from calling read_templates. (#8252)
| * Merge branch 'release-v1.20.0' into developRichard van der Hoff2020-09-061-3/+3
| |\
| | * Unread counts fixes (#8254)Brendan Abolivier2020-09-041-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fixup `ALTER TABLE` database queries Make the new columns nullable, because doing otherwise can wedge a server with a big database, as setting a default value rewrites the table. * Switch back to using the notifications count in the push badge Clients are likely to be confused if we send a push but the badge count is the unread messages one, and not the notifications one. * Changelog
* | | Merge commit '7513006b0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-1/+1
|\| | | | | | | | | | | | | | * commit '7513006b0': In light of #8255, use BIGINTs for destination_rooms (#8256)
| * | In light of #8255, use BIGINTs for destination_rooms (#8256)reivilibre2020-09-041-1/+1
| | |
* | | Merge commit '17fa4c7ca' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-0/+63
|\| | | | | | | | | | | | | | | | | | | | | | | * commit '17fa4c7ca': Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission (#8247) Catch-up after Federation Outage (split, 1) (#8230) Fix type signature in simple_select_one_onecol and friends (#8241) Stop sub-classing object (#8249)
| * | Catch up after Federation Outage (split, 2): Track last successful stream ↵reivilibre2020-09-041-0/+21
| | | | | | | | | | | | | | | ordering after transmission (#8247) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * | Catch-up after Federation Outage (split, 1) (#8230)reivilibre2020-09-041-0/+42
| |/ | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* | Merge commit '9f8abdcc3' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-42/+0
|\| | | | | | | | | * commit '9f8abdcc3': Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
| * Revert "Add experimental support for sharding event persister. (#8170)" (#8242)Brendan Abolivier2020-09-042-42/+0
| | | | | | | | | | | | | | * Revert "Add experimental support for sharding event persister. (#8170)" This reverts commit 82c1ee1c22a87b9e6e3179947014b0f11c0a1ac3. * Changelog
* | Merge commit '112266eaf' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-0/+26
|\| | | | | | | | | | | * commit '112266eaf': Add StreamStore to mypy (#8232) Re-implement unread counts (again) (#8059)
| * Re-implement unread counts (again) (#8059)Brendan Abolivier2020-09-021-0/+26
| |
* | Merge commit '0d4f614fd' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-202-0/+42
|\| | | | | | | | | | | | | | | | | * commit '0d4f614fd': Refactor `_get_e2e_device_keys_for_federation_query_txn` (#8225) Add experimental support for sharding event persister. (#8170) Add /user/{user_id}/shared_rooms/ api (#7785) Do not try to store invalid data in the stats table (#8226) Convert the main methods run by the reactor to async. (#8213)
| * Add experimental support for sharding event persister. (#8170)Erik Johnston2020-09-022-0/+42
| | | | | | | | | | | | This is *not* ready for production yet. Caveats: 1. We should write some tests... 2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
* | Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-0/+25
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '56efa9ec7': (22 commits) Fix rate limiting unit tests. (#8167) Add functions to `MultiWriterIdGen` used by events stream (#8164) Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) Changelog fixes Make StreamIdGen `get_next` and `get_next_mult` async (#8161) Wording fixes to 'name' user admin api filter (#8163) Fix missing double-backtick in RST document Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) Add type hints for state. (#8140) Stop shadow-banned users from sending non-member events. (#8142) Allow capping a room's retention policy (#8104) Add healthcheck for default localhost 8008 port on /health endpoint. (#8147) Fix flaky shadow-ban tests. (#8152) Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991) Do not apply ratelimiting on joins to appservices (#8139) Micro-optimisations to get_auth_chain_ids (#8132) Allow denying or shadow banning registrations via the spam checker (#8034) Stop shadow-banned users from sending invites. (#8095) Be more tolerant of membership events in unknown rooms (#8110) Improve the error code when trying to register using a name reserved for guests. (#8135) ...
| * Allow denying or shadow banning registrations via the spam checker (#8034)Patrick Cloke2020-08-201-0/+25
| |
* | Merge commit 'e04e465b4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-0/+18
|\| | | | | | | | | | | | | | | | | | | * commit 'e04e465b4': Use the default templates when a custom template file cannot be found (#8037) Changelog changes Convert stream database to async/await. (#8074) Add a shadow-banned flag to users. (#8092) Convert pusher databases to async/await. (#8075) Convert receipts and events databases to async/await. (#8076)
| * Add a shadow-banned flag to users. (#8092)Patrick Cloke2020-08-141-0/+18
| |
* | Merge commit 'dc22090a6' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-0/+17
|\| | | | | | | | | | | | | | | * commit 'dc22090a6': Add type hints to synapse.handlers.room (#8090) Remove some unused database functions. (#8085) Convert misc database code to async (#8087) Remove a space at the start of a changelog entry.
| * Remove some unused database functions. (#8085)Patrick Cloke2020-08-141-0/+17
| |
* | Merge commit '2ffd6783c' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-18/+0
|\| | | | | | | | | * commit '2ffd6783c': Revert #7736 (#8039)
| * Revert #7736 (#8039)Brendan Abolivier2020-08-061-18/+0
| |
* | Merge commit '66f24449d' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-192-0/+39
|\| | | | | | | | | * commit '66f24449d': Improve performance of the register endpoint (#8009)
* | Merge commit 'a7bdf98d0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-165-2/+53
|/ | | | | * commit 'a7bdf98d0': Rename database classes to make some sense (#8033)
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-05216-0/+8908