summary refs log tree commit diff
path: root/synapse/handlers/appservice.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-12-13Allow selecting "prejoin" events by state keys (#14642)David Robertson14-695/+983
* Declare new config * Parse new config * Read new config * Don't use trial/our TestCase where it's not needed Before: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m2.277s user 0m2.186s sys 0m0.083s ``` After: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m0.566s user 0m0.508s sys 0m0.056s ``` * Helper to upsert to event fields without exceeding size limits. * Use helper when adding invite/knock state Now that we allow admins to include events in prejoin room state with arbitrary state keys, be a good Matrix citizen and ensure they don't accidentally create an oversized event. * Changelog * Move StateFilter tests should have done this in #14668 * Add extra methods to StateFilter * Use StateFilter * Ensure test file enforces typed defs; alphabetise * Workaround surprising get_current_state_ids * Whoops, fix mypy
2022-12-12Enable `--warn-redundant-casts` option in mypy (#14671)David Robertson5-6/+7
* Enable `--warn-redundant-casts` option in mypy Doesn't do much but helps me sleep better at night. * Changelog * Fix name of the ignore * Fix one more missed cast Not sure why I didn't see this one locally, maybe I needed a poetry update * Remove old comment Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-12-12Fix missing cache invalidation in application service code (#14670)Sean Quah2-2/+7
#11915 introduced the `@cached` `is_interested_in_room` method in Synapse 1.55.0, which depends upon `get_aliases_for_room`. Add a missing cache invalidation callback so that the `is_interested_in_room` cache is invalidated when `get_aliases_for_room` is invalidated. #13787 made `get_rooms_for_user` `@cached`. Add a missing cache invalidation callback so that the `is_interested_in_presence` cache is invalidated when `get_rooms_for_user` is invalidated. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-12-12Move `StateFilter` to `synapse.types` (#14668)David Robertson27-24/+25
* Move `StateFilter` to `synapse.types` * Changelog
2022-12-12Revert the deletion of stale devices due to performance issues. (#14662)reivilibre7-152/+5
2022-12-12Bump packaging from 21.3 to 22.0 (#14657)dependabot[bot]2-22/+5
* Bump packaging from 21.3 to 22.0 Bumps [packaging](https://github.com/pypa/packaging) from 21.3 to 22.0. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pypa/packaging/compare/21.3...22.0) --- updated-dependencies: - dependency-name: packaging dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-12Bump flake8-bugbear from 22.10.27 to 22.12.6 (#14656)dependabot[bot]2-3/+4
* Bump flake8-bugbear from 22.10.27 to 22.12.6 Bumps [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) from 22.10.27 to 22.12.6. - [Release notes](https://github.com/PyCQA/flake8-bugbear/releases) - [Commits](https://github.com/PyCQA/flake8-bugbear/compare/22.10.27...22.12.6) --- updated-dependencies: - dependency-name: flake8-bugbear dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-12Add optional ICU support for user search (#14464)Brendan Abolivier10-6/+166
Fixes #13655 This change uses ICU (International Components for Unicode) to improve boundary detection in user search. This change also adds a new dependency on libicu-dev and pkg-config for the Debian packages, which are available in all supported distros.
2022-12-12Bump phonenumbers from 8.13.1 to 8.13.2 (#14660)dependabot[bot]2-3/+4
* Bump phonenumbers from 8.13.1 to 8.13.2 Bumps [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) from 8.13.1 to 8.13.2. - [Release notes](https://github.com/daviddrysdale/python-phonenumbers/releases) - [Commits](https://github.com/daviddrysdale/python-phonenumbers/compare/v8.13.1...v8.13.2) --- updated-dependencies: - dependency-name: phonenumbers dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-12Bump serde from 1.0.148 to 1.0.150 (#14659)dependabot[bot]2-4/+5
* Bump serde from 1.0.148 to 1.0.150 Bumps [serde](https://github.com/serde-rs/serde) from 1.0.148 to 1.0.150. - [Release notes](https://github.com/serde-rs/serde/releases) - [Commits](https://github.com/serde-rs/serde/compare/v1.0.148...v1.0.150) --- updated-dependencies: - dependency-name: serde dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-12Bump authlib from 1.1.0 to 1.2.0 (#14661)dependabot[bot]2-5/+6
* Bump authlib from 1.1.0 to 1.2.0 Bumps [authlib](https://github.com/lepture/authlib) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/lepture/authlib/releases) - [Changelog](https://github.com/lepture/authlib/blob/master/docs/changelog.rst) - [Commits](https://github.com/lepture/authlib/compare/v1.1.0...v1.2.0) --- updated-dependencies: - dependency-name: authlib dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-12Bump types-pillow from 9.3.0.1 to 9.3.0.4 (#14658)dependabot[bot]2-3/+4
* Bump types-pillow from 9.3.0.1 to 9.3.0.4 Bumps [types-pillow](https://github.com/python/typeshed) from 9.3.0.1 to 9.3.0.4. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-09Handle half-created indices in receipts index background update (#14650)Sean Quah3-48/+60
When Synapse is terminated while running the background update to create the `receipts_graph` or `receipts_linearized` indexes, the indexes may be successfully created (or marked as invalid on postgres) while the background update remains unfinished. When Synapse next starts up, the background update will fail because the index already exists, or exists but is invalid on postgres. Use the existing code to create indices in background updates, since it handles these edge cases. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-12-09Require types in tests.storage. (#14646)Patrick Cloke36-341/+489
Adds missing type hints to `tests.storage` package and does not allow untyped definitions.
2022-12-09Limit the number of devices we delete at once (#14649)Erik Johnston4-4/+43
2022-12-09Delete stale non-e2e devices for users, take 2 (#14595)Erik Johnston5-4/+113
This should help reduce the number of devices e.g. simple bots the repeatedly login rack up. We only delete non-e2e devices as they should be safe to delete, whereas if we delete e2e devices for a user we may accidentally break their ability to receive e2e keys for a message.
2022-12-08Fix html templates to load images only on HTTPS (#14625)Ashish Kumar4-9/+10
This PR changes http-based image URLs to be https in html templates. This impacts the Synapse SSO error page, where browsers report mixed media content warnings. Also, https://matrix.org/img/vector-logo-email.png is currently broken but the URL has been updated to be https anyway. Signed-off-by: Ashish Kumar <ashfame@users.noreply.github.com>
2022-12-08Rebuild the user directory and stats tables. (#14643)Patrick Cloke2-0/+30
Due to the various fixes to the StreamChangeCache it is not safe to trust the information in the user directory or room/user stats tables. Rebuild them as background jobs. In particular see da777207528513c858395758bf4c023da2c2c1a3 (#14639), and 6a8310f3dfe77acf59df2fe3e88a71b85b9b3ecc (#14435). Maybe also be related to fac8a38525387e344e3595a092578e0ffedd49ae (#14592).
2022-12-08Respond with proper error responses on unknown paths. (#14621)Patrick Cloke9-14/+32
Returns a proper 404 with an errcode of M_RECOGNIZED for unknown endpoints per MSC3743.
2022-12-08Check the stream position before checking if the cache is empty. (#14639)Patrick Cloke3-7/+10
An empty cache does not mean the entity has no changed, if it is earlier than the earliest known stream position return that the entity *has* changed since the cache cannot accurately answer that query.
2022-12-08Bump certifi from 2021.10.8 to 2022.12.7 (#14645)dependabot[bot]2-4/+5
* Bump certifi from 2021.10.8 to 2022.12.7 Bumps [certifi](https://github.com/certifi/python-certifi) from 2021.10.8 to 2022.12.7. - [Release notes](https://github.com/certifi/python-certifi/releases) - [Commits](https://github.com/certifi/python-certifi/compare/2021.10.08...2022.12.07) --- updated-dependencies: - dependency-name: certifi dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-07Update admin_faq.mdRichard van der Hoff1-1/+1
2022-12-07Reject receipt requests with invalid room or event IDs. (#14632)Nick Mills-Barrett3-1/+81
If the room or event IDs are empty or of an invalid form they should be rejected.
2022-12-07Unit tests CI speedup (#14610)realtyem2-3/+8
2022-12-07Add `--editable` flag to `complement.sh` which uses an editable install of ↵reivilibre5-20/+162
Synapse for faster turn-around times whilst developing iteratively. (#14548) Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07docker: remove useless cargo install with apt (#14636)Mathieu Velten2-1/+2
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07Fix a bug introduced in v1.67.0 where not specifying a config file or a ↵reivilibre2-2/+4
server URL would lead to the `register_new_matrix_user` script failing. (#14637)
2022-12-07Add Mastodon SSO setup instructions to docs (#14594)villepeh2-0/+42
Fixes https://github.com/matrix-org/synapse/issues/14147
2022-12-07Change `turn_allow_guests` example value to lowercase `true` (#14634)villepeh2-1/+2
* Set `turn_allow_guests` example value to lowercase * Create 14634.doc
2022-12-07Fix a long-standing bug where the user directory would return 1 more row ↵reivilibre4-3/+10
than requested. (#14631)
2022-12-06Faster remote room joins: unblock tasks waiting for full room state when the ↵reivilibre3-0/+77
un-partial-stating of that room is received over the replication stream. [rei:frrj/streams/unpsr] (#14474)
2022-12-06docs: Replace old note about Postgres 10+ being required with link to our ↵Andrew Morgan2-1/+3
deprecation policy (#14590)
2022-12-06Failover on proper error responses. (#14620)Patrick Cloke2-9/+21
When querying a remote server handle a 404/405 with an errcode of M_UNRECOGNIZED as an unimplemented endpoint.
2022-12-06Minor tweak to tense in changelogOlivier Wilkinson (reivilibre)1-1/+1
2022-12-06Tweak changelog v1.73.0Olivier Wilkinson (reivilibre)1-1/+1
2022-12-061.73.0Olivier Wilkinson (reivilibre)3-3/+15
2022-12-06Improve logging and opentracing for to-device message handling (#14598)Richard van der Hoff10-46/+136
A batch of changes intended to make it easier to trace to-device messages through the system. The intention here is that a client can set a property org.matrix.msgid in any to-device message it sends. That ID is then included in any tracing or logging related to the message. (Suggestions as to where this field should be documented welcome. I'm not enthusiastic about speccing it - it's very much an optional extra to help with debugging.) I've also generally improved the data we send to opentracing for these messages.
2022-12-05Better return type for `get_all_entities_changed` (#14604)Erik Johnston8-76/+138
Help callers from using the return value incorrectly by ensuring that callers explicitly check if there was a cache hit or not.
2022-12-05Compare to the earliest known stream pos in the stream change cache. (#14435)Patrick Cloke5-53/+133
The internal methods of the StreamChangeCache were inconsistently treating the earliest known stream position as valid. It is now treated as invalid, meaning the cache cannot determine if an entity at the earliest known stream position has changed or not.
2022-12-05Faster remote room joins: stream the un-partial-stating of rooms over ↵reivilibre8-67/+280
replication. [rei:frrj/streams/unpsr] (#14473)
2022-12-05Fix push.enabled config documentation (#14619)Will Hunt2-2/+3
* Fix push.enabled config documentation * Create 14619.doc * Update 14619.doc
2022-12-05Suppress empty body warnings in room servelets (#14600)David Robertson2-12/+3
* Suppress empty body warnings in room servelets We've already decided to allow empty bodies for backwards compat. The change here stops us from emitting a misleading warning; see also https://github.com/matrix-org/synapse/issues/14478#issuecomment-1319157105 * Changelog
2022-12-05Handle 'go get' deprecation (#14611)realtyem2-1/+2
* Switch out 'go get' for 'go install'. * Changelog
2022-12-05Bump cryptography from 38.0.3 to 38.0.4 (#14616)dependabot[bot]2-27/+28
* Bump cryptography from 38.0.3 to 38.0.4 Bumps [cryptography](https://github.com/pyca/cryptography) from 38.0.3 to 38.0.4. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/38.0.3...38.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-05Bump types-requests from 2.28.11.2 to 2.28.11.5 (#14615)dependabot[bot]2-3/+4
* Bump types-requests from 2.28.11.2 to 2.28.11.5 Bumps [types-requests](https://github.com/python/typeshed) from 2.28.11.2 to 2.28.11.5. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-requests dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-05Bump twine from 4.0.1 to 4.0.2 (#14614)dependabot[bot]2-3/+4
* Bump twine from 4.0.1 to 4.0.2 Bumps [twine](https://github.com/pypa/twine) from 4.0.1 to 4.0.2. - [Release notes](https://github.com/pypa/twine/releases) - [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst) - [Commits](https://github.com/pypa/twine/compare/4.0.1...4.0.2) --- updated-dependencies: - dependency-name: twine dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-05Bump types-setuptools from 65.5.0.3 to 65.6.0.1 (#14613)dependabot[bot]2-3/+4
* Bump types-setuptools from 65.5.0.3 to 65.6.0.1 Bumps [types-setuptools](https://github.com/python/typeshed) from 65.5.0.3 to 65.6.0.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-05Bump phonenumbers from 8.13.0 to 8.13.1 (#14612)dependabot[bot]2-3/+4
* Bump phonenumbers from 8.13.0 to 8.13.1 Bumps [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) from 8.13.0 to 8.13.1. - [Release notes](https://github.com/daviddrysdale/python-phonenumbers/releases) - [Commits](https://github.com/daviddrysdale/python-phonenumbers/compare/v8.13.0...v8.13.1) --- updated-dependencies: - dependency-name: phonenumbers dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-02Bump JasonEtco/create-an-issue from 2.5.0 to 2.8.1 (#14607)dependabot[bot]3-2/+3
* Bump JasonEtco/create-an-issue from 2.5.0 to 2.8.1 Bumps [JasonEtco/create-an-issue](https://github.com/JasonEtco/create-an-issue) from 2.5.0 to 2.8.1. - [Release notes](https://github.com/JasonEtco/create-an-issue/releases) - [Commits](https://github.com/JasonEtco/create-an-issue/compare/5d9504915f79f9cc6d791934b8ef34f2353dd74d...77399b6110ef82b94c1c9f9f615acf9e604f7f56) --- updated-dependencies: - dependency-name: JasonEtco/create-an-issue dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-02Use ClientRestResource on both the main process and workers. (#14528)Patrick Cloke8-119/+71
Add logic to ClientRestResource to decide whether to mount servlets or not based on whether the current process is a worker. This is clearer to see what a worker runs than the completely separate / copy & pasted list of servlets being mounted for workers.
2022-12-02Fix Rust lint CI (#14602)Erik Johnston4-16/+47
2022-12-02Add missing types to tests.util. (#14597)Patrick Cloke21-276/+361
Removes files under tests.util from the ignored by list, then fully types all tests/util/*.py files.
2022-12-02Properly handle unknown results for the stream change cache. (#14592)Patrick Cloke3-16/+22
StreamChangeCache.get_all_changed_entities can return None to signify it does not have information at the given stream position. Two callers (related to device lists and presence) were treating this response the same as an empty list (i.e. there being no updates).
2022-12-02Update worker docs to update preferred settings for pusher and ↵realtyem3-31/+53
federation_sender (#14493) * Fix one typo on line 3700(and apparently do something to other lines, no idea) * Update config_documentation.md with more information about how federation_senders and pushers settings can be handled. Specifically, that the instance map style of config does not require the special other variables that enable and disable functionality and that a single worker CAN be added to the map not only just two or more. * Extra line here for consistency and appearance. * Add link to sygnal repo. * Add deprecation notice to workers.md and point to the newer alternative method of defining this functionality. * Changelog * Correct version number of Synapse the deprecation is happening in. * Update quiet deprecation with simple notice and suggestion.
2022-12-01Bump jsonschema from 4.17.0 to 4.17.3 (#14591)dependabot[bot]2-3/+4
* Bump jsonschema from 4.17.0 to 4.17.3 Bumps [jsonschema](https://github.com/python-jsonschema/jsonschema) from 4.17.0 to 4.17.3. - [Release notes](https://github.com/python-jsonschema/jsonschema/releases) - [Changelog](https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst) - [Commits](https://github.com/python-jsonschema/jsonschema/compare/v4.17.0...v4.17.3) --- updated-dependencies: - dependency-name: jsonschema dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-12-01Bump pyopenssl from 22.0.0 to 22.1.0 (#14561)dependabot[bot]1-5/+5
Bumps [pyopenssl](https://github.com/pyca/pyopenssl) from 22.0.0 to 22.1.0. - [Release notes](https://github.com/pyca/pyopenssl/releases) - [Changelog](https://github.com/pyca/pyopenssl/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/pyopenssl/compare/22.0.0...22.1.0) --- updated-dependencies: - dependency-name: pyopenssl dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-01Bump sentry-sdk from 1.11.0 to 1.11.1 (#14562)dependabot[bot]1-3/+3
Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.11.0 to 1.11.1. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.11.0...1.11.1) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-01Bump types-bleach from 5.0.3 to 5.0.3.1 (#14564)dependabot[bot]1-3/+3
Bumps [types-bleach](https://github.com/python/typeshed) from 5.0.3 to 5.0.3.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-bleach dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-01Bump types-psycopg2 from 2.9.21.1 to 2.9.21.2 (#14558)dependabot[bot]1-3/+3
Bumps [types-psycopg2](https://github.com/python/typeshed) from 2.9.21.1 to 2.9.21.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-psycopg2 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-01Add `push.enabled` option to disable push notification calculation (#14551)Will Hunt5-2/+53
* Add initial option * changelog * Some more linting
2022-12-01Modernize unit tests configuration settings for workers. (#14568)realtyem18-81/+123
Use the newer foo_instances configuration instead of the deprecated flags to enable specific features (e.g. start_pushers).
2022-12-011.73.0rc2 v1.73.0rc2David Robertson4-3/+17
2022-12-01Fix MSC3202 link in changelogDavid Robertson1-1/+1
2022-11-30Cite launchpad bug that says ubuntu's pkgs are old (#14517)David Robertson2-6/+14
* Cite launchpad bug that says ubuntu's pkgs are old * Add some cross-references while I'm here * Changelog
2022-11-30Aggregate unread notif count query for badge count calculation (#14255)Nick Mills-Barrett4-27/+198
Fetch the unread notification counts used by the badge counts in push notifications for all rooms at once (instead of fetching them per room).
2022-11-30Use servers list approx to send read receipts when in partial state (#14549)Mathieu Velten3-1/+6
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-11-30Stop using deprecated `keyIds` param on /key/v2/server (#14525)Richard van der Hoff6-82/+47
Fixes #14523.
2022-11-29Revert "POC delete stale non-e2e devices for users (#14038)" (#14582)David Robertson5-83/+5
2022-11-29Advertise support for Matrix v1.5. (#14576)Patrick Cloke2-0/+2
All features of Matrix v1.5 were already supported: this was mostly a maintenance release.
2022-11-29Update changelog v1.73.0rc1David Robertson1-15/+14
2022-11-29Include fixup PR in changelogDavid Robertson1-1/+1
2022-11-29Fix `UndefinedColumn: column "key_json" does not exist` errors when handling ↵David Robertson1-0/+1
users with more than 50 non-E2E devices (#14580)
2022-11-291.73.0rc1David Robertson56-54/+73
2022-11-29Fix GHA job for pushing the complement-synapse image (#14573)David Robertson3-7/+14
Co-authored-by: Michael Kaye <1917473+michaelkaye@users.noreply.github.com>
2022-11-29POC delete stale non-e2e devices for users (#14038)Erik Johnston5-4/+83
This should help reduce the number of devices e.g. simple bots the repeatedly login rack up. We only delete non-e2e devices as they should be safe to delete, whereas if we delete e2e devices for a user we may accidentally break their ability to receive e2e keys for a message. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-11-28Fix possible variable shadow in `create_new_client_event` (#14575)Shay2-2/+5
2022-11-28Support MSC1767's `content.body` behaviour; Add base rules from MSC3933 (#14524)Travis Ralston7-4/+316
* Support MSC1767's `content.body` behaviour in push rules * Add the base rules from MSC3933 * Changelog entry * Flip condition around for finding `m.markup` * Remove forgotten import
2022-11-28Create MSC1767 (extensible events) room version; Implement MSC3932 (#14521)Travis Ralston4-2/+130
* Add MSC1767's dedicated room version, based on v10 * Only enable MSC1767 room version if the config flag is on Using a similar technique to knocking: https://github.com/matrix-org/synapse/pull/6739/files#diff-3af529eedb0e00279bafb7369370c9654b37792af8eafa0925400e9281d57f0a * Support MSC3932: Extensible events room version feature flag * Changelog entry
2022-11-28Initial support for MSC3931: Room version push rule feature flags (#14520)Travis Ralston8-1/+76
* Add support for MSC3931: Room Version Supports push rule condition * Create experimental flag for future work, and use it to gate MSC3931 * Changelog entry
2022-11-28Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471)Eric Eastwood13-37/+26
Fix https://github.com/matrix-org/synapse/issues/14390 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28Use `device_one_time_keys_count` to match MSC3202 (#14565)Andrew Ferrazzutti9-34/+38
* Use `device_one_time_keys_count` to match MSC3202 Rename the `device_one_time_key_counts` key in responses to `device_one_time_keys_count` to match the name specified by MSC3202. Also change related variable/class names for consistency. Signed-off-by: Andrew Ferrazzutti <andrewf@element.io> * Update changelog.d/14565.misc * Revert name change for `one_time_key_counts` key as this is a different key altogether from `device_one_time_keys_count`, which is used for `/sync` instead of appservice transactions. Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>
2022-11-28Fix logging context warnings due to common usage metrics setup (#14574)Sean Quah2-1/+4
`setup()` is run under the sentinel context manager, so we wrap the initial update in a background process. Before this change, Synapse would log two warnings on startup: Starting db txn 'count_daily_users' from sentinel context Starting db connection from sentinel context: metrics will be lost Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-28Include thread information when sending receipts over federation. (#14466)Patrick Cloke4-64/+198
Include the thread_id field when sending read receipts over federation. This might result in the same user having multiple read receipts per-room, meaning multiple EDUs must be sent to encapsulate those receipts. This restructures the PerDestinationQueue APIs to support multiple receipt EDUs, queue_read_receipt now becomes linear time in the number of queued threaded receipts in the room for the given user, it is expected this is a small number since receipt EDUs are sent as filler in transactions.
2022-11-28Remove option to skip locking of tables during emulated upserts (#14469)Sean Quah10-74/+19
To perform an emulated upsert into a table safely, we must either: * lock the table, * be the only writer upserting into the table * or rely on another unique index being present. When the 2nd or 3rd cases were applicable, we previously avoided locking the table as an optimization. However, as seen in #14406, it is easy to slip up when adding new schema deltas and corrupt the database. The only time we lock when performing emulated upserts is while waiting for background updates on postgres. On sqlite, we do no locking at all. Let's remove the option to skip locking tables, so that we don't shoot ourselves in the foot again. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-28Push complement image to a docker registry (#14509)Michael Kaye2-0/+69
* GHA workflow to build complement images of key branches. * Add changelog.d * GHA workflow to build complement images of key branches. * Add changelog.d * Update complement.yml Remove special casing for michaelk branch. * Update complement.yml Should run on master, develop not main, develop * Rename file to be more obvious * Merge did not go correctly. * Setup 5am builds of develop, limit to one run at once. * Fix crontab---run once at 5AM, not very minute between 5 and 6 * Fix cron syntax again? * Tweak workflow name * Allow manual debug runs * Tweak indentation Ctrl-Alt-L in PyCharm Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: David Robertson <davidr@element.io>
2022-11-28Bump serde_json from 1.0.88 to 1.0.89 (#14560)dependabot[bot]2-2/+3
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-28Bump serde from 1.0.147 to 1.0.148 (#14559)dependabot[bot]2-6/+7
* Bump serde from 1.0.147 to 1.0.148 Bumps [serde](https://github.com/serde-rs/serde) from 1.0.147 to 1.0.148. - [Release notes](https://github.com/serde-rs/serde/releases) - [Commits](https://github.com/serde-rs/serde/compare/v1.0.147...v1.0.148) --- updated-dependencies: - dependency-name: serde dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-28Run Rust CI when Cargo.lock changes too (#14571)David Robertson2-0/+2
* Run Rust CI when Cargo.lock changes too * Changelog
2022-11-28Bump dtolnay/rust-toolchain from 55c7845fad90d0ae8b2e83715cb900e5e861e8cb to ↵dependabot[bot]4-13/+14
e645b0cf01249a964ec099494d38d2da0f0b349f (#14557) * Bump dtolnay/rust-toolchain Bumps [dtolnay/rust-toolchain](https://github.com/dtolnay/rust-toolchain) from 55c7845fad90d0ae8b2e83715cb900e5e861e8cb to e645b0cf01249a964ec099494d38d2da0f0b349f. - [Release notes](https://github.com/dtolnay/rust-toolchain/releases) - [Commits](https://github.com/dtolnay/rust-toolchain/compare/55c7845fad90d0ae8b2e83715cb900e5e861e8cb...e645b0cf01249a964ec099494d38d2da0f0b349f) --- updated-dependencies: - dependency-name: dtolnay/rust-toolchain dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-25Add support for handling avatar with SSO login (#13917)Ashish Kumar6-2/+275
This commit adds support for handling a provided avatar picture URL when logging in via SSO. Signed-off-by: Ashish Kumar <ashfame@users.noreply.github.com> Fixes #9357.
2022-11-24Faster joins: use initial list of servers if we don't have the full state ↵Mathieu Velten2-1/+18
yet (#14408) Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-11-24Fix crash admin media list api when info is None (#14537)schmop2-1/+6
Fixes https://github.com/matrix-org/synapse/issues/14536
2022-11-24Implement message forward pagination from start when no from is given, fixes ↵Benjamin Kampmann4-0/+60
#12383 (#14149) Fixes https://github.com/matrix-org/synapse/issues/12383
2022-11-24Remove legacy Prometheus metrics names. They were deprecated in Synapse ↵reivilibre11-338/+70
v1.69.0 and disabled by default in Synapse v1.71.0. (#14538)
2022-11-24Faster joins: use servers list approximation in `assert_host_in_room` (#14515)Mathieu Velten2-11/+18
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-11-23Add another index to `device_lists_changes_in_room` (#14534)Erik Johnston3-0/+28
This helps avoid reading unnecessarily large amounts of data from the table when querying with a set of room IDs.
2022-11-22Add missing type hints to test.util.caches (#14529)Patrick Cloke7-66/+76
2022-11-22Optimize `filter_events_for_client` for faster `/messages` - v2 (#14527)Eric Eastwood2-20/+80
Fix #14108
2022-11-22Refactor `federation_sender` and `pusher` configuration loading. (#14496)realtyem2-69/+71
To avoid duplicating the same logic for handling legacy configuration settings. This should help in applying similar logic to other worker types.
2022-11-22Add more prompts to bug report form (#14522)David Robertson2-2/+44
2022-11-22Add a type hint for `get_device_handler()` and fix incorrect types. (#14055)Patrick Cloke16-77/+185
This was the last untyped handler from the HomeServer object. Since it was being treated as Any (and thus unchecked) it was being used incorrectly in a few places.
2022-11-22Apply correct editorconfig to .pyi files (#14526)Brendan Abolivier2-1/+2
The current configuration might cause some editors to misbehave when editing stub files.
2022-11-22Track unconverted device list outbound pokes using a position instead (#14516)Sean Quah6-49/+158
When a local device list change is added to `device_lists_changes_in_room`, the `converted_to_destinations` flag is set to `FALSE` and the `_handle_new_device_update_async` background process is started. This background process looks for unconverted rows in `device_lists_changes_in_room`, copies them to `device_lists_outbound_pokes` and updates the flag. To update the `converted_to_destinations` flag, the database performs a `DELETE` and `INSERT` internally, which fragments the table. To avoid this, track unconverted rows using a `(stream ID, room ID)` position instead of the flag. From now on, the `converted_to_destinations` column indicates rows that need converting to outbound pokes, but does not indicate whether the conversion has already taken place. Closes #14037. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-22Parallelize calls to fetch bundled aggregations. (#14510)Patrick Cloke2-32/+52
The bundled aggregations for annotations, references, and edits can be parallelized.
2022-11-22Batch fetch bundled references (#14508)Patrick Cloke6-79/+133
Avoid an n+1 query problem and fetch the bundled aggregations for m.reference relations in a single query instead of a query per event. This applies similar logic for as was previously done for edits in 8b309adb436c162510ed1402f33b8741d71fc058 (#11660; threads in b65acead428653b988351ae8d7b22127a22039cd (#11752); and annotations in 1799a54a545618782840a60950ef4b64da9ee24d (#14491).
2022-11-22Batch fetch bundled annotations (#14491)Patrick Cloke5-141/+202
Avoid an n+1 query problem and fetch the bundled aggregations for m.annotation relations in a single query instead of a query per event. This applies similar logic for as was previously done for edits in 8b309adb436c162510ed1402f33b8741d71fc058 (#11660) and threads in b65acead428653b988351ae8d7b22127a22039cd (#11752).
2022-11-22tweak postgres dep notice v1.72.0David Robertson1-1/+1
2022-11-22Move postgres warning banner to top of readmeDavid Robertson1-2/+2
2022-11-221.72.0David Robertson4-2/+16
2022-11-22Update forgotten references to legacy metrics in the included Grafana ↵reivilibre2-345/+1210
dashboard. (#14477) Fixes https://github.com/matrix-org/synapse/issues/14465
2022-11-21Faster joins: filter out non local events when a room doesn't have its full ↵Mathieu Velten5-13/+43
state (#14404) Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-11-21Fix broken admin API request recommendation link (#14499)Brennan Chapman2-1/+2
Signed-off-by: Brennan Chapman <brennan@chapmanb.com>
2022-11-21Bump serde_json from 1.0.87 to 1.0.88 (#14505)dependabot[bot]2-2/+3
* Bump serde_json from 1.0.87 to 1.0.88 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.87 to 1.0.88. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.87...v1.0.88) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-21Bump phonenumbers from 8.12.56 to 8.13.0 (#14504)dependabot[bot]2-3/+4
* Bump phonenumbers from 8.12.56 to 8.13.0 Bumps [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) from 8.12.56 to 8.13.0. - [Release notes](https://github.com/daviddrysdale/python-phonenumbers/releases) - [Commits](https://github.com/daviddrysdale/python-phonenumbers/compare/v8.12.56...v8.13.0) --- updated-dependencies: - dependency-name: phonenumbers dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-21Bump towncrier from 21.9.0 to 22.8.0 (#14503)dependabot[bot]2-5/+6
* Bump towncrier from 21.9.0 to 22.8.0 Bumps [towncrier](https://github.com/hawkowl/towncrier) from 21.9.0 to 22.8.0. - [Release notes](https://github.com/hawkowl/towncrier/releases) - [Changelog](https://github.com/twisted/towncrier/blob/trunk/NEWS.rst) - [Commits](https://github.com/hawkowl/towncrier/compare/21.9.0...22.8.0) --- updated-dependencies: - dependency-name: towncrier dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-21Bump pygithub from 1.56 to 1.57 (#14500)dependabot[bot]2-5/+6
* Bump pygithub from 1.56 to 1.57 Bumps [pygithub](https://github.com/pygithub/pygithub) from 1.56 to 1.57. - [Release notes](https://github.com/pygithub/pygithub/releases) - [Changelog](https://github.com/PyGithub/PyGithub/blob/master/doc/changes.rst) - [Commits](https://github.com/pygithub/pygithub/compare/v1.56...v1.57) --- updated-dependencies: - dependency-name: pygithub dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-21Bump types-pillow from 9.2.2.1 to 9.3.0.1 (#14502)dependabot[bot]2-3/+4
* Bump types-pillow from 9.2.2.1 to 9.3.0.1 Bumps [types-pillow](https://github.com/python/typeshed) from 9.2.2.1 to 9.3.0.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pillow dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-21Bump sentry-sdk from 1.10.1 to 1.11.0 (#14501)dependabot[bot]2-3/+5
* Bump sentry-sdk from 1.10.1 to 1.11.0 Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.10.1 to 1.11.0. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.10.1...1.11.0) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-20Fixes to `federation_client` dev script (#14479)Richard van der Hoff2-35/+88
* Attempt to fix federation-client devscript handling of .well-known The script was setting the wrong value in the Host header * Fix TLS verification Turns out that actually doing TLS verification isn't that hard. Let's enable it.
2022-11-18Fix /key/v2/server calls with URL-unsafe key IDs (#14490)David Robertson3-1/+14
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-11-18Reduce default third party invite rate limit to 216 invites per day (#14487)Sean Quah2-4/+2
The previous default was the same as the `rc_message` rate limit, which defaults to 17,280 per day. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-17Enable 'strict_equality' checking for mypy (#14452)Andrew Morgan2-0/+2
2022-11-17Fix version that `worker_main_http_uri` is redundant from (#14476)David Robertson3-3/+4
* Fix version that `worker_main_http_uri` is redundant from * Changelog
2022-11-17Fix check to ignore blank lines in incoming TCP replication (#14449)Andrew Morgan3-4/+5
2022-11-17Faster joins: do not wait for full state when creating events to send (#14403)Mathieu Velten3-1/+9
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-11-16Reintroduce #14376, with bugfix for monoliths (#14468)David Robertson14-186/+230
* Add tests for StreamIdGenerator * Drive-by: annotate all defs * Revert "Revert "Remove slaved id tracker (#14376)" (#14463)" This reverts commit d63814fd736fed5d3d45ff3af5e6d3bfae50c439, which in turn reverted 36097e88c4da51fce6556a58c49bd675f4cf20ab. This restores the latter. * Fix StreamIdGenerator not handling unpersisted IDs Spotted by @erikjohnston. Closes #14456. * Changelog Co-authored-by: Nick Mills-Barrett <nick@fizzadar.com> Co-authored-by: Erik Johnston <erik@matrix.org>
2022-11-16Remove need for `worker_main_http_uri` setting to use /keys/upload. (#14400)realtyem7-127/+130
2022-11-16Update changelog v1.72.0rc1Erik Johnston1-2/+2
2022-11-16Point to our deprecation policyErik Johnston1-1/+1
2022-11-16Update changelogErik Johnston1-25/+2
2022-11-16Fix an invalid comparison of `UserPresenceState` to `str` (#14393)Andrew Morgan5-8/+46
2022-11-16Remove redundant types from comments. (#14412)Patrick Cloke55-176/+174
Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.
2022-11-161.72.0rc1Erik Johnston68-66/+102
2022-11-16Fix background updates failing to add unique indexes on receipts (#14453)Sean Quah3-24/+357
As part of the database migration to support threaded receipts, there is a possible window in between `73/08thread_receipts_non_null.sql.postgres` removing the original unique constraints on `receipts_linearized` and `receipts_graph` and the `reeipts_linearized_unique_index` and `receipts_graph_unique_index` background updates from `72/08thread_receipts.sql` completing where the unique constraints on `receipts_linearized` and `receipts_graph` are missing. Any emulated upserts on these tables must therefore be performed with a lock held, otherwise duplicate rows can end up in the tables when there are concurrent emulated upserts. Fix the missing lock. Note that emulated upserts no longer happen by default on sqlite, since the minimum supported version of sqlite supports native upserts by default now. Finally, clean up any duplicate receipts that may have crept in before trying to create the `receipts_graph_unique_index` and `receipts_linearized_unique_index` unique indexes. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-16Revert "Remove slaved id tracker (#14376)" (#14463)Erik Johnston11-74/+176
This reverts commit 36097e88c4da51fce6556a58c49bd675f4cf20ab.
2022-11-16Don't filter state in /context response (#14461)Erik Johnston2-1/+2
We don't filter state usually, so doing so here is a waste of time. This is not much of an issue for clients that enable lazy loading of members, since there will be fewer state events.
2022-11-16Fix stub return type of `PushRuleEvaluator.run` (#14451)Andrew Morgan2-2/+3
2022-11-16Fix HTML templates missing correct HTML tags (#14448)Erik Johnston22-0/+23
2022-11-15Include heroes in partial join responses' state (#14442)David Robertson5-25/+60
* Pull out hero selection logic * Include heroes in partial join response's state * Changelog * Fixup trial test * Remove TODO
2022-11-15Fix typechecking errors introduced in #14128 (#14455)David Robertson3-4/+6
* Fix typechecking errors introduced in #14128 * Changelog * Correct annotations so that context_factory works if you don't use TLS
2022-11-15Send content rules with pattern_type to clients (#14356)DeepBlueV7.X2-3/+14
2022-11-15Support using SSL on worker endpoints. (#14128)Tuomas Ojamies7-53/+100
* Fix missing SSL support in worker endpoints. * Add changelog * SSL for Replication endpoint * Remove unit test change * Refactor listener creation to reduce duplicated code * Fix the logger message * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Update synapse/app/_base.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add config documentation for new TLS option Co-authored-by: Tuomas Ojamies <tojamies@palantir.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-11-15Update docstring to clarify that `get_partial_state_events_batch` does not ↵reivilibre2-1/+10
just give you completely arbitrary partial-state events. (#14417)
2022-11-14Include additional TURN server example into documentation (#14293)sando385-211/+390
* Include eturnal TURN server configuration example and moving specific configuration examples into sub folders. * Update docs/turn-howto.md Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Update docs/setup/turn/coturn.md Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Update docs/setup/turn/eturnal.md Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Fix TURN relaying public IP address hint * lint eturnal installation commands * Adjust synapse setup to link to existing documentation ..avoid redundant information. * remove redundant text * include alpine linux package link * Create 14293.doc * Update 14293.doc add missing dot * Update docs/setup/turn/eturnal.md Co-authored-by: reivilibre <olivier@librepush.net> * Update docs/setup/turn/eturnal.md Co-authored-by: reivilibre <olivier@librepush.net> * Update docs/setup/turn/coturn.md Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com> * Update docs/setup/turn/coturn.md Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com> * Update docs/setup/turn/coturn.md Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com> * Update docs/setup/turn/eturnal.md Co-authored-by: reivilibre <olivier@librepush.net> * Update docs/setup/turn/coturn.md Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com> * Update docs/setup/turn/coturn.md Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com> * Update eturnal.md to link to official documentation ... and to simplify some aspects * Adjust coturn to link to default prefix * Mention eturnalctl location * Update docs/turn-howto.md Co-authored-by: Saarko <sandomir@tutanotal.com> Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> Co-authored-by: reivilibre <olivier@librepush.net> Co-authored-by: Moritz Dietz <moritzdietz@users.noreply.github.com>
2022-11-14Remove slaved id tracker (#14376)Nick Mills-Barrett11-176/+74
This matches the multi instance writer ID generator class which can both handle advancing the current token over replication and by calling the database.
2022-11-14Bump jsonschema from 4.16.0 to 4.17.0 (#14439)dependabot[bot]2-3/+4
* Bump jsonschema from 4.16.0 to 4.17.0 Bumps [jsonschema](https://github.com/python-jsonschema/jsonschema) from 4.16.0 to 4.17.0. - [Release notes](https://github.com/python-jsonschema/jsonschema/releases) - [Changelog](https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst) - [Commits](https://github.com/python-jsonschema/jsonschema/compare/v4.16.0...v4.17.0) --- updated-dependencies: - dependency-name: jsonschema dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump flake8-comprehensions from 3.8.0 to 3.10.1 (#14438)dependabot[bot]2-3/+4
* Bump flake8-comprehensions from 3.8.0 to 3.10.1 Bumps [flake8-comprehensions](https://github.com/adamchainz/flake8-comprehensions) from 3.8.0 to 3.10.1. - [Release notes](https://github.com/adamchainz/flake8-comprehensions/releases) - [Changelog](https://github.com/adamchainz/flake8-comprehensions/blob/main/HISTORY.rst) - [Commits](https://github.com/adamchainz/flake8-comprehensions/compare/3.8.0...3.10.1) --- updated-dependencies: - dependency-name: flake8-comprehensions dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump types-pyopenssl from 22.0.10 to 22.1.0.2 (#14437)dependabot[bot]2-5/+6
* Bump types-pyopenssl from 22.0.10 to 22.1.0.2 Bumps [types-pyopenssl](https://github.com/python/typeshed) from 22.0.10 to 22.1.0.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyopenssl dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump canonicaljson from 1.6.3 to 1.6.4 (#14440)dependabot[bot]2-3/+4
* Bump canonicaljson from 1.6.3 to 1.6.4 Bumps [canonicaljson](https://github.com/matrix-org/python-canonicaljson) from 1.6.3 to 1.6.4. - [Release notes](https://github.com/matrix-org/python-canonicaljson/releases) - [Changelog](https://github.com/matrix-org/python-canonicaljson/blob/main/CHANGES.md) - [Commits](https://github.com/matrix-org/python-canonicaljson/compare/v1.6.3...v1.6.4) --- updated-dependencies: - dependency-name: canonicaljson dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump types-setuptools from 65.5.0.2 to 65.5.0.3 (#14436)dependabot[bot]2-3/+4
* Bump types-setuptools from 65.5.0.2 to 65.5.0.3 Bumps [types-setuptools](https://github.com/python/typeshed) from 65.5.0.2 to 65.5.0.3. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Make Dependabot only bump Rust deps in the lock file (#14434)Erik Johnston3-6/+8
This is to help downstream packagers.
2022-11-14Remove unused type-ignores (#14433)David Robertson3-6/+8
* Remove unused type-ignores Oversights in #14427 and #14429. * Changelog
2022-11-14Clean-up events persistance code (#14411)Patrick Cloke3-8/+4
By removing unused variables and making some arguments required which are always provided.
2022-11-14Bump gitpython from 3.1.27 to 3.1.29 (#14429)dependabot[bot]2-3/+4
* Bump gitpython from 3.1.27 to 3.1.29 Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.27 to 3.1.29. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.27...3.1.29) --- updated-dependencies: - dependency-name: gitpython dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump types-pyyaml from 6.0.12.1 to 6.0.12.2 (#14428)dependabot[bot]2-3/+4
* Bump types-pyyaml from 6.0.12.1 to 6.0.12.2 Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.12.1 to 6.0.12.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump attrs from 21.4.0 to 22.1.0 (#14427)dependabot[bot]2-7/+8
* Bump attrs from 21.4.0 to 22.1.0 Bumps [attrs](https://github.com/python-attrs/attrs) from 21.4.0 to 22.1.0. - [Release notes](https://github.com/python-attrs/attrs/releases) - [Changelog](https://github.com/python-attrs/attrs/blob/main/CHANGELOG.rst) - [Commits](https://github.com/python-attrs/attrs/compare/21.4.0...22.1.0) --- updated-dependencies: - dependency-name: attrs dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump flake8 from 4.0.1 to 5.0.4 (#14431)dependabot[bot]2-20/+21
* Bump flake8 from 4.0.1 to 5.0.4 Bumps [flake8](https://github.com/pycqa/flake8) from 4.0.1 to 5.0.4. - [Release notes](https://github.com/pycqa/flake8/releases) - [Commits](https://github.com/pycqa/flake8/compare/4.0.1...5.0.4) --- updated-dependencies: - dependency-name: flake8 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump types-jsonschema from 4.17.0.0 to 4.17.0.1 (#14430)dependabot[bot]2-3/+6
* Bump types-jsonschema from 4.17.0.0 to 4.17.0.1 Bumps [types-jsonschema](https://github.com/python/typeshed) from 4.17.0.0 to 4.17.0.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-jsonschema dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump blake2 from 0.10.4 to 0.10.5 (#14426)dependabot[bot]3-3/+4
* Bump blake2 from 0.10.4 to 0.10.5 Bumps [blake2](https://github.com/RustCrypto/hashes) from 0.10.4 to 0.10.5. - [Release notes](https://github.com/RustCrypto/hashes/releases) - [Commits](https://github.com/RustCrypto/hashes/compare/blake2-v0.10.4...blake2-v0.10.5) --- updated-dependencies: - dependency-name: blake2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump actions/upload-artifact from 2 to 3 (#14425)dependabot[bot]2-1/+2
* Bump actions/upload-artifact from 2 to 3 Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-14Bump dawidd6/action-download-artifact from 2.24.1 to 2.24.2 (#14424)dependabot[bot]2-1/+2
* Bump dawidd6/action-download-artifact from 2.24.1 to 2.24.2 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.24.1 to 2.24.2. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/b12b127cf24433d14b4f93cee62f5465076ba82a...e6e25ac3a2b93187502a8be1ef9e9603afc34925) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-11Update sample Nginx configuration to HTTP 1.1 (#14414)Brad Jones2-0/+4
Signed-off-by: Brad Jones <brad@kinksters.dating>
2022-11-11Add an Admin API endpoint for looking up users based on 3PID (#14405)Ashish Kumar5-13/+161
2022-11-11Merge/remove `Slaved*` stores into `WorkerStores` (#14375)Nick Mills-Barrett17-377/+202
2022-11-10Remove duplicated code to evict entries. (#14410)Patrick Cloke2-9/+3
This code was factored out to a method, but also left in-place. Calling this twice in a row makes no sense: the first call will reduce the size appropriately, but the loop will immediately exit since the cache size was already reduced.
2022-11-10Fix PostgreSQL sometimes using table scans for `event_search` (#14409)Sean Quah2-0/+34
PostgreSQL may underestimate the number of distinct `room_id`s in `event_search`, which can cause it to use table scans for queries for multiple rooms. Fix this by setting `n_distinct` on the column. Resolves #14402. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-10Quieter logging for stateres failure at missing prev events (#14346)David Robertson2-3/+3
2022-11-09Drop support for Postgres 10 in full text search code. (#14397)Patrick Cloke4-95/+41
2022-11-09Bump dawidd6/action-download-artifact from 2.24.0 to 2.24.1 (#14398)dependabot[bot]2-1/+2
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-09Modernize configure_workers_and_start.py bootstrapping script for ↵realtyem2-15/+15
Dockerfile-workers. (#14294)
2022-11-09Build Debian packages for Ubuntu 22.10 Kinetic Kudu (#14396)Sean Quah2-0/+2
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-09Test against PostgreSQL 15 in CI (#14394)Sean Quah3-2/+12
Resolves #14170. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-08Remove support for PostgreSQL 10 (#14392)Sean Quah5-4/+15
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-08Add all Stream Writer worker types to configure_workers_and_start.py (#14197)realtyem2-7/+70
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-08Allow override for requesting specific worker types for Complement on ↵realtyem4-1/+18
command line. (#14324) * Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested. * Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint. * Update docs to reflect this new ability. * Changelog * Don't rely on soft wrapping to format long strings Good idea dklimpel. Thanks for catching that. Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Small nits just noticed in docs. * Fixup new line in docs. Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
2022-11-08Update CHANGES.md to mention PostgreSQL 10 end of life v1.71.0Sean Quah1-0/+2
2022-11-08Update CHANGES.mdSean Quah1-6/+5
2022-11-081.71.0Sean Quah3-1/+13
2022-11-07Correctly create power level event during initial room creation (#14361)Shay3-4/+26
2022-11-07Fix background update table-scanning `events` (#14374)Richard van der Hoff2-8/+9
When this background update did its last batch, it would try to update all the events that had been inserted since the bgupdate started, which could cause a table-scan. Make sure we limit the update correctly.
2022-11-07Add example on how to load balance /sync requests (#14297)aceArt-GmbH2-3/+6
Signed-off-by: lukas <lukas.walter@aceart.de> Signed-off-by: lukas <lukas.walter@aceart.de>
2022-11-07Do not reject `/sync` requests with unrecognised filter fields (#14369)Sean Quah3-6/+24
For forward compatibility, Synapse needs to ignore fields it does not recognise instead of raising an error. Fixes #14365. Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-07Bump types-jsonschema from 4.4.6 to 4.17.0.0 (#14386)dependabot[bot]2-3/+4
* Bump types-jsonschema from 4.4.6 to 4.17.0.0 Bumps [types-jsonschema](https://github.com/python/typeshed) from 4.4.6 to 4.17.0.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-jsonschema dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump types-pyyaml from 6.0.12 to 6.0.12.1 (#14385)dependabot[bot]2-5/+6
* Bump types-pyyaml from 6.0.12 to 6.0.12.1 Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.12 to 6.0.12.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump cryptography from 36.0.1 to 38.0.3 (#14384)dependabot[bot]2-24/+31
* Bump cryptography from 36.0.1 to 38.0.3 Bumps [cryptography](https://github.com/pyca/cryptography) from 36.0.1 to 38.0.3. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/36.0.1...38.0.3) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump pillow from 9.2.0 to 9.3.0 (#14383)dependabot[bot]2-59/+63
* Bump pillow from 9.2.0 to 9.3.0 Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.2.0 to 9.3.0. - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](https://github.com/python-pillow/Pillow/compare/9.2.0...9.3.0) --- updated-dependencies: - dependency-name: pillow dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump types-setuptools from 65.5.0.1 to 65.5.0.2 (#14382)dependabot[bot]2-3/+4
* Bump types-setuptools from 65.5.0.1 to 65.5.0.2 Bumps [types-setuptools](https://github.com/python/typeshed) from 65.5.0.1 to 65.5.0.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-setuptools dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump pyo3 from 0.17.2 to 0.17.3 (#14381)dependabot[bot]3-11/+12
* Bump pyo3 from 0.17.2 to 0.17.3 Bumps [pyo3](https://github.com/pyo3/pyo3) from 0.17.2 to 0.17.3. - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](https://github.com/pyo3/pyo3/compare/v0.17.2...v0.17.3) --- updated-dependencies: - dependency-name: pyo3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump regex from 1.6.0 to 1.7.0 (#14380)dependabot[bot]3-3/+4
* Bump regex from 1.6.0 to 1.7.0 Bumps [regex](https://github.com/rust-lang/regex) from 1.6.0 to 1.7.0. - [Release notes](https://github.com/rust-lang/regex/releases) - [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.0) --- updated-dependencies: - dependency-name: regex dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump peaceiris/actions-mdbook from 1.1.14 to 1.2.0 (#14379)dependabot[bot]2-1/+2
* Bump peaceiris/actions-mdbook from 1.1.14 to 1.2.0 Bumps [peaceiris/actions-mdbook](https://github.com/peaceiris/actions-mdbook) from 1.1.14 to 1.2.0. - [Release notes](https://github.com/peaceiris/actions-mdbook/releases) - [Changelog](https://github.com/peaceiris/actions-mdbook/blob/main/CHANGELOG.md) - [Commits](https://github.com/peaceiris/actions-mdbook/compare/v1.1.14...adeb05db28a0c0004681db83893d56c0388ea9ea) --- updated-dependencies: - dependency-name: peaceiris/actions-mdbook dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-07Bump dawidd6/action-download-artifact from 2.15.0 to 2.24.0 (#14378)dependabot[bot]2-1/+2
* Bump dawidd6/action-download-artifact from 2.15.0 to 2.24.0 Bumps [dawidd6/action-download-artifact](https://github.com/dawidd6/action-download-artifact) from 2.15.0 to 2.24.0. - [Release notes](https://github.com/dawidd6/action-download-artifact/releases) - [Commits](https://github.com/dawidd6/action-download-artifact/compare/af92a8455a59214b7b932932f2662fdefbd78126...46b4ae883bf0726f5949d025d31cb62c7a5ac70c) --- updated-dependencies: - dependency-name: dawidd6/action-download-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-04Bump flake8-bugbear from 22.9.23 to 22.10.27 (#14329)dependabot[bot]5-9/+12
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-11-04Fix the trigger path for deploying documentation PRs (#14370)Brendan Abolivier2-1/+2
This was missed from #12947
2022-11-04Upload documentation PRs to Netlify (#12947)Michael Telatynski3-0/+69
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> Co-authored-by: Erik Johnston <erik@matrix.org> Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-11-04Fix /refresh endpoint version (#14364)Tulir Asokan3-9/+10
2022-11-04Update CHANGES.md v1.71.0rc2Sean Quah1-11/+7
2022-11-041.71.0rc2Sean Quah7-5/+29
2022-11-04 Include monthly active user metrics in the list of legacy metrics names ↵David Robertson2-0/+4
(#14360)
2022-11-03Cherry-pick "Run trial tests against Python 3.11 (#13812)" and fixup commitDavid Robertson5-94/+146
4f5d492cd6a9438de03d1b768f4c220cb662ac06 The release branch CI is failing because poetry seems unable to install wrapt 1.13.3 when run under CPython 3.11. Develop has already bumped wrapt for 3.11 compatibility. Cherry-pick that commit here to try and get CI going again.
2022-11-03Update legacy `synapse_admin_mau:` metric names in docs (#14358)Will Hunt2-3/+4
* Rename legacy metrics in MAU docs * changelog
2022-11-03Implement MSC3912: Relation-based redactions (#14260)Brendan Abolivier10-28/+486
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-11-02Use maintained action to install Rust in latest deps/twisted trunk jobs (#14351)David Robertson3-18/+13
* Use maintained action to install Rust Part of #14203. Like the changes in #14313. * Changelog
2022-11-02Disable legacy Prometheus metric names by default. They can still be ↵reivilibre5-3/+29
re-enabled for now, but they will be removed altogether in Synapse 1.73.0. (#14353)
2022-11-01Fix issue automation for Needs-Info (#14343)Kat Gerasimova1-27/+27
Run when an issue is labelled with X-Needs-Info only. Add to triage board. Use itemId which is output by actions/add-to-project to run the mutation to update the field value (i.e. move to the right column).
2022-11-01Fix typo in #13320 which could cause log spam (#14347)David Robertson2-1/+2
2022-11-01Enable testing MSC3874 in complement. (#14339)Patrick Cloke3-3/+4
2022-11-01Allow PUT/GET of aliases during faster join (#14292)David Robertson2-2/+3
without blocking on full state.
2022-11-01Bump twisted from 22.8.0 to 22.10.0 (#14340)dependabot[bot]2-12/+13
* Bump twisted from 22.8.0 to 22.10.0 Bumps [twisted](https://github.com/twisted/twisted) from 22.8.0 to 22.10.0. - [Release notes](https://github.com/twisted/twisted/releases) - [Changelog](https://github.com/twisted/twisted/blob/trunk/NEWS.rst) - [Commits](https://github.com/twisted/twisted/compare/twisted-22.8.0...twisted-22.10.0) --- updated-dependencies: - dependency-name: twisted dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-11-01Deal with another batch of GHA warning messages (#14313)David Robertson2-16/+39
2022-11-01Revert a testing commit from #13812David Robertson1-3/+3
It (4f5d492cd6a9438de03d1b768f4c220cb662ac06) should have been reverted before the merge to develop.
2022-11-01Run trial tests against Python 3.11 (#13812)David Robertson5-97/+149
2022-11-01Adjust changelog v1.71.0rc1David Robertson1-25/+34
2022-11-011.71.0rc1David Robertson55-54/+80
2022-11-01Revert "Fix event size checks (#13710)"David Robertson2-6/+5
This reverts commit fab495a9e1442d99e922367f65f41de5eaa488eb. As noted in https://github.com/matrix-org/synapse/pull/13710#issuecomment-1298396007: > We want to see this change land for the protocol's sake (and plan to un-revert it) but want to give this a little more time before releasing this.
2022-11-01Fix type annotation causing import time error in the Complement forking ↵reivilibre2-2/+3
launcher. (#14084) Co-authored-by: David Robertson <davidr@element.io>
2022-10-31Fix dehydrated device REST checks (#14336)David Robertson3-3/+37
2022-10-31Support OIDC backchannel logouts (#11414)Quentin Gliech13-66/+960
If configured an OIDC IdP can log a user's session out of Synapse when they log out of the identity provider. The IdP sends a request directly to Synapse (and must be configured with an endpoint) when a user logs out.
2022-10-31Bump sentry-sdk from 1.5.11 to 1.10.1 (#14330)dependabot[bot]2-10/+13
* Bump sentry-sdk from 1.5.11 to 1.10.1 Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.5.11 to 1.10.1. - [Release notes](https://github.com/getsentry/sentry-python/releases) - [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-python/compare/1.5.11...1.10.1) --- updated-dependencies: - dependency-name: sentry-sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-10-31Bump psycopg2 from 2.9.4 to 2.9.5 (#14331)dependabot[bot]2-12/+13
* Bump psycopg2 from 2.9.4 to 2.9.5 Bumps [psycopg2](https://github.com/psycopg/psycopg2) from 2.9.4 to 2.9.5. - [Release notes](https://github.com/psycopg/psycopg2/releases) - [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS) - [Commits](https://github.com/psycopg/psycopg2/commits) --- updated-dependencies: - dependency-name: psycopg2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-10-31Bump twine from 3.8.0 to 4.0.1 (#14332)dependabot[bot]2-31/+31
* Bump twine from 3.8.0 to 4.0.1 Bumps [twine](https://github.com/pypa/twine) from 3.8.0 to 4.0.1. - [Release notes](https://github.com/pypa/twine/releases) - [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst) - [Commits](https://github.com/pypa/twine/compare/3.8.0...4.0.1) --- updated-dependencies: - dependency-name: twine dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-10-31Bump black from 22.3.0 to 22.10.0 (#14328)dependabot[bot]2-27/+26
* Bump black from 22.3.0 to 22.10.0 Bumps [black](https://github.com/psf/black) from 22.3.0 to 22.10.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](https://github.com/psf/black/compare/22.3.0...22.10.0) --- updated-dependencies: - dependency-name: black dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Changelog Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
2022-10-31Avoid incrementing bg process utime/stime counters by negative durations ↵David Robertson6-5/+35
(#14323)
2022-10-28Prevent federation user keys query from returning device names if disallowed ↵Andrew Morgan3-9/+46
(#14304)
2022-10-28Improve `RawHeaders` type hints (#14303)David Robertson3-9/+24
2022-10-28Switch search SQL to triple-quote strings. (#14311)Patrick Cloke2-89/+100
For ease of reading we switch from concatenated strings to triple quote strings.
2022-10-28Add a note that #14300 did not get a fix for 1.70.1 (#14318)Andrew Morgan1-0/+8
2022-10-28Add docs for an empty `trusted_key_servers` config option (#13999)Dirk Klimpel2-0/+7
* Add docs for an empty `trusted_key_servers` config option * small rewording * Tweak changelog
2022-10-28Tweak changelog v1.70.1Olivier Wilkinson (reivilibre)1-1/+1
2022-10-281.70.1Olivier Wilkinson (reivilibre)5-3/+17
2022-10-28Fix room creation being rate limited too aggressively since Synapse v1.69.0. ↵reivilibre4-9/+70
(#14314) * Introduce a test for the old behaviour which we want to restore * Reintroduce the old behaviour in a simpler way * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Use 1 credit instead of 2 for creating a room: be more lenient than before Notably, the UI in Element Web was still broken after restoring to prior behaviour. After discussion, we agreed that it would be sensible to increase the limit. Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-10-27Check appservice user interest against the local users instead of all users ↵Eric Eastwood7-14/+214
(`get_users_in_room` mis-use) (#13958)
2022-10-27Fix tests for change in PostgreSQL 14 behavior change. (#14310)Patrick Cloke3-7/+15
PostgreSQL 14 changed the behavior of `websearch_to_tsquery` to improve some behaviour. The tests were hitting those edge-cases about handling of hanging double quotes. This fixes the tests to take into account the PostgreSQL version.
2022-10-27Add workers settings to configuration manual (#14086)Dirk Klimpel5-82/+291
* Add workers settings to configuration manual * Update `pusher_instances` * update url to python logger * update headlines * update links after headline change * remove link from `daemon process` There is no docs in Synapse for this * extend example for `federation_sender_instances` and `pusher_instances` * more infos about stream writers * add link to DAG * update `pusher_instances` * update `worker_listeners` * update `stream_writers` * Update `worker_name` Co-authored-by: David Robertson <davidr@element.io>
2022-10-27Fix presence bug introduced in 1.64 by #13313 (#14243)Mathieu Velten2-2/+2
* Fix presence bug introduced in 1.64 by #13313 Signed-off-by: Mathieu Velten <mathieuv@matrix.org> * Add changelog * Add DISTINCT * Apply suggestions from code review Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-10-27Reject history insertion during partial joins (#14291)David Robertson2-0/+8
2022-10-26Refactor MSC3030 `/timestamp_to_event` to move away from our snowflake pull ↵Eric Eastwood7-126/+191
from `destination` pattern (#14096) 1. `federation_client.timestamp_to_event(...)` now handles all `destination` looping and uses our generic `_try_destination_list(...)` helper. 2. Consistently handling `NotRetryingDestination` and `FederationDeniedError` across `get_pdu` , backfill, and the generic `_try_destination_list` which is used for many places we use this pattern. 3. `get_pdu(...)` now returns `PulledPduInfo` so we know which `destination` we ended up pulling the PDU from
2022-10-26Use poetry 1.2 for complement in latest deps (#14305)David Robertson2-2/+2
2022-10-26fix broken avatar checks when server_name contains a port (#13927)Ashish Kumar3-1/+55
Fixes check_avatar_size_and_mime_type() to successfully update avatars on homeservers running on non-default ports which it would mistakenly treat as remote homeserver while validating the avatar's size and mime type. Signed-off-by: Ashish Kumar ashfame@users.noreply.github.com
2022-10-26Fix incorrectly sending authentication tokens to application service as ↵David Robertson3-8/+13
headers (#14301)
2022-10-26Save login tokens in database (#13844)Quentin Gliech11-227/+337
* Save login tokens in database Signed-off-by: Quentin Gliech <quenting@element.io> * Add upgrade notes * Track login token reuse in a Prometheus metric Signed-off-by: Quentin Gliech <quenting@element.io>
2022-10-261.70.0 v1.70.0Olivier Wilkinson (reivilibre)3-1/+13
2022-10-25Unified search query syntax using the full-text search capabilities of the ↵James Salter5-35/+454
underlying DB. (#11635) Support a unified search query syntax which leverages more of the full-text search of each database supported by Synapse. Supports, with the same syntax across Postgresql 11+ and Sqlite: - quoted "search terms" - `AND`, `OR`, `-` (negation) operators - Matching words based on their stem, e.g. searches for "dog" matches documents containing "dogs". This is achieved by - If on postgresql 11+, pass the user input to `websearch_to_tsquery` - If on sqlite, manually parse the query and transform it into the sqlite-specific query syntax. Note that postgresql 10, which is close to end-of-life, falls back to using `phraseto_tsquery`, which only supports a subset of the features. Multiple terms separated by a space are implicitly ANDed. Note that: 1. There is no escaping of full-text syntax that might be supported by the database; e.g. `NOT`, `NEAR`, `*` in sqlite. This runs the risk that people might discover this as accidental functionality and depend on something we don't guarantee. 2. English text is assumed for stemming. To support other languages, either the target language needs to be known at the time of indexing the message (via room metadata, or otherwise), or a separate index for each language supported could be created. Sqlite docs: https://www.sqlite.org/fts3.html#full_text_index_queries Postgres docs: https://www.postgresql.org/docs/11/textsearch-controls.html
2022-10-25Refactor OIDC tests to better mimic an actual OIDC provider. (#13910)Quentin Gliech10-460/+747
This implements a fake OIDC server, which intercepts calls to the HTTP client. Improves accuracy of tests by covering more internal methods. One particular example was the ID token validation, which previously mocked. This uncovered an incorrect dependency: Synapse actually requires at least authlib 0.15.1, not 0.14.0.
2022-10-25Implementation for MSC3664: Pushrules for relations (#11804)DeepBlueV7.X10-17/+454
2022-10-25Tweak changelog v1.70.0rc2Olivier Wilkinson (reivilibre)1-1/+1
2022-10-25Remove unused `@lru_cache` decorator (#13595)Nick Mills-Barrett3-140/+5
* Remove unused `@lru_cache` decorator Spotted this working on something else. Co-authored-by: David Robertson <davidr@element.io>
2022-10-25Cache rust build deps in trial CI (#14287)Erik Johnston2-0/+9
2022-10-251.70.0rc2Olivier Wilkinson (reivilibre)8-6/+25
2022-10-25Enable WAL for SQLite (#13897)asymmetric2-0/+5
Signed-off-by: Lorenzo Manacorda <lorenzo@mailbox.org>
2022-10-24Revert behavior change for bundling edits of non-message events (#14283)Patrick Cloke2-4/+8
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-23 00:10:47 +0000