summary refs log tree commit diff
path: root/synapse/handlers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge tag 'v1.33.0' into babolivier/dinsic_1.41.0Brendan Abolivier2021-08-3147-244/+903
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.33.0 (2021-05-05) =========================== Features -------- - Build Debian packages for Ubuntu 21.04 (Hirsute Hippo). ([\#9909](https://github.com/matrix-org/synapse/issues/9909)) Synapse 1.33.0rc2 (2021-04-29) ============================== Bugfixes -------- - Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](https://github.com/matrix-org/synapse/issues/9900)) Synapse 1.33.0rc1 (2021-04-28) ============================== Features -------- - Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814)) - Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850)) - Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832)) Bugfixes -------- - Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726)) - Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788)) - Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802)) - Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833)) - Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867)) - Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868)) Improved Documentation ---------------------- - Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801)) Internal Changes ---------------- - Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162)) - Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786)) - Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796)) - Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815)) - Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816)) - Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817)) - Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821)) - Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838)) - Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845)) - Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855)) - Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856)) - Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858)) - Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871)) - Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874)) - Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875)) - Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876)) - Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878)) - Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879)) - Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
| * Fix tight loop handling presence replication. (#9900)Erik Johnston2021-04-281-1/+23
| | | | | | | | | | Only affects workers. Introduced in #9819. Fixes #9899.
| * Use current state table for `presence.get_interested_remotes` (#9887)Erik Johnston2021-04-271-7/+2
| | | | | | This should be a lot quicker than asking the state handler.
| * Allow OIDC cookies to work on non-root public baseurls (#9726)Andrew Morgan2021-04-231-5/+17
| | | | | | | | | | Applied a (slightly modified) patch from https://github.com/matrix-org/synapse/issues/9574. As far as I understand this would allow the cookie set during the OIDC flow to work on deployments using public baseurls that do not sit at the URL path root.
| * Make DomainSpecificString an attrs class (#9875)Erik Johnston2021-04-231-0/+5
| |
| * Split presence out of master (#9820)Erik Johnston2021-04-231-20/+36
| |
| * Check for space membership during a remote join of a restricted room (#9814)Patrick Cloke2021-04-233-68/+124
| | | | | | | | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
| * Clear the resync bit after resyncing device lists (#9867)Richard van der Hoff2021-04-221-0/+7
| | | | | | Fixes #9866.
| * Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-225-8/+18
| | | | | | This is no longer required, since we have dropped support for Python 3.5.
| * Rename handler and config modules which end in handler/config. (#9816)Patrick Cloke2021-04-203-4/+1
| |
| * Add presence federation stream (#9819)Erik Johnston2021-04-201-20/+223
| |
| * Fix bug where we sent remote presence states to remote servers (#9850)Erik Johnston2021-04-201-3/+8
| |
| * Fix (final) Bugbear violations (#9838)Jonathan de Jong2021-04-203-10/+7
| |
| * Port "Allow users to click account renewal links multiple times without ↵Andrew Morgan2021-04-192-25/+80
| | | | | | | | | | hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
| * Sanity check identity server passed to bind/unbind. (#9802)Denis Kasak2021-04-191-3/+26
| | | | | | | | Signed-off-by: Denis Kasak <dkasak@termina.org.uk>
| * Don't send normal presence updates over federation replication stream (#9828)Erik Johnston2021-04-191-15/+63
| |
| * User directory: use calculated room membership state instead (#9821)Andrew Morgan2021-04-161-7/+8
| | | | | | | | | | Fixes: #9797. Should help reduce CPU usage on the user directory, especially when memberships change in rooms with lots of state history.
| * Separate creating an event context from persisting it in the federation ↵Patrick Cloke2021-04-141-65/+113
| | | | | | | | | | | | handler (#9800) This refactoring allows adding logic that uses the event context before persisting it.
| * Revert "Check for space membership during a remote join of a restricted ↵Patrick Cloke2021-04-143-227/+129
| | | | | | | | | | | | | | | | room. (#9763)" This reverts commit cc51aaaa7adb0ec2235e027b5184ebda9b660ec4. The PR was prematurely merged and not yet approved.
| * Check for space membership during a remote join of a restricted room. (#9763)Patrick Cloke2021-04-143-129/+227
| | | | | | | | | | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
| * Move some replication processing out of generic_worker (#9796)Erik Johnston2021-04-141-0/+246
| | | | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-1446-46/+0
| | | | | | | | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* | Merge tag 'v1.32.0' into babolivier/dinsic_1.41.0Brendan Abolivier2021-08-3114-209/+511
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.32.0 (2021-04-20) =========================== **Note:** This release requires Python 3.6+ and Postgres 9.6+ or SQLite 3.22+. This release removes the deprecated `GET /_synapse/admin/v1/users/<user_id>` admin API. Please use the [v2 API](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/user_admin_api.rst#query-user-account) instead, which has improved capabilities. This release requires Application Services to use type `m.login.application_service` when registering users via the `/_matrix/client/r0/register` endpoint to comply with the spec. Please ensure your Application Services are up to date. Bugfixes -------- - Fix the log lines of nested logging contexts. Broke in 1.32.0rc1. ([\#9829](https://github.com/matrix-org/synapse/issues/9829)) Synapse 1.32.0rc1 (2021-04-13) ============================== Features -------- - Add a Synapse module for routing presence updates between users. ([\#9491](https://github.com/matrix-org/synapse/issues/9491)) - Add an admin API to manage ratelimit for a specific user. ([\#9648](https://github.com/matrix-org/synapse/issues/9648)) - Include request information in structured logging output. ([\#9654](https://github.com/matrix-org/synapse/issues/9654)) - Add `order_by` to the admin API `GET /_synapse/admin/v2/users`. Contributed by @dklimpel. ([\#9691](https://github.com/matrix-org/synapse/issues/9691)) - Replace the `room_invite_state_types` configuration setting with `room_prejoin_state`. ([\#9700](https://github.com/matrix-org/synapse/issues/9700)) - Add experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9717](https://github.com/matrix-org/synapse/issues/9717), [\#9735](https://github.com/matrix-org/synapse/issues/9735)) - Update experimental support for Spaces: include `m.room.create` in the room state sent with room-invites. ([\#9710](https://github.com/matrix-org/synapse/issues/9710)) - Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. ([\#9766](https://github.com/matrix-org/synapse/issues/9766)) Bugfixes -------- - Prevent `synapse_forward_extremities` and `synapse_excess_extremity_events` Prometheus metrics from initially reporting zero-values after startup. ([\#8926](https://github.com/matrix-org/synapse/issues/8926)) - Fix recently added ratelimits to correctly honour the application service `rate_limited` flag. ([\#9711](https://github.com/matrix-org/synapse/issues/9711)) - Fix longstanding bug which caused `duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"` errors. ([\#9725](https://github.com/matrix-org/synapse/issues/9725)) - Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. ([\#9770](https://github.com/matrix-org/synapse/issues/9770)) - Fix duplicate logging of exceptions thrown during federation transaction processing. ([\#9780](https://github.com/matrix-org/synapse/issues/9780)) Updates to the Docker image --------------------------- - Move opencontainers labels to the final Docker image such that users can inspect them. ([\#9765](https://github.com/matrix-org/synapse/issues/9765)) Improved Documentation ---------------------- - Make the `allowed_local_3pids` regex example in the sample config stricter. ([\#9719](https://github.com/matrix-org/synapse/issues/9719)) Deprecations and Removals ------------------------- - Remove old admin API `GET /_synapse/admin/v1/users/<user_id>`. ([\#9401](https://github.com/matrix-org/synapse/issues/9401)) - Make `/_matrix/client/r0/register` expect a type of `m.login.application_service` when an Application Service registers a user, to align with [the relevant spec](https://spec.matrix.org/unstable/application-service-api/#server-admin-style-permissions). ([\#9548](https://github.com/matrix-org/synapse/issues/9548)) Internal Changes ---------------- - Replace deprecated `imp` module with successor `importlib`. Contributed by Cristina Muñoz. ([\#9718](https://github.com/matrix-org/synapse/issues/9718)) - Experiment with GitHub Actions for CI. ([\#9661](https://github.com/matrix-org/synapse/issues/9661)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9682](https://github.com/matrix-org/synapse/issues/9682)) - Update `scripts-dev/complement.sh` to use a local checkout of Complement, allow running a subset of tests and have it use Synapse's Complement test blacklist. ([\#9685](https://github.com/matrix-org/synapse/issues/9685)) - Improve Jaeger tracing for `to_device` messages. ([\#9686](https://github.com/matrix-org/synapse/issues/9686)) - Add release helper script for automating part of the Synapse release process. ([\#9713](https://github.com/matrix-org/synapse/issues/9713)) - Add type hints to expiring cache. ([\#9730](https://github.com/matrix-org/synapse/issues/9730)) - Convert various testcases to `HomeserverTestCase`. ([\#9736](https://github.com/matrix-org/synapse/issues/9736)) - Start linting mypy with `no_implicit_optional`. ([\#9742](https://github.com/matrix-org/synapse/issues/9742)) - Add missing type hints to federation handler and server. ([\#9743](https://github.com/matrix-org/synapse/issues/9743)) - Check that a `ConfigError` is raised, rather than simply `Exception`, when appropriate in homeserver config file generation tests. ([\#9753](https://github.com/matrix-org/synapse/issues/9753)) - Fix incompatibility with `tox` 2.5. ([\#9769](https://github.com/matrix-org/synapse/issues/9769)) - Enable Complement tests for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): Spaces Summary API. ([\#9771](https://github.com/matrix-org/synapse/issues/9771)) - Use mock from the standard library instead of a separate package. ([\#9772](https://github.com/matrix-org/synapse/issues/9772)) - Update Black configuration to target Python 3.6. ([\#9781](https://github.com/matrix-org/synapse/issues/9781)) - Add option to skip unit tests when building Debian packages. ([\#9793](https://github.com/matrix-org/synapse/issues/9793))
| * Bump black configuration to target py36 (#9781)Dan Callahan2021-04-131-1/+1
| | | | | | Signed-off-by: Dan Callahan <danc@element.io>
| * Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-085-11/+18
| | | | | | | | | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
| * MSC3083: Check for space membership during a local join of restricted rooms. ↵Patrick Cloke2021-04-081-1/+74
| | | | | | | | | | | | | | | | (#9735) When joining a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
| * Add a Synapse Module for configuring presence update routing (#9491)Andrew Morgan2021-04-061-43/+235
| | | | | | | | | | | | | | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.
| * Add type hints to expiring cache. (#9730)Patrick Cloke2021-04-063-18/+8
| |
| * Add type hints to the federation handler and server. (#9743)Patrick Cloke2021-04-061-80/+81
| |
| * Update mypy configuration: `no_implicit_optional = True` (#9742)Jonathan de Jong2021-04-052-3/+6
| |
| * Improve tracing for to device messages (#9686)Erik Johnston2021-04-013-17/+42
| |
| * Replace `room_invite_state_types` with `room_prejoin_state` (#9700)Richard van der Hoff2021-03-301-1/+1
| | | | | | | | | | | | | | `room_invite_state_types` was inconvenient as a configuration setting, because anyone that ever set it would not receive any new types that were added to the defaults. Here, we deprecate the old setting, and replace it with a couple of new settings under `room_prejoin_state`.
| * Make RateLimiter class check for ratelimit overrides (#9711)Erik Johnston2021-03-307-36/+50
| | | | | | | | | | | | | | This should fix a class of bug where we forget to check if e.g. the appservice shouldn't be ratelimited. We also check the `ratelimit_override` table to check if the user has ratelimiting disabled. That table is really only meant to override the event sender ratelimiting, so we don't use any values from it (as they might not make sense for different rate limits), but we do infer that if ratelimiting is disabled for the user we should disabled all ratelimits. Fixes #9663
* | Fix deactivationMathieu Velten2021-08-301-2/+3
| |
* | Rework room freeze and implement unfreezing the room (#100)Brendan Abolivier2021-07-221-4/+6
| | | | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | Unbind using bind_new_user_emails_to_sydent if specified (#98)Mathieu Velten2021-05-211-0/+2
| |
* | fix isort and mypy lintingAndrew Morgan2021-04-261-2/+2
| |
* | Merge commit '78e48f61b' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-0/+3
|\|
| * Make it possible to use dmypy (#9692)Erik Johnston2021-03-261-0/+3
| | | | | | | | | | | | | | | | | | Running `dmypy run` will do a `mypy` check while spinning up a daemon that makes rerunning `dmypy run` a lot faster. `dmypy` doesn't support `follow_imports = silent` and has `local_partial_types` enabled, so this PR enables those options and fixes the issues that were newly raised. Note that `local_partial_types` will be enabled by default in upcoming mypy releases.
* | Merge commit 'c73cc2c2a' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-18/+123
|\|
| * Spaces summary: call out to other servers (#9653)Richard van der Hoff2021-03-241-16/+119
| | | | | | | | | | When we hit an unknown room in the space tree, see if there are other servers that we might be able to poll to get the data. Fixes: #9447
| * Add a type hints for service notices to the HomeServer object. (#9675)Patrick Cloke2021-03-241-2/+4
| |
* | Merge commit '4ecba9bd5' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-2328-73/+174
|\|
| * Federation API for Space summary (#9652)Richard van der Hoff2021-03-231-45/+138
| | | | | | | | | | Builds on the work done in #9643 to add a federation API for space summaries. There's a bit of refactoring of the existing client-server code first, to avoid too much duplication.
| * Import HomeServer from the proper module. (#9665)Patrick Cloke2021-03-2326-26/+26
| |
| * Incorporate reviewBrendan Abolivier2021-03-191-1/+1
| |
| * Merge branch 'develop' into babolivier/msc3026Brendan Abolivier2021-03-192-1/+200
| |\
| * | Fix lintBrendan Abolivier2021-03-191-7/+6
| | |
| * | Move support for MSC3026 behind an experimental flagBrendan Abolivier2021-03-181-2/+10
| | |
| * | Implement MSC3026: busy presence stateBrendan Abolivier2021-03-181-1/+2
| | |
* | | Merge commit '0e3558473' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-1/+200
|\ \ \ | | |/ | |/|
| * | Initial spaces summary API (#9643)Richard van der Hoff2021-03-181-0/+199
| | | | | | | | | This is very bare-bones for now: federation will come soon, while pagination is descoped for now but will come later.
| * | Consistently check whether a password may be set for a user. (#9636)Dirk Klimpel2021-03-181-1/+1
| |/
* | Merge commit 'cc324d53f' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-4/+18
|\|
| * only save remote cross-signing keys if they're different from the current ↵Hubert Chathi2021-03-171-4/+18
| | | | | | | | | | ones (#9634) Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* | Merge commit 'ad721fc55' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-237-7/+45
|\|
| * Fix bad naming of storage function (#9637)Erik Johnston2021-03-172-3/+5
| | | | | | | | | | | | We had two functions named `get_forward_extremities_for_room` and `get_forward_extremeties_for_room` that took different paramters. We rename one of them to avoid confusion.
| * Add type hints to the room member handler. (#9631)Patrick Cloke2021-03-173-4/+14
| |
| * Add SSO attribute requirements for OIDC providers (#9609)Hubbe2021-03-161-0/+13
| | | | | | | | Allows limiting who can login using OIDC via the claims made from the IdP.
| * Return m.change_password.enabled=false if local database is disabled (#9588)Dirk Klimpel2021-03-161-0/+13
| | | | | | | | | | Instead of if the user does not have a password hash. This allows a SSO user to add a password to their account, but only if the local password database is configured.
* | Merge commit 'd315e9644' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-235-3/+14
|\|
| * Pass SSO IdP information to spam checker's registration function (#9626)Andrew Morgan2021-03-161-2/+2
| | | | | | | | | | | | | | Fixes https://github.com/matrix-org/synapse/issues/9572 When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time. This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
| * Handle an empty cookie as an invalid macaroon. (#9620)Patrick Cloke2021-03-161-1/+2
| | | | | | | | | | * Handle an empty cookie as an invalid macaroon. * Newsfragment
| * Add support for stable MSC2858 API (#9617)Richard van der Hoff2021-03-164-0/+10
| | | | | | | | | | The stable format uses different brand identifiers, so we need to support two identifiers for each IdP.
* | Merge commit '1c8a2541d' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-21/+131
|\|
| * Optimise missing prev_event handling (#9601)Richard van der Hoff2021-03-151-21/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: When we receive incoming federation traffic, and notice that we are missing prev_events from the incoming traffic, first we do a `/get_missing_events` request, and then if we still have missing prev_events, we set up new backwards-extremities. To do that, we need to make a `/state_ids` request to ask the remote server for the state at those prev_events, and then we may need to then ask the remote server for any events in that state which we don't already have, as well as the auth events for those missing state events, so that we can auth them. This PR attempts to optimise the processing of that state request. The `state_ids` API returns a list of the state events, as well as a list of all the auth events for *all* of those state events. The optimisation comes from the observation that we are currently loading all of those auth events into memory at the start of the operation, but we almost certainly aren't going to need *all* of the auth events. Rather, we can check that we have them, and leave the actual load into memory for later. (Ideally the federation API would tell us which auth events we're actually going to need, but it doesn't.) The effect of this is to reduce the number of events that I need to load for an event in Matrix HQ from about 60000 to about 22000, which means it can stay in my in-memory cache, whereas previously the sheer number of events meant that all 60K events had to be loaded from db for each request, due to the amount of cache churn. (NB I've already tripled the size of the cache from its default of 10K). Unfortunately I've ended up basically C&Ping `_get_state_for_room` and `_get_events_from_store_or_dest` into a new method, because `_get_state_for_room` is also called during backfill, which expects the auth events to be returned, so the same tricks don't work. That said, I don't really know why that codepath is completely different (ultimately we're doing the same thing in setting up a new backwards extremity) so I've left a TODO suggesting that we clean it up.
* | Merge commit '55da8df07' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-50/+21
|\|
| * Fix additional type hints from Twisted 21.2.0. (#9591)Patrick Cloke2021-03-121-4/+5
| |
| * Improve logging when processing incoming transactions (#9596)Richard van der Hoff2021-03-121-46/+16
| | | | | | Put the room id in the logcontext, to make it easier to understand what's going on.
* | Merge commit 'a7a379006' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-233-21/+36
|\|
| * Convert Requester to attrs (#9586)Richard van der Hoff2021-03-101-2/+3
| | | | | | | | | | | | ... because namedtuples suck Fix up a couple of other annotations to keep mypy happy.
| * Fix the auth provider on the logins metric (#9573)Richard van der Hoff2021-03-101-16/+30
| | | | | | | | | | We either need to pass the auth provider over the replication api, or make sure we report the auth provider on the worker that received the request. I've gone with the latter.
| * Use the chain cover index in get_auth_chain_ids. (#9576)Patrick Cloke2021-03-101-3/+3
| | | | | | | | This uses a simplified version of get_chain_cover_difference to calculate auth chain of events.
* | Merge commit '918f6ed82' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-5/+96
|\|
| * JWT OIDC secrets for Sign in with Apple (#9549)Richard van der Hoff2021-03-091-5/+96
| | | | | | | | | | Apple had to be special. They want a client secret which is generated from an EC key. Fixes #9220. Also fixes #9212 while I'm here.
* | Merge commit '7fdc6cefb' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-231-1/+1
|\|
| * Fix additional type hints. (#9543)Patrick Cloke2021-03-091-1/+1
| | | | | | Type hint fixes due to Twisted 21.2.0 adding type hints.
* | Merge commit '22db45bd4' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-239-69/+116
|\|
| * Add ResponseCache tests. (#9458)Jonathan de Jong2021-03-084-5/+5
| |
| * Create a SynapseReactor type which incorporates the necessary reactor ↵Patrick Cloke2021-03-081-1/+3
| | | | | | | | | | interfaces. (#9528) This helps fix some type hints when running with Twisted 21.2.0.
| * Prometheus metrics for logins and registrations (#9511)Richard van der Hoff2021-03-042-2/+34
| | | | | | Add prom metrics for number of users successfully registering and logging in, by SSO provider.
| * Record the SSO Auth Provider in the login token (#9510)Richard van der Hoff2021-03-043-61/+74
| | | | | | This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)
* | Merge commit 'a5daae2a5' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-16/+18
|\|
| * Prevent presence background jobs from running when presence is disabled (#9530)Aaron Raimist2021-03-031-14/+17
| | | | | | | | | | Prevent presence background jobs from running when presence is disabled Signed-off-by: Aaron Raimist <aaron@raim.ist>
| * Revert "Fix #8518 (sync requests being cached wrongly on timeout) (#9358)"Patrick Cloke2021-03-021-2/+1
| | | | | | | | | | | | | | This reverts commit f5c93fc9931e4029bbd8000f398b6f39d67a8c46. This is being backed out due to a regression (#9507) and additional review feedback being provided.
* | Merge commit 'f00c4e7af' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-3/+3
|\|
| * Use the proper Request in type hints. (#9515)Patrick Cloke2021-03-012-3/+3
| | | | | | | | This also pins the Twisted version in the mypy job for CI until proper type hints are fixed throughout Synapse.
* | Merge commit 'e12077a78' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-1/+9
|\|
| * Ensure pushers are deleted for deactivated accounts (#9285)Erik Johnston2021-02-251-0/+5
| |
| * Fix #8518 (sync requests being cached wrongly on timeout) (#9358)Jonathan de Jong2021-02-241-1/+2
| | | | | | | | | | | | | | This fixes #8518 by adding a conditional check on `SyncResult` in a function when `prev_stream_token == current_stream_token`, as a sanity check. In `CachedResponse.set.<remove>()`, the result is immediately popped from the cache if the conditional function returns "false". This prevents the caching of a timed-out `SyncResult` (that has `next_key` as the stream key that produced that `SyncResult`). The cache is prevented from returning a `SyncResult` that makes the client request the same stream key over and over again, effectively making it stuck in a loop of requesting and getting a response immediately for as long as the cache keeps those values. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* | Merge commit 'e22b71810' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-233-6/+26
|\|
| * Ratelimit cross-user key sharing requests. (#8957)Patrick Cloke2021-02-193-6/+26
| |
| * Fix style checking due to updated black.Patrick Cloke2021-02-191-2/+1
| |
* | Merge commit 'd9f1dccba' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-238-26/+102
|\|
| * Be smarter about which hosts to send presence to when processing room joins ↵Andrew Morgan2021-02-191-14/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#9402) This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually. --- When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed. It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw. This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence: * If it was a local user join, send that user's latest presence to all servers in the room * If it was a remote user join, send the presence for all local users in the room to that homeserver We deduplicate by inserting all of those pending updates into a dictionary of the form: ``` { server_name1: {presence_update1, ...}, server_name2: {presence_update1, presence_update2, ...} } ``` Only after building this dict do we then start sending out presence updates.
| * Add configs to make profile data more private (#9203)AndrewFerr2021-02-192-1/+17
| | | | | | | | | | | | | | Add off-by-default configuration settings to: - disable putting an invitee's profile info in invite events - disable profile lookup via federation Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
* | Merge commit '8ec221710' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-232-13/+9
|\|
| * Add back the guard against the user directory stream position not existing. ↵Patrick Cloke2021-02-181-0/+4
| | | | | | | | | | | | (#9428) As the comment says, this guard was there for when the initial user directory update has yet to happen.
| * Remove dead notify_for_states presence method (#9408)Andrew Morgan2021-02-171-11/+0
| |
| * Fix only handling the last presence state for each user (#9425)Andrew Morgan2021-02-171-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a small bug that I noticed while working on #8956. We have a for-loop which attempts to strip all presence changes for each user except for the final one, as we don't really care about older presence: https://github.com/matrix-org/synapse/blob/9e19c6aab4b5a99039f2ddc7d3120dd3b26c274b/synapse/handlers/presence.py#L368-L371 `new_states_dict` stores this stripped copy of latest presence state for each user, before it is... put into a new variable `new_state`, which is just overridden by the subsequent for loop. I believe this was instead meant to override `new_states`. Without doing so, it effectively meant: 1. The for loop had no effect. 2. We were still processing old presence state for users.
* | Merge commit 'd2f0ec12d' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-2329-336/+540
|\|
| * Support for form_post in OIDC responses (#9376)Richard van der Hoff2021-02-171-22/+52
| | | | | | Apple want to POST the OIDC auth response back to us rather than using query-params; add the necessary support to make that work.
| * Allow OIDC config to override discovered values (#9384)Richard van der Hoff2021-02-161-9/+18
| | | | | | Fixes #9347
| * Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-1629-212/+331
| | | | | | | | | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
| * Clean up caching/locking of OIDC metadata load (#9362)Richard van der Hoff2021-02-161-36/+53
| | | | | | | | Ensure that we lock correctly to prevent multiple concurrent metadata load requests, and generally clean up the way we construct the metadata cache.
| * Handle missing data in power levels events during room upgrade. (#9395)Patrick Cloke2021-02-161-5/+8
| |
| * Remove dead handled_events set in invite_join (#9394)Andrew Morgan2021-02-121-6/+0
| | | | | | | | | | This PR removes a set that was created and [initially used](https://github.com/matrix-org/synapse/commit/1d2a0040cff8d04cdc7d7d09d8f04a5d628fa9dd#diff-0bc92da3d703202f5b9be2d3f845e375f5b1a6bc6ba61705a8af9be1121f5e42R435-R436), but is no longer today. May help cut down a bit on the time it takes to accept invites.
| * Fix some typos.Patrick Cloke2021-02-121-1/+1
| |
| * Merge tag 'v1.27.0rc2' into developPatrick Cloke2021-02-111-0/+4
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.27.0rc2 (2021-02-11) ============================== Features -------- - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297)) Bugfixes -------- - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302)) - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313)) Improved Documentation ---------------------- - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))
| * | Combine the CAS & SAML implementations for required attributes. (#9326)Patrick Cloke2021-02-113-51/+86
| | |
* | | Merge commit 'e40d88cff' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-0/+4
|\ \ \ | | |/ | |/|
| * | Backout changes for automatically calculating the public baseurl. (#9313)Patrick Cloke2021-02-111-0/+4
| | | | | | | | | | | | This breaks some people's configurations (if their Client-Server API is not accessed via port 443).
* | | Merge commit '3f58fc848' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-5/+16
|\ \ \ | | |/ | |/|
| * | Merge pull request #9150 from Yoric/develop-contextDavid Teller2021-02-081-5/+16
| |\ \ | | | | | | | | New API /_synapse/admin/rooms/{roomId}/context/{eventId}
| | * | FIXUP: linterDavid Teller2021-01-281-1/+1
| | | |
| | * | FIXUP: Making get_event_context a bit more paranoidDavid Teller2021-01-281-2/+8
| | | |
| | * | FIXUP: Removing awaitableDavid Teller2021-01-281-3/+3
| | | |
| | * | FIXUP: Don't filter events at all for admin/v1/rooms/.../context/...David Teller2021-01-281-6/+4
| | | |
| | * | New API /_synapse/admin/rooms/{roomId}/context/{eventId}David Teller2021-01-281-2/+9
| | | | | | | | | | | | | | | | Signed-off-by: David Teller <davidt@element.io>
* | | | Merge commit '2ab6e67ab' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-222-4/+12
|\| | |
| * | | Merge remote-tracking branch 'origin/release-v1.27.0' into social_login_hotfixesRichard van der Hoff2021-02-032-4/+12
| |\ \ \ | | | |/ | | |/|
| | * | Honour ratelimit flag for application services for invite ratelimiting (#9302)Erik Johnston2021-02-032-4/+12
| | | |
* | | | Merge commit 'e288499c6' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-223-20/+55
|\| | |
| * | | Social login UI polish (#9301)Richard van der Hoff2021-02-031-2/+14
| | | |
| * | | Add debug for OIDC flow (#9307)Richard van der Hoff2021-02-031-15/+25
| | | |
| * | | Fix formatting for "bad session" error during sso registration flow (#9296)Richard van der Hoff2021-02-031-3/+16
| |/ /
* | | Merge commit '261093072' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-223-6/+6
|\| |
| * | Add an admin API to get the current room state (#9168)Travis Ralston2021-02-021-1/+1
| | | | | | | | | | | | | | | This could arguably replace the existing admin API for `/members`, however that is out of scope of this change. This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
| * | Put SAML callback URI under /_synapse/client. (#9289)Richard van der Hoff2021-02-021-1/+1
| | |
| * | Put OIDC callback URI under /_synapse/client. (#9288)Richard van der Hoff2021-02-011-4/+4
| | |
| * | Merge branch 'social_login' into developRichard van der Hoff2021-02-013-11/+96
| |\ \
* | \ \ Merge commit '351845452' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-223-11/+96
|\ \ \ \ | | |/ / | |/| |
| * | | Collect terms consent from the user during SSO registration (#9276)Richard van der Hoff2021-02-012-0/+46
| | | |
| * | | Improve styling and wording of SSO UIA templates (#9286)Richard van der Hoff2021-02-011-1/+3
| | | | | | | | | | | | fixes #9171
| * | | Make importing display name and email optional (#9277)Richard van der Hoff2021-02-012-10/+47
| | | |
* | | | Merge commit '18ab35284' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-225-5/+67
|\ \ \ \ | | |/ / | |/| |
| * | | Merge branch 'social_login' into developRichard van der Hoff2021-02-012-18/+99
| |\| |
| | * | Replace username picker with a template (#9275)Richard van der Hoff2021-02-011-1/+1
| | | | | | | | | | | | | | | | | | | | There's some prelimiary work here to pull out the construction of a jinja environment to a separate function. I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.
| | * | Improve styling and wording of SSO redirect confirm template (#9272)Richard van der Hoff2021-02-012-2/+32
| | | |
| * | | Prevent email UIA failures from raising a LoginError (#9265)Andrew Morgan2021-02-011-10/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Context, Fixes: https://github.com/matrix-org/synapse/issues/9263 In the past to fix an issue with old Riots re-requesting threepid validation tokens, we raised a `LoginError` during UIA instead of `InteractiveAuthIncompleteError`. This is now breaking the way Tchap logs in - which isn't standard, but also isn't disallowed by the spec. An easy fix is just to remove the 4 year old workaround.
| * | | Ratelimit invites by room and target user (#9258)Erik Johnston2021-01-293-2/+34
| | | |
* | | | Merge commit '9c715a5f1' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-15/+66
|\ \ \ \ | | |/ / | |/| |
| * | | Split out a separate endpoint to complete SSO registration (#9262)Richard van der Hoff2021-02-011-15/+66
| | | | | | | | | | | | There are going to be a couple of paths to get to the final step of SSO reg, and I want the URL in the browser to consistent. So, let's move the final step onto a separate path, which we redirect to.
* | | | Merge commit 'e19396d62' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-225-26/+68
|\ \ \ \ | | |/ / | |/| |
| * | | Merge branch 'social_login' into developRichard van der Hoff2021-01-284-26/+40
| |\| |
| | * | Add 'brand' field to MSC2858 response (#9242)Richard van der Hoff2021-01-274-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | We've decided to add a 'brand' field to help clients decide how to style the buttons. Also, fix up the allowed characters for idp_id, while I'm in the area.
| | * | Support for scraping email addresses from OIDC providers (#9245)Richard van der Hoff2021-01-271-24/+28
| | | |
| * | | Ratelimit 3PID /requestToken API (#9238)Erik Johnston2021-01-281-0/+28
| | | |
* | | | Merge commit 'a78016dad' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-223-133/+193
|\| | |
| * | | Add type hints to E2E handler. (#9232)Patrick Cloke2021-01-283-133/+193
| | |/ | |/| | | | This finishes adding type hints to the `synapse.handlers` module.
* | | Merge commit '2e537a028' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-5/+18
|\| |
| * | Merge branch 'social_login' into developRichard van der Hoff2021-01-271-5/+18
| |\|
| | * Implement MSC2858 support (#9183)Richard van der Hoff2021-01-271-5/+18
| | | | | | | | | Fixes #8928.
* | | Merge commit '1baab2035' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-2212-132/+222
|\| |
| * | Add type hints to various handlers. (#9223)Patrick Cloke2021-01-269-127/+174
| | | | | | | | | | | | With this change all handlers except the e2e_* ones have type hints enabled.
| * | Do not require the CAS service URL setting (use public_baseurl instead). (#9199)Patrick Cloke2021-01-261-5/+1
| | | | | | | | | | | | The current configuration is handled for backwards compatibility, but is considered deprecated.
| * | Precompute joined hosts and store in Redis (#9198)Erik Johnston2021-01-262-0/+47
| |/
* | Merge commit '0cd2938bc' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-225-1/+17
|\|
| * Support icons for Identity Providers (#9154)Richard van der Hoff2021-01-205-1/+17
| |
* | Merge commit 'fa50e4bf4' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-2/+0
|\|
| * Give `public_baseurl` a default value (#9159)Richard van der Hoff2021-01-201-2/+0
| |
* | Merge commit '73b03722f' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-224-10/+173
|\|
| * Allow moving account data and receipts streams off master (#9104)Erik Johnston2021-01-184-10/+173
| |
* | Merge commit '2b467d0b6' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-223-4/+4
|\|
| * Fix bugs in handling clientRedirectUrl, and improve OIDC tests (#9127, #9128)Richard van der Hoff2021-01-182-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Factor out a common TestHtmlParser Looks like I'm doing this in a few different places. * Improve OIDC login test Complete the OIDC login flow, rather than giving up halfway through. * Ensure that OIDC login works with multiple OIDC providers * Fix bugs in handling clientRedirectUrl - don't drop duplicate query-params, or params with no value - allow utf-8 in query-params
| * Ensure the user ID is serialized in the payload instead of used as an ↵Patrick Cloke2021-01-181-1/+1
| | | | | | | | instance name. (#9130)
* | Merge commit '9de6b9411' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-7/+20
|\|
| * Land support for multiple OIDC providers (#9110)Richard van der Hoff2021-01-151-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the final step for supporting multiple OIDC providers concurrently. First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before: oidc_config: enabled: true issuer: "https://oidc_provider" # etc After: oidc_providers: - idp_id: prov1 issuer: "https://oidc_provider" - idp_id: prov2 issuer: "https://another_oidc_provider" The old format is still grandfathered in. With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.
* | Merge commit 'd34c6e127' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-6/+16
|\|
| * Store an IdP ID in the OIDC session (#9109)Richard van der Hoff2021-01-151-6/+16
| | | | | | | | | | Again in preparation for handling more than one OIDC provider, add a new caveat to the macaroon used as an OIDC session cookie, which remembers which OIDC provider we are talking to. In future, when we get a callback, we'll need it to make sure we talk to the right IdP. As part of this, I'm adding an idp_id and idp_name field to the OIDC configuration object. They aren't yet documented, and we'll just use the old values by default.
| * Merge pull request #9091 from matrix-org/rav/error_on_bad_ssoRichard van der Hoff2021-01-152-31/+39
| |\ | | | | | | Give the user a better error when they present bad SSO creds
* | | Merge commit '631dd06f2' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-98/+148
|\| |
| * | Split OidcProvider out of OidcHandler (#9107)Richard van der Hoff2021-01-141-98/+148
| | | | | | | | | | | | | | | | | | | | | The idea here is that we will have an instance of OidcProvider for each configured IdP, with OidcHandler just doing the marshalling of them. For now it's still hardcoded with a single provider.
* | | Merge commit '420031906' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-222-31/+39
|\ \ \ | | |/ | |/|
| * | Move `complete_sso_ui_auth` into SSOHandlerRichard van der Hoff2021-01-132-28/+13
| | | | | | | | | | | | | | | since we're hacking on this code anyway, may as well move it out of the cluttered AuthHandler.
| * | Give the user a better error when they present bad SSO credsRichard van der Hoff2021-01-131-5/+28
| |/ | | | | | | | | | | | | | | | | If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO IdP, try to give them a better error. Previously, the UIA would claim to be successful, but then the operation in question would simply fail with "auth fail". Instead, serve up an error page which explains the failure.
* | Merge commit 'dc3c83a93' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-221-17/+20
|\|
| * Extract OIDCProviderConfig objectRichard van der Hoff2021-01-131-17/+20
| | | | | | | | | | Collect all the config options which related to an OIDC provider into a single object.
* | Fix mypy check failAndrew Morgan2021-04-221-3/+1
| |
* | Merge commit '98a64b7f7' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-226-175/+306
|\|
| * Preparatory refactors of OidcHandler (#9067)Richard van der Hoff2021-01-131-141/+163
| | | | | | | | | | | | | | | | Some light refactoring of OidcHandler, in preparation for bigger things: * remove inheritance from deprecated BaseHandler * add an object to hold the things that go into a session cookie * factor out a separate class for manipulating said cookies
| * Remove user's avatar URL and displayname when deactivated. (#8932)Dirk Klimpel2021-01-122-3/+23
| | | | | | This only applies if the user's data is to be erased.
| * UI Auth via SSO: redirect the user to an appropriate SSO. (#9081)Richard van der Hoff2021-01-123-18/+110
| | | | | | | | | | | | | | If we have integrations with multiple identity providers, when the user does a UI Auth, we need to redirect them to the right one. There are a few steps to this. First of all we actually need to store the userid of the user we are trying to validate in the UIA session, since the /auth/sso/fallback/web request is unauthenticated. Then, once we get the /auth/sso/fallback/web request, we can fish the user id out of the session, and use it to look up the external id mappings, and hence pick an SSO provider for them.
| * Kill off `HomeServer.get_ip_from_request()` (#9080)Richard van der Hoff2021-01-121-7/+2
| | | | | | Homeserver.get_ip_from_request() used to be a bit more complicated, but now it is totally redundant. Let's get rid of it.
| * Remove SynapseRequest.get_user_agent (#9069)Richard van der Hoff2021-01-122-5/+6
| | | | | | | | | | | | | | | | | | | | | | SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests", which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the LoggingContext and write the right entries to the request log). Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a SynapseRequest when there is nothing synapse-specific about the Request at all, and any old twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult. In short: move get_user_agent out to a utility function.
* | Merge commit 'a03d71dc9' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-211-12/+36
|\|
| * Allow running sendToDevice on workers (#9044)Erik Johnston2021-01-071-8/+23
| |
| * Ensure that remote users' device list resyncing always happens on master (#9043)Erik Johnston2021-01-071-4/+13
| | | | | | Currently `DeviceMessageHandler` only ever exists on master, but that is about to change.
* | Merge commit '0312266ee' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-216-27/+163
|\|
| * Handle a display name / avatar URL not included in a federation request. (#9023)Patrick Cloke2021-01-061-2/+2
| | | | | | | | These may be omitted if not set, but Synapse assumed they would be in the response.
| * Implement MSC2176: Updated redaction rules (#8984)Patrick Cloke2021-01-051-1/+1
| | | | | | | | An experimental room version ("org.matrix.msc2176") contains the new redaction rules for testing.
| * Add initial support for a "pick your IdP" page (#9017)Richard van der Hoff2021-01-054-3/+24
| | | | | | | | | | During login, if there are multiple IdPs enabled, offer the user a choice of IdPs.
| * Combine the SSO Redirect Servlets (#9015)Richard van der Hoff2021-01-044-23/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Implement CasHandler.handle_redirect_request ... to make it match OidcHandler and SamlHandler * Clean up interface for OidcHandler.handle_redirect_request Make it accept `client_redirect_url=None`. * Clean up interface for `SamlHandler.handle_redirect_request` ... bring it into line with CAS and OIDC by making it take a Request parameter, move the magic for `client_redirect_url` for UIA into the handler, and fix the return type to be a `str` rather than a `bytes`. * Define a common protocol for SSO auth provider impls * Give SsoIdentityProvider an ID and register them * Combine the SSO Redirect servlets Now that the SsoHandler knows about the identity providers, we can combine the various *RedirectServlets into a single implementation which delegates to the right IdP. * changelog
* | Merge commit '2fe0fb21f' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-212-32/+61
|\|
| * Add type hints to the receipts and user directory handlers. (#8976)Patrick Cloke2021-01-042-32/+61
| |
* | Merge commit '1c9a85056' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-215-44/+80
|\|
| * Use the SSO handler helpers for CAS registration/login. (#8856)Patrick Cloke2021-01-032-39/+77
| |
| * Check if group IDs are valid before using them. (#8977)Patrick Cloke2020-12-301-1/+1
| |
| * Add additional type hints to the storage module. (#8980)Patrick Cloke2020-12-302-4/+2
| |
* | Merge commit '9999eb2d0' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-212-68/+89
|\|
| * Add type hints to admin and room list handlers. (#8973)Patrick Cloke2020-12-292-68/+89
| |
* | Merge commit '4218473f9' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-213-70/+163
|\|
| * Refactor the CAS handler in prep for using the abstracted SSO code. (#8958)Patrick Cloke2020-12-182-66/+158
| | | | | | | | | | | | This makes the CAS handler look more like the SAML/OIDC handlers: * Render errors to users instead of throwing JSON errors. * Internal reorganization.
| * Send the location of the web client to the IS when inviting via 3PIDs. (#8930)Patrick Cloke2020-12-181-0/+5
| | | | | | | | Adds a new setting `email.invite_client_location` which, if defined, is passed to the identity server during invites.
* | Merge commit '28877fade' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-213-48/+297
|\|
| * Implement a username picker for synapse (#8942)Richard van der Hoff2020-12-182-40/+273
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow `UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser back to the client, we redirect to a username-picker resource, which allows the user to enter a username. We *then* complete the SSO flow (including doing the client permission checks). The static resources for the username picker itself (in https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker) are essentially lifted wholesale from https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. As the comment says, we might want to think about making them customisable, but that can be a follow-up. Fixes #8876.
| * Allow re-using a UI auth validation for a period of time (#8970)Patrick Cloke2020-12-181-8/+24
| |
* | Merge commit 'a7a913918' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-203-3/+14
|\|
| * Merge remote-tracking branch 'origin/erikj/as_mau_block' into developErik Johnston2020-12-182-2/+13
| |\
| | * Correctly handle AS registerations and add testErik Johnston2020-12-172-2/+13
| | |
| * | Try and drop stale extremities. (#8929)Erik Johnston2020-12-181-1/+1
| | | | | | | | | | | | If we see stale extremities while persisting events, and notice that they don't change the result of state resolution, we drop them.
* | | Merge commit 'c9c1c9d82' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-201-2/+6
|\| |
| * | Fix a bug that deactivated users appear in the directory (#8933)Dirk Klimpel2020-12-171-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes a bug that deactivated users appear in the directory when their profile information was updated. To change profile information of deactivated users is neccesary for example you will remove displayname or avatar. But they should not appear in directory. They are deactivated. Co-authored-by: Erik Johnston <erikj@jki.re>
* | | Merge commit 'ff5c4da12' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-203-82/+75
|\| |
| * | Push login completion down into SsoHandler (#8941)Richard van der Hoff2020-12-163-82/+75
| | | | | | | | | This is another part of my work towards fixing #8876. It moves some of the logic currently in the SAML and OIDC handlers - in particular the call to `AuthHandler.complete_sso_login` down into the `SsoHandler`.
* | | Merge commit 'bd30cfe86' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-163-8/+10
|\| |
| * | Do not assume that the contents dictionary includes history_visibility. (#8945)Patrick Cloke2020-12-163-8/+10
| | |
* | | Merge commit '01333681b' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-162-14/+35
|\| |
| * | Preparatory refactoring of the SamlHandlerTestCase (#8938)Richard van der Hoff2020-12-151-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * move simple_async_mock to test_utils ... so that it can be re-used * Remove references to `SamlHandler._map_saml_response_to_user` from tests This method is going away, so we can no longer use it as a test point. Instead, factor out a higher-level method which takes a SAML object, and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`. * changelog
| * | Fix startup failure with localdb_enabled: False (#8937)Richard van der Hoff2020-12-141-14/+12
| | |
* | | Merge commit 'f14428b25' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-169-37/+43
|\| |
| * | Allow spam-checker modules to be provide async methods. (#8890)David Teller2020-12-119-22/+21
| | | | | | | | | | | | Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.
| * | Honour AS ratelimit settings for /login requests (#8920)Erik Johnston2020-12-111-3/+4
| | | | | | | | | | | | Fixes #8846.
| * | Don't ratelimit autojoining of rooms (#8921)Erik Johnston2020-12-112-11/+17
| | | | | | | | | Fixes #8866
* | | Merge commit 'c64002e1c' into anoa/dinsic_release_1_31_0Andrew Morgan2021-04-162-28/+50
|\| |
| * | Refactor `SsoHandler.get_mxid_from_sso` (#8900)Richard van der Hoff2020-12-102-28/+50
| | | | | | | | | | | | | | | | | | | | | * Factor out _call_attribute_mapper and _register_mapped_user This is mostly an attempt to simplify `get_mxid_from_sso`. * Move mapping_lock down into SsoHandler.
* | | Merge commit 'cf7d3c90d' into dinsicAndrew Morgan2021-04-166-56/+186
|\| |
| * | Simplify the flow for SSO UIA (#8881)Richard van der Hoff2020-12-085-40/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | * SsoHandler: remove inheritance from BaseHandler * Simplify the flow for SSO UIA We don't need to do all the magic for mapping users when we are doing UIA, so let's factor that out.
| * | Merge tag 'v1.24.0rc2' into developPatrick Cloke2020-12-042-6/+23
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.24.0rc2 (2020-12-04) ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
| * | Merge pull request #8858 from matrix-org/rav/sso_uiaRichard van der Hoff2020-12-021-15/+43
| |\ \ | | | | | | | | UIA: offer only available auth flows
| | * | UIA: offer only available auth flowsRichard van der Hoff2020-12-021-15/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | During user-interactive auth, do not offer password auth to users with no password, nor SSO auth to users with no SSO. Fixes #7559.
| * | | Apply an IP range blacklist to push and key revocation requests. (#8821)Patrick Cloke2020-12-022-4/+4
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).
* | | Port "Add support for no_proxy and case insensitive env variables" from ↵Andrew Morgan2021-03-222-5/+5
| | | | | | | | | | | | | | | | | | | | | mainline to dinsic (#93) This PR is simply porting https://github.com/matrix-org/synapse/pull/9372 to dinsic. I also had to bring in https://github.com/matrix-org/synapse/pull/8821 and https://github.com/matrix-org/synapse/pull/9084 for this code to work properly - a sign that we should merge mainline into dinsic again soon.
* | | Send a ver query parameter for make_knock (#83)Andrew Morgan2021-02-171-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | This informs the remote server of the room versions we support. If the room we're trying to knock on has a version that is not one of our supported room versions, the remote server will return an unsupported room version error. Noticed in https://github.com/matrix-org/matrix-doc/pull/2403#discussion_r577042144 Ported from https://github.com/matrix-org/synapse/pull/6739
* | | Show knock rooms in the public room directory and inform clients they can be ↵Andrew Morgan2021-02-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | knocked on (#82) This PR implements the ["Changes regarding the Public Rooms Directory"](https://github.com/Sorunome/matrix-doc/blob/soru/knock/proposals/2403-knock.md#changes-regarding-the-public-rooms-directory) section of knocking MSC2403. Specifically, it: * Allows rooms with `join_rule` "knock" to be returned by the query behind the public rooms directory * Adds the field `join_rule` to each room entry returned by a public rooms directory query, so clients can know whether to attempt a join or knock on a room This PR is a clone of [the mainline PR](https://github.com/matrix-org/synapse/pull/9359). Complement tests for this change: https://github.com/matrix-org/complement/pull/72.
* | | Add knocking support (#81)Andrew Morgan2021-02-096-54/+461
| | | | | | | | | | | | | | | | | | | | | | | | | | | Implement knocking as defined by https://github.com/matrix-org/matrix-doc/pull/2403 This is the base knocking stuff, taken from https://github.com/matrix-org/synapse/pull/6739 and does not include any public room directory changes. While knocking hasn't merged yet on mainline due to waiting on getting Complement into Synapse's CI, the code has been well-tested.
* | | Remove riot email registration backwards compatibility hackAndrew Morgan2021-01-291-10/+0
| | | | | | | | | | | | | | | | | | | | | | | | This was causing a LoginError to be propagated up to the registration servlet, which was expecting a InteractiveAuthIncompleteError in order to store a password_hash from the client for the session. DINUM relies on this happening. More info here: https://github.com/matrix-org/synapse/issues/9263
* | | Merge commit '693dab487' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-312-6/+23
|\ \ \ | | |/ | |/|
| * | Fix a regression that mapping providers should be able to redirect users. ↵Patrick Cloke2020-12-042-6/+23
| |/ | | | | | | | | (#8878) This was broken in #8801.
* | Merge commit '53b12688d' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-313-48/+53
|\|
| * Fix a regression when grandfathering SAML users. (#8855)Patrick Cloke2020-12-023-47/+52
| | | | | | | | | | | | This was broken in #8801 when abstracting code shared with OIDC. After this change both SAML and OIDC have a concept of grandfathering users, but with different implementations.
| * Add basic SAML tests for mapping users. (#8800)Patrick Cloke2020-12-021-1/+1
| |
* | Merge commit 'd3ed93504' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-75/+313
|\|
| * Create a `PasswordProvider` wrapper object (#8849)Richard van der Hoff2020-12-021-55/+148
| | | | | | | | The idea here is to abstract out all the conditional code which tests which methods a given password provider has, to provide a consistent interface.
| * Support "identifier" dicts in UIA (#8848)Richard van der Hoff2020-12-011-24/+161
| | | | | | | | | | | | | | | | | | | | The spec requires synapse to support `identifier` dicts for `m.login.password` user-interactive auth, which it did not (instead, it required an undocumented `user` parameter.) To fix this properly, we need to pull the code that interprets `identifier` into `AuthHandler.validate_login` so that it can be called from the UIA code. Fixes #5665.
| * Don't offer password login when it is disabled (#8835)Richard van der Hoff2020-12-011-1/+9
| | | | | | Fix a minor bug where we would offer "m.login.password" login if a custom auth provider supported it, even if password login was disabled.
* | Merge commit '09ac0569f' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-1/+2
|\|
| * Allow per-room profile to be used for server notice user (#8799)Mathieu Velten2020-11-301-1/+9
| | | | | | | | This applies even if the feature is disabled at the server level with `allow_per_room_profiles`. The server notice not being a real user it doesn't have an user profile.
| * Simplify the way the `HomeServer` object caches its internal attributes. ↵Jonathan de Jong2020-11-301-1/+2
| | | | | | | | | | (#8565) Changes `@cache_in_self` to use underscore-prefixed attributes.
* | Merge commit 'a090b8620' into anoa/dinsic_release_1_23_1Andrew Morgan2020-12-311-6/+11
|\|
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-09-04 06:28:51 +0000