| Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
|
|
|
|
* Allow providing credentials to HTTPS_PROXY (#9657)
Addresses https://github.com/matrix-org/synapse-dinsic/issues/70
This PR causes `ProxyAgent` to attempt to extract credentials from an `HTTPS_PROXY` env var. If credentials are found, a `Proxy-Authorization` header ([details](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization)) is sent to the proxy server to authenticate against it. The headers are *not* passed to the remote server.
Also added some type hints.
* lint
|
|
|
|
mainline to dinsic (#93)
This PR is simply porting https://github.com/matrix-org/synapse/pull/9372 to dinsic.
I also had to bring in https://github.com/matrix-org/synapse/pull/8821 and https://github.com/matrix-org/synapse/pull/9084 for this code to work properly - a sign that we should merge mainline into dinsic again soon.
|
|
|
|
|
|
|
|
|
|
Small change to the event auth rules that allows transitioning from knock->knock (in case you want to update your knock reason, or change profile information etc).
Coincides with the same commit in [the mainline PR](https://github.com/matrix-org/synapse/pull/6739) and [this update on MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403/commits/49a72862a93d814219968cff814ff63926181645).
|
|
|
|
results (#84)
* Add a config option to prioritise local users in user directory search results (#9383)
This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default.
Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical.
This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.
* Don't mix simple and english psql query types
|
|
This informs the remote server of the room versions we support. If the room we're trying to
knock on has a version that is not one of our supported room versions, the remote server
will return an unsupported room version error.
Noticed in https://github.com/matrix-org/matrix-doc/pull/2403#discussion_r577042144
Ported from https://github.com/matrix-org/synapse/pull/6739
|
|
|
|
This is needed to build the cryptography library, since it does not
provide wheels for ARMv7.
Fixes #9403
Signed-off-by: Dan Callahan <danc@element.io>
|
|
|
|
|
|
|
|
This breaks some people's configurations (if their Client-Server API
is not accessed via port 443).
|
|
knocked on (#82)
This PR implements the ["Changes regarding the Public Rooms Directory"](https://github.com/Sorunome/matrix-doc/blob/soru/knock/proposals/2403-knock.md#changes-regarding-the-public-rooms-directory) section of knocking MSC2403.
Specifically, it:
* Allows rooms with `join_rule` "knock" to be returned by the query behind the public rooms directory
* Adds the field `join_rule` to each room entry returned by a public rooms directory query, so clients can know whether to attempt a join or knock on a room
This PR is a clone of [the mainline PR](https://github.com/matrix-org/synapse/pull/9359). Complement tests for this change: https://github.com/matrix-org/complement/pull/72.
|
|
Implement knocking as defined by https://github.com/matrix-org/matrix-doc/pull/2403
This is the base knocking stuff, taken from https://github.com/matrix-org/synapse/pull/6739
and does not include any public room directory changes.
While knocking hasn't merged yet on mainline due to waiting on getting Complement
into Synapse's CI, the code has been well-tested.
|
|
|
|
|
|
|
|
* Fix the Python 3.5 old-deps build. (#9146)
setuptools 51.0.0 dropped support for Python 3.5.
* Fix Python 3.5 old deps build by using a compatible pip version. (#9217)
Co-authored-by: Dan Callahan <danc@element.io>
pip 21.0 stopped supporting Python 3.5.
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
|
* Adds type hints to the groups servlet and stringutils code.
* Assert the maximum length of some input values for spec compliance.
|
|
* Handle the case of lxml not finding a document tree.
* Parse the document encoding from the XML tag.
|
|
|
|
Signed-off-by: Joseph Arnault <computerdude90042@outlook.com>
Signed-off-by: Dan Callahan <danc@element.io>
Co-authored-by: compu42 <56663749+compu42@users.noreply.github.com>
|
|
|
|
|
|
Fixes some exceptions if the room state isn't quite as expected.
If the expected state events aren't found, try to find them in the
historical room state. If they still aren't found, fallback to a reasonable,
although ugly, value.
|
|
This fixes the Jinja2 templates for the mapping provider.
|
|
* Adds proper dependencies.
* Minor fixes in database layer.
|
|
|
|
|
|
|
|
|
|
* fix validation and don't use built-in validation UI
Co-authored-by: Bruno Windels <brunow@element.io>
|
|
|
|
config: Add detail to auto_join_rooms comment
Signed-off-by: Gary Dykstra <gary@dykstranet.com>
|
|
|
|
Also add a few more IP ranges to the default blacklist.
|
|
|
|
|
|
|
|
|
|
This could arguably replace the existing admin API for `/members`, however that is out of scope of this change.
This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
fixes #9171
|
|
* Fixes a case where no summary text was returned.
* The use of messages_from_person vs. messages_from_person_and_others
was tweaked to depend on whether there was 1 sender or multiple senders,
not based on if there was 1 room or multiple rooms.
|
|
|
|
|
|
Signed-off-by: Jan Christian Grünhage <jan.christian@gruenhage.xyz>
|
|
Context, Fixes: https://github.com/matrix-org/synapse/issues/9263
In the past to fix an issue with old Riots re-requesting threepid validation tokens, we raised a `LoginError` during UIA instead of `InteractiveAuthIncompleteError`. This is now breaking the way Tchap logs in - which isn't standard, but also isn't disallowed by the spec.
An easy fix is just to remove the 4 year old workaround.
|
|
There's some prelimiary work here to pull out the construction of a jinja environment to a separate function.
I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.
|
|
|
|
Fixes #8966.
* Factor out build_synapse_client_resource_tree
Start a function which will mount resources common to all workers.
* Move sso init into build_synapse_client_resource_tree
... so that we don't have to do it for each worker
* Fix SSO-login-via-a-worker
Expose the SSO login endpoints on workers, like the documentation says.
* Update workers config for new endpoints
Add documentation for endpoints recently added (#8942, #9017, #9262)
* remove submit_token from workers endpoints list
this *doesn't* work on workers (yet).
* changelog
* Add a comment about the odd path for SAML2Resource
|
|
There are going to be a couple of paths to get to the final step of SSO reg, and I want the URL in the browser to consistent. So, let's move the final step onto a separate path, which we redirect to.
|
|
* synapse.app.base: only call gc.freeze() on CPython
gc.freeze() is an implementation detail of CPython garbage collector,
and notably does not exist on PyPy.
Rather than playing whack-a-mole and skipping the call when under PyPy,
simply restrict it to CPython because the whole gc module is
implementation-defined.
Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
|
|
|
|
Adds note about updating dh-virtualenv once we drop support for Xenial.
We can't update now, because it needs debhelper 12, while Xenial only
backports 10.
Signed-off-by: Dan Callahan <danc@element.io>
|
|
This was causing a LoginError to be propagated up to the registration servlet, which was
expecting a InteractiveAuthIncompleteError in order to store a password_hash from the
client for the session. DINUM relies on this happening.
More info here: https://github.com/matrix-org/synapse/issues/9263
|
|
|
|
|
|
Reading from the config object is *slow*.
|
|
|
|
This finishes adding type hints to the `synapse.handlers` module.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Signed-off-by: David Teller <davidt@element.io>
|
|
|
|
We've decided to add a 'brand' field to help clients decide how to style the
buttons.
Also, fix up the allowed characters for idp_id, while I'm in the area.
|
|
|
|
|
|
behalf (#9235)
|
|
|
|
* Enables autoescape by default for HTML files.
* Adds a new read_template method for reading a single template.
* Some logic clean-up.
|
|
|
|
|
|
|
|
Fixes #8928.
|
|
the homeserver config (#9229)
If a Synapse module's config block were empty in YAML, thus being translated to a `Nonetype` in Python, then some modules could fail as that None ends up getting passed to their `parse_config` method. Modules are expected to accept a `dict` instead.
This PR ensures that if the user does end up specifying an empty config block (such as what [the default oidc config in the sample config](https://github.com/matrix-org/synapse/blob/5310808d3bebd17275355ecd474bc013e8c7462d/docs/sample_config.yaml#L1816-L1845) states) then `None` is not passed to the module. An empty dict is passed instead.
This code assumes that no existing modules are relying on receiving a `None` config block, but I'd really hope that they aren't.
|
|
With this change all handlers except the e2e_* ones have
type hints enabled.
|
|
The current configuration is handled for backwards compatibility,
but is considered deprecated.
|
|
|
|
Treat unknown encodings (according to lxml) as UTF-8
when generating a preview for HTML documents. This
isn't fully accurate, but will hopefully give a reasonable
title and summary.
|
|
This new version no longer has the problem of adding/removing a blank line in `.pyi` files, which black disagrees with. This would cause `isort` to slightly modify `.pyi` files, before `black` would subsequently modify back directly afterwards.
Relevant `isort` issue: https://github.com/pycqa/isort/issues/1284
|
|
This is done by creating a custom `RedisFactory` subclass that
periodically pings all connections in its pool.
We also ensure that the `replyTimeout` param is non-null, so that we
timeout waiting for the reply to those pings (and thus triggering a
reconnect).
|
|
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
|
Co-authored-by: Erik Johnston <erik@matrix.org>
|
|
|
|
This expands the current shadow-banning feature to be usable via
the admin API and adds documentation for it.
A shadow-banned users receives successful responses to their
client-server API requests, but the events are not propagated into rooms.
Shadow-banning a user should be used as a tool of last resort and may lead
to confusing or broken behaviour for the client.
|
|
Co-authored-by: Dan Callahan <danc@element.io>
pip 21.0 stopped supporting Python 3.5.
|
|
Co-authored-by: Dan Callahan <danc@element.io>
pip 21.0 stopped supporting Python 3.5.
|
|
|
|
|
|
Treat the content as untrusted and do not assume it is of
the proper form.
|
|
Also add a warning on the admin API documentation.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
|
|
|
As we use `execute_values` with the `fetch` parameter.
|
|
If no thumbnail of the requested type exists, return a 404 instead
of erroring. This doesn't quite match the spec (which does not define
what happens if no thumbnail can be found), but is consistent with
what Synapse already does.
|
|
(#9191)
The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.
|
|
|
|
We have seen a failure mode here where if there are many in flight
unfinished IDs then marking an ID as finished takes a lot of CPU (as
calling deque.remove iterates over the list)
|
|
|
|
Introduced in #9104
This wasn't picked up by the tests as this is all fine the first time you run Synapse (after upgrading), but then when you restart the wrong value is pulled from `stream_positions`.
|
|
* Use execute_batch in more places
* Newsfile
|
|
|
|
|
|
|
|
... to avoid clashes with other SSO mechanisms
|
|
`execute_batch` does fewer round trips in postgres than `executemany`, but does not give a correct `txn.rowcount` result after.
|
|
This avoids a warning when uploading packages to PyPI via twine.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Type hints for `FederationClient`.
* Using `async` functions instead of returning `Awaitable` instances.
|
|
Signed-off-by: rht <rhtbot@protonmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If a remote server name is provided, ensure it is something reasonable
before making remote connections to it.
|
|
homeserver.yaml template (#9157)
|
|
Make sure we report the correct config path for errors in the OIDC configs.
|
|
provided (#77)
|
|
|
|
|
|
|
|
|
|
...instead of just creating the exception object and doing nothing with it.
|
|
* Factor out a common TestHtmlParser
Looks like I'm doing this in a few different places.
* Improve OIDC login test
Complete the OIDC login flow, rather than giving up halfway through.
* Ensure that OIDC login works with multiple OIDC providers
* Fix bugs in handling clientRedirectUrl
- don't drop duplicate query-params, or params with no value
- allow utf-8 in query-params
|
|
setuptools 51.0.0 dropped support for Python 3.5.
|
|
instance name. (#9130)
|
|
|
|
0dd2649c1 (#9112) changed the signature of `auth_via_oidc`. Meanwhile,
26d10331e (#9091) introduced a new test which relied on the old signature of
`auth_via_oidc`. The two branches were never tested together until they landed
in develop.
|
|
We do this by allowing a single iteration to process multiple rooms at a
time, as there are often a lot of really tiny rooms, which can massively
slow things down.
|
|
I don't think there's any need to use canonicaljson here.
Fixes: #4475.
|
|
This is the final step for supporting multiple OIDC providers concurrently.
First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before:
oidc_config:
enabled: true
issuer: "https://oidc_provider"
# etc
After:
oidc_providers:
- idp_id: prov1
issuer: "https://oidc_provider"
- idp_id: prov2
issuer: "https://another_oidc_provider"
The old format is still grandfathered in.
With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.
|
|
Protecting media stops it from being quarantined when
e.g. all media in a room is quarantined. This is useful
for sticker packs and other media that is uploaded by
server administrators, but used by many people.
|
|
Previously this code generated unreferenced `Deferred` instances
which caused "Unhandled Deferreds" errors to appear in error
situations.
|
|
`distutils` is pretty much deprecated these days, and replaced with
`setuptools`. It's also annoying because it's you can't `pip install` it, and
it's hard to figure out which debian package we should depend on to make sure
it's there.
Since we only use it for a tiny function anyway, let's just vendor said
function into our codebase.
|
|
|
|
* make the OIDC bits of the test work at a higher level - via the REST api instead of poking the OIDCHandler directly.
* Move it to test_login.py, where I think it fits better.
|
|
Again in preparation for handling more than one OIDC provider, add a new caveat to the macaroon used as an OIDC session cookie, which remembers which OIDC provider we are talking to. In future, when we get a callback, we'll need it to make sure we talk to the right IdP.
As part of this, I'm adding an idp_id and idp_name field to the OIDC configuration object. They aren't yet documented, and we'll just use the old values by default.
|
|
|
|
We passed in a graph to `sorted_topologically` which didn't have an
entry for each node (as we dropped nodes with no edges).
|
|
|
|
(#9115)
|
|
t was doing a sequential scan on `destination_rooms`, which took
minutes.
|
|
|
|
The idea here is that we will have an instance of OidcProvider for each
configured IdP, with OidcHandler just doing the marshalling of them.
For now it's still hardcoded with a single provider.
|
|
A reactor was being passed instead of a whitelist for the BlacklistingAgentWrapper
used by the WellyKnownResolver. This coulld cause exceptions when attempting to
connect to IP addresses that are blacklisted, but in reality this did not have any
observable affect since this code is not used for IP literals.
|
|
since we're hacking on this code anyway, may as well move it out of the
cluttered AuthHandler.
|
|
|
|
If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO
IdP, try to give them a better error.
Previously, the UIA would claim to be successful, but then the operation in
question would simply fail with "auth fail". Instead, serve up an error page
which explains the failure.
|
|
* Add complete test for UI-Auth-via-SSO.
* review comments
|
|
Removes a bare `except Exception` clause and replaces it with
catching a specific exception around the portion that might throw.
|
|
| <
