| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
We might have events in the staging area if the service was restarted while there were unhandled events in the staging area.
Fixes #10295
|
| | |
|
| | |
|
| |
|
| |
Fix link in modules doc to point at instructions on registering a callback instead of ones on registering a web resource.
|
| |
|
| |
Instead of mixing them with user authentication methods.
|
| |
|
|
|
| |
this was a typo introduced in #10282. We don't want to end up doing the
`replace_stream_ordering_column` update after anything that comes up in
migration 60/03.
|
| |
|
| |
Now that we've fixed it.
|
| |
|
| |
Related to #10251
|
| | |
|
| |
|
|
|
| |
The presence router docs include some sample homeserver config. At some point we changed the name of the [config option](https://github.com/matrix-org/synapse/blob/859dc05b3692a3672c1a0db8deaaa9274b6aa6f5/docs/sample_config.yaml#L104-L113), but forgot to update the docs.
I've also added `presence.enabled: true` to the example, as that's the new way to enable presence (the `presence_enabled` option has been deprecated).
|
| | |
|
| |
|
|
| |
We need to rebuild *all* of the indexes that use the current `stream_ordering`
column.
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| | |
This broke in #10272
|
| |\| |
|
| | | |
|
| |\| |
|
| | | |
|
| | | |
|
| | |\
| | |
| | |
| | | |
into release-v1.37
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Fixes #9490
This will break a couple of SyTest that are expecting failures to be added to the response of a federation /send, which obviously doesn't happen now that things are asynchronous.
Two drawbacks:
Currently there is no logic to handle any events left in the staging area after restart, and so they'll only be handled on the next incoming event in that room. That can be fixed separately.
We now only process one event per room at a time. This can be fixed up further down the line.
|
| | | |
| | |
| | | |
This adds a simple best effort locking mechanism that works cross workers.
|
| | | |
| | |
| | |
| | |
| | | |
Previously m.child.room events in non-space rooms would be
treated as part of the room graph, but this is no longer
supported.
|
| | | |
| | |
| | |
| | | |
It was possible for us not to find any rows in a batch, and hence conclude that
we had finished. Let's not do that.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
* Move background update names out to a separate class
`EventsBackgroundUpdatesStore` gets inherited and we don't really want to
further pollute the namespace.
* Migrate stream_ordering to a bigint
* changelog
|
| | | | |
|
| |\| | |
|
| | | | |
|
| | |/ |
|
| | | |
|
| | |
| |
| |
| | |
freenode (#10258)
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently when a new build of the docs is created, an `index.html` file does not exist. Typically this would be generated from a`docs/README.md` file - which we have - however we're currently using [docs/README.md](https://github.com/matrix-org/synapse/blob/394673055db4df49bfd58c2f6118834a6d928563/docs/README.md) to explain the docs and point to the website. It is not part of the content of the website. So we end up not having an `index.html` file, which will result in a 404 page if one tries to navigate to `https://matrix-org.github.io/synapse/<docs_version>/index.html`.
This isn't a really problem for the default version of the documentation (currently `develop`), as [navigating to the top-level root](https://matrix-org.github.io/synapse/) of the website (without specifying a version) will [redirect](https://github.com/matrix-org/synapse/blob/a77e6925f26597958eccf0ef9956cb13c536e57e/index.html#L2) you to the Welcome and Overview page of the `develop` docs version.
However, ideally once we add a GUI for switching between versions, we'll want to send the user to `matrix-org.github.io/synapse/<version>/index.html`, which currently isn't generated.
This PR modifies the CI that builds the docs to simply copy the rendered [Welcome & Overview page](https://matrix-org.github.io/synapse/develop/welcome_and_overview.html) to `index.html`.
|
| | |
| |
| | |
Rather than persisting rejected events via `send_join` and friends, raise a 403 if someone tries to pull a fast one.
|
| | |
| |
| | |
The idea here is to stop people sending things that aren't joins/leaves/knocks through these endpoints: previously you could send anything you liked through them. I wasn't able to find any security holes from doing so, but it doesn't sound like a good thing.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This implements refresh tokens, as defined by MSC2918
This MSC has been implemented client side in Hydrogen Web: vector-im/hydrogen-web#235
The basics of the MSC works: requesting refresh tokens on login, having the access tokens expire, and using the refresh token to get a new one.
Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.37.0rc1 (2021-06-24)
==============================
This release deprecates the current spam checker interface. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new generic module interface.
This release also removes support for fetching and renewing TLS certificates using the ACME v1 protocol, which has been fully decommissioned by Let's Encrypt on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings.
Features
--------
- Implement "room knocking" as per [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403). Contributed by @Sorunome and anoa. ([\#6739](https://github.com/matrix-org/synapse/issues/6739), [\#9359](https://github.com/matrix-org/synapse/issues/9359), [\#10167](https://github.com/matrix-org/synapse/issues/10167), [\#10212](https://github.com/matrix-org/synapse/issues/10212), [\#10227](https://github.com/matrix-org/synapse/issues/10227))
- Add experimental support for backfilling history into rooms ([MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716)). ([\#9247](https://github.com/matrix-org/synapse/issues/9247))
- Implement a generic interface for third-party plugin modules. ([\#10062](https://github.com/matrix-org/synapse/issues/10062), [\#10206](https://github.com/matrix-org/synapse/issues/10206))
- Implement config option `sso.update_profile_information` to sync SSO users' profile information with the identity provider each time they login. Currently only displayname is supported. ([\#10108](https://github.com/matrix-org/synapse/issues/10108))
- Ensure that errors during startup are written to the logs and the console. ([\#10191](https://github.com/matrix-org/synapse/issues/10191))
Bugfixes
--------
- Fix a bug introduced in Synapse v1.25.0 that prevented the `ip_range_whitelist` configuration option from working for federation and identity servers. Contributed by @mikure. ([\#10115](https://github.com/matrix-org/synapse/issues/10115))
- Remove a broken import line in Synapse's `admin_cmd` worker. Broke in Synapse v1.33.0. ([\#10154](https://github.com/matrix-org/synapse/issues/10154))
- Fix a bug introduced in Synapse v1.21.0 which could cause `/sync` to return immediately with an empty response. ([\#10157](https://github.com/matrix-org/synapse/issues/10157), [\#10158](https://github.com/matrix-org/synapse/issues/10158))
- Fix a minor bug in the response to `/_matrix/client/r0/user/{user}/openid/request_token` causing `expires_in` to be a float instead of an integer. Contributed by @lukaslihotzki. ([\#10175](https://github.com/matrix-org/synapse/issues/10175))
- Always require users to re-authenticate for dangerous operations: deactivating an account, modifying an account password, and adding 3PIDs. ([\#10184](https://github.com/matrix-org/synapse/issues/10184))
- Fix a bug introduced in Synpase v1.7.2 where remote server count metrics collection would be incorrectly delayed on startup. Found by @heftig. ([\#10195](https://github.com/matrix-org/synapse/issues/10195))
- Fix a bug introduced in Synapse v1.35.1 where an `allow` key of a `m.room.join_rules` event could be applied for incorrect room versions and configurations. ([\#10208](https://github.com/matrix-org/synapse/issues/10208))
- Fix performance regression in responding to user key requests over federation. Introduced in Synapse v1.34.0rc1. ([\#10221](https://github.com/matrix-org/synapse/issues/10221))
Improved Documentation
----------------------
- Add a new guide to decoding request logs. ([\#8436](https://github.com/matrix-org/synapse/issues/8436))
- Mention in the sample homeserver config that you may need to configure max upload size in your reverse proxy. Contributed by @aaronraimist. ([\#10122](https://github.com/matrix-org/synapse/issues/10122))
- Fix broken links in documentation. ([\#10180](https://github.com/matrix-org/synapse/issues/10180))
- Deploy a snapshot of the documentation website upon each new Synapse release. ([\#10198](https://github.com/matrix-org/synapse/issues/10198))
Deprecations and Removals
-------------------------
- The current spam checker interface is deprecated in favour of a new generic modules system. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new system. ([\#10062](https://github.com/matrix-org/synapse/issues/10062), [\#10210](https://github.com/matrix-org/synapse/issues/10210), [\#10238](https://github.com/matrix-org/synapse/issues/10238))
- Stop supporting the unstable spaces prefixes from MSC1772. ([\#10161](https://github.com/matrix-org/synapse/issues/10161))
- Remove Synapse's support for automatically fetching and renewing certificates using the ACME v1 protocol. This protocol has been fully turned off by Let's Encrypt for existing installations on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings. ([\#10194](https://github.com/matrix-org/synapse/issues/10194))
Internal Changes
----------------
- Update the database schema versioning to support gradual migration away from legacy tables. ([\#9933](https://github.com/matrix-org/synapse/issues/9933))
- Add type hints to the federation servlets. ([\#10080](https://github.com/matrix-org/synapse/issues/10080))
- Improve OpenTracing for event persistence. ([\#10134](https://github.com/matrix-org/synapse/issues/10134), [\#10193](https://github.com/matrix-org/synapse/issues/10193))
- Clean up the interface for injecting OpenTracing over HTTP. ([\#10143](https://github.com/matrix-org/synapse/issues/10143))
- Limit the number of in-flight `/keys/query` requests from a single device. ([\#10144](https://github.com/matrix-org/synapse/issues/10144))
- Refactor EventPersistenceQueue. ([\#10145](https://github.com/matrix-org/synapse/issues/10145))
- Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. ([\#10148](https://github.com/matrix-org/synapse/issues/10148))
- Update the Complement build tags in GitHub Actions to test currently experimental features. ([\#10155](https://github.com/matrix-org/synapse/issues/10155))
- Add a `synapse_federation_soft_failed_events_total` metric to track how often events are soft failed. ([\#10156](https://github.com/matrix-org/synapse/issues/10156))
- Fetch the corresponding complement branch when performing CI. ([\#10160](https://github.com/matrix-org/synapse/issues/10160))
- Add some developer documentation about boolean columns in database schemas. ([\#10164](https://github.com/matrix-org/synapse/issues/10164))
- Add extra logging fields to better debug where events are being soft failed. ([\#10168](https://github.com/matrix-org/synapse/issues/10168))
- Add debug logging for when we enter and exit `Measure` blocks. ([\#10183](https://github.com/matrix-org/synapse/issues/10183))
- Improve comments in structured logging code. ([\#10188](https://github.com/matrix-org/synapse/issues/10188))
- Update [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083) support with modifications from the MSC. ([\#10189](https://github.com/matrix-org/synapse/issues/10189))
- Remove redundant DNS lookup limiter. ([\#10190](https://github.com/matrix-org/synapse/issues/10190))
- Upgrade `black` linting tool to 21.6b0. ([\#10197](https://github.com/matrix-org/synapse/issues/10197))
- Expose OpenTracing trace id in response headers. ([\#10199](https://github.com/matrix-org/synapse/issues/10199))
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| | |
Fixes #10234
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This PR:
* Converts UPGRADE.rst to markdown and moves the contents into the `docs/` directory.
* Updates the contents of UPGRADE.rst to point to the website instead.
* Updates links around the codebase that point to UPGRADE.rst.
`pandoc` + some manual editing was used to convert from RST to md.
|
| | |
| |
| |
| | |
* rename major/minor into the right semver terminology minor/patch (since this was something that got me very confused the first couple of times I've used the script)
* name the release branch based on the new version, not the previous one
|
| | |
| |
| | |
Required some fixes due to merge conflicts with #6739, but nothing too hairy. The first commit is the same as the original (after merge conflict resolution) then two more for compatibility with the latest sync code.
|
| | |
| |
| |
| |
| |
| | |
If a room is remote and we don't have a user in it, always try to join it. It might fail if the room is invite-only, but we don't have a user to invite with, so at this point it's the best we can do.
Fixes #10233 (at least to some extent)
|
| | |
| |
| |
| |
| | |
ensure that events sent via `send_leave` and `send_knock` are sent on to
the rest of the federation.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Drop Origin & Accept from Access-Control-Allow-Headers value
This change drops the Origin and Accept header names from the value of the
Access-Control-Allow-Headers response header sent by Synapse. Per the CORS
protocol, it’s not necessary or useful to include those header names.
Details:
Per-spec at https://fetch.spec.whatwg.org/#forbidden-header-name, Origin
is a “forbidden header name” set by the browser and that frontend
JavaScript code is never allowed to set.
So the value of Access-Control-Allow-Headers isn’t relevant to Origin or
in general to other headers set by the browser itself — the browser
never ever consults the Access-Control-Allow-Headers value to confirm
that it’s OK for the request to include an Origin header.
And per-spec at https://fetch.spec.whatwg.org/#cors-safelisted-request-header,
Accept is a “CORS-safelisted request-header”, which means that browsers
allow requests to contain the Accept header regardless of whether the
Access-Control-Allow-Headers value contains "Accept".
So it’s unnecessary for the Access-Control-Allow-Headers to explicitly
include Accept. Browsers will not perform a CORS preflight for requests
containing an Accept request header.
Related: https://github.com/matrix-org/matrix-doc/pull/3225
Signed-off-by: Michael[tm] Smith <mike@w3.org>
|
| |
|
| |
Introduced in #6739
|
| |
|
| |
So admins aren't surprised if things break when we remove this code in a couple of months.
|
| |
|
| |
Work on https://github.com/matrix-org/matrix-doc/pull/2716
|
| |
|
|
|
|
| |
Implemented config option sso.update_profile_information to keep user's display name in sync with the SSO displayname.
Signed-off-by: Johannes Kanefendt <johannes.kanefendt@krzn.de>
|
| |
|
|
|
| |
We were repeatedly looking up a config option in a loop (using the
unclassed config style), which is expensive enough that it can cause
large CPU usage.
|
| |
|
|
|
| |
An accidental mis-ordering of operations during #6739 technically allowed an incoming knock event over federation in before checking it against any configured Third Party Access Rules modules.
This PR corrects that by performing the TPAR check *before* persisting the event.
|
| |
|
|
|
|
|
|
| |
(#10191)
* Defer stdio redirection until we are about to start the reactor
* Catch and handle exceptions during startup
|
| |
|
|
|
|
|
|
|
| |
This PR will run a new "Deploy release-specific documentation" job whenever a push to a branch name matching `release-v*` occurs. Doing so will create/add to a folder named `vX.Y` on the `gh-pages` branch. Doing so will allow us to build up `major.minor` releases of the docs as we release Synapse.
This is especially useful for having a mechanism for keeping around documentation of old/removed features (for those running older versions of Synapse), without needing to clutter the latest copy of the docs.
After a [discussion](https://matrix.to/#/!XaqDhxuTIlvldquJaV:matrix.org/$rKmkBmQle8OwTlGcoyu0BkcWXdnHW3_oap8BMgclwIY?via=matrix.org&via=vector.modular.im&via=envs.net) in #synapse-dev, we wanted to use tags to trigger the documentation deployments, which I agreed with. However, I soon realised that the bash-foo required to turn a tag of `v1.2.3rc1` into `1.2` was a lot more complex than the branch's `release-v1.2`. So, I've gone with the latter for simplicity.
In the future we'll have some UI on the website to switch between versions, but for now you can simply just change 'develop' to 'v1.2' in the URL.
|
| |
|
|
|
|
|
| |
This could cause a minor data leak if someone defined a non-restricted join rule
with an allow key or used a restricted join rule in an older room version, but this is
unlikely.
Additionally this starts adding unit tests to the spaces summary handler.
|
| | |
|
| |
|
| |
This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
|
| |
|
| |
Fixes: #9480
|
| |
|
|
|
| |
Fixes #9778
ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
|
| |
|
| |
Adds a "type" field and generalize "space" to "room_id".
|
| |
|
| |
Fixes #8518 by telling the ResponseCache not to cache the /sync response if the next_batch param is the same as the since token.
|
| |
|
| |
As I've written in various places in the past (#7113, #9865) I'm pretty sure this is doing nothing useful at all.
|
| |
|
|
|
| |
Reformat all files with the new version.
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
|
| |
|
|
|
|
|
|
|
| |
Fixes https://github.com/matrix-org/synapse/issues/10030.
We were expecting milliseconds where we should have provided a value in seconds.
The impact of this bug isn't too bad. The code is intended to count the number of remote servers that the homeserver can see and report that as a metric. This metric is supposed to run initially 1 second after server startup, and every 60s as well. Instead, it ran 1,000 seconds after server startup, and every 60s after startup.
This fix allows for the correct metrics to be collected immediately, as well as preventing a random collection 1,000s in the future after startup.
|
| |
|
| |
Follow-up to https://github.com/matrix-org/synapse/pull/10156#discussion_r650292223
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
Dangerous actions means deactivating an account, modifying an account
password, or adding a 3PID.
Other actions (deleting devices, uploading keys) can re-use the same UI
auth session if ui_auth.session_timeout is configured.
|
| |
|
|
|
|
|
| |
This doc is short but a useful guide to what the request log lines mean.
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Erik Johnston <erik@matrix.org>
Co-authored-by: Daniele Sluijters <daenney@users.noreply.github.com>
|
| |
|
| |
It can be helpful to know when trying to track down slow requests.
|
| |
|
|
|
|
|
|
|
|
| |
`expires_in` must be an integer according to the OpenAPI spec:
https://github.com/matrix-org/matrix-doc/blob/master/data/api/client-server/definitions/openid_token.yaml#L32
True division (`/`) returns a float instead (`"expires_in": 3600.0`).
Floor division (`//`) returns an integer, so the response is spec compliant.
Signed-off-by: Lukas Lihotzki <lukas@lihotzki.de>
|
| |
|
|
|
| |
* Fix broken links in documentation
* newsfile
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Trace event persistence
When we persist a batch of events, set the parent opentracing span to the that
from the request, so that we can trace all the way in.
* changelog
* When we force tracing, set a baggage item
... so that we can check again later.
* Link in both directions between persist_events spans
|
| |\ |
|
| | | |
|
| | |
| |
| |
| | |
The stable prefixes have been supported since v1.34.0. The unstable
prefixes are not supported by any known clients.
|
| | |
| |
| |
| |
| |
| |
| | |
endpoints. (#10167)
* Room version 7 for knocking.
* Stable prefixes and endpoints (both client and federation) for knocking.
* Removes the experimental configuration flag.
|
| | |
| |
| |
| |
| | |
Add 'federation_ip_range_whitelist'. This allows backwards-compatibility, If 'federation_ip_range_blacklist' is set. Otherwise 'ip_range_whitelist' will be used for federation servers.
Signed-off-by: Michael Kutzner 1mikure@gmail.com
|
| | |
| |
| | |
some cleanup, pulled out of #10134.
|
| | |
| |
| |
| |
| | |
This is the first of two PRs which seek to address #8518. This first PR lays the groundwork by extending ResponseCache; a second PR (#10158) will update the SyncHandler to actually use it, and fix the bug.
The idea here is that we allow the callback given to ResponseCache.wrap to decide whether its result should be cached or not. We do that by (optionally) passing a ResponseCacheContext into it, which it can modify.
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.36.0rc2 (2021-06-11)
==============================
Bugfixes
--------
- Fix a bug which caused presence updates to stop working some time after a restart, when using a presence writer worker. Broke in v1.33.0. ([\#10149](https://github.com/matrix-org/synapse/issues/10149))
- Fix a bug when using federation sender worker where it would send out more presence updates than necessary, leading to high resource usage. Broke in v1.33.0. ([\#10163](https://github.com/matrix-org/synapse/issues/10163))
- Fix a bug where Synapse could send the same presence update to a remote twice. ([\#10165](https://github.com/matrix-org/synapse/issues/10165))
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
It turns out that we were sending the same presence state to a remote
potentially multiple times.
|
| | |
| |
| |
| |
| |
| | |
When using a federation sender we'd send out all local presence updates over
federation even when they shouldn't be.
Fixes #10153.
|
| | |
| |
| | |
Hopefully fixes #10027.
|
| | |
| |
| |
| | |
It turns out that we were sending the same presence state to a remote
potentially multiple times.
|
| | |
| |
| | |
This is essentially an implementation of the proposal made at https://hackmd.io/@richvdh/BJYXQMQHO, though the details have ended up looking slightly different.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This implements similar behavior to sytest where a matching branch is used,
if one exists. This is useful when needing to modify both application code
and tests at the same time. The following rules are used to find a matching
complement branch:
1. Search for the branch name of the pull request. (E.g. feature/foo.)
2. Search for the base branch of the pull request. (E.g. develop or release-vX.Y.)
3. Search for the reference branch of the commit. (E.g. master or release-vX.Y.)
4. Fallback to 'master', the default complement branch name.
|
| | |
| |
| |
| |
| |
| | |
When using a federation sender we'd send out all local presence updates over
federation even when they shouldn't be.
Fixes #10153.
|
| | | |
|
| | |
| |
| | |
Hopefully fixes #10027.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Spawned from missing messages we were seeing on `matrix.org` from a
federated Gtiter bridged room, https://gitlab.com/gitterHQ/webapp/-/issues/2770.
The underlying issue in Synapse is tracked by https://github.com/matrix-org/synapse/issues/10066
where the message and join event race and the message is `soft_failed` before the
`join` event reaches the remote federated server.
Less soft_failed events = better and usually this should only trigger for events
where people are doing bad things and trying to fuzz and fake everything.
|
| | | |
|
| | |
| |
| |
| |
| | |
(#10122)
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | |
| |
| |
| |
| | |
This PR updates the build tags that we perform Complement runs with to match our [buildkite pipeline](https://github.com/matrix-org/pipelines/blob/618b3e90bcae8efd1a71502ae95b7913e6e24665/synapse/pipeline.yml#L570), as well as adding `msc2403` (as it will be required once #9359 is merged). Build tags are what we use to determine which tests to run in Complement (really it determines which test files are compiled into the final binary).
I haven't put in a comment about updating the buildkite side here, as we've decided to migrate fully to GitHub Actions anyhow.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This PR implements the ["Changes regarding the Public Rooms Directory"](https://github.com/Sorunome/matrix-doc/blob/soru/knock/proposals/2403-knock.md#changes-regarding-the-public-rooms-directory) section of knocking MSC2403.
Specifically, it:
* Allows rooms with `join_rule` "knock" to be returned by the query behind the public rooms directory
* Adds the field `join_rule` to each room entry returned by a public rooms directory query, so clients can know whether to attempt a join or knock on a room
Based on https://github.com/matrix-org/synapse/issues/6739. Complement tests for this change: https://github.com/matrix-org/complement/pull/72
|
| | |
| |
| |
| |
| |
| | |
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403
Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
|
| | |
| |
| |
| | |
(#10144)
|
| | |
| |
| |
| |
| |
| |
| | |
* Remove unused helper functions
* Clean up the interface for injecting opentracing over HTTP
* changelog
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Fixes #10140
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches:
release-v1.33.0
release-v1.33.1
release-v1.33.2
Under the new model, all three would share a common branch:
release-v1.33
As before, RCs and actual releases exist as tags on these branches.
This better reflects our support model, e.g., that the "1.33" series had
a formal release followed by two patches / updates.
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
|
| |
signatures/hashes for (#10117)
If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
|
| |
|
| |
Fixes #10123
|
| |
|
| |
Add a couple of extra logs/spans, to give a bit of a better idea.
|
| |
|
|
| |
... otherwise we tend to get a namespace clash between the bg process and the
functions that it calls.
|
| |
|
|
|
| |
Fixes #10121
Signed-off-by: Rohan Sharma <rhnsharma5113@gmail.com>
|
| |
|
| |
Signed-off-by: Chris Castle chris@crc.io
|
| |
|
|
|
|
|
|
|
|
| |
Fixes #1834.
`get_new_events_for_appservice` internally calls `get_events_as_list`, which will filter out any rejected events. If all returned events are filtered out, `_notify_interested_services` will return without updating the last handled stream position. If there are 100 consecutive such events, processing will halt altogether.
Breaking the loop is now done by checking whether we're up-to-date with `current_max` in the loop condition, instead of relying on an empty `events` list.
Signed-off-by: Willem Mulder <14mRh4X0r@gmail.com>
|
| | |
|
| |
|
|
| |
I hope this will improve
https://github.com/matrix-org/synapse/issues/9564.
|
| |
|
|
|
| |
If backfilling is slow then the client may time out and retry, causing
Synapse to start a new `/backfill` before the existing backfill has
finished, duplicating work.
|
| |
|
| |
Fixes #9956.
|
| | |
|
| |
|
|
| |
searchable website (#10086)
|
| |
|
|
|
|
|
|
|
|
| |
This adds quite a lot of OpenTracing decoration for database activity. Specifically it adds tracing at four different levels:
* emit a span for each "interaction" - ie, the top level database function that we tend to call "transaction", but isn't really, because it can end up as multiple transactions.
* emit a span while we hold a database connection open
* emit a span for each database transaction - actual actual transaction.
* emit a span for each database query.
I'm aware this might be quite a lot of overhead, but even just running it on a local Synapse it looks really interesting, and I hope the overhead can be offset just by turning down the sampling frequency and finding other ways of tracing requests of interest (eg, the `force_tracing_for_users` setting).
|
| |
|
|
|
|
|
| |
The existing tracing reports an error each time there is a timeout, which isn't
really representative.
Additionally, we log things about the way `wait_for_events` works
(eg, the result of the callback) to the *parent* span, which is confusing.
|
| |
|
| |
So that they render nicely in mdbook (see #10086), and so that we no longer have a mix of structured text languages in our documentation (excluding files outside of `docs/`).
|
| |
|
|
|
|
| |
[MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432) added this endpoint originally but it has since been included in the spec for nearly a year.
This is progress towards https://github.com/matrix-org/synapse/issues/8334
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
(#10109)
|
| | |
| |
| |
| |
| | |
Related to: #6681, #5956, #10040
Signed-off-by: Dirk Klimpel dirk@klimpel.org
|
| | |
| |
| | |
... using the script from matrix-org/sytest#1052
|
| | | |
|
| | |
| |
| |
| | |
(#10109)
|
| | | |
|
| |\| |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
c.f. https://github.com/matrix-org/synapse/pull/9617 and https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md
Fixes #9627.
|
| | | |
|
| | | |
|
| | |\ |
|
| | | |
| | |
| | |
| | | |
(#10082)
|
| | | | |
|
| | | |
| | |
| | | |
Fixes #10087.
|
| | | |
| | |
| | | |
Empirically, this helped my server considerably when handling gaps in Matrix HQ. The problem was that we would repeatedly call have_seen_events for the same set of (50K or so) auth_events, each of which would take many minutes to complete, even though it's only an index scan.
|
| | | |
| | |
| | | |
Signed-off-by: Brad Murray brad@beeper.com
|
| | | |
| | |
| | | |
... because tags on spans which aren't being sampled get thrown away.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Split out from https://github.com/matrix-org/synapse/pull/9247
Strings:
- `parse_string`
- `parse_string_from_args`
- `parse_strings_from_args`
For comparison with ints:
- `parse_integer`
- `parse_integer_from_args`
Previous discussions:
- https://github.com/matrix-org/synapse/pull/9247#discussion_r573195687
- https://github.com/matrix-org/synapse/pull/9247#discussion_r574214156
- https://github.com/matrix-org/synapse/pull/9247#discussion_r573264791
Signed-off-by: Eric Eastwood <erice@element.io>
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
Implements MSC2414: https://github.com/matrix-org/matrix-doc/pull/2414
See #8551
Signed-off-by: Callum Brown <callum@calcuode.com>
|
| | | |
| | |
| | |
| | | |
(#10082)
|
| |\ \ \
| | |/
| |/|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Synapse 1.35.0rc2 (2021-05-27)
==============================
Bugfixes
--------
- Fix a bug introduced in v1.35.0rc1 when calling the spaces summary API via a GET request. ([\#10079](https://github.com/matrix-org/synapse/issues/10079))
|
| | |/ |
|
| | |
| |
| |
| | |
Fixes a bug due to conflicting PRs which were merged. (One added a new caller to
a method, the other added a new parameter to the same method.)
|
| | |
| |
| | |
Fixes #10068
|
| | | |
|
| | |
| |
| |
| | |
(#10059)
|
| | | |
|
| | |
| |
| |
| |
| | |
we don't need to reload the tls cert if we don't have any tls listeners.
Follow-up to #9280.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Make `invalidate` and `invalidate_many` do the same thing
... so that we can do either over the invalidation replication stream, and also
because they always confused me a bit.
* Kill off `invalidate_many`
* changelog
|
| | |
| |
| |
| | |
In lieu of just always enabling the unstable spaces endpoint and
unstable room version.
|
| | |
| |
| |
| |
| | |
* Fix GitHub Actions lint for newsfragments
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| |
| | |
* Tweak wording of database recommendation in INSTALL.md
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | |
| |
| |
| |
| |
| | |
The `only` field takes a string or list of strings per the Circle docs:
https://circleci.com/docs/2.0/configuration-reference/#branches
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| | |
Signed-off-by: Dirk Klimpel dirk@klimpel.org
|
| |/
|
|
|
|
|
|
| |
(#10046)
* Update CAPTCHA documentation to mention turning off verify origin
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | |
|
| |
|
|
| |
Adds an option to complement.sh to run Synapse in worker
mode (instead of the default monolith mode).
|
| | |
|
| |
|
|
|
| |
Fixes #9970
Signed-off-by: Sergio Miguéns Iglesias lonyelon@lony.xyz
|
| |
|
|
| |
Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
|
| | |
|
| |
|
| |
fixes #9979
|
| |
|
|
|
|
|
| |
`keylen` seems to be a thing that is frequently incorrectly set, and we don't really need it.
The only time it was used was to figure out if we had removed a subtree in `del_multi`, which we can do better by changing `TreeCache.pop` to return a different type (`TreeCacheNode`).
Commits should be independently reviewable.
|
| |
|
|
| |
The previous spaces_enabled flag now defaults to true and
is exposed in the sample config.
|
| | |
|
| |
|
|
|
| |
actually does (#10050)
It looks like a typo copy/paste from `get_state_for_event` above.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Fix /upload 500'ing when presented a very large image
Catch DecompressionBombError and re-raise as ThumbnailErrors
* Set PIL's MAX_IMAGE_PIXELS to match homeserver.yaml
to get it to bomb out quicker, to load less into memory
in the case of super large images
* Add changelog entry for 10029
|
| |
|
|
|
|
|
| |
https://github.com/matrix-org/synapse/issues/9962 uncovered that we accidentally removed all but one of the presence updates that we store in the database when persisting multiple updates. This could cause users' presence state to be stale.
The bug was fixed in #10014, and this PR just adds a test that failed on the old code, and was used to initially verify the bug.
The test attempts to insert some presence into the database in a batch using `PresenceStore.update_presence`, and then simply pulls it out again.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Fixes: https://github.com/matrix-org/synapse/issues/9962
This is a fix for above problem.
I fixed it by swaping the order of insertion of new records and deletion of old ones. This ensures that we don't delete fresh database records as we do deletes before inserts.
Signed-off-by: Marek Matys <themarcq@gmail.com>
|
| | |
|
| |
|
|
|
|
| |
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
|
| |
|
| |
Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
|
| |
|
|
|
|
| |
(#9922)
This finishes up the experimental implementation of MSC3083 by showing
the restricted rooms in the spaces summary (from MSC2946).
|
| |\ |
|
| | |
| |
| | |
Signed-off-by: Javier Junquera Sánchez <javier@junquera.io>
|
| | | |
|
| | |
| |
| | |
Signed-off-by: Savyasachee Jha savya.jha@hawkradius.com
|
| | |
| |
| |
| |
| | |
To be more consistent with similar code. The check now automatically
raises an AuthError instead of passing back a boolean. It also absorbs
some shared logic between callers.
|
| | |
| |
| |
| | |
to them, instead of something in-memory (#9823)
|
| | |
| |
| |
| | |
\x7F was meant to be \0x7E (~) this was originally incorrect
in MSC1772.
|
| | | |
|
| |\| |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| | |
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| | | |
|
| | |
| |
| |
| | |
Turns out these scripts weren't getting linted.
|
| | |
| |
| |
| |
| |
| | |
- use a tuple rather than a list for the iterable that is passed into the
wrapped function, for performance
- test that we can pass an iterable and that keys are correctly deduped.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
It's not obvious that instances of SQLBaseStore each need their own
instances of random.SystemRandom(); let's just use random directly.
Introduced by 52839886d664576831462e033b88e5aba4c019e3
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| | |
Our strtobool already returns a bool, so no need to re-cast here
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We can get away with just catching UnicodeError here.
⋮
+-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
⋮
https://docs.python.org/3/library/exceptions.html#exception-hierarchy
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Functionally identical, but more obviously cryptographically secure.
...Explicit is better than implicit?
Avoids needing to know that SystemRandom() implies a CSPRNG, and
complies with the big scary red box on the documentation for random:
> Warning:
> The pseudo-random generators of this module should not be used for
> security purposes. For security or cryptographic uses, see the
> secrets module.
https://docs.python.org/3/library/random.html
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| | |
Add a config option which allows enabling opentracing by user id, eg for
debugging requests made by a test user.
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
* Correctly ratelimit invites when creating a room
Also allow ratelimiting for more than one action at a time.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This should help ensure that equivalent results are achieved between
homeservers querying for the summary of a space.
This implements modified MSC1772 rules, according to MSC2946.
The different is that the origin_server_ts of the m.room.create event
is not used as a tie-breaker since this might not be known if the
homeserver is not part of the room.
|
| |
|
|
|
|
|
|
|
|
|
| |
Per changes in MSC2946, the C-S and S-S APIs for spaces summary
should use GET requests.
Until this is stable, the POST endpoints still exist.
This does not switch federation requests to use the GET version yet
since it is newly added and already deployed servers might not support
it. When switching to the stable endpoint we should switch to GET
requests.
|
| |
|
|
|
| |
MSC1772 specifies the m.room.create event should be sent as part
of the invite_state. This was done optionally behind an experimental
flag, but is now done by default due to MSC1772 being approved.
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* tests for push rule pattern matching
* tests for acl pattern matching
* factor out common `re.escape`
* Factor out common re.compile
* Factor out common anchoring code
* add word_boundary support to `glob_to_regex`
* Use `glob_to_regex` in push rule evaluator
NB that this drops support for character classes. I don't think anyone ever
used them.
* Improve efficiency of globs with multiple wildcards
The idea here is that we compress multiple `*` globs into a single `.*`. We
also need to consider `?`, since `*?*` is as hard to implement efficiently as
`**`.
* add assertion on regex pattern
* Fix mypy
* Simplify glob_to_regex
* Inline the glob_to_regex helper function
Signed-off-by: Dan Callahan <danc@element.io>
* Moar comments
Signed-off-by: Dan Callahan <danc@element.io>
Co-authored-by: Dan Callahan <danc@element.io>
|
| | |
| |
| |
| |
| | |
Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers.
Signed-off-by: Aaron Raimist <aaron@raim.ist>
|
| | |
| |
| |
| |
| | |
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
|
| | |
| |
| | |
fixes #9960
|
| | |
| |
| |
| |
| |
| | |
We were pulling the full auth chain for the room out of the DB each time
we backfilled, which can be *huge* for large rooms and is totally
unnecessary.
|
| | |
| |
| |
| | |
... and test it.
|
| | |
| |
| | |
Fixes regression in send PDU times introduced in #9905.
|
| | | |
|
| |\| |
|
| | |
| |
| | |
c.f. #9936
|
| | |
| |
| |
| |
| | |
The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate.
It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
|
| | | |
|
| | |
| |
| | |
This reverts commit e9eb3549d32a6f93d07de8dbd5e1ebe54c8d8278.
|
| |\| |
|
| | | |
|
| | |
| |
| | |
Fixes #9936
|
| | | |
|
| | |
| |
| |
| |
| | |
This is an update based on changes to MSC2946. The origin_server_ts
of the m.room.create event is copied into the creation_ts field for each
room returned from the spaces summary.
|
