| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
| |
And fix a bug in the implementation of the updated redaction
format (MSC2174) where the top-level redacts field was not
properly added for backwards-compatibility.
|
| |
|
|
|
|
|
| |
(#15909)
Make it more obvious which Python version runs on a given Linux distribution so when we end up dropping support for a given Python version, we can more easily find the reference to the Python version and remove any references for the distribution. We don't want to be running tests or building packages on a distribution that no longer has a supported Python version.
This way, we can avoid another situation like when we dropped support for Python 3.7 but forgot to drop the Debian Buster references everywhere (https://github.com/matrix-org/synapse/pull/15893)
|
| |
|
|
|
|
|
|
|
| |
Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
|
| | |
|
| |
|
|
|
|
|
| |
Synapse routing of outbound federation traffic (#15806)
`matrix://` is a registered specced scheme nowadays and doesn't make sense for
our internal to Synapse use case anymore. ([discussion]
(https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
|
| |
|
|
|
|
|
|
|
|
|
|
| |
into existing rooms (#15748)
Context for why we're removing the implementation:
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734
Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737
Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
|
| |
|
|
|
| |
See https://github.com/matrix-org/synapse/pull/14095#discussion_r990335492
This is useful because when see that a relevant event is an `outlier` or `soft-failed`, then that's a good unexpected indicator explaining why it's not showing up. `filter_events_for_client` is used in `/sync`, `/messages`, `/context` which are all common end-to-end assertion touch points (also notifications, relations).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fully qualified docker image names for the main Dockerfile and Complement related.
* Fully qualified docker image names for Dockerfiles associated with building Debian release artifacts.
This one is harder and is separate from the other commit in case it wasn't correct or was unwanted. I decided to
do the expansion on the docker images in the Dockerfile itself, instead of the various source places that build
which distribution that is selected, as it would have been more invasive with the scripts breaking up the string
for tagging and such. This one is untested.
* Changelog
* Update docker/Dockerfile-workers
* Update docker/complement/Dockerfile
---------
Co-authored-by: reivilibre <olivier@librepush.net>
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* Ditch dependabot changelog workflow
* Summarise dependabot commits in release script
* Changelog
* Update scripts-dev/release.py
|
| |
|
|
|
| |
The cached decorators always return a Deferred, which was not
properly propagated. It was close enough when wrapping coroutines,
but failed if a bare function was wrapped.
|
| | |
|
| |
|
|
| |
version has been bumped locally (#15466)
|
| |
|
| |
Signed-off-by: Sean Quah <seanq@matrix.org>
|
| |
|
|
|
|
| |
documentation mechanism using Sphinx. (#15265)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
|
|
| |
* Remind maintainer to ask #synapse-dev for changelog
* Changelog
|
| |
|
|
|
| |
* Hack to rebuild the complement editable image
* Changelog
|
| | |
|
| |
|
|
|
|
| |
Also tweak the schema dump script:
- add a note explaining myself how to use it
-Explicitly call `poetry run`, because not everyone uses direnv :(
|
| |
|
|
|
|
| |
(#14742)
|
| |
|
| |
Signed-off-by: Jason Little realtyem@gmail.com
|
| |
|
|
|
| |
* Fix typo in release script
* Changelog
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Enable Complement tests for Faster Remote Room Joins on worker-mode
* (dangerous) Add an override to allow Complement to use FRRJ under workers
* Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Fix race where we didn't send out replication notification
* MORE HACKS
* Fix get_un_partial_stated_rooms_token to take instance_name
* Fix bad merge
* Remove warning
* Correctly advance un_partial_stated_room_stream
* Fix merge
* Add another notify_replication
* Fixups
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Fix presence test
* Newsfile
* Apply suggestions from code review
* Update changelog.d/14752.misc
Co-authored-by: Erik Johnston <erik@matrix.org>
* lint
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* Suppress update check in ruff
* Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
|
| |
|
|
|
| |
ruff is a flake8-compatible Python linter written in Rust.
It supports the flake8 plugins that we use and is significantly
faster in testing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Enable `--warn-redundant-casts` option in mypy
Doesn't do much but helps me sleep better at night.
* Changelog
* Fix name of the ignore
* Fix one more missed cast
Not sure why I didn't see this one locally, maybe I needed a poetry update
* Remove old comment
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
|
|
| |
Synapse for faster turn-around times whilst developing iteratively. (#14548)
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
|
| |
|
|
|
|
|
|
| |
Fix https://github.com/matrix-org/synapse/issues/14390
- Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
- Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`
Complement test changes: https://github.com/matrix-org/complement/pull/559
|
| |
|
|
|
|
|
|
|
|
|
| |
* Attempt to fix federation-client devscript handling of .well-known
The script was setting the wrong value in the Host header
* Fix TLS verification
Turns out that actually doing TLS verification isn't that hard. Let's enable
it.
|
| |
|
|
|
|
|
| |
* Remove unused type-ignores
Oversights in #14427 and #14429.
* Changelog
|
| |
|
| |
Signed-off-by: Sean Quah <seanq@matrix.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
command line. (#14324)
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.
* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.
* Update docs to reflect this new ability.
* Changelog
* Don't rely on soft wrapping to format long strings
Good idea dklimpel. Thanks for catching that.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
* Small nits just noticed in docs.
* Fixup new line in docs.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
|
| | |
|
| |
|
| |
Co-authored-by: Eric Eastwood <erice@element.io>
|
| |
|
| |
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
|
| |
This reverts commit 6d543d6d9f56e39199b7e460d0081b02d61f12be.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Update mypy and mypy-zope
* Unignore assigning to LogRecord attributes
Presumably https://github.com/python/typeshed/pull/8064 makes this ok
Cherry-picked from #13521
* Remove unused ignores due to mypy ParamSpec fixes
https://github.com/python/mypy/pull/12668
Cherry-picked from #13521
* Remove additional unused ignores
* Fix new mypy complaints related to `assertGreater`
Presumably due to https://github.com/python/typeshed/pull/8077
* Changelog
* Reword changelog
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
| |
|
| |
Including another batch of fixes to the schema dump script
|
| |
|
| |
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
|
| | |
|
| |
|
|
|
|
|
| |
* Generate separate snapshots for sqlite, postgres and common
* Cleanup postgres dbs in the TRAP
* Say which logical DB we're applying updates to
* Run background updates on the state DB
* Add new option for accepting a SCHEMA_NUMBER
|
| | |
|
| |
|
|
| |
usable as a guide for the whole process. (#13483)
|
| | |
|
| |
|
|
| |
the correct branches after a release. (#13393)
|
| |
|
| |
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
| |
|
|
|
| |
than having that be a manual process. (#12978)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| | |
|
| |
|
|
| |
Complement. (#13158)
|
| |
|
|
| |
environment when using `complement.sh`. (#13152)
|
| | |
|
| |
|
| |
Add a `-f` argument to `complement.sh` to skip the docker build
|
| | |
|
| |
|
|
| |
to read. (#13058)
|
| | |
|
| | |
|
| |
|
|
|
| |
image. (#12881)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | |
|
| |
|
|
| |
with workers. (#12842)
|
| | |
|
| |
|
|
|
|
|
| |
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
|
| | |
|
| |
|
|
|
|
|
| |
Check we're on the right branch before tagging, and on the right tag before uploading
* Abort if we're on the wrong branch
* Check we have the right tag checked out
* Clarify that `publish` only releases to GitHub
|
| |
|
| |
Not enforced in config yet. One day.
|
| | |
|
| |
|
|
| |
Poetry now manages the project version in pyproject.toml.
|
| | |
|
| |
|
|
|
|
| |
When we join a room via the faster-joins mechanism, we end up with "partial
state" at some points on the event DAG. Many parts of the codebase need to
wait for the full state to load. So, we implement a mechanism to keep track of
which events have partial state, and wait for them to be fully-populated.
|
| |
|
|
|
|
|
| |
* Remove leftover references to setup.py
Missed in #12478.
* Changelog
|
| |
|
|
| |
The release script used to incorrectly write `New synapse release 1.57.0~rc1.`
instead of `New synapse release 1.57.0rc1.`
|
| |
|
|
| |
Co-authored-by: Jan Christian Grünhage <jan.christian@gruenhage.xyz>
Co-authored-by: Marcus Hoffmann <bubu@bubu1.eu>
|
| | |
|
| | |
|
| |
|
| |
This wasn't quite generating the right thing.
|
| |
|
|
|
|
| |
Fixes matrix-org/complement#330 (or it will, once we remove the old files).
It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| | |
As we want to include the previous version in the "No new changes..."
string.
|
| | | |
|
| | |
| |
| | |
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
| | |
| |
| |
| |
| | |
The Complement tests for MSC3030 are now merged, https://github.com/matrix-org/complement/pull/178
Synapse implmentation: https://github.com/matrix-org/synapse/pull/9445
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
* Rename scripts-dev to have suffices
* Update references to `scripts-dev`
* Changelog
* These scripts don't pass mypy
|
| |/
|
|
|
|
|
|
|
|
|
|
| |
(#12118)
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| |
|
|
| |
* Don't use `tox` for `check-sampleconfig`
* Don't use `tox` for check-newsfragment
|
| |
|
|
|
|
|
| |
The complement.sh script relies on the name of the ref matching the name
of the unpacked folder. The branch redirect from renaming the default
branch breaks that assumption.
Signed-off-by: Nicolas Werner <n.werner@famedly.com>
|
| | |
|
| |
|
|
|
| |
Updates complement.sh to read the ref from an environment
variable (defaulting to master) when downloading a complement
bundle for testing.
|
| |
|
| |
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
* Fix AssertionErrors after purging events
If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.
This fixes the situation by rewinding the stream processors.
* `check-newsfragment`: ignore deleted newsfiles
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* update Trove classifiers to remove py36
* stop building bionic
* update dh-virtualenv
* newsfragment
* fix newsfragment
* update version refs
* another try at correct tag
* Update changelog
|
| | |
|
| |
|
|
|
|
| |
This also makes additional updates where the implementation
had drifted from the approved MSC.
Unstable endpoints will be removed at a later data.
|
| |
|
|
|
| |
This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.
|
| |
|
| |
As the tests are currently failing and not run in CI.
|
| |
|
|
| |
By reverting changes from #11166 in this script. Specifically commit
13f084eb58c379b2fc073680818d5931e94626f4.
|
| |
|
|
|
|
|
|
| |
Use $(...) notation instead of legacy backticked `...`.
https://github.com/koalaman/shellcheck/wiki/SC2006
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
|
|
|
|
| |
Double quote to prevent globbing and word splitting.
https://github.com/koalaman/shellcheck/wiki/SC2086
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
|
|
|
|
| |
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
https://github.com/koalaman/shellcheck/wiki/SC2166
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
|
|
|
|
| |
Quote this to prevent word splitting
https://www.shellcheck.net/wiki/SC2046
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
|
|
|
|
| |
Use `cd ... || exit` in case cd fails.
https://github.com/koalaman/shellcheck/wiki/SC2164
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
|
| |
It's been possible to configure a key inline in the homeserver.yaml since
13bc1e0746aa0442aa5d43555cbbc2dc75e8ef43. Update `sign_json` to work with this.
|
| |
|
|
|
|
|
|
|
|
|
| |
* support ubuntu 21.10 indri
* add changelog
* update to correct codename
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
|
| |
|
| |
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| |
|
|
| |
Signed-off-by: Nick Barrett <nick@beeper.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Use sytest:bionic. Sytest:latest is two years old (do we want
CI to push out latest at all?) and comes with Python 3.5, which we
explictly no longer support. The script now runs under PostgreSQL 10
as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root
A few changes to make it speedier to re-run the tests:
- Create blank DB in the container, not the script, so we don't have to
`initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
fetch and install a bunch of packages from PyPI each time
Co-authored-by: reivilibre <olivier@librepush.net>
|
| |
|
| |
When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
|
| |
|
|
|
|
| |
workers. (#10686)
We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
use PULL_REQUEST_NUMBER instead of BUILDKITE_PULL_REQUEST
remove the other user of BUILDKITE_PULL_REQUEST, namely merge_base_branch.sh
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.40.0rc2 (2021-08-04)
==============================
Bugfixes
--------
- Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread. ([\#10517](https://github.com/matrix-org/synapse/issues/10517))
- Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. ([\#10531](https://github.com/matrix-org/synapse/issues/10531))
Internal Changes
----------------
- Fix release script to open the correct URL for the release. ([\#10516](https://github.com/matrix-org/synapse/issues/10516))
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Make historical messages available to federated servers
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
Follow-up to https://github.com/matrix-org/synapse/pull/9247
* Debug message not available on federation
* Add base starting insertion point when no chunk ID is provided
* Fix messages from multiple senders in historical chunk
Follow-up to https://github.com/matrix-org/synapse/pull/9247
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
---
Previously, Synapse would throw a 403,
`Cannot force another user to join.`,
because we were trying to use `?user_id` from a single virtual user
which did not match with messages from other users in the chunk.
* Remove debug lines
* Messing with selecting insertion event extremeties
* Move db schema change to new version
* Add more better comments
* Make a fake requester with just what we need
See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080
* Store insertion events in table
* Make base insertion event float off on its own
See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889
Conflicts:
synapse/rest/client/v1/room.py
* Validate that the app service can actually control the given user
See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455
Conflicts:
synapse/rest/client/v1/room.py
* Add some better comments on what we're trying to check for
* Continue debugging
* Share validation logic
* Add inserted historical messages to /backfill response
* Remove debug sql queries
* Some marker event implemntation trials
* Clean up PR
* Rename insertion_event_id to just event_id
* Add some better sql comments
* More accurate description
* Add changelog
* Make it clear what MSC the change is part of
* Add more detail on which insertion event came through
* Address review and improve sql queries
* Only use event_id as unique constraint
* Fix test case where insertion event is already in the normal DAG
* Remove debug changes
* Add support for MSC2716 marker events
* Process markers when we receive it over federation
* WIP: make hs2 backfill historical messages after marker event
* hs2 to better ask for insertion event extremity
But running into the `sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group`
error
* Add insertion_event_extremities table
* Switch to chunk events so we can auth via power_levels
Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.
So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.
* Switch to chunk events for federation
* Add unstable room version to support new historical PL
* Messy: Fix undefined state_group for federated historical events
```
2021-07-13 02:27:57,810 - synapse.handlers.federation - 1248 - ERROR - GET-4 - Failed to backfill from hs1 because NOT NULL constraint failed: event_to_state_groups.state_group
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1216, in try_backfill
await self.backfill(
File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1035, in backfill
await self._auth_and_persist_event(dest, event, context, backfilled=True)
File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2222, in _auth_and_persist_event
await self._run_push_actions_and_persist_event(event, context, backfilled)
File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2244, in _run_push_actions_and_persist_event
await self.persist_events_and_notify(
File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 3290, in persist_events_and_notify
events, max_stream_token = await self.storage.persistence.persist_events(
File "/usr/local/lib/python3.8/site-packages/synapse/logging/opentracing.py", line 774, in _trace_inner
return await func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 320, in persist_events
ret_vals = await yieldable_gather_results(enqueue, partitioned.items())
File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 237, in handle_queue_loop
ret = await self._per_item_callback(
File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 577, in _persist_event_batch
await self.persist_events_store._persist_events_and_state_updates(
File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 176, in _persist_events_and_state_updates
await self.db_pool.runInteraction(
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 681, in runInteraction
result = await self.runWithConnection(
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 770, in runWithConnection
return await make_deferred_yieldable(
File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 238, in inContext
result = inContext.theWork() # type: ignore[attr-defined]
File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 254, in <lambda>
inContext.theWork = lambda: context.call( # type: ignore[attr-defined]
File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 118, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 83, in callWithContext
return func(*args, **kw)
File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 293, in _runWithConnection
compat.reraise(excValue, excTraceback)
File "/usr/local/lib/python3.8/site-packages/twisted/python/deprecate.py", line 298, in deprecatedFunction
return function(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/twisted/python/compat.py", line 403, in reraise
raise exception.with_traceback(traceback)
File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 284, in _runWithConnection
result = func(conn, *args, **kw)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 765, in inner_func
return func(db_conn, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 549, in new_transaction
r = func(cursor, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/synapse/logging/utils.py", line 69, in wrapped
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 385, in _persist_events_txn
self._store_event_state_mappings_txn(txn, events_and_contexts)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 2065, in _store_event_state_mappings_txn
self.db_pool.simple_insert_many_txn(
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 923, in simple_insert_many_txn
txn.execute_batch(sql, vals)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 280, in execute_batch
self.executemany(sql, args)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 300, in executemany
self._do_execute(self.txn.executemany, sql, *args)
File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 330, in _do_execute
return func(sql, *args)
sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group
```
* Revert "Messy: Fix undefined state_group for federated historical events"
This reverts commit 187ab28611546321e02770944c86f30ee2bc742a.
* Fix federated events being rejected for no state_groups
Add fix from https://github.com/matrix-org/synapse/pull/10439
until it merges.
* Adapting to experimental room version
* Some log cleanup
* Add better comments around extremity fetching code and why
* Rename to be more accurate to what the function returns
* Add changelog
* Ignore rejected events
* Use simplified upsert
* Add Erik's explanation of extra event checks
See https://github.com/matrix-org/synapse/pull/10498#discussion_r680880332
* Clarify that the depth is not directly correlated to the backwards extremity that we return
See https://github.com/matrix-org/synapse/pull/10498#discussion_r681725404
* lock only matters for sqlite
See https://github.com/matrix-org/synapse/pull/10498#discussion_r681728061
* Move new SQL changes to its own delta file
* Clean up upsert docstring
* Bump database schema version (62)
|
| | |
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Synapse 1.39.0rc3 (2021-07-28)
==============================
Bugfixes
--------
- Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477))
- Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485))
- Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486))
Internal Changes
----------------
- Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461))
- Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
|
| | | |
|
| | |
| |
| | |
Updates CI and the helper script t ensures all tests are run (in parallel).
|
| | | |
|
| |/ |
|
| |
|
| |
GHA workflow to build the debs
|
| |
|
|
| |
* rename major/minor into the right semver terminology minor/patch (since this was something that got me very confused the first couple of times I've used the script)
* name the release branch based on the new version, not the previous one
|
| |
|
|
|
| |
ensure that events sent via `send_leave` and `send_knock` are sent on to
the rest of the federation.
|
| |
|
| |
Work on https://github.com/matrix-org/matrix-doc/pull/2716
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches:
release-v1.33.0
release-v1.33.1
release-v1.33.2
Under the new model, all three would share a common branch:
release-v1.33
As before, RCs and actual releases exist as tags on these branches.
This better reflects our support model, e.g., that the "1.33" series had
a formal release followed by two patches / updates.
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
|
| |
|
|
| |
Adds an option to complement.sh to run Synapse in worker
mode (instead of the default monolith mode).
|
| |
|
|
| |
Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
|
| |
|
|
| |
Turns out these scripts weren't getting linted.
|
| | |
|
| | |
|
| |
|
| |
Signed-off-by: Dan Callahan <danc@element.io>
|
| | |
|
| |
|
|
|
|
|
| |
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
|
| |
|
|
| |
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
|
| |
|
| |
Signed-off-by: Dan Callahan <danc@element.io>
|
| |
|
| |
By providing the additional build tag for `msc2946`.
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
(#9735)
When joining a room with join rules set to 'restricted', check if the
user is a member of the spaces defined in the 'allow' key of the join rules.
This only applies to an experimental room version, as defined in MSC3083.
|
| |/ |
|
| |\ |
|
| | | |
|
| |/
|
|
|
|
|
| |
This PR rewrites the original complement.sh script with a number of improvements:
* We can now use a local checkout of Complement (configurable with `COMPLEMENT_DIR`), though the default behaviour still downloads the master branch.
* You can now specify a regex of test names to run, or just run all tests.
* We now use the Synapse test blacklist tag (so all tests will pass).
|
| |
|
| |
For it's obvious performance benefits. `dmypy` support landed in #9692.
|
| |
|
|
|
|
|
|
|
| |
using /usr/bin/env (#9689)
On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work.
`/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my) to use binaries from the PATH as interpreters.
Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
I noticed that I'd occasionally have `scripts-dev/lint.sh` fail when messing about with config options in my PR. The script calls `scripts-dev/config-lint.sh`, which attempts some validation on the sample config.
It does this by using `sed` to edit the sample_config, and then seeing if the file changed using `git diff`.
The problem is: if you changed the sample_config as part of your commit, this script will error regardless.
This PR attempts to change the check so that existing, unstaged changes to the sample_config will not cause the script to report an invalid file.
|
| |
|
|
|
|
|
| |
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md)
- Update `code_style.md` docs around installing black to use the correct version
|
| |
|
|
|
|
|
| |
(#9003)
Remove conflicting sqlite tables that throw sqlite3.OperationalError: object name reserved for internal use: event_search_content when running the twisted unit tests.
Fix #8996
|
| |
|
|
|
| |
(#9191)
The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.
|
| | |
|
| | |
|
| |
|
| |
Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before.
|
| |
|
|
|
|
|
| |
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.
This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
git commit (#8472)
This PR makes several changes to the `./scripts-dev/lint.sh` script, which lints the codebase with a number of tools:
* Adds usage information, with `-h` flag to show it. Otherwise it will show when providing an unknown flag.
* Adds option `-d` which will check both staged and unstaged files that have changed since the last commit and add them to the list of files to lint.
- Note that only files without an extension, or with a `.py` extension will be allowed. This prevents editing bash scripts causing the linters to break on non-python files.
* Improves the print-out of which files/directories are being linted.
|
| |\ |
|
| | | |
|
| |/
|
| |
https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This PR adds a script that:
* Builds the local Synapse checkout using our existing `docker/Dockerfile` image.
* Downloads [Complement](https://github.com/matrix-org/complement/)'s source code.
* Builds the [Synapse.Dockerfile](https://github.com/matrix-org/complement/blob/master/dockerfiles/Synapse.Dockerfile) using the above dockerfile as a base.
* Builds and runs Complement against it.
This set up differs slightly from [that of the dendrite repo](https://github.com/matrix-org/dendrite/blob/master/build/scripts/complement.sh) (`complement.sh`, `Complement.Dockerfile`), which instead stores a separate, but slightly modified, dockerfile in Dendrite's repo rather than running the one stored in Complement's repo. That synapse equivalent to that dockerfile (`Synapse.Dockerfile`) in Complement's repo is just based on top of `matrixdotorg/synapse:latest`, which we opt to build here locally.
Thus copying over the files from Complement's repo wouldn't change any functionality, and would result in two instances of the same files. So just using the dockerfile in Complement's repo was decided upon instead.
|
| |
|
|
|
|
|
| |
This converts calls like super(Foo, self) -> super().
Generated with:
sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
|
| | |
|
| | |
|
| |
|
| |
This requires adding a mypy plugin to fiddle with the type signatures a bit.
|
| |
|
|
| |
Now that the server supports streaming back JSON responses, it would be nice to
show the response as it is streamed, in the test tool.
|
| | |
|
| |
|
|
|
|
| |
This PR:
* Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default.
* Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.
|
| |
|
|
|
|
|
|
|
| |
If there are *no* files with CRLF line endings, then the xargs exits with a
non-zero exit code (as expected), but then, since that is the last thing to
happen in the script, the script as a whole exits non-zero, making the whole
thing fail.
using `if/then/fi` instead of `&& (...)` means that the script exits with a
zero exit code.
|
| |
|
|
|
| |
This PRs adds a script to check for unix-line terminators in the repo. It will be used to address https://github.com/matrix-org/synapse/issues/7943 by adding the check to CI.
I've changed the original script slightly as proposed in https://github.com/matrix-org/pipelines/pull/81#discussion_r460580664
|
| |
|
|
|
|
|
| |
script (#7914)
Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too.
Fixes https://github.com/matrix-org/synapse/issues/7910
|
| | |
|
| |
|
| |
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | | |
|
| |/
|
|
| |
this is no longer needed on python 3
|
| |
|
|
| |
Ensure good comprehension hygiene using flake8-comprehensions.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Raise an exception if there are pending background updates
So we return with a non-0 code
* Changelog
* Port synapse_port_db to async/await
* Port update_database to async/await
* Add version string to mocked homeservers
* Remove unused imports
* Convert overseen bits to async/await
* Fixup logging contexts
* Fix imports
* Add a way to print an error without raising an exception
* Incorporate review
|
| | |
|
| |
|
|
|
| |
This encapsulates config for a given database and is the way to get new
connections.
|
| |\
| |
| | |
Move database config from apps into HomeServer object
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds:
* a test sqlite database
* a configuration file for the sqlite database
* a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`)
as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that:
1. installs Synapse
2. updates the test sqlite database to the latest schema and runs background updates on it
3. creates an empty postgresql database
4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage)
Step `2` is done via a new script located at `scripts-dev/update_database`.
The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.
|
| |
|
|
|
| |
Add a linting script that enforces all boolean values in the default config be lowercase.
This has annoyed me for a while so I decided to fix it.
|
| |
|
|
| |
This doesn't work, and afaict hasn't been used since 2015.
|
| |
|
| |
Add a dev script to cover all the different linting steps.
|
| | |
|
| | |
|
| |
|
|
|
| |
* py3 fixes for federation_client
* .well-known support for federation_client
|
| | |
|
| | |
|
| |\
| |
| | |
Fix yaml warnings by using safe_load
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
* Fix check-newsfragment script
I previously broke this so that it always succeeded...
* more fixes
* fix newsfiles
|
| |
|
|
|
| |
* You need an entry in the debian changelog (and not a regular newsfragment)
for debian packaging changes.
* Regular newsfragments must end in full stops.
|
| | |
|
| |
|
|
| |
Rewrite this in python so that it can be run in parallel.
|
| |\
| |
| |
| |
| | |
matrix-org/babolivier/federation-client-content-type
Add a Content-Type header on POST requests to the federation client script
|
| | | |
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
|
| |
|
|
| |
on py3) (#4068)
|
| | |
|
| |
|
|
| |
Signed-off-by: Schnuffle <schnuffle@github.com>
|
| |
|
|
|
|
|
|
|
|
| |
* changelog
* reduce circleci config
* plus a handy script
* fix regex
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The problem with this script is that it is largely untested, entirely
unmaintained, and running it is likely to make your synapse blow up in
exciting ways.
For example, it leaves a bunch of tables with dead values in it, like
event_to_state_groups.
Having it here sends a message that it is a supported part of
synapse, which is absolutely not the case.
|
| |
|
|
|
|
| |
This appears to have stopped working since matrix.org moved to cloudflare. The
Host header should match the name of the server, not whatever is in the SRV
record.
|
| |\
| |
| | |
nuke-room-from-db.sh: fix deletion from search table
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
| |
so that you don't necessarily need a config file.
|
| |
|
|
| |
Make it read the config file, primarily.
|
| |
|
|
| |
Signed-off-by: Christian W. Zuckschwerdt <christian@zuckschwerdt.org>
|
| | |
|
| | |
|
| | |
|
| | |
|
