summary refs log tree commit diff
path: root/scripts-dev (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Build Debian packages for Ubuntu 23.10 Mantic Minotaur (#16524)David Robertson2023-10-191-0/+1
|
* Update the release script to remind releaser to check for special release ↵reivilibre2023-10-161-0/+4
| | | | | | | | | | | | | | | | | notes. (#16461) * Add reminder to check special release notes board in release script * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Update release.py * Bah, black --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Update complement.sh to match new public API shape (#16466)kegsay2023-10-121-2/+2
| | | | | | | | | | | | | * Update complement.sh to match new public API shape Sister PR to https://github.com/matrix-org/complement/pull/666 Context: https://github.com/matrix-org/complement/issues/654#issuecomment-1746613495 * Changelog * Pedantry * Run complement plz
* mypy plugin to check `@cached` return types (#14911)David Robertson2023-10-021-38/+251
| | | | | | | | Co-authored-by: David Robertson <davidr@element.io> Co-authored-by: Patrick Cloke <patrickc@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org> Assert that the return type of callables wrapped in @cached and @cachedList are cachable (aka immutable).
* Add support for pydantic v2 via pydantic.v1 compat module (#16332)Maxwell G2023-09-251-16/+82
| | | While maintaining support with pydantic v1.
* Merge branch 'master' into developMathieu Velten2023-09-122-17/+30
|\
| * Remove kinetic deb build, it's EOLMathieu Velten2023-09-121-1/+0
| |
| * Support releasing on macOS. (#16266)Patrick Cloke2023-09-071-16/+30
| |
* | Update ruff config (#16283)Patrick Cloke2023-09-081-3/+4
| | | | | | Enable additional checks & clean-up unneeded configuration.
* | Add MSC4040 `matrix-fed` service lookups (#16137)Travis Ralston2023-09-051-0/+12
|/
* Run pyupgrade for python 3.7 & 3.8. (#16110)Patrick Cloke2023-08-155-5/+4
|
* Build packages for Debian Trixie (#15961)Shay2023-07-231-0/+1
|
* Support room version 11 (#15912)Patrick Cloke2023-07-181-1/+1
| | | | | And fix a bug in the implementation of the updated redaction format (MSC2174) where the top-level redacts field was not properly added for backwards-compatibility.
* Make it more obvious which Python version runs on a given Linux distribution ↵Eric Eastwood2023-07-111-7/+11
| | | | | | | (#15909) Make it more obvious which Python version runs on a given Linux distribution so when we end up dropping support for a given Python version, we can more easily find the reference to the Python version and remove any references for the distribution. We don't want to be running tests or building packages on a distribution that no longer has a supported Python version. This way, we can avoid another situation like when we dropped support for Python 3.7 but forgot to drop the Debian Buster references everywhere (https://github.com/matrix-org/synapse/pull/15893)
* Unix Sockets for HTTP Replication (#15708)Jason Little2023-07-111-0/+4
| | | | | | | | | Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at. --------- Co-authored-by: Eric Eastwood <madlittlemods@gmail.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Eric Eastwood <erice@element.io>
* Drop debian buster (#15893)Shay2023-07-101-1/+0
|
* Switch from `matrix://` to `matrix-federation://` scheme for internal ↵Eric Eastwood2023-06-201-2/+2
| | | | | | | Synapse routing of outbound federation traffic (#15806) `matrix://` is a registered specced scheme nowadays and doesn't make sense for our internal to Synapse use case anymore. ([discussion] (https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
* Remove experimental MSC2716 implementation to incrementally import history ↵Eric Eastwood2023-06-161-4/+0
| | | | | | | | | | | | into existing rooms (#15748) Context for why we're removing the implementation: - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734 Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737 Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
* Log when events are (unexpectedly) filtered out of responses in tests (#14213)Eric Eastwood2023-06-011-0/+4
| | | | | See https://github.com/matrix-org/synapse/pull/14095#discussion_r990335492 This is useful because when see that a relevant event is an `outlier` or `soft-failed`, then that's a good unexpected indicator explaining why it's not showing up. `filter_events_for_client` is used in `/sync`, `/messages`, `/context` which are all common end-to-end assertion touch points (also notifications, relations).
* Docker fully qualified image names (#15689)Jason Little2023-05-311-0/+2
| | | | | | | | | | | | | | | | | | | | * Fully qualified docker image names for the main Dockerfile and Complement related. * Fully qualified docker image names for Dockerfiles associated with building Debian release artifacts. This one is harder and is separate from the other commit in case it wasn't correct or was unwanted. I decided to do the expansion on the docker images in the Dockerfile itself, instead of the various source places that build which distribution that is selected, as it would have been more invasive with the scripts breaking up the string for tagging and such. This one is untested. * Changelog * Update docker/Dockerfile-workers * Update docker/complement/Dockerfile --------- Co-authored-by: reivilibre <olivier@librepush.net>
* Tweak release script dependabot wordingDavid Robertson2023-05-301-0/+2
|
* Create dependabot changelogs at release time (#15481)David Robertson2023-05-301-3/+49
| | | | | | | | | * Ditch dependabot changelog workflow * Summarise dependabot commits in release script * Changelog * Update scripts-dev/release.py
* Improve type hints for cached decorator. (#15658)Patrick Cloke2023-05-241-1/+33
| | | | | The cached decorators always return a Deferred, which was not properly propagated. It was close enough when wrapping coroutines, but failed if a bare function was wrapped.
* Allow running Complement integration tests via podman (#15543)Andrew Morgan2023-05-051-12/+26
|
* Update the `check_schema_delta` script to account for when the schema ↵Shay2023-04-251-2/+24
| | | | version has been bumped locally (#15466)
* Build Debian packages for Ubuntu 23.04 Lunar Lobster (#15381)Sean Quah2023-04-031-0/+1
| | | Signed-off-by: Sean Quah <seanq@matrix.org>
* Add developer documentation for the Federation Sender and add a ↵reivilibre2023-03-241-0/+1
| | | | | | documentation mechanism using Sphinx. (#15265) Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Remind maintainer to ask #synapse-dev for changelog (#15303)David Robertson2023-03-211-1/+1
| | | | | * Remind maintainer to ask #synapse-dev for changelog * Changelog
* Hack to rebuild the complement editable image (#15184)David Robertson2023-03-071-1/+11
| | | | | * Hack to rebuild the complement editable image * Changelog
* Configure ruff to automatically fix issues. (#15194)Patrick Cloke2023-03-031-1/+1
|
* Make it easier to use DataGrip w/ Synapse's schema (#14982)David Robertson2023-02-151-3/+17
| | | | | | Also tweak the schema dump script: - add a note explaining myself how to use it -Explicitly call `poetry run`, because not everyone uses direnv :(
* Add check to ensure locked dependencies have source distributions available. ↵reivilibre2023-02-131-0/+58
| | | | | | (#14742)
* Allow enabling the asyncio reactor in complement (#14858)realtyem2023-02-011-0/+5
| | | Signed-off-by: Jason Little realtyem@gmail.com
* Fix typo in release script (#14920)David Robertson2023-01-261-1/+1
| | | | | * Fix typo in release script * Changelog
* Enable Faster Remote Room Joins against worker-mode Synapse. (#14752)reivilibre2023-01-221-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Enable Complement tests for Faster Remote Room Joins on worker-mode * (dangerous) Add an override to allow Complement to use FRRJ under workers * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Fix race where we didn't send out replication notification * MORE HACKS * Fix get_un_partial_stated_rooms_token to take instance_name * Fix bad merge * Remove warning * Correctly advance un_partial_stated_room_stream * Fix merge * Add another notify_replication * Fixups * Create a separate ReplicationNotifier * Fix test * Fix portdb * Create a separate ReplicationNotifier * Fix test * Fix portdb * Fix presence test * Newsfile * Apply suggestions from code review * Update changelog.d/14752.misc Co-authored-by: Erik Johnston <erik@matrix.org> * lint Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Erik Johnston <erik@matrix.org>
* Implement MSC3930: polls push rules (#14787)Andrew Morgan2023-01-191-1/+1
|
* Implement MSC3890: Remotely silence local notifications (#14775)Andrew Morgan2023-01-131-1/+1
|
* Drop unused table `presence` (#14825)Dirk Klimpel2023-01-131-1/+0
|
* Add rust linting commands to `scripts-dev/lint.sh` (#14822)Andrew Morgan2023-01-121-0/+33
|
* Add experimental support for MSC3391: deleting account data (#14714)Andrew Morgan2023-01-011-1/+1
|
* Suppress the update check in the ruff linter. (#14741)reivilibre2022-12-281-1/+2
| | | | | | | | | * Suppress update check in ruff * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Switch to ruff instead of flake8. (#14633)Patrick Cloke2022-12-211-3/+2
| | | | | ruff is a flake8-compatible Python linter written in Rust. It supports the flake8 plugins that we use and is significantly faster in testing.
* Enable `--warn-redundant-casts` option in mypy (#14671)David Robertson2022-12-121-4/+2
| | | | | | | | | | | | | | | | | | | * Enable `--warn-redundant-casts` option in mypy Doesn't do much but helps me sleep better at night. * Changelog * Fix name of the ignore * Fix one more missed cast Not sure why I didn't see this one locally, maybe I needed a poetry update * Remove old comment Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add `--editable` flag to `complement.sh` which uses an editable install of ↵reivilibre2022-12-071-18/+82
| | | | | Synapse for faster turn-around times whilst developing iteratively. (#14548) Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
* Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471)Eric Eastwood2022-11-281-3/+3
| | | | | | | | Fix https://github.com/matrix-org/synapse/issues/14390 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` Complement test changes: https://github.com/matrix-org/complement/pull/559
* Fixes to `federation_client` dev script (#14479)Richard van der Hoff2022-11-201-35/+87
| | | | | | | | | | | * Attempt to fix federation-client devscript handling of .well-known The script was setting the wrong value in the Host header * Fix TLS verification Turns out that actually doing TLS verification isn't that hard. Let's enable it.
* Remove unused type-ignores (#14433)David Robertson2022-11-141-3/+1
| | | | | | | * Remove unused type-ignores Oversights in #14427 and #14429. * Changelog
* Build Debian packages for Ubuntu 22.10 Kinetic Kudu (#14396)Sean Quah2022-11-091-0/+1
| | | Signed-off-by: Sean Quah <seanq@matrix.org>
* Allow override for requesting specific worker types for Complement on ↵realtyem2022-11-081-0/+3
| | | | | | | | | | | | | | | | | | | | | | | command line. (#14324) * Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested. * Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint. * Update docs to reflect this new ability. * Changelog * Don't rely on soft wrapping to format long strings Good idea dklimpel. Thanks for catching that. Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Small nits just noticed in docs. * Fixup new line in docs. Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
* Enable testing MSC3874 in complement. (#14339)Patrick Cloke2022-11-011-1/+1
|
* Mark /relations endpoint as usable on workers. (#14028)Patrick Cloke2022-10-121-2/+5
| | | Co-authored-by: Eric Eastwood <erice@element.io>
* Update mypy and mypy-zope, attempt 3 (#13993)David Robertson2022-09-301-3/+2
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Revert "Update mypy and mypy-zope (#13925)"David Robertson2022-09-301-2/+3
| | | | This reverts commit 6d543d6d9f56e39199b7e460d0081b02d61f12be.
* Update mypy and mypy-zope (#13925)David Robertson2022-09-301-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update mypy and mypy-zope * Unignore assigning to LogRecord attributes Presumably https://github.com/python/typeshed/pull/8064 makes this ok Cherry-picked from #13521 * Remove unused ignores due to mypy ParamSpec fixes https://github.com/python/mypy/pull/12668 Cherry-picked from #13521 * Remove additional unused ignores * Fix new mypy complaints related to `assertGreater` Presumably due to https://github.com/python/typeshed/pull/8077 * Changelog * Reword changelog Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Snapshot schema 72 (#13873)David Robertson2022-09-261-14/+46
| | | Including another batch of fixes to the schema dump script
* Add cache invalidation across workers to module API (#13667)Mathieu Velten2022-09-211-2/+2
| | | Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
* Fix the release script not publishing binary wheels. (#13850)reivilibre2022-09-211-11/+34
|
* Generate separate snapshots for logical databases (#13792)David Robertson2022-09-201-41/+125
| | | | | | | * Generate separate snapshots for sqlite, postgres and common * Cleanup postgres dbs in the TRAP * Say which logical DB we're applying updates to * Run background updates on the state DB * Add new option for accepting a SCHEMA_NUMBER
* Updates to the schema dump script (#13770)David Robertson2022-09-131-32/+16
|
* Extend the release script to wait for GitHub Actions to finish and to be ↵reivilibre2022-09-051-4/+142
| | | | usable as a guide for the whole process. (#13483)
* Reject non-strict types in Pydantic models (#13502)David Robertson2022-08-172-0/+426
|
* Add a `merge-back` command to the release script, which automates merging ↵reivilibre2022-08-021-0/+76
| | | | the correct branches after a release. (#13393)
* Enable Complement CI tests in the 'latest deps' test run. (#13213)reivilibre2022-08-011-0/+1
| | | Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
* Extend the release script to automatically push a new SyTest branch, rather ↵reivilibre2022-07-261-32/+54
| | | | | than having that be a manual process. (#12978) Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Stop building Ubuntu 21.10 (Impish Indri) which is end of life. (#13326)Patrick Cloke2022-07-191-1/+0
|
* Add --build-only option to complement.sh to prevent actually running ↵reivilibre2022-07-071-3/+18
| | | | Complement. (#13158)
* Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` ↵reivilibre2022-07-051-0/+12
| | | | environment when using `complement.sh`. (#13152)
* Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079)reivilibre2022-07-011-1/+6
|
* complement.sh: Permit skipping docker build (#13143)Richard van der Hoff2022-07-011-18/+54
| | | Add a `-f` argument to `complement.sh` to skip the docker build
* Add a CI job to check that schema deltas are in the correct folder. (#13063)Erik Johnston2022-06-151-0/+111
|
* Collapse Docker build commands in Complement CI runs to make the logs easier ↵reivilibre2022-06-151-0/+15
| | | | to read. (#13058)
* Up complement time outs (#13048)Erik Johnston2022-06-141-3/+3
|
* Fix example of running complement.sh. (#12990)Patrick Cloke2022-06-091-1/+1
|
* Merge the Complement testing Docker images into a single, multi-purpose ↵reivilibre2022-06-081-10/+23
| | | | | image. (#12881) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Fixes to MSC3787 implementation (#12858)David Robertson2022-05-241-1/+1
|
* Disable 'faster room join' Complement tests when testing against Synapse ↵reivilibre2022-05-231-1/+8
| | | | with workers. (#12842)
* allow `on_invalidate=None` in `@cached` methods (#12769)David Robertson2022-05-171-9/+16
|
* Tweaks to workers-under-complement (#12637)Richard van der Hoff2022-05-091-2/+11
| | | | | | | * Bump the HS startup timeout * Log prefixes for more processes * Bump the overall timeout
* Add missing space before 'docker' link in release announcement script (#12612)Andrew Morgan2022-05-031-1/+1
|
* Add sanity checks to the release script (#12556)David Robertson2022-05-031-26/+37
| | | | | | | Check we're on the right branch before tagging, and on the right tag before uploading * Abort if we're on the wrong branch * Check we have the right tag checked out * Clarify that `publish` only releases to GitHub
* Make `scripts-dev` pass `mypy --disallow-untyped-defs` (#12356)David Robertson2022-04-275-42/+67
| | | Not enforced in config yet. One day.
* Lint the release scriptDavid Robertson2022-04-261-1/+0
|
* Update release script to be poetry-awareDavid Robertson2022-04-261-47/+21
| | | | Poetry now manages the project version in pyproject.toml.
* build debian package for jammy jellyfish (#12543)Shay2022-04-261-0/+1
|
* Await un-partial-stating after a partial-state join (#12399)Richard van der Hoff2022-04-211-1/+1
| | | | | | When we join a room via the faster-joins mechanism, we end up with "partial state" at some points on the event DAG. Many parts of the codebase need to wait for the full state to load. So, we implement a mechanism to keep track of which events have partial state, and wait for them to be fully-populated.
* Remove leftover references to setup.py (#12514)David Robertson2022-04-201-1/+1
| | | | | | | * Remove leftover references to setup.py Missed in #12478. * Changelog
* Fix typo in deb changelogs from release script (#12497)Richard van der Hoff2022-04-191-1/+1
| | | | The release script used to incorrectly write `New synapse release 1.57.0~rc1.` instead of `New synapse release 1.57.0rc1.`
* Implement MSC3383: include destination in X-Matrix auth header (#11398)Jan Christian Grünhage2022-04-191-1/+6
| | | | Co-authored-by: Jan Christian Grünhage <jan.christian@gruenhage.xyz> Co-authored-by: Marcus Hoffmann <bubu@bubu1.eu>
* Reintroduce the lint targets in the linter script (#12455)David Robertson2022-04-141-2/+14
|
* Fix typos in release script docs (#12450)Erik Johnston2022-04-121-1/+2
|
* Fix up healthcheck generation for workers docker image (#12405)Richard van der Hoff2022-04-111-1/+1
| | | This wasn't quite generating the right thing.
* Move complement setup stuff into the Synapse repo (#12404)Richard van der Hoff2022-04-111-12/+5
| | | | | | Fixes matrix-org/complement#330 (or it will, once we remove the old files). It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
* Stop maintaining a list of lint targets in `lint.sh` (#12420)David Robertson2022-04-081-7/+1
|
* Remove more dead/broken dev scripts (#12355)David Robertson2022-04-033-356/+0
|
* Burn `check_signature` dev script. (#12351)David Robertson2022-04-011-72/+0
|
* Remove `list_url_patterns` dev script (#12349)David Robertson2022-04-011-60/+0
|
* Remove `dockerfile-pgtests` (#12336)David Robertson2022-03-311-19/+0
|
* Update dead links in check-newsfragment.sh (#12331)David Robertson2022-03-301-2/+2
|
* Remove references to "msc2403" (#12165)Andrew Morgan2022-03-301-1/+1
|
* Generate announcement links in release script (#12242)David Robertson2022-03-171-1/+40
|
* Merge remote-tracking branch 'origin/release-v1.54' into developRichard van der Hoff2022-03-041-2/+28
|\
| * Make release script write correct no-op changelog (#12127)Erik Johnston2022-03-021-2/+28
| | | | | | | | As we want to include the previous version in the "No new changes..." string.
* | Move synctl into `synapse._scripts` and expose as an entrypoint (#12140)David Robertson2022-03-041-1/+1
| |
* | Enable MSC2716 Complement tests in Synapse (#12145)Eric Eastwood2022-03-031-1/+1
| | | | | | Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
* | Enable MSC3030 Complement tests in Synapse (#12144)Eric Eastwood2022-03-031-1/+1
| | | | | | | | | | The Complement tests for MSC3030 are now merged, https://github.com/matrix-org/complement/pull/178 Synapse implmentation: https://github.com/matrix-org/synapse/pull/9445
* | Add suffices to scripts in scripts-dev (#12137)David Robertson2022-03-025-4/+2
| | | | | | | | | | | | | | | | | | * Rename scripts-dev to have suffices * Update references to `scripts-dev` * Changelog * These scripts don't pass mypy
* | Move scripts directory inside synapse, exposing as setuptools entry_points ↵David Robertson2022-03-023-15/+8
|/ | | | | | | | | | | | (#12118) * Two scripts are basically entry_points already * Move and rename scripts/* to synapse/_scripts/*.py * Delete sync_room_to_group.pl * Expose entry points in setup.py * Update linter script and config * Fixup scripts & docs mentioning scripts that moved Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Detox, part 1 of N (#12119)David Robertson2022-03-011-1/+1
| | | | * Don't use `tox` for `check-sampleconfig` * Don't use `tox` for check-newsfragment
* Rename default branch of complement.sh to main (#12063)Nicolas Werner2022-02-231-2/+2
| | | | | | | The complement.sh script relies on the name of the ref matching the name of the unpacked folder. The branch redirect from renaming the default branch breaks that assumption. Signed-off-by: Nicolas Werner <n.werner@famedly.com>
* Drop support for EOL Ubuntu 21.04 (#11961)Shay2022-02-101-1/+0
|
* Allow overriding the complement ref. (#11766)Nicolas Werner2022-01-201-6/+8
| | | | | Updates complement.sh to read the ref from an environment variable (defaulting to master) when downloading a complement bundle for testing.
* Use buildkit's cache feature to speed up docker builds (#11691)Richard van der Hoff2022-01-121-0/+4
| | | Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
* Complement environment variable name change and update .gitignore. (#11718)Michael Kaye2022-01-101-1/+1
|
* Fix AssertionErrors after purging events (#11642)Richard van der Hoff2022-01-041-2/+2
| | | | | | | | | | | * Fix AssertionErrors after purging events If you purged a bunch of events from your database, and then restarted synapse without receiving more events, then you would get a bunch of AssertionErrors on restart. This fixes the situation by rewinding the stream processors. * `check-newsfragment`: ignore deleted newsfiles
* Drop Bionic from Debian builds (#11633)Shay2022-01-031-1/+0
| | | | | | | | | | | | | | | | | * update Trove classifiers to remove py36 * stop building bionic * update dh-virtualenv * newsfragment * fix newsfragment * update version refs * another try at correct tag * Update changelog
* `scripts-dev/sign_json`: support for signing events (#11486)Richard van der Hoff2021-12-022-1/+42
|
* Support the stable /hierarchy endpoint from MSC2946 (#11329)Patrick Cloke2021-11-291-1/+1
| | | | | | This also makes additional updates where the implementation had drifted from the approved MSC. Unstable endpoints will be removed at a later data.
* Return the stable `event` field from `/send_join` per MSC3083. (#11413)Patrick Cloke2021-11-291-1/+1
| | | | | This does not remove the unstable field and still parses both. Handling of the unstable field will need to be removed in the future.
* Remove msc2716 from the list of tests for complement. (#11389)Patrick Cloke2021-11-191-1/+1
| | | As the tests are currently failing and not run in CI.
* Fix running complement.sh script. (#11368)Patrick Cloke2021-11-171-3/+3
| | | | By reverting changes from #11166 in this script. Specifically commit 13f084eb58c379b2fc073680818d5931e94626f4.
* Fix Shellcheck SC2006: Use $(...) notationDan Callahan2021-10-225-6/+6
| | | | | | | | Use $(...) notation instead of legacy backticked `...`. https://github.com/koalaman/shellcheck/wiki/SC2006 Signed-off-by: Dan Callahan <danc@element.io>
* Fix Shellcheck SC2086: Quote to prevent splittingDan Callahan2021-10-226-9/+9
| | | | | | | | Double quote to prevent globbing and word splitting. https://github.com/koalaman/shellcheck/wiki/SC2086 Signed-off-by: Dan Callahan <danc@element.io>
* Fix Shellcheck SC2166: test -a is not well definedDan Callahan2021-10-221-1/+1
| | | | | | | | Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. https://github.com/koalaman/shellcheck/wiki/SC2166 Signed-off-by: Dan Callahan <danc@element.io>
* Fix Shellcheck SC2046: Quote to prevent word splitDan Callahan2021-10-224-4/+4
| | | | | | | | Quote this to prevent word splitting https://www.shellcheck.net/wiki/SC2046 Signed-off-by: Dan Callahan <danc@element.io>
* Fix Shellcheck SC2164: exit in case cd fails.Dan Callahan2021-10-222-2/+2
| | | | | | | | Use `cd ... || exit` in case cd fails. https://github.com/koalaman/shellcheck/wiki/SC2164 Signed-off-by: Dan Callahan <danc@element.io>
* Update `sign_json` to support inline key config (#11139)Richard van der Hoff2021-10-201-7/+25
| | | | It's been possible to configure a key inline in the homeserver.yaml since 13bc1e0746aa0442aa5d43555cbbc2dc75e8ef43. Update `sign_json` to work with this.
* Add support for ubuntu 21.10 "Impish Indri" (#11024)Hillery Shay2021-10-121-0/+1
| | | | | | | | | | | * support ubuntu 21.10 indri * add changelog * update to correct codename Co-authored-by: Brendan Abolivier <github@brendanabolivier.com> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
* Release script improvements (#10966)reivilibre2021-10-111-4/+32
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Add `--run-background-updates` option to `update_database` script. (#10954)Nick Barrett2021-10-063-102/+2
| | | | Signed-off-by: Nick Barrett <nick@beeper.com>
* Sign the git tag in release script (#10925)Erik Johnston2021-09-271-1/+1
|
* Update postgresql testing script (#10906)David Robertson2021-09-241-0/+19
| | | | | | | | | | | | | | | | | | - Use sytest:bionic. Sytest:latest is two years old (do we want CI to push out latest at all?) and comes with Python 3.5, which we explictly no longer support. The script now runs under PostgreSQL 10 as a result. - Advertise script in the docs - Move pg testing script to scripts-dev directory - Write to host as the script's exector, not root A few changes to make it speedier to re-run the tests: - Create blank DB in the container, not the script, so we don't have to `initdb` each time - Use a named volume to persist the tox environment, so we don't have to fetch and install a bunch of packages from PyPI each time Co-authored-by: reivilibre <olivier@librepush.net>
* Add a script to update the debian changelog for non-Debian systems (#10778)Brendan Abolivier2021-09-081-0/+64
| | | When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
* Ensure the base Docker image is rebuilt when running complement with ↵Patrick Cloke2021-08-251-7/+7
| | | | | | workers. (#10686) We now always rebuild the matrixdotorg/synapse image, then build the matrixdotorg/synapse-workers image on top of it.
* Build debs for bookworm (#10612)Richard van der Hoff2021-08-171-1/+2
|
* Stop building a debian package for Groovy Gorilla (#10588)David Robertson2021-08-121-1/+0
|
* Also rename in lint.shDavid Robertson2021-08-121-1/+1
|
* Kill off the .buildkite dir completelyDavid Robertson2021-08-101-1/+1
|
* check-newsfragment: pass pr number explicitlyDavid Robertson2021-08-101-1/+1
| | | | | use PULL_REQUEST_NUMBER instead of BUILDKITE_PULL_REQUEST remove the other user of BUILDKITE_PULL_REQUEST, namely merge_base_branch.sh
* Merge tag 'v1.40.0rc2' into developErik Johnston2021-08-051-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.40.0rc2 (2021-08-04) ============================== Bugfixes -------- - Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread. ([\#10517](https://github.com/matrix-org/synapse/issues/10517)) - Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. ([\#10531](https://github.com/matrix-org/synapse/issues/10531)) Internal Changes ---------------- - Fix release script to open the correct URL for the release. ([\#10516](https://github.com/matrix-org/synapse/issues/10516))
| * Fix release script URL (#10516)Erik Johnston2021-08-031-1/+1
| |
* | Add support for MSC2716 marker events (#10498)Eric Eastwood2021-08-041-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make historical messages available to federated servers Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 Follow-up to https://github.com/matrix-org/synapse/pull/9247 * Debug message not available on federation * Add base starting insertion point when no chunk ID is provided * Fix messages from multiple senders in historical chunk Follow-up to https://github.com/matrix-org/synapse/pull/9247 Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 --- Previously, Synapse would throw a 403, `Cannot force another user to join.`, because we were trying to use `?user_id` from a single virtual user which did not match with messages from other users in the chunk. * Remove debug lines * Messing with selecting insertion event extremeties * Move db schema change to new version * Add more better comments * Make a fake requester with just what we need See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080 * Store insertion events in table * Make base insertion event float off on its own See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889 Conflicts: synapse/rest/client/v1/room.py * Validate that the app service can actually control the given user See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455 Conflicts: synapse/rest/client/v1/room.py * Add some better comments on what we're trying to check for * Continue debugging * Share validation logic * Add inserted historical messages to /backfill response * Remove debug sql queries * Some marker event implemntation trials * Clean up PR * Rename insertion_event_id to just event_id * Add some better sql comments * More accurate description * Add changelog * Make it clear what MSC the change is part of * Add more detail on which insertion event came through * Address review and improve sql queries * Only use event_id as unique constraint * Fix test case where insertion event is already in the normal DAG * Remove debug changes * Add support for MSC2716 marker events * Process markers when we receive it over federation * WIP: make hs2 backfill historical messages after marker event * hs2 to better ask for insertion event extremity But running into the `sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group` error * Add insertion_event_extremities table * Switch to chunk events so we can auth via power_levels Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room. * Switch to chunk events for federation * Add unstable room version to support new historical PL * Messy: Fix undefined state_group for federated historical events ``` 2021-07-13 02:27:57,810 - synapse.handlers.federation - 1248 - ERROR - GET-4 - Failed to backfill from hs1 because NOT NULL constraint failed: event_to_state_groups.state_group Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1216, in try_backfill await self.backfill( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1035, in backfill await self._auth_and_persist_event(dest, event, context, backfilled=True) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2222, in _auth_and_persist_event await self._run_push_actions_and_persist_event(event, context, backfilled) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2244, in _run_push_actions_and_persist_event await self.persist_events_and_notify( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 3290, in persist_events_and_notify events, max_stream_token = await self.storage.persistence.persist_events( File "/usr/local/lib/python3.8/site-packages/synapse/logging/opentracing.py", line 774, in _trace_inner return await func(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 320, in persist_events ret_vals = await yieldable_gather_results(enqueue, partitioned.items()) File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 237, in handle_queue_loop ret = await self._per_item_callback( File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 577, in _persist_event_batch await self.persist_events_store._persist_events_and_state_updates( File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 176, in _persist_events_and_state_updates await self.db_pool.runInteraction( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 681, in runInteraction result = await self.runWithConnection( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 770, in runWithConnection return await make_deferred_yieldable( File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 238, in inContext result = inContext.theWork() # type: ignore[attr-defined] File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 254, in <lambda> inContext.theWork = lambda: context.call( # type: ignore[attr-defined] File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 83, in callWithContext return func(*args, **kw) File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 293, in _runWithConnection compat.reraise(excValue, excTraceback) File "/usr/local/lib/python3.8/site-packages/twisted/python/deprecate.py", line 298, in deprecatedFunction return function(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/twisted/python/compat.py", line 403, in reraise raise exception.with_traceback(traceback) File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 284, in _runWithConnection result = func(conn, *args, **kw) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 765, in inner_func return func(db_conn, *args, **kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 549, in new_transaction r = func(cursor, *args, **kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/logging/utils.py", line 69, in wrapped return f(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 385, in _persist_events_txn self._store_event_state_mappings_txn(txn, events_and_contexts) File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 2065, in _store_event_state_mappings_txn self.db_pool.simple_insert_many_txn( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 923, in simple_insert_many_txn txn.execute_batch(sql, vals) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 280, in execute_batch self.executemany(sql, args) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 300, in executemany self._do_execute(self.txn.executemany, sql, *args) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 330, in _do_execute return func(sql, *args) sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group ``` * Revert "Messy: Fix undefined state_group for federated historical events" This reverts commit 187ab28611546321e02770944c86f30ee2bc742a. * Fix federated events being rejected for no state_groups Add fix from https://github.com/matrix-org/synapse/pull/10439 until it merges. * Adapting to experimental room version * Some log cleanup * Add better comments around extremity fetching code and why * Rename to be more accurate to what the function returns * Add changelog * Ignore rejected events * Use simplified upsert * Add Erik's explanation of extra event checks See https://github.com/matrix-org/synapse/pull/10498#discussion_r680880332 * Clarify that the depth is not directly correlated to the backwards extremity that we return See https://github.com/matrix-org/synapse/pull/10498#discussion_r681725404 * lock only matters for sqlite See https://github.com/matrix-org/synapse/pull/10498#discussion_r681728061 * Move new SQL changes to its own delta file * Clean up upsert docstring * Bump database schema version (62)
* Extend the release script to tag and create the releases. (#10496)Erik Johnston2021-08-031-36/+275
|
* Merge tag 'v1.39.0rc3' into developErik Johnston2021-07-281-6/+20
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.39.0rc3 (2021-07-28) ============================== Bugfixes -------- - Fix a bug introduced in Synapse 1.38 which caused an exception at startup when SAML authentication was enabled. ([\#10477](https://github.com/matrix-org/synapse/issues/10477)) - Fix a long-standing bug where Synapse would not inform clients that a device had exhausted its one-time-key pool, potentially causing problems decrypting events. ([\#10485](https://github.com/matrix-org/synapse/issues/10485)) - Fix reporting old R30 stats as R30v2 stats. Introduced in v1.39.0rc1. ([\#10486](https://github.com/matrix-org/synapse/issues/10486)) Internal Changes ---------------- - Fix an error which prevented the Github Actions workflow to build the docker images from running. ([\#10461](https://github.com/matrix-org/synapse/issues/10461)) - Fix release script to correctly version debian changelog when doing RCs. ([\#10465](https://github.com/matrix-org/synapse/issues/10465))
| * Change release script to update debian changelog for RCs (#10465)Erik Johnston2021-07-271-6/+20
| |
* | Use new go test running syntax for complement. (#10488)Patrick Cloke2021-07-271-1/+1
| | | | | | Updates CI and the helper script t ensures all tests are run (in parallel).
* | Enable docker image caching for the deb build (#10431)Richard van der Hoff2021-07-261-9/+29
| |
* | Disable msc2716 until Complement update is merged (#10463)Eric Eastwood2021-07-221-1/+1
|/
* build debs in GHA (#10247)Richard van der Hoff2021-07-121-4/+13
| | | GHA workflow to build the debs
* Fixes to the release script (#10239)Brendan Abolivier2021-06-231-25/+27
| | | | * rename major/minor into the right semver terminology minor/patch (since this was something that got me very confused the first couple of times I've used the script) * name the release branch based on the new version, not the previous one
* Send out invite rejections and knocks over federation (#10223)Richard van der Hoff2021-06-231-1/+1
| | | | | ensure that events sent via `send_leave` and `send_knock` are sent on to the rest of the federation.
* Add endpoints for backfilling history (MSC2716) (#9247)Eric Eastwood2021-06-221-1/+1
| | | Work on https://github.com/matrix-org/matrix-doc/pull/2716
* Name release branches just after major.minor (#10013)Dan Callahan2021-06-081-1/+1
| | | | | | | | | | | | | | | | | | With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches: release-v1.33.0 release-v1.33.1 release-v1.33.2 Under the new model, all three would share a common branch: release-v1.33 As before, RCs and actual releases exist as tags on these branches. This better reflects our support model, e.g., that the "1.33" series had a formal release followed by two patches / updates. Signed-off-by: Dan Callahan <danc@element.io>
* Remove redundant, unmaintained `convert_server_keys` script. (#10055)Richard van der Hoff2021-05-271-108/+0
|
* Run complement with Synapse workers manually. (#10039)Patrick Cloke2021-05-241-3/+22
| | | | Adds an option to complement.sh to run Synapse in worker mode (instead of the default monolith mode).
* Remove tls_fingerprints option (#9280)Jerin J Titus2021-05-241-7/+0
| | | | Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
* Run `black` on the scripts (#9981)Richard van der Hoff2021-05-142-40/+83
| | | | Turns out these scripts weren't getting linted.
* Change the format of access tokens away from macaroons (#5588)Richard van der Hoff2021-05-121-1/+1
|
* Fix make_full_schema to create the db with the right options and user (#9931)Richard van der Hoff2021-05-071-9/+10
|
* Build Debian packages for Ubuntu 21.04 Hirsute (#9909)Dan Callahan2021-04-301-3/+4
| | | Signed-off-by: Dan Callahan <danc@element.io>
* Fix (final) Bugbear violations (#9838)Jonathan de Jong2021-04-202-2/+2
|
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-143-3/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Add release helper script (#9713)Erik Johnston2021-04-131-0/+244
| | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Add option to skip unit tests when building debs (#9793)Dan Callahan2021-04-121-6/+11
| | | Signed-off-by: Dan Callahan <danc@element.io>
* Enable complement tests for MSC2946. (#9771)Patrick Cloke2021-04-091-1/+1
| | | By providing the additional build tag for `msc2946`.
* Merge remote-tracking branch 'origin/develop' into rav/drop_py35Richard van der Hoff2021-04-081-1/+1
|\
| * MSC3083: Check for space membership during a local join of restricted rooms. ↵Patrick Cloke2021-04-081-1/+1
| | | | | | | | | | | | | | | | (#9735) When joining a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* | drop support for stretch and xenialRichard van der Hoff2021-04-081-2/+0
|/
* Merge branch 'release-v1.31.0' into developPatrick Cloke2021-03-311-1/+1
|\
| * Revert "Use 'dmypy run' in lint.sh instead of 'mypy' (#9701)" (#9720)Patrick Cloke2021-03-311-1/+1
| |
* | Rewrite complement.sh (#9685)Andrew Morgan2021-03-311-11/+38
|/ | | | | | | This PR rewrites the original complement.sh script with a number of improvements: * We can now use a local checkout of Complement (configurable with `COMPLEMENT_DIR`), though the default behaviour still downloads the master branch. * You can now specify a regex of test names to run, or just run all tests. * We now use the Synapse test blacklist tag (so all tests will pass).
* Use 'dmypy run' in lint.sh instead of 'mypy' (#9701)Andrew Morgan2021-03-301-1/+1
| | | For it's obvious performance benefits. `dmypy` support landed in #9692.
* Use interpreter from $PATH instead of absolute paths in various scripts ↵Quentin Gliech2021-03-256-7/+7
| | | | | | | | | using /usr/bin/env (#9689) On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work. `/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my) to use binaries from the PATH as interpreters. Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
* federation_client: handle inline signing_keys in hs.yaml (#9647)Richard van der Hoff2021-03-181-54/+17
|
* federation_client: stop adding URL prefix (#9645)Richard van der Hoff2021-03-181-2/+2
|
* Prevent the config-lint script erroring out on any sample_config changes (#9562)Andrew Morgan2021-03-091-2/+7
| | | | | | | | | I noticed that I'd occasionally have `scripts-dev/lint.sh` fail when messing about with config options in my PR. The script calls `scripts-dev/config-lint.sh`, which attempts some validation on the sample config. It does this by using `sed` to edit the sample_config, and then seeing if the file changed using `git diff`. The problem is: if you changed the sample_config as part of your commit, this script will error regardless. This PR attempts to change the check so that existing, unstaged changes to the sample_config will not cause the script to report an invalid file.
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-1/+3
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Remove conflicting sqlite tables that are "reserved" (shadow fts4 tables) ↵Eric Eastwood2021-02-101-0/+11
| | | | | | | (#9003) Remove conflicting sqlite tables that throw sqlite3.OperationalError: object name reserved for internal use: event_search_content when running the twisted unit tests. Fix #8996
* Align the directories linted in CI with the defaults in scripts-dev/lint.sh ↵Andrew Morgan2021-01-211-1/+2
| | | | | (#9191) The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.
* Add type hints to the push module. (#8901)Patrick Cloke2020-12-111-0/+2
|
* Add a script to sign arbitrary json objects. (#8772)Richard van der Hoff2020-11-241-0/+127
|
* Fix cache call signature to accept `on_invalidate`. (#8684)Erik Johnston2020-10-291-11/+27
| | | Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before.
* Support generating structured logs in addition to standard logs. (#8607)Patrick Cloke2020-10-291-1/+1
| | | | | | | This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.
* Run mypy as part of the lint.sh script. (#8633)kleph2020-10-281-0/+1
|
* Add option to scripts-dev/lint.sh to only lint files changed since the last ↵Andrew Morgan2020-10-151-9/+84
| | | | | | | | | | git commit (#8472) This PR makes several changes to the `./scripts-dev/lint.sh` script, which lints the codebase with a number of tools: * Adds usage information, with `-h` flag to show it. Otherwise it will show when providing an unknown flag. * Adds option `-d` which will check both staged and unstaged files that have changed since the last commit and add them to the list of files to lint. - Note that only files without an extension, or with a `.py` extension will be allowed. This prevents editing bash scripts causing the linters to break on non-python files. * Improves the print-out of which files/directories are being linted.
* Merge remote-tracking branch 'origin/release-v1.21.0' into developErik Johnston2020-10-071-0/+1
|\
| * Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475)Patrick Cloke2020-10-071-0/+1
| |
* | Remove docs/sphinx and related references (#8480)Andrew Morgan2020-10-071-1/+0
|/ | | https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...
* Add support for running Complement against the local checkout (#8317)Andrew Morgan2020-09-291-0/+22
| | | | | | | | | | | | This PR adds a script that: * Builds the local Synapse checkout using our existing `docker/Dockerfile` image. * Downloads [Complement](https://github.com/matrix-org/complement/)'s source code. * Builds the [Synapse.Dockerfile](https://github.com/matrix-org/complement/blob/master/dockerfiles/Synapse.Dockerfile) using the above dockerfile as a base. * Builds and runs Complement against it. This set up differs slightly from [that of the dendrite repo](https://github.com/matrix-org/dendrite/blob/master/build/scripts/complement.sh) (`complement.sh`, `Complement.Dockerfile`), which instead stores a separate, but slightly modified, dockerfile in Dendrite's repo rather than running the one stored in Complement's repo. That synapse equivalent to that dockerfile (`Synapse.Dockerfile`) in Complement's repo is just based on top of `matrixdotorg/synapse:latest`, which we opt to build here locally. Thus copying over the files from Complement's repo wouldn't change any functionality, and would result in two instances of the same files. So just using the dockerfile in Complement's repo was decided upon instead.
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-182-2/+2
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Remove obsolete __future__ imports (#8337)Jonathan de Jong2020-09-174-8/+0
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-1/+1
|
* Fix typing for `@cached` wrapped functions (#8240)Erik Johnston2020-09-031-0/+85
| | | This requires adding a mypy plugin to fiddle with the type signatures a bit.
* Update the test federation client to handle streaming responses (#8130)Richard van der Hoff2020-08-261-8/+27
| | | | Now that the server supports streaming back JSON responses, it would be nice to show the response as it is streamed, in the test tool.
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-2/+2
|
* Spruce up the check-newsfragment CI output (#8024)Andrew Morgan2020-08-041-1/+12
| | | | | | This PR: * Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default. * Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.
* Fix exit code for `check_line_terminators.sh` (#7970)Richard van der Hoff2020-07-281-1/+4
| | | | | | | | | If there are *no* files with CRLF line endings, then the xargs exits with a non-zero exit code (as expected), but then, since that is the last thing to happen in the script, the script as a whole exits non-zero, making the whole thing fail. using `if/then/fi` instead of `&& (...)` means that the script exits with a zero exit code.
* Add script for finding files with unix line terminators (#7965)Andrew Morgan2020-07-281-0/+31
| | | | | This PRs adds a script to check for unix-line terminators in the repo. It will be used to address https://github.com/matrix-org/synapse/issues/7943 by adding the check to CI. I've changed the original script slightly as proposed in https://github.com/matrix-org/pipelines/pull/81#discussion_r460580664
* Lint the contrib/ directory in CI and linting scripts, add synctl to linting ↵Andrew Morgan2020-07-201-1/+1
| | | | | | | script (#7914) Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too. Fixes https://github.com/matrix-org/synapse/issues/7910
* Remove Ubuntu Eoan that is now EOL (#7888)Gary Kim2020-07-171-1/+0
|
* isort 5 compatibility (#7786)Will Hunt2020-07-052-2/+2
| | | The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
* Move flake8 to end. Don't exit script on failure (#7738)Andrew Morgan2020-06-251-2/+2
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+1
|
* Check the changelog number in check-newsfragment (#7623)Richard van der Hoff2020-06-031-4/+16
|
* Remove Ubuntu Cosmic and Disco which are both EOL. (#7539)Patrick Cloke2020-05-201-2/+0
|
* Merge branch 'master' into developPatrick Cloke2020-05-191-0/+1
|\
| * update dh-virtualenv (#7526)Richard van der Hoff2020-05-191-0/+1
| |
* | remove `builtins.buffer` code from storage codeRichard van der Hoff2020-05-151-7/+2
|/ | | | this is no longer needed on python 3
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-1/+1
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Use BSD-compatible in-place editing for sed. (#6887)Patrick Cloke2020-02-121-1/+2
|
* Port synapse_port_db to async/await (#6718)Brendan Abolivier2020-01-211-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if there are pending background updates So we return with a non-0 code * Changelog * Port synapse_port_db to async/await * Port update_database to async/await * Add version string to mocked homeservers * Remove unused imports * Convert overseen bits to async/await * Fixup logging contexts * Fix imports * Add a way to print an error without raising an exception * Incorporate review
* Automate generation of the sample and debian log configs (#6627)Richard van der Hoff2020-01-031-0/+10
|
* Add database config class (#6513)Erik Johnston2019-12-181-7/+2
| | | | | This encapsulates config for a given database and is the way to get new connections.
* Merge pull request #6511 from matrix-org/erikj/remove_db_config_from_appsErik Johnston2019-12-121-27/+6
|\ | | | | Move database config from apps into HomeServer object
| * Fix upgrade db scriptErik Johnston2019-12-101-27/+6
| |
* | Add dev script to generate full SQL schema files (#6394)Andrew Morgan2019-12-111-0/+184
|/
* Fix DB scriptsErik Johnston2019-12-051-10/+7
|
* Remove underscore from SQLBaseStore functionsErik Johnston2019-12-041-1/+1
|
* Merge branch 'master' into developRichard van der Hoff2019-11-061-0/+2
|\
| * build debs for eoan and bullseye github/release-v1.5.1 release-v1.5.1Richard van der Hoff2019-11-061-0/+2
| |
* | Document lint.sh & allow application to specified files only (#6312)Neil Pilgrim2019-11-011-3/+11
| |
* | Update CI to run isort on scripts and scripts-dev (#6270)Andrew Morgan2019-10-301-2/+1
| |
* | Add CI for synapse_port_db (#6140)Brendan Abolivier2019-10-281-0/+125
|/ | | | | | | | | | | | | | | | | | This adds: * a test sqlite database * a configuration file for the sqlite database * a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`) as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that: 1. installs Synapse 2. updates the test sqlite database to the latest schema and runs background updates on it 3. creates an empty postgresql database 4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage) Step `2` is done via a new script located at `scripts-dev/update_database`. The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.
* Add config linting script that checks for bool casing (#6203)Andrew Morgan2019-10-232-0/+10
| | | | | Add a linting script that enforces all boolean values in the default config be lowercase. This has annoyed me for a while so I decided to fix it.
* Remove dead check_auth scriptRichard van der Hoff2019-10-111-58/+0
| | | | This doesn't work, and afaict hasn't been used since 2015.
* Add a linting script (#5627)Andrew Morgan2019-07-101-0/+12
| | | Add a dev script to cover all the different linting steps.
* Remove & changelog (#5548)Amber Brown2019-06-261-54/+0
|
* Run Black. (#5482)Amber Brown2019-06-209-61/+54
|
* Updates to the federation_client script (#5447)Richard van der Hoff2019-06-131-5/+38
| | | | | * py3 fixes for federation_client * .well-known support for federation_client
* Unify v1 and v2 REST client APIs (#5226)Amber Brown2019-06-031-3/+1
|
* include disco in deb build target listRichard van der Hoff2019-05-031-0/+1
|
* Merge pull request #4869 from matrix-org/erikj/yaml_loadErik Johnston2019-03-221-1/+1
|\ | | | | Fix yaml warnings by using safe_load
| * Use yaml safe_loadErik Johnston2019-03-221-1/+1
| |
* | Allow newsfragments to end with exclamation marks! (#4912)Richard van der Hoff2019-03-211-2/+2
|/
* Fix check-newsfragment for debian-only changes. (#4825)Richard van der Hoff2019-03-071-7/+5
|
* Include a default configuration file in the 'docs' directory. (#4791)Richard van der Hoff2019-03-041-0/+18
|
* Fix check-newsfragment script (#4750)Richard van der Hoff2019-02-261-3/+8
| | | | | | | | | | * Fix check-newsfragment script I previously broke this so that it always succeeded... * more fixes * fix newsfiles
* Better checks on newsfragments (#4698)Richard van der Hoff2019-02-221-0/+36
| | | | | * You need an entry in the debian changelog (and not a regular newsfragment) for debian packaging changes. * Regular newsfragments must end in full stops.
* Support ACME for certificate provisioning (#4384)Amber Brown2019-01-231-1/+1
|
* Rewrite build_debian_packages github/release-v0.34.1.1 release-v0.34.1.1Richard van der Hoff2019-01-121-0/+154
| | | | Rewrite this in python so that it can be run in parallel.
* Merge pull request #4168 from ↵Brendan Abolivier2018-11-091-2/+7
|\ | | | | | | | | matrix-org/babolivier/federation-client-content-type Add a Content-Type header on POST requests to the federation client script
| * Add a Content-Type header on POST requests to the federation clientBrendan Abolivier2018-11-091-2/+7
| |
* | Delete the disused & unspecced identicon functionality (#4106)Amber Brown2018-10-291-39/+0
|/
* Fix a number of flake8 errorsRichard van der Hoff2018-10-241-1/+1
| | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.
* Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase ↵Amber Brown2018-10-2010-190/+222
| | | | on py3) (#4068)
* Remove Jenkins & other old dev junk (#3988)Amber Brown2018-10-032-66/+0
|
* Replaced all occurences of e.message with str(e)Schnuffle2018-09-271-1/+1
| | | | Signed-off-by: Schnuffle <schnuffle@github.com>
* Reduce the load on our CI (#3957)Amber Brown2018-09-271-0/+9
| | | | | | | | | | * changelog * reduce circleci config * plus a handy script * fix regex
* Remove nuke-room-from-db.sh scriptRichard van der Hoff2018-09-171-57/+0
| | | | | | | | | | | | The problem with this script is that it is largely untested, entirely unmaintained, and running it is likely to make your synapse blow up in exciting ways. For example, it leaves a bunch of tables with dead values in it, like event_to_state_groups. Having it here sends a message that it is a supported part of synapse, which is absolutely not the case.
* Fix federation_client to send the right HostRichard van der Hoff2018-06-121-14/+51
| | | | | | This appears to have stopped working since matrix.org moved to cloudflare. The Host header should match the name of the server, not whatever is in the SRV record.
* Merge pull request #3194 from rubo77/fix-nukeRichard van der Hoff2018-05-091-1/+1
|\ | | | | nuke-room-from-db.sh: fix deletion from search table
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-22 21:16:02 +0000