summary refs log tree commit diff
path: root/scripts-dev (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Support generating structured logs in addition to standard logs. (#8607)Patrick Cloke2020-10-291-1/+1
| | | | | | | This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.
* Run mypy as part of the lint.sh script. (#8633)kleph2020-10-281-0/+1
|
* Add option to scripts-dev/lint.sh to only lint files changed since the last ↵Andrew Morgan2020-10-151-9/+84
| | | | | | | | | | git commit (#8472) This PR makes several changes to the `./scripts-dev/lint.sh` script, which lints the codebase with a number of tools: * Adds usage information, with `-h` flag to show it. Otherwise it will show when providing an unknown flag. * Adds option `-d` which will check both staged and unstaged files that have changed since the last commit and add them to the list of files to lint. - Note that only files without an extension, or with a `.py` extension will be allowed. This prevents editing bash scripts causing the linters to break on non-python files. * Improves the print-out of which files/directories are being linted.
* Merge remote-tracking branch 'origin/release-v1.21.0' into developErik Johnston2020-10-071-0/+1
|\
| * Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475)Patrick Cloke2020-10-071-0/+1
| |
* | Remove docs/sphinx and related references (#8480)Andrew Morgan2020-10-071-1/+0
|/ | | https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...
* Add support for running Complement against the local checkout (#8317)Andrew Morgan2020-09-291-0/+22
| | | | | | | | | | | | This PR adds a script that: * Builds the local Synapse checkout using our existing `docker/Dockerfile` image. * Downloads [Complement](https://github.com/matrix-org/complement/)'s source code. * Builds the [Synapse.Dockerfile](https://github.com/matrix-org/complement/blob/master/dockerfiles/Synapse.Dockerfile) using the above dockerfile as a base. * Builds and runs Complement against it. This set up differs slightly from [that of the dendrite repo](https://github.com/matrix-org/dendrite/blob/master/build/scripts/complement.sh) (`complement.sh`, `Complement.Dockerfile`), which instead stores a separate, but slightly modified, dockerfile in Dendrite's repo rather than running the one stored in Complement's repo. That synapse equivalent to that dockerfile (`Synapse.Dockerfile`) in Complement's repo is just based on top of `matrixdotorg/synapse:latest`, which we opt to build here locally. Thus copying over the files from Complement's repo wouldn't change any functionality, and would result in two instances of the same files. So just using the dockerfile in Complement's repo was decided upon instead.
* Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-182-2/+2
| | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* Remove obsolete __future__ imports (#8337)Jonathan de Jong2020-09-174-8/+0
|
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-1/+1
|
* Fix typing for `@cached` wrapped functions (#8240)Erik Johnston2020-09-031-0/+85
| | | This requires adding a mypy plugin to fiddle with the type signatures a bit.
* Update the test federation client to handle streaming responses (#8130)Richard van der Hoff2020-08-261-8/+27
| | | | Now that the server supports streaming back JSON responses, it would be nice to show the response as it is streamed, in the test tool.
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-2/+2
|
* Spruce up the check-newsfragment CI output (#8024)Andrew Morgan2020-08-041-1/+12
| | | | | | This PR: * Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default. * Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.
* Fix exit code for `check_line_terminators.sh` (#7970)Richard van der Hoff2020-07-281-1/+4
| | | | | | | | | If there are *no* files with CRLF line endings, then the xargs exits with a non-zero exit code (as expected), but then, since that is the last thing to happen in the script, the script as a whole exits non-zero, making the whole thing fail. using `if/then/fi` instead of `&& (...)` means that the script exits with a zero exit code.
* Add script for finding files with unix line terminators (#7965)Andrew Morgan2020-07-281-0/+31
| | | | | This PRs adds a script to check for unix-line terminators in the repo. It will be used to address https://github.com/matrix-org/synapse/issues/7943 by adding the check to CI. I've changed the original script slightly as proposed in https://github.com/matrix-org/pipelines/pull/81#discussion_r460580664
* Lint the contrib/ directory in CI and linting scripts, add synctl to linting ↵Andrew Morgan2020-07-201-1/+1
| | | | | | | script (#7914) Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too. Fixes https://github.com/matrix-org/synapse/issues/7910
* Remove Ubuntu Eoan that is now EOL (#7888)Gary Kim2020-07-171-1/+0
|
* isort 5 compatibility (#7786)Will Hunt2020-07-052-2/+2
| | | The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
* Move flake8 to end. Don't exit script on failure (#7738)Andrew Morgan2020-06-251-2/+2
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+1
|
* Check the changelog number in check-newsfragment (#7623)Richard van der Hoff2020-06-031-4/+16
|
* Remove Ubuntu Cosmic and Disco which are both EOL. (#7539)Patrick Cloke2020-05-201-2/+0
|
* Merge branch 'master' into developPatrick Cloke2020-05-191-0/+1
|\
| * update dh-virtualenv (#7526)Richard van der Hoff2020-05-191-0/+1
| |
* | remove `builtins.buffer` code from storage codeRichard van der Hoff2020-05-151-7/+2
|/ | | | this is no longer needed on python 3
* Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-1/+1
| | | | Ensure good comprehension hygiene using flake8-comprehensions.
* Use BSD-compatible in-place editing for sed. (#6887)Patrick Cloke2020-02-121-1/+2
|
* Port synapse_port_db to async/await (#6718)Brendan Abolivier2020-01-211-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if there are pending background updates So we return with a non-0 code * Changelog * Port synapse_port_db to async/await * Port update_database to async/await * Add version string to mocked homeservers * Remove unused imports * Convert overseen bits to async/await * Fixup logging contexts * Fix imports * Add a way to print an error without raising an exception * Incorporate review
* Automate generation of the sample and debian log configs (#6627)Richard van der Hoff2020-01-031-0/+10
|
* Add database config class (#6513)Erik Johnston2019-12-181-7/+2
| | | | | This encapsulates config for a given database and is the way to get new connections.
* Merge pull request #6511 from matrix-org/erikj/remove_db_config_from_appsErik Johnston2019-12-121-27/+6
|\ | | | | Move database config from apps into HomeServer object
| * Fix upgrade db scriptErik Johnston2019-12-101-27/+6
| |
* | Add dev script to generate full SQL schema files (#6394)Andrew Morgan2019-12-111-0/+184
|/
* Fix DB scriptsErik Johnston2019-12-051-10/+7
|
* Remove underscore from SQLBaseStore functionsErik Johnston2019-12-041-1/+1
|
* Merge branch 'master' into developRichard van der Hoff2019-11-061-0/+2
|\
| * build debs for eoan and bullseye github/release-v1.5.1 release-v1.5.1Richard van der Hoff2019-11-061-0/+2
| |
* | Document lint.sh & allow application to specified files only (#6312)Neil Pilgrim2019-11-011-3/+11
| |
* | Update CI to run isort on scripts and scripts-dev (#6270)Andrew Morgan2019-10-301-2/+1
| |
* | Add CI for synapse_port_db (#6140)Brendan Abolivier2019-10-281-0/+125
|/ | | | | | | | | | | | | | | | | | This adds: * a test sqlite database * a configuration file for the sqlite database * a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`) as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that: 1. installs Synapse 2. updates the test sqlite database to the latest schema and runs background updates on it 3. creates an empty postgresql database 4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage) Step `2` is done via a new script located at `scripts-dev/update_database`. The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.
* Add config linting script that checks for bool casing (#6203)Andrew Morgan2019-10-232-0/+10
| | | | | Add a linting script that enforces all boolean values in the default config be lowercase. This has annoyed me for a while so I decided to fix it.
* Remove dead check_auth scriptRichard van der Hoff2019-10-111-58/+0
| | | | This doesn't work, and afaict hasn't been used since 2015.
* Add a linting script (#5627)Andrew Morgan2019-07-101-0/+12
| | | Add a dev script to cover all the different linting steps.
* Remove & changelog (#5548)Amber Brown2019-06-261-54/+0
|
* Run Black. (#5482)Amber Brown2019-06-209-61/+54
|
* Updates to the federation_client script (#5447)Richard van der Hoff2019-06-131-5/+38
| | | | | * py3 fixes for federation_client * .well-known support for federation_client
* Unify v1 and v2 REST client APIs (#5226)Amber Brown2019-06-031-3/+1
|
* include disco in deb build target listRichard van der Hoff2019-05-031-0/+1
|
* Merge pull request #4869 from matrix-org/erikj/yaml_loadErik Johnston2019-03-221-1/+1
|\ | | | | Fix yaml warnings by using safe_load
| * Use yaml safe_loadErik Johnston2019-03-221-1/+1
| |
* | Allow newsfragments to end with exclamation marks! (#4912)Richard van der Hoff2019-03-211-2/+2
|/
* Fix check-newsfragment for debian-only changes. (#4825)Richard van der Hoff2019-03-071-7/+5
|
* Include a default configuration file in the 'docs' directory. (#4791)Richard van der Hoff2019-03-041-0/+18
|
* Fix check-newsfragment script (#4750)Richard van der Hoff2019-02-261-3/+8
| | | | | | | | | | * Fix check-newsfragment script I previously broke this so that it always succeeded... * more fixes * fix newsfiles
* Better checks on newsfragments (#4698)Richard van der Hoff2019-02-221-0/+36
| | | | | * You need an entry in the debian changelog (and not a regular newsfragment) for debian packaging changes. * Regular newsfragments must end in full stops.
* Support ACME for certificate provisioning (#4384)Amber Brown2019-01-231-1/+1
|
* Rewrite build_debian_packages github/release-v0.34.1.1 release-v0.34.1.1Richard van der Hoff2019-01-121-0/+154
| | | | Rewrite this in python so that it can be run in parallel.
* Merge pull request #4168 from ↵Brendan Abolivier2018-11-091-2/+7
|\ | | | | | | | | matrix-org/babolivier/federation-client-content-type Add a Content-Type header on POST requests to the federation client script
| * Add a Content-Type header on POST requests to the federation clientBrendan Abolivier2018-11-091-2/+7
| |
* | Delete the disused & unspecced identicon functionality (#4106)Amber Brown2018-10-291-39/+0
|/
* Fix a number of flake8 errorsRichard van der Hoff2018-10-241-1/+1
| | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.
* Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase ↵Amber Brown2018-10-2010-190/+222
| | | | on py3) (#4068)
* Remove Jenkins & other old dev junk (#3988)Amber Brown2018-10-032-66/+0
|
* Replaced all occurences of e.message with str(e)Schnuffle2018-09-271-1/+1
| | | | Signed-off-by: Schnuffle <schnuffle@github.com>
* Reduce the load on our CI (#3957)Amber Brown2018-09-271-0/+9
| | | | | | | | | | * changelog * reduce circleci config * plus a handy script * fix regex
* Remove nuke-room-from-db.sh scriptRichard van der Hoff2018-09-171-57/+0
| | | | | | | | | | | | The problem with this script is that it is largely untested, entirely unmaintained, and running it is likely to make your synapse blow up in exciting ways. For example, it leaves a bunch of tables with dead values in it, like event_to_state_groups. Having it here sends a message that it is a supported part of synapse, which is absolutely not the case.
* Fix federation_client to send the right HostRichard van der Hoff2018-06-121-14/+51
| | | | | | This appears to have stopped working since matrix.org moved to cloudflare. The Host header should match the name of the server, not whatever is in the SRV record.
* Merge pull request #3194 from rubo77/fix-nukeRichard van der Hoff2018-05-091-1/+1
|\ | | | | nuke-room-from-db.sh: fix deletion from search table
| * nuke-room-from-db.sh: nuke from table event_search toorubo772018-05-091-0/+1
| |
| * Dont nuke non-existing table event_search_contentrubo772018-05-091-1/+0
| |
* | nuke-room-from-db.sh: added postgresql option and helprubo772018-05-091-1/+11
|/
* federation_client script: Support for posting contentRichard van der Hoff2017-11-281-8/+34
|
* test federation client: Allow server-name and key-file as optionsRichard van der Hoff2017-08-221-8/+28
| | | | so that you don't necessarily need a config file.
* Improvements to the federation test clientRichard van der Hoff2017-08-171-7/+58
| | | | Make it read the config file, primarily.
* bring nuke-room script to current schemaChristian W. Zuckschwerdt2017-02-191-10/+33
| | | | Signed-off-by: Christian W. Zuckschwerdt <christian@zuckschwerdt.org>
* Print newline after result in federation_client scriptErik Johnston2016-08-051-0/+1
|
* Print status code in federation_client.pyErik Johnston2016-08-031-0/+1
|
* Print authorization header for federation_client.pyErik Johnston2016-08-021-5/+6
|
* Add option to definitions.py to search for functions a function refers toMark Haines2016-03-171-1/+22
|
* Add a /replication API for extracting the updates that happened onMark Haines2016-03-011-0/+67
| | | | | | | | | | | | | | | | | | | | | | | | synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.
* Report the v1 and v2 patterns separatelyMark Haines2016-02-051-4/+12
|
* List the URL patterns in synapseMark Haines2016-02-051-0/+54
|
* Add macaroon inspection scriptDaniel Wagner-Hall2016-02-031-0/+24
|
* copyrightsMatthew Hodgson2016-01-072-4/+4
|
* Fix definitions scriptMark Haines2015-12-011-1/+1
|
* Add options to definitions.py to fetch referrers and to output dotMark Haines2015-12-011-6/+39
|
* Fix scripts-dev/definitions.py argparse optionsMark Haines2015-09-231-1/+1
|
* Use argparse for definition finderMark Haines2015-09-231-26/+38
|
* Add dev script for finding where functions are called from, and finding ↵Mark Haines2015-09-221-0/+130
| | | | functions that aren't called at all
* Don't require pdus in check_auth scriptErik Johnston2015-09-091-2/+1
|
* Replace syutil references in scriptsMark Haines2015-08-244-12/+10
|
* Add forgotten .items()Erik Johnston2015-05-191-1/+1
|
* Fix scripts-dev/convert_server_keys.py to have correct formatErik Johnston2015-05-191-1/+4
|
* SYN-376: Add script for converting server keys from v1 to v2Mark Haines2015-05-111-0/+113
|
* Acutally add scripts-dev/Erik Johnston2015-05-0611-0/+549