summary refs log tree commit diff
path: root/scripts (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Fixed code misc. quality issues (#9649)Ankit Dobhal2021-03-221-1/+1
| | | | | - Merge 'isinstance' calls. - Remove unnecessary dict call outside of comprehension. - Use 'sys.exit()' calls.
* Fix SQL delta file taking a long time to run (#9516)Erik Johnston2021-03-021-0/+2
| | | | Fixes #9504
* Include newly added sequences in the port DB script. (#9449)Patrick Cloke2021-02-231-24/+41
| | | And ensure the consistency of `event_auth_chain_id`.
* Add background update for add chain cover index (#9029)Erik Johnston2021-01-141-1/+1
|
* Allow running sendToDevice on workers (#9044)Erik Johnston2021-01-071-0/+27
|
* Fix generate_log_config script (#8952)Richard van der Hoff2020-12-161-1/+3
| | | It used to write an empty file if you gave it a -o arg.
* Fix port script so that it can be run again after failure. (#8755)Erik Johnston2020-11-131-30/+54
| | | | | | | If the script fails (or is CTRL-C'ed) between porting some of the events table and copying of the sequences then the port script will immediately die if run again due to the postgres DB having inconsistencies between sequences and tables. The fix is to move the porting of sequences to before porting the tables, so that there is never a period where the Postgres DB is inconsistent. To do that we need to change how we port the sequences so that it calculates the values from the SQLite DB rather than the Postgres DB. Fixes #8619
* Fix port script fails when DB has no backfilled events. (#8729)Erik Johnston2020-11-111-7/+5
| | | Fixes #8618
* Fix port script to handle foreign key constraints (#8730)Erik Johnston2020-11-111-6/+62
|
* Improve start time by adding index to e2e_cross_signing_keys (#8694)Erik Johnston2020-11-021-0/+2
| | | We do a `SELECT MAX(stream_id) FROM e2e_cross_signing_keys` on startup.
* Cast errors generated during synapse_port_db to str (#8585)Andrew Morgan2020-10-201-2/+3
| | | | | | | | | I noticed in https://github.com/matrix-org/synapse/issues/8575 that the `end_error` variable in `synapse_port_db` is set to an `Exception`, even though later we expect it to be a `str`. This PR simply casts an exception raised to a string. I'm doing this instead of having `end_error` be of type exception as we explicitly set `end_error` to a str here: https://github.com/matrix-org/synapse/blob/d25eb8f3709965d0face01a041d5292490bf0139/scripts/synapse_port_db#L542-L547 This whole file could probably use some heavy refactoring, but until then at least this fix will prevent exception contents from being hidden from us and users.
* Add support for MSC2732: olm fallback keys (#8312)Hubert Chathi2020-10-061-0/+1
|
* Add logging on startup/shutdown (#8448)Erik Johnston2020-10-021-1/+1
| | | | | This is so we can tell what is going on when things are taking a while to start up. The main change here is to ensure that transactions that are created during startup get correctly logged like normal transactions.
* Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410)Dagfinn Ilmari Mannsåker2020-09-281-0/+1
| | | | | | This table was created in #8034 (1.20.0). It references `ui_auth_sessions`, which is ignored, so this one should be too. Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
* Add new sequences to port DB script (#8387)Erik Johnston2020-09-241-0/+24
|
* Mark the shadow_banned column as boolean in synapse_port_db. (#8386)Patrick Cloke2020-09-241-0/+1
|
* Remove obsolete __future__ imports (#8337)Jonathan de Jong2020-09-172-4/+0
|
* Revert #7736 (#8039)Brendan Abolivier2020-08-061-1/+1
|
* Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-39/+39
|
* Re-implement unread counts (#7736)Brendan Abolivier2020-07-291-1/+1
|
* Use a postgres sequence to generate guest user IDsRichard van der Hoff2020-07-161-1/+11
|
* Sync ignored table names in synapse_port_db to current database schema (#7717)Patrick Cloke2020-06-231-4/+12
|
* Allow local media to be marked as safe from being quarantined. (#7718)Patrick Cloke2020-06-221-0/+1
|
* Ignore the UI Auth sessions when porting from sqlite to postgresql (#7711)Patrick Cloke2020-06-171-7/+12
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-3/+1
|
* Add option to move event persistence off master (#7517)Erik Johnston2020-05-221-0/+3
|
* Support any process writing to cache invalidation stream. (#7436)Erik Johnston2020-05-071-2/+2
|
* Avoid attribute error when `password_config` present but empty (#6753)Ivan Vilata-i-Balaguer2020-01-221-1/+1
| | | | | | | | | | The old statement returned `None` for such a `password_config` (like the one created on first run), thus retrieval of the `pepper` key failed with `AttributeError`. Fixes #5315 Signed-off-by: Ivan Vilata i Balaguer <ivan@selidor.net>
* Port synapse_port_db to async/await (#6718)Brendan Abolivier2020-01-211-82/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if there are pending background updates So we return with a non-0 code * Changelog * Port synapse_port_db to async/await * Port update_database to async/await * Add version string to mocked homeservers * Remove unused imports * Convert overseen bits to async/await * Fixup logging contexts * Fix imports * Add a way to print an error without raising an exception * Incorporate review
* Add `local_current_membership` table (#6655)Erik Johnston2020-01-151-1/+1
| | | | | | | Currently we rely on `current_state_events` to figure out what rooms a user was in and their last membership event in there. However, if the server leaves the room then the table may be cleaned up and that information is lost. So lets add a table that separately holds that information.
* Refuse to start if sqlite is older than 3.11.0Richard van der Hoff2020-01-091-4/+12
|
* Modify check_database to take a connection rather than a cursorRichard van der Hoff2020-01-091-18/+7
| | | | We might not need the cursor at all.
* Add a background update to clear tombstoned rooms from the directory (#6648)Richard van der Hoff2020-01-071-0/+5
| | | | | | * Add a background update to clear tombstoned rooms from the directory * use the ABC metaclass
* Automate generation of the sample and debian log configs (#6627)Richard van der Hoff2020-01-031-0/+43
|
* Split state groups into a separate data store (#6296)Erik Johnston2019-12-201-4/+4
|
* Add an export_signing_key script (#6546)Richard van der Hoff2019-12-191-0/+94
| | | | | I want to do some key rotation, and it is silly that we don't have a way to do this.
* Add database config class (#6513)Erik Johnston2019-12-181-34/+24
| | | | | This encapsulates config for a given database and is the way to get new connections.
* Fix port db scriptErik Johnston2019-12-061-34/+2
|
* Fix DB scriptsErik Johnston2019-12-051-5/+9
|
* Move DB pool and helper functions into dedicated Database classErik Johnston2019-12-051-36/+37
|
* Merge pull request #6464 from matrix-org/erikj/make_public_sql_baseErik Johnston2019-12-051-9/+9
|\ | | | | Clean up SQLBaseStore private function usage
| * Remove underscore from SQLBaseStore functionsErik Johnston2019-12-041-9/+9
| |
* | Fix background updates for synapse_port_dbBrendan Abolivier2019-12-041-0/+2
| |
* | Make synapse_port_db exit with a non-0 code if something failedBrendan Abolivier2019-12-041-0/+1
|/
* Fix error when using synapse_port_db on a vanilla synapse dbAndrew Morgan2019-12-021-1/+4
|
* Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-312-4/+4
| | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* Merge pull request #6247 from matrix-org/babolivier/port_db_columnBrendan Abolivier2019-10-241-0/+1
|\ | | | | Add new boolean column to synapse_port_db
| * Add new boolean column to synapse_port_dbBrendan Abolivier2019-10-241-0/+1
| |
* | Fix up changelog v1.5.0rc1Andrew Morgan2019-10-241-7/+9
|/
* Fix import paths in synapse_port_dbBrendan Abolivier2019-10-231-11/+19
|
* Make synapse_port_db correctly create indexes (#6102)Brendan Abolivier2019-10-231-52/+130
| | | | | | | Make `synapse_port_db` correctly create indexes in the PostgreSQL database, by having it run the background updates on the database before migrating the data. To ensure we're migrating the right data, also block the port if the SQLite3 database still has pending or ongoing background updates. Fixes #4877
* Add missing BOOLEAN_COLUMNs to synapse_port_db (#6216)Bart Noordervliet2019-10-181-0/+2
| | | | Small fix to synapse_port_db to be able to convert from database schema v56.
* Update the TLS cipher string and provide configurability for TLS on outgoing ↵Amber Brown2019-06-281-1/+1
| | | | federation (#5550)
* Run Black. (#5482)Amber Brown2019-06-202-6/+6
|
* Expose statistics on extrems to prometheus (#5384)Amber Brown2019-06-131-1/+1
|
* Merge pull request #5325 from matrix-org/babolivier/port_db_account_validityBrendan Abolivier2019-06-101-0/+1
|\ | | | | Add account_validity's email_sent column to the list of boolean columns in synapse_port_db
| * Add account_validity's email_sent column to the list of boolean columns in ↵Brendan Abolivier2019-06-031-0/+1
| | | | | | | | | | | | synapse_port_db Fixes #5306
* | add a script to generate new signing_key filesRichard van der Hoff2019-06-051-0/+37
|/
* drop tables listed in #1830 (#4992)Neil Johnson2019-04-081-5/+0
| | | | | | | | | | | | | | | Tables dropped: * application_services, * application_services_regex, * transaction_id_to_pdu, * stats_reporting * current_state_resets * event_content_hashes * event_destinations * event_edge_hashes * event_signatures * feedback * room_hosts * state_forward_extremities
* Fix issue #4596Anders2019-04-031-1/+1
| | | | | Make synapse_port_db --curses work with Python 3. Signed-off-by: Anders Jensen-Waud <anders@jensenwaud.com>
* Include a default configuration file in the 'docs' directory. (#4791)Richard van der Hoff2019-03-041-0/+11
|
* Consider e2e_room_keys.is_verified column as booleanEric2019-02-201-0/+1
| | | | | | | This column was considered as an int, crashing the whole migration process Signed-off-by: Eric <eric@pedr0.net>
* Add a script to generate a clean config file (#4315)Richard van der Hoff2018-12-221-0/+67
|
* Port hash_password to Python 3 (#4161)Amber Brown2018-11-081-7/+34
| | | | | | * port hash_password * changelog
* Port register_new_matrix_user to Python 3 and add tests (#4085)Amber Brown2018-10-261-202/+2
|
* Fix a number of flake8 errorsRichard van der Hoff2018-10-241-1/+2
| | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.
* Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase ↵Amber Brown2018-10-204-210/+179
| | | | on py3) (#4068)
* add --no-admin flag to registration script (#3836)Bruno Windels2018-09-281-4/+14
|
* Admin API for creating new users (#3415)Amber Brown2018-07-201-3/+29
|
* Replace some type checks with six type checksAdrian Tschira2018-04-071-1/+3
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Port script: Set up state_group_id_seqRichard van der Hoff2018-04-061-0/+13
| | | | Fixes https://github.com/matrix-org/synapse/issues/3050.
* Port script: clean up a bitRichard van der Hoff2018-04-061-25/+29
| | | | | Improve logging and comments. Group all the stuff to do with inspecting tables together rather than creating the port tables in the middle.
* Port script: avoid nasty errors when setting upRichard van der Hoff2018-04-061-8/+6
| | | | We really shouldn't spit out "Failed to create port table", it looks scary.
* Script to move remote media to another media storeRichard van der Hoff2018-01-311-0/+133
|
* sanity checksMatthew Hodgson2017-11-211-1/+4
|
* don't double-invite in sync_room_to_group.plMatthew Hodgson2017-11-191-3/+6
|
* quick and dirty room membership<->group membership sync scriptMatthew Hodgson2017-11-171-0/+39
|
* Less s'sErik Johnston2017-11-141-2/+2
|
* Add new boolean columns to port scriptErik Johnston2017-11-141-0/+7
|
* Make the port script drop NUL values in all tablesDavid Baker2017-10-311-7/+19
| | | | | | Postgres doesn't support NULs in strings so it makes the script throw an exception and stop if any values contain \0. Drop them with appropriate warning.
* Add groups table to BOOLEAN_COLUMNS in synapse_port_dbLuke Barnard2017-10-261-0/+1
|
* Fix port scriptErik Johnston2017-10-261-0/+1
| | | | | | | We changed _simple_update_one_txn to use _simple_update_txn but didn't yank it out in the port script. Fixes #2565
* Drop search values with nul charactersRichard van der Hoff2017-10-021-4/+7
| | | | | | https://github.com/matrix-org/synapse/issues/2187 contains a report of a port failing due to nul characters somewhere in the search table. Let's try dropping the offending rows.
* Extend commentErik Johnston2017-07-201-1/+2
|
* Fix port script for user directory tablesErik Johnston2017-07-201-0/+18
|
* add missing boolean to synapse_port_dbMatthew2017-06-161-0/+1
|
* fix synapse_port scriptMatthew2017-06-161-1/+1
|
* Don't assume postgres tables are in the public schema during db portJohn Kristensen2017-03-171-3/+1
| | | | | | | When fetching the list of tables from the postgres database during the db port, it is assumed that the tables are in the public schema. This is not always the case, so lets just rely on postgres to determine the default schema to use.
* Fix typoKevin Liu2017-02-111-1/+1
| | | This is what I get for not proofreading
* Fix synapse_port_db failure (fixes #1902)Kevin Liu2017-02-111-0/+1
| | | | | See https://matrix.to/#/!cURbafjkfsMDVwdRDQ:matrix.org/$148686272020hCgRD:potatofrom.space Signed-off-by: Kevin Liu <kevin@potatofrom.space>
* Update port script with recently added tablesErik Johnston2016-09-271-0/+9
| | | | | | This also fixes a bug where the port script would explode when it encountered the newly added boolean column `public_room_list_stream.visibility`
* Allow upgrading from old port_from_sqlite3 formatErik Johnston2016-08-041-0/+21
|
* Handle the fact that some tables have negative rowid rowsErik Johnston2016-08-041-43/+111
|
* Merge pull request #909 from matrix-org/erikj/shared_secretErik Johnston2016-07-061-6/+26
|\ | | | | Add an admin option to shared secret registration (breaks backwards compat)
| * Add null separator to hmacErik Johnston2016-07-061-0/+2
| |
| * Add an admin option to shared secret registrationErik Johnston2016-07-051-3/+16
| |
| * Protect password when registering using shared secretErik Johnston2016-07-051-3/+8
| |
* | Update hash_password scriptKent Shikama2016-07-061-1/+17
|/ | | | Signed-off-by: Kent Shikama <kent@kentshikama.com>
* Update port scriptErik Johnston2016-04-221-42/+81
|
* Don't require config to create databaseErik Johnston2016-04-061-8/+5
|
* Password reset docs and scriptRichard van der Hoff2016-03-162-1/+39
| | | | | Replace the bash/perl gen_password script with a python one, and write a note on how to use it.
* Mark AS users with their AS's IDDaniel Wagner-Hall2016-02-111-2/+7
|
* copyrightsMatthew Hodgson2016-01-072-2/+2
|
* throwaway 1-liner for generating password hashesMatthew Hodgson2015-12-101-0/+1
|
* SpellingErik Johnston2015-11-191-1/+1
|
* Fix database port script to work with new event_search tableErik Johnston2015-11-181-11/+44
|
* Remove unused _execute_and_decode from scripts/synapse_port_dbMark Haines2015-09-231-2/+0
|
* Merge branch 'master' into developMark Haines2015-09-071-6/+9
|\ | | | | | | | | Conflicts: setup.py
| * Make port script handle empty sent_transactions tableErik Johnston2015-09-021-6/+9
| |
* | update logger to match new ambiguous script name...Matthew Hodgson2015-09-031-1/+1
|/
* Clean up scripts/Erik Johnston2015-08-274-373/+0
|
* Revert register_new_matrix_user to use v1 apiErik Johnston2015-05-221-2/+3
|
* Support reading directly from a configErik Johnston2015-05-081-0/+3
|
* Fix port_from_sqlite_to_postgres after changes to storage layer.Erik Johnston2015-05-081-4/+2
|
* Re-add scripts/register_new_matrix_userErik Johnston2015-05-061-0/+153
|
* Seperate scripts/ into scripts/ and scripts-dev/, where scripts/* are ↵Erik Johnston2015-05-0613-550/+2
| | | | automatically added to the package
* Check that postgres database has correct charset setErik Johnston2015-04-291-0/+4
|
* Add better help to scripts/port_from_sqlite_to_postgres.pyErik Johnston2015-04-281-5/+27
|
* Quickly fix dodgy est. time remainingErik Johnston2015-04-281-7/+14
|
* Correctly handle total/remaining counts in the presence of ↵Erik Johnston2015-04-281-1/+1
| | | | sent_transasctions table
* Correctly handle total/remaining counts in the presence of ↵Erik Johnston2015-04-281-123/+167
| | | | sent_transasctions table
* Remove accidentally committed debug hardcode hackErik Johnston2015-04-281-1/+2
|
* Typo in port scriptErik Johnston2015-04-281-1/+1
|
* Split setuping up and processing of tablesErik Johnston2015-04-271-8/+23
|
* Shuffle progress stuffErik Johnston2015-04-271-201/+174
|
* Don't port over all of the sent_transactions tableErik Johnston2015-04-271-38/+120
|
* PEP8Erik Johnston2015-04-171-2/+1
|
* Fix port script after storage changes. Add very simple (off by default) ↵Erik Johnston2015-04-171-171/+409
| | | | curses UI to see progress.
* Remove mysql/maria supportErik Johnston2015-04-151-34/+24
|
* Change port script to work with postgresErik Johnston2015-04-151-47/+67
|
* Support running porting script multiple timesErik Johnston2015-04-131-8/+100
|
* Add beginnings of migration sqlite->maria db scriptErik Johnston2015-04-101-0/+271
|
* SYN-67: Finish up implementing new database schema managementErik Johnston2015-03-041-54/+0
|
* Run deltas and bump user_version in upgrade scriptKegan Dougal2015-03-021-5/+23
|
* Add upgrade scriptKegan Dougal2015-03-021-0/+36
|
* Correctly handle all the places that can throw exceptionsErik Johnston2015-02-121-2/+5
|
* add some options and docMatthew Hodgson2015-02-071-5/+20
|
* create identicons for new users by default as default avatars, and provide ↵Matthew Hodgson2015-02-071-0/+24
| | | | script to update existing avatarless users
* Print out the auth events on failureErik Johnston2015-02-051-1/+1
|
* Add script to check and auth chain and current state of a roomErik Johnston2015-02-041-0/+65
|
* Move scripts into scriptsMark Haines2015-01-225-0/+83
|
* Add copyrighter script for sqlMark Haines2015-01-121-0/+33
|
* Update copyright noticesMark Haines2015-01-061-1/+1
|
* Change upgrade script to not check hashes or signaturesErik Johnston2014-12-161-46/+53
|
* Fix upgrade script to run all the missing deltas.Erik Johnston2014-12-161-0/+20
|
* Bump UPGRADES and CHANGESErik Johnston2014-12-161-0/+2
|
* Fix bugs in upgrade script.Erik Johnston2014-12-161-1/+5
| | | | | Handle the case when there are colons in server_name. Handle http exceptions more gracefully.
* Rename upgrade scriptErik Johnston2014-12-161-0/+0
|
* Include database bump in upgrade scriptErik Johnston2014-12-161-0/+2
|
* Merge pull request #25 from matrix-org/events_refactorMark Haines2014-12-162-0/+299
|\ | | | | Event refactor
| * Update upgrade scriptErik Johnston2014-12-161-2/+11
| |
| * Finish up upgrade scriptErik Johnston2014-12-151-30/+94
| |
| * Add beginnings of upgrade scriptErik Johnston2014-12-151-0/+223
| |
| * Fix bug where we ignored event_edge_hashes tableErik Johnston2014-12-151-0/+3
| |
* | Add a script for talking matrix federation adding X-Matrix AuthorizationMark Haines2014-12-151-0/+143
|/ | | | headers.
* Use correct default port in scripts/check_signature.pyErik Johnston2014-11-261-1/+1
|
* Fix bugs in generating event signatures and hashingErik Johnston2014-11-032-5/+8
|
* Script for checking event hashesMark Haines2014-11-031-0/+43
|
* handle server names with embeded portsMark Haines2014-11-031-0/+4
|
* script for checking signatures on signed jsonMark Haines2014-11-031-0/+70
|
* Add script to hash exisitng historyMark Haines2014-10-171-0/+69
|
* move gendoc into matrix-doc projectMatthew Hodgson2014-10-173-794/+0
|
* target live siteMatthew Hodgson2014-09-041-2/+2
|
* fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-032-3/+5
| | | | hasn't been incorporated in time for launch.
* doc generatorMatthew Hodgson2014-09-033-0/+792
|
* add in copyrights to everything, not just the synapse subdir, and add a ↵Matthew Hodgson2014-08-131-0/+33
copyrighter.pl whilst we're at it
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-05 10:48:06 +0000