summary refs log tree commit diff
path: root/synapse/_scripts (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Delete server-side backup keys when deactivating an account. (#15181)Shay2023-04-041-0/+2
|
* Make cleaning up pushers depend on the device_id instead of the token_id ↵Quentin Gliech2023-03-241-1/+5
| | | | | | | | | | | | | | (#15280) This makes it so that we rely on the `device_id` to delete pushers on logout, instead of relying on the `access_token_id`. This ensures we're not removing pushers on token refresh, and prepares for a world without access token IDs (also known as the OIDC). This actually runs the `set_device_id_for_pushers` background update, which was forgotten in #13831. Note that for backwards compatibility it still deletes pushers based on the `access_token` until the background update finishes.
* Add a primitive helper script for listing worker endpoints. (#15243)reivilibre2023-03-231-0/+302
| | | | Co-authored-by: Patrick Cloke <patrickc@matrix.org>
* Add a check to SQLite port DB script to ensure that the sqlite database ↵Shay2023-03-221-0/+8
| | | | passed to the script exists before trying to port from it (#15306)
* Revert "check sqlite database file exists before porting/#14692" (#15301)Shay2023-03-211-1/+1
|
* Refactor media modules. (#15146)Patrick Cloke2023-02-271-1/+1
| | | | | | | * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-223-4/+0
|
* Teach portdb about `un_partial_stated_event_stream` (#15108)David Robertson2023-02-201-33/+52
| | | | | | | | | | | | * Sort BOOLEAN_COLUMNS and APPEND_ONLY_TABLES So I can see if a given table is present in logarithmic time, rather than linear. * Teach portdb about `un_partial_stated_event_streams` * Comments comments comments * Changelog
* Always notify replication when a stream advances (#14877)Erik Johnston2023-01-201-0/+4
| | | This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.
* Check sqlite database file exists before porting. (#14692)Jeyachandran Rathnam2022-12-221-1/+1
| | | | To avoid creating an empty SQLite file if the given path is incorrect.
* Fix a bug introduced in v1.67.0 where not specifying a config file or a ↵reivilibre2022-12-071-2/+3
| | | | server URL would lead to the `register_new_matrix_user` script failing. (#14637)
* let update_synapse_database run on a multi-database configurations (#13422)Finn2022-10-191-8/+0
| | | | | | | | | | | | | | | | | * Allow sharded database in db migrate script Signed-off-by: Finn Herzfeld <finn@beeper.com> * Update changelog.d/13422.bugfix Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Remove check entirely * remove unused import Signed-off-by: Finn Herzfeld <finn@beeper.com> Co-authored-by: finn <finn@beeper.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add an API for listing threads in a room. (#13394)Patrick Cloke2022-10-131-0/+2
| | | | | | | | | Implement the /threads endpoint from MSC3856. This is currently unstable and behind an experimental configuration flag. It includes a background update to backfill data, results from the /threads endpoint will be partial until that finishes.
* Allow admins to require a manual approval process before new accounts can be ↵Brendan Abolivier2022-09-291-1/+1
| | | | used (using MSC3866) (#13556)
* Support enabling/disabling pushers (from MSC3881) (#13799)Brendan Abolivier2022-09-211-0/+1
| | | Partial implementation of MSC3881
* Generate separate snapshots for logical databases (#13792)David Robertson2022-09-201-4/+10
| | | | | | | * Generate separate snapshots for sqlite, postgres and common * Cleanup postgres dbs in the TRAP * Say which logical DB we're applying updates to * Run background updates on the state DB * Add new option for accepting a SCHEMA_NUMBER
* Add receipts event stream ordering (#13703)Nick Mills-Barrett2022-09-131-0/+2
|
* Support `registration_shared_secret` in a file (#13614)Richard van der Hoff2022-08-251-3/+42
| | | | A new `registration_shared_secret_path` option. This is kinda handy for k8s deployments and things.
* register_new_matrix_user: read server url from config (#13616)Richard van der Hoff2022-08-251-6/+51
| | | | Fixes https://github.com/matrix-org/synapse/issues/3672: `https://localhost:8448` is virtually never right.
* Merge branch 'master' into developBrendan Abolivier2022-07-191-1/+1
|\
| * Remove 'anonymised' from the phone home stats documentation (#13321)Andrew Morgan2022-07-191-1/+1
| |
* | Drop unused tables from groups/communities. (#12967)Patrick Cloke2022-07-131-16/+0
|/ | | | These tables have been unused since Synapse v1.61.0, although schema version 72 was added in Synapse v1.62.0.
* Ensure portdb selects _all_ rows with negative rowids (#13226)David Robertson2022-07-111-1/+4
|
* Check that `auto_vacuum` is disabled when porting a SQLite database to ↵reivilibre2022-07-071-0/+34
| | | | Postgres, as `VACUUM`s must not be performed between runs of the script. (#13195)
* Merge remote-tracking branch 'origin/release-v1.62' into developPatrick Cloke2022-06-301-4/+2
|\
| * Add index to help delete old push actions (#13141)Erik Johnston2022-06-301-4/+2
| |
* | Improve performance of getting unread counts in rooms (#13119)Erik Johnston2022-06-291-0/+3
|/
* Speed up `get_unread_event_push_actions_by_room` (#13005)Erik Johnston2022-06-151-0/+4
| | | | | | | | | | | Fixes #11887 hopefully. The core change here is that `event_push_summary` now holds a summary of counts up until a much more recent point, meaning that the range of rows we need to count in `event_push_actions` is much smaller. This needs two major changes: 1. When we get a receipt we need to recalculate `event_push_summary` rather than just delete it 2. The logic for deleting `event_push_actions` is now divorced from calculating `event_push_summary`. In future it would be good to calculate `event_push_summary` while we persist a new event (it should just be a case of adding one to the relevant rows in `event_push_summary`), as that will further simplify the get counts logic and remove the need for us to periodically update `event_push_summary` in a background job.
* Replace noop background updates with DELETE. (#12954)Patrick Cloke2022-06-131-2/+0
| | | | Removes the `register_noop_background_update` and deletes the background updates directly in a delta file.
* Fix Synapse git info missing in version strings (#12973)David Robertson2022-06-072-8/+4
|
* Remove remaining pieces of groups code. (#12966)Patrick Cloke2022-06-061-0/+1
| | | | | * Remove an unused stream ID generator. * Remove the now unused remote profile cache.
* Remove groups code from synapse_port_db. (#12899)Patrick Cloke2022-06-031-8/+15
|
* Remove most groups datastore code. (#12895)Patrick Cloke2022-05-311-2/+2
| | | | The remaining piece is a background update that is needed for backwards compatibility.
* hash_password: raise an error if no config file is specified (#12789)Jae Lo Presti2022-05-191-5/+5
|
* Add missing type hints to config classes. (#12402)Patrick Cloke2022-04-111-3/+1
|
* Disallow untyped defs in synapse._scripts (#12422)David Robertson2022-04-1110-109/+185
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Of note: * No untyped defs in `register_new_matrix_user` This one might be contraversial. `request_registration` has three dependency-injection arguments used for testing. I'm removing the injection of the `requests` module and using `unitest.mock.patch` in the test cases instead. Doing `reveal_type(requests)` and `reveal_type(requests.get)` before the change: ``` synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "Any" synapse/_scripts/register_new_matrix_user.py:46: note: Revealed type is "Any" ``` And after: ``` synapse/_scripts/register_new_matrix_user.py:44: note: Revealed type is "types.ModuleType" synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "def (url: Union[builtins.str, builtins.bytes], params: Union[Union[_typeshed.SupportsItems[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], typing.Iterable[Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]]], builtins.str, builtins.bytes], None] =, data: Union[Any, None] =, headers: Union[Any, None] =, cookies: Union[Any, None] =, files: Union[Any, None] =, auth: Union[Any, None] =, timeout: Union[Any, None] =, allow_redirects: builtins.bool =, proxies: Union[Any, None] =, hooks: Union[Any, None] =, stream: Union[Any, None] =, verify: Union[Any, None] =, cert: Union[Any, None] =, json: Union[Any, None] =) -> requests.models.Response" ``` * Drive-by comment in `synapse.storage.types` * No untyped defs in `synapse_port_db` This was by far the most painful. I'm happy to break this up into smaller pieces for review if it's not managable as-is.
* Make `synapse._scripts` pass typechecks (#12421)David Robertson2022-04-084-38/+49
|
* Track device list updates per room. (#12321)Erik Johnston2022-04-041-0/+1
| | | | | | | | | | | | | | This is a first step in dealing with #7721. The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things: 1. we can defer calculating the set of remote servers that need to be poked about the change; and 2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed. However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled. There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly). Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table.
* Move synctl into `synapse._scripts` and expose as an entrypoint (#12140)David Robertson2022-03-041-0/+360
|
* Move scripts directory inside synapse, exposing as setuptools entry_points ↵David Robertson2022-03-028-0/+1851
| | | | | | | | | | | | (#12118) * Two scripts are basically entry_points already * Move and rename scripts/* to synapse/_scripts/*.py * Delete sync_room_to_group.pl * Expose entry points in setup.py * Update linter script and config * Fixup scripts & docs mentioning scripts that moved Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Add a flag to the `synapse_review_recent_signups` script to ignore and ↵lukasdenk2022-01-171-3/+15
| | | | filter appservice users. (#11675)
* Add type hints to synapse._scripts (#11297)Patrick Cloke2021-11-102-18/+28
|
* Fixed config parse bug in review_recent_signups (#11191)Samuel Philipp2021-10-271-2/+7
|
* Remove shebang line from module files (#10415)V024602021-07-291-1/+0
| | | Signed-off-by: Kai A. Hiller <V02460@gmail.com>
* Add script for getting info about recently registered users (#10290)Erik Johnston2021-07-061-0/+175
|
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Strip trailing / from server_url in register_new_matrix_user (#8823)Dmitry Borodaenko2020-11-261-1/+1
| | | | | | | | | When server URL provided to register_new_matrix_user includes path component (e.g. "http://localhost:8008/"), the command fails with "ERROR! Received 400 Bad Request". Stripping trailing slash from the server_url command argument makes sure combined endpoint URL remains valid. Signed-off-by: Dmitry Borodaenko angdraug@debian.org
* Remove deprecated `/_matrix/client/*/admin` endpoints (#8785)Dirk Klimpel2020-11-251-1/+1
| | | These are now only available via `/_synapse/admin/v1`.
* Remove obsolete __future__ imports (#8337)Jonathan de Jong2020-09-171-2/+0
|
* Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-2/+0
|
* Replace instance variations of homeserver with correct case/spacingAndrew Morgan2019-11-121-3/+3
|
* Run Black. (#5482)Amber Brown2019-06-201-11/+13
|
* Fix flake8 (#4519)Amber Brown2019-01-301-2/+2
|
* create support user (#4141)Neil Johnson2018-12-141-3/+16
| | | | | | Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
* Port register_new_matrix_user to Python 3 and add tests (#4085)Amber Brown2018-10-262-0/+215