summary refs log tree commit diff
path: root/tests (unfollow)
Commit message (Collapse)AuthorFilesLines
2021-11-30Merge trust_identity_server_for_password_resets PRs v1.48.0Brendan Abolivier1-2/+1
2021-11-30Move notices upBrendan Abolivier1-5/+4
2021-11-30Fixup changelogBrendan Abolivier1-1/+1
2021-11-301.48.0Brendan Abolivier3-1/+13
2021-11-25Fix docker hub name v1.48.0rc1Brendan Abolivier1-1/+1
2021-11-25Incorporate review from synapse-devBrendan Abolivier1-10/+12
2021-11-25Fixup changelogBrendan Abolivier1-13/+12
2021-11-251.48.0rc1Brendan Abolivier72-71/+90
2021-11-25Improve performance of `remove_{hidden,deleted}_devices_from_device_inbox` ↵Brendan Abolivier4-148/+84
(#11421) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-24Lower minumum batch size to 1 for background updates (#11422)Brendan Abolivier4-13/+23
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-23Rename unstable `access_token_lifetime` configuration option to ↵reivilibre6-16/+36
`refreshable_access_token_lifetime` to make it clear it only concerns refreshable access tokens. (#11388)
2021-11-23Add missing type hints to config base classes (#11377)Patrick Cloke13-109/+184
2021-11-23Remove code invalidated by deprecated config flag ↵Shay4-41/+4
'trust_identity_servers_for_password_resets' (#11395) * remove background update code related to deprecated config flag * changelog entry * update changelog * Delete 11394.removal Duplicate, wrong number * add no-op background update and change newfragment so it will be consolidated with associated work * remove unused code * Remove code associated with deprecated flag from legacy docker dynamic config file Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-23Refactor the code to inject bundled relations during serialization. (#11408)Patrick Cloke8-73/+92
2021-11-22Add config for customizing the claim used for JWT logins. (#11361)Kostas6-35/+57
Allows specifying a different claim (from the default "sub") to use when calculating the localpart of the Matrix ID used during the JWT login.
2021-11-22Store arbitrary relations from events. (#11391)Patrick Cloke6-45/+193
Instead of only known relation types. This also reworks the background update for thread relations to crawl events and search for any relation type, not just threaded relations.
2021-11-19Update README.mdShay1-2/+4
2021-11-19Add an admin API to run background jobs. (#11352)Dirk Klimpel9-43/+280
Instead of having admins poke into the database directly. Can currently run jobs to populate stats and to populate the user directory.
2021-11-19Add CVE number v1.47.1Sean Quah1-1/+1
2021-11-19Fix checking whether a room can be published on creation. (#11392)Patrick Cloke4-56/+95
If `room_list_publication_rules` was configured with a rule with a non-wildcard alias and a room was created with an alias then an internal server error would have been thrown. This fixes the error and properly applies the publication rules during room creation.
2021-11-19Refer to 1.47.1 without the vSean Quah1-1/+1
2021-11-19Update 1.47.1 release date in CHANGES.mdSean Quah1-1/+1
2021-11-191.47.1Sean Quah3-1/+30
2021-11-19Prevent the media store from writing outside of the configured directorySean Quah5-50/+483
Also tighten validation of server names by forbidding invalid characters in IPv6 addresses and empty domain labels.
2021-11-19Remove dead code from acme support. (#11393)Patrick Cloke2-50/+1
2021-11-19Remove msc2716 from the list of tests for complement. (#11389)Patrick Cloke2-1/+2
As the tests are currently failing and not run in CI.
2021-11-19Keep fallback key marked as used if it's re-uploaded (#11382)Hubert Chathi3-12/+72
2021-11-19Publish a `develop` docker image (#11380)Richard van der Hoff2-1/+5
I'd find it helpful to have a docker image corresponding to current develop, without having to build my own.
2021-11-19Fix verification of objects signed with old local keys (#11379)Richard van der Hoff3-31/+95
Fixes a bug introduced in #11129: objects signed by the local server, but with keys other than the current one, could not be successfully verified. We need to check the key id in the signature, and track down the right key.
2021-11-18Prevent historical state from being pushed to an application service via ↵Eric Eastwood4-2/+39
`/transactions` (MSC2716) (#11265) Mark historical state from the MSC2716 `/batch_send` endpoint as `historical` which makes it `backfilled` and have a negative `stream_ordering` so it doesn't get queried by `/transactions`. Fix https://github.com/matrix-org/synapse/issues/11241 Complement tests: https://github.com/matrix-org/complement/pull/221
2021-11-18Remove legacy code related to deprecated ↵Shay5-47/+18
`trust_identity_server_for_password_resets` config flag (#11333) * remove code legacy code related to deprecated config flag "trust_identity_server_for_password_resets" from synapse/config/emailconfig.py * remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/config/registration.py * remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/handlers/identity.py * add tests to ensure config error is thrown and synapse refuses to start when depreciated config flag is found * add changelog * slightly change behavior to only check for deprecated flag if set to 'true' * Update changelog.d/11333.misc Co-authored-by: reivilibre <oliverw@matrix.org> Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-18Add dedicated admin API for blocking a room (#11324)Dirk Klimpel6-0/+404
2021-11-18Add/Unerase annotations to Module API (#11341)reivilibre2-23/+135
2021-11-18Rename `get_refresh_token_for_user_id` to `create_refresh_token_for_user_id` ↵reivilibre3-3/+4
(#11370)
2021-11-18Use auto_attribs for RefreshTokenLookupResult (#11386)reivilibre2-7/+8
2021-11-18Do not allow MSC3440 threads to fork threads (#11161)Patrick Cloke4-8/+176
Adds validation to the Client-Server API to ensure that the potential thread head does not relate to another event already. This results in not allowing a thread to "fork" into other threads. If the target event is unknown for some reason (maybe it isn't visible to your homeserver), but is the target of other events it is assumed that the thread can be created from it. Otherwise, it is rejected as an unknown event.
2021-11-18Docs: Quote wildcard `federation_certificate_verification_whitelist` (#11381)Nicolai Søborg3-4/+5
Otherwise I get this beautiful stacktrace: ``` python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/root/synapse/synapse/app/homeserver.py", line 455, in <module> main() File "/root/synapse/synapse/app/homeserver.py", line 445, in main hs = setup(sys.argv[1:]) File "/root/synapse/synapse/app/homeserver.py", line 345, in setup config = HomeServerConfig.load_or_generate_config( File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config config_dict = read_config_files(config_files) File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files yaml_config = yaml.safe_load(file_stream) File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load return load(stream, SafeLoader) File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load return loader.get_single_data() File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data node = self.get_single_node() File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node document = self.compose_document() File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document node = self.compose_node(None, None) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node item_value = self.compose_node(node, item_key) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node node = self.compose_sequence_node(anchor) File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node while not self.check_event(SequenceEndEvent): File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event self.current_event = self.state() File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry return self.parse_block_sequence_entry() File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry if not self.check_token(BlockEntryToken, BlockEndToken): File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token self.fetch_more_tokens() File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens return self.fetch_alias() File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias self.tokens.append(self.scan_anchor(AliasToken)) File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor raise ScannerError("while scanning an %s" % name, start_mark, yaml.scanner.ScannerError: while scanning an alias in "/etc/matrix/homeserver.yaml", line 614, column 5 expected alphabetic or numeric character, but found '.' in "/etc/matrix/homeserver.yaml", line 614, column 6 ``` Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
2021-11-17Add type annotations to `synapse.metrics` (#10847)Sean Quah12-85/+173
2021-11-17Add support for `/_matrix/media/v3` APIs (#11371)Aaron R6-51/+60
* Add support for `/_matrix/media/v3` APIs Signed-off-by: Aaron Raimist <aaron@raim.ist> * Update `workers.md` to use v3 client and media APIs Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-17Rename `get_access_token_for_user_id` method to ↵reivilibre7-15/+16
`create_access_token_for_user_id` (#11369)
2021-11-17Fix running complement.sh script. (#11368)Patrick Cloke2-3/+4
By reverting changes from #11166 in this script. Specifically commit 13f084eb58c379b2fc073680818d5931e94626f4.
2021-11-171.47.0 v1.47.0David Robertson3-1/+13
2021-11-16Fix incorrect return value in tests. (#11359)Patrick Cloke2-1/+2
2021-11-16Avoid sharing room hierarchy responses between users (#11355)Sean Quah3-2/+65
Different users may be allowed to see different rooms within a space, so sharing responses between users is inadvisable.
2021-11-16fix up changelog language v1.47.0rc3Andrew Morgan1-1/+1
2021-11-16mark the migration file migration as a bugAndrew Morgan1-6/+1
2021-11-161.47.0rc3Andrew Morgan6-4/+22
2021-11-16Add support for `/_matrix/client/v3` APIs (#11318)Aaron R4-3/+5
This is one of the changes required to support Matrix 1.1 Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-16Add most missing type hints to synapse.util (#11328)Patrick Cloke10-165/+161
2021-11-16Rename `remove_deleted_devices_from_device_inbox` to ensure it is always run ↵Andrew Morgan2-1/+14
(#11353) Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-16Properly register all callback hooks for legacy password authentication ↵reivilibre2-13/+14
providers (#11340) Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-16Add ability to un-shadow-ban via the admin API. (#11347)Patrick Cloke5-12/+53
2021-11-16Misc typing fixes for tests, part 2 of N (#11330)David Robertson5-29/+66
2021-11-16change 'Home Server' to one word 'homeserver' (#11320)Ashwin Nair4-15/+16
Signed-off-by: Ashwin S. Nair <58840757+Ashwin-exe@users.noreply.github.com>
2021-11-15Run _upgrade_existing_database on workers if at current schema_version (#11346)Andrew Morgan3-19/+74
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-15Database storage profile passes mypy (#11342)David Robertson4-8/+21
It already seems to pass mypy. I wonder what changed, given that it was on the exclusion list. So this commit consists of me ensuring `--disallow-untyped-defs` passes and a minor fixup to a function that returned either `True` or `None`.
2021-11-15Get directory db file to pass mypy (#11339)David Robertson4-7/+11
2021-11-15Move sql file for `remove_deleted_devices_from_device_inbox` into v65 (#11303)Dirk Klimpel2-1/+2
2021-11-15Add support for the stable version of MSC2778 (#11335)Tulir Asokan4-4/+16
* Add support for the stable version of MSC2778 Signed-off-by: Tulir Asokan <tulir@maunium.net> * Expect m.login.application_service in login and password provider tests Signed-off-by: Tulir Asokan <tulir@maunium.net>
2021-11-12Remove unused tables `room_stats_historical` and `user_stats_historical` ↵Shay3-1/+24
(#11280) * remove unused tables room_stats_historical and user_stats_historical * update changelog number * Bump schema compat version comment * make linter happy * Update comment to give more info Co-authored-by: reivilibre <oliverw@matrix.org> Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-12Annotations for state_deltas.py (#11316)David Robertson3-4/+17
I was sad that I couldn't do better for `_curr_state_delta_stream_cache`. At least it's explicitly called out in a comment with #TODO.
2021-11-12Rollback #11322 due to wrong syntax in mypy.ini. (#11332)Patrick Cloke2-2/+15
This was only checking the __init__ files in modules instead of all files in a module, which don't pass yet.
2021-11-12Test room alias deletion (#11327)David Robertson5-31/+91
* Prefer `HTTPStatus` over plain `int` This is an Opinion that no-one has seemed to object to yet. * `--disallow-untyped-defs` for `tests.rest.client.test_directory` * Improve synapse's annotations for deleting aliases * Test case for deleting a room alias * Changelog
2021-11-12Change display names/avatar URLs to None if they contain null bytes before ↵Shay3-4/+56
storing in DB (#11230) * change display names/avatar URLS to None if they contain null bytes * add changelog * add POC test, requested changes * add a saner test and remove old one * update test to verify that display name has been changed to None * make test less fragile
2021-11-12Add type hints to media repository storage module (#11311)Patrick Cloke4-62/+89
2021-11-12Attempt to annotate events_forward_extremities (#11314)David Robertson4-8/+17
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore the former implicitly inherits from the latter, so they should be ordered like this when used.
2021-11-12Misc typing fixes for `tests`, part 1 of N (#11323)David Robertson7-29/+53
* Annotate HomeserverTestCase.servlets * Correct annotation of federation_auth_origin * Use AnyStr custom_headers instead of a Union This allows (str, str) and (bytes, bytes). This disallows (str, bytes) and (bytes, str) * DomainSpecificString.SIGIL is a ClassVar
2021-11-12Generalize the disallowed_untyped_defs in mypy.ini (#11322)Patrick Cloke2-14/+3
2021-11-12Add more type hints to synapse.util. (#11321)Patrick Cloke3-15/+24
2021-11-12Suggest using `/etc/matrix-synapse/conf.d/` for configuration with Debian ↵reivilibre2-0/+7
packages (#11281)
2021-11-12Correct target of link to the modules page from the Password Auth Providers ↵reivilibre1-1/+1
page (#11309)
2021-11-12Convert delete room admin API to async endpoint (#11223)Dirk Klimpel9-91/+1317
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-11Annotations for user_erasure_store (#11313)David Robertson3-5/+9
I'm not sure why this was excluded---it seemed to be passing for me. But it's easy enough to fixup.
2021-11-11Get db signatures file to pass mypy (#11312)David Robertson4-10/+8
2021-11-11Correct type hint for room_batch.py (#11310)David Robertson3-11/+12
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11Move sql file for `remove_deleted_devices_from_device_inbox` into v65 (#11303)Dirk Klimpel2-1/+2
2021-11-11Add type hints to some storage classes (#11307)Patrick Cloke9-54/+116
2021-11-10Fix error in thumbnail generation (#11288)Neeeflix2-4/+5
Signed-off-by: Jonas Zeunert <jonas@zeunert.org>
2021-11-10Add missing type hints to `synapse.app`. (#11287)Patrick Cloke17-190/+223
2021-11-10Add type hints to synapse._scripts (#11297)Patrick Cloke5-20/+30
2021-11-10Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298)Andrew Morgan5-3/+25
2021-11-10Remove redundant parameters on `_check_event_auth` (#11292)Richard van der Hoff3-12/+1
as of #11012, these parameters are unused.
2021-11-10Changelog tweak from feedback v1.47.0rc2Olivier Wilkinson (reivilibre)1-1/+1
2021-11-10Move Debian changelog entries to rc2 since rc1 was not publishedOlivier Wilkinson (reivilibre)1-7/+4
2021-11-101.47.0rc2Olivier Wilkinson (reivilibre)3-1/+14
2021-11-10Correct the Debian changelogOlivier Wilkinson (reivilibre)1-1/+1
2021-11-09Fix typos in the `username_available` admin API documentation. (#11286)Stanislav Motylkov2-1/+2
2021-11-09Split out federated PDU retrieval into a non-cached version (#11242)Eric Eastwood2-22/+59
Context: https://github.com/matrix-org/synapse/pull/11114/files#r741643968
2021-11-09Ignore missing imports for parameterized. (#11285)Patrick Cloke2-0/+4
This was due to a conflict between #11282, which changed mypy configuration, and #11228, a normal change.
2021-11-09Require mypy for synapse/ & tests/ unless excluded (#11282)Dan Callahan3-77/+168
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-09Clarifications and small fixes to to-device related code (#11247)Andrew Morgan6-17/+78
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09Update __init__.py v1.47.0rc1Olivier Wilkinson (reivilibre)1-1/+1
2021-11-09Changelog tweaks from reviewOlivier Wilkinson (reivilibre)1-5/+5
2021-11-09Allow admins to proactively block rooms (#11228)David Robertson6-21/+103
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-09Support filtering by relations per MSC3440 (#11236)Patrick Cloke15-110/+680
Adds experimental support for `relation_types` and `relation_senders` fields for filters.
2021-11-09Make Deprecations and Removals more prominentOlivier Wilkinson (reivilibre)1-7/+7
2021-11-09Changelog tweaksOlivier Wilkinson (reivilibre)1-6/+6
2021-11-091.47.0rc1Olivier Wilkinson (reivilibre)59-59/+90
2021-11-09Fix typo in `RelationAggregationPaginationServlet` error response (#11278)Andrew Morgan2-1/+4
2021-11-09Include cross-signing signatures when syncing remote devices for the first ↵Erik Johnston3-86/+277
time (#11234) When fetching remote devices for the first time, we did not correctly include the cross signing keys in the returned results. c.f. #11159
2021-11-09Require body for read receipts with user-agent exceptions (#11157)rogersheu3-3/+40
Co-authored-by: reivilibre <olivier@librepush.net>
2021-11-08Rename to more clear `get_insertion_event_id_by_batch_id` (MSC2716) (#11244)Eric Eastwood4-3/+4
`get_insertion_event_by_batch_id` -> `get_insertion_event_id_by_batch_id` Split out from https://github.com/matrix-org/synapse/pull/11114
2021-11-08Add some background update admin APIs (#11263)Erik Johnston8-18/+468
Fixes #11259
2021-11-08Fix typo in comment from #11255. (#11276)Patrick Cloke2-1/+2
2021-11-08Default value for `public_baseurl` (#11210)Richard van der Hoff16-73/+62
We might as well use a default value for `public_baseurl` based on `server_name` - in many cases, it will be correct.
2021-11-08Address review feedback from #11269 (#11273)Dan Callahan2-2/+3
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-08Blacklist new sytest validation test (#11270)Erik Johnston2-0/+4
2021-11-08Wrong DTLS port in "Troubleshooting" (#11268)jmcparland1-1/+1
Port 5349, not 5479.
2021-11-08Handle federation inbound instances being killed more gracefully (#11262)Erik Johnston3-10/+27
* Make lock better handle process being killed If the process gets killed and restarted (so that it didn't have a chance to drop its locks gracefully) then there may still be locks in the DB that are for the same instance that haven't yet timed out but are safe to delete. We handle this case by a) checking if the current instance already has taken out the lock, and b) if not then ignoring locks that are for the same instance. * Periodically check for old staged events This is to protect against other instances dying and their locks timing out.
2021-11-07Minor cleanup to Debian packaging (#11269)Dan Callahan11-53/+10
* Remove unused Vagrant scripts * Change package Architecture to any * Preinstall the wheel package when building venvs. Addresses the following warnings during Debian builds: Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed. Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed. Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed. Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed. Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed. Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed. Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed. Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed. * Allow /etc/default/matrix-synapse to be missing Per the systemd.exec manpage, prefixing an EnvironmentFile with "-": > indicates that if the file does not exist, it will not be read and no > error or warning message is logged. Signed-off-by: Dan Callahan <danc@element.io>
2021-11-05Add doc to integrate synapse with LemonLDAP OIDC (#11257)Julian2-0/+39
Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>
2021-11-05Fix rolling back when using workers (#11255)Erik Johnston3-11/+82
Fixes #11252
2021-11-04Make minor correction to type of auth_checkers callbacks (#11253)reivilibre3-2/+5
2021-11-04Additional test for `cachedList` (#11246)Richard van der Hoff2-0/+44
I was trying to understand how `cachedList` works, and ended up writing this extra test. I figure we may as well keep it.
2021-11-04Track ongoing event fetches correctly in the presence of failure (#11240)Sean Quah2-22/+35
When an event fetcher aborts due to an exception, `_event_fetch_ongoing` must be decremented, otherwise the event fetcher would never be replaced. If enough event fetchers were to fail, no more events would be fetched and requests would get stuck waiting for events.
2021-11-03Add a linearizer on (appservice, stream) when handling ephemeral events. ↵Nick Barrett3-18/+103
(#11207) Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-03Enable passing typing stream writers as a list. (#11237)Nick Barrett8-16/+24
This makes the typing stream writer config match the other stream writers that only currently support a single worker.
2021-11-03Remove a debug statement from tests. (#11239)Patrick Cloke2-1/+1
2021-11-03Add twine and towncrier as dev dependencies (#11233)Erik Johnston2-0/+4
We don't pin them as we execute them as commands, rather than use them as libs.
2021-11-03fix a small typo in the delete room api docsAndrew Morgan1-1/+1
2021-11-03Support sending no `state_events_at_start` in the MSC2716 `/batch_send` ↵Eric Eastwood2-12/+18
endpoint (#11188) As brought up by @tulir, https://matrix.to/#/!SBYNQlpqkwJzFIdzxI:nevarro.space/$Gwnb2ZvXHc3poYXuBhho0cmoYq4KJ11Jh3m5s8kjNOM?via=nevarro.space&via=beeper.com&via=matrix.org This use case only works if the user is already joined in the current room state at the given `?prev_event_id`
2021-11-02Add index to `local_group_updates.stream_id` (#11231)Erik Johnston4-1/+37
This should speed up startup times and generally increase performance of groups.
2021-11-02Add remaining type hints to `synapse.events`. (#11098)Patrick Cloke15-110/+185
2021-11-02Update changelog v1.46.0Erik Johnston1-2/+2
2021-11-021.46.0Erik Johnston4-4/+16
2021-11-02Delete messages for hidden devices from `device_inbox` (#11199)Dirk Klimpel4-0/+186
2021-11-02Fix providing a `RoomStreamToken` instance to ↵Andrew Morgan5-37/+30
`_notify_app_services_ephemeral` (#11137) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-02Add search by room ID and room alias to List Room admin API (#11099)Dirk Klimpel4-53/+76
Fixes: #10874 Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-02ObservableDeferred: run observers in order (#11229)Richard van der Hoff4-20/+88
2021-11-01Update outdated links in `PULL_REQUEST_TEMPLATE.md` (#11225)Dirk Klimpel2-4/+6
2021-11-01Handle missing Content-Type header when accessing remote media (#11200)Shay4-4/+29
* add code to handle missing content-type header and a test to verify that it works * add handling for missing content-type in the /upload endpoint as well * slightly refactor test code to put private method in approriate place * handle possible null value for content-type when pulling from the local db * add changelog * refactor test and add code to handle missing content-type in cached remote media * requested changes * Update changelog.d/11200.bugfix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01Add `use_float=true` to ijson calls in Synapse (#11217)Shay2-0/+4
* add use_float=true to ijson calls * lints * add changelog * Update changelog.d/11217.bugfix Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-01Fix a bug in unit test `test_block_room_and_not_purge` (#11226)Dirk Klimpel2-2/+3
2021-11-01Make `check_event_allowed` module API callback not fail open (accept events) ↵reivilibre5-17/+24
when an exception is raised (#11033)
2021-11-01Remove deprecated delete room admin API (#11213)Dirk Klimpel6-124/+79
Remove deprecated delete room admin API, `POST /_synapse/admin/v1/rooms/<room_id>/delete`
2021-11-01Support for serving server well-known files (#11211)Richard van der Hoff8-47/+159
Fixes https://github.com/matrix-org/synapse/issues/8308
2021-11-01Add domain specific matching for haproxy config (#11128)Brett Bethke2-1/+2
2021-11-01Docker: avoid changing userid unnecessarily (#11209)Richard van der Hoff3-23/+28
* Docker image: avoid changing user during `generate` The intention was always that the config files get written as the initial user (normally root) - only the data directory needs to be writable by Synapse. This got changed in https://github.com/matrix-org/synapse/pull/5970, but that seems to have been a mistake. * Avoid changing user if no explicit UID is given * changelog
2021-11-01Support Client-Server API r0.6.1 (#11097)Aaron R2-0/+2
Fixes #11064 Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-01Improve code formatting and fix a few typos in docs (#11221)Sumner Evans20-168/+233
* Labeled a lot more code blocks with the appropriate type * Fixed a couple of minor typos (missing/extraneous commas) Signed-off-by: Sumner Evans <me@sumnerevans.com>
2021-11-01 Add metrics to the threadpools (#11178)Erik Johnston4-1/+49
2021-11-01Test that `ClientIpStore` combines database and in-memory data correctly ↵Sean Quah2-0/+207
(#11179)
2021-11-01Stop synapse from saving messages in device_inbox for hidden devices. (#10097)JohannesKleine2-2/+7
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-01Deprecate user_may_create_room_with_invites (#11206)Brendan Abolivier3-36/+14
2021-10-29Fix comments referencing v1.46.0 from PR #10969. (#11212)Dirk Klimpel3-2/+3
#10969 was merged after 1.46.0rc1 was cut and will be included in v1.47.0rc1 instead.
2021-10-29Add a module API method to retrieve state from a room (#11204)Brendan Abolivier3-1/+74
2021-10-29Clarify lack of Windows support in documentation (#11198)Sean Quah3-12/+14
2021-10-28Additional type hints for relations database class. (#11205)Patrick Cloke3-15/+25
2021-10-28Add knock information in admin exported data (#11171)Rafael Gonçalves5-2/+99
Signed-off-by: Rafael Goncalves <rafaelgoncalves@riseup.net>
2021-10-28Fetch verify key locally rather than trying to do so over federation if ↵Shay3-29/+58
origin and host are the same. (#11129) * add tests for fetching key locally * add logic to check if origin server is same as host and fetch verify key locally rather than over federation * add changelog * slight refactor, add docstring, change changelog entry * Make changelog entry one line * remove verify_json_locally and push locality check to process_request, add function process_request_locally * remove leftover code reference * refactor to add common call to 'verify_json and associated handling code * add type hint to process_json * add some docstrings + very slight refactor
2021-10-28Add a ModuleApi method to update a user's membership in a room (#11147)Brendan Abolivier3-2/+225
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28Type hints for the remaining two files in `synapse.http`. (#11164)David Robertson6-37/+56
* Teach MyPy that the sentinel context is False This means that if `ctx: LoggingContextOrSentinel` then `bool(ctx)` narrows us to `ctx:LoggingContext`, which is a really neat find! * Annotate RequestMetrics - Raise errors for sentry if we use the sentinel context - Ensure we don't raise an error and carry on, but not recording stats - Include stack trace in the error case to lower Sean's blood pressure * Make mypy pass for synapse.http.request_metrics * Make synapse.http.connectproxyclient pass mypy Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28docs/openid: Add Authentik documentation. (#11151)Skyler Mäntysaari2-0/+35
2021-10-27Annotate `log_function` decorator (#10943)reivilibre12-18/+58
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-27Fixed config parse bug in review_recent_signups (#11191)Samuel Philipp2-2/+8
2021-10-27Force deb compression with `xz`. (#11197)Richard van der Hoff2-0/+12
Fixes a problem where `impish` packages could not be processed by `reprepro`.
2021-10-27Refactor `Filter` to handle fields according to data being filtered. (#11194)Patrick Cloke5-69/+87
This avoids filtering against fields which cannot exist on an event source. E.g. presence updates don't have a room.
2021-10-27Delete messages from `device_inbox` table when deleting device (#10969)Dirk Klimpel6-15/+256
Fixes: #9346
2021-10-27Fix URL preview errors when previewing XML documents. (#11196)Patrick Cloke3-3/+22
2021-10-27Include the stable identifier for MSC3288. (#11187)Patrick Cloke2-0/+3
Includes both the stable and unstable identifier to store-invite calls to the identity server. In the future we should remove the unstable identifier.
2021-10-27Update release dateErik Johnston1-1/+1
2021-10-27Update changelog with new changes v1.46.0rc1Erik Johnston3-4/+2
2021-10-27Shut down the DNS threadpool (#11190)Sean Quah2-0/+2
The DNS threadpool must be explicitly stopped, otherwise Synapse will hang indefinitely when asked to shut down.
2021-10-27Fix thread BG update to not seq scan event_json (#11192)Erik Johnston2-1/+2
For some reason the query optimiser decided to seq scan both tables, rather than index scanning `event_json`.
2021-10-26Update CHANGES.mdSean Quah1-16/+8
2021-10-26Move #10975 to bugfix section in changelogSean Quah1-1/+1
2021-10-26Update CHANGES.mdSean Quah1-3/+3
2021-10-26Implement an `on_new_event` callback (#11126)Brendan Abolivier8-12/+165
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-26Update CHANGES.mdSean Quah1-2/+4
2021-10-261.46.0rc1Sean Quah61-59/+81
2021-10-26Move DNS lookups into separate thread pool (#11177)Erik Johnston3-1/+149
This is to stop large bursts of lookups starving out other users of the thread pools. Fixes #11049.
2021-10-26Add a background update for updating MSC3440 relation threads. (#11181)Patrick Cloke3-2/+102
2021-10-26Document the version each module API method was added to Synapse (#11183)Brendan Abolivier2-11/+89
2021-10-26Enable changing user type via users admin API (#11174)Jason Robinson5-2/+80
Users admin API can now also modify user type in addition to allowing it to be set on user creation. Signed-off-by: Jason Robinson <jasonr@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-25Fix cyclic import in the module API (#11180)Brendan Abolivier2-2/+5
Introduced in #10548 See https://github.com/matrix-org/synapse-email-account-validity/runs/3979337154?check_suite_focus=true for an example of a module's CI choking over this issue.
2021-10-25Don't set new room alias before potential 403 (#10930)AndrewFerr4-12/+113
Fixes: #10929 Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-10-25Ensure that we correctly auth events returned by `send_join` (#11012)Richard van der Hoff2-86/+61
This is the final piece of the jigsaw for #9595. As with other changes before this one (eg #10771), we need to make sure that we auth the auth events in the right order, and actually check that their predecessors haven't been rejected. To do this I've reused the existing code we use when persisting outliers elsewhere. I've removed the code for attempting to fetch missing auth_events - the events should have been present in the send_join response, so the likely reason they are missing is that we couldn't verify them, so requesting them again is unlikely to help. Instead, we simply drop any state which relies on those auth events, as we do at a backwards-extremity. See also matrix-org/complement#216 for a test for this.
2021-10-25Fix module API's `get_user_ip_and_agents` function when run on workers (#11112)Sean Quah3-40/+91
2021-10-22ChangelogDan Callahan2-0/+7
Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2006: Use $(...) notationDan Callahan10-15/+15
Use $(...) notation instead of legacy backticked `...`. https://github.com/koalaman/shellcheck/wiki/SC2006 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2129: Consider using {..} >> fileDan Callahan1-60/+63
Consider using { cmd1; cmd2; } >> file instead of individual redirects. https://github.com/koalaman/shellcheck/wiki/SC2129 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2086: Quote to prevent splittingDan Callahan13-42/+42
Double quote to prevent globbing and word splitting. https://github.com/koalaman/shellcheck/wiki/SC2086 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2012: Use find instead of lsDan Callahan1-1/+1
Use find instead of ls to better handle non-alphanumeric filenames. https://github.com/koalaman/shellcheck/wiki/SC2012 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2016: Single quotes don't expandDan Callahan1-1/+2
Expressions don't expand in single quotes, use double quotes for that. https://github.com/koalaman/shellcheck/wiki/SC2016 This specifically warned about the '$aregis...' part of the sed script. Which is a relatively obscure use of sed. Splitting this into two commands makes its intent more obvious and avoids contravening Shellcheck's lints. Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC1091: Can't follow fileDan Callahan2-0/+2
Not following: (error message here) https://github.com/koalaman/shellcheck/wiki/SC1091 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC1001: Meaningless char escapesDan Callahan1-1/+1
This \o will be a regular 'o' in this context. https://github.com/koalaman/shellcheck/wiki/SC1001 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2089 and SC2090: Quotes in varsDan Callahan1-2/+4
SC2089: Quotes/backslashes will be treated literally. Use an array. https://github.com/koalaman/shellcheck/wiki/SC2089 SC2090: Quotes/backslashes in this variable will not be respected. https://github.com/koalaman/shellcheck/wiki/SC2090 Putting literal JSON in a variable mistakenly triggers these warnings. Instead of adding ignore directives, this can be avoided by inlining the JSON data into the curl invocation. Since the variable is only used in this one location, inlining is fine. Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2155: Declare + export separatelyDan Callahan1-1/+2
Declare and assign separately to avoid masking return values. https://github.com/koalaman/shellcheck/wiki/SC2155 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2166: test -a is not well definedDan Callahan1-1/+1
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. https://github.com/koalaman/shellcheck/wiki/SC2166 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2154: variable possibly undefinedDan Callahan1-1/+1
var is referenced but not assigned. https://github.com/koalaman/shellcheck/wiki/SC2154 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2064: Use single quotes on trapsDan Callahan1-1/+1
Use single quotes, otherwise this expands now rather than when signalled. https://github.com/koalaman/shellcheck/wiki/SC2064 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2115: Ensure never expands to /*Dan Callahan1-2/+2
Use "${var:?}" to ensure this never expands to /* . https://github.com/koalaman/shellcheck/wiki/SC2115 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2046: Quote to prevent word splitDan Callahan8-8/+8
Quote this to prevent word splitting https://www.shellcheck.net/wiki/SC2046 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2164: exit in case cd fails.Dan Callahan3-6/+6
Use `cd ... || exit` in case cd fails. https://github.com/koalaman/shellcheck/wiki/SC2164 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Add type hints for most `HomeServer` parameters (#11095)Sean Quah58-143/+342
2021-10-22Fix synapse.config module "read" command (#11145)Jason Robinson5-68/+138
`synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse. The "read" command broke in https://github.com/matrix-org/synapse/pull/10916 as it now requires passing in "server.server_name" for example. Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok. Signed-off-by: Jason Robinson <jasonr@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-22Add more information what happens when a user is deactivated (#11083)Dirk Klimpel2-0/+11
2021-10-21Add a thread relation type per MSC3440. (#11088)Patrick Cloke8-8/+119
Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
2021-10-21Fix adding excluded users to the private room sharing tables when joining a ↵David Robertson3-29/+67
room (#11143) * We only need to fetch users in private rooms * Filter out `user_id` at the top * Discard excluded users in the top loop We weren't doing this in the "First, if they're our user" branch so this is a bugfix. * The caller must check that `user_id` is included This is in the docstring. There are two call sites: - one in `_handle_room_publicity_change`, which explicitly checks before calling; - and another in `_handle_room_membership_event`, which returns early if the user is excluded. So this change is safe. * Test joining a private room with an excluded user * Tweak an existing test * Changelog * test docstring * lint
2021-10-21Improve docstrings for methods related to sending EDUs to application ↵Andrew Morgan7-23/+148
services (#11138)
2021-10-21Add missing type hints to synapse.crypto. (#11146)Patrick Cloke5-18/+36
And require type hints for this module.
2021-10-21fix relative link in docker readme (#11144)Richard van der Hoff2-1/+3
relative links don't work when it's on dockerhub.
2021-10-21Fix setting a user's external_id via the admin API returns 500 and deletes ↵Dirk Klimpel4-37/+321
users existing external mappings if that external ID is already mapped (#11051) Fixes #10846
2021-10-20Update `sign_json` to support inline key config (#11139)Richard van der Hoff2-7/+26
It's been possible to configure a key inline in the homeserver.yaml since 13bc1e0746aa0442aa5d43555cbbc2dc75e8ef43. Update `sign_json` to work with this.
2021-10-20Consider IP whitelist for identity server resolution (#11120)Robert Edström2-1/+4
Signed-off-by: Robert Edström <github@legogris.se>
2021-10-20Clean up `_update_auth_events_and_context_for_auth` (#11122)Richard van der Hoff2-114/+38
Remove some redundant code, and generally simplify.
2021-10-20Show error when timestamp in seconds is provided to the /purge_media_cache ↵Aaron R4-13/+133
API (#11101)
2021-10-20Remove false warning about copying the log config to a homeserver.yaml (#11092)Travis Ralston2-6/+7
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-20Document the version of Synapse each module callback was introduced in (#11132)Brendan Abolivier6-0/+45
* Mention callbacks introduced in v1.37.0 According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062 * Mention callbacks introduced in v1.39.0 According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884 * Mention callbacks introduced in v1.42.0 According to https://github.com/matrix-org/synapse/pull/10524 * Mention callbacks introduced in v1.44.0 and v1.45.0 As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894 * Mention callbacks introduced in v1.46.0 According to https://github.com/matrix-org/synapse/pull/10548
2021-10-20Remove link to #10947 from changelog v1.45.1Sean Quah1-1/+1
2021-10-201.45.1Sean Quah4-2/+16
2021-10-20Revert change to counting of deactivated users towards the monthly active ↵Sean Quah4-62/+4
users limit (#11127) Temporarily revert "Add functionality to remove deactivated users from the monthly_active_users table (#10947)". This reverts commit eda8c88b84ee7506379a71ac2a7a88c08b759d43.
2021-10-19Add missing type hints to event fetching. (#11121)Patrick Cloke2-61/+82
Updates the event rows returned from the database to be attrs classes instead of dictionaries.
2021-10-19Fix instances of [example]{.title-ref} in the upgrade notes (#11118)Andrew Morgan2-27/+28
2021-10-19Be less inconsistent about v1.2.3 versus 1.2.3 v1.45.0David Robertson1-5/+5
2021-10-19Duplicate known issues under 1.45 releaseDavid Robertson1-0/+7
2021-10-191.45.0David Robertson4-2/+18
2021-10-19Reword changelog regarding a suspected regression (#11117)Dan Callahan2-3/+15
Signed-off-by: Dan Callahan <danc@element.io>
2021-10-19Move _persist_auth_tree into FederationEventHandler (#11115)Richard van der Hoff3-125/+120
This is just a lift-and-shift, because it fits more naturally here. We do rename it to `process_remote_join` at the same time though.
2021-10-19Rename `_auth_and_persist_fetched_events` (#11116)Richard van der Hoff2-14/+10
... to `_auth_and_persist_outliers`, since that reflects its purpose better.
2021-10-19Include rejected status when we log events. (#11008)Richard van der Hoff2-6/+11
If we find ourselves dealing with rejected events, we proably want to know about it. Let's include it in the stringification of the event so that it gets logged.
2021-10-18Add missing type hints to synapse.api. (#11109)Patrick Cloke10-99/+84
* Convert UserPresenceState to attrs. * Remove args/kwargs from error classes and explicitly pass msg/errorcode.
2021-10-18Check auth on received events' auth_events (#11001)Richard van der Hoff2-2/+98
Currently, when we receive an event whose auth_events differ from those we expect, we state-resolve between the two state sets, and check that the event passes auth based on the resolved state. This means that it's possible for us to accept events which don't pass auth at their declared auth_events (or where the auth events themselves were rejected), leading to problems down the line like #10083. This change means we will: * ignore any events where we cannot find the auth events * reject any events whose auth events were rejected * reject any events which do not pass auth at their declared auth_events. Together with a whole raft of previous work, this is a partial fix to #9595. Fixes #6643. Based on #11009.
2021-10-18Check *all* auth events for room id and rejection (#11009)Richard van der Hoff8-85/+122
This fixes a bug where we would accept an event whose `auth_events` include rejected events, if the rejected event was shadowed by another `auth_event` with same `(type, state_key)`. The approach is to pass a list of auth events into `check_auth_rules_for_event` instead of a dict, which of course means updating the call sites. This is an extension of #10956.
2021-10-18Document Synapse's behaviour when dealing with multiple modules (#11096)Brendan Abolivier8-8/+154
Document Synapse's behaviour when multiple modules register the same callback/web resource/etc. Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-18`_run_push_actions_and_persist_event`: handle no min_depth (#11014)Richard van der Hoff3-11/+20
Make sure that we correctly handle rooms where we do not yet have a `min_depth`, and also add some comments and logging.
2021-10-18Fix broken export-data admin command and add a test for it to CI (#11078)Hillery Shay4-8/+93
Fix broken export-data admin command and add a test for it to CI
2021-10-18Don't remove local users from dir when the leave their last room (#11103)David Robertson3-5/+59
2021-10-15Correctly exclude users when making a room public or private (#11075)David Robertson4-83/+148
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-15Move experimental & retention config out of the server module. (#11070)Patrick Cloke10-255/+290
2021-10-15Fix logging context warnings when losing replication connection (#10984)Sean Quah3-10/+27
Instead of triggering `__exit__` manually on the replication handler's logging context, use it as a context manager so that there is an `__enter__` call to balance the `__exit__`.
2021-10-15Update doc of the allowed characters for registration tokens (#11093)Dirk Klimpel2-1/+2
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-14Fix 500 error on `/messages` when we accumulate more than 5 backward ↵Eric Eastwood4-12/+79
extremities (#11027) Found while working on the Gitter backfill script and noticed it only happened after we sent 7 batches, https://gitlab.com/gitterHQ/webapp/-/merge_requests/2229#note_665906390 When there are more than 5 backward extremities for a given depth, backfill will throw an error because we sliced the extremity list to 5 but then try to iterate over the full list. This causes us to look for state that we never fetched and we get a `KeyError`. Before when calling `/messages` when there are more than 5 backward extremities: ``` Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 258, in _async_render_wrapper callback_return = await self._async_render(request) File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 446, in _async_render callback_return = await raw_callback_return File "/usr/local/lib/python3.8/site-packages/synapse/rest/client/room.py", line 580, in on_GET msgs = await self.pagination_handler.get_messages( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/pagination.py", line 396, in get_messages await self.hs.get_federation_handler().maybe_backfill( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 133, in maybe_backfill return await self._maybe_backfill_inner(room_id, current_depth, limit) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 386, in _maybe_backfill_inner likely_extremeties_domains = get_domains_from_state(states[e_id]) KeyError: '$zpFflMEBtZdgcMQWTakaVItTLMjLFdKcRWUPHbbSZJl' ```
2021-10-14Ensure each charset is attempted only once during media preview. (#11089)Patrick Cloke3-14/+64
There's no point in trying more than once since it is guaranteed to continually fail.
2021-10-14Attempt different character encodings when previewing a URL. (#11077)Patrick Cloke3-67/+80
This follows similar logic to BeautifulSoup where we attempt different character encodings until we find one which works.
2021-10-14Fix-up some type hints in the relations tests. (#11076)Patrick Cloke5-51/+64
2021-10-14Add a test for a workaround concerning the behaviour of third-party rule ↵reivilibre2-6/+51
modules and `SynapseError`s. (#11071)
2021-10-14it appeared in 1.44, not 45rc1 v1.45.0rc2David Robertson1-1/+1
2021-10-14mentioned -> which appearedDavid Robertson1-1/+1
2021-10-14Refer to the bugs mentioned in 1.45.0rc1 noteDavid Robertson1-0/+2
2021-10-141.45.0rc2David Robertson10-9/+28
2021-10-13Resolve and share `state_groups` for all historical events in batch ↵Eric Eastwood8-47/+114
(MSC2716) (#10975) Resolve and share `state_groups` for all historical events in batch. This also helps for showing the appropriate avatar/displayname in Element and will work whenever `/messages` has one of the historical messages as the first message in the batch. This does have the flaw where if you just insert a single historical event somewhere, it probably won't resolve the state correctly from `/messages` or `/context` since it will grab a non historical event above or below with resolved state which never included the historical state back then. For the same reasions, this also does not work in Element between the transition from actual messages to historical messages. In the Gitter case, this isn't really a problem since all of the historical messages are in one big lump at the beginning of the room. For a future iteration, might be good to look at `/messages` and `/context` to additionally add the `state` for any historical messages in that batch. --- How are the `state_groups` shared? To illustrate the `state_group` sharing, see this example: **Before** (new `state_group` for every event 😬, very inefficient): ``` # Tests from https://github.com/matrix-org/complement/pull/206 $ COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1 COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/should_resolve_member_state_events_for_historical_events create_new_client_event m.room.member event=$_JXfwUDIWS6xKGG4SmZXjSFrizhARM7QblhATVWWUcA state_group=None create_new_client_event org.matrix.msc2716.insertion event=$1ZBfmBKEjg94d-vGYymKrVYeghwBOuGJ3wubU1-I9y0 state_group=9 create_new_client_event org.matrix.msc2716.insertion event=$Mq2JvRetTyclPuozRI682SAjYp3GqRuPc8_cH5-ezPY state_group=10 create_new_client_event m.room.message event=$MfmY4rBQkxrIp8jVwVMTJ4PKnxSigpG9E2cn7S0AtTo state_group=11 create_new_client_event m.room.message event=$uYOv6V8wiF7xHwOMt-60d1AoOIbqLgrDLz6ZIQDdWUI state_group=12 create_new_client_event m.room.message event=$PAbkJRMxb0bX4A6av463faiAhxkE3FEObM1xB4D0UG4 state_group=13 create_new_client_event org.matrix.msc2716.batch event=$Oy_S7AWN7rJQe_MYwGPEy6RtbYklrI-tAhmfiLrCaKI state_group=14 ``` **After** (all events in batch sharing `state_group=10`) (the base insertion event has `state_group=8` which matches the `prev_event` we're inserting next to): ``` # Tests from https://github.com/matrix-org/complement/pull/206 $ COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1 COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/should_resolve_member_state_events_for_historical_events create_new_client_event m.room.member event=$PWomJ8PwENYEYuVNoG30gqtybuQQSZ55eldBUSs0i0U state_group=None create_new_client_event org.matrix.msc2716.insertion event=$e_mCU7Eah9ABF6nQU7lu4E1RxIWccNF05AKaTT5m3lw state_group=9 create_new_client_event org.matrix.msc2716.insertion event=$ui7A3_GdXIcJq0C8GpyrF8X7B3DTjMd_WGCjogax7xU state_group=10 create_new_client_event m.room.message event=$EnTIM5rEGVezQJiYl62uFBl6kJ7B-sMxWqe2D_4FX1I state_group=10 create_new_client_event m.room.message event=$LGx5jGONnBPuNhAuZqHeEoXChd9ryVkuTZatGisOPjk state_group=10 create_new_client_event m.room.message event=$wW0zwoN50lbLu1KoKbybVMxLbKUj7GV_olozIc5i3M0 state_group=10 create_new_client_event org.matrix.msc2716.batch event=$5ZB6dtzqFBCEuMRgpkU201Qhx3WtXZGTz_YgldL6JrQ state_group=10 ```
2021-10-13Fix upgrade dead links (#11069)David Robertson2-4/+5
2021-10-13Rearrange the user_directory's `_handle_deltas` function (#11035)David Robertson2-57/+79
* Pull out `_handle_room_membership_event` * Discard excluded users early * Rearrange logic so the change is membership is effectively switched over. See PR for rationale.
2021-10-13Remove dead code from `MediaFilePaths` (#11056)Sean Quah2-17/+1
2021-10-13Add type hints to synapse.events.*. (#11066)Patrick Cloke11-145/+208
Except `synapse/events/__init__.py`, which will be done in a follow-up.
2021-10-13Port the Password Auth Providers module interface to the new generic ↵Azrenbeth13-225/+790
interface (#10548) Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13Be more lenient when parsing the version for oEmbed responses. (#11065)Patrick Cloke5-8/+60
2021-10-13Stop user directory from failing if it encounters users not in the `users` ↵David Robertson13-93/+921
table. (#11053) The following scenarios would halt the user directory updater: - user joins room - user leaves room - user present in room which switches from private to public, or vice versa. for two classes of users: - appservice senders - users missing from the user table. If this happened, the user directory would be stuck, unable to make forward progress. Exclude both cases from the user directory, so that we ignore them. Co-authored-by: Eric Eastwood <erice@element.io> Co-authored-by: reivilibre <oliverw@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13Mark Module API error imports as re-exported and mark Synapse as containing ↵reivilibre4-2/+11
type annotations (#11054)
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-14 05:45:33 +0000