summary refs log tree commit diff
path: root/contrib/docker (unfollow)
Commit message (Collapse)AuthorFilesLines
2021-10-22Fix Shellcheck SC2155: Declare + export separatelyDan Callahan1-1/+2
Declare and assign separately to avoid masking return values. https://github.com/koalaman/shellcheck/wiki/SC2155 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2166: test -a is not well definedDan Callahan1-1/+1
Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. https://github.com/koalaman/shellcheck/wiki/SC2166 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2154: variable possibly undefinedDan Callahan1-1/+1
var is referenced but not assigned. https://github.com/koalaman/shellcheck/wiki/SC2154 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2064: Use single quotes on trapsDan Callahan1-1/+1
Use single quotes, otherwise this expands now rather than when signalled. https://github.com/koalaman/shellcheck/wiki/SC2064 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2115: Ensure never expands to /*Dan Callahan1-2/+2
Use "${var:?}" to ensure this never expands to /* . https://github.com/koalaman/shellcheck/wiki/SC2115 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2046: Quote to prevent word splitDan Callahan8-8/+8
Quote this to prevent word splitting https://www.shellcheck.net/wiki/SC2046 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Fix Shellcheck SC2164: exit in case cd fails.Dan Callahan3-6/+6
Use `cd ... || exit` in case cd fails. https://github.com/koalaman/shellcheck/wiki/SC2164 Signed-off-by: Dan Callahan <danc@element.io>
2021-10-22Add type hints for most `HomeServer` parameters (#11095)Sean Quah58-143/+342
2021-10-22Fix synapse.config module "read" command (#11145)Jason Robinson5-68/+138
`synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse. The "read" command broke in https://github.com/matrix-org/synapse/pull/10916 as it now requires passing in "server.server_name" for example. Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok. Signed-off-by: Jason Robinson <jasonr@matrix.org> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-22Add more information what happens when a user is deactivated (#11083)Dirk Klimpel2-0/+11
2021-10-21Add a thread relation type per MSC3440. (#11088)Patrick Cloke8-8/+119
Adds experimental support for MSC3440's `io.element.thread` relation type (and the aggregation for it).
2021-10-21Fix adding excluded users to the private room sharing tables when joining a ↵David Robertson3-29/+67
room (#11143) * We only need to fetch users in private rooms * Filter out `user_id` at the top * Discard excluded users in the top loop We weren't doing this in the "First, if they're our user" branch so this is a bugfix. * The caller must check that `user_id` is included This is in the docstring. There are two call sites: - one in `_handle_room_publicity_change`, which explicitly checks before calling; - and another in `_handle_room_membership_event`, which returns early if the user is excluded. So this change is safe. * Test joining a private room with an excluded user * Tweak an existing test * Changelog * test docstring * lint
2021-10-21Improve docstrings for methods related to sending EDUs to application ↵Andrew Morgan7-23/+148
services (#11138)
2021-10-21Add missing type hints to synapse.crypto. (#11146)Patrick Cloke5-18/+36
And require type hints for this module.
2021-10-21fix relative link in docker readme (#11144)Richard van der Hoff2-1/+3
relative links don't work when it's on dockerhub.
2021-10-21Fix setting a user's external_id via the admin API returns 500 and deletes ↵Dirk Klimpel4-37/+321
users existing external mappings if that external ID is already mapped (#11051) Fixes #10846
2021-10-20Update `sign_json` to support inline key config (#11139)Richard van der Hoff2-7/+26
It's been possible to configure a key inline in the homeserver.yaml since 13bc1e0746aa0442aa5d43555cbbc2dc75e8ef43. Update `sign_json` to work with this.
2021-10-20Consider IP whitelist for identity server resolution (#11120)Robert Edström2-1/+4
Signed-off-by: Robert Edström <github@legogris.se>
2021-10-20Clean up `_update_auth_events_and_context_for_auth` (#11122)Richard van der Hoff2-114/+38
Remove some redundant code, and generally simplify.
2021-10-20Show error when timestamp in seconds is provided to the /purge_media_cache ↵Aaron R4-13/+133
API (#11101)
2021-10-20Remove false warning about copying the log config to a homeserver.yaml (#11092)Travis Ralston2-6/+7
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-20Document the version of Synapse each module callback was introduced in (#11132)Brendan Abolivier6-0/+45
* Mention callbacks introduced in v1.37.0 According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062 * Mention callbacks introduced in v1.39.0 According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884 * Mention callbacks introduced in v1.42.0 According to https://github.com/matrix-org/synapse/pull/10524 * Mention callbacks introduced in v1.44.0 and v1.45.0 As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894 * Mention callbacks introduced in v1.46.0 According to https://github.com/matrix-org/synapse/pull/10548
2021-10-20Remove link to #10947 from changelog v1.45.1Sean Quah1-1/+1
2021-10-201.45.1Sean Quah4-2/+16
2021-10-20Revert change to counting of deactivated users towards the monthly active ↵Sean Quah4-62/+4
users limit (#11127) Temporarily revert "Add functionality to remove deactivated users from the monthly_active_users table (#10947)". This reverts commit eda8c88b84ee7506379a71ac2a7a88c08b759d43.
2021-10-19Add missing type hints to event fetching. (#11121)Patrick Cloke2-61/+82
Updates the event rows returned from the database to be attrs classes instead of dictionaries.
2021-10-19Fix instances of [example]{.title-ref} in the upgrade notes (#11118)Andrew Morgan2-27/+28
2021-10-19Be less inconsistent about v1.2.3 versus 1.2.3 v1.45.0David Robertson1-5/+5
2021-10-19Duplicate known issues under 1.45 releaseDavid Robertson1-0/+7
2021-10-191.45.0David Robertson4-2/+18
2021-10-19Reword changelog regarding a suspected regression (#11117)Dan Callahan2-3/+15
Signed-off-by: Dan Callahan <danc@element.io>
2021-10-19Move _persist_auth_tree into FederationEventHandler (#11115)Richard van der Hoff3-125/+120
This is just a lift-and-shift, because it fits more naturally here. We do rename it to `process_remote_join` at the same time though.
2021-10-19Rename `_auth_and_persist_fetched_events` (#11116)Richard van der Hoff2-14/+10
... to `_auth_and_persist_outliers`, since that reflects its purpose better.
2021-10-19Include rejected status when we log events. (#11008)Richard van der Hoff2-6/+11
If we find ourselves dealing with rejected events, we proably want to know about it. Let's include it in the stringification of the event so that it gets logged.
2021-10-18Add missing type hints to synapse.api. (#11109)Patrick Cloke10-99/+84
* Convert UserPresenceState to attrs. * Remove args/kwargs from error classes and explicitly pass msg/errorcode.
2021-10-18Check auth on received events' auth_events (#11001)Richard van der Hoff2-2/+98
Currently, when we receive an event whose auth_events differ from those we expect, we state-resolve between the two state sets, and check that the event passes auth based on the resolved state. This means that it's possible for us to accept events which don't pass auth at their declared auth_events (or where the auth events themselves were rejected), leading to problems down the line like #10083. This change means we will: * ignore any events where we cannot find the auth events * reject any events whose auth events were rejected * reject any events which do not pass auth at their declared auth_events. Together with a whole raft of previous work, this is a partial fix to #9595. Fixes #6643. Based on #11009.
2021-10-18Check *all* auth events for room id and rejection (#11009)Richard van der Hoff8-85/+122
This fixes a bug where we would accept an event whose `auth_events` include rejected events, if the rejected event was shadowed by another `auth_event` with same `(type, state_key)`. The approach is to pass a list of auth events into `check_auth_rules_for_event` instead of a dict, which of course means updating the call sites. This is an extension of #10956.
2021-10-18Document Synapse's behaviour when dealing with multiple modules (#11096)Brendan Abolivier8-8/+154
Document Synapse's behaviour when multiple modules register the same callback/web resource/etc. Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-18`_run_push_actions_and_persist_event`: handle no min_depth (#11014)Richard van der Hoff3-11/+20
Make sure that we correctly handle rooms where we do not yet have a `min_depth`, and also add some comments and logging.
2021-10-18Fix broken export-data admin command and add a test for it to CI (#11078)Hillery Shay4-8/+93
Fix broken export-data admin command and add a test for it to CI
2021-10-18Don't remove local users from dir when the leave their last room (#11103)David Robertson3-5/+59
2021-10-15Correctly exclude users when making a room public or private (#11075)David Robertson4-83/+148
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-15Move experimental & retention config out of the server module. (#11070)Patrick Cloke10-255/+290
2021-10-15Fix logging context warnings when losing replication connection (#10984)Sean Quah3-10/+27
Instead of triggering `__exit__` manually on the replication handler's logging context, use it as a context manager so that there is an `__enter__` call to balance the `__exit__`.
2021-10-15Update doc of the allowed characters for registration tokens (#11093)Dirk Klimpel2-1/+2
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-14Fix 500 error on `/messages` when we accumulate more than 5 backward ↵Eric Eastwood4-12/+79
extremities (#11027) Found while working on the Gitter backfill script and noticed it only happened after we sent 7 batches, https://gitlab.com/gitterHQ/webapp/-/merge_requests/2229#note_665906390 When there are more than 5 backward extremities for a given depth, backfill will throw an error because we sliced the extremity list to 5 but then try to iterate over the full list. This causes us to look for state that we never fetched and we get a `KeyError`. Before when calling `/messages` when there are more than 5 backward extremities: ``` Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 258, in _async_render_wrapper callback_return = await self._async_render(request) File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 446, in _async_render callback_return = await raw_callback_return File "/usr/local/lib/python3.8/site-packages/synapse/rest/client/room.py", line 580, in on_GET msgs = await self.pagination_handler.get_messages( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/pagination.py", line 396, in get_messages await self.hs.get_federation_handler().maybe_backfill( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 133, in maybe_backfill return await self._maybe_backfill_inner(room_id, current_depth, limit) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 386, in _maybe_backfill_inner likely_extremeties_domains = get_domains_from_state(states[e_id]) KeyError: '$zpFflMEBtZdgcMQWTakaVItTLMjLFdKcRWUPHbbSZJl' ```
2021-10-14Ensure each charset is attempted only once during media preview. (#11089)Patrick Cloke3-14/+64
There's no point in trying more than once since it is guaranteed to continually fail.
2021-10-14Attempt different character encodings when previewing a URL. (#11077)Patrick Cloke3-67/+80
This follows similar logic to BeautifulSoup where we attempt different character encodings until we find one which works.
2021-10-14Fix-up some type hints in the relations tests. (#11076)Patrick Cloke5-51/+64
2021-10-14Add a test for a workaround concerning the behaviour of third-party rule ↵reivilibre2-6/+51
modules and `SynapseError`s. (#11071)
2021-10-14it appeared in 1.44, not 45rc1 v1.45.0rc2David Robertson1-1/+1
2021-10-14mentioned -> which appearedDavid Robertson1-1/+1
2021-10-14Refer to the bugs mentioned in 1.45.0rc1 noteDavid Robertson1-0/+2
2021-10-141.45.0rc2David Robertson10-9/+28
2021-10-13Resolve and share `state_groups` for all historical events in batch ↵Eric Eastwood8-47/+114
(MSC2716) (#10975) Resolve and share `state_groups` for all historical events in batch. This also helps for showing the appropriate avatar/displayname in Element and will work whenever `/messages` has one of the historical messages as the first message in the batch. This does have the flaw where if you just insert a single historical event somewhere, it probably won't resolve the state correctly from `/messages` or `/context` since it will grab a non historical event above or below with resolved state which never included the historical state back then. For the same reasions, this also does not work in Element between the transition from actual messages to historical messages. In the Gitter case, this isn't really a problem since all of the historical messages are in one big lump at the beginning of the room. For a future iteration, might be good to look at `/messages` and `/context` to additionally add the `state` for any historical messages in that batch. --- How are the `state_groups` shared? To illustrate the `state_group` sharing, see this example: **Before** (new `state_group` for every event 😬, very inefficient): ``` # Tests from https://github.com/matrix-org/complement/pull/206 $ COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1 COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/should_resolve_member_state_events_for_historical_events create_new_client_event m.room.member event=$_JXfwUDIWS6xKGG4SmZXjSFrizhARM7QblhATVWWUcA state_group=None create_new_client_event org.matrix.msc2716.insertion event=$1ZBfmBKEjg94d-vGYymKrVYeghwBOuGJ3wubU1-I9y0 state_group=9 create_new_client_event org.matrix.msc2716.insertion event=$Mq2JvRetTyclPuozRI682SAjYp3GqRuPc8_cH5-ezPY state_group=10 create_new_client_event m.room.message event=$MfmY4rBQkxrIp8jVwVMTJ4PKnxSigpG9E2cn7S0AtTo state_group=11 create_new_client_event m.room.message event=$uYOv6V8wiF7xHwOMt-60d1AoOIbqLgrDLz6ZIQDdWUI state_group=12 create_new_client_event m.room.message event=$PAbkJRMxb0bX4A6av463faiAhxkE3FEObM1xB4D0UG4 state_group=13 create_new_client_event org.matrix.msc2716.batch event=$Oy_S7AWN7rJQe_MYwGPEy6RtbYklrI-tAhmfiLrCaKI state_group=14 ``` **After** (all events in batch sharing `state_group=10`) (the base insertion event has `state_group=8` which matches the `prev_event` we're inserting next to): ``` # Tests from https://github.com/matrix-org/complement/pull/206 $ COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1 COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/should_resolve_member_state_events_for_historical_events create_new_client_event m.room.member event=$PWomJ8PwENYEYuVNoG30gqtybuQQSZ55eldBUSs0i0U state_group=None create_new_client_event org.matrix.msc2716.insertion event=$e_mCU7Eah9ABF6nQU7lu4E1RxIWccNF05AKaTT5m3lw state_group=9 create_new_client_event org.matrix.msc2716.insertion event=$ui7A3_GdXIcJq0C8GpyrF8X7B3DTjMd_WGCjogax7xU state_group=10 create_new_client_event m.room.message event=$EnTIM5rEGVezQJiYl62uFBl6kJ7B-sMxWqe2D_4FX1I state_group=10 create_new_client_event m.room.message event=$LGx5jGONnBPuNhAuZqHeEoXChd9ryVkuTZatGisOPjk state_group=10 create_new_client_event m.room.message event=$wW0zwoN50lbLu1KoKbybVMxLbKUj7GV_olozIc5i3M0 state_group=10 create_new_client_event org.matrix.msc2716.batch event=$5ZB6dtzqFBCEuMRgpkU201Qhx3WtXZGTz_YgldL6JrQ state_group=10 ```
2021-10-13Fix upgrade dead links (#11069)David Robertson2-4/+5
2021-10-13Rearrange the user_directory's `_handle_deltas` function (#11035)David Robertson2-57/+79
* Pull out `_handle_room_membership_event` * Discard excluded users early * Rearrange logic so the change is membership is effectively switched over. See PR for rationale.
2021-10-13Remove dead code from `MediaFilePaths` (#11056)Sean Quah2-17/+1
2021-10-13Add type hints to synapse.events.*. (#11066)Patrick Cloke11-145/+208
Except `synapse/events/__init__.py`, which will be done in a follow-up.
2021-10-13Port the Password Auth Providers module interface to the new generic ↵Azrenbeth13-225/+790
interface (#10548) Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13Be more lenient when parsing the version for oEmbed responses. (#11065)Patrick Cloke5-8/+60
2021-10-13Stop user directory from failing if it encounters users not in the `users` ↵David Robertson13-93/+921
table. (#11053) The following scenarios would halt the user directory updater: - user joins room - user leaves room - user present in room which switches from private to public, or vice versa. for two classes of users: - appservice senders - users missing from the user table. If this happened, the user directory would be stuck, unable to make forward progress. Exclude both cases from the user directory, so that we ignore them. Co-authored-by: Eric Eastwood <erice@element.io> Co-authored-by: reivilibre <oliverw@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13Mark Module API error imports as re-exported and mark Synapse as containing ↵reivilibre4-2/+11
type annotations (#11054)
2021-10-12Always dump logs from trial during CI. (#11068)Patrick Cloke3-0/+9
Instead of only dumping them if trial passes.
2021-10-12Simplify the user admin API tests (#11048)Dirk Klimpel2-255/+147
2021-10-12Add support for ubuntu 21.10 "Impish Indri" (#11024)Hillery Shay2-0/+2
* support ubuntu 21.10 indri * add changelog * update to correct codename Co-authored-by: Brendan Abolivier <github@brendanabolivier.com> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2021-10-12Add tests for `MediaFilePaths` (#11057)Sean Quah3-0/+240
2021-10-12Update `_wrap_in_base_path` type hints to preserve function arguments (#11055)Sean Quah2-3/+7
2021-10-12Fix formatting string when oEmbed errors occur. (#11061)Patrick Cloke2-1/+2
2021-10-12Fix race in `MultiWriterIdGenerator` (#11045)Erik Johnston2-15/+68
The race allowed the current position to advance too far when stream IDs are still being persisted. This happened when it received a new stream ID from a remote write between a new stream ID being allocated and it being added to the set of unpersisted stream IDs. Fixes #9424.
2021-10-12Reset global cache state before cache tests. (#11036)Patrick Cloke2-13/+12
This reverts #11019 and structures the code a bit more like it was before #10985. The global cache state must be reset before running the tests since other test cases might have configured caching (and thus touched the global state).
2021-10-12Add type hints to `synapse.storage.databases.main.client_ips` (#10972)Sean Quah5-45/+121
2021-10-12Fixup changelog v1.45.0rc1Brendan Abolivier1-2/+2
2021-10-12TypoBrendan Abolivier1-1/+1
2021-10-12Add a link to the upgrade notesBrendan Abolivier1-0/+2
2021-10-12Fix inconsistent behavior of `get_last_client_by_ip` (#10970)Sean Quah3-4/+53
Make `get_last_client_by_ip` return the same dictionary structure regardless of whether the data has been persisted to the database. This change will allow slightly cleaner type hints to be applied later on.
2021-10-12Update upgrade notesBrendan Abolivier1-0/+9
2021-10-12Fix opentracing and Prometheus metrics for replication requests (#10996)Sean Quah3-76/+87
This commit fixes two bugs to do with decorators not instrumenting `ReplicationEndpoint`'s `send_request` correctly. There are two decorators on `send_request`: Prometheus' `Gauge.track_inprogress()` and Synapse's `opentracing.trace`. `Gauge.track_inprogress()` does not have any support for async functions when used as a decorator. Since async functions behave like regular functions that return coroutines, only the creation of the coroutine was covered by the metric and none of the actual body of `send_request`. `Gauge.track_inprogress()` returns a regular, non-async function wrapping `send_request`, which is the source of the next bug. The `opentracing.trace` decorator would normally handle async functions correctly, but since the wrapped `send_request` is a non-async function, the decorator ends up suffering from the same issue as `Gauge.track_inprogress()`: the opentracing span only measures the creation of the coroutine and none of the actual function body. Using `Gauge.track_inprogress()` as a context manager instead of a decorator resolves both bugs.
2021-10-12Add warning about known issuesBrendan Abolivier1-0/+2
2021-10-12Fixup changelogBrendan Abolivier1-7/+7
2021-10-121.45.0rc1Brendan Abolivier66-66/+84
2021-10-12Add an approximate difference method to StateFilters (#10825)reivilibre3-3/+683
2021-10-11disallow-untyped-defs for synapse.push (#11023)David Robertson7-10/+28
2021-10-11Include the requirements for [mypy,lint] in [dev] (#11034)reivilibre4-11/+17
2021-10-11Pass through `SynapseError`s that are raised from experimental ↵reivilibre2-0/+10
`check_event_allowed` callback of the module API (#11042) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-11Remove unnecessary list comprehension in `synapse_port_db` to fix linting in ↵reivilibre2-1/+2