summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* Always update AS last_pos, even on no events (#10107)14mRh4X0r2021-06-073-17/+15
| | | | | | | | | | Fixes #1834. `get_new_events_for_appservice` internally calls `get_events_as_list`, which will filter out any rejected events. If all returned events are filtered out, `_notify_interested_services` will return without updating the last handled stream position. If there are 100 consecutive such events, processing will halt altogether. Breaking the loop is now done by checking whether we're up-to-date with `current_max` in the loop condition, instead of relying on an empty `events` list. Signed-off-by: Willem Mulder <14mRh4X0r@gmail.com>
* Add missing type hints to the admin API servlets (#10105)Dirk Klimpel2021-06-076-40/+48
|
* Delete completes to-device messages earlier in /sync (#10124)Richard van der Hoff2021-06-072-10/+12
| | | | I hope this will improve https://github.com/matrix-org/synapse/issues/9564.
* Don't try and backfill the same room in parallel. (#10116)Erik Johnston2021-06-042-0/+9
| | | | | If backfilling is slow then the client may time out and retry, causing Synapse to start a new `/backfill` before the existing backfill has finished, duplicating work.
* Limit number of events in a replication request (#10118)Erik Johnston2021-06-042-2/+4
| | | Fixes #9956.
* Enable Prometheus metrics for the jaeger client library (#10112)Richard van der Hoff2021-06-043-1/+5
|
* Compile and render Synapse's docs into a browsable, mobile-friendly and ↵Andrew Morgan2021-06-0335-54/+978
| | | | searchable website (#10086)
* Add OpenTracing for database activity. (#10113)Richard van der Hoff2021-06-033-33/+60
| | | | | | | | | | This adds quite a lot of OpenTracing decoration for database activity. Specifically it adds tracing at four different levels: * emit a span for each "interaction" - ie, the top level database function that we tend to call "transaction", but isn't really, because it can end up as multiple transactions. * emit a span while we hold a database connection open * emit a span for each database transaction - actual actual transaction. * emit a span for each database query. I'm aware this might be quite a lot of overhead, but even just running it on a local Synapse it looks really interesting, and I hope the overhead can be offset just by turning down the sampling frequency and finding other ways of tracing requests of interest (eg, the `force_tracing_for_users` setting).
* Improve opentracing annotations for Notifier (#10111)Richard van der Hoff2021-06-032-33/+34
| | | | | | | The existing tracing reports an error each time there is a timeout, which isn't really representative. Additionally, we log things about the way `wait_for_events` works (eg, the result of the callback) to the *parent* span, which is confusing.
* Convert admin api docs to markdown (#10089)Andrew Morgan2021-06-039-1132/+1160
| | | So that they render nicely in mdbook (see #10086), and so that we no longer have a mix of structured text languages in our documentation (excluding files outside of `docs/`).
* r0.6.1 support: /rooms/:roomId/aliases endpoint (#9224)Travis Ralston2021-06-033-3/+3
| | | | | | [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432) added this endpoint originally but it has since been included in the spec for nearly a year. This is progress towards https://github.com/matrix-org/synapse/issues/8334
* Merge branch 'master' into developPatrick Cloke2021-06-034-2/+16
|\
| * Clarify changelog. v1.35.1 github/release-v1.35 release-v1.35Patrick Cloke2021-06-031-1/+1
| |
| * 1.35.1Patrick Cloke2021-06-034-2/+16
| |
| * Do not show invite-only rooms in spaces summary (unless joined/invited). ↵Patrick Cloke2021-06-032-10/+10
| | | | | | | | (#10109)
* | Add new admin APIs to remove media by media ID from quarantine. (#10044)Dirk Klimpel2021-06-025-10/+201
| | | | | | | | | | Related to: #6681, #5956, #10040 Signed-off-by: Dirk Klimpel dirk@klimpel.org
* | github actions: summarize Sytest results in an easy-to-read format (#10094)Richard van der Hoff2021-06-022-2/+3
| | | | | | ... using the script from matrix-org/sytest#1052
* | Rewrite the KeyRing (#10035)Erik Johnston2021-06-028-492/+393
| |
* | Do not show invite-only rooms in spaces summary (unless joined/invited). ↵Patrick Cloke2021-06-022-10/+10
| | | | | | | | (#10109)
* | Make /sync do less state res (#10102)Erik Johnston2021-06-022-2/+3
| |
* | Merge branch 'master' into developAndrew Morgan2021-06-015-2/+50
|\|
| * Merge v1.35.0rc3 into v1.35.0 due to incorrect tagging v1.35.0Andrew Morgan2021-06-011-5/+1
| |
| * Indicate that there were no functional changes since v1.35.0rc3Andrew Morgan2021-06-011-0/+2
| |
| * 1.35.0Andrew Morgan2021-06-014-2/+16
| |
| * Announce deprecation of experimental `msc2858_enabled` option. (#10101)Erik Johnston2021-06-011-0/+1
| | | | | | | | | | c.f. https://github.com/matrix-org/synapse/pull/9617 and https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md Fixes #9627.
| * CHANGELOGErik Johnston2021-05-281-2/+2
| |
| * 1.35.0rc3Erik Johnston2021-05-285-4/+17
| |
| * Merge remote-tracking branch 'origin/release-v1.35.0' into release-v1.35Erik Johnston2021-05-283-2/+10
| |\
| * | Limit the number of events sent over replication when persisting events. ↵Brendan Abolivier2021-05-282-7/+11
| | | | | | | | | | | | (#10082)
| * | Log method and path when dropping request due to size limit (#10091)Erik Johnston2021-05-282-1/+4
| | |
| * | Allow response of `/send_join` to be larger. (#10093)Erik Johnston2021-05-283-1/+21
| | | | | | | | | Fixes #10087.
* | | add a cache to have_seen_event (#9953)Richard van der Hoff2021-06-0110-20/+205
| | | | | | | | | Empirically, this helped my server considerably when handling gaps in Matrix HQ. The problem was that we would repeatedly call have_seen_events for the same set of (50K or so) auth_events, each of which would take many minutes to complete, even though it's only an index scan.
* | | Fix opentracing inject to use the SpanContext, not the Span (#10074)Brad Murray2021-06-012-5/+6
| | | | | | | | | Signed-off-by: Brad Murray brad@beeper.com
* | | Set opentracing priority before setting other tags (#10092)Richard van der Hoff2021-05-285-11/+32
| | | | | | | | | ... because tags on spans which aren't being sampled get thrown away.
* | | Log method and path when dropping request due to size limit (#10091)Erik Johnston2021-05-282-1/+4
| | |
* | | Add `parse_strings_from_args` to get `prev_events` array (#10048)Eric Eastwood2021-05-282-43/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Split out from https://github.com/matrix-org/synapse/pull/9247 Strings: - `parse_string` - `parse_string_from_args` - `parse_strings_from_args` For comparison with ints: - `parse_integer` - `parse_integer_from_args` Previous discussions: - https://github.com/matrix-org/synapse/pull/9247#discussion_r573195687 - https://github.com/matrix-org/synapse/pull/9247#discussion_r574214156 - https://github.com/matrix-org/synapse/pull/9247#discussion_r573264791 Signed-off-by: Eric Eastwood <erice@element.io>
* | | Move some more endpoints off master (#10084)Erik Johnston2021-05-274-6/+10
| | |
* | | Make reason and score optional for report_event (#10077)Callum Brown2021-05-276-13/+105
| | | | | | | | | | | | | | | | | | Implements MSC2414: https://github.com/matrix-org/matrix-doc/pull/2414 See #8551 Signed-off-by: Callum Brown <callum@calcuode.com>
* | | Limit the number of events sent over replication when persisting events. ↵Brendan Abolivier2021-05-272-7/+11
| | | | | | | | | | | | (#10082)
* | | Merge tag 'v1.35.0rc2' into developErik Johnston2021-05-273-2/+11
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | Synapse 1.35.0rc2 (2021-05-27) ============================== Bugfixes -------- - Fix a bug introduced in v1.35.0rc1 when calling the spaces summary API via a GET request. ([\#10079](https://github.com/matrix-org/synapse/issues/10079))
| * | 1.35.0rc2 v1.35.0rc2 github/release-v1.35.0 release-v1.35.0Erik Johnston2021-05-273-2/+10
| |/
| * Pass the origin when calculating the spaces summary over GET. (#10079)Patrick Cloke2021-05-272-1/+2
| | | | | | | | Fixes a bug due to conflicting PRs which were merged. (One added a new caller to a method, the other added a new parameter to the same method.)
* | Fix up `BatchingQueue` (#10078)Erik Johnston2021-05-273-24/+125
| | | | | | Fixes #10068
* | Clarify security note regarding the domain Synapse is hosted on. (#9221)Denis Kasak2021-05-272-11/+36
| |
* | Improve the error message printed by synctl when synapse fails to start. ↵Richard van der Hoff2021-05-272-4/+9
| | | | | | | | (#10059)
* | Remove redundant, unmaintained `convert_server_keys` script. (#10055)Richard van der Hoff2021-05-272-108/+1
| |
* | Remove redundant code to reload tls cert (#10054)Richard van der Hoff2021-05-274-25/+6
| | | | | | | | | | we don't need to reload the tls cert if we don't have any tls listeners. Follow-up to #9280.
* | Combine `LruCache.invalidate` and `invalidate_many` (#9973)Richard van der Hoff2021-05-2712-52/+52
| | | | | | | | | | | | | | | | | | | | * Make `invalidate` and `invalidate_many` do the same thing ... so that we can do either over the invalidation replication stream, and also because they always confused me a bit. * Kill off `invalidate_many` * changelog
* | Remove the experimental spaces enabled flag. (#10063)Patrick Cloke2021-05-266-49/+9
| | | | | | | | In lieu of just always enabling the unstable spaces endpoint and unstable room version.
* | Fix GitHub Actions lint for newsfragments (#10069)Dan Callahan2021-05-262-0/+7
| | | | | | | | | | * Fix GitHub Actions lint for newsfragments Signed-off-by: Dan Callahan <danc@element.io>
* | Tweak wording of database recommendation in INSTALL.md (#10057)Aaron Raimist2021-05-262-5/+8
| | | | | | | | | | | | * Tweak wording of database recommendation in INSTALL.md Signed-off-by: Aaron Raimist <aaron@raim.ist>
* | Tell CircleCI to build Docker images from `main` (#9906)Dan Callahan2021-05-262-1/+2
| | | | | | | | | | | | The `only` field takes a string or list of strings per the Circle docs: https://circleci.com/docs/2.0/configuration-reference/#branches Signed-off-by: Dan Callahan <danc@element.io>
* | Add an admin API for unprotecting local media from quarantine (#10040)Dirk Klimpel2021-05-265-5/+151
| | | | | | Signed-off-by: Dirk Klimpel dirk@klimpel.org
* | Update CAPTCHA documentation to mention turning off verify origin feature ↵Aaron Raimist2021-05-262-22/+29
|/ | | | | | | | (#10046) * Update CAPTCHA documentation to mention turning off verify origin Signed-off-by: Aaron Raimist <aaron@raim.ist>
* 1.35.0rc1 v1.35.0rc1Erik Johnston2021-05-2539-38/+65
|
* Run complement with Synapse workers manually. (#10039)Patrick Cloke2021-05-243-7/+27
| | | | Adds an option to complement.sh to run Synapse in worker mode (instead of the default monolith mode).
* Add missing type hints to synapse.util (#9982)Patrick Cloke2021-05-249-25/+39
|
* Fix docker image to not log at `/homeserver.log` (#10045)Sergio Miguéns2021-05-242-1/+4
| | | | | Fixes #9970 Signed-off-by: Sergio Miguéns Iglesias lonyelon@lony.xyz
* Remove tls_fingerprints option (#9280)Jerin J Titus2021-05-246-95/+1
| | | | Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
* Fix --no-daemonize for synctl with workers (#9995)Richard van der Hoff2021-05-242-70/+33
|
* Fix off-by-one-error in synapse_port_db (#9991)Richard van der Hoff2021-05-245-9/+10
| | | fixes #9979
* Remove `keylen` from `LruCache`. (#9993)Richard van der Hoff2021-05-2411-55/+80
| | | | | | | `keylen` seems to be a thing that is frequently incorrectly set, and we don't really need it. The only time it was used was to figure out if we had removed a subtree in `del_multi`, which we can do better by changing `TreeCache.pop` to return a different type (`TreeCacheNode`). Commits should be independently reviewable.
* Enable experimental spaces by default. (#10011)Patrick Cloke2021-05-244-2/+35
| | | | The previous spaces_enabled flag now defaults to true and is exposed in the sample config.
* Add missing entry to the table of contents of room admin API (#10043)Dirk Klimpel2021-05-242-0/+2
|
* Fix `get_state_ids_for_event` return type typo to match what the function ↵Eric Eastwood2021-05-242-1/+2
| | | | | actually does (#10050) It looks like a typo copy/paste from `get_state_for_event` above.
* Don't hammer the database for destination retry timings every ~5mins (#10036)Erik Johnston2021-05-2110-76/+62
|
* Fix /upload 500'ing when presented a very large image (#10029)Michael Telatynski2021-05-213-0/+12
| | | | | | | | | | | | * Fix /upload 500'ing when presented a very large image Catch DecompressionBombError and re-raise as ThumbnailErrors * Set PIL's MAX_IMAGE_PIXELS to match homeserver.yaml to get it to bomb out quicker, to load less into memory in the case of super large images * Add changelog entry for 10029
* Add a test for update_presence (#10033)Andrew Morgan2021-05-212-1/+47
| | | | | | | https://github.com/matrix-org/synapse/issues/9962 uncovered that we accidentally removed all but one of the presence updates that we store in the database when persisting multiple updates. This could cause users' presence state to be stale. The bug was fixed in #10014, and this PR just adds a test that failed on the old code, and was used to initially verify the bug. The test attempts to insert some presence into the database in a batch using `PresenceStore.update_presence`, and then simply pulls it out again.
* Remove unused properties from the SpaceSummaryHandler. (#10038)Patrick Cloke2021-05-212-2/+1
|
* Fixed removal of new presence stream states (#10014)Marek Matys2021-05-212-9/+10
| | | | | | | | | Fixes: https://github.com/matrix-org/synapse/issues/9962 This is a fix for above problem. I fixed it by swaping the order of insertion of new records and deletion of old ones. This ensures that we don't delete fresh database records as we do deletes before inserts. Signed-off-by: Marek Matys <themarcq@gmail.com>
* Add a batching queue implementation. (#10017)Erik Johnston2021-05-213-0/+323
|
* Add `Keyring.verify_events_for_server` and reduce memory usage (#10018)Erik Johnston2021-05-203-22/+94
| | | | | | Also add support for giving a callback to generate the JSON object to verify. This should reduce memory usage, as we no longer have the event in memory in dict form (which has a large memory footprint) for extend periods of time.
* Use ijson to parse the response to `/send_join`, reducing memory usage. (#9958)Erik Johnston2021-05-207-58/+227
| | | Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
* Allow a user who could join a restricted room to see it in spaces summary. ↵Patrick Cloke2021-05-204-54/+254
| | | | | | (#9922) This finishes up the experimental implementation of MSC3083 by showing the restricted rooms in the spaces summary (from MSC2946).
* Merge branch 'master' of github.com:matrix-org/synapse into developAndrew Morgan2021-05-202-1/+2
|\
| * Update user_directory.md (#10016)Javier Junquera Sánchez2021-05-202-1/+2
| | | | | | Signed-off-by: Javier Junquera Sánchez <javier@junquera.io>
* | Update the contrib grafana dashboard (#10001)Erik Johnston2021-05-192-644/+3558
| |
* | Hardened systemd unit files (#9803)Savyasachee Jha2021-05-193-0/+102
| | | | | | Signed-off-by: Savyasachee Jha savya.jha@hawkradius.com
* | Refactor checking restricted join rules (#10007)Patrick Cloke2021-05-184-51/+50
| | | | | | | | | | To be more consistent with similar code. The check now automatically raises an AuthError instead of passing back a boolean. It also absorbs some shared logic between callers.
* | Use a database table to hold the users that should have full presence sent ↵Andrew Morgan2021-05-1811-158/+479
| | | | | | | | to them, instead of something in-memory (#9823)
* | Fix the allowed range of valid ordering characters for spaces. (#10002)Patrick Cloke2021-05-172-2/+3
| | | | | | | | \x7F was meant to be \0x7E (~) this was originally incorrect in MSC1772.
* | Clarify comments in the space summary handler. (#9974)Patrick Cloke2021-05-172-5/+47
| |
* | Merge branch 'master' into developErik Johnston2021-05-174-15/+25
|\|
| * Changelog v1.34.0 github/release-v1.34.0 release-v1.34.0Erik Johnston2021-05-171-6/+6
| |
| * Update UPGRADE.rstErik Johnston2021-05-171-7/+4
| |
| * 1.34.0Erik Johnston2021-05-173-3/+16
| |
* | Split multiplart email sending into a dedicated handler (#9977)Brendan Abolivier2021-05-175-90/+122
| | | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* | Update postgres docs (#9989)Richard van der Hoff2021-05-143-103/+98
| |
* | Run `black` on the scripts (#9981)Richard van der Hoff2021-05-148-76/+141
| | | | | | | | Turns out these scripts weren't getting linted.
* | Minor `@cachedList` enhancements (#9975)Richard van der Hoff2021-05-146-20/+31
| | | | | | | | | | | | - use a tuple rather than a list for the iterable that is passed into the wrapped function, for performance - test that we can pass an iterable and that keys are correctly deduped.
* | Remove unnecessary SystemRandom from SQLBaseStore (#9987)Dan Callahan2021-05-143-3/+3
| | | | | | | | | | | | | | | | It's not obvious that instances of SQLBaseStore each need their own instances of random.SystemRandom(); let's just use random directly. Introduced by 52839886d664576831462e033b88e5aba4c019e3 Signed-off-by: Dan Callahan <danc@element.io>
* | Remove superfluous call to bool() (#9986)Dan Callahan2021-05-142-1/+2
| | | | | | | | | | Our strtobool already returns a bool, so no need to re-cast here Signed-off-by: Dan Callahan <danc@element.io>
* | Simplify exception handling in is_ascii. (#9985)Dan Callahan2021-05-142-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We can get away with just catching UnicodeError here. ⋮ +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError ⋮ https://docs.python.org/3/library/exceptions.html#exception-hierarchy Signed-off-by: Dan Callahan <danc@element.io>
* | Use Python's secrets module instead of random (#9984)Dan Callahan2021-05-142-8/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Functionally identical, but more obviously cryptographically secure. ...Explicit is better than implicit? Avoids needing to know that SystemRandom() implies a CSPRNG, and complies with the big scary red box on the documentation for random: > Warning: > The pseudo-random generators of this module should not be used for > security purposes. For security or cryptographic uses, see the > secrets module. https://docs.python.org/3/library/random.html Signed-off-by: Dan Callahan <danc@element.io>
* | Support enabling opentracing by user (#9978)Richard van der Hoff2021-05-145-17/+56
| | | | | | | | | | Add a config option which allows enabling opentracing by user id, eg for debugging requests made by a test user.
* | Update minimum supported version in postgres.md (#9988)Andrew Morgan2021-05-142-1/+2
| |
* | Update SSO mapping providers documentation about unique IDs. (#9980)Patrick Cloke2021-05-132-7/+12
|/
* Incorporate changes from review v1.34.0rc1Brendan Abolivier2021-05-121-2/+6
|
* Refer and link to the upgrade notes rather than to the file nameBrendan Abolivier2021-05-121-1/+1
|
* 1.34.0rc1Brendan Abolivier2021-05-1236-35/+62
|
* Correctly ratelimit invites when creating a room (#9968)Brendan Abolivier2021-05-126-12/+157
| | | | | * Correctly ratelimit invites when creating a room Also allow ratelimiting for more than one action at a time.
* Change the format of access tokens away from macaroons (#5588)Richard van der Hoff2021-05-129-103/+78
|
* Run cache_joined_hosts_for_event in background (#9951)Erik Johnston2021-05-122-5/+41
|
* Tests for to-device messages (#9965)Richard van der Hoff2021-05-112-0/+202
|
* Sort child events according to MSC1772 for the spaces summary API. (#9954)Patrick Cloke2021-05-113-2/+151
| | | | | | | | | This should help ensure that equivalent results are achieved between homeservers querying for the summary of a space. This implements modified MSC1772 rules, according to MSC2946. The different is that the origin_server_ts of the m.room.create event is not used as a tie-breaker since this might not be known if the homeserver is not part of the room.
* Support fetching the spaces summary via GET over federation. (#9947)Patrick Cloke2021-05-114-0/+29
| | | | | | | | | | | Per changes in MSC2946, the C-S and S-S APIs for spaces summary should use GET requests. Until this is stable, the POST endpoints still exist. This does not switch federation requests to use the GET version yet since it is newly added and already deployed servers might not support it. When switching to the stable endpoint we should switch to GET requests.
* Send the `m.room.create` stripped event with invites (support MSC1772). (#9966)Patrick Cloke2021-05-115-5/+34
| | | | | MSC1772 specifies the m.room.create event should be sent as part of the invite_state. This was done optionally behind an experimental flag, but is now done by default due to MSC1772 being approved.
* Merge branch 'master' into developBrendan Abolivier2021-05-1110-70/+319
|\
| * Use link to advisory rather than to the CVE repo v1.33.2 github/release-v1.33.2 release-v1.33.2Brendan Abolivier2021-05-111-1/+1
| |
| * 1.33.2Brendan Abolivier2021-05-114-2/+23
| |
| * Merge pull request from GHSA-x345-32rc-8h85Richard van der Hoff2021-05-116-68/+296
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests for push rule pattern matching * tests for acl pattern matching * factor out common `re.escape` * Factor out common re.compile * Factor out common anchoring code * add word_boundary support to `glob_to_regex` * Use `glob_to_regex` in push rule evaluator NB that this drops support for character classes. I don't think anyone ever used them. * Improve efficiency of globs with multiple wildcards The idea here is that we compress multiple `*` globs into a single `.*`. We also need to consider `?`, since `*?*` is as hard to implement efficiently as `**`. * add assertion on regex pattern * Fix mypy * Simplify glob_to_regex * Inline the glob_to_regex helper function Signed-off-by: Dan Callahan <danc@element.io> * Moar comments Signed-off-by: Dan Callahan <danc@element.io> Co-authored-by: Dan Callahan <danc@element.io>
* | Add config option to hide device names over federation (#9945)Aaron Raimist2021-05-114-1/+20
| | | | | | | | | | Now that cross signing exists there is much less of a need for other people to look at devices and verify them individually. This PR adds a config option to allow you to prevent device display names from being shared with other servers. Signed-off-by: Aaron Raimist <aaron@raim.ist>
* | Add debug logging for issue #9533 (#9959)Richard van der Hoff2021-05-116-2/+42
| | | | | | | | | | Hopefully this will help us track down where to-device messages are getting lost/delayed.
* | Fix `m.room_key_request` to-device messages (#9961)Richard van der Hoff2021-05-114-26/+32
| | | | | | fixes #9960
* | Improve performance of backfilling in large rooms. (#9935)Erik Johnston2021-05-102-69/+55
| | | | | | | | | | | | We were pulling the full auth chain for the room out of the DB each time we backfilled, which can be *huge* for large rooms and is totally unnecessary.
* | Fix port_db on empty db (#9930)Richard van der Hoff2021-05-106-54/+69
| | | | | | | | ... and test it.
* | Always cache 'event_to_prev_state_group' (#9950)Erik Johnston2021-05-072-6/+8
| | | | | | Fixes regression in send PDU times introduced in #9905.
* | Fix make_full_schema to create the db with the right options and user (#9931)Richard van der Hoff2021-05-072-9/+11
| |
* | Merge remote-tracking branch 'origin/release-v1.33.2' into developErik Johnston2021-05-072-1/+2
|\|
| * Unpin attrs dep after new version has been released (#9946)Erik Johnston2021-05-072-1/+2
| | | | | | c.f. #9936
* | Reorganise the database schema directories (#9932)Richard van der Hoff2021-05-07284-47/+81
| | | | | | | | | | The hope here is that by moving all the schema files into synapse/storage/schema, it gets a bit easier for newcomers to navigate. It certainly got easier for me to write a helpful README. There's more to do on that front, but I'll follow up with other PRs for that.
* | Export jemalloc stats to prometheus when used (#9882)Erik Johnston2021-05-064-0/+200
| |
* | Revert "Leave out optional keys from /sync (#9919)" (#9940)Erik Johnston2021-05-064-51/+50
| | | | | | This reverts commit e9eb3549d32a6f93d07de8dbd5e1ebe54c8d8278.
* | Merge remote-tracking branch 'origin/master' into developErik Johnston2021-05-064-2/+18
|\|
| * 1.33.1 v1.33.1 github/release-v1.33.1 release-v1.33.1Erik Johnston2021-05-064-2/+16
| |
| * Pin attrs to <21.1.0 (#9937)Erik Johnston2021-05-062-1/+3
| | | | | | Fixes #9936
* | Follow-up to #9915 to correct the identifier for room types.Patrick Cloke2021-05-051-1/+1
| |
* | Include the time of the create event in Spaces Summary. (#9928)Patrick Cloke2021-05-052-0/+2
| | | | | | | | | | This is an update based on changes to MSC2946. The origin_server_ts of the m.room.create event is copied into the creation_ts field for each room returned from the spaces summary.
* | Increase perf of handling presence when joining large rooms. (#9916)Erik Johnston2021-05-053-82/+87
| |
* | Support stable MSC1772 spaces identifiers. (#9915)Patrick Cloke2021-05-053-2/+10
| | | | | | | | Support both the unstable and stable identifiers. A future release will disable the unstable identifiers.
* | Optionally track memory usage of each LruCache (#9881)Erik Johnston2021-05-058-1/+97
| | | | | | | | | | This will double count slightly in the presence of interned strings. It's off by default as it can consume a lot of resources.
* | Limit how often GC happens by time. (#9902)Erik Johnston2021-05-056-3/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse can be quite memory intensive, and unless care is taken to tune the GC thresholds it can end up thrashing, causing noticable performance problems for large servers. We fix this by limiting how often we GC a given generation, regardless of current counts/thresholds. This does not help with the reverse problem where the thresholds are set too high, but that should only happen in situations where they've been manually configured. Adds a `gc_min_seconds_between` config option to override the defaults. Fixes #9890.
* | Don't set the external cache if its been done recently (#9905)Erik Johnston2021-05-053-5/+34
| |
* | Use get_current_users_in_room from store and not StateHandler (#9910)Erik Johnston2021-05-0512-17/+26
| |
* | Docker healthcheck timings - add startup delay and changed interval (#9913)Christopher May-Townsend2021-05-053-4/+16
| | | | | | | | | | * Add healthcheck startup delay by 5secs and reduced interval check to 15s to reduce waiting time for docker aware edge routers bringing an instance online
* | Leave out optional keys from /sync (#9919)DeepBlueV7.X2021-05-054-50/+51
| | | | | | | | | | | | This leaves out all optional keys from /sync. This should be fine for all clients tested against conduit already, but it may break some clients, as such we should check, that at least most of them don't break horribly and maybe back out some of the individual changes. (We can probably always leave out groups for example, while the others may cause more issues.) Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* | Merge branch 'master' into developBrendan Abolivier2021-05-054-4/+20
|\|
| * 1.33.0 v1.33.0 github/release-v1.33.0 release-v1.33.0Brendan Abolivier2021-05-054-2/+16
| |
| * Build Debian packages for Ubuntu 21.04 Hirsute (#9909)Dan Callahan2021-04-302-3/+5
| | | | | | Signed-off-by: Dan Callahan <danc@element.io>
* | Time external cache response time (#9904)Erik Johnston2021-05-042-10/+27
| |
* | Add port parameter to the sample config for psycopg2 args (#9911)Andrew Morgan2021-05-043-0/+3
| | | | | | Adds the `port` option with the default value to the sample config file.
* | Merge tag 'v1.33.0rc2' into developAndrew Morgan2021-04-293-2/+10
|\| | | | | | | | | | | | | | | | | | | Synapse 1.33.0rc2 (2021-04-29) ============================== Bugfixes -------- - Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](https://github.com/matrix-org/synapse/issues/9900))
| * typo in changelog v1.33.0rc2Andrew Morgan2021-04-291-1/+1
| |
| * 1.33.0rc2Andrew Morgan2021-04-293-2/+10
| |
* | Add missing type hints to handlers and fix a Spam Checker type hint. (#9896)Patrick Cloke2021-04-298-54/+82
| | | | | | | | | | The user_may_create_room_alias method on spam checkers declared the room_alias parameter as a str when in reality it is passed a RoomAlias object.
* | Delete room endpoint (#9889)ThibF2021-04-295-64/+128
| | | | | | | | | | | | Support the delete of a room through DELETE request and mark previous request as deprecated through documentation. Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>
* | Merge remote-tracking branch 'origin/release-v1.33.0' into developErik Johnston2021-04-283-1/+46
|\|
| * Fix tight loop handling presence replication. (#9900)Erik Johnston2021-04-283-1/+46
| | | | | | | | | | Only affects workers. Introduced in #9819. Fixes #9899.
* | Use the parent's logging context name for runWithConnection. (#9895)Patrick Cloke2021-04-282-1/+4
| | | | | | | | | | This fixes a regression where the logging context for runWithConnection was reported as runWithConnection instead of the connection name, e.g. "POST-XYZ".
* | Merge tag 'v1.33.0rc1' into developAndrew Morgan2021-04-2841-173/+147
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.33.0rc1 (2021-04-28) ============================== Features -------- - Update experimental support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083): restricting room access via group membership. ([\#9800](https://github.com/matrix-org/synapse/issues/9800), [\#9814](https://github.com/matrix-org/synapse/issues/9814)) - Add experimental support for handling presence on a worker. ([\#9819](https://github.com/matrix-org/synapse/issues/9819), [\#9820](https://github.com/matrix-org/synapse/issues/9820), [\#9828](https://github.com/matrix-org/synapse/issues/9828), [\#9850](https://github.com/matrix-org/synapse/issues/9850)) - Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](https://github.com/matrix-org/synapse/issues/9832)) Bugfixes -------- - Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](https://github.com/matrix-org/synapse/issues/9726)) - Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](https://github.com/matrix-org/synapse/issues/9788)) - Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](https://github.com/matrix-org/synapse/issues/9802)) - Limit the size of HTTP responses read over federation. ([\#9833](https://github.com/matrix-org/synapse/issues/9833)) - Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](https://github.com/matrix-org/synapse/issues/9867)) - Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](https://github.com/matrix-org/synapse/issues/9868)) Improved Documentation ---------------------- - Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](https://github.com/matrix-org/synapse/issues/9801)) Internal Changes ---------------- - Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](https://github.com/matrix-org/synapse/issues/9162)) - Apply `pyupgrade` across the codebase. ([\#9786](https://github.com/matrix-org/synapse/issues/9786)) - Move some replication processing out of `generic_worker`. ([\#9796](https://github.com/matrix-org/synapse/issues/9796)) - Replace `HomeServer.get_config()` with inline references. ([\#9815](https://github.com/matrix-org/synapse/issues/9815)) - Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](https://github.com/matrix-org/synapse/issues/9816)) - Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](https://github.com/matrix-org/synapse/issues/9817)) - Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](https://github.com/matrix-org/synapse/issues/9821)) - Small speed up for joining large remote rooms. ([\#9825](https://github.com/matrix-org/synapse/issues/9825)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](https://github.com/matrix-org/synapse/issues/9838)) - Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](https://github.com/matrix-org/synapse/issues/9845)) - Limit length of accepted email addresses. ([\#9855](https://github.com/matrix-org/synapse/issues/9855)) - Remove redundant `synapse.types.Collection` type definition. ([\#9856](https://github.com/matrix-org/synapse/issues/9856)) - Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](https://github.com/matrix-org/synapse/issues/9858)) - Disable invite rate-limiting by default when running the unit tests. ([\#9871](https://github.com/matrix-org/synapse/issues/9871)) - Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](https://github.com/matrix-org/synapse/issues/9874)) - Make `DomainSpecificString` an `attrs` class. ([\#9875](https://github.com/matrix-org/synapse/issues/9875)) - Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](https://github.com/matrix-org/synapse/issues/9876)) - Remove redundant `_PushHTTPChannel` test class. ([\#9878](https://github.com/matrix-org/synapse/issues/9878)) - Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](https://github.com/matrix-org/synapse/issues/9879)) - Small performance improvement around handling new local presence updates. ([\#9887](https://github.com/matrix-org/synapse/issues/9887))
| * Reword account validity template change to sound less like a bugfix v1.33.0rc1Andrew Morgan2021-04-281-1/+1
| |
| * 1.33.0rc1Andrew Morgan2021-04-2836-35/+54
| |
| * Revert "Experimental Federation Speedup (#9702)"Andrew Morgan2021-04-285-138/+93
| | | | | | | | This reverts commit 05e8c70c059f8ebb066e029bc3aa3e0cefef1019.
* | Reduce memory footprint of caches (#9886)Erik Johnston2021-04-282-18/+60
| |
* | Add type hints to presence handler (#9885)Erik Johnston2021-04-282-70/+90
|/
* Remove various bits of compatibility code for Python <3.6 (#9879)Andrew Morgan2021-04-2716-98/+29
| | | I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
* Pass errors back to the client when trying multiple federation destinations. ↵Patrick Cloke2021-04-272-58/+61
| | | | | | | | (#9868) This ensures that something like an auth error (403) will be returned to the requester instead of attempting to try more servers, which will likely result in the same error, and then passing back a generic 400 error.
* Use current state table for `presence.get_interested_remotes` (#9887)Erik Johnston2021-04-272-7/+3
| | | This should be a lot quicker than asking the state handler.
* Improved validation for received requests (#9817)Richard van der Hoff2021-04-2315-31/+174
| | | | | | * Simplify `start_listening` callpath * Correctly check the size of uploaded files
* Kill off `_PushHTTPChannel`. (#9878)Richard van der Hoff2021-04-233-121/+20
| | | | | First of all, a fixup to `FakeChannel` which is needed to make it work with the default HTTP channel implementation. Secondly, it looks like we no longer need `_PushHTTPChannel`, because as of #8013, the producer that gets attached to the `HTTPChannel` is now an `IPushProducer`. This is good, because it means we can remove a whole load of test-specific boilerplate which causes variation between tests and production.
* Allow OIDC cookies to work on non-root public baseurls (#9726)Andrew Morgan2021-04-233-9/+22
| | | | | Applied a (slightly modified) patch from https://github.com/matrix-org/synapse/issues/9574. As far as I understand this would allow the cookie set during the OIDC flow to work on deployments using public baseurls that do not sit at the URL path root.
* pass a reactor into SynapseSite (#9874)Richard van der Hoff2021-04-237-24/+43
|
* Add type hints to auth and auth_blocking. (#9876)Patrick Cloke2021-04-234-44/+48
|
* Make DomainSpecificString an attrs class (#9875)Erik Johnston2021-04-234-8/+24
|
* Remove room and user invite ratelimits in default unit test config (#9871)Andrew Morgan2021-04-232-0/+5
|
* Split presence out of master (#9820)Erik Johnston2021-04-2317-245/+245
|
* Check for space membership during a remote join of a restricted room (#9814)Patrick Cloke2021-04-236-68/+131
| | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* Only store data in caches, not "smart" objects (#9845)Erik Johnston2021-04-233-141/+182
|
* Limit the size of HTTP responses read over federation. (#9833)Richard van der Hoff2021-04-234-8/+110
|
* Handle all new rate limits in demo scripts (#9858)manuroe2021-04-222-12/+43
|
* Limit length of accepted email addresses (#9855)Erik Johnston2021-04-226-7/+100
|
* Clear the resync bit after resyncing device lists (#9867)Richard van der Hoff2021-04-223-10/+17
| | | Fixes #9866.
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-2226-62/+77
| | | This is no longer required, since we have dropped support for Python 3.5.
* Merge branch 'master' into developAndrew Morgan2021-04-224-4/+30
|\
| * Merge tag 'v1.32.2'Andrew Morgan2021-04-224-4/+30
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.32.2 (2021-04-22) =========================== This release includes a fix for a regression introduced in 1.32.0. Bugfixes -------- - Fix a regression in Synapse 1.32.0 and 1.32.1 which caused `LoggingContext` errors in plugins. ([\#9857](https://github.com/matrix-org/synapse/issues/9857))
| | * A regression can't be introduced twice v1.32.2 github/release-v1.32.2 release-v1.32.2Andrew Morgan2021-04-221-2/+2
| | |
| | * Note regression was in 1.32.0 and 1.32.1Andrew Morgan2021-04-221-2/+2
| | |
| | * Update dates in changelogsAndrew Morgan2021-04-212-2/+2
| | |
| | * 1.32.2Andrew Morgan2021-04-214-2/+18
| | |
| | * Merge branch 'release-v1.32.1' of github.com:matrix-org/synapse into ↵Andrew Morgan2021-04-212-9/+16
| | |\ | | | | | | | | | | | | release-v1.32.2
| | * | Make LoggingContext's name optional (#9857)Richard van der Hoff2021-04-212-3/+13
| | | | | | | | | | | | | | | | Fixes https://github.com/matrix-org/synapse-s3-storage-provider/issues/55
* | | | Merge branch 'master' of github.com:matrix-org/synapse into developAndrew Morgan2021-04-212-9/+16
|\| | |
| * | | Merge branch 'release-v1.32.1' of github.com:matrix-org/synapseAndrew Morgan2021-04-212-9/+16
| |\ \ \ | | | |/ | | |/|
| | * | Note LoggingContext signature change incompatibility in 1.32.0 (#9859) github/release-v1.32.1 release-v1.32.1Andrew Morgan2021-04-212-9/+16
| | |/ | | | | | | | | | | | | 1.32.0 also introduced an incompatibility with Synapse modules that make use of `synapse.logging.context.LoggingContext`, such as [synapse-s3-storage-provider](https://github.com/matrix-org/synapse-s3-storage-provider). This PR adds a note to the 1.32.0 changelog and upgrade notes about it.
* | | Merge branch 'master' into developAndrew Morgan2021-04-216-15/+49
|\| |
| * | Merge tag 'v1.32.1'Andrew Morgan2021-04-216-15/+49
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.32.1 (2021-04-21) =========================== This release fixes [a regression](https://github.com/matrix-org/synapse/issues/9853) in Synapse 1.32.0 that caused connected Prometheus instances to become unstable. If you ran Synapse 1.32.0 with Prometheus metrics, first upgrade to Synapse 1.32.1 and follow [these instructions](https://github.com/matrix-org/synapse/pull/9854#issuecomment-823472183) to clean up any excess writeahead logs. Bugfixes -------- - Fix a regression in Synapse 1.32.0 which caused Synapse to report large numbers of Prometheus time series, potentially overwhelming Prometheus instances. ([\#9854](https://github.com/matrix-org/synapse/issues/9854))
| | * Clarify 1.32.0/1 changelog and upgrade notes v1.32.1Andrew Morgan2021-04-212-8/+7
| | |
| | * Add regression notes to CHANGES.md; fix link in 1.32.0 changelogAndrew Morgan2021-04-211-2/+6
| | |
| | * Add link to fixing prometheus to 1.32.0 upgrade notes; 1.32.1 has a fixAndrew Morgan2021-04-212-2/+6
| | |
| | * Fix typo in link to regression in 1.32.0 upgrade notesAndrew Morgan2021-04-211-1/+1
| | |
| | * 1.32.1Andrew Morgan2021-04-214-2/+16
| | |
| | * Mention Prometheus metrics regression in v1.32.0Andrew Morgan2021-04-212-0/+15
| | |
| | * Stop BackgroundProcessLoggingContext making new prometheus timeseries (#9854)Richard van der Hoff2021-04-213-5/+18
| | | | | | | | | | | | This undoes part of b076bc276e881b262048307b6a226061d96c4a8d.
* | | Rename handler and config modules which end in handler/config. (#9816)Patrick Cloke2021-04-2017-35/+43
| | |
* | | Merge branch 'master' into developAndrew Morgan2021-04-202-0/+15
|\| |
| * | Mention Prometheus metrics regression in v1.32.0 github/release-v1.32.0 release-v1.32.0Andrew Morgan2021-04-202-0/+15
| |/
* | Merge branch 'master' into developAndrew Morgan2021-04-201-0/+9
|\|
| * Further tweaking on gpg signing key noticeAndrew Morgan2021-04-201-2/+5
| |
| * Add note about expired Debian gpg signing keys to CHANGES.mdAndrew Morgan2021-04-201-0/+6
| |
* | Merge branch 'master' into developAndrew Morgan2021-04-2011-40/+62
|\|
| * Update v1.32.0 changelog. It's m.login.application_service, not plural v1.32.0Andrew Morgan2021-04-201-1/+1
| |
| * 1.32.0Andrew Morgan2021-04-204-6/+18
| |
| * Add Application Service registration type requirement + py35, pg95 ↵Andrew Morgan2021-04-201-0/+18
| | | | | | | | | | | | | | deprecation notices to v1.32.0 upgrade notes (#9849) Fixes https://github.com/matrix-org/synapse/issues/9846. Adds important removal information from the top of https://github.com/matrix-org/synapse/releases/tag/v1.32.0rc1 into UPGRADE.rst.
| * Always use the name as the log ID. (#9829)Patrick Cloke2021-04-208-34/+26
| | | | | | | | | | As far as I can tell our logging contexts are meant to log the request ID, or sometimes the request ID followed by a suffix (this is generally stored in the name field of LoggingContext). There's also code to log the name@memory location, but I'm not sure this is ever used. This simplifies the code paths to require every logging context to have a name and use that in logging. For sub-contexts (created via nested_logging_contexts, defer_to_threadpool, Measure) we use the current context's str (which becomes their name or the string "sentinel") and then potentially modify that (e.g. add a suffix).
* | Add presence federation stream (#9819)Erik Johnston2021-04-206-31/+426
| |
* | Fix bug where we sent remote presence states to remote servers (#9850)Erik Johnston2021-04-203-3/+13
| |
* | Fix (final) Bugbear violations (#9838)Jonathan de Jong2021-04-2023-49/+46
| |
* | Port "Allow users to click account renewal links multiple times without ↵Andrew Morgan2021-04-1918-263/+496
| | | | | | | | | | hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).
* | Sanity check identity server passed to bind/unbind. (#9802)Denis Kasak2021-04-193-3/+59
| | | | | | | | Signed-off-by: Denis Kasak <dkasak@termina.org.uk>
* | Don't send normal presence updates over federation replication stream (#9828)Erik Johnston2021-04-195-183/+75
| |
* | User directory: use calculated room membership state instead (#9821)Andrew Morgan2021-04-163-7/+36
| | | | | | | | | | Fixes: #9797. Should help reduce CPU usage on the user directory, especially when memberships change in rooms with lots of state history.
* | Small speed up joining large remote rooms (#9825)Erik Johnston2021-04-162-21/+34
| | | | | | | | | | There are a couple of points in `persist_events` where we are doing a query per event in series, which we can replace.
* | remove `HomeServer.get_config` (#9815)Richard van der Hoff2021-04-149-18/+16
| | | | | | | | Every single time I want to access the config object, I have to remember whether or not we use `get_config`. Let's just get rid of it.
* | Separate creating an event context from persisting it in the federation ↵Patrick Cloke2021-04-143-67/+118
| | | | | | | | | | | | handler (#9800) This refactoring allows adding logic that uses the event context before persisting it.
* | Revert "Check for space membership during a remote join of a restricted ↵Patrick Cloke2021-04-147-238/+131
| | | | | | | | | | | | | | | | room. (#9763)" This reverts commit cc51aaaa7adb0ec2235e027b5184ebda9b660ec4. The PR was prematurely merged and not yet approved.
* | Check for space membership during a remote join of a restricted room. (#9763)Patrick Cloke2021-04-147-131/+238
| | | | | | | | | | | | | | When receiving a /send_join request for a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* | Experimental Federation Speedup (#9702)Jonathan de Jong2021-04-145-97/+129
| | | | | | | | | | This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max). Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* | Move some replication processing out of generic_worker (#9796)Erik Johnston2021-04-146-483/+486
| | | | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | More robust handling of the Content-Type header for thumbnail generation (#9788)rkfg2021-04-143-0/+5
| | | | | | | | Signed-off-by: Sergey Shpikin <rkfg@rkfg.me>
* | Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-14651-651/+1
| | | | | | | | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* | Add a dockerfile for running a set of Synapse worker processes (#9162)Andrew Morgan2021-04-1411-6/+867
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds a Dockerfile and some supporting files to the `docker/` directory. The Dockerfile's intention is to spin up a container with: * A Synapse main process. * Any desired worker processes, defined by a `SYNAPSE_WORKERS` environment variable supplied at runtime. * A redis for worker communication. * A nginx for routing traffic. * A supervisord to start all worker processes and monitor them if any go down. Note that **this is not currently intended to be used in production**. If you'd like to use Synapse workers with Docker, instead make use of the official image, with one worker per container. The purpose of this dockerfile is currently to allow testing Synapse in worker mode with the [Complement](https://github.com/matrix-org/complement/) test suite. `configure_workers_and_start.py` is where most of the magic happens in this PR. It reads from environment variables (documented in the file) and creates all necessary config files for the processes. It is the entrypoint of the Dockerfile, and thus is run any time the docker container is spun up, recreating all config files in case you want to use a different set of workers. One can specify which workers they'd like to use by setting the `SYNAPSE_WORKERS` environment variable (as a comma-separated list of arbitrary worker names) or by setting it to `*` for all worker processes. We will be using the latter in CI. Huge thanks to @MatMaul for helping get this all working :tada: This PR is paired with its equivalent on the Complement side: https://github.com/matrix-org/complement/pull/62. Note, for the purpose of testing this PR before it's merged: You'll need to (re)build the base Synapse docker image for everything to work (`matrixdotorg/synapse:latest`). Then build the worker-based docker image on top (`matrixdotorg/synapse:workers`).
* | Add note to docker docs explaining platform support (#9801)Andrew Morgan2021-04-142-3/+7
|/ | | | | | | Context is in https://github.com/matrix-org/synapse/issues/9764#issuecomment-818615894. I struggled to find a more official link for this. The problem occurs when using WSL1 instead of WSL2, which some Windows platforms (at least Server 2019) still don't have. Docker have updated their documentation to paint a much happier picture now given WSL2's support. The last sentence here can probably be removed once WSL1 is no longer around... though that will likely not be for a very long time.
* Update changelog for v1.32.0 v1.32.0rc1Andrew Morgan2021-04-131-1/+1
|
* 1.32.0rc1Andrew Morgan2021-04-1337-36/+73
|
* Add release helper script (#9713)Erik Johnston2021-04-133-0/+252
| | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Bump black configuration to target py36 (#9781)Dan Callahan2021-04-1311-15/+16
| | | Signed-off-by: Dan Callahan <danc@element.io>
* Add an admin API to manage ratelimit for a specific user (#9648)Dirk Klimpel2021-04-136-6/+573
|
* Drop Python 3.5 from Trove classifier metadata. (#9782)Dan Callahan2021-04-122-1/+1
| | | | | * Drop Python 3.5 from Trove classifier metadata. Signed-off-by: Dan Callahan <danc@element.io>
* Add option to skip unit tests when building debs (#9793)Dan Callahan2021-04-124-13/+34
| | | Signed-off-by: Dan Callahan <danc@element.io>
* Require AppserviceRegistrationType (#9548)Will Hunt2021-04-125-23/+60
| | | This change ensures that the appservice registration behaviour follows the spec. We decided to do this for Dendrite, so it made sense to also make a PR for synapse to correct the behaviour.
* Use mock from the stdlib. (#9772)Patrick Cloke2021-04-0982-126/+86
|
* Fix duplicate logging of exceptions in transaction processing (#9780)Richard van der Hoff2021-04-092-7/+4
| | | There's no point logging this twice.
* Enable complement tests for MSC2946. (#9771)Patrick Cloke2021-04-092-1/+2
| | | By providing the additional build tag for `msc2946`.
* Proof of concept for GitHub Actions (#9661)Dan Callahan2021-04-092-0/+323
| | | Signed-off-by: Dan Callahan <danc@element.io>
* Remove old admin API `GET /_synapse/admin/v1/users/<user_id>` (#9401)Dirk Klimpel2021-04-095-27/+16
| | | | | | | | | | | | | Related: #8334 Deprecated in: #9429 - Synapse 1.28.0 (2021-02-25) `GET /_synapse/admin/v1/users/<user_id>` has no - unit tests - documentation API in v2 is available (#5925 - 12/2019, v1.7.0). API is misleading. It expects `user_id` and returns a list of all users. Signed-off-by: Dirk Klimpel dirk@klimpel.org
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-0838-113/+224
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Merge pull request #9766 from matrix-org/rav/drop_py35Richard van der Hoff2021-04-0813-111/+24
|\ | | | | Require py36, Postgres 9.6, and sqlite 3.22
| * Merge remote-tracking branch 'origin/develop' into rav/drop_py35Richard van der Hoff2021-04-0867-825/+2182
| |\ | |/ |/|
* | Merge pull request #9769 from matrix-org/rav/fix_bionicRichard van der Hoff2021-04-083-11/+18
|\ \ | | | | | | Fix incompatibility with bionic
* \ \ Merge branch 'erikj/fix_stalled_catchup' into developErik Johnston2021-04-082-2/+5
|\ \ \
| * | | Fix sharded federation sender sometimes using 100% CPU.Erik Johnston2021-04-082-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | We pull all destinations requiring catchup from the DB in batches. However, if all those destinations get filtered out (due to the federation sender being sharded), then the `last_processed` destination doesn't get updated, and we keep requesting the same set repeatedly.
* | | | Put opencontainers labels to the final image (#9765)Johannes Wienke2021-04-082-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | They don't make any sense on the intermediate builder image. The final images needs them to be of use for anyone. Signed-off-by: Johannes Wienke <languitar@semipol.de>
* | | | MSC3083: Check for space membership during a local join of restricted rooms. ↵Patrick Cloke2021-04-083-2/+76
|/ / / | | | | | | | | | | | | | | | | | | | | | (#9735) When joining a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.
* | | Record more information into structured logs. (#9654)Patrick Cloke2021-04-0810-88/+255
| | | | | | | | | | | | Records additional request information into the structured logs, e.g. the requester, IP address, etc.
* | | Don't report anything from GaugeBucketCollector metrics until data is ↵Andrew Morgan2021-04-062-3/+14
| | | | | | | | | | | | | | | present (#8926) This PR modifies `GaugeBucketCollector` to only report data once it has been updated, rather than initially reporting a value of 0. Fixes zero values being reported for some metrics on startup until a background job to update the metric's value runs later.
* | | Add a Synapse Module for configuring presence update routing (#9491)Andrew Morgan2021-04-0614-64/+1282
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state.
* | | Add type hints to expiring cache. (#9730)Patrick Cloke2021-04-068-54/+65
| | |
* | | Fix reported bugbear: too broad exception assertion (#9753)Andrew Morgan2021-04-062-2/+4
| | |
* | | Remove outdated constraint on remote_media_cache_thumbnails (#9725)Richard van der Hoff2021-04-063-3/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The `remote_media_cache_thumbnails_media_origin_media_id_thumbna_key` constraint is superceded by `remote_media_repository_thumbn_media_origin_id_width_height_met` (which adds `thumbnail_method` to the unique key). PR #7124 made an attempt to remove the old constraint, but got the name wrong, so it didn't work. Here we update the bg update and rerun it. Fixes #8649.
* | | Merge branch 'master' into developErik Johnston2021-04-068-8/+72
|\ \ \
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-22 04:52:39 +0000