summary refs log tree commit diff
path: root/synapse/rest/client/transactions.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2021-11-12Delete local RoomAccessRules module (#112)Brendan Abolivier3-2211/+0
2021-11-04Remove now useless config optionBrendan Abolivier1-3/+0
2021-11-04LintBrendan Abolivier2-12/+58
2021-11-04Delete custom user_may_create_roomBrendan Abolivier2-57/+1
Because we already do these checks with RoomAccessRules
2021-11-04Update testsBrendan Abolivier1-47/+123
2021-11-04Update DomainRuleChecker to use the new interfaceBrendan Abolivier1-32/+108
2021-11-03Fix debian package builds. (#10931) (#109)Brendan Abolivier2-2/+4
This was due to dh-virtualenv builds being broken due to Shpinx removing deprecated APIs. Co-authored-by: Erik Johnston <erik@matrix.org>
2021-11-01Add a module API method to retrieve state from a room (#11204)Brendan Abolivier3-2/+81
2021-10-26Converge with mainlineBrendan Abolivier5-129/+26
Bring other callbacks to party with mainline, and fixup code calling to the various callbacks.
2021-10-26Add a spamchecker method to allow or deny 3pid invites (#10894)Brendan Abolivier5-4/+153
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
2021-10-26Add a spamchecker callback to allow or deny room joins (#10910)Brendan Abolivier5-6/+156
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-10-26Add a spamchecker callback to allow or deny room creation based on invites ↵Brendan Abolivier5-17/+358
(#10898) This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
2021-09-27Revert shadow HS support (#104)Brendan Abolivier9-272/+70
Revert shadow HS support added in matrix-org/synapse#4145 Fixes matrix-org/matrix-dinsic#803 Part of that PR isn't reverted because it relates to matrix-org/matrix-dinsic#793
2021-09-27Remove duplicate jobBrendan Abolivier1-3/+0
Forgot to include this commit in #106
2021-09-27Use correct tag for sytest (#106)Brendan Abolivier2-9/+6
2021-09-22Use the BaseReporter super-class for _WrappedRustReporter (#105)Brendan Abolivier2-1/+12
* Use the BaseReporter super-class for _WrappedRustReporter. (#10799) This fixes mypy errors with jaeger-client >= 4.7.0 and should be a no-op for versions before that. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-09-21Fix typo again v1.43.0David Robertson1-1/+1
2021-09-21Specify MSC name; fix typoDavid Robertson1-3/+3
one day I'll learn how to spell hierarchy
2021-09-21Point to upgrade notesDavid Robertson1-0/+2
2021-09-21Move deprecation notice from 1.43 rc to releaseDavid Robertson1-3/+2
2021-09-21 1.43.0David Robertson3-1/+13
2021-09-17Add hyperlink to #9424 in changelog v1.43.0rc2David Robertson1-1/+1
2021-09-17 1.43.0rc2David Robertson4-2/+16
2021-09-15Cleanup opentracing logging for syncs (#10828)Erik Johnston2-126/+142
We added a bunch of spans in #10704, but this ended up adding a lot of redundant spans for rooms where nothing changed, so instead we only start the span if there might be something interesting going on.
2021-09-14update changelog wording v1.43.0rc1Andrew Morgan1-7/+7
2021-09-14changelog improvementsAndrew Morgan1-3/+3
2021-09-14update changelog wordingAndrew Morgan1-1/+1
2021-09-14update wordingAndrew Morgan1-1/+1
2021-09-14 1.43.0rc1Andrew Morgan54-52/+76
2021-09-13Use direct references for some configuration variables (#10798)Patrick Cloke66-133/+152
Instead of proxying through the magic getter of the RootConfig object. This should be more performant (and is more explicit).
2021-09-13Fix copy-paste error in the password section of the sample-config. (#10804)BramvdnHeuvel3-2/+3
2021-09-13Use the BaseReporter super-class for _WrappedRustReporter. (#10799)Patrick Cloke2-1/+12
This fixes mypy errors with jaeger-client >= 4.7.0 and should be a no-op for versions before that.
2021-09-10Add types to synapse.util. (#10601)reivilibre41-254/+401
2021-09-10Fix 2 typos in docs/log_contexts.md (#10795)reivilibre2-2/+3
2021-09-10Handle room upgrades for spaces (#10774)Patrick Cloke3-6/+81
By copying the `room_type` field of the create event and migrating any non-empty `m.space.child` events to the new room that is created.
2021-09-10Easy refactors of the user directory (#10789)David Robertson8-37/+90
No functional changes here. This came out as I was working to tackle #5677
2021-09-10Remove fixed and flakey tests from the sytest blacklist (#10788)David Robertson3-18/+2
We want to blacklist only known, consistent failures. We should deflake tests rather than ignoring them.
2021-09-10Ask consent on SSO registration with default mxid (#10733)AndrewFerr3-23/+63
Fixes #10732: consent flow skipped during SSO user registration if username is left at default Signed-off-by: Andrew Ferrazzutti fair@miscworks.net
2021-09-10Don't needlessly batch in `add_event_to_cache` (#10784)Erik Johnston2-27/+25
We've already batched up the events previously, and assume in other places in the events.py file that we have. Removing this makes it easier to adjust the batch sizes in one place.
2021-09-09Remove unstable MSC2858 API, including `experimental.msc2858_enabled` config ↵Sean9-127/+19
option (#10693) Signed-off-by: Sean Quah <seanq@element.io>
2021-09-09Prefer room v9 for restricted rooms. (#10772)Patrick Cloke2-1/+2
Hint to clients via the room capabilities API (MSC3244) that room version 9 should be preferred for creating a room with restricted join rules (instead of room version 8).
2021-09-08Get rid of `_auth_and_persist_event` (#10781)Richard van der Hoff3-63/+43
This is only called in two places, and the code seems much clearer without it.
2021-09-08Improve the modules doc (#10758)Brendan Abolivier10-404/+537
* Split up the documentation in several files rather than one huge one * Add examples for each callback category * Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632 * Add titles to callbacks so they're easier to find in the navigation panels and link to
2021-09-08Fix frontend_proxy jinja script in docker workers (#10783)kegsay2-1/+2
2021-09-08Add a script to update the debian changelog for non-Debian systems (#10778)Brendan Abolivier2-0/+65
When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
2021-09-08Document that /spaces & /hierarchy can be routed to workers. (#10648)Patrick Cloke4-25/+33
Also refactors some of the registration of endpoints on workers.
2021-09-08Skip handling of push actions for outlier events (#10780)Erik Johnston3-4/+19
Outlier events don't ever have push actions associated with them, so we can skip some expensive queries during event persistence.
2021-09-08Add a constant for m.federate. (#10775)Patrick Cloke8-10/+28
2021-09-08Change logging of puppeted requests to better differentiate users (#10779)Erik Johnston2-1/+2
This used to be a comma and got accidentally changed to a period in #9654, but a pipe character is more easier to parse visually.
2021-09-08Request JSON for oEmbed requests (and ignore XML only providers). (#10759)Patrick Cloke4-8/+98
This adds the format to the request arguments / URL to ensure that JSON data is returned (which is all that Synapse supports). This also adds additional error checking / filtering to the configuration file to ignore XML-only providers.
2021-09-08Add some assertions about outliers (#10773)Richard van der Hoff2-71/+78
I think I have finally teased apart the codepaths which handle outliers, and those that handle non-outliers. Let's add some assertions to demonstrate my newfound knowledge.
2021-09-08Persist auth events before the events that rely on them (#10771)Richard van der Hoff2-36/+66
If we're persisting an event E which has auth_events A1, A2, then we ought to make sure that we correctly auth and persist A1 and A2, before we blindly accept E. This PR does part of that - it persists the auth events first - but it does not fully solve the problem, because we still don't check that the auth events weren't rejected.
2021-09-07Add "No significant changes" to changelog v1.42.0Azrenbeth1-0/+2
2021-09-07Move upgrade notice up in changelogAzrenbeth1-6/+1
2021-09-071.42.0Azrenbeth3-1/+13
2021-09-07Use attrs internally for the URL preview code & add documentation. (#10753)Patrick Cloke5-119/+132
2021-09-07Return stripped m.space.child events via the space summary. (#10760)Patrick Cloke2-14/+13
The full event content cannot be trusted from this API (as no auth chain, etc.) is processed over federation. Returning the full event content was a bug as MSC2946 specifies that only the stripped state should be returned. This also avoids calculating aggregations / annotations which go unused.
2021-09-07Underscore-prefix private fields in `FederationEventHandler` (#10746)Richard van der Hoff2-71/+74
2021-09-06Add logging to help debug #9424 (#10704)Azrenbeth2-13/+55
2021-09-06Add config option to use non-default manhole password and keys (#10643)Azrenbeth9-17/+161
2021-09-06Improve changelog wording v1.42.0rc2Brendan Abolivier1-1/+1
2021-09-06Expand on why users should read upgrade notesBrendan Abolivier1-1/+4
2021-09-06Move the upgrade notes reminder up to rc2Brendan Abolivier1-3/+2
2021-09-06 1.42.0rc2Brendan Abolivier5-3/+22
2021-09-06Add a warning when using deprecated template_dir settings (#10768)Brendan Abolivier4-0/+42
The deprecation itself happened in #10596 which shipped with Synapse v1.41.0. However, it doesn't seem fair to suddenly drop support for these settings in ~4-6w without being more vocal about said deprecation.
2021-09-06Stop using BaseHandler in `FederationEventHandler` (#10745)Richard van der Hoff3-10/+12
It's now only used in a couple of places, so we can drop it altogether.
2021-09-06Allow `room_alias_name` parameter to be handled by /createRoom calls on ↵Andrew Morgan2-2/+3
workers (#10757)
2021-09-06Ease searching for M_TOO_LARGE-related error codes (#10750)Andrew Morgan2-9/+7
2021-09-06Doc: Clarifying undoing room shutdowns (#10480)David Teller2-15/+28
2021-09-06Move `maybe_kick_guest_users` out of `BaseHandler` (#10744)Richard van der Hoff9-85/+125
This is part of my ongoing war against BaseHandler. I've moved kick_guest_users into RoomMemberHandler (since it calls out to that handler anyway), and split maybe_kick_guest_users into the two places it is called.
2021-09-06Pull out encrypted_by_default tests from user_directory tests (#10752)David Robertson4-95/+111
2021-09-06Speed up persisting redacted events (#10756)Erik Johnston2-11/+12
2021-09-04Allow room creator to send MSC2716 related events in existing room versions ↵Eric Eastwood4-10/+61
(#10566) * Allow room creator to send MSC2716 related events in existing room versions Discussed at https://github.com/matrix-org/matrix-doc/pull/2716/#discussion_r682474869 Restoring `get_create_event_for_room_txn` from, https://github.com/matrix-org/synapse/pull/10245/commits/44bb3f0cf5cb365ef9281554daceeecfb17cc94d * Add changelog * Stop people from trying to redact MSC2716 events in unsupported room versions * Populate rooms.creator column for easy lookup > From some [out of band discussion](https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$p2fKESoFst038x6pOOmsY0C49S2gLKMr0jhNMz_JJz0?via=jki.re&via=matrix.org), my plan is to use `rooms.creator`. But currently, we don't fill in `creator` for remote rooms when a user is invited to a room for example. So we need to add some code to fill in `creator` wherever we add to the `rooms` table. And also add a background update to fill in the rows missing `creator` (we can use the same logic that `get_create_event_for_room_txn` is doing by looking in the state events to get the `creator`). > > https://github.com/matrix-org/synapse/pull/10566#issuecomment-901616642 * Remove and switch away from get_create_event_for_room_txn * Fix no create event being found because no state events persisted yet * Fix and add tests for rooms creator bg update * Populate rooms.creator field for easy lookup Part of https://github.com/matrix-org/synapse/pull/10566 - Fill in creator whenever we insert into the rooms table - Add background update to backfill any missing creator values * Add changelog * Fix usage * Remove extra delta already included in #10697 * Don't worry about setting creator for invite * Only iterate over rows missing the creator See https://github.com/matrix-org/synapse/pull/10697#discussion_r695940898 * Use constant to fetch room creator field See https://github.com/matrix-org/synapse/pull/10697#discussion_r696803029 * More protection from other random types See https://github.com/matrix-org/synapse/pull/10697#discussion_r696806853 * Move new background update to end of list See https://github.com/matrix-org/synapse/pull/10697#discussion_r696814181 * Fix query casing * Fix ambiguity iterating over cursor instead of list Fix `psycopg2.ProgrammingError: no results to fetch` error when tests run with Postgres. ``` SYNAPSE_POSTGRES=1 SYNAPSE_TEST_LOG_LEVEL=INFO python -m twisted.trial tests.storage.databases.main.test_room ``` --- We use `txn.fetchall` because it will return the results as a list or an empty list when there are no results. Docs: > `cursor` objects are iterable, so, instead of calling explicitly fetchone() in a loop, the object itself can be used: > > https://www.psycopg.org/docs/cursor.html#cursor-iterable And I'm guessing iterating over a raw cursor does something weird when there are no results. --- Test CI failure: https://github.com/matrix-org/synapse/pull/10697/checks?check_run_id=3468916530 ``` tests.test_visibility.FilterEventsForServerTestCase.test_large_room =============================================================================== [FAIL] Traceback (most recent call last): File "/home/runner/work/synapse/synapse/tests/storage/databases/main/test_room.py", line 85, in test_background_populate_rooms_creator_column self.get_success( File "/home/runner/work/synapse/synapse/tests/unittest.py", line 500, in get_success return self.successResultOf(d) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/trial/_synctest.py", line 700, in successResultOf self.fail( twisted.trial.unittest.FailTest: Success result expected on <Deferred at 0x7f4022f3eb50 current result: None>, found failure result instead: Traceback (most recent call last): File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 701, in errback self._startRunCallbacks(fail) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 764, in _startRunCallbacks self._runCallbacks() File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 858, in _runCallbacks current.result = callback( # type: ignore[misc] File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 1751, in gotResult current_context.run(_inlineCallbacks, r, gen, status) --- <exception caught here> --- File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 1657, in _inlineCallbacks result = current_context.run( File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/failure.py", line 500, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/home/runner/work/synapse/synapse/synapse/storage/background_updates.py", line 224, in do_next_background_update await self._do_background_update(desired_duration_ms) File "/home/runner/work/synapse/synapse/synapse/storage/background_updates.py", line 261, in _do_background_update items_updated = await update_handler(progress, batch_size) File "/home/runner/work/synapse/synapse/synapse/storage/databases/main/room.py", line 1399, in _background_populate_rooms_creator_column end = await self.db_pool.runInteraction( File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 686, in runInteraction result = await self.runWithConnection( File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 791, in runWithConnection return await make_deferred_yieldable( File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 858, in _runCallbacks current.result = callback( # type: ignore[misc] File "/home/runner/work/synapse/synapse/tests/server.py", line 425, in <lambda> d.addCallback(lambda x: function(*args, **kwargs)) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 293, in _runWithConnection compat.reraise(excValue, excTraceback) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/deprecate.py", line 298, in deprecatedFunction return function(*args, **kwargs) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/compat.py", line 404, in reraise raise exception.with_traceback(traceback) File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 284, in _runWithConnection result = func(conn, *args, **kw) File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 786, in inner_func return func(db_conn, *args, **kwargs) File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 554, in new_transaction r = func(cursor, *args, **kwargs) File "/home/runner/work/synapse/synapse/synapse/storage/databases/main/room.py", line 1375, in _background_populate_rooms_creator_column_txn for room_id, event_json in txn: psycopg2.ProgrammingError: no results to fetch ``` * Move code not under the MSC2716 room version underneath an experimental config option See https://github.com/matrix-org/synapse/pull/10566#issuecomment-906437909 * Add ordering to rooms creator background update See https://github.com/matrix-org/synapse/pull/10697#discussion_r696815277 * Add comment to better document constant See https://github.com/matrix-org/synapse/pull/10697#discussion_r699674458 * Use constant field
2021-09-03Speed up MultiWriterIdGenerator when lots of IDs are in flight. (#10755)Erik Johnston4-2/+124
2021-09-03Support MSC3375: room version 9. (#10747)Patrick Cloke4-1/+79
2021-09-03Add a partial index to `presence_stream` to speed up startups (#10748)Sean4-1/+43
Signed-off-by: Sean Quah <seanq@element.io>
2021-09-03Use `execute_values` more in PostgreSQL (#10754)Erik Johnston2-19/+43
`execute_values` is a faster version of `execute_batch`.
2021-09-03Raise an error if an unknown preset is used to create a room. (#10738)Patrick Cloke2-1/+7
Raises a 400 error instead of a 500 if an unknown preset is passed from a client to create a room.
2021-09-03Additional type hints for client REST servlets (part 5) (#10736)Patrick Cloke8-68/+146
Additionally this enforce type hints on all function signatures inside of the synapse.rest.client package.
2021-09-03Incorporate reviewBrendan Abolivier3-13/+9
2021-09-03Fix bug with reusing 'txn' when persisting event. (#10743)Erik Johnston2-1/+8
This will only happen when a server has multiple out of band membership events in a single room.
2021-09-02Use correct federation path for knockingBrendan Abolivier1-12/+1
2021-09-02Fix documentation of directory name for remote thumbnails (#10556)cuttingedge11092-1/+2
2021-09-02LintBrendan Abolivier2-2/+1
2021-09-02Import mock from unittestBrendan Abolivier4-4/+4
2021-09-02Fix test_rewritten_id_serverBrendan Abolivier1-0/+5
2021-09-02LintBrendan Abolivier2-4/+6
2021-09-02Appease mypyBrendan Abolivier4-6/+12
2021-09-02Load legacy modules in testsBrendan Abolivier2-0/+5
2021-09-02LintBrendan Abolivier2-5/+7
2021-09-02Sample configBrendan Abolivier1-66/+0
2021-09-01Ignore rooms with unknown room versions in the spaces summary. (#10727)Patrick Cloke3-2/+40
This avoids breaking the entire endpoint if a room with an unsupported room version is encountered.
2021-09-01Consider the `origin_server_ts` of the `m.space.child` event when ordering ↵Patrick Cloke3-12/+22
rooms. (#10730) This updates the ordering of the returned events from the spaces summary API to that defined in MSC2946 (which updates MSC1772). Previously a step was skipped causing ordering to be inconsistent with clients.
2021-09-01LintBrendan Abolivier3-5/+13
2021-09-01Don't ratelimit when admin updates profileBrendan Abolivier2-3/+8
2021-09-01Fix one more testBrendan Abolivier1-4/+0
2021-09-01Fix more testsBrendan Abolivier5-11/+21
2021-09-01Fix testsBrendan Abolivier4-11/+7
2021-09-01Additional type hints for client REST servlets (part 4) (#10728)Patrick Cloke9-100/+145
2021-09-01Populate `rooms.creator` field for easy lookup (#10697)Eric Eastwood6-4/+213
Part of https://github.com/matrix-org/synapse/pull/10566 - Fill in creator whenever we insert into the rooms table - Add background update to backfill any missing creator values
2021-09-01Fix cyclic dependency with the legacy account validity workBrendan Abolivier1-3/+7
2021-09-01Remove account validity related schema updateBrendan Abolivier1-18/+0
Because DINUM had it before mainline, so the file names don't match. Will need some fiddling with the database before updating to a mainlined version.
2021-09-01Remove knock-related schema updateBrendan Abolivier1-20/+0
Because DINUM had it before mainline, so the file names don't match. Will need some fiddling with the database before updating to a mainlined version.
2021-09-01LintBrendan Abolivier4-23/+18
2021-09-01Make minor changes to changelog v1.42.0rc1Olivier Wilkinson (reivilibre)1-2/+2
2021-09-01Correctly include room avatars in email notifications (#10658)David Robertson3-6/+71
Judging by the template, this was intended ages ago, but we never actually passed an avatar URL to the template. So let's provide one. Closes #1546. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-09-01Remove buildkite pipelineBrendan Abolivier11-717/+0
2021-09-01Add buildkite pipeline backBrendan Abolivier10-0/+682
2021-09-01Improve changelogOlivier Wilkinson (reivilibre)1-3/+3
Expand OIDC to OpenID Connect.
2021-09-01Improve changelogOlivier Wilkinson (reivilibre)1-7/+6
2021-09-01Skip the final GC on shutdown to improve restart times (#10712)Sean2-0/+8
Use `gc.freeze()` on exit to exclude all existing objects from the final GC. In testing, this sped up shutdown by up to a few seconds. `gc.freeze()` runs in constant time, so there is little chance of performance regression. Signed-off-by: Sean Quah <seanq@element.io>
2021-09-011.42.0rc1Olivier Wilkinson (reivilibre)51-52/+78
2021-09-01Fix iteration in _remove_deleted_email_pushers background job. (#10734)Andrew Morgan3-1/+47
2021-08-31Allow configuration of the oEmbed URLs. (#10714)Patrick Cloke8-252/+463
This adds configuration options (under an `oembed` section) to configure which URLs are matched to use oEmbed for URL previews.
2021-08-31Additional type hints for the client REST servlets (part 3). (#10707)Patrick Cloke7-150/+306
2021-08-31Move the sessions delta to the latest schema version. (#10725)Patrick Cloke2-0/+1
This was erroneously put under schema version 62 instead of 63.
2021-08-31 1.41.1 v1.41.1Richard van der Hoff3-1/+39
2021-08-31Fix up unit tests (#10723)Richard van der Hoff2-2/+16