| Commit message (Collapse) | Author | Files | Lines |
|
See https://github.com/matrix-org/synapse/pull/11114#discussion_r799354680
|
|
|
|
parameter (#11911)
|
|
The Content-Length header should be treated as an int, not
a string. This shouldn't have any user-facing change.
|
|
|
|
|
|
|
|
|
|
|
|
See:
- https://github.com/matrix-org/synapse/pull/11114#discussion_r799350801
- https://github.com/matrix-org/synapse/pull/11114#discussion_r799354680
|
|
|
|
Since #11811 there has been general Complement flakiness around networking.
It seems like tests are hitting the wrong containers. In an effort to diagnose
the cause of this, as well as reduce its impact on this project, set the
parallelsim to 1 (no parallelism) when running tests.
If this fixes the flakiness then this indicates the cause and I can diagnose
this further. If this doesn't fix the flakiness then that implies some kind
of test pollution which also helps to diagnose this further.
|
|
|
|
This document isn't really relevant any more, and its existence is more confusing than helpful.
|
|
Also remove support for non-HTTP(S) web_client_location.
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
|
|
`join` test helper. (#11616)
|
|
The idea here is to set the parent span for incoming federation requests to the
*outgoing* span on the other end. That means that you can see (most of) the
full end-to-end flow when you have a process that includes federation requests.
However, in order not to lose information, we still want a link to the
`incoming-federation-request` span from the servlet, so we have to create
another span to do exactly that.
|
|
`start_active_span` was inconsistent as to whether it would activate the span
immediately, or wait for `scope.__enter__` to happen (it depended on whether
the current logcontext already had an associated scope). The inconsistency was
rather confusing if you were hoping to set up a couple of separate spans before
activating either.
Looking at the other implementations of opentracing `ScopeManager`s, the
intention is that it *should* be activated immediately, as the name
implies. Indeed, the idea is that you don't have to use the scope as a
contextmanager at all - you can just call `.close` on the result. Hence, our
cleanup has to happen in `.close` rather than `.__exit__`.
So, the main change here is to ensure that `start_active_span` does activate
the span, and that `scope.close()` does close the scope.
We also add some tests, which requires a `tracer` param so that we don't have
to rely on the global variable in unit tests.
|
|
(#11878)
The get_users_in_room and get_users_in_room_with_profiles
are now only invalidated when the membership of a room changes,
instead of during any state change in the room.
|
|
library. (#11832)
Co-authored-by: David Robertson <davidr@element.io>
|
|
* Fix losing incoming EDUs if debug logging enabled
Fixes #11889. Homeservers should only be affected if the
`synapse.8631_debug` logger was enabled for DEBUG mode.
I am not sure if this merits a bugfix release: I think the logging can
be disabled in config if anyone is affected? But it is still pretty bad.
|
|
We disabled coverage calculation for most of CI in #11017, but the olddeps
build uses a separate script and got forgotten.
|
|
Should have been caught in #10826.
|
|
Manually reverts the merge from cdbb8e6d6e36e0b6bc36e676d8fe66c96986b399.
|
|
users. (#11655)
|
|
helper. (#11615)
|
|
|
|
Fixes #10701
|
|
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
|
It's already under deps & removals
|
|
|
|
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
MSC1849 was replaced by MSC2675, which was merged.
The configuration flag, which defaulted to true, is no
longer useful.
|
|
|
|
Only allow files which file size and content types match configured
limits to be set as avatar.
Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
|
|
This was missed in https://github.com/matrix-org/synapse/pull/11799
|
|
No point recreating these for each call, since they are frozen
|
|
|
|
|
|
* Include 'prev_content' field in AS events
Signed-off-by: Vaishnav Nair <nairvaishnav007@icloud.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
|
This is in the context of mainlining the Tchap fork of Synapse. Currently in Tchap usernames are derived from the user's email address (extracted from the UIA results, more specifically the m.login.email.identity step).
This change also exports the check_username method from the registration handler as part of the module API, so that a module can check if the username it's trying to generate is correct and doesn't conflict with an existing one, and fallback gracefully if not.
Co-authored-by: David Robertson <davidr@element.io>
|
|
This is some odds and ends found during the review of #11791
and while continuing to work in this code:
* Return attrs classes instead of dictionaries from some methods
to improve type safety.
* Call `get_bundled_aggregations` fewer times.
* Adds a missing assertion in the tests.
* Do not return empty bundled aggregations for an event (preferring
to not include the bundle at all, as the docstring states).
|
|
* Don't print HTTPStatus.* in "Processed..." logs
Fixes #11812. See also #7118 and
https://github.com/matrix-org/synapse/pull/7188#r401719326 in
particular.
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
|
|
|
This is mostly motivated by the tchap use case, where usernames are automatically generated from the user's email address (in a way that allows figuring out the email address from the username). Therefore, it's an issue if we respond to requests on /register and /register/available with M_USER_IN_USE, because it can potentially leak email addresses (which include the user's real name and place of work).
This commit adds a flag to inhibit the M_USER_IN_USE errors that are raised both by /register/available, and when providing a username early into the registration process. This error will still be raised if the user completes the registration process but the username conflicts. This is particularly useful when using modules (https://github.com/matrix-org/synapse/pull/11790 adds a module callback to set the username of users at registration) or SSO, since they can ensure the username is unique.
More context is available in the PR that introduced this behaviour to synapse-dinsic: matrix-org/synapse-dinsic#48 - as well as the issue in the matrix-dinsic repo: matrix-org/matrix-dinsic#476
|
|
|
|
Similar to #11817.
In `_create_power_level_validator` we
- retrieve `validator`. This is a class implementing the
`jsonschema.protocols.Validator` interface. In other words,
`validator: Type[jsonschema.protocols.Validator]`.
- we then create an second validator class by modifying the original
`validator`. We return that class, which is also of type
`Type[jsonschema.protocols.Validator]`.
So the original annotation was incorrect: it claimed we were returning
an instance of jsonSchema.Draft7Validator, not the class (or a subclass)
itself. (Strictly speaking this is incorrect, because `POWER_LEVELS_SCHEMA`
isn't pinned to a particular version of JSON Schema. But there are other
complications with the type stubs if you try to fix this; I felt like
the change herein was a decent compromise that better expresses intent).
(I suspect/hope the typeshed project would welcome an effort to improve
the jsonschema stubs. Let's see if I get some spare time.)
|
|
(#11816)
* add check that gc.freeze is available before calling
* newsfragment
* lint
* Update comment
Co-authored-by: Dan Callahan <danc@element.io>
Co-authored-by: Dan Callahan <danc@element.io>
|
|
|
|
* CI: run Complement on the VM, not inside Docker
This requires https://github.com/matrix-org/complement/pull/289
We now run Complement on the VM instead of inside a Docker container.
This is to allow Complement to bind to any high-numbered port when it
starts up its own federation servers. We want to do this to allow for
more concurrency when running complement tests. Previously, Complement
only ever bound to `:8448` when running its own federation server. This
prevented multiple federation tests running at the same time as they would
fight each other on the port. This did however allow Complement to run
in Docker, as the host could just port forward `:8448` to allow homeserver
containers to communicate to Complement. Now that we are using random
ports however, we cannot use Docker to run Complement. This ends up
being a good thing because:
- Running Complement tests locally is closer to how they run in CI.
- Allows the `CI` env var to be removed in Complement.
- Slightly speeds up runs as we don't need to pull down the Complement
image prior to running tests. This assumes GHA caches actions sensibly.
* Changelog
* Full stop
* Update .github/workflows/tests.yml
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Review comments
* Update .github/workflows/tests.yml
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
|
|
|
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
|
* Make functions in python deltas optional
It's annoying to always have to write stubs for these.
* Documentation for delta files
* changelog
|
|
* Docs: add missing PR submission process how-tos
The documentation says that in order to submit a pull request you have to run the linter and links to [Run the linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters). IMO "Run the linters" should explain that development dependencies are a pre-requisite.
I also included `pip install wheel` which I had to run inside my virtual environment on ubuntu before I `pip install -e ".[all,dev]"` would succeed.
|
|
It had already accounted for 1.50.2 (ordered chronologically rather than
sem-ver-ically); it just seems this wasn't merged into master when we
released 1.50.2.
|
|
|
|
* Fix get federation status of destination if no error occured
|
|
|
|
|
|
|
|
|
|
PyNaCl's recent 1.5.0 release on PyPi includes arm64 wheels, which means our
arm64 docker images now build in a sensible amount of time, so we can skip the
amd64-only build.
|
|
This prevents the URL preview code from reading
a stream forever.
|
|
Images which are data URLs will no longer break URL
previews and will properly be "downloaded" and
thumbnailed.
|
|
|
|
upon user deactivation. (#11621)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
|
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Richard van der Hoff <richard@matrix.org>
|
|
|
|
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Richard van der Hoff <richard@matrix.org>
|
|
* remove reference in comments to python3.6
* upgrade tox python env in script
* bump python version in example for completeness
* upgrade python version requirement in setup doc
* upgrade necessary python version in __init__.py
* upgrade python version in setup.py
* newsfragment
* drops refs to bionic and replace with focal
* bump refs to postgres 9.6 to 10
* fix hanging ci
* try installing tzdata first
* revert change made in b979f336
* ignore new random mypy error while debugging other error
* fix lint error for temporary workaround
* revert change to install list
* try passing env var
* export debian frontend var?
* move line and add comment
* bump pillow dependency
* bump lxml depenency
* install libjpeg-dev for pillow
* bump automat version to one compatible with py3.8
* add libwebp for pillow
* bump twisted trunk python version
* change suffix of newsfragment
* remove redundant python 3.7 checks
* lint
|
|
The PR was cherrypicked into v1.51.0rc1.
|
|
|
|
... and start populating them for new events
|
|
|
|
|
|
|
|
|
|
This is a follow-up to #10565.
|
|
Preparation for dropping this table altogether. Part of #6574.
|
|
This is more efficient, since we only have to look up `state_key` in the event
dict once, rather than three (!) times.
|
|
whose key is prefixed with the user ID (#11788)
|
|
|
|
|
|
To not change the behaviour during the deprecation period.
Follow-up to #11774.
|
|
simultaneously (#11765)
|
|
(#11770)
|
|
#11595 dropped support for python 3.6, but forgot to update this doc.
|
|
`web_client_location`. (#11774)
This changes the behaviour of the root endpoint to redirect
directly to the configuration of `web_client_location` if it is
given an HTTP(S) URL.
|
|
Debug for #8631.
I'm having a hard time tracking down what's going wrong in that issue.
In the reported example, I could see server A sending federation traffic
to server B and all was well. Yet B reports out-of-sync device updates
from A.
I couldn't see what was _in_ the events being sent from A to B. So I
have added some crude logging to track
- when we have updates to send to a remote HS
- the edus we actually accumulate to send
- when a federation transaction includes a device list update edu
- when such an EDU is received
This is a bit of a sledgehammer.
|
|
Updates complement.sh to read the ref from an environment
variable (defaulting to master) when downloading a complement
bundle for testing.
|
|
`FederationClient.get_room_hierarchy()` caches its return values, so
refactor the code to avoid modifying the returned room summary.
|
|
flame graph
|
|
|
|
|
|
This gets us down to the ~0.8s per 100 event batch times!
|
|
~1.3s to ~1.1s time savings by re-using some calculations
that will be the same for every event type in the batch
TODO: Find a way to make store_state_group_id_for_event_id batch persist
in the db which brings it down to about 0.8s
|
|
A couple of surprises for me here, so thought I'd document them
|
|
|
|
|
|
By scraping Open Graph information from the HTML even
when an autodiscovery endpoint is found. The results are
then combined to capture as much information as possible
from the page.
|
|
Per updates to MSC3440.
This is implement as a separate method since it needs to be cached
on a per-user basis, instead of a per-thread basis.
|
|
|
|
|
|
|
|
|
|
I've never found this terribly useful. I think it was added in the early days
of Synapse, without much thought as to what would actually be useful to log,
and has just been cargo-culted ever since.
Rather, it tends to clutter up debug logs with useless information.
|
|
|
|
|
|
filter appservice users. (#11675)
|
|
* Warn against using Let's Encrypt certs for encrypted TURN
This helps to avoid client-side issues:
* https://github.com/vector-im/element-android/issues/1533
* https://github.com/vector-im/element-ios/issues/2712
Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
|
|
Fixes #11741
|
|
... and a minor thinko fix in the sample config.
|
|
The existing implementation of the `python_twisted_reactor_tick_time` metric is pretty useless, because it *only*
measures the time taken to execute timed calls and callbacks from threads. That neglects everything that
happens off the back of I/O, which is obviously quite a lot for us.
To improve this, I've hooked into a different place in the reactor - in particular, where it calls `epoll`. That call is
the only place it should wait for something to happen - the rest of the loop *should* be quick.
I've also removed `python_twisted_reactor_pending_calls`, because I don't believe anyone ever looks at it, and
it's a nuisance to populate.
|
|
Always add state.room_id after the configurable ORDER BY. Otherwise,
for any sort, certain pages can contain results from
other pages. (Especially when sorting by creator, since there may
be many rooms by the same creator)
* Document different order direction of numerical fields
"joined_members", "joined_local_members", "version" and "state_events"
are ordered in descending direction by default (dir=f). Added a note
in tests to explain the differences in ordering.
Signed-off-by: Daniël Sonck <daniel@sonck.nl>
|
|
dictionary (#11576)
|
|
* Fix sample_config.yaml in regards track_puppeted_user_ips
Closes #11741
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
|
|
|
|
|
This should be (slightly) more efficient and it is simpler
to have a single method for inserting multiple values.
|
|
documentation claims that you can use the %(app)s variable in password_reset and email_validation subjects, but if you do you end up with an error 500
Co-authored-by: br4nnigan <10244835+br4nnigan@users.noreply.github.com>
|
|
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
|
fail because too many EDUs were produced for device updates. (#11730)
Co-authored-by: David Robertson <davidr@element.io>
|
|
* change spec url in config files
* Create 11739.txt
* .txt -> .doc
|
|
|
|
|
|
Rather than hooking into the reactor loop, just add a timed task that runs every 100 ms to do the garbage collection.
Part 1 of a quest to simplify the reactor monkey-patching.
|
|
|
|
|
|
Currently when puppeting another user, the user doing the puppeting is
tracked for client IPs and MAU (if configured).
When tracking MAU is important, it becomes necessary to be possible to
also track the client IPs and MAU of puppeted users. As an example a
client that manages user creation and creation of tokens via the Synapse
admin API, passing those tokens for the client to use.
This PR adds optional configuration to enable tracking of puppeted users
into monthly active users. The default behaviour stays the same.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
|
|
be sent to remote homeservers if there were too many to send at once. (#11729)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
|
|
* Deal with mypy errors w/ type-hinted pynacl 1.5.0
Fixes #11644.
I really don't like that we're monkey patching pynacl SignedKey
instances with alg and version objects. But I'm too scared to make the
changes necessary right now.
(Ideally I would replace `signedjson.types.SingingKey` with a runtime class which
wraps or inherits from `nacl.signing.SigningKey`.) C.f. https://github.com/matrix-org/python-signedjson/issues/16
|
|
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
|
|
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
|
|
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
|
|
|
|
|
|
|
|
* Deal with mypy errors w/ type-hinted pynacl 1.5.0
Fixes #11644.
I really don't like that we're monkey patching pynacl SignedKey
instances with alg and version objects. But I'm too scared to make the
changes necessary right now.
(Ideally I would replace `signedjson.types.SingingKey` with a runtime class which
wraps or inherits from `nacl.signing.SigningKey`.) C.f. https://github.com/matrix-org/python-signedjson/issues/16
|
|
* Prettier complement logs
* Changelog
|
|
By returning all of the m.space.child state of the space, not just
the first 50. The number of rooms returned is still capped at 50.
For the federation API this implies that the requesting server will
need to individually query for any other rooms it is not joined to.
|
|
* Optionally use an on-disk sqlite db in tests
When debugging a test it is sometimes useful to inspect the state of the
DB. This is not easy when the db is in-memory: one cannot attach the
sqlite CLI to another process's DB.
With this change, if SYNAPSE_TEST_PERSIST_SQLITE_DB is set, we use
`_trial_temp/test.db` as our sqlite database. One can then use
`sqlite3 _trial_temp/test.db` and query to your heart's content.
The DB is destroyed and recreated between different test cases.
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
|
(#11701)
|
|
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.
Overall this should cause no change in behavior, but is prep work
for other improvements.
|
|
|
|
This was removed from MSC2674 before that was approved
and is not used by any known clients.
|
|
On Python 2, indexing a byte-string gives back a byte-string,
while on Python 3 it gives back the ASCII equivalent as an int.
|
|
|
|
federation (#11530)
* add some tests to verify we are stripping unauthorized fields out of unsigned
* add function to strip unauthorized fields from the unsigned object of event
* newsfragment
* update newsfragment number
* add check to on_send_membership_event
* refactor tests
* fix lint error
* slightly refactor tests and add some comments
* slight refactor
* refactor tests
* fix import error
* slight refactor
* remove unsigned filtration code from synapse/handlers/federation_event.py
* lint
* move unsigned filtering code to event base
* refactor tests
* update newsfragment
* requested changes
* remove unused retun values
|
|
`glob_to_regex` in `synapse.util`, where it moved from. (#11696)
|
|
error to be returned, not `M_UNKNOWN` (#11672)
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
as per
https://github.com/matrix-org/synapse/pull/11532#discussion_r769123269
|
|
Fixes minor discrepancies between the /hierarchy endpoint described
in MSC2946 and the implementation.
Note that the changes impact the stable and unstable /hierarchy and
unstable /spaces endpoints for both client and federation APIs.
|
|
|
|
|
|
* newsfragment
* fix newsfragment number
* update changelog
* remove extra space
|
|
|
|
|
|
|
|
|
|
|
|
If we've never made a request to a remote homeserver, we should cache the response---even if the response is "this user has no devices".
|
|
|
|
* `_auth_and_persist_outliers`: mark persisted events as outliers
Mark any events that get persisted via `_auth_and_persist_outliers` as, well,
outliers.
Currently this will be a no-op as everything will already be flagged as an
outlier, but I'm going to change that.
* `process_remote_join`: stop flagging as outlier
The events are now flagged as outliers later on, by `_auth_and_persist_outliers`.
* `send_join`: remove `outlier=True`
The events created here are returned in the result of `send_join` to
`FederationHandler.do_invite_join`. From there they are passed into
`FederationEventHandler.process_remote_join`, which passes them to
`_auth_and_persist_outliers`... which sets the `outlier` flag.
* `get_event_auth`: remove `outlier=True`
stop flagging the events returned by `get_event_auth` as outliers. This method
is only called by `_get_remote_auth_chain_for_event`, which passes the results
into `_auth_and_persist_outliers`, which will flag them as outliers.
* `_get_remote_auth_chain_for_event`: remove `outlier=True`
we pass all the events into `_auth_and_persist_outliers`, which will now flag
the events as outliers.
* `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter
This param is now never set to True, so we can remove it.
* `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param
This is no longer set anywhere, so we can remove it.
* `get_pdu`: remove unused `outlier` parameter
... and chase it down into `get_pdu_from_destination_raw`.
* `event_from_pdu_json`: remove redundant `outlier` param
This is never set to `True`, so can be removed.
* changelog
* update docstring
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
#11505 (#11687)
Co-authored-by: Sean Quah <seanq@element.io>
|
|
before the names of methods, as appropriate. (#11681)
|
|
|
|
Co-authored-by: reivilibre <olivier@librepush.net>
|
|
this should not be a case-insensitive match.
|
|
* Fix AssertionErrors after purging events
If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.
This fixes the situation by rewinding the stream processors.
* `check-newsfragment`: ignore deleted newsfiles
|
|
Events returned by `backfill` should not be flagged as outliers.
Fixes:
```
AssertionError: null
File "synapse/handlers/federation.py", line 313, in try_backfill
dom, room_id, limit=100, extremities=extremities
File "synapse/handlers/federation_event.py", line 517, in backfill
await self._process_pulled_events(dest, events, backfilled=True)
File "synapse/handlers/federation_event.py", line 642, in _process_pulled_events
await self._process_pulled_event(origin, ev, backfilled=backfilled)
File "synapse/handlers/federation_event.py", line 669, in _process_pulled_event
assert not event.internal_metadata.is_outlier()
```
See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/231992
Fixes #8894.
|
|
* Push `get_room_{min,max_stream_ordering}` into StreamStore
Both implementations of this are identical, so we may as well push it down and
get rid of the abstract base class nonsense.
* Remove redundant `StreamStore` class
This is empty now
* Remove redundant `get_current_events_token`
This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid
of it.
* newsfile
|
|
"Unknown room" can mean a multitude of things here. To help with debugging, add
some more words to the exception text.
|
|
Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.
|
|
* update Trove classifiers to remove py36
* stop building bionic
* update dh-virtualenv
* newsfragment
* fix newsfragment
* update version refs
* another try at correct tag
* Update changelog
|
|
To improve type hints throughout the code.
|
|
|
|
exclusion list. (#11657)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
|
|
By using cast and making ignores more specific.
|
|
Both of those APIs return state events, which will not have bundled
aggregations added anyway.
|
|
|
|
* update black version
* run updated version of black on code
* newsfragment
* enumerate python versions
|
|
|
|
Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
|
|
* remove python 3.6 and postgres 9.6 from github workflow
* remove python 3.6 env from tox
* newsfragment
* correct postgres version
* add py310 to tox env list
|
|
postgres 10 _+_
|
|
|
|
... to work around breakage on buster
(https://github.com/Marco-Sulla/python-frozendict/issues/41)
|
|
`tests.server.setup_test_homeserver`. (#11503)
|
|
|
|
|
|
|
|
|
|
* Wrap `auth.get_user_by_req` in an opentracing span
give `get_user_by_req` its own opentracing span, since it can result in a
non-trivial number of sub-spans which it is useful to group together.
This requires a bit of reorganisation because it also sets some tags (and may
force tracing) on the servlet span.
* Emit opentracing span for encoding json responses
This can be a significant time sink.
* Rename all sync spans with a prefix
* Write an opentracing span for encoding sync response
* opentracing span to group generate_room_entries
* opentracing spans within sync.encode_response
* changelog
* Use the `trace` decorator instead of context managers
|
|
|
|
And make bundling aggregations opt-in, instead of opt-out to avoid
having APIs to include extraneous data (and being much heavier than
necessary).
|
|
This adds some opentracing annotations to ResponseCache, to make it easier to see what's going on; in particular, it adds a link back to the initial trace which is actually doing the work of generating the response.
|
|
* remove `start_active_span_from_request`
Instead, pull out a separate function, `span_context_from_request`, to extract
the parent span, which we can then pass into `start_active_span` as
normal. This seems to be clearer all round.
* Remove redundant tags from `incoming-federation-request`
These are all wrapped up inside a parent span generated in AsyncResource, so
there's no point duplicating all the tags that are set there.
* Leave request spans open until the request completes
It may take some time for the response to be encoded into JSON, and that JSON
to be streamed back to the client, and really we want that inside the top-level
span, so let's hand responsibility for closure to the SynapseRequest.
* opentracing logs for HTTP request events
* changelog
|
|
* Disable aggregation bundling on `/sync` responses
A partial revert of #11478. This turns out to have had a significant CPU impact
on initial-sync handling. For now, let's disable it, until we find a more
efficient way of achieving this.
* Fix tests.
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
|
|
Instead of returning 404 errors with HTML bodies when an unknown
prefix was requested (e.g. /matrix/client/v1 before Synapse v1.49.0).
|
|
Instead of the backported version.
|
|
|
|
See https://github.com/matrix-org/synapse/pull/11114#discussion_r741687508
|
|
|
|
|
|
|
|
|
|
As suggested in https://github.com/matrix-org/matrix-react-sdk/pull/7372#discussion_r769523369
|
|
See https://github.com/matrix-org/synapse/pull/11114#discussion_r760843910
|
|
|
|
|
|
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
|
|
|
|
masquerade as specific devices. (#11538)
|
|
|
|
|
|
|
|
This reverts commit 158d73ebdd61eef33831ae5f6990acf07244fc55.
|
|
|
|
|
|
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
|
|
|
|
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
|
|
* Splits the logic for parsing HTML from the resource handling code.
* Fix a circular import in the oEmbed code (which uses the HTML parsing code).
* Renames some of the HTML parsing methods to:
* Make it clear which methods are "internal" to the module.
* Clarify what the methods do.
|
|
|
|
|
|
|
|
raising an exception. (#11565)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
|
|
The event still needs to have `auth_events` defined to be valid.
Split out from https://github.com/matrix-org/synapse/pull/11114
|
|
batch (MSC2716) (#11487)
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
We did some work on making sure the `state_groups` were shared in
https://github.com/matrix-org/synapse/pull/10975
|
|
C.f. https://github.com/matrix-org/synapse/pull/11494#pullrequestreview-827780886
|
|
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
|
|
this field is never read, so we may as well stop populating it.
|
|
This should fix pushers with an email in non-canonical form is used as
the pushkey.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
And expand some type hints in the receipts storage module.
|
|
|
|
Co-authored-by: David Robertson <davidr@element.io>
|
|
`tests.rest.client.test_auth`. (#11520)
|
|
Somehow I'd managed to get my database in a pickle with stream ids. These
changes were useful to debug.
|