| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| |
| | |
* commit '4e874ed59':
Remove unnecessary maybeDeferred calls (#8044)
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit '4dd27e6d1':
Reduce unnecessary whitespace in JSON. (#7372)
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit 'fe6cfc80e':
Convert some util functions to async (#8035)
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit '0a86850ba':
Stop the parent process flushing the logs on exit (#8012)
|
| | |
| |
| | |
This solves the problem that the first few lines are logged twice on matrix.org. Hopefully the comments explain it.
|
| |\|
| |
| |
| |
| | |
* commit '916cf2d43':
re-implement daemonize (#8011)
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This has long been something I've wanted to do. Basically the `Daemonize` code
is both too flexible and not flexible enough, in that it offers a bunch of
features that we don't use (changing UID, closing FDs in the child, logging to
syslog) and doesn't offer a bunch that we could do with (redirecting stdout/err
to a file instead of /dev/null; having the parent not exit until the child is
running).
As a first step, I've lifted the Daemonize code and removed the bits we don't
use. This should be a non-functional change. Fixing everything else will come
later.
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit 'de119063f': (31 commits)
Convert room list handler to async/await. (#7912)
Element CSS and logo in email templates (#7919)
Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
Remove unused code from synapse.logging.utils. (#7897)
Fix a typo in the sample config. (#7890)
Fix deprecation warning: import ABC from collections.abc (#7892)
Change sample config's postgres user to synapse_user (#7889)
Fix deprecation warning due to invalid escape sequences (#7895)
Remove Ubuntu Eoan that is now EOL (#7888)
Fix the trace function for async functions. (#7872)
Add help for creating a user via docker (#7885)
Switch to Debian:Slim from Alpine for the docker image (#7839)
Stop using 'device_max_stream_id' (#7882)
Fix TypeError in synapse.notifier (#7880)
Add a default limit (of 100) to get/sync operations. (#7858)
Change "unknown room ver" logging to warning. (#7881)
Convert device handler to async/await (#7871)
Convert synapse.app to async/await. (#7868)
Convert _base, profile, and _receipts handlers to async/await (#7860)
Add admin endpoint to get members in a room. (#7842)
...
|
| | | |
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit '43726783e': (22 commits)
1.17.0rc1
Fix some spelling mistakes / typos. (#7811)
`update_membership` declaration: now always returns an event id. (#7809)
Improve stacktraces from exceptions in background processes (#7808)
Fix `can only concatenate list (not "tuple") to list` exception (#7810)
Pass original request headers from workers to the main process. (#7797)
Generate real events when we reject invites (#7804)
Add `HomeServer.signing_key` property (#7805)
Revert "Update the installation docs on apt-transport-https (#7801)"
Do not use simplejson in Synapse. (#7800)
Stop passing bytes when dumping JSON (#7799)
Update the installation docs on apt-transport-https (#7801)
shuffle changelog slightly
Change Caddy links (old is deprecated) (#7789)
Stop populating unused table `local_invites`. (#7793)
Refactor getting replication updates from database v2. (#7740)
Add libwebp dependency to Dockerfile (#7791)
Add documentation for JWT login type and improve sample config. (#7776)
Convert the appservice handler to async/await. (#7775)
Don't ignore `set_tweak` actions with no explicit `value`. (#7766)
...
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit '5cdca53aa':
Merge different Resource implementation classes (#7732)
Fix inconsistent handling of upper and lower cases of email addresses. (#7021)
Allow YAML config file to contain None (#7779)
Fix a typo.
Move 1.15.2 after 1.16.0rc2.
1.16.0rc2
Remove an extraneous space.
Add links to the fixes.
Fix tense in the release notes.
Hack to add push priority to push notifications (#7765)
Add early returns to `_check_for_soft_fail` (#7769)
Use symbolic names for replication stream names (#7768)
Type checking for `FederationHandler` (#7770)
Fix new metric where we used ms instead of seconds (#7771)
Fix incorrect error message when database CTYPE was set incorrectly. (#7760)
Pin link in CHANGES.md
Fixes to CHANGES.md
|
| | |
| |
| | |
fixes #7016
|
| |\|
| |
| |
| |
| | |
* commit '231252516':
Fix "argument of type 'ObservableDeferred' is not iterable" error (#7708)
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit 'a3f11567d':
Replace all remaining six usage with native Python 3 equivalents (#7704)
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit '03619324f':
Create a ListenerConfig object (#7681)
Fix changelog wording
1.15.1
Wrap register_device coroutine in an ensureDeferred (#7684)
Ensure the body is a string before comparing push rules. (#7701)
Ensure etag is a string for GET room_keys/version response (#7691)
Update m.id.phone to use 'phone' instead of 'number' (#7687)
Fix "There was no active span when trying to log." error (#7698)
Enable 3PID add/bind/unbind endpoints on r0 routes
Discard RDATA from already seen positions. (#7648)
Replace iteritems/itervalues/iterkeys with native versions. (#7692)
Fix warnings about losing log context during UI auth. (#7688)
Fix a typo when comparing the URI & method during UI Auth. (#7689)
Remove "user_id" from GET /presence. (#7606)
Increase the default SAML session expirary time to 15 minutes. (#7664)
fix typo in sample_config.yaml (#7652)
Take out a lock before modifying _CACHES (#7663)
Add option to enable encryption by default for new rooms (#7639)
Clean-up the fallback login code. (#7657)
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
dinsic-release-v1.15.x
* 'release-v1.15.0' of github.com:matrix-org/synapse: (55 commits)
1.15.0
Fix some attributions
Update CHANGES.md
1.15.0rc1
Revert "1.15.0rc1"
1.15.0rc1
Fix bug in account data replication stream. (#7656)
Convert the registration handler to async/await. (#7649)
Accept device information at the login fallback endpoint. (#7629)
Convert user directory handler and related classes to async/await. (#7640)
Add an option to disable autojoin for guest accounts (#6637)
Clarifications to the admin api documentation (#7647)
Update to the stable SSO prefix for UI Auth. (#7630)
Fix type information on `assert_*_is_admin` methods (#7645)
Remove some unused constants. (#7644)
Typo fixes.
Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263)
Add device management to admin API (#7481)
Attempt to fix PhoneHomeStatsTestCase.test_performance_100 being flaky. (#7634)
Support CS API v0.6.0 (#6585)
...
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:
* Rather undocumented, and
* causing a *lot* of config checks
This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation.
Best to be reviewed commit-by-commit.
|
| | |
| |
| |
| |
| |
| | |
Instead of storing and sending an ACK for every single row we send
synchronously, we instead do it asynchronously while batching up
updates.
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
dinsic-release-v1.14.x
* 'release-v1.14.0' of github.com:matrix-org/synapse: (108 commits)
Fix typo in PR link
Update debian changelog
1.14.0
Improve changelog wording
1.14.0rc2
Fix sample config docs error (#7581)
Fix up comments
Fix specifying cache factors via env vars with * in name. (#7580)
Don't apply cache factor to event cache. (#7578)
Ensure ReplicationStreamer is always started when replication enabled. (#7579)
Remove the changes to the debian changelog
Not full release yet, this is rc1
Merge event persistence move changelog entries
More changelog fix
Changelog fixes
1.14.0
Replace device_27_unique_idx bg update with a fg one (#7562)
Fix incorrect exception handling in KeyUploadServlet.on_POST (#7563)
Fix recording of federation stream token (#7564)
Simplify reap_monthly_active_users (#7558)
...
|
| | |
| |
| |
| | |
This is already correctly done when we instansiate the cache, but wasn't
when it got reloaded (which always happens at least once on startup).
|
| | |
| |
| | |
`Failure()` is more cunning than `Failure(e)`.
|
| | | |
|
| | |
| |
| |
| | |
this is a no-op on python 3.
|
| | |
| |
| |
| | |
this is a no-op on python 3.
|
| | | |
|
| | |
| |
| |
| | |
variables (#6391)
|
| |\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
dinsic-release-v1.14.x
* 'release-v1.13.0' of github.com:matrix-org/synapse: (257 commits)
Update changelog based on feedback.
Move warnings in the changelog and re-iterate changes to branches.
1.13.0
update dh-virtualenv (#7526)
1.13.0rc3
Hash passwords earlier in the registration process (#7523)
1.13.0rc2
1.13.0rc2
Stop `get_joined_users` corruption from custom statuses (#7376)
Do not validate that the client dict is stable during UI Auth. (#7483)
Fix new flake8 errors (#7489)
Don't UPGRADE database rows
RST indenting
Put rollback instructions in upgrade notes
Fix changelog typo
Oh yeah, RST
Absolute URL it is then
Fix upgrade notes link
Provide summary of upgrade issues in changelog. Fix )
Move next version notes from changelog to upgrade notes
...
|
| | |
| |
| |
| |
| | |
Currently we copy `users_who_share_room` needlessly about three times,
which is expensive when the set is large (which it can easily be).
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
First some background: StreamChangeCache is used to keep track of what "entities" have
changed since a given stream ID. So for example, we might use it to keep track of when the last
to-device message for a given user was received [1], and hence whether we need to pull any to-device messages from the database on a sync [2].
Now, it turns out that StreamChangeCache didn't support more than one thing being changed at
a given stream_id (this was part of the problem with #7206). However, it's entirely valid to send
to-device messages to more than one user at a time.
As it turns out, this did in fact work, because *some* methods of StreamChangeCache coped
ok with having multiple things changing on the same stream ID, and it seems we never actually
use the methods which don't work on the stream change caches where we allow multiple
changes at the same stream ID. But that feels horribly fragile, hence: let's update
StreamChangeCache to properly support this, and add some typing and some more tests while
we're at it.
[1]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L301
[2]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L47-L51
|
| | |
| |
| |
| |
| |
| | |
Other parts of the code (such as the StreamChangeCache) assume that there will
not be multiple changes with the same stream id.
This code was introduced in #7024, and I hope this fixes #7206.
|
| | |
| |
| |
| | |
make sure we clear out all but one update for the user
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Pull Sentinel out of LoggingContext
... and drop a few unnecessary references to it
* Factor out LoggingContext.current_context
move `current_context` and `set_context` out to top-level functions.
Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.
* move copy-to-parent into `stop`
this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.
* Replace `LoggingContext.alive` with `finished`
Turn `alive` into `finished` and make it a bit better defined.
|
| |\|
| |
| |
| |
| |
| |
| | |
* commit '509e381af':
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
changelog
Increase MAX_EVENTS_BEHIND for replication clients
|
| | |
| |
| |
| | |
Ensure good comprehension hygiene using flake8-comprehensions.
|
| |\|
| |
| |
| |
| | |
* commit 'ed630ea17':
Reduce amount of logging at INFO level. (#6862)
|
| | |
| |
| |
| |
| |
| |
| |
| | |
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
|
| |\|
| |
| |
| |
| | |
* commit 'ae5b3104f':
Fix stacktraces when using ObservableDeferred and async/await (#6836)
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit '9f7aaf90b':
Validate client_secret parameter (#6767)
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| | |
* commit '59dc87c61':
changelog
Log saml assertions rather than the whole response
move batch_iter to a separate module
|
| | |
| |
| |
| |
| |
| | |
... since the whole response is huge.
We even need to break up the assertions, since kibana otherwise truncates them.
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| | |
* commit '608bf7d74':
changelog
Handle `config` not being set for synapse plugin modules
Cleanups and additions to the module API
|
| | |
| |
| |
| |
| | |
Some modules don't need any config, so having to define a `config` property
just to keep the loader happy is a bit annoying.
|
| |\|
| |
| |
| |
| | |
* commit 'bc7de8765':
Persist auth/state events at backwards extremities when we fetch them (#6526)
|
| | |
| |
| | |
The main point here is to make sure that the state returned by _get_state_in_room has been authed before we try to use it as state in the room.
|
| |\|
| |
| |
| |
| | |
* commit 'cb2db1799':
look up cross-signing keys from the DB in bulk (#6486)
|
| | | |
|
| |\|
| |
| |
| |
| |
| | |
* commit 'e3f528c54':
Newsfile
Remove SnapshotCache in favour of ResponseCache
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit '18660a34d':
Fix inaccurate per-block metrics (#6491)
|
| | |
| |
| |
| |
| | |
`Measure` incorrectly assumed that it was the only thing being done by the parent `LoggingContext`. For instance, during a "renew group attestations" operation, hundreds of `outbound_request` calls could take place in parallel, all using the same `LoggingContext`. This would mean that any resources used during *any* of those calls would be reported against *all* of them, producing wildly inaccurate results.
Instead, we now give each `Measure` block its own `LoggingContext` (using the parent `LoggingContext` mechanism to ensure that the log lines look correct and that the metrics are ultimately propogated to the top level for reporting against requests/backgrond tasks).
|
| |\|
| |
| |
| |
| |
| |
| | |
* commit 'a9b393340':
Fixup functions to consistently return deferreds
Newsfile
Port SyncHandler to async/await
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| | |
* commit '6e1b40dc2':
sample config
Add changelog
A couple more instances
Replace instance variations of homeserver with correct case/spacing
|
| | | |
|
| |\|
| |
| |
| |
| | |
* commit 'affcc2cc3':
Fix LruCache callback deduplication (#6213)
|
| | | |
|
| |\| |
|
| | |
| |
| | |
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
|
| |\| |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
This makes it easier to use in an async/await world.
Also fixes a bug where cache descriptors would occaisonally return a raw
value rather than a deferred.
|
| |\|
| |
| |
| |
| | |
* commit 'b39ca49db':
Handle FileNotFound error in checking git repository version (#6284)
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| |
| | |
* commit 'b4465564c':
Don't return coroutines
Make concurrently_execute work with async/await
Newsfile
Port federation_server to async/await
|
| | | |
|
| |\|
| |
| |
| |
| |
| |
| | |
* commit '561133c3c':
Update docstring
Newsfile
Quick fix to ensure cache descriptors always return deferreds
|
| | | |
|
| | | |
|
| |\| |
|
| | | |
|
| |\| |
|
| | |
| |
| | |
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |\| |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| | |
* type checking fixes
* changelog
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
We have set the max retry interval to a value larger than a postgres or
sqlite int can hold, which caused exceptions when updating the
destinations table.
To fix postgres we need to change the column to a bigint, and for sqlite
we lower the max interval to 2**62 (which is still incredibly long).
|
| |\| |
|
| | |\ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Fixes a bug where the default attribute maps were prioritised over
user-specified ones, resulting in incorrect mappings.
The problem is that if you call SPConfig.load() multiple times, it adds new
attribute mappers to a list. So by calling it with the default config first,
and then the user-specified config, we would always get the default mappers
before the user-specified mappers.
To solve this, let's merge the config dicts first, and then pass them to
SPConfig.
|
| |\ \ \
| | |/
| |/| |
|
| | | |
| | |
| | |
| | | |
Track the time that a server started failing at, for general analysis purposes.
|
| |\| | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
Essentially the intention here is to end up blacklisting servers which never
respond to federation requests.
Fixes https://github.com/matrix-org/synapse/issues/5113.
|
| |\| | |
|
| | | |
| | |
| | |
| | | |
This was intended to introduce an element of jitter; instead it gave you a
30/60 chance of resetting to zero.
|
| |\| | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted.
Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140).
Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.
|
| |\| | |
|
| | | |
| | |
| | |
| | | |
* remove some unused code
* make things which were constants into constants for efficiency and clarity
|
| | | |
| | |
| | |
| | |
| | | |
This reverts commit 71fc04069a5770a204c3514e0237d7374df257a8.
This broke 3PID invites as #5892 was required for it to work correctly.
|
| | | |
| | |
| | |
| | |
| | | |
This reverts commit 978f263e7c5d1eb440efaf07abc5009408ade25d, reversing
changes made to 4f6ee99818d9c338944a10585d0aea4c7349d456.
|
| |\| | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Fixes https://github.com/matrix-org/synapse/issues/5861
Adds support for the v2 lookup API as defined in [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134). Currently this is only used for 3PID invites.
Sytest PR: https://github.com/matrix-org/sytest/pull/679
|
| |\| | |
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| | |
This gives a bit of a grace period where we can attempt to refetch a
remote `well-known`, while still using the cached result if that fails.
Hopefully this will make the well-known resolution a bit more torelant
of failures, rather than it immediately treating failures as "no result"
and caching that for an hour.
|
| | | |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| | |
There was some inconsistent behaviour in the caching layer around how
exceptions were handled - particularly synchronously-thrown ones.
This seems to be most easily handled by pushing the creation of
ObservableDeferreds down from CacheDescriptor to the Cache.
|
| |\| |
|
| | |
| |
| |
| |
| |
| | |
* Add a prometheus metric for active cache lookups.
* changelog
|
| |\| |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
The version of a module isn't going to change over the lifetime of the
process (assuming no funky hot reloading is going on, which it isn't),
so let's just cache the result to avoid spawning lots of git
subprocesses.
Fixes #5672.
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| | |
- Put the default window_size back to 1000ms (broken by #5181)
- Make the `rc_federation` config actually do something
- fix an off-by-one error in the 'concurrent' limit
- Avoid creating an unused `_PerHostRatelimiter` object for every single
incoming request
|
| |\|
| |
| |
| | |
(#5617)
|
| | |
| |
| |
| |
| |
| |
| |
| | |
(#5617)
* Improve the backwards compatibility re-exports of synapse.logging.context.
* reexport logformatter too
|
| |\| |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
* Fix 'utime went backwards' errors on daemonization.
Fixes #5608
* remove spurious debug
|
| |\| |
|
| | |
| |
| |
| | |
Fixes #5602, #5603
|
| |\| |
|
| | | |
|
| |\| |
|
| | |
| |
| |
| |
| |
| |
| | |
Closes #4583
Does slightly less than #5045, which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.
Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
|
| |\| |
|
| | |
| |
| |
| |
| | |
Sentry will catch the errors if they happen, so that should be good enough, and
woun't make things explode if we hit the error condition.
|
| | |\ |
|
| | | |
| | |
| | |
| | | |
Check that our clocks go forward.
|
| |\ \ \
| | |/
| |/| |
|
| | |/ |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |\| |
|
| | |
| |
| | |
Fixes a regression introduced in #5335.
|
| | | |
|
| | | |
|
| |\| |
|
| | | |
|
| | |\
| | |
| | | |
Allow client event serialization to be async
|
| | | |
| | |
| | | |
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
|
| | | | |
|
| | |/ |
|
| |\| |
|
| | |\ |
|
| | | | |
|
| |\| | |
|
| | | | |
|
| | |/
| |
| | |
Avoid sending syntax errors from the manhole to sentry.
|
| | |
| |
| |
| | |
* Prevent exception when hs or shadow_hs missing
|
| | | |
|
| |\|
| |
| |
| | |
erikj/dinsic-merged-master
|
| | | |
|
| | |\
| | |
| | | |
Implement workaround for login error.
|
| | | |
| | |
| | |
| | | |
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
|
| | |/ |
|
| | | |
|
| | |
| |
| |
| | |
This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
|
| |\ \
| | |
| | |
| | | |
dinsic_anoa/info_split
|
| | | |
| | |
| | | |
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
|
| |/ / |
|
| | | |
|
| |\| |
|
| | |\
| | |
| | |
| | | |
erikj/alias_disallow_list
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.
While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.
Fixes #4064
|
| | | |
| | |
| | |
| | | |
on py3) (#4068)
|
| | | | |
|
| | | | |
|
| | | | |
|
| | |/ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
If a looping call function errors, then it kills the loop entirely.
Currently it throws away the exception logs, so we should make it
actually log them.
Fixes #3929
|
| | | |
|
| | | |
|
| | |\ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
transactions (#3959)
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.
This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
|
| | |\| |
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
(This appears to have been a longstanding bug, but one which has been made more
of a problem by #3932 and #3933).
(This was originally done by Erik as part of #3933. I'm cherry-picking it
because really it's a fix in its own right)
|
| | | |
| | |
| | |
| | |
| | |
| | | |
It used to try and produce an estimate, which was sometimes negative.
This caused metrics to be sad, so lets always just calculate it from
scratch.
|
| | | |
| | |
| | |
| | | |
Hopefully helps with #3931
|
| | |/ |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:
- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
them into other, more cryptic, exceptions.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
The existing deferred timeout helper function (and the one into twisted)
suffer from a bug when a deferred's canceller throws an exception, #3842.
The new helper function doesn't suffer from this problem.
|
| | |
| |
| |
| |
| | |
Turns out deferred.cancel sometimes throws, so we do that last to ensure
that we always do resolve the new deferred.
|
| | |
| |
| |
| | |
This is an attempt to mitigate #3842 by adding yet-another-timeout
|
| | | |
|
| | |
| |
| |
| |
| | |
Newer versions of openssh client refuse to connect to the old key due to
its length.
|
| |\| |
|
| | |
| |
| |
| |
| | |
This fixes bugs introduced in #3700, by making sure that we behave sanely
when an incoming connection is closed before the headers are read.
|
| | |
| |
| |
| |
| | |
Make the logcontext filter not explode if it somehow ends up with a logcontext
of None, since that infinite-loops the whole logging system.
|
| | | |
|
| | |\ |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
Turns out that cancellation of inlineDeferreds didn't really work properly
until Twisted 18.7. This commit refactors Linearizer.queue to avoid
inlineCallbacks.
|
| | |/ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Because it was complicated and annoyed me. I suspect this will be more
efficient too.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
|
| | | |
|
| | |
| |
| |
| |
| | |
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
|
| | |
| |
| |
| |
| | |
* give them names, to improve logging
* use a deque rather than a list for efficiency
|
| | |
| |
| |
| | |
Fixes #3570
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is more involved than it might otherwise be, because the current
implementation just drops its logcontexts and runs everything in the sentinel
context.
It turns out that we aren't actually using a bunch of the functionality here
(notably suppress_failures and the fact that Distributor.fire returns a
deferred), so the easiest way to fix this is actually by simplifying a bunch of
code.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This fixes #3518, and ensures that we get useful logs and metrics for lots of
things that happen in the background.
(There are certainly more things that happen in the background; these are just
the common ones I've found running a single-process synapse locally).
|
| |\| |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
The get_entities_changed function was changed to return all changed
entities since the given stream position, rather than only those changed
from a given list of entities. This resulted in the function incorrectly
returning large numbers of entities that, for example, caused large
increases in database usage.
|
| | |\
| | |
| | | |
Don't return unknown entities in get_entities_changed
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The stream cache keeps track of all entities that have changed since
a particular stream position, so get_entities_changed does not need to
return unknown entites when given a larger stream position.
This makes it consistent with the behaviour of has_entity_changed.
|
| | |/
| |
| |
| |
| |
| |
| |
| | |
popitem removes the *most recent* item by default [1]. We want the oldest.
Fixes #3524
[1]: https://docs.python.org/2/library/collections.html#collections.OrderedDict.popitem
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This line shows up as about 5% of cpu time on a synchrotron:
not_known_entities = set(entities) - set(self._entity_to_key)
Presumably the problem here is that _entity_to_key can be largeish, and
building a set for its keys every time this function is called is slow.
Here we rewrite the logic to avoid building so many sets.
|
| | |
| |
| |
| |
| | |
Let's try to include time spent in the DB threads in the per-request/block cpu
usage metrics.
|
| | |
| |
| |
| |
| | |
Factor out the resource usage tracking out to a separate object, which can be
passed around and copied independently of the logcontext itself.
|
| | | |
|
| |\| |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
When _get_state_for_groups is given a wildcard filter, just do a complete
lookup. Hopefully this will give us the best of both worlds by not filling up
the ram if we only need one or two keys, but also making the cache still work
for the federation reader usecase.
|
| | |\
| | |
| | | |
Log number of events fetched from DB
|
| | | |
| | |
| | |
| | | |
so that we can stub it for the sentinel and not have a billion failing UTs
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
When we finish processing a request, log the number of events we fetched from
the database to handle it.
[I'm trying to figure out which requests are responsible for large amounts of
event cache churn. It may turn out to be more helpful to add counts to the
prometheus per-request/block metrics, but that is an extension to this code
anyway.]
|
| | |/ |
|
