summary refs log tree commit diff
path: root/synapse/push (unfollow)
Commit message (Collapse)AuthorFilesLines
2020-07-30Add deprecation warningsOlivier Wilkinson (reivilibre)2-0/+36
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-07-301.18.0Olivier Wilkinson (reivilibre)4-2/+16
2020-07-29Update worker docs with recent enhancements (#7969)Erik Johnston14-235/+413
2020-07-281.18.0rc2 v1.18.0rc2Richard van der Hoff5-4/+17
2020-07-28Typing worker needs to handle stream update requests (#7967)Erik Johnston2-1/+2
IIRC this doesn't break tests because its only hit on reconnection, or something. Basically, when a process needs to fetch missing updates for the `typing` stream it needs to query the writer instance via HTTP (as we don't write typing notifications to the DB), the problem was that the endpoint (`streams`) was only registered on master and specifically not on the typing writer worker.
2020-07-27Handle replication commands synchronously where possible (#7876)Richard van der Hoff5-86/+113
Most of the stuff we do for replication commands can be done synchronously. There's no point spinning up background processes if we're not going to need them.
2020-07-27update changelog v1.18.0rc1Richard van der Hoff1-16/+4
2020-07-271.18.0rc1Richard van der Hoff76-76/+106
2020-07-27Fix error reporting when using `opentracing.trace` (#7961)Erik Johnston3-12/+4
2020-07-27Fix typing replication not being handled on master (#7959)Erik Johnston4-7/+12
Handling of incoming typing stream updates from replication was not hooked up on master, effecting set ups where typing was handled on a different worker. This is really only a problem if the master process is also handling sync requests, which is unlikely for those that are at the stage of moving typing off. The other observable effect is that if a worker restarts or a replication connect drops then the typing worker will issue a `POSITION typing`, triggering master process to try and stream *all* typing updates from position 0. Fixes #7907
2020-07-27Remove hacky error handling for inlineDeferreds. (#7950)Patrick Cloke3-21/+13
2020-07-27Convert tests/rest/admin/test_room.py to unix file endings (#7953)Andrew Morgan2-1447/+1448
Converts tests/rest/admin/test_room.py to have unix file endings after they were accidentally changed in #7613. Keeping the same changelog as #7613 as it hasn't gone out in a release yet.
2020-07-27Support oEmbed for media previews. (#7920)Patrick Cloke3-53/+355
Fixes previews of Twitter URLs by using their oEmbed endpoint to grab content.
2020-07-24Convert state resolution to async/await (#7942)Patrick Cloke18-184/+198
2020-07-24Fix up types and comments that refer to Deferreds. (#7945)Patrick Cloke9-157/+174
2020-07-24Do not convert async functions to Deferreds in the interactive_auth_handler ↵Patrick Cloke2-26/+22
(#7944)
2020-07-24Convert more of the media code to async/await (#7873)Patrick Cloke4-34/+47
2020-07-24Return an empty body for OPTIONS requests. (#7886)Patrick Cloke3-25/+12
2020-07-24Downgrade warning on client disconnect to INFO (#7928)Richard van der Hoff3-61/+3
Clients disconnecting before we finish processing the request happens from time to time. We don't need to yell about it
2020-07-23Convert presence handler helpers to async/await. (#7939)Patrick Cloke3-24/+24
2020-07-23Update the auth providers to be async. (#7935)Patrick Cloke4-112/+118
2020-07-23Put a cache on `/state_ids` (#7931)Richard van der Hoff2-2/+12
If we send out an event which refers to `prev_events` which other servers in the federation are missing, then (after a round or two of backfill attempts), they will end up asking us for `/state_ids` at a particular point in the DAG. As per https://github.com/matrix-org/synapse/issues/7893, this is quite expensive, and we tend to see lots of very similar requests around the same time. We can therefore handle this much more efficiently by using a cache, which (a) ensures that if we see the same request from multiple servers (or even the same server, multiple times), then they share the result, and (b) any other servers that miss the initial excitement can also benefit from the work. [It's interesting to note that `/state` has a cache for exactly this reason. `/state` is now essentially unused and replaced with `/state_ids`, but evidently when we replaced it we forgot to add a cache to the new endpoint.]
2020-07-23Abort federation requests if the client disconnects early (#7930)Richard van der Hoff2-0/+7
For inbound federation requests, if a given remote server makes too many requests at once, we start stacking them up rather than processing them immediatedly. However, that means that there is a fair chance that the requesting server will disconnect before we start processing the request. In that case, if it was a read-only request (ie, a GET request), there is absolutely no point in building a response (and some requests are quite expensive to handle). Even in the case of a POST request, one of two things will happen: * Most likely, the requesting server will retry the request and we'll get the information anyway. * Even if it doesn't, the requesting server has to assume that we didn't get the memo, and act accordingly. In short, we're better off aborting the request at this point rather than ploughing on with what might be a quite expensive request.
2020-07-23Reorder database docs to promote postgresql. (#7933)Michael Kaye2-5/+6
2020-07-23Convert the federation agent and related code to async/await. (#7874)Patrick Cloke5-53/+51
2020-07-22Follow-up to admin API to re-activate accounts (#7908)Patrick Cloke3-6/+43
2020-07-22Convert the message handler to async/await. (#7884)Patrick Cloke10-238/+273
2020-07-22Update the dates for ACME v1 EOLBrendan Abolivier2-2/+4
As per https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430
2020-07-22Skip serializing /sync response if client has disconnected (#7927)Richard van der Hoff2-0/+7
... it's a load of work which may be entirely redundant.
2020-07-22Add debugging to sync response generation (#7929)Richard van der Hoff3-2/+19
2020-07-22Remove an unused prometheus metric (#7878)Richard van der Hoff2-3/+2
2020-07-22Track command processing as a background process (#7879)Richard van der Hoff4-3/+40
I'm going to be doing more stuff synchronously, and I don't want to lose the CPU metrics down the sofa.
2020-07-22Clean up PreserveLoggingContext (#7877)Richard van der Hoff2-16/+14
This had some dead code and some just plain wrong docstrings.
2020-07-22fix an incorrect commentRichard van der Hoff1-2/+2
2020-07-21Convert room list handler to async/await. (#7912)Patrick Cloke3-41/+32
2020-07-21Element CSS and logo in email templates (#7919)Jason Robinson4-0/+12
Use Element CSS and logo in notification emails when app name is Element. Signed-off-by: Jason Robinson <jasonr@matrix.org>
2020-07-20Lint the contrib/ directory in CI and linting scripts, add synctl to linting ↵Andrew Morgan11-82/+71
script (#7914) Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too. Fixes https://github.com/matrix-org/synapse/issues/7910
2020-07-20Remove unused code from synapse.logging.utils. (#7897)Karthikeyan Singaravelan2-126/+2
2020-07-20Fix a typo in the sample config. (#7890)Adrian3-2/+3
2020-07-20Fix deprecation warning: import ABC from collections.abc (#7892)Karthikeyan Singaravelan5-6/+7
2020-07-20Change sample config's postgres user to synapse_user (#7889)Andrew Morgan3-2/+3
The [postgres setup docs](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md#set-up-database) recommend setting up your database with user `synapse_user`. However, uncommenting the postgres defaults in the sample config leave you with user `synapse`. This PR switches the sample config to recommend `synapse_user`. Took a me a second to figure this out, so assume this will beneficial to others.
2020-07-20Fix deprecation warning due to invalid escape sequences (#7895)Karthikeyan Singaravelan2-4/+5
* Fix deprecation warnings due to invalid escape sequences. * Add changelog Signed-off-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
2020-07-17Remove Ubuntu Eoan that is now EOL (#7888)Gary Kim2-1/+1
2020-07-17Fix the trace function for async functions. (#7872)Patrick Cloke2-23/+41
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-07-17Add help for creating a user via docker (#7885)Michael Kaye2-0/+16
2020-07-17Switch to Debian:Slim from Alpine for the docker image (#7839)Christopher May-Townsend3-40/+30
As mentioned in #7397, switching to a debian base should help with multi-arch work to save time on compiling. This is unashamedly based on #6373, but without the extra functionality. Switch python version back to generic 3.7 to always pull the latest. Essentially, keeping this as small as possible. The image is bigger though unfortunately.
2020-07-17Stop using 'device_max_stream_id' (#7882)Erik Johnston4-5/+3
It serves no purpose and updating everytime we write to the device inbox stream means all such transactions will conflict, causing lots of transaction failures and retries.
2020-07-17Fix TypeError in synapse.notifier (#7880)Erik Johnston2-0/+9
Fixes #7774
2020-07-17Add a default limit (of 100) to get/sync operations. (#7858)Patrick Cloke4-4/+18
2020-07-17Change "unknown room ver" logging to warning. (#7881)Erik Johnston2-1/+2
It's somewhat expected for us to have unknown room versions in the database due to room version experiments.
2020-07-17Convert device handler to async/await (#7871)Patrick Cloke6-166/+162
2020-07-17Convert synapse.app to async/await. (#7868)Patrick Cloke4-41/+37
2020-07-17Convert _base, profile, and _receipts handlers to async/await (#7860)Patrick Cloke6-59/+53
2020-07-16Add admin endpoint to get members in a room. (#7842)Michael Albert5-1/+107
2020-07-16Consistently use `db_to_json` to convert from database values to JSON ↵Patrick Cloke22-82/+80
objects. (#7849)
2020-07-16Combine nginx federation server blocks (#7823)Luke Faraone1-11/+5
I'm pretty sure there's no technical reason these have to be distinct server blocks, so collapse into one and go with the more terse location block. Signed-off-by: Luke W Faraone <luke@faraone.cc>
2020-07-16Optimise queueing of inbound replication commands (#7861)Richard van der Hoff2-116/+216
When we get behind on replication, we tend to stack up background processes behind a linearizer. Bg processes are heavy (particularly with respect to prometheus metrics) and linearizers aren't terribly efficient once the queue gets long either. A better approach is to maintain a queue of requests to be processed, and nominate a single process to work its way through the queue. Fixes: #7444
2020-07-16Reject attempts to join empty rooms over federation (#7859)Richard van der Hoff2-2/+14
We shouldn't allow others to make_join through us if we've left the room; reject such attempts with a 404. Fixes #7835. Fixes #6958.
2020-07-16Allow moving typing off master (#7869)Erik Johnston10-178/+284
2020-07-16Add ability to run multiple pusher instances (#7855)Erik Johnston9-82/+293
This reuses the same scheme as federation sender sharding
2020-07-16