summary refs log tree commit diff
path: root/contrib/scripts (unfollow)
Commit message (Collapse)AuthorFilesLines
2021-03-09Fix additional type hints. (#9543)Patrick Cloke9-18/+32
Type hint fixes due to Twisted 21.2.0 adding type hints.
2021-03-09Handle image transparency better when thumbnailing. (#9473)Patrick Cloke3-11/+30
Properly uses RGBA mode for 1- and 8-bit images with transparency (instead of RBG mode).
2021-03-09Add a list of hashes to ignore during git blame. (#9560)Patrick Cloke3-0/+10
The hashes are from commits due to auto-formatting, e.g. running black. git can be configured to use this automatically by running the following: git config blame.ignoreRevsFile .git-blame-ignore-revs
2021-03-09Fixup sample configErik Johnston1-2/+1
After 0764d0c6e575793ca506cf021aff3c4b9e0a5972
2021-03-09Prevent the config-lint script erroring out on any sample_config changes (#9562)Andrew Morgan2-2/+8
I noticed that I'd occasionally have `scripts-dev/lint.sh` fail when messing about with config options in my PR. The script calls `scripts-dev/config-lint.sh`, which attempts some validation on the sample config. It does this by using `sed` to edit the sample_config, and then seeing if the file changed using `git diff`. The problem is: if you changed the sample_config as part of your commit, this script will error regardless. This PR attempts to change the check so that existing, unstaged changes to the sample_config will not cause the script to report an invalid file.
2021-03-09Add logging to ObservableDeferred callbacks (#9523)Jonathan de Jong2-8/+19
2021-03-08quick config comment tweak to clarify allow_profile_lookup_over_federationMatthew Hodgson1-2/+1
2021-03-08Add ResponseCache tests. (#9458)Jonathan de Jong10-20/+156
2021-03-08Warn that /register will soon require a type when called with an access ↵Will Hunt2-0/+7
token (#9559) This notice is giving a heads up to the planned spec compliance fix https://github.com/matrix-org/synapse/pull/9548.
2021-03-08Add type hints to purge room and server notice admin API. (#9520)Dirk Klimpel3-15/+24
2021-03-08Add a basic test for purging rooms. (#9541)Patrick Cloke2-26/+46
Unfortunately this doesn't test re-joining the room since that requires having another homeserver to query over federation, which isn't easily doable in unit tests.
2021-03-08Fixup changelog v1.29.0 release-v1.29.0Erik Johnston1-0/+3
2021-03-08 1.29.0Erik Johnston3-5/+11
2021-03-08Create a SynapseReactor type which incorporates the necessary reactor ↵Patrick Cloke8-12/+32
interfaces. (#9528) This helps fix some type hints when running with Twisted 21.2.0.
2021-03-06Update reverse proxy to add OpenBSD relayd example configuration. (#9508)Leo Bärring3-4/+53
Update reverse proxy to add OpenBSD relayd example configuration. Signed-off-by: Leo Bärring <leo.barring@protonmail.com>
2021-03-05Add additional SAML2 upgrade notes (#9550)Ben Banfield-Zanin2-0/+8
2021-03-04Replace `last_*_pdu_age` metrics with timestamps (#9540)Richard van der Hoff4-12/+11
Following the advice at https://prometheus.io/docs/practices/instrumentation/#timestamps-not-time-since, it's preferable to export unix timestamps, not ages. There doesn't seem to be any particular naming convention for timestamp metrics.
2021-03-04Prometheus metrics for logins and registrations (#9511)Richard van der Hoff4-4/+43
Add prom metrics for number of users successfully registering and logging in, by SSO provider.
2021-03-04Record the SSO Auth Provider in the login token (#9510)Richard van der Hoff13-151/+258
This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)
2021-03-04Fix link in UPGRADES v1.29.0rc1Erik Johnston1-3/+3
2021-03-04Fix changelogErik Johnston1-6/+3
2021-03-04 1.29.0rc1Erik Johnston42-41/+55
2021-03-03Fix additional type hints from Twisted upgrade. (#9518)Patrick Cloke12-61/+96
2021-03-03Set X-Forwarded-Proto header when frontend-proxy proxies a request (#9539)Richard van der Hoff2-2/+14
Should fix some remaining warnings
2021-03-03Fix 'rejected_events_metadata' background update (#9537)Erik Johnston2-1/+4
Turns out matrix.org has an event that has duplicate auth events (which really isn't supposed to happen, but here we are). This caused the background update to fail due to `UniqueViolation`.
2021-03-03Purge chain cover tables when purging events. (#9498)Patrick Cloke3-10/+38
2021-03-03Add type hints to user admin API. (#9521)Dirk Klimpel4-35/+63
2021-03-03Bump the mypy and mypy-zope versions. (#9529)Patrick Cloke4-3/+4
2021-03-03Make deleting stale pushers a background update (#9536)Erik Johnston3-1/+55
2021-03-03Update nginx reverse-proxy docs (#9512)Richard van der Hoff2-0/+3
Turns out nginx overwrites the Host header by default.
2021-03-03Prevent presence background jobs from running when presence is disabled (#9530)Aaron Raimist2-14/+18
Prevent presence background jobs from running when presence is disabled Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-03-02Revert "Fix #8518 (sync requests being cached wrongly on timeout) (#9358)"Patrick Cloke3-35/+3
This reverts commit f5c93fc9931e4029bbd8000f398b6f39d67a8c46. This is being backed out due to a regression (#9507) and additional review feedback being provided.
2021-03-02Re-run rejected metadata background update. (#9503)Erik Johnston2-0/+10
It landed in schema version 58 after 59 had been created, causing some servers to not run it. The main effect of was that not all rooms had their chain cover calculated correctly. After the BG updates complete the chain covers will get fixed when a new state event in the affected rooms is received.
2021-03-02Fix SQL delta file taking a long time to run (#9516)Erik Johnston4-5/+60
Fixes #9504
2021-03-02Add type hints to device and event report admin API (#9519)Dirk Klimpel3-16/+40
2021-03-01Fix a bug when a room alias is given to the admin join endpoint (#9506)Patrick Cloke2-58/+75
2021-03-01(Hopefully) stop leaking file descriptors in media repo. (#9497)Patrick Cloke3-38/+85
By consuming the response if the headers imply that the content is too large.
2021-03-01Use the proper Request in type hints. (#9515)Patrick Cloke18-31/+38
This also pins the Twisted version in the mypy job for CI until proper type hints are fixed throughout Synapse.
2021-02-26Allow bytecode again (#9502)Jonathan de Jong8-10/+16
In #75, bytecode was disabled (from a bit of FUD back in `python<2.4` days, according to dev chat), I think it's safe enough to enable it again. Added in `__pycache__/` and `.pyc`/`.pyd` to `.gitignore`, to extra-insure compiled files don't get committed. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-02-26Add support for no_proxy and case insensitive env variables (#9372)Tim Leung6-64/+114
### Changes proposed in this PR - Add support for the `no_proxy` and `NO_PROXY` environment variables - Internally rely on urllib's [`proxy_bypass_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2519) - Extract env variables using urllib's `getproxies`/[`getproxies_environment`](https://github.com/python/cpython/blob/bdb941be423bde8b02a5695ccf51c303d6204bed/Lib/urllib/request.py#L2488) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment This does contain behaviour changes for consumers so making sure these are called out: - `no_proxy`/`NO_PROXY` is now respected - lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY` Related to #9306 which also uses `ProxyAgent` Signed-off-by: Timothy Leung tim95@hotmail.co.uk
2021-02-26SSO: redirect to public URL before setting cookies (#9436)Richard van der Hoff7-28/+130
... otherwise, we don't get the cookie back.
2021-02-26Call out the need for an X-Forwarded-Proto in the upgrade notes (#9501)Richard van der Hoff4-1/+28
2021-02-25Test that we require validated email for email pushers (#9496)Erik Johnston3-2/+39
2021-02-25Ensure pushers are deleted for deactivated accounts (#9285)Erik Johnston4-0/+70
2021-02-25Fixup changelog v1.28.0 release-v1.28.0Erik Johnston1-1/+1
2021-02-25Fixup changelogErik Johnston1-4/+5
2021-02-25 1.28.0Erik Johnston4-2/+16
2021-02-24Add support for X-Forwarded-Proto (#9472)Richard van der Hoff3-28/+94
rewrite XForwardedForRequest to set `isSecure()` based on `X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.
2021-02-24Fix typo in spam checker documentationAndrew Morgan1-1/+1
2021-02-24Add SQL delta for deleting stale pushers (#9479)Erik Johnston2-0/+20
2021-02-24Fix #8518 (sync requests being cached wrongly on timeout) (#9358)Jonathan de Jong3-3/+35
This fixes #8518 by adding a conditional check on `SyncResult` in a function when `prev_stream_token == current_stream_token`, as a sanity check. In `CachedResponse.set.<remove>()`, the result is immediately popped from the cache if the conditional function returns "false". This prevents the caching of a timed-out `SyncResult` (that has `next_key` as the stream key that produced that `SyncResult`). The cache is prevented from returning a `SyncResult` that makes the client request the same stream key over and over again, effectively making it stuck in a loop of requesting and getting a response immediately for as long as the cache keeps those values. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-02-24Clean up `ShardedWorkerHandlingConfig` (#9466)Erik Johnston14-63/+128
* Split ShardedWorkerHandlingConfig This is so that we have a type level understanding of when it is safe to call `get_instance(..)` (as opposed to `should_handle(..)`). * Remove special cases in ShardedWorkerHandlingConfig. `ShardedWorkerHandlingConfig` tried to handle the various different ways it was possible to configure federation senders and pushers. This led to special cases that weren't hit during testing. To fix this the handling of the different cases is moved from there and `generic_worker` into the worker config class. This allows us to have the logic in one place and allows the rest of the code to ignore the different cases.
2021-02-24Refactor to ensure we call check_consistency (#9470)Erik Johnston8-28/+72
The idea here is to stop people forgetting to call `check_consistency`. Folks can still just pass in `None` to the new args in `build_sequence_generator`, but hopefully they won't.
2021-02-23Add back the deprecated SAML endpoint. (#9474)Patrick Cloke2-1/+7
2021-02-23Add a comment about systemd-python. (#9464)Richard van der Hoff2-0/+4
This confused me for a while.
2021-02-23Include newly added sequences in the port DB script. (#9449)Patrick Cloke4-26/+55
And ensure the consistency of `event_auth_chain_id`.
2021-02-22Fix deleting pushers when using sharded pushers. (#9465)Erik Johnston10-67/+94
2021-02-22Remove vestiges of uploads_path config (#9462)Richard van der Hoff5-4/+1
`uploads_path` was a thing that was never used; most of it was removed in #6628 but a few vestiges remained.
2021-02-22Add an `order_by` field to list users' media admin API. (#8978)Dirk Klimpel5-29/+325
2021-02-22example systemd config: propagate reloads to units (#9463)Richard van der Hoff3-0/+3
It should be possible to reload `synapse.target` to have the reload propagate to all the synapse units.
2021-02-22Remove cache for get_shared_rooms_for_users (#9416)Andrew Morgan3-37/+43
This PR remove the cache for the `get_shared_rooms_for_users` storage method (the db method driving the experimental "what rooms do I share with this user?" feature: [MSC2666](https://github.com/matrix-org/matrix-doc/pull/2666)). Currently subsequent requests to the endpoint will return the same result, even if your shared rooms with that user have changed. The cache was added in https://github.com/matrix-org/synapse/pull/7785, but we forgot to ensure it was invalidated appropriately. Upon attempting to invalidate it, I found that the cache had to be entirely invalidated whenever a user (remote or local) joined or left a room. This didn't make for a very useful cache, especially for a function that may or may not be called very often. Thus, I've opted to remove it instead of invalidating it.
2021-02-22Clean up the user directory sample config section (#9385)Andrew Morgan3-50/+67
The user directory sample config section was a little messy, and didn't adhere to our [recommended config format guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format). This PR cleans that up a bit.
2021-02-19Ratelimit cross-user key sharing requests. (#8957)Patrick Cloke9-17/+67
2021-02-19Add testErik Johnston1-3/+66
2021-02-19Add documentation and type hints to parse_duration. (#9432)Patrick Cloke2-2/+16
2021-02-19Fix style checking due to updated black.Patrick Cloke2-3/+5
2021-02-19Update release date. v1.28.0rc1Patrick Cloke1-1/+1
2021-02-19Support not providing an IdP icon when choosing a username. (#9440)Patrick Cloke2-1/+2
2021-02-19Be smarter about which hosts to send presence to when processing room joins ↵Andrew Morgan4-19/+54
(#9402) This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually. --- When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed. It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw. This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence: * If it was a local user join, send that user's latest presence to all servers in the room * If it was a remote user join, send the presence for all local users in the room to that homeserver We deduplicate by inserting all of those pending updates into a dictionary of the form: ``` { server_name1: {presence_update1, ...}, server_name2: {presence_update1, presence_update2, ...} } ``` Only after building this dict do we then start sending out presence updates.
2021-02-19Add a config option to prioritise local users in user directory search ↵Andrew Morgan5-9/+159
results (#9383) This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default. Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical. This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.
2021-02-19Return a 404 if we don't have the original file</