summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* Record mappings from saml users in an external tableRichard van der Hoff2019-09-136-10/+276
| | | | | | We want to assign unique mxids to saml users based on an incrementing suffix. For that to work, we need to record the allocated mxid in a separate table.
* Make the sample saml config closer to our standardsRichard van der Hoff2019-09-132-102/+121
| | | | It' still not great, thanks to the nested dictionaries, but it's better.
* 1.3.1 v1.3.1 github/release-v1.3.1 release-v1.3.1Richard van der Hoff2019-08-175-5/+28
|
* Drop dependency on sdnotify (#5871)Richard van der Hoff2019-08-173-11/+38
| | | | | | ... to save OSes which don't use it from having to maintain a port. Fixes #5865.
* add attributionRichard van der Hoff2019-08-161-1/+1
|
* Add changelog entry.Chris Moos2019-08-161-0/+1
|
* Fix issue with Synapse not starting up. Fixes #5866.Chris Moos2019-08-161-1/+1
| | | | Signed-off-by: Chris Moos <chris@chrismoos.com>
* Merge tag 'v1.3.0'Brendan Abolivier2019-08-15237-2739/+4310
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.3.0 (2019-08-15) ========================== Bugfixes -------- - Fix 500 Internal Server Error on `publicRooms` when the public room list was cached. ([\#5851](https://github.com/matrix-org/synapse/issues/5851)) Synapse 1.3.0rc1 (2019-08-13) ========================== Features -------- - Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login. ([\#5686](https://github.com/matrix-org/synapse/issues/5686)) - Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. ([\#5732](https://github.com/matrix-org/synapse/issues/5732)) - Synapse will no longer serve any media repo admin endpoints when `enable_media_repo` is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. ([\#5754](https://github.com/matrix-org/synapse/issues/5754), [\#5848](https://github.com/matrix-org/synapse/issues/5848)) - Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. ([\#5783](https://github.com/matrix-org/synapse/issues/5783)) - Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. ([\#5807](https://github.com/matrix-org/synapse/issues/5807)) Bugfixes -------- - Fix UISIs during homeserver outage. ([\#5693](https://github.com/matrix-org/synapse/issues/5693), [\#5789](https://github.com/matrix-org/synapse/issues/5789)) - Fix stack overflow in server key lookup code. ([\#5724](https://github.com/matrix-org/synapse/issues/5724)) - start.sh no longer uses deprecated cli option. ([\#5725](https://github.com/matrix-org/synapse/issues/5725)) - Log when we receive an event receipt from an unexpected origin. ([\#5743](https://github.com/matrix-org/synapse/issues/5743)) - Fix debian packaging scripts to correctly build sid packages. ([\#5775](https://github.com/matrix-org/synapse/issues/5775)) - Correctly handle redactions of redactions. ([\#5788](https://github.com/matrix-org/synapse/issues/5788)) - Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. ([\#5798](https://github.com/matrix-org/synapse/issues/5798)) - Fix check that tombstone is a state event in push rules. ([\#5804](https://github.com/matrix-org/synapse/issues/5804)) - Fix error when trying to login as a deactivated user when using a worker to handle login. ([\#5806](https://github.com/matrix-org/synapse/issues/5806)) - Fix bug where user `/sync` stream could get wedged in rare circumstances. ([\#5825](https://github.com/matrix-org/synapse/issues/5825)) - The purge_remote_media.sh script was fixed. ([\#5839](https://github.com/matrix-org/synapse/issues/5839)) Deprecations and Removals ------------------------- - Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration. ([\#5678](https://github.com/matrix-org/synapse/issues/5678), [\#5729](https://github.com/matrix-org/synapse/issues/5729)) - Remove non-functional 'expire_access_token' setting. ([\#5782](https://github.com/matrix-org/synapse/issues/5782)) Internal Changes ---------------- - Make Jaeger fully configurable. ([\#5694](https://github.com/matrix-org/synapse/issues/5694)) - Add precautionary measures to prevent future abuse of `window.opener` in default welcome page. ([\#5695](https://github.com/matrix-org/synapse/issues/5695)) - Reduce database IO usage by optimising queries for current membership. ([\#5706](https://github.com/matrix-org/synapse/issues/5706), [\#5738](https://github.com/matrix-org/synapse/issues/5738), [\#5746](https://github.com/matrix-org/synapse/issues/5746), [\#5752](https://github.com/matrix-org/synapse/issues/5752), [\#5770](https://github.com/matrix-org/synapse/issues/5770), [\#5774](https://github.com/matrix-org/synapse/issues/5774), [\#5792](https://github.com/matrix-org/synapse/issues/5792), [\#5793](https://github.com/matrix-org/synapse/issues/5793)) - Improve caching when fetching `get_filtered_current_state_ids`. ([\#5713](https://github.com/matrix-org/synapse/issues/5713)) - Don't accept opentracing data from clients. ([\#5715](https://github.com/matrix-org/synapse/issues/5715)) - Speed up PostgreSQL unit tests in CI. ([\#5717](https://github.com/matrix-org/synapse/issues/5717)) - Update the coding style document. ([\#5719](https://github.com/matrix-org/synapse/issues/5719)) - Improve database query performance when recording retry intervals for remote hosts. ([\#5720](https://github.com/matrix-org/synapse/issues/5720)) - Add a set of opentracing utils. ([\#5722](https://github.com/matrix-org/synapse/issues/5722)) - Cache result of get_version_string to reduce overhead of `/version` federation requests. ([\#5730](https://github.com/matrix-org/synapse/issues/5730)) - Return 'user_type' in admin API user endpoints results. ([\#5731](https://github.com/matrix-org/synapse/issues/5731)) - Don't package the sytest test blacklist file. ([\#5733](https://github.com/matrix-org/synapse/issues/5733)) - Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. ([\#5736](https://github.com/matrix-org/synapse/issues/5736)) - Blacklist some flakey tests in worker mode. ([\#5740](https://github.com/matrix-org/synapse/issues/5740)) - Fix some error cases in the caching layer. ([\#5749](https://github.com/matrix-org/synapse/issues/5749)) - Add a prometheus metric for pending cache lookups. ([\#5750](https://github.com/matrix-org/synapse/issues/5750)) - Stop trying to fetch events with event_id=None. ([\#5753](https://github.com/matrix-org/synapse/issues/5753)) - Convert RedactionTestCase to modern test style. ([\#5768](https://github.com/matrix-org/synapse/issues/5768)) - Allow looping calls to be given arguments. ([\#5780](https://github.com/matrix-org/synapse/issues/5780)) - Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. ([\#5785](https://github.com/matrix-org/synapse/issues/5785)) - Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. ([\#5787](https://github.com/matrix-org/synapse/issues/5787)) - Remove some spurious exceptions from the logs where we failed to talk to a remote server. ([\#5790](https://github.com/matrix-org/synapse/issues/5790)) - Improve performance when making `.well-known` requests by sharing the SSL options between requests. ([\#5794](https://github.com/matrix-org/synapse/issues/5794)) - Disable codecov GitHub comments on PRs. ([\#5796](https://github.com/matrix-org/synapse/issues/5796)) - Don't allow clients to send tombstone events that reference the room it's sent in. ([\#5801](https://github.com/matrix-org/synapse/issues/5801)) - Deny redactions of events sent in a different room. ([\#5802](https://github.com/matrix-org/synapse/issues/5802)) - Deny sending well known state types as non-state events. ([\#5805](https://github.com/matrix-org/synapse/issues/5805)) - Handle incorrectly encoded query params correctly by returning a 400. ([\#5808](https://github.com/matrix-org/synapse/issues/5808)) - Handle pusher being deleted during processing rather than logging an exception. ([\#5809](https://github.com/matrix-org/synapse/issues/5809)) - Return 502 not 500 when failing to reach any remote server. ([\#5810](https://github.com/matrix-org/synapse/issues/5810)) - Reduce global pauses in the events stream caused by expensive state resolution during persistence. ([\#5826](https://github.com/matrix-org/synapse/issues/5826)) - Add a lower bound to well-known lookup cache time to avoid repeated lookups. ([\#5836](https://github.com/matrix-org/synapse/issues/5836)) - Whitelist history visbility sytests in worker mode tests. ([\#5843](https://github.com/matrix-org/synapse/issues/5843))
| * 1.3.0 v1.3.0 github/release-v1.3.0 release-v1.3.0Brendan Abolivier2019-08-155-6/+14
| |
| * Remove libsqlite3-dev from required build dependencies. (#5766)Andrew Morgan2019-08-153-5/+4
| |
| * Merge pull request #5851 from matrix-org/rei/roomdir_maybedeferredreivilibre2019-08-152-2/+10
| |\ | | | | | | Room Directory: Wrap `get_local_public_room_list` call in `maybeDeferred`
| | * Update changelog.d/5851.bugfix reivilibre2019-08-141-2/+2
| | | | | | | | | | | | | | | Use imperative Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * NewsfileOlivier Wilkinson (reivilibre)2019-08-141-0/+2
| | | | | | | | | | | | Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| | * Wrap `get_local_public_room_list` call in `maybeDeferred` because itOlivier Wilkinson (reivilibre)2019-08-141-2/+8
| |/ | | | | | | | | | | | | is cached and so does not always return a `Deferred`. `await` does not silently pass-through non-Deferreds like `yield` used to. Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
| * 1.3.0rc1 v1.3.0rc1Brendan Abolivier2019-08-1363-62/+75
| |
| * Merge pull request #5848 from matrix-org/hawkowl/fix-mediarepo-worker-startupBrendan Abolivier2019-08-132-1/+2
| |\ | | | | | | Fix mediarepo worker startup
| | * changelogAmber H. Brown2019-08-141-0/+1
| | |
| | * fix config being a dict, actuallyAmber H. Brown2019-08-141-1/+1
| |/
| * Merge pull request #5809 from matrix-org/erikj/handle_pusher_stopErik Johnston2019-08-134-21/+55
| |\ | | | | | | Handle pusher being deleted during processing.
| | * Remove redundant returnErik Johnston2019-08-131-1/+0
| | |
| | * Clarify docstringErik Johnston2019-08-131-1/+1
| | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * NewsfileErik Johnston2019-08-011-0/+1
| | |
| | * Handle pusher being deleted during processing.Erik Johnston2019-08-013-21/+55
| | | | | | | | | | | | | | | Instead of throwing a StoreError lets break out of processing loop and mark the pusher as stopped.
| * | Don't load the media repo when configured to use an external media repo (#5754)Amber Brown2019-08-1310-91/+188
| | |
| * | Merge pull request #5836 from matrix-org/erikj/lower_bound_ttl_well_knownErik Johnston2019-08-134-171/+222
| |\ \ | | | | | | | | Add a lower bound to well-known TTL.
| | * | Make default well known cache global again.Erik Johnston2019-08-131-1/+4
| | | |
| | * | NewsfileErik Johnston2019-08-121-0/+1
| | | |
| | * | Move well known lookup into a separate clasErik Johnston2019-08-073-173/+216
| | | |
| | * | Add a lower bound for TTL on well known results.Erik Johnston2019-08-062-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It costs both us and the remote server for us to fetch the well known for every single request we send, so we add a minimum cache period. This is set to 5m so that we still honour the basic premise of "refetch frequently".
| * | | Merge pull request #5826 from matrix-org/erikj/reduce_event_pausesErik Johnston2019-08-132-128/+143
| |\ \ \ | | | | | | | | | | Don't unnecessarily block notifying of new events.
| | * | | NewsfileErik Johnston2019-08-061-0/+1
| | | | |
| | * | | Don't unnecessarily block notifying of new events.Erik Johnston2019-08-061-128/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When persisting events we calculate new stream orderings up front. Before we notify about an event all events with lower stream orderings must have finished being persisted. This PR moves the assignment of stream ordering till *after* calculated the new current state and split the batch of events into separate chunks for persistence. This means that if it takes a long time to calculate new current state then it will not block events in other rooms being notified about. This should help reduce some global pauses in the events stream which can last for tens of seconds (if not longer), caused by some particularly expensive state resolutions.
| * | | | Merge pull request #5843 from matrix-org/erikj/workers_hist_visErik Johnston2019-08-122-4/+1
| |\ \ \ \ | | | | | | | | | | | | Whitelist history visbility sytests for worker mode
| | * | | | NewsfileErik Johnston2019-08-121-0/+1
| | | | | |
| | * | | | Whitelist history visbility sytests for worker modeErik Johnston2019-08-121-4/+0
| | | | | |
| * | | | | Merge pull request #5839 from tcitworld/fix-purge-remote-media-scriptErik Johnston2019-08-122-1/+2
| |\ \ \ \ \ | | |/ / / / | |/| | | | Fix curl command typo in purge_remote_media.sh
| | * | | | Changelogs should end in '.' or '!'Erik Johnston2019-08-121-1/+1
| | | | | |
| | * | | | Fix curl command typo in purge_remote_media.shThomas Citharel2019-08-092-1/+2
| | | |/ / | | |/| | | | | | | | | | | | | | | | | | | | | | Was verbose option instead of -X, command didn't work Signed-off-by: Thomas Citharel <tcit@tcit.fr>
| * | | | Merge pull request #5788 from matrix-org/rav/metaredactionsRichard van der Hoff2019-08-123-101/+183
| |\ \ \ \ | | |/ / / | |/| | | Fix handling of redactions of redactions
| | * | | changelogRichard van der Hoff2019-07-301-0/+1
| | | | |
| | * | | Handle loops in redaction eventsRichard van der Hoff2019-07-302-60/+106
| | | | |
| | * | | recursively fetch redactionsRichard van der Hoff2019-07-301-32/+36
| | | | |
| | * | | split _get_events_from_db out of _enqueue_eventsRichard van der Hoff2019-07-291-32/+51
| | | | |
| | * | | Return dicts from _fetch_event_listRichard van der Hoff2019-07-291-15/+27
| | | | |
| * | | | Merge pull request #5825 from matrix-org/erikj/fix_empty_limited_syncErik Johnston2019-08-062-6/+38
| |\ \ \ \ | | | | | | | | | | | | Handle TimelineBatch being limited and empty.
| | * | | | NewsfileErik Johnston2019-08-061-0/+1
| | | | | |
| | * | | | Handle TimelineBatch being limited and empty.Erik Johnston2019-08-061-6/+37
| | | |/ / | | |/| | | | | | | | | | | | | | | | | | | | | | This hopefully addresses #5407 by gracefully handling an empty but limited TimelineBatch. We also add some logging to figure out how this is happening.
| * / | | Return 404 instead of 403 when retrieving an event without perms (#5798)Andrew Morgan2019-08-062-3/+12
| |/ / / | | | | | | | | | | | | | | | | Part of fixing matrix-org/sytest#652 Sytest PR: matrix-org/sytest#667
| * | | Account validity: allow defining HTML templates to serve the us… (#5807)Brendan Abolivier2019-08-018-11/+119
| |\ \ \ | | | | | | | | | | Account validity: allow defining HTML templates to serve the user on account renewal attempt
| | * | | LintBrendan Abolivier2019-08-013-17/+10
| | | | |
| | * | | Sample configBrendan Abolivier2019-08-011-0/+10
| | | | |
| | * | | ChangelogBrendan Abolivier2019-08-011-0/+1
| | | | |
| | * | | Add testsBrendan Abolivier2019-08-011-0/+37
| | | | |
| | * | | Allow defining HTML templates to serve the user on account renewalBrendan Abolivier2019-08-015-9/+76
| | | | |
| * | | | Merge pull request #5810 from matrix-org/erikj/no_server_reachableErik Johnston2019-08-012-4/+4
| |\ \ \ \ | | | | | | | | | | | | Return 502 not 500 when failing to reach any remote server.
| | * | | | NewsfileErik Johnston2019-08-011-0/+1
| | | | | |
| | * | | | Return 502 not 500 when failing to reach any remote server.Erik Johnston2019-08-011-4/+3
| | | | | |
| * | | | | Have ClientReaderSlavedStore inherit RegistrationStore (#5806)Andrew Morgan2019-08-012-21/+22
| | | | | | | | | | | | | | | | | | Fixes #5803
| * | | | | Merge pull request #5808 from matrix-org/erikj/parse_decode_errorErik Johnston2019-08-012-1/+7
| |\ \ \ \ \ | | |/ / / / | |/| | | | Handle incorrectly encoded query params correctly
| | * | | | NewsfileErik Johnston2019-08-011-0/+1
| | | | | |
| | * | | | Handle incorrectly encoded query params correctlyErik Johnston2019-08-011-1/+6
| | | |_|/ | | |/| |
| * | | | Merge pull request #5805 from matrix-org/erikj/validate_stateErik Johnston2019-08-012-2/+10
| |\ \ \ \ | | | | | | | | | | | | Validate well known state events are state events.
| | * | | | NewsfileErik Johnston2019-07-311-0/+1
| | | | | |
| | * | | | Validate well-known state events are state events.Erik Johnston2019-07-311-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lets disallow sending things like memberships, topics etc as non-state events.
| * | | | | Merge pull request #5801 from matrix-org/erikj/recursive_tombstoneErik Johnston2019-08-012-0/+10
| |\ \ \ \ \ | | |_|/ / / | |/| | | | Don't allow clients to send tombstones that reference the same room
| | * | | | Fix spelling.Erik Johnston2019-08-012-2/+4
| | |/ / / | | | | | | | | | | | | | | | Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
| | * | | NewsfileErik Johnston2019-07-311-0/+1
| | | | |
| | * | | Don't allow clients to send tombstones that reference the same roomErik Johnston2019-07-311-0/+7
| | | | |
| * | | | Merge pull request #5802 from matrix-org/erikj/deny_redacting_different_roomErik Johnston2019-08-012-1/+4
| |\ \ \ \ | | | | | | | | | | | | Deny redaction of events in a different room.
| | * | | | NewsfileErik Johnston2019-07-311-0/+1
| | | | | |
| | * | | | Deny redaction of events in a different room.Erik Johnston2019-07-311-1/+3
| | |/ / / | | | | | | | | | | | | | | | | | | | | We already correctly filter out such redactions, but we should also deny them over the CS API.
| * | | | Merge pull request #5790 from matrix-org/erikj/groups_request_errorsErik Johnston2019-08-013-32/+61
| |\ \ \ \ | | | | | | | | | | | | Handle RequestSendFailed exception correctly in more places.
| | * | | | NewsfileErik Johnston2019-07-301-0/+1
| | | | | |
| | * | | | Don't log as exception when failing durig backfillErik Johnston2019-07-301-0/+3
| | | | | |
| | * | | | Correctly handle errors doing requests to group serversErik Johnston2019-07-291-32/+57
| | | | | |
| * | | | | Merge pull request #5804 from matrix-org/erikj/match_against_state_keyErik Johnston2019-08-012-1/+8
| |\ \ \ \ \ | | |_|/ / / | |/| | | | Explicitly check that tombstone is a state event before notifying.
| | * | | | NewsfileErik Johnston2019-07-311-0/+1
| | | | | |
| | * | | | Explicitly check that tombstone is a state event before notifying.Erik Johnston2019-07-311-1/+7
| |/ / / /
| * | | | Change user deactivated errcode to USER_DEACTIVATED and use it (#5686)Andrew Morgan2019-07-312-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | This is intended as an amendment to #5674 as using M_UNKNOWN as the errcode makes it hard for clients to differentiate between an invalid password and a deactivated user (the problem we were trying to solve in the first place). M_UNKNOWN was originally chosen as it was presumed than an MSC would have to be carried out to add a new code, but as Synapse often is the testing bed for new MSC implementations, it makes sense to try it out first in the wild and then add it into the spec if it is successful. Thus this PR return a new M_USER_DEACTIVATED code when a deactivated user attempts to login.
| * | | | Remove duplicate return statementAndrew Morgan2019-07-311-1/+0
| | | | |
| * | | | Merge pull request #5794 from matrix-org/erikj/share_ssl_options_for_well_knownErik Johnston2019-07-314-17/+20
| |\ \ \ \ | | | | | | | | | | | | Share SSL options for well-known requests
| | * | | | NewsfileErik Johnston2019-07-311-0/+1
| | | | | |
| | * | | | Share SSL options for well-known requestsErik Johnston2019-07-313-17/+19
| | | | | |
| * | | | | Remove DelayedCall debugging from test runs (#5787)Amber Brown2019-07-312-6/+1
| | | | | |
| * | | | | Merge pull request #5796 from matrix-org/erikj/disable_codecov_reportErik Johnston2019-07-312-2/+2
| |\ \ \ \ \ | | |/ / / / | |/| | | | Disable codecov reports to GH comments.
| | * | | | NewsfileErik Johnston2019-07-311-0/+1
| | | | | |
| | * | | | Disable codecov reports to GH comments.Erik Johnston2019-07-311-2/+1
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | The double posting is really annoying, and I don't think anyone is actually reading them. The commit statuses should give a good summary and will link to a full report.
| * | | | Merge pull request #5793 from matrix-org/erikj/fix_bg_updateBrendan Abolivier2019-07-303-1/+23
| |\ \ \ \ | | |_|/ / | |/| | | Don't recreate current_state_events.membership column
| | * | | NewsfileErik Johnston2019-07-301-0/+1
| | | | |
| | * | | Don't recreate columnErik Johnston2019-07-302-1/+22
| | | | |
| * | | | Merge pull request #5792 from matrix-org/erikj/fix_bg_updateErik Johnston2019-07-303-1/+2
| |\| | | | | | | | | | | | | Fix current_state_events membership background update.
| | * | | NewsfileErik Johnston2019-07-301-0/+1
| | | | |
| | * | | Fix current_state_events membership background update.Erik Johnston2019-07-302-1/+1
| |/ / / | | | | | | | | | | | | | | | | | | | | Turns out not all rooms are in `rooms`, so lets fetch the room list from `current_state_events`. We move the delta file to force it to be run again.
| * | | Merge pull request #5789 from matrix-org/erikj/fix_error_handling_keysErik Johnston2019-07-302-4/+4
| |\ \ \ | | | | | | | | | | Fix error handling when fetching remote device keys
| | * | | NewsfileErik Johnston2019-07-301-0/+1
| | | | |
| | * | | Unwrap errorErik Johnston2019-07-301-1/+2
| | | | |
| | * | | Fix error handling when fetching remote device keysErik Johnston2019-07-301-3/+1
| |/ / /
| * | | Merge pull request #5746 from matrix-org/erikj/test_bg_update_currnet_stateErik Johnston2019-07-302-1/+37
| |\ \ \ | | | | | | | | | | Add unit test for current state membership bg update
| | * | | NewsfileErik Johnston2019-07-231-0/+1
| | | | |
| | * | | Add unit test for current state membership bg updateErik Johnston2019-07-231-1/+36
| | | | |
| * | | | Remove non-functional 'expire_access_token' setting (#5782)Richard van der Hoff2019-07-308-37/+9
| | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | The `expire_access_token` didn't do what it sounded like it should do. What it actually did was make Synapse enforce the 'time' caveat on macaroons used as access tokens, but since our access token macaroons never contained such a caveat, it was always a no-op. (The code to add 'time' caveats was removed back in v0.18.5, in #1656)
| * | | Room Complexity Client Implementation (#5783)Amber Brown2019-07-308-14/+298
| | |/ | |/|
| * | Merge pull request #5774 from matrix-org/erikj/fix_rejected_membershipErik Johnston2019-07-292-0/+7
| |\ \ | | | | | | | | Fix room summary when rejected events are in state
| | * | NewsfileErik Johnston2019-07-261-0/+1
| | | |
| | * | Fix room summary when rejected events are in stateErik Johnston2019-07-261-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Annoyingly, `current_state_events` table can include rejected events, in which case the membership column will be null. To work around this lets just always filter out null membership for now.
| * | | Merge pull request #5752 from matrix-org/erikj/forgotten_userErik Johnston2019-07-293-35/+124
| |\ \ \ | | | | | | | | | | Remove some more joins on room_memberships
| | * | | Expand commentErik Johnston2019-07-291-0/+5
| | | | |
| | * | | Remove lost commentErik Johnston2019-07-291-7/+0
| | |/ /
| | * | NewsfileErik Johnston2019-07-241-0/+1
| | | |
| | * | Remove join when calculating room summaries.Erik Johnston2019-07-241-13/+21
| | | |
| | * | Remove join on room_memberships when fetching rooms for user.Erik Johnston2019-07-241-22/+36
| | | |
| | * | Add function to get all forgotten rooms for userErik Johnston2019-07-242-0/+68
| | |/ | | | | | | | | | | | | | | | This will allow us to efficiently filter out rooms that have been forgotten in other queries without having to join against the `room_memberships` table.
| * | Merge pull request #5770 from matrix-org/erikj/fix_current_state_event_sqliteErik Johnston2019-07-293-4/+6
| |\ \ | | | | | | | | Fix current_state bg update to work on old SQLite
| | * | NewsfileErik Johnston2019-07-291-0/+1
| | | |
| | * | Update old deps unit test to use old sqlite3Erik Johnston2019-07-291-2/+3
| | | |
| | * | Fix current_state bg update to work on old SQLiteErik Johnston2019-07-291-2/+2
| | | |
| * | | Move some timeout checking logs to DEBUG #5785 Amber Brown2019-07-303-2/+3
| | | |
| * | | Update the device list cache when keys/query is called (#5693)Jorik Schellekens2019-07-293-74/+137
| | | |
| * | | Fix debian packages for sid being called buster. (#5775)Erik Johnston2019-07-303-1/+8
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix debian packages for sid being called buster. I don't know why the sid images return buster as its codename in `lsb_release` but it does, so lets just grab the codename from the distro we pass into dockerfile * Newsfile
| * | Merge pull request #5780 from matrix-org/baboliver/loopingcall-argsBrendan Abolivier2019-07-292-2/+5
| |\ \ | | | | | | | | Add ability to pass arguments to looping calls
| | * | ChangelogBrendan Abolivier2019-07-291-0/+1
| | | |
| | * | Add kwargs and docBrendan Abolivier2019-07-291-2/+4
| | | |
| | * | Add ability to pass arguments to looping callsBrendan Abolivier2019-07-291-2/+2
| |/ /
| * | Merge branch 'master' into developRichard van der Hoff2019-07-266-4/+38
| |\ \
| * \ \ Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatchRichard van der Hoff2019-07-263-4/+38
| |\ \ \ | | | | | | | | | | Log when we receive a /make_* request from a different origin
| * | | | Convert RedactionTestCase to modern test style (#5768)Richard van der Hoff2019-07-262-36/+39
| | | | |
| * | | | Merge pull request #5767 from matrix-org/rav/redactions/cross_room_idRichard van der Hoff2019-07-252-0/+28
| |\ \ \ \ | | | | | | | | | | | | log when a redaction attempts to redact an event in a different room
| * | | | | Fix some error cases in the caching layer. (#5749)Richard van der Hoff2019-07-253-35/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There was some inconsistent behaviour in the caching layer around how exceptions were handled - particularly synchronously-thrown ones. This seems to be most easily handled by pushing the creation of ObservableDeferreds down from CacheDescriptor to the Cache.
| * | | | | Merge branch 'master' into developAndrew Morgan2019-07-253-3/+12
| |\ \ \ \ \
| * \ \ \ \ \ Merge tag 'v1.2.0rc2' into developAndrew Morgan2019-07-2411-32/+101
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bugfixes -------- - Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
| * \ \ \ \ \ \ Merge pull request #5743 from matrix-org/erikj/log_origin_receipts_mismatchErik Johnston2019-07-242-13/+23
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Log when we receive receipt from a different origin
| | * | | | | | | Update changelog.d/5743.bugfixErik Johnston2019-07-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | | | NewsfileErik Johnston2019-07-231-0/+1
| | | | | | | | |
| | * | | | | | | Log when we receive receipt from a different originErik Johnston2019-07-231-13/+22
| | | | | | | | |
| * | | | | | | | Stop trying to fetch events with event_id=None. (#5753)Richard van der Hoff2019-07-244-11/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `None` is not a valid event id, so queuing up a database fetch for it seems like a silly thing to do. I considered making `get_event` return `None` if `event_id is None`, but then its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
| * | | | | | | | Add a prometheus metric for active cache lookups. (#5750)Richard van der Hoff2019-07-243-2/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add a prometheus metric for active cache lookups. * changelog
| * | | | | | | | Make Jaeger fully configurable (#5694)Jorik Schellekens2019-07-234-4/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Allow Jaeger to be configured * Update sample config
| * | | | | | | | Replace returnValue with return (#5736)Amber Brown2019-07-23177-1514/+1360
| | |_|_|_|_|_|/ | |/| | | | | |
| * | | | | | | Opentracing Utils (#5722)Jorik Schellekens2019-07-233-101/+357
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add decerators for tracing functions * Use the new clean contexts * Context and edu utils * Move opentracing setters * Move whitelisting * Sectioning comments * Better args wrapper * Docstrings Co-Authored-By: Erik Johnston <erik@matrix.org> * Remove unused methods. * Don't use global * One tracing decorator to rule them all.
| * | | | | | | Merge pull request #5732 from matrix-org/erikj/sdnotifyErik Johnston2019-07-2317-14/+67
| |\ \ \ \ \ \ \ | | |/ / / / / / | |/| | | | | | Add process hooks to tell systemd our state.
| | * | | | | | Also update systemd-with-workers contrib examplesErik Johnston2019-07-232-2/+4
| | | | | | | |
| | * | | | | | Update example systemd service fileErik Johnston2019-07-221-1/+3
| | | | | | | |
| | * | | | | | Call startup commands as system triggers.Erik Johnston2019-07-2211-11/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This helps ensures that we only consider ourselves "up" once all the startup functions have completed.
| | * | | | | | NewsfileErik Johnston2019-07-221-0/+1
| | | | | | | |
| | * | | | | | Add process hooks to tell systemd our state.Erik Johnston2019-07-222-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #5676.
| * | | | | | | Merge pull request #5740 from matrix-org/erikj/worker_flakey_testsErik Johnston2019-07-232-1/+8
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Mark flakey tests as blacklisted for worker mode
| | * | | | | | | NewsfileErik Johnston2019-07-231-0/+1
| | | | | | | | |
| | * | | | | | | Mark flakey tests as blacklisted for worker modeErik Johnston2019-07-231-1/+7
| |/ / / / / / /
| * | | | | | | Merge pull request #5738 from matrix-org/erikj/faster_updateErik Johnston2019-07-232-18/+31
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Speed up current state background update.
| | * | | | | | | NewsfileErik Johnston2019-07-231-0/+1
| | | | | | | | |
| | * | | | | | | Speed up current state background update.Erik Johnston2019-07-231-18/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Turns out that storing huge JSON arrays in the progress JSON isn't something that postgres particularly likes.
| * | | | | | | | Merge pull request #5733 from matrix-org/erikj/exlude_sytest_blacklistErik Johnston2019-07-232-1/+2
| |\ \ \ \ \ \ \ \ | | |/ / / / / / / | |/| | | | | | | Don't package sytest-blacklist file.
| | * | | | | | | NewsfileErik Johnston2019-07-221-0/+1
| | | | | | | | |
| | * | | | | | | Don't package sytest-blacklist file.Erik Johnston2019-07-221-1/+1
| |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | I don't think its useful, and I don't even know where it would end up.
| * | | | | | | Fix logging in workers (#5729)Amber Brown2019-07-224-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | This also adds a worker blacklist.
| * | | | | | | Merge pull request #5730 from matrix-org/erikj/cache_versionsErik Johnston2019-07-222-2/+22
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Cache get_version_string.
| | * | | | | | | Update changelog.d/5730.miscErik Johnston2019-07-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | | | NewsfileErik Johnston2019-07-221-0/+1
| | | | | | | | |
| | * | | | | | | Cache get_version_string.Erik Johnston2019-07-221-2/+21
| | |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The version of a module isn't going to change over the lifetime of the process (assuming no funky hot reloading is going on, which it isn't), so let's just cache the result to avoid spawning lots of git subprocesses. Fixes #5672.
| * | | | | | | Merge pull request #5731 from matrix-org/jaywink/admin-user-list-user-typeJason Robinson2019-07-222-3/+4
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Add `user_type` to returned fields in admin API user list endpoints
| | * | | | | | | Add `user_type` to returned fields in admin API user list endpointsJason Robinson2019-07-222-3/+4
| | |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mostly user type will be empty (normal user) but there is also the "support" user type. Signed-off-by: Jason Robinson <jasonr@matrix.org>
| * | | | | | | Fix stack overflow in Keyring (#5724)Richard van der Hoff2019-07-223-80/+42
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactor Keyring._start_key_lookups There's an awful lot of deferreds and dictionaries flying around here. The whole thing can be made much simpler and achieve the same effect. * Add a delay to key lookup lock release to fix stack overflow A tactical call_later here should fix #5723 * changelog
| | * | | | | | | changelogRichard van der Hoff2019-07-191-0/+1
| | | | | | | | |
| | * | | | | | | Add a delay to key lookup lock release to fix stack overflowRichard van der Hoff2019-07-191-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A tactical call_later here should fix #5723
| | * | | | | | | Refactor Keyring._start_key_lookupsRichard van der Hoff2019-07-192-80/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's an awful lot of deferreds and dictionaries flying around here. The whole thing can be made much simpler and achieve the same effect.
| * | | | | | | | Merge tag 'v1.2.0rc1' into developAndrew Morgan2019-07-2254-56/+82
| |\ \ \ \ \ \ \ \ | | |_|/ / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | v1.2.0rc1 Features -------- - Add support for opentracing. ([\#5544](https://github.com/matrix-org/synapse/issues/5544), [\#5712](https://github.com/matrix-org/synapse/issues/5712)) - Add ability to pull all locally stored events out of synapse that a particular user can see. ([\#5589](https://github.com/matrix-org/synapse/issues/5589)) - Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. ([\#5597](https://github.com/matrix-org/synapse/issues/5597)) - Add `sender` and `origin_server_ts` fields to `m.replace`. ([\#5613](https://github.com/matrix-org/synapse/issues/5613)) - Add default push rule to ignore reactions. ([\#5623](https://github.com/matrix-org/synapse/issues/5623)) - Include the original event when asking for its relations. ([\#5626](https://github.com/matrix-org/synapse/issues/5626)) - Implement `session_lifetime` configuration option, after which access tokens will expire. ([\#5660](https://github.com/matrix-org/synapse/issues/5660)) - Return "This account has been deactivated" when a deactivated user tries to login. ([\#5674](https://github.com/matrix-org/synapse/issues/5674)) - Enable aggregations support by default ([\#5714](https://github.com/matrix-org/synapse/issues/5714)) Bugfixes -------- - Fix 'utime went backwards' errors on daemonization. ([\#5609](https://github.com/matrix-org/synapse/issues/5609)) - Various minor fixes to the federation request rate limiter. ([\#5621](https://github.com/matrix-org/synapse/issues/5621)) - Forbid viewing relations on an event once it has been redacted. ([\#5629](https://github.com/matrix-org/synapse/issues/5629)) - Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format. ([\#5638](https://github.com/matrix-org/synapse/issues/5638)) - Fix newly-registered users not being able to lookup their own profile without joining a room. ([\#5644](https://github.com/matrix-org/synapse/issues/5644)) - Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`. ([\#5654](https://github.com/matrix-org/synapse/issues/5654)) - Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated. ([\#5658](https://github.com/matrix-org/synapse/issues/5658)) - Fix some problems with authenticating redactions in recent room versions. ([\#5699](https://github.com/matrix-org/synapse/issues/5699), [\#5700](https://github.com/matrix-org/synapse/issues/5700), [\#5707](https://github.com/matrix-org/synapse/issues/5707)) - Ignore redactions of m.room.create events. ([\#5701](https://github.com/matrix-org/synapse/issues/5701)) Updates to the Docker image --------------------------- - Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). ([\#5619](https://github.com/matrix-org/synapse/issues/5619)) - Add missing space in default logging file format generated by the Docker image. ([\#5620](https://github.com/matrix-org/synapse/issues/5620)) Improved Documentation ---------------------- - Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! ([\#5397](https://github.com/matrix-org/synapse/issues/5397)) - --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. ([\#5651](https://github.com/matrix-org/synapse/issues/5651)) - Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! ([\#5661](https://github.com/matrix-org/synapse/issues/5661)) - Minor tweaks to postgres documentation. ([\#5675](https://github.com/matrix-org/synapse/issues/5675)) Deprecations and Removals ------------------------- - Remove support for the `invite_3pid_guest` configuration setting. ([\#5625](https://github.com/matrix-org/synapse/issues/5625)) Internal Changes ---------------- - Move logging code out of `synapse.util` and into `synapse.logging`. ([\#5606](https://github.com/matrix-org/synapse/issues/5606), [\#5617](https://github.com/matrix-org/synapse/issues/5617)) - Add a blacklist file to the repo to blacklist certain sytests from failing CI. ([\#5611](https://github.com/matrix-org/synapse/issues/5611)) - Make runtime errors surrounding password reset emails much clearer. ([\#5616](https://github.com/matrix-org/synapse/issues/5616)) - Remove dead code for persiting outgoing federation transactions. ([\#5622](https://github.com/matrix-org/synapse/issues/5622)) - Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI. ([\#5627](https://github.com/matrix-org/synapse/issues/5627)) - Move RegistrationHandler.get_or_create_user to test code. ([\#5628](https://github.com/matrix-org/synapse/issues/5628)) - Add some more common python virtual-environment paths to the black exclusion list. ([\#5630](https://github.com/matrix-org/synapse/issues/5630)) - Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details. ([\#5636](https://github.com/matrix-org/synapse/issues/5636)) - Unblacklist some user_directory sytests. ([\#5637](https://github.com/matrix-org/synapse/issues/5637)) - Factor out some redundant code in the login implementation. ([\#5639](https://github.com/matrix-org/synapse/issues/5639)) - Update ModuleApi to avoid register(generate_token=True). ([\#5640](https://github.com/matrix-org/synapse/issues/5640)) - Remove access-token support from `RegistrationHandler.register`, and rename it. ([\#5641](https://github.com/matrix-org/synapse/issues/5641)) - Remove access-token support from `RegistrationStore.register`, and rename it. ([\#5642](https://github.com/matrix-org/synapse/issues/5642)) - Improve logging for auto-join when a new user is created. ([\#5643](https://github.com/matrix-org/synapse/issues/5643)) - Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. ([\#5645](https://github.com/matrix-org/synapse/issues/5645)) - Fix a small typo in a code comment. ([\#5655](https://github.com/matrix-org/synapse/issues/5655)) - Clean up exception handling around client access tokens. ([\#5656](https://github.com/matrix-org/synapse/issues/5656)) - Add a mechanism for per-test homeserver configuration in the unit tests. ([\#5657](https://github.com/matrix-org/synapse/issues/5657)) - Inline issue_access_token. ([\#5659](https://github.com/matrix-org/synapse/issues/5659)) - Update the sytest BuildKite configuration to checkout Synapse in `/src`. ([\#5664](https://github.com/matrix-org/synapse/issues/5664)) - Add a `docker` type to the towncrier configuration. ([\#5673](https://github.com/matrix-org/synapse/issues/5673)) - Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces. ([\#5689](https://github.com/matrix-org/synapse/issues/5689)) - Documentation for opentracing. ([\#5703](https://github.com/matrix-org/synapse/issues/5703))
| * | | | | | | | Demo uses deprecated cli option (#5725)Jorik Schellekens2019-07-222-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove deprecated 'verbose' cli arg * Create 5725.bugfix
| * | | | | | | | Merge branch 'release-v1.2.0' into developJorik Schellekens2019-07-225-24/+230
| |\ \ \ \ \ \ \ \
| * | | | | | | | | Revert "Remove deprecated 'verbose' cli arg"Jorik Schellekens2019-07-191-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit dc7cf81267e464858c74e6215184de0c634e2b26.
| * | | | | | | | | Remove deprecated 'verbose' cli argJorik Schellekens2019-07-191-4/+3
| | | | | | | | | |
| * | | | | | | | | Merge pull request #5720 from matrix-org/erikj/transactions_upsertErik Johnston2019-07-192-0/+21
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Use upsert when updating destination retry interval
| | * | | | | | | | | NewsfileErik Johnston2019-07-191-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Use upsert when updating destination retry intervalErik Johnston2019-07-191-0/+20
| | | |_|/ / / / / / | | |/| | | | | | |
| * | | | | | | | | Merge pull request #5713 from matrix-org/erikj/use_cache_for_filtered_stateErik Johnston2019-07-192-2/+7
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Delegate to cached version when using get_filtered_current_state_ids
| | * | | | | | | | | NewsfileErik Johnston2019-07-181-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Delegate to cached version when using get_filtered_current_state_idsErik Johnston2019-07-181-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the case where it gets called with `StateFilter.all()`
| * | | | | | | | | | Merge pull request #5706 from matrix-org/erikj/add_memberships_to_current_stateErik Johnston2019-07-198-41/+195
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Add membership column to current_state_events table
| | * | | | | | | | | | LoggingTransaction accepts None for callback lists.Erik Johnston2019-07-193-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Its a bit disingenuousto give LoggingTransaction lists to append callbacks to if we're not going to run the callbacks.
| | * | | | | | | | | | Update comment for new columnErik Johnston2019-07-191-0/+3
| | | | | | | | | | | |
| | * | | | | | | | | | NewsfileErik Johnston2019-07-181-0/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Use the current_state_events.membership columnErik Johnston2019-07-181-17/+37
| | | | | | | | | | | |
| | * | | | | | | | | | Track if current_state_events.membership is up to dateErik Johnston2019-07-181-0/+45
| | | | | | | | | | | |
| | * | | | | | | | | | Add background update for current_state_events.membership columnErik Johnston2019-07-182-0/+54
| | | | | | | | | | | |
| | * | | | | | | | | | Add membership column to current_state_events table.Erik Johnston2019-07-185-20/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It turns out that doing a join is surprisingly expensive for the DB to do when room_membership table is larger than the disk cache.
| * | | | | | | | | | | Don't accept opentracing data from clients. (#5715)Jorik Schellekens2019-07-192-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Don't accept opentracing data from clients. * newsfile
| * | | | | | | | | | | Add 'rel' attribute to default welcome page. (#5695)Neil Johnson2019-07-192-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | add rel attribute as a precaution against reverse tabnabbing in future
| * | | | | | | | | | | Update the coding style doc (#5719)Richard van der Hoff2019-07-192-30/+95
| | |_|/ / / / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A few fixes and removal of duplicated stuff, but mostly a bunch of the words on the config file.
| * | | | | | | | | | Speed up the PostgreSQL unit tests (#5717)Amber Brown2019-07-192-3/+10
| | | | | | | | | | |
| * | | | | | | | | | Remove non-dedicated logging options and command line arguments (#5678)Amber Brown2019-07-193-75/+13
| | | | | | | | | | |
* | | | | | | | | | | make user creation steps clearerNeil Johnson2019-08-081-5/+4
| |_|_|_|_|_|_|_|_|/ |/| | | | | | | | |
* | | | | | | | | | yet more changelog attribution fixes v1.2.1Richard van der Hoff2019-07-261-2/+2
| | | | | | | | | |
* | | | | | | | | | correct attributions in changelog github/release-v1.2.1 release-v1.2.1Richard van der Hoff2019-07-261-1/+4
| | | | | | | | | |
* | | | | | | | | | correct attributions in changelogRichard van der Hoff2019-07-261-1/+2
| | | | | | | | | |
* | | | | | | | | | 1.2.1Richard van der Hoff2019-07-265-4/+30
| | | | | | | | | |
* | | | | | | | | | Merge branch 'erikj/log_leave_origin_mismatch' into release-v1.2.1Richard van der Hoff2019-07-263-4/+38
|\ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|_|_|/ | |/| | | | | | | |
| * | | | | | | | | Log when we receive a /make_* request from a different originRichard van der Hoff2019-07-263-4/+38
| | |_|_|_|_|/ / / | |/| | | | | | |
* | | | | | | | | Log when we receive receipt from a different originErik Johnston2019-07-261-13/+22
| | | | | | | | |
* | | | | | | | | Fix DoS when there is a cycle in redaction eventsRichard van der Hoff2019-07-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure that synapse doesn't explode when a redaction redacts itself, or there is a larger cycle.
* | | | | | | | | Merge branch 'rav/redactions/cross_room_id' into release-v1.2.1Richard van der Hoff2019-07-252-0/+28
|\ \ \ \ \ \ \ \ \ | |_|_|_|_|_|_|/ / |/| | | | | | | / | | |_|_|_|_|_|/ | |/| | | | | |
| * | | | | | | log when a redaction attempts to redact an event in a different roomRichard van der Hoff2019-07-252-0/+28
|/ / / / / / /
* / / / / / / 1.2.0 v1.2.0 github/release-v1.2.0 release-v1.2.0Andrew Morgan2019-07-253-3/+12
|/ / / / / /
* | | | | | 1.2.0rc2 v1.2.0rc2Andrew Morgan2019-07-243-2/+10
| | | | | |
* | | | | | Fix servlet metric names (#5734)Jorik Schellekens2019-07-2410-31/+92
| |_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths
* | | | | Make changelog slightly more readable v1.2.0rc1Andrew Morgan2019-07-221-1/+1
| | | | |
* | | | | 1.2.0rc1Andrew Morgan2019-07-2254-56/+82
| |_|_|/ |/| | |
* | | | Opentracing Documentation (#5703)Jorik Schellekens2019-07-225-24/+230
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Opentracing survival guide * Update decorator names in doc * Doc cleanup These are all alterations as a result of comments in #5703, it includes mostly typos and clarifications. The most interesting changes are: - Split developer and user docs into two sections - Add a high level description of OpenTracing * newsfile * Move contributer specific info to docstring. * Sample config. * Trailing whitespace. * Update 5703.misc * Apply suggestions from code review Mostly just rewording parts of the docs for clarity. Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | | towncrierNeil Johnson2019-07-181-0/+1
| | |
* | | enable aggregations support by defaultNeil Johnson2019-07-181-1/+1
| |/ |/|
* | Clean up opentracing configuration options (#5712)Richard van der Hoff2019-07-187-63/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clean up config settings and dead code. This is mostly about cleaning up the config format, to bring it into line with our conventions. In particular: * There should be a blank line after `## Section ##' headings * There should be a blank line between each config setting * There should be a `#`-only line between a comment and the setting it describes * We don't really do the `# #` style commenting-out of whole sections if we can help it * rename `tracer_enabled` to `enabled` While we're here, do more config parsing upfront, which makes it easier to use later on. Also removes redundant code from LogContextScopeManager. Also changes the changelog fragment to a `feature` - it's exciting!
* | Support Prometheus_client 0.4.0+ (#5636)Amber Brown2019-07-1820-50/+399
| |
* | Remove the ability to query relations when the original event was redacted. ↵Andrew Morgan2019-07-185-39/+180
| | | | | | | | | | | | | | (#5629) Fixes #5594 Forbid viewing relations on an event once it has been redacted.
* | Convert synapse.federation.transport.server to async (#5689)Richard van der Hoff2019-07-182-242/+189
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Convert BaseFederationServlet._wrap to async Empirically, this fixes some lost stacktraces. It should be safe because the wrapped function is called from JsonResource._async_render, which is already async. * Convert the rest of synapse.federation.transport.server to async We may as well do the whole file while we're here. * changelog * flake8
* | Ignore redactions of m.room.create events (#5701)Richard van der Hoff2019-07-175-24/+57
| |
* | Improve `Depends` specs in debian package. (#5675)Richard van der Hoff2019-07-176-3/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is basically a contrived way of adding a `Recommends` on `libpq5`, to fix #5653. The way this is supposed to happen in debhelper is to run `dh_shlibdeps`, which in turn runs `dpkg-shlibdeps`, which spits things out into `debian/<package>.substvars` whence they can later be included by `control`. Previously, we had disabled `dh_shlibdeps`, mostly because `dpkg-shlibdeps` gets confused about PIL's interdependent objects, but that's not really the right thing to do and there is another way to work around that. Since we don't always use postgres, we don't necessarily want a hard Depends on libpq5, so I've actually ended up adding an explicit invocation of `dpkg-shlibdeps` for `psycopg2`. I've also updated the build-depends list for the package, which was missing a couple of entries.
* | More refactoring in `get_events_as_list` (#5707)Richard van der Hoff2019-07-174-27/+198
| | | | | | | | | | | | | | | | We can now use `_get_events_from_cache_or_db` rather than going right back to the database, which means that (a) we can benefit from caching, and (b) it opens the way forward to more extensive checks on the original event. We now always require the original event to exist before we will serve up a redaction.
* | Fix redaction authentication (#5700)Richard van der Hoff2019-07-172-85/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensures that redactions are correctly authenticated for recent room versions. There are a few things going on here: * `_fetch_event_rows` is updated to return a dict rather than a list of rows. * Rather than returning multiple copies of an event which was redacted multiple times, it returns the redactions as a list within the dict. * It also returns the actual rejection reason, rather than merely the fact that it was rejected, so that we don't have to query the table again in `_get_event_from_row`. * The redaction handling is factored out of `_get_event_from_row`, and now checks if any of the redactions are valid.
* | Refactor `get_events_as_list` (#5699)Richard van der Hoff2019-07-172-48/+75
|/ | | | | | | | | | | A couple of changes here: * get rid of a redundant `allow_rejected` condition - we should already have filtered out any rejected events before we get to that point in the code, and the redundancy is confusing. Instead, let's stick in an assertion just to make double-sure we aren't leaking rejected events by mistake. * factor out a `_get_events_from_cache_or_db` method, which is going to be important for a forthcoming fix to redactions.
* Merge pull request #5597 from matrix-org/erikj/admin_api_cmdErik Johnston2019-07-168-13/+357
|\ | | | | Create basic admin command app
| * Remove pointless descriptionErik Johnston2019-07-161-1/+0
| |
| * Clean up arg name and remove lying commentErik Johnston2019-07-161-7/+5
| |
| * Fix typo in synapse/app/admin_cmd.pyErik Johnston2019-07-161-1/+1
| | | | | | Co-Authored-By: Aaron Raimist <aaron@raim.ist>
| * Fix invoking add_argument from homeserver.pyErik Johnston2019-07-151-1/+1
| |
| * s/exfiltrate_user_data/export_user_data/Erik Johnston2019-07-151-1/+1
| |
| * Add FileExfiltrationWriterErik Johnston2019-07-151-1/+69
| |
| * Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_api_cmdErik Johnston2019-07-15293-2140/+3248
| |\ | |/ |/|
* | Return a different error from Invalid Password when a user is deactivated ↵Andrew Morgan2019-07-153-0/+26
| | | | | | | | | | (#5674) Return `This account has been deactivated` instead of `Invalid password` when a user is deactivated.
* | Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_dataErik Johnston2019-07-156-7/+423
|\ \ | | | | | | Add basic function to get all data for a user out of synapse
| * | Fixup docstringsErik Johnston2019-07-121-7/+5
| | |
| * | pep8Erik Johnston2019-07-051-4/+0
| | |
| * | Remove FileExfiltrationWriterErik Johnston2019-07-051-63/+0
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-07-05217-1407/+1732
| |\ \ | | | | | | | | | | | | erikj/admin_exfiltrate_data
| * | | Assume key existence. Update docstringsErik Johnston2019-07-052-9/+11
| | | |
| * | | Fixup from review comments.Erik Johnston2019-07-042-22/+27
| | | |
| * | | Apply comment fixups from code reviewErik Johnston2019-07-031-6/+6
| | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | | | Add a `docker` type to the towncrier configuration (#5673)Richard van der Hoff2019-07-127-14/+27
| | | | | | | | | | | | ... and certain other changelog-related fixes
* | | | Implement access token expiry (#5660)Richard van der Hoff2019-07-1214-31/+253
| | | | | | | | | | | | | | | | Record how long an access token is valid for, and raise a soft-logout once it expires.
* | | | fix typo: backgroud -> backgroundAndrew Morgan2019-07-122-5/+5
| | | |
* | | | fix changelog nameRichard van der Hoff2019-07-121-0/+0
| | | |
* | | | Update reverse_proxy.rst (#5397)Ulrik Günther2019-07-122-0/+3
| | | | | | | | | | | | | | | | | | | | Updates reverse_proxy.rst with information about nginx' URI normalisation.
* | | | Add missing space in default logging file format generated by the Docker ↵Slavi Pantaleev2019-07-122-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | image (#5620) This adds a missing space, without which log lines appear uglier. Signed-off-by: Slavi Pantaleev <slavi@devture.com>
* | | | Upgrade Alpine Linux used in the Docker image (3.8 -> 3.10) (#5619)Slavi Pantaleev2019-07-122-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Alpine Linux 3.8 is still supported, but it seems like it's quite outdated now. While Python should be the same on both, all other libraries, etc., are much newer in Alpine 3.9 and 3.10. Signed-off-by: Slavi Pantaleev <slavi@devture.com>
* | | | Add a mechanism for per-test configs (#5657)Richard van der Hoff2019-07-122-1/+55
| | | | | | | | | | | | | | | | | | | | | | | | It's useful to be able to tweak the homeserver config to be used for each test. This PR adds a mechanism to do so.
* | | | Use /src for checking out synapse during sytests (#5664)Amber Brown2019-07-112-0/+4
| | | |
* | | | Improved docs on setting up Postgresql (#5661)Lrizika2019-07-112-4/+16
| | | | | | | | | | | | | | | | | | | | Added that synapse_user needs a database to access before it can auth Noted you'll need to enable password auth, linked to pg_hba.conf docs
* | | | small typo fix (#5655)Andrew Morgan2019-07-112-1/+2
| | | |
* | | | Clean up exception handling for access_tokens (#5656)Richard van der Hoff2019-07-116-100/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First of all, let's get rid of `TOKEN_NOT_FOUND_HTTP_STATUS`. It was a hack we did at one point when it was possible to return either a 403 or a 401 if the creds were missing. We always return a 401 in these cases now (thankfully), so it's not needed. Let's also stop abusing `AuthError` for these cases. Honestly they have nothing that relates them to the other places that `AuthError` is used, other than the fact that they are loosely under the 'Auth' banner. It makes no sense for them to share exception classes. Instead, let's add a couple of new exception classes: `InvalidClientTokenError` and `MissingClientTokenError`, for the `M_UNKNOWN_TOKEN` and `M_MISSING_TOKEN` cases respectively - and an `InvalidClientCredentialsError` base class for the two of them.
* | | | Add basic opentracing support (#5544)Jorik Schellekens2019-07-1112-12/+633
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter
* | | | Inline issue_access_token (#5659)Richard van der Hoff2019-07-113-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this is only used in one place, so it's clearer if we inline it and reduce the API surface. Also, fixes a buglet where we would create an access token even if we were about to block the user (we would never return the AT, so the user could never use it, but it was still created and added to the db.)
* | | | Merge pull request #5658 from matrix-org/babolivier/is-jsonBrendan Abolivier2019-07-102-1/+2
|\ \ \ \ | | | | | | | | | | Send 3PID bind requests as JSON data
| * | | | TypoBrendan Abolivier2019-07-101-1/+1
| | | | |