summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'shhs' of ssh://github.com/matrix-org/synapse into shhs shhs-v1.2.1.1 github/shhs shhsAmber H. Brown2019-07-268-65/+124
|\
| * Merge tag 'v1.2.1' into shhs shhs-v1.2.1Amber Brown2019-07-267-19/+124
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.2.1 (2019-07-26) ========================== Security update --------------- This release includes *four* security fixes: - Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. ([\#5767](https://github.com/matrix-org/synapse/issues/5767)) - Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to `@lrizika:matrix.org` for identifying and responsibly disclosing this issue. ([0f2ecb961](https://github.com/matrix-org/synapse/commit/0f2ecb961)) - Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to @Dylanger for identifying and responsibly disclosing this issue. ([\#5744](https://github.com/matrix-org/synapse/issues/5744)) - Fix a vulnerability where a federated server could spoof read-receipts from users on other servers. Thanks to @Dylanger for identifying this issue too. ([\#5743](https://github.com/matrix-org/synapse/issues/5743)) Additionally, the following fix was in Synapse **1.2.0**, but was not correctly identified during the original release: - It was possible for a room moderator to send a redaction for an `m.room.create` event, which would downgrade the room to version 1. Thanks to `/dev/ponies` for identifying and responsibly disclosing this issue! ([\#5701](https://github.com/matrix-org/synapse/issues/5701))
| | * correct attributions in changelog github/release-v1.2.1 release-v1.2.1Richard van der Hoff2019-07-261-1/+4
| | |
| | * correct attributions in changelogRichard van der Hoff2019-07-261-1/+2
| | |
| | * 1.2.1Richard van der Hoff2019-07-265-4/+30
| | |
| | * Merge branch 'erikj/log_leave_origin_mismatch' into release-v1.2.1Richard van der Hoff2019-07-263-4/+38
| | |\
| | | * Log when we receive a /make_* request from a different originRichard van der Hoff2019-07-263-4/+38
| | | |
| | * | Log when we receive receipt from a different originErik Johnston2019-07-261-13/+22
| | | |
| | * | Fix DoS when there is a cycle in redaction eventsRichard van der Hoff2019-07-261-0/+4
| | | | | | | | | | | | | | | | | | | | Make sure that synapse doesn't explode when a redaction redacts itself, or there is a larger cycle.
| | * | Merge branch 'rav/redactions/cross_room_id' into release-v1.2.1Richard van der Hoff2019-07-252-0/+28
| | |\ \
| | | * | log when a redaction attempts to redact an event in a different roomRichard van der Hoff2019-07-252-0/+28
| | |/ /
| * | | don't have a circleci configAmber Brown2019-07-261-46/+0
| | | |
* | | | fix merging forward shhs-v1.2.0.2Amber H. Brown2019-07-261-1/+1
| | | |
* | | | fix this shhs-v1.2.0.1Amber H. Brown2019-07-261-1/+1
|/ / /
* | | Merge tag 'v1.2.0' into shhs shhs-v1.2Amber H. Brown2019-07-26111-1284/+4042
|\| | | | | | | | | | | No changes since v1.2.0rc2.
| * | 1.2.0 v1.2.0 github/release-v1.2.0 release-v1.2.0Andrew Morgan2019-07-253-3/+12
| |/
| * 1.2.0rc2 v1.2.0rc2Andrew Morgan2019-07-243-2/+10
| |
| * Fix servlet metric names (#5734)Jorik Schellekens2019-07-2410-31/+92
| | | | | | | | | | | | | | | | | | | | * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths
| * Make changelog slightly more readable v1.2.0rc1Andrew Morgan2019-07-221-1/+1
| |
| * 1.2.0rc1Andrew Morgan2019-07-2254-56/+82
| |
| * Opentracing Documentation (#5703)Jorik Schellekens2019-07-225-24/+230
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Opentracing survival guide * Update decorator names in doc * Doc cleanup These are all alterations as a result of comments in #5703, it includes mostly typos and clarifications. The most interesting changes are: - Split developer and user docs into two sections - Add a high level description of OpenTracing * newsfile * Move contributer specific info to docstring. * Sample config. * Trailing whitespace. * Update 5703.misc * Apply suggestions from code review Mostly just rewording parts of the docs for clarity. Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * towncrierNeil Johnson2019-07-181-0/+1
| |
| * enable aggregations support by defaultNeil Johnson2019-07-181-1/+1
| |
| * Clean up opentracing configuration options (#5712)Richard van der Hoff2019-07-187-63/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clean up config settings and dead code. This is mostly about cleaning up the config format, to bring it into line with our conventions. In particular: * There should be a blank line after `## Section ##' headings * There should be a blank line between each config setting * There should be a `#`-only line between a comment and the setting it describes * We don't really do the `# #` style commenting-out of whole sections if we can help it * rename `tracer_enabled` to `enabled` While we're here, do more config parsing upfront, which makes it easier to use later on. Also removes redundant code from LogContextScopeManager. Also changes the changelog fragment to a `feature` - it's exciting!
| * Support Prometheus_client 0.4.0+ (#5636)Amber Brown2019-07-1820-50/+399
| |
| * Remove the ability to query relations when the original event was redacted. ↵Andrew Morgan2019-07-185-39/+180
| | | | | | | | | | | | | | (#5629) Fixes #5594 Forbid viewing relations on an event once it has been redacted.
| * Convert synapse.federation.transport.server to async (#5689)Richard van der Hoff2019-07-182-242/+189
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Convert BaseFederationServlet._wrap to async Empirically, this fixes some lost stacktraces. It should be safe because the wrapped function is called from JsonResource._async_render, which is already async. * Convert the rest of synapse.federation.transport.server to async We may as well do the whole file while we're here. * changelog * flake8
| * Ignore redactions of m.room.create events (#5701)Richard van der Hoff2019-07-175-24/+57
| |
| * Improve `Depends` specs in debian package. (#5675)Richard van der Hoff2019-07-176-3/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is basically a contrived way of adding a `Recommends` on `libpq5`, to fix #5653. The way this is supposed to happen in debhelper is to run `dh_shlibdeps`, which in turn runs `dpkg-shlibdeps`, which spits things out into `debian/<package>.substvars` whence they can later be included by `control`. Previously, we had disabled `dh_shlibdeps`, mostly because `dpkg-shlibdeps` gets confused about PIL's interdependent objects, but that's not really the right thing to do and there is another way to work around that. Since we don't always use postgres, we don't necessarily want a hard Depends on libpq5, so I've actually ended up adding an explicit invocation of `dpkg-shlibdeps` for `psycopg2`. I've also updated the build-depends list for the package, which was missing a couple of entries.
| * More refactoring in `get_events_as_list` (#5707)Richard van der Hoff2019-07-174-27/+198
| | | | | | | | | | | | | | | | We can now use `_get_events_from_cache_or_db` rather than going right back to the database, which means that (a) we can benefit from caching, and (b) it opens the way forward to more extensive checks on the original event. We now always require the original event to exist before we will serve up a redaction.
| * Fix redaction authentication (#5700)Richard van der Hoff2019-07-172-85/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensures that redactions are correctly authenticated for recent room versions. There are a few things going on here: * `_fetch_event_rows` is updated to return a dict rather than a list of rows. * Rather than returning multiple copies of an event which was redacted multiple times, it returns the redactions as a list within the dict. * It also returns the actual rejection reason, rather than merely the fact that it was rejected, so that we don't have to query the table again in `_get_event_from_row`. * The redaction handling is factored out of `_get_event_from_row`, and now checks if any of the redactions are valid.
| * Refactor `get_events_as_list` (#5699)Richard van der Hoff2019-07-172-48/+75
| | | | | | | | | | | | | | | | | | | | | | A couple of changes here: * get rid of a redundant `allow_rejected` condition - we should already have filtered out any rejected events before we get to that point in the code, and the redundancy is confusing. Instead, let's stick in an assertion just to make double-sure we aren't leaking rejected events by mistake. * factor out a `_get_events_from_cache_or_db` method, which is going to be important for a forthcoming fix to redactions.
| * Merge pull request #5597 from matrix-org/erikj/admin_api_cmdErik Johnston2019-07-168-13/+357
| |\ | | | | | | Create basic admin command app
| | * Remove pointless descriptionErik Johnston2019-07-161-1/+0
| | |
| | * Clean up arg name and remove lying commentErik Johnston2019-07-161-7/+5
| | |
| | * Fix typo in synapse/app/admin_cmd.pyErik Johnston2019-07-161-1/+1
| | | | | | | | | Co-Authored-By: Aaron Raimist <aaron@raim.ist>
| | * Fix invoking add_argument from homeserver.pyErik Johnston2019-07-151-1/+1
| | |
| | * s/exfiltrate_user_data/export_user_data/Erik Johnston2019-07-151-1/+1
| | |
| | * Add FileExfiltrationWriterErik Johnston2019-07-151-1/+69
| | |
| | * Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_api_cmdErik Johnston2019-07-15293-2140/+3248
| | |\ | | |/ | |/|
| * | Return a different error from Invalid Password when a user is deactivated ↵Andrew Morgan2019-07-153-0/+26
| | | | | | | | | | | | | | | (#5674) Return `This account has been deactivated` instead of `Invalid password` when a user is deactivated.
| * | Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_dataErik Johnston2019-07-156-7/+423
| |\ \ | | | | | | | | Add basic function to get all data for a user out of synapse
| | * | Fixup docstringsErik Johnston2019-07-121-7/+5
| | | |
| | * | pep8Erik Johnston2019-07-051-4/+0
| | | |
| | * | Remove FileExfiltrationWriterErik Johnston2019-07-051-63/+0
| | | |
| | * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-07-05217-1407/+1732
| | |\ \ | | | | | | | | | | | | | | | erikj/admin_exfiltrate_data
| | * | | Assume key existence. Update docstringsErik Johnston2019-07-052-9/+11
| | | | |
| | * | | Fixup from review comments.Erik Johnston2019-07-042-22/+27
| | | | |
| | * | | Apply comment fixups from code reviewErik Johnston2019-07-031-6/+6
| | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * | | | Add a `docker` type to the towncrier configuration (#5673)Richard van der Hoff2019-07-127-14/+27
| | | | | | | | | | | | | | | ... and certain other changelog-related fixes
| * | | | Implement access token expiry (#5660)Richard van der Hoff2019-07-1214-31/+253
| | | | | | | | | | | | | | | | | | | | Record how long an access token is valid for, and raise a soft-logout once it expires.
| * | | | fix typo: backgroud -> backgroundAndrew Morgan2019-07-122-5/+5
| | | | |
| * | | | fix changelog nameRichard van der Hoff2019-07-121-0/+0
| | | | |
| * | | | Update reverse_proxy.rst (#5397)Ulrik Günther2019-07-122-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | Updates reverse_proxy.rst with information about nginx' URI normalisation.
| * | | | Add missing space in default logging file format generated by the Docker ↵Slavi Pantaleev2019-07-122-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | image (#5620) This adds a missing space, without which log lines appear uglier. Signed-off-by: Slavi Pantaleev <slavi@devture.com>
| * | | | Upgrade Alpine Linux used in the Docker image (3.8 -> 3.10) (#5619)Slavi Pantaleev2019-07-122-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Alpine Linux 3.8 is still supported, but it seems like it's quite outdated now. While Python should be the same on both, all other libraries, etc., are much newer in Alpine 3.9 and 3.10. Signed-off-by: Slavi Pantaleev <slavi@devture.com>
| * | | | Add a mechanism for per-test configs (#5657)Richard van der Hoff2019-07-122-1/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's useful to be able to tweak the homeserver config to be used for each test. This PR adds a mechanism to do so.
| * | | | Use /src for checking out synapse during sytests (#5664)Amber Brown2019-07-112-0/+4
| | | | |
| * | | | Improved docs on setting up Postgresql (#5661)Lrizika2019-07-112-4/+16
| | | | | | | | | | | | | | | | | | | | | | | | | Added that synapse_user needs a database to access before it can auth Noted you'll need to enable password auth, linked to pg_hba.conf docs
| * | | | small typo fix (#5655)Andrew Morgan2019-07-112-1/+2
| | | | |
| * | | | Clean up exception handling for access_tokens (#5656)Richard van der Hoff2019-07-116-100/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First of all, let's get rid of `TOKEN_NOT_FOUND_HTTP_STATUS`. It was a hack we did at one point when it was possible to return either a 403 or a 401 if the creds were missing. We always return a 401 in these cases now (thankfully), so it's not needed. Let's also stop abusing `AuthError` for these cases. Honestly they have nothing that relates them to the other places that `AuthError` is used, other than the fact that they are loosely under the 'Auth' banner. It makes no sense for them to share exception classes. Instead, let's add a couple of new exception classes: `InvalidClientTokenError` and `MissingClientTokenError`, for the `M_UNKNOWN_TOKEN` and `M_MISSING_TOKEN` cases respectively - and an `InvalidClientCredentialsError` base class for the two of them.
| * | | | Add basic opentracing support (#5544)Jorik Schellekens2019-07-1112-12/+633
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter
| * | | | Inline issue_access_token (#5659)Richard van der Hoff2019-07-113-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this is only used in one place, so it's clearer if we inline it and reduce the API surface. Also, fixes a buglet where we would create an access token even if we were about to block the user (we would never return the AT, so the user could never use it, but it was still created and added to the db.)
| * | | | Merge pull request #5658 from matrix-org/babolivier/is-jsonBrendan Abolivier2019-07-102-1/+2
| |\ \ \ \ | | | | | | | | | | | | Send 3PID bind requests as JSON data
| | * | | | TypoBrendan Abolivier2019-07-101-1/+1
| | | | | |
| | * | | | Rename changelog fileBrendan Abolivier2019-07-101-0/+0
| | | | | |
| | * | | | Send 3PID bind requests as JSON dataBrendan Abolivier2019-07-102-1/+2
| | | | | |
| * | | | | Remove access-token support from RegistrationStore.register (#5642)Richard van der Hoff2019-07-109-79/+30
| |/ / / / | | | | | | | | | | | | | | | | | | | | The 'token' param is no longer used anywhere except the tests, so let's kill that off too.
| * | | | Don't bundle aggregations when retrieving the original event (#5654)Andrew Morgan2019-07-102-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A fix for PR #5626, which returned the original event content as part of a call to /relations. Only problem was that we were attempting to aggregate the relations on top of it when we did so. We now set bundle_aggregations to False in the get_event call. We also do this when pulling the relation events as well, because edits of edits are not something we'd like to support here.
| * | | | Add a linting script (#5627)Andrew Morgan2019-07-102-0/+13
| | | | | | | | | | | | | | | Add a dev script to cover all the different linting steps.
| * | | | Correct pep517 flag in readme (#5651)Bruno Windels2019-07-102-1/+2
| | | | |
| * | | | Merge pull request #5638 from matrix-org/babolivier/invite-jsonBrendan Abolivier2019-07-092-4/+19
| |\ \ \ \ | | | | | | | | | | | | Use JSON when querying the IS's /store-invite endpoint
| | * \ \ \ Merge branch 'develop' into babolivier/invite-jsonBrendan Abolivier2019-07-08241-1918/+2403
| | |\ \ \ \
| | * | | | | ChangelogBrendan Abolivier2019-07-081-0/+1
| | | | | | |
| | * | | | | LintBrendan Abolivier2019-07-081-1/+2
| | | | | | |
| | * | | | | Use application/json when querying the IS's /store-invite endpointBrendan Abolivier2019-07-081-4/+17
| | | | | | |
| * | | | | | remove unused and unnecessary check for FederationDeniedError (#5645)Hubert Chathi2019-07-092-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FederationDeniedError is a subclass of SynapseError, which is a subclass of CodeMessageException, so if e is a FederationDeniedError, then this check for FederationDeniedError will never be reached since it will be caught by the check for CodeMessageException above. The check for CodeMessageException does almost the same thing as this check (since FederationDeniedError initialises with code=403 and msg="Federation denied with %s."), so may as well just keep allowing it to handle this case.
| * | | | | | Include the original event in /relations (#5626)Andrew Morgan2019-07-094-4/+12
| | | | | | | | | | | | | | | | | | | | | When asking for the relations of an event, include the original event in the response. This will mostly be used for efficiently showing edit history, but could be useful in other circumstances.
| * | | | | | Merge pull request #5644 from matrix-org/babolivier/profile-allow-selfBrendan Abolivier2019-07-093-0/+52
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | Allow newly-registered users to lookup their own profiles
| | * | | | | | LintBrendan Abolivier2019-07-081-2/+2
| | | | | | | |
| | * | | | | | Add test caseBrendan Abolivier2019-07-081-0/+47
| | | | | | | |
| | * | | | | | ChangelogBrendan Abolivier2019-07-081-0/+1
| | | | | | | |
| | * | | | | | Allow newly-registered users to lookup their own profilesBrendan Abolivier2019-07-081-0/+4
| | | |/ / / / | | |/| | | | | | | | | | | | | | | | | | When a user creates an account and the 'require_auth_for_profile_requests' config flag is set, and a client that performed the registration wants to lookup the newly-created profile, the request will be denied because the user doesn't share a room with themselves yet.
| * | | | | | Remove access-token support from RegistrationHandler.register (#5641)Richard van der Hoff2019-07-088-81/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Nothing uses this now, so we can remove the dead code, and clean up the API. Since we're changing the shape of the return value anyway, we take the opportunity to give the method a better name.
| * | | | | | Unblacklist some user_directory sytests (#5637)Richard van der Hoff2019-07-092-4/+1
| | | | | | | | | | | | | | | | | | | | | Fixes https://github.com/matrix-org/synapse/issues/2306
| * | | | | | Better logging for auto-join. (#5643)Richard van der Hoff2019-07-082-0/+8
| |/ / / / / | | | | | | | | | | | | It was pretty unclear what was going on, so I've added a couple of log lines.
| * | | | | Update ModuleApi to avoid register(generate_token=True) (#5640)Richard van der Hoff2019-07-082-8/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update ModuleApi to avoid register(generate_token=True) This is the only place this is still used, so I'm trying to kill it off. * changelog
| * | | | | Factor out some redundant code in the login impl (#5639)Richard van der Hoff2019-07-082-39/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Factor out some redundant code in the login impl Also fixes a redundant access_token which was generated during jwt login. * changelog
| * | | | | Move get_or_create_user to test code (#5628)Richard van der Hoff2019-07-083-60/+60
| | | | | | | | | | | | | | | | | | This is only used in tests, so...
| * | | | | Add a few more common environment directory names to black exclusion (#5630)Andrew Morgan2019-07-082-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add a few more common environment directory names to black exclusion * Add changelog
| * | | | | Add default push rule to ignore reactions (#5623)J. Ryan Stinnett2019-07-052-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a default push rule following the proposal in [MSC2153](https://github.com/matrix-org/matrix-doc/pull/2153). See also https://github.com/vector-im/riot-web/issues/10208 See also https://github.com/matrix-org/matrix-js-sdk/pull/976
| * | | | | Add origin_server_ts and sender fields to m.replace (#5613)Andrew Morgan2019-07-053-7/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Riot team would like some extra fields as part of m.replace, so here you go. Fixes: #5598
| * | | | | Remove support for invite_3pid_guest. (#5625)Richard van der Hoff2019-07-057-196/+3
| | |_|/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has never been documented, and I'm not sure it's ever been used outside sytest. It's quite a lot of poorly-maintained code, so I'd like to get rid of it. For now I haven't removed the database table; I suggest we leave that for a future clearout.
| * | | | remove dead transaction persist code (#5622)Richard van der Hoff2019-07-054-69/+1
| | | | | | | | | | | | | | | | | | | | this hasn't done anything for years
| * | | | Fixes to the federation rate limiter (#5621)Richard van der Hoff2019-07-056-16/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Put the default window_size back to 1000ms (broken by #5181) - Make the `rc_federation` config actually do something - fix an off-by-one error in the 'concurrent' limit - Avoid creating an unused `_PerHostRatelimiter` object for every single incoming request
| | | | * Use set_defaults(func=) styleErik Johnston2019-07-151-11/+8
| | | | |
| | | | * Move creation of ArgumentParser to callerErik Johnston2019-07-152-11/+8
| | | | |
| | | | * Fix up commentsErik Johnston2019-07-152-3/+3
| | | | |
| | | | * Change add_arguments to be a static methodErik Johnston2019-07-155-5/+39
| | | | |
| | | | * NewsfileErik Johnston2019-07-021-0/+1
| | | | |
| | | | * Add basic admin cmd appErik Johnston2019-07-023-6/+257
| | | |/
| | | * NewsfileErik Johnston2019-07-021-0/+1
| | | |
| | | * Add basic function to get all data for a user out of synapseErik Johnston2019-07-024-1/+478
| | | |
* | | | dockerfile updateAmber H. Brown2019-07-262-69/+3
| | | |
* | | | add a waitAmber H. Brown2019-07-181-0/+2
| | | |
* | | | fix shhs-v1.1.1.7-optAmber H. Brown2019-07-171-1/+1
| | | |
* | | | fix shhs-v1.1.1.6-optAmber H. Brown2019-07-171-1/+1
| | | |
* | | | fixAmber H. Brown2019-07-171-1/+1
| | | |
* | | | fixAmber H. Brown2019-07-171-2/+2
| | | |
* | | | fixAmber H. Brown2019-07-172-5/+4
| | | |
* | | | try nowAmber H. Brown2019-07-171-3/+3
| | | |
* | | | try nowAmber H. Brown2019-07-171-2/+2
| | | |
* | | | see if we can do a build!Amber H. Brown2019-07-172-1/+84
| | | |
* | | | Merge remote-tracking branch 'origin/develop' into shhs shhs-v1.1.1Amber H. Brown2019-07-05109-1126/+1109
|\| | |
| * | | Make errors about email password resets much clearer (#5616)Andrew Morgan2019-07-052-11/+9
| | | | | | | | | | | | The runtime errors that dealt with local email password resets talked about config options that users may not even have in their config file yet (if upgrading). Instead, the cryptic errors are now replaced with hopefully much more helpful ones.
| * | | Improve the backwards compatibility re-exports of synapse.logging.context ↵Amber Brown2019-07-054-5/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#5617) * Improve the backwards compatibility re-exports of synapse.logging.context. * reexport logformatter too
| * | | Add a sytest blacklist file (#5611)Andrew Morgan2019-07-055-148/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add a sytest blacklist file * Add changelog * Add blacklist to manifest
| * | | Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2019-07-043-6/+26
| |\ \ \
| * | | | Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-0498-233/+249
| | | | |
| * | | | Fix 'utime went backwards' errors on daemonization. (#5609)Richard van der Hoff2019-07-033-31/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix 'utime went backwards' errors on daemonization. Fixes #5608 * remove spurious debug
* | | | | lintingAmber H. Brown2019-07-051-1/+0
| | | | |
* | | | | lintingAmber H. Brown2019-07-052-26/+0
| | | | |
* | | | | lintingAmber H. Brown2019-07-056-27/+25
| | | | |
* | | | | Merge tag 'v1.1.0' into shhs shhs-v1.1Amber H. Brown2019-07-053-4/+20
|\ \ \ \ \ | | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.1.0 (2019-07-04) ========================== As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details. This release also deprecates the use of environment variables to configure the docker image. See the [docker README](https://github.com/matrix-org/synapse/blob/release-v1.1.0/docker/README.md#legacy-dynamic-configuration-file-support) for more details. No changes since 1.1.0rc2. Synapse 1.1.0rc2 (2019-07-03) ============================= Bugfixes -------- - Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. ([\#5593](https://github.com/matrix-org/synapse/issues/5593)) - Removed the `SYNAPSE_SMTP_*` docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. ([\#5596](https://github.com/matrix-org/synapse/issues/5596)) - Fix a number of "Starting txn from sentinel context" warnings. ([\#5605](https://github.com/matrix-org/synapse/issues/5605)) Internal Changes ---------------- - Update github templates. ([\#5552](https://github.com/matrix-org/synapse/issues/5552)) Synapse 1.1.0rc1 (2019-07-02) ============================= As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details. Features -------- - Added possibilty to disable local password authentication. Contributed by Daniel Hoffend. ([\#5092](https://github.com/matrix-org/synapse/issues/5092)) - Add monthly active users to phonehome stats. ([\#5252](https://github.com/matrix-org/synapse/issues/5252)) - Allow expired user to trigger renewal email sending manually. ([\#5363](https://github.com/matrix-org/synapse/issues/5363)) - Statistics on forward extremities per room are now exposed via Prometheus. ([\#5384](https://github.com/matrix-org/synapse/issues/5384), [\#5458](https://github.com/matrix-org/synapse/issues/5458), [\#5461](https://github.com/matrix-org/synapse/issues/5461)) - Add --no-daemonize option to run synapse in the foreground, per issue #4130. Contributed by Soham Gumaste. ([\#5412](https://github.com/matrix-org/synapse/issues/5412), [\#5587](https://github.com/matrix-org/synapse/issues/5587)) - Fully support SAML2 authentication. Contributed by [Alexander Trost](https://github.com/galexrt) - thank you! ([\#5422](https://github.com/matrix-org/synapse/issues/5422)) - Allow server admins to define implementations of extra rules for allowing or denying incoming events. ([\#5440](https://github.com/matrix-org/synapse/issues/5440), [\#5474](https://github.com/matrix-org/synapse/issues/5474), [\#5477](https://github.com/matrix-org/synapse/issues/5477)) - Add support for handling pagination APIs on client reader worker. ([\#5505](https://github.com/matrix-org/synapse/issues/5505), [\#5513](https://github.com/matrix-org/synapse/issues/5513), [\#5531](https://github.com/matrix-org/synapse/issues/5531)) - Improve help and cmdline option names for --generate-config options. ([\#5512](https://github.com/matrix-org/synapse/issues/5512)) - Allow configuration of the path used for ACME account keys. ([\#5516](https://github.com/matrix-org/synapse/issues/5516), [\#5521](https://github.com/matrix-org/synapse/issues/5521), [\#5522](https://github.com/matrix-org/synapse/issues/5522)) - Add --data-dir and --open-private-ports options. ([\#5524](https://github.com/matrix-org/synapse/issues/5524)) - Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. ([\#5534](https://github.com/matrix-org/synapse/issues/5534)) - The minimum TLS version used for outgoing federation requests can now be set with `federation_client_minimum_tls_version`. ([\#5550](https://github.com/matrix-org/synapse/issues/5550)) - Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. ([\#5559](https://github.com/matrix-org/synapse/issues/5559)) - Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. ([\#5476](https://github.com/matrix-org/synapse/issues/5476)) Bugfixes -------- - Fix bug processing incoming events over federation if call to `/get_missing_events` fails. ([\#5042](https://github.com/matrix-org/synapse/issues/5042)) - Prevent more than one room upgrade happening simultaneously on the same room. ([\#5051](https://github.com/matrix-org/synapse/issues/5051)) - Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. ([\#5325](https://github.com/matrix-org/synapse/issues/5325)) - Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. ([\#5387](https://github.com/matrix-org/synapse/issues/5387)) - Fix email notifications for unnamed rooms with multiple people. ([\#5388](https://github.com/matrix-org/synapse/issues/5388)) - Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. ([\#5389](https://github.com/matrix-org/synapse/issues/5389)) - Fix handling of failures fetching remote content to not log failures as exceptions. ([\#5390](https://github.com/matrix-org/synapse/issues/5390)) - Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. ([\#5394](https://github.com/matrix-org/synapse/issues/5394)) - Fix missing invite state after exchanging 3PID invites over federaton. ([\#5464](https://github.com/matrix-org/synapse/issues/5464)) - Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. ([\#5498](https://github.com/matrix-org/synapse/issues/5498)) - Fix logging error when a tampered event is detected. ([\#5500](https://github.com/matrix-org/synapse/issues/5500)) - Fix bug where clients could tight loop calling `/sync` for a period. ([\#5507](https://github.com/matrix-org/synapse/issues/5507)) - Fix bug with `jinja2` preventing Synapse from starting. Users who had this problem should now simply need to run `pip install matrix-synapse`. ([\#5514](https://github.com/matrix-org/synapse/issues/5514)) - Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. ([\#5523](https://github.com/matrix-org/synapse/issues/5523)) - Fixed m.login.jwt using unregistred user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. ([\#5555](https://github.com/matrix-org/synapse/issues/5555), [\#5586](https://github.com/matrix-org/synapse/issues/5586)) - Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. ([\#5576](https://github.com/matrix-org/synapse/issues/5576)) Updates to the Docker image --------------------------- - Add ability to change Docker containers [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) with the `TZ` variable. ([\#5383](https://github.com/matrix-org/synapse/issues/5383)) - Update docker image to use Python 3.7. ([\#5546](https://github.com/matrix-org/synapse/issues/5546)) - Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. ([\#5561](https://github.com/matrix-org/synapse/issues/5561), [\#5562](https://github.com/matrix-org/synapse/issues/5562), [\#5566](https://github.com/matrix-org/synapse/issues/5566), [\#5567](https://github.com/matrix-org/synapse/issues/5567)) - Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. ([\#5547](https://github.com/matrix-org/synapse/issues/5547)) - Send synapse logs to the docker logging system, by default. ([\#5565](https://github.com/matrix-org/synapse/issues/5565)) - Open the non-TLS port by default. ([\#5568](https://github.com/matrix-org/synapse/issues/5568)) - Fix failure to start under docker with SAML support enabled. ([\#5490](https://github.com/matrix-org/synapse/issues/5490)) - Use a sensible location for data files when generating a config file. ([\#5563](https://github.com/matrix-org/synapse/issues/5563)) Deprecations and Removals ------------------------- - Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. ([\#5425](https://github.com/matrix-org/synapse/issues/5425)) - PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. ([\#5448](https://github.com/matrix-org/synapse/issues/5448)) - Remove support for cpu_affinity setting. ([\#5525](https://github.com/matrix-org/synapse/issues/5525)) Improved Documentation ---------------------- - Improve README section on performance troubleshooting. ([\#4276](https://github.com/matrix-org/synapse/issues/4276)) - Add information about how to install and run `black` on the codebase to code_style.rst. ([\#5537](https://github.com/matrix-org/synapse/issues/5537)) - Improve install docs on choosing server_name. ([\#5558](https://github.com/matrix-org/synapse/issues/5558)) Internal Changes ---------------- - Add logging to 3pid invite signature verification. ([\#5015](https://github.com/matrix-org/synapse/issues/5015)) - Update example haproxy config to a more compatible setup. ([\#5313](https://github.com/matrix-org/synapse/issues/5313)) - Track deactivated accounts in the database. ([\#5378](https://github.com/matrix-org/synapse/issues/5378), [\#5465](https://github.com/matrix-org/synapse/issues/5465), [\#5493](https://github.com/matrix-org/synapse/issues/5493)) - Clean up code for sending federation EDUs. ([\#5381](https://github.com/matrix-org/synapse/issues/5381)) - Add a sponsor button to the repo. ([\#5382](https://github.com/matrix-org/synapse/issues/5382), [\#5386](https://github.com/matrix-org/synapse/issues/5386)) - Don't log non-200 responses from federation queries as exceptions. ([\#5383](https://github.com/matrix-org/synapse/issues/5383)) - Update Python syntax in contrib/ to Python 3. ([\#5446](https://github.com/matrix-org/synapse/issues/5446)) - Update federation_client dev script to support `.well-known` and work with python3. ([\#5447](https://github.com/matrix-org/synapse/issues/5447)) - SyTest has been moved to Buildkite. ([\#5459](https://github.com/matrix-org/synapse/issues/5459)) - Demo script now uses python3. ([\#5460](https://github.com/matrix-org/synapse/issues/5460)) - Synapse can now handle RestServlets that return coroutines. ([\#5475](https://github.com/matrix-org/synapse/issues/5475), [\#5585](https://github.com/matrix-org/synapse/issues/5585)) - The demo servers talk to each other again. ([\#5478](https://github.com/matrix-org/synapse/issues/5478)) - Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. ([\#5480](https://github.com/matrix-org/synapse/issues/5480)) - Synapse's codebase is now formatted by `black`. ([\#5482](https://github.com/matrix-org/synapse/issues/5482)) - Some cleanups and sanity-checking in the CPU and database metrics. ([\#5499](https://github.com/matrix-org/synapse/issues/5499)) - Improve email notification logging. ([\#5502](https://github.com/matrix-org/synapse/issues/5502)) - Fix "Unexpected entry in 'full_schemas'" log warning. ([\#5509](https://github.com/matrix-org/synapse/issues/5509)) - Improve logging when generating config files. ([\#5510](https://github.com/matrix-org/synapse/issues/5510)) - Refactor and clean up Config parser for maintainability. ([\#5511](https://github.com/matrix-org/synapse/issues/5511)) - Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the `synapse/` folder within it. ([\#5543](https://github.com/matrix-org/synapse/issues/5543)) - Update v1.0.0 release changelog to include more information about changes to password resets. ([\#5545](https://github.com/matrix-org/synapse/issues/5545)) - Remove non-functioning check_event_hash.py dev script. ([\#5548](https://github.com/matrix-org/synapse/issues/5548)) - Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. ([\#5550](https://github.com/matrix-org/synapse/issues/5550)) - Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. ([\#5557](https://github.com/matrix-org/synapse/issues/5557)) - Reduce the amount of stuff we send in the docker context. ([\#5564](https://github.com/matrix-org/synapse/issues/5564)) - Point the reverse links in the Purge History contrib scripts at the intended location. ([\#5570](https://github.com/matrix-org/synapse/issues/5570))
| * | | | Update changelog v1.1.0 github/release-v1.1.0 release-v1.1.0Erik Johnston2019-07-041-0/+10
| | | | |
| * | | | 1.1.0Erik Johnston2019-07-044-4/+9
| | | | |
| * | | | Merge pull request #5615 from matrix-org/anoa/fix_changelog_email_resetsErik Johnston2019-07-042-1/+2
| |\ \ \ \ | | |/ / / | |/| | | Suggest people use a config file for Docker instead of env vars
| | * | | Suggest people use a config file for Docker instead of env varsAndrew Morgan2019-07-042-1/+2
| |/ / /
* | | | Merge remote-tracking branch 'origin/develop' into shhsAmber H. Brown2019-07-03108-202/+464
|\| | |
| * | | 1.1.0rc2 v1.1.0rc2Richard van der Hoff2019-07-036-5/+18
| | | |
| * | | Fix a number of "Starting txn from sentinel context" warnings (#5605)Richard van der Hoff2019-07-035-6/+35
| | | | | | | | | | | | | | | | Fixes #5602, #5603
| * | | Remove SMTP_* env var functionality from docker conf (#5596)Andrew Morgan2019-07-032-19/+1
| | | | | | | | | | | | | | | | | | | | Removes any `SMTP_*` docker container environment variables from having any effect on the default config. Fixes https://github.com/matrix-org/synapse/issues/5430
| * | | Fix media repo breaking (#5593)Amber Brown2019-07-026-24/+58
| | | |
| * | | tweak changelog v1.1.0rc1Richard van der Hoff2019-07-021-10/+13
| | | |
| * | | Merge remote-tracking branch 'origin/master' into release-v1.1.0Richard van der Hoff2019-07-023-5/+7
| |\ \ \
| | * \ \ Merge pull request #5552 from matrix-org/rav/github_templatesRichard van der Hoff2019-06-263-5/+7
| | |\ \ \ | | | | | | | | | | | | Update github templates
| | | * | | changelogRichard van der Hoff2019-06-251-0/+1
| | | | | |
| | | * | | Update github templatesRichard van der Hoff2019-06-251-2/+3
| | | | | |
| | | * | | Update SUPPORT.mdRichard van der Hoff2019-06-251-3/+3
| | | | | |
| * | | | | prepare v1.1.0rc1Richard van der Hoff2019-07-0291-94/+106
| | | | | |
| * | | | | Complete the SAML2 implementation (#5422)Richard van der Hoff2019-07-027-45/+231
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * SAML2 Improvements and redirect stuff Signed-off-by: Alexander Trost <galexrt@googlemail.com> * Code cleanups and simplifications. Also: share the saml client between redirect and response handlers. * changelog * Revert redundant changes to static js * Move all the saml stuff out to a centralised handler * Add support for tracking SAML2 sessions. This allows us to correctly handle `allow_unsolicited: False`. * update sample config * cleanups * update sample config * rename BaseSSORedirectServlet for consistency * Address review comments
| | * \ \ \ \ Merge branch 'develop' into rav/saml2_clientRichard van der Hoff2019-07-0155-450/+832
| | |\ \ \ \ \
| | * | | | | | Address review commentsRichard van der Hoff2019-07-011-5/+7
| | | | | | | |
| | * | | | | | rename BaseSSORedirectServlet for consistencyRichard van der Hoff2019-06-271-3/+3
| | | | | | | |
| | * | | | | | update sample configRichard van der Hoff2019-06-271-6/+13
| | | | | | | |
| | * | | | | | cleanupsRichard van der Hoff2019-06-273-11/+18
| | | | | | | |
| | * | | | | | update sample configRichard van der Hoff2019-06-261-0/+13
| | | | | | | |
| | * | | | | | Add support for tracking SAML2 sessions.Richard van der Hoff2019-06-262-3/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to correctly handle `allow_unsolicited: False`.
| | * | | | | | Move all the saml stuff out to a centralised handlerRichard van der Hoff2019-06-264-50/+96
| | | | | | | |
| | * | | | | | Revert redundant changes to static jsRichard van der Hoff2019-06-261-4/+2
| | | | | | | |
| | * | | | | | Merge branch 'develop' into rav/saml2_clientRichard van der Hoff2019-06-26478-11486/+18913
| | |\ \ \ \ \ \
| | * | | | | | | changelogRichard van der Hoff2019-06-111-0/+1
| | | | | | | | |
| | * | | | | | | Merge tag 'v1.0.0rc3' into rav/saml2_clientRichard van der Hoff2019-06-116-2/+14
| | |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.0.0rc3 (2019-06-10) ============================= Security: Fix authentication bug introduced in 1.0.0rc1. Please upgrade to rc3 immediately
| | * | | | | | | | Code cleanups and simplifications.Richard van der Hoff2019-06-116-50/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also: share the saml client between redirect and response handlers.
| | * | | | | | | | Merge remote-tracking branch 'origin/develop' into rav/saml2_clientRichard van der Hoff2019-06-10157-1096/+5758
| | |\ \ \ \ \ \ \ \
| | * | | | | | | | | SAML2 Improvements and redirect stuffAlexander Trost2019-06-025-2/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Alexander Trost <galexrt@googlemail.com>
| * | | | | | | | | | Merge pull request #5587 from matrix-org/erikj/fix_synctlErik Johnston2019-07-022-6/+7
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Fix --no-daemonize flag for synctl
| | * | | | | | | | | | NewsfileErik Johnston2019-07-021-0/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Fix --no-daemonize flag for synctlErik Johnston2019-07-021-6/+6
| | | |_|_|_|_|_|_|_|/ | | |/| | | | | | | |
| * | | | | | | | | | Add ability to set timezone for Docker container (#5383)Amir Zarrinkafsh2019-07-023-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Amir Zarrinkafsh <nightah@me.com>
| * | | | | | | | | | Fix JWT login with new users (#5586)PauRE2019-07-022-6/+4
| |/ / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* | | | | | | | | | Merge remote-tracking branch 'origin/develop' into shhsAmber H. Brown2019-07-02461-11322/+17745
|\| | | | | | | | |
| * | | | | | | | | fix async/await consentresource (#5585)Amber Brown2019-07-012-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #5582
| * | | | | | | | | Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloopErik Johnston2019-07-012-2/+20
| |\ \ \ \ \ \ \ \ \ | | |_|_|_|_|/ / / / | |/| | | | | | | | Fix sync tightloop bug.
| | * | | | | | | | Fixup commentErik Johnston2019-07-011-6/+13
| | | | | | | | | |
| | * | | | | | | | NewsfileErik Johnston2019-06-211-0/+1
| | | | | | | | | |
| | * | | | | | | | Fix sync tightloop bug.Erik Johnston2019-06-211-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If, for some reason, presence updates take a while to persist then it can trigger clients to tightloop calling `/sync` due to the presence handler returning updates but not advancing the stream token. Fixes #5503.
| * | | | | | | | | Make the http server handle coroutine-making REST servlets (#5475)Amber Brown2019-06-2912-174/+162
| | | | | | | | | |
| * | | | | | | | | Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimitBrendan Abolivier2019-06-282-0/+2
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Don't update the ratelimiter before sending a 3PID invite
| | * | | | | | | | | Only ratelimit when sending the emailBrendan Abolivier2019-06-281-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
| | * | | | | | | | | ChangelogBrendan Abolivier2019-06-281-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Don't update the ratelimiter before sending a 3PID inviteBrendan Abolivier2019-06-281-1/+1
| | | |_|_|_|_|_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
| * | | | | | | | | Don't log GC 0s at INFO (#5557)Amber Brown2019-06-282-1/+5
| | | | | | | | | |
| * | | | | | | | | Update the TLS cipher string and provide configurability for TLS on outgoing ↵Amber Brown2019-06-287-9/+190
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | federation (#5550)
| * | | | | | | | | Added possibilty to disable local password authentication (#5092)Daniel Hoffend2019-06-275-1/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Daniel Hoffend <dh@dotlan.net>
| * | | | | | | | | Include systemd-python in Debian package to allow logging to journal (#5261)Silke Hofstra2019-06-272-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Silke Hofstra <silke@slxh.eu>
| * | | | | | | | | Make it clearer that the template dir is relative to synapse's root dir (#5543)Andrew Morgan2019-06-273-2/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Helps address #5444
| * | | | | | | | | Merge pull request #5559 from matrix-org/erikj/refactor_changed_devicesErik Johnston2019-06-274-37/+98
| |\ \ \ \ \ \ \ \ \ | | |/ / / / / / / / | |/| | | | | | | | Refactor devices changed query to pull less from DB
| | * | | | | | | | Review commentsErik Johnston2019-06-272-7/+5
| | | | | | | | | |
| | * | | | | | | | Move changelogErik Johnston2019-06-261-0/+0
| | | | | | | | | |
| | * | | | | | | | Refactor and comment sync device list codeErik Johnston2019-06-261-17/+53
| | | | | | | | | |
| | * | | | | | | | Use batch_iter and correct docstringErik Johnston2019-06-261-15/+12
| | | | | | | | | |
| | * | | | | | | | Rename get_users_whose_devices_changedErik Johnston2019-06-263-5/+5
| | | | | | | | | |
| | * | | | | | | | NewsfileErik Johnston2019-06-261-0/+1
| | | | | | | | | |
| | * | | | | | | | Refactor get_user_ids_changed to pull less from DBErik Johnston2019-06-263-28/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a client asks for users whose devices have changed since a token we used to pull *all* users from the database since the token, which could easily be thousands of rows for old tokens. This PR changes this to only check for changes for users the client is actually interested in. Fixes #5553
| * | | | | | | | | Docker image: Add a migrate_config mode (#5567)Richard van der Hoff2019-06-274-17/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... to help people escape env var hell
| * | | | | | | | | Docker image: open the non-TLS port by default. (#5568)Richard van der Hoff2019-06-272-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's not much point in binding to localhost when it's in a docker container.
| * | | | | | | | | Deprecate the env var way of running the docker image (#5566)Richard van der Hoff2019-06-274-154/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is mostly a documentation change, but also adds a default value for SYNAPSE_CONFIG_PATH, so that running from the generated config is the default, and will Just Work provided your config is in the right place.
| * | | | | | | | | Fix JWT login (#5555)PauRE2019-06-273-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix JWT login with register Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Add pyjwt conditional dependency Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Added changelog file Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Improved changelog description Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
| * | | | | | | | | Merge pull request #5565 from matrix-org/rav/docker/fix_log_configRichard van der Hoff2019-06-273-1/+14
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Docker: generate our own log config
| | * | | | | | | | | changelogRichard van der Hoff2019-06-261-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Docker: generate our own log configRichard van der Hoff2019-06-261-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running under docker, we want to use docker's own logging stuff rather than losing the logs somewhere on the container's filesystem, so let's use log configs that spit logs out to stdout instead.
| | * | | | | | | | | Add support for SYNAPSE_CONFIG_DIRRichard van der Hoff2019-06-262-0/+5
| | | | | | | | | | |
| * | | | | | | | | | Reduce the amount of stuff we send in the docker context (#5564)Richard van der Hoff2019-06-272-9/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this makes docker builds a bit faster.
| * | | | | | | | | | Merge pull request #5563 from matrix-org/rav/docker/data_dirRichard van der Hoff2019-06-273-4/+25
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Docker image: add support for SYNAPSE_DATA_DIR parameter
| | * | | | | | | | | changelogRichard van der Hoff2019-06-261-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Docker image: add support for SYNAPSE_DATA_DIR parameterRichard van der Hoff2019-06-262-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #4830.
| | * | | | | | | | | document supported env vars for docker 'generate' optionRichard van der Hoff2019-06-261-0/+7
| | | | | | | | | | |
| * | | | | | | | | | Merge pull request #5562 from matrix-org/rav/docker/no-generate-keysRichard van der Hoff2019-06-272-8/+24
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Docker: only run --generate-keys when generating config on-the-fly.
| | * | | | | | | | | | changelogRichard van der Hoff2019-06-261-0/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Docker: only run --generate-keys when generating config on-the-fly.Richard van der Hoff2019-06-261-7/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't want to generate any missing configs when running from a precanned config. (There's a strong argument that we don't want to do this at all, since generating a new signing key on each invocation sounds disasterous, but I don't fancy unpicking that for now.)
| | * | | | | | | | | | improve logging for generate_config_from_templateRichard van der Hoff2019-06-261-1/+2
| | |/ / / / / / / / /
| * | | | | | | | | | Merge pull request #5561 from matrix-org/rav/docker/refactorRichard van der Hoff2019-06-272-62/+118
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the docker/start.py script
| | * | | | | | | | | changelogRichard van der Hoff2019-06-261-0/+1
| | | | | | | | | | |
| | * | | | | | | | | isortRichard van der Hoff2019-06-251-4/+5
| | | | | | | | | | |
| | * | | | | | | | | Add a main() functionRichard van der Hoff2019-06-251-8/+11
| | | | | | | | | | |
| | * | | | | | | | | Factor out a run_generate_config functionRichard van der Hoff2019-06-251-17/+28
| | | | | | | | | | |
| | * | | | | | | | | Factor out "generate_config_from_template"Richard van der Hoff2019-06-251-41/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and inline generate_secrets
| * | | | | | | | | | Merge pull request #5570 from almereyda/patch-2Richard van der Hoff2019-06-272-2/+3
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Update purge_api README
| | * | | | | | | | | | fix changelogRichard van der Hoff2019-06-271-1/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Create 5570.miscjon r2019-06-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Jon Richter <jon@allmende.io>
| | * | | | | | | | | | Update purge_api READMEjon r2019-06-271-2/+2
| | | |_|_|_|_|_|/ / / | | |/| | | | | | | | | | | | | | | | | | | This points the reverse links at the intended location.
| * | | | | | | | | | Merge pull request #5313 from twrist/patch-1Richard van der Hoff2019-06-272-2/+5
| |\ \ \ \ \ \ \ \ \ \ | | |/ / / / / / / / / | |/| | | | | | | | | Update HAProxy example rules
| | * | | | | | | | | Create 5313.miscIke Johnson2019-06-021-0/+1
| | | | | | | | | | |
| | * | | | | | | | | Update HAProxy example rulesIke Johnson2019-06-021-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These new rules allow a user to instead route only matrix traffic, allowing them to run matrix on the domain without affecting their existing websites
| * | | | | | | | | | Improve docs on choosing server_name (#5558)Richard van der Hoff2019-06-262-13/+19
| | |_|/ / / / / / / | |/| | | | | | | | | | | | | | | | | | Fixes #4901
| * | | | | | | | | Remove & changelog (#5548)Amber Brown2019-06-262-54/+1
| | | | | | | | | |
| * | | | | | | | | Merge branch 'master' into developAndrew Morgan2019-06-253-2/+7
| |\ \ \ \ \ \ \ \ \ | | |_|/ / / / / / / | |/| | | | | | / / | | | |_|_|_|_|/ / | | |/| | | | | | | | | | | | | | | * master: Fix broken link in MSC1711 FAQ Update changelog to better expain password reset change (#5545)
| | * | | | | | | Merge branch 'release-v1.0.0'Andrew Morgan2019-06-252-1/+6
| | |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * release-v1.0.0: Update changelog to better expain password reset change (#5545)
| | | * | | | | | | Update changelog to better expain password reset change (#5545) github/release-v1.0.0 release-v1.0.0Andrew Morgan2019-06-242-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Updates the v1.0.0 changelog to provide more information to those upgrading to v1.0 on why they may receive errors or find their password reset abilities have now been disabled. Helps address #5444
| | * | | | | | | | Fix broken link in MSC1711 FAQRichard van der Hoff2019-06-251-1/+1
| | | |_|_|_|_|_|/ | | |/| | | | | |
| * | | | | | | | Prevent multiple upgrades on the same room at once (#5051)Andrew Morgan2019-06-253-52/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Closes #4583 Does slightly less than #5045, which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time. Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
| * | | | | | | | Merge branch 'develop' of github.com:matrix-org/synapse into developRichard van der Hoff2019-06-255-8/+7
| |\ \ \ \ \ \ \ \
| | * | | | | | | | Increase default log level for docker image to INFO. (#5547)Richard van der Hoff2019-06-252-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #3370.
| | * | | | | | | | Update docker image to use Python 3.7. (#5546)Richard van der Hoff2019-06-253-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python 3.7 is apparently faster than 3.6, and should be mature enough.
| * | | | | | | | | Grafana dashboard updatesRichard van der Hoff2019-06-251-1/+6814
| |\ \ \ \ \ \ \ \ \ | | |/ / / / / / / / | |/| | | | | | | |
| | * | | | | | | | Add extremities graphs to grafana dashboardRichard van der Hoff2019-06-251-23/+832
| | | | | | | | | |
| | * | | | | | | | format json for grafana dashboardRichard van der Hoff2019-06-251-1/+6005
| |/ / / / / / / /
| * | | | | | | | Add info about black to code_style.rst (#5537)Andrew Morgan2019-06-242-45/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #5533 Adds information about how to install and run black on the codebase.
| * | | | | | | | Merge pull request #5524 from matrix-org/rav/new_cmdline_optionsRichard van der Hoff2019-06-244-6/+43
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Add --data-dir and --open-private-ports options.
| | * | | | | | | | changelogRichard van der Hoff2019-06-241-0/+1
| | | | | | | | | |
| | * | | | | | | | Add "--open-private-ports" cmdline optionRichard van der Hoff2019-06-243-6/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is helpful when generating a config file for running synapse under docker.
| | * | | | | | | | Add --data-directory commandline argumentRichard van der Hoff2019-06-241-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't necessarily want to put the data in the cwd.
| * | | | | | | | | Merge pull request #5523 from matrix-org/rav/arg_defaultsRichard van der Hoff2019-06-2430-107/+88
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | Stop conflating generated config and default config
| | * | | | | | | | changelogRichard van der Hoff2019-06-241-0/+1
| | | | | | | | | |
| | * | | | | | | | Don't load the generated config as the default.Richard van der Hoff2019-06-2429-94/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's too confusing.
| | * | | | | | | | Ensure that all config options have sensible defaultsRichard van der Hoff2019-06-244-13/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will enable us to skip the unintuitive behaviour where the generated config and default config are the same thing.
| * | | | | | | | | Merge pull request #5499 from matrix-org/rav/cleanup_metricsRichard van der Hoff2019-06-243-29/+53
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Cleanups and sanity-checking in cpu and db metrics
| | * | | | | | | | | blackRichard van der Hoff2019-06-241-6/+2
| | | | | | | | | | |
| | * | | | | | | | | Avoid raising exceptions in metricsRichard van der Hoff2019-06-241-8/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sentry will catch the errors if they happen, so that should be good enough, and woun't make things explode if we hit the error condition.
| | * | | | | | | | | Merge branch 'develop' into rav/cleanup_metricsRichard van der Hoff2019-06-24397-10437/+9513
| | |\ \ \ \ \ \ \ \ \
| | * | | | | | | | | | changelogRichard van der Hoff2019-06-191-0/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Sanity-checking for metrics updatesRichard van der Hoff2019-06-191-7/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Check that our clocks go forward.
| | * | | | | | | | | | Simplify PerformanceCounters.update interfaceRichard van der Hoff2019-06-191-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we already have the duration for the update, so may as well use it rather than passing extra params around and recalculating it.
| | * | | | | | | | | | Remove unused _get_event_countersRichard van der Hoff2019-06-191-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has been redundant since cdb3757942fefdcdc3d33b9c6d7c9e44decefd6f.
| * | | | | | | | | | | Merge pull request #5534 from matrix-org/babolivier/federation-publicroomsBrendan Abolivier2019-06-246-20/+49
| |\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | Split public rooms directory auth config in two
| | * | | | | | | | | | | Split public rooms directory auth config in twoBrendan Abolivier2019-06-246-20/+49
| | | | | | | | | | | | |
| * | | | | | | | | | | | Merge pull request #5531 from matrix-org/erikj/workers_pagination_tokenErik Johnston2019-06-243-17/+20
| |\ \ \ \ \ \ \ \ \ \ \ \ | | |_|_|_|/ / / / / / / / | |/| | | | | | | | | | | Fix /messages on workers when no from param specified.
| | * | | | | | | | | | | NewsfileErik Johnston2019-06-241-0/+1
| | | | | | | | | | | | |
| | * | | | | | | | | | | Fix /messages on workers when no from param specified.Erik Johnston2019-06-242-17/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If no `from` param is specified we calculate and use the "current token" that inlcuded typing, presence, etc. These are unused during pagination and are not available on workers, so we simply don't calculate them.