summary refs log tree commit diff
path: root/synapse/replication (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2020-01-314-5/+12
|\
| * Fix sending server up commands from workers (#6811)Erik Johnston2020-01-301-0/+4
| | | | | | | | Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
| * Detect unknown remote devices and mark cache as stale (#6776)Erik Johnston2020-01-281-1/+1
| | | | | | | | We just mark the fact that the cache may be stale in the database for now.
| * Propagate cache invalidates from workers to other workers. (#6748)Erik Johnston2020-01-272-4/+7
| | | | | | Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesBrendan Abolivier2020-01-2211-175/+235
|\|
| * Allow streaming cache invalidate all to workers. (#6749)Erik Johnston2020-01-222-6/+27
| |
| * Wake up transaction queue when remote server comes back online (#6706)Erik Johnston2020-01-174-0/+44
| | | | | | | | | | This will be used to retry outbound transactions to a remote server if we think it might have come back up.
| * Port synapse.replication.tcp to async/await (#6666)Erik Johnston2020-01-165-85/+63
| | | | | | | | | | | | | | | | | | | | * Port synapse.replication.tcp to async/await * Newsfile * Correctly document type of on_<FOO> functions as async * Don't be overenthusiastic with the asyncing....
| * Add `local_current_membership` table (#6655)Erik Johnston2020-01-151-1/+1
| | | | | | | | | | | | | | Currently we rely on `current_state_events` to figure out what rooms a user was in and their last membership event in there. However, if the server leaves the room then the table may be cleaned up and that information is lost. So lets add a table that separately holds that information.
| * Fixup synapse.replication to pass mypy checks (#6667)Erik Johnston2020-01-1410-86/+103
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesRichard van der Hoff2020-01-063-3/+8
|\|
| * Reduce the reconnect time when replication fails. (#6617)Richard van der Hoff2020-01-031-1/+2
| |
| * Change EventContext to use the Storage class (#6564)Erik Johnston2019-12-202-2/+6
| |
* | Merge branch 'release-v1.7.0' of github.com:matrix-org/synapse into ↵Neil Johnson2019-12-1014-30/+47
|\| | | | | | | matrix-org-hotfixes
| * Change DataStores to accept 'database' param.Erik Johnston2019-12-0613-26/+39
| |
| * _CURRENT_STATE_CACHE_NAME is publicErik Johnston2019-12-041-2/+2
| |
| * Move cache invalidation to main data storeErik Johnston2019-12-041-1/+2
| |
| * Propagate reason in remotely rejected invitesErik Johnston2019-11-281-2/+5
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-11-263-5/+85
|\|
| * Prevent account_data content from being sent over TCP replication (#6333)Andrew Morgan2019-11-261-4/+3
| |\
| | * lintAndrew Morgan2019-11-081-2/+1
| | |
| | * Remove content from being sent for account data rdata streamAndrew Morgan2019-11-081-3/+3
| | |
| * | Merge pull request #6332 from matrix-org/erikj/query_devices_fixErik Johnston2019-11-262-1/+82
| |\ \ | | |/ | |/| Fix caching devices for remote servers in worker.
| | * Fixup docsErik Johnston2019-11-261-1/+5
| | |
| | * Fix caching devices for remote servers in worker.Erik Johnston2019-11-052-1/+78
| | | | | | | | | | | | | | | | | | | | | | | | When the `/keys/query` API is hit on client_reader worker Synapse may decide that it needs to resync some remote deivces. Usually this happens on master, and then gets cached. However, that fails on workers and so it falls back to fetching devices from remotes directly, which may in turn fail if the remote is down.
* | | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-11-1912-59/+157
|\| |
| * | Address review commentsAndrew Morgan2019-11-061-1/+1
| | |
| * | Don't forget to ratelimit calls outside of RegistrationHandlerAndrew Morgan2019-11-061-0/+2
| |/
| * document the REPLICATE command a bit better (#6305)Richard van der Hoff2019-11-043-9/+95
| | | | | | | | since I found myself wonder how it works
| * Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notifyHubert Chathi2019-10-314-4/+4
| |\
| | * Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-314-4/+4
| | | | | | | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
| * | clean up code a bitHubert Chathi2019-10-311-5/+9
| | |
| * | make user signatures a separate streamHubert Chathi2019-10-303-2/+25
| | |
| * | Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notifyHubert Chathi2019-10-306-44/+26
| |\|
| | * Port replication http server endpoints to async/awaitErik Johnston2019-10-296-44/+26
| | |
| * | make notification of signatures work with workersHubert Chathi2019-10-241-0/+1
| |/
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-10-2517-27/+32
|\|
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-221-0/+3
| |\ | | | | | | | | | erikj/refactor_stores
| | * Merge branch 'develop' into uhoreg/e2e_cross-signing_mergedHubert Chathi2019-09-071-10/+11
| | |\
| | * | add user signature stream change cache to slaved device storeHubert Chathi2019-09-041-0/+3
| | | |
| * | | Move storage classes into a main "data store".Erik Johnston2019-10-2117-27/+29
| | |/ | |/| | | | | | | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* | | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-09-061-1/+6
|\| |
| * | Trace how long it takes for the send trasaction to complete, including ↵Jorik Schellekens2019-09-051-1/+6
| | | | | | | | | | | | retrys (#5986)
* | | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-09-052-28/+9
|\| |
| * | Add opentracing to all client servlets (#5983)Jorik Schellekens2019-09-051-10/+6
| |/
| * Remove bind_email and bind_msisdn (#5964)Andrew Morgan2019-09-041-18/+3
| | | | | | Removes the `bind_email` and `bind_msisdn` parameters from the `/register` C/S API endpoint as per [MSC2140: Terms of Service for ISes and IMs](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R107).
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-09-046-15/+15
|\|
| * Remove unnecessary parentheses around return statements (#5931)Andrew Morgan2019-08-306-15/+15
| | | | | | | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-08-291-2/+14
|\|
| * Opentracing across workers (#5771)Jorik Schellekens2019-08-221-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Propagate opentracing contexts across workers Also includes some Convenience modifications to opentracing for servlets, notably: - Add boolean to skip the whitelisting check on inject extract methods. - useful when injecting into carriers locally. Otherwise we'd always have to include our own servername and whitelist our servername - start_active_span_from_request instead of header - Add boolean to decide whether to extract context from a request to a servlet
| * Revert "Add "require_consent" parameter for registration"Brendan Abolivier2019-08-221-2/+0
| | | | | | | | This reverts commit 3320aaab3a9bba3f5872371aba7053b41af9d0a0.
| * Add "require_consent" parameter for registrationHalf-Shot2019-08-221-0/+2
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-07-308-20/+20
|\|
| * Merge tag 'v1.2.0rc2' into developAndrew Morgan2019-07-241-1/+1
| |\ | | | | | | | | | | | | | | | | | | Bugfixes -------- - Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))
| * | Replace returnValue with return (#5736)Amber Brown2019-07-238-20/+20
| | |
* | | Merge branch 'release-v1.2.0' of github.com:matrix-org/synapse into ↵Andrew Morgan2019-07-241-1/+1
|\ \ \ | | |/ | |/| | | | matrix-org-hotfixes
| * | Fix servlet metric names (#5734)Jorik Schellekens2019-07-241-1/+1
| |/ | | | | | | | | | | | | | | | | | | * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-07-173-72/+1
|\|
| * Remove access-token support from RegistrationHandler.register (#5641)Richard van der Hoff2019-07-081-6/+0
| | | | | | | | | | | | | | | | Nothing uses this now, so we can remove the dead code, and clean up the API. Since we're changing the shape of the return value anyway, we take the opportunity to give the method a better name.
| * Remove support for invite_3pid_guest. (#5625)Richard van der Hoff2019-07-051-65/+0
| | | | | | | | | | | | | | | | | | This has never been documented, and I'm not sure it's ever been used outside sytest. It's quite a lot of poorly-maintained code, so I'd like to get rid of it. For now I haven't removed the database table; I suggest we leave that for a future clearout.
| * Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-06-2126-355/+357
|\|
| * Run Black. (#5482)Amber Brown2019-06-2026-355/+357
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-06-171-1/+9
|\|
| * Handle failing to talk to master over replicationErik Johnston2019-06-071-1/+9
| |
* | Merge branch 'develop' into matrix-org-hotfixesRichard van der Hoff2019-05-203-8/+18
|\|
| * Fixup bsaed on review commentsErik Johnston2019-05-171-1/+1
| |
| * Add basic editing supportErik Johnston2019-05-161-0/+1
| |
| * Fix relations in worker modeErik Johnston2019-05-163-8/+17
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesRichard van der Hoff2019-05-021-17/+4
|\|
| * Replace SlavedKeyStore with a shimRichard van der Hoff2019-04-081-14/+4
| | | | | | | | | | since we're pulling everything out of KeyStore anyway, we may as well simplify it.
| * Remove unused server_tls_certificates functions (#5028)Richard van der Hoff2019-04-081-3/+0
| | | | | | | | These have been unused since #4120, and with the demise of perspectives, it is unlikely that they will ever be used again.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-04-041-10/+0
|\|
| * Remove presence lists (#4989)Neil Johnson2019-04-031-10/+0
| | | | | | Remove presence list support as per MSC 1819
* | Merge branch 'develop' into matrix-org-hotfixesRichard van der Hoff2019-04-028-113/+299
|\|
| * Fix sync bug when accepting invites (#4956)Richard van der Hoff2019-04-021-9/+22
| | | | | | | | | | | | | | | | | | | | Hopefully this time we really will fix #4422. We need to make sure that the cache on `get_rooms_for_user_with_stream_ordering` is invalidated *before* the SyncHandler is notified for the new events, and we can now do so reliably via the `events` stream.
| * Combine the CurrentStateDeltaStream into the EventStreamRichard van der Hoff2019-03-273-23/+33
| |
| * Make EventStream rows have a typeRichard van der Hoff2019-03-273-16/+94
| | | | | | | | ... as a precursor to combining it with the CurrentStateDelta stream.
| * Skip building a ROW_TYPE when building updatesRichard van der Hoff2019-03-271-2/+2
| | | | | | | | | | We're about to turn it straight into a JSON object anyway so building a ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
| * Add parse_row method to replication stream classRichard van der Hoff2019-03-273-3/+19
| | | | | | | | This will allow individual stream classes to override how a row is parsed.
| * move FederationStream out to its own fileRichard van der Hoff2019-03-274-23/+43
| |
| * move EventsStream out to its own fileRichard van der Hoff2019-03-273-23/+42
| |
| * Move replication.tcp.streams into a packageRichard van der Hoff2019-03-272-33/+51
| |
| * Fix/improve some docstrings in the replication code. (#4949)Richard van der Hoff2019-03-272-7/+19
| |
* | Merge branch 'develop' into matrix-org-hotfixesRichard van der Hoff2019-03-253-15/+32
|\|
| * Fix ClientReplicationStreamProtocol.__str__ (#4929)Richard van der Hoff2019-03-252-4/+5
| | | | | | | | | | | | | | | | `__str__` depended on `self.addr`, which was absent from ClientReplicationStreamProtocol, so attempting to call str on such an object would raise an exception. We can calculate the peer addr from the transport, so there is no need for addr anyway.
| * Fix bug where read-receipts lost their timestamps (#4927)Richard van der Hoff2019-03-252-11/+27
| | | | | | | | | | Make sure that they are sent correctly over the replication stream. Fixes: #4898
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-03-211-1/+17
|\|
| * Add a config option for torture-testing worker replication. (#4902)Richard van der Hoff2019-03-201-1/+17
| | | | | | Setting this to 50 or so makes a bunch of sytests fail in worker mode.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-03-061-3/+28
|\|
| * Prefill client IPs cache on workersErik Johnston2019-03-061-0/+2
| |
| * Merge pull request #4792 from matrix-org/anoa/replication_tokensAndrew Morgan2019-03-061-3/+28
| |\ | | | | | | Support batch updates in the worker sender
| | * Simplify token replication logicAndrew Morgan2019-03-051-23/+14
| | |
| | * Clean up logic and add commentsAndrew Morgan2019-03-041-11/+18
| | |
| | * Clearer branching, fix missing list clearAndrew Morgan2019-03-041-4/+11
| | |
| | * Prevent replication wedgingAndrew Morgan2019-03-041-4/+24
| | |
* | | Remove redundant changes from synapse/replication/tcp/streams.py (#4813)Richard van der Hoff2019-03-061-2/+2
| | | | | | | | | | | | | | | This was some hacky code (introduced in c10c71e70d) to make the presence stream do nothing on hotfixes. We now ensure that no replication clients subscribe to the presence stream, so this is redundant.
* | | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-03-058-45/+76
|\| |
| * | Add rate-limiting on registration (#4735)Brendan Abolivier2019-03-051-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test
| * | Fixup slave storesErik Johnston2019-03-043-36/+26
| |/
| * When presence is enabled don't send over replicationErik Johnston2019-02-271-2/+5
| |
| * Merge pull request #4749 from matrix-org/erikj/replication_connection_backoffErik Johnston2019-02-273-5/+39
| |\ | | | | | | Fix tightloop over connecting to replication server
| | * Move connecting logic into ClientReplicationStreamProtocolErik Johnston2019-02-272-18/+17
| | |
| | * Increase the max delay between retry attemptsErik Johnston2019-02-261-1/+1
| | | | | | | | | | | | | | | Otherwise if you have many workers they can easily take out master with their connection attempts
| | * Fix tightloop over connecting to replication serverErik Johnston2019-02-262-4/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the client failed to process incoming commands during the initial set up of the replication connection it would immediately disconnect and reconnect, resulting in a tightloop. This can happen, for example, when subscribing to a stream that has a row that is too long in the backlog. The fix here is to not consider the connection successfully set up until the client has succesfully subscribed and caught up with the streams. This ensures that the retry logic timers aren't reset until then, meaning that if an error does happen during start up the client will continue backing off before retrying again.
* | | Remove #4733 debug (#4767)Richard van der Hoff2019-03-041-4/+0
| | | | | | | | | | | | We don't need any of this stuff now; this brings protocol.py back into line with develop for the hotfixes branch.
* | | Merge branch 'develop' into matrix-org-hotfixesRichard van der Hoff2019-02-271-1/+16
|\| |
| * | Limit cache invalidation replication line length (#4748)Erik Johnston2019-02-271-1/+16
| |/
* | Add logging when sending errorErik Johnston2019-02-251-0/+1
| |
* | Merge pull request #4734 from matrix-org/rav/repl_debugErik Johnston2019-02-251-0/+3
|\ \ | | | | | | Add some debug to help with #4733
| * | Add some debug to help with #4733Richard van der Hoff2019-02-251-0/+3
| |/
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-02-221-6/+1
|\|
| * Fix state cache invalidation on workersErik Johnston2019-02-221-6/+1
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-02-203-3/+58
|\|
| * Fix registration on workers (#4682)Erik Johnston2019-02-203-3/+58
| | | | | | | | | | | | | | | | | | | | * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-02-194-8/+180
|\|
| * Batch cache invalidation over replicationErik Johnston2019-02-181-7/+12
| | | | | | | | | | | | | | | | | | | | Currently whenever the current state changes in a room invalidate a lot of caches, which cause *a lot* of traffic over replication. Instead, lets batch up all those invalidations and send a single poke down the replication streams. Hopefully this will reduce load on the master process by substantially reducing traffic.
| * Move register_device into handlerErik Johnston2019-02-181-14/+3
| |
| * Split out registration to workerErik Johnston2019-02-183-1/+179
| | | | | | | | | | | | | | | | This allows registration to be handled by a worker, though the actual write to the database still happens on master. Note: due to the in-memory session map all registration requests must be handled by the same worker.
* | Revert "Merge pull request #4654 from matrix-org/hawkowl/registration-worker"Erik Johnston2019-02-192-67/+1
| | | | | | | | | | This reverts commit 5bd2e2c31dbfed2e69800ee72aef80f7e4bda210, reversing changes made to d97c3a6ce651f7ff2ffb8b7ba5abd1e292b62896.
* | Revert "Merge pull request #4655 from matrix-org/hawkowl/registration-worker"Erik Johnston2019-02-193-67/+2
| | | | | | | | | | This reverts commit 93555af5c91f2a242bb890a52cfd8e0ca303d34a, reversing changes made to 5bd2e2c31dbfed2e69800ee72aef80f7e4bda210.
* | Emergency changesErik Johnston2019-02-152-3/+3
| |
* | fixAmber Brown2019-02-161-1/+2
| |
* | use a device replication thingyAmber Brown2019-02-162-1/+65
| |
* | pep8Amber Brown2019-02-161-5/+0
| |
* | add all the filesAmber Brown2019-02-161-0/+70
| |
* | isortAmber Brown2019-02-161-1/+1
| |
* | add cache remover endpoint and wire it upAmber Brown2019-02-161-1/+2
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-01-301-1/+4
|\|
| * Fix replication for room v3 (#4523)Erik Johnston2019-01-301-1/+4
| | | | | | | | | | | | | | | | | | * Fix replication for room v3 We were not correctly quoting the path fragments over http replication, which meant that it exploded when the event IDs had a slash in them * Newsfile
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2019-01-301-1/+1
|\|
| * Fix receiving events from federation via a workerErik Johnston2019-01-291-1/+1
| | | | | | | | This bug was introduced in PR #4470, commit 678a92cb56d547dcadffa723e29b4855a27d0901
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesAndrew Morgan2019-01-292-4/+12
|\|
| * Replace missed usages of FrozenEventErik Johnston2019-01-252-4/+12
| |
| * Revert "Require event format version to parse or create events"Erik Johnston2019-01-252-12/+4
| |
| * Replace missed usages of FrozenEventErik Johnston2019-01-242-4/+12
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-10-291-2/+2
|\|
| * Don't truncate command name in metricsErik Johnston2018-10-291-2/+2
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesAmber Brown2018-10-291-1/+1
|\|
| * Make the replication logger quieter (#4108)Amber Brown2018-10-291-1/+1
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixesAmber Brown2018-10-156-30/+30
|\|
| * Make workers work on Py3 (#4027)Amber Brown2018-10-136-30/+30
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-09-142-8/+17
|\|
| * Fix minor typo in exceptionTravis Ralston2018-09-131-1/+1
| |
| * merge (#3576)Amber Brown2018-09-141-7/+16
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-09-111-12/+12
|\|
| * Remove conn_idErik Johnston2018-09-041-2/+2
| |
| * Remove conn_id from repl prometheus metricsErik Johnston2018-09-031-10/+10
| | | | | | | | | | `conn_id` gets set to a random string, and so we end up filling up prometheus with tonnes of data series, which is bad.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-08-204-16/+44
|\|
| * Merge pull request #3713 from matrix-org/erikj/fixup_fed_loggingErik Johnston2018-08-201-1/+1
| |\ | | | | | | Fix logging bug in EDU handling over replication
| | * Fix logging bug in EDU handling over replicationErik Johnston2018-08-171-1/+1
| | |
| * | Logcontexts for replication command handlersRichard van der Hoff2018-08-173-15/+43
| |/ | | | | | | | | | | | | | | | | | | Run the handlers for replication commands as background processes. This should improve the visibility in our metrics, and reduce the number of "running db transaction from sentinel context" warnings. Ideally it means converting the things that fire off deferreds into the night into things that actually return a Deferred when they are done. I've made a bit of a stab at this, but it will probably be leaky.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-08-157-256/+638
|\|
| * Use federation handler function rather than duplicateErik Johnston2018-08-151-41/+3
| | | | | | | | This involves renaming _persist_events to be a public function.
| * Rename slave TransactionStore to SlaveTransactionStoreErik Johnston2018-08-151-1/+1
| |
| * Move clean_room_for_join to masterErik Johnston2018-08-091-0/+35
| |
| * Fixup doc commentsErik Johnston2018-08-091-0/+17
| |
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-094-17/+63
| |\ | | | | | | | | | erikj/split_federation
| | * Merge pull request #3632 from matrix-org/erikj/refactor_repl_servletErik Johnston2018-08-093-243/+374
| | |\ | | | | | | | | Add helper base class for generating new replication endpoints
| | | * Fixup wording and remove dead codeErik Johnston2018-08-091-2/+1
| | | |
| | | * Rename POST param to METHODErik Johnston2018-08-082-13/+22
| | | |
| | | * Fixup logging and docstringsErik Johnston2018-08-082-2/+40
| | | |
| | * | Basic support for room versioningRichard van der Hoff2018-08-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event
| * | | Import all functions from TransactionStoreErik Johnston2018-08-061-11/+2
| | | |
| * | | Add EDU/query handling over replicationErik Johnston2018-08-061-1/+1
| | | |
| * | | Add replication APIs for persisting federation eventsErik Johnston2018-08-062-1/+247
| | |/ | |/|
| * | Fix isortErik Johnston2018-08-061-4/+1
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-031-4/+3
| |\| | | | | | | | | | erikj/refactor_repl_servlet
| * | Use new helper base class for membership requestsErik Johnston2018-07-311-171/+91
| | |
| * | Use new helper base class for ReplicationSendEventRestServletErik Johnston2018-07-311-79/+36
| | |
| * | Add helper base class for generating new replication endpointsErik Johnston2018-07-311-0/+208
| | | | | | | | | | | | | | | This will hopefully reduce the boiler plate required to implement new internal HTTP requests.
* | | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-08-064-23/+21
|\ \ \ | | |/ | |/|
| * | Kill off MatrixCodeMessageExceptionRichard van der Hoff2018-08-012-16/+12
| |/ | | | | | | | | | | | | | | | | | | | | This code brings the SimpleHttpClient into line with the MatrixFederationHttpClient by having it raise HttpResponseExceptions when a request fails (rather than trying to parse for matrix errors and maybe raising MatrixCodeMessageException). Then, whenever we were checking for MatrixCodeMessageException and turning them into SynapseErrors, we now need to check for HttpResponseExceptions and call to_synapse_error.
| * Fix unit testsRichard van der Hoff2018-07-251-1/+1
| | | | | | | | | | | | on_notifier_poke no longer runs synchonously, so we have to do a different hack to make sure that the replication data has been sent. Let's actually listen for its arrival.
| * Wrap a number of things that run in the backgroundRichard van der Hoff2018-07-251-6/+8
| | | | | | | | | | This will reduce the number of "Starting db connection from sentinel context" warnings, and will help with our metrics.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-07-231-2/+5
|\|
| * Fix missing attributes on workers.Erik Johnston2018-07-231-2/+5
| | | | | | | | | | This was missed during the transition from attribute to getter for getting state from context.
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-07-121-1/+1
|\|
| * Use stream cache in get_linearized_receipts_for_roomErik Johnston2018-07-101-1/+1
| | | | | | | | | | This avoids us from uncessarily hitting the database when there has been no change for the room
* | Merge branch 'develop' into matrix-org-hotfixesRichard van der Hoff2018-07-1024-66/+91
|\|
| * run isortAmber Brown2018-07-0924-66/+91
| |
* | Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixeshera2018-06-281-6/+10
|\|
| * Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-6/+10
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixesErik Johnston2018-06-253-5/+7
|\|
| * Merge pull request #3441 from matrix-org/erikj/redo_erasureErik Johnston2018-06-251-0/+2
| |\ | | | | | | Fix user erasure and re-enable
| | * Add UserErasureWorkerStore to workersErik Johnston2018-06-251-0/+2
| | |
| * | Remove all global reactor imports & pass it around explicitly (#3424)Amber Brown2018-06-252-5/+5
| |/
* | Prefill client_ip_last_seen in replicationErik Johnston2018-06-221-0/+2
| |
* | Increase MAX_EVENTS_BEHIND for replication clientsErik Johnston2018-06-221-1/+1
|/
* Pass around the reactor explicitly (#3385)Amber Brown2018-06-221-3/+3
|
* Fix tcp protocol metrics naming (#3410)Amber Brown2018-06-211-18/+35
|
* Fix replication metricsRichard van der Hoff2018-06-041-2/+2
| | | | fix bug introduced in #3256
* Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-282-8/+9
|\
| * Merge pull request #3244 from NotAFile/py3-six-4Amber Brown2018-05-242-5/+7
| |\ | | | | | | replace some iteritems with six
| | * replace some iteritems with sixAdrian Tschira2018-05-192-5/+7
| | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* | | more cleanupAmber Brown2018-05-222-6/+10
| | |
* | | fix the test failuresAmber Brown2018-05-221-1/+1
| | |
* | | cleanups, self-registrationAmber Brown2018-05-221-4/+5
| | |
* | | Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-221-0/+2
|\| |
| * | Send users a server notice about consentRichard van der Hoff2018-05-221-0/+2
| |/ | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.
* | rest of the changesAmber Brown2018-05-211-16/+14
| |
* | replacing portionsAmber Brown2018-05-211-54/+34
|/
* make imports localAdrian Tschira2018-04-282-4/+4
| | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* Refactor ResponseCache usageRichard van der Hoff2018-04-121-12/+6
| | | | | | | | | | | | | | | Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production.
* Add metrics for ResponseCacheRichard van der Hoff2018-04-101-1/+1
|
* Fix json encoding bug in replicationRichard van der Hoff2018-04-031-1/+1
| | | | json encoders have an encode method, not a dumps method.
* Use static JSONEncodersRichard van der Hoff2018-03-291-3/+5
| | | | | using json.dumps with custom options requires us to create a new JSONEncoder on each call. It's more efficient to create one upfront and reuse it.
* Explicitly use simplejsonErik Johnston2018-03-201-7/+7
|
* Fix replication after switch to simplejsonErik Johnston2018-03-191-2/+4
| | | | | Turns out that simplejson serialises namedtuple's as dictionaries rather than tuples by default.
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-191-1/+1
|\
| * Replace ujson with simplejsonErik Johnston2018-03-151-1/+1
| |
* | Merge pull request #2992 from matrix-org/erikj/implement_member_workreErik Johnston2018-03-142-3/+336
|\ \ | | | | | | Implement RoomMemberWorkerHandler
| * | Fix importsErik Johnston2018-03-142-7/+4
| | |
| * | s/join/joined/ in notify_user_membership_changeErik Johnston2018-03-141-3/+3
| | |
| * | Implement RoomMemberWorkerHandlerErik Johnston2018-03-132-0/+336
| | |
* | | Split up ProfileStoreErik Johnston2018-03-131-0/+21
|/ /
* | extra_users is actually a list of UserIDsErik Johnston2018-03-131-4/+4
| |
* | Split Directory storeErik Johnston2018-03-051-5/+3
| |
* | Split registration storeErik Johnston2018-03-021-15/+3
| |
* | Merge pull request #2928 from matrix-org/erikj/read_marker_cachesErik Johnston2018-03-011-1/+1
|\ \ | | | | | | Fix typo in getting replication account data processing
| * | Fix typo in getting replication account data processingErik Johnston2018-03-011-1/+1
| | |
* | | Merge pull request #2925 from matrix-org/erikj/split_sig_fedErik Johnston2018-03-011-47/+9
|\ \ \ | | | | | | | | Split out SignatureStore and EventFederationStore
| * | | Remove unused DataStoreErik Johnston2018-03-011-1/+0
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_sig_fedErik Johnston2018-03-012-65/+11
| |\ \ \
| * | | | Split out SignatureStore and EventFederationStoreErik Johnston2018-03-011-42/+8
| | | | |
* | | | | Merge pull request #2927 from matrix-org/erikj/read_marker_cachesErik Johnston2018-03-011-0/+4
|\ \ \ \ \ | |_|/ / / |/| | / / | | |/ / | |/| | Improve caching for read_marker API
| * | | Improve caching for read_marker APIErik Johnston2018-03-011-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | We add a new storage function to get a paritcular type of room account data. This allows us to prefill the cache when updating that acount data.
| * | | Add some caches to help read marker APIErik Johnston2018-03-011-0/+1
| |/ /
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-03-012-19/+4
|\ \ \ | | | | | | | | | | | | erikj/split_stream_store
| * \ \ Merge pull request #2923 from matrix-org/erikj/stream_ago_workerErik Johnston2018-03-011-1/+0
| |\ \ \ | | | | | | | | | | Calculate stream_ordering_month_ago correctly on workers
| | * | | Calculate stream_ordering_month_ago correctly on workersErik Johnston2018-03-011-1/+0
| | |/ /
| * / / Split up RoomStoreErik Johnston2018-03-011-17/+4
| |/ /
* / / Split out stream storeErik Johnston2018-03-011-45/+9
|/ /
* | Log in the correct placesErik Johnston2018-03-011-2/+4
| |
* | Don't do preserve_fn for every requestErik Johnston2018-03-011-1/+2
| |
* | Add some loggingErik Johnston2018-03-011-0/+2
| |
* | Make repl send_event idempotent and retry on timeoutsErik Johnston2018-03-011-6/+38
| | | | | | | | | | | | If we treated timeouts as failures on the worker we would attempt to clean up e.g. push actions while the master might still process the event.
* | Correctly send ratelimit and extra_users paramsErik Johnston2018-03-011-1/+13
| |
* | Calculate push actions on workerErik Johnston2018-02-281-1/+1
| |
* | Merge pull request #2904 from matrix-org/erikj/receipt_cache_invalidationErik Johnston2018-02-271-0/+2
|\ \ | | | | | | Fix missing invalidations for receipt storage
| * | Fix missing invalidations for receipt storageErik Johnston2018-02-211-0/+2
| | |
* | | Merge pull request #2903 from matrix-org/erikj/split_roommember_storeErik Johnston2018-02-271-28/+5
|\ \ \ | | | | | | | | Split out RoomMemberStore
| * \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-234-84/+23
| |\ \ \ | | | | | | | | | | | | | | | erikj/split_roommember_store
| * | | | Update copyrightErik Johnston2018-02-231-0/+1
| | | | |
| * | | | Split out RoomMemberStoreErik Johnston2018-02-211-27/+3
| | | | |
* | | | | Merge pull request #2901 from matrix-org/erikj/split_as_storesErik Johnston2018-02-271-28/+7
|\ \ \ \ \ | |_|/ / / |/| | | | Split AS stores
| * | | | Update copyrightErik Johnston2018-02-231-0/+1
| | | | |
| * | | | Split AS storesErik Johnston2018-02-211-28/+6
| |/ / /
* | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-234-74/+21
|\ \ \ \ | | | | | | | | | | | | | | | erikj/split_event_push_actions
| * \ \ \ Merge pull request #2902 from matrix-org/erikj/split_events_storeErik Johnston2018-02-231-11/+3
| |\ \ \ \ | | | | | | | | | | | | Split out get_events and co into a worker store
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-09-07 13:13:58 +0000