summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #3826 from matrix-org/rav/logging_for_keyringAmber Brown2018-09-121-0/+1
|\ | | | | add some logging for the keyring queue
| * clearer logging when things fail, tooRichard van der Hoff2018-09-061-0/+1
| |
* | Speed up lazy loading (#3827)Matthew Hodgson2018-09-122-0/+69
| | | | | | | | | | * speed up room summaries by pulling their data from room_memberships rather than room state * disable LL for incr syncs, and log incr sync stats (#3840)
* | ensure guests never enter mau listNeil Johnson2018-09-061-4/+3
| |
* | guest users should not be part of mau totalNeil Johnson2018-09-051-1/+6
|/
* move threepid checker to config, add missing yieldsNeil Johnson2018-08-311-14/+0
|
* ensure post registration auth checks do not fail erroneouslyNeil Johnson2018-08-311-1/+14
|
* Port storage/ to Python 3 (#3725)Amber Brown2018-08-3110-31/+74
|
* Implement trail usersErik Johnston2018-08-232-1/+32
|
* Add missing yieldErik Johnston2018-08-231-1/+2
|
* Merge pull request #3659 from matrix-org/erikj/split_profilesErik Johnston2018-08-222-31/+31
|\ | | | | Allow profile updates to happen on workers
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-179-134/+217
| |\ | | | | | | | | | erikj/split_profiles
| * | Remote profile cache should remain in master workerErik Johnston2018-08-171-4/+2
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-097-34/+272
| |\ \ | | | | | | | | | | | | erikj/split_profiles
| * | | Allow ratelimiting on workersErik Johnston2018-08-071-29/+29
| | | |
| * | | Allow profile changes to happen on workersErik Johnston2018-08-071-2/+4
| | | |
* | | | Merge pull request #3673 from matrix-org/erikj/refactor_state_handlerErik Johnston2018-08-221-1/+3
|\ \ \ \ | | | | | | | | | | Refactor state module to support multiple room versions
| * \ \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-209-134/+217
| |\ \ \ \ | | | |_|/ | | |/| | | | | | | erikj/refactor_state_handler
| * | | | Choose state algorithm based on room versionErik Johnston2018-08-091-1/+3
| | |_|/ | |/| |
* | | | Split the state_group_cache in two (#3726)Matthew Hodgson2018-08-221-17/+141
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | Splits the state_group_cache in two. One half contains normal state events; the other contains member events. The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching. Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.
* | | call reap on start up and fix under reaping bugNeil Johnson2018-08-161-1/+4
| | |
* | | initial cut at a room summary API (#3574)Matthew Hodgson2018-08-163-8/+8
| | |
* | | speed up /members and add at= and membership params (#3568)Matthew Hodgson2018-08-152-3/+65
| | |
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-154-24/+41
|\ \ \ | | | | | | | | | | | | erikj/split_federation
| * | | adding missing yieldNeil Johnson2018-08-141-1/+1
| | | |
| * | | Merge pull request #3670 from matrix-org/neilj/mau_sync_blockNeil Johnson2018-08-141-5/+5
| |\ \ \ | | | | | | | | | | Block ability to read via sync if mau limit exceeded
| | * | | rename _user_last_seen_monthly_activeNeil Johnson2018-08-091-5/+5
| | |/ /
| * | | Merge remote-tracking branch 'origin/develop' into ↵Amber Brown2018-08-141-0/+5
| |\ \ \ | | | | | | | | | | | | | | | neilj/fix_reap_users_in_postgres
| | * | | Run tests under PostgreSQL (#3423)Amber Brown2018-08-131-0/+5
| | | | |
| * | | | fix sqlite/postgres incompatibility in reap_monthly_active_usersNeil Johnson2018-08-111-16/+28
| |/ / /
| * / / Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-102-2/+2
| |/ /
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-097-34/+272
|\| | | | | | | | | | | erikj/split_federation
| * | Merge pull request #3654 from matrix-org/rav/room_versionsRichard van der Hoff2018-08-081-3/+30
| |\ \ | | | | | | | | Support for room versioning
| | * | Basic support for room versioningRichard van der Hoff2018-08-031-3/+30
| | |/ | | | | | | | | | | | | | | | | | | | | | This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-083-5/+10
| |\ \ | | | | | | | | | | | | neilj/reserved_users
| | * \ Merge pull request #3633 from matrix-org/neilj/mau_trackerNeil Johnson2018-08-085-28/+195
| | |\ \ | | | | | | | | | | API for monthly_active_users table
| | | * | typosNeil Johnson2018-08-082-3/+3
| | | | |
| | * | | Fix occasional glitches in the synapse_event_persisted_position metricRichard van der Hoff2018-08-071-3/+8
| | | |/ | | |/| | | | | | | | | | | | | Every so often this metric glitched to a negative number. I'm assuming it was due to backfilled events.
| * | | prevent total number of reserved users being too largeNeil Johnson2018-08-071-1/+3
| | | |
| * | | implement reserved users for mau limitsNeil Johnson2018-08-071-11/+34
| | | |
| * | | WIP building out mau reserved usersNeil Johnson2018-08-061-1/+15
| | |/ | |/|
| * | Fix postgres compatibility bugNeil Johnson2018-08-061-4/+15
| | |
| * | make use of _simple_select_one_onecol, improved commentsNeil Johnson2018-08-062-8/+15
| | |
| * | bug fixesNeil Johnson2018-08-031-2/+1
| | |
| * | wip commit - tests failingNeil Johnson2018-08-032-38/+49
| | |
| * | fix caching and testsNeil Johnson2018-08-031-48/+43
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_trackerNeil Johnson2018-08-033-8/+27
| |\|
| * | typoNeil Johnson2018-08-021-1/+1
| | |
| * | wip attempt at cachingNeil Johnson2018-08-021-9/+47
| | |
| * | do mau checks based on monthly_active_users tableNeil Johnson2018-08-021-7/+8
| | |
| * | remove unused count_monthly_usersNeil Johnson2018-08-021-25/+0
| | |
| * | insertion into monthly_active_usersNeil Johnson2018-08-023-7/+35
| | |
| * | Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-022-1/+3
| |\ \ | | | | | | | | | | | | neilj/mau_tracker
| | * | self.db_conn unusedNeil Johnson2018-08-011-1/+0
| | | |
| | * | normalise reaping queryNeil Johnson2018-08-011-3/+38
| | | |
| * | | Revert "change monthly_active_users table to be a single column"Neil Johnson2018-08-022-4/+10
| |/ / | | | | | | | | | This reverts commit ec716a35b219d147dee51733b55573952799a549.
| * | change monthly_active_users table to be a single columnNeil Johnson2018-08-012-10/+4
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_trackerNeil Johnson2018-08-017-11/+47
| |\ \
| * | | clean upNeil Johnson2018-08-011-15/+3
| | | |
| * | | fix commentNeil Johnson2018-08-011-1/+1
| | | |
| * | | api into monthly_active_users tableNeil Johnson2018-07-314-2/+114
| | | |
* | | | Pull in necessary stores in federation_readerErik Johnston2018-08-061-1/+0
| | | |
* | | | Move necessary storage functions to worker classesErik Johnston2018-08-063-98/+100
| |_|/ |/| |
* | | Merge branch 'master' into developRichard van der Hoff2018-08-022-6/+15
|\ \ \
| * | | Avoid extra db lookupsRichard van der Hoff2018-08-022-35/+15
| | | | | | | | | | | | | | | | | | | | Since we're about to look up the events themselves anyway, we can skip the extra db queries here.
| * | | Validation for events/rooms in fed requestsRichard van der Hoff2018-08-021-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When we get a federation request which refers to an event id, make sure that said event is in the room the caller claims it is in. (patch supplied by @turt2live)
* | | | Merge pull request #3621 from matrix-org/erikj/split_fed_storeErik Johnston2018-08-021-2/+12
|\ \ \ \ | |_|_|/ |/| | | Split out DB writes in federation handler
| * | | update docsErik Johnston2018-08-011-1/+1
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-3011-67/+225
| |\ \ \ | | | |/ | | |/| | | | | erikj/split_fed_store
| * | | Split out DB writes in federation handlerErik Johnston2018-07-251-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | This will allow us to easily add an internal replication API to proxy these reqeusts to master, so that we can move federation APIs to workers.
* | | | Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limitsNeil Johnson2018-08-012-1/+27
|\ \ \ \ | | | | | | | | | | Initial impl of capping MAU
| * | | | improve clarityNeil Johnson2018-08-011-4/+5
| | | | |
| * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-016-6/+10
| |\ \ \ \ | | | | | | | | | | | | | | | | | | neilj/mau_sign_in_log_in_limits
| * | | | | remove errant printNeil Johnson2018-08-011-1/+0
| | | | | |
| * | | | | make count_monthly_users async synapse/handlers/auth.pyNeil Johnson2018-08-011-13/+13
| | | | | |
| * | | | | coding styleNeil Johnson2018-07-312-3/+2
| | | | | |
| * | | | | actually close connNeil Johnson2018-07-301-4/+8
| | | | | |
| * | | | | factor out metrics from __init__ to app/homeserverNeil Johnson2018-07-301-24/+13
| | | | | |
| * | | | | limit register and sign in on number of monthly usersNeil Johnson2018-07-301-0/+34
| | |_|/ / | |/| | |
* | | | | Python 3: Convert some unicode/bytes uses (#3569)Amber Brown2018-08-022-5/+11
| |/ / / |/| | |
* | | | Merge pull request #3612 from matrix-org/rav/store_heirarchyRichard van der Hoff2018-07-316-6/+10
|\ \ \ \ | |/ / / |/| | | Make EventStore inherit from EventFederationStore
| * | | reinstate explicit include of EventsWorkerStoreRichard van der Hoff2018-07-311-1/+3
| | | |
| * | | Make EventStore inherit from EventFederationStoreRichard van der Hoff2018-07-266-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (since it uses methods therein) Turns out that we had a bunch of things which were incorrectly importing EventWorkerStore from events.py rather than events_worker.py, which broke once I removed the import into events.py.
* | | | make /context lazyload & filter aware (#3567)Matthew Hodgson2018-07-271-3/+11
| | | | | | | | | | | | make /context lazyload & filter aware.
* | | | Merge pull request #3614 from matrix-org/rav/stop_populating_event_contentRichard van der Hoff2018-07-263-2/+98
|\ \ \ \ | | | | | | | | | | Stop populating events.content
| * | | | Create the column nullableRichard van der Hoff2018-07-262-9/+13
| | | | | | | | | | | | | | | | | | | | | | | | | There's no real point in ever making the column non-nullable, and doing so breaks the sytests.
| * | | | Stop populating events.contentRichard van der Hoff2018-07-262-1/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This field is no longer read from, so we should stop populating it. Once we're happy that this doesn't break everything, and a rollback is unlikely, we can think about dropping the column.
* | | | | Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_idRichard van der Hoff2018-07-262-8/+6
|\ \ \ \ \ | | | | | | | | | | | | Remove some redundant joins on event_edges.room_id
| * | | | | Remove some redundant joins on event_edges.room_idRichard van der Hoff2018-07-262-8/+6
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've long passed the point where it's possible to have the same event_id in different tables, so these join conditions are redundant: we can just join on event_id. event_edges is of non-trivial size, and the room_id column is wasteful, so let's stop reading from it. In future, we can stop writing to it, and then drop it.
* | | | | Merge pull request #3610 from matrix-org/rav/fix_looping_callsRichard van der Hoff2018-07-265-6/+8
|\ \ \ \ \ | |/ / / / |/| | | | Fix some looping_call calls which were broken in #3604
| * | | | Fix some looping_call calls which were broken in #3604Richard van der Hoff2018-07-265-6/+8
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.
* / / / comment on event_edgesRichard van der Hoff2018-07-261-1/+2
|/ / /
* | | switch missing_types to be a boolMatthew Hodgson2018-07-251-4/+4
| | |
* | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-258-125/+187
|\| |
| * | Merge pull request #3603 from matrix-org/erikj/handle_outliersErik Johnston2018-07-251-6/+8
| |\ \ | | | | | | | | Correctly handle outliers during persist events
| | * | Actually fix it by adding continueErik Johnston2018-07-251-0/+1
| | | |
| | * | Fix typo in conditionalErik Johnston2018-07-251-1/+1
| | | |
| | * | Correctly handle outliers during persist eventsErik Johnston2018-07-251-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We incorrectly asserted that all contexts must have a non None state group without consider outliers. This would usually be fine as the assertion would never be hit, as there is a shortcut during persistence if the forward extremities don't change. However, if the outlier is being persisted with non-outlier events, the function would be called and the assertion would be hit. Fixes #3601
| * | | Merge pull request #3606 from matrix-org/rav/logcontext_fixes_once_moreRichard van der Hoff2018-07-252-5/+10
| |\ \ \ | | | | | | | | | | Fix another logcontext leak in _persist_events
| | * | | Fix another logcontext leak in _persist_eventsRichard van der Hoff2018-07-252-5/+10
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | We need to run the errback in the sentinel context to avoid losing our own context. Also: add logging to runInteraction to help identify where "Starting db connection from sentinel context" warnings are coming from
| * | | Merge pull request #3607 from matrix-org/rav/fix_persist_events_integrity_errorRichard van der Hoff2018-07-251-1/+1
| |\ \ \ | | | | | | | | | | Fix occasional 'tuple index out of range' error
| | * | | Fix occasional 'tuple index out of range' errorRichard van der Hoff2018-07-251-1/+1
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | This fixes a bug in _delete_existing_rows_txn which was introduced in #3435 (though it's been on matrix-org-hotfixes for *years*). This code is only called when there is some sort of conflict the first time we try to persist an event, so it only happens rarely. Still, the exceptions are annoying.
| * / / Wrap a number of things that run in the backgroundRichard van der Hoff2018-07-254-10/+26
| |/ / | | | | | | | | | | | | This will reduce the number of "Starting db connection from sentinel context" warnings, and will help with our metrics.
| * | Pull out did_forget to worker storeErik Johnston2018-07-241-25/+27
| | |
| * | Merge pull request #3595 from matrix-org/erikj/use_deltasErik Johnston2018-07-241-53/+118
| |\ \ | | | | | | | | Use deltas to calculate current state deltas
| | * | Expand on docstring comment about return valueErik Johnston2018-07-241-0/+5
| | | |
| | * | Remove unnecessary iteritemsErik Johnston2018-07-241-1/+1
| | | |
| | * | Fixup comment (and indent)Erik Johnston2018-07-241-16/+20
| | | |
| | * | Don't fetch state from the database unless neededErik Johnston2018-07-241-12/+18
| | | |
| | * | Have _get_new_state_after_events return deltaErik Johnston2018-07-241-17/+48
| | | | | | | | | | | | | | | | | | | | | | | | If we have a delta from the existing to new current state, then we can reuse that rather than manually working it out by fetching both lots of state.
| | * | Don't require to_delete to have event_idsErik Johnston2018-07-241-36/+55
| | | |
| * | | Merge pull request #3583 from matrix-org/rav/remove_who_forgot_in_roomRichard van der Hoff2018-07-242-28/+0
| |\ \ \ | | |/ / | |/| | Remove redundant checks on room forgottenness
| | * | Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_roomRichard van der Hoff2018-07-242-15/+31
| | |\ \
| | * \ \ Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_roomRichard van der Hoff2018-07-235-26/+70
| | |\ \ \
| | * | | | Remove redundant checks on room forgottennessRichard van der Hoff2018-07-232-28/+0
| | | | | | | | | | | | | | | | | | | | | | | | Fixes #3550
* | | | | | incorporate more review.Matthew Hodgson2018-07-251-16/+14
| | | | | |
* | | | | | handle the edge case for _get_some_state_from_cache where types is []Matthew Hodgson2018-07-241-1/+7
| | | | | |
* | | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-241-12/+4
|\| | | | |
| * | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-242-13/+24
| |\ \ \ \ \ | | | |_|/ / | | |/| | | | | | | | | erikj/speed_up_calculate_state_delta
| * | | | | Remove unnecessary ifErik Johnston2018-07-241-2/+1
| | | | | |
| * | | | | Speed up _calculate_state_deltaErik Johnston2018-07-241-10/+3
| | | | | |
* | | | | | consider non-filter_type types as wildcards, thus missing from the ↵Matthew Hodgson2018-07-241-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | state-group-cache
* | | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-242-14/+25
|\ \ \ \ \ \ | | |/ / / / | |/| | | |
| * | | | | Merge branch 'develop' into rav/logcontext_fixesRichard van der Hoff2018-07-241-11/+21
| |\ \ \ \ \
| | * | | | | fix idiocyRichard van der Hoff2018-07-241-2/+8
| | | | | | |
| | * | | | | Add some measure blocks to persist_eventsRichard van der Hoff2018-07-231-11/+15
| | |/ / / / | | | | | | | | | | | | | | | | | | ... to help us figure out where 40% of CPU is going
| * / / / / Logcontext fixesRichard van der Hoff2018-07-242-3/+4
| |/ / / / | | | | | | | | | | | | | | | Fix some random logcontext leaks.
* | | | | incorporate more reviewMatthew Hodgson2018-07-241-27/+9
| | | | |
* | | | | handle case where types is [] on postgres correctlyMatthew Hodgson2018-07-231-1/+1
| | | | |
* | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-235-26/+75
|\| | | |
| * | | | CommentsErik Johnston2018-07-231-0/+3
| | | | |
| * | | | Only get cached state from context in persist_eventErik Johnston2018-07-231-1/+3
| | |/ / | |/| | | | | | | | | | | | | | | | | | We don't want to bother pulling out the current state from the DB since until we know we have to. Checking the context for state is just an optimisation.
| * | | Use new gettersErik Johnston2018-07-233-5/+11
| | | |
| * | | Merge branch 'develop' into matthew/sync_deleted_devicesMatthew Hodgson2018-07-236-23/+49
| |\| |
| * | | add trailing commaMatthew Hodgson2018-07-231-1/+1
| | | |
| * | | yes, we do need to invalidate the device_id_exists_cache when deleting a ↵Matthew Hodgson2018-07-191-1/+0
| | | | | | | | | | | | | | | | remote device
| * | | spell out that include_deleted_devices requires include_all_devicesMatthew Hodgson2018-07-191-1/+5
| | | |
| * | | shift to using an explicit deleted flag on m.device_list_update EDUsMatthew Hodgson2018-07-122-20/+25
| | | | | | | | | | | | | | | | and generally make it work.
| * | | WIP to announce deleted devices over federationMatthew Hodgson2018-07-122-13/+43
| | | | | | | | | | | | | | | | | | | | | | | | Previously we queued up the poke correctly when the device was deleted, but then the actual EDU wouldn't get sent, as the device was no longer known. Instead, we now send EDUs for deleted devices too if there's a poke for them.
* | | | incorporate reviewMatthew Hodgson2018-07-231-12/+8
| | | |
* | | | fix thinkos; unbreak testsMatthew Hodgson2018-07-191-2/+2
| | | |
* | | | incorporate reviewMatthew Hodgson2018-07-191-3/+4
| | | |
* | | | add a filtered_types param to limit filtering to specific typesMatthew Hodgson2018-07-191-53/+60
| | | |
* | | | merge develop pydoc for _get_state_for_groupsMatthew Hodgson2018-07-1950-375/+550
|\ \ \ \ | | |/ / | |/| |
| * | | revert 00bc979Richard van der Hoff2018-07-191-4/+3
| | | | | | | | | | | | | | | | | | | | ... we've fixed the things that caused the warnings, so we should reinstate the warning.
| * | | Merge remote-tracking branch 'origin/release-v0.33.0' into developRichard van der Hoff2018-07-191-3/+4
| |\ \ \ | | | |/ | | |/|
| | * | Disable logcontext warningRichard van der Hoff2018-07-191-3/+4
| | | | | | | | | | | | | | | | Temporary workaround to #3518 while we release 0.33.0.
| * | | Run things as background processesRichard van der Hoff2018-07-184-16/+29
| |/ / | | | | | | | | | | | | | | | | | | | | | This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).
| * | Merge pull request #3505 from matrix-org/erikj/receipts_cahceErik Johnston2018-07-121-6/+19
| |\ \ | | |/ | |/| Use stream cache in get_linearized_receipts_for_room
| | * Update return value docstringErik Johnston2018-07-121-1/+1
| | |
| | * Use 'is not None' and add commentsErik Johnston2018-07-101-2/+6
| | |
| | * Use stream cache in get_linearized_receipts_for_roomErik Johnston2018-07-101-4/+13
| | | | | | | | | | | | | | | This avoids us from uncessarily hitting the database when there has been no change for the room
| * | s/becuase/because/gMatthew Hodgson2018-07-101-2/+2
| |/
| * Attempt to include db threads in cpu usage stats (#3496)Richard van der Hoff2018-07-102-18/+17
| | | | | | | | | | Let's try to include time spent in the DB threads in the per-request/block cpu usage metrics.
| * Add CPU metrics for _fetch_event_listRichard van der Hoff2018-07-091-19/+32
| | | | | | | | | | add a Measure block on _fetch_event_list, in the hope that we can better measure CPU usage here.
| * run isortAmber Brown2018-07-0949-246/+244
| |
| * Invalidate cache on correct threadErik Johnston2018-07-021-1/+2
| |
| * Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-2819-28/+35
| |
| * Fix error on deleting users pending deactivationDavid Baker2018-06-261-1/+3
| | | | | | | | Use simple_delete instead of simple_delete_one as commented
| * Revert "Revert "Merge pull request #3431 from ↵Erik Johnston2018-06-253-0/+126
| | | | | | | | | | | | matrix-org/rav/erasure_visibility"" This reverts commit 1d009013b3c3e814177afc59f066e02a202b21cd.
| * Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"Richard van der Hoff2018-06-223-126/+0
| | | | | | | | | | This reverts commit ce0d911156b355c5bf452120bfb08653dad96497, reversing changes made to b4a5d767a94f1680d07edfd583aae54ce422573e.
| * Deleting from event_push_actions needs to use an indexMark Haines2018-06-221-1/+8
| |
| * Merge pull request #3432 from matrix-org/rav/joined_hosts_cache_non_iterableRichard van der Hoff2018-06-221-1/+1
| |\ | | | | | | Make _get_joined_hosts_cache cache non-iterable
| | * Make _get_joined_hosts_cache cache non-iterableErik Johnston2018-06-221-1/+1
| | |
| * | Merge pull request #3430 from matrix-org/rav/configurable_push_action_rotationRichard van der Hoff2018-06-221-3/+5
| |\ \ | | |/ | |/| Make push actions rotation configurable
| | * Make push actions rotation configurableErik Johnston2018-06-221-3/+5
| | |
| * | Merge pull request #3431 from matrix-org/rav/erasure_visibilityErik Johnston2018-06-223-0/+126
| |\ \ | | | | | | | | Support hiding events from deleted users
| | * | UserErasureStoreRichard van der Hoff2018-06-123-0/+126
| | | | | | | | | | | | | | | | to store which users have been erased
| * | | Merge pull request #3427 from matrix-org/erikj/remove_filtersErik Johnston2018-06-221-20/+1
| |\| | | | |/ | |/| remove dead filter_events_for_clients
| | * remove dead filter_events_for_clientsRichard van der Hoff2018-06-121-20/+1
| | | | | | | | | | | | | | | | | | This is only used by filter_events_for_client, so we can simplify the whole thing by just doing one user at a time, and removing a dead storage function to boot.
| * | Disable partial state group caching for wildcard lookupsRichard van der Hoff2018-06-221-13/+43
| | | | | | | | | | | | | | | | | | | | | When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase.
| * | Merge pull request #3382 from matrix-org/rav/optimise_state_groupsRichard van der Hoff2018-06-221-5/+1
| |\ \ | | | | | | | | Optimise state_group_cache update
| | * | Optimise state_group_cache updateRichard van der Hoff2018-06-111-5/+1
| | | | | | | | | | | | | | | | | | | | (1) matrix-org-hotfixes has removed the intern calls; let's do the same here. (2) remove redundant iteritems() so we can used an optimised db update.
| * | | Merge pull request #3419 from matrix-org/rav/events_per_requestRichard van der Hoff2018-06-221-0/+4
| |\ \ \ | | | | | | | | | | Log number of events fetched from DB
| | * | | Indirect evt_count updates via method callRichard van der Hoff2018-06-221-1/+1
| | | | | | | | | | | | | | | | | | | | so that we can stub it for the sentinel and not have a billion failing UTs
| | * | | Log number of events fetched from DBRichard van der Hoff2018-06-211-0/+4
| | | |/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we finish processing a request, log the number of events we fetched from the database to handle it. [I'm trying to figure out which requests are responsible for large amounts of event cache churn. It may turn out to be more helpful to add counts to the prometheus per-request/block metrics, but that is an extension to this code anyway.]
| * / | Pass around the reactor explicitly (#3385)Amber Brown2018-06-224-9/+9
| |/ /
| * | Merge pull request #3276 from matrix-org/dbkr/unbindDavid Baker2018-06-111-9/+0
| |\ \ | | | | | | | | Remove email addresses / phone numbers from ID servers when they're removed from synapse
| | * \ Merge remote-tracking branch 'origin/develop' into dbkr/unbindDavid Baker2018-05-241-9/+6
| | |\ \
| | * | | Hit the 3pid unbind endpoint on deactivationDavid Baker2018-05-231-9/+0
| | | | |
* | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-06-102-29/+3
|\| | | |
| * | | | Fix sql error in _get_state_groups_from_groupsRichard van der Hoff2018-06-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | If this was called with a `(type, None)` entry in types (which is supposed to return all state of type `type`), it would explode with a sql error.
| * | | | Merge pull request #3324 from matrix-org/rav/remove_dead_methodRichard van der Hoff2018-06-041-26/+0
| |\ \ \ \ | | | | | | | | | | | | Remove was_forgotten_at
| | * | | | Remove was_forgotten_atRichard van der Hoff2018-06-011-26/+0
| | | |_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | This is unused. IT MUST DIE!!!1 ̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢ ̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
| * / | | Add hacky cache factor override systemErik Johnston2018-06-041-2/+2
| |/ / /
* | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-06-0412-92/+179
|\| | |
| * | | Merge pull request #3264 from matrix-org/neil/sign-up-statsNeil Johnson2018-06-013-1/+56
| |\ \ \ | | | | | | | | | | daily user type phone home stats
| | * | | remove unnecessary INSERTNeil Johnson2018-06-011-1/+0
| | | | |
| | * | | create users index on creation_tsNeil Johnson2018-05-291-0/+20
| | | | |
| | * | | create index in backgroundNeil Johnson2018-05-291-0/+7
| | | | |
| | * | | bump versionNeil Johnson2018-05-291-1/+1
| | | | |
| | * | | daily user type phone home statsNeil Johnson2018-05-221-0/+29
| | |/ /
| * | | Handle RRs which arrive before their eventsRichard van der Hoff2018-06-011-25/+19
| | | |
| * | | Ignore depth when updating read-receiptsRichard van der Hoff2018-06-011-30/+37
| | | | | | | | | | | | | | | | Order read receipts by stream ordering instead of depth
| * | | Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-315-54/+59
| | | | | | | | | | | | | | | | they're not meant to be lazy (#3307)
| * | | Merge pull request #3303 from NotAFile/py3-memoryviewAmber Brown2018-05-303-6/+30
| |\ \ \ | | | | | | | | | | use memoryview in py3
| | * | | add remaining memoryview changesAdrian Tschira2018-05-292-4/+12
| | | | |
| | * | | use memoryview in py3Adrian Tschira2018-05-292-2/+18
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * | | | Merge pull request #3281 from NotAFile/py3-six-isinstanceAmber Brown2018-05-301-2/+3
| |\ \ \ \ | | | | | | | | | | | | remaining isintance fixes
| | * | | | remaining isintance fixesAdrian Tschira2018-05-241-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * | | | | Exempt AS-registered users from doing gdprRichard van der Hoff2018-05-291-0/+1
| | |/ / / | |/| | |
* | | | | add pydocMatthew Hodgson2018-05-291-22/+54
| | | | |
* | | | | merge developMatthew Hodgson2018-05-2940-519/+1165
|\| | | |
| * | | | update metrics to be in secondsAmber Brown2018-05-281-15/+15
| | | | |
| * | | | Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-2810-43/+60
| |\ \ \ \
| | * \ \ \ Merge pull request #3279 from NotAFile/py3-more-iteritemsAmber Brown2018-05-241-4/+6
| | |\ \ \ \ | | | | | | | | | | | | | | more six iteritems
| | | * | | | more iteritemsAdrian Tschira2018-05-241-4/+6
| | | |/ / /
| | * / / / Misc. py3 fixesAdrian Tschira2018-05-243-3/+3
| | |/ / / | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| | * | | Merge pull request #3278 from NotAFile/py3-storage-baseAmber Brown2018-05-241-17/+22
| | |\ \ \ | | | | | | | | | | | | Py3 storage/_base.py
| | | * | | Py3 storage/_base.pyAdrian Tschira2018-05-241-17/+22
| | | | |/ | | | |/| | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| | * | | Merge pull request #3244 from NotAFile/py3-six-4Amber Brown2018-05-244-9/+16
| | |\ \ \ | | | | | | | | | | | | replace some iteritems with six
| | | * | | replace some iteritems with sixAdrian Tschira2018-05-194-9/+16
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| | * | | | Merge pull request #3247 from NotAFile/py3-miscAmber Brown2018-05-241-1/+7
| | |\ \ \ \ | | | |_|/ / | | |/| | | Misc Python3 fixes
| | | * | | use stand-in value if maxint is not availableAdrian Tschira2018-05-191-1/+7
| | | |/ / | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| | * | | Remove unused string formatting paramErik Johnston2018-05-221-1/+0
| | | | |
| | * | | Don't support limitless paginationErik Johnston2018-05-221-8/+6
| | | |/ | | |/| | | | | | | | | | | | | | | | | | | | | The pagination storage function supported not specifiying a limit on the number of events returned. This was triggered when using the search or context API with a limit of zero, which the storage function took to mean not being limited.
| * | | cleanupAmber Brown2018-05-221-3/+6
| | | |
| * | | Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-2211-75/+286
| |\| |
| | * | Send users a server notice about consentRichard van der Hoff2018-05-222-7/+59
| | | | | | | | | | | | | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.
| | * | Tighter filtering for user_daily_visitsNeil Johnson2018-05-181-1/+5
| | |/
| | * fix psql compatability bugNeil Johnson2018-05-181-1/+1
| | |
| | * Merge pull request #3235 from matrix-org/rav/fix_receipts_deferredRichard van der Hoff2018-05-181-7/+11
| | |\ | | | | | | | | Fix error in handling receipts
| | | * Fix error in handling receiptsRichard van der Hoff2018-05-181-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes an error which has been happening ever since #2158 (v0.21.0-rc1): > TypeError: argument of type 'ObservableDeferred' is not iterable fixes #3234
| | * | Merge pull request #3221 from matrix-org/erikj/purge_tokenErik Johnston2018-05-181-24/+53
| | |\ \ | | | |/ | | |/| Make purge_history operate on tokens
| | | * Clarify commentErik Johnston2018-05-161-1/+2
| | | |
| | | * CommentsErik Johnston2018-05-161-0/+8
| | | |
| | | * Move and rename variableErik Johnston2018-05-161-3/+3
| | | |
| | | * Make purge_history operate on tokensErik Johnston2018-05-151-8/+9
| | | | | | | | | | | | | | | | As we're soon going to change how topological_ordering works
| | | * Use events_to_purge table rather than tokenErik Johnston2018-05-151-15/+34
| | | |
| | * | Merge pull request #3212 from matrix-org/erikj/epa_streamErik Johnston2018-05-172-38/+16
| | |\ \ | | | | | | | | | | Use stream rather depth ordering for push actions
| | | * | Use stream rather depth ordering for push actionsErik Johnston2018-05-112-38/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This simplifies things as it is, but will also allow us to change the way we traverse topologically without having to update the way push actions work.
| | * | | Merge pull request #3163 from matrix-org/cohort_analyticsNeil Johnson2018-05-165-5/+113
| | |\ \ \ | | | | | | | | | | | | user visit data
| | | * | | remove empty lineNeil Johnson2018-05-161-1/+0
| | | | | |
| | | * | | style and further contraining queryNeil Johnson2018-05-161-8/+12
| | | | | |
| | | * | | Limit query load of generate_user_daily_visitsNeil Johnson2018-05-151-13/+47
| | | | | | | | | | | | | | | | | | | | | | | | The aim is to keep track of when it was last called and only query from that point in time
| | | * | | instead of inserting user daily visit data at the end of the day, instead ↵Neil Johnson2018-05-141-40/+14
| | | | | | | | | | | | | | | | | | | | | | | | insert incrementally through the day
| | | * | | Merge branch 'develop' of https://github.com/matrix-org/synapse into ↵Neil Johnson2018-05-1410-290/+380
| | | |\| | | | | | | | | | | | | | | | | | | | cohort_analytics
| | | * | | remove user agent from data model, will just join on user_ipsNeil Johnson2018-05-011-12/+11
| | | | | |
| | | * | | add inidexes based on usageNeil Johnson2018-05-011-6/+2
| | | | | |
| | | * | | pep8Neil Johnson2018-04-251-3/+4
| | | | | |
| | | * | | Generate user daily statsNeil Johnson2018-04-255-5/+106
| | | | | |
| | * | | | Merge pull request #3213 from matrix-org/rav/consent_handlerRichard van der Hoff2018-05-162-0/+36
| | |\ \ \ \ | | | |_|_|/ | | |/| | | ConsentResource to gather policy consent from users
| | | * | | ConsentResource to gather policy consent from usersRichard van der Hoff2018-05-152-0/+36
| | | | |/ | | | |/| | | | | | | | | | | | | | | | Hopefully there are enough comments and docs in this that it makes sense on its own.
| * | / | rest of the changesAmber Brown2018-05-212-27/+18
| |/ / /
| * | | Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivateDavid Baker2018-05-142-0/+64
| |\ \ \ | | |/ / | |/| | Part user from rooms on account deactivate
| | * | Many docstringsDavid Baker2018-05-101-0/+12
| | | |
| | * | Add the schema fileDavid Baker2018-05-091-0/+25
| | | |
| | * | Part deactivated users in the backgroundDavid Baker2018-05-091-0/+27
| | | | | | | | | | | | | | | | | | | | One room at a time so we don't take out the whole server with leave events, and restart at server restart.
| * | | Merge pull request #3199 from matrix-org/erikj/pagination_syncErik Johnston2018-05-091-39/+36
| |\ \ \ | | | | | | | | | | Refactor sync APIs to reuse pagination API
| | * | | Refactor sync APIs to reuse pagination APIErik Johnston2018-05-091-38/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sync API often returns events in a topological rather than stream ordering, e.g. when the user joined the room or on initial sync. When this happens we can reuse existing pagination storage functions.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-21 00:16:46 +0000