summary refs log tree commit diff
path: root/synapse/storage/roommember.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Run Black. (#5482)Amber Brown2019-06-201-4/+4
|
* Fetch membership counts all at onceErik Johnston2019-05-301-22/+11
|
* Room Statistics (#4338)Amber Brown2019-05-211-0/+32
|
* Run `black` on some storage modules that the stats branch touches (#4959)Amber Brown2019-03-291-86/+87
|
* Use term 'out of band membership' insteadErik Johnston2019-01-241-3/+3
|
* Clarify the invite flowsErik Johnston2019-01-231-4/+7
|
* Store rejected remote invite events as outliersErik Johnston2019-01-231-4/+1
| | | | | | | | | | | Currently they're stored as non-outliers even though the server isn't in the room, which can be problematic in places where the code assumes it has the state for all non outlier events. In particular, there is an edge case where persisting the leave event triggers a state resolution, which requires looking up the room version from state. Since the server doesn't have the state, this causes an exception to be thrown.
* Speed up lazy loading (#3827)Matthew Hodgson2018-09-121-0/+65
| | | | | * speed up room summaries by pulling their data from room_memberships rather than room state * disable LL for incr syncs, and log incr sync stats (#3840)
* Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-101-1/+1
|
* Make EventStore inherit from EventFederationStoreRichard van der Hoff2018-07-261-1/+1
| | | | | | | | (since it uses methods therein) Turns out that we had a bunch of things which were incorrectly importing EventWorkerStore from events.py rather than events_worker.py, which broke once I removed the import into events.py.
* Pull out did_forget to worker storeErik Johnston2018-07-241-25/+27
|
* Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_roomRichard van der Hoff2018-07-231-2/+5
|\
| * Use new gettersErik Johnston2018-07-231-2/+5
| |
* | Remove redundant checks on room forgottennessRichard van der Hoff2018-07-231-15/+0
|/ | | | Fixes #3550
* run isortAmber Brown2018-07-091-10/+9
|
* Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-281-1/+1
|
* Make _get_joined_hosts_cache cache non-iterableErik Johnston2018-06-221-1/+1
|
* Remove was_forgotten_atRichard van der Hoff2018-06-011-26/+0
| | | | | | This is unused. IT MUST DIE!!!1 ̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢ ̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
* more iteritemsAdrian Tschira2018-05-241-4/+6
|
* Remove uses of events.contentRichard van der Hoff2018-03-291-2/+4
|
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-191-1/+1
|\
| * Replace ujson with simplejsonErik Johnston2018-03-151-1/+1
| |
* | Fix race in sync when joining roomErik Johnston2018-03-071-1/+26
| | | | | | | | | | | | | | | | | | | | | | The race happens when the user joins a room at the same time as doing a sync. We fetch the current token and then get the rooms the user is in. If the join happens after the current token, but before we get the rooms we end up sending down a partial room entry in the sync. This is fixed by looking at the stream ordering of the membership returned by get_rooms_for_user, and handling the case when that stream ordering is after the current token.
* | Move storage functions for push calculationsErik Johnston2018-02-271-12/+12
| | | | | | | | This will allow push actions for an event to be calculated on workers.
* | Update copyrightErik Johnston2018-02-231-0/+1
| |
* | Split out RoomMemberStoreErik Johnston2018-02-211-179/+181
|/
* Make __init__ consitstent across Store heirarchyRichard van der Hoff2017-11-131-2/+2
| | | | | | Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-231-1/+1
| | | | what could possibly go wrong
* Fix up commentErik Johnston2017-10-091-1/+2
|
* Update commentsErik Johnston2017-10-031-0/+7
|
* Ignore incoming events for rooms that we have leftErik Johnston2017-10-031-0/+32
| | | | | | | | | | | | | | | | | When synapse receives an event for a room its not in over federation, it double checks with the remote server to see if it is in fact in the room. This is done so that if the server has forgotten about the room (usually as a result of the database being dropped) it can recover from it. However, in the presence of state resets in large rooms, this can cause a lot of work for servers that have legitimately left. As a hacky solution that supports both cases we drop incoming events for rooms that we have explicitly left. This means that we no longer support the case of servers having forgotten that they've rejoined a room, but that is sufficiently rare that we're not going to support it for now.
* Add cache for is_host_joinedErik Johnston2017-06-131-1/+1
|
* Tweak SQLErik Johnston2017-06-091-3/+6
|
* Change is_host_joined to use current_state tableErik Johnston2017-06-091-29/+23
| | | | | | This bypasses a bug where using the state groups to figure out if a host is in a room sometimes errors if the servers isn't in the room. (For example when the server rejected an invite to a remote room)
* CommentsErik Johnston2017-06-071-0/+9
|
* Remove spurious log linesErik Johnston2017-06-071-1/+0
|
* Handle None state group correctlyErik Johnston2017-05-261-1/+4
|
* Faster cache for get_joined_hostsErik Johnston2017-05-251-21/+72
|
* CommentErik Johnston2017-05-171-0/+5
|
* Short circuit when we have delta idsErik Johnston2017-05-171-3/+19
|
* Speed up get_joined_hostsErik Johnston2017-05-161-1/+2
|
* Don't update event cache hit ratio from get_joined_usersErik Johnston2017-05-081-0/+4
| | | | | Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.
* Merge pull request #2176 from matrix-org/erikj/faster_get_joinedErik Johnston2017-05-051-16/+42
|\ | | | | Make get_joined_users faster
| * FixesErik Johnston2017-05-051-3/+4
| |
| * Add commentErik Johnston2017-05-041-0/+3
| |
| * Make get_joined_users fasterErik Johnston2017-05-031-16/+38
| |
* | Merge pull request #2185 from matrix-org/erikj/smaller_cachesErik Johnston2017-05-051-2/+2
|\ \ | | | | | | Optimise caches for single key
| * | Reduce size of get_users_in_roomErik Johnston2017-05-041-2/+2
| |/
* | Intern host stringsErik Johnston2017-05-041-1/+2
| |
* | Make caches biggerErik Johnston2017-05-041-1/+1
| |
* | Fix comment and num argsErik Johnston2017-05-041-1/+1
| |
* | Add cache for get_current_hosts_in_roomErik Johnston2017-05-021-0/+38
|/
* CommentErik Johnston2017-04-251-0/+2
|
* Reduce size of joined_user cacheErik Johnston2017-04-251-8/+14
| | | | | | | | The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.
* Cache hosts in roomErik Johnston2017-03-241-0/+10
|
* Replace some calls to cursor_to_dictErik Johnston2017-03-241-34/+9
| | | | | cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.
* CommentsErik Johnston2017-03-161-0/+2
|
* Don't recreate so many setsErik Johnston2017-03-161-5/+6
|
* Bump cache sizes for common membership queriesErik Johnston2017-02-021-3/+3
|
* CommentErik Johnston2017-02-021-1/+2
|
* Make presence.get_new_events a bit fasterErik Johnston2017-02-021-0/+16
| | | | We do this by caching the set of users a user shares rooms with.
* Merge pull request #1868 from matrix-org/erikj/replication_cacheErik Johnston2017-02-011-2/+0
|\ | | | | Only invalidate membership caches based on the cache stream
| * Only invalidate membership caches based on the cache streamErik Johnston2017-01-311-2/+0
| | | | | | | | | | Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.
* | Add m.room.member type to queryErik Johnston2017-02-011-1/+1
| |
* | Add an index to make membership queries fasterErik Johnston2017-01-311-1/+1
|/
* Don't have such a large cacheErik Johnston2017-01-301-1/+1
|
* Use get_users_in_room and declare it iterableErik Johnston2017-01-301-1/+1
|
* Up cache max entries for stateErik Johnston2017-01-161-1/+1
|
* Increase cache size limitErik Johnston2017-01-161-1/+1
|
* Optionally measure size of cache by sum of length of valuesErik Johnston2017-01-131-1/+2
|
* fix annoying typosMatthew Hodgson2017-01-051-2/+2
|
* Fixup membership queryErik Johnston2016-12-141-2/+2
|
* Fix background update that prematurely stoppedErik Johnston2016-12-121-1/+1
|
* Add /room/<room_id>/joined_members APIErik Johnston2016-12-081-3/+13
| | | | | | This returns the currently joined members in the room with their display names and avatar urls. This is more efficient than /members for large rooms where you don't need the full events.
* Add profile data to the room_membership table for joinsErik Johnston2016-12-081-0/+86
|
* Reduce batch size to be under SQL limitErik Johnston2016-09-091-1/+1
|
* Correctly handle the difference between prev and current stateErik Johnston2016-08-311-4/+23
|
* Use state handler instead of get_users_in_room/get_joined_hostsErik Johnston2016-08-261-9/+2
|
* Add is_host_joined to slave storageErik Johnston2016-08-261-3/+4
|
* Cache check_host_in_roomErik Johnston2016-08-261-0/+35
|
* Add descErik Johnston2016-08-251-1/+2
|
* Replace context.current_state with context.current_state_idsErik Johnston2016-08-251-2/+43
|
* Add some invalidations to a cache_streamErik Johnston2016-08-151-5/+7
|
* Remove dead code.Mark Haines2016-06-071-7/+0
| | | | | | | | | | | Loading push rules now happens in the datastore, so we can remove the methods that loaded them outside the datastore. The ``waiting_for_join_list`` in federation handler is populated by anything, so can be removed. The ``_get_members_events_txn`` method isn't called from anywhere so can be removed.
* Make get_joined_hosts_for_room use get_users_in_roomErik Johnston2016-06-031-16/+3
|
* Use state to calculate get_users_in_roomErik Johnston2016-06-011-3/+0
|
* get_room_members is unused nowMark Haines2016-05-161-18/+0
|
* Spell "domain" correctlyMark Haines2016-05-161-2/+2
| | | | s/domian/domain/g
* Add and use get_domian_from_idErik Johnston2016-05-091-5/+2
|
* Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actionsDavid Baker2016-04-111-0/+3
|\ | | | | Change pushers to use the event_actions table
| * pep8David Baker2016-04-061-1/+3
| |
| * Make pushers use the event_push_actions table instead of listening on an ↵David Baker2016-04-061-0/+1
| | | | | | | | | | | | | | event stream & running the rules again. Sytest passes, but remaining to do: * Make badges work again * Remove old, unused code
* | Remove some unused functions (#711)Mark Haines2016-04-081-33/+0
|/ | | | | | | | * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want
* Fix stuck invitesErik Johnston2016-04-051-0/+19
| | | | | | | | If rejecting a remote invite fails with an error response don't fail the entire request; instead mark the invite as locally rejected. This fixes the bug where users can get stuck invites which they can neither accept nor reject.
* Docs and indentsErik Johnston2016-04-041-2/+16
|
* Add upgrade path, rename tableErik Johnston2016-04-041-3/+3
|
* Store invites in a separate table.Erik Johnston2016-04-041-21/+90
|
* Make get_invites return RoomsForUserErik Johnston2016-03-231-6/+4
|
* Remove dead code left over from presence changesMark Haines2016-03-171-24/+0
|
* Cache get_room_changes_for_userErik Johnston2016-01-291-0/+4
|
* Up get_rooms_for_user cache sizeErik Johnston2016-01-281-1/+1
|
* Push: Use storage apis that are cachedErik Johnston2016-01-251-0/+1
|
* Allow filtering events for multiple users at onceErik Johnston2016-01-181-0/+13
|
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Fix typoDaniel Wagner-Hall2015-12-141-1/+1
|
* Add caches for whether a room has been forgotten by a userMark Haines2015-12-101-4/+7
|
* Fix SQL for postgres againDaniel Wagner-Hall2015-11-231-1/+1
|
* Fix SQL for postgresDaniel Wagner-Hall2015-11-231-1/+1
|
* Ignore forgotten rooms in v2 syncDaniel Wagner-Hall2015-11-191-1/+1
|
* Simplify codeDaniel Wagner-Hall2015-11-191-7/+5
|
* Apply forgetting properly to historical eventsDaniel Wagner-Hall2015-11-181-2/+32
|
* Allow users to forget roomsDaniel Wagner-Hall2015-11-171-0/+36
|
* Include banned rooms in the archived section of v2 syncMark Haines2015-10-211-2/+2
|
* Add rooms that the user has left under archived in v2 sync.Mark Haines2015-10-191-0/+13
|
* Add a get_invites_for_user method to the storage to find out the rooms a ↵Mark Haines2015-10-131-0/+14
| | | | user is invited to
* Merge pull request #288 from matrix-org/markjh/unused_definitionsMark Haines2015-09-281-6/+0
|\ | | | | Remove some of the unused definitions from synapse
| * synapse/storage/roommember.py:_get_members_query was unusedMark Haines2015-09-231-6/+0
| |
* | Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txnMark Haines2015-09-241-6/+6
|/
* Include the event_id and stream_ordering of membership events when looking ↵Mark Haines2015-09-081-2/+4
| | | | up which rooms a user is in
* Move all the caches into their own package, synapse.util.cachesErik Johnston2015-08-111-1/+2
|
* Change Cache to not use *args in its interfaceErik Johnston2015-08-071-3/+3
|
* Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room'Erik Johnston2015-08-051-2/+2
|
* Add bulk insert events APIErik Johnston2015-06-251-27/+17
|
* Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2015-05-221-0/+2
|\
| * Add caches for things requested by the pushersMark Haines2015-05-211-0/+2
| |
* | Move fetching of events into their own transactionsErik Johnston2015-05-141-21/+18
|/
* Collect the invalidate callbacks on the transaction object rather than ↵Mark Haines2015-05-051-5/+3
| | | | passing around a separate list
* Invalidate the caches from the correct threadMark Haines2015-05-051-3/+5
|
* More join conditionsErik Johnston2015-04-301-2/+6
|
* Add more conditions on JOINs to make postgres go a little faster.Erik Johnston2015-04-301-1/+1
|
* Add get_rooms_for_user cacheErik Johnston2015-04-301-0/+2
|
* PEP8Erik Johnston2015-04-071-1/+0
|
* Don't use room hosts tableErik Johnston2015-03-241-25/+4
|
* Sanitize RoomMemberStoreErik Johnston2015-03-231-63/+61
|
* Merge branch 'develop' of github.com:matrix-org/synapse into mysqlErik Johnston2015-03-201-1/+2
|\
| * Give sensible names for '_simple_...' transactionsErik Johnston2015-03-201-1/+2
| |
* | Convert storage layer to be mysql compatibleErik Johnston2015-03-191-1/+1
|/
* PEP8Kegan Dougal2015-03-021-1/+1
|
* Wrap all of get_app_service_rooms in a txn.Kegan Dougal2015-03-021-17/+19
|
* RoomMemberStore no longer needs a _user_rooms_cache memberPaul "LeoNerd" Evans2015-02-231-5/+0
|
* Take named arguments to @cached() decorator, add a 'max_entries' limitPaul "LeoNerd" Evans2015-02-191-1/+1
|
* Move @cached decorator out into synapse.storage._base; add minimal docsPaul "LeoNerd" Evans2015-02-191-27/+1
|
* Pull out the 'get_rooms_for_user' cache logic into a reüsable @cached decoratorPaul "LeoNerd" Evans2015-02-191-24/+29
|
* Use consumeErrors=True on all DeferredLists.Erik Johnston2015-02-171-1/+1
| | | | | | This is so that the DeferredLists actually consume the error instead of propogating down the non-existent errback chain. This should reduce the number of unhandled errors we are seeing.
* Added another TODO notePaul "LeoNerd" Evans2015-02-111-0/+3
|
* Cache the result of a get_rooms_for_user query, to make ↵Paul "LeoNerd" Evans2015-02-111-7/+34
| | | | user_rooms_intersect() much lighter in the read-common case
* First step of making user_rooms_intersect() faster - implement in ↵Paul "LeoNerd" Evans2015-02-111-18/+20
| | | | intersection logic in Python code terms of a DB query that is cacheable per user
* Replace hs.parse_userid with UserID.from_stringMark Haines2015-01-231-2/+3
|
* Merge branch 'erikj-perf' of github.com:matrix-org/synapse into developErik Johnston2015-01-061-21/+49
|\
| * Name 'user_rooms_intersect' transactionErik Johnston2015-01-061-16/+18
| |
| * We don't need the full events for get_rooms_for_user_where_membership_isErik Johnston2015-01-061-5/+31
| |
* | Merge pull request #28 from matrix-org/erikj-perfMark Haines2015-01-061-7/+14
|\| | | | | Database performance improvements.
| * Add RoomMemberStore.get_users_in_room, so that we can get the list of joined ↵Erik Johnston2015-01-061-0/+13
| | | | | | | | users without having to retrieve the full events
| * Test some ideas that might help performance a bitErik Johnston2014-12-171-7/+1
| |
* | Update copyright noticesMark Haines2015-01-061-1/+1
|/
* Workaround for non-uniqueness of room member events in the database ↵Paul "LeoNerd" Evans2014-12-021-1/+3
| | | | confusing HAVING COUNT() test of room membership intersection (with thanks to Tom Molesworth)
* Add a few missing yields, Move deferred lists inside PreserveLoggingContext ↵Mark Haines2014-11-201-2/+2
| | | | because they don't interact well with the logging contexts
* Fix pep8 warningsMark Haines2014-10-301-3/+5
|
* pyflakes cleanupErik Johnston2014-09-301-1/+0
|
* Add LIMIT to scalar subqueryErik Johnston2014-09-251-1/+2
|
* Rename deletions to redactionsErik Johnston2014-09-241-3/+3
|
* Add m.room.deletion. If an event is deleted it will be returned to clients ↵Erik Johnston2014-09-231-3/+10
| | | | 'pruned', i.e. all client specified keys will be removed.
* Merge remote-tracking branch 'origin/develop' into test-sqlite-memoryPaul "LeoNerd" Evans2014-09-151-2/+2
|\
| * Fix bug where we relied on the current_state_events being updated when we ↵Erik Johnston2014-09-121-2/+2
| | | | | | | | are handling type specific persistence
* | Merge remote-tracking branch 'origin/develop' into test-sqlite-memoryPaul "LeoNerd" Evans2014-09-121-12/+45
|\| | | | | | | | | Conflicts: synapse/storage/pdu.py
| * Fix bug where we incorrectly removed a remote host from the list of hosts in ↵Erik Johnston2014-09-121-12/+45
| | | | | | | | a room when any user from that host left that room even if they weren't the last user from that host in that room
* | More accurate docs / clearer paramter names in RoomMemberStorePaul "LeoNerd" Evans2014-09-121-6/+7
| |
* | Revert recent changes to RoomMemberStorePaul "LeoNerd" Evans2014-09-121-27/+9
| |
* | Add a .runInteraction() method on SQLBaseStore itself to wrap the .db_poolPaul "LeoNerd" Evans2014-09-121-0/+5
| |
* | Add a better _store_room_member_txn() method that takes separated fields ↵Paul "LeoNerd" Evans2014-09-111-8/+21
| | | | | | | | instead of an event object; also add FIXME comment about a big bug in the logic
* | Rename _store_room_member_txn to _store_room_member_from_event_txn so we can ↵Paul "LeoNerd" Evans2014-09-111-1/+1
|/ | | | create another, more sensible function of that name
* When getting a state event also include the previous contentErik Johnston2014-09-061-2/+2
|
* fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org ↵Matthew Hodgson2014-09-031-1/+1
| | | | hasn't been incorporated in time for launch.
* Rename the 'do_users_share_a_room' to something slightly less verb-soundingPaul "LeoNerd" Evans2014-09-031-1/+1
|
* add _get_room_member, fix datastore methodsMark Haines2014-08-271-2/+20
|
* Merge branch 'develop' into storage_transactionsMark Haines2014-08-261-10/+27
|\ | | | | | | | | | | | | Conflicts: synapse/api/auth.py synapse/handlers/room.py synapse/storage/__init__.py
| * Fix pyflakes errorsMark Haines2014-08-261-7/+2
| |
| * Add a do_users_share_a_room method and use that in the presence handler.Erik Johnston2014-08-261-0/+21
| |
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Kegan Dougal2014-08-261-1/+1
| |\ | | | | | | | | | client_server_url_rename
| | * Turn off spammy loggingErik Johnston2014-08-221-1/+1
| | |
| * | Removed member list servlet: now using generic state paths.Kegan Dougal2014-08-261-2/+3
| |/
* / Move the event storage into a single transactionMark Haines2014-08-261-5/+5
|/
* PEP8 cleanupsErik Johnston2014-08-151-1/+0
|
* Start chagning the events stream to work with the new DB schemaErik Johnston2014-08-151-1/+7
|
* Fix up typos and correct sql queriesErik Johnston2014-08-141-13/+13
|
* Start fixing places that use the data store.Erik Johnston2014-08-141-1/+1
|
* Rename _execute_queryErik Johnston2014-08-141-1/+1
|
* Fix up RoomMemberStore to work with the new schema.Erik Johnston2014-08-141-95/+69
|
* add in whitespace after copyright statements to improve legibilityMatthew Hodgson2014-08-131-0/+1
|
* Reference Matrix Home Servermatrix.org2014-08-121-0/+171
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-10 14:13:13 +0000