summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-237-8/+8
| | | | what could possibly go wrong
* Fix logcontext handling for persist_eventsRichard van der Hoff2017-10-171-7/+17
| | | | | | | | * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments
* Implement GET /groups/$groupId/invited_usersLuke Barnard2017-10-161-0/+12
|
* peeeeeeeeep8888888888888888888888888888Erik Johnston2017-10-111-1/+1
|
* Fix group stream replicationErik Johnston2017-10-111-7/+7
| | | | | The stream update functions expect the storage function to return a list of tuples.
* Fix schema delta versionsErik Johnston2017-10-113-1/+1
|
* Merge pull request #2466 from matrix-org/erikj/groups_mergedErik Johnston2017-10-116-20/+1529
|\ | | | | Initial Group Implementation
| * Merge branch 'develop' into erikj/groups_mergedDavid Baker2017-10-024-23/+129
| |\
| * | Add remove room APIErik Johnston2017-09-261-0/+23
| | |
| * | Add unique index to group_rooms tableErik Johnston2017-09-261-1/+1
| | |
| * | Remove user from group summary when the leave the groupErik Johnston2017-09-211-0/+8
| | |
| * | Correctly return next tokenErik Johnston2017-09-201-1/+2
| | |
| * | Fix initial syncErik Johnston2017-09-201-1/+9
| | |
| * | Add user profiles to summary from group serverErik Johnston2017-08-251-1/+1
| | |
| * | Add remote profile cacheErik Johnston2017-08-252-0/+126
| | |
| * | Add _simple_updateErik Johnston2017-08-251-19/+32
| | |
| * | Merge pull request #2410 from matrix-org/erikj/groups_publiciseErik Johnston2017-08-212-0/+32
| |\ \ | | | | | | | | Add ability to publicise group membership
| | * | Use BOOLEAN rather than TEXT typeErik Johnston2017-08-211-1/+1
| | | |
| | * | Add bulk group publicised lookup APIErik Johnston2017-08-091-0/+14
| | | |
| | * | Allow update group publicityErik Johnston2017-08-081-0/+15
| | | |
| | * | Store whether the user wants to publicise their membership of a groupErik Johnston2017-08-082-0/+3
| | | |
| * | | Groups: Fix mising json.load in initial syncErik Johnston2017-08-211-1/+7
| |/ /
| * | Update commentErik Johnston2017-07-241-2/+2
| | |
| * | Use join rather than joined, etc.Erik Johnston2017-07-241-2/+2
| | |
| * | Fix all the typosErik Johnston2017-07-241-8/+7
| | |
| * | Include users membership in group in summary APIErik Johnston2017-07-241-0/+55
| | |
| * | Check users/rooms are in group before adding to summaryErik Johnston2017-07-241-0/+25
| | |
| * | Merge pull request #2378 from matrix-org/erikj/group_sync_supportErik Johnston2017-07-213-5/+110
| |\ \ | | | | | | | | Add groups to sync stream
| | * | Fix replication. And notifyErik Johnston2017-07-201-0/+23
| | | |
| | * | Add groups to sync streamErik Johnston2017-07-203-5/+87
| | | |
| * | | Merge pull request #2377 from matrix-org/erikj/group_profile_updateErik Johnston2017-07-201-0/+11
| |\ \ \ | | |/ / | |/| | Add update group profile API
| | * | Fix a storage descErik Johnston2017-07-201-1/+1
| | | |
| | * | Add update group profile APIErik Johnston2017-07-201-0/+11
| | | |
| * | | Fix typoErik Johnston2017-07-201-1/+1
| |/ /
| * | Fix typosErik Johnston2017-07-181-8/+17
| | |
| * | CommentErik Johnston2017-07-181-0/+14
| | |
| * | Remove sync stuffErik Johnston2017-07-182-29/+1
| | |
| * | Remove unused tablesErik Johnston2017-07-181-7/+0
| | |
| * | Remove sync stuffErik Johnston2017-07-182-70/+0
| | |
| * | Add local group server supportErik Johnston2017-07-173-0/+195
| | |
| * | Add DEFAULT_ROLE_IDErik Johnston2017-07-141-4/+5
| | |
| * | Correctly increment ordersErik Johnston2017-07-141-2/+2
| | |
| * | Remove unused functionsErik Johnston2017-07-131-152/+0
| | |
| * | CommentsErik Johnston2017-07-131-12/+24
| | |
| * | CommentsErik Johnston2017-07-122-5/+41
| | |
| * | Add group summary APIsErik Johnston2017-07-122-0/+699
| | |
| * | CommentErik Johnston2017-07-121-1/+1
| | |
| * | Move commentErik Johnston2017-07-111-2/+2
| | |
| * | Rename column to attestation_jsonErik Johnston2017-07-112-5/+5
| | |
| * | CommentsErik Johnston2017-07-111-2/+2
| | |
| * | CommentsErik Johnston2017-07-112-4/+34
| | |
| * | Initial group server implementationErik Johnston2017-07-103-1/+359
| | |
* | | Fix up commentErik Johnston2017-10-091-1/+2
| | |
* | | Invalidate cacheErik Johnston2017-10-031-0/+3
| | |
* | | Update commentsErik Johnston2017-10-031-0/+7
| | |
* | | Ignore incoming events for rooms that we have leftErik Johnston2017-10-031-0/+32
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When synapse receives an event for a room its not in over federation, it double checks with the remote server to see if it is in fact in the room. This is done so that if the server has forgotten about the room (usually as a result of the database being dropped) it can recover from it. However, in the presence of state resets in large rooms, this can cause a lot of work for servers that have legitimately left. As a hacky solution that supports both cases we drop incoming events for rooms that we have explicitly left. This means that we no longer support the case of servers having forgotten that they've rejoined a room, but that is sufficiently rare that we're not going to support it for now.
* | Up the limits on number of url cache entries to delete at one timeErik Johnston2017-09-281-2/+2
| |
* | Add old indicesErik Johnston2017-09-281-0/+2
| |
* | Only pull out local media that were for url cacheErik Johnston2017-09-281-1/+1
| |
* | Fix typoErik Johnston2017-09-281-3/+3
| |
* | Change expires column to expires_tsErik Johnston2017-09-282-8/+27
| |
* | More bracketsErik Johnston2017-09-281-1/+1
| |
* | Delete expired url cache dataErik Johnston2017-09-283-1/+79
| |
* | Invalidate signing key cache when we gat an updateRichard van der Hoff2017-09-201-17/+24
|/ | | | This might make the cache slightly more efficient.
* Perf: Don't filter events for pushErik Johnston2017-07-071-0/+13
| | | | | We know the users are joined and we can explicitly check for if they are ignoring the user, so lets do that.
* Merge pull request #2309 from matrix-org/erikj/user_ip_replErik Johnston2017-07-061-3/+5
|\ | | | | Fix up user_ip replication commands
| * Fix upErik Johnston2017-06-271-3/+5
| |
* | Define CACHE_SIZE_FACTOR onceErik Johnston2017-07-042-8/+3
| |
* | Make into listErik Johnston2017-06-291-1/+1
| |
* | Prefill forward extrems and event to state groupsErik Johnston2017-06-292-2/+13
|/
* Fix up client ips to read from pending dataErik Johnston2017-06-272-21/+59
|
* Batch upsert user ipsErik Johnston2017-06-271-19/+38
|
* Fix up indices for users_who_share_roomsErik Johnston2017-06-261-1/+2
|
* Store URL cache preview downloads seperatelyErik Johnston2017-06-232-2/+22
| | | | This makes it easier to clear old media out at a later date
* Add descErik Johnston2017-06-221-0/+1
|
* Merge pull request #2296 from matrix-org/erikj/dont_appserver_sharErik Johnston2017-06-211-15/+20
|\ | | | | Don't work out users who share room with appservice users
| * Initialise exclusive_user_regexErik Johnston2017-06-211-15/+20
| |
* | Merge branch 'develop' into avoid_duplicate_filtersKrombel2017-06-2125-223/+1652
|\|
| * Handle thumbnail urlsErik Johnston2017-06-191-12/+14
| |
| * Add API to quarantine mediaErik Johnston2017-06-193-2/+89
| |
| * Add shutdown room APIErik Johnston2017-06-193-0/+59
| |
| * Merge pull request #2286 from matrix-org/erikj/split_out_user_dirErik Johnston2017-06-161-0/+18
| |\ | | | | | | Split out user directory to a separate process
| | * Initial worker implErik Johnston2017-06-161-0/+18
| | |
| * | Merge pull request #2280 from matrix-org/erikj/share_room_user_dirErik Johnston2017-06-163-31/+299
| |\| | | | | | | Include users who you share a room with in user directory
| | * Include users who share room with requester in user directoryErik Johnston2017-06-151-22/+39
| | |
| | * Implement updating users who share rooms on the flyErik Johnston2017-06-151-1/+110
| | |
| | * Implement initial population of users who share rooms tableErik Johnston2017-06-151-7/+117
| | |
| | * Add DB schema for tracking users who share roomsErik Johnston2017-06-152-1/+33
| | |
| * | Merge pull request #2281 from matrix-org/erikj/phone_home_statsErik Johnston2017-06-153-70/+70
| |\ \ | | |/ | |/| Fix phone home stats
| | * Add some more statsErik Johnston2017-06-152-0/+28
| | |
| | * Fix phone home statsErik Johnston2017-06-142-71/+43
| | |
| * | Fix user directory insertion due to missing room_idErik Johnston2017-06-131-1/+2
| | |
| * | Fix up sqlErik Johnston2017-06-131-10/+12
| | |
| * | Fix user dir to not assume existence of userErik Johnston2017-06-131-22/+37
| | |
| * | Merge pull request #2274 from matrix-org/erikj/cache_is_host_joinedErik Johnston2017-06-132-1/+6
| |\ \ | | | | | | | | Add cache for is_host_joined
| | * | Add cache for is_host_joinedErik Johnston2017-06-132-1/+6
| | | |
| * | | Tweak the ranking of PG user dir searchErik Johnston2017-06-131-13/+40
| |/ /
| * | Merge pull request #2266 from matrix-org/erikj/host_in_roomErik Johnston2017-06-121-29/+26
| |\ \ | | | | | | | | Change is_host_joined to use current_state table
| | * | Tweak SQLErik Johnston2017-06-091-3/+6
| | | |
| | * | Change is_host_joined to use current_state tableErik Johnston2017-06-091-29/+23
| | | | | | | | | | | | | | | | | | | | | | | | This bypasses a bug where using the state groups to figure out if a host is in a room sometimes errors if the servers isn't in the room. (For example when the server rejected an invite to a remote room)
| * | | Fix replicationErik Johnston2017-06-091-1/+1
| | | |
| * | | CommentsErik Johnston2017-06-091-0/+3
| | | |
| * | | Cache state deltasErik Johnston2017-06-091-3/+12
| |/ /
| * | Ensure we don't use unpersisted state group as prev groupErik Johnston2017-06-081-0/+13
| | |
| * | Merge pull request #2259 from matrix-org/erikj/fix_state_woesErik Johnston2017-06-073-10/+24
| |\ \ | | | | | | | | Fix bug where state_group tables got corrupted
| | * | Fix bug where state_group tables got corruptedErik Johnston2017-06-073-10/+24
| | |/ | | | | | | | | | | | | | | | | | | | | | This is due to the fact that we prefilled caches using txn.call_after, which always gets called including on error. We fix this by making txn.call_after only fire when a transaction completes successfully, which is what we want most of the time anyway.
| * | Merge pull request #2248 from matrix-org/erikj/state_fixupErik Johnston2017-06-072-21/+122
| |\ \ | | | | | | | | Faster cache for get_joined_hosts
| | * | CommentsErik Johnston2017-06-072-0/+15
| | | |
| | * | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixupErik Johnston2017-06-077-9/+624
| | |\ \
| | * | | Remove spurious log linesErik Johnston2017-06-071-1/+0
| | | | |
| | * | | Handle None state group correctlyErik Johnston2017-05-261-1/+4
| | | | |
| | * | | Faster cache for get_joined_hostsErik Johnston2017-05-252-21/+105
| | | | |
| * | | | When pruning, delete from device_lists_outbound_last_successErik Johnston2017-06-071-0/+8
| | | | |
| * | | | Keep pruning background taskErik Johnston2017-06-071-0/+45
| | | | |
| * | | | Split up device_lists_outbound_pokes table for faster updates.Erik Johnston2017-06-072-58/+57
| | |/ / | |/| |
| * | | Increase size of IP cacheErik Johnston2017-06-071-1/+6
| | | |
| * | | Handle profile updates in user directoryErik Johnston2017-06-011-0/+49
| | | |
| * | | Fix sqliteErik Johnston2017-06-011-1/+1
| | | |
| * | | Split the table in twoErik Johnston2017-06-013-4/+88
| | | |
| * | | Tweak search queryErik Johnston2017-06-011-1/+1
| | | |
| * | | Tweak search queryErik Johnston2017-06-011-2/+2
| | | |
| * | | Remove spurious inlineCallbacksErik Johnston2017-06-011-1/+0
| | | |
| * | | Bug fixes and loggingErik Johnston2017-06-011-1/+1
| | | | | | | | | | | | | | | | | | | | - Check if room is public when a user joins before adding to user dir - Fix typo of field name "content.join_rules" -> "content.join_rule"
| * | | Order by if they have profile infoErik Johnston2017-06-011-2/+11
| | | |
| * | | Use prefix matchingErik Johnston2017-06-011-5/+29
| | | |
| * | | Comment on why arbitrary commentsErik Johnston2017-05-311-2/+6
| | | |
| * | | Use unique indicesErik Johnston2017-05-311-2/+2
| | | |
| * | | Convert to intErik Johnston2017-05-311-0/+1
| | | |
| * | | Fix testsErik Johnston2017-05-311-0/+1
| | | |
| * | | Fix typoErik Johnston2017-05-311-1/+1
| | | |
| * | | Limit number of things we fetch out of the dbErik Johnston2017-05-311-9/+30
| | | |
| * | | Add stream change cacheErik Johnston2017-05-313-1/+19
| | | |
| * | | Split out directory and search tablesErik Johnston2017-05-312-29/+56
| | | |
| * | | Pull max id from correct tableErik Johnston2017-05-311-0/+8
| | | |
| * | | Handle the server leaving a public roomErik Johnston2017-05-312-0/+15
| | | |
| * | | Add commentsErik Johnston2017-05-312-4/+37
| | | |
| * | | Weight differentlyErik Johnston2017-05-311-10/+24
| | | |
| * | | Add searchErik Johnston2017-05-311-0/+35
| | | |
| * | | Update room column when room becomes unpublicErik Johnston2017-05-311-0/+10
| | | |
| * | | Add user_directory to databaseErik Johnston2017-05-313-0/+216
| | | |
| * | | Add commentErik Johnston2017-05-301-1/+1
| | | |
| * | | Add clobbered event_idErik Johnston2017-05-302-1/+3
| | | |
| * | | Add current_state_delta_stream tableErik Johnston2017-05-302-8/+48
| |/ /
| * | Merge pull request #2247 from matrix-org/erikj/auth_eventErik Johnston2017-05-244-7/+104
| |\ \ | | | | | | | | Only store event_auth for state events
| | * | Add background task to clear out old event_authErik Johnston2017-05-243-1/+74
| | | |
| | * | Only store event_auth for state eventsErik Johnston2017-05-242-6/+30
| | | |
| * | | Remove redundant invalidationErik Johnston2017-05-231-3/+0
| | | |
| * | | Missed an invalidationErik Johnston2017-05-231-0/+3
| | | |
| * | | Stream count_e2e_one_time_keys cache invalidationErik Johnston2017-05-221-13/+18
| |/ /
| * | Fix invalidation of get_users_with_read_receipts_in_roomErik Johnston2017-05-191-1/+3
| | |
| * | Increase size of push rule cachesErik Johnston2017-05-181-2/+2
| | |
| * | Make get_if_app_services_interested_in_user fasterErik Johnston2017-05-182-10/+23
| | |
| * | Merge pull request #2230 from matrix-org/erikj/speed_up_get_stateErik Johnston2017-05-171-29/+11
| |\ \ | | | | | | | | Make get_state_groups_from_groups faster.
| | * | Make get_state_groups_from_groups faster.Erik Johnston2017-05-171-29/+11
| | |/ | | | | | | | | | | | | | | | | | | | | | | | | Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.
| * | Merge pull request #2216 from slipeer/app_services_interested_in_userErik Johnston2017-05-172-3/+6
| |\ \ | | |/ | |/| Fix users claimed non-exclusively by an app service don't get notific…
| | * Fix users claimed non-exclusively by an app service don't get notifications ↵Slipeer2017-05-112-3/+6
| | | | | | | | | | | | #2211
| * | CommentErik Johnston2017-05-171-0/+5
| | |
| * | Short circuit when we have delta idsErik Johnston2017-05-171-3/+19
| | |
| * | Speed up get_joined_hostsErik Johnston2017-05-161-1/+2
| | |
| * | Merge pull request #2224 from matrix-org/erikj/prefill_stateErik Johnston2017-05-163-9/+26
| |\ \ | | | | | | | | Prefill state caches
| | * | Remove spurious merge artifactsErik Johnston2017-05-161-13/+0
| | | |
| | * | Take a copy before prefilling, as it may be a frozendictErik Johnston2017-05-161-1/+1
| | | |
| | * | Update commentErik Johnston2017-05-151-1/+2
| | | |
| | * | CommentsErik Johnston2017-05-152-3/+6
| | | |
| | * | Add more granular event send metricsErik Johnston2017-05-151-0/+12
| | | |
| | * | Prefill state cachesErik Johnston2017-05-153-6/+20
| | | |
* | | | replaced json.dumps with encode_canonical_jsonKrombel2017-06-211-1/+2
| | | |
* | | | Merge branch 'develop' into avoid_duplicate_filtersKrombel2017-05-164-12/+63
|\| | |
| * | | Add a comment to old deltaRichard van der Hoff2017-05-111-0/+4
| | | |
| * | | Don't create event_search index on sqliteRichard van der Hoff2017-05-112-3/+11
| | | | | | | | | | | | | | | | ... because the table is virtual
| * | | Add more logging for purgingRichard van der Hoff2017-05-111-7/+14
| | | | | | | | | | | | | | | | Log the number of events we will be deleting at info.
| * | | Add an index to event_searchRichard van der Hoff2017-05-113-3/+35
| |/ / | | | | | | | | | - to make the purge API quicker
* | | avoid access-error if no filter_id matchesKrombel2017-05-151-3/+3
| | |
* | | insert whitespace to fix travis buildKrombel2017-05-121-1/+1
| | |
* | | add check to only add a new filter if the same filter does not exist previouslyKrombel2017-05-111-0/+9
|/ / | | | | | | Signed-off-by: Matthias Kesler <krombel@krombel.de>
* | Tidy purge code and add some commentsRichard van der Hoff2017-05-111-12/+19
| | | | | | | | Try to make this clearer with more comments and some variable renames
* | Don't de-delta state groups we're about to deleteRichard van der Hoff2017-05-101-11/+9
| |
* | add some logging to purge_historyRichard van der Hoff2017-05-101-4/+21
|/
* Merge pull request #2208 from matrix-org/erikj/ratelimit_overridErik Johnston2017-05-102-1/+57
|\ | | | | Add per user ratelimiting overrides
| * Add per user ratelimiting overridesErik Johnston2017-05-102-1/+57
| |
* | Allow clients to upload one-time-keys with new sigsRichard van der Hoff2017-05-091-20/+27
|/ | | | | | | When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
* Merge pull request #2203 from matrix-org/erikj/event_cache_hit_ratioErik Johnston2017-05-082-2/+21
|\ | | | | Don't update event cache hit ratio from get_joined_users
| * Expand docstring a bitErik Johnston2017-05-081-1/+7
| |
| * Don't update event cache hit ratio from get_joined_usersErik Johnston2017-05-082-2/+15
| | | | | | | | | | Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.
* | Merge pull request #2201 from matrix-org/erikj/store_device_cacheErik Johnston2017-05-081-3/+22
|\ \ | | | | | | Cache check to see if device exists
| * | Add missing yieldsErik Johnston2017-05-081-2/+4
| | |
| * | Invalidate cache on device deletionErik Johnston2017-05-081-2/+6
| | |
| * | CommentErik Johnston2017-05-081-0/+2
| | |
| * | Cache check to see if device existsErik Johnston2017-05-081-1/+12
| |/
* / Cache one time key countsErik Johnston2017-05-081-0/+9
|/
* Increase client_ip cache sizeErik Johnston2017-05-081-0/+1
|
* Revert "Remove unused import"Erik Johnston2017-05-081-0/+1
| | | | This reverts commit ab37bef83bebd7cdaeb7cfd98553d18883d09103.
* Revert "We don't care about forgotten rooms"Erik Johnston2017-05-081-0/+12
| | | | This reverts commit ad8b316939d59230526e60660caf9094cff62c8f.
* Revert "Speed up filtering of a single event in push"Erik Johnston2017-05-082-16/+2
| | | | This reverts commit 421fdf74609439edaaffce117436e6a6df147841.
* Merge pull request #2176 from matrix-org/erikj/faster_get_joinedErik Johnston2017-05-051-16/+42
|\ | | | | Make get_joined_users faster
| * FixesErik Johnston2017-05-051-3/+4
| |
| * Add commentErik Johnston2017-05-041-0/+3
| |
| * Make get_joined_users fasterErik Johnston2017-05-031-16/+38
| |
* | Merge pull request #2185 from matrix-org/erikj/smaller_cachesErik Johnston2017-05-051-2/+2
|\ \ | | | | | | Optimise caches for single key
| * | Reduce size of get_users_in_roomErik Johnston2017-05-041-2/+2
| | |
* | | Merge pull request #2174 from matrix-org/erikj/current_cache_hostsErik Johnston2017-05-051-0/+39
|\ \ \ | | | | | | | | Add cache for get_current_hosts_in_room
| * | | Intern host stringsErik Johnston2017-05-041-1/+2
| | | |
| * | | Make caches biggerErik Johnston2017-05-041-1/+1
| | | |
| * | | Fix comment and num argsErik Johnston2017-05-041-1/+1
| | | |
| * | | Add cache for get_current_hosts_in_roomErik Johnston2017-05-021-0/+38
| | |/ | |/|
* | | Revert "Prefill state caches"Erik Johnston2017-05-043-27/+9
| |/ |/|
* | Update commentErik Johnston2017-05-031-1/+2
| |
* | CommentsErik Johnston2017-05-032-3/+6
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_stateErik Johnston2017-05-031-0/+18
|\ \
| * | Add more granular event send metricsErik Johnston2017-05-021-0/+16
| |/
* / Prefill state cachesErik Johnston2017-05-023-6/+18
|/
* Merge pull request #2080 from matrix-org/erikj/filter_speedErik Johnston2017-04-282-2/+16
|\ | | | | Speed up filtering of a single event in push
| * Remove unused importErik Johnston2017-04-281-1/+0
| |
| * We don't care about forgotten roomsErik Johnston2017-04-281-12/+0
| |
| * Speed up filtering of a single event in pushErik Johnston2017-04-282-2/+29
| |
* | Fix bgupdate error if index already exists (#2167)Richard van der Hoff2017-04-271-30/+53
|/ | | | | | | | | When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).
* Make state caches cache in asciiErik Johnston2017-04-251-2/+3
|
* CommentErik Johnston2017-04-251-0/+2
|
* Reduce size of joined_user cacheErik Johnston2017-04-251-8/+14
| | | | | | | | The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.
* Reduce _get_state_group_for_event cache sizeErik Johnston2017-04-251-1/+1
|
* Reduce cache size by not storing deferredsErik Johnston2017-04-251-4/+7
| | | | | | | | | | | | | | | | | | | | Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.
* Remove unused cacheErik Johnston2017-04-241-6/+1
|
* Merge pull request #2120 from matrix-org/luke/read-markersLuke Barnard2017-04-131-0/+22
|\ | | | | Implement Read Marker API
| * Correct logic in is_event_afterLuke Barnard2017-04-131-1/+1
| |
| * Simplify is_event_after logicLuke Barnard2017-04-121-10/+3
| |
| * travis flake8..Luke Barnard2017-04-111-0/+1
| |
| * Refactor event ordering check to events storeLuke Barnard2017-04-111-0/+28
| |
* | Bail on where clause insteadErik Johnston2017-04-111-3/+4
| |
* | Don't bother with outer check for nowErik Johnston2017-04-111-3/+0
| |
* | Fix getting latest device IP for user with no devicesErik Johnston2017-04-111-0/+5
|/
* CommentErik Johnston2017-04-071-0/+9
|
* Speed up get_current_state_idsErik Johnston2017-04-071-10/+17
| | | | | | | | | | | | Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.
* Deduplicate new deviceinbox rows for replicationErik Johnston2017-04-041-4/+7
|
* Fix up docsErik Johnston2017-03-311-1/+1
|
* Add new storage functions for new replicationErik Johnston2017-03-303-3/+133
| | | | | The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.
* Use txn.fetchall() so we can reuse txnErik Johnston2017-03-301-1/+1
|
* Increase cache size for _get_state_group_for_eventErik Johnston2017-03-291-1/+1
|
* Decrank last commitErik Johnston2017-03-291-4/+5
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-03-2917-164/+164
|\ | | | | | | erikj/e2e_one_time_upsert
| * Merge pull request #2063 from matrix-org/erikj/device_list_batchErik Johnston2017-03-281-0/+4
| |\ | | | | | | Batch sending of device list pokes
| | * Batch sending of device list pokesErik Johnston2017-03-241-0/+4
| | |
| * | TypoErik Johnston2017-03-271-1/+1
| | |
| * | Short circuit if all new events have same state groupErik Johnston2017-03-271-36/+51
| |/
| * Merge pull request #2062 from matrix-org/erikj/presence_replicationErik Johnston2017-03-241-2/+2
| |\ | | | | | | Use presence replication stream to invalidate cache
| | * Actually call invalidateErik Johnston2017-03-241-1/+1
| | |
| | * Use presence replication stream to invalidate cacheErik Johnston2017-03-241-2/+2
| | | | | | | | | | | | | | | | | | Instead of using the cache invalidation replication stream to invalidate the _get_presence_cache, we can instead rely on the presence replication stream. This reduces the amount of replication traffic considerably.
| * | Cache hosts in roomErik Johnston2017-03-241-0/+10
| |/
| * Replace some calls to cursor_to_dictErik Johnston2017-03-242-38/+13
| | | | | | | | | | cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.
| * Use iter(items|values)Erik Johnston2017-03-243-51/+48
| |
| * User Cursor.__iter__ instead of fetchallErik Johnston2017-03-2315-39/+38
| | | | | | | | This prevents unnecessary construction of lists
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-18 10:32:48 +0000