summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Make get_joined_users fasterErik Johnston2017-05-031-16/+38
|
* Merge pull request #2080 from matrix-org/erikj/filter_speedErik Johnston2017-04-282-2/+16
|\ | | | | Speed up filtering of a single event in push
| * Remove unused importErik Johnston2017-04-281-1/+0
| |
| * We don't care about forgotten roomsErik Johnston2017-04-281-12/+0
| |
| * Speed up filtering of a single event in pushErik Johnston2017-04-282-2/+29
| |
* | Fix bgupdate error if index already exists (#2167)Richard van der Hoff2017-04-271-30/+53
|/ | | | | | | | | When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).
* Make state caches cache in asciiErik Johnston2017-04-251-2/+3
|
* CommentErik Johnston2017-04-251-0/+2
|
* Reduce size of joined_user cacheErik Johnston2017-04-251-8/+14
| | | | | | | | The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.
* Reduce _get_state_group_for_event cache sizeErik Johnston2017-04-251-1/+1
|
* Reduce cache size by not storing deferredsErik Johnston2017-04-251-4/+7
| | | | | | | | | | | | | | | | | | | | Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.
* Remove unused cacheErik Johnston2017-04-241-6/+1
|
* Merge pull request #2120 from matrix-org/luke/read-markersLuke Barnard2017-04-131-0/+22
|\ | | | | Implement Read Marker API
| * Correct logic in is_event_afterLuke Barnard2017-04-131-1/+1
| |
| * Simplify is_event_after logicLuke Barnard2017-04-121-10/+3
| |
| * travis flake8..Luke Barnard2017-04-111-0/+1
| |
| * Refactor event ordering check to events storeLuke Barnard2017-04-111-0/+28
| |
* | Bail on where clause insteadErik Johnston2017-04-111-3/+4
| |
* | Don't bother with outer check for nowErik Johnston2017-04-111-3/+0
| |
* | Fix getting latest device IP for user with no devicesErik Johnston2017-04-111-0/+5
|/
* CommentErik Johnston2017-04-071-0/+9
|
* Speed up get_current_state_idsErik Johnston2017-04-071-10/+17
| | | | | | | | | | | | Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.
* Deduplicate new deviceinbox rows for replicationErik Johnston2017-04-041-4/+7
|
* Fix up docsErik Johnston2017-03-311-1/+1
|
* Add new storage functions for new replicationErik Johnston2017-03-303-3/+133
| | | | | The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.
* Use txn.fetchall() so we can reuse txnErik Johnston2017-03-301-1/+1
|
* Increase cache size for _get_state_group_for_eventErik Johnston2017-03-291-1/+1
|
* Decrank last commitErik Johnston2017-03-291-4/+5
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-03-2917-164/+164
|\ | | | | | | erikj/e2e_one_time_upsert
| * Merge pull request #2063 from matrix-org/erikj/device_list_batchErik Johnston2017-03-281-0/+4
| |\ | | | | | | Batch sending of device list pokes
| | * Batch sending of device list pokesErik Johnston2017-03-241-0/+4
| | |
| * | TypoErik Johnston2017-03-271-1/+1
| | |
| * | Short circuit if all new events have same state groupErik Johnston2017-03-271-36/+51
| |/
| * Merge pull request #2062 from matrix-org/erikj/presence_replicationErik Johnston2017-03-241-2/+2
| |\ | | | | | | Use presence replication stream to invalidate cache
| | * Actually call invalidateErik Johnston2017-03-241-1/+1
| | |
| | * Use presence replication stream to invalidate cacheErik Johnston2017-03-241-2/+2
| | | | | | | | | | | | | | | | | | Instead of using the cache invalidation replication stream to invalidate the _get_presence_cache, we can instead rely on the presence replication stream. This reduces the amount of replication traffic considerably.
| * | Cache hosts in roomErik Johnston2017-03-241-0/+10
| |/
| * Replace some calls to cursor_to_dictErik Johnston2017-03-242-38/+13
| | | | | | | | | | cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.
| * Use iter(items|values)Erik Johnston2017-03-243-51/+48
| |
| * User Cursor.__iter__ instead of fetchallErik Johnston2017-03-2315-39/+38
| | | | | | | | This prevents unnecessary construction of lists
* | Correctly look up keyErik Johnston2017-03-291-6/+5
| |
* | The algorithm is part of the key idErik Johnston2017-03-281-4/+4
| |
* | Raise a more helpful exceptionErik Johnston2017-03-231-2/+4
| |
* | Don't user upsert to persist new one time keysErik Johnston2017-03-231-9/+48
|/ | | | | Instead we no-op duplicate one time key uploads, an error if the key_id already exists but encodes a different key.
* fix up some key verif docstringsRichard van der Hoff2017-03-211-2/+3
|
* Merge pull request #2033 from matrix-org/erikj/repl_speedErik Johnston2017-03-211-17/+14
|\ | | | | Don't send the full event json over replication
| * Don't send the full event json over replicationErik Johnston2017-03-171-17/+14
| |
* | Merge pull request #2027 from matrix-org/rav/logcontext_leaksRichard van der Hoff2017-03-201-12/+3
|\ \ | | | | | | A few fixes to logcontext things
| * | Remove broken use of clock.call_laterRichard van der Hoff2017-03-181-12/+3
| |/ | | | | | | | | | | | | | | | | background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.
* | Avoid resetting state on rejected eventsRichard van der Hoff2017-03-172-4/+14
| | | | | | | | | | | | | | When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.
* | Refactoring and cleanupsRichard van der Hoff2017-03-172-72/+225
|/ | | | | | | | | | | | A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.
* CommentsErik Johnston2017-03-161-0/+2
|
* Don't recreate so many setsErik Johnston2017-03-161-5/+6
|
* Fix assertion to stop transaction queue getting wedgedRichard van der Hoff2017-03-153-4/+18
| | | | | | | | ... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-03-143-3/+86
|\ | | | | | | erikj/public_list_speed
| * Fix current_state_events table to not lieErik Johnston2017-03-141-3/+28
| | | | | | | | | | If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.
| * Implement _simple_delete_many_txn, use it to delete devicesLuke Barnard2017-03-132-0/+58
| | | | | | | | | | | | (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.
* | Assume rooms likely haven't changedErik Johnston2017-03-131-0/+3
| |
* | Get current state by using current_state_events tableErik Johnston2017-03-102-11/+21
|/
* Noop repated delete device inbox calls from /syncErik Johnston2017-03-101-1/+35
|
* Remove spurious SQL logging (#1972)Richard van der Hoff2017-03-081-8/+0
| | | | | looks like the upsert function was accidentally sending sql logging to the general logger. We already log the sql in `txn.execute`.
* Select distinct devices from DBErik Johnston2017-03-061-1/+1
| | | | | Otherwise we might pull out tonnes of duplicate user_ids and this can make synapse sad.
* Fix upErik Johnston2017-03-032-8/+8
|
* Add index to device_lists_streamErik Johnston2017-03-013-4/+28
|
* Clobber old device list stream entriesErik Johnston2017-03-011-0/+10
|
* Merge pull request #1959 from matrix-org/erikj/intern_onceErik Johnston2017-02-281-3/+2
|\ | | | | Intern table column names once
| * Intern table column names onceErik Johnston2017-02-281-3/+2
| |
* | Add stream_id index to device_lists_outbound_pokesErik Johnston2017-02-282-1/+17
|/ | | | As this is used for replication streaming
* Fix up txn nameErik Johnston2017-02-281-1/+1
|
* Remove needless checkErik Johnston2017-02-271-2/+0
|
* It should be allErik Johnston2017-02-271-1/+1
|
* Don't fetch current state in common caseErik Johnston2017-02-271-0/+17
| | | | | | Currently we fetch the list of current state events whenever we send something in a room. This is overkill for the common case of persisting a simple chain of non-state events, so lets handle that case specially.
* Merge pull request #1954 from matrix-org/erikj/cache_device2Erik Johnston2017-02-271-45/+73
|\ | | | | Cache get_user_devices_from_cache
| * Cache get_user_devices_from_cacheErik Johnston2017-02-271-45/+73
| |
* | Strip newlines from SQL queriesErik Johnston2017-02-231-0/+6
| |
* | Remove lock from rotate notifsErik Johnston2017-02-221-3/+0
|/
* Store looping callErik Johnston2017-02-211-1/+3
|
* Fix up notif rotationErik Johnston2017-02-181-14/+22
|
* Use TEXT rather than VARCHARDavid Baker2017-02-171-6/+6
| | | | While we're changing anyway
* Make the pushers lang field column longerDavid Baker2017-02-171-0/+39
| | | | | | To accommodate things like zh-Hans-CN Fixes https://github.com/vector-im/riot-ios/issues/1031
* CommentErik Johnston2017-02-161-1/+4
|
* CommentErik Johnston2017-02-161-7/+14
|
* Store the default push actions in a more efficient mannerErik Johnston2017-02-161-11/+40
|
* Merge pull request #1916 from matrix-org/erikj/push_actions_deleteErik Johnston2017-02-163-57/+248
|\ | | | | Aggregate event push actions
| * Less aggressive timersErik Johnston2017-02-141-2/+2
| |
| * Aggregate event push actionsErik Johnston2017-02-143-57/+248
| |
* | Merge pull request #1921 from matrix-org/rav/fix_key_changesRichard van der Hoff2017-02-152-2/+29
|\ \ | |/ |/| Fix bugs in the /keys/changes api
| * Fix bugs in the /keys/changes apiRichard van der Hoff2017-02-142-2/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.
* | Merge pull request #1873 from matrix-org/erikj/delete_push_actionsErik Johnston2017-02-141-3/+8
|\ \ | |/ |/| Be more agressive about purging old room event_push_actions
| * Be more agressive about purging old room event_push_actionsErik Johnston2017-02-011-3/+8
| |
* | Cache get_presence storageErik Johnston2017-02-131-3/+11
| |
* | Make None check explicitErik Johnston2017-02-081-1/+1
| |
* | Merge pull request #1892 from matrix-org/erikj/rejection_fwd_extremErik Johnston2017-02-081-9/+6
|\ \ | | | | | | Ignore new rejected events when working out forward extremities.
| * | Ignore new rejected events when working out forward extremeties.Erik Johnston2017-02-081-9/+6
| | |
* | | Merge pull request #1784 from morteza-araby/user-adminErik Johnston2017-02-062-0/+235
|\ \ \ | |/ / |/| | Administration functionalities
| * | admin,storage: added more administrator functionalitiesMorteza Araby2017-02-022-0/+235
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | administrators can now: - Set displayname of users - Update user avatars - Search for users by user_id - Browse all users in a paginated API - Reset user passwords - Deactivate users Helpers for doing paginated queries has also been added to storage Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
* | Bump cache sizes for common membership queriesErik Johnston2017-02-021-3/+3
| |
* | CommentErik Johnston2017-02-021-1/+2
| |
* | Make presence.get_new_events a bit fasterErik Johnston2017-02-021-0/+16
|/ | | | We do this by caching the set of users a user shares rooms with.
* Doc argsErik Johnston2017-02-011-0/+4
|
* CommentErik Johnston2017-02-011-0/+3
|
* Include newly joined users in /keys/changes APIErik Johnston2017-02-011-0/+7
|
* Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_eventsErik Johnston2017-02-011-0/+2
|\ | | | | Add a small cache get_all_new_events
| * Add a small cache get_all_new_eventsErik Johnston2017-02-011-0/+2
| |
* | Merge pull request #1868 from matrix-org/erikj/replication_cacheErik Johnston2017-02-012-22/+0
|\ \ | | | | | | Only invalidate membership caches based on the cache stream
| * | Only invalidate membership caches based on the cache streamErik Johnston2017-01-312-22/+0
| | | | | | | | | | | | | | | Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.
* | | Add m.room.member type to queryErik Johnston2017-02-011-1/+1
| | |
* | | Merge pull request #1867 from matrix-org/erikj/member_indexErik Johnston2017-02-013-1/+26
|\ \ \ | |_|/ |/| | Add an index to make membership queries faster
| * | Add an index to make membership queries fasterErik Johnston2017-01-313-1/+26
| |/
* / Up get_latest_event_ids_in_room cacheErik Johnston2017-02-011-1/+1
|/
* Merge pull request #1864 from matrix-org/erikj/device_list_fixesErik Johnston2017-01-311-8/+26
|\ | | | | Fix clearing out old device list outbound pokes
| * Fix deletion of old sent devices correctlyErik Johnston2017-01-311-5/+17
| |
| * Only fetch with row ts and count > 1Erik Johnston2017-01-311-1/+2
| |
| * Fix clearing out old device list outbound pokesErik Johnston2017-01-311-2/+7
| |
* | Merge pull request #1862 from matrix-org/erikj/presence_updateErik Johnston2017-01-311-2/+6
|\ \ | |/ |/| Use DB cache of joined users for presence
| * Stream cache invalidations for room membership storage functionsErik Johnston2017-01-301-2/+6
| |
* | Don't have such a large cacheErik Johnston2017-01-301-1/+1
| |
* | Fix unit testsErik Johnston2017-01-301-2/+10
| |
* | Noop device key changes if they're the sameErik Johnston2017-01-302-11/+40
| |
* | Fix queryErik Johnston2017-01-301-1/+1
|/
* Use get_users_in_room and declare it iterableErik Johnston2017-01-301-1/+1
|
* Rename funcErik Johnston2017-01-301-1/+1
|
* Clear out old destination pokes.Erik Johnston2017-01-301-0/+42
|
* Always use the latest stream_id, sent or unsentErik Johnston2017-01-301-4/+4
|
* Add ts column to device_lists_outbound_pokesErik Johnston2017-01-272-1/+5
|
* Hook device list updates to replicationErik Johnston2017-01-271-0/+15
|
* CommentErik Johnston2017-01-271-0/+2
|
* CommentErik Johnston2017-01-271-0/+2
|
* SQL param orderingErik Johnston2017-01-271-1/+1
|
* User if rather than forErik Johnston2017-01-271-1/+4
|
* Better nameErik Johnston2017-01-271-1/+1
|
* Remove unused paramErik Johnston2017-01-271-2/+1
|
* Fix on sqlite: use left rather than outer joinErik Johnston2017-01-261-3/+3
|
* Handle users leaving roomsErik Johnston2017-01-261-0/+8
|
* Add commentsErik Johnston2017-01-262-3/+42
|
* Implement device key caching over federationErik Johnston2017-01-263-26/+199
|
* Fix up sending of m.device_list_update edusErik Johnston2017-01-251-20/+20
|
* Add basic implementation of local device list changesErik Johnston2017-01-255-21/+244
|
* TypoErik Johnston2017-01-231-1/+1
|
* Refactor to calculate state delta outside transactionErik Johnston2017-01-231-87/+118
|
* CommentsErik Johnston2017-01-201-8/+19
|
* Insert delta of current_state_events to be more efficientErik Johnston2017-01-202-78/+52
|
* SpellingErik Johnston2017-01-201-1/+1
|
* Update all call sites after renameErik Johnston2017-01-201-1/+1
|
* CommentsErik Johnston2017-01-201-0/+11
|
* Calculate the forward extremeties onceErik Johnston2017-01-202-126/+92
|
* Derive current_state_events from state groupsErik Johnston2017-01-201-70/+118
|
* Lower loading events log to DEBUGErik Johnston2017-01-171-2/+2
|
* Increase state_group_cache_sizeErik Johnston2017-01-171-1/+1
|
* Up cache max entries for stateErik Johnston2017-01-162-2/+2
|
* Increase cache size limitErik Johnston2017-01-162-2/+2
|
* Optionally measure size of cache by sum of length of valuesErik Johnston2017-01-132-2/+3
|
* Merge pull request #1793 from matrix-org/erikj/change_device_inbox_indexErik Johnston2017-01-133-3/+55
|\ | | | | Change device_inbox stream index to include user
| * Add commentErik Johnston2017-01-131-0/+1
| |
| * Explicitly close the cursorErik Johnston2017-01-101-0/+1
| |
| * Don't disable autocommitErik Johnston2017-01-101-8/+4
| |
| * Change device_inbox stream index to include userErik Johnston2017-01-103-3/+57
| | | | | | | | | | This makes fetching the nost recently changed users much tricker, and brings it in line with e.g. presence_stream indices.
* | Limit number of entries to prefill from cacheErik Johnston2017-01-102-3/+6
|/ | | | | | | | | Some tables, like device_inbox, take a long time to query at startup for the stream change cache prefills. This is likely because they are slower growing streams and so are more fragmented on disk. For now, lets pull fewer entries out to make startup quicker. In future, we should add a better index to make it even faster.
* Merge branch 'release-v0.18.6' into developMatthew Hodgson2017-01-052-3/+3
|\
| * fix annoying typosMatthew Hodgson2017-01-052-3/+3
| |
* | Merge branch 'release-v0.18.6' into developMatthew Hodgson2017-01-051-0/+2
|\|
| * More logging for the linearizer and for get_eventsMark Haines2017-01-051-0/+2
| |
* | Add /account/3pid/delete endpointDavid Baker2016-12-201-0/+11
|/ | | | Also fix a typo in a comment
* Add some logging for syncing to_device eventsRichard van der Hoff2016-12-151-1/+2
| | | | | Attempt to track down the loss of to_device events (https://github.com/vector-im/riot-web/issues/2711 etc).
* Fixup membership queryErik Johnston2016-12-141-2/+2
|
* Merge pull request #1697 from matrix-org/erikj/fix_bg_memberErik Johnston2016-12-121-1/+1
|\ | | | | Fix background update that prematurely stopped
| * Fix background update that prematurely stoppedErik Johnston2016-12-121-1/+1
| |
* | Merge pull request #1676 from matrix-org/erikj/room_listErik Johnston2016-12-122-20/+196
|\ \ | |/ |/| Add new API appservice specific public room list
| * Add cache to get_public_room_ids_at_stream_idErik Johnston2016-12-121-0/+2
| |
| * Notify replication. Use correct network_idErik Johnston2016-12-121-1/+3
| |
| * DocstringErik Johnston2016-12-121-0/+12
| |
| * CommentsErik Johnston2016-12-071-0/+2
| |
| * Add new API appservice specific public room listErik Johnston2016-12-062-20/+178
| |
* | Merge pull request #1680 from matrix-org/erikj/joined_roomsErik Johnston2016-12-094-3/+123
|\ \ | | | | | | Add new room membership APIs
| * | Add /room/<room_id>/joined_members APIErik Johnston2016-12-081-3/+13
| | | | | | | | | | | | | | | | | | This returns the currently joined members in the room with their display names and avatar urls. This is more efficient than /members for large rooms where you don't need the full events.
| * | Add profile data to the room_membership table for joinsErik Johnston2016-12-084-0/+110
| |/
* / Fix result of insert_receiptRichard van der Hoff2016-12-081-1/+1
|/ | | | This should fix the absence of notifications when new receipts arrive.
* Rip out more refresh_token codeRichard van der Hoff2016-11-302-67/+0
| | | | | | | | We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.
* Don't send old events as federationErik Johnston2016-11-241-1/+1
|
* CommentErik Johnston2016-11-231-1/+3
|
* More efficient notif count queriesErik Johnston2016-11-232-8/+48
|
* Merge pull request #1635 from matrix-org/erikj/split_out_fed_txnErik Johnston2016-11-238-29/+156
|\ | | | | Split out federation transaction sending to a worker
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2016-11-231-0/+8
| |\ | | | | | | | | | erikj/split_out_fed_txn
| * | Fix tests and flake8Erik Johnston2016-11-221-1/+0
| | |
| * | Invalidate retry cache in both directionsErik Johnston2016-11-221-12/+36
| | |
| * | Fix testsErik Johnston2016-11-211-2/+2
| | |
| * | Store federation stream positions in the databaseErik Johnston2016-11-213-4/+52
| | |
| * | Handle sending events and device messages over federationErik Johnston2016-11-174-13/+62
| | |
| * | Hook up the send queue and create a federation sender workerErik Johnston2016-11-161-0/+7
| | |
* | | Ignore AS users when fetching push rulesErik Johnston2016-11-232-2/+18
| |/ |/| | | | | | | By ignoring AS users early on when fetching push rules for a room we can avoid needlessly hitting the DB and filling up the caches.
* | Fail with a coherent error message if `/sync?filter=` is invalidKegan Dougal2016-11-211-0/+8
|/
* Merge pull request #1617 from matrix-org/erikj/intern_state_dictErik Johnston2016-11-151-1/+4
|\ | | | | Correctly intern keys in state cache
| * Correctly intern keys in state cacheErik Johnston2016-11-081-1/+4
| |
* | New Flake8 fixesErik Johnston2016-11-151-0/+1
|/
* CommentErik Johnston2016-11-041-0/+1
|
* Remove unused but buggy functionErik Johnston2016-11-041-11/+1
|
* Remove spurious commentErik Johnston2016-11-031-2/+2
|
* Replace postgres GIN with GISTErik Johnston2016-11-033-1/+45
| | | | | This is because GIN can be slow to write too, especially when the table gets large.
* Continue to clean up received_transactionsErik Johnston2016-11-011-0/+14
|
* Removed unused stuffErik Johnston2016-10-311-163/+3
|
* Increase batching of sent transaction insertsErik Johnston2016-10-311-1/+1
| | | | | | This should further reduce the number of individual inserts, transactions and updates that are required for keeping sent_transactions up to date.
* Use most recently added binding, not most recently seen user.Erik Johnston2016-10-271-19/+11
|
* Fix user_threepids schema deltaErik Johnston2016-10-271-1/+38
| | | | | | | | | | | The delta `37/user_threepids.sql` aimed to update all the email addresses to be lower case, however duplicate emails may exist in the table already. This commit adds a step where the delta moves the duplicate emails to a new `medium` `email_old`. Only the most recently used account keeps the binding intact. We move rather than delete so that we retain some record of which emails were associated with which account.
* Fix incredubly slow back pagination queryErik Johnston2016-10-241-6/+13
| | | | | | | If a client didn't specify a from token when paginating backwards synapse would attempt to query the (global) maximum topological token. This a) doesn't make much sense since they're room specific and b) there are no indices that lets postgres do this efficiently.
* Merge pull request #1164 from pik/error-codesErik Johnston2016-10-191-1/+0
|\ | | | | Clarify Error codes for GET /filter/
| * Refactor test_filter to use real DataStorepik2016-10-181-1/+0
| | | | | | | | * add tests for filter api errors
* | Latest delta is now 37David Baker2016-10-191-0/+0
| |
* | Merge remote-tracking branch 'origin/develop' into ↵David Baker2016-10-193-2/+82
|\ \ | | | | | | | | | dbkr/password_reset_case_insensitive
| * | Bump schema versionErik Johnston2016-10-172-1/+1
| | |
| * | Remove TODOErik Johnston2016-10-171-2/+0
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_authErik Johnston2016-10-1712-72/+429
| |\ \
| * | | Drop some unused indicesErik Johnston2016-10-173-17/+83
| | | |
| * | | Remove redundant event_auth indexErik Johnston2016-09-131-0/+16
| | | |
* | | | Convert emails to lowercase when storingDavid Baker2016-10-191-1/+8
| | | | | | | | | | | | | | | | And db migration sql to convert existing addresses.
* | | | Revert part of 6207399David Baker2016-10-141-20/+10
| | | | | | | | | | | | | | | | | | | | older sqlite doesn't support indexes on expressions, lets just store things lowercase in the db
* | | | Merge remote-tracking branch 'origin/develop' into ↵David Baker2016-10-141-0/+3
|\ \ \ \ | | |/ / | |/| | | | | | dbkr/password_reset_case_insensitive
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2016-10-113-44/+64
| |\ \ \ | | | | | | | | | | | | | | | erikj/replication_noop
| * | | | Reduce DB hits for replicationErik Johnston2016-09-231-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.
* | | | | Make password reset email field case insensitiveDavid Baker2016-10-142-10/+36
| |/ / / |/| | |
* | | | storage/appservice: make appservice methods only relying on the cache ↵Patrik Oldsberg2016-10-061-6/+6
| |_|/ |/| | | | | | | | synchronous
* | | Merge pull request #1145 from matrix-org/erikj/fix_reindexErik Johnston2016-09-291-15/+29
|\ \ \ | | | | | | | | Fix background reindex of origin_server_ts
| * | | Fix background reindex of origin_server_tsErik Johnston2016-09-271-15/+29
| |/ / | | | | | | | | | | | | | | | | | | | | | The storage function `_get_events_txn` was removed everywhere except from this background reindex. The function was removed due to it being (almost) completely unused while also being large and complex. Therefore, instead of resurrecting `_get_events_txn` we manually reimplement the bits that are needed directly.
* | | Remove duplicationErik Johnston2016-09-271-31/+19
| | |
* | | Remove unused variableErik Johnston2016-09-271-3/+0
| | |
* | | Fix perf of fetching state in SQLiteErik Johnston2016-09-261-20/+41
|/ /
* | Match against event_id, rather than room_idErik Johnston2016-09-211-1/+1
| |
* | Remove spurious AS clauseErik Johnston2016-09-211-1/+1
| |
* | Update correct tableErik Johnston2016-09-211-6/+10
| |
* | Readd entries to public_room_list_stream that were deletedErik Johnston2016-09-212-1/+23
| |
* | Fix _delete_old_forward_extrem_cache queryErik Johnston2016-09-211-5/+4
| |
* | CommentErik Johnston2016-09-161-0/+2
| |
* | Filter remote rooms lists locallyErik Johnston2016-09-161-1/+1
| |
* | Handling expiring stream extrems correctly.Erik Johnston2016-09-152-3/+22
| |
* | Change get_pos_of_last_change to return upper boundErik Johnston2016-09-151-3/+2
| |
* | Use stream_change cache to make get_forward_extremeties_for_room cache more ↵Erik Johnston2016-09-151-1/+10
| | | | | | | | effective
* | Add cache to get_forward_extremeties_for_roomErik Johnston2016-09-151-0/+1
| |
* | Merge pull request #1121 from matrix-org/erikj/public_room_paginateErik Johnston2016-09-156-15/+288
|\ \ | | | | | | Add pagination support to publicRooms
| * | Stream public room changes down replicationErik Johnston2016-09-151-0/+16
| | |
| * | Base public room list off of public_rooms streamErik Johnston2016-09-151-0/+52
| | |
| * | Calculate the public room list from a stream_orderingErik Johnston2016-09-141-0/+3
| | |
| * | Add a room visibility streamErik Johnston2016-09-144-16/+100
| | |
| * | Add a DB index to figure out past state at a stream ordering in a roomErik Johnston2016-09-142-0/+118
| |/
* | Fix reindexErik Johnston2016-09-141-15/+16
| |
* | Ensure we don't mutate state cache entriesErik Johnston2016-09-141-20/+25
|/
* Remove where clauseErik Johnston2016-09-121-1/+0
|
* Index contains_url for file search queriesErik Johnston2016-09-122-0/+25
|
* Add WHERE clause support to index creationErik Johnston2016-09-122-10/+17
|
* Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrentErik Johnston2016-09-121-1/+1
|\ | | | | Create new index concurrently
| * Create new index concurrentlyErik Johnston2016-09-121-1/+1
| |
* | Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fastErik Johnston2016-09-122-1/+29
|\ \ | |/ |/| Optimise /notifications query
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-14 10:56:42 +0000