summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #6484 from matrix-org/erikj/port_sync_handlerErik Johnston2019-12-092-3/+3
|\ | | | | Port SyncHandler to async/await
| * Fixup functions to consistently return deferredsErik Johnston2019-12-062-3/+3
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-12-063-47/+71
|\ \ | | | | | | | | | erikj/make_database_class
| * | Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925)Manuel Stahl2019-12-053-44/+71
| |/
* | Remove unused varErik Johnston2019-12-061-2/+0
| |
* | Move background update handling out of storeErik Johnston2019-12-0517-146/+163
| |
* | CommentsErik Johnston2019-12-053-5/+16
| |
* | Move DB pool and helper functions into dedicated Database classErik Johnston2019-12-0544-2192/+2271
|/
* Merge pull request #6464 from matrix-org/erikj/make_public_sql_baseErik Johnston2019-12-0543-507/+508
|\ | | | | Clean up SQLBaseStore private function usage
| * Remove underscore from SQLBaseStore functionsErik Johnston2019-12-0443-500/+492
| |
| * Don't call SQLBaseStore methods from outside storesErik Johnston2019-12-042-8/+17
| |
* | Merge pull request #6470 from matrix-org/babolivier/port_db_ci_failureBrendan Abolivier2019-12-041-2/+10
|\ \ | | | | | | Make synapse_port_db exit with a non-0 code if something failed
| * | Fix background updates for synapse_port_dbBrendan Abolivier2019-12-041-2/+10
| |/
* / get rid of (most of) have_events from ↵Richard van der Hoff2019-12-041-34/+0
|/ | | | | | | | | | | | | _update_auth_events_and_context_for_auth (#6468) have_events was a map from event_id to rejection reason (or None) for events which are in our local database. It was used as filter on the list of event_ids being passed into get_events_as_list. However, since get_events_as_list will ignore any event_ids that are unknown or rejected, we can equivalently just leave it to get_events_as_list to do the filtering. That means that we don't have to keep `have_events` up-to-date, and can use `have_seen_events` instead of `get_seen_events_with_rejection` in the one place we do need it.
* Merge pull request #6454 from matrix-org/erikj/clean_base_StoreErik Johnston2019-12-047-190/+218
|\ | | | | Move things out of SQLBaseStore
| * Revert "Move get_user_count_txn out of base store"Erik Johnston2019-12-042-12/+12
| | | | | | | | | | | | This reverts commit 00f0d67566cdfe8eae44aeae1c982c42a255cfcd. Its going to get removed soon, so lets not make merge conflicts.
| * _CURRENT_STATE_CACHE_NAME is publicErik Johnston2019-12-041-3/+3
| |
| * Move get_user_count_txn out of base storeErik Johnston2019-12-042-12/+12
| |
| * Move cache invalidation to main data storeErik Johnston2019-12-045-112/+141
| |
| * Move event fetch vars to EventWorkStoreErik Johnston2019-12-044-14/+15
| |
| * Move account validity bg updates to registration storeErik Johnston2019-12-042-66/+64
| |
* | Merge pull request #6329 from matrix-org/babolivier/context_filtersBrendan Abolivier2019-12-041-0/+3
|\ \ | |/ |/| Filter state, events_before and events_after in /context requests
| * Merge branch 'develop' into babolivier/context_filtersBrendan Abolivier2019-12-0422-305/+1024
| |\
| * | Update copyrightsBrendan Abolivier2019-11-051-0/+3
| | |
* | | Merge pull request #6451 from matrix-org/uhoreg/cross_signing_signatures_indexErik Johnston2019-12-042-1/+24
|\ \ \ | | | | | | | | make cross signing signature index non-unique
| * | | apply changes from reviewHubert Chathi2019-12-031-1/+2
| | | |
| * | | make cross signing signature index non-uniqueHubert Chathi2019-12-022-1/+23
| | |/ | |/|
* | | Fix exception when a cross-signed device is deleted (#6462)Richard van der Hoff2019-12-041-4/+19
| | | | | | | | | | | | | | | (hopefully) ... and deobfuscate the relevant bit of code.
* | | Add ephemeral messages support (MSC2228) (#6409)Brendan Abolivier2019-12-032-6/+141
|/ / | | | | | | | | | | | | | | Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are: * the feature is hidden behind a configuration flag (`enable_ephemeral_messages`) * self-destruction doesn't happen for state events * only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one) * doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
* | Remove local threepids on account deactivation (#6426)Andrew Morgan2019-11-281-0/+13
| |
* | add etag and count to key backup endpoints (#5858)Hubert Chathi2019-11-272-66/+177
| |
* | Merge branch 'develop' into babolivier/message_retentionBrendan Abolivier2019-11-2618-239/+547
|\ \
| * | Don't construct a setErik Johnston2019-11-261-6/+3
| | |
| * | Fix find_next_generated_user_id_localpartErik Johnston2019-11-261-10/+7
| | |
| * | Prevent account_data content from being sent over TCP replication (#6333)Andrew Morgan2019-11-261-3/+3
| |\ \
| | * | Remove content from being sent for account data rdata streamAndrew Morgan2019-11-081-3/+3
| | | |
| * | | Make sure that we close cursors before returning from a query (#6408)Richard van der Hoff2019-11-252-10/+43
| | | | | | | | | | | | | | | | | | | | | | | | There are lots of words in the comment as to why this is a good idea. Fixes #6403.
| * | | Merge tag 'v1.6.0rc2' into developRichard van der Hoff2019-11-251-18/+25
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.6.0rc2 (2019-11-25) ============================= Bugfixes -------- - Fix a bug which could cause the background database update hander for event labels to get stuck in a loop raising exceptions. ([\#6407](https://github.com/matrix-org/synapse/issues/6407))
| | * | | Fix exceptions from background database update for event labels. (#6407)Richard van der Hoff2019-11-251-18/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add some exception handling here so that events whose json cannot be parsed are ignored rather than getting us stuck in a loop. Fixes #6404.
| * | | | Clean up newline quote marks around the codebase (#6362)Andrew Morgan2019-11-2110-22/+16
| |/ / /
| * | | Replace UPDATE with UPSERT on device_max_stream_id table (#6363)Andrew Morgan2019-11-151-2/+15
| | | |
| * | | Fix guest -> real account upgrade with account validity enabled (#6359)Andrew Morgan2019-11-141-6/+3
| |/ /
| * | Merge pull request #6340 from matrix-org/babolivier/pagination_queryBrendan Abolivier2019-11-081-8/+32
| |\ \ | | | | | | | | Fix the SQL SELECT query in _paginate_room_events_txn
| | * | Incorporate reviewBrendan Abolivier2019-11-081-12/+12
| | | |
| | * | LintBrendan Abolivier2019-11-071-2/+2
| | | |
| | * | Only join on event_labels if we're filtering on labelsBrendan Abolivier2019-11-071-9/+24
| | | |
| | * | Handle lack of filterBrendan Abolivier2019-11-071-5/+3
| | | |
| | * | Fix the SQL SELECT query in _paginate_room_events_txnBrendan Abolivier2019-11-071-2/+13
| | | | | | | | | | | | | | | | Doing a SELECT DISTINCT when paginating is quite expensive, because it requires the engine to do sorting on the entire events table. However, we only need to run it if we're filtering on 2+ labels, so this PR is changing the request so that DISTINCT is only used then.
| * | | Merge pull request #6295 from matrix-org/erikj/split_purge_historyErik Johnston2019-11-084-172/+319
| |\ \ \ | | | | | | | | | | Split purge API into events vs state and add PurgeEventsStorage
| | * | | Move type annotation into docstringErik Johnston2019-11-081-3/+3
| | | | |
| | * | | Fix deleting state groups during room purge.Erik Johnston2019-11-061-13/+14
| | | | | | | | | | | | | | | | | | | | And fix the tests to actually test that things got deleted.
| | * | | Use correct type annotationErik Johnston2019-11-061-2/+2
| | | | |
| | * | | Change to not require a state_groups.room_id index.Erik Johnston2019-11-044-53/+45
| | | | | | | | | | | | | | | | | | | | | | | | | This does mean that we won't clean up orphaned state groups (i.e. state groups that were persisted but the associated event wasn't).
| | * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-11-0418-54/+259
| | |\ \ \ | | | | |/ | | | |/| | | | | | erikj/split_purge_history
| | * | | Fix up commentErik Johnston2019-11-011-2/+1
| | | | |
| | * | | Update log line to lie a little lessErik Johnston2019-10-311-1/+1
| | | | |
| | * | | Add state_groups.room_id indexErik Johnston2019-10-312-0/+24
| | | | |
| | * | | DocstringsErik Johnston2019-10-311-3/+11
| | | | |
| | * | | Fix purge room APIErik Johnston2019-10-311-5/+8
| | | | |
| | * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-317-12/+243
| | |\ \ \ | | | | | | | | | | | | | | | | | | erikj/split_purge_history
| | * | | | Split purge API into events vs stateErik Johnston2019-10-304-175/+295
| | | | | |
| * | | | | Merge pull request #6310 from matrix-org/babolivier/msc2326_bg_updateBrendan Abolivier2019-11-072-0/+80
| |\ \ \ \ \ | | | | | | | | | | | | | | MSC2326: Add background update to take previous events into account
| | * | | | | Update synapse/storage/data_stores/main/events_bg_updates.pyBrendan Abolivier2019-11-071-1/+1
| | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | Update synapse/storage/data_stores/main/events_bg_updates.pyBrendan Abolivier2019-11-071-1/+1
| | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | Copy resultsBrendan Abolivier2019-11-071-1/+3
| | | | | | |
| | * | | | | Revert "Back to using cursor_to_dict"Brendan Abolivier2019-11-071-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 1186612d6cd728e6b7ed7806579db3cea7410b54.
| | * | | | | Back to using cursor_to_dictBrendan Abolivier2019-11-071-9/+9
| | | | | | |
| | * | | | | Initialise value before loopingBrendan Abolivier2019-11-071-4/+8
| | | | | | |
| | * | | | | Incorporate reviewBrendan Abolivier2019-11-071-14/+9
| | | | | | |
| | * | | | | Fix field nameBrendan Abolivier2019-11-061-1/+1
| | | | | | |
| | * | | | | Update insertBrendan Abolivier2019-11-061-1/+6
| | | | | | |
| | * | | | | Print out the actual number of affected rowsBrendan Abolivier2019-11-041-4/+4
| | | | | | |
| | * | | | | Correctly order resultsBrendan Abolivier2019-11-041-5/+5
| | | | | | |
| | * | | | | TODOBrendan Abolivier2019-11-041-0/+2
| | | | | | |
| | * | | | | Fix exit conditionBrendan Abolivier2019-11-041-1/+3
| | | | | | |
| | * | | | | LintBrendan Abolivier2019-11-041-6/+3
| | | | | | |
| | * | | | | Don't try to process events we already have a label forBrendan Abolivier2019-11-041-1/+2
| | | | | | |
| | * | | | | Use a sensible default value for labelsBrendan Abolivier2019-11-041-1/+1
| | | | | | |
| | * | | | | Use the right format for rowsBrendan Abolivier2019-11-041-1/+1
| | | | | | |
| | * | | | | MSC2326: Add background update to take previous events into accountBrendan Abolivier2019-11-042-0/+72
| | | |_|/ / | | |/| | |
| * | | | | Merge pull request #6235 from matrix-org/anoa/room_upgrade_groupsAndrew Morgan2019-11-072-1/+20
| |\ \ \ \ \
| | * | | | | Re-add docstring, with caveats detailedAndrew Morgan2019-11-042-2/+6
| | | | | | |
| | * | | | | Transfer upgraded rooms on groupsAndrew Morgan2019-11-041-0/+15
| | |/ / / /
| * | | | / Numeric ID checker now checks @0, don't ratelimit on checkingAndrew Morgan2019-11-061-4/+4
| | |_|_|/ | |/| | |
| * | | | Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-1/+1
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
* | | | Fix worker modeBrendan Abolivier2019-11-191-56/+56
| | | |
* | | | LintBrendan Abolivier2019-11-191-25/+24
| | | |
* | | | Implement per-room message retention policiesBrendan Abolivier2019-11-043-0/+288
|/ / /
* | | Merge pull request #6301 from matrix-org/babolivier/msc2326Brendan Abolivier2019-11-013-2/+75
|\ \ \ | | | | | | | | Implement MSC2326 (label based filtering)
| * | | Incorporate reviewBrendan Abolivier2019-11-012-1/+7
| | | |
| * | | Update synapse/storage/data_stores/main/schema/delta/56/event_labels.sqlBrendan Abolivier2019-11-011-1/+1
| | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| * | | Add more data to the event_labels table and fix the indexesBrendan Abolivier2019-11-013-5/+21
| | | |
| * | | LintBrendan Abolivier2019-11-011-1/+1
| | | |
| * | | Incorporate reviewBrendan Abolivier2019-11-011-2/+10
| | | |
| * | | LintBrendan Abolivier2019-10-301-7/+1
| | | |
| * | | Add more integration testingBrendan Abolivier2019-10-301-1/+1
| | | |
| * | | Add index on labelBrendan Abolivier2019-10-301-1/+3
| | | |
| * | | Implement filteringBrendan Abolivier2019-10-301-0/+9
| | | |
| * | | Store labels for new eventsBrendan Abolivier2019-10-301-1/+19
| | | |
| * | | Add database table for keeping track of labels on eventsBrendan Abolivier2019-10-291-0/+20
| | | |
* | | | Merge pull request #6313 from matrix-org/uhoreg/cross_signing_fix_sqlite_schemaHubert Chathi2019-11-011-0/+42
|\ \ \ \ | | | | | | | | | | fix hidden field in devices table for older sqlite
| * | | | fix hidden field in devices table for older sqliteHubert Chathi2019-10-311-0/+42
| | | | |
* | | | | Update synapse/storage/data_stores/main/devices.pyHubert Chathi2019-10-311-1/+1
| | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* | | | | Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-3119-40/+300
|\| | | |
| * | | | Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notifyHubert Chathi2019-10-3118-39/+272
| |\ \ \ \
| | * | | | Update black to 19.10b0 (#6304)Amber Brown2019-11-0113-27/+29
| | | |_|/ | | |/| | | | | | | * update version of black and also fix the mypy config being overridden
| | * | | Merge pull request #6294 from matrix-org/erikj/add_state_storageErik Johnston2019-10-313-1/+236
| | |\ \ \ | | | | | | | | | | | | Add StateGroupStorage interface
| | | * | | Apply suggestions from code review Erik Johnston2019-10-311-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix docstring Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | | * | | Add StateGroupStorage interfaceErik Johnston2019-10-303-1/+235
| | | | |/ | | | |/|
| | * | | Fix /purge_room API.Erik Johnston2019-10-311-1/+0
| | | | | | | | | | | | | | | | | | | | It fails trying to clean the `topic` table which was recently removed.
| | * | | Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-313-5/+5
| | | | | | | | | | | | | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
| | * | | fix delete_existing for _persist_events (#6300)Richard van der Hoff2019-10-302-5/+2
| | |/ / | | | | | | | | this is part of _retry_on_integrity_error, so should only be on _persist_events_and_state_updates
| * | | make user signatures a separate streamHubert Chathi2019-10-302-12/+25
| | | |
| * | | Merge branch 'develop' into uhoreg/cross_signing_fix_workers_notifyHubert Chathi2019-10-3013-724/+889
| |\| |
| * | | make notification of signatures work with workersHubert Chathi2019-10-242-2/+16
| | | |
* | | | rename get_devices_by_remote to get_device_updates_by_remoteHubert Chathi2019-10-301-4/+4
| | | |
* | | | Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-3012-657/+822
|\ \ \ \ | | |/ / | |/| |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-3012-84/+126
| |\ \ \ | | | | | | | | | | | | | | | erikj/split_out_persistence_store
| | * | | Port receipt and read markers to async/waitErik Johnston2019-10-291-4/+3
| | | |/ | | |/|
| | * | Merge pull request #6263 from matrix-org/erikj/caches_return_deferredsErik Johnston2019-10-291-1/+1
| | |\ \ | | | | | | | | | | Quick fix to ensure cache descriptors always return deferreds
| | | * | Quick fix to ensure cache descriptors always return deferredsErik Johnston2019-10-281-1/+1
| | | | |
| | * | | Merge branch 'release-v1.5.0' of github.com:matrix-org/synapse into developErik Johnston2019-10-291-4/+8
| | |\ \ \
| | | * | | Make room directory search case insensitiveErik Johnston2019-10-291-4/+8
| | | | | |
| | * | | | Add CI for synapse_port_db (#6140)Brendan Abolivier2019-10-281-3/+6
| | | |/ / | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds: * a test sqlite database * a configuration file for the sqlite database * a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`) as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that: 1. installs Synapse 2. updates the test sqlite database to the latest schema and runs background updates on it 3. creates an empty postgresql database 4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage) Step `2` is done via a new script located at `scripts-dev/update_database`. The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.
| | * | | Convert EventContext to attrs (#6218)Richard van der Hoff2019-10-281-1/+6
| | | | | | | | | | | | | | | | | | | | * make EventContext use an attr
| | * | | Merge pull request #6253 from matrix-org/uhoreg/e2e_backup_delete_keysHubert Chathi2019-10-252-0/+33
| | |\ \ \ | | | | | | | | | | | | delete keys when deleting backup versions
| | | * | | delete keys when deleting backupsHubert Chathi2019-10-242-0/+33
| | | | |/ | | | |/|
| | * | | Merge branch 'release-v1.5.0' of github.com:matrix-org/synapse into developErik Johnston2019-10-251-67/+67
| | |\ \ \ | | | | |/ | | | |/|
| | | * | Fix /keys/query API on workers.Erik Johnston2019-10-251-67/+67
| | | |/ | | | | | | | | | | | | | | | | The necessary getters were added only to the master store and not the worker stores.
| | * | Merge pull request #6251 from matrix-org/michaelkaye/debug_guard_loggingErik Johnston2019-10-251-3/+1
| | |\ \ | | | | | | | | | | Reduce debug logging overhead
| | | * | Address codestyle concernsMichael Kaye2019-10-241-3/+1
| | | | |
| | | * | use %r to __repr__ objectsMichael Kaye2019-10-241-1/+1
| | | |/ | | | | | | | | | | | | This avoids calculating __repr__ unless we are going to log.
| | * / Make user/room stats log line less verbose.Michael Kaye2019-10-241-1/+1
| | |/
| * | Review commentsErik Johnston2019-10-303-10/+26
| | |
| * | Add DataStores and Storage classes.Erik Johnston2019-10-233-4/+34
| | |
| * | Move persist_events out from main data store.Erik Johnston2019-10-232-634/+711
| | | | | | | | | | | | | | | | | | This is in preparation for splitting out of state_groups_state from the main store into it own one, as persisting events depends on calculating state.
* | | apply changes as a result of PR reviewHubert Chathi2019-10-301-43/+36
| | |
* | | Merge branch 'develop' into cross-signing_federationHubert Chathi2019-10-24241-4341/+4597
|\ \ \ | | |/ | |/|
| * | Move schema delta files to the correct data store.Erik Johnston2019-10-242-0/+0
| |/ | | | | | | They were put in the global schema delta directory due to a bad merge.
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-221-11/+37
| |\ | | | | | | | | | erikj/refactor_stores
| * | Clean up prepare_database.py a bit and add commentsErik Johnston2019-10-221-7/+30
| | |
| * | Delete background_update table creation in main data_storeErik Johnston2019-10-221-21/+0
| | |
| * | Move README into synapse/storage/__init__.pyErik Johnston2019-10-222-13/+12
| | |
| * | Add a basic README to synapse.storageErik Johnston2019-10-221-0/+13
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-226-12/+337
| |\ \ | | | | | | | | | | | | erikj/refactor_stores
| * | | Fix schema management to work with multiple data stores.Erik Johnston2019-10-211-41/+108
| | | |
| * | | Move storage classes into a main "data store".Erik Johnston2019-10-21239-4292/+4466
| | | | | | | | | | | | | | | | | | | | This is in preparation for having multiple data stores that offer different functionality, e.g. splitting out state or event storage.
* | | | move get_e2e_cross_signing_key to EndToEndKeyWorkerStore so it works with ↵Hubert Chathi2019-10-231-67/+67
| | | | | | | | | | | | | | | | workers
* | | | vendor-prefix the EDU name until MSC1756 is merged into the specHubert Chathi2019-10-221-1/+2
| | | |
* | | | update to work with newer code, and fix formattingHubert Chathi2019-10-221-1/+1
| | | |
* | | | make black happyHubert Chathi2019-10-221-22/+32
| | | |
* | | | don't crash if the user doesn't have cross-signing keysHubert Chathi2019-10-221-14/+25
| | | |
* | | | implement federation parts of cross-signingHubert Chathi2019-10-221-5/+51
| |_|/ |/| |
* | | Merge branch 'develop' into cross-signing_sig_uploadHubert Chathi2019-10-1838-908/+1949
|\ \ \ | | |/ | |/|
| * | Merge branch 'develop' into uhoreg/e2e_cross-signing_mergedHubert Chathi2019-10-1838-903/+1945
| |\|
| | * Merge pull request #6193 from matrix-org/uhoreg/interpret_device_key_in_storageHubert Chathi2019-10-111-4/+14
| | |\ | | | | | | | | make storage layer in charge of interpreting the device key data
| | | * expand on commentHubert Chathi2019-10-111-1/+2
| | | |
| | | * make sure we actually return somethingHubert Chathi2019-10-101-2/+4
| | | |
| | | * make storage layer in charge of interpreting the device key dataHubert Chathi2019-10-101-2/+9
| | | |
| | * | Fix MAU reaping where reserved users are specified. (#6168)Neil Johnson2019-10-111-39/+62
| | | |
| | * | Merge pull request #6156 from matrix-org/erikj/postgres_anyErik Johnston2019-10-1014-109/+189
| | |\ \ | | | | | | | | | | Use Postgres ANY for selecting many values.
| | | * | Fix SQLite take 2Erik Johnston2019-10-101-1/+1
| | | | |
| | | * | Fix SQLiteErik Johnston2019-10-101-1/+1
| | | | |
| | | * | Remove add_in_list_sql_clauseErik Johnston2019-10-104-34/+25
| | | | |
| | | * | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_anyErik Johnston2019-10-101-3/+4
| | | |\ \
| | | * | | Replace IN usage with helper funcsErik Johnston2019-10-1011-97/+138
| | | | | |
| | | * | | Add helper funcs to use postgres ANYErik Johnston2019-10-101-8/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This means that we can write queries with `col = ANY(?)`, which helps postgres.
| | * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-101-3/+4
| | |\ \ \ \ | | | | |/ / | | | |/| | | | | | | | erikj/disable_sql_bytes
| | | * | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_innerErik Johnston2019-10-102-35/+61
| | | |\| |
| | | * | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_innerErik Johnston2019-10-0923-706/+1180
| | | |\ \ \ | | | | | |/ | | | | |/|
| | | * | | Patch inlinecallbacks for log contextsErik Johnston2019-09-272-4/+5
| | | | | |
| | * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-1016-725/+1030
| | |\ \ \ \ | | | | |_|/ | | | |/| | | | | | | | erikj/disable_sql_bytes
| | | * | | Fix races in room stats (and other) updates. (#6187)Richard van der Hoff2019-10-101-9/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hopefully this will fix the occasional failures we were seeing in the room directory. The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8. We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
| | | * | | Rewrite the user_filter migration again (#6184)Richard van der Hoff2019-10-101-26/+32
| | | | |/ | | | |/| | | | | | | | | | | | | | | | you can't plausibly ALTER TABLE in sqlite, so we create the new table with the right schema to start with.
| | | * | Merge pull request #6185 from matrix-org/erikj/fix_censored_evnetsErik Johnston2019-10-092-4/+28
| | | |\ \ | | | | | | | | | | | | Fix inserting bytes as text in `censor_redactions`
| | | * | | Fix RegistrationStoreBrendan Abolivier2019-10-081-1/+1
| | | | | |
| | | * | | Cleanup client_ipsBrendan Abolivier2019-10-081-7/+0
| | | | | |
| | | * | | Don't end up with 4 classes in registrationBrendan Abolivier2019-10-081-52/+50
| | | | | |
| | | * | | LintBrendan Abolivier2019-10-073-3/+3
| | | | | |
| | | * | | Move roommember's bg updates to a dedicated storeBrendan Abolivier2019-10-071-108/+114
| | | | | |
| | | * | | Move user_directory's bg updates to a dedicated storeBrendan Abolivier2019-10-071-84/+94
| | | | | |
| | | * | | Move state's bg updates to a dedicated storeBrendan Abolivier2019-10-071-190/+204
| | | | | |
| | | * | | Move search's bg updates to a dedicated storeBrendan Abolivier2019-10-071-25/+31
| | | | | |
| | | * | | Move registration's bg updates to a dedicated storeBrendan Abolivier2019-10-071-95/+103
| | | | | |
| | | * | | Move media_repository's bg updates to a dedicated storeBrendan Abolivier2019-10-071-3/+9
| | | | | |
| | | * | | Move devices's bg updates to a dedicated storeBrendan Abolivier2019-10-071-22/+27
| | | | | |
| | | * | | Move deviceinbox's bg updates to a dedicated storeBrendan Abolivier2019-10-071-15/+22
| | | | | |
| | | * | | Move client_ips's bg updates to a dedicated storeBrendan Abolivier2019-10-071-94/+106
| | | | | |
| | | * | | add some logging to the rooms stats updates, to try to track down a flaky ↵Richard van der Hoff2019-10-071-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | test (#6167)
| | | * | | Merge pull request #6175 from matrix-org/babolivier/fix_unique_user_filter_indexBrendan Abolivier2019-10-071-2/+2
| | | |\ \ \ | | | | | | | | | | | | | | Fix unique_user_filter_index schema update
| | | | * | | Fix unique_user_filter_index schema updateBrendan Abolivier2019-10-071-2/+2
| | | | | | |
| | | * | | | Merge pull request #6159 from matrix-org/erikj/cache_membershipsErik Johnston2019-10-071-19/+45
| | | |\ \ \ \ | | | | |/ / / | | | |/| | | Cache room membership lookups in _get_joined_users_from_context
| | | | * | | Fix bug where we didn't pull out event IDErik Johnston2019-10-071-1/+1
| | | | | | |
| | | | * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-031-6/+15
| | | | |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | erikj/cache_memberships
| | | | * | | | Use the right error....Erik Johnston2019-10-031-1/+1
| | | | | | | |
| | | | * | | | pep8Erik Johnston2019-10-031-1/+0
| | | | | | | |
| | | | * | | | Fixup names and commentsErik Johnston2019-10-031-6/+7
| | | | | | | |
| | | | * | | | cachedList descriptor doesn't like typingErik Johnston2019-10-031-2/+2
| | | | | | | |
| | | | * | | | Cache room membership lookups in _get_joined_users_from_contextErik Johnston2019-10-031-19/+45
| | | | | | | |
| | | * | | | | Update `user_filters` table to have a unique index, and non-null columns (#1172)Alexander Maznev2019-10-041-0/+46
| | | | |/ / / | | | |/| | |
| | | * | | | add some metrics on the federation sender (#6160)Richard van der Hoff2019-10-031-6/+15
| | | |/ / /
| | | * | | Merge pull request #6154 from matrix-org/erikj/fix_appservice_paginationErik Johnston2019-10-021-18/+18
| | | |\ \ \ | | | | | | | | | | | | | | Fix appservice room list pagination
| | | | * | | Fix appservice room list paginationErik Johnston2019-10-021-18/+18
| | | | | | |
| | | * | | | Merge pull request #6153 from matrix-org/erikj/fix_room_list_non_federatableErik Johnston2019-10-021-0/+3
| | | |\ \ \ \ | | | | | | | | | | | | | | | | Fix not showing non-federatable rooms to remote room list queries
| | | | * | | | Fix not showing non-federatable rooms to remote room list queriesErik Johnston2019-10-021-0/+3
| | | | |/ / /
| | | * | | | Merge pull request #6148 from matrix-org/erikj/find_next_generatedErik Johnston2019-10-021-1/+3
| | | |\ \ \ \ | | | | |/ / / | | | |/| | | Bound find_next_generated_user_id DB query.
| | | | * | | Bound find_next_generated_user_id DB query.Erik Johnston2019-10-021-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We can easily bound the set of user IDs we pull out of the DB, so lets do that.
| | | * | | | Fix public room list pagination.Erik Johnston2019-10-021-17/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We incorrectly used `room_id` as to bound the result set, even though we order by `joined_members, room_id`, leading to incorrect results after pagination.
| | | * | | | Land improved room list based on room stats (#6019)Erik Johnston2019-10-022-77/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Use room_stats and room_state for room directory search
| | | * | | | Fix yields and copy instead of move push rules on room upgrade (#6144)Andrew Morgan2019-10-021-10/+6
| | | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Copy push rules during a room upgrade from the old room to the new room, instead of deleting them from the old room. For instance, we've defined upgrading of a room multiple times to be possible, and push rules won't be transferred on the second upgrade if they're deleted during the first. Also fix some missing yields that probably broke things quite a bit.
| | | * | | Merge branch 'release-v1.4.0' of github.com:matrix-org/synapse into developErik Johnston2019-10-025-18/+142
| | | |\ \ \
| | | * \ \ \ Merge branch 'erikj/fixup_devices_last_seen_query' of ↵Erik Johnston2019-10-013-7/+54
| | | |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | github.com:matrix-org/synapse into develop
| | | * | | | | Drop unused tables (#6115)Richard van der Hoff2019-09-301-0/+20
| | | | |_|_|/ | | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These tables are unused since #5893 (as amended by #6047), so we can now drop them. Fixes #6048.
| | * | | | | Do the update as a background indexErik Johnston2019-10-092-9/+51
| | | | | | |
| | * | | | | Disable bytes usage with postgresErik Johnston2019-10-083-3/+10
| | | |_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | More often than not passing bytes to `txn.execute` is a bug (where we meant to pass a string) that just happens to work if `BYTEA_OUTPUT` is set to `ESCAPE`. However, this is a bit of a footgun so we want to instead error when this happens, and force using `bytearray` if we actually want to use bytes.
| | * | | | Fix existing hex encoded json values in DBErik Johnston2019-10-081-0/+26
| | | | | |
| | * | | | Fix inserting bytes as textErik Johnston2019-10-081-4/+2
| | | |_|/ | | |/| |
| | * | | Merge pull request #6146 from matrix-org/erikj/fix_destination_retry_timingsErik Johnston2019-10-021-0/+18
| | |\ \ \ | | | | | | | | | | | | Fix errors storing large retry intervals.
| | | * | | Fix errors storing large retry intervals.Erik Johnston2019-10-021-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have set the max retry interval to a value larger than a postgres or sqlite int can hold, which caused exceptions when updating the destinations table. To fix postgres we need to change the column to a bigint, and for sqlite we lower the max interval to 2**62 (which is still incredibly long).
| | * | | | Merge pull request #6145 from matrix-org/erikj/fix_censored_redactionsErik Johnston2019-10-021-0/+14
| | |\ \ \ \ | | | |/ / / | | |/| | | Fix fetching censored redactions from DB
| | | * | | Fix fetching censored redactions from DBErik Johnston2019-10-021-0/+14
| | | | |/ | | | |/| | | | | | | | | | | | | | | | | | | | | Fetching a censored redactions caused an exception due to the code expecting redactions to have a `redact` key, which redacted redactions don't have.
| | * | | Use `received_ts` to find uncensored redacted eventsErik Johnston2019-10-011-14/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Joining against `events` and ordering by `stream_ordering` is inefficient as it forced scanning the entirety of the redactions table. This isn't the case if we use `redactions.received_ts` column as we can then use an index.
| | * | | Add received_ts column to redactions.Erik Johnston2019-10-013-9/+92
| | | | | | | | | | | | | | | | | | | | | | | | | This will allow us to efficiently search for uncensored redactions in the DB before a given time.
| | * | | Don't repeatedly attempt to censor events we don't have.Erik Johnston2019-10-011-1/+17
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we don't set `have_censored` column if we don't have the target event of a redaction, which means we repeatedly attempt to censor the same non-existant event. When we persist non-redacted events we unset the `have_censored` column for any redactions that target said event.
| | * / Fix `devices_last_seen` background update.Erik Johnston2019-09-303-7/+54
| | |/ | | | | | | | | | Fixes #6134.
| | * Fix dummy event insertion consent bug (#6053)Neil Johnson2019-09-261-3/+15
| | | | | | | | | Fixes #5905
| | * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-09-254-9/+94
| | |\ | | | | | | | | | | | | erikj/cleanup_user_ips_2
| | | * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-09-253-8/+93
| | | |\ | | | | | | | | | | | | | | | erikj/cleanup_user_ips
| | | | * Merge pull request #6092 from ↵Brendan Abolivier2019-09-242-6/+6
| | | | |\ | | | | | | | | | | | | | | | | | | | | | | | | matrix-org/babolivier/background_update_deactivated_return Fix the return value in the users_set_deactivated_flag background job
| | | | | * Incorporate reviewBrendan Abolivier2019-09-241-3/+3
| | | | | |
| | | | | * Fix loggingBrendan Abolivier2019-09-241-1/+1
| | | | | |
| | | | | * TypoBrendan Abolivier2019-09-231-1/+1
| | | | | |
| | | | | * Fix the return value in the users_set_deactivated_flag background jobBrendan Abolivier2019-09-231-4/+4
| | | | | |
| | | | * | Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_workRichard van der Hoff2019-09-241-1/+1
| | | | |\ \
| | | | | * | remove unused parameter to get_user_id_by_threepid (#6099)Richard van der Hoff2019-09-241-1/+1
| | | | | |/ | | | | | | | | | | | | Added in #5377, apparently in error
| | | | * | Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_workRichard van der Hoff2019-09-242-28/+45
| | | | |\|
| | | | | * Implement MSC2290 (#6043)Andrew Morgan2019-09-231-1/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implements MSC2290. This PR adds two new endpoints, /unstable/account/3pid/add and /unstable/account/3pid/bind. Depending on the progress of that MSC the unstable prefix may go away. This PR also removes the blacklist on some 3PID tests which occurs in #6042, as the corresponding Sytest PR changes them to use the new endpoints. Finally, it also modifies the account deactivation code such that it doesn't just try to deactivate 3PIDs that were bound to the user's account, but any 3PIDs that were bound through the homeserver on that user's account.
| | | | * | Merge branch 'develop' into rav/saml_mapping_workRichard van der Hoff2019-09-1928-552/+1649
| | | | |\ \
| | | | * | | Record mappings from saml users in an external tableRichard van der Hoff2019-09-132-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We want to assign unique mxids to saml users based on an incrementing suffix. For that to work, we need to record the allocated mxid in a separate table.
| | | * | | | Use if `is not None`Erik Johnston2019-09-251-1/+1
| | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | Review commentsErik Johnston2019-09-252-5/+2
| | | | | | |
| | * | | | | Prune rows in user_ips older than configured periodErik Johnston2019-09-241-8/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Defaults to pruning everything older than 28d.
| | * | | | | Add has_completed_background_updateErik Johnston2019-09-241-1/+24
| | |/ / / / | | | | | | | | | | | | | | | | | | This allows checking if a specific background update has completed.
| | * | | | Query devices table for last seen info.Erik Johnston2019-09-231-49/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a) simpler than querying user_ips directly and b) means we can purge older entries from user_ips without losing the required info. The storage functions now no longer return the access_token, since it was unused.
| | * | | | Add BG update to populate devices last seen infoErik Johnston2019-09-232-0/+55
| | | | | |
| | * | | | Add last seen info to devices table.Erik Johnston2019-09-232-0/+36
| | | |_|/ | | |/| | | | | | | | | | | | | | | | | This allows us to purge old user_ips entries without having to preserve the latest last seen info for active devices.
| | * | | Allow HS to send emails when adding an email to the HS (#6042)Andrew Morgan2019-09-201-7/+24
| | | | |
| | * | | Undo the deletion of some tables (#6047)Richard van der Hoff2019-09-191-20/+0
| | | |/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a partial revert of #5893. The problem is that if we drop these tables in the same release as removing the code that writes to them, it prevents users users from being able to roll back to a previous release. So let's leave the tables in place for now, and remember to drop them in a subsequent release. (Note that these tables haven't been *read* for *years*, so any missing rows resulting from a temporary upgrade to vNext won't cause a problem.)
| | * | Add 'failure_ts' column to 'destinations' table (#6016)Richard van der Hoff2019-09-172-7/+41
| | | | | | | | | | | | | | | | Track the time that a server started failing at, for general analysis purposes.
| | * | Clean up some code in the retry logic (#6017)Richard van der Hoff2019-09-111-20/+0
| | | | | | | | | | | | | | | | * remove some unused code * make things which were constants into constants for efficiency and clarity
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-17 16:36:15 +0000