summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Implement the SHHS complexity API (#5216)Amber Brown2019-05-301-1/+49
|
* Correctly filter out extremities with soft failed prevs (#5274)Erik Johnston2019-05-291-3/+79
| | | | | | | | | | When we receive a soft failed event we, correctly, *do not* update the forward extremity table with the event. However, if we later receive an event that references the soft failed event we then need to remove the soft failed events prev events from the forward extremities table, otherwise we just build up forward extremities. Fixes #5269
* Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schemaBrendan Abolivier2019-05-281-0/+3
|\ | | | | Fix schema update for account validity
| * Fix schema update for account validityBrendan Abolivier2019-05-281-0/+3
| |
* | Fix logging for room stats background updateTravis Ralston2019-05-251-1/+1
| |
* | Store key validity time in the storage layerRichard van der Hoff2019-05-232-10/+44
| | | | | | | | | | | | | | | | This is a first step to checking that the key is valid at the required moment. The idea here is that, rather than passing VerifyKey objects in and out of the storage layer, we instead pass FetchKeyResult objects, which simply wrap the VerifyKey and add a valid_until_ts field.
* | Rewrite store_server_verify_key to store several keys at once (#5234)Richard van der Hoff2019-05-231-26/+39
| | | | | | | | | | Storing server keys hammered the database a bit. This replaces the implementation which stored a single key, with one which can do many updates at once.
* | Simplifications and comments in do_auth (#5227)Richard van der Hoff2019-05-231-1/+1
| | | | | | | | | | I was staring at this function trying to figure out wtf it was actually doing. This is (hopefully) a non-functional refactor which makes it a bit clearer.
* | Room Statistics (#4338)Amber Brown2019-05-216-3/+597
| |
* | Merge pull request #5203 from matrix-org/erikj/aggregate_by_senderErik Johnston2019-05-211-1/+1
|\ \ | | | | | | Only count aggregations from distinct senders
| * | Only count aggregations from distinct sendersErik Johnston2019-05-171-1/+1
| | | | | | | | | | | | As a user isn't allowed to send a single emoji more than once.
* | | Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactionsErik Johnston2019-05-211-3/+45
|\ \ \ | | | | | | | | Block attempts to annotate the same event twice
| * | | Block attempts to annotate the same event twiceErik Johnston2019-05-201-3/+45
| | | |
* | | | Exclude soft-failed events from fwd-extremity candidates. (#5146)Richard van der Hoff2019-05-211-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When considering the candidates to be forward-extremities, we must exclude soft failures. Hopefully fixes #5090.
* | | | Merge pull request #5204 from ↵Brendan Abolivier2019-05-212-12/+62
|\ \ \ \ | |/ / / |/| | / | | |/ | |/| matrix-org/babolivier/account_validity_expiration_date Add startup background job for account validity
| * | Do the select and insert in a single transactionBrendan Abolivier2019-05-211-10/+6
| | |
| * | Add startup background job for account validityBrendan Abolivier2019-05-172-12/+66
| | | | | | | | | | | | If account validity is enabled in the server's configuration, this job will run at startup as a background job and will stick an expiration date to any registered account missing one.
* | | Rename relation types to match MSCErik Johnston2019-05-201-4/+4
| | |
* | | Merge pull request #5209 from matrix-org/erikj/reactions_baseErik Johnston2019-05-204-4/+476
|\ \ \ | | | | | | | | Land basic reaction and edit support.
| * | | Correctly update aggregation counts after redactionErik Johnston2019-05-202-0/+20
| | |/ | |/|
| * | Fixup bsaed on review commentsErik Johnston2019-05-171-15/+17
| | |
| * | Add basic editing supportErik Johnston2019-05-161-2/+58
| | |
| * | Move parsing of tokens out of storage layerErik Johnston2019-05-161-14/+2
| | |
| * | Indirect tuple conversionErik Johnston2019-05-161-0/+6
| | |
| * | Fix relations in worker modeErik Johnston2019-05-162-5/+11
| | |
| * | Add cache to relationsErik Johnston2019-05-161-0/+8
| | |
| * | Add aggregations APIErik Johnston2019-05-161-6/+219
| | |
| * | Actually check for None rather falseyErik Johnston2019-05-151-2/+2
| | |
| * | Add simple pagination APIErik Johnston2019-05-151-0/+80
| | |
| * | Add simple send_relation API and track in DBErik Johnston2019-05-154-0/+93
| | |
* | | Spelling and clarificationsErik Johnston2019-05-171-5/+9
| | |
* | | Update docstringErik Johnston2019-05-161-1/+12
|/ /
* / Make generating SQL bounds for pagination genericErik Johnston2019-05-151-61/+118
|/ | | | | This will allow us to reuse the same structure when we paginate e.g. relations
* Update docstring with correct typeErik Johnston2019-05-151-1/+1
| | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Expose DataStore._get_events as get_events_as_listErik Johnston2019-05-145-29/+53
| | | | This is in preparation for reaction work which requires it.
* Limit the number of EDUs in transactions to 100 as expected by receiver (#5138)Quentin Dufour2019-05-091-1/+1
| | | Fixes #3951.
* Fix infinite loop in presence handlerRichard van der Hoff2019-04-261-0/+18
| | | | Fixes #5102
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-04-1769-3786/+2805
|\ | | | | | | babolivier/account_expiration
| * Merge pull request #5047 from matrix-org/babolivier/account_expirationBrendan Abolivier2019-04-172-18/+159
| |\ | | | | | | Send out emails with links to extend an account's validity period
| * \ Merge pull request #5033 from matrix-org/erikj/fix_schema_deltaErik Johnston2019-04-101-1/+3
| |\ \ | | | | | | | | Fix schema upgrade when dropping tables
| | * | Fix schema upgrade when dropping tablesErik Johnston2019-04-091-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We need to drop tables in the correct order due to foreign table constraints (on `application_services`), otherwise the DROP TABLE command will fail. Introduced in #4992.
| * | | Merge pull request #5027 from matrix-org/babolivier/account_expirationBrendan Abolivier2019-04-092-0/+54
| |\ \ \ | | |/ / | |/| | Add time-based account expiration
| * | | Rewrite Datastore.get_server_verify_keysRichard van der Hoff2019-04-091-28/+46
| | | | | | | | | | | | | | | | Rewrite this so that it doesn't hammer the database.
| * | | Remove unused server_tls_certificates functions (#5028)Richard van der Hoff2019-04-083-59/+5
| | | | | | | | | | | | | | | | These have been unused since #4120, and with the demise of perspectives, it is unlikely that they will ever be used again.
| * | | drop tables listed in #1830 (#4992)Neil Johnson2019-04-0824-752/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tables dropped: * application_services, * application_services_regex, * transaction_id_to_pdu, * stats_reporting * current_state_resets * event_content_hashes * event_destinations * event_edge_hashes * event_signatures * feedback * room_hosts * state_forward_extremities
| * | | Merge pull request #5001 from matrix-org/rav/keyring_cleanupsRichard van der Hoff2019-04-081-2/+2
| |\ \ \ | | | | | | | | | | Cleanups in the Keyring
| | * | | Fix docstring on get_server_keys_jsonRichard van der Hoff2019-04-041-2/+2
| | | | |
| * | | | Clean up the database pagination code (#5007)Amber Brown2019-04-052-67/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * rewrite & simplify * changelog * cleanup potential sql injection
| * | | | Add functions to delete a groupErik Johnston2019-04-031-0/+37
| | | | |
| * | | | Fix grammar and document get_current_users_in_room (#4998)Andrew Morgan2019-04-031-1/+1
| |/ / /
| * | | Remove presence lists (#4989)Neil Johnson2019-04-034-96/+20
| | | | | | | | | | | | Remove presence list support as per MSC 1819
| * | | Merge pull request #4982 from matrix-org/erikj/msc1915Erik Johnston2019-04-032-0/+141
| |\ \ \ | | | | | | | | | | Implement MSC1915 - 3PID unbind APIs
| | * | | Fixup docstringsErik Johnston2019-04-022-5/+5
| | | | |
| | * | | Grandfather in existing user threepidsErik Johnston2019-04-012-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We assume, as we did before, that users bound their threepid to one of the trusted identity servers. So we simply fill the new table with all threepids in `user_threepids` joined with the trusted identity servers.
| | * | | Track IS used to bind 3PIDsErik Johnston2019-04-012-0/+104
| | | | | | | | | | | | | | | | | | | | | | | | | This will then be used to know which IS to default to when unbinding the threepid.
| * | | | Run black on the rest of the storage module (#4996)Amber Brown2019-04-0341-2455/+2130
| | | | |
| * | | | Make starting pushers faster during start upErik Johnston2019-04-021-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We start all pushers on start up and immediately start a background process to fetch push to send. This makes start up incredibly painful when dealing with many pushers. Instead, let's do a quick fast DB check to see if there *may* be push to send and only start the background processes for those pushers. We also stagger starting up and doing those checks so that we don't try and handle all pushers at once.
| * | | | Fix sync bug when accepting invites (#4956)Richard van der Hoff2019-04-022-11/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hopefully this time we really will fix #4422. We need to make sure that the cache on `get_rooms_for_user_with_stream_ordering` is invalidated *before* the SyncHandler is notified for the new events, and we can now do so reliably via the `events` stream.
| * | | | Collect room-version variations into one place (#4969)Richard van der Hoff2019-04-011-1/+2
| |/ / / | | | | | | | | | | | | Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.
| * | | Merge pull request #4955 from matrix-org/rav/merge_state_into_eventsRichard van der Hoff2019-03-281-3/+0
| |\ \ \ | | | | | | | | | | Combine the CurrentStateDeltaStream into the EventStream
| | * | | Combine the CurrentStateDeltaStream into the EventStreamRichard van der Hoff2019-03-271-3/+0
| | | | |
| * | | | Run `black` on some storage modules that the stats branch touches (#4959)Amber Brown2019-03-292-380/+335
| |/ / /
* | | / Add management endpoints for account validityBrendan Abolivier2019-04-171-12/+17
| |_|/ |/| |
* | | Send out emails with links to extend an account's validity periodBrendan Abolivier2019-04-172-18/+159
| |/ |/|
* | Add account expiration featureBrendan Abolivier2019-04-093-1/+55
|/
* Use the state event amount for userdir import batching, not room count (#4944)Amber Brown2019-03-271-8/+20
|
* Fix bug where read-receipts lost their timestamps (#4927)Richard van der Hoff2019-03-251-1/+3
| | | | | Make sure that they are sent correctly over the replication stream. Fixes: #4898
* Refactor out state delta handling into its own class (#4917)Amber Brown2019-03-252-64/+76
|
* Fix upsertErik Johnston2019-03-211-0/+1
|
* comment block_roomErik Johnston2019-03-201-0/+9
|
* Allow blocking a room multiple timesErik Johnston2019-03-201-2/+4
|
* Batching in the user directory import (#4900)Amber Brown2019-03-211-0/+13
|
* Fix user directory background update (#4887)Erik Johnston2019-03-191-1/+1
|
* Migrate the user directory initial population to a background task (#4864)Amber Brown2019-03-193-96/+312
|
* Merge pull request #4846 from matrix-org/hawkowl/userdir-searchErik Johnston2019-03-144-75/+92
|\ | | | | Improve searching in the userdir
| * fixupAmber Brown2019-03-131-1/+1
| |
| * fixupAmber Brown2019-03-132-55/+11
| |
| * fixupAmber Brown2019-03-121-1/+1
| |
| * fixupAmber Brown2019-03-121-1/+2
| |
| * fixupAmber Brown2019-03-122-113/+69
| |
| * fixupAmber Brown2019-03-121-1/+1
| |
| * use the old methodAmber Brown2019-03-121-3/+2
| |
| * fixAmber Brown2019-03-121-1/+1
| |
| * pep8Amber Brown2019-03-121-2/+1
| |
| * fixesAmber Brown2019-03-122-9/+56
| |
| * Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-searchAmber Brown2019-03-112-15/+2
| |\
| * | initialAmber Brown2019-03-113-3/+62
| | |
* | | Transfer local user's push rules on room upgrade (#4838)Andrew Morgan2019-03-121-0/+57
| |/ |/| | | Transfer push rules (notifications) on room upgrade
* | Add comment to schemaErik Johnston2019-03-081-0/+2
| |
* | Make `prev_state` field optionalErik Johnston2019-03-081-15/+0
|/ | | | | | | The `prev_state` field on events is not specced and so synapse shouldn't explode if an event is missing the field. Fixes #4787
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-03-083-181/+156
|\ | | | | | | erikj/soft_fail_impl
| * Rewrite userdir to be faster (#4537)Amber Brown2019-03-072-181/+137
| |
| * Add docstrings from matrix-org-hotfixesErik Johnston2019-03-061-0/+19
| |
* | Implement soft failErik Johnston2019-03-061-0/+1
|/
* Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_roomErik Johnston2019-03-051-0/+22
|\ | | | | Stop backpaginating when events not visible
| * s/get_forward_events/get_successor_events/Erik Johnston2019-03-041-2/+2
| |
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-03-044-43/+120
| |\ | | | | | | | | | erikj/stop_fed_not_in_room
| * | Stop backpaginating when events not visibleErik Johnston2019-02-201-0/+22
| | |
* | | Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keysErik Johnston2019-03-053-541/+546
|\ \ \ | | | | | | | | Allow /keys/{changes,query} API to run on worker
| * | | Fixup slave storesErik Johnston2019-03-043-541/+546
| | |/ | |/|
* / | Add some debug about processing read receipts.Richard van der Hoff2019-03-041-6/+20
|/ / | | | | | | | | I'm hoping to establish which rooms are having lots of RRs sent for them, and how old the events are when they are sent.
* | Move /account/3pid to client_readerErik Johnston2019-02-271-33/+33
| |
* | Limit cache invalidation replication line length (#4748)Erik Johnston2019-02-271-4/+11
| |
* | Add database version to phonehome stats. (#4753)Richard van der Hoff2019-02-273-0/+42
| |
* | Fix state cache invalidation on workersErik Johnston2019-02-221-6/+34
|/
* UPSERT many functionality (#4644)Amber Brown2019-02-201-12/+134
|
* Merge pull request #4671 from matrix-org/erikj/state_cache_invalidationErik Johnston2019-02-192-25/+58
|\ | | | | Batch cache invalidation over replication
| * DocsErik Johnston2019-02-191-2/+3
| |
| * Use itertoolsErik Johnston2019-02-191-2/+2
| |
| * DocumentationErik Johnston2019-02-191-4/+4
| |
| * Batch cache invalidation over replicationErik Johnston2019-02-182-25/+57
| | | | | | | | | | | | | | | | | | | | Currently whenever the current state changes in a room invalidate a lot of caches, which cause *a lot* of traffic over replication. Instead, lets batch up all those invalidations and send a single poke down the replication streams. Hopefully this will reduce load on the master process by substantially reducing traffic.
* | Split /login into client_readerErik Johnston2019-02-181-41/+41
|/
* Split out registration to workerErik Johnston2019-02-181-115/+115
| | | | | | | | This allows registration to be handled by a worker, though the actual write to the database still happens on master. Note: due to the in-memory session map all registration requests must be handled by the same worker.
* Add configurable room list publishing rulesErik Johnston2019-02-141-0/+25
| | | | | This allows specifying who and what is allowed to be published onto the public room list
* Run `black` on user directory code (#4635)Amber Brown2019-02-131-87/+66
|
* Merge pull request #4627 from matrix-org/erikj/user_ips_analyzeErik Johnston2019-02-122-3/+31
|\ | | | | Analyze user_ips before running deduplication
| * Fix pep8Erik Johnston2019-02-121-1/+1
| |
| * Analyze user_ips before running deduplicationErik Johnston2019-02-122-3/+31
| | | | | | | | | | | | | | Due to the table locks taken out by the naive upsert, the table statistics may be out of date. During deduplication it is important that the correct index is used as otherwise a full table scan may be incorrectly used, which can end up thrashing the database badly.
* | Reduce user_ips bloat during dedupe background updateErik Johnston2019-02-121-3/+60
|/ | | | | | | | | | | | The background update to remove duplicate rows naively deleted and reinserted the duplicates. For large tables with a large number of duplicates this causes a lot of bloat (with postgres), as the inserted rows are appended to the table, since deleted rows will not be overwritten until a VACUUM has happened. This should hopefully also help ensure that the query in the last batch uses the correct index, as inserting a large number of new rows without analyzing will upset the query planner.
* Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updatingErik Johnston2019-02-111-0/+21
|\ | | | | add updating of backup versions
| * add new endpoint to update backup versionsHubert Chathi2019-02-061-0/+21
| |
* | Fix 'no unique or exclusion constraint' error (#4591)Richard van der Hoff2019-02-081-6/+21
| | | | | | | | | | Add more tables to the list of tables which need a background update to complete before we can upsert into them, which fixes a race against the background updates.
* | Filter user directory state query to a subset of state events (#4462)Andrew Morgan2019-02-051-4/+12
|/ | | | | | * Filter user directory state query to a subset of state events * Add changelog
* Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatableAndrew Morgan2019-01-311-11/+27
|\ | | | | Copy over non-federatable trait on room upgrade
| * Raise an exception instead of returning NoneAndrew Morgan2019-01-311-2/+9
| |
| * New function for getting room's create eventAndrew Morgan2019-01-311-11/+20
| |
* | Fix infinite loop when an event is redacted in a v3 room (#4535)Richard van der Hoff2019-01-311-5/+32
|/
* Fix flake8 (#4519)Amber Brown2019-01-303-86/+86
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-01-291-0/+23
|\ | | | | | | erikj/redactions_eiah
| * Refactor event building into EventBuilderErik Johnston2019-01-291-0/+23
| | | | | | | | | | This is so that everything is done in one place, making it easier to change the event format based on room version
* | Update synapse/storage/events_worker.pyErik Johnston2019-01-291-1/+1
| |
* | Check redaction state when event is pulled out of the databaseErik Johnston2019-01-291-0/+13
| |
* | Fix typoErik Johnston2019-01-291-1/+1
| |
* | Remove unused argErik Johnston2019-01-291-2/+1
| |
* | Update synapse/storage/events_worker.pyRichard van der Hoff2019-01-291-1/+1
| | | | | | Co-Authored-By: erikjohnston <erikj@jki.re>
* | Implement rechecking of redactionsErik Johnston2019-01-291-1/+25
|/
* Fix typo in upserts code (#4505)Amber Brown2019-01-291-1/+1
| | | | | | * fix obvious problem :| * changelog
* Fix UPSERTs on SQLite 3.24+ (#4477)Amber Brown2019-01-283-12/+20
|
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Andrew Morgan2019-01-282-10/+9
|\ | | | | | | anoa/dm_room_upgrade
| * Merge pull request #4470 from matrix-org/erikj/require_format_versionErik Johnston2019-01-251-6/+2
| |\ | | | | | | Require event format version to parse or create events
| | * Replace missed usages of FrozenEventErik Johnston2019-01-251-6/+2
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-01-252-2/+25
| |\ \ | | | | | | | | | | | | erikj/sqlite_native_upsert
| | * | Don't require sqlite3 when using postgres (#4466)Andrew Morgan2019-01-251-2/+1
| | | |
| * | | Disable native upserts for sqlite, as they don't workErik Johnston2019-01-251-3/+7
| | |/ | |/|
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Andrew Morgan2019-01-2512-64/+334
|\ \ \ | | |/ | |/| | | | anoa/dm_room_upgrade
| * | Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_roomsAndrew Morgan2019-01-251-0/+24
| |\ \ | | |/ | |/| Ability to search entire room history after upgrading room
| | * Merge branch 'anoa/full_search_upgraded_rooms' of ↵Andrew Morgan2019-01-241-1/+1
| | |\ | | | | | | | | | | | | github.com:matrix-org/synapse into anoa/full_search_upgraded_rooms
| | | * Apply suggestions from code reviewRichard van der Hoff2019-01-241-1/+1
| | | | | | | | | | | | Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
| | * | Remove redundant create event None checkAndrew Morgan2019-01-241-3/+0
| | |/
| | * Do not return in a deferred functionAndrew Morgan2019-01-221-3/+3
| | |
| | * Ensure new filter is actually createdAndrew Morgan2019-01-221-1/+0
| | |
| | * Fix a bug with single-room search searching all roomsAndrew Morgan2019-01-221-1/+28
| | | | | | | | | | | | | | | * Create a new method for getting predecessor rooms * Remove formatting change
| | * Search for messages across predecessor roomsAndrew Morgan2019-01-181-0/+1
| | | | | | | | | | | | Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
| * | Revert "Require event format version to parse or create events"Erik Johnston2019-01-251-2/+6
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-01-248-33/+238
| |\ \ | | | | | | | | | | | | erikj/require_format_version
| | * \ Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invitesErik Johnston2019-01-241-4/+4
| | |\ \ | | | | | | | | | | Store rejected remote invite events as outliers
| | | * | Use term 'out of band membership' insteadErik Johnston2019-01-241-3/+3
| | | | |
| | * | | Merge pull request #4437 from matrix-org/erikj/event_format_version_v2Erik Johnston2019-01-243-4/+32
| | |\ \ \ | | | | | | | | | | | | Add support for persisting event format versions
| | * | | | Fix UPSERT check (#4459)Amber Brown2019-01-241-5/+28
| | | | | |
| | * | | | Use native UPSERTs where possible (#4306)Amber Brown2019-01-247-30/+212
| | | | | |
| * | | | | Replace missed usages of FrozenEventErik Johnston2019-01-241-6/+2
| | | | | |
| * | | | | Merge branch 'erikj/fixup_rejecting_invites' of ↵Erik Johnston2019-01-231-4/+4
| |\ \ \ \ \ | | | |_|/ / | | |/| | | | | | | | | github.com:matrix-org/synapse into erikj/require_format_version
| | * | | | Clarify the invite flowsErik Johnston2019-01-231-4/+7
| | | | | |
| | * | | | Store rejected remote invite events as outliersErik Johnston2019-01-231-4/+1
| | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently they're stored as non-outliers even though the server isn't in the room, which can be problematic in places where the code assumes it has the state for all non outlier events. In particular, there is an edge case where persisting the leave event triggers a state resolution, which requires looking up the room version from state. Since the server doesn't have the state, this causes an exception to be thrown.
| * | / / Merge branch 'erikj/get_pdu_versions' into erikj/require_format_versionErik Johnston2019-01-231-1/+5
| |\| | | | | |/ / | |/| |
| | * | Fixup removal of duplicate `user_ips` rows (#4432)Erik Johnston2019-01-231-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove unnecessary ORDER BY clause * Add logging * Newsfile
| * | | isortErik Johnston2019-01-231-1/+1
| | | |
| * | | Add support for persisting event format versionsErik Johnston2019-01-233-4/+32
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we only have the one event format version defined, but this adds the necessary infrastructure to persist and fetch the format versions alongside the events. We specify the format version rather than the room version as: 1. We don't necessarily know the room version, existing events may be either v1 or v2. 2. We'd need to be careful to prevent/handle correctly if different events in the same room reported to be of different versions, which sounds annoying.
| * | Refactor to rewrite the SQL insteadErik Johnston2019-01-221-13/+16
| | |
| * | Don't shadow paramsErik Johnston2019-01-221-4/+3
| | |
| * | Fix bug when removing duplicate rows from user_ipsErik Johnston2019-01-221-13/+20
| | | | | | | | | | | | | | | | | | | | | | | | This was caused by accidentally overwritting a `last_seen` variable in a for loop, causing the wrong value to be written to the progress table. The result of which was that we didn't scan sections of the table when searching for duplicates, and so some duplicates did not get deleted.
* | | Fix typosAndrew Morgan2019-01-221-1/+1
|/ /
* / Fix race when persisting create event (#4404)Erik Johnston2019-01-181-1/+12
|/ | | | | | | | | * Fix race when persisting create event When persisting a chunk of DAG it is sometimes requried to do a state resolution, which requires knowledge of the room version. If this happens while we're persisting the create event then we need to use that event rather than attempting to look it up in the database.
* Remove duplicates in the user_ips table and add an index (#4370)Amber Brown2019-01-122-4/+160
|
* Fix adding new rows instead of updating them if one of the key values is a ↵Amber Brown2019-01-091-1/+9
| | | | NULL in upserts. (#4369)
* Ensure synchrotrons can access is_support_user in the storage layerNeil Johnson2019-01-021-25/+25
|
* Log roomid along with Unknown room (#4297)Will Hunt2018-12-241-1/+1
|
* create support user (#4141)Neil Johnson2018-12-143-4/+83
| | | | | | Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
* Settings Fix deleting e2e room keys on xenial (#4295)Amber Brown2018-12-141-1/+1
|
* Initialise user displayname from SAML2 data (#4272)Richard van der Hoff2018-12-071-7/+13
| | | | | When we register a new user from SAML2 data, initialise their displayname correctly.
* drop undocumented dependency on dateutil (#4266)Richard van der Hoff2018-12-061-7/+6
| | | | It turns out we were relying on dateutil being pulled in transitively by pysaml2. There's no need for that bloat.
* Add an option to disable search for homeservers which may not be interested ↵Travis Ralston2018-12-041-0/+6
| | | | | | in it (#4230) This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
* add more detail to logging regarding "More than one row matched" error (#4234)Ben Parsons2018-12-041-6/+6
|
* Fix UnicodeDecodeError when postgres is not configured in english (#4253)Richard van der Hoff2018-12-041-7/+8
| | | | This is a bit of a half-assed effort at fixing https://github.com/matrix-org/synapse/issues/4252. Fundamentally the right answer is to drop support for Python 2.
* Drop sent_transactionsAaron Raimist2018-12-016-74/+3
| | | | Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Neilj/fix mau initial reserved users (#4211)Neil Johnson2018-11-281-2/+3
| | | | | | | | * fix transaction wrapping bug that caused get_user_id_by_threepid_txn to fail * towncrier * white space
* Ignore __pycache__ directories in schema delta dirRichard van der Hoff2018-11-201-1/+1
| | | | Now that we use py3, compiled python ends up in __pycache__ rather than *.pyc.
* Add option to track MAU stats (but not limit people) (#3830)Travis Ralston2018-11-151-34/+40
|
* Merge remote-tracking branch 'origin/develop' into ↵David Baker2018-11-098-65/+274
|\ | | | | | | dbkr/e2e_backup_versions_are_numbers
| * Lets convert bytes to unicode insteadErik Johnston2018-11-081-5/+3
| |
| * Fix noop checks when updating device keysErik Johnston2018-11-081-0/+5
| | | | | | | | | | | | | | | | | | | | Clients often reupload their device keys (for some reason) so its important for the server to check for no-ops before sending out device list update notifications. The check is broken in python 3 due to the fact comparing bytes and unicode always fails, and that we write bytes to the DB but get unicode when we read.
| * Add helpers for getting prev and auth events (#4139)Erik Johnston2018-11-062-6/+6
| | | | | | | | | | | | | | * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions.
| * Add unique indexes to a couple of tablesRichard van der Hoff2018-11-023-6/+88
| | | | | | | | | | The indexes on device_lists_remote_extremeties can be unique, and they therefore should, to ensure that the db remains consistent.
| * Remove redundant database locks for device list updatesRichard van der Hoff2018-11-021-5/+40
| | | | | | | | We can rely on the application-level per-user linearizer.
| * Merge pull request #4006 from matrix-org/erikj/purge_state_groupsErik Johnston2018-10-304-48/+137
| |\ | | | | | | Delete unreferenced state groups during purge
| | * Move _find_unreferenced_groupsErik Johnston2018-10-292-81/+83
| | |
| | * Fix set operations thinkoErik Johnston2018-10-291-3/+4
| | |
| | * Don't make temporary listErik Johnston2018-10-291-3/+2
| | |
| | * Make SQL a bit cleanerErik Johnston2018-10-291-3/+2
| | |
| | * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-10-2610-379/+635
| | |\ | | | | | | | | | | | | erikj/purge_state_groups
| | * | pep8Erik Johnston2018-10-191-1/+1
| | | |
| | * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-10-196-56/+461
| | |\ \ | | | | | | | | | | | | | | | erikj/purge_state_groups
| | * | | Batch process handling state groupsErik Johnston2018-10-192-101/+92
| | | | |
| | * | | Fix up commentsErik Johnston2018-10-121-3/+3
| | | | |
| | * | | Add state_group index to event_to_state_groupsErik Johnston2018-10-043-1/+27
| | | | | | | | | | | | | | | | | | | | | | | | | This is needed to efficiently check for unreferenced state groups during purge.
| | * | | Delete unreferened state groups during purgeErik Johnston2018-10-042-6/+77
| | | | |
* | | | | pep8David Baker2018-11-091-1/+2
| | | | |
* | | | | add docsDavid Baker2018-11-091-1/+4
| | | | |
* | | | | Cast to int here tooDavid Baker2018-10-301-0/+5
| | | | |
* | | | | Cast bacjup version to int when queryingDavid Baker2018-10-301-1/+6
| | | | |
* | | | | Convert version back to a stringDavid Baker2018-10-301-0/+1
| | | | |
* | | | | Try & make it work on postgresDavid Baker2018-10-301-2/+2
| | | | |
* | | | | Make e2e backup versions numeric in the DBDavid Baker2018-10-292-1/+54
|/ / / / | | | | | | | | | | | | | | | | We were doing max(version) which does not do what we wanted on a column of type TEXT.
* | | | Deduplicate device updates sent over replicationErik Johnston2018-10-291-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently send several kHz of device list updates over replication occisonally, which often causes the replications streams to lag and then get dropped. A lot of those updates will actually be duplicates, since we don't send e.g. device_ids across replication, so let's deduplicate it when we pull them out of the database.
* | | | preserve room visibilityRichard van der Hoff2018-10-251-1/+1
| |_|/ |/| |
* | | Refactor state group lookup to reduce DB hits (#4011)Erik Johnston2018-10-252-327/+520
| | | | | | | | | | | | | | | | | | | | | | | | Currently when fetching state groups from the data store we make two hits two the database: once for members and once for non-members (unless request is filtered to one or the other). This adds needless load to the datbase, so this PR refactors the lookup to make only a single database hit.
* | | Merge pull request #4081 from matrix-org/neilj/fix_mau_initNeil Johnson2018-10-252-23/+83
|\ \ \ | | | | | | | | fix race condiftion in calling initialise_reserved_users
| * | | add new lineNeil Johnson2018-10-251-0/+1
| | | |
| * | | improve commentsNeil Johnson2018-10-251-6/+6
| | | |
| * | | fix style inconsistenciesNeil Johnson2018-10-242-15/+47
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_initNeil Johnson2018-10-242-22/+25
| |\ \ \
| * | | | remove white spaceNeil Johnson2018-10-231-2/+1
| | | | |
| * | | | fix race condiftion in calling initialise_reserved_usersNeil Johnson2018-10-232-17/+45
| | | | |
* | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-10-2512-51/+423
|\ \ \ \ \ | | |/ / / | |/| | | | | | | | matthew/autocreate_autojoin
| * | | | Merge pull request #4082 from matrix-org/rav/fix_pep8Richard van der Hoff2018-10-241-1/+1
| |\ \ \ \ | | | | | | | | | | | | Fix a number of flake8 errors
| | * | | | Fix a number of flake8 errorsRichard van der Hoff2018-10-241-1/+1
| | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.
| * | | | Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebaseErik Johnston2018-10-241-21/+24
| |\ \ \ \ | | |/ / / | |/| | | Add v2 state resolution algorithm
| | * | | Update event_auth table for rejected eventsErik Johnston2018-10-161-15/+21
| | | | |
| | * | | Add v2 state res algorithm.Erik Johnston2018-10-161-6/+3
| | | | | | | | | | | | | | | | | | | | We hook this up to the vdh test room version.
| * | | | Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase ↵Amber Brown2018-10-205-6/+6
| | |_|/ | |/| | | | | | | | | | on py3) (#4068)
| * | | Fix incorrect truncation in get_missing_eventsRichard van der Hoff2018-10-161-22/+16
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id.
| * | Merge pull request #4019 from matrix-org/dbkr/e2e_backupsDavid Baker2018-10-153-0/+361
| |\ \ | | | | | | | | E2E backups
| | * | Misc PR feedback bitsDavid Baker2018-10-121-2/+2
| | | |
| | * | Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backupsDavid Baker2018-10-0916-116/+330
| | |\ \
| | * | | Don't reuse backup versionsDavid Baker2018-10-052-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since we don't actually delete the keys, just mark the versions as deleted in the db rather than actually deleting them, then we won't reuse versions. Fixes https://github.com/vector-im/riot-web/issues/7448
| | * | | WIP e2e key backupsDavid Baker2018-09-131-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Continues from uhoreg's branch This just fixed the errcode on /room_keys/version if no backup and updates the schema delta to be on the latest so it gets run
| | * | | try to make flake8 and isort happyHubert Chathi2018-09-061-2/+4
| | | | |
| | * | | Merge branch 'develop' into e2e_backupsHubert Chathi2018-08-2414-195/+674
| | |\ \ \
| | * | | | allow session_data to be any JSON instead of just a stringHubert Chathi2018-08-211-2/+4
| | | | | |
| | * | | | allow auth_data to be any JSON instead of a stringHubert Chathi2018-08-211-2/+5
| | | | | |
| | * | | | missing importMatthew Hodgson2018-08-121-0/+1
| | | | | |
| | * | | | 404 nicely if you try to interact with a missing current versionMatthew Hodgson2018-08-121-16/+35
| | | | | |
| | * | | | flake8Matthew Hodgson2018-08-121-1/+1
| | | | | |
| | * | | | implement remaining tests and make them workMatthew Hodgson2018-08-121-1/+2
| | | | | |
| | * | | | fix idiocies and so make tests passMatthew Hodgson2018-08-122-3/+4
| | | | | |
| | * | | | don't needlessly return user_idMatthew Hodgson2018-08-121-1/+0
| | | | | |
| | * | | | fix flakesMatthew Hodgson2018-08-121-2/+4
| | | | | |
| | * | | | fix typosMatthew Hodgson2018-08-121-1/+1
| | | | | |
| | * | | | add a tonne of docstring; make upload_room_keys properly assert versionMatthew Hodgson2018-08-121-20/+31
| | | | | |
| | * | | | add storage docstring; remove unused set_e2e_room_keysMatthew Hodgson2018-08-121-36/+83
| | | | | |
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-25 18:15:51 +0000