summary refs log tree commit diff
path: root/synapse/storage (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | | Servlet to purge old rooms (#5845)Richard van der Hoff2019-08-221-0/+137
| | | | | |
* | | | | | Drop some unused tables. (#5893)Andrew Morgan2020-02-213-47/+22
|\| | | | |
| * | | | | Drop some unused tables. (#5893)Richard van der Hoff2019-08-213-47/+22
| | | | | | | | | | | | | | | | | | These tables are never used, so we may as well drop them.
* | | | | | Merge pull request #5857 from matrix-org/uhoreg/fix_e2e_room_keys_indexAndrew Morgan2020-02-202-1/+19
|\| | | | |
| * | | | | Merge pull request #5857 from matrix-org/uhoreg/fix_e2e_room_keys_indexHubert Chathi2019-08-162-1/+19
| |\ \ \ \ \ | | |/ / / / | |/| | | | add the version field to the index for e2e_room_keys
| | * | | | update set_e2e_room_key to agree with fixed indexHubert Chathi2019-08-141-1/+1
| | | | | |
| | * | | | add the version field to the index for e2e_room_keysHubert Chathi2019-08-141-0/+18
| | | | | |
* | | | | | Merge pull request #5809 from matrix-org/erikj/handle_pusher_stopAndrew Morgan2020-02-201-8/+22
|\| | | | |
| * | | | | Merge pull request #5809 from matrix-org/erikj/handle_pusher_stopErik Johnston2019-08-131-8/+22
| |\ \ \ \ \ | | | | | | | | | | | | | | Handle pusher being deleted during processing.
| | * | | | | Clarify docstringErik Johnston2019-08-131-1/+1
| | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | Handle pusher being deleted during processing.Erik Johnston2019-08-011-8/+22
| | | |_|/ / | | |/| | | | | | | | | | | | | | | | | | | | | Instead of throwing a StoreError lets break out of processing loop and mark the pusher as stopped.
* | | | | | Merge pull request #5826 from matrix-org/erikj/reduce_event_pausesAndrew Morgan2020-02-201-128/+142
|\| | | | |
| * | | | | Merge pull request #5826 from matrix-org/erikj/reduce_event_pausesErik Johnston2019-08-131-128/+142
| |\ \ \ \ \ | | | | | | | | | | | | | | Don't unnecessarily block notifying of new events.
| | * | | | | Don't unnecessarily block notifying of new events.Erik Johnston2019-08-061-128/+142
| | | |/ / / | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When persisting events we calculate new stream orderings up front. Before we notify about an event all events with lower stream orderings must have finished being persisted. This PR moves the assignment of stream ordering till *after* calculated the new current state and split the batch of events into separate chunks for persistence. This means that if it takes a long time to calculate new current state then it will not block events in other rooms being notified about. This should help reduce some global pauses in the events stream which can last for tens of seconds (if not longer), caused by some particularly expensive state resolutions.
* | | | | | Cross-signing [1/4] -- hidden devices (#5759)Andrew Morgan2020-02-203-10/+48
|\ \ \ \ \ \ | | |_|_|_|/ | |/| | | |
| * | | | | Cross-signing [1/4] -- hidden devices (#5759)Hubert Chathi2019-08-123-10/+48
| | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | * allow devices to be marked as "hidden" This is a prerequisite for cross-signing, as it allows us to create other things that live within the device namespace, so they can be used for signatures.
* | | | | Merge pull request #5788 from matrix-org/rav/metaredactionsAndrew Morgan2020-02-201-101/+112
|\ \ \ \ \ | | |/ / / | |/| | |
| * | | | Merge pull request #5788 from matrix-org/rav/metaredactionsRichard van der Hoff2019-08-121-101/+112
| |\ \ \ \ | | |/ / / | |/| | | Fix handling of redactions of redactions
| | * | | Handle loops in redaction eventsRichard van der Hoff2019-07-301-60/+36
| | | | |
| | * | | recursively fetch redactionsRichard van der Hoff2019-07-301-32/+36
| | | | |
| | * | | split _get_events_from_db out of _enqueue_eventsRichard van der Hoff2019-07-291-32/+51
| | | | |
| | * | | Return dicts from _fetch_event_listRichard van der Hoff2019-07-291-15/+27
| | | | |
* | | | | Have ClientReaderSlavedStore inherit RegistrationStore (#5806)Andrew Morgan2020-02-201-21/+21
|\| | | |
| * | | | Have ClientReaderSlavedStore inherit RegistrationStore (#5806)Andrew Morgan2019-08-011-21/+21
| | |/ / | |/| | | | | | Fixes #5803
* | | | Merge pull request #5793 from matrix-org/erikj/fix_bg_updateAndrew Morgan2020-02-202-1/+22
|\| | |
| * | | Don't recreate columnErik Johnston2019-07-302-1/+22
| | | |
* | | | Merge pull request #5792 from matrix-org/erikj/fix_bg_updateAndrew Morgan2020-02-202-1/+1
|\| | |
| * | | Fix current_state_events membership background update.Erik Johnston2019-07-302-1/+1
| |/ / | | | | | | | | | | | | | | | Turns out not all rooms are in `rooms`, so lets fetch the room list from `current_state_events`. We move the delta file to force it to be run again.
* | | Merge pull request #5774 from matrix-org/erikj/fix_rejected_membershipAndrew Morgan2020-02-201-0/+6
|\| |
| * | Merge pull request #5774 from matrix-org/erikj/fix_rejected_membershipErik Johnston2019-07-291-0/+6
| |\ \ | | | | | | | | Fix room summary when rejected events are in state
| | * | Fix room summary when rejected events are in stateErik Johnston2019-07-261-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Annoyingly, `current_state_events` table can include rejected events, in which case the membership column will be null. To work around this lets just always filter out null membership for now.
* | | | Merge pull request #5752 from matrix-org/erikj/forgotten_userAndrew Morgan2020-02-202-35/+123
|\| | |
| * | | Merge pull request #5752 from matrix-org/erikj/forgotten_userErik Johnston2019-07-292-35/+123
| |\ \ \ | | | | | | | | | | Remove some more joins on room_memberships
| | * | | Expand commentErik Johnston2019-07-291-0/+5
| | | | |
| | * | | Remove lost commentErik Johnston2019-07-291-7/+0
| | |/ /
| | * | Remove join when calculating room summaries.Erik Johnston2019-07-241-13/+21
| | | |
| | * | Remove join on room_memberships when fetching rooms for user.Erik Johnston2019-07-241-22/+36
| | | |
| | * | Add function to get all forgotten rooms for userErik Johnston2019-07-242-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | This will allow us to efficiently filter out rooms that have been forgotten in other queries without having to join against the `room_memberships` table.
* | | | Merge pull request #5770 from matrix-org/erikj/fix_current_state_event_sqliteAndrew Morgan2020-02-201-2/+2
|\| | |
| * | | Fix current_state bg update to work on old SQLiteErik Johnston2019-07-291-2/+2
| | | |
| * | | Merge branch 'master' into developRichard van der Hoff2019-07-261-0/+4
| |\ \ \
* | \ \ \ Merge branch 'erikj/log_leave_origin_mismatch' into release-v1.2.1Andrew Morgan2020-02-191-0/+4
|\ \ \ \ \ | | |/ / / | |/| | |
| * | | | Fix DoS when there is a cycle in redaction eventsRichard van der Hoff2019-07-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Make sure that synapse doesn't explode when a redaction redacts itself, or there is a larger cycle.
* | | | | Merge pull request #5767 from matrix-org/rav/redactions/cross_room_idAndrew Morgan2020-02-191-0/+27
|\ \ \ \ \ | | |/ / / | |/| | |
| * | | | Merge pull request #5767 from matrix-org/rav/redactions/cross_room_idRichard van der Hoff2019-07-251-0/+27
| |\| | | | | | | | | | | | | log when a redaction attempts to redact an event in a different room
| | * | | log when a redaction attempts to redact an event in a different roomRichard van der Hoff2019-07-251-0/+27
| | | | |
* | | | | Stop trying to fetch events with event_id=None. (#5753)Andrew Morgan2020-02-192-10/+15
|\| | | |
| * | | | Stop trying to fetch events with event_id=None. (#5753)Richard van der Hoff2019-07-242-10/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `None` is not a valid event id, so queuing up a database fetch for it seems like a silly thing to do. I considered making `get_event` return `None` if `event_id is None`, but then its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
* | | | | Replace returnValue with return (#5736)Andrew Morgan2020-02-1937-362/+321
|\| | | |
| * | | | Replace returnValue with return (#5736)Amber Brown2019-07-2337-362/+321
| | |/ / | |/| |
* | | | Merge pull request #5738 from matrix-org/erikj/faster_updateAndrew Morgan2020-02-191-18/+30
|\| | |
| * | | Speed up current state background update.Erik Johnston2019-07-231-18/+30
| | | | | | | | | | | | | | | | | | | | Turns out that storing huge JSON arrays in the progress JSON isn't something that postgres particularly likes.
* | | | Merge pull request #5731 from matrix-org/jaywink/admin-user-list-user-typeAndrew Morgan2020-02-191-3/+3
|\| | |
| * | | Add `user_type` to returned fields in admin API user list endpointsJason Robinson2019-07-221-3/+3
| | |/ | |/| | | | | | | | | | | | | | | | Mostly user type will be empty (normal user) but there is also the "support" user type. Signed-off-by: Jason Robinson <jasonr@matrix.org>
* | | Merge pull request #5720 from matrix-org/erikj/transactions_upsertAndrew Morgan2020-02-191-0/+20
|\| |
| * | Merge pull request #5720 from matrix-org/erikj/transactions_upsertErik Johnston2019-07-191-0/+20
| |\ \ | | | | | | | | Use upsert when updating destination retry interval
| | * | Use upsert when updating destination retry intervalErik Johnston2019-07-191-0/+20
| | |/
* | | Merge pull request #5713 from matrix-org/erikj/use_cache_for_filtered_stateAndrew Morgan2020-02-191-2/+6
|\| |
| * | Merge pull request #5713 from matrix-org/erikj/use_cache_for_filtered_stateErik Johnston2019-07-191-2/+6
| |\ \ | | | | | | | | Delegate to cached version when using get_filtered_current_state_ids
| | * | Delegate to cached version when using get_filtered_current_state_idsErik Johnston2019-07-181-2/+6
| | |/ | | | | | | | | | In the case where it gets called with `StateFilter.all()`
* | | Merge pull request #5706 from matrix-org/erikj/add_memberships_to_current_stateAndrew Morgan2020-02-194-21/+99
|\| |
| * | Merge pull request #5706 from matrix-org/erikj/add_memberships_to_current_stateErik Johnston2019-07-197-41/+194
| |\ \ | | |/ | |/| Add membership column to current_state_events table
| | * LoggingTransaction accepts None for callback lists.Erik Johnston2019-07-193-6/+16
| | | | | | | | | | | | | | | Its a bit disingenuousto give LoggingTransaction lists to append callbacks to if we're not going to run the callbacks.
| | * Update comment for new columnErik Johnston2019-07-191-0/+3
| | |
| | * Use the current_state_events.membership columnErik Johnston2019-07-181-17/+37
| | |
| | * Track if current_state_events.membership is up to dateErik Johnston2019-07-181-0/+45
| | |
* | | Add background update for current_state_events.membership columnAndrew Morgan2020-02-192-0/+54
|\ \ \ | | |/ | |/|
| * | Add background update for current_state_events.membership columnErik Johnston2019-07-182-0/+54
| | |
* | | Add membership column to current_state_events table.Andrew Morgan2020-02-195-20/+41
|\| |
| * | Add membership column to current_state_events table.Erik Johnston2019-07-185-20/+41
| | | | | | | | | | | | | | | It turns out that doing a join is surprisingly expensive for the DB to do when room_membership table is larger than the disk cache.
* | | Ignore redactions of m.room.create events (#5701)Andrew Morgan2020-02-191-0/+12
|\ \ \ | | |/ | |/|
| * | Ignore redactions of m.room.create events (#5701)Richard van der Hoff2019-07-171-0/+12
| | |
* | | More refactoring in `get_events_as_list` (#5707)Andrew Morgan2020-02-191-36/+36
|\| |
| * | More refactoring in `get_events_as_list` (#5707)Richard van der Hoff2019-07-171-27/+37
| | | | | | | | | | | | | | | | | | | | | | | | We can now use `_get_events_from_cache_or_db` rather than going right back to the database, which means that (a) we can benefit from caching, and (b) it opens the way forward to more extensive checks on the original event. We now always require the original event to exist before we will serve up a redaction.
* | | Fix redaction authentication (#5700)Andrew Morgan2020-02-191-94/+138
|\| |
| * | Fix redaction authentication (#5700)Richard van der Hoff2019-07-171-85/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensures that redactions are correctly authenticated for recent room versions. There are a few things going on here: * `_fetch_event_rows` is updated to return a dict rather than a list of rows. * Rather than returning multiple copies of an event which was redacted multiple times, it returns the redactions as a list within the dict. * It also returns the actual rejection reason, rather than merely the fact that it was rejected, so that we don't have to query the table again in `_get_event_from_row`. * The redaction handling is factored out of `_get_event_from_row`, and now checks if any of the redactions are valid.
* | | Refactor `get_events_as_list` (#5699)Andrew Morgan2020-02-171-44/+67
|\| |
| * | Refactor `get_events_as_list` (#5699)Richard van der Hoff2019-07-171-48/+74
| |/ | | | | | | | | | | | | | | | | | | | | A couple of changes here: * get rid of a redundant `allow_rejected` condition - we should already have filtered out any rejected events before we get to that point in the code, and the redundancy is confusing. Instead, let's stick in an assertion just to make double-sure we aren't leaking rejected events by mistake. * factor out a `_get_events_from_cache_or_db` method, which is going to be important for a forthcoming fix to redactions.
* | Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_dataAndrew Morgan2020-02-172-6/+28
|\|
| * Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_dataErik Johnston2019-07-152-6/+28
| |\ | | | | | | Add basic function to get all data for a user out of synapse
| | * Fixup docstringsErik Johnston2019-07-121-7/+5
| | |
| | * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-07-056-39/+25
| | |\ | | | | | | | | | | | | erikj/admin_exfiltrate_data
| | * | Assume key existence. Update docstringsErik Johnston2019-07-051-6/+10
| | | |
| | * | Add basic function to get all data for a user out of synapseErik Johnston2019-07-021-0/+20
| | | |
* | | | Implement access token expiry (#5660)Andrew Morgan2020-02-172-9/+36
|\| | |
| * | | Implement access token expiry (#5660)Richard van der Hoff2019-07-122-5/+32
| | | | | | | | | | | | | | | | Record how long an access token is valid for, and raise a soft-logout once it expires.
| * | | fix typo: backgroud -> backgroundAndrew Morgan2019-07-121-4/+4
| | | |
* | | | small typo fix (#5655)Andrew Morgan2020-02-171-1/+1
|\| | |
| * | | small typo fix (#5655)Andrew Morgan2019-07-111-1/+1
| | | |
* | | | Remove access-token support from RegistrationStore.register (#5642)Andrew Morgan2020-02-171-20/+4
|\| | |
| * | | Remove access-token support from RegistrationStore.register (#5642)Richard van der Hoff2019-07-101-20/+4
| | | | | | | | | | | | | | | | | | | | The 'token' param is no longer used anywhere except the tests, so let's kill that off too.
* | | | Include the original event in /relations (#5626)Andrew Morgan2020-02-171-1/+1
|\| | |
| * | | Include the original event in /relations (#5626)Andrew Morgan2019-07-091-1/+1
| | | | | | | | | | | | When asking for the relations of an event, include the original event in the response. This will mostly be used for efficiently showing edit history, but could be useful in other circumstances.
* | | | Remove support for invite_3pid_guest. (#5625)Andrew Morgan2020-02-171-47/+0
|\| | |
| * | | Remove support for invite_3pid_guest. (#5625)Richard van der Hoff2019-07-051-47/+0
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | This has never been documented, and I'm not sure it's ever been used outside sytest. It's quite a lot of poorly-maintained code, so I'd like to get rid of it. For now I haven't removed the database table; I suggest we leave that for a future clearout.
* | | remove dead transaction persist code (#5622)Andrew Morgan2020-02-171-28/+0
|\| |
| * | remove dead transaction persist code (#5622)Richard van der Hoff2019-07-051-28/+0
| | | | | | | | | | | | this hasn't done anything for years
* | | Fix with isort==4.3.21 and black==19.3b0Andrew Morgan2020-02-141-2/+0
| | |
* | | Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Andrew Morgan2020-02-144-7/+7
|\| |
| * | Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-044-7/+7
| | |
* | | Fix a number of "Starting txn from sentinel context" warnings (#5605)Andrew Morgan2020-02-142-4/+18
|\| |
| * | Fix a number of "Starting txn from sentinel context" warnings (#5605)Richard van der Hoff2019-07-032-4/+18
| |/ | | | | | | Fixes #5602, #5603
* | Merge pull request #5559 from matrix-org/erikj/refactor_changed_devicesAndrew Morgan2020-02-141-12/+38
|\|
| * Review commentsErik Johnston2019-06-271-3/+1
| |
| * Use batch_iter and correct docstringErik Johnston2019-06-261-15/+12
| |
| * Rename get_users_whose_devices_changedErik Johnston2019-06-261-3/+3
| |
| * Refactor get_user_ids_changed to pull less from DBErik Johnston2019-06-261-10/+41
| | | | | | | | | | | | | | | | | | | | | | When a client asks for users whose devices have changed since a token we used to pull *all* users from the database since the token, which could easily be thousands of rows for old tokens. This PR changes this to only check for changes for users the client is actually interested in. Fixes #5553
* | Merge pull request #5499 from matrix-org/rav/cleanup_metricsAndrew Morgan2020-02-141-22/+13
|\|
| * Merge pull request #5499 from matrix-org/rav/cleanup_metricsRichard van der Hoff2019-06-241-22/+13
| |\ | | | | | | Cleanups and sanity-checking in cpu and db metrics
| | * blackRichard van der Hoff2019-06-241-6/+2
| | |
| | * Merge branch 'develop' into rav/cleanup_metricsRichard van der Hoff2019-06-2433-254/+306
| | |\
| | * | Simplify PerformanceCounters.update interfaceRichard van der Hoff2019-06-191-11/+12
| | | | | | | | | | | | | | | | | | | | we already have the duration for the update, so may as well use it rather than passing extra params around and recalculating it.
| | * | Remove unused _get_event_countersRichard van der Hoff2019-06-191-7/+1
| | | | | | | | | | | | | | | | This has been redundant since cdb3757942fefdcdc3d33b9c6d7c9e44decefd6f.
* | | | Merge pull request #5498 from matrix-org/rav/fix_clock_reversalAndrew Morgan2020-02-131-6/+14
|\| | |
| * | | Merge pull request #5498 from matrix-org/rav/fix_clock_reversalRichard van der Hoff2019-06-241-6/+14
| |\ \ \ | | |_|/ | |/| | Use monotonic clock where possible for metrics
| | * | Use monotonic clock where possible for metricsRichard van der Hoff2019-06-191-6/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes intermittent errors observed on Apple hardware which were caused by time.clock() appearing to go backwards when called from different threads. Also fixes a bug where database activity times were logged as 1/1000 of their correct ratio due to confusion between milliseconds and seconds.
* | | | Fix "Unexpected entry in 'full_schemas'" log warning (#5509)Andrew Morgan2020-02-131-1/+1
|\| | |
| * | | Fix "Unexpected entry in 'full_schemas'" log warning (#5509)Richard van der Hoff2019-06-211-1/+1
| | | | | | | | | | | | | | | | | | | | There is a README.txt which always sets off this warning, which is a bit alarming when you first start synapse. I don't think we need to warn about this.
* | | | BlackAndrew Morgan2020-02-131-3/+3
|\| | |
| * | | BlackErik Johnston2019-06-201-2/+2
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-06-2034-276/+286
| |\ \ \ | | | | | | | | | | | | | | | erikj/histogram_extremities
| | * | | Run Black. (#5482)Amber Brown2019-06-2033-253/+252
| | | | |
| | * | | Merge pull request #5480 from matrix-org/erikj/extremities_dummy_eventsErik Johnston2019-06-191-0/+29
| | |\ \ \ | | | | | | | | | | | | Add experimental option to reduce extremities.
| | * | | | Fix typo in deactivation background jobBrendan Abolivier2019-06-191-1/+1
| | | |_|/ | | |/| |
| * | | | Add descriptions and remove redundant set(..)Erik Johnston2019-06-191-3/+5
| | | | |
* | | | | Dinsic Blacking with black==18.6b2Andrew Morgan2020-02-1134-321/+307
| | | | |
* | | | | Add experimental option to reduce extremities.Andrew Morgan2020-02-101-0/+29
|\ \ \ \ \ | | |_|/ / | |/| | |
| * | | | Add experimental option to reduce extremities.Erik Johnston2019-06-181-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds new config option `cleanup_extremities_with_dummy_events` which periodically sends dummy events to rooms with more than 10 extremities. THIS IS REALLY EXPERIMENTAL.
* | | | | Use consistent bucketsAndrew Morgan2020-02-101-1/+23
|\ \ \ \ \ | | |/ / / | |/| | |
| * | | | Add metrics for len of new extremities persisted.Erik Johnston2019-06-171-1/+23
| |/ / / | | | | | | | | | | | | | | | | Of new events being persisted add metrics for total size of forward extremities and number of unchanged, "stale" extremities.
* | | | Remove Postgres 9.4 support (#5448)Andrew Morgan2020-02-102-24/+6
|\ \ \ \ | | |/ / | |/| |
| * | | Remove Postgres 9.4 support (#5448)Amber Brown2019-06-182-24/+6
| |/ /
* | | Merge pull request #5461 from matrix-org/erikj/histograms_are_cumalitiveAndrew Morgan2020-02-101-1/+2
|\| |
| * | Merge pull request #5461 from matrix-org/erikj/histograms_are_cumalitiveErik Johnston2019-06-141-1/+2
| |\ \ | | | | | | | | Prometheus histograms are cumalative
| | * | Prometheus histograms are cumalativeErik Johnston2019-06-141-1/+2
| | | |
| * | | Fix background job for deactivated flagBrendan Abolivier2019-06-141-2/+4
| | | |
| * | | Don't send renewal emails to deactivated usersBrendan Abolivier2019-06-142-2/+16
| |/ /
| * | Track deactivated accounts in the database (#5378)Brendan Abolivier2019-06-142-0/+133
| | |
* | | Merge remote-tracking branch 'dinsic/dinsic' into dinsic-release-v1.1.0Andrew Morgan2020-02-103-61/+89
|\ \ \
| * | | Hide expired users from user directory, optionally show on renewal (#13)Andrew Morgan2019-11-141-0/+23
| | | |
| * | | Add a /user/:user_id/info servlet to give user deactivated/expired ↵Andrew Morgan2019-11-141-6/+3
| | | | | | | | | | | | | | | | information (#12)
| * | | Consider every room as having no retention policy if the feature is disabledBrendan Abolivier2019-09-241-0/+8
| | | |
| * | | Move get_retention_policy_for_room to RoomWorkerStoreBrendan Abolivier2019-09-241-55/+55
| | | |
* | | | Expose statistics on extrems to prometheus (#5384)Andrew Morgan2020-02-101-13/+31
|\ \ \ \ | | |/ / | |/| |
| * | | Expose statistics on extrems to prometheus (#5384)Amber Brown2019-06-131-13/+31
| | | |
* | | | Remove Python 2.7 support. (#5425)Andrew Morgan2020-02-101-15/+29
|\| | | | |/ / |/| |
| * | Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into developErik Johnston2019-06-112-1/+5
| |\|
| * | add monthly active users to phonehome stats (#5252)Neil Johnson2019-06-101-15/+29
| | | | | | | | | | | | * add monthly active users to phonehome stats
* | | Merge branch 'dinsic' into babolivier/dinsic-message-retentionBrendan Abolivier2019-08-281-17/+32
|\ \ \
| * | | Ignore redactions of redactions in get_events_as_listBrendan Abolivier2019-07-311-4/+1
| | | |
| * | | Ignore invalid redactions in _get_event_from_rowBrendan Abolivier2019-07-311-0/+5
| | | |
| * | | Should now work, unless we can't find the redaction event which happens for ↵Brendan Abolivier2019-07-301-18/+18
| | | | | | | | | | | | | | | | some reason (need to investigate)
| * | | Don't make the checks depend on recheck_redactionBrendan Abolivier2019-07-301-20/+23
| | | |
| * | | Do checks soonerBrendan Abolivier2019-07-291-21/+19
| | | |
| * | | Check room ID and type of redacted eventBrendan Abolivier2019-07-291-10/+22
| | | |
* | | | Implement per-room message retention policiesBrendan Abolivier2019-08-283-0/+288
|/ / /
* | | Fix typo in deactivation background jobBrendan Abolivier2019-06-191-1/+1
| | |
* | | Fix background job for deactivated flagBrendan Abolivier2019-06-141-2/+4
| | |
* | | Don't send renewal emails to deactivated usersBrendan Abolivier2019-06-142-2/+16
| | |
* | | Track deactivated accounts in the database (#5378)Brendan Abolivier2019-06-142-0/+133
| | |
* | | Make index uniqueBrendan Abolivier2019-06-141-0/+1
| | |
* | | LintBrendan Abolivier2019-06-131-1/+1
| | |
* | | Fix snafuBrendan Abolivier2019-06-131-2/+3
| | |
* | | Create the index in a background updateBrendan Abolivier2019-06-132-3/+15
| | |
* | | Merge branch 'dinsic' into babolivier/fix_profile_replicationBrendan Abolivier2019-06-1225-327/+4814
|\ \ \
| * | | Regenerate full schemasBrendan Abolivier2019-06-122-2/+17
| | | |
| * | | Merge branch 'master' into dinsicBrendan Abolivier2019-06-1220-74/+3625
| |\ \ \ | | | |/ | | |/|
| | * | Fix defaults on checking threepidsErik Johnston2019-06-101-1/+1
| | | |
| | * | Fix bug sending federation transactions with lots of EDUsErik Johnston2019-06-101-0/+4
| | |/ | | | | | | | | | | | | | | | If we try and send a transaction with lots of EDUs and we run out of space, we call get_new_device_msgs_for_remote with a limit of 0, which then failed.
| | * Fix key verification when key stored with null valid_until_msErik Johnston2019-06-101-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some keys are stored in the synapse database with a null valid_until_ms which caused an exception to be thrown when using that key. We fix this by treating nulls as zeroes, i.e. they keys will match verification requests with a minimum_valid_until_ms of zero (i.e. don't validate ts) but will not match requests with a non-zero minimum_valid_until_ms. Fixes #5391.
| | * Prevent multiple device list updates from breaking a batch send (#5156)Andrew Morgan2019-06-061-29/+123
| | | | | | | | | fixes #5153
| | * Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1Erik Johnston2019-06-065-2/+2352
| | |\ | | | | | | | | Make a full SQL schema
| | | * fix maybeAmber H. Brown2019-06-071-20/+2
| | | |
| | | * just user dir?Amber H. Brown2019-06-071-1/+19
| | | |
| | | * remove background updates that arent neededAmber H. Brown2019-06-071-33/+1
| | | |
| | | * add more commentsAmber Brown2019-06-061-0/+5
| | | |
| | | * add stuff in bg updatesAmber Brown2019-06-041-10/+62
| | | |
| | | * WHY IS THIS CALLED A SLIGHTLY DIFFERENT THINGAmber Brown2019-06-031-0/+1
| | | |
| | | * more fixAmber Brown2019-06-031-0/+38
| | | |
| | | * fix schemasAmber Brown2019-06-034-61/+20
| | | |
| | | * full schemaAmber Brown2019-06-033-0/+2315
| | | |
| | | * prepareAmber Brown2019-06-031-2/+14
| | | |
| | * | Add ability to perform password reset via email without trusting the ↵Andrew Morgan2019-06-064-5/+324
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | identity server (#5377) Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option. This PR is a culmination of 3 smaller PRs which have each been separately reviewed: * #5308 * #5345 * #5368
| | * | Merge pull request #5221 from matrix-org/erikj/fix_worker_sytestErik Johnston2019-06-061-1/+11
| | |\ \ | | | | | | | | | | Fix get_max_topological_token to never return None
| | | * | Better wordsErik Johnston2019-06-061-1/+1
| | | | |
| | | * | Fix get_max_topological_token to never return NoneErik Johnston2019-05-211-1/+11
| | | | |
| | * | | Fix background updates to handle redactions/rejections (#5352)Erik Johnston2019-06-061-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix background updates to handle redactions/rejections In background updates based on current state delta stream we need to handle that we may not have all the events (or at least that `get_events` may raise an exception).
| | * | | FixErik Johnston2019-06-031-1/+2
| | | | |
| | * | | Ignore room state with null bytes in for room statsErik Johnston2019-06-031-0/+15
| | | |/ | | |/|
| | * | Merge pull request #5276 from matrix-org/babolivier/account_validity_job_deltaErik Johnston2019-05-311-2/+20
| | |\ \ | | | | | | | | | | Allow configuring a range for the account validity startup job
| | * \ \ Merge pull request #5294 from matrix-org/erikj/speed_up_room_statsErik Johnston2019-05-314-87/+111
| | |\ \ \ | | | | | | | | | | | | Speed up room stats background update
| | | * | | Add indices. Remove room_ids accidentally addedErik Johnston2019-05-312-13/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have to do this by re-inserting a background update and recreating tables, as the tables only get created during a background update and will later be deleted. We also make sure that we remove any entries that should have been removed but weren't due to a race that has been fixed in a previous commit.
| | | * | | Join against events to use its room_id indexErik Johnston2019-05-301-1/+6
| | | | | |
| | | * | | Move deletion from table inside txnErik Johnston2019-05-301-2/+5
| | | | | |
| | | * | | Fetch membership counts all at onceErik Johnston2019-05-302-38/+18
| | | | | |
| | | * | | Get events all at onceErik Johnston2019-05-301-33/+26
| | | | | |
| | * | | | Add index to temp tableErik Johnston2019-05-301-0/+1
| | |/ / /
| | * | | Merge pull request #5278 from matrix-org/erikj/cleanup_bad_extremitiesErik Johnston2019-05-305-181/+436
| | |\ \ \ | | | | | | | | | | | | Add DB bg update to cleanup extremities.
| | | * | | Update synapse/storage/events_bg_updates.pyErik Johnston2019-05-301-1/+1
| | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | | * | | Rename constantErik Johnston2019-05-301-3/+3
| | | | | |
| | | * | | Move event background updates to a separate fileErik Johnston2019-05-303-369/+405
| | | | | |
| | | * | | Fixup comments and loggingErik Johnston2019-05-302-9/+15
| | | | | |
| | | * | | Log actual number of entries deletedErik Johnston2019-05-292-5/+13
| | | | | |
| | | * | | Add DB bg update to cleanup extremities.Erik Johnston2019-05-292-0/+205
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to #5269 we may have extremities in our DB that we shouldn't have, so lets add a cleanup task such to remove those.
| | * | | | Implement the SHHS complexity API (#5216)Amber Brown2019-05-301-1/+49
| | |/ / /
| | * | | Correctly filter out extremities with soft failed prevs (#5274)Erik Johnston2019-05-291-3/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we receive a soft failed event we, correctly, *do not* update the forward extremity table with the event. However, if we later receive an event that references the soft failed event we then need to remove the soft failed events prev events from the forward extremities table, otherwise we just build up forward extremities. Fixes #5269
| | * | | Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schemaBrendan Abolivier2019-05-281-0/+3
| | |\ \ \ | | | | | | | | | | | | Fix schema update for account validity
| | * | | | Fix logging for room stats background updateTravis Ralston2019-05-251-1/+1
| | | | | |
| | * | | | Store key validity time in the storage layerRichard van der Hoff2019-05-232-10/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a first step to checking that the key is valid at the required moment. The idea here is that, rather than passing VerifyKey objects in and out of the storage layer, we instead pass FetchKeyResult objects, which simply wrap the VerifyKey and add a valid_until_ts field.
| | * | | | Rewrite store_server_verify_key to store several keys at once (#5234)Richard van der Hoff2019-05-231-26/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Storing server keys hammered the database a bit. This replaces the implementation which stored a single key, with one which can do many updates at once.
| | * | | | Simplifications and comments in do_auth (#5227)Richard van der Hoff2019-05-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I was staring at this function trying to figure out wtf it was actually doing. This is (hopefully) a non-functional refactor which makes it a bit clearer.
| | * | | | Room Statistics (#4338)Amber Brown2019-05-216-3/+597
| | | | | |
| * | | | | Merge branch 'master' into dinsicBrendan Abolivier2019-06-118-251/+1174
| |\ \ \ \ \
| | * | | | | Add index to temp tableErik Johnston2019-05-301-0/+1
| | | | | | |
| | * | | | | Update synapse/storage/events_bg_updates.pyErik Johnston2019-05-301-1/+1
| | | | | | | | | | | | | | | | | | | | | Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
| | * | | | | Rename constantErik Johnston2019-05-301-3/+3
| | | | | | |
| | * | | | | Move event background updates to a separate fileErik Johnston2019-05-303-369/+405
| | | | | | |
| | * | | | | Fixup comments and loggingErik Johnston2019-05-302-9/+15
| | | | | | |
| | * | | | | Log actual number of entries deletedErik Johnston2019-05-302-5/+13
| | | | | | |
| | * | | | | Add DB bg update to cleanup extremities.Erik Johnston2019-05-302-0/+205
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to #5269 we may have extremities in our DB that we shouldn't have, so lets add a cleanup task such to remove those.
| | * | | | | Correctly filter out extremities with soft failed prevs (#5274)Erik Johnston2019-05-301-3/+79
| | |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we receive a soft failed event we, correctly, *do not* update the forward extremity table with the event. However, if we later receive an event that references the soft failed event we then need to remove the soft failed events prev events from the forward extremities table, otherwise we just build up forward extremities. Fixes #5269
| | * | | | Merge pull request #5203 from matrix-org/erikj/aggregate_by_senderErik Johnston2019-05-211-1/+1
| | |\ \ \ \ | | | | | | | | | | | | | | Only count aggregations from distinct senders
| | | * | | | Only count aggregations from distinct sendersErik Johnston2019-05-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | As a user isn't allowed to send a single emoji more than once.
| | * | | | | Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactionsErik Johnston2019-05-211-3/+45
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | Block attempts to annotate the same event twice
| | | * | | | | Block attempts to annotate the same event twiceErik Johnston2019-05-201-3/+45
| | | | |_|_|/ | | | |/| | |
| | * | | | | Exclude soft-failed events from fwd-extremity candidates. (#5146)Richard van der Hoff2019-05-211-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When considering the candidates to be forward-extremities, we must exclude soft failures. Hopefully fixes #5090.
| | * | | | | Merge pull request #5204 from ↵Brendan Abolivier2019-05-212-12/+62
| | |\ \ \ \ \ | | | |/ / / / | | |/| | | | | | | | | | | | | | | | | | matrix-org/babolivier/account_validity_expiration_date Add startup background job for account validity
| | * | | | | Rename relation types to match MSCErik Johnston2019-05-201-4/+4
| | | | | | |
| | * | | | | Merge pull request #5209 from matrix-org/erikj/reactions_baseErik Johnston2019-05-204-4/+476
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | Land basic reaction and edit support.
| | | * | | | | Correctly update aggregation counts after redactionErik Johnston2019-05-202-0/+20
| | | | |/ / / | | | |/| | |
| | | * | | | Fixup bsaed on review commentsErik Johnston2019-05-171-15/+17
| | | | | | |
| | | * | | | Add basic editing supportErik Johnston2019-05-161-2/+58
| | | | | | |
| | | * | | | Move parsing of tokens out of storage layerErik Johnston2019-05-161-14/+2
| | | | | | |
| | | * | | | Indirect tuple conversionErik Johnston2019-05-161-0/+6
| | | | | | |
| | | * | | | Fix relations in worker modeErik Johnston2019-05-162-5/+11
| | | | | | |
| | | * | | | Add cache to relationsErik Johnston2019-05-161-0/+8
| | | | | | |
| | | * | | | Add aggregations APIErik Johnston2019-05-161-6/+219
| | | | | | |
| | | * | | | Actually check for None rather falseyErik Johnston2019-05-151-2/+2
| | | | | | |
| | | * | | | Add simple pagination APIErik Johnston2019-05-151-0/+80
| | | | | | |
| | | * | | | Add simple send_relation API and track in DBErik Johnston2019-05-154-0/+93
| | | | | | |
| | * | | | | Spelling and clarificationsErik Johnston2019-05-171-5/+9
| | | | | | |
| | * | | | | Update docstringErik Johnston2019-05-161-1/+12
| | |/ / / /
| | * | | | Make generating SQL bounds for pagination genericErik Johnston2019-05-151-61/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will allow us to reuse the same structure when we paginate e.g. relations
* | | | | | Move schema update to 54 and add a warning commentBrendan Abolivier2019-06-102-1/+3
| | | | | |
* | | | | | New lineBrendan Abolivier2019-06-101-1/+1
| | | | | |
* | | | | | Add unique index to the profile_replication_status table.Brendan Abolivier2019-06-102-1/+17
|/ / / / /
* | | | | Merge branch 'babolivier/account_validity_job_delta' of ↵ dinsic_2019-05-31Erik Johnston2019-05-311-2/+20
|\ \ \ \ \ | | |_|_|/ | |/| | | | | | | | github.com:matrix-org/synapse into dinsic
| * | | | Move delta from +10% to -10%Brendan Abolivier2019-05-311-2/+2
| | | | |
| * | | | Make max_delta equal to period * 10%Brendan Abolivier2019-05-311-4/+3
| | | | |
| * | | | Allow configuring a range for the account validity startup jobBrendan Abolivier2019-05-281-2/+21
| | |_|/ | |/| | | | | | | | | | | | | | When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal. In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch.
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-08-09 10:18:34 +0000