summary refs log tree commit diff
path: root/synapse (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move presence handling into handle_new_client_eventErik Johnston2018-02-071-6/+6
| | | | As we want to have it run on the main synapse instance
* Add event_creator workerErik Johnston2018-02-073-1/+190
|
* Add replication http endpoint for event sendingErik Johnston2018-02-078-13/+304
|
* Remove pointless ratelimit checkErik Johnston2018-02-071-5/+0
| | | | | | The intention was for the check to be called as early as possible in the request, but actually was called just before the main ratelimit check, so was fairly pointless.
* Merge pull request #2847 from matrix-org/erikj/separate_event_creationErik Johnston2018-02-068-188/+211
|\ | | | | Split event creation into a separate handler
| * Update copyrightErik Johnston2018-02-066-1/+6
| |
| * s/_create_new_client_event/create_new_client_event/Erik Johnston2018-02-062-8/+8
| |
| * Update places where we create eventsErik Johnston2018-02-057-38/+42
| |
| * Split event creation into a separate handlerErik Johnston2018-02-051-146/+160
| |
* | Store state groups separately from events (#2784)Erik Johnston2018-02-0610-120/+237
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* | Merge pull request #2849 from matrix-org/rav/clean_up_state_deltaRichard van der Hoff2018-02-051-10/+23
|\ \ | | | | | | Remove redundant return value from _calculate_state_delta
| * | Remove redundant return value from _calculate_state_deltaRichard van der Hoff2018-02-051-10/+23
| | | | | | | | | | | | | | | we already have the state from _get_new_state_after_events, so returning it from _calculate_state_delta is just confusing.
* | | Factor out common code for search insertRichard van der Hoff2018-02-051-33/+62
| | | | | | | | | | | | | | | we can reuse the same code as is used for event insert, for doing the background index population.
* | | Move store_event_search_txn to SearchStoreRichard van der Hoff2018-02-052-35/+41
|/ / | | | | | | | | ... as a precursor to making event storing and doing the bg update share some code.
* | Merge pull request #2844 from matrix-org/rav/evicted_metricsRichard van der Hoff2018-02-054-5/+44
|\ \ | |/ |/| montoring metrics for number of cache evictions
| * report metrics on number of cache evictionsRichard van der Hoff2018-02-054-5/+44
| |
* | Merge pull request #2791 from matrix-org/erikj/media_storage_refactorErik Johnston2018-02-053-12/+47
|\ \ | | | | | | Ensure media is in local cache before thumbnailing
| * | Use better file consumerErik Johnston2018-01-181-5/+10
| | |
| * | Ensure media is in local cache before thumbnailingErik Johnston2018-01-183-10/+40
| | |
* | | Merge pull request #2845 from matrix-org/rav/urlcache_error_handlingRichard van der Hoff2018-02-021-21/+34
|\ \ \ | | | | | | | | Handle url_previews with no content-type
| * | | Handle url_previews with no content-typeRichard van der Hoff2018-02-021-21/+34
| | |/ | |/| | | | | | | | | | | | | | | | avoid failing with an exception if the remote server doesn't give us a Content-Type header. Also, clean up the exception handling a bit.
* | | Merge pull request #2842 from matrix-org/rav/state_resolution_handlerRichard van der Hoff2018-02-024-65/+116
|\ \ \ | | | | | | | | Factor out resolve_state_groups to a separate handler
| * | | Factor out resolve_state_groups to a separate handlerRichard van der Hoff2018-02-013-53/+105
| | | | | | | | | | | | | | | | | | | | | | | | We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.
| * | | Rename resolve_state_groups -> resolve_state_groups_for_eventsRichard van der Hoff2018-02-012-12/+11
| | | | | | | | | | | | | | | | (to make way for a method that actually just does the state group resolution)
* | | | Merge pull request #2841 from matrix-org/rav/refactor_calc_state_deltaRichard van der Hoff2018-02-021-18/+39
|\ \ \ \ | | | | | | | | | | factor _get_new_state_after_events out of _calculate_state_delta
| * | | | factor _get_new_state_after_events out of _calculate_state_deltaRichard van der Hoff2018-01-311-18/+39
| |/ / / | | | | | | | | | | | | This reduces the scope of a bunch of variables
* | | | Merge pull request #2836 from matrix-org/rav/resolve_state_events_docstringRichard van der Hoff2018-02-021-8/+28
|\ \ \ \ | | | | | | | | | | Docstring fixes
| * | | | doc arg types for _seperateRichard van der Hoff2018-02-011-0/+15
| | | | |
| * | | | More docstring fixesRichard van der Hoff2018-02-011-4/+4
| | | | | | | | | | | | | | | | | | | | Fix a couple of errors in docstrings
| * | | | Fix docstring for StateHandler.resolve_state_groupsRichard van der Hoff2018-02-011-4/+9
| | | | | | | | | | | | | | | | | | | | The return type was a complete lie, so fix it
* | | | | Merge branch 'develop' into travis/admin-list-mediaTravis Ralston2018-02-0140-195/+427
|\ \ \ \ \ | | |_|/ / | |/| | |
| * | | | Merge pull request #2837 from matrix-org/rav/fix_quarantine_mediaRichard van der Hoff2018-02-011-1/+1
| |\ \ \ \ | | |_|/ / | |/| | | Fix sql error in quarantine_media
| | * | | Fix sql error in quarantine_mediaRichard van der Hoff2018-01-301-1/+1
| | |/ /
| * | | Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_failRichard van der Hoff2018-01-311-2/+1
| |\ \ \ | | | | | | | | | | Remove spurious log argument
| | * | | Remove spurious log argumentRichard van der Hoff2018-01-301-2/+1
| | |/ / | | | | | | | | | | | | ... which would cause scary-looking and unhelpful errors in the log on dns fail
| * | | Merge pull request #2835 from matrix-org/rav/remove_event_type_paramRichard van der Hoff2018-01-311-6/+14
| |\ \ \ | | | | | | | | | | Remove unused "event_type" param on state.get_current_state_ids
| | * | | docstring for get_current_state_idsRichard van der Hoff2018-01-291-0/+13
| | | | |
| | * | | Remove unused "event_type" param on state.get_current_state_idsRichard van der Hoff2018-01-291-6/+1
| | |/ / | | | | | | | | | | | | | | | | | | | | this param doesn't seem to be used, and is a bit pointless anyway because it can easily be replicated by the caller. It is also horrible, because it changes the return type of the method.
| * | | Merge pull request #2834 from ↵Richard van der Hoff2018-01-311-3/+3
| |\ \ \ | | | | | | | | | | | | | | | | | | | | matrix-org/rav/better_persist_event_exception_handling Improve exception handling in persist_event
| | * | | Improve exception handling in persist_eventRichard van der Hoff2018-01-291-3/+3
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. use `deferred.errback()` instead of `deferred.errback(e)`, which means that a Failure object will be constructed using the current exception state, *including* its stack trace - so the stack trace is saved in the Failure, leading to better exception reports. 2. Set `consumeErrors=True` on the ObservableDeferred, because we know that there will always be at least one observer - which avoids a spurious "CRITICAL: unhandled exception in Deferred" error in the logs
| * | | Merge pull request #2833 from matrix-org/rav/pusher_hacksRichard van der Hoff2018-01-311-7/+30
| |\ \ \ | | | | | | | | | | Logging and metrics for the http pusher
| | * | | Better logging when pushes failRichard van der Hoff2018-01-291-2/+8
| | | | |
| | * | | add appid/device_display_name to to pusher loggingRichard van der Hoff2018-01-291-2/+2
| | | | |
| | * | | Fix logging and add user_idRichard van der Hoff2018-01-221-2/+3
| | | | |
| | * | | logging and debug for http pusherRichard van der Hoff2018-01-221-4/+20
| | | | |
| * | | | Merge pull request #2817 from matrix-org/rav/http_conn_poolRichard van der Hoff2018-01-311-1/+13
| |\ \ \ \ | | |_|/ / | |/| | | Use a connection pool for the SimpleHttpClient
| | * | | Increase http conn pool sizeRichard van der Hoff2018-01-291-1/+6
| | |/ /
| | * | Use a connection pool for the SimpleHttpClientRichard van der Hoff2018-01-201-1/+8
| | | | | | | | | | | | | | | | | | | | In particular I hope this will help the pusher, which makes many requests to sygnal, and is currently negotiating SSL for each one.
| * | | Fix SQL for user searchRichard van der Hoff2018-01-271-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | fix some syntax errors for user search when search_all_users is enabled fixes #2801, hopefully
| * | | Merge pull request #2830 from matrix-org/rav/factor_out_get_connRichard van der Hoff2018-01-2711-130/+17
| |\ \ \ | | | | | | | | | | Factor out get_db_conn to HomeServer base class
| | * | | Factor out get_db_conn to HomeServer base classRichard van der Hoff2018-01-2611-130/+17
| | | | | | | | | | | | | | | | | | | | | | | | | This function is identical to all subclasses, so we may as well push it up to the base class to reduce duplication (and make use of it in the tests)
| * | | | Merge pull request #2827 from ↵Neil Johnson2018-01-261-1/+2
| |\ \ \ \ | | |/ / / | |/| | | | | | | | | | | | | matrix-org/fix_server_500_on_public_rooms_call_when_no_rooms_exist Fix server 500 on public rooms call when no rooms exist
| | * | | add white space lineNeil Johnson2018-01-261-0/+1
| | | | |
| | * | | rather than try reconstruct the results object, better to guard against the ↵Neil Johnson2018-01-261-6/+2
| | | | | | | | | | | | | | | | | | | | xrange step argument being 0
| | * | | fix return type, should be a dictNeil Johnson2018-01-251-1/+1
| | | | |
| | * | | fix PEP8 violationNeil Johnson2018-01-251-1/+1
| | | | |
| | * | | remove white spaceNeil Johnson2018-01-251-1/+0
| | | | |
| | * | | synapse 500s on a call to publicRooms in the case where the number of public ↵Neil Johnson2018-01-251-0/+5
| | | | | | | | | | | | | | | | | | | | rooms is zero, the specific cause is due to xrange trying to use a step value of zero, but if the total room number really is zero then it makes sense to just bail and save the extra processing
| * | | | fix thinko on 3pid whitelistingMatthew Hodgson2018-01-241-2/+2
| | | | |
| * | | | Merge pull request #2816 from matrix-org/rav/metrics_commentsRichard van der Hoff2018-01-231-1/+6
| |\ \ \ \ | | | | | | | | | | | | Add some comments about the reactor tick time metric
| | * | | | Add some comments about the reactor tick time metricRichard van der Hoff2018-01-191-1/+6
| | | | | |
| * | | | | add ?ts massaging for ASes (#2754)Matthew Hodgson2018-01-231-6/+11
| | |/ / / | |/| | | | | | | | blindly implement ?ts for AS. untested
| * | | | Add federation_domain_whitelist option (#2820)Matthew Hodgson2018-01-2213-7/+145
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
| * | | | Merge pull request #2813 from matrix-org/matthew/registrations_require_3pidMatthew Hodgson2018-01-227-23/+177
| |\ \ \ \ | | |/ / / | |/| | | add registrations_require_3pid and allow_local_3pids
| | * | | fix typo (thanks sytest)Matthew Hodgson2018-01-191-1/+1
| | | | |
| | * | | oops, check all login typesMatthew Hodgson2018-01-191-14/+11
| | | | |
| | * | | fix PR nitpickingMatthew Hodgson2018-01-191-3/+6
| | | | |
| | * | | trailing commasMatthew Hodgson2018-01-192-7/+7
| | | | |
| | * | | rewrite based on PR feedback:Matthew Hodgson2018-01-197-89/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * [ ] split config options into allowed_local_3pids and registrations_require_3pid * [ ] simplify and comment logic for picking registration flows * [ ] fix docstring and move check_3pid_allowed into a new util module * [ ] use check_3pid_allowed everywhere @erikjohnston PTAL
| | * | | fix up v1, and improve errorsMatthew Hodgson2018-01-194-20/+65
| | | | |
| | * | | fix pep8Matthew Hodgson2018-01-191-2/+1
| | | | |
| | * | | add registrations_require_3pidMatthew Hodgson2018-01-195-13/+110
| | | | | | | | | | | | | | | | | | | | | | | | | lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with. Typically useful for stopping people from registering with non-work emails
* | | | | pep8Travis Ralston2018-01-311-9/+12
| | | | | | | | | | | | | | | Signed-off-by: Travis Ralston <travpc@gmail.com>
* | | | | Documentation and namingTravis Ralston2018-01-311-7/+26
| | | | | | | | | | | | | | | Signed-off-by: Travis Ralston <travpc@gmail.com>
* | | | | Appease the linterTravis Ralston2018-01-201-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | These are ids anyways, not mxc uris. Signed-off-by: Travis Ralston <travpc@gmail.com>
* | | | | Add an admin route to get all the media in a roomTravis Ralston2018-01-202-56/+97
|/ / / / | | | | | | | | | | | | | | | | This is intended to be used by administrators to monitor the media that is passing through their server, if they wish. Signed-off-by: Travis Ralston <travpc@gmail.com>
* | | | Merge pull request #2814 from matrix-org/rav/fix_urlcache_thumbsRichard van der Hoff2018-01-191-0/+8
|\ \ \ \ | | | | | | | | | | Use the right path for url_preview thumbnails
| * | | | Use the right path for url_preview thumbnailsRichard van der Hoff2018-01-191-0/+8
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | This was introduced by #2627: we were overwriting the original media for url previews with the thumbnails :/ (fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)
* | | | Fix storage provider bug introduced when renamed to store_localErik Johnston2018-01-181-1/+1
| | | |
* | | | Merge pull request #2812 from matrix-org/erikj/media_storage_provider_configErik Johnston2018-01-183-33/+101
|\ \ \ \ | | | | | | | | | | Make storage providers configurable
| * | | | Fix passing wrong config to provider constructorErik Johnston2018-01-181-1/+1
| | | | |
| * | | | Remove duplicate directory testErik Johnston2018-01-181-4/+0
| | | | |
| * | | | Missing staticmethodErik Johnston2018-01-181-0/+1
| | | | |
| * | | | Fixup commentsErik Johnston2018-01-182-3/+7
| | | | |
| * | | | Make storage providers more configurableErik Johnston2018-01-183-31/+98
| | | | |
* | | | | Merge pull request #2804 from matrix-org/erikj/file_consumerErik Johnston2018-01-181-0/+139
|\ \ \ \ \ | | |_|_|/ | |/| | | Add decent impl of a FileConsumer
| * | | | Do logcontexts correctlyErik Johnston2018-01-181-2/+2
| | | | |
| * | | | Move test stuff to testsErik Johnston2018-01-181-25/+1
| | | | |
| * | | | Make all fields privateErik Johnston2018-01-181-31/+31
| | | | |
| * | | | Ensure we registerProducer isn't called twiceErik Johnston2018-01-181-0/+3
| | | | |
| * | | | Fix _notify_empty typoErik Johnston2018-01-181-1/+1
| | | | |
| * | | | Move definition of paused_producer to __init__Erik Johnston2018-01-181-2/+4
| | | | |
| * | | | Fix commentsErik Johnston2018-01-181-3/+3
| | | | |
| * | | | Add decent impl of a FileConsumerErik Johnston2018-01-171-0/+158
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind
* | / / Fix typo in thumbnail resource causing access times to be incorrectErik Johnston2018-01-181-2/+2
| |/ / |/| |
* | | Fix bugs in block metricsRichard van der Hoff2018-01-181-2/+4
| | | | | | | | | | | | ... which I introduced in #2785
* | | Merge pull request #2809 from matrix-org/rav/metrics_errorsRichard van der Hoff2018-01-181-1/+8
|\ \ \ | |_|/ |/| | better exception logging in callbackmetrics
| * | better exception logging in callbackmetricsRichard van der Hoff2018-01-181-1/+8
| | | | | | | | | | | | when we fail to render a metric, give a clue as to which metric it was
* | | Merge pull request #2805 from matrix-org/rav/log_state_resRichard van der Hoff2018-01-171-0/+6
|\ \ \ | |/ / |/| | Log room when doing state resolution
| * | Log room when doing state resolutionRichard van der Hoff2018-01-171-0/+6
| | | | | | | | | | | | Mostly because it helps figure out what is prompting the resolution
* | | Merge pull request #2783 from matrix-org/erikj/media_last_accessedErik Johnston2018-01-175-11/+64
|\ \ \ | | | | | | | | Keep track of last access time for local media
| * | | Remove lost commentErik Johnston2018-01-171-3/+0
| | | |
| * | | Update last access time when thumbnails are viewedErik Johnston2018-01-171-0/+2
| | | |
| * | | Keep track of last access time for local mediaErik Johnston2018-01-174-11/+65
| | |/ | |/|
* | | Merge pull request #2803 from matrix-org/matthew/fix-userdir-sqlMatthew Hodgson2018-01-171-2/+7
|\ \ \ | | | | | | | | fix SQL when searching all users
| * | | fix SQL when searching all usersMatthew Hodgson2018-01-171-2/+7
| | | |
* | | | Split resolve_events into two functionsRichard van der Hoff2018-01-172-20/+29
|/ / / | | | | | | | | | ... so that the return type doesn't depend on the arg types
* | | Merge pull request #2795 from matrix-org/rav/track_db_schedulingRichard van der Hoff2018-01-175-4/+43
|\ \ \ | | | | | | | | Track DB scheduling delay per-request
| * \ \ Merge remote-tracking branch 'origin/develop' into rav/track_db_schedulingRichard van der Hoff2018-01-175-34/+78
| |\ \ \
| * | | | Track DB scheduling delay per-requestRichard van der Hoff2018-01-165-4/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing.
| * | | | Merge branch 'rav/db_txn_time_in_millis' into rav/track_db_schedulingRichard van der Hoff2018-01-164-16/+34
| |\ \ \ \
* | \ \ \ \ Merge pull request #2797 from matrix-org/rav/user_id_checkingRichard van der Hoff2018-01-172-8/+19
|\ \ \ \ \ \ | |_|_|/ / / |/| | | | | Sanity checking for user ids
| * | | | | Sanity checking for user idsRichard van der Hoff2018-01-172-8/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Check the user_id passed to a couple of APIs for validity, to avoid "IndexError: list index out of range" exception which looks scary and results in a 500 rather than a more useful error. Fixes #1432, among other things
* | | | | | Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millisRichard van der Hoff2018-01-174-10/+17
|\ \ \ \ \ \ | | | | | | | | | | | | | | Track db txn time in millisecs
| * | | | | | Merge branch 'develop' into rav/db_txn_time_in_millisRichard van der Hoff2018-01-174-25/+69
| |\| | | | | | | |_|/ / / | |/| | | |
| * | | | | Track db txn time in millisecsRichard van der Hoff2018-01-164-10/+17
| | | | | | | | | | | | | | | | | | | | | | | | ... to reduce the amount of floating-point foo we do.
* | | | | | Merge pull request #2796 from matrix-org/rav/fix_closed_connection_errorsRichard van der Hoff2018-01-171-9/+9
|\ \ \ \ \ \ | | | | | | | | | | | | | | Fix 'NoneType' object has no attribute 'writeHeaders'
| * \ \ \ \ \ Merge branch 'develop' into rav/fix_closed_connection_errorsRichard van der Hoff2018-01-175-32/+87
| |\ \ \ \ \ \ | | | |/ / / / | | |/| | | |
| * | | | | | Fix 'NoneType' object has no attribute 'writeHeaders'Richard van der Hoff2018-01-161-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid throwing a (harmless) exception when we try to write an error response to an http request where the client has disconnected. This comes up as a CRITICAL error in the logs which tends to mislead people into thinking there's an actual problem
* | | | | | | Merge branch 'develop' into rav/rework_run_interactionRichard van der Hoff2018-01-175-32/+87
|\ \ \ \ \ \ \ | | |/ / / / / | |/| | / / / | |_|_|/ / / |/| | | | |
| * | | | | Merge remote-tracking branch 'origin/develop' into rav/optimise_logging_contextRichard van der Hoff2018-01-174-25/+69
| |\ \ \ \ \ | | |_|/ / / | |/| | / / | | | |/ / | | |/| |
| | * | | Merge pull request #2789 from matrix-org/erikj/fix_thumbnailsErik Johnston2018-01-173-24/+68
| | |\ \ \ | | | | | | | | | | | | Fix thumbnailing remote files
| | | * | | Add docstringErik Johnston2018-01-171-1/+1
| | | | | |
| | | * | | Use local varsErik Johnston2018-01-161-6/+2
| | | | | |
| | | * | | Change _generate_thumbnails to take media_typeErik Johnston2018-01-162-7/+10
| | | | | |
| | | * | | Move setting of file_id up to callerErik Johnston2018-01-162-13/+14
| | | | | |
| | | * | | Fix typoErik Johnston2018-01-161-7/+10
| | | | | |
| | | * | | Fix up log linesErik Johnston2018-01-162-7/+10
| | | | | |
| | | * | | Correctly use server_name/file_id when generating/fetching remote thumbnailsErik Johnston2018-01-162-5/+8
| | | | | |
| | | * | | Log when we respond with 404Erik Johnston2018-01-162-1/+8
| | | | | |
| | | * | | Fix thumbnailing remote filesErik Johnston2018-01-162-2/+30
| | | | | |
| | * | | | fix typoMatthew Hodgson2018-01-161-1/+1
| | | |/ / | | |/| |
| * / | | Optimise LoggingContext creation and copyingRichard van der Hoff2018-01-161-7/+18
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.
* / / / rework runInteraction in terms of runConnectionRichard van der Hoff2018-01-161-20/+31
|/ / / | | | | | | | | | ... so that we can share the code
* | | Merge pull request #2785 from matrix-org/rav/reorganise_metrics_againRichard van der Hoff2018-01-163-39/+151
|\ \ \ | | | | | | | | Reorganise request and block metrics
| * | | Reorganise request and block metricsRichard van der Hoff2018-01-152-25/+84
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to circumvent the number of duplicate foo:count metrics increasing without bounds, it's time for a rearrangement. The following are all deprecated, and replaced with synapse_util_metrics_block_count: synapse_util_metrics_block_timer:count synapse_util_metrics_block_ru_utime:count synapse_util_metrics_block_ru_stime:count synapse_util_metrics_block_db_txn_count:count synapse_util_metrics_block_db_txn_duration:count The following are all deprecated, and replaced with synapse_http_server_response_count: synapse_http_server_requests synapse_http_server_response_time:count synapse_http_server_response_ru_utime:count synapse_http_server_response_ru_stime:count synapse_http_server_response_db_txn_count:count synapse_http_server_response_db_txn_duration:count The following are renamed (the old metrics are kept for now, but deprecated): synapse_util_metrics_block_timer:total -> synapse_util_metrics_block_time_seconds synapse_util_metrics_block_ru_utime:total -> synapse_util_metrics_block_ru_utime_seconds synapse_util_metrics_block_ru_stime:total -> synapse_util_metrics_block_ru_stime_seconds synapse_util_metrics_block_db_txn_count:total -> synapse_util_metrics_block_db_txn_count synapse_util_metrics_block_db_txn_duration:total -> synapse_util_metrics_block_db_txn_duration_seconds synapse_http_server_response_time:total -> synapse_http_server_response_time_seconds synapse_http_server_response_ru_utime:total -> synapse_http_server_response_ru_utime_seconds synapse_http_server_response_ru_stime:total -> synapse_http_server_response_ru_stime_seconds synapse_http_server_response_db_txn_count:total -> synapse_http_server_response_db_txn_count synapse_http_server_response_db_txn_duration:total synapse_http_server_response_db_txn_duration_seconds
| * | | mechanism to render metrics with alternative namesRichard van der Hoff2018-01-151-13/+40
| | | |
| * | | Add some comments to metrics classesRichard van der Hoff2018-01-151-1/+27
| | | |
* | | | Merge pull request #2790 from matrix-org/rav/preserve_event_logcontext_leakRichard van der Hoff2018-01-161-1/+8
|\ \ \ \ | | | | | | | | | | Fix a logcontext leak in persist_events
| * | | | Fix a logcontext leak in persist_eventsRichard van der Hoff2018-01-161-1/+8
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ObserveableDeferred expects its callbacks to be called without any logcontexts, whereas it turns out we were calling them with the logcontext of the request which initiated the persistence loop. It seems wrong that we are attributing work done in the persistence loop to the request that happened to initiate it, so let's solve this by dropping the logcontext for it. (I'm not sure this actually causes any real problems other than messages in the debug log, but let's clean it up anyway)
* | | | Merge pull request #2787 from matrix-org/rav/worker_event_countsRichard van der Hoff2018-01-162-0/+11
|\ \ \ \ | | | | | | | | | | Metrics for events processed in appservice and fed sender
| * | | | Metrics for events processed in appservice and fed senderRichard van der Hoff2018-01-152-0/+11
| |/ / / | | | | | | | | | | | | More metrics I wished I'd had
* | | | Merge pull request #2786 from matrix-org/rav/rdata_metricsRichard van der Hoff2018-01-161-5/+14
|\ \ \ \ | |_|/ / |/| | | Metrics for number of RDATA commands received
| * | | Metrics for number of RDATA commands receivedRichard van der Hoff2018-01-151-5/+14
| |/ / | | | | | | | | | I found myself wishing we had this.
* | | Merge pull request #2767 from matrix-org/erikj/media_storage_refactorErik Johnston2018-01-167-346/+843
|\ \ \ | |/ / |/| | Refactor MediaRepository to separate out storage
| * | Fix up commentsErik Johnston2018-01-122-7/+16
| | |
| * | Correctly reraise exceptionErik Johnston2018-01-121-2/+4
| | |
| * | Make Responder a context managerErik Johnston2018-01-122-9/+10
| | |
| * | Add missing class varErik Johnston2018-01-121-0/+3
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-01-127-28/+68
| |\ \ | | | | | | | | | | | | erikj/media_storage_refactor
| * | | Remove unnecessary conditionErik Johnston2018-01-121-4/+1
| | | |
| * | | Remove unused variablesErik Johnston2018-01-121-3/+0
| | | |
| * | | Make class var localErik Johnston2018-01-121-3/+3
| | | |
| * | | CommentsErik Johnston2018-01-123-15/+65
| | | |
| * | | Add StorageProvider conceptErik Johnston2018-01-093-9/+162
| | | |
| * | | Make PreviewUrlResource use MediaStorageErik Johnston2018-01-092-8/+14
| | | |
| * | | Make ThumbnailResource use MediaStorageErik Johnston2018-01-092-48/+68
| | | |
| * | | Use MediaStorage for remote mediaErik Johnston2018-01-092-136/+156
| | | |
| * | | Use MediaStorage for local filesErik Johnston2018-01-092-119/+73
| | | |
| * | | Add MediaStorage classErik Johnston2018-01-091-0/+198
| | | |
| * | | Add some helper classesErik Johnston2018-01-091-0/+73
| | | |
| * | | Split out add_file_headersErik Johnston2018-01-091-28/+42
| | | |
* | | | Make Counter render floatsRichard van der Hoff2018-01-121-3/+10
| |_|/ |/| | | | | | | | | | | | | | | | | | | | Prometheus handles all metrics as floats, and sometimes we store non-integer values in them (notably, durations in seconds), so let's render them as floats too. (Note that the standard client libraries also treat Counters as floats.)
* | | Reinstate media download on thumbnail requestRichard van der Hoff2018-01-121-0/+5
| | | | | | | | | | | | | | | We need to actually download the remote media when we get a request for a thumbnail.
* | | Merge pull request #2774 from matrix-org/erikj/synctlErik Johnston2018-01-122-5/+31
|\ \ \ | |_|/ |/| | When using synctl with workers, don't start the main synapse automatically
| * | Actually make it workErik Johnston2018-01-122-2/+15
| | |
| * | RefactorErik Johnston2018-01-121-9/+4
| | |
| * | When using synctl with workers, don't start the main synapse automaticallyErik Johnston2018-01-111-8/+26
| | |
* | | Merge pull request #2766 from matrix-org/rav/room_eventRichard van der Hoff2018-01-111-3/+26
|\ \ \ | |/ / |/| | Add /room/{id}/event/{id} to synapse
| * | Add /room/{id}/event/{id} to synapseRichard van der Hoff2018-01-091-3/+26
| | | | | | | | | | | | | | | | | | Turns out that there is a valid usecase for retrieving event by id (notably having received a push), but event ids should be scoped to room, so /event/{id} is wrong.
* | | Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-01-111-1/+1
|\ \ \
| * | | Bump version and changelog release-v0.26.0Erik Johnston2018-01-051-1/+1
| | | |
| * | | Bump changelog and versionErik Johnston2017-12-131-1/+1
| | | |
* | | | Merge pull request #2773 from matrix-org/erikj/hash_bgErik Johnston2018-01-103-10/+18
|\ \ \ \ | | | | | | | | | | Do bcrypt hashing in a background thread
| * | | | Do bcrypt hashing in a background threadErik Johnston2018-01-103-10/+18
| | | | |
* | | | | fix order of operations derp and also use `.get` to default to {}Michael Telatynski2018-01-101-2/+3
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
* | | | | Fix publicised groups API (singular) over federationMichael Telatynski2018-01-101-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | which was missing its fed client API, since there is no other API it might as well reuse the bulk one and unwrap it Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
* | | | | Merge pull request #2770 from matrix-org/rav/fix_request_metricsRichard van der Hoff2018-01-101-11/+19
|\ \ \ \ \ | |_|_|_|/ |/| | | | Update http request metrics before calling servlet
| * | | | Update http request metrics before calling servletRichard van der Hoff2018-01-091-11/+19
| | |_|/ | |/| | | | | | | | | | | | | | Make sure that we set the servlet name in the metrics object *before* calling the servlet, in case the servlet throws an exception.
* | | | Remove dead TODOErik Johnston2018-01-091-2/+0
| | | |
* | | | Remove dead code related to default thumbnailsErik Johnston2018-01-092-76/+3
|/ / /
* | | Merge pull request #2763 from matrix-org/rav/fix_config_utsRichard van der Hoff2018-01-091-2/+2
|\ \ \ | | | | | | | | Fix broken config UTs
| * | | Fix broken config UTsRichard van der Hoff2018-01-091-2/+2
| | | | | | | | | | | | | | | | | | | | https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests.
* | | | Make indentation of generated log config consistentRichard van der Hoff2018-01-091-17/+17
|/ / / | | | | | | | | | (we had a mix of 2- and 4-space indents)
* | | Fix templating error with unban permission messageTravis Ralston2018-01-071-1/+1
| | | | | | | | | | | | | | | Fixes https://github.com/matrix-org/synapse/issues/2759 Signed-off-by: Travis Ralston <travpc@gmail.com>
* | | Remove 'verbosity'/'log_file' from generated cfgRichard van der Hoff2018-01-051-8/+4
| | | | | | | | | | | | | | | | | | ... because these only really exist to confuse people nowadays. Also bring log config more into line with the generated log config, by making `level_for_storage` apply to the `synapse.storage.SQL` logger rather than `synapse.storage`.
* | | Merge pull request #2744 from matrix-org/rav/login_loggingRichard van der Hoff2018-01-051-4/+10
|\ \ \ | | | | | | | | Better logging when login can't find a 3pid
| * | | Better logging when login can't find a 3pidRichard van der Hoff2017-12-201-4/+10
| | | |
* | | | Check missing fields in event_from_pdu_jsonRichard van der Hoff2017-12-301-0/+7
| | | | | | | | | | | | | | | | Return a 400 rather than a 500 when somebody messes up their send_join
* | | | Factor out `event_from_pdu_json`Richard van der Hoff2017-12-303-37/+44
| | | | | | | | | | | | | | | | | | | | turns out we have two copies of this, and neither needs to be an instance method
* | | | federation_server: clean up importsRichard van der Hoff2017-12-301-14/+11
| | | |
* | | | federation_client: clean up importsRichard van der Hoff2017-12-301-13/+10
|/ / /
* | | Merge pull request #2683 from seckrv/fix_pwd_auth_prov_typoRichard van der Hoff2017-12-181-2/+2
|\ \ \ | | | | | | | | synapse/config/password_auth_providers: Fixed bracket typo
| * | | synapse/config/password_auth_providers: Fixed bracket typoRichard von Seck2017-11-161-2/+2
| | | | | | | | | | | | | | | | Signed-off-by: Richard von Seck <richard.von-seck@gmx.net>
* | | | Implement listen_tcp method in remaining workersSilke2017-12-1810-192/+155
| | | | | | | | | | | | | | | | Signed-off-by: Silke <silke@slxh.eu>
* | | | Remove logger argument and do not catch replication listenerSilke2017-12-182-24/+18
| | | | | | | | | | | | | | | | Signed-off-by: Silke <silke@slxh.eu>
* | | | Add methods for listening on multiple addressesSilke Hofstra2017-12-172-51/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP and reactor.listenSSL for multiple addresses. Signed-off-by: Silke Hofstra <silke@slxh.eu>
* | | | Allow binds to both :: and 0.0.0.0Silke Hofstra2017-12-172-42/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed. This causes a warning explaining the behaviour. Configuration changed to match. See #2232 Signed-off-by: Silke Hofstra <silke@slxh.eu>
* | | | Adapt the default config to bind on IPv6.Willem Mulder2017-12-171-8/+9
| |_|/ |/| | | | | | | | | | | | | | | | | | | | | | | Most deployments are on Linux (or Mac OS), so this would actually bind on both IPv4 and IPv6. Resolves #1886. Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-12-0763-781/+1462
|\ \ \ | | | | | | | | | | | | erikj/createroom_content
| * \ \ Merge pull request #2723 from matrix-org/matthew/search-all-local-usersMatthew Hodgson2017-12-0510-36/+221
| |\ \ \ | | | | | | | | | | Add all local users to the user_directory and optionally search them
| | * | | fix StoreError syntaxMatthew Hodgson2017-12-051-1/+1
| | | | |
| | * | | speed up the rate of initial spam for usersMatthew Hodgson2017-12-041-11/+12
| | | | |
| | * | | better docMatthew Hodgson2017-12-041-1/+4
| | | | |
| | * | | fix pep8 and testsMatthew Hodgson2017-12-043-8/+6
| | | | |
| | * | | switch to a simpler 'search_all_users' button as per review feedbackMatthew Hodgson2017-12-045-37/+30
| | | | |
| | * | | Merge branch 'develop' into matthew/search-all-local-usersMatthew Hodgson2017-11-3031-179/+497
| | |\ \ \ | | | | |/ | | | |/|
| | * | | specify default user_directory_include_patternMatthew Hodgson2017-11-301-0/+1
| | | | |
| | * | | remove null constraint on user_dir.room_idMatthew Hodgson2017-11-301-0/+35
| | | | |
| | * | | fix alternation operator for FTS4 - how did this ever work!?Matthew Hodgson2017-11-301-1/+1
| | | | |
| | * | | fix thinkos galoreMatthew Hodgson2017-11-303-20/+41
| | | | |
| | * | | kick the user_directory index when new users registerMatthew Hodgson2017-11-291-0/+8
| | | | |
| | * | | untested WIP but might actually workMatthew Hodgson2017-11-296-20/+96
| | | | |
| | * | | Add user_directory_include_pattern config param to expand search results to ↵Matthew Hodgson2017-11-295-9/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | additional users Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive. It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users... ...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too. Committing it here as a WIP
| * | | | Fix error when deleting devicesRichard van der Hoff2017-12-051-1/+1
| | | | | | | | | | | | | | | | | | | | This was introduced in d7ea8c4 / PR #2728
| * | | | support custom login types for validating usersRichard van der Hoff2017-12-051-24/+57
| | | | | | | | | | | | | | | | | | | | | | | | | Wire the custom login type support from password providers into the UI-auth user-validation flows.
| * | | | Factor out a validate_user_via_ui_auth methodRichard van der Hoff2017-12-053-74/+102
| | | | | | | | | | | | | | | | | | | | Collect together all the places that validate a logged-in user via UI auth.
| * | | | Refactor UI auth implementationRichard van der Hoff2017-12-056-44/+96
| | |/ / | |/| | | | | | | | | | | | | | Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.
| * | | Merge pull request #2721 from matrix-org/rav/get_user_by_access_token_commentsRichard van der Hoff2017-11-291-1/+5
| |\ \ \ | | | | | | | | | | Improve comments on get_user_by_access_token
| | * | | Improve comments on get_user_by_access_tokenRichard van der Hoff2017-11-291-1/+5
| | | | | | | | | | | | | | | | | | | | because I have to reverse-engineer this every time.
| * | | | Delete devices in various logout situationsRichard van der Hoff2017-11-295-5/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure that we delete devices whenever a user is logged out due to any of the following situations: * /logout * /logout_all * change password * deactivate account (by the user or by an admin) * invalidate access token from a dynamic module Fixes #2672.
| * | | | Move set_password into its own handlerRichard van der Hoff2017-11-296-19/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Non-functional refactoring to move set_password. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * | | | Move deactivate_account into its own handlerRichard van der Hoff2017-11-296-22/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Non-functional refactoring to move deactivate_account. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * | | | Remove pushers when deleting access tokensRichard van der Hoff2017-11-293-18/+32
| | | | | | | | | | | | | | | | | | | | | | | | | Whenever an access token is invalidated, we should remove the associated pushers.
| * | | | Merge pull request #2718 from matrix-org/rav/notify_logcontextsRichard van der Hoff2017-11-292-9/+18
| |\ \ \ \ | | | | | | | | | | | | Clear logcontext before starting fed txn queue runner
| | * | | | Clear logcontext before starting fed txn queue runnerRichard van der Hoff2017-11-282-9/+18
| | |/ / / | | | | | | | | | | | | | | | | | | | | These processes take a long time compared to the request, so there is lots of "Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
| * | | | Merge pull request #2719 from matrix-org/rav/handle_missing_hashesRichard van der Hoff2017-11-291-3/+10
| |\ \ \ \ | | | | | | | | | | | | Fix 500 when joining matrix-dev
| | * | | | Fix 500 when joining matrix-devRichard van der Hoff2017-11-291-3/+10
| | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | matrix-dev has an event (`$/6ANj/9QWQyd71N6DpRQPf+SDUu11+HVMeKSpMzBCwM:zemos.net`) which has no `hashes` member. Check for missing `hashes` element in events.
| * / / / Allow guest access to group APIs for readingLuke Barnard2017-11-281-11/+11
| |/ / /
| * | | Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_errorRichard van der Hoff2017-11-276-17/+89
| |\ \ \ | | | | | | | | | | Fix error on sqlite 3.7
| | * | | fix sql failsRichard van der Hoff2017-11-221-1/+1
| | | | |
| | * | | Check database in has_completed_background_updatesRichard van der Hoff2017-11-223-12/+33
| | | | | | | | | | | | | | | | | | | | so that the right thing happens on workers.
| | * | | Fix error on sqlite 3.7Richard van der Hoff2017-11-215-8/+59
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.
| * | | Merge pull request #2713 from matrix-org/rav/no_upsert_foreverRichard van der Hoff2017-11-271-0/+7
| |\ \ \ | | | | | | | | | | Avoid retrying forever on IntegrityError
| | * | | Avoid retrying forever on IntegrityErrorRichard van der Hoff2017-11-271-0/+7
| | | | |
| * | | | Merge pull request #2711 from matrix-org/rav/fix_dns_errhandlerRichard van der Hoff2017-11-271-2/+4
| |\ \ \ \ | | | | | | | | | | | | Fix error handling on dns lookup
| | * | | | Fix error handling on dns lookupRichard van der Hoff2017-11-241-2/+4
| | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | pass the right arguments to the errback handler Fixes "TypeError('eb() takes exactly 2 arguments (1 given)',)"
| * | | | Merge pull request #2710 from matrix-org/rav/remove_dead_codeRichard van der Hoff2017-11-271-2/+1
| |\ \ \ \ | | | | | | | | | | | | Tiny code cleanups
| | * | | | Add a comment which might save some confusionRichard van der Hoff2017-11-241-0/+1
| | | | | |
| | * | | | Remove dead sync_callbackRichard van der Hoff2017-11-241-2/+0
| | |/ / / | | | | | | | | | | | | | | | This is never used; let's remove it to stop confusing things.
| * | | | Merge pull request #2708 from matrix-org/rav/replication_logcontext_leaksRichard van der Hoff2017-11-271-2/+4
| |\ \ \ \ | | | | | | | | | | | | Fix some logcontext leaks in replication resource
| | * | | | Fix some logcontext leaks in replication resourceRichard van der Hoff2017-11-231-2/+4
| | |/ / / | | | | | | | | | | | | | | | | | | | | The @measure_func annotations rely on the wrapped function respecting the logcontext rules. Add the necessary yields to make this work.
| * | | | Merge pull request #2707 from matrix-org/rav/fix_urlpreviewRichard van der Hoff2017-11-271-1/+5
| |\ \ \ \ | | | | | | | | | | | | Fix OPTIONS on preview_url
| | * | | | Fix OPTIONS on preview_urlRichard van der Hoff2017-11-231-1/+5
| | |/ / / | | | | | | | | | | | | | | | Fixes #2706
| * / / / Improve tracebacks on exceptionsRichard van der Hoff2017-11-271-3/+9
| |/ / / | | | | | | | | | | | | | | | | Use failure.Failure to recover our failure, which will give us a useful stacktrace, unlike the rethrown exception.
| * | | Add config option to disable media_repo on main synapseRichard van der Hoff2017-11-223-8/+26
| | | | | | | | | | | | | | | | ... to stop us doing the cache cleanup jobs on the master.
| * | | Build MediaRepositoryResource as a homeserver dependencyRichard van der Hoff2017-11-224-5/+19
| | | | | | | | | | | | | | | | | | | | | | | | This avoids the scenario where we have four different PreviewUrlResources configured on a single app, each of which have their own caches and cache clearing jobs.
| * | | Merge pull request #2698 from matrix-org/rav/remove_dead_dependenciesRichard van der Hoff2017-11-211-16/+15
| |\ \ \ | | | | | | | | | | Clean up dependency list
| | * | | Clean up dependency listRichard van der Hoff2017-11-211-16/+15
| | |/ / | | | | | | | | | | | | | | | | remove those that aren't used at all, and replace the ones that don't have builders with simple getters rather than dynamically-generated methods.
| * | | Merge pull request #2689 from matrix-org/rav/unlock_account_data_upsertRichard van der Hoff2017-11-211-36/+49
| |\ \ \ | | |/ / | |/| | Avoid locking account_data tables for upserts
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-10-31 01:25:08 +0000