summary refs log tree commit diff
path: root/synapse/handlers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #3233 from matrix-org/rav/remove_dead_codeRichard van der Hoff2018-05-181-55/+0
|\ | | | | Remove unused `update_external_syncs`
| * Remove unused `update_external_syncs`Richard van der Hoff2018-05-171-55/+0
| | | | | | | | This method isn't used anywhere. Burninate it.
* | Merge pull request #3221 from matrix-org/erikj/purge_tokenErik Johnston2018-05-181-6/+6
|\ \ | |/ |/| Make purge_history operate on tokens
| * Make purge_history operate on tokensErik Johnston2018-05-151-6/+6
| | | | | | | | As we're soon going to change how topological_ordering works
* | Move RoomCreationHandler out of synapse.handlers.HandlersRichard van der Hoff2018-05-172-6/+7
|/ | | | | | | Handlers is deprecated nowadays, so let's move this out before I add a new dependency on it. Also fix the docstrings on create_room.
* Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivateDavid Baker2018-05-141-2/+79
|\ | | | | Part user from rooms on account deactivate
| * Catch failure to part user from roomDavid Baker2018-05-101-7/+13
| |
| * Many docstringsDavid Baker2018-05-101-0/+24
| |
| * Oops, don't call function passed to run_in_backgroundDavid Baker2018-05-101-1/+1
| |
| * Prefix internal functionsDavid Baker2018-05-091-7/+7
| |
| * Indent failDavid Baker2018-05-091-1/+1
| |
| * Part deactivated users in the backgroundDavid Baker2018-05-091-1/+34
| | | | | | | | | | One room at a time so we don't take out the whole server with leave events, and restart at server restart.
| * Part user from rooms on account deactivateDavid Baker2018-05-081-1/+15
| | | | | | | | | | | | | | This implements this very crudely: this probably isn't viable because parting a user from all their rooms could take a long time, and if the HS gets restarted in that time the process will be aborted.
* | Merge pull request #3199 from matrix-org/erikj/pagination_syncErik Johnston2018-05-091-6/+18
|\ \ | | | | | | Refactor sync APIs to reuse pagination API
| * | Add comment to sync as to why code path is splitErik Johnston2018-05-091-0/+5
| | |
| * | Refactor sync APIs to reuse pagination APIErik Johnston2018-05-091-6/+13
| | | | | | | | | | | | | | | | | | The sync API often returns events in a topological rather than stream ordering, e.g. when the user joined the room or on initial sync. When this happens we can reuse existing pagination storage functions.
* | | Fix returned token is no longer a tupleErik Johnston2018-05-091-1/+1
|/ /
* / Refactor get_recent_events_for_room return typeErik Johnston2018-05-092-6/+6
|/ | | | | | There is no reason to return a tuple of tokens when the last token is always the token passed as an argument. Changing it makes it consistent with other storage APIs
* Merge branch 'release-v0.28.1' into developRichard van der Hoff2018-05-011-1/+5
|\
| * Apply some limits to depth to counter abuseRichard van der Hoff2018-05-011-1/+5
| | | | | | | | | | | | | | * When creating a new event, cap its depth to 2^63 - 1 * When receiving events, reject any without a sensible depth As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
* | Merge pull request #3143 from matrix-org/rav/remove_redundant_preserve_fnRichard van der Hoff2018-04-301-4/+2
|\ \ | | | | | | Remove redundant call to preserve_fn
| * | Remove redundant call to preserve_fnRichard van der Hoff2018-04-301-4/+2
| | | | | | | | | | | | submit_event_for_as doesn't return a deferred anyway, so this is pointless.
* | | Merge pull request #3153 from NotAFile/py3-httplibRichard van der Hoff2018-04-301-3/+3
|\ \ \ | | | | | | | | move httplib import to six
| * | | move httplib import to sixAdrian Tschira2018-04-281-3/+3
| | | |
* | | | Merge pull request #3151 from NotAFile/py3-xrange-1Richard van der Hoff2018-04-301-1/+3
|\ \ \ \ | | | | | | | | | | Move more xrange to six
| * \ \ \ Merge branch 'develop' into py3-xrange-1Richard van der Hoff2018-04-306-21/+32
| |\ \ \ \ | | | |/ / | | |/| |
| * | | | Move more xrange to sixAdrian Tschira2018-04-281-1/+3
| | |/ / | |/| | | | | | | | | | | | | | | | | | plus a bonus next() Signed-off-by: Adrian Tschira <nota@notafile.com>
* | | | Merge pull request #3150 from NotAFile/py3-listcomp-yieldRichard van der Hoff2018-04-301-5/+9
|\ \ \ \ | |_|/ / |/| | | Don't yield in list comprehensions
| * | | Don't yield in list comprehensionsAdrian Tschira2018-04-281-5/+9
| |/ / | | | | | | | | | | | | | | | I've tried to grep for more of this with no success. Signed-off-by: Adrian Tschira <nota@notafile.com>
* | | pep8; remove spurious importRichard van der Hoff2018-04-271-1/+1
| | |
* | | Merge remote-tracking branch 'origin/develop' into rav/use_run_in_backgroundRichard van der Hoff2018-04-276-74/+112
|\| |
| * | Merge pull request #3142 from matrix-org/rav/reraiseRichard van der Hoff2018-04-272-13/+24
| |\ \ | | | | | | | | reraise exceptions more carefully
| | * | reraise exceptions more carefullyRichard van der Hoff2018-04-272-13/+24
| | |/ | | | | | | | | | | | | | | | | | | We need to be careful (under python 2, at least) that when we reraise an exception after doing some error handling, we actually reraise the original exception rather than anything that might have been raised (and handled) during the error handling.
| * | Merge pull request #3138 from matrix-org/rav/catch_unhandled_exceptionsRichard van der Hoff2018-04-274-59/+86
| |\ \ | | | | | | | | Improve exception handling for background processes
| | * | Improve exception handling for background processesRichard van der Hoff2018-04-274-59/+86
| | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.
| * / Add missing consumeErrorsRichard van der Hoff2018-04-271-2/+2
| |/ | | | | | | | | In general we want defer.gatherResults to consumeErrors, rather than having exceptions hanging around and getting logged as CRITICAL unhandled errors.
* / Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-276-20/+31
|/ | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* Merge pull request #3118 from matrix-org/rav/reject_prev_eventsRichard van der Hoff2018-04-231-7/+71
|\ | | | | Reject events which have lots of prev_events
| * Remove spurious paramRichard van der Hoff2018-04-231-1/+0
| |
| * accept stupid events over backfillRichard van der Hoff2018-04-201-4/+9
| |
| * Add some commentsRichard van der Hoff2018-04-181-3/+12
| |
| * Check events on backfill tooRichard van der Hoff2018-04-181-15/+42
| |
| * Reject events which have too many auth_events or prev_eventsRichard van der Hoff2018-04-181-4/+28
| | | | | | | | | | ... this should protect us from being dossed by people making silly events (deliberately or otherwise)
* | Merge pull request #3107 from NotAFile/py3-bool-nonzeroRichard van der Hoff2018-04-201-0/+7
|\ \ | | | | | | add __bool__ alias to __nonzero__ methods
| * | add __bool__ alias to __nonzero__ methodsAdrian Tschira2018-04-151-0/+7
| |/ | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Merge pull request #3093 from matrix-org/rav/response_cache_wrapRichard van der Hoff2018-04-202-35/+19
|\ \ | | | | | | Refactor ResponseCache usage
| * | Refactor ResponseCache usageRichard van der Hoff2018-04-122-35/+19
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production.
* | Merge pull request #3117 from matrix-org/rav/refactor_have_eventsRichard van der Hoff2018-04-201-19/+12
|\ \ | | | | | | Refactor store.have_events
| * | Refactor store.have_eventsRichard van der Hoff2018-04-201-19/+12
| |/ | | | | | | | | | | It turns out that most of the time we were calling have_events, we were only using half of the result. Replace have_events with have_seen_events and get_rejection_reasons, so that we can see what's going on a bit more clearly.
* / Avoid creating events with huge numbers of prev_eventsRichard van der Hoff2018-04-162-38/+53
|/ | | | | | In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events.
* Merge pull request #3092 from matrix-org/rav/response_cache_metricsRichard van der Hoff2018-04-122-3/+4
|\ | | | | Add metrics for ResponseCache
| * Add metrics for ResponseCacheRichard van der Hoff2018-04-102-3/+4
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-04-121-2/+19
|\ \ | | | | | | | | | erikj/processed_event_lag
| * \ Merge pull request #3088 from matrix-org/erikj/as_parallelErik Johnston2018-04-121-2/+19
| |\ \ | | | | | | | | Send events to ASes concurrently
| | * | Send events to ASes concurrentlyErik Johnston2018-04-111-2/+19
| | | |
* | | | Set all metrics at the same timeErik Johnston2018-04-121-4/+4
| | | |
* | | | Track last processed event received_tsErik Johnston2018-04-111-0/+10
| | | |
* | | | Track where event stream processing have gotten up toErik Johnston2018-04-111-0/+4
|/ / /
* | | Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sendsErik Johnston2018-04-101-49/+55
|\ \ \ | |/ / |/| | Limit concurrent event sends for a room
| * | Note why we're limiting concurrent event sendsErik Johnston2018-04-101-0/+5
| | |
| * | Limit concurrent event sends for a roomErik Johnston2018-04-091-49/+50
| | |
* | | Merge pull request #2996 from krombel/allow_auto_join_roomsRichard van der Hoff2018-04-101-2/+34
|\ \ \ | |_|/ |/| | move handling of auto_join_rooms to RegisterHandler
| * | Merge branch 'develop' of into allow_auto_join_roomsKrombel2018-03-283-8/+14
| |\ \
| * | | move handling of auto_join_rooms to RegisterHandlerKrombel2018-03-141-2/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de>
* | | | Merge pull request #3080 from matrix-org/rav/fix_500_on_rejoinRichard van der Hoff2018-04-091-0/+8
|\ \ \ \ | |_|_|/ |/| | | Return a 404 rather than a 500 on rejoining empty rooms
| * | | Return a 404 rather than a 500 on rejoining empty roomsRichard van der Hoff2018-04-091-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Filter ourselves out of the server list before checking for an empty remote host list, to fix 500 error Fixes #2141
* | | | Implement group join APIDavid Baker2018-04-061-1/+39
|/ / /
* | | Merge pull request #3045 from matrix-org/dbkr/group_joinableLuke Barnard2018-04-051-0/+3
|\ \ \ | | | | | | | | Add joinability for groups
| * | | Use join_policy API instead of joinableLuke Barnard2018-04-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The API is now under /groups/$group_id/setting/m.join_policy and expects a JSON blob of the shape ```json { "m.join_policy": { "type": "invite" } } ``` where "invite" could alternatively be "open".
| * | | Add joinability for groupsDavid Baker2018-03-281-0/+3
| | | | | | | | | | | | | | | | | | | | Adds API to set the 'joinable' flag, and corresponding flag in the table.
* | | | Merge pull request #3000 from NotAFile/change-except-styleRichard van der Hoff2018-04-041-3/+3
|\ \ \ \ | | | | | | | | | | Replace old style error catching with 'as' keyword
| * | | | replace old style error catching with 'as' keywordNotAFile2018-03-151-3/+3
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | This is both easier to read and compatible with python3 (not that that matters) Signed-off-by: Adrian Tschira <nota@notafile.com>
* | | | Merge pull request #3049 from matrix-org/rav/use_staticjsonRichard van der Hoff2018-04-031-2/+2
|\ \ \ \ | | | | | | | | | | Use static JSONEncoders
| * | | | Use static JSONEncodersRichard van der Hoff2018-03-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | using json.dumps with custom options requires us to create a new JSONEncoder on each call. It's more efficient to create one upfront and reuse it.
* | | | | Use simplejson throughoutRichard van der Hoff2018-03-291-3/+5
|/ / / / | | | | | | | | | | | | Let's use simplejson rather than json, for consistency.
* | | | Merge pull request #3034 from matrix-org/rav/fix_key_claim_errorsRichard van der Hoff2018-03-281-28/+28
|\ \ \ \ | |_|_|/ |/| | | Fix error when claiming e2e keys from offline servers
| * | | Stringify exceptions for keys/{query,claim}Richard van der Hoff2018-03-271-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure we stringify any exceptions we return from keys/query and keys/claim, to avoid a 'not JSON serializable' error later Fixes #3010
| * | | factor out exception handling for keys/claim and keys/queryRichard van der Hoff2018-03-271-28/+25
| | |/ | |/| | | | | | | this stuff is badly c&p'ed
* | | Also do check inside linearizerErik Johnston2018-03-271-3/+4
| | |
* | | PEP8Erik Johnston2018-03-261-1/+3
| | |
* | | Linearize calls to _generate_user_idErik Johnston2018-03-261-4/+7
|/ /
* | Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-192-4/+4
|\ \ | |/ |/|
| * Replace ujson with simplejsonErik Johnston2018-03-152-4/+4
| |
* | Merge pull request #2992 from matrix-org/erikj/implement_member_workreErik Johnston2018-03-141-0/+102
|\ \ | | | | | | Implement RoomMemberWorkerHandler
| * | s/join/joined/ in notify_user_membership_changeErik Johnston2018-03-141-1/+1
| | |
| * | Split RoomMemberWorkerHandler to separate fileErik Johnston2018-03-142-78/+102
| | |
| * | Implement RoomMemberWorkerHandlerErik Johnston2018-03-131-0/+78
| | |
* | | Merge pull request #2989 from matrix-org/erikj/profile_cache_masterErik Johnston2018-03-141-1/+4
|\ \ \ | | | | | | | | Only update remote profile cache on master
| * | | Only update remote profile cache on masterErik Johnston2018-03-131-1/+4
| | | |
* | | | _remote_join and co take a requesterErik Johnston2018-03-131-4/+4
| |/ / |/| |
* | | Merge pull request #2987 from matrix-org/erikj/split_room_member_handlerErik Johnston2018-03-131-98/+185
|\ \ \ | |/ / |/| | Split RoomMemberHandler into base and master class
| * | Add missing param to docstringsErik Johnston2018-03-131-0/+3
| | |
| * | Correct import orderErik Johnston2018-03-131-3/+3
| | |
| * | Move user_*_room distributor stuff to master classErik Johnston2018-03-131-5/+50
| | | | | | | | | | | | | | | I added yields when calling user_left_room, but they shouldn't matter on the master process as they always return None anyway.
| * | Split RoomMemberHandler into base and master classErik Johnston2018-03-131-96/+135
| | | | | | | | | | | | | | | | | | | | | | | | The intention here is to split the class into the bits that can be done on workers and the bits that have to be done on the master. In future there will also be a class that can be run on the worker, which will delegate work to the master when necessary.
* | | Merge pull request #2978 from matrix-org/erikj/refactor_replication_layerErik Johnston2018-03-136-6/+6
|\ \ \ | | | | | | | | Remove ReplicationLayer and user Client/Server directly
| * | | s/replication_client/federation_client/Erik Johnston2018-03-136-6/+6
| | | |
* | | | Merge pull request #2981 from matrix-org/erikj/factor_remote_leaveErik Johnston2018-03-131-22/+54
|\ \ \ \ | | |/ / | |/| | Factor out _remote_reject_invite in RoomMember
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-03-1311-31/+49
| |\ \ \ | | | | | | | | | | | | | | | erikj/factor_remote_leave
| * | | | Add docstringErik Johnston2018-03-131-0/+26
| | | | |
| * | | | Factor out _remote_reject_invite in RoomMemberErik Johnston2018-03-131-22/+28
| | | | |
* | | | | Merge pull request #2979 from matrix-org/erikj/no_handlersErik Johnston2018-03-138-12/+7
|\ \ \ \ \ | |_|/ / / |/| | / / | | |/ / | |/| | Don't build handlers on workers unnecessarily
| * | | Split replication layer into twoErik Johnston2018-03-138-12/+7
| | | |
* | | | Merge pull request #2980 from matrix-org/erikj/rm_privErik Johnston2018-03-131-7/+7
|\ \ \ \ | | | | | | | | | | Make RoomMemberHandler functions private that can be
| * | | | Make functions private that can beErik Johnston2018-03-131-7/+7
| | |/ / | |/| |
* | | | Merge pull request #2982 from matrix-org/erikj/fix_extra_usersErik Johnston2018-03-131-1/+1
|\ \ \ \ | | | | | | | | | | extra_users is actually a list of UserIDs
| * | | | extra_users is actually a list of UserIDsErik Johnston2018-03-131-1/+1
| |/ / /
* | | | Merge pull request #2983 from matrix-org/erikj/rename_register_3pidErik Johnston2018-03-132-11/+25
|\ \ \ \ | |_|/ / |/| | | Refactor get_or_register_3pid_guest
| * | | Refactor get_or_register_3pid_guestErik Johnston2018-03-132-11/+25
| |/ /
* / / Split out edu/query registration to a separate classErik Johnston2018-03-138-12/+16
|/ /
* | Add transactional API to history purgeRichard van der Hoff2018-03-121-5/+99
| | | | | | | | Make the purge request return quickly, and allow scripts to poll for updates.
* | Return an error when doing two purges on a roomRichard van der Hoff2018-03-121-3/+14
| | | | | | | | Queuing up purges doesn't sound like a good thing.
* | Fix up log messageErik Johnston2018-03-071-1/+1
| |
* | Fix typoErik Johnston2018-03-071-2/+2
| |
* | Fix race in sync when joining roomErik Johnston2018-03-071-28/+75
| | | | | | | | | | | | | | | | | | | | | | The race happens when the user joins a room at the same time as doing a sync. We fetch the current token and then get the rooms the user is in. If the join happens after the current token, but before we get the rooms we end up sending down a partial room entry in the sync. This is fixed by looking at the stream ordering of the membership returned by get_rooms_for_user, and handling the case when that stream ordering is after the current token.
* | Merge pull request #2949 from krombel/use_bcrypt_checkpwRichard van der Hoff2018-03-061-2/+4
|\ \ | | | | | | use bcrypt.checkpw
| * | use bcrypt.checkpwKrombel2018-03-051-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | in bcrypt 3.1.0 checkpw got introduced (already 2 years ago) This makes use of that with enhancements which might get introduced by that Signed-Off-by: Matthias Kesler <krombel@krombel.de>
* | | Merge pull request #2946 from matrix-org/rav/timestamp_to_purgeRichard van der Hoff2018-03-061-9/+5
|\ \ \ | | | | | | | | Implement purge_history by timestamp
| * | | Provide a means to pass a timestamp to purge_historyRichard van der Hoff2018-03-051-9/+5
| |/ /
* / / Remove ability for AS users to call /events and /syncErik Johnston2018-03-052-10/+9
|/ / | | | | | | | | | | | | This functionality has been deprecated for a while as well as being broken for a while. Instead of fixing it lets just remove it entirely. See: https://github.com/matrix-org/matrix-doc/issues/1144
* | Add missing yield during 3pid signature checksErik Johnston2018-03-021-1/+1
| |
* | Merge pull request #2927 from matrix-org/erikj/read_marker_cachesErik Johnston2018-03-011-3/+3
|\ \ | | | | | | Improve caching for read_marker API
| * | Improve caching for read_marker APIErik Johnston2018-03-011-3/+3
| | | | | | | | | | | | | | | | | | We add a new storage function to get a paritcular type of room account data. This allows us to prefill the cache when updating that acount data.
* | | Move back to hs.is_mineErik Johnston2018-03-011-7/+6
| | |
* | | Move RoomMemberHandler out of HandlersErik Johnston2018-03-016-31/+37
|/ /
* | Check event auth on the workerErik Johnston2018-03-011-16/+16
| |
* | Correctly send ratelimit and extra_users paramsErik Johnston2018-03-011-0/+2
| |
* | Fixup commentsErik Johnston2018-03-011-4/+4
| |
* | Calculate push actions on workerErik Johnston2018-02-281-25/+61
| |
* | Delete from push_actions_staging in federation tooErik Johnston2018-02-201-9/+17
| |
* | Ensure that we delete staging push actions on errorsErik Johnston2018-02-151-3/+9
| |
* | Merge pull request #2854 from matrix-org/erikj/event_create_workerErik Johnston2018-02-131-6/+22
|\ \ | | | | | | Create a worker for event creation
| * | Move presence handling into handle_new_client_eventErik Johnston2018-02-071-6/+6
| | | | | | | | | | | | As we want to have it run on the main synapse instance
| * | Add replication http endpoint for event sendingErik Johnston2018-02-071-0/+16
| | |
* | | Merge pull request #2858 from matrix-org/rav/purge_updatesRichard van der Hoff2018-02-091-2/+2
|\ \ \ | |/ / |/| | delete_local_events for purge_room_history
| * | delete_local_events for purge_historyRichard van der Hoff2018-02-091-2/+2
| | | | | | | | | | | | Add a flag which makes the purger delete local events
| * | rename delete_old_state -> purge_historyRichard van der Hoff2018-02-091-1/+1
| | | | | | | | | | | | (beacause it deletes more than state)
* | | Remove pointless ratelimit checkErik Johnston2018-02-071-5/+0
| | | | | | | | | | | | | | | | | | The intention was for the check to be called as early as possible in the request, but actually was called just before the main ratelimit check, so was fairly pointless.
* | | Merge pull request #2847 from matrix-org/erikj/separate_event_creationErik Johnston2018-02-065-179/+193
|\ \ \ | | | | | | | | Split event creation into a separate handler
| * | | Update copyrightErik Johnston2018-02-064-1/+4
| | | |
| * | | s/_create_new_client_event/create_new_client_event/Erik Johnston2018-02-062-8/+8
| | | |
| * | | Update places where we create eventsErik Johnston2018-02-054-29/+26
| | | |
| * | | Split event creation into a separate handlerErik Johnston2018-02-051-146/+160
| | | |
* | | | Store state groups separately from events (#2784)Erik Johnston2018-02-061-7/+17
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* | | Rename resolve_state_groups -> resolve_state_groups_for_eventsRichard van der Hoff2018-02-011-6/+5
| | | | | | | | | | | | (to make way for a method that actually just does the state group resolution)
* | | add white space lineNeil Johnson2018-01-261-0/+1
| | |
* | | rather than try reconstruct the results object, better to guard against the ↵Neil Johnson2018-01-261-6/+2
| | | | | | | | | | | | xrange step argument being 0
* | | fix return type, should be a dictNeil Johnson2018-01-251-1/+1
| | |
* | | fix PEP8 violationNeil Johnson2018-01-251-1/+1
| | |
* | | remove white spaceNeil Johnson2018-01-251-1/+0
| | |
* | | synapse 500s on a call to publicRooms in the case where the number of public ↵Neil Johnson2018-01-251-0/+5
| | | | | | | | | | | | rooms is zero, the specific cause is due to xrange trying to use a step value of zero, but if the total room number really is zero then it makes sense to just bail and save the extra processing
* | | Add federation_domain_whitelist option (#2820)Matthew Hodgson2018-01-223-1/+15
| | | | | | | | | | | | | | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
* | | rewrite based on PR feedback:Matthew Hodgson2018-01-191-10/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | * [ ] split config options into allowed_local_3pids and registrations_require_3pid * [ ] simplify and comment logic for picking registration flows * [ ] fix docstring and move check_3pid_allowed into a new util module * [ ] use check_3pid_allowed everywhere @erikjohnston PTAL
* | | fix up v1, and improve errorsMatthew Hodgson2018-01-191-1/+12
|/ /
* | Sanity checking for user idsRichard van der Hoff2018-01-172-8/+19
| | | | | | | | | | | | | | | | Check the user_id passed to a couple of APIs for validity, to avoid "IndexError: list index out of range" exception which looks scary and results in a 500 rather than a more useful error. Fixes #1432, among other things
* | Metrics for events processed in appservice and fed senderRichard van der Hoff2018-01-151-0/+7
| | | | | | | | More metrics I wished I'd had
* | Merge pull request #2773 from matrix-org/erikj/hash_bgErik Johnston2018-01-103-10/+18
|\ \ | | | | | | Do bcrypt hashing in a background thread
| * | Do bcrypt hashing in a background threadErik Johnston2018-01-103-10/+18
| | |
* | | fix order of operations derp and also use `.get` to default to {}Michael Telatynski2018-01-101-2/+3
| | | | | | | | | | | | Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
* | | Fix publicised groups API (singular) over federationMichael Telatynski2018-01-101-4/+4
| |/ |/| | | | | | | | | | | which was missing its fed client API, since there is no other API it might as well reuse the bulk one and unwrap it Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2017-12-0712-116/+389
|\ \ | | | | | | | | | erikj/createroom_content
| * \ Merge pull request #2723 from matrix-org/matthew/search-all-local-usersMatthew Hodgson2017-12-053-16/+78
| |\ \ | | | | | | | | Add all local users to the user_directory and optionally search them
| | * | speed up the rate of initial spam for usersMatthew Hodgson2017-12-041-11/+12
| | | |
| | * | fix pep8 and testsMatthew Hodgson2017-12-042-4/+3
| | | |
| | * | switch to a simpler 'search_all_users' button as per review feedbackMatthew Hodgson2017-12-043-6/+6
| | | |
| | * | Merge branch 'develop' into matthew/search-all-local-usersMatthew Hodgson2017-11-304-37/+139
| | |\|
| | * | fix thinkos galoreMatthew Hodgson2017-11-301-4/+6
| | | |
| | * | kick the user_directory index when new users registerMatthew Hodgson2017-11-291-0/+8
| | | |
| | * | untested WIP but might actually workMatthew Hodgson2017-11-292-4/+56
| | | |
| | * | Add user_directory_include_pattern config param to expand search results to ↵Matthew Hodgson2017-11-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | additional users Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive. It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users... ...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too. Committing it here as a WIP
| * | | support custom login types for validating usersRichard van der Hoff2017-12-051-24/+57
| | | | | | | | | | | | | | | | | | | | Wire the custom login type support from password providers into the UI-auth user-validation flows.
| * | | Factor out a validate_user_via_ui_auth methodRichard van der Hoff2017-12-051-0/+43
| | | | | | | | | | | | | | | | Collect together all the places that validate a logged-in user via UI auth.
| * | | Refactor UI auth implementationRichard van der Hoff2017-12-051-17/+29
| | |/ | |/| | | | | | | | | | Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.
| * | Delete devices in various logout situationsRichard van der Hoff2017-11-293-1/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure that we delete devices whenever a user is logged out due to any of the following situations: * /logout * /logout_all * change password * deactivate account (by the user or by an admin) * invalidate access token from a dynamic module Fixes #2672.
| * | Move set_password into its own handlerRichard van der Hoff2017-11-292-16/+45
| | | | | | | | | | | | | | | | | | Non-functional refactoring to move set_password. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * | Move deactivate_account into its own handlerRichard van der Hoff2017-11-292-16/+44
| | | | | | | | | | | | | | | | | | Non-functional refactoring to move deactivate_account. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.
| * | Remove pushers when deleting access tokensRichard van der Hoff2017-11-291-4/+12
| |/ | | | | | | | | Whenever an access token is invalidated, we should remove the associated pushers.
| * Merge pull request #2686 from matrix-org/luke/as-flairLuke Barnard2017-11-171-0/+11
| |\ | | | | | | Add automagical AS Publicised Group(s)
| | * Make sure we check AS groups for lookup on bulkLuke Barnard2017-11-161-0/+5
| | |
| | * Add automagical AS Publicised Group(s)Luke Barnard2017-11-161-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | via registration file "users" namespace: ```YAML ... namespaces: users: - exclusive: true regex: '.*luke.*' group_id: '+all_the_lukes:hsdomain' ... ``` This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.
| * | typoMatthew Hodgson2017-11-171-1/+1
| | |
| * | Fix auth handler #2678Jurek2017-11-161-1/+1
| |/
| * Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcsRichard van der Hoff2017-11-152-4/+4
| |\ | | | | | | Remove preserve_context_over_{fn, deferred}
| | * Remove preserve_context_over_{fn, deferred}Richard van der Hoff2017-11-142-4/+4
| | | | | | | | | | | | | | | Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off.
| * | Reshuffle room list request codeRichard van der Hoff2017-11-141-27/+24
| | | | | | | | | | | | | | | | | | I'm not entirely sure if this will actually help anything, but it simplifies the code and might give further clues about why room list search requests are blowing out the get_current_state_ids caches.
| * | Add a load of logging to the room_list handlerRichard van der Hoff2017-11-141-0/+13
| |/ | | | | | | So we can see what it gets up to.
* | Copy dict in update_membership tooErik Johnston2017-12-071-0/+4
| |
* | PEP8Erik Johnston2017-11-281-1/+1
| |
* | Fix wrong avatars when inviting multiple users when creating roomErik Johnston2017-11-281-5/+5
| | | | | | We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user
* | Fix auth handler #2678Jurek2017-11-171-1/+1
|/
* Revert "Allow upper-case characters in mxids"Richard van der Hoff2017-11-091-3/+3
| | | | This reverts commit b70b64690330c25cbd04c1b2cacf8276b566efc8.
* Allow upper-case characters in mxidsRichard van der Hoff2017-11-091-3/+3
| | | | Because we're never going to be able to fix this :'(
* Have an explicit API to update room configErik Johnston2017-11-081-0/+1
|
* Revert "Modify group room association API to allow modification of is_public"Erik Johnston2017-11-081-2/+2
|
* Merge pull request #2631 from xyzz/fix_appservice_event_backlogErik Johnston2017-11-081-5/+1
|\ | | | | Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
| * Remove useless assignment in notify_interested_servicesIlya Zhuravlev2017-11-071-1/+0
| |
| * Fix appservices being backlogged and not receiving new events due to a bug ↵Ilya Zhuravlev2017-11-021-4/+1
| | | | | | | | in notify_interested_services
* | Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_resRichard van der Hoff2017-11-081-20/+46
|\ \ | | | | | | Fix bug in state group storage
| * | s/items/iteritems/Richard van der Hoff2017-11-081-2/+2
| | |
| * | Update deltas when doing auth resolutionRichard van der Hoff2017-11-071-3/+7
| | | | | | | | | | | | | | | Fixes a bug where the persisted state groups were different to those actually being used after auth resolution.
| * | factor out _update_context_for_auth_eventsRichard van der Hoff2017-11-071-20/+42
| | | | | | | | | | | | This is duplicated, so let's factor it out before fixing it
* | | Merge pull request #2643 from matrix-org/matthew/user_dir_typosMatthew Hodgson2017-11-071-2/+2
|\ \ \ | |/ / |/| | Fix various embarrassing typos around user_directory and add some doc.
| * | s/intial/initial/Matthew Hodgson2017-11-041-2/+2
| |/
* | Merge pull request #2636 from farialima/me-masterErik Johnston2017-11-071-13/+8
|\ \ | |/ |/| Fix for #2635: correctly update rooms avatar/display name when modified by admin
| * removed unused import flagged by flake8aFrancois Granade2017-11-031-1/+0
| |
| * Fix for issue 2635: correctly update rooms avatar/display name when modified ↵Francois Granade2017-11-031-12/+8
| | | | | | | | by admin
* | Factor _AccountHandler proxy out to ModuleApiRichard van der Hoff2017-11-021-69/+3
| | | | | | | | | | We're going to need to use this from places that aren't password auth, so let's move it to a proper class.
* | Merge pull request #2624 from matrix-org/rav/password_provider_notify_logoutDavid Baker2017-11-021-2/+24
|\ \ | | | | | | Notify auth providers on logout
| * | Notify auth providers on logoutRichard van der Hoff2017-11-011-2/+24
| | | | | | | | | | | | Provide a hook by which auth providers can be notified of logouts.
* | | Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providersDavid Baker2017-11-021-6/+11
|\ \ \ | | | | | | | | Allow password_auth_providers to return a callback
| * | | Fix user-interactive password authRichard van der Hoff2017-11-011-1/+3
| | | | | | | | | | | | | | | | this got broken in the previous commit
| * | | Allow password_auth_providers to return a callbackRichard van der Hoff2017-11-011-5/+8
| |/ / | | | | | | | | | ... so that they have a way to record access tokens.
* | | Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providersDavid Baker2017-11-021-0/+16
|\ \ \ | |/ / |/| | Let auth providers get to the database
| * | Let auth providers get to the databaseRichard van der Hoff2017-10-311-0/+16
| | | | | | | | | | | | Somewhat open to abuse, but also somewhat unavoidable :/
* | | Merge pull request #2620 from matrix-org/rav/auth_non_passwordRichard van der Hoff2017-11-011-22/+96
|\ \ \ | | | | | | | | Let password auth providers handle arbitrary login types
| * | | Let password auth providers handle arbitrary login typesRichard van der Hoff2017-11-011-22/+96
| |/ / | | | | | | | | | | | | Provide a hook where password auth providers can say they know about other login types, and get passed the relevant parameters
* | | Merge remote-tracking branch 'origin/develop' into ↵David Baker2017-11-013-13/+8
|\ \ \ | | | | | | | | | | | | rav/refactor_accesstoken_delete
| * \ \ Merge pull request #2615 from matrix-org/rav/break_auth_device_depDavid Baker2017-11-011-9/+6
| |\ \ \ | | | | | | | | | | Break dependency of auth_handler on device_handler
| | * | | Break dependency of auth_handler on device_handlerRichard van der Hoff2017-11-011-9/+6
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I'm going to need to make the device_handler depend on the auth_handler, so I need to break this dependency to avoid a cycle. It turns out that the auth_handler was only using the device_handler in one place which was an edge case which we can more elegantly handle by throwing an error rather than fixing it up.
| * | | Merge pull request #2613 from matrix-org/rav/kill_refresh_tokensDavid Baker2017-11-011-2/+0
| |\ \ \ | | | | | | | | | | Remove the last vestiges of refresh_tokens
| | * | | Remove the last vestiges of refresh_tokensRichard van der Hoff2017-10-311-2/+0
| | |/ /
| * / / Modify group room association API to allow modification of is_publicLuke Barnard2017-10-311-2/+2
| |/ / | | | | | | | | | also includes renamings to make things more consistent.
* / / Move access token deletion into auth handlerRichard van der Hoff2017-11-013-5/+52
|/ / | | | | | | | | | | | | Also move duplicated deactivation code into the auth handler. I want to add some hooks when we deactivate an access token, so let's bring it all in here so that there's somewhere to put it.
* | Refactor some logic from LoginRestServlet into AuthHandlerRichard van der Hoff2017-10-311-28/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | I'm going to need some more flexibility in handling login types in password auth providers, so as a first step, move some stuff from LoginRestServlet into AuthHandler. In particular, we pass everything other than SAML, JWT and token logins down to the AuthHandler, which now has responsibility for checking the login type and fishing the password out of the login dictionary, as well as qualifying the user_id if need be. Ideally SAML, JWT and token would go that way too, but there's no real need for it right now and I'm trying to minimise impact. This commit *should* be non-functional.
* | Merge pull request #2589 from matrix-org/rav/as_deactivate_accountRichard van der Hoff2017-10-271-1/+1
|\ \ | | | | | | Allow ASes to deactivate their own users
| * | Allow ASes to deactivate their own usersRichard van der Hoff2017-10-271-1/+1
| |/
* | Update room_list.pyErik Johnston2017-10-261-1/+1
| |
* | Do logcontexts outside ResponseCacheErik Johnston2017-10-252-5/+8
| |
* | Add logging and fix log contexts for publicRoomsErik Johnston2017-10-251-0/+2
|/
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-238-17/+17
| | | | what could possibly go wrong
* Merge pull request #2561 from matrix-org/rav/id_checkingRichard van der Hoff2017-10-231-9/+6
|\ | | | | Updates to ID checking
| * Allow = in mxids and groupidsRichard van der Hoff2017-10-201-3/+2
| | | | | | | | ... because the spec says we should.
| * Disallow capital letters in useridsRichard van der Hoff2017-10-201-6/+4
| | | | | | | | | | Factor out a common function for checking user ids and group ids, which forbids capitals.
* | Remove pointless create() methodRichard van der Hoff2017-10-202-3/+3
|/ | | | | It just calls the constructor, so we may as well kill it rather than having random codepaths.
* _create_rererouter for get_invited_users_in_groupLuke Barnard2017-10-161-17/+2
|
* Implement GET /groups/$groupId/invited_usersLuke Barnard2017-10-161-0/+17
|
* Log a warning when no profile for invited memberLuke Barnard2017-10-161-2/+5
| | | | And return empty profile
* Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messagesDavid Baker2017-10-121-2/+6
|\ | | | | Allow error strings from spam checker
| * fix isinstanceDavid Baker2017-10-111-1/+1
| |
| * Allow error strings from spam checkerDavid Baker2017-10-111-2/+6
| |
* | Fix fetching remote summariesErik Johnston2017-10-111-2/+2
| |
* | Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontextRichard van der Hoff2017-10-111-0/+4
|\ \ | | | | | | fix a logcontext leak in read receipt handling
| * | fix a logcontext leak in read receipt handlingRichard van der Hoff2017-10-111-0/+4
| | |
* | | Fix attestations to check correct server nameErik Johnston2017-10-111-10/+20
| |/ |/|
* | Merge pull request #2466 from matrix-org/erikj/groups_mergedErik Johnston2017-10-118-17/+584
|\ \ | | | | | | Initial Group Implementation
| * \ Merge branch 'develop' into erikj/groups_mergedDavid Baker2017-10-024-8/+121
| |\ \
| * | | Add remove room APIErik Johnston2017-09-261-0/+1
| | | |
| * | | Add is_publicised to group summaryErik Johnston2017-09-261-25/+31
| | | |
| * | | Ensure that creator of group sees group down /syncErik Johnston2017-09-192-4/+31
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_mergedErik Johnston2017-09-193-33/+163
| |\ \ \