summary refs log tree commit diff
path: root/synapse/handlers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Include event when resolving state for missing prevsRichard van der Hoff2018-09-271-2/+16
| | | | | | | | | | | If we have a forward extremity for a room as `E`, and you receive `A`, `B`, s.t. `A -> B -> E`, and `B` also points to an unknown event `X`, then we need to do state res between `X` and `E`. When that happens, we need to make sure we include `X` in the state that goes into the state res alg. Fixes #3934.
* Include state from remote servers in pdu handlingRichard van der Hoff2018-09-271-4/+19
| | | | | | | | If we've fetched state events from remote servers in order to resolve the state for a new event, we need to actually pass those events into resolve_events_with_factory (so that it can do the state res) and then persist the ones we need - otherwise other bits of the codebase get confused about why we have state groups pointing to non-existent events.
* Fix "unhashable type: 'list'" exception in federation handlingRichard van der Hoff2018-09-271-7/+12
| | | | | | | | | | | | | get_state_groups returns a map from state_group_id to a list of FrozenEvents, so was very much the wrong thing to be putting as one of the entries in the list passed to resolve_events_with_factory (which expects maps from (event_type, state_key) to event id). We actually want get_state_groups_ids().values() rather than get_state_groups(). This fixes the main problem in #3923, but there are other problems with this bit of code which get discovered once you do so.
* more commentsRichard van der Hoff2018-09-271-4/+3
|
* Clarifications in FederationHandlerRichard van der Hoff2018-09-271-2/+11
| | | | | | | * add some comments on things that look a bit bogus * rename this `state` variable to avoid confusion with the `state` used elsewhere in this function. (There was no actual conflict, but it was a confusing bit of spaghetti.)
* Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2Richard van der Hoff2018-09-271-0/+4
|\ | | | | Logging improvements
| * Logging improvementsRichard van der Hoff2018-09-261-0/+4
| | | | | | | | Some logging tweaks to help with debugging incoming federation transactions
* | Include eventid in log lines when processing incoming federation ↵Richard van der Hoff2018-09-271-26/+39
|/ | | | | | | | | | | transactions (#3959) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.
* Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pduAmber Brown2018-09-261-23/+46
|\ | | | | Comments and interface cleanup for on_receive_pdu
| * Comments and interface cleanup for on_receive_pduRichard van der Hoff2018-09-201-23/+46
| | | | | | | | | | | | | | | | Add some informative comments about what's going on here. Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of `get_missing` and use `sent_to_us_directly` consistently.
* | Only lazy load self-members on initial syncMatthew Hodgson2018-09-251-5/+8
| | | | | | Given we have disabled lazy loading for incr syncs in #3840, we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
* | Always LL ourselves if we're in a room to simplify clients (#3916)Matthew Hodgson2018-09-201-0/+4
|/ | | | Should fix https://github.com/vector-im/riot-web/issues/7209
* Improve the logging when handling a federation transaction (#3904)Richard van der Hoff2018-09-191-55/+109
| | | | | | | | | | Let's try to rationalise the logging that happens when we are processing an incoming transaction, to make it easier to figure out what is going wrong when they take ages. In particular: - make everything start with a [room_id event_id] prefix - make sure we log a warning when catching exceptions rather than just turning them into other, more cryptic, exceptions.
* Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeoutAmber Brown2018-09-191-1/+30
|\ | | | | Bump timeout on get_missing_events request
| * Bump timeout on get_missing_events requestRichard van der Hoff2018-09-181-1/+30
| |
* | Use directory server for room joins (#3899)Richard van der Hoff2018-09-181-0/+5
|/ | | | | | When we do a join, always try the server we used for the alias lookup first. Fixes #2418
* Merge pull request #3879 from matrix-org/matthew/fix-autojoinRichard van der Hoff2018-09-181-0/+1
|\ | | | | don't ratelimit autojoins
| * don't ratelimit autojoinsMatthew Hodgson2018-09-151-0/+1
| |
* | Reinstate missing null checkRichard van der Hoff2018-09-171-1/+1
| |
* | Return a 404 when deleting unknown room aliasRichard van der Hoff2018-09-171-3/+16
|/ | | | | | As per https://github.com/matrix-org/matrix-doc/issues/1675 Fixes https://github.com/matrix-org/synapse/issues/2782
* don't filter membership events based on history visibility (#3874)Matthew Hodgson2018-09-141-8/+1
| | | | | | | | don't filter membership events based on history visibility as we will already have filtered the messages in the timeline, and state events are always visible. and because @erikjohnston said so.
* show heroes if a room has a 'deleted' name/canonical_alias (#3851)Matthew Hodgson2018-09-121-2/+15
|
* argh pep8Matthew Hodgson2018-09-121-1/+1
|
* correctly log gappy sync metricsMatthew Hodgson2018-09-121-1/+1
|
* fix loglineMatthew Hodgson2018-09-121-1/+1
|
* Speed up lazy loading (#3827)Matthew Hodgson2018-09-121-30/+88
| | | | | * speed up room summaries by pulling their data from room_memberships rather than room state * disable LL for incr syncs, and log incr sync stats (#3840)
* Don't send empty tags list down syncErik Johnston2018-09-061-0/+5
|
* Send existing room tags down sync on joinErik Johnston2018-09-061-0/+8
| | | | | | | | When a user joined a room any existing tags were not sent down the sync stream. Ordinarily this isn't a problem because the user needs to be in the room to have set tags in it, however synapse will sometimes add tags for a user to a room, e.g. for server notices, which need to come down sync.
* Merge pull request #3795 from matrix-org/erikj/faster_sync_stateAmber Brown2018-09-071-8/+8
|\ | | | | User iter* during sync state calculations
| * User iter* during sync state calculationsErik Johnston2018-09-051-8/+8
| |
* | Port handlers/ to Python 3 (#3803)Amber Brown2018-09-076-19/+22
|/
* Merge pull request #3737 from matrix-org/erikj/remove_redundant_state_funcErik Johnston2018-09-031-1/+1
|\ | | | | Remove unnecessary resolve_events_with_state_map
| * Remove unnecessary resolve_events_with_state_mapErik Johnston2018-08-221-1/+1
| | | | | | | | | | We only ever used the synchronous resolve_events_with_state_map in one place, which is trivial to replace with the async version.
* | fix bug where preserved threepid user comes to sign up and server is mau blockedNeil Johnson2018-08-311-1/+2
| |
* | don't return non-LL-member state in incremental sync state blocks (#3760)Matthew Hodgson2018-08-281-1/+8
|/ | | don't return non-LL-member state in incremental sync state blocks
* Merge pull request #3659 from matrix-org/erikj/split_profilesErik Johnston2018-08-222-9/+26
|\ | | | | Allow profile updates to happen on workers
| * Rename WorkerProfileHandler to BaseProfileHandlerErik Johnston2018-08-221-3/+10
| |
| * Add assert to ensure handler is only run on masterErik Johnston2018-08-221-0/+2
| |
| * Remove redundant varsErik Johnston2018-08-221-3/+0
| |
| * Add some fixmesErik Johnston2018-08-171-0/+4
| |
| * Call UserDirectoryHandler methods directlyErik Johnston2018-08-171-21/+5
| | | | | | | | | | | | | | | | Turns out that the user directory handling is fairly racey as a bunch of stuff assumes that the processing happens on master, which it doesn't when there is a synapse.app.user_dir worker. So lets just call the function directly until we actually get round to fixing it, since it doesn't make the situation any worse.
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-1715-84/+409
| |\ | | | | | | | | | erikj/split_profiles
| * | Split ProfileHandler into master and workerErik Johnston2018-08-171-7/+14
| | |
| * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-096-55/+112
| |\ \ | | | | | | | | | | | | erikj/split_profiles
| * | | Allow profile changes to happen on workersErik Johnston2018-08-071-5/+21
| | | |
* | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-2016-109/+445
|\ \ \ \ | | | | | | | | | | | | | | | erikj/refactor_state_handler
| * \ \ \ Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updatesRichard van der Hoff2018-08-173-17/+11
| |\ \ \ \ | | | | | | | | | | | | Fix logcontexts for running pushers
| | * | | | Fix logcontexts for running pushersRichard van der Hoff2018-08-173-17/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First of all, avoid resetting the logcontext before running the pushers, to fix the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning. Instead, give them their own "background process" logcontexts.
| * | | | | Integrate presence from hotfixes (#3694)Amber Brown2018-08-183-8/+25
| | |_|_|/ | |/| | |
| * | | | block event creation and room creation on hitting resource limitsNeil Johnson2018-08-162-1/+9
| |/ / /
| * | | lazyload aware /messages (#3589)Matthew Hodgson2018-08-161-1/+34
| | | |
| * | | initial cut at a room summary API (#3574)Matthew Hodgson2018-08-161-10/+149
| | | |
| * | | Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+mausNeil Johnson2018-08-151-2/+2
| |\ \ \ | | | | | | | | | | Fix Mau off by one errors
| | * \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-154-31/+108
| | |\ \ \ | | | | | | | | | | | | | | | | | | neilj/fix_off_by_1+maus
| | * | | | support admin_email config and pass through into blocking errors, return ↵Neil Johnson2018-08-141-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | AuthError in all cases
| | * | | | fix off by 1s on mauNeil Johnson2018-08-141-2/+2
| | | | | |
| | * | | | Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-141-10/+0
| | |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | neilj/fix_off_by_1+maus
| | * \ \ \ \ Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-141-13/+14
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | neilj/fix_off_by_1+maus
| | * \ \ \ \ \ Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-141-2/+11
| | |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | neilj/fix_off_by_1+maus
| * | | | | | | | speed up /members and add at= and membership params (#3568)Matthew Hodgson2018-08-151-12/+76
| | |_|_|_|/ / / | |/| | | | | |
| * | | | | | | Merge pull request #3653 from matrix-org/erikj/split_federationErik Johnston2018-08-151-17/+57
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Move more federation APIs to workers
| | * \ \ \ \ \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-1514-27/+78
| | |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | erikj/split_federation
| | * | | | | | | | Use federation handler function rather than duplicateErik Johnston2018-08-151-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This involves renaming _persist_events to be a public function.
| | * | | | | | | | Move clean_room_for_join to masterErik Johnston2018-08-091-2/+14
| | | | | | | | | |
| | * | | | | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-096-55/+112
| | |\ \ \ \ \ \ \ \ | | | | |_|_|_|_|_|/ | | | |/| | | | | | | | | | | | | | | | erikj/split_federation
| | * | | | | | | | Add EDU/query handling over replicationErik Johnston2018-08-061-11/+13
| | | | | | | | | |
| | * | | | | | | | Add replication APIs for persisting federation eventsErik Johnston2018-08-061-9/+35
| | | |_|_|_|_|_|/ | | |/| | | | | |
| * | | | | | | | Merge pull request #3687 from matrix-org/neilj/admin_emailNeil Johnson2018-08-151-14/+4
| |\ \ \ \ \ \ \ \ | | |_|_|/ / / / / | |/| | | | | | | support admin_email config and pass through into blocking errors,
| | * | | | | | | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_emailNeil Johnson2018-08-141-2/+11
| | |\ \ \ \ \ \ \
| | * | | | | | | | remove blank lineNeil Johnson2018-08-141-1/+0
| | | |_|_|_|_|_|/ | | |/| | | | | |
| | * | | | | | | Update register.pyNeil Johnson2018-08-141-10/+0
| | | |_|_|_|_|/ | | |/| | | | | | | | | | | | | remove comments
| | * | | | | | support admin_email config and pass through into blocking errors, return ↵Neil Johnson2018-08-131-13/+14
| | | |_|_|/ / | | |/| | | | | | | | | | | | | | | | | | AuthError in all cases
| * | | | | | Merge pull request #3667 from matrix-org/erikj/fixup_unbindErik Johnston2018-08-153-14/+51
| |\ \ \ \ \ \ | | |_|/ / / / | |/| | | | | Don't fail requests to unbind 3pids for non supporting ID servers
| | * | | | | Log when we 3pid/unbind request failsErik Johnston2018-08-091-0/+2
| | | | | | |
| | * | | | | Don't fail requests to unbind 3pids for non supporting ID serversErik Johnston2018-08-083-14/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Older identity servers may not support the unbind 3pid request, so we shouldn't fail the requests if we received one of 400/404/501. The request still fails if we receive e.g. 500 responses, allowing clients to retry requests on transient identity server errors that otherwise do support the API. Fixes #3661
| * | | | | | Merge pull request #3670 from matrix-org/neilj/mau_sync_blockNeil Johnson2018-08-141-2/+11
| |\ \ \ \ \ \ | | |_|/ / / / | |/| | | | / | | | |_|_|/ | | |/| | | Block ability to read via sync if mau limit exceeded
| | * | | | only block on sync where user is not part of the mau cohortNeil Johnson2018-08-091-1/+6
| | | | | |
| | * | | | block sync if auth checks failNeil Johnson2018-08-091-7/+5
| | | | | |
| | * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-095-34/+69
| | |\ \ \ \ | | | | |_|/ | | | |/| | | | | | | | neilj/mau_sync_block
| | * | | | sync auth blockingNeil Johnson2018-08-081-5/+11
| | | | | |
| * | | | | Revert "support admin_email config and pass through into blocking errors, ↵Neil Johnson2018-08-131-14/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | return AuthError in all cases" This reverts commit 0d43f991a19840a224d3dac78d79f13d78212ee6.
| * | | | | support admin_email config and pass through into blocking errors, return ↵Neil Johnson2018-08-131-13/+14
| | | | | | | | | | | | | | | | | | | | | | | | AuthError in all cases
| * | | | | Run tests under PostgreSQL (#3423)Amber Brown2018-08-131-0/+5
| | | | | |
| * | | | | Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678)Amber Brown2018-08-1011-11/+11
| | |/ / / | |/| | |
* | | | | Remove redundant room_version checksErik Johnston2018-08-201-1/+1
| | | | |
* | | | | Revert spurious changeErik Johnston2018-08-201-2/+2
| | | | |
* | | | | Choose state algorithm based on room versionErik Johnston2018-08-092-4/+9
|/ / / /
* | | | Merge pull request #3632 from matrix-org/erikj/refactor_repl_servletErik Johnston2018-08-092-31/+21
|\ \ \ \ | | |_|/ | |/| | Add helper base class for generating new replication endpoints
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-08-035-141/+210
| |\ \ \ | | | | | | | | | | | | | | | erikj/refactor_repl_servlet
| * | | | Use new helper base class for membership requestsErik Johnston2018-07-311-24/+17
| | | | |
| * | | | Use new helper base class for ReplicationSendEventRestServletErik Johnston2018-07-311-7/+4
| | | | |
* | | | | Merge pull request #3664 from matrix-org/rav/federation_metricsRichard van der Hoff2018-08-081-0/+10
|\ \ \ \ \ | | | | | | | | | | | | more metrics for the federation and appservice senders
| * | | | | more metrics for the federation and appservice sendersRichard van der Hoff2018-08-071-0/+10
| | |_|_|/ | |/| | |
* | | | | Merge pull request #3654 from matrix-org/rav/room_versionsRichard van der Hoff2018-08-082-3/+38
|\ \ \ \ \ | |_|_|_|/ |/| | | | Support for room versioning
| * | | | Don't expose default_room_version as config optRichard van der Hoff2018-08-071-1/+2
| | | | |
| * | | | include known room versions in outgoing make_joinsRichard van der Hoff2018-08-061-2/+11
| | | | |
| * | | | Basic support for room versioningRichard van der Hoff2018-08-031-1/+26
| | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event
* | | | Merge pull request #3633 from matrix-org/neilj/mau_trackerNeil Johnson2018-08-082-21/+6
|\ \ \ \ | |_|/ / |/| | | API for monthly_active_users table
| * | | py3 fixNeil Johnson2018-08-031-1/+1
| | | |
| * | | bug fixesNeil Johnson2018-08-032-17/+6
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_trackerNeil Johnson2018-08-033-114/+140
| |\| |
| * | | do mau checks based on monthly_active_users tableNeil Johnson2018-08-022-12/+8
| | | |
* | | | Return M_NOT_FOUND when a profile could not be found. (#3596)Will Hunt2018-08-031-31/+58
| |/ / |/| |
* | | Merge pull request #3639 from matrix-org/rav/refactor_error_handlingRichard van der Hoff2018-08-021-17/+8
|\ \ \ | | | | | | | | Clean up handling of errors from outbound requests
| * | | Kill off MatrixCodeMessageExceptionRichard van der Hoff2018-08-011-17/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This code brings the SimpleHttpClient into line with the MatrixFederationHttpClient by having it raise HttpResponseExceptions when a request fails (rather than trying to parse for matrix errors and maybe raising MatrixCodeMessageException). Then, whenever we were checking for MatrixCodeMessageException and turning them into SynapseErrors, we now need to check for HttpResponseExceptions and call to_synapse_error.
* | | | Merge branch 'master' into developRichard van der Hoff2018-08-022-10/+44
|\ \ \ \
| * | | | Check room visibility for /event/ requestsRichard van der Hoff2018-08-021-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make sure that the user has permission to view the requeseted event for /event/{eventId} and /room/{roomId}/event/{eventId} requests. Also check that the event is in the given room for /room/{roomId}/event/{eventId}, for sanity.
| * | | | Avoid extra db lookupsRichard van der Hoff2018-08-021-27/+11
| | | | | | | | | | | | | | | | | | | | | | | | | Since we're about to look up the events themselves anyway, we can skip the extra db queries here.
| * | | | Validation for events/rooms in fed requestsRichard van der Hoff2018-08-021-1/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we get a federation request which refers to an event id, make sure that said event is in the room the caller claims it is in. (patch supplied by @turt2live)
* | | | | Merge pull request #3621 from matrix-org/erikj/split_fed_storeErik Johnston2018-08-021-87/+88
|\ \ \ \ \ | |_|_|/ / |/| | | | Split out DB writes in federation handler
| * | | | update docsErik Johnston2018-08-011-3/+5
| | | | |
| * | | | _persist_auth_tree no longer returns anythingErik Johnston2018-08-011-1/+1
| | | | |
| * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-306-30/+177
| |\ \ \ \ | | | |_|/ | | |/| | | | | | | erikj/split_fed_store
| * | | | Split out DB writes in federation handlerErik Johnston2018-07-251-83/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will allow us to easily add an internal replication API to proxy these reqeusts to master, so that we can move federation APIs to workers.
* | | | | Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limitsNeil Johnson2018-08-012-3/+35
|\ \ \ \ \ | |_|_|_|/ |/| | | | Initial impl of capping MAU
| * | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Neil Johnson2018-08-011-14/+14
| |\ \ \ \ | | | | | | | | | | | | | | | | | | neilj/mau_sign_in_log_in_limits
| * | | | | count_monthly_users() asyncNeil Johnson2018-08-011-4/+7
| | | | | |
| * | | | | make count_monthly_users async synapse/handlers/auth.pyNeil Johnson2018-08-011-4/+5
| | | | | |
| * | | | | coding styleNeil Johnson2018-07-311-1/+2
| | | | | |
| * | | | | limit register and sign in on number of monthly usersNeil Johnson2018-07-302-2/+29
| | |_|/ / | |/| | |
* | | | | Python 3: Convert some unicode/bytes uses (#3569)Amber Brown2018-08-022-10/+21
| |/ / / |/| | |
* | | | rename replication_layer to federation_clientRichard van der Hoff2018-07-311-14/+14
|/ / / | | | | | | | | | | | | I have HAD ENOUGH of trying to remember wtf a replication layer is in terms of classes.
* | | make /context lazyload & filter aware (#3567)Matthew Hodgson2018-07-272-4/+22
| | | | | | | | | make /context lazyload & filter aware.
* | | Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pidRichard van der Hoff2018-07-271-0/+4
|\ \ \ | | | | | | | | if inviter_display_name == ""||None then default to inviter MXID
| * \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Michael Telatynski2018-07-2431-1534/+3975
| |\ \ \ | | | | | | | | | | | | | | | t3chguy/default_inviter_display_name_3pid
| * | | | if inviter_display_name == ""||None then default to inviter MXIDMichael Telatynski2018-06-131-0/+4
| | | | | | | | | | | | | | | | | | | | to prevent email invite from "None"
* | | | | Deduplicate redundant lazy-loaded members (#3331)Matthew Hodgson2018-07-261-25/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * attempt at deduplicating lazy-loaded members as per the proposal; we can deduplicate redundant lazy-loaded members which are sent in the same sync sequence. we do this heuristically rather than requiring the client to somehow tell us which members it has chosen to cache, by instead caching the last N members sent to a client, and not sending them again. For now we hardcode N to 100. Each cache for a given (user,device) tuple is in turn cached for up to X minutes (to avoid the caches building up). For now we hardcode X to 30. * add include_redundant_members filter option & make it work * remove stale todo * add tests for _get_some_state_from_cache * incorporate review
* | | | | Fix some looping_call calls which were broken in #3604Richard van der Hoff2018-07-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.
* | | | | Merge pull request #2970 from matrix-org/matthew/filter_membersMatthew Hodgson2018-07-261-14/+102
|\ \ \ \ \ | | | | | | | | | | | | Implement the lazy_load_members room state filter parameter
| * \ \ \ \ Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-253-13/+21
| |\ \ \ \ \ | | | |_|/ / | | |/| | |
| * | | | | clarify comment on p_idsMatthew Hodgson2018-07-241-1/+2
| | | | | |
| * | | | | rather than adding ll_ids, remove them from p_idsMatthew Hodgson2018-07-241-5/+6
| | | | | |
| * | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-246-282/+307
| |\ \ \ \ \
| * | | | | | incorporate more reviewMatthew Hodgson2018-07-241-3/+9
| | | | | | |
| * | | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-07-234-43/+77
| |\ \ \ \ \ \
| * | | | | | | incorporate reviewMatthew Hodgson2018-07-231-40/+27
| | | | | | | |
| * | | | | | | incorporate reviewMatthew Hodgson2018-07-191-17/+27
| | | | | | | |
| * | | | | | | add a filtered_types param to limit filtering to specific typesMatthew Hodgson2018-07-191-29/+36
| | | | | | | |
| * | | | | | | merge develop pydoc for _get_state_for_groupsMatthew Hodgson2018-07-1930-416/+441
| |\ \ \ \ \ \ \
| * \ \ \ \ \ \ \ Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-06-101-0/+4
| |\ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ \ Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-06-0411-30/+45
| |\ \ \ \ \ \ \ \ \
| * | | | | | | | | | more commentsMatthew Hodgson2018-06-041-0/+7
| | | | | | | | | | |
| * | | | | | | | | | add pydocMatthew Hodgson2018-05-291-5/+13
| | | | | | | | | | |
| * | | | | | | | | | merge developMatthew Hodgson2018-05-2920-472/+970
| |\ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | make lazy_load_members configurable in filtersMatthew Hodgson2018-03-191-9/+9
| | | | | | | | | | | |
| * | | | | | | | | | | reshuffle todo & commentsMatthew Hodgson2018-03-191-2/+10
| | | | | | | | | | | |
| * | | | | | | | | | | only get member state IDs for incremental syncs if we're filteringMatthew Hodgson2018-03-181-5/+5
| | | | | | | | | | | |
| * | | | | | | | | | | remove debugMatthew Hodgson2018-03-161-4/+0
| | | | | | | | | | | |
| * | | | | | | | | | | fix tsm search againMatthew Hodgson2018-03-161-5/+4
| | | | | | | | | | | |
| * | | | | | | | | | | add state_ids for timeline entriesMatthew Hodgson2018-03-161-4/+14
| | | | | | | | | | | |
| * | | | | | | | | | | simplify timeline_start_membersMatthew Hodgson2018-03-161-14/+13
| | | | | | | | | | | |
| * | | | | | | | | | | make incr syncs workMatthew Hodgson2018-03-161-3/+4
| | | | | | | | | | | |
| * | | | | | | | | | | Merge branch 'develop' into matthew/filter_membersMatthew Hodgson2018-03-143-101/+293
| |\ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | add copyright to nudge CIMatthew Hodgson2018-03-141-1/+2
| | | | | | | | | | | | |
| * | | | | | | | | | | | oopsMatthew Hodgson2018-03-131-1/+1
| | | | | | | | | | | | |
| * | | | | | | | | | | | make it workMatthew Hodgson2018-03-131-3/+3
| | | | | | | | | | | | |
| * | | | | | | | | | | | merge proper fix to bug 2969Matthew Hodgson2018-03-1313-73/+223
| |\ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | ensure we always include the members for a given timeline blockMatthew Hodgson2018-03-131-4/+32
| | | | | | | | | | | | | |
| * | | | | | | | | | | | | typosMatthew Hodgson2018-03-111-2/+2
| | | | | | | | | | | | | |
| * | | | | | | | | | | | | WIP experiment in lazyloading room membersMatthew Hodgson2018-03-111-12/+31
| | | | | | | | | | | | | |
* | | | | | | | | | | | | | Fix a minor documentation typo in on_make_leaveTravis Ralston2018-07-251-1/+1
| |_|_|_|_|_|_|_|_|/ / / / |/| | | | | | | | | | | |
* | | | | | | | | | | | | Fix updating of cached remote profilesRichard van der Hoff2018-07-251-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | _update_remote_profile_cache was missing its `defer.inlineCallbacks`, so when it was called, would just return a generator object, without actually running any of the method body.
* | | | | | | | | | | | | Replace usage of get_current_toke with StreamToken.STARTErik Johnston2018-07-241-5/+7
| |_|_|_|_|_|_|_|_|_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to handle /context/ requests on the client_reader worker without having to pull in all the various stream handlers (e.g. precence, typing, pushers etc). The only thing the token gets used for is pagination, and that ignores everything but the room portion of the token.
* | | | | | | | | | | | Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groupsRichard van der Hoff2018-07-241-7/+6
|\ \ \ \ \ \ \ \ \ \ \ \ | |_|_|_|_|_|_|_|_|/ / / |/| | | | | | | | | | | Fixes and optimisations for resolve_state_groups
| * | | | | | | | | | | Handle delta_ids being None in _update_context_for_auth_eventsRichard van der Hoff2018-07-231-7/+6
| | |_|_|_|_|_|_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | it's easier to create the new state group as a delta from the existing one. (There's an outside chance this will help with https://github.com/matrix-org/synapse/issues/3364)
* | | | | | | | | | | Merge branch 'develop' into rav/logcontext_fixesRichard van der Hoff2018-07-244-264/+282
|\ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | Merge branch 'develop' into erikj/client_apis_moveErik Johnston2018-07-244-32/+66
| |\| | | | | | | | | |
| * | | | | | | | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-07-232-11/+11
| |\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | erikj/client_apis_move
| * | | | | | | | | | | | Move PaginationHandler to its own fileErik Johnston2018-07-202-240/+267
| | | | | | | | | | | | |
| * | | | | | | | | | | | Move check_in_room_or_world_readable to AuthErik Johnston2018-07-201-34/+6
| | | | | | | | | | | | |
| * | | | | | | | | | | | Move RoomContextHandler out of HandlersErik Johnston2018-07-182-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is in preparation for moving GET /context/ to a worker
| * | | | | | | | | | | | Split MessageHandler into read only and writersErik Johnston2018-07-182-133/+150
| | |_|_|_|_|_|_|_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will let us call the read only parts from workers, and so be able to move some APIs off of master, e.g. the `/state` API.
* | | | | | | | | | | | another couple of logcontext leaksRichard van der Hoff2018-07-242-5/+10
| | | | | | | | | | | |
* | | | | | | | | | | | Logcontext fixesRichard van der Hoff2018-07-241-13/+15
| |_|/ / / / / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix some random logcontext leaks.
* | | | | | | | | | | Fix missing attributes on workers.Erik Johnston2018-07-231-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was missed during the transition from attribute to getter for getting state from context.
* | | | | | | | | | | Add support for updating stateErik Johnston2018-07-231-9/+23
| | | | | | | | | | |
* | | | | | | | | | | Use new gettersErik Johnston2018-07-234-21/+40
| |/ / / / / / / / / |/| | | | | | | | |
* | | | | | | | | | Make the rest of the .iterwhatever go away (#3562)Amber Brown2018-07-211-9/+9
| | | | | | | | | |
* | | | | | | | | | Combine Limiter and LinearizerRichard van der Hoff2018-07-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Linearizer was effectively a Limiter with max_count=1, so rather than maintaining two sets of code, let's combine them.
* | | | | | | | | | Improvements to the LimiterRichard van der Hoff2018-07-201-1/+1
|/ / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * give them names, to improve logging * use a deque rather than a list for efficiency
* | | | | | | | | Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacksRichard van der Hoff2018-07-181-43/+0
|\ \ \ \ \ \ \ \ \ | |_|_|_|_|_|_|_|/ |/| | | | | | | | Remove event re-signing hacks
| * | | | | | | | Merge branch 'develop' into rav/drop_re_signing_hacksRichard van der Hoff2018-07-0412-111/+313
| |\ \ \ \ \ \ \ \
| * | | | | | | | | Remove event re-signing hacksRichard van der Hoff2018-06-071-43/+0
| | |_|_|_|_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These "temporary fixes" have been here three and a half years, and I can't find any events in the matrix.org database where the calculated signature differs from what's in the db. It's time for them to go away.
* | | | | | | | | Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_serverRichard van der Hoff2018-07-171-138/+6
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Refactor and optimze filter_events_for_server
| * | | | | | | | | Move filter_events_for_server out of FederationHandlerRichard van der Hoff2018-07-161-138/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for easier unit testing.
* | | | | | | | | | typoMatthew Hodgson2018-07-131-3/+3
|/ / / / / / / / /
* | | | | | | | | s/becuase/because/gMatthew Hodgson2018-07-102-2/+2
| | | | | | | | |
* | | | | | | | | run isortAmber Brown2018-07-0930-184/+186
| |/ / / / / / / |/| | | | | | |
* | | | | | | | Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checkingErik Johnston2018-06-291-28/+68
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Check the state of prev_events a bit more thoroughly when coming over federation
| * | | | | | | | try and clean upAmber Brown2018-06-271-3/+6
| | | | | | | | |
| * | | | | | | | pep8Amber Brown2018-06-271-6/+4
| | | | | | | | |
| * | | | | | | | cleanupsAmber Brown2018-06-271-13/+15
| | | | | | | | |
| * | | | | | | | handle federation not telling us about prev_eventsAmber Brown2018-06-271-25/+62
| | | | | | | | |
* | | | | | | | | Attempt to be more performant on PyPy (#3462)Amber Brown2018-06-284-9/+8
|/ / / / / / / /
* | | | | | | | Merge pull request #3448 from matrix-org/matthew/gdpr-deactivate-admin-apiMatthew Hodgson2018-06-261-0/+1
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | add GDPR erase param to deactivate API
| * | | | | | | | add GDPR erase param to deactivate APIMatthew Hodgson2018-06-261-0/+1
| | | | | | | | |
* | | | | | | | | Merge pull request #3442 from matrix-org/matthew/allow-unconsented-partsErik Johnston2018-06-251-2/+6
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | allow non-consented users to still part rooms (to let us autopart them)
| * | | | | | | | | allow non-consented users to still part rooms (to let us autopart them)Matthew Hodgson2018-06-251-2/+6
| |/ / / / / / / /
* | | | | | | | | Actually fix itErik Johnston2018-06-251-3/+3
| | | | | | | | |
* | | | | | | | | CommentErik Johnston2018-06-251-0/+8
| | | | | | | | |
* | | | | | | | | Fix bug with assuming wrong typeErik Johnston2018-06-251-2/+2
| | | | | | | | |
* | | | | | | | | Add fast path to _filter_events_for_serverErik Johnston2018-06-251-1/+34
|/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most rooms have a trivial history visibility like "shared" or "world_readable", especially large rooms, so lets not bother getting the full membership of those rooms in that case.
* | | | | | | | typosMatthew Hodgson2018-06-251-1/+1
| | | | | | | |
* | | | | | | | typoMatthew Hodgson2018-06-251-1/+1
| | | | | | | |
* | | | | | | | Merge pull request #3441 from matrix-org/erikj/redo_erasureErik Johnston2018-06-252-1/+19
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Fix user erasure and re-enable
| * | | | | | | | Revert "Revert "Merge pull request #3431 from ↵Erik Johnston2018-06-252-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | matrix-org/rav/erasure_visibility"" This reverts commit 1d009013b3c3e814177afc59f066e02a202b21cd.
* | | | | | | | | Remove all global reactor imports & pass it around explicitly (#3424)Amber Brown2018-06-253-6/+6
|/ / / / / / / /
* | | | | | | | Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"Richard van der Hoff2018-06-222-19/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit ce0d911156b355c5bf452120bfb08653dad96497, reversing changes made to b4a5d767a94f1680d07edfd583aae54ce422573e.
* | | | | | | | Also log number of events for serach contextErik Johnston2018-06-221-0/+5
| | | | | | | |
* | | | | | | | Add some logging to search queriesErik Johnston2018-06-221-0/+9
| | | | | | | |
* | | | | | | | Merge pull request #3431 from matrix-org/rav/erasure_visibilityErik Johnston2018-06-222-1/+19
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Support hiding events from deleted users
| * | | | | | | | Filter out erased messagesRichard van der Hoff2018-06-121-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Redact any messges sent by erased users.
| * | | | | | | | mark accounts as erased when requestedRichard van der Hoff2018-06-121-1/+6
| | | | | | | | |
* | | | | | | | | Merge pull request #3428 from matrix-org/erikj/persisted_pduErik Johnston2018-06-221-18/+26
|\| | | | | | | | | | | | | | | | | | | | | | | | | | Simplify get_persisted_pdu
| * | | | | | | | simplify get_persisted_pduRichard van der Hoff2018-06-121-18/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | it doesn't make much sense to use get_persisted_pdu on the receive path: just get the event straight from the store.
* | | | | | | | | Pass around the reactor explicitly (#3385)Amber Brown2018-06-223-15/+25
| | | | | | | | |
* | | | | | | | | Merge pull request #3399 from costacruise/masterRichard van der Hoff2018-06-201-2/+6
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Add error code to room creation error
| * | | | | | | | | Add error code to room creation errorMichael Wagner2018-06-141-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This error code is mentioned in the documentation at https://matrix.org/docs/api/client-server/#!/Room32creation/createRoom
| * | | | | | | | | Fix event filtering in get_missing_events handlerRichard van der Hoff2018-06-081-0/+4
| | |/ / / / / / / | |/| | | | | | |
* | | | | | | | | Remove run_on_reactor (#3395)Amber Brown2018-06-145-26/+5
| |/ / / / / / / |/| | | | | | |
* | | | | | | | Merge pull request #3276 from matrix-org/dbkr/unbindDavid Baker2018-06-113-2/+82
|\ \ \ \ \ \ \ \ | |_|_|_|_|_|/ / |/| | | | | | | Remove email addresses / phone numbers from ID servers when they're removed from synapse
| * | | | | | | doc fixesDavid Baker2018-06-051-1/+1
| | | | | | | |
| * | | | | | | Fix commentDavid Baker2018-06-041-1/+1
| | | | | | | |
| * | | | | | | Missing yieldDavid Baker2018-06-041-1/+1
| | | | | | | |
| * | | | | | | docstringDavid Baker2018-06-041-1/+9
| | | | | | | |
| * | | | | | | Merge remote-tracking branch 'origin/develop' into dbkr/unbindDavid Baker2018-05-245-16/+46
| |\ \ \ \ \ \ \
| * | | | | | | | pep8David Baker2018-05-243-7/+12
| | | | | | | | |
| * | | | | | | | commentDavid Baker2018-05-241-0/+4
| | | | | | | | |
| * | | | | | | | Unbind 3pids when they're deleted tooDavid Baker2018-05-241-0/+8
| | | | | | | | |
| * | | | | | | | Hit the 3pid unbind endpoint on deactivationDavid Baker2018-05-232-1/+56
| | | | | | | | |
* | | | | | | | | Fix event filtering in get_missing_events handlerRichard van der Hoff2018-06-081-0/+4
| |_|/ / / / / / |/| | | | | | |
* | | | | | | | Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-3110-29/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | they're not meant to be lazy (#3307)
* | | | | | | | Update some comments and docstrings in SyncHandlerRichard van der Hoff2018-05-291-1/+11
| | | | | | | |
* | | | | | | | Exempt AS-registered users from doing gdprRichard van der Hoff2018-05-291-0/+3
| |_|_|_|_|/ / |/| | | | | |
* | | | | | | Merge remote-tracking branch 'origin/develop' into 3218-official-promAmber Brown2018-05-2810-61/+105
|\ \ \ \ \ \ \
| * \ \ \ \ \ \ Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_roomRichard van der Hoff2018-05-251-10/+14
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Let users leave the server notice room after joining
| | * | | | | | | Let users leave the server notice room after joiningRichard van der Hoff2018-05-251-10/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | They still can't reject invites, but we let them leave it.
| * | | | | | | | Misc. py3 fixesAdrian Tschira2018-05-242-8/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * | | | | | | | Merge pull request #3244 from NotAFile/py3-six-4Amber Brown2018-05-246-27/+35
| |\ \ \ \ \ \ \ \ | | |_|_|/ / / / / | |/| | | | | | | replace some iteritems with six
| | * | | | | | | replace some iteritems with sixAdrian Tschira2018-05-196-27/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com>
| * | | | | | | | Remove users from user directory on deactivateDavid Baker2018-05-242-0/+11
| | | | | | | | |
| * | | | | | | | Merge pull request #3261 from matrix-org/erikj/pagination_fixesErik Johnston2018-05-241-9/+22
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Fix federation backfill bugs
| | * | | | | | | | Use iter* methodsErik Johnston2018-05-221-7/+7
| | | | | | | | | |
| | * | | | | | | | Fix that states is a dict of dictsErik Johnston2018-05-221-1/+4
| | | | | | | | | |
| | * | | | | | | | get_domains_from_state returns list of tuplesErik Johnston2018-05-221-2/+12
| | | | | | | | | |
| * | | | | | | | | Merge pull request #3267 from matrix-org/erikj/iter_filterErik Johnston2018-05-241-5/+5
| |\ \ \ \ \ \ \ \ \ | | |_|_|/ / / / / / | |/| | | | | | | | Use iter* methods for _filter_events_for_server
| | * | | | | | | | s/values/itervalues/Erik Johnston2018-05-231-1/+1
| | | | | | | | | |
| | * | | | | | | | Use iter* methods for _filter_events_for_serverErik Johnston2018-05-231-4/+4
| | | | | | | | | |
| * | | | | | | | | Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uriRichard van der Hoff2018-05-231-1/+4
| |\ \ \ \ \ \ \ \ \
| | * | | | | | | | | Support for putting %(consent_uri)s in messagesRichard van der Hoff2018-05-231-1/+4
| | |/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make it possible to put the URI in the error message and the server notice that get sent by the server
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-16 21:09:37 +0000