summary refs log tree commit diff
path: root/synapse/handlers/federation.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge commit '3ee17585c' into dinsicAndrew Morgan2020-12-311-63/+3
|\
| * Move third_party_rules check to event creation timeRichard van der Hoff2020-10-131-44/+2
| | | | | | | | | | Rather than waiting until we handle the event, call the ThirdPartyRules check when we fist create the event.
| * Remove redundant calls to third_party_rules in `on_send_{join,leave}`Richard van der Hoff2020-10-131-19/+1
| | | | | | | | | | There's not much point in calling these *after* we have decided to accept them into the DAG.
* | Merge commit '74976a8e4' into dinsicAndrew Morgan2020-12-311-3/+9
|\|
| * Fix message duplication if something goes wrong after persisting the event ↵Erik Johnston2020-10-131-3/+6
| | | | | | | | | | (#8476) Should fix #3365.
| * Remove stream ordering from Metadata dict (#8452)Richard van der Hoff2020-10-051-0/+3
| | | | | | | | | | | | | | | | There's no need for it to be in the dict as well as the events table. Instead, we store it in a separate attribute in the EventInternalMetadata object, and populate that on load. This means that we can rely on it being correctly populated for any event which has been persited to the database.
* | Merge commit 'cc40a59b4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-5/+8
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'cc40a59b4': 1.21.0 Add prometheus metrics to track federation delays (#8430) Make token serializing/deserializing async (#8427) Allow additional SSO properties to be passed to the client (#8413) changelog Add an improved "forward extremities" metric Rewrite BucketCollector Fix _exposition.py to stop stripping samples Drop support for ancient prometheus_client (#8426) Various clean ups to room stream tokens. (#8423) changelog Report state res metrics to Prometheus and log Move Measure calls into `resolve_events_with_store` Expose a `get_resource_usage` method in `Measure` Move `resolve_events_with_store` into StateResolutionHandler
| * Move `resolve_events_with_store` into StateResolutionHandlerRichard van der Hoff2020-09-291-5/+8
| |
* | Merge commit '8238b55e0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-4/+9
|\| | | | | | | | | | | | | | | * commit '8238b55e0': Update description of server_name config option (#8415) Discard an empty upload_name before persisting an uploaded file (#7905) Don't table scan events on worker startup (#8419) Mypy fixes for `synapse.handlers.federation` (#8422)
| * Mypy fixes for `synapse.handlers.federation` (#8422)Richard van der Hoff2020-09-291-4/+9
| | | | | | For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
* | Merge commit 'f43c66d23' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'f43c66d23': Add support for running Complement against the local checkout (#8317) Filter out appservices from mau count (#8404) Only assert valid next_link params when provided (#8417) Add metrics to track success/otherwise of replication requests (#8406) Fix handling of connection timeouts in outgoing http requests (#8400) Changelog Don't check whether a 3pid is allowed to register during password reset Add checks for postgres sequence consistency (#8402) Create a mechanism for marking tests "logcontext clean" (#8399) Add `ui_auth_sessions_ips` table to `synapse_port_db` ignore list (#8410) A pair of tiny cleanups in the federation request code. (#8401) typo
| * A pair of tiny cleanups in the federation request code. (#8401)Richard van der Hoff2020-09-281-1/+1
| |
* | Merge commit '31acc5c30' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-6/+10
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '31acc5c30': Escape the error description on the sso_error template. (#8405) Fix occasional "Re-starting finished log context" from keyring (#8398) Allow existing users to login via OpenID Connect. (#8345) Fix schema delta for servers that have not backfilled (#8396) Fix MultiWriteIdGenerator's handling of restarts. (#8374) s/URLs/variables in changelog s/accidentally/incorrectly in changelog Update changelog wording Add type annotations to SimpleHttpClient (#8372) Add new sequences to port DB script (#8387) Add EventStreamPosition type (#8388) Mark the shadow_banned column as boolean in synapse_port_db. (#8386)
| * Add EventStreamPosition type (#8388)Erik Johnston2020-09-241-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea is to remove some of the places we pass around `int`, where it can represent one of two things: 1. the position of an event in the stream; or 2. a token that partitions the stream, used as part of the stream tokens. The valid operations are then: 1. did a position happen before or after a token; 2. get all events that happened before or after a token; and 3. get all events between two tokens. (Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
* | Merge commit '4325be1a5' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-211-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '4325be1a5': Fix missing null character check on guest_access room state Fixed a bug with reactivating users with the admin API (#8362) Admin API for reported events (#8217) Fix wording of deprecation notice in changelog Deprecation warning for synapse admin api being accessible under /_matrix Create function to check for long names in devices (#8364) Add a comment re #1691 Fix a bad merge from release-v1.20.0. (#8354) Admin API for querying rooms where a user is a member (#8306) Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322) Simplify super() calls to Python 3 syntax. (#8344) Allow appservice users to /login (#8320) Update test logging to be able to accept braces (#8335) Move lint dependencies to extras_require (#8330)
| * Merge tag 'v1.20.0rc5' into developPatrick Cloke2020-09-181-8/+57
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.20.0rc5 (2020-09-18) ============================== In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3. Features -------- - Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343)) Bugfixes -------- - Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342)) - Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349)) Internal Changes ---------------- - Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285))
| * | Simplify super() calls to Python 3 syntax. (#8344)Patrick Cloke2020-09-181-1/+1
| | | | | | | | | | | | | | | | | | | | | This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
* | | Merge commit '43f2b67e4' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-8/+57
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | * commit '43f2b67e4': Intelligently select extremities used in backfill. (#8349) Add flags to /versions about whether new rooms are encrypted by default. (#8343) Fix ratelimiting for federation `/send` requests. (#8342) blacklist MSC2753 sytests until it's implemented in synapse (#8285)
| * | Intelligently select extremities used in backfill. (#8349)Erik Johnston2020-09-181-8/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of just using the most recent extremities let's pick the ones that will give us results that the pagination request cares about, i.e. pick extremities only if they have a smaller depth than the pagination token. This is useful when we fail to backfill an extremity, as we no longer get stuck requesting that same extremity repeatedly.
* | | Merge commit '837293c31' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-18/+31
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '837293c31': Remove obsolete __future__ imports (#8337) Use admin_patterns for all admin APIs. (#8331) Fix a potential bug of UnboundLocalError (#8329) Switch metaclass initialization to python 3-compatible syntax (#8326) Catch-up after Federation Outage (split, 4): catch-up loop (#8272) Use slots in attrs classes where possible (#8296) Fix typos in comments. Add the topic and avatar to the room details admin API (#8305) Improve SAML error messages (#8248) Add experimental support for sharding event persister. Again. (#8294) Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) Use TLSv1.2 for fake servers in tests (#8208) Add /_synapse/client to the reverse proxy docs (#8227) Clean up `Notifier.on_new_room_event` code path (#8288)
| * | Use slots in attrs classes where possible (#8296)Patrick Cloke2020-09-141-1/+1
| | | | | | | | | | | | | | | slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.
| * | Add experimental support for sharding event persister. Again. (#8294)Erik Johnston2020-09-141-14/+30
| | | | | | | | | | | | | | | | | | This is *not* ready for production yet. Caveats: 1. We should write some tests... 2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
| * | Clean up `Notifier.on_new_room_event` code path (#8288)Erik Johnston2020-09-101-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it. This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens. The valid operations here are: 1. Is a position before or after a token 2. Fetching all events between two tokens 3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C. Future PR will change the token type to a dedicated type.
* | | Merge commit 'a3a90ee03' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-43/+2
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'a3a90ee03': Show a confirmation page during user password reset (#8004) Do not error when thumbnailing invalid files (#8236) Remove some unused distributor signals (#8216) Fixup pusher pool notifications (#8287) Revert "Fixup pusher pool notifications" Fixup pusher pool notifications
| * | Remove some unused distributor signals (#8216)Patrick Cloke2020-09-091-42/+1
| | | | | | | | | | | | | | | Removes the `user_joined_room` and stops calling it since there are no observers. Also cleans-up some other unused signals and related code.
| * | Fixup pusher pool notifications (#8287)Erik Johnston2020-09-091-1/+1
| | | | | | | | | | | | | | | `pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed. I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
| * | Revert "Fixup pusher pool notifications"Erik Johnston2020-09-091-1/+1
| | | | | | | | | | | | This reverts commit e7fd336a53a4ca489cdafc389b494d5477019dc0.
| * | Fixup pusher pool notificationsErik Johnston2020-09-091-1/+1
| |/
* | Merge commit '9f8abdcc3' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-30/+14
|\| | | | | | | | | * commit '9f8abdcc3': Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
| * Revert "Add experimental support for sharding event persister. (#8170)" (#8242)Brendan Abolivier2020-09-041-30/+14
| | | | | | | | | | | | | | * Revert "Add experimental support for sharding event persister. (#8170)" This reverts commit 82c1ee1c22a87b9e6e3179947014b0f11c0a1ac3. * Changelog
* | Merge commit '208e1d3eb' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-5/+5
|\| | | | | | | | | | | | | | | | | | | * commit '208e1d3eb': Fix typing for `@cached` wrapped functions (#8240) Remove useless changelog about reverting a #8239. Revert pinning of setuptools (#8239) Fix typing for SyncHandler (#8237) wrap `_get_e2e_device_keys_and_signatures_txn` in a non-txn method (#8231) Add an overload for simple_select_one_onecol_txn. (#8235)
| * Fix typing for `@cached` wrapped functions (#8240)Erik Johnston2020-09-031-5/+5
| | | | | | This requires adding a mypy plugin to fiddle with the type signatures a bit.
* | Merge commit '0d4f614fd' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-14/+30
|\| | | | | | | | | | | | | | | | | * commit '0d4f614fd': Refactor `_get_e2e_device_keys_for_federation_query_txn` (#8225) Add experimental support for sharding event persister. (#8170) Add /user/{user_id}/shared_rooms/ api (#7785) Do not try to store invalid data in the stats table (#8226) Convert the main methods run by the reactor to async. (#8213)
| * Add experimental support for sharding event persister. (#8170)Erik Johnston2020-09-021-14/+30
| | | | | | | | | | | | This is *not* ready for production yet. Caveats: 1. We should write some tests... 2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
* | Merge commit '5c03134d0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-8/+16
|\| | | | | | | | | | | | | * commit '5c03134d0': Convert additional database code to async/await. (#8195) Define StateMap as immutable and add a MutableStateMap type. (#8183) Move and refactor LoginRestServlet helper methods (#8182)
| * Convert additional database code to async/await. (#8195)Patrick Cloke2020-08-281-2/+2
| |
| * Define StateMap as immutable and add a MutableStateMap type. (#8183)Patrick Cloke2020-08-281-6/+14
| |
* | Merge commit '56efa9ec7' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-201-4/+6
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '56efa9ec7': (22 commits) Fix rate limiting unit tests. (#8167) Add functions to `MultiWriterIdGen` used by events stream (#8164) Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) Changelog fixes Make StreamIdGen `get_next` and `get_next_mult` async (#8161) Wording fixes to 'name' user admin api filter (#8163) Fix missing double-backtick in RST document Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) Add type hints for state. (#8140) Stop shadow-banned users from sending non-member events. (#8142) Allow capping a room's retention policy (#8104) Add healthcheck for default localhost 8008 port on /health endpoint. (#8147) Fix flaky shadow-ban tests. (#8152) Don't fail /submit_token requests on incorrect session ID if request_token_inhibit_3pid_errors is turned on (#7991) Do not apply ratelimiting on joins to appservices (#8139) Micro-optimisations to get_auth_chain_ids (#8132) Allow denying or shadow banning registrations via the spam checker (#8034) Stop shadow-banned users from sending invites. (#8095) Be more tolerant of membership events in unknown rooms (#8110) Improve the error code when trying to register using a name reserved for guests. (#8135) ...
| * Add type hints for state. (#8140)Patrick Cloke2020-08-241-4/+6
| |
* | Merge commit 'c9c544cda' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-11/+5
|\| | | | | | | | | | | | | | | | | | | | | | | | | * commit 'c9c544cda': Remove `ChainedIdGenerator`. (#8123) Switch the JSON byte producer from a pull to a push producer. (#8116) Updated docs: Added note about missing 308 redirect support. (#8120) Be stricter about JSON that is accepted by Synapse (#8106) Convert runWithConnection to async. (#8121) Remove the unused inlineCallbacks code-paths in the caching code (#8119) Separate `get_current_token` into two. (#8113) Convert events worker database to async/await. (#8071) Add a link to the matrix-synapse-rest-password-provider. (#8111)
| * Convert events worker database to async/await. (#8071)Patrick Cloke2020-08-181-11/+5
| |
* | Merge commit 'd4a7829b1' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-191-1/+1
|\| | | | | | | | | * commit 'd4a7829b1': Convert synapse.api to async/await (#8031)
| * Convert synapse.api to async/await (#8031)Patrick Cloke2020-08-061-1/+1
| |
* | Merge commit 'a7bdf98d0' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-161-1/+1
|\| | | | | | | | | * commit 'a7bdf98d0': Rename database classes to make some sense (#8033)
| * Rename database classes to make some sense (#8033)Erik Johnston2020-08-051-1/+1
| |
* | Merge commit '3950ae51e' into anoa/dinsic_release_1_21_xAndrew Morgan2020-10-161-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '3950ae51e': Ensure that remove_pusher is always async (#7981) Ensure the msg property of HttpResponseException is a string. (#7979) Remove from the event_relations table when purging historical events. (#7978) Add additional logging for SAML sessions. (#7971) Add MSC reference to changelog for #7736 Re-implement unread counts (#7736) Various improvements to the docs (#7899) Convert storage layer to async/await. (#7963) Add an option to disable purge in delete room admin API (#7964) Move some log lines from default logger to sql/transaction loggers (#7952) Use the JSON module from the std library instead of simplejson. (#7936) Fix exit code for `check_line_terminators.sh` (#7970) Option to allow server admins to join complex rooms (#7902) Fix typo in metrics docs (#7966) Add script for finding files with unix line terminators (#7965) Convert the remaining media repo code to async / await. (#7947) Convert a synapse.events to async/await. (#7949) Convert groups and visibility code to async / await. (#7951) Convert push to async/await. (#7948)
| * Convert a synapse.events to async/await. (#7949)Patrick Cloke2020-07-271-1/+1
| |
* | Merge commit 'f88c48f3b' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-12/+10
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'f88c48f3b': 1.18.0rc1 Fix error reporting when using `opentracing.trace` (#7961) Fix typing replication not being handled on master (#7959) Remove hacky error handling for inlineDeferreds. (#7950) Convert tests/rest/admin/test_room.py to unix file endings (#7953) Support oEmbed for media previews. (#7920) Convert state resolution to async/await (#7942) Fix up types and comments that refer to Deferreds. (#7945) Do not convert async functions to Deferreds in the interactive_auth_handler (#7944) Convert more of the media code to async/await (#7873) Return an empty body for OPTIONS requests. (#7886) Downgrade warning on client disconnect to INFO (#7928) Convert presence handler helpers to async/await. (#7939) Update the auth providers to be async. (#7935) Put a cache on `/state_ids` (#7931)
| * Remove hacky error handling for inlineDeferreds. (#7950)Patrick Cloke2020-07-271-9/+5
| |
| * Fix up types and comments that refer to Deferreds. (#7945)Patrick Cloke2020-07-241-3/+5
| |
* | Merge commit 'de119063f' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-3/+14
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'de119063f': (31 commits) Convert room list handler to async/await. (#7912) Element CSS and logo in email templates (#7919) Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914) Remove unused code from synapse.logging.utils. (#7897) Fix a typo in the sample config. (#7890) Fix deprecation warning: import ABC from collections.abc (#7892) Change sample config's postgres user to synapse_user (#7889) Fix deprecation warning due to invalid escape sequences (#7895) Remove Ubuntu Eoan that is now EOL (#7888) Fix the trace function for async functions. (#7872) Add help for creating a user via docker (#7885) Switch to Debian:Slim from Alpine for the docker image (#7839) Stop using 'device_max_stream_id' (#7882) Fix TypeError in synapse.notifier (#7880) Add a default limit (of 100) to get/sync operations. (#7858) Change "unknown room ver" logging to warning. (#7881) Convert device handler to async/await (#7871) Convert synapse.app to async/await. (#7868) Convert _base, profile, and _receipts handlers to async/await (#7860) Add admin endpoint to get members in a room. (#7842) ...
| * Fix deprecation warning: import ABC from collections.abc (#7892)Karthikeyan Singaravelan2020-07-201-1/+1
| |
| * Reject attempts to join empty rooms over federation (#7859)Richard van der Hoff2020-07-161-2/+13
| | | | | | | | | | | | We shouldn't allow others to make_join through us if we've left the room; reject such attempts with a 404. Fixes #7835. Fixes #6958.
* | Merge commit 'f1245dc3c' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-8/+19
|\| | | | | | | | | * commit 'f1245dc3c': Fix resync remote devices on receive PDU in worker mode. (#7815)
| * Fix resync remote devices on receive PDU in worker mode. (#7815)Erik Johnston2020-07-101-8/+19
| | | | | | | | | | | | The replication client requires that arguments are given as keyword arguments, which was not done in this case. We also pull out the logic so that we can catch and handle any exceptions raised, rather than leaving them unhandled.
* | Merge commit 'e29c44340' into anoa/dinsic_release_1_18_xAndrew Morgan2020-08-031-12/+37
|\| | | | | | | | | * commit 'e29c44340': Fix recursion error when fetching auth chain over federation (#7817)
| * Fix recursion error when fetching auth chain over federation (#7817)Erik Johnston2020-07-101-12/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When fetching the state of a room over federation we receive the event IDs of the state and auth chain. We then fetch those events that we don't already have. However, we used a function that recursively fetched any missing auth events for the fetched events, which can lead to a lot of recursion if the server is missing most of the auth chain. This work is entirely pointless because would have queued up the missing events in the auth chain to be fetched already. Let's just diable the recursion, since it only gets called from one place anyway.
* | Merge commit '43726783e' into dinsicAndrew Morgan2020-08-031-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '43726783e': (22 commits) 1.17.0rc1 Fix some spelling mistakes / typos. (#7811) `update_membership` declaration: now always returns an event id. (#7809) Improve stacktraces from exceptions in background processes (#7808) Fix `can only concatenate list (not "tuple") to list` exception (#7810) Pass original request headers from workers to the main process. (#7797) Generate real events when we reject invites (#7804) Add `HomeServer.signing_key` property (#7805) Revert "Update the installation docs on apt-transport-https (#7801)" Do not use simplejson in Synapse. (#7800) Stop passing bytes when dumping JSON (#7799) Update the installation docs on apt-transport-https (#7801) shuffle changelog slightly Change Caddy links (old is deprecated) (#7789) Stop populating unused table `local_invites`. (#7793) Refactor getting replication updates from database v2. (#7740) Add libwebp dependency to Dockerfile (#7791) Add documentation for JWT login type and improve sample config. (#7776) Convert the appservice handler to async/await. (#7775) Don't ignore `set_tweak` actions with no explicit `value`. (#7766) ...
| * Add `HomeServer.signing_key` property (#7805)Richard van der Hoff2020-07-081-1/+1
| | | | | | ... instead of duplicating `config.signing_key[0]` everywhere
* | Merge commit '5cdca53aa' into dinsicAndrew Morgan2020-08-031-81/+85
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '5cdca53aa': Merge different Resource implementation classes (#7732) Fix inconsistent handling of upper and lower cases of email addresses. (#7021) Allow YAML config file to contain None (#7779) Fix a typo. Move 1.15.2 after 1.16.0rc2. 1.16.0rc2 Remove an extraneous space. Add links to the fixes. Fix tense in the release notes. Hack to add push priority to push notifications (#7765) Add early returns to `_check_for_soft_fail` (#7769) Use symbolic names for replication stream names (#7768) Type checking for `FederationHandler` (#7770) Fix new metric where we used ms instead of seconds (#7771) Fix incorrect error message when database CTYPE was set incorrectly. (#7760) Pin link in CHANGES.md Fixes to CHANGES.md
| * Merge branch 'master' into developPatrick Cloke2020-07-021-3/+3
| |\
| * | Add early returns to `_check_for_soft_fail` (#7769)Richard van der Hoff2020-07-011-64/+55
| | | | | | | | | | | | my editor was complaining about unset variables, so let's add some early returns to fix that and reduce indentation/cognitive load.
| * | Type checking for `FederationHandler` (#7770)Richard van der Hoff2020-07-011-17/+30
| | | | | | | | | fix a few things to make this pass mypy.
* | | Merge commit 'e8c36e527' into dinsicAndrew Morgan2020-08-031-3/+3
|\ \ \ | | |/ | |/| | | | | | | | | | | | | * commit 'e8c36e527': 1.15.2 Correctly handle outliers as prev events over federation Ensure that HTML pages served from Synapse include headers to avoid embedding.
| * | Correctly handle outliers as prev events over federationErik Johnston2020-07-021-3/+3
| | |
* | | Merge commit 'dc80a0762' into dinsicAndrew Morgan2020-08-031-0/+1
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'dc80a0762': 1.16.0rc1 Back out MSC2625 implementation (#7761) Additional configuration options for auto-join rooms (#7763) Add some metrics for inbound and outbound federation processing times (#7755) Explain the purpose of the "tests" conditional dependency requirement (#7751) Add another yield point to state res v2 (#7746) Move flake8 to end. Don't exit script on failure (#7738) Make tox actions work on Debian 10 (#7703) Yield during large v2 state res. (#7735) add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675) Set Content-Length for Metrics requests (#7730) Sync ignored table names in synapse_port_db to current database schema (#7717) Allow local media to be marked as safe from being quarantined. (#7718) Convert directory handler to async/await (#7727) Speed up state res v2 across large state differences. (#7725)
| * | Yield during large v2 state res. (#7735)Erik Johnston2020-06-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | State res v2 across large data sets can be very CPU intensive, and if all the relevant events are in the cache the algorithm will run from start to finish within a single reactor tick. This can result in blocking the reactor tick for several seconds, which can have major repercussions on other requests. To fix this we simply add the occaisonal `sleep(0)` during iterations to yield execution until the next reactor tick. The aim is to only do this for large data sets so that we don't impact otherwise quick resolutions.=
* | | Merge commit 'a3f11567d' into dinsicAndrew Morgan2020-08-031-5/+4
|\| | | | | | | | | | | | | | * commit 'a3f11567d': Replace all remaining six usage with native Python 3 equivalents (#7704)
| * | Replace all remaining six usage with native Python 3 equivalents (#7704)Dagfinn Ilmari Mannsåker2020-06-161-5/+4
| | |
* | | Merge commit '03619324f' into dinsicAndrew Morgan2020-08-031-14/+20
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '03619324f': Create a ListenerConfig object (#7681) Fix changelog wording 1.15.1 Wrap register_device coroutine in an ensureDeferred (#7684) Ensure the body is a string before comparing push rules. (#7701) Ensure etag is a string for GET room_keys/version response (#7691) Update m.id.phone to use 'phone' instead of 'number' (#7687) Fix "There was no active span when trying to log." error (#7698) Enable 3PID add/bind/unbind endpoints on r0 routes Discard RDATA from already seen positions. (#7648) Replace iteritems/itervalues/iterkeys with native versions. (#7692) Fix warnings about losing log context during UI auth. (#7688) Fix a typo when comparing the URI & method during UI Auth. (#7689) Remove "user_id" from GET /presence. (#7606) Increase the default SAML session expirary time to 15 minutes. (#7664) fix typo in sample_config.yaml (#7652) Take out a lock before modifying _CACHES (#7663) Add option to enable encryption by default for new rooms (#7639) Clean-up the fallback login code. (#7657)
| * | Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-12/+10
| | |
| * | Add option to enable encryption by default for new rooms (#7639)Andrew Morgan2020-06-101-2/+10
| |/ | | | | | | | | | | | | | | | | Fixes https://github.com/matrix-org/synapse/issues/2431 Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used. Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637 Based on #7637
* | Merge branch 'release-v1.15.0' of github.com:matrix-org/synapse into ↵Andrew Morgan2020-06-241-1/+1
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dinsic-release-v1.15.x * 'release-v1.15.0' of github.com:matrix-org/synapse: (55 commits) 1.15.0 Fix some attributions Update CHANGES.md 1.15.0rc1 Revert "1.15.0rc1" 1.15.0rc1 Fix bug in account data replication stream. (#7656) Convert the registration handler to async/await. (#7649) Accept device information at the login fallback endpoint. (#7629) Convert user directory handler and related classes to async/await. (#7640) Add an option to disable autojoin for guest accounts (#6637) Clarifications to the admin api documentation (#7647) Update to the stable SSO prefix for UI Auth. (#7630) Fix type information on `assert_*_is_admin` methods (#7645) Remove some unused constants. (#7644) Typo fixes. Allow new users to be registered via the admin API even if the monthly active user limit has been reached (#7263) Add device management to admin API (#7481) Attempt to fix PhoneHomeStatsTestCase.test_performance_100 being flaky. (#7634) Support CS API v0.6.0 (#6585) ...
| * Fix exceptions when fetching events from a down host. (#7622)Erik Johnston2020-06-031-1/+1
| | | | | | We already caught some exceptions, but not all.
* | Merge branch 'release-v1.14.0' of github.com:matrix-org/synapse into ↵Andrew Morgan2020-06-101-32/+52
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dinsic-release-v1.14.x * 'release-v1.14.0' of github.com:matrix-org/synapse: (108 commits) Fix typo in PR link Update debian changelog 1.14.0 Improve changelog wording 1.14.0rc2 Fix sample config docs error (#7581) Fix up comments Fix specifying cache factors via env vars with * in name. (#7580) Don't apply cache factor to event cache. (#7578) Ensure ReplicationStreamer is always started when replication enabled. (#7579) Remove the changes to the debian changelog Not full release yet, this is rc1 Merge event persistence move changelog entries More changelog fix Changelog fixes 1.14.0 Replace device_27_unique_idx bg update with a fg one (#7562) Fix incorrect exception handling in KeyUploadServlet.on_POST (#7563) Fix recording of federation stream token (#7564) Simplify reap_monthly_active_users (#7558) ...
| * Add option to move event persistence off master (#7517)Erik Johnston2020-05-221-6/+10
| |
| * Add ability to wait for replication streams (#7542)Erik Johnston2020-05-221-10/+23
| | | | | | | | | | | | | | The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room). Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on. People probably want to look at this commit by commit.
| * Fix exception reporting due to HTTP request errors. (#7556)Erik Johnston2020-05-221-0/+7
| | | | | | | | These are business as usual errors, rather than stuff we want to log at error.
| * Convert federation handler to async/await. (#7459)Patrick Cloke2020-05-111-18/+14
| |
* | Merge branch 'release-v1.13.0' of github.com:matrix-org/synapse into ↵Andrew Morgan2020-06-101-59/+36
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dinsic-release-v1.14.x * 'release-v1.13.0' of github.com:matrix-org/synapse: (257 commits) Update changelog based on feedback. Move warnings in the changelog and re-iterate changes to branches. 1.13.0 update dh-virtualenv (#7526) 1.13.0rc3 Hash passwords earlier in the registration process (#7523) 1.13.0rc2 1.13.0rc2 Stop `get_joined_users` corruption from custom statuses (#7376) Do not validate that the client dict is stable during UI Auth. (#7483) Fix new flake8 errors (#7489) Don't UPGRADE database rows RST indenting Put rollback instructions in upgrade notes Fix changelog typo Oh yeah, RST Absolute URL it is then Fix upgrade notes link Provide summary of upgrade issues in changelog. Fix ) Move next version notes from changelog to upgrade notes ...
| * async/await is_server_admin (#7363)Andrew Morgan2020-05-011-11/+10
| |
| * Convert some of the federation handler methods to async/await. (#7338)Patrick Cloke2020-04-241-25/+24
| |
| * Rewrite prune_old_outbound_device_pokes for efficiency (#7159)Richard van der Hoff2020-03-301-23/+2
| | | | | | | | make sure we clear out all but one update for the user
* | apply lintingAndrew Morgan2020-05-141-2/+1
| |
* | Store room version on invite (#6983)Andrew Morgan2020-03-241-0/+12
|\| | | | | | | | | * commit '3e99528f2': Store room version on invite (#6983)
| * Store room version on invite (#6983)Richard van der Hoff2020-02-261-0/+12
| | | | | | | | | | When we get an invite over federation, store the room version in the rooms table. The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
* | Remove redundant store_room call (#6979)Andrew Morgan2020-03-241-23/+0
|\| | | | | | | | | * commit '691659568': Remove redundant store_room call (#6979)
| * Remove redundant store_room call (#6979)Richard van der Hoff2020-02-241-23/+0
| | | | | | | | | | `_process_received_pdu` is only called by `on_receive_pdu`, which ignores any events for unknown rooms, so this is redundant.
* | Upsert room version when we join over federation (#6968)Andrew Morgan2020-03-241-10/+12
|\| | | | | | | | | * commit 'a301934f4': Upsert room version when we join over federation (#6968)
| * Upsert room version when we join over federation (#6968)Richard van der Hoff2020-02-241-10/+12
| | | | | | | | | | | | | | | | This is intended as a precursor to storing room versions when we receive an invite over federation, but has the happy side-effect of fixing #3374 at last. In short: change the store_room with try/except to a proper upsert which updates the right columns.
* | Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Andrew Morgan2020-03-241-9/+9
|\| | | | | | | | | | | | | * commit '509e381af': Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) changelog Increase MAX_EVENTS_BEHIND for replication clients
| * Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)Patrick Cloke2020-02-211-9/+9
| | | | | | | | Ensure good comprehension hygiene using flake8-comprehensions.
* | Filter the results of user directory searching via the spam checker (#6888)Andrew Morgan2020-03-231-1/+1
|\| | | | | | | | | * commit '49f877d32': Filter the results of user directory searching via the spam checker (#6888)
* | Limit the number of events that can be requested when backfilling events (#6864)Andrew Morgan2020-03-231-0/+4
|\| | | | | | | | | * commit '7765bf398': Limit the number of events that can be requested when backfilling events (#6864)
| * Limit the number of events that can be requested when backfilling events (#6864)Patrick Cloke2020-02-061-0/+4
| | | | | | Limit the maximum number of events requested when backfilling events.
* | pass room version into FederationClient.send_join (#6854)Andrew Morgan2020-03-231-2/+1
|\| | | | | | | | | * commit 'b0c8bdd49': pass room version into FederationClient.send_join (#6854)
| * pass room version into FederationClient.send_join (#6854)Richard van der Hoff2020-02-061-2/+1
| | | | | | | | ... which allows us to sanity-check the create event.
* | Merge pull request #6823 from matrix-org/rav/redact_changes/5Andrew Morgan2020-03-231-6/+2
|\| | | | | | | | | | | | | | | | | | | * commit '9bcd37146': Update changelog Update changelog Update changelog Newsfile Simplify `room_version` handling in `FederationClient.send_invite` Pass room version object into `FederationClient.get_pdu`
| * Merge pull request #6823 from matrix-org/rav/redact_changes/5Richard van der Hoff2020-02-061-6/+2
| |\ | | | | | | pass room versions around
| | * Pass room version object into `FederationClient.get_pdu`Richard van der Hoff2020-02-051-6/+2
| | |
| * | Merge tag 'v1.10.0rc2' into developErik Johnston2020-02-061-14/+60
| |\ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse 1.10.0rc2 (2020-02-06) ============================== Bugfixes -------- - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844)) - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848)) Internal Changes ---------------- - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
* | | Check sender_key matches on inbound encrypted events. (#6850)Andrew Morgan2020-03-231-13/+59
|\ \ \ | | |/ | |/| | | | | | | * commit 'a58860e48': Check sender_key matches on inbound encrypted events. (#6850)
| * | Check sender_key matches on inbound encrypted events. (#6850)Erik Johnston2020-02-051-13/+59
| | | | | | | | | If they don't then the device lists are probably out of sync.
* | | Fix detecting unknown devices from remote encrypted events. (#6848)Andrew Morgan2020-03-231-1/+1
|\| | | | | | | | | | | | | | * commit '6475382d8': Fix detecting unknown devices from remote encrypted events. (#6848)
| * | Fix detecting unknown devices from remote encrypted events. (#6848)Erik Johnston2020-02-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | We were looking at the wrong event type (`m.room.encryption` vs `m.room.encrypted`). Also fixup the duplicate `EvenTypes` entries. Introduced in #6776.
* | | Merge pull request #6837 from matrix-org/rav/federation_asyncAndrew Morgan2020-03-231-228/+201
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'c7d6d5c69': (27 commits) make FederationHandler.send_invite async make FederationHandler.on_get_missing_events async changelog make FederationHandler.user_joined_room async make FederationHandler._clean_room_for_join async make FederationHandler._notify_persisted_event async make FederationHandler.persist_events_and_notify async make FederationHandler._make_and_verify_event async make FederationHandler.do_remotely_reject_invite async make FederationHandler._check_for_soft_fail async make FederationHandler._persist_auth_tree async make FederationHandler.do_invite_join async make FederationHandler.on_event_auth async make FederationHandler.on_exchange_third_party_invite_request async make FederationHandler.construct_auth_difference async make FederationHandler._update_context_for_auth_events async make FederationHandler._update_auth_events_and_context_for_auth async make FederationHandler.do_auth async make FederationHandler._prep_event async make FederationHandler._handle_new_event async ...
| * | make FederationHandler.send_invite asyncRichard van der Hoff2020-02-031-3/+2
| | |
| * | make FederationHandler.on_get_missing_events asyncRichard van der Hoff2020-02-031-5/+4
| | |
| * | make FederationHandler.user_joined_room asyncRichard van der Hoff2020-02-031-3/+3
| | |
| * | make FederationHandler._clean_room_for_join asyncRichard van der Hoff2020-02-031-4/+4
| | |
| * | make FederationHandler._notify_persisted_event asyncRichard van der Hoff2020-02-031-4/+6
| | |
| * | make FederationHandler.persist_events_and_notify asyncRichard van der Hoff2020-02-031-10/+10
| | |
| * | make FederationHandler._make_and_verify_event asyncRichard van der Hoff2020-02-031-5/+10
| | |
| * | make FederationHandler.do_remotely_reject_invite asyncRichard van der Hoff2020-02-031-6/+7
| | |
| * | make FederationHandler._check_for_soft_fail asyncRichard van der Hoff2020-02-031-13/+9
| | |
| * | make FederationHandler._persist_auth_tree asyncRichard van der Hoff2020-02-031-11/+7
| | |
| * | make FederationHandler.do_invite_join asyncRichard van der Hoff2020-02-031-16/+14
| | |
| * | make FederationHandler.on_event_auth asyncRichard van der Hoff2020-02-031-5/+4
| | |
| * | make FederationHandler.on_exchange_third_party_invite_request asyncRichard van der Hoff2020-02-031-14/+12
| | |
| * | make FederationHandler.construct_auth_difference asyncRichard van der Hoff2020-02-031-3/+4
| | |
| * | make FederationHandler._update_context_for_auth_events asyncRichard van der Hoff2020-02-031-10/+10
| | |
| * | make FederationHandler._update_auth_events_and_context_for_auth asyncRichard van der Hoff2020-02-031-20/+21
| | |
| * | make FederationHandler.do_auth asyncRichard van der Hoff2020-02-031-10/+14
| | |
| * | make FederationHandler._prep_event asyncRichard van der Hoff2020-02-031-23/+10
| | |
| * | make FederationHandler._handle_new_event asyncRichard van der Hoff2020-02-031-6/+7
| | |
| * | make FederationHandler._handle_new_events asyncRichard van der Hoff2020-02-031-8/+6
| | |
| * | make FederationHandler.on_make_leave_request asyncRichard van der Hoff2020-02-031-13/+10
| | |
| * | make FederationHandler.on_send_leave_request asyncRichard van der Hoff2020-02-031-5/+3
| | |
| * | make FederationHandler.on_make_join_request asyncRichard van der Hoff2020-02-031-13/+10
| | |
| * | make FederationHandler.on_invite_request asyncRichard van der Hoff2020-02-031-5/+4
| | |
| * | make FederationHandler.on_send_join_request asyncRichard van der Hoff2020-02-031-9/+7
| | |
| * | make FederationHandler.on_query_auth asyncRichard van der Hoff2020-02-031-7/+6
| |/
* | pass room_version into compute_event_signature (#6807)Andrew Morgan2020-03-231-1/+4
|\| | | | | | | | | * commit '7f93eb190': pass room_version into compute_event_signature (#6807)
| * pass room_version into compute_event_signature (#6807)Richard van der Hoff2020-01-311-1/+4
| |
* | Merge pull request #6820 from matrix-org/rav/get_room_version_idAndrew Morgan2020-03-231-9/+9
|\| | | | | | | | | | | | | * commit '160522e32': newsfile Add `get_room_version` method s/get_room_version/get_room_version_id/
| * Merge pull request #6820 from matrix-org/rav/get_room_version_idRichard van der Hoff2020-01-311-9/+9
| |\ | | | | | | Make `get_room_version` return a RoomVersion object
| | * s/get_room_version/get_room_version_id/Richard van der Hoff2020-01-311-9/+9
| | | | | | | | | | | | | | | ... to make way for a forthcoming get_room_version which returns a RoomVersion object.
* | | Fix bug with getting missing auth event during join 500'ed (#6810)Andrew Morgan2020-03-231-1/+5
|\| | | | | | | | | | | | | | * commit '7d846e870': Fix bug with getting missing auth event during join 500'ed (#6810)
| * | Fix bug with getting missing auth event during join 500'ed (#6810)Erik Johnston2020-01-311-1/+5
| |/
* | pass room version into FederationHandler.on_invite_request (#6805)Andrew Morgan2020-03-231-3/+3
|\| | | | | | | | | * commit '46a446828': pass room version into FederationHandler.on_invite_request (#6805)
| * pass room version into FederationHandler.on_invite_request (#6805)Richard van der Hoff2020-01-301-3/+3
| |
* | Resync remote device list when detected as stale. (#6786)Andrew Morgan2020-03-231-2/+16
|\| | | | | | | | | * commit 'b66032705': Resync remote device list when detected as stale. (#6786)
| * Resync remote device list when detected as stale. (#6786)Erik Johnston2020-01-301-2/+16
| |
* | Detect unknown remote devices and mark cache as stale (#6776)Andrew Morgan2020-03-231-0/+20
|\| | | | | | | | | * commit 'e17a11066': Detect unknown remote devices and mark cache as stale (#6776)
| * Detect unknown remote devices and mark cache as stale (#6776)Erik Johnston2020-01-281-0/+20
| | | | | | | | We just mark the fact that the cache may be stale in the database for now.
* | Pass room version object into event_auth.check and check_redaction (#6788)Andrew Morgan2020-03-231-7/+11
|\| | | | | | | | | * commit 'a8ce7aeb4': Pass room version object into event_auth.check and check_redaction (#6788)
| * Pass room version object into event_auth.check and check_redaction (#6788)Richard van der Hoff2020-01-281-7/+11
| | | | | | | | | | | | | | These are easier to work with than the strings and we normally have one around. This fixes `FederationHander._persist_auth_tree` which was passing a RoomVersion object into event_auth.check instead of a string.
* | Add `rooms.room_version` column (#6729)Andrew Morgan2020-03-231-15/+50
|\| | | | | | | | | * commit '8df862e45': Add `rooms.room_version` column (#6729)
| * Add `rooms.room_version` column (#6729)Erik Johnston2020-01-271-15/+50
| | | | | | This is so that we don't have to rely on pulling it out from `current_state_events` table.
* | Add StateMap type alias (#6715)Andrew Morgan2020-03-231-6/+4
|\| | | | | | | | | * commit 'd386f2f33': Add StateMap type alias (#6715)
| * Add StateMap type alias (#6715)Erik Johnston2020-01-161-6/+4
| |
* | Merge pull request #6652 from matrix-org/babolivier/depth_missing_eventsAndrew Morgan2020-03-201-2/+2
|\| | | | | | | | | | | * commit '0ab5853ec': Changelog Fix conditions failing if min_depth = 0
| * Fix conditions failing if min_depth = 0Brendan Abolivier2020-01-071-2/+2
| | | | | | | | This could result in Synapse not fetching prev_events for new events in the room if it has missed some events.
| * Merge branch 'master' into developRichard van der Hoff2019-12-201-1/+4
| |\
* | | Fix small extra argument passed to get_eventsAndrew Morgan2020-03-201-2/+9
| | |
* | | Change EventContext to use the Storage class (#6564)Andrew Morgan2020-03-201-7/+7
|\| | | | | | | | | | | | | | * commit 'fa780e972': Change EventContext to use the Storage class (#6564)
| * | Change EventContext to use the Storage class (#6564)Erik Johnston2019-12-201-7/+7
| | |
* | | Fix exceptions when attempting to backfill (#6576)Andrew Morgan2020-03-201-1/+4
|\ \ \ | | |/ | |/| | | | | | | * commit '03d3792f3': Fix exceptions when attempting to backfill (#6576)
| * | Fix exceptions when attempting to backfill (#6576)Richard van der Hoff2019-12-201-1/+4
| | | | | | | | | Fixes #6575
* | | Add database config class (#6513)Andrew Morgan2020-03-201-35/+33
|\ \ \ | | |/ | |/| | | | | | | | | | * commit '2284eb3a5': Add database config class (#6513) too many parens
| * | Merge release-v1.7.1 into developRichard van der Hoff2019-12-181-0/+1
| |\|
* | | Exclude rejected state events when calculating state at backwards extrems ↵Andrew Morgan2020-03-191-1/+1
|\ \ \ | | |/ | |/| | | | | | | | | | | | | (#6527) * commit 'bbb75ff6e': Exclude rejected state events when calculating state at backwards extrems (#6527)
| * | Exclude rejected state events when calculating state at backwards extrems ↵Richard van der Hoff2019-12-161-1/+1
| | | | | | | | | | | | | | | (#6527) This fixes a weird bug where, if you were determined enough, you could end up with a rejected event forming part of the state at a backwards-extremity. Authing that backwards extrem would then lead to us trying to pull the rejected event from the db (with allow_rejected=False), which would fail with a 404.
* | | Persist auth/state events at backwards extremities when we fetch them (#6526)Andrew Morgan2020-03-191-39/+14
|\| | | | | | | | | | | | | | * commit 'ff773ff72': Persist auth/state events at backwards extremities when we fetch them (#6526)
| * | Persist auth/state events at backwards extremities when we fetch them (#6526)Richard van der Hoff2019-12-161-163/+80
| | | | | | | | | | | | The main point here is to make sure that the state returned by _get_state_in_room has been authed before we try to use it as state in the room.
| * | sanity-checking for events used in state res (#6531)Richard van der Hoff2019-12-161-0/+1
| | | | | | | | | | | | | | | When we perform state resolution, check that all of the events involved are in the right room.
* | | Check the room_id of events when fetching room state/auth (#6524)Andrew Morgan2020-03-191-24/+54
|\| | | | | | | | | | | | | | * commit '35bbe4ca7': Check the room_id of events when fetching room state/auth (#6524)
| * | Check the room_id of events when fetching room state/auth (#6524)Richard van der Hoff2019-12-161-24/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we request the state/auth_events to populate a backwards extremity (on backfill or in the case of missing events in a transaction push), we should check that the returned events are in the right room rather than blindly using them in the room state or auth chain. Given that _get_events_from_store_or_dest takes a room_id, it seems clear that it should be sanity-checking the room_id of the requested events, so let's do it there.
* | | Add `include_event_in_state` to _get_state_for_room (#6521)Andrew Morgan2020-03-191-18/+21
|\| | | | | | | | | | | | | | * commit '20d5ba16e': Add `include_event_in_state` to _get_state_for_room (#6521)
| * | Add `include_event_in_state` to _get_state_for_room (#6521)Richard van der Hoff2019-12-161-18/+21
| | | | | | | | | | | | | | | | | | Make it return the state *after* the requested event, rather than the one before it. This is a bit easier and requires fewer calls to get_events_from_store_or_dest.
* | | Move get_state methods into FederationHandler (#6503)Andrew Morgan2020-03-191-101/+93
|\| | | | | | | | | | | | | | * commit 'be294d6fd': Move get_state methods into FederationHandler (#6503)
| * | Move get_state methods into FederationHandler (#6503)Richard van der Hoff2019-12-161-6/+95
| | | | | | | | | | | | | | | This is a non-functional refactor as a precursor to some other work.
* | | Exclude rejected state events when calculating state at backwards extrems ↵Andrew Morgan2020-03-191-1/+1
|\ \ \ | | |/ | |/| | | | | | | | | | | | | (#6527) * commit '6920d8889': Exclude rejected state events when calculating state at backwards extrems (#6527)
| * | Exclude rejected state events when calculating state at backwards extrems ↵Richard van der Hoff2019-12-161-1/+1
| | | | | | | | | | | | | | | (#6527) This fixes a weird bug where, if you were determined enough, you could end up with a rejected event forming part of the state at a backwards-extremity. Authing that backwards extrem would then lead to us trying to pull the rejected event from the db (with allow_rejected=False), which would fail with a 404.
* | | Persist auth/state events at backwards extremities when we fetch them (#6526)Andrew Morgan2020-03-191-167/+80
|\| | | | | | | | | | | | | | * commit 'bc7de8765': Persist auth/state events at backwards extremities when we fetch them (#6526)
| * | Persist auth/state events at backwards extremities when we fetch them (#6526)Richard van der Hoff2019-12-161-167/+80
| | | | | | | | | The main point here is to make sure that the state returned by _get_state_in_room has been authed before we try to use it as state in the room.
* | | sanity-checking for events used in state res (#6531)Andrew Morgan2020-03-191-0/+1
|\| | | | | | | | | | | | | | * commit '1da15f05f': sanity-checking for events used in state res (#6531)
| * | sanity-checking for events used in state res (#6531)Richard van der Hoff2019-12-131-0/+1
| | | | | | | | | | | | | | | | | | | | | When we perform state resolution, check that all of the events involved are in the right room.
* | | Check the room_id of events when fetching room state/auth (#6524)Andrew Morgan2020-03-191-23/+51
|\| | | | | | | | | | | | | | * commit '25f124432': Check the room_id of events when fetching room state/auth (#6524)
| * | Check the room_id of events when fetching room state/auth (#6524)Richard van der Hoff2019-12-121-23/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we request the state/auth_events to populate a backwards extremity (on backfill or in the case of missing events in a transaction push), we should check that the returned events are in the right room rather than blindly using them in the room state or auth chain. Given that _get_events_from_store_or_dest takes a room_id, it seems clear that it should be sanity-checking the room_id of the requested events, so let's do it there.
* | | Add `include_event_in_state` to _get_state_for_room (#6521)Andrew Morgan2020-03-191-22/+28
|\| | | | | | | | | | | | | | * commit '204535651': Add `include_event_in_state` to _get_state_for_room (#6521)
| * | Add `include_event_in_state` to _get_state_for_room (#6521)Richard van der Hoff2019-12-111-22/+28
| | | | | | | | | | | | | | | Make it return the state *after* the requested event, rather than the one before it. This is a bit easier and requires fewer calls to get_events_from_store_or_dest.
* | | Merge pull request #6517 from matrix-org/rav/event_auth/13Andrew Morgan2020-03-191-84/+75
|\| | | | | | | | | | | | | | | | | | | | | | | | | | * commit '894d2adda': changelog convert to async: FederationHandler._process_received_pdu convert to async: FederationHandler._get_state_for_room convert to async: FederationHandler.on_receive_pdu Convert federation backfill to async
| * | convert to async: FederationHandler._process_received_pduRichard van der Hoff2019-12-111-11/+10
| | | | | | | | | | | | also fix user_joined_room to consistently return deferreds
| * | convert to async: FederationHandler._get_state_for_roomRichard van der Hoff2019-12-111-21/+21
| | | | | | | | | | | | ... and _get_events_from_store_or_dest
| * | convert to async: FederationHandler.on_receive_pduRichard van der Hoff2019-12-111-27/+22
| | | | | | | | | | | | | | | | | | | | | and associated functions: * on_receive_pdu * handle_queued_pdus * get_missing_events_for_pdu
| * | Convert federation backfill to asyncRichard van der Hoff2019-12-111-25/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | PaginationHandler.get_messages is only called by RoomMessageListRestServlet, which is async. Chase the code path down from there: - FederationHandler.maybe_backfill (and nested try_backfill) - FederationHandler.backfill
* | | Clean up some logging (#6515)Andrew Morgan2020-03-191-18/+19
|\| | | | | | | | | | | | | | * commit '7c429f92d': Clean up some logging (#6515)
| * | Clean up some logging (#6515)Richard van der Hoff2019-12-111-18/+19
| | | | | | | | | | | | This just makes some of the logging easier to follow when things start going wrong.
* | | Prevent redacted events from appearing in message search (#6377)Andrew Morgan2020-03-191-2/+5
|\| | | | | | | | | | | | | | * commit 'fc316a489': Prevent redacted events from appearing in message search (#6377)
| * | Prevent redacted events from appearing in message search (#6377)Andrew Morgan2019-12-111-2/+5
| | |
* | | Prevent message search in upgraded rooms we're not in (#6385)Andrew Morgan2020-03-191-2/+2
|\| | | | | | | | | | | | | | * commit 'ea0f0ad41': Prevent message search in upgraded rooms we're not in (#6385)
| * | Prevent message search in upgraded rooms we're not in (#6385)Andrew Morgan2019-12-111-2/+2
| | |
* | | Move get_state methods into FederationHandler (#6503)Andrew Morgan2020-03-191-6/+95
|\| | | | | | | | | | | | | | * commit 'f8bc2ae88': Move get_state methods into FederationHandler (#6503)
| * | Move get_state methods into FederationHandler (#6503)Richard van der Hoff2019-12-101-6/+95
| |/ | | | | | | | | This is a non-functional refactor as a precursor to some other work.
* | Stronger typing in the federation handler (#6480)Andrew Morgan2020-03-191-24/+57
|\| | | | | | | | | * commit '63d6ad106': Stronger typing in the federation handler (#6480)
| * Stronger typing in the federation handler (#6480)Richard van der Hoff2019-12-051-24/+57
| | | | | | | | | | replace the event_info dict with an attrs thing
* | Sanity-check the rooms of auth events before pulling them in. (#6472)Andrew Morgan2020-03-191-9/+25
|\| | | | | | | | | * commit 'e1f4c83f4': Sanity-check the rooms of auth events before pulling them in. (#6472)
| * Sanity-check the rooms of auth events before pulling them in. (#6472)Richard van der Hoff2019-12-051-9/+25
| |
* | get rid of (most of) have_events from ↵Andrew Morgan2020-03-191-38/+24
|\| | | | | | | | | | | | | _update_auth_events_and_context_for_auth (#6468) * commit 'e203874ca': get rid of (most of) have_events from _update_auth_events_and_context_for_auth (#6468)
| * get rid of (most of) have_events from ↵Richard van der Hoff2019-12-041-38/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | _update_auth_events_and_context_for_auth (#6468) have_events was a map from event_id to rejection reason (or None) for events which are in our local database. It was used as filter on the list of event_ids being passed into get_events_as_list. However, since get_events_as_list will ignore any event_ids that are unknown or rejected, we can equivalently just leave it to get_events_as_list to do the filtering. That means that we don't have to keep `have_events` up-to-date, and can use `have_seen_events` instead of `get_seen_events_with_rejection` in the one place we do need it.
* | Add ephemeral messages support (MSC2228) (#6409)Andrew Morgan2020-03-191-0/+8
|\| | | | | | | | | * commit '54dd5dc12': Add ephemeral messages support (MSC2228) (#6409)
| * Add ephemeral messages support (MSC2228) (#6409)Brendan Abolivier2019-12-031-0/+8
| | | | | | | | | | | | | | | | Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are: * the feature is hidden behind a configuration flag (`enable_ephemeral_messages`) * self-destruction doesn't happen for state events * only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one) * doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
* | Fix: Pillow error when uploading RGBA image (#3325) (#6241)Andrew Morgan2020-03-191-2/+2
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '81731c6e7': Fix: Pillow error when uploading RGBA image (#3325) (#6241) Add User-Interactive Auth to /account/3pid/add (#6119) Lint Changelog Discard retention policies when retrieving state blacklist more tests Newsfile Add tests Propagate reason in remotely rejected invites MSC2367 Allow reason field on all member events
| * Propagate reason in remotely rejected invitesErik Johnston2019-11-281-2/+2
| |
| * Merge pull request #6358 from matrix-org/babolivier/message_retentionBrendan Abolivier2019-11-271-2/+2
| |\ | | | | | | Implement message retention policies (MSC1763)
| | * Merge branch 'develop' into babolivier/message_retentionBrendan Abolivier2019-11-261-3/+8
| | |\
| | * | Fix 3PID invite exchangeBrendan Abolivier2019-11-191-1/+1
| | | |
| | * | Implement per-room message retention policiesBrendan Abolivier2019-11-041-1/+1
| | | |
* | | | Merge pull request #6343 from matrix-org/rav/event_auth/4Andrew Morgan2020-03-181-37/+44
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '651d930f1': remove confusing fixme newsfile Use get_events_as_list rather than lots of calls to get_event Update some docstrings and comments Simplify _update_auth_events_and_context_for_auth
| * | | remove confusing fixmeRichard van der Hoff2019-11-261-6/+0
| | | |
| * | | Merge remote-tracking branch 'origin/develop' into rav/event_auth/4Richard van der Hoff2019-11-181-2/+2
| |\ \ \ | | | |/ | | |/|
| * | | Use get_events_as_list rather than lots of calls to get_eventRichard van der Hoff2019-11-081-16/+8
| | | | | | | | | | | | | | | | It's more efficient and clearer.
| * | | Update some docstrings and commentsRichard van der Hoff2019-11-081-8/+31
| | | |
| * | | Simplify _update_auth_events_and_context_for_authRichard van der Hoff2019-11-081-11/+9
| | | | | | | | | | | | | | | | | | | | move event_key calculation into _update_context_for_auth_events, since it's only used there.
* | | | Replace instance variations of homeserver with correct case/spacing (#6357)Andrew Morgan2020-03-181-2/+2
|\ \ \ \ | | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | * commit '6e1b40dc2': sample config Add changelog A couple more instances Replace instance variations of homeserver with correct case/spacing
| * | | Replace instance variations of homeserver with correct case/spacingAndrew Morgan2019-11-121-2/+2
| |/ /
* | | Fix bug which caused rejected events to be stored with the wrong room state ↵Andrew Morgan2020-03-161-0/+1
|\| | | | | | | | | | | | | | | | | | | | (#6320) * commit '807ec3bd9': Fix bug which caused rejected events to be stored with the wrong room state (#6320)
| * | Fix bug which caused rejected events to be stored with the wrong room state ↵Richard van der Hoff2019-11-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (#6320) Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.
* | | Add some checks that we aren't using state from rejected events (#6330)Andrew Morgan2020-03-161-1/+5
|\| | | | | | | | | | | | | | * commit '0e3ab8afd': Add some checks that we aren't using state from rejected events (#6330)
| * | Add some checks that we aren't using state from rejected events (#6330)Richard van der Hoff2019-11-051-1/+5
| |/ | | | | | | | | | | | | | | | | | | | | | | | | * Raise an exception if accessing state for rejected events Add some sanity checks on accessing state_group etc for rejected events. * Skip calculating push actions for rejected events It didn't actually cause any bugs, because rejected events get filtered out at various later points, but there's not point in trying to calculate the push actions for a rejected event.
* | Merge pull request #6301 from matrix-org/babolivier/msc2326Andrew Morgan2020-03-161-19/+19
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'f496d2587': Incorporate review Factor out an _AsyncEventContextImpl (#6298) Update synapse/storage/data_stores/main/schema/delta/56/event_labels.sql Add more data to the event_labels table and fix the indexes Add unstable feature flag Lint Incorporate review Lint Changelog Add integration tests for /messages Add more integration testing Add integration tests for sync Add unit tests Add index on label Implement filtering Store labels for new events Add database table for keeping track of labels on events
| * Factor out an _AsyncEventContextImpl (#6298)Richard van der Hoff2019-11-011-19/+19
| | | | | | | | | | | | The intention here is to make it clearer which fields we can expect to be populated when: notably, that the _event_type etc aren't used for the synchronous impl of EventContext.
* | Depublish a room from the public rooms list when it is upgraded (#6232)Andrew Morgan2020-03-161-1/+29
|\| | | | | | | | | * commit 'ace947e8d': Depublish a room from the public rooms list when it is upgraded (#6232)
| * Depublish a room from the public rooms list when it is upgraded (#6232)Andrew Morgan2019-11-011-1/+29
| |
* | Update black to 19.10b0 (#6304)Andrew Morgan2020-03-161-4/+5
|\| | | | | | | | | * commit '020add509': Update black to 19.10b0 (#6304)
| * Update black to 19.10b0 (#6304)Amber Brown2019-11-011-4/+5
| | | | | | * update version of black and also fix the mypy config being overridden
* | Merge pull request #6294 from matrix-org/erikj/add_state_storageAndrew Morgan2020-03-161-9/+10
|\|
| * Merge pull request #6294 from matrix-org/erikj/add_state_storageErik Johnston2019-10-311-9/+10
| |\ | | | | | | Add StateGroupStorage interface
| | * Port to use state storageErik Johnston2019-10-301-9/+10
| | |
* | | Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2020-03-161-16/+20
|\| |
| * | Remove usage of deprecated logger.warn method from codebase (#6271)Andrew Morgan2019-10-311-16/+20
| |/ | | | | Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
* | Merge pull request #6240 from matrix-org/erikj/split_out_persistence_storeAndrew Morgan2020-03-161-1/+2
|\|
| * Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2019-10-301-3/+19
| |\ | | | | | | | | | erikj/split_out_persistence_store
| * | Use new EventPersistenceStoreErik Johnston2019-10-231-1/+2
| | |
* | | Fix log line that was printing undefined value (#6278)Andrew Morgan2020-03-161-1/+1
|\ \ \ | | |/ | |/| | | | * commit 'a2276d4d3':
| * | Fix log line that was printing undefined value (#6278)Andrew Morgan2019-10-301-1/+1
| | |
* | | Improve signature checking on some federation APIs (#6262)Andrew Morgan2020-03-161-2/+18
|\| | | | | | | | | | | | | | * commit '172f264ed': Improve signature checking on some federation APIs (#6262)
| * | Improve signature checking on some federation APIs (#6262)Richard van der Hoff2019-10-281-2/+18
| |/ | | | | | | | | Make sure that we check that events sent over /send_join, /send_leave, and /invite, are correctly signed and come from the expected servers.
* | Remove Auth.check method (#6217)Andrew Morgan2020-03-161-3/+4
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '93eaeec75': Remove Auth.check method (#6217) add note about database upgrade use the right function for when we're already in runInteraction add user signature stream change cache to slaved device store black use stream ID generator instead of timestamp make isort happy apply PR review suggestions fix formatting add changelog make changes from PR review don't need to return the hidden column any more really fix queries to work with Postgres PostgreSQL, Y U no like? remove extra SQL query param add changelog file apply changes from PR review allow uploading keys for cross-signing allow devices to be marked as "hidden"
| * Remove Auth.check method (#6217)Richard van der Hoff2019-10-181-3/+4
| | | | | | This method was somewhat redundant, and confusing.
* | Merge pull request #6214 from matrix-org/rav/event_auth/1Andrew Morgan2020-03-161-102/+0
|\| | | | | | | | | | | | | * commit '47ada4dff': changelog rip out some unreachable code Remove dead check_auth script
| * rip out some unreachable codeRichard van der Hoff2019-10-171-102/+0
| | | | | | | | The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
| * TypoBrendan Abolivier2019-10-041-1/+1
| |
| * LintBrendan Abolivier2019-10-041-3/+1
| |
| * Incorporate reviewBrendan Abolivier2019-10-041-14/+8
| |
| * Add test caseBrendan Abolivier2019-10-031-1/+1
| |
| * LintBrendan Abolivier2019-10-021-1/+3
| |
| * Don't 500 code when trying to exchange a revoked 3PID inviteBrendan Abolivier2019-10-021-2/+13
| | | | | | | | | | | | | | | | While this is not documented in the spec (but should be), Riot (and other clients) revoke 3PID invites by sending a m.room.third_party_invite event with an empty ({}) content to the room's state. When the invited 3PID gets associated with a MXID, the identity server (which doesn't know about revocations) sends down to the MXID's homeserver all of the undelivered invites it has for this 3PID. The homeserver then tries to talk to the inviting homeserver in order to exchange these invite for m.room.member events. When one of the invite is revoked, the inviting homeserver responds with a 500 error because it tries to extract a 'display_name' property from the content, which is empty. This might cause the invited server to consider that the server is down and not try to exchange other, valid invites (or at least delay it). This fix handles the case of revoked invites by avoiding trying to fetch a 'display_name' from the original invite's content, and letting the m.room.member event fail the auth rules (because, since the original invite's content is empty, it doesn't have public keys), which results in sending a 403 with the correct error message to the invited server.
* | Remove origin parameter from add_display_name_to_third_party_invite and add ↵Andrew Morgan2020-02-251-1/+6
|\| | | | | | | params to docstring (#6010)
generated by cgit-magenta 1.5.1 (git 2.48.1) at 2025-08-09 14:47:48 +0000