summary refs log tree commit diff
path: root/synapse/handlers/appservice.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Some refactors around receipts stream (#16426)Erik Johnston2023-10-041-2/+2
|
* Return immutable objects for cachedList decorators (#16350)Patrick Cloke2023-09-191-4/+5
|
* Add support for claiming multiple OTKs at once. (#15468)Patrick Cloke2023-04-271-6/+8
| | | | | | | MSC3983 provides a way to request multiple OTKs at once from appservices, this extends this concept to the Client-Server API. Note that this will likely be spit out into a separate MSC, but is currently part of MSC3983.
* Add unstable /keys/claim endpoint which always returns fallback keys. (#15462)Patrick Cloke2023-04-251-8/+5
| | | | | | | | | | | | | It can be useful to always return the fallback key when attempting to claim keys. This adds an unstable endpoint for `/keys/claim` which always returns fallback keys in addition to one-time-keys. The fallback key(s) are not marked as "used" unless there are no corresponding OTKs. This is currently defined in MSC3983 (although likely to be split out to a separate MSC). The endpoint shape may change or be requested differently (i.e. a keyword parameter on the current endpoint), but the core logic should be reasonable.
* Implement MSC3984 to proxy /keys/query requests to appservices. (#15321)Patrick Cloke2023-03-301-0/+61
| | | | | If enabled, for users which are exclusively owned by an application service then the appservice will be queried for devices in addition to any information stored in the Synapse database.
* Implement MSC3983 to proxy /keys/claim queries to appservices. (#15314)Patrick Cloke2023-03-281-1/+73
| | | | | | Experimental support for MSC3983 is behind a configuration flag. If enabled, for users which are exclusively owned by an application service then the appservice will be queried for one-time keys *if* there are none uploaded to Synapse.
* Bump black from 22.12.0 to 23.1.0 (#15103)dependabot[bot]2023-02-221-1/+1
|
* Improve logging and opentracing for to-device message handling (#14598)Richard van der Hoff2022-12-061-3/+0
| | | | | | | A batch of changes intended to make it easier to trace to-device messages through the system. The intention here is that a client can set a property org.matrix.msgid in any to-device message it sends. That ID is then included in any tracing or logging related to the message. (Suggestions as to where this field should be documented welcome. I'm not enthusiastic about speccing it - it's very much an optional extra to help with debugging.) I've also generally improved the data we send to opentracing for these messages.
* Better return type for `get_all_entities_changed` (#14604)Erik Johnston2022-12-051-2/+2
| | | | Help callers from using the return value incorrectly by ensuring that callers explicitly check if there was a cache hit or not.
* Fix a bug where redactions were not being sent over federation if we did not ↵Shay2022-10-111-3/+6
| | | | have the original event. (#13813)
* Federation Sender & Appservice Pusher Stream Optimisations (#13251)Nick Mills-Barrett2022-07-151-5/+6
| | | | | | | | | | | | | * Replace `get_new_events_for_appservice` with `get_all_new_events_stream` The functions were near identical and this brings the AS worker closer to the way federation senders work which can allow for multiple workers to handle AS traffic. * Pull received TS alongside events when processing the stream This avoids an extra query -per event- when both federation sender and appservice pusher process events.
* Additional constants for EDU types. (#12884)Patrick Cloke2022-05-271-2/+2
| | | Instead of hard-coding strings in many places.
* Add `StreamKeyType` class and replace string literals with constants (#12567)Andrew Morgan2022-05-161-19/+20
|
* Add the `notify_appservices_from_worker` configuration option (superseding ↵reivilibre2022-05-061-1/+1
| | | | `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452)
* Bound ephemeral events by key (#12544)Nick Mills-Barrett2022-04-261-2/+2
| | | | Co-authored-by: Brad Murray <bradtgmurray@gmail.com> Co-authored-by: Andrew Morgan <andrewm@element.io>
* Refactor and convert `Linearizer` to async (#12357)Sean Quah2022-04-051-4/+2
| | | | | | | | | | | Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah <seanq@element.io>
* Send device list updates to application services (MSC3202) - part 1 (#11881)Andrew Morgan2022-03-301-6/+142
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Rename various ApplicationServices interested methods (#11915)Andrew Morgan2022-03-031-2/+2
|
* Remove `HomeServer.get_datastore()` (#12031)Richard van der Hoff2022-02-231-1/+1
| | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733
* Remove optional state of `ApplicationService.is_interested`'s `store` ↵Andrew Morgan2022-02-071-1/+1
| | | | parameter (#11911)
* Send to-device messages to application services (#11215)Andrew Morgan2022-02-011-17/+119
| | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Convert all namedtuples to attrs. (#11665)Patrick Cloke2021-12-301-2/+2
| | | To improve type hints throughout the code.
* Clarifications and small fixes to to-device related code (#11247)Andrew Morgan2021-11-091-4/+20
| | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Add a linearizer on (appservice, stream) when handling ephemeral events. ↵Nick Barrett2021-11-031-18/+51
| | | | | (#11207) Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
* Fix providing a `RoomStreamToken` instance to ↵Andrew Morgan2021-11-021-5/+17
| | | | | `_notify_app_services_ephemeral` (#11137) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Improve docstrings for methods related to sending EDUs to application ↵Andrew Morgan2021-10-211-12/+82
| | | | services (#11138)
* Use direct references for some configuration variables (part 3) (#10885)Patrick Cloke2021-09-231-1/+1
| | | | | | | | This avoids the overhead of searching through the various configuration classes by directly referencing the class that the attributes are in. It also improves type hints since mypy can now resolve the types of the configuration variables.
* Add type hints for event streams. (#10856)Patrick Cloke2021-09-211-3/+3
|
* Require type hints in the handlers module. (#10831)Patrick Cloke2021-09-201-9/+9
| | | | | | | Adds missing type hints to methods in the synapse.handlers module and requires all methods to have type hints there. This also removes the unused construct_auth_difference method from the FederationHandler.
* Add types to synapse.util. (#10601)reivilibre2021-09-101-0/+3
|
* Only return an appservice protocol if it has a service providing it. (#10532)Will Hunt2021-08-051-4/+3
| | | | | | If there are no services providing a protocol, omit it completely instead of returning an empty dictionary. This fixes a long-standing spec compliance bug.
* Use inline type hints in `handlers/` and `rest/`. (#10382)Jonathan de Jong2021-07-161-3/+3
|
* Always update AS last_pos, even on no events (#10107)14mRh4X0r2021-06-071-13/+12
| | | | | | | | | | Fixes #1834. `get_new_events_for_appservice` internally calls `get_events_as_list`, which will filter out any rejected events. If all returned events are filtered out, `_notify_interested_services` will return without updating the last handled stream position. If there are 100 consecutive such events, processing will halt altogether. Breaking the loop is now done by checking whether we're up-to-date with `current_max` in the loop condition, instead of relying on an empty `events` list. Signed-off-by: Willem Mulder <14mRh4X0r@gmail.com>
* Remove `synapse.types.Collection` (#9856)Richard van der Hoff2021-04-221-2/+2
| | | This is no longer required, since we have dropped support for Python 3.5.
* Remove redundant "coding: utf-8" lines (#9786)Jonathan de Jong2021-04-141-1/+0
| | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
* Bugbear: Add Mutable Parameter fixes (#9682)Jonathan de Jong2021-04-081-2/+2
| | | | | | | Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
* Import HomeServer from the proper module. (#9665)Patrick Cloke2021-03-231-1/+1
|
* Update black, and run auto formatting over the codebase (#9381)Eric Eastwood2021-02-161-1/+3
| | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version
* Improve appservice handler to send only the most recent read receipts when ↵Will Hunt2020-11-181-1/+1
| | | | | | | | | | | | | | | | | | | no stream_id is stored. (#8744) * Make this line debug (it's noisy) * Don't include from_key for presence if we are at 0 * Limit read receipts for all rooms to 100 * changelog.d/8744.bugfix * Allow from_key to be None * Update 8744.bugfix * The from_key is superflous * Update comment
* Add type hints to application services. (#8655)Patrick Cloke2020-10-281-34/+41
|
* Merge branch 'release-v1.22.0' into developErik Johnston2020-10-261-3/+4
|\
| * Fix get|set_type_stream_id_for_appservice store functions (#8648)Will Hunt2020-10-261-6/+6
| |
| * Properly handle presence events for application services. (#8656)Patrick Cloke2020-10-261-4/+5
| |
* | Fix get|set_type_stream_id_for_appservice store functions (#8648)Will Hunt2020-10-261-6/+6
| |
* | Start fewer opentracing spans (#8640)Erik Johnston2020-10-261-7/+43
|/ | | | | | | #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.
* Send some ephemeral events to appservices (#8437)Will Hunt2020-10-151-3/+106
| | | Optionally sends typing, presence, and read receipt information to appservices.
* Fix not sending events over federation when using sharded event persisters ↵Erik Johnston2020-10-141-4/+7
| | | | | | | | | | | | | | | | | (#8536) * Fix outbound federaion with multiple event persisters. We incorrectly notified federation senders that the minimum persisted stream position had advanced when we got an `RDATA` from an event persister. Notifying of federation senders already correctly happens in the notifier, so we just delete the offending line. * Change some interfaces to use RoomStreamToken. By enforcing use of `RoomStreamTokens` we make it less likely that people pass in random ints that they got from somewhere random.
* Stop sub-classing object (#8249)Patrick Cloke2020-09-041-1/+1
|
* Convert run_as_background_process inner function to async. (#8032)Patrick Cloke2020-08-061-1/+1
|
* Convert appservice to async. (#7973)Patrick Cloke2020-07-301-5/+5
|
* Convert the appservice handler to async/await. (#7775)Patrick Cloke2020-07-061-42/+32
|
* Fix new metric where we used ms instead of seconds (#7771)Erik Johnston2020-07-011-1/+1
| | | | Introduced in #7755, not yet released.
* Add some metrics for inbound and outbound federation processing times (#7755)Erik Johnston2020-06-301-0/+6
|
* Replace iteritems/itervalues/iterkeys with native versions. (#7692)Patrick Cloke2020-06-151-3/+1
|
* Update black to 19.10b0 (#6304)Amber Brown2019-11-011-1/+4
| | | * update version of black and also fix the mypy config being overridden
* Remove double return statements (#5962)Andrew Morgan2019-09-031-2/+0
| | | | | | | | | | Remove all the "double return" statements which were a result of us removing all the instances of ``` defer.returnValue(...) return ``` statements when we switched to python3 fully.
* Replace returnValue with return (#5736)Amber Brown2019-07-231-11/+11
|
* Move logging utilities out of the side drawer of util/ and into logging/ (#5606)Amber Brown2019-07-041-1/+1
|
* Run Black. (#5482)Amber Brown2019-06-201-38/+36
|
* Remove unnecessary extra function call layerErik Johnston2018-10-081-15/+3
|
* Use errback pattern and catch async failuresErik Johnston2018-10-081-1/+6
|
* more metrics for the federation and appservice sendersRichard van der Hoff2018-08-071-0/+10
|
* another couple of logcontext leaksRichard van der Hoff2018-07-241-1/+4
|
* run isortAmber Brown2018-07-091-7/+6
|
* Consistently use six's iteritems and wrap lazy keys/values in list() if ↵Amber Brown2018-05-311-1/+3
| | | | they're not meant to be lazy (#3307)
* cleanupAmber Brown2018-05-221-3/+6
|
* replacing portionsAmber Brown2018-05-211-13/+6
|
* Merge pull request #3143 from matrix-org/rav/remove_redundant_preserve_fnRichard van der Hoff2018-04-301-4/+2
|\ | | | | Remove redundant call to preserve_fn
| * Remove redundant call to preserve_fnRichard van der Hoff2018-04-301-4/+2
| | | | | | | | submit_event_for_as doesn't return a deferred anyway, so this is pointless.
* | Merge pull request #3150 from NotAFile/py3-listcomp-yieldRichard van der Hoff2018-04-301-5/+9
|\ \ | |/ |/| Don't yield in list comprehensions
| * Don't yield in list comprehensionsAdrian Tschira2018-04-281-5/+9
| | | | | | | | | | | | I've tried to grep for more of this with no success. Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Use run_in_background in preference to preserve_fnRichard van der Hoff2018-04-271-1/+4
|/ | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.
* Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-04-121-2/+19
|\ | | | | | | erikj/processed_event_lag
| * Send events to ASes concurrentlyErik Johnston2018-04-111-2/+19
| |
* | Set all metrics at the same timeErik Johnston2018-04-121-4/+4
| |
* | Track last processed event received_tsErik Johnston2018-04-111-0/+10
| |
* | Track where event stream processing have gotten up toErik Johnston2018-04-111-0/+4
|/
* Metrics for events processed in appservice and fed senderRichard van der Hoff2018-01-151-0/+7
| | | | More metrics I wished I'd had
* Remove preserve_context_over_{fn, deferred}Richard van der Hoff2017-11-141-2/+2
| | | | | Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off.
* Remove useless assignment in notify_interested_servicesIlya Zhuravlev2017-11-071-1/+0
|
* Fix appservices being backlogged and not receiving new events due to a bug ↵Ilya Zhuravlev2017-11-021-4/+1
| | | | in notify_interested_services
* storage/appservice: make appservice methods only relying on the cache ↵Patrik Oldsberg2016-10-061-11/+9
| | | | synchronous
* Reject malformed 3PE query metadata results earlier in AS API handling codePaul "LeoNerd" Evans2016-09-091-9/+5
|
* Minor fixes from PR commentsPaul "LeoNerd" Evans2016-09-091-2/+2
|
* Efficiency fix for lookups of a single protocolPaul "LeoNerd" Evans2016-09-091-1/+4
|
* Don't corrupt shared cache on subsequent protocol requestsPaul "LeoNerd" Evans2016-09-091-0/+2
|
* Collect up all the "instances" lists of individual AS protocol results into ↵Paul "LeoNerd" Evans2016-09-081-1/+29
| | | | one combined answer to the client
* Merge pull request #1041 from matrix-org/paul/third-party-lookupPaul Evans2016-08-251-0/+10
|\ | | | | Extend 3PE lookup APIs for metadata query
| * Don't need toplevel cache on 3PE lookup metadata any morePaul "LeoNerd" Evans2016-08-251-8/+0
| |
| * Actually query over AS API for 3PE lookup metadataPaul "LeoNerd" Evans2016-08-251-13/+1
| |
| * Move static knowledge of protocol metadata into AS handler; cache the resultPaul "LeoNerd" Evans2016-08-241-2/+24
| |
| * Initial hack at the 3PN protocols metadata lookup APIPaul "LeoNerd" Evans2016-08-241-0/+8
| |
* | Preserve some logcontextsErik Johnston2016-08-241-4/+4
|/
* Merge pull request #1026 from matrix-org/paul/thirdpartylookupPaul Evans2016-08-181-0/+24
|\ | | | | 3rd party entity lookup
| * Move validation logic for AS 3PE query response into ApplicationServiceApi ↵Paul "LeoNerd" Evans2016-08-181-44/+2
| | | | | | | | class, to keep the handler logic neater
| * More warnings about invalid results from AS 3PE queryPaul "LeoNerd" Evans2016-08-181-2/+7
| |
| * Avoid so much copypasta between 3PU and 3PL query by unifying around a ↵Paul "LeoNerd" Evans2016-08-181-26/+9
| | | | | | | | ThirdPartyEntityKind enumeration
| * Minor syntax neateningsPaul "LeoNerd" Evans2016-08-181-14/+8
| |
| * Appease pep8Paul "LeoNerd" Evans2016-08-181-0/+2
| |
| * Copypasta the 3PU support code to also do 3PLPaul "LeoNerd" Evans2016-08-181-3/+30
| |
| * Log a warning if an AS yields an invalid 3PU lookup resultPaul "LeoNerd" Evans2016-08-181-1/+6
| |
| * Scattergather the call out to ASes; validate received resultsPaul "LeoNerd" Evans2016-08-181-7/+34
| |
| * Filter 3PU lookups by only ASes that declare knowledge of that protocolPaul "LeoNerd" Evans2016-08-181-2/+4
| |
| * Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookupPaul "LeoNerd" Evans2016-08-181-44/+55
| |\
| * | Extend individual list results into the main return list, don't appendPaul "LeoNerd" Evans2016-08-181-1/+1
| | |
| * | Thread 3PU lookup through as far as the AS API object; which currently noöps itPaul "LeoNerd" Evans2016-08-171-0/+21
| | |
* | | Add appservice workerErik Johnston2016-08-181-38/+51
| |/ |/|
* | Make AppserviceHandler stream events from databaseErik Johnston2016-08-181-21/+44
| | | | | | | | | | | | | | | | | | This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.
* | Make notify_interested_services fasterErik Johnston2016-08-171-23/+8
| |
* | Measure notify_interested_servicesErik Johnston2016-08-171-19/+22
|/
* Move the AS handler out of the Handlers object.Mark Haines2016-05-311-10/+5
| | | | | | Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.
* copyrightsMatthew Hodgson2016-01-071-1/+1
|
* Fix bug where synapse was sending AS user queries incorrectly.Kegan Dougal2015-06-171-1/+1
| | | | | | Bug introduced in 92b20713d7c6346aeb20dc09963081e472752bb5 which reversed the comparison when checking if a user existed in the users table. Added UTs to prevent this happening again.
* Remove unused importMark Haines2015-05-221-1/+1
|
* Make the appservice use 'users_in_room' rather than get_room_members since ↵Mark Haines2015-05-221-4/+1
| | | | it is cached
* More missed get_user_by_id API changesDavid Baker2015-05-011-1/+1
|
* Fix more AS sender ID thinkos.Kegan Dougal2015-04-011-1/+8
| | | | | | Specifically, the ASes own user ID wasn't being treated as 'exclusive' so a human could nab it. Also, the HS would needlessly send user queries to the AS for its own user ID.
* Remove more reg/unreg methods. Read config not database for cache.Kegan Dougal2015-03-311-37/+0
|
* Set the service ID as soon as it is known.Kegan Dougal2015-03-161-1/+1
|
* Minor PR comment tweaks.Kegan Dougal2015-03-161-2/+8
|
* Use seconds; start gluing in the AS scheduler into the AS handler.Kegan Dougal2015-03-091-3/+14
|
* Notify ASes for events sent by other users in a room which an AS user is a ↵Kegan Dougal2015-02-111-5/+16
| | | | part of.
* Fix bugs so lazy room joining works as intended.Kegan Dougal2015-02-091-3/+3
|
* Register a user account for the AS when the AS registers. Add 'sender' ↵Kegan Dougal2015-02-091-1/+7
| | | | column to AS table.
* Dependency inject ApplicationServiceApi when creating ↵Kegan Dougal2015-02-051-3/+2
| | | | ApplicationServicesHandler.
* Fix user query checks. HS>AS pushing now works.Kegan Dougal2015-02-051-13/+31
|
* Glue AS work to general event notifications. Add more exception handling ↵Kegan Dougal2015-02-051-14/+25
| | | | when poking ASes.
* Add unknown room alias check. Call it from directory_handler.get_associationKegan Dougal2015-02-051-43/+54
|
* Add unknown user ID check. Use store.get_aliases_for_room(room_id) when ↵Kegan Dougal2015-02-051-5/+19
| | | | searching for services by alias.
* Add hs_token column and generate a different token f.e application service.Kegan Dougal2015-02-051-3/+6
|
* Impl push_bulk functionKegan Dougal2015-02-051-1/+1
|
* Add query_user/alias APIs.Kegan Dougal2015-02-041-10/+16
|
* Begin to add unit tests for appservice glue and regex testing.Kegan Dougal2015-02-041-2/+6
|
* Add stub ApplicationServiceApi and glue it with the handler.Kegan Dougal2015-02-041-3/+15
|
* Add appservice package and move ApplicationService into it.Kegan Dougal2015-02-031-1/+3
|
* Implement restricted namespace checks. Begin fleshing out the main hook for ↵Kegan Dougal2015-02-031-0/+19
| | | | notifying application services.
* Add namespace constants. Add restrict_to option to limit namespace checks.Kegan Dougal2015-02-031-10/+15
|
* Implement regex checks for app services.Kegan Dougal2015-02-031-2/+23
| | | | | Expose handler.get_services_for_event which manages the checks for all services.
* Implement txns for AS (un)registration.Kegan Dougal2015-02-021-1/+2
|
* Add basic application_services SQL, and hook up parts of the appservice ↵Kegan Dougal2015-01-281-3/+4
| | | | store to read from it.
* Log when ASes are registered/unregistered.Kegan Dougal2015-01-271-0/+2
|
* Use ApplicationService when registering.Kegan Dougal2015-01-271-8/+5
|
* Add stub methods, TODOs and docstrings for application services.Kegan Dougal2015-01-271-3/+22
|
* Add appservice handler and store. Glue together rest > handler > store.Kegan Dougal2015-01-271-0/+49
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-09-21 14:27:25 +0000