Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Add a locality to a few presence metrics (#15952) | Jason Little | 2023-07-18 | 1 | -13/+24 |
| | |||||
* | Remove some unused `server_name` fields (#15723) | Sean Quah | 2023-06-06 | 1 | -1/+0 |
| | | | Signed-off-by: Sean Quah <seanq@matrix.org> | ||||
* | Bump black from 22.12.0 to 23.1.0 (#15103) | dependabot[bot] | 2023-02-22 | 1 | -2/+0 |
| | |||||
* | Fix a bug in the send_local_online_presence_to module API (#14880) | Patrick Cloke | 2023-01-25 | 1 | -6/+12 |
| | | | | | | | Destination was being used incorrectly (a single destination instead of a list of destinations was being passed). This also updates some of the types in the area to not use Collection[str], which is a footgun. | ||||
* | Add an early return when handling no-op presence updates. (#14855) | Erik Johnston | 2023-01-16 | 1 | -0/+5 |
| | | | This stops us from incrementing the presence stream position for no-op updates. | ||||
* | Better return type for `get_all_entities_changed` (#14604) | Erik Johnston | 2022-12-05 | 1 | -5/+7 |
| | | | | Help callers from using the return value incorrectly by ensuring that callers explicitly check if there was a cache hit or not. | ||||
* | Properly handle unknown results for the stream change cache. (#14592) | Patrick Cloke | 2022-12-02 | 1 | -2/+2 |
| | | | | | | StreamChangeCache.get_all_changed_entities can return None to signify it does not have information at the given stream position. Two callers (related to device lists and presence) were treating this response the same as an empty list (i.e. there being no updates). | ||||
* | Fix an invalid comparison of `UserPresenceState` to `str` (#14393) | Andrew Morgan | 2022-11-16 | 1 | -1/+1 |
| | |||||
* | Remove redundant types from comments. (#14412) | Patrick Cloke | 2022-11-16 | 1 | -1/+1 |
| | | | | | | | Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in. | ||||
* | Bump flake8-bugbear from 22.9.23 to 22.10.27 (#14329) | dependabot[bot] | 2022-11-04 | 1 | -2/+4 |
| | | | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> | ||||
* | Do not allow a None-limit on PaginationConfig. (#14146) | Patrick Cloke | 2022-10-14 | 1 | -1/+3 |
| | | | | | | | The callers either set a default limit or manually handle a None-limit later on (by setting a default value). Update the callers to always instantiate PaginationConfig with a default limit and then assume the limit is non-None. | ||||
* | Update `get_users_in_room` mis-use to get hosts with dedicated ↵ | Eric Eastwood | 2022-08-24 | 1 | -2/+1 |
| | | | | | `get_current_hosts_in_room` (#13605) See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755 | ||||
* | Refactor presence so we can prune user in room caches (#13313) | Erik Johnston | 2022-07-25 | 1 | -76/+36 |
| | | | | | | | | See #10826 and #10786 for context as to why we had to disable pruning on those caches. Now that `get_users_who_share_room_with_user` is called frequently only for presence, we just need to make calls to it less frequent and then we can remove the various levels of caching that is going on. | ||||
* | Wait for lazy join to complete when getting current state (#12872) | Erik Johnston | 2022-06-01 | 1 | -1/+5 |
| | |||||
* | Additional constants for EDU types. (#12884) | Patrick Cloke | 2022-05-27 | 1 | -3/+5 |
| | | | Instead of hard-coding strings in many places. | ||||
* | Add `StreamKeyType` class and replace string literals with constants (#12567) | Andrew Morgan | 2022-05-16 | 1 | -3/+3 |
| | |||||
* | Prevent memory leak from reoccurring when presence is disabled. (#12656) | Erik Johnston | 2022-05-06 | 1 | -15/+27 |
| | |||||
* | Prevent a sync request from removing a user's busy presence status (#12213) | David Baker | 2022-04-13 | 1 | -10/+46 |
| | | | | | | | | | | In trying to use the MSC3026 busy presence status, the user's status would be set back to 'online' next time they synced. This change makes it so that syncing does not affect a user's presence status if it is currently set to 'busy': it must be removed through the presence API. The MSC defers to implementations on the behaviour of busy presence, so this ought to remain compatible with the MSC. | ||||
* | Refactor and convert `Linearizer` to async (#12357) | Sean Quah | 2022-04-05 | 1 | -2/+2 |
| | | | | | | | | | | | Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah <seanq@element.io> | ||||
* | Start application service stream token tracking from 1 (#12193) | Andrew Morgan | 2022-03-30 | 1 | -1/+1 |
| | | | Co-authored-by: Erik Johnston <erik@matrix.org> | ||||
* | Remove unnecessary pass statements. (#12206) | Patrick Cloke | 2022-03-11 | 1 | -2/+0 |
| | |||||
* | Rename get_tcp_replication to get_replication_command_handler. (#12192) | Patrick Cloke | 2022-03-10 | 1 | -2/+2 |
| | | | | | | Since the object it returns is a ReplicationCommandHandler. This is clean-up from adding support to Redis where the command handler was added as an additional layer of abstraction from the TCP protocol. | ||||
* | Remove `HomeServer.get_datastore()` (#12031) | Richard van der Hoff | 2022-02-23 | 1 | -2/+2 |
| | | | | | | | The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733 | ||||
* | Add type hints to `synapse/storage/databases/main` (#11984) | Dirk Klimpel | 2022-02-21 | 1 | -12/+14 |
| | |||||
* | Remove `log_function` and its uses (#11761) | Richard van der Hoff | 2022-01-18 | 1 | -2/+0 |
| | | | | | | | I've never found this terribly useful. I think it was added in the early days of Synapse, without much thought as to what would actually be useful to log, and has just been cargo-culted ever since. Rather, it tends to clutter up debug logs with useless information. | ||||
* | Remove redundant `get_current_events_token` (#11643) | Richard van der Hoff | 2022-01-04 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | * Push `get_room_{min,max_stream_ordering}` into StreamStore Both implementations of this are identical, so we may as well push it down and get rid of the abstract base class nonsense. * Remove redundant `StreamStore` class This is empty now * Remove redundant `get_current_events_token` This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid of it. * newsfile | ||||
* | Correctly register shutdown handler for presence workers (#11518) | David Robertson | 2021-12-07 | 1 | -1/+1 |
| | | | Fixes #11517 | ||||
* | Annotate `log_function` decorator (#10943) | reivilibre | 2021-10-27 | 1 | -0/+2 |
| | | | Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> | ||||
* | Improve docstrings for methods related to sending EDUs to application ↵ | Andrew Morgan | 2021-10-21 | 1 | -5/+29 |
| | | | | services (#11138) | ||||
* | Add missing type hints to synapse.api. (#11109) | Patrick Cloke | 2021-10-18 | 1 | -1/+1 |
| | | | | * Convert UserPresenceState to attrs. * Remove args/kwargs from error classes and explicitly pass msg/errorcode. | ||||
* | Fix issue causing sending presence to ASes to fail (due to incomplete type ↵ | reivilibre | 2021-09-30 | 1 | -1/+3 |
| | | | | annotations) (#10944) | ||||
* | Add type hints for event streams. (#10856) | Patrick Cloke | 2021-09-21 | 1 | -3/+5 |
| | |||||
* | Require type hints in the handlers module. (#10831) | Patrick Cloke | 2021-09-20 | 1 | -18/+27 |
| | | | | | | | Adds missing type hints to methods in the synapse.handlers module and requires all methods to have type hints there. This also removes the unused construct_auth_difference method from the FederationHandler. | ||||
* | Use direct references for some configuration variables (part 2) (#10812) | Patrick Cloke | 2021-09-15 | 1 | -6/+6 |
| | |||||
* | Add types to synapse.util. (#10601) | reivilibre | 2021-09-10 | 1 | -2/+3 |
| | |||||
* | Additional type hints for REST servlets (part 2). (#10674) | Patrick Cloke | 2021-08-26 | 1 | -0/+5 |
| | | | Applies the changes from #10665 to additional modules. | ||||
* | Do not remove `status_msg` when user going offline (#10550) | Dirk Klimpel | 2021-08-09 | 1 | -7/+4 |
| | | | Signed-off-by: Dirk Klimpel dirk@klimpel.org | ||||
* | Use inline type hints in `handlers/` and `rest/`. (#10382) | Jonathan de Jong | 2021-07-16 | 1 | -15/+13 |
| | |||||
* | Only send a presence state to a destination once (#10165) | Erik Johnston | 2021-06-11 | 1 | -13/+12 |
| | | | | It turns out that we were sending the same presence state to a remote potentially multiple times. | ||||
* | Fix sending presence over federation when using workers (#10163) | Erik Johnston | 2021-06-11 | 1 | -6/+19 |
| | | | | | | When using a federation sender we'd send out all local presence updates over federation even when they shouldn't be. Fixes #10153. | ||||
* | Use a database table to hold the users that should have full presence sent ↵ | Andrew Morgan | 2021-05-18 | 1 | -26/+110 |
| | | | | to them, instead of something in-memory (#9823) | ||||
* | Increase perf of handling presence when joining large rooms. (#9916) | Erik Johnston | 2021-05-05 | 1 | -72/+82 |
| | |||||
* | Use get_current_users_in_room from store and not StateHandler (#9910) | Erik Johnston | 2021-05-05 | 1 | -1/+1 |
| | |||||
* | Merge remote-tracking branch 'origin/release-v1.33.0' into develop | Erik Johnston | 2021-04-28 | 1 | -1/+23 |
|\ | |||||
| * | Fix tight loop handling presence replication. (#9900) | Erik Johnston | 2021-04-28 | 1 | -1/+23 |
| | | | | | | | | | | Only affects workers. Introduced in #9819. Fixes #9899. | ||||
* | | Add type hints to presence handler (#9885) | Erik Johnston | 2021-04-28 | 1 | -70/+89 |
|/ | |||||
* | Use current state table for `presence.get_interested_remotes` (#9887) | Erik Johnston | 2021-04-27 | 1 | -7/+2 |
| | | | This should be a lot quicker than asking the state handler. | ||||
* | Split presence out of master (#9820) | Erik Johnston | 2021-04-23 | 1 | -20/+36 |
| | |||||
* | Remove `synapse.types.Collection` (#9856) | Richard van der Hoff | 2021-04-22 | 1 | -1/+2 |
| | | | This is no longer required, since we have dropped support for Python 3.5. | ||||
* | Add presence federation stream (#9819) | Erik Johnston | 2021-04-20 | 1 | -20/+223 |
| | |||||
* | Fix bug where we sent remote presence states to remote servers (#9850) | Erik Johnston | 2021-04-20 | 1 | -3/+8 |
| | |||||
* | Don't send normal presence updates over federation replication stream (#9828) | Erik Johnston | 2021-04-19 | 1 | -15/+63 |
| | |||||
* | Move some replication processing out of generic_worker (#9796) | Erik Johnston | 2021-04-14 | 1 | -0/+246 |
| | | | Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> | ||||
* | Remove redundant "coding: utf-8" lines (#9786) | Jonathan de Jong | 2021-04-14 | 1 | -1/+0 |
| | | | | | | | Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>` | ||||
* | Bump black configuration to target py36 (#9781) | Dan Callahan | 2021-04-13 | 1 | -1/+1 |
| | | | Signed-off-by: Dan Callahan <danc@element.io> | ||||
* | Add a Synapse Module for configuring presence update routing (#9491) | Andrew Morgan | 2021-04-06 | 1 | -43/+235 |
| | | | | | | | | | | | | At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though. This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around. A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync. The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being: * Sending state for a specific set or all known users to a defined set of local and remote users. * The ability to trigger an initial sync for specific users, so they receive all current state. | ||||
* | Incorporate review | Brendan Abolivier | 2021-03-19 | 1 | -1/+1 |
| | |||||
* | Fix lint | Brendan Abolivier | 2021-03-19 | 1 | -7/+6 |
| | |||||
* | Move support for MSC3026 behind an experimental flag | Brendan Abolivier | 2021-03-18 | 1 | -2/+10 |
| | |||||
* | Implement MSC3026: busy presence state | Brendan Abolivier | 2021-03-18 | 1 | -1/+2 |
| | |||||
* | Prevent presence background jobs from running when presence is disabled (#9530) | Aaron Raimist | 2021-03-03 | 1 | -14/+17 |
| | | | | | Prevent presence background jobs from running when presence is disabled Signed-off-by: Aaron Raimist <aaron@raim.ist> | ||||
* | Be smarter about which hosts to send presence to when processing room joins ↵ | Andrew Morgan | 2021-02-19 | 1 | -14/+42 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (#9402) This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually. --- When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed. It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw. This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence: * If it was a local user join, send that user's latest presence to all servers in the room * If it was a remote user join, send the presence for all local users in the room to that homeserver We deduplicate by inserting all of those pending updates into a dictionary of the form: ``` { server_name1: {presence_update1, ...}, server_name2: {presence_update1, presence_update2, ...} } ``` Only after building this dict do we then start sending out presence updates. | ||||
* | Remove dead notify_for_states presence method (#9408) | Andrew Morgan | 2021-02-17 | 1 | -11/+0 |
| | |||||
* | Fix only handling the last presence state for each user (#9425) | Andrew Morgan | 2021-02-17 | 1 | -2/+5 |
| | | | | | | | | | | | | | | This is a small bug that I noticed while working on #8956. We have a for-loop which attempts to strip all presence changes for each user except for the final one, as we don't really care about older presence: https://github.com/matrix-org/synapse/blob/9e19c6aab4b5a99039f2ddc7d3120dd3b26c274b/synapse/handlers/presence.py#L368-L371 `new_states_dict` stores this stripped copy of latest presence state for each user, before it is... put into a new variable `new_state`, which is just overridden by the subsequent for loop. I believe this was instead meant to override `new_states`. Without doing so, it effectively meant: 1. The for loop had no effect. 2. We were still processing old presence state for users. | ||||
* | Update black, and run auto formatting over the codebase (#9381) | Eric Eastwood | 2021-02-16 | 1 | -10/+5 |
| | | | | | | | - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](https://github.com/matrix-org/synapse/blob/80d6dc9783aa80886a133756028984dbf8920168/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version | ||||
* | Use TYPE_CHECKING instead of magic MYPY variable. (#8770) | Patrick Cloke | 2020-11-17 | 1 | -3/+2 |
| | |||||
* | Fix cache call signature to accept `on_invalidate`. (#8684) | Erik Johnston | 2020-10-29 | 1 | -5/+7 |
| | | | Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before. | ||||
* | Fix typos and spelling errors. (#8639) | Patrick Cloke | 2020-10-23 | 1 | -2/+2 |
| | |||||
* | Clean up types for PaginationConfig (#8250) | Erik Johnston | 2020-09-08 | 1 | -3/+0 |
| | | | This removes `SourcePaginationConfig` and `get_pagination_rows`. The reasoning behind this is that these generic classes/functions erased the types of the IDs it used (i.e. instead of passing around `StreamToken` it'd pass in e.g. `token.room_key`, which don't have uniform types). | ||||
* | Stop sub-classing object (#8249) | Patrick Cloke | 2020-09-04 | 1 | -1/+1 |
| | |||||
* | Add type hints for state. (#8140) | Patrick Cloke | 2020-08-24 | 1 | -3/+3 |
| | |||||
* | Convert stream database to async/await. (#8074) | Patrick Cloke | 2020-08-17 | 1 | -1/+1 |
| | |||||
* | Rename database classes to make some sense (#8033) | Erik Johnston | 2020-08-05 | 1 | -2/+2 |
| | |||||
* | Convert state resolution to async/await (#7942) | Patrick Cloke | 2020-07-24 | 1 | -2/+2 |
| | |||||
* | Convert presence handler helpers to async/await. (#7939) | Patrick Cloke | 2020-07-23 | 1 | -23/+20 |
| | |||||
* | Refactor getting replication updates from database. (#7636) | Erik Johnston | 2020-06-16 | 1 | -3/+26 |
| | | | The aim here is to make it easier to reason about when streams are limited and when they're not, by moving the logic into the database functions themselves. This should mean we can kill of `db_query_to_update_function` function. | ||||
* | Replace iteritems/itervalues/iterkeys with native versions. (#7692) | Patrick Cloke | 2020-06-15 | 1 | -8/+6 |
| | |||||
* | Add option to move event persistence off master (#7517) | Erik Johnston | 2020-05-22 | 1 | -0/+6 |
| | |||||
* | Ignore incoming presence updates when presence is disabled (#7508) | Andrew Morgan | 2020-05-15 | 1 | -2/+4 |
| | |||||
* | Stop the master relaying USER_SYNC for other workers (#7318) | Richard van der Hoff | 2020-04-22 | 1 | -95/+115 |
| | | | | | | | Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257. | ||||
* | Remove concept of a non-limited stream. (#7011) | Erik Johnston | 2020-03-20 | 1 | -2/+2 |
| | |||||
* | Port PresenceHandler to async/await (#6991) | Erik Johnston | 2020-02-26 | 1 | -105/+87 |
| | |||||
* | Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) | Patrick Cloke | 2020-02-21 | 1 | -3/+3 |
| | | | | Ensure good comprehension hygiene using flake8-comprehensions. | ||||
* | Fix an error which was thrown by the PresenceHandler _on_shutdown handler. ↵ | Richard van der Hoff | 2020-01-06 | 1 | -7/+2 |
| | | | | (#6640) | ||||
* | Add database config class (#6513) | Erik Johnston | 2019-12-18 | 1 | -1/+1 |
| | | | | | This encapsulates config for a given database and is the way to get new connections. | ||||
* | Fix presence timeouts when synchrotron restarts. (#6212) | Erik Johnston | 2019-10-18 | 1 | -4/+9 |
| | | | | | | | | | * Fix presence timeouts when synchrotron restarts. Handling timeouts would fail if there was an external process that had timed out, e.g. a synchrotron restarting. This was due to a couple of variable name typoes. Fixes #3715. | ||||
* | Fix races in room stats (and other) updates. (#6187) | Richard van der Hoff | 2019-10-10 | 1 | -4/+12 |
| | | | | | | | Hopefully this will fix the occasional failures we were seeing in the room directory. The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8. We can solve this easily by only processing up to the stream_id where we know all events have been persisted. | ||||
* | fix typo | Matthew Hodgson | 2019-09-03 | 1 | -1/+1 |
| | |||||
* | Remove unnecessary parentheses around return statements (#5931) | Andrew Morgan | 2019-08-30 | 1 | -2/+2 |
| | | | | | Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :) | ||||
* | Move some timeout checking logs to DEBUG #5785 | Amber Brown | 2019-07-30 | 1 | -1/+1 |
| | |||||
* | Replace returnValue with return (#5736) | Amber Brown | 2019-07-23 | 1 | -31/+23 |
| | |||||
* | Move logging utilities out of the side drawer of util/ and into logging/ (#5606) | Amber Brown | 2019-07-04 | 1 | -2/+2 |
| | |||||
* | Fixup comment | Erik Johnston | 2019-07-01 | 1 | -6/+13 |
| | |||||
* | Fix sync tightloop bug. | Erik Johnston | 2019-06-21 | 1 | -2/+12 |
| | | | | | | | | If, for some reason, presence updates take a while to persist then it can trigger clients to tightloop calling `/sync` due to the presence handler returning updates but not advancing the stream token. Fixes #5503. | ||||
* | Run Black. (#5482) | Amber Brown | 2019-06-20 | 1 | -153/+158 |
| | |||||
* | Fix missing logcontext for PresenceHandler.on_shutdown. (#5369) | Richard van der Hoff | 2019-06-06 | 1 | -1/+7 |
| | | | | Fixes some warnings, and a scary-looking stacktrace when sytest kills the process. | ||||
* | Fix background updates to handle redactions/rejections (#5352) | Erik Johnston | 2019-06-06 | 1 | -4/+7 |
| | | | | | | | | * Fix background updates to handle redactions/rejections In background updates based on current state delta stream we need to handle that we may not have all the events (or at least that `get_events` may raise an exception). | ||||
* | Fix "db txn 'update_presence' from sentinel context" log messages (#5275) | Richard van der Hoff | 2019-05-28 | 1 | -52/+47 |
| | | | | | Fixes #4414. | ||||
* | Fix infinite loop in presence handler | Richard van der Hoff | 2019-04-26 | 1 | -0/+5 |
| | | | | Fixes #5102 | ||||
* | Fix grammar and document get_current_users_in_room (#4998) | Andrew Morgan | 2019-04-03 | 1 | -1/+1 |
| | |||||
* | Remove presence lists (#4989) | Neil Johnson | 2019-04-03 | 1 | -165/+2 |
| | | | Remove presence list support as per MSC 1819 | ||||
* | Use an assert | Erik Johnston | 2019-03-28 | 1 | -2/+1 |
| | |||||
* | Review comments | Erik Johnston | 2019-03-28 | 1 | -44/+62 |
| | |||||
* | Use event streams to calculate presence | Erik Johnston | 2019-03-27 | 1 | -29/+130 |
| | | | | | | | | | | | | | | | | | Primarily this fixes a bug in the handling of remote users joining a room where the server sent out the presence for all local users in the room to all servers in the room. We also change to using the state delta stream, rather than the distributor, as it will make it easier to split processing out of the master process (as well as being more flexible). Finally, when sending presence states to newly joined servers we filter out old presence states to reduce the number sent. Initially we filter out states that are offline and have a last active more than a week ago, though this can be changed down the line. Fixes #3962 | ||||
* | Avoid rebuilding Edu objects in worker mode (#4770) | Richard van der Hoff | 2019-03-04 | 1 | -3/+3 |
| | | | | | In worker mode, on the federation sender, when we receive an edu for sending over the replication socket, it is parsed into an Edu object. There is no point extracting the contents of it so that we can then immediately build another Edu. | ||||
* | Integrate presence from hotfixes (#3694) | Amber Brown | 2018-08-18 | 1 | -7/+19 |
| | |||||
* | Run tests under PostgreSQL (#3423) | Amber Brown | 2018-08-13 | 1 | -0/+5 |
| | |||||
* | Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678) | Amber Brown | 2018-08-10 | 1 | -1/+1 |
| | |||||
* | run isort | Amber Brown | 2018-07-09 | 1 | -11/+10 |
| | |||||
* | Remove all global reactor imports & pass it around explicitly (#3424) | Amber Brown | 2018-06-25 | 1 | -2/+2 |
| | |||||
* | Consistently use six's iteritems and wrap lazy keys/values in list() if ↵ | Amber Brown | 2018-05-31 | 1 | -6/+6 |
| | | | | they're not meant to be lazy (#3307) | ||||
* | Merge remote-tracking branch 'origin/develop' into 3218-official-prom | Amber Brown | 2018-05-28 | 1 | -7/+8 |
|\ | |||||
| * | Merge pull request #3244 from NotAFile/py3-six-4 | Amber Brown | 2018-05-24 | 1 | -7/+8 |
| |\ | | | | | | | replace some iteritems with six | ||||
| | * | replace some iteritems with six | Adrian Tschira | 2018-05-19 | 1 | -7/+8 |
| | | | | | | | | | | | | Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | | cleanup | Amber Brown | 2018-05-22 | 1 | -4/+8 |
| | | | |||||
* | | | cleanups, self-registration | Amber Brown | 2018-05-22 | 1 | -3/+4 |
| | | | |||||
* | | | Merge remote-tracking branch 'origin/develop' into 3218-official-prom | Amber Brown | 2018-05-22 | 1 | -56/+5 |
|\| | | |||||
| * | | Stub out ServerNoticesSender on the workers | Richard van der Hoff | 2018-05-22 | 1 | -4/+0 |
| | | | | | | | | | | | | | | | ... and have the sync endpoints call it directly rather than obsure indirection via PresenceHandler | ||||
| * | | Send users a server notice about consent | Richard van der Hoff | 2018-05-22 | 1 | -1/+9 |
| |/ | | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so. | ||||
| * | Remove unused `update_external_syncs` | Richard van der Hoff | 2018-05-17 | 1 | -55/+0 |
| | | | | | | | | This method isn't used anywhere. Burninate it. | ||||
* | | replacing portions | Amber Brown | 2018-05-21 | 1 | -29/+29 |
|/ | |||||
* | Improve exception handling for background processes | Richard van der Hoff | 2018-04-27 | 1 | -4/+15 |
| | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background. | ||||
* | Split replication layer into two | Erik Johnston | 2018-03-13 | 1 | -1/+0 |
| | |||||
* | Split out edu/query registration to a separate class | Erik Johnston | 2018-03-13 | 1 | -4/+6 |
| | |||||
* | typo | Matthew Hodgson | 2017-11-17 | 1 | -1/+1 |
| | |||||
* | replace 'except:' with 'except Exception:' | Richard van der Hoff | 2017-10-23 | 1 | -1/+1 |
| | | | | what could possibly go wrong | ||||
* | Make presence use cached users/hosts in room | Erik Johnston | 2017-05-16 | 1 | -3/+3 |
| | |||||
* | Reuse get_interested_parties | Erik Johnston | 2017-04-12 | 1 | -16/+5 |
| | |||||
* | Move get_interested_parties | Erik Johnston | 2017-04-11 | 1 | -35/+34 |
| | |||||
* | Comments | Erik Johnston | 2017-04-11 | 1 | -2/+3 |
| | |||||
* | Move get_interested_remotes back to presence handler | Erik Johnston | 2017-04-11 | 1 | -3/+49 |
| | |||||
* | Reduce federation presence replication traffic | Erik Johnston | 2017-04-10 | 1 | -40/+14 |
| | | | | | | | | This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination. | ||||
* | Use iteritems | Erik Johnston | 2017-04-06 | 1 | -2/+2 |
| | |||||
* | Add a timestamp to USER_SYNC command | Erik Johnston | 2017-03-31 | 1 | -7/+7 |
| | | | | This timestamp is used to indicate when the user last sync'd | ||||
* | Fix up presence | Erik Johnston | 2017-03-31 | 1 | -1/+3 |
| | |||||
* | Remove user from process_presence when stops syncing | Erik Johnston | 2017-03-30 | 1 | -0/+1 |
| | |||||
* | Add functions to presence to support remote syncs | Erik Johnston | 2017-03-30 | 1 | -0/+66 |
| | | | | | | | The TCP replication protocol streams deltas of who has started or stopped syncing. This is different from the HTTP API which periodically sends the full list of users who are syncing. This commit adds support for the new TCP style of sending deltas. | ||||
* | Cache hosts in room | Erik Johnston | 2017-03-24 | 1 | -2/+1 |
| | |||||
* | Don't recreate so many sets | Erik Johnston | 2017-03-16 | 1 | -8/+9 |
| | |||||
* | Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence | Erik Johnston | 2017-03-15 | 1 | -1/+4 |
|\ | | | | | Add fallback to last_active_ts if it beats the last sync time on a presence timeout. | ||||
| * | Add a great comment to handle_timeout for active vs sync times. | Will Hunt | 2017-03-15 | 1 | -0/+2 |
| | | |||||
| * | Add fallback to last_active_ts if it beats the last sync time. | Will Hunt | 2017-03-15 | 1 | -1/+2 |
| | | |||||
* | | Cache set of users whose presence the other user should see | Erik Johnston | 2017-03-15 | 1 | -13/+27 |
| | | |||||
* | | Comment | Erik Johnston | 2017-03-15 | 1 | -0/+3 |
| | | |||||
* | | Format presence events on the edges instead of reformatting them multiple times | Erik Johnston | 2017-03-15 | 1 | -15/+15 |
|/ | |||||
* | Cache get_presence storage | Erik Johnston | 2017-02-13 | 1 | -1/+1 |
| | |||||
* | Make presence.get_new_events a bit faster | Erik Johnston | 2017-02-02 | 1 | -29/+15 |
| | | | | We do this by caching the set of users a user shares rooms with. | ||||
* | Use DB cache of joined users for presence | Erik Johnston | 2017-01-30 | 1 | -3/+3 |
| | |||||
* | Use new federation_sender DI | Erik Johnston | 2016-11-16 | 1 | -5/+6 |
| | |||||
* | Fix typo "persiting" | Paul "LeoNerd" Evans | 2016-09-13 | 1 | -2/+2 |
| | |||||
* | Clobber EDUs in send queue | Erik Johnston | 2016-09-09 | 1 | -15/+5 |
| | |||||
* | Deduplicate presence in _update_states | Erik Johnston | 2016-09-09 | 1 | -0/+6 |
| | |||||
* | Check the user_id for presence/typing matches origin | Erik Johnston | 2016-09-08 | 1 | -0/+7 |
| | |||||
* | Record counts of state changes | Erik Johnston | 2016-09-06 | 1 | -0/+4 |
| | |||||
* | Short circuit if presence is the same | Erik Johnston | 2016-09-06 | 1 | -0/+3 |
| | |||||
* | Fiddle should_notify to better report stats | Erik Johnston | 2016-09-06 | 1 | -12/+7 |
| | |||||
* | Record why we have chosen to notify | Erik Johnston | 2016-09-05 | 1 | -0/+8 |
| | |||||
* | Fix typo in log line | Erik Johnston | 2016-09-01 | 1 | -1/+1 |
| | |||||
* | Merge pull request #1055 from matrix-org/erikj/occaisonally_persist | Erik Johnston | 2016-08-30 | 1 | -0/+28 |
|\ | | | | | Occaisonally persist unpersisted presence updates | ||||
| * | Check correct variable | Erik Johnston | 2016-08-30 | 1 | -1/+1 |
| | | |||||
| * | Occaisonally persist unpersisted presence updates | Erik Johnston | 2016-08-30 | 1 | -0/+28 |
| | | |||||
* | | Don't notify for online -> online transitions. | Erik Johnston | 2016-08-30 | 1 | -1/+6 |
|/ | | | | | Specifically, if currently_active remains true then we should not notify if only the last active time changes. | ||||
* | Use state handler instead of get_users_in_room/get_joined_hosts | Erik Johnston | 2016-08-26 | 1 | -4/+9 |
| | |||||
* | Make synchrotron accept /events | Erik Johnston | 2016-08-12 | 1 | -8/+19 |
| | |||||
* | Don't change status_msg on /sync | Will Hunt | 2016-08-10 | 1 | -3/+6 |
| | |||||
* | .values() returns list of sets | Erik Johnston | 2016-06-06 | 1 | -1/+2 |
| | |||||
* | Fix typos | Erik Johnston | 2016-06-06 | 1 | -32/+36 |
| | |||||
* | Fire after 30s not 8h | Erik Johnston | 2016-06-06 | 1 | -1/+1 |
| | |||||
* | Change metric style | Erik Johnston | 2016-06-03 | 1 | -5/+4 |
| | |||||
* | Presence metrics. Change def of small delta | Erik Johnston | 2016-06-03 | 1 | -5/+10 |
| | |||||
* | Allow external processes to mark a user as syncing. (#812) | Mark Haines | 2016-06-02 | 1 | -13/+106 |
| | | | | | | | | | | | | * Add infrastructure to the presence handler to track sync requests in external processes * Expire stale entries for dead external processes * Add an http endpoint for making users as syncing Add some docstrings and comments. * Fixes | ||||
* | Merge pull request #787 from matrix-org/markjh/liberate_presence_handler | Mark Haines | 2016-05-17 | 1 | -19/+21 |
|\ | | | | | Move the presence handler out of the Handlers object | ||||
| * | Don't inherit PresenceHandler from BaseHandler, remove references to self.hs ↵ | Mark Haines | 2016-05-16 | 1 | -19/+21 |
| | | | | | | | | from presence handler | ||||
| * | Move the presence handler out of the Handlers object | Mark Haines | 2016-05-16 | 1 | -1/+1 |
| | | |||||
* | | Spell "domain" correctly | Mark Haines | 2016-05-16 | 1 | -2/+2 |
|/ | | | | s/domian/domain/g | ||||
* | Add and use get_domian_from_id | Erik Johnston | 2016-05-09 | 1 | -2/+2 |
| | |||||
* | Reduce database inserts when sending transactions | Erik Johnston | 2016-05-06 | 1 | -1/+1 |
| | |||||
* | Remove another unused function from presence | Mark Haines | 2016-03-17 | 1 | -4/+0 |
| | |||||
* | Remove dead code left over from presence changes | Mark Haines | 2016-03-17 | 1 | -4/+0 |
| | |||||
* | Merge pull request #612 from matrix-org/erikj/cache_size | Erik Johnston | 2016-03-02 | 1 | -0/+4 |
|\ | | | | | Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 | ||||
| * | Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 | Erik Johnston | 2016-03-01 | 1 | -0/+4 |
| | | |||||
* | | Add a /replication API for extracting the updates that happened on | Mark Haines | 2016-03-01 | 1 | -0/+19 |
|/ | | | | | | | | | | | | | | | | | | | | | | | | synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates. | ||||
* | Check presence token interval is less than 100, rather than the token itself | Erik Johnston | 2016-02-23 | 1 | -4/+4 |
| | |||||
* | Merge pull request #602 from matrix-org/erikj/presence | Erik Johnston | 2016-02-23 | 1 | -19/+45 |
|\ | | | | | Change the way we figure out presence updates for small deltas | ||||
| * | Handle get_all_entities_changed returning None | Erik Johnston | 2016-02-23 | 1 | -0/+3 |
| | | |||||
| * | Add comments | Erik Johnston | 2016-02-23 | 1 | -0/+4 |
| | | |||||
| * | Change the way we figure out presence updates for small deltas | Erik Johnston | 2016-02-23 | 1 | -17/+36 |
| | | |||||
* | | Merge pull request #600 from matrix-org/erikj/presence | Erik Johnston | 2016-02-23 | 1 | -26/+30 |
|\| | | | | | Measure PresenceEventSource.get_new_events | ||||
| * | Don't include defer.returnValue in Measure block | Erik Johnston | 2016-02-23 | 1 | -9/+12 |
| | | |||||
| * | Measure PresenceEventSource.get_new_events | Erik Johnston | 2016-02-23 | 1 | -34/+35 |
| | | |||||
* | | Pick up currently_active across federation | Erik Johnston | 2016-02-23 | 1 | -0/+1 |
|/ | |||||
* | Correctly filter states | Erik Johnston | 2016-02-19 | 1 | -2/+2 |
| | |||||
* | Only send presence updates to remote hosts if user is ours | Erik Johnston | 2016-02-19 | 1 | -2/+10 |
| | |||||
* | Add Measures to presence | Erik Johnston | 2016-02-19 | 1 | -57/+62 |
| | |||||
* | Add presence metric | Erik Johnston | 2016-02-19 | 1 | -0/+3 |
| | |||||
* | Remove spurious extra metrics | Erik Johnston | 2016-02-19 | 1 | -1/+0 |
| | |||||
* | Don't set currently_active for remote presence | Erik Johnston | 2016-02-19 | 1 | -6/+12 |
| | |||||
* | More metrics | Erik Johnston | 2016-02-19 | 1 | -0/+6 |
| | |||||
* | Add presence metrics | Erik Johnston | 2016-02-19 | 1 | -0/+8 |
| | |||||
* | Remove invalid arg. | Erik Johnston | 2016-02-18 | 1 | -1/+0 |
| | |||||
* | Add unit test | Erik Johnston | 2016-02-18 | 1 | -93/+187 |
| | |||||
* | Add stuff pulled from the DB to the cache | Erik Johnston | 2016-02-18 | 1 | -2/+4 |
| | |||||
* | PEP8 | Erik Johnston | 2016-02-18 | 1 | -2/+2 |
| | |||||
* | Prefix TS fields with _ts | Erik Johnston | 2016-02-18 | 1 | -27/+27 |
| | |||||
* | Remove status_msg when going offline. Don't offline -> online if you send a ↵ | Erik Johnston | 2016-02-18 | 1 | -5/+15 |
| | | | | message | ||||
* | Check presence state is a valid one | Erik Johnston | 2016-02-18 | 1 | -0/+6 |
| | |||||
* | Move if statement | Erik Johnston | 2016-02-18 | 1 | -8/+8 |
| | |||||
* | Comment | Erik Johnston | 2016-02-18 | 1 | -0/+2 |
| | |||||
* | Remove spurious comment | Erik Johnston | 2016-02-17 | 1 | -1/+0 |
| | |||||
* | Initial cut | Erik Johnston | 2016-02-17 | 1 | -1031/+631 |
| | |||||
* | Fix up logcontexts | Erik Johnston | 2016-02-08 | 1 | -9/+11 |
| | |||||
* | Fix flake8 warnings for new flake8 | Daniel Wagner-Hall | 2016-02-02 | 1 | -1/+1 |
| | |||||
* | copyrights | Matthew Hodgson | 2016-01-07 | 1 | -1/+1 |
| | |||||
* | Wrap calls to distributor.fire in appropriately named functions so that ↵ | Mark Haines | 2015-12-01 | 1 | -7/+11 |
| | | | | static analysis can work out want is calling what | ||||
* | Rename presence_handler.send_invite to presence_handler.send_presence_invite ↵ | Mark Haines | 2015-12-01 | 1 | -1/+1 |
| | | | | to distinguish it from normal invites | ||||
* | Allow guest users to join and message rooms | Daniel Wagner-Hall | 2015-11-10 | 1 | -1/+2 |
| | |||||
* | Open up /events to anonymous users for room events only | Daniel Wagner-Hall | 2015-11-05 | 1 | -2/+2 |
| | | | | Squash-merge of PR #345 from daniel/anonymousevents | ||||
* | Fix some races in the synapse presence handler caused by not yielding on ↵ | Mark Haines | 2015-10-09 | 1 | -3/+8 |
| | | | | deferreds | ||||
* | Typo | Erik Johnston | 2015-08-18 | 1 | -1/+1 |
| | |||||
* | None check | Erik Johnston | 2015-08-18 | 1 | -0/+2 |
| | |||||
* | More helpful variable names | Erik Johnston | 2015-08-18 | 1 | -7/+7 |
| | |||||
* | Doc strings | Erik Johnston | 2015-08-18 | 1 | -0/+28 |
| | |||||
* | Remove debug try/catch | Erik Johnston | 2015-08-18 | 1 | -47/+43 |
| | |||||
* | Implement a batched presence_handler.get_state and use it | Erik Johnston | 2015-08-17 | 1 | -0/+63 |
| | |||||
* | Consolidate duplicate code in notifier | Erik Johnston | 2015-07-02 | 1 | -1/+1 |
| | |||||
* | Don't do unecessary db ops in presence.get_state | Erik Johnston | 2015-06-16 | 1 | -14/+14 |
| | |||||
* | Add a cache for getting the presence list for a user | Mark Haines | 2015-05-22 | 1 | -9/+15 |
| | |||||
* | Remove spurious TODO comment | Mark Haines | 2015-05-22 | 1 | -1/+0 |
| | |||||
* | Removed unused 'is_visible' method | Mark Haines | 2015-05-20 | 1 | -27/+0 |
| | |||||
* | Clean up the presence_list checking logic a bit | Mark Haines | 2015-05-20 | 1 | -12/+9 |
| | |||||
* | Speed up the get_pagination_rows as well | Mark Haines | 2015-05-18 | 1 | -3/+17 |
| | |||||
* | Cache the most recent serial for each room | Mark Haines | 2015-05-18 | 1 | -6/+22 |
| | |||||
* | Move updating the serial and state of the presence cache into a single function | Mark Haines | 2015-05-18 | 1 | -17/+43 |
| | |||||
* | Merge branch 'notifier_performance' into markjh/presence_performance | Mark Haines | 2015-05-18 | 1 | -0/+4 |
|\ | |||||
| * | Merge branch 'develop' into notifier_performance | Mark Haines | 2015-05-18 | 1 | -49/+220 |
| |\ | |||||
| * \ | Merge branch 'notifier_unify' into notifier_performance | Mark Haines | 2015-05-14 | 1 | -20/+6 |
| |\ \ | |||||
| * | | | Don't bother checking for updates if the stream token hasn't advanced for a user | Mark Haines | 2015-05-13 | 1 | -0/+4 |
| | | | | |||||
* | | | | Add more doc string, reduce C+P boilerplate for getting room list | Mark Haines | 2015-05-18 | 1 | -6/+22 |
| |_|/ |/| | | |||||
* | | | More whitespace | Mark Haines | 2015-05-15 | 1 | -0/+1 |
| | | | |||||
* | | | Add some missed argument types, cleanup the whitespace a bit | Mark Haines | 2015-05-14 | 1 | -6/+17 |
| | | | |||||
* | | | More doc-strings | Mark Haines | 2015-05-14 | 1 | -39/+202 |
| | | | |||||
* | | | Remove unused arguments and doc PresenceHandler.push_update_to_clients | Mark Haines | 2015-05-14 | 1 | -12/+8 |
| |/ |/| | |||||
* | | Revert "Improvement to performance of presence event stream handling" | Mark Haines | 2015-05-12 | 1 | -20/+6 |
|/ | |||||
* | Merge pull request #147 from matrix-org/presence-performance | Mark Haines | 2015-05-12 | 1 | -6/+20 |
|\ | | | | | Improvement to performance of presence event stream handling | ||||
| * | Use the presence cachemap ordering to early-abort the iteration loop | Paul "LeoNerd" Evans | 2015-05-07 | 1 | -2/+7 |
| | | |||||
| * | Store presence cachemap in an ordered dict, so that the newer serials will ↵ | Paul "LeoNerd" Evans | 2015-05-07 | 1 | -4/+13 |
| | | | | | | | | be at the end | ||||
* | | Fix up leak. Add warnings. | Erik Johnston | 2015-05-08 | 1 | -4/+6 |
| | | |||||
* | | Change the way we do logging contexts so that they survive divergences | Erik Johnston | 2015-05-08 | 1 | -19/+15 |
|/ | |||||
* | Fix off by one in presence token handling | Erik Johnston | 2015-04-29 | 1 | -1/+1 |
| | |||||
* | Ensure we never miss any presence updates | Erik Johnston | 2015-04-28 | 1 | -1/+7 |
| |