Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Add metrics for number of outgoing EDUs, by type (#4695) | Richard van der Hoff | 2019-02-20 | 1 | -4/+18 |
| | |||||
* | Merge pull request #4420 from matrix-org/jaywink/openid-listener | Erik Johnston | 2019-02-11 | 1 | -42/+95 |
|\ | | | | | New listener resource for the federation API "openid/userinfo" endpoint | ||||
| * | Document `servlet_groups` parameters | Jason Robinson | 2019-01-23 | 1 | -0/+23 |
| | | | | | | | | Signed-off-by: Jason Robinson <jasonr@matrix.org> | ||||
| * | Split federation OpenID userinfo endpoint out of the federation resource | Jason Robinson | 2019-01-23 | 1 | -42/+72 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the OpenID userinfo endpoint to be active even if the federation resource is not active. The OpenID userinfo endpoint is called by integration managers to verify user actions using the client API OpenID access token. Without this verification, the integration manager cannot know that the access token is valid. The OpenID userinfo endpoint will be loaded in the case that either "federation" or "openid" resource is defined. The new "openid" resource is defaulted to active in default configuration. Signed-off-by: Jason Robinson <jasonr@matrix.org> | ||||
* | | Reject large transactions on federation (#4513) | Andrew Morgan | 2019-01-31 | 1 | -0/+16 |
| | | | | | | | | | | | | | | | | | | | | * Reject large transactions on federation * Add changelog * lint * Simplify large transaction handling | ||||
* | | No vdh tests! | Erik Johnston | 2019-01-29 | 1 | -2/+1 |
| | | |||||
* | | Enable room version v3 | Erik Johnston | 2019-01-29 | 1 | -2/+9 |
| | | |||||
* | | Update comment | Erik Johnston | 2019-01-29 | 1 | -3/+3 |
| | | |||||
* | | Only check event ID domain for signatures for V1 events | Erik Johnston | 2019-01-29 | 3 | -31/+44 |
| | | | | | | | | | | In future version events won't have an event ID, so we won't be able to do this check. | ||||
* | | Use event origin for filtering incoming events | Erik Johnston | 2019-01-29 | 1 | -4/+7 |
| | | | | | | | | | | | | We only process events sent to us from a server if the event ID matches the server, to help guard against federation storms. We replace this with a check against the event origin. | ||||
* | | Use snder and not event ID domain to check if ours | Erik Johnston | 2019-01-29 | 1 | -1/+1 |
| | | | | | | | | | | | | The transaction queue only sends out events that we generate. This was done by checking domain of event ID, but that can no longer be used. Instead, we may as well use the sender field. | ||||
* | | Merge pull request #4481 from matrix-org/erikj/event_builder | Erik Johnston | 2019-01-29 | 1 | -14/+6 |
|\ \ | | | | | | | Refactor event building into EventBuilder | ||||
| * | | Refactor event building into EventBuilder | Erik Johnston | 2019-01-29 | 1 | -14/+6 |
| | | | | | | | | | | | | | | | This is so that everything is done in one place, making it easier to change the event format based on room version | ||||
* | | | Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature | Erik Johnston | 2019-01-29 | 1 | -1/+1 |
|\ \ \ | | | | | | | | | Fixup calls to `comput_event_signature` | ||||
| * | | | Fixup calls to `comput_event_signature` | Erik Johnston | 2019-01-29 | 1 | -1/+1 |
| |/ / | | | | | | | | | | | | | | | | | | | | | | We currently pass FrozenEvent instead of `dict` to `compute_event_signature`, which works by accident due to `dict(event)` producing the correct result. This fixes PR #4493 commit 855a151 | ||||
* / / | Implement fallback for V2 invite API | Erik Johnston | 2019-01-28 | 2 | -14/+89 |
|/ / | | | | | | | | | If the room version is either 1 or 2 then a server should retry failed `/v2/invite` requests with the v1 API | ||||
* | | Require event format version to parse or create events | Erik Johnston | 2019-01-25 | 4 | -41/+65 |
| | | |||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 | Erik Johnston | 2019-01-25 | 3 | -13/+64 |
|\ \ | |||||
| * \ | Merge pull request #4448 from matrix-org/erikj/get_pdu_versions | Erik Johnston | 2019-01-24 | 3 | -11/+51 |
| |\ \ | | | | | | | | | Add room_version param to get_pdu | ||||
| | * | | Review comments | Erik Johnston | 2019-01-24 | 1 | -1/+2 |
| | | | | |||||
| | * | | Add room_version param to get_pdu | Erik Johnston | 2019-01-23 | 3 | -11/+50 |
| | |/ | | | | | | | | | | | | | When we add new event format we'll need to know the event format or room version when parsing events. | ||||
| * | | Update make_membership_event docs | Erik Johnston | 2019-01-24 | 1 | -2/+5 |
| | | | |||||
| * | | Remove unnecessary '_sign_event' | Erik Johnston | 2019-01-23 | 1 | -1/+8 |
| | | | |||||
| * | | Store rejected remote invite events as outliers | Erik Johnston | 2019-01-23 | 1 | -2/+8 |
| |/ | | | | | | | | | | | | | | | | | | | | | Currently they're stored as non-outliers even though the server isn't in the room, which can be problematic in places where the code assumes it has the state for all non outlier events. In particular, there is an edge case where persisting the leave event triggers a state resolution, which requires looking up the room version from state. Since the server doesn't have the state, this causes an exception to be thrown. | ||||
* | | Review comments | Erik Johnston | 2019-01-24 | 1 | -1/+2 |
| | | |||||
* | | Implement MSC 1813 - Add room version to make APIs | Erik Johnston | 2019-01-23 | 2 | -6/+23 |
|/ | | | | | We also implement `make_membership_event` converting the returned room version to an event format version. | ||||
* | Add /v2/invite federation API | Erik Johnston | 2019-01-15 | 2 | -6/+40 |
| | |||||
* | Add groundwork for new versions of federation APIs | Erik Johnston | 2019-01-15 | 2 | -67/+71 |
| | |||||
* | Don't log stack traces for HTTP error responses | Erik Johnston | 2019-01-08 | 1 | -1/+6 |
| | |||||
* | Refactor request sending to have better excpetions (#4358) | Erik Johnston | 2019-01-08 | 1 | -5/+14 |
| | | | | | | | | | | | | | | * Correctly retry and back off if we get a HTTPerror response * Refactor request sending to have better excpetions MatrixFederationHttpClient blindly reraised exceptions to the caller without differentiating "expected" failures (e.g. connection timeouts etc) versus more severe problems (e.g. programming errors). This commit adds a RequestSendFailed exception that is raised when "expected" failures happen, allowing the TransactionQueue to log them as warnings while allowing us to log other exceptions as actual exceptions. | ||||
* | Update synapse/federation/federation_server.py | Richard van der Hoff | 2018-11-09 | 1 | -1/+1 |
| | | | Co-Authored-By: erikjohnston <erikj@jki.re> | ||||
* | Update synapse/federation/federation_server.py | Richard van der Hoff | 2018-11-09 | 1 | -1/+1 |
| | | | Co-Authored-By: erikjohnston <erikj@jki.re> | ||||
* | Drop PDUs of unknown rooms | Erik Johnston | 2018-11-08 | 1 | -1/+23 |
| | | | | | | When we receive events over federation we will need to know the room version to be able to correctly handle them, e.g. once we start changing event formats. Currently, we attempt to handle events in unknown rooms. | ||||
* | Add helpers for getting prev and auth events (#4139) | Erik Johnston | 2018-11-06 | 1 | -3/+1 |
| | | | | | | | * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions. | ||||
* | Clean up event accesses and tests | Erik Johnston | 2018-11-02 | 1 | -3/+0 |
| | | | | | This is in preparation to refactor FrozenEvent to support different event formats for different room versions | ||||
* | Remove the unused /pull federation API (#4118) | Amber Brown | 2018-10-31 | 2 | -14/+0 |
| | |||||
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2018-10-25 | 1 | -1/+1 |
|\ | | | | | | | erikj/alias_disallow_list | ||||
| * | Fix a number of flake8 errors | Richard van der Hoff | 2018-10-24 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there. | ||||
* | | Add config option to control alias creation | Erik Johnston | 2018-10-19 | 1 | -14/+2 |
|/ | |||||
* | Fix incorrect truncation in get_missing_events | Richard van der Hoff | 2018-10-16 | 2 | -6/+4 |
| | | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id. | ||||
* | Various cleanups in the federation client code (#4031) | Richard van der Hoff | 2018-10-16 | 2 | -23/+23 |
| | | | | | | | | | | | | | | - Improve logging: log things in the right order, include destination and txids in all log lines, don't log successful responses twice - Fix the docstring on TransportLayerClient.send_transaction - Don't use treq.request, which is overcomplicated for our purposes: just use a twisted.web.client.Agent. - simplify the logic for setting up the bodyProducer - fix bytes/str confusions | ||||
* | Merge pull request #3964 from matrix-org/rav/remove_localhost_checks | Richard van der Hoff | 2018-09-28 | 1 | -30/+6 |
|\ | | | | | remove spurious federation checks on localhost | ||||
| * | Fix complete fail to do the right thing | Richard van der Hoff | 2018-09-28 | 1 | -1/+2 |
| | | |||||
| * | remove spurious federation checks on localhost | Richard van der Hoff | 2018-09-26 | 1 | -31/+6 |
| | | | | | | | | | | | | | | | | | | | | | | There's really no point in checking for destinations called "localhost" because there is nothing stopping people creating other DNS entries which point to 127.0.0.1. The right fix for this is https://github.com/matrix-org/synapse/issues/3953. Blocking localhost, on the other hand, means that you get a surprise when trying to connect a test server on localhost to an existing server (with a 'normal' server_name). | ||||
* | | more comments | Richard van der Hoff | 2018-09-27 | 1 | -2/+0 |
| | | |||||
* | | Include eventid in log lines when processing incoming federation ↵ | Richard van der Hoff | 2018-09-27 | 1 | -15/+17 |
| | | | | | | | | | | | | | | | | | | | | | | transactions (#3959) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines. | ||||
* | | fix docstring for FederationClient.get_state_for_room | Richard van der Hoff | 2018-09-26 | 1 | -4/+4 |
|/ | | | | trivial fixes for docstring | ||||
* | Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu | Amber Brown | 2018-09-26 | 1 | -1/+1 |
|\ | | | | | Comments and interface cleanup for on_receive_pdu | ||||
| * | Comments and interface cleanup for on_receive_pdu | Richard van der Hoff | 2018-09-20 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | Add some informative comments about what's going on here. Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of `get_missing` and use `sent_to_us_directly` consistently. | ||||
* | | Fix some instances of ExpiringCache not expiring cache items | Erik Johnston | 2018-09-21 | 1 | -16/+12 |
|/ | | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead. | ||||
* | Fix handling of redacted events from federation | Erik Johnston | 2018-09-13 | 1 | -1/+1 |
| | | | | | | | | | | If we receive an event that doesn't pass their content hash check (e.g. due to already being redacted) then we hit a bug which causes an exception to be raised, which then promplty stops the event (and request) from being processed. This effects all sorts of federation APIs, including joining rooms with a redacted state event. | ||||
* | Port federation/ to py3 (#3847) | Amber Brown | 2018-09-12 | 3 | -19/+18 |
| | |||||
* | Merge pull request #3826 from matrix-org/rav/logging_for_keyring | Amber Brown | 2018-09-12 | 1 | -7/+27 |
|\ | | | | | add some logging for the keyring queue | ||||
| * | clearer logging when things fail, too | Richard van der Hoff | 2018-09-06 | 1 | -7/+27 |
| | | |||||
* | | Limit the number of PDUs/EDUs per fedreation transaction | Erik Johnston | 2018-09-06 | 1 | -0/+12 |
|/ | |||||
* | Merge branch 'master' into develop | Richard van der Hoff | 2018-09-06 | 4 | -31/+125 |
|\ | |||||
| * | Fix origin handling for pushed transactions | Richard van der Hoff | 2018-09-05 | 3 | -15/+15 |
| | | | | | | | | | | Use the actual origin for push transactions, rather than whatever the remote server claimed. | ||||
| * | Check that signatures on events are valid | Richard van der Hoff | 2018-09-05 | 1 | -16/+110 |
| | | | | | | | | | | | | We should check that both the sender's server, and the server which created the event_id (which may be different from whatever the remote server has told us the origin is), have signed the event. | ||||
* | | Port http/ to Python 3 (#3771) | Amber Brown | 2018-09-06 | 1 | -5/+5 |
| | | |||||
* | | fix #3445 | Krombel | 2018-08-29 | 1 | -2/+2 |
| | | | | | | | | | | itervalues(d) calls d.itervalues() [PY2] and d.values() [PY3] but SortedDict only implements d.values() | ||||
* | | Merge pull request #3735 from matrix-org/travis/federation-spelling | Richard van der Hoff | 2018-08-22 | 1 | -1/+1 |
|\ \ | | | | | | | limt -> limit | ||||
| * | | limt -> limit | Travis Ralston | 2018-07-31 | 1 | -1/+1 |
| | | | |||||
* | | | Log more detail when we fail to authenticate request | Erik Johnston | 2018-08-21 | 1 | -3/+3 |
| | | | |||||
* | | | Don't log exceptions when failing to fetch server keys | Erik Johnston | 2018-08-21 | 1 | -2/+2 |
| |/ |/| | | | | | | | Not being able to resolve or connect to remote servers is an expected error, so we shouldn't log at ERROR with stacktraces. | ||||
* | | Integrate presence from hotfixes (#3694) | Amber Brown | 2018-08-18 | 1 | -0/+4 |
| | | |||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2018-08-15 | 1 | -4/+4 |
|\ \ | | | | | | | | | | erikj/split_federation | ||||
| * | | Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678) | Amber Brown | 2018-08-10 | 1 | -4/+4 |
| | | | |||||
* | | | Fixup doc comments | Erik Johnston | 2018-08-09 | 1 | -0/+11 |
| | | | |||||
* | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2018-08-09 | 5 | -13/+132 |
|\| | | | | | | | | | | | erikj/split_federation | ||||
| * | | Merge pull request #3664 from matrix-org/rav/federation_metrics | Richard van der Hoff | 2018-08-08 | 1 | -1/+9 |
| |\ \ | | | | | | | | | more metrics for the federation and appservice senders | ||||
| | * | | more metrics for the federation and appservice senders | Richard van der Hoff | 2018-08-07 | 1 | -1/+9 |
| | | | | |||||
| * | | | Check m.room.create for sane room_versions | Richard van der Hoff | 2018-08-06 | 1 | -3/+23 |
| | | | | |||||
| * | | | include known room versions in outgoing make_joins | Richard van der Hoff | 2018-08-06 | 2 | -4/+9 |
| | | | | |||||
| * | | | sanity check response from make_join | Richard van der Hoff | 2018-08-03 | 1 | -1/+3 |
| | | | | |||||
| * | | | Enforce compatibility when processing make_join requests | Richard van der Hoff | 2018-08-03 | 2 | -4/+41 |
| | | | | | | | | | | | | | | | | | | | | | | | | Reject make_join requests from servers which do not support the room version. Also include the room version in the response. | ||||
| * | | | Docstrings for BaseFederationServlet | Richard van der Hoff | 2018-08-03 | 1 | -0/+47 |
| |/ / | | | | | | | | | | ... to save me reverse-engineering this stuff again. | ||||
* / / | Add EDU/query handling over replication | Erik Johnston | 2018-08-06 | 1 | -0/+43 |
|/ / | |||||
* | | Merge pull request #3639 from matrix-org/rav/refactor_error_handling | Richard van der Hoff | 2018-08-02 | 1 | -12/+17 |
|\ \ | | | | | | | Clean up handling of errors from outbound requests | ||||
| * | | Refactor code for turning HttpResponseException into SynapseError | Richard van der Hoff | 2018-08-01 | 1 | -2/+2 |
| | | | | | | | | | | | | | | | | | | | | | | | | This commit replaces SynapseError.from_http_response_exception with HttpResponseException.to_synapse_error. The new method actually returns a ProxiedRequestError, which allows us to pass through additional metadata from the API call. | ||||
| * | | Be more careful which errors we send back over the C-S API | Richard van der Hoff | 2018-08-01 | 1 | -12/+17 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We really shouldn't be sending all CodeMessageExceptions back over the C-S API; it will include things like 401s which we shouldn't proxy. That means that we need to explicitly turn a few HttpResponseExceptions into SynapseErrors in the federation layer. The effect of the latter is that the matrix errcode will get passed through correctly to calling clients, which might help with some of the random M_UNKNOWN errors when trying to join rooms. | ||||
* | | | Merge pull request #3638 from ↵ | Richard van der Hoff | 2018-08-02 | 1 | -129/+148 |
|\| | | | | | | | | | | | | | | matrix-org/rav/refactor_federation_client_exception_handling Factor out exception handling in federation_client | ||||
| * | | Factor out exception handling in federation_client | Richard van der Hoff | 2018-08-01 | 1 | -129/+148 |
| | | | | | | | | | | | | | | | Factor out the error handling from make_membership_event, send_join, and send_leave, so that it can be shared. | ||||
* | | | Merge branch 'master' into develop | Richard van der Hoff | 2018-08-02 | 1 | -0/+1 |
|\ \ \ | |/ / |/| | | |||||
| * | | Validation for events/rooms in fed requests | Richard van der Hoff | 2018-08-02 | 1 | -0/+1 |
| | | | | | | | | | | | | | | | | | | | | | When we get a federation request which refers to an event id, make sure that said event is in the room the caller claims it is in. (patch supplied by @turt2live) | ||||
* | | | Python 3: Convert some unicode/bytes uses (#3569) | Amber Brown | 2018-08-02 | 1 | -1/+1 |
| |/ |/| | |||||
* | | Remove pdu_failures from transactions | Travis Ralston | 2018-07-30 | 5 | -96/+7 |
| | | | | | | The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this. | ||||
* | | Update the send_leave path to be an event_id | Travis Ralston | 2018-07-26 | 1 | -2/+2 |
| | | | | | | It's still not used, however the parameter is an event ID not a transaction ID. | ||||
* | | Improve logging for exceptions handling PDUs | Richard van der Hoff | 2018-07-23 | 1 | -3/+8 |
| | | | | | | | | when we get an exception handling a federation PDU, log the whole stacktrace. | ||||
* | | Run things as background processes | Richard van der Hoff | 2018-07-18 | 1 | -9/+6 |
| | | | | | | | | | | | | | | | | This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally). | ||||
* | | Resource tracking for background processes | Richard van der Hoff | 2018-07-18 | 1 | -5/+7 |
|/ | | | | | | | | | | | | | | | | This introduces a mechanism for tracking resource usage by background processes, along with an example of how it will be used. This will help address #3518, but more importantly will give us better insights into things which are happening but not being shown up by the request metrics. We *could* do this with Measure blocks, but: - I think having them pulled out as a completely separate metric class will make it easier to distinguish top-level processes from those which are nested. - I want to be able to report on in-flight background processes, and I don't think we want to do this for *all* Measure blocks. | ||||
* | rename assert_params_in_request to assert_params_in_dict | Krombel | 2018-07-13 | 1 | -2/+2 |
| | | | | | | the method "assert_params_in_request" does handle dicts and not requests. A request body has to be parsed to json before this method can be used | ||||
* | run isort | Amber Brown | 2018-07-09 | 9 | -64/+57 |
| | |||||
* | Implementation of server_acls | Richard van der Hoff | 2018-07-04 | 2 | -4/+154 |
| | | | | | ... as described at https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw. | ||||
* | More server_name validation | Richard van der Hoff | 2018-07-04 | 1 | -2/+3 |
| | | | | | | | | We need to do a bit more validation when we get a server name, but don't want to be re-doing it all over the shop, so factor out a separate parse_and_validate_server_name, and do the extra validation. Also, use it to verify the server name in the config file. | ||||
* | Reject invalid server names (#3480) | Richard van der Hoff | 2018-07-03 | 1 | -22/+44 |
| | | | | | Make sure that server_names used in auth headers are sane, and reject them with a sensible error code, before they disappear off into the depths of the system. | ||||
* | Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking | Erik Johnston | 2018-06-29 | 1 | -1/+3 |
|\ | | | | | Check the state of prev_events a bit more thoroughly when coming over federation | ||||
| * | handle federation not telling us about prev_events | Amber Brown | 2018-06-27 | 1 | -1/+3 |
| | | |||||
* | | Attempt to be more performant on PyPy (#3462) | Amber Brown | 2018-06-28 | 1 | -1/+1 |
|/ | |||||
* | Merge pull request #3428 from matrix-org/erikj/persisted_pdu | Erik Johnston | 2018-06-22 | 1 | -12/+1 |
|\ | | | | | Simplify get_persisted_pdu | ||||
| * | simplify get_persisted_pdu | Richard van der Hoff | 2018-06-12 | 1 | -12/+1 |
| | | | | | | | | | | it doesn't make much sense to use get_persisted_pdu on the receive path: just get the event straight from the store. | ||||
* | | Populate synapse_federation_client_sent_pdu_destinations:count again (#3386) | Amber Brown | 2018-06-21 | 1 | -3/+7 |
| | | |||||
* | | Remove run_on_reactor (#3395) | Amber Brown | 2018-06-14 | 1 | -4/+0 |
|/ | |||||
* | federation/send_queue.py: fix usage of sortedcontainers.SortedDict | Ivan Shapovalov | 2018-06-06 | 1 | -27/+22 |
| | |||||
* | Port to sortedcontainers (with tests!) (#3332) | Amber Brown | 2018-06-06 | 1 | -7/+7 |
| | |||||
* | federation/send_queue.py: fix usage of LaterGauge | Ivan Shapovalov | 2018-06-03 | 1 | -1/+1 |
| | | | | | Fixes a startup crash due to commit df9f72d9e5fe264b86005208e0f096156eb03e4b "replacing portions". | ||||
* | Consistently use six's iteritems and wrap lazy keys/values in list() if ↵ | Amber Brown | 2018-05-31 | 3 | -5/+7 |
| | | | | they're not meant to be lazy (#3307) | ||||
* | fixes | Amber Brown | 2018-05-23 | 1 | -4/+4 |
| | |||||
* | cleanup | Amber Brown | 2018-05-22 | 1 | -3/+2 |
| | |||||
* | cleanup pep8 errors | Amber Brown | 2018-05-22 | 2 | -6/+21 |
| | |||||
* | fixes | Amber Brown | 2018-05-22 | 1 | -3/+3 |
| | |||||
* | replacing portions | Amber Brown | 2018-05-21 | 4 | -53/+35 |
| | |||||
* | Merge pull request #3209 from damir-manapov/master | Richard van der Hoff | 2018-05-11 | 1 | -2/+0 |
|\ | | | | | transaction_id, destination defined twice | ||||
| * | transaction_id, destination twice | Damir Manapov | 2018-05-10 | 1 | -2/+0 |
| | | |||||
* | | Merge branch 'release-v0.28.1' into develop | Richard van der Hoff | 2018-05-01 | 1 | -3/+18 |
|\| | |||||
| * | Apply some limits to depth to counter abuse | Richard van der Hoff | 2018-05-01 | 1 | -3/+18 |
| | | | | | | | | | | | | | | * When creating a new event, cap its depth to 2^63 - 1 * When receiving events, reject any without a sensible depth As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI | ||||
* | | Merge branch 'develop' into py3-xrange-1 | Richard van der Hoff | 2018-04-30 | 1 | -2/+3 |
|\ \ | |||||
| * \ | Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background | Richard van der Hoff | 2018-04-27 | 2 | -3/+12 |
| |\ \ | |||||
| * | | | Use run_in_background in preference to preserve_fn | Richard van der Hoff | 2018-04-27 | 1 | -2/+3 |
| | |/ | |/| | | | | | | | | | | | | | While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object. | ||||
* | | | Move more xrange to six | Adrian Tschira | 2018-04-28 | 1 | -1/+3 |
| |/ |/| | | | | | | | | | plus a bonus next() Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Improve exception handling for background processes | Richard van der Hoff | 2018-04-27 | 2 | -3/+12 |
|/ | | | | | | | | | | | | | | | | | | There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background. | ||||
* | Merge pull request #3118 from matrix-org/rav/reject_prev_events | Richard van der Hoff | 2018-04-23 | 1 | -2/+23 |
|\ | | | | | Reject events which have lots of prev_events | ||||
| * | Add some comments | Richard van der Hoff | 2018-04-18 | 1 | -2/+23 |
| | | |||||
* | | Merge pull request #3106 from NotAFile/py3-six-itervalues-1 | Richard van der Hoff | 2018-04-20 | 2 | -9/+13 |
|\ \ | | | | | | | Use six.itervalues in some places | ||||
| * | | Use six.itervalues in some places | Adrian Tschira | 2018-04-15 | 2 | -9/+13 |
| |/ | | | | | | | | | | | There's more where that came from Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Merge pull request #3093 from matrix-org/rav/response_cache_wrap | Richard van der Hoff | 2018-04-20 | 1 | -11/+11 |
|\ \ | | | | | | | Refactor ResponseCache usage | ||||
| * | | Reinstate linearizer for federation_server.on_context_state_request | Richard van der Hoff | 2018-04-20 | 1 | -5/+11 |
| | | | |||||
| * | | Refactor ResponseCache usage | Richard van der Hoff | 2018-04-12 | 1 | -11/+5 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production. | ||||
* | | | Refactor store.have_events | Richard van der Hoff | 2018-04-20 | 1 | -1/+1 |
| |/ |/| | | | | | | | | | It turns out that most of the time we were calling have_events, we were only using half of the result. Replace have_events with have_seen_events and get_rejection_reasons, so that we can see what's going on a bit more clearly. | ||||
* | | fix federation_domain_whitelist | Matthew Hodgson | 2018-04-13 | 1 | -6/+6 |
| | | | | | | | | we were checking the wrong server_name on inbound requests | ||||
* | | revert last to PR properly | Matthew Hodgson | 2018-04-13 | 1 | -6/+6 |
| | | |||||
* | | correctly auth inbound federation_domain_whitelist reqs | Matthew Hodgson | 2018-04-13 | 1 | -6/+6 |
| | | |||||
* | | Revert "Use sortedcontainers instead of blist" | Richard van der Hoff | 2018-04-13 | 1 | -7/+7 |
|/ | | | | | | | | | | | This reverts commit 9fbe70a7dc3afabfdac176ba1f4be32dd44602aa. It turns out that sortedcontainers.SortedDict is not an exact match for blist.sorteddict; in particular, `popitem()` removes things from the opposite end of the dict. This is trivial to fix, but I want to add some unit tests, and potentially some more thought about it, before we do so. | ||||
* | Merge pull request #3092 from matrix-org/rav/response_cache_metrics | Richard van der Hoff | 2018-04-12 | 1 | -1/+1 |
|\ | | | | | Add metrics for ResponseCache | ||||
| * | Add metrics for ResponseCache | Richard van der Hoff | 2018-04-10 | 1 | -1/+1 |
| | | |||||
* | | Set all metrics at the same time | Erik Johnston | 2018-04-12 | 1 | -6/+6 |
| | | |||||
* | | Track last processed event received_ts | Erik Johnston | 2018-04-11 | 1 | -0/+11 |
| | | |||||
* | | Track where event stream processing have gotten up to | Erik Johnston | 2018-04-11 | 1 | -0/+4 |
| | | |||||
* | | Merge pull request #2760 from Valodim/pypy | Richard van der Hoff | 2018-04-11 | 1 | -7/+7 |
|\ \ | | | | | | | Synapse on PyPy | ||||
| * | | Use sortedcontainers instead of blist | Vincent Breitmoser | 2018-04-10 | 1 | -7/+7 |
| |/ | | | | | | | | | | | | | | | This commit drop-in replaces blist with SortedContainers. They are written in pure python so work with pypy, but perform as good as native implementations, at least in a couple benchmarks: http://www.grantjenks.com/docs/sortedcontainers/performance.html | ||||
* | | Merge pull request #3078 from matrix-org/erikj/federation_sender | Erik Johnston | 2018-04-10 | 1 | -15/+41 |
|\ \ | | | | | | | Send federation events concurrently | ||||
| * | | Use run_in_background instead | Erik Johnston | 2018-04-10 | 1 | -1/+1 |
| | | | |||||
| * | | Preserve log contexts correctly | Erik Johnston | 2018-04-10 | 1 | -1/+4 |
| | | | |||||
| * | | Log event ID on exception | Erik Johnston | 2018-04-10 | 1 | -1/+4 |
| | | | |||||
| * | | Handle all events in a room correctly | Erik Johnston | 2018-04-09 | 1 | -1/+2 |
| | | | |||||
| * | | Send federation events concurrently | Erik Johnston | 2018-04-09 | 1 | -4/+18 |
| | | | |||||
| * | | Handle exceptions in get_hosts_for_room when sending events over federation | Erik Johnston | 2018-04-09 | 1 | -11/+16 |
| |/ | |||||
* | | Ensure slashes are escaped | Erik Johnston | 2018-04-10 | 1 | -1/+1 |
| | | |||||
* | | URL quote path segments over federation | Erik Johnston | 2018-04-10 | 1 | -48/+80 |
|/ | |||||
* | pep8 | Luke Barnard | 2018-04-06 | 1 | -1/+1 |
| | |||||
* | Fix federation client `set_group_joinable` typo | Luke Barnard | 2018-04-06 | 1 | -1/+1 |
| | |||||
* | Implement group join API | David Baker | 2018-04-06 | 2 | -0/+31 |
| | |||||
* | use PUT instead of POST for federating groups/m.join_policy | Krombel | 2018-04-06 | 2 | -2/+2 |
| | |||||
* | Use "/settings/" (plural) | Luke Barnard | 2018-04-05 | 2 | -2/+2 |
| | |||||
* | Use join_policy API instead of joinable | Luke Barnard | 2018-04-03 | 2 | -6/+6 |
| | | | | | | | | | | | | | | | | | The API is now under /groups/$group_id/setting/m.join_policy and expects a JSON blob of the shape ```json { "m.join_policy": { "type": "invite" } } ``` where "invite" could alternatively be "open". | ||||
* | pep8 | David Baker | 2018-03-28 | 1 | -2/+1 |
| | |||||
* | Add joinability for groups | David Baker | 2018-03-28 | 2 | -0/+37 |
| | | | | | Adds API to set the 'joinable' flag, and corresponding flag in the table. | ||||
* | Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer | Erik Johnston | 2018-03-13 | 1 | -1/+1 |
|\ | | | | | Remove ReplicationLayer and user Client/Server directly | ||||
| * | s/replication_server/federation_server | Erik Johnston | 2018-03-13 | 1 | -1/+1 |
| | | |||||
* | | Merge pull request #2979 from matrix-org/erikj/no_handlers | Erik Johnston | 2018-03-13 | 4 | -69/+2 |
|\| | | | | | Don't build handlers on workers unnecessarily | ||||
| * | Remove unused ReplicationLayer | Erik Johnston | 2018-03-13 | 2 | -59/+0 |
| | | |||||
| * | Split replication layer into two | Erik Johnston | 2018-03-13 | 2 | -10/+2 |
| | | |||||
* | | Merge pull request #2977 from matrix-org/erikj/replication_move_props | Erik Johnston | 2018-03-13 | 4 | -22/+13 |
|\| | | | | | Move property setting from ReplicationLayer to base classes | ||||
| * | Move property setting from ReplicationLayer to FederationBase | Erik Johnston | 2018-03-13 | 4 | -22/+13 |
| | | |||||
* | | Fix docstring types | Erik Johnston | 2018-03-13 | 1 | -2/+2 |
|/ | |||||
* | Split out edu/query registration to a separate class | Erik Johnston | 2018-03-13 | 1 | -48/+69 |
| | |||||
* | Add federation_domain_whitelist option (#2820) | Matthew Hodgson | 2018-01-22 | 4 | -3/+18 |
| | | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network | ||||
* | Metrics for events processed in appservice and fed sender | Richard van der Hoff | 2018-01-15 | 1 | -0/+4 |
| | | | | More metrics I wished I'd had | ||||
* | Check missing fields in event_from_pdu_json | Richard van der Hoff | 2017-12-30 | 1 | -0/+7 |
| | | | | Return a 400 rather than a 500 when somebody messes up their send_join | ||||
* | Factor out `event_from_pdu_json` | Richard van der Hoff | 2017-12-30 | 3 | -37/+44 |
| | | | | | turns out we have two copies of this, and neither needs to be an instance method | ||||
* | federation_server: clean up imports | Richard van der Hoff | 2017-12-30 | 1 | -14/+11 |
| | |||||
* | federation_client: clean up imports | Richard van der Hoff | 2017-12-30 | 1 | -13/+10 |
| | |||||
* | Clear logcontext before starting fed txn queue runner | Richard van der Hoff | 2017-11-28 | 1 | -2/+8 |
| | | | | | These processes take a long time compared to the request, so there is lots of "Entering|Restoring dead context" in the logs. Let's try to shut it up a bit. | ||||
* | Remove preserve_context_over_{fn, deferred} | Richard van der Hoff | 2017-11-14 | 1 | -2/+2 |
| | | | | | Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off. | ||||
* | Add brackets | Erik Johnston | 2017-11-09 | 1 | -2/+4 |
| | |||||
* | Have an explicit API to update room config | Erik Johnston | 2017-11-08 | 2 | -1/+36 |
| | |||||
* | Revert "Modify group room association API to allow modification of is_public" | Erik Johnston | 2017-11-08 | 2 | -7/+6 |
| | |||||
* | Update docs for updating room group association | Luke Barnard | 2017-11-01 | 1 | -1/+1 |
| | |||||
* | Flake8 | Luke Barnard | 2017-10-31 | 1 | -3/+4 |
| | |||||
* | Modify group room association API to allow modification of is_public | Luke Barnard | 2017-10-31 | 2 | -4/+4 |
| | | | | also includes renamings to make things more consistent. | ||||
* | Do logcontexts outside ResponseCache | Erik Johnston | 2017-10-25 | 1 | -3/+5 |
| | |||||
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Erik Johnston | 2017-10-24 | 1 | -3/+3 |
|\ | | | | | | | erikj/group_fed_update_profile | ||||
| * | replace 'except:' with 'except Exception:' | Richard van der Hoff | 2017-10-23 | 1 | -3/+3 |
| | | | | | | | | what could possibly go wrong | ||||
* | | Correctly wire in update group profile over federation | Erik Johnston | 2017-10-23 | 2 | -11/+31 |
|/ | |||||
* | Fix typo | Erik Johnston | 2017-10-19 | 1 | -1/+1 |
| | |||||
* | Fix typo in group attestation handling | Erik Johnston | 2017-10-19 | 1 | -1/+1 |
| | |||||
* | Fix 500 error when we get an error handling a PDU | Richard van der Hoff | 2017-10-17 | 1 | -1/+0 |
| | | | | | | | | | | FederationServer doesn't have a send_failure (and nor does its subclass, ReplicationLayer), so this was failing. I'm not really sure what the idea behind send_failure is, given (a) we don't do anything at the other end with it except log it, and (b) we also send back the failure via the transaction response. I suspect there's a whole lot of dead code around it, but for now I'm just removing the broken bit. | ||||
* | Delint | Luke Barnard | 2017-10-16 | 1 | -0/+2 |
| | |||||
* | Implement GET /groups/$groupId/invited_users | Luke Barnard | 2017-10-16 | 2 | -1/+30 |
| | |||||
* | Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling | Richard van der Hoff | 2017-10-11 | 1 | -2/+3 |
|\ | | | | | log pdu_failures from incoming transactions | ||||
| * | log pdu_failures from incoming transactions | Richard van der Hoff | 2017-10-11 | 1 | -2/+3 |
| | | | | | | | | | | | | | | ... even if we have no EDUs. This appears to have been introduced in 476899295f5fd6cff64799bcbc84cd4bf9005e33. | ||||
* | | Fix typo in invite to group | Erik Johnston | 2017-10-11 | 1 | -1/+1 |
| | | |||||
* | | Merge pull request #2466 from matrix-org/erikj/groups_merged | Erik Johnston | 2017-10-11 | 2 | -1/+892 |
|\ \ | | | | | | | Initial Group Implementation | ||||
| * \ | Merge branch 'develop' into erikj/groups_merged | David Baker | 2017-10-02 | 2 | -69/+78 |
| |\ \ | |||||
| * | | | Add remove room API | Erik Johnston | 2017-09-26 | 2 | -1/+25 |
| | | | | |||||
| * | | | Add bulk group publicised lookup API | Erik Johnston | 2017-08-09 | 2 | -0/+32 |
| | | | | |||||
| * | | | Add update group profile API | Erik Johnston | 2017-07-20 | 1 | -0/+12 |
| | | | | |||||
| * | | | Comments | Erik Johnston | 2017-07-18 | 1 | -1/+1 |
| | | | | |||||
| * | | | Comments | Erik Johnston | 2017-07-18 | 1 | -0/+40 |
| | | | | |||||
| * | | | Fix typos | Erik Johnston | 2017-07-18 | 1 | -2/+2 |
| | | | | |||||
| * | | | Correctly parse query params | Erik Johnston | 2017-07-18 | 1 | -19/+19 |
| | | | | |||||
| * | | | Add client apis | Erik Johnston | 2017-07-18 | 1 | -9/+187 |
| | | | | |||||
| * | | | Comments | Erik Johnston | 2017-07-18 | 1 | -15/+19 |
| | | | | |||||
| * | | | Add local group server support | Erik Johnston | 2017-07-17 | 2 | -0/+121 |
| | | | | |||||
| * | | | Ensure category and role ids are non-null | Erik Johnston | 2017-07-14 | 1 | -0/+24 |
| | | | | |||||
| * | | | Comments | Erik Johnston | 2017-07-13 | 1 | -0/+8 |
| | | | | |||||
| * | | | Add summary APIs to federation | Erik Johnston | 2017-07-13 | 1 | -11/+223 |
| | | | | |||||
| * | | | Add group summary APIs | Erik Johnston | 2017-07-12 | 1 | -0/+17 |
| | | | | |||||
| * | | | Add federation API for adding room to group | Erik Johnston | 2017-07-11 | 1 | -0/+18 |
| | | | | |||||
| * | | | Comment | Erik Johnston | 2017-07-11 | 2 | -0/+26 |
| | | | | |||||
| * | | | Initial group server implementation | Erik Johnston | 2017-07-10 | 2 | -1/+176 |
| | | | | |||||
* | | | | fed server: process PDUs for different rooms in parallel | Richard van der Hoff | 2017-10-09 | 1 | -19/+34 |
| | | | | | | | | | | | | | | | | | | | | With luck, this will give a real-time improvement when there are many rooms and the server ends up calling out to fetch missing events. | ||||
* | | | | Fed server: use a linearizer for ongoing transactions | Richard van der Hoff | 2017-10-09 | 1 | -1/+29 |
| |_|/ |/| | | | | | | | | | | | We don't want to process the same transaction multiple times concurrently, so use a linearizer. | ||||
* | | | fed server: refactor on_incoming_transaction | Richard van der Hoff | 2017-10-09 | 1 | -24/+29 |
| | | | | | | | | | | | | | | | Move as much as possible to after the have_responded check, and reduce the number of times we iterate over the pdu list. | ||||
* | | | Fed server: Move origin-check code to _handle_received_pdu | Richard van der Hoff | 2017-10-09 | 1 | -24/+24 |
| | | | | | | | | | | | | | | | | | | | | | The response-building code expects there to be an entry in the `results` list for each entry in the pdu_list, so the early `continue` was messing this up. That doesn't really matter, because all that the federation client does is log any errors, but it's pretty poor form. | ||||
* | | | Fix up logcontext handling in (federation) TransactionQueue | Richard van der Hoff | 2017-10-06 | 1 | -16/+32 |
| |/ |/| | | | | | | | Avoid using preserve_context_over_function, which has problems with respect to logcontexts. | ||||
* | | Make the spam checker a module | David Baker | 2017-09-26 | 1 | -3/+2 |
| | | |||||
* | | Fix logcontexts in _check_sigs_and_hashes | Richard van der Hoff | 2017-09-20 | 2 | -60/+62 |
| | | |||||
* | | Remove redundant `preserve_fn` | Richard van der Hoff | 2017-09-20 | 1 | -2/+2 |
| | | | | | | | | | | | | preserve_fn is a no-op unless the wrapped function returns a Deferred. verify_json_objects_for_server returns a list, so this is doing nothing. | ||||
* | | PoC for filtering spammy events (#2456) | Richard van der Hoff | 2017-09-19 | 1 | -16/+24 |
|/ | | | Demonstration of how you might add some hooks to filter out spammy events. | ||||
* | Separate federation servlet into different lists | Erik Johnston | 2017-07-05 | 1 | -8/+16 |
| | |||||
* | Remove spurious log lines | Erik Johnston | 2017-06-07 | 1 | -1/+0 |
| | |||||
* | Faster cache for get_joined_hosts | Erik Johnston | 2017-05-25 | 1 | -0/+2 |
| | |||||
* | Make presence use cached users/hosts in room | Erik Johnston | 2017-05-16 | 1 | -1/+1 |
| | |||||
* | Do some logging when one-time-keys get claimed | Richard van der Hoff | 2017-05-09 | 1 | -0/+10 |
| | | | | | might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened. | ||||
* | Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more | Erik Johnston | 2017-05-05 | 1 | -0/+9 |
|\ | | | | | Always mark remotes as up if we receive a signed request from them | ||||
| * | Use preserve_fn and add logs | Erik Johnston | 2017-05-05 | 1 | -1/+3 |
| | | |||||
| * | Always mark remotes as up if we receive a signed request from them | Erik Johnston | 2017-05-05 | 1 | -0/+7 |
| | | |||||
* | | Add cache for get_current_hosts_in_room | Erik Johnston | 2017-05-02 | 1 | -5/+1 |
|/ | |||||
* | Try harder when sending leave events | Richard van der Hoff | 2017-04-21 | 1 | -1/+19 |
| | | | | | When we're rejecting invites, ignore the backoff data, so that we have a better chance of not getting the room out of sync. | ||||
* | Fix some lies, and other clarifications, in docstrings | Richard van der Hoff | 2017-04-21 | 2 | -2/+68 |
| | | | | | The documentation on get_json has been wrong ever since the very first commit to synapse... | ||||
* | Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl | Erik Johnston | 2017-04-12 | 2 | -45/+110 |
|\ | | | | | Reduce federation replication traffic | ||||
| * | Comment | Erik Johnston | 2017-04-12 | 1 | -2/+1 |
| | | |||||
| * | Reuse get_interested_parties | Erik Johnston | 2017-04-12 | 1 | -3/+3 |
| | | |||||
| * | Comment | Erik Johnston | 2017-04-11 | 2 | -1/+9 |
| | | |||||
| * | Comments | Erik Johnston | 2017-04-11 | 1 | -7/+7 |
| | | |||||
| * | Move get_interested_remotes back to presence handler | Erik Johnston | 2017-04-11 | 1 | -35/+6 |
| | | |||||
| * | Comments | Erik Johnston | 2017-04-11 | 1 | -1/+14 |
| | | |||||
| * | Reduce federation presence replication traffic | Erik Johnston | 2017-04-10 | 2 | -36/+110 |
| | | | | | | | | | | | | | | | | This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination. | ||||
* | | Merge pull request #2121 from matrix-org/paul/sent-transactions-metric | Paul Evans | 2017-04-12 | 1 | -0/+3 |
|\ \ | | | | | | | Add a counter metric for successfully-sent transactions | ||||
| * | | Add a counter metric for successfully-sent transactions | Paul "LeoNerd" Evans | 2017-04-11 | 1 | -0/+3 |
| |/ | |||||
* | | Add some comments | Erik Johnston | 2017-04-12 | 1 | -0/+15 |
| | | |||||
* | | Use generators | Erik Johnston | 2017-04-10 | 1 | -2/+2 |
| | | |||||
* | | Dedupe KeyedEdu and Devices federation repl traffic | Erik Johnston | 2017-04-10 | 1 | -6/+9 |
|/ | |||||
* | Rename variable | Erik Johnston | 2017-04-10 | 1 | -7/+7 |
| | |||||
* | Revert to sending the same data type as before | Erik Johnston | 2017-04-10 | 1 | -2/+2 |
| | |||||
* | Change name | Erik Johnston | 2017-04-10 | 1 | -2/+2 |
| | |||||
* | Comments | Erik Johnston | 2017-04-10 | 1 | -4/+6 |
| | |||||
* | Comment | Erik Johnston | 2017-04-07 | 1 | -2/+15 |
| | |||||
* | Add logging | Erik Johnston | 2017-04-07 | 1 | -0/+8 |
| | |||||
* | Fix up federation SendQueue and document types | Erik Johnston | 2017-04-07 | 1 | -26/+220 |
| |