summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* Remove redundant run_as_background_process() from pusherpoolRichard van der Hoff2018-10-227-50/+14
| | | | | | | `on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher` now always return synchronously, so we can remove the `defer.gatherResults` on their results, and the `run_as_background_process` wrappers can be removed too because the PusherPool methods will now complete quickly enough.
* Give pushers their own background logcontextRichard van der Hoff2018-10-222-58/+54
| | | | | | | | Each pusher has its own loop which runs for as long as it has work to do. This should run in its own background thread with its own logcontext, as other similar loops elsewhere in the system do - which means that CPU usage is consistently attributed to that loop, rather than to whatever request happened to start the loop.
* move get_all_pushers call downRichard van der Hoff2018-10-221-3/+9
| | | | simplifies the interface to _start_pushers
* Factor PusherPool._start_pusher out of _start_pushersRichard van der Hoff2018-10-221-18/+33
| | | | | ... and use it from start_pusher_by_id. This mostly simplifies start_pusher_by_id.
* Rename _refresh_pusherRichard van der Hoff2018-10-222-3/+4
| | | | | This is public (or at least, called from outside the class), so ought to have a better name.
* Merge branch 'rav/fix_email_templates_4065' into developRichard van der Hoff2018-10-192-8/+0
|\
| * Remove notes on fallback for email_templatesRichard van der Hoff2018-10-192-8/+0
| | | | | | | | This fallback didn't work, and was removed in #4069.
* | Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065Richard van der Hoff2018-10-193-28/+21
|\| | | | | Calculate absolute path for email templates
| * Calculate absolute path for email templatesRichard van der Hoff2018-10-193-28/+21
| |
* | Fix manhole on py3 (pt 2) (#4067)Amber Brown2018-10-192-0/+3
| |
* | Make the metrics less racy (#4061)Amber Brown2018-10-193-16/+22
| |
* | Merge pull request #4060 from matrix-org/hawkowl/ssh-key-py3Richard van der Hoff2018-10-192-2/+3
|\ \ | | | | | | Make manhole work on Python 3 again
| * | changelogAmber Brown2018-10-191-0/+1
| | |
| * | make a bytestringAmber Brown2018-10-191-2/+2
|/ /
* | Merge pull request #4057 from matrix-org/rav/use_correct_pythonAmber Brown2018-10-192-1/+3
|\ \ | | | | | | Use the right python when starting workers
| * | Use the right python when starting workersRichard van der Hoff2018-10-182-1/+3
|/ / | | | | | | | | We should use the same python to start the workers as we do for the main synapse (ie, the same one used to run synctl.)
* | Merge branch 'master' into developRichard van der Hoff2018-10-181-0/+17
|\|
| * Add warnings about the upgrade to 0.33.7 github/release-v0.33.7 release-v0.33.7Richard van der Hoff2018-10-181-0/+17
| |
* | remove redundant changelog fileRichard van der Hoff2018-10-181-1/+0
| | | | | | | | this change has been released
* | Merge branch 'master' into developRichard van der Hoff2018-10-1824-18/+101
|\|
| * 0.33.7 v0.33.7Richard van der Hoff2018-10-182-3/+6
| |
| * fix changelog formatting v0.33.7rc2Richard van der Hoff2018-10-171-1/+4
| |
| * v0.33.7rc2Richard van der Hoff2018-10-171-1/+1
| |
| * prep changelogRichard van der Hoff2018-10-173-13/+25
| |
| * Fix incorrect truncation in get_missing_eventsRichard van der Hoff2018-10-175-35/+26
| | | | | | | | | | | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id.
| * Merge pull request #4052 from matrix-org/rav/ship_resources_as_package_dataRichard van der Hoff2018-10-1713-6/+53
| |\ | | | | | | Ship the email templates as package_data
| | * Put the warning blob at the top of the fileRichard van der Hoff2018-10-171-6/+9
| | |
| | * Ship the email templates as package_dataRichard van der Hoff2018-10-1713-6/+50
| |/ | | | | | | | | | | | | | | | | | | move the example email templates into the synapse package so that they can be used as package data, which should mean that all of the packaging mechanisms (pip, docker, debian, arch, etc) should now come with the example templates. In order to grandfather in people who relied on the templates being in the old place, check for that situation and fall back to using the defaults if the templates directory does not exist.
| * update changelog v0.33.7rc1Amber Brown2018-10-1511-10/+28
| |
| * update changelogAmber Brown2018-10-151-2/+1
| |
| * version bumpAmber Brown2018-10-151-1/+1
| |
* | Merge pull request #4049 from matrix-org/erikj/synctl_colourErik Johnston2018-10-182-1/+11
|\ \ | | | | | | Only colourise synctl output when attached to tty
| * | Assume isatty is always defined, and catch AttributeError. Also don't bother ↵Erik Johnston2018-10-171-2/+6
| | | | | | | | | | | | checking colour==Normal
| * | NewsfileErik Johnston2018-10-171-0/+1
| | |
| * | Only colourise synctl output when attached to ttyErik Johnston2018-10-171-1/+6
| | |
* | | Merge pull request #4050 from matrix-org/erikj/fix_py37_iterationErik Johnston2018-10-172-2/+6
|\ \ \ | |/ / |/| | Fix bug where we raised StopIteration in a generator
| * | NewsfileErik Johnston2018-10-171-0/+1
| | |
| * | Fix bug where we raised StopIteration in a generatorErik Johnston2018-10-171-2/+5
|/ / | | | | | | This made python 3.7 unhappy
* | Fix roomlist since tokens on Python 3 (#4046)Will Hunt2018-10-174-3/+50
| | | | | | Thanks @Half-Shot !!!
* | Merge pull request #4045 from matrix-org/rav/fix_get_missing_eventsRichard van der Hoff2018-10-175-35/+26
|\ \ | | | | | | Fix incorrect truncation in get_missing_events
| * | Fix incorrect truncation in get_missing_eventsRichard van der Hoff2018-10-165-35/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | It's quite important that get_missing_events returns the *latest* events in the room; however we were pulling event ids out of the database until we got *at least* 10, and then taking the *earliest* of the results. We also shouldn't really be relying on depth, and should be checking the room_id.
* | | Merge pull request #4041 from matrix-org/rav/run_tests_in_dockerAmber Brown2018-10-173-43/+51
|\ \ \ | |/ / |/| | run the circle builds in docker containers
| * | Use wget rather than curlRichard van der Hoff2018-10-161-2/+2
| | | | | | | | | | | | the docker image doesn't have wget
| * | changelogRichard van der Hoff2018-10-161-0/+1
| | |
| * | run the circle builds in docker containersRichard van der Hoff2018-10-161-41/+48
|/ / | | | | | | Docker containers spin up faster than entire VMs.
* / Various cleanups in the federation client code (#4031)Richard van der Hoff2018-10-164-61/+64
|/ | | | | | | | | | | | | | - Improve logging: log things in the right order, include destination and txids in all log lines, don't log successful responses twice - Fix the docstring on TransportLayerClient.send_transaction - Don't use treq.request, which is overcomplicated for our purposes: just use a twisted.web.client.Agent. - simplify the logic for setting up the bodyProducer - fix bytes/str confusions
* Merge pull request #4019 from matrix-org/dbkr/e2e_backupsDavid Baker2018-10-1510-0/+1442
|\ | | | | E2E backups
| * missed oneDavid Baker2018-10-121-0/+1
| |
| * Linting soothes the savage PEP8 monsterDavid Baker2018-10-121-4/+9
| |
| * Misc PR feedback bitsDavid Baker2018-10-121-2/+2
| |
| * Don't mangle exceptionsDavid Baker2018-10-121-4/+2
| |
| * Split /room_keys/version into 2 servletsDavid Baker2018-10-121-9/+21
| |
| * PR feedback pt. 1David Baker2018-10-123-29/+22
| |
| * Fix mergefailDavid Baker2018-10-091-1/+1
| |
| * Add changelogDavid Baker2018-10-091-0/+1
| |
| * isortDavid Baker2018-10-091-1/+1
| |
| * Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backupsDavid Baker2018-10-09210-3221/+5551
| |\
| * | Apparently this blank line is Very ImportantDavid Baker2018-10-091-0/+1
| | |
| * | be python3 compatibleDavid Baker2018-10-091-2/+3
| | |
| * | Don't reuse backup versionsDavid Baker2018-10-052-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | Since we don't actually delete the keys, just mark the versions as deleted in the db rather than actually deleting them, then we won't reuse versions. Fixes https://github.com/vector-im/riot-web/issues/7448
| * | WIP e2e key backupsDavid Baker2018-09-132-4/+10
| | | | | | | | | | | | | | | | | | | | | Continues from uhoreg's branch This just fixed the errcode on /room_keys/version if no backup and updates the schema delta to be on the latest so it gets run
| * | try to make flake8 and isort happyHubert Chathi2018-09-065-8/+15
| | |
| * | update to newer Synapse APIsHubert Chathi2018-08-242-1/+2
| | |
| * | Merge branch 'develop' into e2e_backupsHubert Chathi2018-08-24232-4107/+7197
| |\ \
| * | | allow session_data to be any JSON instead of just a stringHubert Chathi2018-08-211-2/+4
| | | |
| * | | allow auth_data to be any JSON instead of a stringHubert Chathi2018-08-211-2/+5
| | | |
| * | | missing importMatthew Hodgson2018-08-121-1/+1
| | | |
| * | | missing importMatthew Hodgson2018-08-121-0/+1
| | | |
| * | | use parse_stringMatthew Hodgson2018-08-121-3/+3
| | | |
| * | | 404 nicely if you try to interact with a missing current versionMatthew Hodgson2018-08-122-16/+57
| | | |
| * | | support DELETE /version with no argsMatthew Hodgson2018-08-122-3/+4
| | | |
| * | | flake8Matthew Hodgson2018-08-122-9/+14
| | | |
| * | | implement remaining tests and make them workMatthew Hodgson2018-08-124-33/+287
| | | |
| * | | lintingMatthew Hodgson2018-08-121-25/+23
| | | |
| * | | fix idiocies and so make tests passMatthew Hodgson2018-08-123-11/+15
| | | |
| * | | first cut at a UTMatthew Hodgson2018-08-121-0/+141
| | | |
| * | | don't needlessly return user_idMatthew Hodgson2018-08-121-1/+0
| | | |
| * | | improve docstringMatthew Hodgson2018-08-121-1/+5
| | | |
| * | | switch get_current_version_info back to being get_version_infoMatthew Hodgson2018-08-121-4/+7
| | | |
| * | | fix flakesMatthew Hodgson2018-08-122-4/+6
| | | |
| * | | fix typosMatthew Hodgson2018-08-122-2/+3
| | | |
| * | | add a tonne of docstring; make upload_room_keys properly assert versionMatthew Hodgson2018-08-122-9/+113
| | | |
| * | | add a tonne of docstring; make upload_room_keys properly assert versionMatthew Hodgson2018-08-121-20/+31
| | | |
| * | | more docstring for the e2e_room_keys restMatthew Hodgson2018-08-122-2/+51
| | | |
| * | | add storage docstring; remove unused set_e2e_room_keysMatthew Hodgson2018-08-121-36/+83
| | | |
| * | | fix factoring out of _should_replace_room_keyMatthew Hodgson2018-08-121-3/+4
| | | |
| * | | rename room_key_version table correctly, and fix opt argsMatthew Hodgson2018-08-122-16/+16
| | | |
| * | | blindly incorporate PR review - needs testing & fixingMatthew Hodgson2018-08-125-79/+99
| | | |
| * | | make /room_keys/version workMatthew Hodgson2018-08-124-15/+38
| | | |
| * | | implement /room_keys/version too (untested)Matthew Hodgson2018-08-125-6/+171
| | | |
| * | | document the APIMatthew Hodgson2018-08-121-0/+133
| | | |
| * | | make it work and fix pep8Matthew Hodgson2018-08-127-82/+134
| | | |
| * | | interim WIP checkin; doesn't build yetMatthew Hodgson2018-08-123-19/+84
| | | |
| * | | total WIP skeleton for /room_keys APIMatthew Hodgson2018-08-124-0/+289
| | | |
* | | | Merge pull request #4033 from intelfx/py37-hotfixesAmber Brown2018-10-153-3/+5
|\ \ \ \ | | | | | | | | | | py3: python3.7 hotfixes
| * | | | synapse/app: frontend_proxy.py: actually make workers work on py3Ivan Shapovalov2018-10-142-2/+3
| | | | |
| * | | | synapse/visibility.py: fix SyntaxError on py3.7Ivan Shapovalov2018-10-142-1/+2
|/ / / /
* | | | Make workers work on Py3 (#4027)Amber Brown2018-10-1313-62/+64
| | | |
* | | | Comments on get_all_new_events_streamRichard van der Hoff2018-10-121-1/+15
| | | | | | | | | | | | | | | | just some docstrings to clarify the behaviour here
* | | | Merge pull request #4022 from matrix-org/erikj/metrics_lazy_loaded_countErik Johnston2018-10-102-8/+33
|\ \ \ \ | | | | | | | | | | Add metric to count lazy member sync requests
| * | | | Add description to counter metricErik Johnston2018-10-101-1/+5
| | | | |
| * | | | Update newsfileErik Johnston2018-10-101-1/+1
| | | | |
| * | | | Track number of non-empty sync responses insteadErik Johnston2018-10-101-9/+15
| | | | |
| * | | | Append _total to metric and fix up spellingErik Johnston2018-10-101-2/+2
| | | | |
| * | | | NewsfileErik Johnston2018-10-091-0/+1
| | | | |
| * | | | Add metric to count lazy member sync requestsErik Johnston2018-10-091-6/+20
|/ / / /
* | | | Merge pull request #4008 from matrix-org/erikj/log_looping_exceptionsErik Johnston2018-10-093-13/+36
|\ \ \ \ | |_|_|/ |/| | | Log looping call exceptions
| * | | Remove unnecessary extra function call layerErik Johnston2018-10-082-31/+16
| | | |
| * | | Use errback pattern and catch async failuresErik Johnston2018-10-082-15/+35
| | | |
| * | | NewsfileErik Johnston2018-10-051-0/+1
| | | |
| * | | Log looping call exceptionsErik Johnston2018-10-051-1/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a looping call function errors, then it kills the loop entirely. Currently it throws away the exception logs, so we should make it actually log them. Fixes #3929
* | | | Merge pull request #4017 from matrix-org/rav/optimise_filter_events_for_serverRichard van der Hoff2018-10-092-7/+7
|\ \ \ \ | |/ / / |/| | | Optimisation for filter_events_for_server
| * | | Optimisation for filter_events_for_serverRichard van der Hoff2018-10-082-7/+7
|/ / / | | | | | | | | | | | | | | | | | | We're better off hashing just the event_id than the whole ((type, state_key), event_id) tuple - so use a dict instead of a set. Also, iteritems > items.
* | | Merge pull request #3995 from matrix-org/rav/no_deextrem_outliersRichard van der Hoff2018-10-042-33/+70
|\ \ \ | | | | | | | | Fix bug in forward_extremity update logic
| * | | actually exclude outliersRichard van der Hoff2018-10-031-1/+2
| | | |
| * | | fix newsfile nameRichard van der Hoff2018-10-031-0/+0
| | | |
| * | | remove debuggingRichard van der Hoff2018-10-021-10/+0
| | | |
| * | | Fix bug in forward_extremity update logicRichard van der Hoff2018-10-022-33/+79
| | | | | | | | | | | | | | | | | | | | An event does not stop being a forward_extremity just because an outlier or rejected event refers to it.
* | | | Merge branch 'master' into developAmber Brown2018-10-0463-66/+88
|\ \ \ \
| * | | | changelog v0.33.6 github/release-v0.33.6 release-v0.33.6Amber Brown2018-10-042-1/+9
| | | | |
| * | | | full versionAmber Brown2018-10-041-1/+1
| | | | |
| * | | | Merge pull request #4002 from matrix-org/rav/pin_prometheusRichard van der Hoff2018-10-032-1/+5
| |\ \ \ \ | | | | | | | | | | | | Pin to prometheus_client<0.4 to avoid renaming all of our metrics
| | * | | | Pin to prometheus_client<0.4 to avoid renaming all of our metricsRichard van der Hoff2018-10-032-1/+5
| |/ / / /
| * | | | Merge pull request #3998 from ↵Richard van der Hoff2018-10-031-1/+1
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | matrix-org/michaelkaye/build_docker_images_of_rc_tags Docker build all tags starting vX.Y.Z, including release candidates
| | * | | | Docker build all tags starting vX.Y.Z, inc RCsMichael Kaye2018-10-031-1/+1
| |/ / / / | | | | | | | | | | | | | | | | | | | | Note the regex must match the complete string anyway, so the leading ^ was useless.
| * | | | some changelog fixesAmber Brown2018-10-031-8/+4
| | | | |
| * | | | changelog v0.33.6rc1Amber Brown2018-10-0360-61/+75
| | | | |
| * | | | versionAmber Brown2018-10-031-1/+1
| |/ / /
* | | | Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetionErik Johnston2018-10-032-5/+3
|\ \ \ \ | | | | | | | | | | Fix handling of rejected threepid invites
| * | | | Sanitise error messages when user doesn't have permission to inviteErik Johnston2018-10-031-5/+2
| | | | |
| * | | | NewsfileErik Johnston2018-10-031-0/+1
| | | | |
| * | | | Fix handling of rejected threepid invitesErik Johnston2018-10-031-1/+1
| |/ / /
* | | | Merge pull request #3996 from matrix-org/erikj/fix_bg_iterationErik Johnston2018-10-032-2/+7
|\ \ \ \ | | | | | | | | | | Fix exception in background metrics collection
| * | | | NewsfileErik Johnston2018-10-031-0/+1
| | | | |
| * | | | Fix exception in background metrics collectionErik Johnston2018-10-031-2/+6
| |/ / / | | | | | | | | | | | | | | | | We attempted to iterate through a list on a separate thread without doing the necessary copying.
* | | | Merge pull request #3997 from matrix-org/erikj/fix_profile_error_handlingErik Johnston2018-10-032-6/+3
|\ \ \ \ | |/ / / |/| | | Fix exception handling in fetching remote profiles
| * | | NewsfileErik Johnston2018-10-031-0/+1
| | | |
| * | | Fix exception handling in fetching remote profilesErik Johnston2018-10-031-6/+2
|/ / /
* | | Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cacheErik Johnston2018-10-023-1/+47
|\ \ \ | | | | | | | | Fix bug when invalidating destination retry timings
| * | | Add testsErik Johnston2018-10-021-0/+45
| | | |
| * | | NewsfileErik Johnston2018-10-021-0/+1
| | | |
| * | | Fix bug when invalidating destination retry timingsErik Johnston2018-10-021-1/+1
| | | |
* | | | Remove Jenkins & other old dev junk (#3988)Amber Brown2018-10-0312-278/+4
|/ / /
* | | Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_errorRichard van der Hoff2018-10-022-2/+3
|\ \ \ | | | | | | | | Fix error handling for missing auth_event
| * \ \ Merge branch 'develop' into rav/fix_missing_create_event_errorRichard van der Hoff2018-10-0168-442/+779
| |\ \ \
| * | | | changelogRichard van der Hoff2018-09-261-0/+1
| | | | |
| * | | | Fix error handling for missing auth_eventRichard van der Hoff2018-09-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we were authorizing an event, if there was no `m.room.create` in its auth_events, we would raise a SynapseError with a cryptic message, which then meant that we would bail out of processing any incoming events, rather than storing a rejection for the faulty event and moving on. We should treat the absent event the same as any other auth failure, by raising an AuthError, so that the event is marked as rejected.
* | | | | Disable frozen dicts by default (#3987)Amber Brown2018-10-024-5/+18
| | | | |
* | | | | Merge pull request #3990 from matrix-org/erikj/fix_logging_lost_connectionErik Johnston2018-10-022-6/+22
|\ \ \ \ \ | | | | | | | | | | | | Fix error when logging incomplete requests
| * | | | | NewsfileErik Johnston2018-10-021-0/+1
| | | | | |
| * | | | | Fix error when logging incomplete requestsErik Johnston2018-10-021-6/+21
|/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a connection is lost before a request is read from Request, Twisted sets `method` (and `uri`) attributes to dummy values. These dummy values have incorrect types (i.e. they're not bytes), and so things like `__repr__` would raise an exception. To fix this we had a helper method to return the method with a consistent type.
* | | | | Merge pull request #3989 from matrix-org/rav/better_stacktracesRichard van der Hoff2018-10-023-22/+24
|\ \ \ \ \ | | | | | | | | | | | | Avoid reraise, to improve stacktraces
| * | | | | Avoid reraise, to improve stacktracesRichard van der Hoff2018-10-013-22/+24
| | |/ / / | |/| | |
* | | | | Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_stateErik Johnston2018-10-022-2/+3
|\ \ \ \ \ | |/ / / / |/| | | | Fix lazy loaded sync with rejected state events
| * | | | NewsfileErik Johnston2018-10-011-0/+1
| | | | |
| * | | | Fix lazy loaded sync with rejected state eventsErik Johnston2018-10-011-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In particular, we assume that the name and canonical alias events in the state have not been rejected. In practice this may not be the case (though we should probably think about fixing that) so lets ensure that we gracefully handle that case, rather than 404'ing the sync request like we do now.
* | | | | Update instructions to point to pip install (#3985)Amber Brown2018-10-024-61/+29
| | | | |
* | | | | Merge pull request #3968 from matrix-org/rav/fix_federation_errorsRichard van der Hoff2018-10-013-118/+47
|\ \ \ \ \ | | | | | | | | | | | | Fix exceptions when handling incoming transactions
| * \ \ \ \ Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errorsRichard van der Hoff2018-09-2731-38/+117
| |\ \ \ \ \
| * | | | | | Remove redundant, failing, testRichard van der Hoff2018-09-271-106/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This test didn't do what it claimed to do, and what it claimed to do was the same as test_cant_hide_direct_ancestors anyway. This stuff is tested by sytest anyway.
| * | | | | | changelogRichard van der Hoff2018-09-271-0/+1
| | | | | | |
| * | | | | | Include event when resolving state for missing prevsRichard van der Hoff2018-09-271-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we have a forward extremity for a room as `E`, and you receive `A`, `B`, s.t. `A -> B -> E`, and `B` also points to an unknown event `X`, then we need to do state res between `X` and `E`. When that happens, we need to make sure we include `X` in the state that goes into the state res alg. Fixes #3934.
| * | | | | | Include state from remote servers in pdu handlingRichard van der Hoff2018-09-271-4/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we've fetched state events from remote servers in order to resolve the state for a new event, we need to actually pass those events into resolve_events_with_factory (so that it can do the state res) and then persist the ones we need - otherwise other bits of the codebase get confused about why we have state groups pointing to non-existent events.
| * | | | | | Fix "unhashable type: 'list'" exception in federation handlingRichard van der Hoff2018-09-271-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | get_state_groups returns a map from state_group_id to a list of FrozenEvents, so was very much the wrong thing to be putting as one of the entries in the list passed to resolve_events_with_factory (which expects maps from (event_type, state_key) to event id). We actually want get_state_groups_ids().values() rather than get_state_groups(). This fixes the main problem in #3923, but there are other problems with this bit of code which get discovered once you do so.
* | | | | | | Fix userconsent on Python 3 (#3938)Amber Brown2018-10-026-96/+240
| |_|/ / / / |/| | | | |
* | | | | | Merge pull request #3933 from matrix-org/erikj/destination_retry_cacheErik Johnston2018-10-013-3/+45
|\ \ \ \ \ \ | | | | | | | | | | | | | | Add a five minute cache to get_destination_retry_timings
| * | | | | | Correctly match 'dict.pop' apiErik Johnston2018-10-011-3/+11
| | | | | | |
| * | | | | | Don't update eviction metrics on explicit removalErik Johnston2018-10-011-5/+0
| | | | | | |
| * | | | | | Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cacheRichard van der Hoff2018-09-2851-426/+696
| |\ \ \ \ \ \ | | | |/ / / / | | |/| | | |
| * | | | | | Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cacheRichard van der Hoff2018-09-266-187/+440
| |\ \ \ \ \ \
| * \ \ \ \ \ \ Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cacheRichard van der Hoff2018-09-2551-62/+128
| |\ \ \ \ \ \ \
| * | | | | | | | Fix ExpiringCache.__len__ to be accurateErik Johnston2018-09-211-12/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It used to try and produce an estimate, which was sometimes negative. This caused metrics to be sad, so lets always just calculate it from scratch.
| * | | | | | | | NewsfileErik Johnston2018-09-211-0/+1
| | | | | | | | |
| * | | | | | | | Add a five minute cache to get_destination_retry_timingsErik Johnston2018-09-212-1/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hopefully helps with #3931
| * | | | | | | | Make ExpiringCache slightly more performantErik Johnston2018-09-211-1/+5
| | | | | | | | |
* | | | | | | | | Further reduce the size of the docker image (#3972)Richard van der Hoff2018-10-012-25/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rewrite the dockerfile as a multistage build: this means we can get rid of a whole load of cruft which we don't need.
* | | | | | | | | add --no-admin flag to registration script (#3836)Bruno Windels2018-09-282-4/+15
| | | | | | | | |
* | | | | | | | | Merge pull request #3964 from matrix-org/rav/remove_localhost_checksRichard van der Hoff2018-09-283-30/+8
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | remove spurious federation checks on localhost
| * | | | | | | | | Fix complete fail to do the right thingRichard van der Hoff2018-09-282-1/+3
| | | | | | | | | |
| * | | | | | | | | changelogRichard van der Hoff2018-09-261-0/+1
| | | | | | | | | |
| * | | | | | | | | remove spurious federation checks on localhostRichard van der Hoff2018-09-261-31/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's really no point in checking for destinations called "localhost" because there is nothing stopping people creating other DNS entries which point to 127.0.0.1. The right fix for this is https://github.com/matrix-org/synapse/issues/3953. Blocking localhost, on the other hand, means that you get a surprise when trying to connect a test server on localhost to an existing server (with a 'normal' server_name).
* | | | | | | | | | Merge pull request #3980 from matrix-org/rav/remove_broken_cache_callRichard van der Hoff2018-09-282-1/+1
|\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | Remove redundant call to start_get_pdu_cache
| * | | | | | | | | | Remove redundant call to start_get_pdu_cacheRichard van der Hoff2018-09-282-1/+1
| | |_|_|_|/ / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I think this got forgotten in #3932. We were getting away with it because it was the last call in this function.
* / | | | | | | | | build python 3 docker images on circle CI (#3976)Jan Christian Grünhage2018-09-282-5/+12
|/ / / / / / / / /
* | | | | | | | | Merge pull request #3794 from matrix-org/erikj/faster_typingRichard van der Hoff2018-09-272-3/+20
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Improve performance of getting typing updates for replication
| * \ \ \ \ \ \ \ \ Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typingErik Johnston2018-09-19177-1345/+2545
| |\ \ \ \ \ \ \ \ \
| * | | | | | | | | | NewsfileErik Johnston2018-09-191-0/+1
| | | | | | | | | | |
| * | | | | | | | | | Improve performance of getting typing updates for replicationErik Johnston2018-09-051-3/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fetching the list of all new typing notifications involved iterating over all rooms and comparing their serial. Lets move to using a stream change cache, like we do for other streams.
* | | | | | | | | | | Merge pull request #3946 from matrix-org/michaelkaye/automate_docker_hub_uploadRichard van der Hoff2018-09-272-0/+27
|\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Build and push docker image to hub automatically
| * | | | | | | | | | | Do the changelog.d danceMichael Kaye2018-09-271-0/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Make a ":latest" tag, and a SHA1 commit ID one too.Michael Kaye2018-09-271-3/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Latest is horrible and makes debugging what has happened anywhere a nightmare. We push a latest because of demand for it, but we'll also push a SHA1 commit id so those wanting to know what they're running (and be able to roll back if required) can use those instead. Note that latest here is defined as "most recent master commit" not "most recent released version", as the actual semantics of making latest correct while still being able to build bugfixed releases of previous versions is just ARGH. So we define it as "master" not "latest release".
| * | | | | | | | | | | Also, don't run this job on any branchesMichael Kaye2018-09-271-1/+3
| | | | | | | | | | | |
| * | | | | | | | | | | Make username configurable in the UI tooMichael Kaye2018-09-271-1/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Fix typoMichael Kaye2018-09-271-1/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Build and push docker image to hubMichael Kaye2018-09-271-0/+11
| | | | | | | | | | | |
* | | | | | | | | | | | Merge pull request #3967 from matrix-org/rav/federation_handler_cleanupsRichard van der Hoff2018-09-273-4/+11
|\ \ \ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|_|/ / / / | |/| | | | | | | | | | Clarifications in FederationHandler
| * | | | | | | | | | | more commentsRichard van der Hoff2018-09-272-6/+3
| | | | | | | | | | | |
| * | | | | | | | | | | changelogRichard van der Hoff2018-09-271-0/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Clarifications in FederationHandlerRichard van der Hoff2018-09-271-2/+11
| |/ / / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add some comments on things that look a bit bogus * rename this `state` variable to avoid confusion with the `state` used elsewhere in this function. (There was no actual conflict, but it was a confusing bit of spaghetti.)
* | | | | | | | | | | Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_processAmber Brown2018-09-272-2/+7
|\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Run notify_app_services as a bg process
| * | | | | | | | | | | changelogRichard van der Hoff2018-09-261-0/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Run notify_app_services as a bg processRichard van der Hoff2018-09-261-2/+6
| | |_|_|/ / / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This ensures that its resource usage metrics get recorded somewhere rather than getting lost. (It also fixes an error when called from a nested logging context which completes before the bg process)
* | | | | | | | | | | Merge pull request #3970 from schnuffle/develop-py3Amber Brown2018-09-2719-19/+20
|\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Replaced all occurences of e.message with str(e)
| * | | | | | | | | | | Update 3970.bugfixAmber Brown2018-09-271-1/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Added changelog fragmentSchnuffle2018-09-271-0/+1
| | | | | | | | | | | |
| * | | | | | | | | | | Replaced all occurences of e.message with str(e)Schnuffle2018-09-2718-19/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Schnuffle <schnuffle@github.com>
* | | | | | | | | | | | Run our oldest supported configuration in CI (#3952)Amber Brown2018-09-274-13/+38
| | | | | | | | | | | |
* | | | | | | | | | | | Merge pull request #3961 from matrix-org/neilj/lock_mau_upsertsRichard van der Hoff2018-09-272-1/+5
|\ \ \ \ \ \ \ \ \ \ \ \ | |_|_|/ / / / / / / / / |/| | | | | | | | | | | fix #3854 MAU transaction errors
| * | | | | | | | | | | towncrierNeil Johnson2018-09-261-0/+1
| | | | | | | | | | | |
| * | | | | | | | | | | fix #3854Neil Johnson2018-09-261-1/+4
| | |_|_|_|_|_|_|/ / / | |/| | | | | | | | |
* | | | | | | | | | | Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2Richard van der Hoff2018-09-273-2/+17
|\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Logging improvements
| * | | | | | | | | | | Logging improvementsRichard van der Hoff2018-09-263-2/+17
| | |_|/ / / / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some logging tweaks to help with debugging incoming federation transactions
* | | | | | | | | | | Include eventid in log lines when processing incoming federation ↵Richard van der Hoff2018-09-276-57/+115
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | transactions (#3959) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.
* | | | | | | | | | | docstrings and unittests for storage.state (#3958)Richard van der Hoff2018-09-273-8/+62
| |_|/ / / / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | I spent ages trying to figure out how I was going mad...
* | | | | | | | | | Merge pull request #3963 from matrix-org/rav/get_state_for_room_docstringAmber Brown2018-09-272-4/+5
|\ \ \ \ \ \ \ \ \ \ | |/ / / / / / / / / |/| | | | | | | | | fix docstring for FederationClient.get_state_for_room
| * | | | | | | | | changelogRichard van der Hoff2018-09-261-0/+1
| | | | | | | | | |
| * | | | | | | | | fix docstring for FederationClient.get_state_for_roomRichard van der Hoff2018-09-261-4/+4
|/ / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | trivial fixes for docstring
* / / / / / / / / Reduce the load on our CI (#3957)Amber Brown2018-09-273-8/+30
|/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * changelog * reduce circleci config * plus a handy script * fix regex
* | | | | | | | Merge branch 'develop' into rav/hacky_cache_factor_fixRichard van der Hoff2018-09-266-315/+320
|\ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_lenRichard van der Hoff2018-09-263-20/+25
| |\ \ \ \ \ \ \ \ | | | |_|_|_|/ / / | | |/| | | | | | Fix ExpiringCache.__len__ to be accurate
| | * | | | | | | Log which cache is throwing exceptionsRichard van der Hoff2018-09-261-10/+17
| | | | | | | | |
| | * | | | | | | changelogRichard van der Hoff2018-09-261-0/+1
| | | | | | | | |
| | * | | | | | | Fix ExpiringCache.__len__ to be accurateErik Johnston2018-09-261-10/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It used to try and produce an estimate, which was sometimes negative. This caused metrics to be sad, so lets always just calculate it from scratch. (This appears to have been a longstanding bug, but one which has been made more of a problem by #3932 and #3933). (This was originally done by Erik as part of #3933. I'm cherry-picking it because really it's a fix in its own right)
| * | | | | | | | Merge pull request #3948 from matrix-org/rav/no_symlink_synctlAmber Brown2018-09-263-285/+285
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Move synctl into top dir to avoid a symlink
| | * | | | | | | | changelogRichard van der Hoff2018-09-251-0/+1
| | | | | | | | | |
| | * | | | | | | | Move synctl into top dir to avoid a symlinkRichard van der Hoff2018-09-252-285/+284
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | symlinks apparently break setuptools on python3 and alpine (https://bugs.python.org/issue31940), so let's stop using a symlink and just use the file directly.
* | | | | | | | | | Actuall set cache factors in workersErik Johnston2018-09-261-5/+15
|/ / / / / / / / /
* | | | | | | | | Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pduAmber Brown2018-09-263-24/+48
|\ \ \ \ \ \ \ \ \ | |_|/ / / / / / / |/| | | | | | | | Comments and interface cleanup for on_receive_pdu
| * | | | | | | | changelogRichard van der Hoff2018-09-201-0/+1
| | | | | | | | |
| * | | | | | | | Comments and interface cleanup for on_receive_pduRichard van der Hoff2018-09-202-24/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add some informative comments about what's going on here. Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of `get_missing` and use `sent_to_us_directly` consistently.
* | | | | | | | | Merge pull request #3911 from matrix-org/jcgruenhage/docker-support-python3Richard van der Hoff2018-09-253-2/+5
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | make python 3 work in the docker container
| * | | | | | | | | Update DockerfileJan Christian Grünhage2018-09-251-1/+0
| | | | | | | | | |
| * | | | | | | | | Update .dockerignoreJan Christian Grünhage2018-09-251-1/+0
| | | | | | | | | |
| * | | | | | | | | add changelogJan Christian Grünhage2018-09-201-0/+1
| | | | | | | | | |
| * | | | | | | | | make python 3 work in the docker containerJan Christian Grünhage2018-09-203-2/+6
| | | | | | | | | |
* | | | | | | | | | Update grafana dashboardRichard van der Hoff2018-09-251-175/+417
| |_|_|_|_|_|/ / / |/| | | | | | | |
* | | | | | | | | Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_cachesRichard van der Hoff2018-09-2517-39/+13
|\ \ \ \ \ \ \ \ \ | | |_|_|_|_|/ / / | |/| | | | | | | Fix some instances of ExpiringCache not expiring cache items
| * | | | | | | | NewsfileErik Johnston2018-09-211-0/+1
| | | | | | | | |
| * | | | | | | | Fix some instances of ExpiringCache not expiring cache itemsErik Johnston2018-09-2116-39/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.
* | | | | | | | | added "media-src: 'self'" to CSP for resources (#3578)Jérémy Farnaud2018-09-252-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Synapse doesn’t allow for media resources to be played directly from Chrome. It is a problem for users on other networks (e.g. IRC) communicating with Matrix users through a gateway. The gateway sends them the raw URL for the resource when a Matrix user uploads a video and the video cannot be played directly in Chrome using that URL. Chrome argues it is not authorized to play the video because of the Content Security Policy. Chrome checks for the "media-src" policy which is missing, and defauts to the "default-src" policy which is "none". As Synapse already sends "object-src: 'self'" I thought it wouldn’t be a problem to add "media-src: 'self'" to the CSP to fix this problem.
* | | | | | | | | Merge pull request #3925 from matrix-org/erikj/fix_producers_unregisteredRichard van der Hoff2018-09-252-14/+36
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | Fix spurious exceptions when client closes conncetion
| * | | | | | | | | NewsfileErik Johnston2018-09-201-0/+1
| | | | | | | | | |
| * | | | | | | | | Fix spurious exceptions when client closes conncetionErik Johnston2018-09-201-14/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a HTTP handler throws an exception while processing a request we automatically write a JSON error response. If the handler had already started writing a response twisted throws an exception. We should check for this case and simple abort the connection if there was an error after the response had started being written.
* | | | | | | | | | Merge pull request #3947 from matrix-org/rav/attr_versionRichard van der Hoff2018-09-252-1/+4
|\ \ \ \ \ \ \ \ \ \ | |_|_|_|_|/ / / / / |/| | | | | | | | | We require attrs 16.0.0
| * | | | | | | | | changelogRichard van der Hoff2018-09-251-0/+1
| | | | | | | | | |
| * | | | | | | | | We require attrs 16.0.0Richard van der Hoff2018-09-251-1/+3
|/ / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | Ref: https://github.com/matrix-org/synapse/issues/3945
* | | | | | | | | Only lazy load self-members on initial syncMatthew Hodgson2018-09-252-5/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | Given we have disabled lazy loading for incr syncs in #3840, we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
* | | | | | | | | Merge branch 'master' into developAmber Brown2018-09-253-3/+13
|\ \ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ \ Merge tag 'v0.33.5.1'Amber Brown2018-09-253-3/+13
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Internal Changes ---------------- - Fix incompatibility with older Twisted version in tests. Thanks @OlegGirko! ([\#3940](https://github.com/matrix-org/synapse/issues/3940))
| | * | | | | | | | | changelog v0.33.5.1 github/release-v0.33.5.1 release-v0.33.5.1Amber Brown2018-09-252-2/+10
| | | | | | | | | | |
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-11 13:57:28 +0000