summary refs log tree commit diff
path: root/tests (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'develop' of github.com:matrix-org/synapse into erikj/room_chunksErik Johnston2018-05-233-3/+85
|\
| * Reject attempts to send event before privacy consent is givenRichard van der Hoff2018-05-221-0/+1
| | | | | | | | | | | | Returns an M_CONSENT_NOT_GIVEN error (cf https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet given.
| * Move consent config parsing into ConsentConfigRichard van der Hoff2018-05-221-1/+1
| | | | | | | | turns out we need to reuse this, so it's better in the config class.
| * Send users a server notice about consentRichard van der Hoff2018-05-222-3/+9
| | | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.
| * Fix logcontext leak in HttpTransactionCacheRichard van der Hoff2018-05-211-0/+21
| | | | | | | | ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
| * Stop the transaction cache caching failuresRichard van der Hoff2018-05-211-0/+54
| | | | | | | | | | | | The transaction cache has some code which tries to stop it caching failures, but if the callback function failed straight away, then things would happen backwards and we'd end up with the failure stuck in the cache.
* | Make insert_* functions internal and reorder funcsErik Johnston2018-05-171-8/+8
| | | | | | | | | | This makes it clearer what the public interface is vs what subclasses need to implement.
* | Improve documentation and commentsErik Johnston2018-05-171-0/+4
| |
* | Begin adding implementing room chunksErik Johnston2018-05-172-0/+239
|/ | | | | | This commit adds the necessary tables and columns, as well as an implementation of an online topological sorting algorithm to maintain an absolute ordering of the room chunks.
* Use stream rather depth ordering for push actionsErik Johnston2018-05-111-2/+2
| | | | | | This simplifies things as it is, but will also allow us to change the way we traverse topologically without having to update the way push actions work.
* Merge pull request #3161 from NotAFile/remove-v1authRichard van der Hoff2018-05-034-13/+18
|\ | | | | Make Client-Server API return 403 for invalid token
| * Burminate v1authAdrian Tschira2018-04-304-13/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Merge pull request #3175 from matrix-org/erikj/escape_metric_valuesErik Johnston2018-05-031-1/+20
|\ \ | | | | | | Escape label values in prometheus metrics
| * | Escape label values in prometheus metricsErik Johnston2018-05-021-1/+20
| | |
* | | Merge branch 'develop' into rav/more_logcontext_leaksRichard van der Hoff2018-05-023-3/+47
|\ \ \
| * | | Fix a couple of logcontext leaks in unit testsRichard van der Hoff2018-05-022-3/+9
| |/ / | | | | | | | | | | | | | | | ... which were making other, innocent, tests, fail. Plus remove a spurious unittest.DEBUG which was making the output noisy.
| * / Fix incorrect reference to StringIORichard van der Hoff2018-05-021-0/+38
| |/ | | | | | | This was introduced in 4f2f5171
* | Remove spurious unittest.DEBUGRichard van der Hoff2018-05-021-1/+0
| |
* | Fix a class of logcontext leaksRichard van der Hoff2018-05-021-11/+56
|/ | | | | | | | | | | | | | | | So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the *result* of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
* Merge pull request #3108 from NotAFile/py3-six-urlparseRichard van der Hoff2018-04-302-10/+9
|\ | | | | Use six.moves.urlparse
| * Use six.moves.urlparseAdrian Tschira2018-04-152-10/+9
| | | | | | | | | | | | The imports were shuffled around a bunch in py3 Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Merge branch 'develop' into rav/deferred_timeoutRichard van der Hoff2018-04-275-5/+6
|\ \
| * \ Merge pull request #3109 from NotAFile/py3-tests-fixRichard van der Hoff2018-04-235-5/+6
| |\ \ | | | | | | | | Make tests py3 compatible
| | * | Make tests py3 compatibleAdrian Tschira2018-04-165-5/+6
| | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a mixed commit that fixes various small issues * print parentheses * 01 is invalid syntax (it was octal in py2) * [x for i in 1, 2] is invalid syntax * six moves Signed-off-by: Adrian Tschira <nota@notafile.com>
* | / Use deferred.addTimeout instead of time_bound_deferredRichard van der Hoff2018-04-231-33/+0
|/ / | | | | | | This doesn't feel like a wheel we need to reinvent.
* | Merge pull request #3104 from NotAFile/py3-unittest-configRichard van der Hoff2018-04-201-0/+4
|\ \ | | | | | | Add some more variables to the unittest config
| * | Add some more variables to the unittest configAdrian Tschira2018-04-151-0/+4
| |/ | | | | | | | | | | | | These worked accidentally before (python2 doesn't complain if you compare incompatible types) but under py3 this blows up spectacularly Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Merge pull request #3103 from NotAFile/py3-baseexcepton-messageRichard van der Hoff2018-04-201-6/+6
|\ \ | | | | | | Use str(e) instead of e.message
| * | Use str(e) instead of e.messageAdrian Tschira2018-04-151-6/+6
| |/ | | | | | | | | | | | | Doing this I learned e.message was pretty shortlived, added in 2.6, they realized it was a bad idea and deprecated it in 2.7 Signed-off-by: Adrian Tschira <nota@notafile.com>
* / Avoid creating events with huge numbers of prev_eventsRichard van der Hoff2018-04-161-0/+68
|/ | | | | | In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events.
* Fix testsErik Johnston2018-04-111-0/+1
|
* Merge pull request #2996 from krombel/allow_auto_join_roomsRichard van der Hoff2018-04-101-0/+1
|\ | | | | move handling of auto_join_rooms to RegisterHandler
| * move handling of auto_join_rooms to RegisterHandlerKrombel2018-03-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de>
* | Merge pull request #3061 from NotAFile/add-some-byte-stringsRichard van der Hoff2018-04-091-1/+1
|\ \ | | | | | | Add b prefixes to some strings that are bytes in py3
| * | Add b prefixes to some strings that are bytes in py3Adrian Tschira2018-04-041-1/+1
| |/ | | | | | | | | | | This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com>
* | Merge pull request #3016 from silkeh/improve-service-lookupsRichard van der Hoff2018-04-091-27/+2
|\ \ | | | | | | Improve handling of SRV records for federation connections
| * | Remove address resolution of hosts in SRV recordsSilke2018-04-041-27/+2
| |/ | | | | | | Signed-off-by: Silke Hofstra <silke@slxh.eu>
* / Fix overzealous cache invalidationRichard van der Hoff2018-04-051-0/+46
|/ | | | | Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate.
* s/replication_client/federation_client/Erik Johnston2018-03-139-20/+20
|
* s/replication_server/federation_serverErik Johnston2018-03-131-1/+1
|
* Fix testsErik Johnston2018-03-1310-21/+22
|
* Fix testsErik Johnston2018-03-132-10/+8
|
* Fix comment typoRichard van der Hoff2018-03-051-1/+1
|
* Test and fix find_first_stream_ordering_after_tsRichard van der Hoff2018-03-051-0/+67
| | | | It seemed to suffer from a bunch of off-by-one errors.
* Move RoomMemberHandler out of HandlersErik Johnston2018-03-011-1/+1
|
* Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actionsErik Johnston2018-02-262-5/+7
|\ | | | | Batch inserts into event_push_actions_staging
| * Fix unit testsErik Johnston2018-02-202-5/+7
| |
* | Fix testErik Johnston2018-02-201-1/+1
|/
* Fix unit testErik Johnston2018-02-151-4/+6
|
* Remove context.push_actionsErik Johnston2018-02-151-1/+4
|
* Merge pull request #2847 from matrix-org/erikj/separate_event_creationErik Johnston2018-02-062-8/+6
|\ | | | | Split event creation into a separate handler
| * s/_create_new_client_event/create_new_client_event/Erik Johnston2018-02-062-4/+4
| |
| * Updates testsErik Johnston2018-02-052-8/+6
| |
* | Store state groups separately from events (#2784)Erik Johnston2018-02-062-69/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* | Merge pull request #2844 from matrix-org/rav/evicted_metricsRichard van der Hoff2018-02-051-0/+12
|\ \ | |/ |/| montoring metrics for number of cache evictions
| * report metrics on number of cache evictionsRichard van der Hoff2018-02-051-0/+12
| |
* | Fix broken unit test for media storageErik Johnston2018-02-051-1/+6
| |
* | Merge pull request #2791 from matrix-org/erikj/media_storage_refactorErik Johnston2018-02-053-0/+109
|\ \ | | | | | | Ensure media is in local cache before thumbnailing
| * | Add unit testsErik Johnston2018-01-183-0/+109
| | |
* | | Factor out resolve_state_groups to a separate handlerRichard van der Hoff2018-02-011-1/+3
| |/ |/| | | | | | | | | We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.
* | Add tests for user directory searchRichard van der Hoff2018-01-272-0/+89
| |
* | Make it possible to run tests against postgresRichard van der Hoff2018-01-271-9/+34
| |
* | Create dbpool as normal in testsRichard van der Hoff2018-01-272-45/+19
| | | | | | | | | | | | | | | | ... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a bit of a mystery. For some reason this makes one of the tests run slightly slower, so bump the sleep(). Sorry.
* | Run on_new_connection for unit testsRichard van der Hoff2018-01-271-7/+17
| | | | | | | | | | Configure the connectionpool used for unit tests to run the `on_new_connection` function.
* | Remove unused/bitrotted MemoryDataStoreRichard van der Hoff2018-01-261-151/+13
| | | | | | | | This isn't used, and looks thoroughly bitrotted.
* | Remove spurious unittest.DEBUGRichard van der Hoff2018-01-261-1/+0
| |
* | Merge pull request #2821 from matrix-org/rav/matthew_test_fixesRichard van der Hoff2018-01-221-2/+6
|\ \ | | | | | | Matthew's fixes to the unit tests
| * | Matthew's fixes to the unit testsRichard van der Hoff2018-01-221-2/+6
| | | | | | | | | | | | Extracted from https://github.com/matrix-org/synapse/pull/2820
* | | Add federation_domain_whitelist option (#2820)Matthew Hodgson2018-01-221-0/+1
|/ / | | | | | | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
* | Merge pull request #2813 from matrix-org/matthew/registrations_require_3pidMatthew Hodgson2018-01-221-0/+1
|\ \ | | | | | | add registrations_require_3pid and allow_local_3pids
| * | mock registrations_require_3pidMatthew Hodgson2018-01-191-0/+1
| | |
* | | Merge pull request #2804 from matrix-org/erikj/file_consumerErik Johnston2018-01-181-0/+176
|\ \ \ | |/ / |/| / | |/ Add decent impl of a FileConsumer
| * Move test stuff to testsErik Johnston2018-01-181-8/+46
| |
| * Add decent impl of a FileConsumerErik Johnston2018-01-171-0/+138
| | | | | | | | | | | | | | | | | | | | Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind
* | Optimise LoggingContext creation and copyingRichard van der Hoff2018-01-162-15/+15
|/ | | | | | | | It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.
* Merge pull request #2765 from matrix-org/rav/fix_room_utsRichard van der Hoff2018-01-093-4/+9
|\ | | | | Fix flaky test_rooms UTs
| * enable twisted delayedcall debugging in UTsRichard van der Hoff2018-01-091-1/+5
| |
| * Remove dead codeRichard van der Hoff2018-01-091-3/+0
| | | | | | | | pointless function is pointless
| * Disable user_directory updates for UTsRichard van der Hoff2018-01-091-0/+4
| | | | | | | | | | Fix flakiness in the UTs caused by the user_directory being updated in the background
* | Fix broken config UTsRichard van der Hoff2018-01-091-1/+16
|/ | | | | https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests.
* Merge pull request #2723 from matrix-org/matthew/search-all-local-usersMatthew Hodgson2017-12-051-1/+13
|\ | | | | Add all local users to the user_directory and optionally search them
| * try make tests work a bit more...Matthew Hodgson2017-12-041-0/+12
| |
| * erik told me toMatthew Hodgson2017-12-041-1/+1
| |
* | Refactor UI auth implementationRichard van der Hoff2017-12-051-4/+7
|/ | | | | Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.
* Add extra space before inline commentLuke Barnard2017-11-161-1/+1
|
* Fix testsLuke Barnard2017-11-161-0/+1
|
* Fix tests for Store.__init__ updateRichard van der Hoff2017-11-135-11/+11
| | | | Fix the test to pass the right number of args to the Store constructors
* Fix appservice tests to account for new behavior of notify_interested_servicesIlya Zhuravlev2017-11-021-3/+12
|
* Fix tests for refresh_token removalRichard van der Hoff2017-11-011-3/+3
|
* fix testsRichard van der Hoff2017-10-311-0/+1
|
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-232-5/+5
| | | | what could possibly go wrong
* Validate group ids when parsingRichard van der Hoff2017-10-211-1/+23
| | | | | May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.
* Merge pull request #2546 from matrix-org/rav/remove_dead_event_injectorRichard van der Hoff2017-10-171-76/+0
|\ | | | | Remove dead class
| * Remove dead classRichard van der Hoff2017-10-171-76/+0
| | | | | | | | This isn't used anywhere.
* | Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldableRichard van der Hoff2017-10-171-0/+38
|\ \ | | | | | | Add some tests for make_deferred_yieldable
| * | Fix name of test_logcontextRichard van der Hoff2017-10-171-0/+0
| | | | | | | | | | | | The file under test is logcontext.py, not log_context.py
| * | Add some tests for make_deferred_yieldableRichard van der Hoff2017-10-171-0/+38
| |/
* / Fix testDavid Baker2017-10-171-0/+1
|/
* Merge pull request #2532 from matrix-org/rav/fix_linearizerRichard van der Hoff2017-10-111-4/+24
|\ | | | | Fix stackoverflow and logcontexts from linearizer
| * Fix stackoverflow and logcontexts from linearizerRichard van der Hoff2017-10-111-4/+24
| | | | | | | | | | | | | | 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.
* | Merge branch 'develop' into erikj/groups_mergedDavid Baker2017-10-023-13/+187
|\|
| * Do an AAAA lookup on SRV record targets (#2462)Richard van der Hoff2017-09-221-4/+22
| | | | | | | | | | Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405
| * Add some more tests for KeyringRichard van der Hoff2017-09-201-37/+140
| |
| * Fix concurrent server_key requests (#2458)Richard van der Hoff2017-09-191-3/+55
| | | | | | | | Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.
| * Add a config option to block all room invites (#2457)Richard van der Hoff2017-09-191-0/+1
| | | | | | | | | | - allows sysadmins the ability to lock down their servers so that people can't send their users room invites.
* | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_mergedErik Johnston2017-09-191-0/+74
|\|
| * Clean up and document handling of logcontexts in Keyring (#2452)Richard van der Hoff2017-09-181-0/+74
| | | | | | | | | | | | I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls...
* | Split out profile handler to fix testsErik Johnston2017-08-253-7/+5
| |
* | Add groups to sync streamErik Johnston2017-07-201-2/+2
|/
* Merge pull request #2309 from matrix-org/erikj/user_ip_replErik Johnston2017-07-062-6/+2
|\ | | | | Fix up user_ip replication commands
| * Fix upErik Johnston2017-06-272-6/+2
| |
* | Update testErik Johnston2017-07-041-1/+1
|/
* Fix up client ips to read from pending dataErik Johnston2017-06-271-4/+1
|
* Merge pull request #2281 from matrix-org/erikj/phone_home_statsErik Johnston2017-06-151-115/+0
|\ | | | | Fix phone home stats
| * Remove unhelpful testErik Johnston2017-06-151-115/+0
| |
* | Fix typing testsErik Johnston2017-05-261-0/+2
| |
* | Only load jinja2 templates onceErik Johnston2017-05-221-0/+1
| | | | | | | | | | Instead of every time a new email pusher is created, as loading jinja2 templates is slow.
* | Make get_state_groups_from_groups faster.Erik Johnston2017-05-171-1/+1
|/ | | | | | | | | Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.
* Do some logging when one-time-keys get claimedRichard van der Hoff2017-05-091-0/+34
| | | | | might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.
* Allow clients to upload one-time-keys with new sigsRichard van der Hoff2017-05-091-0/+98
| | | | | | | When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
* Add more granular event send metricsErik Johnston2017-05-022-3/+3
|
* Remove test of replication resourceErik Johnston2017-04-111-204/+0
|
* Merge pull request #2097 from matrix-org/erikj/repl_tcp_clientErik Johnston2017-04-051-8/+22
|\ | | | | Move to using TCP replication
| * Change slave storage to use new replication interfaceErik Johnston2017-04-031-8/+22
| | | | | | | | | | | | | | As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.
* | Merge pull request #2075 from matrix-org/erikj/cache_speedErik Johnston2017-03-313-2/+42
|\ \ | | | | | | Speed up cached function access
| * | Fix up testsErik Johnston2017-03-303-2/+42
| | |
* | | Merge pull request #2076 from matrix-org/erikj/as_perfErik Johnston2017-03-311-1/+3
|\ \ \ | |/ / |/| | Make AS's faster
| * | Compile the regex's used in ASesErik Johnston2017-03-281-1/+3
| |/
* / Fix the logcontext handling in the cache wrappers (#2077)Richard van der Hoff2017-03-301-0/+91
|/ | | | | | | The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.
* Merge pull request #2054 from matrix-org/erikj/user_iter_cursorErik Johnston2017-03-241-2/+2
|\ | | | | Reduce some CPU work on DB threads
| * User Cursor.__iter__ instead of fetchallErik Johnston2017-03-231-2/+2
| | | | | | | | This prevents unnecessary construction of lists
* | Merge pull request #2050 from matrix-org/rav/federation_backoffRichard van der Hoff2017-03-233-1/+5
|\ \ | | | | | | push federation retry limiter down to matrixfederationclient
| * | fix testsRichard van der Hoff2017-03-232-1/+3
| | |
| * | push federation retry limiter down to matrixfederationclientRichard van der Hoff2017-03-231-0/+2
| | | | | | | | | | | | | | | rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.
* | | Merge pull request #2052 from matrix-org/rav/time_bound_deferredRichard van der Hoff2017-03-231-0/+33
|\ \ \ | | | | | | | | Fix time_bound_deferred to throw the right exception
| * | | Fix time_bound_deferred to throw the right exceptionRichard van der Hoff2017-03-231-0/+33
| |/ / | | | | | | | | | | | | | | | Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.
* | | use jsonschema.FormatChecker for RoomID and UserID stringspik2017-03-232-6/+13
| | | | | | | | | | | | | | | | | | * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>
* | | Add valid filter tests, flake8, fix typopik2017-03-231-5/+49
| | | | | | | | | | | | Signed-off-by: pik <alexander.maznev@gmail.com>
* | | check_valid_filter using JSONSchemapik2017-03-231-1/+17
| |/ |/| | | | | | | | | * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>
* | Fix caching of remote servers' signature keysRichard van der Hoff2017-03-223-0/+153
|/ | | | | | | | | The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
* Merge pull request #2033 from matrix-org/erikj/repl_speedErik Johnston2017-03-211-1/+1
|\ | | | | Don't send the full event json over replication
| * Fix unit testErik Johnston2017-03-201-1/+1
| |
* | Stop preserve_fn leaking context into the reactorRichard van der Hoff2017-03-181-0/+61
|/ | | | | | | | Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.
* Modify test_user_sync so it doesn't look at last_active_ts over ↵Will Hunt2017-03-151-1/+1
| | | | last_user_sync_ts
* Aggregate event push actionsErik Johnston2017-02-141-0/+86
|
* Fix email push in pusher workerErik Johnston2017-02-022-12/+7
| | | | | | This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.
* Remote membership tests for replicationErik Johnston2017-01-311-43/+0
| | | | | This is because it now relies of the caches stream, which only works on postgres. We are trying to test with sqlite.
* Fix unit testsErik Johnston2017-01-301-4/+4
|
* Fix unit testsErik Johnston2017-01-261-0/+17
|
* Implement device key caching over federationErik Johnston2017-01-264-14/+27
|
* Add basic implementation of local device list changesErik Johnston2017-01-252-2/+5
|
* Insert delta of current_state_events to be more efficientErik Johnston2017-01-201-29/+0
|
* Derive current_state_events from state groupsErik Johnston2017-01-201-28/+17
|
* Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_splitErik Johnston2017-01-182-2/+25
|\ | | | | Optimise state resolution
| * Optimise state resolutionErik Johnston2017-01-172-2/+25
| |
* | Tidy up testErik Johnston2017-01-171-12/+12
| |
* | Speed up cache size calculationErik Johnston2017-01-172-16/+20
| | | | | | | | | | | | | | | | | | | | | | Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.
* | Add ExpiringCache testsErik Johnston2017-01-161-0/+84
| |
* | Optionally measure size of cache by sum of length of valuesErik Johnston2017-01-131-0/+25
|/
* Linearize updates to membership via PUT /state/Erik Johnston2017-01-092-3/+6
|
* Fix crash in url preview when html tag has no textMarcin Bachry2016-12-141-0/+50
| | | | Signed-off-by: Marcin Bachry <hegel666@gmail.com>
* Merge pull request #1620 from matrix-org/erikj/concurrent_room_accessErik Johnston2016-12-121-0/+70
|\ | | | | Limit the number of events that can be created on a given room concurrently
| * Add Limiter: limit concurrent access to resourceErik Johnston2016-11-101-0/+70
| |
* | Fix unit testsErik Johnston2016-12-091-0/+4
| |
* | Prevent user tokens being used as guest tokens (#1675)Richard van der Hoff2016-12-061-15/+78
| | | | | | | | | | Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.
* | test_preview: Fix incorrect wrappingJohannes Löthberg2016-12-051-2/+2
| | | | | | | | | | | | | | The old test expected an incorrect wrapping due to the preview function not using unicode properly, so it got the wrong length. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* | Make test_preview use unicode stringsJohannes Löthberg2016-12-051-68/+68
| | | | | | | | Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* | fix importsRichard van der Hoff2016-11-301-3/+0
| |
* | Rip out more refresh_token codeRichard van der Hoff2016-11-301-55/+0
| | | | | | | | | | | | | | | | We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.
* | Merge branch 'develop' into rav/no_more_refresh_tokensRichard van der Hoff2016-11-302-7/+5
|\ \
| * | Stop putting a time caveat on access tokensRichard van der Hoff2016-11-292-7/+5
| | | | | | | | | | | | | | | | | | | | | | | | The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.
* | | Stop generating refresh tokensRichard van der Hoff2016-11-281-8/+4
|/ / | | | | | | | | | | | | Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.
* | Shuffle receipt handler around so that worker apps don't need to load itErik Johnston2016-11-231-1/+1
| |
* | Fix testsErik Johnston2016-11-231-0/+1
| |
* | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2016-11-231-1/+169
|\ \ | | | | | | | | | erikj/split_out_fed_txn
| * | Review commentsKegan Dougal2016-11-221-10/+2
| | |
| * | More testsKegan Dougal2016-11-221-1/+56
| | |
| * | Add remaining testsKegan Dougal2016-11-211-4/+70
| | |
| * | Start adding some testsKegan Dougal2016-11-211-3/+37
| | |
| * | Move event_fields filtering to serialize_eventKegan Dougal2016-11-211-0/+21
| | | | | | | | | | | | Also make it an inclusive not exclusive filter, as the spec demands.
* | | Fix tests and flake8Erik Johnston2016-11-221-0/+1
| | |
* | | Fix testsErik Johnston2016-11-212-5/+19
|/ /
* / Clean transactions based on time. Add HttpTransactionCache tests.Kegan Dougal2016-11-141-0/+69
|/
* Merge pull request #1098 from matrix-org/markjh/bearer_tokenMark Haines2016-10-255-14/+29
|\ | | | | Allow clients to supply access_tokens as headers
| * Fix unit testsMark Haines2016-09-125-14/+29
| |
* | Merge pull request #1164 from pik/error-codesErik Johnston2016-10-192-41/+88
|\ \ | | | | | | Clarify Error codes for GET /filter/
| * | Refactor test_filter to use real DataStorepik2016-10-181-43/+81
| | | | | | | | | | | | * add tests for filter api errors
| * | Fix MockHttpRequest always returning M_UNKNOWN errcode in testingpik2016-10-141-1/+1
| | |
| * | Error codes for filtersAlexander Maznev2016-10-141-3/+12
| | | | | | | | | | | | | | | | | | * add tests Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
* | | Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_authErik Johnston2016-10-122-3/+15
|\ \ \ | | | | | | | | Implement pluggable password auth
| * | | Implement pluggable password authErik Johnston2016-10-032-3/+15
| |/ / | | | | | | | | | | | | | | | Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2016-10-117-41/+29
|\ \ \ | | | | | | | | | | | | erikj/replication_noop
| * | | rest/client/v1/register: use the correct requester in createUserPatrik Oldsberg2016-10-062-24/+14
| | | | | | | | | | | | | | | | Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
| * | | storage/appservice: make appservice methods only relying on the cache ↵Patrik Oldsberg2016-10-062-7/+4
| |/ / | | | | | | | | | synchronous
| * / Time out typing over federationErik Johnston2016-09-233-10/+11
| |/
* / Reduce DB hits for replicationErik Johnston2016-09-232-3/+3
|/ | | | | | | | | | | | | | Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.
* Fix unit testsMark Haines2016-09-061-0/+8
|
* Correctly handle the difference between prev and current stateErik Johnston2016-08-313-32/+20
|
* Don't notify for online -> online transitions.Erik Johnston2016-08-301-0/+47
| | | | | Specifically, if currently_active remains true then we should not notify if only the last active time changes.
* Merge pull request #1049 from matrix-org/erikj/presence_users_in_roomErik Johnston2016-08-302-41/+6
|\ | | | | Use state handler instead of get_users_in_room/get_joined_hosts
| * Fix unit testsErik Johnston2016-08-262-41/+6
| |
* | Merge remote-tracking branch 'origin/develop' into markjh/direct_to_deviceMark Haines2016-08-262-51/+78
|\|
| * Fix testsErik Johnston2016-08-261-5/+46
| |
| * Replace context.current_state with context.current_state_idsErik Johnston2016-08-252-48/+34
| |
* | More 0_0 in testsMark Haines2016-08-251-2/+2
|/
* Ensure invalidation list does not grow unboundedlyErik Johnston2016-08-192-0/+88
|
* Rename to on_invalidateErik Johnston2016-08-191-2/+2
|
* Make cache_context an explicit optionErik Johnston2016-08-191-2/+2
|
* Remove lru optionErik Johnston2016-08-191-1/+1
|
* Add concept of cache contextsErik Johnston2016-08-192-0/+179
|
* Make AppserviceHandler stream events from databaseErik Johnston2016-08-181-3/+6
| | | | | | | | | This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.
* Make notify_interested_services fasterErik Johnston2016-08-172-68/+54
|
* Clean up _ServiceQueuerErik Johnston2016-08-171-1/+1
|
* Update unit testsErik Johnston2016-08-171-0/+2
|
* Fix up preview URL API. Add tests.Erik Johnston2016-08-161-1/+79
| | | | | | | | | This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text.
* PEP8Richard van der Hoff2016-08-081-0/+1
|
* Fix login with m.login.tokenRichard van der Hoff2016-08-081-4/+49
| | | | | login with token (as used by CAS auth) was broken by 067596d, such that it always returned a 401.
* Test summarizationErik Johnston2016-08-041-0/+139
|
* PEP8Richard van der Hoff2016-08-031-2/+0
|
* Include device name in /keys/query responseRichard van der Hoff2016-08-031-0/+92
| | | | Add an 'unsigned' section which includes the device display name.
* keys/query: return all users which were asked forRichard van der Hoff2016-08-031-0/+46
| | | | | | In the situation where all of a user's devices get deleted, we want to indicate this to a client, so we want to return an empty dictionary, rather than nothing at all.
* Create separate methods for getting messages to pushMark Haines2016-07-281-0/+41
| | | | | | | | | | | for the email and http pushers rather than trying to make a single method that will work with their conflicting requirements. The http pusher needs to get the messages in ascending stream order, and doesn't want to miss a message. The email pusher needs to get the messages in descending timestamp order, and doesn't mind if it misses messages.
* Add `create_requester` functionRichard van der Hoff2016-07-264-26/+22
| | | | | Wrap the `Requester` constructor with a function which provides sensible defaults, and use it throughout
* Delete refresh tokens when deleting devicesRichard van der Hoff2016-07-261-0/+34
|
* Implement updating devicesRichard van der Hoff2016-07-262-0/+52
| | | | You can update the displayname of devices now.
* Implement deleting devicesRichard van der Hoff2016-07-262-6/+30
|
* Merge pull request #945 from matrix-org/rav/background_reindexRichard van der Hoff2016-07-251-5/+15
|\ | | | | Create index on user_ips in the background
| * PEP8Richard van der Hoff2016-07-251-1/+1
| |
| * Fix background_update testsRichard van der Hoff2016-07-251-6/+16
| | | | | | | | | | | | A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.
* | Slightly saner logging for unittestsRichard van der Hoff2016-07-251-4/+7
|/ | | | | | | 1. Give the handler used for logging in unit tests a formatter, so that the output is slightly more meaningful 2. Log some synapse.storage stuff, because it's useful.
* make /devices return a listRichard van der Hoff2016-07-211-4/+7
| | | | Turns out I specced this to return a list of devices rather than a dict of them
* Fix PEP8 errorsRichard van der Hoff2016-07-211-0/+1
|
* Implement GET /device/{deviceId}Richard van der Hoff2016-07-211-10/+27
|
* Fix PEP8 errorsRichard van der Hoff2016-07-202-3/+1
|
* Merge branch 'develop' into rav/get_devices_apiRichard van der Hoff2016-07-202-4/+19
|\ | | | | | | (pick up PR #938 in the hope of fixing the UTs)
| * Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ipsRichard van der Hoff2016-07-201-1/+9
| |\ | | | | | | Record device_id in client_ips
| | * Record device_id in client_ipsRichard van der Hoff2016-07-201-1/+9
| | | | | | | | | | | | | | | Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.
| * | Register a device_id in the /v2/register flow.Richard van der Hoff2016-07-201-3/+10
| |/ | | | | | | | | | | | | | | | | | | This doesn't cover *all* of the registration flows, but it does cover the most common ones: in particular: shared_secret registration, appservice registration, and normal user/pass registration. Pull device_id from the registration parameters. Register the device in the devices table. Associate the device with the returned access and refresh tokens. Profit.
* / GET /devices endpointRichard van der Hoff2016-07-203-12/+199
|/ | | | | | | implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.
* Merge pull request #932 from matrix-org/rav/register_refactorDavid Baker2016-07-201-1/+5
|\ | | | | Further registration refactoring
| * Further registration refactoringRichard van der Hoff2016-07-191-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.
* | Merge pull request #922 from matrix-org/erikj/file_api2Erik Johnston2016-07-202-6/+7
|\ \ | |/ |/| Feature: Add filter to /messages. Add 'contains_url' to filter.
| * Add filter param to /messages APIErik Johnston2016-07-142-6/+7
| |
* | Merge pull request #931 from matrix-org/rav/refactor_registerDavid Baker2016-07-191-1/+2
|\ \ | | | | | | rest/client/v2_alpha/register.py: Refactor flow somewhat.
| * | rest/client/v2_alpha/register.py: Refactor flow somewhat.Richard van der Hoff2016-07-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is meant to be an *almost* non-functional change, with the exception that it fixes what looks a lot like a bug in that it only calls `auth_handler.add_threepid` and `add_pusher` once instead of three times. The idea is to move the generation of the `access_token` out of `registration_handler.register`, because `access_token`s now require a device_id, and we only want to generate a device_id once registration has been successful.
* | | Add device_id support to /loginRichard van der Hoff2016-07-182-7/+89
|/ / | | | | | | | | | | | | | | | | | | | | | | | | Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.
* / Bug fix: expire invalid access tokensNegar Fazeli2016-07-132-3/+32
|/
* Add ReadWriteLockErik Johnston2016-07-051-0/+85
|
* Remove room name & alias testDavid Baker2016-06-241-41/+0
| | | | as get_room_name_and_alias is now gone
* Rework ldap integration with ldap3Martin Weinelt2016-06-221-0/+1
| | | | | | | | | | | | | | | | | | | Use the pure-python ldap3 library, which eliminates the need for a system dependency. Offer both a `search` and `simple_bind` mode, for more sophisticated ldap scenarios. - `search` tries to find a matching DN within the `user_base` while employing the `user_filter`, then tries the bind when a single matching DN was found. - `simple_bind` tries the bind against a specific DN by combining the localpart and `user_base` Offer support for STARTTLS on a plain connection. The configuration was changed to reflect these new possibilities. Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>