| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| | |
Use six.moves.urlparse
|
| | |
| |
| |
| |
| |
| | |
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \ |
|
| | |\ \
| | | |
| | | | |
Make tests py3 compatible
|
| | | |/
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is a mixed commit that fixes various small issues
* print parentheses
* 01 is invalid syntax (it was octal in py2)
* [x for i in 1, 2] is invalid syntax
* six moves
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |/ /
| |
| |
| | |
This doesn't feel like a wheel we need to reinvent.
|
| |\ \
| | |
| | | |
Add some more variables to the unittest config
|
| | |/
| |
| |
| |
| |
| |
| | |
These worked accidentally before (python2 doesn't complain if you
compare incompatible types) but under py3 this blows up spectacularly
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Use str(e) instead of e.message
|
| | |/
| |
| |
| |
| |
| |
| | |
Doing this I learned e.message was pretty shortlived, added in 2.6,
they realized it was a bad idea and deprecated it in 2.7
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |/
|
|
|
|
| |
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
|
| | |
|
| |\
| |
| | |
move handling of auto_join_rooms to RegisterHandler
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work
This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.
This is related to issue #2725
Signed-Off-by: Matthias Kesler <krombel@krombel.de>
|
| |\ \
| | |
| | | |
Add b prefixes to some strings that are bytes in py3
|
| | |/
| |
| |
| |
| |
| | |
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Improve handling of SRV records for federation connections
|
| | |/
| |
| |
| | |
Signed-off-by: Silke Hofstra <silke@slxh.eu>
|
| |/
|
|
|
| |
Fixes an issue where a cache invalidation would invalidate *all* pending
entries, rather than just the entry that we intended to invalidate.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
It seemed to suffer from a bunch of off-by-one errors.
|
| | |
|
| |\
| |
| | |
Batch inserts into event_push_actions_staging
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |\
| |
| | |
Split event creation into a separate handler
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
|
| |\ \
| |/
|/| |
montoring metrics for number of cache evictions
|
| | | |
|
| | | |
|
| |\ \
| | |
| | | |
Ensure media is in local cache before thumbnailing
|
| | | | |
|
| | |/
|/|
| |
| |
| |
| | |
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a
bit of a mystery.
For some reason this makes one of the tests run slightly slower, so bump the
sleep(). Sorry.
|
| | |
| |
| |
| |
| | |
Configure the connectionpool used for unit tests to run the `on_new_connection`
function.
|
| | |
| |
| |
| | |
This isn't used, and looks thoroughly bitrotted.
|
| | | |
|
| |\ \
| | |
| | | |
Matthew's fixes to the unit tests
|
| | | |
| | |
| | |
| | | |
Extracted from https://github.com/matrix-org/synapse/pull/2820
|
| |/ /
| |
| |
| |
| |
| | |
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
|
| |\ \
| | |
| | | |
add registrations_require_3pid and allow_local_3pids
|
| | | | |
|
| |\ \ \
| |/ /
|/| /
| |/ |
Add decent impl of a FileConsumer
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.
Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
|
| |/
|
|
|
|
|
|
| |
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
|
| |\
| |
| | |
Fix flaky test_rooms UTs
|
| | | |
|
| | |
| |
| |
| | |
pointless function is pointless
|
| | |
| |
| |
| |
| | |
Fix flakiness in the UTs caused by the user_directory being updated in the
background
|
| |/
|
|
|
| |
https://github.com/matrix-org/synapse/pull/2755 broke log-config generation,
which in turn broke the unit tests.
|
| |\
| |
| | |
Add all local users to the user_directory and optionally search them
|
| | | |
|
| | | |
|
| |/
|
|
|
| |
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
|
| | |
|
| | |
|
| |
|
|
| |
Fix the test to pass the right number of args to the Store constructors
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
what could possibly go wrong
|
| |
|
|
|
| |
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
|
| |\
| |
| | |
Remove dead class
|
| | |
| |
| |
| | |
This isn't used anywhere.
|
| |\ \
| | |
| | | |
Add some tests for make_deferred_yieldable
|
| | | |
| | |
| | |
| | | |
The file under test is logcontext.py, not log_context.py
|
| | |/ |
|
| |/ |
|
| |\
| |
| | |
Fix stackoverflow and logcontexts from linearizer
|
| | |
| |
| |
| |
| |
| |
| | |
1. make it not blow out the stack when there are more than 50 things waiting
for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505.
2. Make it not mess up the log contexts.
|
| |\| |
|
| | |
| |
| |
| |
| | |
Support SRV records which point at AAAA records, as well as A records.
Fixes https://github.com/matrix-org/synapse/issues/2405
|
| | | |
|
| | |
| |
| |
| | |
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
|
| | |
| |
| |
| |
| | |
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
|
| |\| |
|
| | |
| |
| |
| |
| |
| | |
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
|
| | | |
|
| |/ |
|
| |\
| |
| | |
Fix up user_ip replication commands
|
| | | |
|
| |/ |
|
| | |
|
| |\
| |
| | |
Fix phone home stats
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
|
| |/
|
|
|
|
|
|
|
| |
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
|
| |
|
|
|
| |
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
|
| |
|
|
|
|
|
| |
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
|
| | |
|
| | |
|
| |\
| |
| | |
Move to using TCP replication
|
| | |
| |
| |
| |
| |
| |
| | |
As the TCP replication uses a slightly different API and streams than
the HTTP replication.
This breaks HTTP replication.
|
| |\ \
| | |
| | | |
Speed up cached function access
|
| | | | |
|
| |\ \ \
| |/ /
|/| | |
Make AS's faster
|
| | |/ |
|
| |/
|
|
|
|
|
| |
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
|
| |\
| |
| | |
Reduce some CPU work on DB threads
|
| | |
| |
| |
| | |
This prevents unnecessary construction of lists
|
| |\ \
| | |
| | | |
push federation retry limiter down to matrixfederationclient
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | | |
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
|
| |\ \ \
| | | |
| | | | |
Fix time_bound_deferred to throw the right exception
|
| | |/ /
| | |
| | |
| | |
| | |
| | | |
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
|
| | | |
| | |
| | |
| | |
| | |
| | | |
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| | | |
| | |
| | |
| | | |
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| | |/
|/|
| |
| |
| |
| | |
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| |/
|
|
|
|
|
|
|
| |
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
|
| |\
| |
| | |
Don't send the full event json over replication
|
| | | |
|
| |/
|
|
|
|
|
|
| |
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
|
| |
|
|
| |
last_user_sync_ts
|
| | |
|
| |
|
|
|
|
| |
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
|
| |
|
|
|
| |
This is because it now relies of the caches stream, which only works on
postgres. We are trying to test with sqlite.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Optimise state resolution
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
| |
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
|
| |\
| |
| | |
Limit the number of events that can be created on a given room concurrently
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
|
| | |
| |
| |
| |
| |
| |
| | |
The old test expected an incorrect wrapping due to the preview function
not using unicode properly, so it got the wrong length.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| | |
| |
| |
| | |
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
|
| |/ /
| |
| |
| |
| |
| |
| | |
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
|
| | | |
|
| | | |
|
| |\ \
| | |
| | |
| | | |
erikj/split_out_fed_txn
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
Also make it an inclusive not exclusive filter, as the spec demands.
|
| | | | |
|
| |/ / |
|
| |/ |
|
| |\
| |
| | |
Allow clients to supply access_tokens as headers
|
| | | |
|
| |\ \
| | |
| | | |
Clarify Error codes for GET /filter/
|
| | | |
| | |
| | |
| | | |
* add tests for filter api errors
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
* add tests
Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
|
| |\ \ \
| | | |
| | | | |
Implement pluggable password auth
|
| | |/ /
| | |
| | |
| | |
| | |
| | | |
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
|
| |\ \ \
| | | |
| | | |
| | | | |
erikj/replication_noop
|
| | | | |
| | | |
| | | |
| | | | |
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
|
| | |/ /
| | |
| | |
| | | |
synchronous
|
| | |/ |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
|
| | |
|
| | |
|
| |
|
|
|
| |
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
|
| |\
| |
| | |
Use state handler instead of get_users_in_room/get_joined_hosts
|
| | | |
|
| |\| |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
|
| | |
|
| |
|
|
|
| |
login with token (as used by CAS auth) was broken by 067596d, such that it
always returned a 401.
|
| | |
|
| | |
|
| |
|
|
| |
Add an 'unsigned' section which includes the device display name.
|
| |
|
|
|
|
| |
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
|
| |
|
|
|
|
|
|
|
|
|
| |
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
|
| |
|
|
|
| |
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
|
| | |
|
| |
|
|
| |
You can update the displayname of devices now.
|
| | |
|
| |\
| |
| | |
Create index on user_ips in the background
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
|
| |/
|
|
|
|
|
| |
1. Give the handler used for logging in unit tests a formatter, so that the
output is slightly more meaningful
2. Log some synapse.storage stuff, because it's useful.
|
| |
|
|
| |
Turns out I specced this to return a list of devices rather than a dict of them
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| | |
(pick up PR #938 in the hope of fixing the UTs)
|
| | |\
| | |
| | | |
Record device_id in client_ips
|
| | | |
| | |
| | |
| | |
| | | |
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
|
| |/
|
|
|
|
|
| |
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
|
| |\
| |
| | |
Further registration refactoring
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
|
| |\ \
| |/
|/| |
Feature: Add filter to /messages. Add 'contains_url' to filter.
|
| | | |
|
| |\ \
| | |
| | | |
rest/client/v2_alpha/register.py: Refactor flow somewhat.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
|
| |/ |
|
| | |
|
| |
|
|
| |
as get_room_name_and_alias is now gone
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.
Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
employing the `user_filter`, then tries the bind when a single
matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
localpart and `user_base`
Offer support for STARTTLS on a plain connection.
The configuration was changed to reflect these new possibilities.
Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
|
| |
|
|
| |
Fix the relevant unit test cases
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
|
| |
|
| |
Rather than storing them as UserID objects.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
|
| | |
|
| |
|
|
| |
Otherwise it tries to start the remote public room list updating looping call which breaks.
|
| |
|
|
|
|
| |
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
|
| |
|
|
|
| |
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| |/ |
|
| |\
| |
| | |
Add a slaved datastore for account data
|
