| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | | |
Fixes #3570
|
| | |/ / / / |
|
| | | | | | |
|
| |/ / / / |
|
| |\ \ \ \
| | | | |
| | | | | |
Run things as background processes
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This is more involved than it might otherwise be, because the current
implementation just drops its logcontexts and runs everything in the sentinel
context.
It turns out that we aren't actually using a bunch of the functionality here
(notably suppress_failures and the fact that Distributor.fire returns a
deferred), so the easiest way to fix this is actually by simplifying a bunch of
code.
|
| |/ / / / |
|
| | | | | |
|
| |\ \ \ \
| | | | |
| | | | | |
Refactor and optimze filter_events_for_server
|
| | | | | | |
|
| | | | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | | |
for easier unit testing.
|
| | | | | | |
|
| | | | | | |
|
| |\ \ \ \ \
| |_|_|_|/
|/| | | | |
Don't return unknown entities in get_entities_changed
|
| | |/ / / |
|
| | | | | |
|
| |/ / / |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Newer syntax
attr.ib(factory=dict)
is just a syntactic sugar for
attr.ib(default=attr.Factory(dict))
It was introduced in newest version of attrs package (18.1.0)
and doesn't work with older versions.
We should either require minimum version of attrs to be 18.1.0,
or use older (slightly more verbose) syntax.
Requiring newest version is not a good solution because
Linux distributions may have older version of attrs (17.4.0 in Fedora 28),
and requiring to build (and package)
newer version just to use newer syntactic sugar in only one test
is just too much.
It's much better to fix that test to use older syntax.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
|
| | |/
|/| |
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | | |
... as described at
https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.
|
| |\| |
| | |
| | | |
More server_name validation
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We need to do a bit more validation when we get a server name, but don't want
to be re-doing it all over the shop, so factor out a separate
parse_and_validate_server_name, and do the extra validation.
Also, use it to verify the server name in the config file.
|
| |/ /
| |
| |
| |
| |
| |
| |
| | |
a61738b removed a call to run_on_reactor from a unit test, but that call was
doing something useful, in making the function in question asynchronous.
Reinstate the call and add a check that we are testing what we wanted to be
testing.
|
| | |
| |
| |
| |
| | |
Make sure that server_names used in auth headers are sane, and reject them with
a sensible error code, before they disappear off into the depths of the system.
|
| |\ \
| | |
| | | |
Check the state of prev_events a bit more thoroughly when coming over federation
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| |/ / |
|
| |/ |
|
| |\
| |
| | |
remove dead filter_events_for_clients
|
| | |
| |
| |
| |
| |
| | |
This is only used by filter_events_for_client, so we can simplify the whole
thing by just doing one user at a time, and removing a dead storage function to
boot.
|
| | |
| |
| |
| |
| |
| |
| | |
When _get_state_for_groups is given a wildcard filter, just do a complete
lookup. Hopefully this will give us the best of both worlds by not filling up
the ram if we only need one or two keys, but also making the cache still work
for the federation reader usecase.
|
| | | |
|
| |\ \ |
|
| | | |
| | |
| | |
| | | |
We need power levels for this test to do what it is supposed to do.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Make it so that, before there is a power-levels event in the room, you need a
power level of at least 50 to send state.
Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192
|
| | |/
|/| |
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet
given.
|
| | |
| |
| |
| | |
turns out we need to reuse this, so it's better in the config class.
|
| | |
| |
| |
| |
| | |
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
|
| | |
| |
| |
| | |
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
|
| | |
| |
| |
| |
| |
| | |
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
|
| | |
| |
| |
| |
| |
| | |
This simplifies things as it is, but will also allow us to change the
way we traverse topologically without having to update the way push
actions work.
|
| | | |
|
| |/ |
|
| |\
| |
| | |
Make Client-Server API return 403 for invalid token
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This closes #2602
v1auth was created to account for the differences in status code between
the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid
tokens). However since those protocols were merged, this makes the r0
version/endpoint internally inconsistent, and violates the
specification for the r0 endpoint.
This might break clients that rely on this inconsistency with the
specification. This is said to affect the legacy angular reference
client. However, I feel that restoring parity with the spec is more
important. Either way, it is critical to inform developers about this
change, in case they rely on the illegal behaviour.
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Escape label values in prometheus metrics
|
| | | | |
|
| |\ \ \ |
|
| | |/ /
| | |
| | |
| | |
| | |
| | | |
... which were making other, innocent, tests, fail.
Plus remove a spurious unittest.DEBUG which was making the output noisy.
|
| | |/
| |
| |
| | |
This was introduced in 4f2f5171
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
So, it turns out that if you have a first `Deferred` `D1`, you can add a
callback which returns another `Deferred` `D2`, and `D2` must then complete
before any further callbacks on `D1` will execute (and later callbacks on `D1`
get the *result* of `D2` rather than `D2` itself).
So, `D1` might have `called=True` (as in, it has started running its
callbacks), but any new callbacks added to `D1` won't get run until `D2`
completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield`
will 'block'.
In conclusion: some of our assumptions in `logcontext` were invalid. We need to
make sure that we don't optimise out the logcontext juggling when this
situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
|
| |\
| |
| | |
Use six.moves.urlparse
|
| | |
| |
| |
| |
| |
| | |
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \ |
|
| | |\ \
| | | |
| | | | |
Make tests py3 compatible
|
| | | |/
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is a mixed commit that fixes various small issues
* print parentheses
* 01 is invalid syntax (it was octal in py2)
* [x for i in 1, 2] is invalid syntax
* six moves
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |/ /
| |
| |
| | |
This doesn't feel like a wheel we need to reinvent.
|
| |\ \
| | |
| | | |
Add some more variables to the unittest config
|
| | |/
| |
| |
| |
| |
| |
| | |
These worked accidentally before (python2 doesn't complain if you
compare incompatible types) but under py3 this blows up spectacularly
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Use str(e) instead of e.message
|
| | |/
| |
| |
| |
| |
| |
| | |
Doing this I learned e.message was pretty shortlived, added in 2.6,
they realized it was a bad idea and deprecated it in 2.7
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |/
|
|
|
|
| |
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
|
| | |
|
| |\
| |
| | |
move handling of auto_join_rooms to RegisterHandler
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work
This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.
This is related to issue #2725
Signed-Off-by: Matthias Kesler <krombel@krombel.de>
|
| |\ \
| | |
| | | |
Add b prefixes to some strings that are bytes in py3
|
| | |/
| |
| |
| |
| |
| | |
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
|
| |\ \
| | |
| | | |
Improve handling of SRV records for federation connections
|
| | |/
| |
| |
| | |
Signed-off-by: Silke Hofstra <silke@slxh.eu>
|
| |/
|
|
|
| |
Fixes an issue where a cache invalidation would invalidate *all* pending
entries, rather than just the entry that we intended to invalidate.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
It seemed to suffer from a bunch of off-by-one errors.
|
| | |
|
| |\
| |
| | |
Batch inserts into event_push_actions_staging
|
| | | |
|
| |/ |
|
| | |
|
| | |
|
| |\
| |
| | |
Split event creation into a separate handler
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
|
| |\ \
| |/
|/| |
montoring metrics for number of cache evictions
|
| | | |
|
| | | |
|
| |\ \
| | |
| | | |
Ensure media is in local cache before thumbnailing
|
| | | | |
|
| | |/
|/|
| |
| |
| |
| | |
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a
bit of a mystery.
For some reason this makes one of the tests run slightly slower, so bump the
sleep(). Sorry.
|
| | |
| |
| |
| |
| | |
Configure the connectionpool used for unit tests to run the `on_new_connection`
function.
|
| | |
| |
| |
| | |
This isn't used, and looks thoroughly bitrotted.
|
| | | |
|
| |\ \
| | |
| | | |
Matthew's fixes to the unit tests
|
| | | |
| | |
| | |
| | | |
Extracted from https://github.com/matrix-org/synapse/pull/2820
|
| |/ /
| |
| |
| |
| |
| | |
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
|
| |\ \
| | |
| | | |
add registrations_require_3pid and allow_local_3pids
|
| | | | |
|
| |\ \ \
| |/ /
|/| /
| |/ |
Add decent impl of a FileConsumer
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.
Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
|
| |/
|
|
|
|
|
|
| |
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
|
| |\
| |
| | |
Fix flaky test_rooms UTs
|
| | | |
|
| | |
| |
| |
| | |
pointless function is pointless
|
| | |
| |
| |
| |
| | |
Fix flakiness in the UTs caused by the user_directory being updated in the
background
|
| |/
|
|
|
| |
https://github.com/matrix-org/synapse/pull/2755 broke log-config generation,
which in turn broke the unit tests.
|
| |\
| |
| | |
Add all local users to the user_directory and optionally search them
|
| | | |
|
| | | |
|
| |/
|
|
|
| |
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
|
| | |
|
| | |
|
| |
|
|
| |
Fix the test to pass the right number of args to the Store constructors
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
what could possibly go wrong
|
| |
|
|
|
| |
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
|
| |\
| |
| | |
Remove dead class
|
| | |
| |
| |
| | |
This isn't used anywhere.
|
| |\ \
| | |
| | | |
Add some tests for make_deferred_yieldable
|
| | | |
| | |
| | |
| | | |
The file under test is logcontext.py, not log_context.py
|
| | |/ |
|
| |/ |
|
| |\
| |
| | |
Fix stackoverflow and logcontexts from linearizer
|
| | |
| |
| |
| |
| |
| |
| | |
1. make it not blow out the stack when there are more than 50 things waiting
for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505.
2. Make it not mess up the log contexts.
|
| |\| |
|
| | |
| |
| |
| |
| | |
Support SRV records which point at AAAA records, as well as A records.
Fixes https://github.com/matrix-org/synapse/issues/2405
|
| | | |
|
| | |
| |
| |
| | |
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
|
| | |
| |
| |
| |
| | |
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
|
| |\| |
|
| | |
| |
| |
| |
| |
| | |
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
|
| | | |
|
| |/ |
|
| |\
| |
| | |
Fix up user_ip replication commands
|
| | | |
|
| |/ |
|
| | |
|
| |\
| |
| | |
Fix phone home stats
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
|
| |/
|
|
|
|
|
|
|
| |
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
|
| |
|
|
|
| |
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
|
| |
|
|
|
|
|
| |
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
|
| | |
|
| | |
|
| |\
| |
| | |
Move to using TCP replication
|
| | |
| |
| |
| |
| |
| |
| | |
As the TCP replication uses a slightly different API and streams than
the HTTP replication.
This breaks HTTP replication.
|
| |\ \
| | |
| | | |
Speed up cached function access
|
| | | | |
|
| |\ \ \
| |/ /
|/| | |
Make AS's faster
|
| | |/ |
|
| |/
|
|
|
|
|
| |
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
|
| |\
| |
| | |
Reduce some CPU work on DB threads
|
| | |
| |
| |
| | |
This prevents unnecessary construction of lists
|
| |\ \
| | |
| | | |
push federation retry limiter down to matrixfederationclient
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | | |
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
|
| |\ \ \
| | | |
| | | | |
Fix time_bound_deferred to throw the right exception
|
| | |/ /
| | |
| | |
| | |
| | |
| | | |
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
|
| | | |
| | |
| | |
| | |
| | |
| | | |
* use a valid filter in rest/client/v2_alpha test
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| | | |
| | |
| | |
| | | |
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| | |/
|/|
| |
| |
| |
| | |
* add invalid filter tests
Signed-off-by: pik <alexander.maznev@gmail.com>
|
| |/
|
|
|
|
|
|
|
| |
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.
By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
|
| |\
| |
| | |
Don't send the full event json over replication
|
| | | |
|
| |/
|
|
|
|
|
|
| |
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
|
| |
|
|
| |
last_user_sync_ts
|
| | |
|
| |
|
|
|
|
| |
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
|
| |
|
|
|
| |
This is because it now relies of the caches stream, which only works on
postgres. We are trying to test with sqlite.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| | |
Optimise state resolution
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
|
| | | |
|
| |/ |
|
| | |
|
| |
|
|
| |
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
|
| |\
| |
| | |
Limit the number of events that can be created on a given room concurrently
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
|
| | |
| |
| |
| |
| |
| |
| | |
The old test expected an incorrect wrapping due to the preview function
not using unicode properly, so it got the wrong length.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| | |
| |
| |
| | |
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
|
| |/ /
| |
| |
| |
| |
| |
| | |
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
|
| | | |
|
| | | |
|
| |\ \
| | |
| | |
| | | |
erikj/split_out_fed_txn
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
Also make it an inclusive not exclusive filter, as the spec demands.
|
| | | | |
|
| |/ / |
|
| |/ |
|
| |\
| |
| | |
Allow clients to supply access_tokens as headers
|
| | | |
|
| |\ \
| | |
| | | |
Clarify Error codes for GET /filter/
|
| | | |
| | |
| | |
| | | |
* add tests for filter api errors
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
* add tests
Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
|
| |\ \ \
| | | |
| | | | |
Implement pluggable password auth
|
| | |/ /
| | |
| | |
| | |
| | |
| | | |
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
|
| |\ \ \
| | | |
| | | |
| | | | |
erikj/replication_noop
|
| | | | |
| | | |
| | | |
| | | | |
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
|
| | |/ /
| | |
| | |
| | | |
synchronous
|
| | |/ |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
|
| | |
|
| | |
|
| |
|
|
|
| |
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
|
| |\
| |
| | |
Use state handler instead of get_users_in_room/get_joined_hosts
|
| | | |
|
