Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Merge pull request #3654 from matrix-org/rav/room_versions | Richard van der Hoff | 2018-08-08 | 1 | -0/+4 |
|\ | | | | | Support for room versioning | ||||
| * | Basic support for room versioning | Richard van der Hoff | 2018-08-03 | 1 | -0/+4 |
| | | | | | | | | | | | | | | | | This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event | ||||
* | | mock config.max_mau_value | Neil Johnson | 2018-08-08 | 1 | -0/+1 |
| | | |||||
* | | implement reserved users for mau limits | Neil Johnson | 2018-08-07 | 1 | -1/+58 |
| | | |||||
* | | add default mau_limits_reserved_threepids | Neil Johnson | 2018-08-06 | 1 | -0/+1 |
| | | |||||
* | | remove unused import | Neil Johnson | 2018-08-03 | 1 | -1/+1 |
| | | |||||
* | | bug fixes | Neil Johnson | 2018-08-03 | 2 | -8/+3 |
| | | |||||
* | | wip commit - tests failing | Neil Johnson | 2018-08-03 | 2 | -13/+13 |
| | | |||||
* | | fix (lots of) py3 test failures | Neil Johnson | 2018-08-03 | 2 | -1/+1 |
| | | |||||
* | | fix py3 test failure | Neil Johnson | 2018-08-03 | 1 | -0/+1 |
| | | |||||
* | | fix caching and tests | Neil Johnson | 2018-08-03 | 1 | -13/+37 |
| | | |||||
* | | do mau checks based on monthly_active_users table | Neil Johnson | 2018-08-02 | 3 | -42/+68 |
| | | |||||
* | | remove unused count_monthly_users | Neil Johnson | 2018-08-02 | 1 | -65/+0 |
| | | |||||
* | | fix test, update constructor call | Neil Johnson | 2018-08-02 | 1 | -1/+1 |
| | | |||||
* | | insertion into monthly_active_users | Neil Johnson | 2018-08-02 | 1 | -3/+63 |
| | | |||||
* | | Revert "change monthly_active_users table to be a single column" | Neil Johnson | 2018-08-02 | 1 | -3/+3 |
| | | | | | | | | This reverts commit ec716a35b219d147dee51733b55573952799a549. | ||||
* | | change monthly_active_users table to be a single column | Neil Johnson | 2018-08-01 | 1 | -3/+3 |
| | | |||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker | Neil Johnson | 2018-08-01 | 7 | -32/+310 |
|\| | |||||
| * | Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits | Neil Johnson | 2018-08-01 | 3 | -12/+184 |
| |\ | | | | | | | Initial impl of capping MAU | ||||
| | * | fix isort | Neil Johnson | 2018-08-01 | 1 | -1/+0 |
| | | | |||||
| | * | fix test for py3 | Neil Johnson | 2018-08-01 | 1 | -0/+4 |
| | | | |||||
| | * | Merge branch 'develop' of github.com:matrix-org/synapse into ↵ | Neil Johnson | 2018-08-01 | 2 | -1/+101 |
| | |\ | | | | | | | | | | | | | neilj/mau_sign_in_log_in_limits | ||||
| | * | | fix known broken test | Neil Johnson | 2018-08-01 | 1 | -25/+20 |
| | | | | |||||
| | * | | make count_monthly_users async synapse/handlers/auth.py | Neil Johnson | 2018-08-01 | 2 | -21/+28 |
| | | | | |||||
| | * | | coding style | Neil Johnson | 2018-07-31 | 2 | -3/+2 |
| | | | | |||||
| | * | | fix user_ips counting | Neil Johnson | 2018-07-31 | 1 | -9/+33 |
| | | | | |||||
| | * | | monthly_active_tests | Neil Johnson | 2018-07-30 | 1 | -0/+48 |
| | | | | |||||
| | * | | limit register and sign in on number of monthly users | Neil Johnson | 2018-07-30 | 2 | -1/+97 |
| | | | | |||||
| * | | | Python 3: Convert some unicode/bytes uses (#3569) | Amber Brown | 2018-08-02 | 2 | -19/+25 |
| | |/ | |/| | |||||
| * | | Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator | Richard van der Hoff | 2018-08-01 | 1 | -0/+101 |
| |\ \ | | | | | | | | | Rewrite cache list decorator | ||||
| | * | | fix invalidation | Richard van der Hoff | 2018-07-27 | 1 | -1/+41 |
| | | | | |||||
| | * | | Rewrite cache list decorator | Richard van der Hoff | 2018-07-27 | 1 | -0/+61 |
| | |/ | | | | | | | | | | | | | Because it was complicated and annoyed me. I suspect this will be more efficient too. | ||||
| * / | Remove pdu_failures from transactions | Travis Ralston | 2018-07-30 | 1 | -1/+0 |
| |/ | | | | | The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this. | ||||
* / | api into monthly_active_users table | Neil Johnson | 2018-07-31 | 1 | -0/+42 |
|/ | |||||
* | Merge branch 'develop' into matthew/filter_members | Matthew Hodgson | 2018-07-25 | 1 | -7/+30 |
|\ | |||||
| * | Fix unit tests | Richard van der Hoff | 2018-07-25 | 1 | -7/+30 |
| | | | | | | | | | | | | on_notifier_poke no longer runs synchonously, so we have to do a different hack to make sure that the replication data has been sent. Let's actually listen for its arrival. | ||||
* | | flake8 | Matthew Hodgson | 2018-07-25 | 1 | -12/+12 |
| | | |||||
* | | add tests for _get_some_state_from_cache | Matthew Hodgson | 2018-07-25 | 1 | -0/+150 |
| | | |||||
* | | incorporate more review | Matthew Hodgson | 2018-07-24 | 1 | -0/+9 |
| | | |||||
* | | Merge branch 'develop' into matthew/filter_members | Matthew Hodgson | 2018-07-23 | 7 | -87/+427 |
|\| | |||||
| * | Fix unit tests | Erik Johnston | 2018-07-23 | 2 | -15/+40 |
| | | |||||
| * | Make the rest of the .iterwhatever go away (#3562) | Amber Brown | 2018-07-21 | 1 | -2/+1 |
| | | |||||
| * | Test and fix support for cancellation in Linearizer | Richard van der Hoff | 2018-07-20 | 1 | -0/+31 |
| | | |||||
| * | Merge pull request #3571 from matrix-org/rav/limiter_fixes | Richard van der Hoff | 2018-07-20 | 2 | -70/+47 |
| |\ | | | | | | | A set of improvements to the Limiter | ||||
| | * | Combine Limiter and Linearizer | Richard van der Hoff | 2018-07-20 | 2 | -70/+47 |
| | | | | | | | | | | | | | | | Linearizer was effectively a Limiter with max_count=1, so rather than maintaining two sets of code, let's combine them. | ||||
| | * | Add a sleep to the Limiter to fix stack overflows. | Richard van der Hoff | 2018-07-20 | 1 | -4/+4 |
| | | | | | | | | | | | | Fixes #3570 | ||||
| * | | Admin API for creating new users (#3415) | Amber Brown | 2018-07-20 | 2 | -0/+308 |
| |/ | |||||
* | | make test work | Matthew Hodgson | 2018-07-19 | 1 | -37/+46 |
| | | |||||
* | | first cut of a UT for testing state store (untested) | Matthew Hodgson | 2018-07-19 | 1 | -0/+151 |
|/ | |||||
* | Merge pull request #3556 from matrix-org/rav/background_processes | Richard van der Hoff | 2018-07-19 | 1 | -52/+4 |
|\ | | | | | Run things as background processes | ||||
| * | Make Distributor run its processes as a background process | Richard van der Hoff | 2018-07-18 | 1 | -52/+4 |
| | | | | | | | | | | | | | | | | | | | | | | This is more involved than it might otherwise be, because the current implementation just drops its logcontexts and runs everything in the sentinel context. It turns out that we aren't actually using a bunch of the functionality here (notably suppress_failures and the fact that Distributor.fire returns a deferred), so the easiest way to fix this is actually by simplifying a bunch of code. | ||||
* | | Move v1-only APIs into their own module & isolate deprecated ones (#3460) | Amber Brown | 2018-07-19 | 5 | -86/+27 |
|/ | |||||
* | Fix visibility of events from erased users over federation | Richard van der Hoff | 2018-07-17 | 1 | -0/+63 |
| | |||||
* | Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server | Richard van der Hoff | 2018-07-17 | 1 | -0/+261 |
|\ | | | | | Refactor and optimze filter_events_for_server | ||||
| * | fix imports | Richard van der Hoff | 2018-07-17 | 1 | -0/+1 |
| | | |||||
| * | Add a test which profiles filter_events_for_server in a large room | Richard van der Hoff | 2018-07-16 | 1 | -2/+155 |
| | | |||||
| * | Move filter_events_for_server out of FederationHandler | Richard van der Hoff | 2018-07-16 | 1 | -0/+107 |
| | | | | | | | | for easier unit testing. | ||||
* | | Refactor REST API tests to use explicit reactors (#3351) | Amber Brown | 2018-07-17 | 10 | -931/+935 |
| | | |||||
* | | Add unit test | Erik Johnston | 2018-07-17 | 1 | -0/+16 |
| | | |||||
* | | Merge pull request #3530 from matrix-org/erikj/stream_cache | Amber Brown | 2018-07-17 | 1 | -3/+3 |
|\ \ | | | | | | | Don't return unknown entities in get_entities_changed | ||||
| * | | Fixup unit test | Erik Johnston | 2018-07-13 | 1 | -3/+3 |
| |/ | |||||
* | | Make auth & transactions more testable (#3499) | Amber Brown | 2018-07-14 | 1 | -1/+4 |
| | | |||||
* | | Fix unit tests | Richard van der Hoff | 2018-07-13 | 1 | -0/+1 |
|/ | |||||
* | Use more portable syntax using attrs package. | Oleg Girko | 2018-07-10 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | Newer syntax attr.ib(factory=dict) is just a syntactic sugar for attr.ib(default=attr.Factory(dict)) It was introduced in newest version of attrs package (18.1.0) and doesn't work with older versions. We should either require minimum version of attrs to be 18.1.0, or use older (slightly more verbose) syntax. Requiring newest version is not a good solution because Linux distributions may have older version of attrs (17.4.0 in Fedora 28), and requiring to build (and package) newer version just to use newer syntactic sugar in only one test is just too much. It's much better to fix that test to use older syntax. Signed-off-by: Oleg Girko <ol@infoserver.lv> | ||||
* | run isort | Amber Brown | 2018-07-09 | 76 | -216/+273 |
| | |||||
* | Merge remote-tracking branch 'hera/rav/server_acls' into develop | Neil Johnson | 2018-07-05 | 2 | -0/+57 |
|\ | |||||
| * | Implementation of server_acls | Richard van der Hoff | 2018-07-04 | 2 | -0/+57 |
| | | | | | | | | | | ... as described at https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw. | ||||
* | | Merge pull request #3483 from matrix-org/rav/more_server_name_validation | Erik Johnston | 2018-07-05 | 1 | -4/+13 |
|\| | | | | | More server_name validation | ||||
| * | More server_name validation | Richard van der Hoff | 2018-07-04 | 1 | -4/+13 |
| | | | | | | | | | | | | | | | | We need to do a bit more validation when we get a server name, but don't want to be re-doing it all over the shop, so factor out a separate parse_and_validate_server_name, and do the extra validation. Also, use it to verify the server name in the config file. | ||||
* | | Reinstate lost run_on_reactor in unit test | Richard van der Hoff | 2018-07-04 | 1 | -2/+15 |
|/ | | | | | | | | a61738b removed a call to run_on_reactor from a unit test, but that call was doing something useful, in making the function in question asynchronous. Reinstate the call and add a check that we are testing what we wanted to be testing. | ||||
* | Reject invalid server names (#3480) | Richard van der Hoff | 2018-07-03 | 2 | -0/+46 |
| | | | | | Make sure that server_names used in auth headers are sane, and reject them with a sensible error code, before they disappear off into the depths of the system. | ||||
* | Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking | Erik Johnston | 2018-06-29 | 2 | -1/+247 |
|\ | | | | | Check the state of prev_events a bit more thoroughly when coming over federation | ||||
| * | pep8 | Amber Brown | 2018-06-27 | 1 | -5/+2 |
| | | |||||
| * | cleanups | Amber Brown | 2018-06-27 | 1 | -1/+4 |
| | | |||||
| * | stylistic cleanup | Amber Brown | 2018-06-27 | 1 | -6/+18 |
| | | |||||
| * | cleanups | Amber Brown | 2018-06-27 | 1 | -7/+6 |
| | | |||||
| * | handle federation not telling us about prev_events | Amber Brown | 2018-06-27 | 2 | -1/+236 |
| | | |||||
* | | add test | Matthew Hodgson | 2018-06-28 | 1 | -0/+33 |
| | | |||||
* | | fix tests | Matthew Hodgson | 2018-06-28 | 1 | -3/+15 |
|/ | |||||
* | Better testing framework for homeserver-using things (#3446) | Amber Brown | 2018-06-27 | 2 | -0/+309 |
| | |||||
* | Merge pull request #3427 from matrix-org/erikj/remove_filters | Erik Johnston | 2018-06-22 | 1 | -8/+0 |
|\ | | | | | remove dead filter_events_for_clients | ||||
| * | remove dead filter_events_for_clients | Richard van der Hoff | 2018-06-12 | 1 | -8/+0 |
| | | | | | | | | | | | | This is only used by filter_events_for_client, so we can simplify the whole thing by just doing one user at a time, and removing a dead storage function to boot. | ||||
* | | Disable partial state group caching for wildcard lookups | Richard van der Hoff | 2018-06-22 | 1 | -6/+6 |
| | | | | | | | | | | | | | | When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase. | ||||
* | | Pass around the reactor explicitly (#3385) | Amber Brown | 2018-06-22 | 7 | -21/+30 |
| | | |||||
* | | Merge branch 'master' into develop | Richard van der Hoff | 2018-06-14 | 2 | -2/+165 |
|\ \ | |||||
| * | | Fix broken unit test | Richard van der Hoff | 2018-06-14 | 1 | -2/+14 |
| | | | | | | | | | | | | We need power levels for this test to do what it is supposed to do. | ||||
| * | | remove spurious debug | Richard van der Hoff | 2018-06-14 | 1 | -2/+0 |
| | | | |||||
| * | | Make default state_default 50 | Richard van der Hoff | 2018-06-14 | 1 | -0/+153 |
| | | | | | | | | | | | | | | | | | | | | | Make it so that, before there is a power-levels event in the room, you need a power level of at least 50 to send state. Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192 | ||||
* | | | Remove run_on_reactor (#3395) | Amber Brown | 2018-06-14 | 2 | -4/+0 |
| |/ |/| | |||||
* | | Port to sortedcontainers (with tests!) (#3332) | Amber Brown | 2018-06-06 | 1 | -0/+198 |
|/ | |||||
* | Put python's logs into Trial when running unit tests (#3319) | Amber Brown | 2018-06-04 | 1 | -13/+27 |
| | |||||
* | fix tests | Richard van der Hoff | 2018-05-29 | 1 | -0/+1 |
| | |||||
* | fix another dumb typo | Matthew Hodgson | 2018-05-29 | 1 | -2/+2 |
| | |||||
* | fix dumb typo | Matthew Hodgson | 2018-05-29 | 1 | -4/+4 |
| | |||||
* | Merge remote-tracking branch 'origin/develop' into 3218-official-prom | Amber Brown | 2018-05-22 | 4 | -5/+87 |
|\ | |||||
| * | Reject attempts to send event before privacy consent is given | Richard van der Hoff | 2018-05-22 | 1 | -0/+1 |
| | | | | | | | | | | | | Returns an M_CONSENT_NOT_GIVEN error (cf https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet given. | ||||
| * | Move consent config parsing into ConsentConfig | Richard van der Hoff | 2018-05-22 | 1 | -1/+1 |
| | | | | | | | | turns out we need to reuse this, so it's better in the config class. | ||||
| * | Send users a server notice about consent | Richard van der Hoff | 2018-05-22 | 2 | -3/+9 |
| | | | | | | | | | | When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so. | ||||
| * | Fix logcontext leak in HttpTransactionCache | Richard van der Hoff | 2018-05-21 | 1 | -0/+21 |
| | | | | | | | | ONE DAY I WILL PURGE THE WORLD OF THIS EVIL | ||||
| * | Stop the transaction cache caching failures | Richard van der Hoff | 2018-05-21 | 1 | -0/+54 |
| | | | | | | | | | | | | The transaction cache has some code which tries to stop it caching failures, but if the callback function failed straight away, then things would happen backwards and we'd end up with the failure stuck in the cache. | ||||
| * | Use stream rather depth ordering for push actions | Erik Johnston | 2018-05-11 | 1 | -2/+2 |
| | | | | | | | | | | | | This simplifies things as it is, but will also allow us to change the way we traverse topologically without having to update the way push actions work. | ||||
* | | rest of the changes | Amber Brown | 2018-05-21 | 2 | -192/+0 |
| | | |||||
* | | replacing portions | Amber Brown | 2018-05-21 | 1 | -0/+3 |
|/ | |||||
* | Merge pull request #3161 from NotAFile/remove-v1auth | Richard van der Hoff | 2018-05-03 | 4 | -13/+18 |
|\ | | | | | Make Client-Server API return 403 for invalid token | ||||
| * | Burminate v1auth | Adrian Tschira | 2018-04-30 | 4 | -13/+18 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Merge pull request #3175 from matrix-org/erikj/escape_metric_values | Erik Johnston | 2018-05-03 | 1 | -1/+20 |
|\ \ | | | | | | | Escape label values in prometheus metrics | ||||
| * | | Escape label values in prometheus metrics | Erik Johnston | 2018-05-02 | 1 | -1/+20 |
| | | | |||||
* | | | Merge branch 'develop' into rav/more_logcontext_leaks | Richard van der Hoff | 2018-05-02 | 3 | -3/+47 |
|\ \ \ | |||||
| * | | | Fix a couple of logcontext leaks in unit tests | Richard van der Hoff | 2018-05-02 | 2 | -3/+9 |
| |/ / | | | | | | | | | | | | | | | | ... which were making other, innocent, tests, fail. Plus remove a spurious unittest.DEBUG which was making the output noisy. | ||||
| * / | Fix incorrect reference to StringIO | Richard van der Hoff | 2018-05-02 | 1 | -0/+38 |
| |/ | | | | | | | This was introduced in 4f2f5171 | ||||
* | | Remove spurious unittest.DEBUG | Richard van der Hoff | 2018-05-02 | 1 | -1/+0 |
| | | |||||
* | | Fix a class of logcontext leaks | Richard van der Hoff | 2018-05-02 | 1 | -11/+56 |
|/ | | | | | | | | | | | | | | | | So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the *result* of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`. | ||||
* | Merge pull request #3108 from NotAFile/py3-six-urlparse | Richard van der Hoff | 2018-04-30 | 2 | -10/+9 |
|\ | | | | | Use six.moves.urlparse | ||||
| * | Use six.moves.urlparse | Adrian Tschira | 2018-04-15 | 2 | -10/+9 |
| | | | | | | | | | | | | The imports were shuffled around a bunch in py3 Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Merge branch 'develop' into rav/deferred_timeout | Richard van der Hoff | 2018-04-27 | 5 | -5/+6 |
|\ \ | |||||
| * \ | Merge pull request #3109 from NotAFile/py3-tests-fix | Richard van der Hoff | 2018-04-23 | 5 | -5/+6 |
| |\ \ | | | | | | | | | Make tests py3 compatible | ||||
| | * | | Make tests py3 compatible | Adrian Tschira | 2018-04-16 | 5 | -5/+6 |
| | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a mixed commit that fixes various small issues * print parentheses * 01 is invalid syntax (it was octal in py2) * [x for i in 1, 2] is invalid syntax * six moves Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | / | Use deferred.addTimeout instead of time_bound_deferred | Richard van der Hoff | 2018-04-23 | 1 | -33/+0 |
|/ / | | | | | | | This doesn't feel like a wheel we need to reinvent. | ||||
* | | Merge pull request #3104 from NotAFile/py3-unittest-config | Richard van der Hoff | 2018-04-20 | 1 | -0/+4 |
|\ \ | | | | | | | Add some more variables to the unittest config | ||||
| * | | Add some more variables to the unittest config | Adrian Tschira | 2018-04-15 | 1 | -0/+4 |
| |/ | | | | | | | | | | | | | These worked accidentally before (python2 doesn't complain if you compare incompatible types) but under py3 this blows up spectacularly Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Merge pull request #3103 from NotAFile/py3-baseexcepton-message | Richard van der Hoff | 2018-04-20 | 1 | -6/+6 |
|\ \ | | | | | | | Use str(e) instead of e.message | ||||
| * | | Use str(e) instead of e.message | Adrian Tschira | 2018-04-15 | 1 | -6/+6 |
| |/ | | | | | | | | | | | | | Doing this I learned e.message was pretty shortlived, added in 2.6, they realized it was a bad idea and deprecated it in 2.7 Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* / | Avoid creating events with huge numbers of prev_events | Richard van der Hoff | 2018-04-16 | 1 | -0/+68 |
|/ | | | | | | In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events. | ||||
* | Fix tests | Erik Johnston | 2018-04-11 | 1 | -0/+1 |
| | |||||
* | Merge pull request #2996 from krombel/allow_auto_join_rooms | Richard van der Hoff | 2018-04-10 | 1 | -0/+1 |
|\ | | | | | move handling of auto_join_rooms to RegisterHandler | ||||
| * | move handling of auto_join_rooms to RegisterHandler | Krombel | 2018-03-14 | 1 | -0/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de> | ||||
* | | Merge pull request #3061 from NotAFile/add-some-byte-strings | Richard van der Hoff | 2018-04-09 | 1 | -1/+1 |
|\ \ | | | | | | | Add b prefixes to some strings that are bytes in py3 | ||||
| * | | Add b prefixes to some strings that are bytes in py3 | Adrian Tschira | 2018-04-04 | 1 | -1/+1 |
| |/ | | | | | | | | | | | This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com> | ||||
* | | Merge pull request #3016 from silkeh/improve-service-lookups | Richard van der Hoff | 2018-04-09 | 1 | -27/+2 |
|\ \ | | | | | | | Improve handling of SRV records for federation connections | ||||
| * | | Remove address resolution of hosts in SRV records | Silke | 2018-04-04 | 1 | -27/+2 |
| |/ | | | | | | | Signed-off-by: Silke Hofstra <silke@slxh.eu> | ||||
* / | Fix overzealous cache invalidation | Richard van der Hoff | 2018-04-05 | 1 | -0/+46 |
|/ | | | | | Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate. | ||||
* | s/replication_client/federation_client/ | Erik Johnston | 2018-03-13 | 9 | -20/+20 |
| | |||||
* | s/replication_server/federation_server | Erik Johnston | 2018-03-13 | 1 | -1/+1 |
| | |||||
* | Fix tests | Erik Johnston | 2018-03-13 | 10 | -21/+22 |
| | |||||
* | Fix tests | Erik Johnston | 2018-03-13 | 2 | -10/+8 |
| | |||||
* | Fix comment typo | Richard van der Hoff | 2018-03-05 | 1 | -1/+1 |
| | |||||
* | Test and fix find_first_stream_ordering_after_ts | Richard van der Hoff | 2018-03-05 | 1 | -0/+67 |
| | | | | It seemed to suffer from a bunch of off-by-one errors. | ||||
* | Move RoomMemberHandler out of Handlers | Erik Johnston | 2018-03-01 | 1 | -1/+1 |
| | |||||
* | Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actions | Erik Johnston | 2018-02-26 | 2 | -5/+7 |
|\ | | | | | Batch inserts into event_push_actions_staging | ||||
| * | Fix unit tests | Erik Johnston | 2018-02-20 | 2 | -5/+7 |
| | | |||||
* | | Fix test | Erik Johnston | 2018-02-20 | 1 | -1/+1 |
|/ | |||||
* | Fix unit test | Erik Johnston | 2018-02-15 | 1 | -4/+6 |
| | |||||
* | Remove context.push_actions | Erik Johnston | 2018-02-15 | 1 | -1/+4 |
| | |||||
* | Merge pull request #2847 from matrix-org/erikj/separate_event_creation | Erik Johnston | 2018-02-06 | 2 | -8/+6 |
|\ | | | | | Split event creation into a separate handler | ||||
| * | s/_create_new_client_event/create_new_client_event/ | Erik Johnston | 2018-02-06 | 2 | -4/+4 |
| | | |||||
| * | Updates tests | Erik Johnston | 2018-02-05 | 2 | -8/+6 |
| | | |||||
* | | Store state groups separately from events (#2784) | Erik Johnston | 2018-02-06 | 2 | -69/+89 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints | ||||
* | | Merge pull request #2844 from matrix-org/rav/evicted_metrics | Richard van der Hoff | 2018-02-05 | 1 | -0/+12 |
|\ \ | |/ |/| | montoring metrics for number of cache evictions | ||||
| * | report metrics on number of cache evictions | Richard van der Hoff | 2018-02-05 | 1 | -0/+12 |
| | | |||||
* | | Fix broken unit test for media storage | Erik Johnston | 2018-02-05 | 1 | -1/+6 |
| | | |||||
* | | Merge pull request #2791 from matrix-org/erikj/media_storage_refactor | Erik Johnston | 2018-02-05 | 3 | -0/+109 |
|\ \ | | | | | | | Ensure media is in local cache before thumbnailing | ||||
| * | | Add unit tests | Erik Johnston | 2018-01-18 | 3 | -0/+109 |
| | | | |||||
* | | | Factor out resolve_state_groups to a separate handler | Richard van der Hoff | 2018-02-01 | 1 | -1/+3 |
| |/ |/| | | | | | | | | | We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer. | ||||
* | | Add tests for user directory search | Richard van der Hoff | 2018-01-27 | 2 | -0/+89 |
| | | |||||
* | | Make it possible to run tests against postgres | Richard van der Hoff | 2018-01-27 | 1 | -9/+34 |
| | | |||||
* | | Create dbpool as normal in tests | Richard van der Hoff | 2018-01-27 | 2 | -45/+19 |
| | | | | | | | | | | | | | | | | ... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a bit of a mystery. For some reason this makes one of the tests run slightly slower, so bump the sleep(). Sorry. | ||||
* | | Run on_new_connection for unit tests | Richard van der Hoff | 2018-01-27 | 1 | -7/+17 |
| | | | | | | | | | | Configure the connectionpool used for unit tests to run the `on_new_connection` function. | ||||
* | | Remove unused/bitrotted MemoryDataStore | Richard van der Hoff | 2018-01-26 | 1 | -151/+13 |
| | | | | | | | | This isn't used, and looks thoroughly bitrotted. | ||||
* | | Remove spurious unittest.DEBUG | Richard van der Hoff | 2018-01-26 | 1 | -1/+0 |
| | | |||||
* | | Merge pull request #2821 from matrix-org/rav/matthew_test_fixes | Richard van der Hoff | 2018-01-22 | 1 | -2/+6 |
|\ \ | | | | | | | Matthew's fixes to the unit tests | ||||
| * | | Matthew's fixes to the unit tests | Richard van der Hoff | 2018-01-22 | 1 | -2/+6 |
| | | | | | | | | | | | | Extracted from https://github.com/matrix-org/synapse/pull/2820 | ||||
* | | | Add federation_domain_whitelist option (#2820) | Matthew Hodgson | 2018-01-22 | 1 | -0/+1 |
|/ / | | | | | | | | | | | Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network | ||||
* | | Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid | Matthew Hodgson | 2018-01-22 | 1 | -0/+1 |
|\ \ | | | | | | | add registrations_require_3pid and allow_local_3pids | ||||
| * | | mock registrations_require_3pid | Matthew Hodgson | 2018-01-19 | 1 | -0/+1 |
| | | | |||||
* | | | Merge pull request #2804 from matrix-org/erikj/file_consumer | Erik Johnston | 2018-01-18 | 1 | -0/+176 |
|\ \ \ | |/ / |/| / | |/ | Add decent impl of a FileConsumer | ||||
| * | Move test stuff to tests | Erik Johnston | 2018-01-18 | 1 | -8/+46 |
| | | |||||
| * | Add decent impl of a FileConsumer | Erik Johnston | 2018-01-17 | 1 | -0/+138 |
| | | | | | | | | | | | | | | | | | | | | Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind | ||||
* | | Optimise LoggingContext creation and copying | Richard van der Hoff | 2018-01-16 | 2 | -15/+15 |
|/ | | | | | | | | It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit. | ||||
* | Merge pull request #2765 from matrix-org/rav/fix_room_uts | Richard van der Hoff | 2018-01-09 | 3 | -4/+9 |
|\ | | | | | Fix flaky test_rooms UTs | ||||
| * | enable twisted delayedcall debugging in UTs | Richard van der Hoff | 2018-01-09 | 1 | -1/+5 |
| | | |||||
| * | Remove dead code | Richard van der Hoff | 2018-01-09 | 1 | -3/+0 |
| | | | | | | | | pointless function is pointless | ||||
| * | Disable user_directory updates for UTs | Richard van der Hoff | 2018-01-09 | 1 | -0/+4 |
| | | | | | | | | | | Fix flakiness in the UTs caused by the user_directory being updated in the background | ||||
* | | Fix broken config UTs | Richard van der Hoff | 2018-01-09 | 1 | -1/+16 |
|/ | | | | | https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests. | ||||
* | Merge pull request #2723 from matrix-org/matthew/search-all-local-users | Matthew Hodgson | 2017-12-05 | 1 | -1/+13 |
|\ | | | | | Add all local users to the user_directory and optionally search them | ||||
| * | try make tests work a bit more... | Matthew Hodgson | 2017-12-04 | 1 | -0/+12 |
| | | |||||
| * | erik told me to | Matthew Hodgson | 2017-12-04 | 1 | -1/+1 |
| | | |||||
* | | Refactor UI auth implementation | Richard van der Hoff | 2017-12-05 | 1 | -4/+7 |
|/ | | | | | Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper. | ||||
* | Add extra space before inline comment | Luke Barnard | 2017-11-16 | 1 | -1/+1 |
| | |||||
* | Fix tests | Luke Barnard | 2017-11-16 | 1 | -0/+1 |
| | |||||
* | Fix tests for Store.__init__ update | Richard van der Hoff | 2017-11-13 | 5 | -11/+11 |
| | | | | Fix the test to pass the right number of args to the Store constructors | ||||
* | Fix appservice tests to account for new behavior of notify_interested_services | Ilya Zhuravlev | 2017-11-02 | 1 | -3/+12 |
| | |||||
* | Fix tests for refresh_token removal | Richard van der Hoff | 2017-11-01 | 1 | -3/+3 |
| | |||||
* | fix tests | Richard van der Hoff | 2017-10-31 | 1 | -0/+1 |
| | |||||
* | replace 'except:' with 'except Exception:' | Richard van der Hoff | 2017-10-23 | 2 | -5/+5 |
| | | | | what could possibly go wrong | ||||
* | Validate group ids when parsing | Richard van der Hoff | 2017-10-21 | 1 | -1/+23 |
| | | | | | May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place. | ||||
* | Merge pull request #2546 from matrix-org/rav/remove_dead_event_injector | Richard van der Hoff | 2017-10-17 | 1 | -76/+0 |
|\ | | | | | Remove dead class | ||||
| * | Remove dead class | Richard van der Hoff | 2017-10-17 | 1 | -76/+0 |
| | | | | | | | | This isn't used anywhere. | ||||
* | | Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldable | Richard van der Hoff | 2017-10-17 | 1 | -0/+38 |
|\ \ | | | | | | | Add some tests for make_deferred_yieldable | ||||
| * | | Fix name of test_logcontext | Richard van der Hoff | 2017-10-17 | 1 | -0/+0 |
| | | | | | | | | | | | | The file under test is logcontext.py, not log_context.py | ||||
| * | | Add some tests for make_deferred_yieldable | Richard van der Hoff | 2017-10-17 | 1 | -0/+38 |
| |/ | |||||
* / | Fix test | David Baker | 2017-10-17 | 1 | -0/+1 |
|/ | |||||
* | Merge pull request #2532 from matrix-org/rav/fix_linearizer | Richard van der Hoff | 2017-10-11 | 1 | -4/+24 |
|\ | | | | | Fix stackoverflow and logcontexts from linearizer | ||||
| * | Fix stackoverflow and logcontexts from linearizer | Richard van der Hoff | 2017-10-11 | 1 | -4/+24 |
| | | | | | | | | | | | | | | 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts. | ||||
* | | Merge branch 'develop' into erikj/groups_merged | David Baker | 2017-10-02 | 3 | -13/+187 |
|\| | |||||
| * | Do an AAAA lookup on SRV record targets (#2462) | Richard van der Hoff | 2017-09-22 | 1 | -4/+22 |
| | | | | | | | | | | Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405 | ||||
| * | Add some more tests for Keyring | Richard van der Hoff | 2017-09-20 | 1 | -37/+140 |
| | | |||||
| * | Fix concurrent server_key requests (#2458) | Richard van der Hoff | 2017-09-19 | 1 | -3/+55 |
| | | | | | | | | Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time. | ||||
| * | Add a config option to block all room invites (#2457) | Richard van der Hoff | 2017-09-19 | 1 | -0/+1 |
| | | | | | | | | | | - allows sysadmins the ability to lock down their servers so that people can't send their users room invites. | ||||
* | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged | Erik Johnston | 2017-09-19 | 1 | -0/+74 |
|\| | |||||
| * | Clean up and document handling of logcontexts in Keyring (#2452) | Richard van der Hoff | 2017-09-18 | 1 | -0/+74 |
| | | | | | | | | | | | | I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls... | ||||
* | | Split out profile handler to fix tests | Erik Johnston | 2017-08-25 | 3 | -7/+5 |
| | | |||||
* | | Add groups to sync stream | Erik Johnston | 2017-07-20 | 1 | -2/+2 |
|/ | |||||
* | Merge pull request #2309 from matrix-org/erikj/user_ip_repl | Erik Johnston | 2017-07-06 | 2 | -6/+2 |
|\ | | | | | Fix up user_ip replication commands | ||||
| * | Fix up | Erik Johnston | 2017-06-27 | 2 | -6/+2 |
| | | |||||
* | | Update test | Erik Johnston | 2017-07-04 | 1 | -1/+1 |
|/ | |||||
* | Fix up client ips to read from pending data | Erik Johnston | 2017-06-27 | 1 | -4/+1 |
| | |||||
* | Merge pull request #2281 from matrix-org/erikj/phone_home_stats | Erik Johnston | 2017-06-15 | 1 | -115/+0 |
|\ | | | | | Fix phone home stats | ||||
| * | Remove unhelpful test | Erik Johnston | 2017-06-15 | 1 | -115/+0 |
| | | |||||
* | | Fix typing tests | Erik Johnston | 2017-05-26 | 1 | -0/+2 |
| | | |||||
* | | Only load jinja2 templates once | Erik Johnston | 2017-05-22 | 1 | -0/+1 |
| | | | | | | | | | | Instead of every time a new email pusher is created, as loading jinja2 templates is slow. | ||||
* | | Make get_state_groups_from_groups faster. | Erik Johnston | 2017-05-17 | 1 | -1/+1 |
|/ | | | | | | | | | Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy. | ||||
* | Do some logging when one-time-keys get claimed | Richard van der Hoff | 2017-05-09 | 1 | -0/+34 |
| | | | | | might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened. | ||||
* | Allow clients to upload one-time-keys with new sigs | Richard van der Hoff | 2017-05-09 | 1 | -0/+98 |
| | | | | | | | When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully. | ||||
* | Add more granular event send metrics | Erik Johnston | 2017-05-02 | 2 | -3/+3 |
| | |||||
* | Remove test of replication resource | Erik Johnston | 2017-04-11 | 1 | -204/+0 |
| | |||||
* | Merge pull request #2097 from matrix-org/erikj/repl_tcp_client | Erik Johnston | 2017-04-05 | 1 | -8/+22 |
|\ | | | | | Move to using TCP replication | ||||
| * | Change slave storage to use new replication interface | Erik Johnston | 2017-04-03 | 1 | -8/+22 |
| | | | | | | | | | | | | | | As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication. | ||||
* | | Merge pull request #2075 from matrix-org/erikj/cache_speed | Erik Johnston | 2017-03-31 | 3 | -2/+42 |
|\ \ | | | | | | | Speed up cached function access | ||||
| * | | Fix up tests | Erik Johnston | 2017-03-30 | 3 | -2/+42 |
| | | | |||||
* | | | Merge pull request #2076 from matrix-org/erikj/as_perf | Erik Johnston | 2017-03-31 | 1 | -1/+3 |
|\ \ \ | |/ / |/| | | Make AS's faster | ||||
| * | | Compile the regex's used in ASes | Erik Johnston | 2017-03-28 | 1 | -1/+3 |
| |/ | |||||
* / | Fix the logcontext handling in the cache wrappers (#2077) | Richard van der Hoff | 2017-03-30 | 1 | -0/+91 |
|/ | | | | | | | The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken. | ||||
* | Merge pull request #2054 from matrix-org/erikj/user_iter_cursor | Erik Johnston | 2017-03-24 | 1 | -2/+2 |
|\ | | | | | Reduce some CPU work on DB threads | ||||
| * | User Cursor.__iter__ instead of fetchall | Erik Johnston | 2017-03-23 | 1 | -2/+2 |
| | | | | | | | | This prevents unnecessary construction of lists | ||||
* | | Merge pull request #2050 from matrix-org/rav/federation_backoff | Richard van der Hoff | 2017-03-23 | 3 | -1/+5 |
|\ \ | | | | | | | push federation retry limiter down to matrixfederationclient | ||||
| * | | fix tests | Richard van der Hoff | 2017-03-23 | 2 | -1/+3 |
| | | | |||||
| * | | push federation retry limiter down to matrixfederationclient | Richard van der Hoff | 2017-03-23 | 1 | -0/+2 |
| | | | | | | | | | | | | | | | rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter. | ||||
* | | | Merge pull request #2052 from matrix-org/rav/time_bound_deferred | Richard van der Hoff | 2017-03-23 | 1 | -0/+33 |
|\ \ \ | | | | | | | | | Fix time_bound_deferred to throw the right exception | ||||
| * | | | Fix time_bound_deferred to throw the right exception | Richard van der Hoff | 2017-03-23 | 1 | -0/+33 |
| |/ / | | | | | | | | | | | | | | | | Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing. | ||||
* | | | use jsonschema.FormatChecker for RoomID and UserID strings | pik | 2017-03-23 | 2 | -6/+13 |
| | | | | | | | | | | | | | | | | | | * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com> | ||||
* | | | Add valid filter tests, flake8, fix typo | pik | 2017-03-23 | 1 | -5/+49 |
| | | | | | | | | | | | | Signed-off-by: pik <alexander.maznev@gmail.com> | ||||
* | | | check_valid_filter using JSONSchema | pik | 2017-03-23 | 1 | -1/+17 |
| |/ |/| | | | | | | | | | * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com> | ||||
* | | Fix caching of remote servers' signature keys | Richard van der Hoff | 2017-03-22 | 3 | -0/+153 |
|/ | | | | | | | | | The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`. | ||||
* | Merge pull request #2033 from matrix-org/erikj/repl_speed | Erik Johnston | 2017-03-21 | 1 | -1/+1 |
|\ | | | | | Don't send the full event json over replication | ||||
| * | Fix unit test | Erik Johnston | 2017-03-20 | 1 | -1/+1 |
| | | |||||
* | | Stop preserve_fn leaking context into the reactor | Richard van der Hoff | 2017-03-18 | 1 | -0/+61 |
|/ | | | | | | | | Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs. | ||||
* | Modify test_user_sync so it doesn't look at last_active_ts over ↵ | Will Hunt | 2017-03-15 | 1 | -1/+1 |
| | | | | last_user_sync_ts | ||||
* | Aggregate event push actions | Erik Johnston | 2017-02-14 | 1 | -0/+86 |
| | |||||
* | Fix email push in pusher worker | Erik Johnston | 2017-02-02 | 2 | -12/+7 |
| | | | | | | This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending. | ||||
* | Remote membership tests for replication | Erik Johnston | 2017-01-31 | 1 | -43/+0 |
| | | | | | This is because it now relies of the caches stream, which only works on postgres. We are trying to test with sqlite. | ||||
* | Fix unit tests | Erik Johnston | 2017-01-30 | 1 | -4/+4 |
| | |||||
* | Fix unit tests | Erik Johnston | 2017-01-26 | 1 | -0/+17 |
| | |||||
* | Implement device key caching over federation | Erik Johnston | 2017-01-26 | 4 | -14/+27 |
| | |||||
* | Add basic implementation of local device list changes | Erik Johnston | 2017-01-25 | 2 | -2/+5 |
| | |||||
* | Insert delta of current_state_events to be more efficient | Erik Johnston | 2017-01-20 | 1 | -29/+0 |
| | |||||
* | Derive current_state_events from state groups | Erik Johnston | 2017-01-20 | 1 | -28/+17 |
| | |||||
* | Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split | Erik Johnston | 2017-01-18 | 2 | -2/+25 |
|\ | | | | | Optimise state resolution | ||||
| * | Optimise state resolution | Erik Johnston | 2017-01-17 | 2 | -2/+25 |
| | | |||||
* | | Tidy up test | Erik Johnston | 2017-01-17 | 1 | -12/+12 |
| | | |||||
* | | Speed up cache size calculation | Erik Johnston | 2017-01-17 | 2 | -16/+20 |
| | | | | | | | | | | | | | | | | | | | | | | Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory. |