summary refs log tree commit diff
path: root/synapse (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Explicitly use simplejsonErik Johnston2018-03-201-7/+7
|
* Fix replication after switch to simplejsonErik Johnston2018-03-191-2/+4
| | | | | Turns out that simplejson serialises namedtuple's as dictionaries rather than tuples by default.
* Merge pull request #3005 from matrix-org/erikj/fix_cache_sizeErik Johnston2018-03-192-5/+16
|\ | | | | Fix bug where state cache used lots of memory
| * Add commentsErik Johnston2018-03-191-0/+7
| |
| * Fix bug where state cache used lots of memoryErik Johnston2018-03-152-5/+9
| | | | | | | | | | | | | | | | | | The state cache bases its size on the sum of the size of entries. The size of the entry is calculated once on insertion, so it is important that the size of entries does not change. The DictionaryCache modified the entries size, which caused the state cache to incorrectly think it was smaller than it actually was.
* | Merge branch 'master' of github.com:matrix-org/synapse into developErik Johnston2018-03-1925-37/+34
|\ \
| * | Remove wrong commentErik Johnston2018-03-161-1/+0
| | |
| * | Bump version and changelogErik Johnston2018-03-151-1/+1
| | |
| * | Replace ujson with simplejsonErik Johnston2018-03-1525-38/+37
| | |
* | | spell out not to massively increase bcrypt roundsMatthew Hodgson2018-03-191-1/+3
| | |
* | | Replace some ujson with simplejson to make it workErik Johnston2018-03-164-4/+5
| |/ |/|
* | Register membership/state servlets in event_creatorErik Johnston2018-03-141-1/+13
| |
* | Merge pull request #2992 from matrix-org/erikj/implement_member_workreErik Johnston2018-03-144-4/+440
|\ \ | | | | | | Implement RoomMemberWorkerHandler
| * | Fix importsErik Johnston2018-03-142-7/+4
| | |
| * | s/join/joined/ in notify_user_membership_changeErik Johnston2018-03-142-4/+4
| | |
| * | Split RoomMemberWorkerHandler to separate fileErik Johnston2018-03-143-81/+104
| | |
| * | Implement RoomMemberWorkerHandlerErik Johnston2018-03-134-2/+418
| | |
* | | Merge pull request #2989 from matrix-org/erikj/profile_cache_masterErik Johnston2018-03-141-1/+4
|\ \ \ | | | | | | | | Only update remote profile cache on master
| * | | Only update remote profile cache on masterErik Johnston2018-03-131-1/+4
| | | |
* | | | Merge pull request #2988 from matrix-org/erikj/split_profile_storeErik Johnston2018-03-142-24/+47
|\ \ \ \ | | | | | | | | | | Split up ProfileStore
| * | | | Split up ProfileStoreErik Johnston2018-03-132-24/+47
| |/ / /
* | | | Merge pull request #2991 from matrix-org/erikj/fixup_rmErik Johnston2018-03-141-4/+4
|\ \ \ \ | | | | | | | | | | _remote_join and co take a requester
| * | | | _remote_join and co take a requesterErik Johnston2018-03-131-4/+4
| | |/ / | |/| |
* | | | Merge pull request #2993 from matrix-org/erikj/is_blockedErik Johnston2018-03-141-13/+17
|\ \ \ \ | | | | | | | | | | Add is_blocked to worker store
| * | | | Add is_blocked to worker storeErik Johnston2018-03-131-13/+17
| |/ / /
* / / / fix bug #2926 (loading all state for a given type from the DB if the ↵Matthew Hodgson2018-03-131-7/+27
|/ / / | | | | | | | | | | | | state_key is None) (#2990) Fixes a regression that had crept in where the caching layer upholds requests for loading state which is filtered by type (but not by state_key), but the DB layer itself would interpret a missing state_key as a request to filter by null state_key rather than returning all state_keys.
* | | Merge pull request #2987 from matrix-org/erikj/split_room_member_handlerErik Johnston2018-03-132-100/+189
|\ \ \ | |/ / |/| | Split RoomMemberHandler into base and master class
| * | Raise, don't return, exceptionErik Johnston2018-03-131-1/+1
| | |
| * | Add missing param to docstringsErik Johnston2018-03-131-0/+3
| | |
| * | Correct import orderErik Johnston2018-03-131-3/+3
| | |
| * | Move user_*_room distributor stuff to master classErik Johnston2018-03-131-5/+50
| | | | | | | | | | | | | | | I added yields when calling user_left_room, but they shouldn't matter on the master process as they always return None anyway.
| * | Split RoomMemberHandler into base and master classErik Johnston2018-03-132-98/+139
| | | | | | | | | | | | | | | | | | | | | | | | The intention here is to split the class into the bits that can be done on workers and the bits that have to be done on the master. In future there will also be a class that can be run on the worker, which will delegate work to the master when necessary.
* | | Merge pull request #2978 from matrix-org/erikj/refactor_replication_layerErik Johnston2018-03-139-12/+12
|\ \ \ | | | | | | | | Remove ReplicationLayer and user Client/Server directly
| * | | s/replication_client/federation_client/Erik Johnston2018-03-138-9/+9
| | | |
| * | | s/replication_server/federation_serverErik Johnston2018-03-132-3/+3
| | | |
* | | | Merge pull request #2981 from matrix-org/erikj/factor_remote_leaveErik Johnston2018-03-131-22/+54
|\ \ \ \ | | |/ / | |/| | Factor out _remote_reject_invite in RoomMember
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-03-1319-110/+146
| |\ \ \ | | | | | | | | | | | | | | | erikj/factor_remote_leave
| * | | | Add docstringErik Johnston2018-03-131-0/+26
| | | | |
| * | | | Factor out _remote_reject_invite in RoomMemberErik Johnston2018-03-131-22/+28
| | | | |
* | | | | Merge pull request #2979 from matrix-org/erikj/no_handlersErik Johnston2018-03-1319-91/+19
|\ \ \ \ \ | |_|/ / / |/| | | | Don't build handlers on workers unnecessarily
| * | | | Don't build handlers on workers unnecessarilyErik Johnston2018-03-135-5/+0
| | |/ / | |/| |
| * | | Remove unused ReplicationLayerErik Johnston2018-03-132-59/+0
| | | |
| * | | Split replication layer into twoErik Johnston2018-03-1312-27/+19
| | | |
* | | | Merge pull request #2980 from matrix-org/erikj/rm_privErik Johnston2018-03-131-7/+7
|\ \ \ \ | | | | | | | | | | Make RoomMemberHandler functions private that can be
| * | | | Make functions private that can beErik Johnston2018-03-131-7/+7
| | |/ / | |/| |
* | | | Merge pull request #2982 from matrix-org/erikj/fix_extra_usersErik Johnston2018-03-132-5/+5
|\ \ \ \ | | | | | | | | | | extra_users is actually a list of UserIDs
| * | | | extra_users is actually a list of UserIDsErik Johnston2018-03-132-5/+5
| |/ / /
* | | | Merge pull request #2983 from matrix-org/erikj/rename_register_3pidErik Johnston2018-03-132-11/+25
|\ \ \ \ | | | | | | | | | | Refactor get_or_register_3pid_guest
| * | | | Refactor get_or_register_3pid_guestErik Johnston2018-03-132-11/+25
| |/ / /
* | | | Merge pull request #2984 from matrix-org/erikj/fix_rest_regeixErik Johnston2018-03-131-1/+1
|\ \ \ \ | | | | | | | | | | RoomMembershipRestServlet doesn't handle /forget
| * | | | RoomMembershipRestServlet doesn't handle /forgetErik Johnston2018-03-131-1/+1
| |/ / / | | | | | | | | | | | | | | | | Due to the order we register the REST handlers `/forget` was handled by the correct handler.
* | | | Merge pull request #2975 from matrix-org/rav/measure_persist_eventsRichard van der Hoff2018-03-131-4/+5
|\ \ \ \ | | | | | | | | | | Add Measure block for persist_events
| * | | | Add Measure block for persist_eventsRichard van der Hoff2018-03-131-4/+5
| |/ / / | | | | | | | | | | | | This seems like a useful thing to measure.
* | | | Merge pull request #2977 from matrix-org/erikj/replication_move_propsErik Johnston2018-03-134-22/+13
|\ \ \ \ | | |/ / | |/| | Move property setting from ReplicationLayer to base classes
| * | | Move property setting from ReplicationLayer to FederationBaseErik Johnston2018-03-134-22/+13
| | | |
* | | | Fix docstring typesErik Johnston2018-03-131-2/+2
|/ / /
* / / Split out edu/query registration to a separate classErik Johnston2018-03-1310-60/+90
|/ /
* | Add transactional API to history purgeRichard van der Hoff2018-03-122-8/+134
| | | | | | | | Make the purge request return quickly, and allow scripts to poll for updates.
* | Return an error when doing two purges on a roomRichard van der Hoff2018-03-121-3/+14
| | | | | | | | Queuing up purges doesn't sound like a good thing.
* | Merge pull request #2961 from matrix-org/rav/run_in_backgroundRichard van der Hoff2018-03-121-24/+29
|\ \ | | | | | | Factor run_in_background out from preserve_fn
| * | Factor run_in_background out from preserve_fnRichard van der Hoff2018-03-081-24/+29
| | | | | | | | | | | | | | | It annoys me that we create temporary function objects when there's really no need for it. Let's factor the gubbins out of preserve_fn and start using it.
* | | Merge pull request #2965 from matrix-org/rav/request_loggingRichard van der Hoff2018-03-122-34/+96
|\ \ \ | | | | | | | | Add a metric which increments when a request is received
| * | | Add some docstrings to help figure this outRichard van der Hoff2018-03-091-2/+26
| | | |
| * | | Add a metric which increments when a request is receivedRichard van der Hoff2018-03-092-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | It's useful to know when there are peaks in incoming requests - which isn't quite the same as there being peaks in outgoing responses, due to the time taken to handle requests.
| * | | refactor JsonResourceRichard van der Hoff2018-03-091-32/+46
| | | | | | | | | | | | | | | | | | | | rephrase the OPTIONS and unrecognised request handling so that they look similar to the common flow.
* | | | Fix up log messageErik Johnston2018-03-071-1/+1
| | | |
* | | | Fix typoErik Johnston2018-03-071-2/+2
| | | |
* | | | Fix race in sync when joining roomErik Johnston2018-03-073-30/+102
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The race happens when the user joins a room at the same time as doing a sync. We fetch the current token and then get the rooms the user is in. If the join happens after the current token, but before we get the rooms we end up sending down a partial room entry in the sync. This is fixed by looking at the stream ordering of the membership returned by get_rooms_for_user, and handling the case when that stream ordering is after the current token.
* | | Merge pull request #2949 from krombel/use_bcrypt_checkpwRichard van der Hoff2018-03-062-3/+5
|\ \ \ | | | | | | | | use bcrypt.checkpw
| * | | use bcrypt.checkpwKrombel2018-03-052-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in bcrypt 3.1.0 checkpw got introduced (already 2 years ago) This makes use of that with enhancements which might get introduced by that Signed-Off-by: Matthias Kesler <krombel@krombel.de>
* | | | Merge pull request #2946 from matrix-org/rav/timestamp_to_purgeRichard van der Hoff2018-03-063-12/+87
|\ \ \ \ | | | | | | | | | | Implement purge_history by timestamp
| * | | | Provide a means to pass a timestamp to purge_historyRichard van der Hoff2018-03-053-12/+87
| |/ / /
* | | | Merge pull request #2948 from matrix-org/erikj/kill_as_syncErik Johnston2018-03-064-163/+14
|\ \ \ \ | | | | | | | | | | Remove ability for AS users to call /events and /sync
| * | | | Remove ability for AS users to call /events and /syncErik Johnston2018-03-054-163/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This functionality has been deprecated for a while as well as being broken for a while. Instead of fixing it lets just remove it entirely. See: https://github.com/matrix-org/matrix-doc/issues/1144
* | | | | Merge pull request #2947 from matrix-org/erikj/split_directory_storeErik Johnston2018-03-052-28/+30
|\ \ \ \ \ | |_|/ / / |/| | | | Split Directory store
| * | | | Fix cache invalidation on deletionErik Johnston2018-03-051-1/+4
| | | | |
| * | | | Split Directory storeErik Johnston2018-03-052-27/+26
| |/ / /
* | | | Merge pull request #2943 from ↵Richard van der Hoff2018-03-051-12/+71
|\ \ \ \ | |/ / / |/| | | | | | | | | | | matrix-org/rav/fix_find_first_stream_ordering_after_ts Test and fix find_first_stream_ordering_after_ts
| * | | Test and fix find_first_stream_ordering_after_tsRichard van der Hoff2018-03-051-15/+53
| | | | | | | | | | | | | | | | It seemed to suffer from a bunch of off-by-one errors.
| * | | Add find_first_stream_ordering_after_tsRichard van der Hoff2018-03-051-0/+21
| | | | | | | | | | | | | | | | Expose this as a public function which can be called outside a txn
* | | | Merge pull request #2934 from matrix-org/erikj/cache_fixErik Johnston2018-03-052-13/+15
|\ \ \ \ | | | | | | | | | | Fix bug with delayed cache invalidation stream
| * | | | Fix bug with delayed cache invalidation streamErik Johnston2018-03-022-13/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We poked the notifier before updated the current token for the cache invalidation stream. This mean that sometimes the update wouldn't be sent until the next time a cache was invalidated.
* | | | | Merge pull request #2929 from matrix-org/erikj/split_regististration_storeErik Johnston2018-03-052-72/+64
|\ \ \ \ \ | |/ / / / |/| | | | Split registration store
| * | | | Split registration storeErik Johnston2018-03-022-72/+64
| | | | |
* | | | | Add missing yield during 3pid signature checksErik Johnston2018-03-021-1/+1
|/ / / /
* | | | Merge pull request #2928 from matrix-org/erikj/read_marker_cachesErik Johnston2018-03-011-1/+1
|\ \ \ \ | | |/ / | |/| | Fix typo in getting replication account data processing
| * | | Fix typo in getting replication account data processingErik Johnston2018-03-011-1/+1
| | | |
* | | | Merge pull request #2925 from matrix-org/erikj/split_sig_fedErik Johnston2018-03-013-180/+152
|\ \ \ \ | |_|/ / |/| | | Split out SignatureStore and EventFederationStore
| * | | Remove unused DataStoreErik Johnston2018-03-011-1/+0
| | | |
| * | | Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_sig_fedErik Johnston2018-03-017-445/+428
| |\ \ \
| * | | | Stub out broken function only used for cacheErik Johnston2018-03-011-1/+3
| | | | |
| * | | | Split out SignatureStore and EventFederationStoreErik Johnston2018-03-013-174/+148
| | | | |
* | | | | Merge pull request #2927 from matrix-org/erikj/read_marker_cachesErik Johnston2018-03-014-5/+46
|\ \ \ \ \ | | |_|/ / | |/| | | Improve caching for read_marker API
| * | | | Fewer lies are betterErik Johnston2018-03-011-2/+2
| | | | |
| * | | | Improve caching for read_marker APIErik Johnston2018-03-013-3/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We add a new storage function to get a paritcular type of room account data. This allows us to prefill the cache when updating that acount data.
| * | | | Add some caches to help read marker APIErik Johnston2018-03-013-2/+5
| |/ / /
* | | | Merge pull request #2926 from matrix-org/erikj/member_handler_moveErik Johnston2018-03-0110-39/+51
|\ \ \ \ | |_|/ / |/| | | Move RoomMemberHandler out of Handlers
| * | | Move back to hs.is_mineErik Johnston2018-03-011-7/+6
| | | |
| * | | Move RoomMemberHandler out of HandlersErik Johnston2018-03-0110-44/+57
| |/ /
* | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-03-016-225/+221
|\ \ \ | | | | | | | | | | | | erikj/split_stream_store
| * \ \ Merge pull request #2923 from matrix-org/erikj/stream_ago_workerErik Johnston2018-03-013-80/+85
| |\ \ \ | | | | | | | | | | Calculate stream_ordering_month_ago correctly on workers
| | * | | Default stream_ordering_*_ago to NoneErik Johnston2018-03-011-2/+2
| | | | |
| | * | | Fix comment typoErik Johnston2018-03-011-1/+1
| | | | |
| | * | | Calculate stream_ordering_month_ago correctly on workersErik Johnston2018-03-013-80/+85
| | |/ /
| * | | Merge pull request #2922 from matrix-org/erikj/split_room_storeErik Johnston2018-03-012-135/+125
| |\ \ \ | | | | | | | | | | Split up RoomStore
| | * | | Split up RoomStoreErik Johnston2018-03-012-135/+125
| | |/ /
| * | | Merge pull request #2921 from matrix-org/rav/unyielding_make_deferred_yieldableRichard van der Hoff2018-03-011-9/+11
| |\ \ \ | | |/ / | |/| | Rewrite make_deferred_yieldable avoiding inlineCallbacks
| | * | Rewrite make_deferred_yieldable avoiding inlineCallbacksRichard van der Hoff2018-03-011-9/+11
| | | | | | | | | | | | | | | | | | | | ... because (a) it's actually simpler (b) it might be marginally more performant?
* | | | Remove unused variablesErik Johnston2018-03-011-8/+1
| | | |
* | | | Document abstract class and method betterErik Johnston2018-03-011-8/+13
| | | |
* | | | Split out stream storeErik Johnston2018-03-013-210/+202
|/ / /
* | | Log in the correct placesErik Johnston2018-03-011-2/+4
| | |
* | | Don't do preserve_fn for every requestErik Johnston2018-03-011-1/+2
| | |
* | | Add some loggingErik Johnston2018-03-011-0/+2
| | |
* | | Make repl send_event idempotent and retry on timeoutsErik Johnston2018-03-011-6/+38
| | | | | | | | | | | | | | | | | | If we treated timeouts as failures on the worker we would attempt to clean up e.g. push actions while the master might still process the event.
* | | Check event auth on the workerErik Johnston2018-03-011-16/+16
| | |
* | | Correctly send ratelimit and extra_users paramsErik Johnston2018-03-012-1/+15
| | |
* | | Fixup commentsErik Johnston2018-03-011-4/+4
| | |
* | | Calculate push actions on workerErik Johnston2018-02-283-26/+70
|/ /
* | Move storage functions for push calculationsErik Johnston2018-02-276-98/+101
| | | | | | | | This will allow push actions for an event to be calculated on workers.
* | Merge pull request #2904 from matrix-org/erikj/receipt_cache_invalidationErik Johnston2018-02-272-14/+16
|\ \ | | | | | | Fix missing invalidations for receipt storage
| * | Fix missing invalidations for receipt storageErik Johnston2018-02-212-14/+16
| | |
* | | Merge pull request #2903 from matrix-org/erikj/split_roommember_storeErik Johnston2018-02-272-207/+187
|\ \ \ | | | | | | | | Split out RoomMemberStore
| * \ \ Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-2313-594/+647
| |\ \ \ | | | | | | | | | | | | | | | erikj/split_roommember_store
| * | | | Update copyrightErik Johnston2018-02-233-0/+3
| | | | |
| * | | | Split out RoomMemberStoreErik Johnston2018-02-212-206/+184
| | | | |
* | | | | Merge pull request #2901 from matrix-org/erikj/split_as_storesErik Johnston2018-02-272-37/+29
|\ \ \ \ \ | | | | | | | | | | | | Split AS stores
| * | | | | Add commentErik Johnston2018-02-271-0/+4
| | | | | |
| * | | | | Update copyrightErik Johnston2018-02-232-0/+2
| | | | | |
| * | | | | Split AS storesErik Johnston2018-02-212-37/+23
| |/ / / /
* | | | | Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actionsErik Johnston2018-02-262-23/+46
|\ \ \ \ \ | | | | | | | | | | | | Batch inserts into event_push_actions_staging
| * | | | | Update commentsErik Johnston2018-02-211-3/+4
| | | | | |
| * | | | | Batch inserts into event_push_actions_stagingErik Johnston2018-02-202-23/+45
| | | | | |
* | | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-2615-687/+747
|\ \ \ \ \ \ | | |_|/ / / | |/| | | | | | | | | | erikj/handle_unpersisted_events_push
| * | | | | Merge branch 'develop' of github.com:matrix-org/synapse into ↵Erik Johnston2018-02-2312-507/+570
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | erikj/split_event_push_actions
| | * \ \ \ \ Merge pull request #2902 from matrix-org/erikj/split_events_storeErik Johnston2018-02-233-370/+405
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | Split out get_events and co into a worker store
| | | * | | | | Split EventsWorkerStore into separate fileErik Johnston2018-02-233-361/+401
| | | | | | | |
| | | * | | | | Update copyrightErik Johnston2018-02-232-0/+2
| | | | | | | |
| | | * | | | | Remove redundant clockErik Johnston2018-02-231-3/+0
| | | | | | | |
| | | * | | | | _event_persist_queue shouldn't be in worker storeErik Johnston2018-02-231-4/+4
| | | | |/ / / | | | |/| | |
| | | * | | | Split out get_events and co into a worker storeErik Johnston2018-02-212-356/+352
| | | | | | |
| | * | | | | Merge pull request #2899 from matrix-org/erikj/split_pushersErik Johnston2018-02-232-13/+10
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | Split PusherStore
| | | * | | | | Update copyrightErik Johnston2018-02-232-0/+2
| | | | | | | |
| | | * | | | | Split PusherStoreErik Johnston2018-02-212-13/+8
| | | |/ / / /
| | * | | | | Merge pull request #2898 from matrix-org/erikj/split_push_rules_storeErik Johnston2018-02-233-45/+64
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | Split PushRulesStore
| | | * | | | | Update copyrightErik Johnston2018-02-233-0/+3
| | | | | | | |
| | | * | | | | Split PushRulesStoreErik Johnston2018-02-213-45/+61
| | | |/ / / /
| | * | | | | Merge pull request #2897 from matrix-org/erikj/split_account_dataErik Johnston2018-02-234-69/+74
| | |\ \ \ \ \ | | | |_|_|/ / | | |/| | | | Split AccountDataStore and TagStore
| | | * | | | Update copyrightErik Johnston2018-02-233-0/+3
| | | | | | |
| | | * | | | Use absolute importsErik Johnston2018-02-233-5/+5
| | | | | | |
| | | * | | | Split AccountDataStore and TagStoreErik Johnston2018-02-214-67/+69
| | | |/ / /
| | * | | | Merge pull request #2893 from matrix-org/erikj/delete_from_staging_fedErik Johnston2018-02-211-9/+17
| | |\ \ \ \ | | | |/ / / | | |/| | | Delete from push_actions_staging in federation too
| | | * | | Delete from push_actions_staging in federation tooErik Johnston2018-02-201-9/+17
| | | |/ /
| * | | | Update copyrightErik Johnston2018-02-232-0/+2
| | | | |
| * | | | Split out EventPushActionWorkerStoreErik Johnston2018-02-212-91/+76
| |/ / /
| * | | Raise exception in abstract methodErik Johnston2018-02-201-1/+1
| | | |
| * | | Fix commentErik Johnston2018-02-202-2/+2
| | | |
| * | | Use abstract base class to access stream IDsErik Johnston2018-02-202-17/+34
| | | |
| * | | Split ReceiptsStoreErik Johnston2018-02-203-76/+69
| | | |
* | | | Actually use new paramErik Johnston2018-02-211-1/+3
| | | |
* | | | Ensure all push actions are deleted from stagingErik Johnston2018-02-202-2/+19
| | | |
* | | | Refactor _set_push_actions_for_event_and_users_txn to use events_and_contextsErik Johnston2018-02-202-33/+41
|/ / /
* | | Merge pull request #2868 from matrix-org/erikj/refactor_media_storageErik Johnston2018-02-201-26/+11
|\ \ \ | |/ / |/| | Make store_file use store_into_file
| * | Make store_file use store_into_fileErik Johnston2018-02-141-26/+11
| | |
* | | Update pynacl dependency to 1.2.1 or higherPascal Bach2018-02-191-1/+1
| | | | | | | | | | | | Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>
* | | (Really) fix tablescan of event_push_actions on purgeRichard van der Hoff2018-02-161-1/+0
| | | | | | | | | | | | | | | commit 278d21b5 added new code to avoid the tablescan, but didn't remove the old :/
* | | Fix typo of double is_highlightErik Johnston2018-02-161-1/+1
| | |
* | | CommentsErik Johnston2018-02-163-1/+9
| | |
* | | Ensure that we delete staging push actions on errorsErik Johnston2018-02-152-3/+25
| | |
* | | Remove context.push_actionsErik Johnston2018-02-154-19/+7
| | |
* | | Update event_push_actions table from staging tableErik Johnston2018-02-152-22/+39
| | |
* | | Store push actions in staging areaErik Johnston2018-02-153-0/+54
| | |
* | | Don't serialize current state over replicationErik Johnston2018-02-153-8/+51
| | |
* | | Merge pull request #2872 from matrix-org/erikj/event_worker_dont_logErik Johnston2018-02-151-1/+10
|\ \ \ | | | | | | | | Don't log errors propogated from send_event
| * | | Don't log errors propogated from send_eventErik Johnston2018-02-151-1/+10
| | | |
* | | | Fix state group storage bug in workersErik Johnston2018-02-151-41/+41
|/ / / | | | | | | | | | | | | We needed to move `_count_state_group_hops_txn` to the StateGroupWorkerStore.
* | | Merge pull request #2867 from matrix-org/rav/rework_purgeRichard van der Hoff2018-02-151-35/+83
|\ \ \ | | | | | | | | purge_history cleanups
| * | | purge_history: fix sqlite syntax errorRichard van der Hoff2018-02-141-1/+4
| | | | | | | | | | | | | | | | apparently sqlite insists on indexes being named
| * | | purge_history: handle sqlite asshatteryRichard van der Hoff2018-02-141-19/+27
| | | | | | | | | | | | | | | | | | | | apparently creating a temporary table commits the transaction. because that's a useful thing.
| * | | purge_history: fix index useRichard van der Hoff2018-02-141-0/+14
| | | | | | | | | | | | | | | | | | | | event_push_actions doesn't have an index on event_id, so we need to specify room_id.
| * | | Rework event purge to use a temporary tableRichard van der Hoff2018-02-141-35/+58
| | | | | | | | | | | | | | | | | | | | ... which should speed things up by reducing the amount of data being shuffled across the connection
* | | | Merge pull request #2769 from matrix-org/matthew/hit_the_ginRichard van der Hoff2018-02-145-20/+110
|\ \ \ \ | |_|/ / |/| | | switch back from GIST to GIN indexes
| * | | remove overzealous exception handlingRichard van der Hoff2018-02-141-18/+10
| | | |
| * | | Merge branch 'matthew/gin_work_mem' into matthew/hit_the_ginRichard van der Hoff2018-02-1390-1492/+3304
| |\ \ \
| | * \ \ Merge branch 'develop' into matthew/gin_work_memRichard van der Hoff2018-02-1329-549/+995
| | |\ \ \
| | * | | | Factor out common code for search insertRichard van der Hoff2018-02-041-33/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we can reuse the same code as is used for event insert, for doing the background index population.
| | * | | | Clean up work_mem handlingRichard van der Hoff2018-02-031-11/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add some comments and improve exception handling when twiddling work_mem for the search update
| | * | | | Move store_event_search_txn to SearchStoreRichard van der Hoff2018-02-032-37/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... as a precursor to making event storing and doing the bg update share some code.
| | * | | | Merge branch 'develop' into matthew/gin_work_memRichard van der Hoff2018-02-0372-902/+2203
| | |\ \ \ \
| | * | | | | oopshera2018-01-092-2/+2
| | | | | | |
| | * | | | | oops, tweak work_mem when actually storingMatthew Hodgson2018-01-091-0/+2
| | | | | | |
| | * | | | | avoid 80s GIN inserts by tweaking work_memMatthew Hodgson2018-01-091-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | see https://github.com/matrix-org/synapse/issues/2753 for details
| * | | | | | move search reindex to schema 47Richard van der Hoff2018-02-132-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're up to schema v47 on develop now, so this will have to go in there to have an effect. This might cause an error if somebody has already run it in the v46 guise, and runs it again in the v47 guise, because it will cause a duplicate entry in the bbackground_updates table. On the other hand, the entry is removed once it is complete, and it is unlikely that anyone other than matrix.org has run it on v46. The update itself is harmless to re-run because it deliberately copes with the index already existing.
| * | | | | | GIN reindex: Fix syntax errors, improve exception handlingRichard van der Hoff2018-02-131-13/+27
| | | | | | |
| * | | | | | Reinstate event_search_postgres_gist handlerRichard van der Hoff2018-02-023-6/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | People may have queued updates for this, so we can't just delete it.
| * | | | | | fix GIST->GIN switchMatthew Hodgson2018-01-093-14/+37
| | | | | | |
| * | | | | | switch back from GIST to GIN indexesMatthew Hodgson2018-01-091-8/+13
| |/ / / / /
* | | | | | Merge pull request #2854 from matrix-org/erikj/event_create_workerErik Johnston2018-02-1311-20/+500
|\ \ \ \ \ \ | |_|_|_|/ / |/| | | | | Create a worker for event creation
| * | | | | Move presence handling into handle_new_client_eventErik Johnston2018-02-071-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | As we want to have it run on the main synapse instance
| * | | | | Add event_creator workerErik Johnston2018-02-073-1/+190
| | | | | |
| * | | | | Add replication http endpoint for event sendingErik Johnston2018-02-078-13/+304
| | | | | |
* | | | | | Fix log message in purge_historyRichard van der Hoff2018-02-131-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | (we don't just remove remote events)
* | | | | | Merge pull request #2864 from matrix-org/rav/persist_event_cachingRichard van der Hoff2018-02-132-60/+70
|\ \ \ \ \ \ | | | | | | | | | | | | | | Use StateResolutionHandler to resolve state in persist_events
| * | | | | | style nitRichard van der Hoff2018-02-131-1/+1
| | | | | | |
| * | | | | | Use StateResolutionHandler to resolve state in persist eventsRichard van der Hoff2018-02-051-48/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and thus benefit (hopefully) from its cache.
| * | | | | | Flatten _get_new_state_after_eventsRichard van der Hoff2018-02-051-44/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | rejig the if statements to simplify the logic and reduce indentation
| * | | | | | Check that events being persisted have state_groupRichard van der Hoff2018-02-051-4/+9
| | | | | | |
| * | | | | | Add event_map param to resolve_state_groupsRichard van der Hoff2018-02-052-4/+31
| | | | | | |
* | | | | | | Fix typos in purge api & docRichard van der Hoff2018-02-131-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * It's supposed to be purge_local_events, not ..._history * Fix the doc to have valid json
* | | | | | | Merge pull request #2695 from okurz/feature/allow_recent_pysamlRichard van der Hoff2018-02-131-1/+1
|\ \ \ \ \ \ \ | |_|_|_|_|/ / |/| | | | | | Allow use of higher versions of saml2
| * | | | | | Allow use of higher versions of saml2Oliver Kurz2017-11-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The package was pinned to <4.0 with 07cf96eb because "from saml2 import config" did not work. This seems to have been fixed in the mean time in the saml2 package and therefore should not stop to use a more recent version. Signed-off-by: Oliver Kurz <okurz@suse.de>
* | | | | | | Merge pull request #2857 from matrix-org/erikj/upload_storeErik Johnston2018-02-121-0/+6
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | Tell storage providers about new file so they can upload
| * | | | | | | Tell storage providers about new file so they can uploadErik Johnston2018-02-071-0/+6
| | | | | | | |
* | | | | | | | Merge pull request #2858 from matrix-org/rav/purge_updatesRichard van der Hoff2018-02-094-39/+88
|\ \ \ \ \ \ \ \ | |_|_|_|/ / / / |/| | | | | | | delete_local_events for purge_room_history
| * | | | | | | purge: move room_depth update to endRichard van der Hoff2018-02-091-6/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... to avoid locking the table for too long
| * | | | | | | delete_local_events for purge_historyRichard van der Hoff2018-02-094-13/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a flag which makes the purger delete local events
| * | | | | | | purge: Move cache invalidation to more appropriate placeRichard van der Hoff2018-02-091-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | it was a bit of a non-sequitur there
| * | | | | | | bump purge logging to infoRichard van der Hoff2018-02-091-13/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this thing takes ages and the only sign of any progress is the logs, so having some logs is useful.
| * | | | | | | rename delete_old_state -> purge_historyRichard van der Hoff2018-02-092-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (beacause it deletes more than state)
* | | | | | | | Remove pointless ratelimit checkErik Johnston2018-02-071-5/+0
| |/ / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The intention was for the check to be called as early as possible in the request, but actually was called just before the main ratelimit check, so was fairly pointless.
* | | | | | | Merge pull request #2847 from matrix-org/erikj/separate_event_creationErik Johnston2018-02-068-188/+211
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | Split event creation into a separate handler
| * | | | | | | Update copyrightErik Johnston2018-02-066-1/+6
| | | | | | | |
| * | | | | | | s/_create_new_client_event/create_new_client_event/Erik Johnston2018-02-062-8/+8
| | | | | | | |
| * | | | | | | Update places where we create eventsErik Johnston2018-02-057-38/+42
| | | | | | | |
| * | | | | | | Split event creation into a separate handlerErik Johnston2018-02-051-146/+160
| | | | | | | |
* | | | | | | | Store state groups separately from events (#2784)Erik Johnston2018-02-0610-120/+237
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints
* | | | | | | | Merge pull request #2849 from matrix-org/rav/clean_up_state_deltaRichard van der Hoff2018-02-051-10/+23
|\ \ \ \ \ \ \ \ | | |_|_|/ / / / | |/| | | | | | Remove redundant return value from _calculate_state_delta
| * | | | | | | Remove redundant return value from _calculate_state_deltaRichard van der Hoff2018-02-051-10/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we already have the state from _get_new_state_after_events, so returning it from _calculate_state_delta is just confusing.
* | | | | | | | Factor out common code for search insertRichard van der Hoff2018-02-051-33/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we can reuse the same code as is used for event insert, for doing the background index population.
* | | | | | | | Move store_event_search_txn to SearchStoreRichard van der Hoff2018-02-052-35/+41
|/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... as a precursor to making event storing and doing the bg update share some code.
* | | | | | | Merge pull request #2844 from matrix-org/rav/evicted_metricsRichard van der Hoff2018-02-054-5/+44
|\ \ \ \ \ \ \ | |/ / / / / / |/| | | | | | montoring metrics for number of cache evictions
| * | | | | | report metrics on number of cache evictionsRichard van der Hoff2018-02-054-5/+44
| | |_|_|/ / | |/| | | |
* | | | | | Merge pull request #2791 from matrix-org/erikj/media_storage_refactorErik Johnston2018-02-053-12/+47
|\ \ \ \ \ \ | | | | | | | | | | | | | | Ensure media is in local cache before thumbnailing
| * | | | | | Use better file consumerErik Johnston2018-01-181-5/+10
| | | | | | |
| * | | | | | Ensure media is in local cache before thumbnailingErik Johnston2018-01-183-10/+40
| | | | | | |
* | | | | | | Merge pull request #2845 from matrix-org/rav/urlcache_error_handlingRichard van der Hoff2018-02-021-21/+34
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | Handle url_previews with no content-type
| * | | | | | | Handle url_previews with no content-typeRichard van der Hoff2018-02-021-21/+34
| | |/ / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | avoid failing with an exception if the remote server doesn't give us a Content-Type header. Also, clean up the exception handling a bit.
* | | | | | | Merge pull request #2842 from matrix-org/rav/state_resolution_handlerRichard van der Hoff2018-02-024-65/+116
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | Factor out resolve_state_groups to a separate handler
| * | | | | | | Factor out resolve_state_groups to a separate handlerRichard van der Hoff2018-02-013-53/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.
| * | | | | | | Rename resolve_state_groups -> resolve_state_groups_for_eventsRichard van der Hoff2018-02-012-12/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (to make way for a method that actually just does the state group resolution)
* | | | | | | | Merge pull request #2841 from matrix-org/rav/refactor_calc_state_deltaRichard van der Hoff2018-02-021-18/+39
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | factor _get_new_state_after_events out of _calculate_state_delta
| * | | | | | | | factor _get_new_state_after_events out of _calculate_state_deltaRichard van der Hoff2018-01-311-18/+39
| |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | This reduces the scope of a bunch of variables
* | | | | | | | Merge pull request #2836 from matrix-org/rav/resolve_state_events_docstringRichard van der Hoff2018-02-021-8/+28
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Docstring fixes
| * | | | | | | | doc arg types for _seperateRichard van der Hoff2018-02-011-0/+15
| | | | | | | | |
| * | | | | | | | More docstring fixesRichard van der Hoff2018-02-011-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix a couple of errors in docstrings
| * | | | | | | | Fix docstring for StateHandler.resolve_state_groupsRichard van der Hoff2018-02-011-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The return type was a complete lie, so fix it
* | | | | | | | | Merge branch 'develop' into travis/admin-list-mediaTravis Ralston2018-02-0140-195/+427
|\ \ \ \ \ \ \ \ \ | | |_|/ / / / / / | |/| | | | | | |
| * | | | | | | | Merge pull request #2837 from matrix-org/rav/fix_quarantine_mediaRichard van der Hoff2018-02-011-1/+1
| |\ \ \ \ \ \ \ \ | | |_|/ / / / / / | |/| | | | | | | Fix sql error in quarantine_media
| | * | | | | | | Fix sql error in quarantine_mediaRichard van der Hoff2018-01-301-1/+1
| | |/ / / / / /
| * | | | | | | Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_failRichard van der Hoff2018-01-311-2/+1
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | Remove spurious log argument
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-12-22 22:10:46 +0000