summary refs log tree commit diff
path: root/synapse/rest/media (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Remove unnecessary conditionErik Johnston2018-01-121-4/+1
|
* Remove unused variablesErik Johnston2018-01-121-3/+0
|
* Make class var localErik Johnston2018-01-121-3/+3
|
* CommentsErik Johnston2018-01-123-15/+65
|
* Add StorageProvider conceptErik Johnston2018-01-093-9/+162
|
* Make PreviewUrlResource use MediaStorageErik Johnston2018-01-092-8/+14
|
* Make ThumbnailResource use MediaStorageErik Johnston2018-01-092-48/+68
|
* Use MediaStorage for remote mediaErik Johnston2018-01-092-136/+156
|
* Use MediaStorage for local filesErik Johnston2018-01-092-119/+73
|
* Add MediaStorage classErik Johnston2018-01-091-0/+198
|
* Add some helper classesErik Johnston2018-01-091-0/+73
|
* Split out add_file_headersErik Johnston2018-01-091-28/+42
|
* Remove dead TODOErik Johnston2018-01-091-2/+0
|
* Remove dead code related to default thumbnailsErik Johnston2018-01-091-73/+3
|
* Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_errorRichard van der Hoff2017-11-271-3/+7
|\ | | | | Fix error on sqlite 3.7
| * Check database in has_completed_background_updatesRichard van der Hoff2017-11-221-1/+1
| | | | | | | | so that the right thing happens on workers.
| * Fix error on sqlite 3.7Richard van der Hoff2017-11-211-3/+7
| | | | | | | | | | | | | | | | | | | | | | Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.
* | Fix OPTIONS on preview_urlRichard van der Hoff2017-11-231-1/+5
|/ | | | Fixes #2706
* Bit more loggingRichard van der Hoff2017-11-101-0/+2
|
* Cache failures in url_preview handlerRichard van der Hoff2017-11-101-41/+45
| | | | | Reshuffle the caching logic in the url_preview handler so that failures are cached (and to generally simplify things and fix the logcontext leaks).
* Ignore noscript tags when generating URL previewsMaxime Vaillancourt2017-10-251-1/+8
|
* replace 'except:' with 'except Exception:'Richard van der Hoff2017-10-233-5/+5
| | | | what could possibly go wrong
* Fix a logcontext leak in the media repoRichard van der Hoff2017-10-231-1/+4
|
* Fix typo in thumbnail generationErik Johnston2017-10-191-2/+5
|
* fix thumbnailing (#2548)Krombel2017-10-171-2/+2
| | | | | | | in commit 0e28281a the code for thumbnailing got refactored and the renaming of this variables was not done correctly. Signed-Off-by: Matthias Kesler <krombel@krombel.de>
* Add try/finally block to close t_byte_sourceErik Johnston2017-10-131-24/+41
|
* Remove old commentErik Johnston2017-10-131-1/+0
|
* Don't reuse source but instead copy from primary media store to backupErik Johnston2017-10-132-22/+9
|
* PEP8Erik Johnston2017-10-131-1/+1
|
* Move type into keyErik Johnston2017-10-131-3/+3
|
* CommentErik Johnston2017-10-131-1/+1
|
* Remove spurious additionErik Johnston2017-10-131-4/+0
|
* Don't needlessly rename variableErik Johnston2017-10-131-13/+13
|
* Add paths back in...Erik Johnston2017-10-131-4/+4
|
* Remove unnecessary diffErik Johnston2017-10-131-3/+1
|
* Make things be absolute paths againErik Johnston2017-10-131-7/+7
|
* s/write_to_file/write_to_file_and_backup/Erik Johnston2017-10-131-5/+5
|
* Fix upErik Johnston2017-10-132-43/+32
|
* Fix up thumbnailing functionErik Johnston2017-10-133-75/+73
|
* Fix typoErik Johnston2017-10-131-1/+1
|
* Fix up commentsErik Johnston2017-10-131-2/+26
|
* Move makedirs to threadErik Johnston2017-10-131-2/+1
|
* Use make_deferred_yieldableErik Johnston2017-10-131-7/+7
|
* Fix commentsErik Johnston2017-10-122-1/+4
|
* Don't close prematurely..Erik Johnston2017-10-121-2/+5
|
* getvalue closes bufferErik Johnston2017-10-121-11/+10
|
* Get len before closeErik Johnston2017-10-121-6/+11
|
* Don't close file prematurelyErik Johnston2017-10-122-8/+18
|
* Copy everything to backupErik Johnston2017-10-124-76/+148
|
* TypoErik Johnston2017-10-121-1/+1
|
* Remove dead codeErik Johnston2017-10-121-3/+0
|
* Fix typosErik Johnston2017-10-121-20/+26
|
* Fix typoErik Johnston2017-10-121-1/+1
|
* Basic implementation of backup media storeErik Johnston2017-10-123-126/+113
|
* Only log if we've removed mediaErik Johnston2017-09-281-2/+4
|
* Clear up expired url cache every 10sErik Johnston2017-09-281-1/+1
|
* Actually assign result to variableErik Johnston2017-09-281-1/+1
|
* Fix typo in url cache expiry timerErik Johnston2017-09-281-1/+1
|
* Support new and old style media id formatsErik Johnston2017-09-281-31/+81
|
* Change expires column to expires_tsErik Johnston2017-09-281-2/+2
|
* Delete expired url cache dataErik Johnston2017-09-282-4/+129
|
* Store URL cache preview downloads seperatelyErik Johnston2017-06-235-18/+72
| | | | This makes it easier to clear old media out at a later date
* Add API to quarantine mediaErik Johnston2017-06-193-3/+5
|
* Throw exception when not retrying when downloading mediaErik Johnston2017-06-131-0/+1
|
* actually trim oversize og:description metaMatthew Hodgson2017-05-221-0/+2
|
* Don't log exceptions for NotRetryingDestinationErik Johnston2017-05-151-1/+3
|
* Address review commentsRichard van der Hoff2017-03-141-15/+17
| | | | | | | - don't blindly proxy all HTTPRequestExceptions - log unexpected exceptions at error - avoid `isinstance` - improve docs on `from_http_response_exception`
* Fix routing loop when fetching remote mediaRichard van der Hoff2017-03-132-4/+38
| | | | | | | | | | | When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502.
* Fix dynamic thumbnails aspectJurek2017-02-241-0/+3
|
* Log which files we saved attachments to in the media_repositoryMark Haines2017-01-103-0/+11
|
* Name linearizer's for better logsErik Johnston2017-01-091-1/+1
|
* Fix crash in url preview when html tag has no textMarcin Bachry2016-12-141-1/+4
| | | | Signed-off-by: Marcin Bachry <hegel666@gmail.com>
* preview_url_resource: Ellipsis must be in unicode stringJohannes Löthberg2016-12-011-1/+1
| | | | Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
* Set CORs headers on responses from the media repoMark Haines2016-11-022-2/+4
|
* Add quotes and be explicity about script-srcErik Johnston2016-09-051-1/+2
|
* Allow PDF to be rendered from media repoErik Johnston2016-09-051-1/+7
|
* Set `Content-Security-Policy` on media repoErik Johnston2016-08-171-0/+1
| | | | | This is to inform browsers that they should sandbox the returned media. This is particularly cruical for javascript/HTML files.
* Add None check to _iterate_over_textErik Johnston2016-08-171-1/+1
|
* Flake8Erik Johnston2016-08-161-4/+1
|
* Fix up preview URL API. Add tests.Erik Johnston2016-08-161-157/+196
| | | | | | | | | This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text.
* Don't include html comments in descriptionErik Johnston2016-08-051-1/+2
|
* TypoErik Johnston2016-08-041-1/+1
|
* Test summarizationErik Johnston2016-08-041-52/+54
|
* Don't infer paragrahs from newlinesErik Johnston2016-08-021-2/+1
|
* Comment on why we cloneErik Johnston2016-08-021-0/+1
|
* Spelling.Erik Johnston2016-08-021-3/+3
|
* Make it actually compileErik Johnston2016-08-021-1/+1
|
* Change the way we summarize URLsErik Johnston2016-08-021-11/+67
| | | | | | | | Using XPath is slow on some machines (for unknown reasons), so use a different approach to get a list of text nodes. Try to generate a summary that respect paragraph and then word boundaries, adding ellipses when appropriate.
* Remove raceErik Johnston2016-06-291-11/+0
|
* Implement purge_media_cache admin APIErik Johnston2016-06-292-20/+64
|
* Track approximate last access time for remote mediaErik Johnston2016-06-291-0/+24
|
* Remove the legacy v0 content upload API.Mark Haines2016-06-211-110/+2
| | | | | | The existing content can still be downloaded. The last upload to the matrix.org server was in January 2015, so it is probably safe to remove the upload API.
* Line lengthsErik Johnston2016-06-151-3/+3
|
* Handle floats as img widthsErik Johnston2016-06-151-1/+1
|
* Handle og props with not contentErik Johnston2016-06-151-1/+2
|
* 502 on /thumbnail when can't contact remote serverErik Johnston2016-06-091-4/+10
|
* Clean up the blacklist/whitelist handling.Mark Haines2016-05-161-33/+30
| | | | | | | Always set the config key with an empty list, even if a list isn't specified. This means that the codepaths are the same for both the empty list and for a missing key. Since the behaviour is the same for both cases this makes the code somewhat easier to reason about.
* Report per request metrics for all of the things using request_handlerMark Haines2016-04-284-4/+7
|
* Add self.media_repo to PreviewUrlResourceErik Johnston2016-04-191-0/+1
|
* _make_dirs was moved to MediaRepositoryErik Johnston2016-04-191-1/+1
|
* Add store to PreviewUrlResourceErik Johnston2016-04-191-0/+1
|
* Reorder importsErik Johnston2016-04-191-7/+8
|
* Move MediaRepository to media_repository moduleErik Johnston2016-04-195-489/+496
|
* Split out BaseMediaResource into MediaRepositoryErik Johnston2016-04-196-138/+180
| | | | | | | | | This is so that a single MediaRepository can be shared across all resources, rather than having a "copy" per resource. In particular this allows us to guard against both the thumbnail and download resource triggering a download of remote content at the same time.
* explicitly pass in the charset from Content-Type to lxml to fix cyrillic ↵Matthew Hodgson2016-04-151-6/+18
| | | | woes better
* fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, ↵Matthew Hodgson2016-04-151-16/+8
| | | | rather than relying on lxml's heuristics which seem to get it wrong
* fix urlparse import thinko breaking tiny URLsMatthew Hodgson2016-04-141-5/+5
|
* comment out 2c838f6459db35ad9812a83184d85a06ca5d940a due to risk of ↵Matthew Hodgson2016-04-141-16/+16
| | | | https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve
* Sanitize the optional dependencies for spider APIErik Johnston2016-04-132-27/+3
|
* PEP8Erik Johnston2016-04-111-1/+0
|
* fix typos and needless try/except from PR reviewMatthew Hodgson2016-04-111-143/+140
|
* actually throw meaningful errorsMatthew Hodgson2016-04-081-9/+24
|
* Fix pep8 warningMark Haines2016-04-081-3/+4
|
* more PR feedbackMatthew Hodgson2016-04-081-4/+8
|
* Add url_preview_enabled config option to turn on/off preview_url endpoint. ↵Matthew Hodgson2016-04-082-18/+64
| | | | | | | | | defaults to off. Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered. Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered. Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist Add commentary and generally address PR feedback
* report image size (bytewise) in OG metaMatthew Hodgson2016-04-031-0/+2
|
* char encodingMatthew Hodgson2016-04-031-0/+1
|
* pep8Matthew Hodgson2016-04-032-55/+73
|
* fix etag typing error. fix timestamp typing errorMatthew Hodgson2016-04-031-2/+2
|
* rebase all image URLsMatthew Hodgson2016-04-031-4/+4
|
* remove stale todoMatthew Hodgson2016-04-031-3/+0
|
* Ensure only one download for a given URL is active at a timeMatthew Hodgson2016-04-031-1/+18
|
* add a persistent cache of URL lookups, and fix up the in-memory one to workMatthew Hodgson2016-04-031-10/+54
|
* support gzip compression, and don't pass through error msgsMatthew Hodgson2016-04-021-2/+3
|
* how was _respond_default_thumbnail ever meant to work?Matthew Hodgson2016-04-021-0/+2
|
* pass back SVGs as their own thumbnailsMatthew Hodgson2016-04-021-0/+20
|
* handle missing dimensions without crashingMatthew Hodgson2016-04-021-4/+13
|
* make meta comparisons case insensitiveMatthew Hodgson2016-04-021-2/+2
|
* refactor calc_og; spider image URLs; fix xpath; add a (broken) ↵Matthew Hodgson2016-04-021-81/+121
| | | | expiringcache; loads of other fixes
* fix assorted redirect, unicode and screenscraping bugsMatthew Hodgson2016-04-011-78/+96
|
* handle spidered relative images correctlyMatthew Hodgson2016-03-311-2/+12
|
* prevent choking on invalid utf-8, and handle image thumbnailing smarterMatthew Hodgson2016-03-311-5/+18
|
* synthesise basig OG metadata from pages lacking itMatthew Hodgson2016-03-311-0/+47
|
* handle requests with missing content-length headers (e.g. YouTube)Matthew Hodgson2016-03-311-2/+2
|
* fix bugsMatthew Hodgson2016-03-291-5/+6
|
* debugMatthew Hodgson2016-03-291-3/+2
|
* make it workMatthew Hodgson2016-03-292-55/+77
|
* make it buildMatthew Hodgson2016-03-291-8/+9
|
* Merge branch 'develop' into matthew/preview_urlsMatthew Hodgson2016-03-272-8/+11
|\
| * Catch the exceptions thrown by twisted when you write to a closed connectionMark Haines2016-02-122-4/+4
| |
| * Preserve log context over when deferring to thread pool in media repoErik Johnston2016-02-031-4/+7
| |
* | initial WIP of a tentative preview_url endpoint - incomplete, untested, ↵Matthew Hodgson2016-01-242-0/+167
|/ | | | experimental, etc. just putting it here for safekeeping for now
* Merge pull request #478 from matrix-org/daniel/userobjectDaniel Wagner-Hall2016-01-112-5/+5
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a User object I'm sick of passing around more and more things as tuple items around the whole world, and needing to edit every call site every time there is more information about a user. So pass them around together as an object. This object has incredibly poorly named fields because we have a convention that `user` indicates a UserID object, and `user_id` indicates a string. I tried to clean up the whole repo to fix this, but gave up. So instead, I introduce a second convention. A user_object is a User, and a user_id_object is a UserId. I may have cried a little bit.
| * Introduce a Requester objectDaniel Wagner-Hall2016-01-112-5/+5
| | | | | | | | | | | | | | | | | | This tracks data about the entity which made the request. This is instead of passing around a tuple, which requires call-site modifications every time a new piece of optional context is passed around. I tried to introduce a User object. I gave up.
* | Only use cropped thumbnails when asked for a cropped thumbnail.Mark Haines2016-01-071-1/+1
|/ | | | Even though ones cropped with scale might be technically valid.
* copyrightsMatthew Hodgson2016-01-079-9/+9
|
* Allow guests to upgrade their accountsDaniel Wagner-Hall2016-01-051-1/+1
|
* Use larger thumbnail rather than smaller.Erik Johnston2016-01-051-4/+13
|
* Allow guests to register and call /events?room_id=Daniel Wagner-Hall2015-11-042-2/+2
| | | | | | | This follows the same flows-based flow as regular registration, but as the only implemented flow has no requirements, it auto-succeeds. In the future, other flows (e.g. captcha) may be required, so clients should treat this like the regular registration flow choices.
* Merge branch 'release-v0.10.0' into developMark Haines2015-08-282-15/+51
|\ | | | | | | | | | | | | Conflicts: synapse/handlers/auth.py synapse/python_dependencies.py synapse/rest/client/v1/login.py
| * ConsistencyErik Johnston2015-08-271-3/+3
| |
| * Check for an internationalised filename firstErik Johnston2015-08-271-8/+14
| |
| * None check the correct variableErik Johnston2015-08-271-2/+3
| |
| * Check for NoneErik Johnston2015-08-271-1/+1
| |
| * Give a sensible error message if the filename is invalid UTF-8Mark Haines2015-08-262-2/+8
| |
| * Handle unicode filenames given when downloading or received over federationMark Haines2015-08-261-5/+21
| |
| * Allow non-ascii filenames for attachmentsMark Haines2015-08-262-8/+15
| |
* | Remove completely unused concepts from codebaseDaniel Wagner-Hall2015-08-252-2/+2
|/ | | | | | | | | | Removes device_id and ClientInfo device_id is never actually written, and the matrix.org DB has no non-null entries for it. Right now, it's just cluttering up code. This doesn't remove the columns from the database, because that's fiddly.
* Merge remote-tracking branch 'origin/develop' into ↵Mark Haines2015-08-131-34/+43
|\ | | | | | | erikj/generate_presice_thumbnails
| * Generate local thumbnails on a threadErik Johnston2015-07-231-34/+43
| |
* | Add config option for setting the list of thumbnail sizes to precalculateMark Haines2015-08-121-16/+2
| |
* | Make a config option for whether to generate new thumbnail sizes dynamicallyMark Haines2015-08-122-7/+19
| |
* | Factor out thumbnail()Erik Johnston2015-07-231-56/+40
| |
* | Fix remote thumbnailingErik Johnston2015-07-232-7/+11
| |
* | Always return a thumbnail of the requested size.Erik Johnston2015-07-232-2/+162
|/ | | | | | Before, we returned a thumbnail that was at least as big (if possible) as the requested size. Now, if we don't have a thumbnail of the given size we generate (and persist) one of that size.
* Pick larger than desired thumbnail for 'crop'Erik Johnston2015-07-231-1/+2
|
* bump up image quality a bit more as it looks crapMatthew Hodgson2015-07-141-1/+1
|
* If user supplies filename in URL when downloading from media repo, use that ↵Erik Johnston2015-07-033-9/+14
| | | | name in Content Disposition
* Enforce ascii filenames for uploadsErik Johnston2015-06-302-1/+6
|
* Add Content-Disposition headers to media repo v1 downloadsErik Johnston2015-06-303-7/+30
|
* Merge pull request #175 from matrix-org/erikj/thumbnail_threadErik Johnston2015-06-031-45/+53
|\ | | | | Thumbnail images on a seperate thread
| * Thumbnail images on a seperate threadErik Johnston2015-06-021-45/+53
| |
* | SYN-403: Make content repository use its own http client.Erik Johnston2015-06-031-1/+2
|/
* Change the way we create observers to deferreds so that we don't get spammed ↵Erik Johnston2015-05-081-2/+6
| | | | by 'unhandled errors'
* Implement locks using create_observer for fetching media and server keysErik Johnston2015-04-271-1/+3
|
* Copyright noticesMark Haines2015-04-241-0/+14
|
* Set a version_string in BaseMediaResource so that the request_handler ↵Mark Haines2015-04-211-0/+1
| | | | wrapper works
* Combine the request wrappers in rest/media/v1 and http/server into a single ↵Mark Haines2015-04-214-131/+62
| | | | wrapper decorator
* SYN-282: Don't log tracebacks for client errorsMark Haines2015-02-181-1/+1
|
* Blunty replace json with simplejsonErik Johnston2015-02-111-1/+1
|
* Fix code-styleMark Haines2015-02-101-2/+2
|
* Code-style fixesMark Haines2015-02-101-3/+4
|
* ...and here's the actual impl. git fail.Matthew Hodgson2015-02-071-25/+32
|
* thou shalt specify a content-lengthMatthew Hodgson2015-02-071-0/+1
|
* i hate weakly typed languagesMatthew Hodgson2015-02-071-1/+1
|
* Add Cache-Control header to identiconMark Haines2015-02-021-0/+3
|
* Fix setting identicon width and heightMark Haines2015-02-021-2/+2
|
* Spell height more correctlyMark Haines2015-02-021-1/+1
|
* Add a media/v1/identicon resource for generating identicons using pydenticonMark Haines2015-02-022-0/+50
|
* Extract the id token of the token when authing users, include the token and ↵Mark Haines2015-01-282-2/+2
| | | | device_id in the internal meta data for the event along with the transaction id when sending events
* Return the device_id from get_auth_by_reqMark Haines2015-01-282-2/+2
|
* Move rest APIs back under the rest directoryMark Haines2015-01-2211-0/+1248
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-05 16:46:08 +0000