Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Fix up comments | Erik Johnston | 2017-10-13 | 1 | -2/+26 |
| | |||||
* | Move makedirs to thread | Erik Johnston | 2017-10-13 | 1 | -2/+1 |
| | |||||
* | Use make_deferred_yieldable | Erik Johnston | 2017-10-13 | 1 | -7/+7 |
| | |||||
* | Fix comments | Erik Johnston | 2017-10-12 | 2 | -1/+4 |
| | |||||
* | Don't close prematurely.. | Erik Johnston | 2017-10-12 | 1 | -2/+5 |
| | |||||
* | getvalue closes buffer | Erik Johnston | 2017-10-12 | 1 | -11/+10 |
| | |||||
* | Get len before close | Erik Johnston | 2017-10-12 | 1 | -6/+11 |
| | |||||
* | Don't close file prematurely | Erik Johnston | 2017-10-12 | 2 | -8/+18 |
| | |||||
* | Copy everything to backup | Erik Johnston | 2017-10-12 | 4 | -76/+148 |
| | |||||
* | Typo | Erik Johnston | 2017-10-12 | 1 | -1/+1 |
| | |||||
* | Remove dead code | Erik Johnston | 2017-10-12 | 1 | -3/+0 |
| | |||||
* | Fix typos | Erik Johnston | 2017-10-12 | 1 | -20/+26 |
| | |||||
* | Fix typo | Erik Johnston | 2017-10-12 | 1 | -1/+1 |
| | |||||
* | Basic implementation of backup media store | Erik Johnston | 2017-10-12 | 3 | -126/+113 |
| | |||||
* | Only log if we've removed media | Erik Johnston | 2017-09-28 | 1 | -2/+4 |
| | |||||
* | Clear up expired url cache every 10s | Erik Johnston | 2017-09-28 | 1 | -1/+1 |
| | |||||
* | Actually assign result to variable | Erik Johnston | 2017-09-28 | 1 | -1/+1 |
| | |||||
* | Fix typo in url cache expiry timer | Erik Johnston | 2017-09-28 | 1 | -1/+1 |
| | |||||
* | Support new and old style media id formats | Erik Johnston | 2017-09-28 | 1 | -31/+81 |
| | |||||
* | Change expires column to expires_ts | Erik Johnston | 2017-09-28 | 1 | -2/+2 |
| | |||||
* | Delete expired url cache data | Erik Johnston | 2017-09-28 | 2 | -4/+129 |
| | |||||
* | Store URL cache preview downloads seperately | Erik Johnston | 2017-06-23 | 5 | -18/+72 |
| | | | | This makes it easier to clear old media out at a later date | ||||
* | Add API to quarantine media | Erik Johnston | 2017-06-19 | 3 | -3/+5 |
| | |||||
* | Throw exception when not retrying when downloading media | Erik Johnston | 2017-06-13 | 1 | -0/+1 |
| | |||||
* | actually trim oversize og:description meta | Matthew Hodgson | 2017-05-22 | 1 | -0/+2 |
| | |||||
* | Don't log exceptions for NotRetryingDestination | Erik Johnston | 2017-05-15 | 1 | -1/+3 |
| | |||||
* | Address review comments | Richard van der Hoff | 2017-03-14 | 1 | -15/+17 |
| | | | | | | | - don't blindly proxy all HTTPRequestExceptions - log unexpected exceptions at error - avoid `isinstance` - improve docs on `from_http_response_exception` | ||||
* | Fix routing loop when fetching remote media | Richard van der Hoff | 2017-03-13 | 2 | -4/+38 |
| | | | | | | | | | | | When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502. | ||||
* | Fix dynamic thumbnails aspect | Jurek | 2017-02-24 | 1 | -0/+3 |
| | |||||
* | Log which files we saved attachments to in the media_repository | Mark Haines | 2017-01-10 | 3 | -0/+11 |
| | |||||
* | Name linearizer's for better logs | Erik Johnston | 2017-01-09 | 1 | -1/+1 |
| | |||||
* | Fix crash in url preview when html tag has no text | Marcin Bachry | 2016-12-14 | 1 | -1/+4 |
| | | | | Signed-off-by: Marcin Bachry <hegel666@gmail.com> | ||||
* | preview_url_resource: Ellipsis must be in unicode string | Johannes Löthberg | 2016-12-01 | 1 | -1/+1 |
| | | | | Signed-off-by: Johannes Löthberg <johannes@kyriasis.com> | ||||
* | Set CORs headers on responses from the media repo | Mark Haines | 2016-11-02 | 2 | -2/+4 |
| | |||||
* | Add quotes and be explicity about script-src | Erik Johnston | 2016-09-05 | 1 | -1/+2 |
| | |||||
* | Allow PDF to be rendered from media repo | Erik Johnston | 2016-09-05 | 1 | -1/+7 |
| | |||||
* | Set `Content-Security-Policy` on media repo | Erik Johnston | 2016-08-17 | 1 | -0/+1 |
| | | | | | This is to inform browsers that they should sandbox the returned media. This is particularly cruical for javascript/HTML files. | ||||
* | Add None check to _iterate_over_text | Erik Johnston | 2016-08-17 | 1 | -1/+1 |
| | |||||
* | Flake8 | Erik Johnston | 2016-08-16 | 1 | -4/+1 |
| | |||||
* | Fix up preview URL API. Add tests. | Erik Johnston | 2016-08-16 | 1 | -157/+196 |
| | | | | | | | | | This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text. | ||||
* | Don't include html comments in description | Erik Johnston | 2016-08-05 | 1 | -1/+2 |
| | |||||
* | Typo | Erik Johnston | 2016-08-04 | 1 | -1/+1 |
| | |||||
* | Test summarization | Erik Johnston | 2016-08-04 | 1 | -52/+54 |
| | |||||
* | Don't infer paragrahs from newlines | Erik Johnston | 2016-08-02 | 1 | -2/+1 |
| | |||||
* | Comment on why we clone | Erik Johnston | 2016-08-02 | 1 | -0/+1 |
| | |||||
* | Spelling. | Erik Johnston | 2016-08-02 | 1 | -3/+3 |
| | |||||
* | Make it actually compile | Erik Johnston | 2016-08-02 | 1 | -1/+1 |
| | |||||
* | Change the way we summarize URLs | Erik Johnston | 2016-08-02 | 1 | -11/+67 |
| | | | | | | | | Using XPath is slow on some machines (for unknown reasons), so use a different approach to get a list of text nodes. Try to generate a summary that respect paragraph and then word boundaries, adding ellipses when appropriate. | ||||
* | Remove race | Erik Johnston | 2016-06-29 | 1 | -11/+0 |
| | |||||
* | Implement purge_media_cache admin API | Erik Johnston | 2016-06-29 | 2 | -20/+64 |
| | |||||
* | Track approximate last access time for remote media | Erik Johnston | 2016-06-29 | 1 | -0/+24 |
| | |||||
* | Remove the legacy v0 content upload API. | Mark Haines | 2016-06-21 | 1 | -110/+2 |
| | | | | | | The existing content can still be downloaded. The last upload to the matrix.org server was in January 2015, so it is probably safe to remove the upload API. | ||||
* | Line lengths | Erik Johnston | 2016-06-15 | 1 | -3/+3 |
| | |||||
* | Handle floats as img widths | Erik Johnston | 2016-06-15 | 1 | -1/+1 |
| | |||||
* | Handle og props with not content | Erik Johnston | 2016-06-15 | 1 | -1/+2 |
| | |||||
* | 502 on /thumbnail when can't contact remote server | Erik Johnston | 2016-06-09 | 1 | -4/+10 |
| | |||||
* | Clean up the blacklist/whitelist handling. | Mark Haines | 2016-05-16 | 1 | -33/+30 |
| | | | | | | | Always set the config key with an empty list, even if a list isn't specified. This means that the codepaths are the same for both the empty list and for a missing key. Since the behaviour is the same for both cases this makes the code somewhat easier to reason about. | ||||
* | Report per request metrics for all of the things using request_handler | Mark Haines | 2016-04-28 | 4 | -4/+7 |
| | |||||
* | Add self.media_repo to PreviewUrlResource | Erik Johnston | 2016-04-19 | 1 | -0/+1 |
| | |||||
* | _make_dirs was moved to MediaRepository | Erik Johnston | 2016-04-19 | 1 | -1/+1 |
| | |||||
* | Add store to PreviewUrlResource | Erik Johnston | 2016-04-19 | 1 | -0/+1 |
| | |||||
* | Reorder imports | Erik Johnston | 2016-04-19 | 1 | -7/+8 |
| | |||||
* | Move MediaRepository to media_repository module | Erik Johnston | 2016-04-19 | 5 | -489/+496 |
| | |||||
* | Split out BaseMediaResource into MediaRepository | Erik Johnston | 2016-04-19 | 6 | -138/+180 |
| | | | | | | | | | This is so that a single MediaRepository can be shared across all resources, rather than having a "copy" per resource. In particular this allows us to guard against both the thumbnail and download resource triggering a download of remote content at the same time. | ||||
* | explicitly pass in the charset from Content-Type to lxml to fix cyrillic ↵ | Matthew Hodgson | 2016-04-15 | 1 | -6/+18 |
| | | | | woes better | ||||
* | fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, ↵ | Matthew Hodgson | 2016-04-15 | 1 | -16/+8 |
| | | | | rather than relying on lxml's heuristics which seem to get it wrong | ||||
* | fix urlparse import thinko breaking tiny URLs | Matthew Hodgson | 2016-04-14 | 1 | -5/+5 |
| | |||||
* | comment out 2c838f6459db35ad9812a83184d85a06ca5d940a due to risk of ↵ | Matthew Hodgson | 2016-04-14 | 1 | -16/+16 |
| | | | | https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve | ||||
* | Sanitize the optional dependencies for spider API | Erik Johnston | 2016-04-13 | 2 | -27/+3 |
| | |||||
* | PEP8 | Erik Johnston | 2016-04-11 | 1 | -1/+0 |
| | |||||
* | fix typos and needless try/except from PR review | Matthew Hodgson | 2016-04-11 | 1 | -143/+140 |
| | |||||
* | actually throw meaningful errors | Matthew Hodgson | 2016-04-08 | 1 | -9/+24 |
| | |||||
* | Fix pep8 warning | Mark Haines | 2016-04-08 | 1 | -3/+4 |
| | |||||
* | more PR feedback | Matthew Hodgson | 2016-04-08 | 1 | -4/+8 |
| | |||||
* | Add url_preview_enabled config option to turn on/off preview_url endpoint. ↵ | Matthew Hodgson | 2016-04-08 | 2 | -18/+64 |
| | | | | | | | | | defaults to off. Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered. Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered. Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist Add commentary and generally address PR feedback | ||||
* | report image size (bytewise) in OG meta | Matthew Hodgson | 2016-04-03 | 1 | -0/+2 |
| | |||||
* | char encoding | Matthew Hodgson | 2016-04-03 | 1 | -0/+1 |
| | |||||
* | pep8 | Matthew Hodgson | 2016-04-03 | 2 | -55/+73 |
| | |||||
* | fix etag typing error. fix timestamp typing error | Matthew Hodgson | 2016-04-03 | 1 | -2/+2 |
| | |||||
* | rebase all image URLs | Matthew Hodgson | 2016-04-03 | 1 | -4/+4 |
| | |||||
* | remove stale todo | Matthew Hodgson | 2016-04-03 | 1 | -3/+0 |
| | |||||
* | Ensure only one download for a given URL is active at a time | Matthew Hodgson | 2016-04-03 | 1 | -1/+18 |
| | |||||
* | add a persistent cache of URL lookups, and fix up the in-memory one to work | Matthew Hodgson | 2016-04-03 | 1 | -10/+54 |
| | |||||
* | support gzip compression, and don't pass through error msgs | Matthew Hodgson | 2016-04-02 | 1 | -2/+3 |
| | |||||
* | how was _respond_default_thumbnail ever meant to work? | Matthew Hodgson | 2016-04-02 | 1 | -0/+2 |
| | |||||
* | pass back SVGs as their own thumbnails | Matthew Hodgson | 2016-04-02 | 1 | -0/+20 |
| | |||||
* | handle missing dimensions without crashing | Matthew Hodgson | 2016-04-02 | 1 | -4/+13 |
| | |||||
* | make meta comparisons case insensitive | Matthew Hodgson | 2016-04-02 | 1 | -2/+2 |
| | |||||
* | refactor calc_og; spider image URLs; fix xpath; add a (broken) ↵ | Matthew Hodgson | 2016-04-02 | 1 | -81/+121 |
| | | | | expiringcache; loads of other fixes | ||||
* | fix assorted redirect, unicode and screenscraping bugs | Matthew Hodgson | 2016-04-01 | 1 | -78/+96 |
| | |||||
* | handle spidered relative images correctly | Matthew Hodgson | 2016-03-31 | 1 | -2/+12 |
| | |||||
* | prevent choking on invalid utf-8, and handle image thumbnailing smarter | Matthew Hodgson | 2016-03-31 | 1 | -5/+18 |
| | |||||
* | synthesise basig OG metadata from pages lacking it | Matthew Hodgson | 2016-03-31 | 1 | -0/+47 |
| | |||||
* | handle requests with missing content-length headers (e.g. YouTube) | Matthew Hodgson | 2016-03-31 | 1 | -2/+2 |
| | |||||
* | fix bugs | Matthew Hodgson | 2016-03-29 | 1 | -5/+6 |
| | |||||
* | debug | Matthew Hodgson | 2016-03-29 | 1 | -3/+2 |
| | |||||
* | make it work | Matthew Hodgson | 2016-03-29 | 2 | -55/+77 |
| | |||||
* | make it build | Matthew Hodgson | 2016-03-29 | 1 | -8/+9 |
| | |||||
* | Merge branch 'develop' into matthew/preview_urls | Matthew Hodgson | 2016-03-27 | 2 | -8/+11 |
|\ | |||||
| * | Catch the exceptions thrown by twisted when you write to a closed connection | Mark Haines | 2016-02-12 | 2 | -4/+4 |
| | | |||||
| * | Preserve log context over when deferring to thread pool in media repo | Erik Johnston | 2016-02-03 | 1 | -4/+7 |
| | | |||||
* | | initial WIP of a tentative preview_url endpoint - incomplete, untested, ↵ | Matthew Hodgson | 2016-01-24 | 2 | -0/+167 |
|/ | | | | experimental, etc. just putting it here for safekeeping for now | ||||
* | Merge pull request #478 from matrix-org/daniel/userobject | Daniel Wagner-Hall | 2016-01-11 | 2 | -5/+5 |
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a User object I'm sick of passing around more and more things as tuple items around the whole world, and needing to edit every call site every time there is more information about a user. So pass them around together as an object. This object has incredibly poorly named fields because we have a convention that `user` indicates a UserID object, and `user_id` indicates a string. I tried to clean up the whole repo to fix this, but gave up. So instead, I introduce a second convention. A user_object is a User, and a user_id_object is a UserId. I may have cried a little bit. | ||||
| * | Introduce a Requester object | Daniel Wagner-Hall | 2016-01-11 | 2 | -5/+5 |
| | | | | | | | | | | | | | | | | | | This tracks data about the entity which made the request. This is instead of passing around a tuple, which requires call-site modifications every time a new piece of optional context is passed around. I tried to introduce a User object. I gave up. | ||||
* | | Only use cropped thumbnails when asked for a cropped thumbnail. | Mark Haines | 2016-01-07 | 1 | -1/+1 |
|/ | | | | Even though ones cropped with scale might be technically valid. | ||||
* | copyrights | Matthew Hodgson | 2016-01-07 | 9 | -9/+9 |
| | |||||
* | Allow guests to upgrade their accounts | Daniel Wagner-Hall | 2016-01-05 | 1 | -1/+1 |
| | |||||
* | Use larger thumbnail rather than smaller. | Erik Johnston | 2016-01-05 | 1 | -4/+13 |
| | |||||
* | Allow guests to register and call /events?room_id= | Daniel Wagner-Hall | 2015-11-04 | 2 | -2/+2 |
| | | | | | | | This follows the same flows-based flow as regular registration, but as the only implemented flow has no requirements, it auto-succeeds. In the future, other flows (e.g. captcha) may be required, so clients should treat this like the regular registration flow choices. | ||||
* | Merge branch 'release-v0.10.0' into develop | Mark Haines | 2015-08-28 | 2 | -15/+51 |
|\ | | | | | | | | | | | | | Conflicts: synapse/handlers/auth.py synapse/python_dependencies.py synapse/rest/client/v1/login.py | ||||
| * | Consistency | Erik Johnston | 2015-08-27 | 1 | -3/+3 |
| | | |||||
| * | Check for an internationalised filename first | Erik Johnston | 2015-08-27 | 1 | -8/+14 |
| | | |||||
| * | None check the correct variable | Erik Johnston | 2015-08-27 | 1 | -2/+3 |
| | | |||||
| * | Check for None | Erik Johnston | 2015-08-27 | 1 | -1/+1 |
| | | |||||
| * | Give a sensible error message if the filename is invalid UTF-8 | Mark Haines | 2015-08-26 | 2 | -2/+8 |
| | | |||||
| * | Handle unicode filenames given when downloading or received over federation | Mark Haines | 2015-08-26 | 1 | -5/+21 |
| | | |||||
| * | Allow non-ascii filenames for attachments | Mark Haines | 2015-08-26 | 2 | -8/+15 |
| | | |||||
* | | Remove completely unused concepts from codebase | Daniel Wagner-Hall | 2015-08-25 | 2 | -2/+2 |
|/ | | | | | | | | | | Removes device_id and ClientInfo device_id is never actually written, and the matrix.org DB has no non-null entries for it. Right now, it's just cluttering up code. This doesn't remove the columns from the database, because that's fiddly. | ||||
* | Merge remote-tracking branch 'origin/develop' into ↵ | Mark Haines | 2015-08-13 | 1 | -34/+43 |
|\ | | | | | | | erikj/generate_presice_thumbnails | ||||
| * | Generate local thumbnails on a thread | Erik Johnston | 2015-07-23 | 1 | -34/+43 |
| | | |||||
* | | Add config option for setting the list of thumbnail sizes to precalculate | Mark Haines | 2015-08-12 | 1 | -16/+2 |
| | | |||||
* | | Make a config option for whether to generate new thumbnail sizes dynamically | Mark Haines | 2015-08-12 | 2 | -7/+19 |
| | | |||||
* | | Factor out thumbnail() | Erik Johnston | 2015-07-23 | 1 | -56/+40 |
| | | |||||
* | | Fix remote thumbnailing | Erik Johnston | 2015-07-23 | 2 | -7/+11 |
| | | |||||
* | | Always return a thumbnail of the requested size. | Erik Johnston | 2015-07-23 | 2 | -2/+162 |
|/ | | | | | | Before, we returned a thumbnail that was at least as big (if possible) as the requested size. Now, if we don't have a thumbnail of the given size we generate (and persist) one of that size. | ||||
* | Pick larger than desired thumbnail for 'crop' | Erik Johnston | 2015-07-23 | 1 | -1/+2 |
| | |||||
* | bump up image quality a bit more as it looks crap | Matthew Hodgson | 2015-07-14 | 1 | -1/+1 |
| | |||||
* | If user supplies filename in URL when downloading from media repo, use that ↵ | Erik Johnston | 2015-07-03 | 3 | -9/+14 |
| | | | | name in Content Disposition | ||||
* | Enforce ascii filenames for uploads | Erik Johnston | 2015-06-30 | 2 | -1/+6 |
| | |||||
* | Add Content-Disposition headers to media repo v1 downloads | Erik Johnston | 2015-06-30 | 3 | -7/+30 |
| | |||||
* | Merge pull request #175 from matrix-org/erikj/thumbnail_thread | Erik Johnston | 2015-06-03 | 1 | -45/+53 |
|\ | | | | | Thumbnail images on a seperate thread | ||||
| * | Thumbnail images on a seperate thread | Erik Johnston | 2015-06-02 | 1 | -45/+53 |
| | | |||||
* | | SYN-403: Make content repository use its own http client. | Erik Johnston | 2015-06-03 | 1 | -1/+2 |
|/ | |||||
* | Change the way we create observers to deferreds so that we don't get spammed ↵ | Erik Johnston | 2015-05-08 | 1 | -2/+6 |
| | | | | by 'unhandled errors' | ||||
* | Implement locks using create_observer for fetching media and server keys | Erik Johnston | 2015-04-27 | 1 | -1/+3 |
| | |||||
* | Copyright notices | Mark Haines | 2015-04-24 | 1 | -0/+14 |
| | |||||
* | Set a version_string in BaseMediaResource so that the request_handler ↵ | Mark Haines | 2015-04-21 | 1 | -0/+1 |
| | | | | wrapper works | ||||
* | Combine the request wrappers in rest/media/v1 and http/server into a single ↵ | Mark Haines | 2015-04-21 | 4 | -131/+62 |
| | | | | wrapper decorator | ||||
* | SYN-282: Don't log tracebacks for client errors | Mark Haines | 2015-02-18 | 1 | -1/+1 |
| | |||||
* | Blunty replace json with simplejson | Erik Johnston | 2015-02-11 | 1 | -1/+1 |
| | |||||
* | Fix code-style | Mark Haines | 2015-02-10 | 1 | -2/+2 |
| | |||||
* | Code-style fixes | Mark Haines | 2015-02-10 | 1 | -3/+4 |
| | |||||
* | ...and here's the actual impl. git fail. | Matthew Hodgson | 2015-02-07 | 1 | -25/+32 |
| | |||||
* | thou shalt specify a content-length | Matthew Hodgson | 2015-02-07 | 1 | -0/+1 |
| | |||||
* | i hate weakly typed languages | Matthew Hodgson | 2015-02-07 | 1 | -1/+1 |
| | |||||
* | Add Cache-Control header to identicon | Mark Haines | 2015-02-02 | 1 | -0/+3 |
| | |||||
* | Fix setting identicon width and height | Mark Haines | 2015-02-02 | 1 | -2/+2 |
| | |||||
* | Spell height more correctly | Mark Haines | 2015-02-02 | 1 | -1/+1 |
| | |||||
* | Add a media/v1/identicon resource for generating identicons using pydenticon | Mark Haines | 2015-02-02 | 2 | -0/+50 |
| | |||||
* | Extract the id token of the token when authing users, include the token and ↵ | Mark Haines | 2015-01-28 | 2 | -2/+2 |
| | | | | device_id in the internal meta data for the event along with the transaction id when sending events | ||||
* | Return the device_id from get_auth_by_req | Mark Haines | 2015-01-28 | 2 | -2/+2 |
| | |||||
* | Move rest APIs back under the rest directory | Mark Haines | 2015-01-22 | 11 | -0/+1248 |