diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 4fcd2b7852..44338a78b3 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -34,7 +34,7 @@
- [Application Services](application_services.md)
- [Server Notices](server_notices.md)
- [Consent Tracking](consent_tracking.md)
- - [URL Previews](url_previews.md)
+ - [URL Previews](development/url_previews.md)
- [User Directory](user_directory.md)
- [Message Retention Policies](message_retention_policies.md)
- [Pluggable Modules](modules.md)
diff --git a/docs/development/url_previews.md b/docs/development/url_previews.md
new file mode 100644
index 0000000000..bbe05e281c
--- /dev/null
+++ b/docs/development/url_previews.md
@@ -0,0 +1,51 @@
+URL Previews
+============
+
+The `GET /_matrix/media/r0/preview_url` endpoint provides a generic preview API
+for URLs which outputs [Open Graph](https://ogp.me/) responses (with some Matrix
+specific additions).
+
+This does have trade-offs compared to other designs:
+
+* Pros:
+ * Simple and flexible; can be used by any clients at any point
+* Cons:
+ * If each homeserver provides one of these independently, all the HSes in a
+ room may needlessly DoS the target URI
+ * The URL metadata must be stored somewhere, rather than just using Matrix
+ itself to store the media.
+ * Matrix cannot be used to distribute the metadata between homeservers.
+
+When Synapse is asked to preview a URL it does the following:
+
+1. Checks against a URL blacklist (defined as `url_preview_url_blacklist` in the
+ config).
+2. Checks the in-memory cache by URLs and returns the result if it exists. (This
+ is also used to de-duplicate processing of multiple in-flight requests at once.)
+3. Kicks off a background process to generate a preview:
+ 1. Checks the database cache by URL and timestamp and returns the result if it
+ has not expired and was successful (a 2xx return code).
+ 2. Checks if the URL matches an oEmbed pattern. If it does, fetch the oEmbed
+ response. If this is an image, replace the URL to fetch and continue. If
+ if it is HTML content, use the HTML as the document and continue.
+ 3. If it doesn't match an oEmbed pattern, downloads the URL and stores it
+ into a file via the media storage provider and saves the local media
+ metadata.
+ 5. If the media is an image:
+ 1. Generates thumbnails.
+ 2. Generates an Open Graph response based on image properties.
+ 6. If the media is HTML:
+ 1. Decodes the HTML via the stored file.
+ 2. Generates an Open Graph response from the HTML.
+ 3. If an image exists in the Open Graph response:
+ 1. Downloads the URL and stores it into a file via the media storage
+ provider and saves the local media metadata.
+ 2. Generates thumbnails.
+ 3. Updates the Open Graph response based on image properties.
+ 7. Stores the result in the database cache.
+4. Returns the result.
+
+The in-memory cache expires after 1 hour.
+
+Expired entries in the database cache (and their associated media files) are
+deleted every 10 seconds. The default expiration time is 1 hour from download.
diff --git a/docs/url_previews.md b/docs/url_previews.md
deleted file mode 100644
index 665554e165..0000000000
--- a/docs/url_previews.md
+++ /dev/null
@@ -1,76 +0,0 @@
-URL Previews
-============
-
-Design notes on a URL previewing service for Matrix:
-
-Options are:
-
- 1. Have an AS which listens for URLs, downloads them, and inserts an event that describes their metadata.
- * Pros:
- * Decouples the implementation entirely from Synapse.
- * Uses existing Matrix events & content repo to store the metadata.
- * Cons:
- * Which AS should provide this service for a room, and why should you trust it?
- * Doesn't work well with E2E; you'd have to cut the AS into every room
- * the AS would end up subscribing to every room anyway.
-
- 2. Have a generic preview API (nothing to do with Matrix) that provides a previewing service:
- * Pros:
- * Simple and flexible; can be used by any clients at any point
- * Cons:
- * If each HS provides one of these independently, all the HSes in a room may needlessly DoS the target URI
- * We need somewhere to store the URL metadata rather than just using Matrix itself
- * We can't piggyback on matrix to distribute the metadata between HSes.
-
- 3. Make the synapse of the sending user responsible for spidering the URL and inserting an event asynchronously which describes the metadata.
- * Pros:
- * Works transparently for all clients
- * Piggy-backs nicely on using Matrix for distributing the metadata.
- * No confusion as to which AS
- * Cons:
- * Doesn't work with E2E
- * We might want to decouple the implementation of the spider from the HS, given spider behaviour can be quite complicated and evolve much more rapidly than the HS. It's more like a bot than a core part of the server.
-
- 4. Make the sending client use the preview API and insert the event itself when successful.
- * Pros:
- * Works well with E2E
- * No custom server functionality
- * Lets the client customise the preview that they send (like on FB)
- * Cons:
- * Entirely specific to the sending client, whereas it'd be nice if /any/ URL was correctly previewed if clients support it.
-
- 5. Have the option of specifying a shared (centralised) previewing service used by a room, to avoid all the different HSes in the room DoSing the target.
-
-Best solution is probably a combination of both 2 and 4.
- * Sending clients do their best to create and send a preview at the point of sending the message, perhaps delaying the message until the preview is computed? (This also lets the user validate the preview before sending)
- * Receiving clients have the option of going and creating their own preview if one doesn't arrive soon enough (or if the original sender didn't create one)
-
-This is a bit magical though in that the preview could come from two entirely different sources - the sending HS or your local one. However, this can always be exposed to users: "Generate your own URL previews if none are available?"
-
-This is tantamount also to senders calculating their own thumbnails for sending in advance of the main content - we are trusting the sender not to lie about the content in the thumbnail. Whereas currently thumbnails are calculated by the receiving homeserver to avoid this attack.
-
-However, this kind of phishing attack does exist whether we let senders pick their thumbnails or not, in that a malicious sender can send normal text messages around the attachment claiming it to be legitimate. We could rely on (future) reputation/abuse management to punish users who phish (be it with bogus metadata or bogus descriptions). Bogus metadata is particularly bad though, especially if it's avoidable.
-
-As a first cut, let's do #2 and have the receiver hit the API to calculate its own previews (as it does currently for image thumbnails). We can then extend/optimise this to option 4 as a special extra if needed.
-
-API
----
-
-```
-GET /_matrix/media/r0/preview_url?url=http://wherever.com
-200 OK
-{
- "og:type" : "article"
- "og:url" : "https://twitter.com/matrixdotorg/status/684074366691356672"
- "og:title" : "Matrix on Twitter"
- "og:image" : "https://pbs.twimg.com/profile_images/500400952029888512/yI0qtFi7_400x400.png"
- "og:description" : "“Synapse 0.12 is out! Lots of polishing, performance & bugfixes: /sync API, /r0 prefix, fulltext search, 3PID invites https://t.co/5alhXLLEGP”"
- "og:site_name" : "Twitter"
-}
-```
-
-* Downloads the URL
- * If HTML, just stores it in RAM and parses it for OG meta tags
- * Download any media OG meta tags to the media repo, and refer to them in the OG via mxc:// URIs.
- * If a media filetype we know we can thumbnail: store it on disk, and hand it to the thumbnailer. Generate OG meta tags from the thumbnailer contents.
- * Otherwise, don't bother downloading further.
|
