summary refs log tree commit diff
path: root/synapse
diff options
context:
space:
mode:
authorAaron Raimist <aaron@raim.ist>2020-12-08 16:51:03 -0600
committerGitHub <noreply@github.com>2020-12-08 22:51:03 +0000
commitcd9e72b185e36ac3f18ab1fe567fdeee87bfcb8e (patch)
treedde78414e8ab929c0c003fbc20dcab5ee2cef077 /synapse
parentBetter formatting for config errors from modules (#8874) (diff)
downloadsynapse-cd9e72b185e36ac3f18ab1fe567fdeee87bfcb8e.tar.xz
Add X-Robots-Tag header to stop crawlers from indexing media (#8887)
Fixes / related to: https://github.com/matrix-org/synapse/issues/6533

This should do essentially the same thing as a robots.txt file telling robots to not index the media repo. https://developers.google.com/search/reference/robots_meta_tag

Signed-off-by: Aaron Raimist <aaron@raim.ist>
Diffstat (limited to 'synapse')
-rw-r--r--synapse/rest/media/v1/_base.py5
1 files changed, 5 insertions, 0 deletions
diff --git a/synapse/rest/media/v1/_base.py b/synapse/rest/media/v1/_base.py
index 67aa993f19..47c2b44bff 100644
--- a/synapse/rest/media/v1/_base.py
+++ b/synapse/rest/media/v1/_base.py
@@ -155,6 +155,11 @@ def add_file_headers(request, media_type, file_size, upload_name):
     request.setHeader(b"Cache-Control", b"public,max-age=86400,s-maxage=86400")
     request.setHeader(b"Content-Length", b"%d" % (file_size,))
 
+    # Tell web crawlers to not index, archive, or follow links in media. This
+    # should help to prevent things in the media repo from showing up in web
+    # search results.
+    request.setHeader(b"X-Robots-Tag", "noindex, nofollow, noarchive, noimageindex")
+
 
 # separators as defined in RFC2616. SP and HT are handled separately.
 # see _can_encode_filename_as_token.