From be84aeacf620d9eb01de07a52d67bd4035985fbd Mon Sep 17 00:00:00 2001 From: reivilibre Date: Tue, 2 Aug 2022 10:56:24 +0000 Subject: deploy: c2f48712265de92bd833a15f86935bed1f1efbe5 --- latest/development/cas.html | 2 +- latest/development/contributing_guide.html | 2 +- latest/development/database_schema.html | 2 +- latest/development/demo.html | 2 +- latest/development/dependencies.html | 17 +- latest/development/experimental_features.html | 2 +- latest/development/git.html | 2 +- .../development/internal_documentation/index.html | 2 +- latest/development/releases.html | 2 +- latest/development/reviews.html | 2 +- latest/development/room-dag-concepts.html | 2 +- latest/development/saml.html | 2 +- .../synapse_architecture/cancellation.html | 2 +- latest/development/url_previews.html | 273 --------------------- 14 files changed, 28 insertions(+), 286 deletions(-) delete mode 100644 latest/development/url_previews.html (limited to 'latest/development') diff --git a/latest/development/cas.html b/latest/development/cas.html index f8228d64a2..3fa6697f71 100644 --- a/latest/development/cas.html +++ b/latest/development/cas.html @@ -76,7 +76,7 @@ diff --git a/latest/development/contributing_guide.html b/latest/development/contributing_guide.html index bdebf87817..c98d112e67 100644 --- a/latest/development/contributing_guide.html +++ b/latest/development/contributing_guide.html @@ -76,7 +76,7 @@ diff --git a/latest/development/database_schema.html b/latest/development/database_schema.html index 2b2c5ceaec..f6b579226e 100644 --- a/latest/development/database_schema.html +++ b/latest/development/database_schema.html @@ -76,7 +76,7 @@ diff --git a/latest/development/demo.html b/latest/development/demo.html index 88089929c4..8a63aa8b2c 100644 --- a/latest/development/demo.html +++ b/latest/development/demo.html @@ -76,7 +76,7 @@ diff --git a/latest/development/dependencies.html b/latest/development/dependencies.html index 168648a8a0..5bab1846ce 100644 --- a/latest/development/dependencies.html +++ b/latest/development/dependencies.html @@ -76,7 +76,7 @@ @@ -321,6 +321,21 @@ be required.

because build is a standardish tool which doesn't require poetry. (It's what we use in CI too). However, you could try poetry build too.

+

Troubleshooting

+

Check the version of poetry with poetry --version.

+

At the time of writing, the 1.2 series is beta only. We have seen some examples +where the lockfiles generated by 1.2 prereleasese aren't interpreted correctly +by poetry 1.1.x. For now, use poetry 1.1.14, which includes a critical +change needed to remain +compatible with PyPI.

+

It can also be useful to check the version of poetry-core in use. If you've +installed poetry with pipx, try pipx runpip poetry list | grep poetry-core.

+

Clear caches: poetry cache clear --all pypi.

+

Poetry caches a bunch of information about packages that isn't readily available +from PyPI. (This is what makes poetry seem slow when doing the first +poetry install.) Try poetry cache list and poetry cache clear --all <name of cache> to see if that fixes things.

+

Try --verbose or --dry-run arguments.

+

Sometimes useful to see what poetry's internal logic is.

diff --git a/latest/development/experimental_features.html b/latest/development/experimental_features.html index e58d22dba1..81a98d560a 100644 --- a/latest/development/experimental_features.html +++ b/latest/development/experimental_features.html @@ -76,7 +76,7 @@ diff --git a/latest/development/git.html b/latest/development/git.html index 1f6ac5c907..d51a4da527 100644 --- a/latest/development/git.html +++ b/latest/development/git.html @@ -76,7 +76,7 @@ diff --git a/latest/development/internal_documentation/index.html b/latest/development/internal_documentation/index.html index 1de6b5daee..086f5a7504 100644 --- a/latest/development/internal_documentation/index.html +++ b/latest/development/internal_documentation/index.html @@ -76,7 +76,7 @@ diff --git a/latest/development/releases.html b/latest/development/releases.html index 502adcad8e..12565c42ba 100644 --- a/latest/development/releases.html +++ b/latest/development/releases.html @@ -76,7 +76,7 @@ diff --git a/latest/development/reviews.html b/latest/development/reviews.html index bf32996243..77b510813b 100644 --- a/latest/development/reviews.html +++ b/latest/development/reviews.html @@ -76,7 +76,7 @@ diff --git a/latest/development/room-dag-concepts.html b/latest/development/room-dag-concepts.html index 69dc0aa4a3..fd5a054056 100644 --- a/latest/development/room-dag-concepts.html +++ b/latest/development/room-dag-concepts.html @@ -76,7 +76,7 @@ diff --git a/latest/development/saml.html b/latest/development/saml.html index 6c99649b81..cc70db5d39 100644 --- a/latest/development/saml.html +++ b/latest/development/saml.html @@ -76,7 +76,7 @@ diff --git a/latest/development/synapse_architecture/cancellation.html b/latest/development/synapse_architecture/cancellation.html index 0cbce2f205..f5c41c5fe4 100644 --- a/latest/development/synapse_architecture/cancellation.html +++ b/latest/development/synapse_architecture/cancellation.html @@ -76,7 +76,7 @@ diff --git a/latest/development/url_previews.html b/latest/development/url_previews.html deleted file mode 100644 index 8814f5c9da..0000000000 --- a/latest/development/url_previews.html +++ /dev/null @@ -1,273 +0,0 @@ - - - - - - URL Previews - Synapse - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - - - - - - -
-
- -
- -
- -

URL Previews

-

The GET /_matrix/media/r0/preview_url endpoint provides a generic preview API -for URLs which outputs Open Graph responses (with some Matrix -specific additions).

-

This does have trade-offs compared to other designs:

-
    -
  • Pros: -
      -
    • Simple and flexible; can be used by any clients at any point
      • -
    -
    • -
  • Cons: -
      -
    • If each homeserver provides one of these independently, all the HSes in a -room may needlessly DoS the target URI
      • -
    • The URL metadata must be stored somewhere, rather than just using Matrix -itself to store the media.
      • -
    • Matrix cannot be used to distribute the metadata between homeservers.
      • -
    -
    • -
-

When Synapse is asked to preview a URL it does the following:

-
    -
  1. Checks against a URL blacklist (defined as url_preview_url_blacklist in the -config).
    2. -
  2. Checks the in-memory cache by URLs and returns the result if it exists. (This -is also used to de-duplicate processing of multiple in-flight requests at once.)
    3. -
  3. Kicks off a background process to generate a preview: -
      -
    1. Checks the database cache by URL and timestamp and returns the result if it -has not expired and was successful (a 2xx return code).
      2. -
    2. Checks if the URL matches an oEmbed pattern. If it -does, update the URL to download.
      3. -
    3. Downloads the URL and stores it into a file via the media storage provider -and saves the local media metadata.
      4. -
    4. If the media is an image: -
        -
      1. Generates thumbnails.
        2. -
      2. Generates an Open Graph response based on image properties.
        3. -
      -
      5. -
    5. If the media is HTML: -
        -
      1. Decodes the HTML via the stored file.
        2. -
      2. Generates an Open Graph response from the HTML.
        3. -
      3. If a JSON oEmbed URL was found in the HTML via autodiscovery: -
          -
        1. Downloads the URL and stores it into a file via the media storage provider -and saves the local media metadata.
          2. -
        2. Convert the oEmbed response to an Open Graph response.
          3. -
        3. Override any Open Graph data from the HTML with data from oEmbed.
          4. -
        -
        4. -
      4. If an image exists in the Open Graph response: -
          -
        1. Downloads the URL and stores it into a file via the media storage -provider and saves the local media metadata.
          2. -
        2. Generates thumbnails.
          3. -
        3. Updates the Open Graph response based on image properties.
          4. -
        -
        5. -
      -
      6. -
    6. If the media is JSON and an oEmbed URL was found: -
        -
      1. Convert the oEmbed response to an Open Graph response.
        2. -
      2. If a thumbnail or image is in the oEmbed response: -
          -
        1. Downloads the URL and stores it into a file via the media storage -provider and saves the local media metadata.
          2. -
        2. Generates thumbnails.
          3. -
        3. Updates the Open Graph response based on image properties.
          4. -
        -
        3. -
      -
      7. -
    7. Stores the result in the database cache.
      8. -
    -
    4. -
  4. Returns the result.
    5. -
-

The in-memory cache expires after 1 hour.

-

Expired entries in the database cache (and their associated media files) are -deleted every 10 seconds. The default expiration time is 1 hour from download.

- -
- - -
-
- - - -
- - - - - - - - - - - - - \ No newline at end of file -- cgit 1.5.1