From 5f8173dd80359da2c289b36e2a1c9d0a3b73c950 Mon Sep 17 00:00:00 2001 From: David Robertson Date: Tue, 12 Apr 2022 10:16:01 +0100 Subject: Workaround pip bug installing latest treq and not-latest twisted from hashes (#12439) The requirements file generated by `poetry export` isn't correctly processed by `pip install -r requirements.txt`. It contains twisted and treq, both pinned to 22.2.0. When `pip` installs treq, it notices that `Twisted[tls]` is required. It then tries to acquire the latest twisted release, only to fail (because this hash isn't listed in the requirements file).From e.g. https://github.com/matrix-org/synapse/runs/5977154990?check_suite_focus=true > ``` > #15 9.204 Collecting Twisted[tls]>=18.7.0 > #15 9.205 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not: > #15 9.205 Twisted[tls]>=18.7.0 from https://files.pythonhosted.org/packages/db/99/38622ff95bb740bcc991f548eb46295bba62fcb6e907db1987c4d92edd09/Twisted-22.4.0-py3-none-any.whl#sha256=f9f7a91f94932477a9fc3b169d57f54f96c6e74a23d78d9ce54039a7f48928a2 (from treq==22.2.0->-r /synapse/requirements.txt (line 724)) > #15 ERROR: executor failed running [/bin/sh -c pip install --prefix="/install" --no-warn-script-location -r /synapse/requirements.txt]: exit code: 1 > ``` The underlying pip issue is https://github.com/pypa/pip/issues/9644. A comment notes that one can avoid this behaviour with by `pip install`ing with the `--no-deps` flag. Let us do so. (At first glance, the problem looks like https://github.com/python-poetry/poetry/issues/5311, but that was a bug in `poetry install`; this is `poetry export`, whose behaviour is fine AFAICS). --- changelog.d/12439.misc | 1 + docker/Dockerfile | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 changelog.d/12439.misc diff --git a/changelog.d/12439.misc b/changelog.d/12439.misc new file mode 100644 index 0000000000..43bc64ed55 --- /dev/null +++ b/changelog.d/12439.misc @@ -0,0 +1 @@ +Bundle locked versions of dependencies into the Docker image. diff --git a/docker/Dockerfile b/docker/Dockerfile index 6009da7db7..6f87702cc8 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -93,7 +93,7 @@ RUN \ # This is aiming at installing the `[tool.poetry.depdendencies]` from pyproject.toml. COPY --from=requirements /synapse/requirements.txt /synapse/ RUN --mount=type=cache,target=/root/.cache/pip \ - pip install --prefix="/install" --no-warn-script-location -r /synapse/requirements.txt + pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requirements.txt # Copy over the rest of the synapse source code. COPY synapse /synapse/synapse/ -- cgit 1.4.1