Build the CI MinIO container from the GHCR mirror (PP-5244) - #3767
Conversation
|
Claude finished @dbernstein's task in 56s —— View job SummaryI found nothing to change. The PR swaps the unpullable |
e3bdc75 to
dff8b78
Compare
|
MinIO has withdrawn anonymous public access to its server image from both Docker Hub and quay.io, so `FROM quay.io/minio/minio:latest` now fails with a 401 before the test suite starts. Pinning an older tag does not help (the whole repository is gated), and neither does a local cache (tox_docker passes pull=True on every build). Points the CI MinIO container at the Palace mirror instead, published from the ci-scripts repo. The mirror is an unmodified MinIO build, so the credentials, ports and command below the FROM line are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dff8b78 to
374f27c
Compare
Comments Outside DiffThese findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.
|
## Description Adds a mirror of the upstream MinIO server image, published as `ghcr.io/thepalaceproject/palace-ci-minio`: - `images/minio/Dockerfile` — assembles the image from MinIO's official GitHub release binaries (`minio` `RELEASE.2025-09-07T16-13-09Z`, `mc` `RELEASE.2025-08-13T08-35-41Z`) plus the AGPL license files, each pinned by sha256 via BuildKit's `ADD --checksum`. - `.github/workflows/build-minio-mirror.yml` — builds `linux/amd64` + `linux/arm64` and pushes to GHCR. Runs only when the Dockerfile or the workflow changes, or on demand. - A README section covering what the image is and how to consume it. This lives in ci-scripts rather than circulation because the image is shared by three repos and belongs to none of them in particular. GHCR packages are namespaced by org, not repo, so one mirror serves all three. Supersedes ThePalaceProject/circulation#3766, which is now closed. ### Design notes - **Assembled from release binaries, not pulled and re-tagged.** Both upstream registries are closed to anonymous pulls, so there is no image left to `docker pull`, and `dl.min.io` is gone too (410). GitHub release assets are the one channel MinIO still serves anonymously. The binaries are byte-for-byte identical to the ones inside the last upstream image CI used — verified below. - **Bare passthrough, no Palace configuration.** circulation and library-registry use one set of credentials, the library registry uses another. Baking any of that in would fork the image between repos immediately, so each repo keeps its own Dockerfile and changes only its `FROM` line. - **Pinned to an immutable tag, no `latest`.** Following a moving upstream tag is part of how we got here. `RELEASE.2025-09-07T16-13-09Z` is also the last MinIO release to publish binaries at all — later tags ship no assets — so there is no newer version to move to. - **Only `main` publishes.** Pull requests and `workflow_dispatch` runs from a branch build both architectures and verify the checksums, then discard the result. An unmerged branch must not be able to overwrite a tag three other repos pin their CI to. - **Public visibility**, consistent with `circ-webapp`, `circ-scripts`, `circ-exec` and `circ-baseimage`, so no `docker login` for developers and no package-access grants for CI. - **`palace-` rather than `circ-` prefix**, because the image is shared across three repos. ###⚠️ One-time manual step after merge GHCR packages are created **private**. Once the workflow has run, set `palace-ci-minio` to public (Org → Packages → `palace-ci-minio` → Package settings). Until that is done, the three consumer PRs will still fail. Consumer PRs, all draft, each a single `FROM` line: ThePalaceProject/circulation#3767, ThePalaceProject/library-registry#1066, ThePalaceProject/virtual-library-card#1016. ## Motivation and Context All tox-docker CI jobs in circulation fail while building the MinIO test container, and the other two repos will hit the same wall on their next run: ``` docker.errors.BuildError: unauthorized: access to the requested resource is not authorized ``` MinIO has progressively withdrawn public distribution — Docker Hub around 13 Sept (worked around in ThePalaceProject/circulation#3728 by moving to quay.io), and quay.io around 24 Sept. Anonymous probes on 24 Sept: | Probe | Result | | --- | --- | | `quay.io/minio/minio:latest` manifest | 401 | | `quay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z` (pinned old tag) | 401 | | `registry-1.docker.io/minio/minio:latest` manifest | 401 | | Docker Hub API for `minio/minio` | `object not found` | | `dl.min.io` server/client binaries | 410 | | `quay.io/prometheus/busybox:latest` (control) | 200 | Quay issues an anonymous pull token and then refuses the manifest, so the repository is gated rather than the network; the control confirms anonymous pulls work from the same machine. Two consequences: pinning a digest or an older tag will not help, because the whole repository is closed; and a local cache will not help, because `tox_docker` passes `pull=True` on every build (`tox_docker/plugin.py`), forcing a fresh pull. Local `tox` fails identically, so developers cannot run the suites either. ### Retirement This mirror is a bridge, not a destination — the intent is to drop MinIO for a maintained S3-compatible image. It should be short-lived for two reasons: we do not want to become a de-facto public distributor of a frozen MinIO build; and GitHub does not allow self-service deletion of a public package once any version exceeds 5,000 downloads, above which it becomes a Support request. With `pull=True` on every build, ephemeral runners and three repos pulling, that threshold arrives quickly. When the time comes: ``` gh api -X DELETE /orgs/ThePalaceProject/packages/container/palace-ci-minio ``` ## How Has This Been Tested? Verified locally on macOS / Docker 29.6.1 (arm64 host). **The mirrored artifacts are identical to the withdrawn upstream image.** A cached copy of `quay.io/minio/minio:latest` (labelled `RELEASE.2025-09-07T16-13-09Z`) was still present locally; its binaries and license files hash the same as what the mirror downloads: ``` # inside the cached upstream image (arm64) # published by MinIO on GitHub 5c83cd2c…f03d /usr/bin/minio 5c83cd2c…f03d minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z 14c8c961…c12c /usr/bin/mc 14c8c961…c12c mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z 0d96a4ff…bcb0 /licenses/LICENSE 0d96a4ff…bcb0 minio/minio@RELEASE.2025-09-07T16-13-09Z:LICENSE 113b8c63…8542 /licenses/CREDITS 113b8c63…8542 minio/minio@RELEASE.2025-09-07T16-13-09Z:CREDITS ``` **Build.** Builds for `linux/amd64` and `linux/arm64`; both architectures' checksums verify (a mismatch fails the build by design). The final stage has no `RUN` instructions, so it cross-builds without QEMU — confirmed on a CI runner that advertised only `linux/amd64…/386` as supported. **Runtime.** `minio --version` and `mc --version` report the expected releases; the server comes up and `/minio/health/live` returns 200 in ~3s; the console on :9001 returns 200; `curl` inside the container returns 200 against the health endpoint (the check `docker-compose.yml` uses); and `mc alias set` / `mc mb` / `mc anonymous set download` all succeed (the operations virtual-library-card's entrypoint needs). `/licenses/LICENSE` and `/licenses/CREDITS` are present and hash-identical to upstream's. **End-to-end through tox.** Because `pull=True` rejects a local-only tag, the image was pushed to a throwaway `registry:2` on localhost and circulation's `docker/Dockerfile.minio.ci` temporarily pointed at it, so the whole chain was exercised — tox building the consumer Dockerfile, pulling the mirror from a real registry, starting the container and running the tests: ``` py312-docker: docker> build .../docker/Dockerfile.minio.ci target 'minio' py312-docker: docker> run 'sha256:fe9e8121edcb' (from 'minio-circ') py312-docker: commands[0]> pytest -m minio --no-cov -q 14 passed in 13.22s ``` The same local-registry substitution was used to build and run library-registry's `docker/Dockerfile.minio.ci` (healthy, `mc` works) and virtual-library-card's `ci/minio/Dockerfile` (entrypoint completes — bucket created, anonymous download policy set, anonymous bucket read returns 200). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3767 +/- ##
==========================================
- Coverage 93.72% 93.71% -0.01%
==========================================
Files 510 510
Lines 46510 46510
Branches 6313 6313
==========================================
- Hits 43590 43589 -1
- Misses 1886 1887 +1
Partials 1034 1034 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This shouldn't be here. I'm talking to Claude about how it might have popped up. Regardless it has nothing to do with this PR. |
Description
Switches
docker/Dockerfile.minio.cifromquay.io/minio/minio:latest, which no longer permitsanonymous pulls, to the Palace mirror:
That is the whole change: one
FROMline, plus a comment recording why it points somewhereunusual. The mirror is an unmodified MinIO build, so the credentials, ports and command below the
FROMare untouched.The mirror is published and public:
ghcr.io/thepalaceproject/palace-ci-miniois live at digestsha256:0e8998d7ba6f742bac3c6ad22f354979b987d5d31c0a95b9b0dcad9d73e32751withlinux/amd64andlinux/arm64, and an anonymous (unauthenticated) manifest request returns 200.It is published from ci-scripts rather than from
here, because the image is shared by three repos and belongs to none of them in particular.
Companion one-line changes: ThePalaceProject/library-registry#1066 and
ThePalaceProject/virtual-library-card#1016. The three are independent and can merge in any order.
Motivation and Context
JIRA
All tox-docker CI jobs currently fail on
mainwhile building the MinIO test container:MinIO has withdrawn anonymous public access to its server image from Docker Hub (~13 Sept, worked
around in #3728 by moving to quay.io) and now from quay.io as well.
Ten checks fail on
main: Tests (Py 3.12 / 3.13 / 3.14), OpenSearch 2.19, OpenSearch 3.5, Unittests (amd64 / arm64), Integration test (amd64 / arm64) and the Backwards compatibility test. Lint,
mypy, CodeQL, the Migration test and the Docker build are unaffected because they do not use
tox-docker. Local
toxfails identically, so developers cannot run the suites either.Pinning an older tag does not help — the whole upstream repository is gated, not just
:latest—and neither does a local Docker cache, because
tox_dockerpassespull=Trueon every build.Full background, registry probes and the retirement plan are in ThePalaceProject/ci-scripts#2.
How Has This Been Tested?
This PR's own CI is the acceptance test, and it passes. All ten checks listed above are green
against the published mirror, including the Backwards compatibility test and both Integration and
Unit test jobs on amd64 and arm64.
Locally,
tox -e py312-docker -- -m miniowas run against the realghcr.io/thepalaceproject/palace-ci-minioreference with the image first cleared from the localDocker cache, so the pull genuinely came from GHCR:
Before the mirror was published, the same chain was exercised by pushing the image to a throwaway
registry:2on localhost and temporarily pointing thisFROMline at it —tox_dockerpassespull=Trueon every build, so a local-only tag is rejected and a real registry is required eitherway. That run also gave 14 passed.
Checklist
🤖 Generated with Claude Code