Mirror the MinIO CI image to GHCR - #3766
dbernstein wants to merge 2 commits into
Conversation
MinIO has withdrawn anonymous public access to its server image: first from Docker Hub (~13 Sept), then from quay.io (~24 Sept). Both registries now answer 401 to anonymous manifest requests for every tag, and dl.min.io answers 410. Any Dockerfile doing `FROM <registry>/minio/minio` fails before tests run, which is every tox-docker job in circulation, library-registry and virtual-library-card. Pinning an older tag does not help (the whole repository is gated, not just :latest), and neither does a local cache (tox_docker passes pull=True on every build, forcing a fresh pull). This adds a mirror published as ghcr.io/thepalaceproject/palace-ci-minio, built from MinIO's GitHub release assets — the one channel still served anonymously. The binaries are byte-for-byte identical to the ones inside the last upstream image CI used, verified by comparing their sha256 against /usr/bin/minio and /usr/bin/mc extracted from quay.io/minio/minio:latest. The image is a bare passthrough with no Palace configuration, so the three consuming repos keep their own differing MinIO setups and change only a FROM line. It is pinned to an immutable release tag and publishes no `latest`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @dbernstein's task in 2m 51s —— View job SummaryThe approach is sound and clearly explained. Building from checksum-pinned GitHub release binaries is the right choice now that both registries are closed, and a final stage with no DetailsMinor:
|
The workflow previously only ran on pushes to main, so a broken Dockerfile or workflow would surface after merge — precisely when the image needs to publish. Building on PRs that touch either file catches that in review instead. Pull request runs build both architectures and verify the pinned checksums, then discard the result: the registry login, the push and the published-manifest check are all skipped, so nothing is written to GHCR from a PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| platforms: linux/amd64,linux/arm64 | ||
| # On a pull request this builds both architectures and verifies the pinned checksums, | ||
| # then throws the result away. Only main and manual runs publish. | ||
| push: ${{ github.event_name != 'pull_request' }} |
There was a problem hiding this comment.
A manual dispatch from a non-main branch still logs in and pushes. If that branch changes the Dockerfile without changing MINIO_RELEASE, it overwrites the release tag pinned by the consumer in PR #3767. CI can then pull image contents that have not been merged. Restrict publishing to the intended branch, or give branch builds distinct tags.
| # also replace the matching `--checksum=` value, or the build will fail (by design). | ||
| # Checksums come from the `.sha256sum` asset published alongside each binary. | ||
|
|
||
| ARG BASE_IMAGE=registry.access.redhat.com/ubi9/ubi-minimal:9.6 |
There was a problem hiding this comment.
The workflow says rebuilding produces the same image, but ubi-minimal:9.6 is specified by tag rather than digest. If that base tag changes, a manual rebuild can publish different base layers under the same MinIO release tag. This makes the pinned image less predictable for consumers; pin the base by digest if rebuilds must preserve its contents.
|
Superseded by ThePalaceProject/ci-scripts#2 — the mirror is published from #3767 stays open here and is now based on |
## Description Adds a mirror of the upstream MinIO server image, published as `ghcr.io/thepalaceproject/palace-ci-minio`: - `images/minio/Dockerfile` — assembles the image from MinIO's official GitHub release binaries (`minio` `RELEASE.2025-09-07T16-13-09Z`, `mc` `RELEASE.2025-08-13T08-35-41Z`) plus the AGPL license files, each pinned by sha256 via BuildKit's `ADD --checksum`. - `.github/workflows/build-minio-mirror.yml` — builds `linux/amd64` + `linux/arm64` and pushes to GHCR. Runs only when the Dockerfile or the workflow changes, or on demand. - A README section covering what the image is and how to consume it. This lives in ci-scripts rather than circulation because the image is shared by three repos and belongs to none of them in particular. GHCR packages are namespaced by org, not repo, so one mirror serves all three. Supersedes ThePalaceProject/circulation#3766, which is now closed. ### Design notes - **Assembled from release binaries, not pulled and re-tagged.** Both upstream registries are closed to anonymous pulls, so there is no image left to `docker pull`, and `dl.min.io` is gone too (410). GitHub release assets are the one channel MinIO still serves anonymously. The binaries are byte-for-byte identical to the ones inside the last upstream image CI used — verified below. - **Bare passthrough, no Palace configuration.** circulation and library-registry use one set of credentials, the library registry uses another. Baking any of that in would fork the image between repos immediately, so each repo keeps its own Dockerfile and changes only its `FROM` line. - **Pinned to an immutable tag, no `latest`.** Following a moving upstream tag is part of how we got here. `RELEASE.2025-09-07T16-13-09Z` is also the last MinIO release to publish binaries at all — later tags ship no assets — so there is no newer version to move to. - **Only `main` publishes.** Pull requests and `workflow_dispatch` runs from a branch build both architectures and verify the checksums, then discard the result. An unmerged branch must not be able to overwrite a tag three other repos pin their CI to. - **Public visibility**, consistent with `circ-webapp`, `circ-scripts`, `circ-exec` and `circ-baseimage`, so no `docker login` for developers and no package-access grants for CI. - **`palace-` rather than `circ-` prefix**, because the image is shared across three repos. ###⚠️ One-time manual step after merge GHCR packages are created **private**. Once the workflow has run, set `palace-ci-minio` to public (Org → Packages → `palace-ci-minio` → Package settings). Until that is done, the three consumer PRs will still fail. Consumer PRs, all draft, each a single `FROM` line: ThePalaceProject/circulation#3767, ThePalaceProject/library-registry#1066, ThePalaceProject/virtual-library-card#1016. ## Motivation and Context All tox-docker CI jobs in circulation fail while building the MinIO test container, and the other two repos will hit the same wall on their next run: ``` docker.errors.BuildError: unauthorized: access to the requested resource is not authorized ``` MinIO has progressively withdrawn public distribution — Docker Hub around 13 Sept (worked around in ThePalaceProject/circulation#3728 by moving to quay.io), and quay.io around 24 Sept. Anonymous probes on 24 Sept: | Probe | Result | | --- | --- | | `quay.io/minio/minio:latest` manifest | 401 | | `quay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z` (pinned old tag) | 401 | | `registry-1.docker.io/minio/minio:latest` manifest | 401 | | Docker Hub API for `minio/minio` | `object not found` | | `dl.min.io` server/client binaries | 410 | | `quay.io/prometheus/busybox:latest` (control) | 200 | Quay issues an anonymous pull token and then refuses the manifest, so the repository is gated rather than the network; the control confirms anonymous pulls work from the same machine. Two consequences: pinning a digest or an older tag will not help, because the whole repository is closed; and a local cache will not help, because `tox_docker` passes `pull=True` on every build (`tox_docker/plugin.py`), forcing a fresh pull. Local `tox` fails identically, so developers cannot run the suites either. ### Retirement This mirror is a bridge, not a destination — the intent is to drop MinIO for a maintained S3-compatible image. It should be short-lived for two reasons: we do not want to become a de-facto public distributor of a frozen MinIO build; and GitHub does not allow self-service deletion of a public package once any version exceeds 5,000 downloads, above which it becomes a Support request. With `pull=True` on every build, ephemeral runners and three repos pulling, that threshold arrives quickly. When the time comes: ``` gh api -X DELETE /orgs/ThePalaceProject/packages/container/palace-ci-minio ``` ## How Has This Been Tested? Verified locally on macOS / Docker 29.6.1 (arm64 host). **The mirrored artifacts are identical to the withdrawn upstream image.** A cached copy of `quay.io/minio/minio:latest` (labelled `RELEASE.2025-09-07T16-13-09Z`) was still present locally; its binaries and license files hash the same as what the mirror downloads: ``` # inside the cached upstream image (arm64) # published by MinIO on GitHub 5c83cd2c…f03d /usr/bin/minio 5c83cd2c…f03d minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z 14c8c961…c12c /usr/bin/mc 14c8c961…c12c mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z 0d96a4ff…bcb0 /licenses/LICENSE 0d96a4ff…bcb0 minio/minio@RELEASE.2025-09-07T16-13-09Z:LICENSE 113b8c63…8542 /licenses/CREDITS 113b8c63…8542 minio/minio@RELEASE.2025-09-07T16-13-09Z:CREDITS ``` **Build.** Builds for `linux/amd64` and `linux/arm64`; both architectures' checksums verify (a mismatch fails the build by design). The final stage has no `RUN` instructions, so it cross-builds without QEMU — confirmed on a CI runner that advertised only `linux/amd64…/386` as supported. **Runtime.** `minio --version` and `mc --version` report the expected releases; the server comes up and `/minio/health/live` returns 200 in ~3s; the console on :9001 returns 200; `curl` inside the container returns 200 against the health endpoint (the check `docker-compose.yml` uses); and `mc alias set` / `mc mb` / `mc anonymous set download` all succeed (the operations virtual-library-card's entrypoint needs). `/licenses/LICENSE` and `/licenses/CREDITS` are present and hash-identical to upstream's. **End-to-end through tox.** Because `pull=True` rejects a local-only tag, the image was pushed to a throwaway `registry:2` on localhost and circulation's `docker/Dockerfile.minio.ci` temporarily pointed at it, so the whole chain was exercised — tox building the consumer Dockerfile, pulling the mirror from a real registry, starting the container and running the tests: ``` py312-docker: docker> build .../docker/Dockerfile.minio.ci target 'minio' py312-docker: docker> run 'sha256:fe9e8121edcb' (from 'minio-circ') py312-docker: commands[0]> pytest -m minio --no-cov -q 14 passed in 13.22s ``` The same local-registry substitution was used to build and run library-registry's `docker/Dockerfile.minio.ci` (healthy, `mc` works) and virtual-library-card's `ci/minio/Dockerfile` (entrypoint completes — bucket created, anonymous download policy set, anonymous bucket read returns 200). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 participant
Description
Adds a mirror of the upstream MinIO server image, published as
ghcr.io/thepalaceproject/palace-ci-minio:docker/Dockerfile.minio.mirror— assembles the image from MinIO's official GitHub releasebinaries (
minioRELEASE.2025-09-07T16-13-09Z,mcRELEASE.2025-08-13T08-35-41Z), eachpinned by sha256 via BuildKit's
ADD --checksum..github/workflows/build-minio-mirror.yml— buildslinux/amd64+linux/arm64and pushes toGHCR. Runs only when the Dockerfile or the workflow changes, or on demand.
No consumer is switched over in this PR — that is the stacked follow-up,
#3767.
Design notes
closed to anonymous pulls, so there is no image left to
docker pull. GitHub release assets arethe one channel MinIO still serves anonymously. The binaries are byte-for-byte identical to the
ones inside the last upstream image CI used — verified below.
different minio credentials and a
and a custom
entrypoint.sh. Baking any of that in would fork the image between reposimmediately, so each repo keeps its own Dockerfile and changes only its
FROMline.latest. Following a moving upstream tag is part of how wegot here.
circ-webapp,circ-scripts,circ-execandcirc-baseimage, so nodocker loginfor developers and no package-access grants for CI.palace-rather thancirc-prefix, because the image is shared across three repos eventhough circulation hosts the publishing workflow.
GHCR packages are created private. Once the workflow has run, set
palace-ci-minioto public(Org → Packages →
palace-ci-minio→ Package settings). Until that is done, the stacked PR and thesibling repos will still fail.
Motivation and Context
All tox-docker CI jobs fail while building the MinIO test container:
MinIO has progressively withdrawn public distribution — Docker Hub around 13 Sept (worked around in
#3728 by moving to quay.io), and quay.io around 24 Sept. Anonymous
probes on 24 Sept:
quay.io/minio/minio:latestmanifestquay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z(pinned old tag)registry-1.docker.io/minio/minio:latestmanifestminio/minioobject not founddl.min.ioserver/client binariesquay.io/prometheus/busybox:latest(control)Quay issues an anonymous pull token and then refuses the manifest, so the repository is gated rather
than the network; the control confirms anonymous pulls work from the same machine. Two consequences:
pinning a digest or an older tag will not help, because the whole repository is closed; and a local
cache will not help, because
tox_dockerpassespull=Trueon every build(
tox_docker/plugin.py), forcing a fresh pull.This affects three repositories — circulation (
docker/Dockerfile.minio.ci), library-registry(byte-identical file) and virtual-library-card (
ci/minio/Dockerfile). Localtoxfailsidentically, so developers cannot run the suites either.
Retirement
This mirror is a bridge, not a destination — see the linked spike for replacing MinIO outright. It
should be short-lived for two reasons: we do not want to become a de-facto public distributor of a
frozen MinIO build; and GitHub does not allow self-service deletion of a public package once any
version exceeds 5,000 downloads, above which it becomes a Support request. With
pull=Trueon everybuild, ephemeral runners and three repos pulling, that threshold arrives quickly.
When the time comes:
How Has This Been Tested?
Verified locally on macOS / Docker 29.6.1 (arm64 host).
The mirrored binaries are identical to the withdrawn upstream image. A cached copy of
quay.io/minio/minio:latest(labelledRELEASE.2025-09-07T16-13-09Z) was still present locally;its binaries hash the same as the GitHub release assets the mirror downloads:
Build. Builds for
linux/amd64andlinux/arm64; both architectures' checksums verify (amismatch fails the build by design). The final stage has no
RUNinstructions, so it cross-buildswithout QEMU.
Runtime.
minio --versionandmc --versionreport the expected releases; the server comes upand
/minio/health/livereturns 200 in ~2s; the console on :9001 returns 200;curlinside thecontainer returns 200 against the health endpoint (the check
docker-compose.ymluses); andmc alias set/mc mb/mc anonymous set downloadall succeed (the operationsvirtual-library-card's entrypoint needs).
End-to-end through tox. Because
pull=Truerejects a local-only tag, the image was pushed to athrowaway
registry:2on localhost anddocker/Dockerfile.minio.citemporarily pointed at it, sothe whole chain was exercised — tox building the consumer Dockerfile, pulling the mirror from a real
registry, starting the container and running the tests:
The same local-registry substitution was used to build and run library-registry's
docker/Dockerfile.minio.ci(healthy,mcworks) and virtual-library-card'sci/minio/Dockerfile(entrypoint completes — bucket created, anonymous download policy set, anonymous bucket read returns
200).
Checklist
🤖 Generated with Claude Code