Skip to content

Release pipeline: versioned GHCR image, attested deploy bundle, GitHub Release (v1.0.0) - #76

Merged
WittsEnd2 merged 9 commits into
mainfrom
release-pipeline
Oct 27, 2026
Merged

WittsEnd2 merged 9 commits into
mainfrom
release-pipeline

Conversation

@WittsEnd2

Copy link
Copy Markdown
Member

Adds a tag-driven release pipeline, so the public project ships named, verifiable versions. Production is unchanged: it still builds on the host from docker-compose.prod.yml.

On a pushed vX.Y.Z tag (vX.Y.Z-pre for prereleases), .github/workflows/release.yml:

  1. verify: the tag equals __version__ and the pyproject version, the CHANGELOG section with Upgrade notes exists, and the tag is on main.
  2. test: reuses ci.yml via workflow_call, so the full unit and integration gate runs.
  3. publish-image: ghcr.io/ragnar-security/lab-playground, linux/amd64 and arm64, with BuildKit provenance and SBOM plus a GitHub build-provenance attestation. The X.Y and latest aliases are computed from the protected tags (release_version.py aliases), never from timing.
  4. attest-bundle and release: a reproducible deploy bundle, built from git objects so untracked operator files can never enter it, plus SHA256SUMS, an attestation, and the GitHub Release with CHANGELOG notes and bound verify commands.

Security (Codex security audit: two rounds, final APPROVE):

  • Only tag pushes publish. Every job with write permissions is tag-push-only and runs in the release environment (v* tags only, approval by lab-playground-maintainers, admins cannot bypass). Tag creation is restricted to that team by a ruleset.
  • workflow_dispatch is always a read-only dry run.
  • Every action in both workflows is pinned to a commit SHA; checkouts don't persist credentials.
  • All publishing runs share one concurrency queue (queue: max, GitHub 2026-05-07; a narrow actionlint ignore until actionlint supports it).
  • The docs and release notes verify with --signer-workflow .../release.yml --source-ref refs/tags/vX.Y.Z (plus --source-digest in the notes).
  • Residual, documented in docs/RELEASING.md: GitHub has no push rulesets for public repos, so any repo writer could author a workflow that uses GITHUB_TOKEN to write packages. Such artifacts fail the bound verification.

Other changes:

  • /auth/health reports version and revision.
  • Dockerfile.prod gets OCI labels.
  • The pyproject version (0.1.0) and __version__ (1.0.0) disagreed; both are now 1.0.0, with a consistency test.
  • CHANGELOG.md.
  • A docker-compose.release.yml override (image instead of build; nothing else changes).
  • Preflight refuses a moving LAB_PLAYGROUND_VERSION.
  • The release helpers are kept out of the runtime image.
  • Docs: docs/RELEASING.md, docs/DEPLOY-FROM-RELEASE.md, README, PRODUCTION_DEPLOYMENT.md.

Verification:

  • TDD, red then green; 1117 passed, 5 skipped; actionlint clean.
  • amd64 smoke test on a dev VM: image labels and version; a stack run from the unpacked bundle with the override had healthy containers, /auth/health returning version and revision, 401s on protected routes, and all six migrations OK.
  • Bundle: planted operator files are excluded, and builds are byte-identical across machines.
  • Next, after merge: a dry run of release.yml from main (the first arm64 build), then tag v1.0.0.

WittsEnd2 and others added 9 commits October 27, 2026 15:04
pyproject.toml said 0.1.0 while app/__init__.py said 1.0.0. Both are now
1.0.0, the first release, and uv.lock records the same. A test fails when
the package, pyproject.toml, uv.lock or the top released CHANGELOG section
disagree.

CHANGELOG.md follows Keep a Changelog. Every release section carries an
'### Upgrade notes' subsection; 1.0.0's lists the six migrations, the new
optional environment variables, the deployment_scope requirement on
/destroy and /extend, and the single-worker rule.

scripts/release_notes.py prints a version's section (the GitHub Release
body) and fails when it is missing, empty, or has no non-empty upgrade
notes. scripts/release_version.py prints the version once both copies
agree. Both are stdlib-only for the release workflow.

Co-Authored-By: Claude <noreply@anthropic.com>
/auth/health now returns version (app.__version__) and revision, read from
APP_REVISION; empty or unset reads as null. Dockerfile.prod turns the
APP_REVISION build argument into that runtime variable and sets the OCI
version, revision, source and licenses labels, after every other layer so
a new commit keeps the dependency cache. Source builds on the server pass
no arguments, so they report revision null; nothing else changes for them.

Co-Authored-By: Claude <noreply@anthropic.com>
docker-compose.release.yml swaps the app's build for
ghcr.io/ragnar-security/lab-playground:${LAB_PLAYGROUND_VERSION:?}, using
'build: !reset null' so Compose pulls instead of building. A test runs
docker compose config and checks that the merged stack differs from
docker-compose.prod.yml only in that build/image swap, and that the
production file alone still builds.

scripts/build_release_bundle.py packs the deploy files from the git tree of
a revision, never the working tree, so untracked operator files (.env,
templates.yaml, data/, scripts/site.env, config/headscale-acl.json) and
uncommitted edits cannot enter it. Members are sorted, timestamped with the
commit time, owned by root without names, and gzipped without a name or
mtime, so a rebuild of a tag is byte-identical; it also writes SHA256SUMS.
Tests plant operator files and glob-matching untracked files in a clone,
and build twice. app/production_policy.py ships too, because
preflight_env.py loads it by path.

Co-Authored-By: Claude <noreply@anthropic.com>
A pushed vX.Y.Z tag runs: verify (tag = v + version in both places,
CHANGELOG notes valid, commit on origin/main), test (ci.yml through a new
workflow_call trigger, so the release reruns the same suite), image
(buildx, linux/amd64 and linux/arm64, GHCR tags X.Y.Z, X.Y and latest,
provenance mode=max, SBOM, build provenance attestation pushed to the
registry), bundle (deploy tarball, attested) and release (GitHub Release
with the notes, bundle and SHA256SUMS). workflow_dispatch defaults to a dry
run that builds and publishes nothing.

Top-level permissions are empty; each job gets only what it needs, and no
job holds both an OIDC token and contents: write. Every action is pinned to
a commit SHA. Values from the event reach scripts through env only.

ci.yml: the wait loop no longer uses 'A && B || C' and an unused variable,
which actionlint's shellcheck flagged; its behaviour is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
docs/RELEASING.md is the maintainer checklist: bump both versions and the
CHANGELOG, merge, tag main, watch the workflow, verify the attestations,
and make the GHCR package public after the first release. README gains
'Install from a release'; PRODUCTION_DEPLOYMENT.md gains 'Running a
released image', with the source-build redeploy still the default.

Co-Authored-By: Claude <noreply@anthropic.com>
The release workflow gates on ci.yml, and its docker-push job runs next to
the Docker Hub token, yet ci.yml used moving tags (actions/checkout@v4,
docker/login-action@v3, ...). Each is now pinned to the commit of the
newest release within the same major version, resolved through annotated
tags with gh api:

  actions/checkout             v4.4.0   11d5960a326750d5838078e36cf38b85af677262
  actions/setup-python         v5.6.0   a26af69be951a213d495a4c3e4e4022e16d87065
  astral-sh/setup-uv           v6.8.0   d0cc045d04ccac9d8b7881df0226f9e82c39688e
  docker/setup-buildx-action   v3.12.0  8d2750c68a42422c14e847fe6c8ac0403b4cbd6f
  docker/login-action          v3.7.0   c94ce9fb468520275223c153574b00df6fe4bcc9
  docker/build-push-action     v6.19.2  10e90e3645eae34f1e60eeb005ba3a3d33f178e8

No CI job pushes to git, so every checkout sets persist-credentials: false.

Co-Authored-By: Claude <noreply@anthropic.com>
docker-compose.release.yml rejects an unset or empty version but would
pull LAB_PLAYGROUND_VERSION=latest, so any restart could change what
production runs. When the variable is set, preflight_env.py now requires an
exact release (1.2.3 or 1.2.3-rc.1), optionally pinned as @sha256:<64 hex>.
Source-build hosts leave it unset and are not affected.

Co-Authored-By: Claude <noreply@anthropic.com>
Dockerfile.prod copies scripts/ for init_db and the migrations; the release
tooling (build_release_bundle.py, release_notes.py, release_version.py)
runs only in the workflow and on maintainers' machines.

Co-Authored-By: Claude <noreply@anthropic.com>
…verification

Security review findings:

- Any repository writer could publish: workflow_dispatch with
  dry_run=false, or a dispatch of an edited release.yml from a branch.
  workflow_dispatch now has no inputs and is always a dry run. Publishing
  happens only in publish-image, attest-bundle and release, which run only
  for a pushed tag (event push, ref_type tag) and use the 'release'
  environment (v* tags only, maintainer approval). Every other job holds
  contents: read only; the dry-run image build is a separate job. Prerelease
  tags (vX.Y.Z-*) now trigger the workflow and publish the same way.
- Concurrent releases could roll back latest and X.Y. All tag-push runs share
  the release-publish concurrency group with queue: max, so none is
  cancelled while waiting; dry runs queue per ref. The aliases come from
  the ruleset-protected tags: 'release_version.py aliases <version>' emits
  X.Y only for the highest stable release of X.Y and latest only for the
  highest stable release overall (numeric ordering, prereleases excluded).
  The GitHub Release is marked latest on the same rule.
- Verification trusted any workflow on any ref of the repository. Release
  notes, README, RELEASING, DEPLOY-FROM-RELEASE and PRODUCTION_DEPLOYMENT
  now pass --signer-workflow .../release.yml and --source-ref
  refs/tags/vX.Y.Z; the release notes also pass --source-digest.

The bundle attestation re-checks SHA256SUMS before signing. RELEASING.md
documents the approval step and the residual risk: a repository writer can
still push to GHCR from a branch workflow; bound verification rejects that,
and closing it is a maintainer decision.

.github/actionlint.yaml ignores only 'unexpected key "queue"' in
release.yml: actionlint 1.7.12 predates the key (GitHub, May 2026).

tests/test_workflow_security.py pins these rules: SHA-pinned actions,
non-persisted credentials, privileged jobs only on tag pushes in the
release environment, read-only dry-run jobs, the shared queue, aliases from
tags, and bound verification commands.

Co-Authored-By: Claude <noreply@anthropic.com>
@WittsEnd2
WittsEnd2 requested a review from a team as a code owner October 27, 2026 19:33
@WittsEnd2
WittsEnd2 merged commit 0caa102 into main Oct 27, 2026
3 checks passed
@WittsEnd2
WittsEnd2 deleted the release-pipeline branch October 27, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant