Skip to content

🤖 ci: publish the sandbox image on push to main, once per inputs key - #7671

Merged
ThomasK33 merged 3 commits into
mainfrom
tests/sandbox-image-publish-upsert
Oct 27, 2026
Merged

ThomasK33 merged 3 commits into
mainfrom
tests/sandbox-image-publish-upsert

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Summary

The "Bug-bash sandbox image" workflow now publishes the image by itself. A push to main that changes an image input runs the publish job. The job creates the image only if the key has none yet: each inputs key gets one stable tag, ghcr.io/coder/xum-bugbash-sandbox:inputs-<key>. A key that has its tag pushes nothing. This is PR 1 of 2. Runners still trust only the digest in image.json until PR 2.

Background

An image change takes four steps today, and two of them need a person. #7362 changed the Dockerfile. Its story lanes refused the stale image, so it needed an admin merge, and the merge queue then stayed red for 3.5 hours. After that, a maintainer ran the manual publish on main, and #7369 pinned the new digest. Four images went through that process between Oct 8 and Oct 23.

The plan, which the user approved: main publishes once per key with no person (this PR). Then every sandbox job pulls inputs-<key>, checks its build attestation (gh attestation verify --source-ref refs/heads/main), and builds locally when there is no verified image (PR 2). image.json and the BUGBASH_SANDBOX_BUILD_STALE opt-in go away in PR 2.

Implementation

  1. New trigger: push to main, filtered to the image inputs (build.sh --key hashes the Dockerfile, build.sh and the playwright-core version in bun.lock) and to this workflow file. A change to this file therefore publishes the current key once. Makefile is not in the push filter. A Makefile change does not change the key, and an existing key needs no publish.
  2. Create if missing: the publish step looks up the stable tag with docker buildx imagetools inspect. If the tag exists, the step writes "exists" to the summary and stops. Only a : not found error counts as missing. Any other lookup error fails the job. If the tag is missing, the step runs the publish as before.
  3. Tag last: Attest signs the new digest. Only then does a new step, "Tag inputs-", point the stable tag at it with imagetools create --prefer-index=false. That flag copies the manifest as it is. The default wraps a single manifest in a new index with a new, unsigned digest. The step reads the tag's digest back and fails unless it equals the signed digest. The stable tag marks the key as done, so a failed Attest leaves no tag, and a re-run publishes again.
  4. No races: the existing concurrency group runs one publish at a time and never cancels it. A second push of the same key waits, then finds the tag and stops.
  5. Docs: AGENTS.md and the bug-bash skill now say that the merge push publishes the image. A manual run on main retries a failed publish.

This PR does not change any image input, so the inputs key stays 2bf9e12d…, the key in image.json. No old-style publish is needed for this PR.

Validation

  • I ran the step's lookup branch against GHCR with an empty Docker config, with these results:
    • The current key's stable tag: PUBLISH: … missing.
    • An existing published tag: SKIP: … exists.
    • An unreachable registry: FAIL with exit 1, and the DNS error printed.
  • I ran the "Tag inputs-" step's script against a local registry, with a single-platform Docker v2 manifest like the published image. It kept the source digest. Without --prefer-index=false, the tag got a different digest.
  • actionlint 1.7.13: clean. make lint-zizmor and make static-check: pass.
  • After the merge, I will check two runs. The merge push must publish inputs-2bf9e12d3ac7dcb10397698df2792bf1b244b3fcbe68225378302ef695ba302a. Then a manual run on main must report "exists" and push nothing. I will link both runs here.

Risks

Low. Publishing stays main-only, from the run's own commit, with a fresh build and an attestation. A new path is the one trigger: every bun.lock change on main starts a run that ends after the lookup, at about one minute of runner time each. A failed lookup fails the run and pushes nothing.

Decisions in the approved plan
  1. A job can build the image itself in CI and on a developer machine.
  2. Only main pushes images, after a merge. The merge queue does not push.
  3. A job uses a registry image only if its attestation shows that main's publish workflow built it. Otherwise it builds locally.
  4. Two PRs: this publish change first, then the runner, build.sh and docs.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh • Cost: $263.47

The publish job of the Bug-bash sandbox image workflow now also runs on pushes to main that change an image input. It skips a key whose stable tag inputs-<key> exists, and otherwise builds fresh, pushes, points the stable tag at the build's digest and attests it.
@mintlify

mintlify Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
Mux 🟢 Ready View Preview Oct 27, 2026, 8:42 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T09:00:06.599662Z 70bca07 Manual request
🔒 Security Review ✅ Completed 2026-10-27T09:00:34.065812Z 70bca07 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37b9627f09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/bugbash-sandbox-image.yml Outdated
Comment thread .github/workflows/bugbash-sandbox-image.yml
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e stories (Haiku): 14 passed

22 agent steps · 7 replayed from cache · 15 model calls · 46.1k tokens (24% cached)
Spend: $0.09 of the $1.00 cap (proxy ledger)

All 14 tests in 9 files
Test Agent Time
🟢 chatRestart.e2e.ts · 1 passed 1 step · 1 call 5.5s
🟢 a chat survives a restart of the server 1 step · 1 call 5.5s
🟢 compaction.e2e.ts · 1 passed 1 step · 1 call 11.5s
🟢 /compact replaces the old chat with a summary 1 step · 1 call 11.5s
🟢 pluginTrustRestart.e2e.ts · 1 passed 4 steps · 2 calls 10.5s
🟢 a trusted plugin hook stays trusted after the server restarts 4 steps · 2 calls 10.5s
🟢 settingsRoundTrip.e2e.ts · 1 passed 2 steps · 1 call 8.8s
🟢 a chat survives trips through Settings 2 steps · 1 call 8.8s
🟢 sidebarPerWorkspace.e2e.ts · 1 passed 4 steps · 1 call 4.2s
🟢 a new workspace starts with its right sidebar collapsed while another keeps it open 4 steps · 1 call 4.2s
🟢 smoke.e2e.ts · 1 passed 2 steps · 1 call 2.3s
🟢 the agent opens the playground workspace 2 steps · 1 call 2.3s
🟢 streamErrors.e2e.ts · 1 passed 1 step · 1 call 6.7s
🟢 stream errors show in the chat, and the chat recovers 1 step · 1 call 6.7s
🟢 todoNudge.e2e.ts · 1 passed 1 step · 1 call 6.3s
🟢 the agent finishes open TODO items after the turn-end nudge 1 step · 1 call 6.3s
🟢 touchAlignment.e2e.ts · 6 passed 6 steps · 6 calls 24.7s
🟢 the Settings close button is level with the title on a touch screen 390px wide 1 step · 1 call 4.1s
🟢 the Settings close button is level with the title on a touch screen 768px wide 1 step · 1 call 3.3s
🟢 Backup settings checkboxes are small and level with their labels on a touch phone 1 step · 1 call 4.2s
🟢 the heartbeat checkbox is small and level with its label on a touch phone 1 step · 1 call 3.7s
🟢 a reply's model and time stay on one line on a touch phone 1 step · 1 call 2.9s
🟢 the workspace actions menu stays on the title row on a touch phone 1 step · 1 call 6.4s

e2e 0.20.0 · 1m 24s · web · run artifacts

🟢 e2e Electron stories (Haiku): 3 passed

3 agent steps · 3 model calls · 8.0k tokens (22% cached)
Spend: $0.02 of the $1.00 cap (proxy ledger)

All 3 tests in 3 files
Test Agent Time
🟢 electron/deepLink.e2e.ts · 1 passed 1 step · 1 call 6.3s
🟢 a deep link of a second launch starts a chat in the running app 1 step · 1 call 6.3s
🟢 electron/menuBar.e2e.ts · 1 passed 1 step · 1 call 5.3s
🟢 the workspace menu bar is the desktop title bar 1 step · 1 call 5.3s
🟢 electron/smoke.e2e.ts · 1 passed 1 step · 1 call 6.7s
🟢 Xum Electron opens the playground workspace 1 step · 1 call 6.7s

e2e 0.20.0 · 20.6s · electron · run artifacts

🟢 e2e agent repros (Haiku): no tests selected

Spend: $0.00 of the $1.00 cap (proxy ledger)

e2e 0.20.0 · 5.4s · 2 targets (web, phone) · run artifacts

…ested digest

imagetools create wraps a single manifest in a new index by default, so the stable tag got an unattested digest. --prefer-index=false copies the manifest, and the step checks the tag's digest. The tag is the completion marker, so it now comes after Attest: a failed attestation leaves no tag and a re-run publishes again.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: ffafa560a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: ffafa560a9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

jq inside echo arguments does not stop a bash -e script, so a bad record set an output and Attest no longer failed on it. Assign first.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 70bca07231

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 70bca07231

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

Readiness record

  • Final commit: 70bca07231. Required and Codex Comments passed on it.
  • Review rounds: 3.
    • Round 1 had 2 findings. imagetools create wrapped the manifest, so the stable tag got an unsigned digest. The tag also came before Attest, so a failed attestation left a stuck key. Both are fixed in ffafa560a9, and both threads are answered and resolved.
    • Round 2 had no findings.
    • The clean-context advisor found a fail-open: jq inside echo arguments did not stop the step. It is fixed in 70bca07231.
    • Round 3 (code and security) had no findings. No threads are open.
  • Local checks: actionlint 1.7.13, make lint-zizmor, prettier and make static-check pass.
    • The lookup branch ran against GHCR: a missing tag goes to publish, an existing tag skips, and an unreachable registry fails with exit 1.
    • The "Tag inputs-" script ran against a local registry: the digest was kept, and the digest check passed.
    • A record with no digest now fails the step (exit 1).
  • Advisor (clean context): ready with tracked follow-ups.
  • Follow-ups:
    • Makefile is not in the push paths. A Makefile change does not change the key, so this is intended.
    • A stable tag with the wrong digest would make the publish job skip that key. 🤖 tests: resolve the sandbox image by stable tag and attestation, else build locally #7672 makes such a tag visible: jobs warn and build locally. A person then deletes the tag.
    • The first merge push publishes a second image for key 2bf9…. This is expected.
    • This merge push is the first real run of build.sh --push in this flow. I will link the run here.
  • Decision: ready. Landing through the merge queue. PR 2 is 🤖 tests: resolve the sandbox image by stable tag and attestation, else build locally #7672, in stack #7673.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit f3299a8 Oct 27, 2026
36 of 37 checks passed
@ThomasK33
ThomasK33 deleted the tests/sandbox-image-publish-upsert branch October 27, 2026 09:26
@ThomasK33

Copy link
Copy Markdown
Member Author

Post-merge check of the publish workflow on main:

  1. Push run 40480740229 on f3299a8f78 built the image and pushed it. It attested sha256:fb7221e7c56dbdf14b36a848433be4d015fd0d7da0a2a686f53a64cbbf34d458 (attestation 60301668). Then it pointed ghcr.io/coder/xum-bugbash-sandbox:inputs-2bf9e12d3ac7dcb10397698df2792bf1b244b3fcbe68225378302ef695ba302a at that digest. docker buildx imagetools inspect on the tag returns the same digest.
  2. gh attestation verify oci://…@sha256:fb72… --bundle-from-oci --repo coder/xum --source-ref refs/heads/main --signer-workflow coder/xum/.github/workflows/bugbash-sandbox-image.yml exits 0. The certificate names this push run.
  3. Manual run 40481127901 on main reported "…:inputs-2bf9… exists: nothing to publish." It skipped Attest and the tag step and pushed nothing.

The Attest step in the push run left one warning: the storage record needs artifact-metadata: write. Runners do not read storage records, so #7672 sets create-storage-record: false (c85bf67).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant