Repository navigation
ci(pullfrog): give reviews the dev shell's tools - #751
Conversation
Quest-Claim: 4ab0f0a294b24007
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Quest-Claim: c5b5931035e623f5
…/pullfrog-setup # Conflicts: # quest/m0/pullfrog-setup.md
Rind needs no setup now that its vectors are vendored; a dispatched test run on its main passed. Studio, rust2, and web-transport get the same steps in their own PRs; the quest waits on those and the post-merge runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 5 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
ℹ️ One rough edge worth a look — nothing blocking.
Reviewed changes
Reviewed the Pullfrog workflow's new Nix setup steps and the rewritten pullfrog-setup quest.
- Dev shell on the agent's PATH: installs Nix with the same pinned installer
check.ymluses, then prepends the/nix/storeentries ofmain'snix print-dev-envPATHthroughGITHUB_PATH, colon-joined on one line so order is preserved. - Job token scrubbed: deletes the installer's
access-tokensline from/etc/nix/nix.confafter the dev shell is built, and fails the job ifgithub.tokenremains anywhere under/etc/nix. - Quest narrowed:
quest/m0/pullfrog-setup.mdis resized to XS, records rind as done, and leaves one dispatched run per repository plus the three outside merges underWaiting.
claude-opus-5-5 | 𝕏
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the one commit since the prior Pullfrog review (cedeb9a).
- Failing
print-dev-envnow fails the step: the PATH step setsshell: bash, which runs with-eo pipefail. A failednix print-dev-envno longer writes an empty line toGITHUB_PATH. This addresses the prior thread, which is resolved.
claude-sonnet-5-5 | 𝕏
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the one commit since the prior Pullfrog review (cedeb9a).
- Token scrub widened: the step now strips
access-tokensfrom every file under/etc/nix(includingnix.custom.conf), not justnix.conf. - Receipt locked down:
/nix/receipt.json, whose plan also records the token, is set to0600. The existinggrep -rqFguard over/etc/nixstill fails the job if the token survives there.
claude-sonnet-5-5 | 𝕏

Summary
Works quest/m0/pullfrog-setup.md as far as it can go before outside merges.
mainhasthirdparty/vectors/{wycheproof,cavp,cctv}/, no.gitmodules, and no submodule entries. The dispatched Pullfrog run (run 43743879414) rancargo test --workspace --no-fail-fastonmain: 1785 passed, 0 failed, 53 ignored on purpose (BINSEC, SDE, slow RustCrypto).quest,just, taplo, nextest, cargo-hack...), so the rind route of needing no tool isn't available. Theirpullfrog.ymlnow, before the agent:./.github/actions/nixwrapping it),/nix/storeentries ofmain'snix print-dev-envPATHonGITHUB_PATH,access-tokensline the installer writes with the job'sGITHUB_TOKENinto the world-readable files under/etc/nix(nix.confandnix.custom.conf), restricts/nix/receipt.json, which holds the install plan, to root, and fails if the token is still anywhere under/etc/nix.Steps 2 and 3 run under
shell: bash(pipefail), so a failednix print-dev-envor an empty PATH fails the step.main, one dispatched run that runs its checks, with its setup time, andWaitingitems for the three outside merges. Resized S to XS.Choices made without asking (the Plan's "Open for the implementer" items):
hooks.setup. Pullfrog runshooks.setupin its shell sandbox (setpriv --no-new-privs, all capabilities dropped, per itsutils/lifecycle.tsandutils/sandbox.ts), so it cannotsudoto install Nix, and setting it needs the maintainer's console login; workflow steps run before the action with the runner's privileges.GITHUB_PATH, not the whole dev env onGITHUB_ENV. Pullfrog's restricted shell (filterEnvin itsutils/secrets.ts) passesPATHbut drops variables outside its allowlist, soGITHUB_ENVexports would never reach the agent.nixitself is on PATH too, sonix develop --commandstill covers anything needing the shell's other variables.pullfrogenvironment admits onlymain, so the dev shell is always built from the default branch; Pullfrog'scheckout_prmoves to a PR's head only after these steps. The rule against running a non-writer's PR code reaches the agent through the repositories'quest contextreference line, which now resolves sincequestis on PATH.Public API
None.
Validation
actionlintpasses on all four workflows.nix print-dev-env | jq | grep /nix/storepipeline on each repository's flake, with only that PATH plus/usr/bin:/bin, resolvesjustandquestin all four, and each repository's own check tools (studio and rust2: cargo, bun, taplo, cargo-nextest, shfmt; web-transport: cargo with its wasm32 target, bun, cargo-hack, cargo-shear, cargo-sort, wasm-bindgen; quest: bun, actionlint, shfmt).ACTIONS_ID_TOKEN_REQUEST_*removal, and whencheckout_prruns; read the pinned installer's source for where it writesaccess-tokens.false | jq | grep | pasteexits 1 underbash -eo pipefailand 0 underbash -e. The token scrub'sfind ... -exec sedleaves no token in a mocknix.confplusnix.custom.conf. The installer source (latest, which the action downloads) puts extra conf, includingaccess-tokens, innix.custom.confand serializes it into the receipt (pending_nix_config), which is written at the default mode.maincan do; that is the quest's remaining work.just checkandjust testpass.Follow-ups
None.
Models
Size: S
Implementer: Claude Code, claude-opus-5-5, unknown effort (arm A)
Implementer: Claude Code, claude-sonnet-5-5, unknown effort (iterate, XS, arm A)
🤖 Generated with Claude Code
(Written by Claude Opus 5.5)
Review
Rewritten by
quest reviewafter each round.Head: 4f90321
Base: origin/main at 361549a
Risk: high,
it changes .github/workflows/pullfrog.yml, which is always high: workflows run with the repository's secrets and decide what merges and shipsClassified: high by the always-high paths
Reviewer: Codex, gpt-6.1-sol, medium effort
Session:
01a1e067-c81a-7082-903c-f6d0fa8a08de, round 2Score: 8/10,
The incremental workflow changes appear sound by inspection with no actionable defects, but execution was not verified because builds and tests were prohibited.Rounds: 6; findings 0: fixed 0, refuted 0, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, low effort, shadow
Session:
01a1d32d-ba0c-7733-9a31-ed2f0f32eab9Rounds: 1; findings 0: fixed 0, refuted 0, open 0