Repository navigation
fix: check a review lane boundary before the model runs - #30440
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: one fail-closed probe in an existing step, aimed at a boundary that was assumed but never checked (#13414), with no new surface. What this change shipsInventory (4 items) — 4 justifiedIntent: stop the fork review lane before its model starts on a runner where the model's sandboxed shell could read another job process's environment. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 356728c Description read[DESCRIPTION-READ] 52fe5467deaeb5d9cfce4c7987dd4b7e9397b870cd896559cc9c44f0d7ab6adf That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns: the step now tests the missing boundary directly, with positive controls on both sides, so a pass cannot come from a probe that ran too early or a broken sandbox. Notes
[DESIGN-REVIEWED] 356728c Description read[DESCRIPTION-READ] 192f16f02c80529561375ee658620e8e3dadfb41c02836e1a0405e3213f933d9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
The fork review lane hands its model a shell on an untrusted diff. Before the model starts, the lane now proves that this shell cannot reach what other job processes hold, and refuses to review when it can, instead of assuming the boundary holds on the runner it landed on. On a runner where the boundary holds the lane runs exactly as before.
fee1ac7 to
356728c
Compare
|
self-added: yes
|
|
macOS Tests / Backend Tests (macOS) (6) on fee1ac7 failed in |
There was a problem hiding this comment.
🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:
Approve. Before the fork review model gets a shell, the fork review lane now proves that the model's sandbox cannot read another job process's environment. If the sandbox can read it, the lane stops instead of passing quietly. Fork PR authors see no difference on a runner where the boundary holds.
Evidence at 356728c:
- The probe sits in the existing "Verify the review sandbox fails closed" step (fork-gpt-review.yml:827-850), which runs before both GPT passes (:853, :1223). Every new branch exits 1: the marker never appears unconfined (:832-839), the checkout is unreadable under the sandbox (:842-845), or the sibling's environ is readable under the sandbox (:846-849). The sibling starts under
env -iwith only the marker, so it carries nothing from the job. - The diff changes no trigger, permission, secret reference or checkout ref. The step's env is unchanged.
- test_ai_review_workflows_credential_isolation.py: 63 passed. Mutations were run one at a time, each restored before the next. Removing the environ refusal reddens test_it_fails_closed_when_another_process_environment_is_readable[False]. Replacing the unconfined marker wait reddens test_it_fails_closed_when_the_marked_sibling_never_shows_its_marker, which shows the earlier race finding is fixed. Moving the step after the discovery pass reddens test_the_step_exists_and_runs_before_both_codex_passes.
- CI: 82 passed, 12 skipped, none failing. PR Readiness green. Merges cleanly onto main.
Not verified here: the probe has not yet run on a hosted runner, because this lane runs from the default branch. The claim that the pinned codex sandbox blocks the read in both Linux modes rests on the manual run in the description. Please watch the first fork review after merge. If it fails at the new probe, the runner's sandbox does not hold that boundary, and that needs a fix or a revert rather than a rerun.
Non-blocking follow-ups: nothing reddens if the checkout-read control (:842-845) is removed. The step name and the closing echo still mention only sudo and the credential file.
Problem / Motivation
Goal: The fork review lane refuses to start its model when the runner's sandbox would let that model reach what other job processes hold, and runs exactly as before when it would not.
The fork review lane gives its model a shell on an untrusted fork diff. Before the model starts, the lane already proves two of the boundaries that shell must respect. A third one was assumed, never checked on the runner: if it failed there, nothing went red.
Why it matters
A prompt-injected reviewer on a fork PR could copy something another job process holds into a public review. The repository wants that case to stop the lane, not to pass silently.
Not a goal
What changed (motivation → approach → change)
.githubself-check step gains one probe. It starts a marked sibling process and waits, outside the sandbox, until its marker is readable. It fails the job closed when the sandbox can read that marker, when the marker never shows outside the sandbox, or when the sandbox cannot read the checkout it must review (either would make the refusal meaningless).The CI review lanes page in
docsdescribes the probe. Thetestfile covers all three outcomes.Backwards compatibility
Compatible: on a runner where the boundary holds, every probe passes and the lane runs as before. The lane stops only on a runner where the model could reach what it must not.
Removes nothing: no input, key, record or capability is removed or tightened. The only new outcome is a refused run on a runner whose sandbox does not hold.
Tests
The new tests run the real step script against stand-in sandboxes. A sandbox that can reach other processes must fail the step, and one that cannot must pass it. A sibling whose marker never appears must fail the step too, so a sandboxed read can never pass just because it came too early. Both failing cases fail with the fix reverted.
Proof: PROVEN test/test_ai_review_workflows_credential_isolation.py::TestReviewSandboxFailsClosed::test_it_fails_closed_when_another_process_environment_is_readable[False], test/test_ai_review_workflows_credential_isolation.py::TestReviewSandboxFailsClosed::test_it_fails_closed_when_the_marked_sibling_never_shows_its_marker (head 8557115916c4)
Determinism: 15/15 runs of the three probe cases at
-n 4; the related review-workflow test files, 2395 passed at-n 4.Manual verification
The step's own script was run on a Linux host that can create user namespaces, against the pinned review CLI:
The hosted-runner check is the step itself: every fork review run measures the boundary on its own runner before the model starts.
Related Issues
Closes #13414
Sibling sweep
Siblings: none -- the only other lane that hands a model a shell reviews same-repository branches only, so no untrusted author can steer it.
Pattern harvest
Not generalizable: one review lane's pre-model check missed one boundary; the other lanes give their model no shell.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)