Repository navigation
ci(fork-review): verify the sandbox read restriction before the lane runs - #19812
cruisercohen wants to merge 1 commit into
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The probe checks a read-deny rule that no review pass uses, and "Fixes #13414" would close the issue while the Watch
[DESIGN-REVIEWED] 4edb8e9 Description read[DESCRIPTION-READ] 336339a022ebaa14b2a52e46f45635a17aeab760c57924f553f41281216152f8 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🔴 BLOCKPremise-level review of First-Principles-Verdict: BLOCK Probe 3 checks a read-deny rule the review passes never apply, and the author's own measurement says the existing Not justified as shipped
What this change shipsInventory (2 items) — 0 justifiedIntent: make the fork review lane refuse to run when its sandbox's read confinement isn't really enforced. This is a FIX.
BlockersProbe verifies a fence nothing uses; its zero option costs nobody anything [FIRST-PRINCIPLES-REVIEWED] 4edb8e9 Description read[DESCRIPTION-READ] 57059db48b93edff199a0996e5f5fab451c2c0f86c9fa5e078c3660d16048d44 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsFinding: the new read-restriction probe never tests the configuration the review passes run under. FINDING — .github/workflows/fork-gpt-review.yml:846 — Probe 3 runs [OPUS-REVIEWED] 4edb8e9 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
3bf211d to
34d3d45
Compare
…runs The fork review lane hands the model a shell on an untrusted diff under a read-only sandbox, but the pre-model self-check never exercised a denied read, so it could not tell the enforcing backend's filesystem confinement apart from a weaker fallback that leaves reads unrestricted. Add a probe to the existing verify step that applies a permission profile denying a sentinel file and confirms, through the same confinement the passes use, that the denied read is refused while an ordinary read still works. When the enforcing backend cannot apply the restriction, the step now fails closed with an actionable message instead of reviewing behind an unenforced fence. Fixes #13414
34d3d45 to
4edb8e9
Compare
|
Closing this draft unmerged. The review lanes found that the probe checks a deny profile that no review pass actually loads, so this PR can't land as scoped. The real /proc deny for the fork lane waits on the hosted-runner measurement that #13414 asks for; #13414 stays open for that. The branch is kept. |
Problem / Motivation
Goal: Make the fork review lane prove its sandbox read restriction is actually enforced before the model runs, and fail closed when it cannot.
The fork review lane hands the model a shell on an untrusted diff under a read-only sandbox. The step that checks the sandbox before the model runs only verified that privilege escalation and the credential file were blocked — it never exercised a denied read, so it could not tell the enforcing sandbox backend apart from a weaker fallback that leaves the read restriction unenforced.
Why it matters
A review that proceeds behind a read restriction that is configured but not actually in force gives false confidence. The lane runs on untrusted fork input, so the self-check should confirm the restriction holds — or refuse to run — rather than assume it.
Not a goal
What changed (motivation → approach → change)
The pre-model self-check asserted sudo was blocked and the credential file was unreadable, but read-only mode permits every read, so no denied read was ever tested — the check could not distinguish the enforcing backend from a fallback that does not confine reads.
Measured against the pinned review CLI on a hosted runner: the enforcing backend reliably refuses a denied read of a regular path, while the restriction is simply not applied when that backend is unavailable (the sandbox then fails to start rather than silently downgrading).
So the self-check now adds a probe that applies a read-deny profile to a throwaway sentinel file and confirms, through the same confinement the passes use, that the sentinel read is refused while an ordinary read still works. If the enforcing backend cannot apply the restriction, the step fails closed with an actionable message instead of reviewing behind a fence that is not in force. The change is in
.github/workflows/fork-gpt-review.yml(the verify step) plus a workflow-shape test.Backwards compatibility
Compatible: adds a verification probe to an existing step; it changes no review-pass behavior and no configuration the passes rely on. On a healthy runner the probe passes (verified end to end); it only turns a silent weaker-fence case into a clear, failing one.
Tests
A new workflow-shape test asserts the verify step applies a read-deny profile, does not suppress it with a sandbox-mode override, and fails closed both when a denied read succeeds and when the enforcing backend cannot be established. It fails on the pre-change workflow and passes after.
Manual verification
The exact verify-step script was run on a hosted runner with the pinned CLI: all probes pass and the step exits 0 when the enforcing backend is available, and the sandbox fails to start (so the step fails closed) when it is not.
Issue link
Fixes #13414