Repository navigation
fix(ci): name the scope-review validate refusal on the check-run itself - #13647
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Same-repo lane still shows the generic sentence: the description claims both lanes surface the refusal cause, but only the fork lane is wired. Watch
[DESIGN-REVIEWED] 232a0ba Description read[DESCRIPTION-READ] b171a28808c0af0b87a9abc5251dd36a7ab8ab73911f1f4269e398cd051cc51c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS Description says both lanes surface the refusal's cause; only the fork lane is wired — the same-repo lane's identical refusals still read the generic sentence. Not justified as shipped
What this change shipsInventory (6 items) — 4 justifiedIntent: FIX — make the Security Scope Review check itself say why a
[FIRST-PRINCIPLES-REVIEWED] 232a0ba Description read[DESCRIPTION-READ] adc77403cf1e0a965a535bb59c051a2977f81d0595696aa901d2a174c1408f52 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNo findings that block. FINDING — .github/workflows/fork-security-scope-review.yml:2471 — the unconditional [OPUS-REVIEWED] 232a0ba Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: the shared conclusion ladder in ADJUDICATED: 10 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: posix — the workflow's own additions ( UNADJUDICATED: the tightening lives entirely in the lane's own verdict policy, which the classifier cannot adjudicate — it settles shell command lines through the deny composite, and no hunk touches that composite. The guard: the [SCOPE-REVIEWED] 232a0ba |
Board disclosure: the one red is a floored rate-limit casualty, not a verdict
The floor is per-SHA and, on the same-repo lane, cannot discount the earlier row: both rows No lane is a branch-protection gate here ( |
A `validate` refusal dispatches no adjudication leg, so the fold reads `no-report` and the shared table answers with the generic sentence for that row. The refusal already writes a code that names its own cause, as a job output `publish` reads; the table now takes it as an input and selects the cause for the check-run title plus the action that clears it for the summary. The conclusion and the `settled` answer are unchanged for every row: a named refusal selects a more specific `why`, never a softer verdict, and the row sits ahead of the green nothing-new reading so the two can only fail closed. Both halves are authored in the table and keyed by exact match, so no value of the refusal code reaches a published verdict and an unrecognized code falls through to the generic sentence. The check-run summary no longer sends a reader to a PR comment for confirmed rows on a run that folded none. The comment pointer stays in the arm that has a deterministic body to point at. Closes #13617
7b6aa80 to
232a0ba
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
Reviewed. No permissions: block touched (publish keeps actions:read, checks:write, contents:read, pull-requests:write), no new secrets.* reference, trigger stays workflow_run so the default-branch definition runs, and scripts/scope_candidates.py runs from the default-branch checkout with persist-credentials false and no ref. VALIDATE_RC is only the base-authored rc=corpus-* echoes matched by exact key against the script's own table, so no caller string reaches GITHUB_OUTPUT. Coupling test pins the workflow's rc codes to the table keys.
Problem / Motivation
When the security-scope review refuses to run because a candidate operation carries a credential shape (
rc=corpus-credential) or the normalized corpus could not be probed (rc=corpus-uncheckable), no adjudication leg is dispatched, so the fold readsno-report. The lane's generic sentence for that state — "no leg reported, and this was not the nothing-to-adjudicate short circuit" — is true of a broken lane, of a regression in the author's own diff, and of these refusals alike. The real cause reached an author only through a job annotation, four hops from the check they read.Goal: Name a
validaterefusal cause on the Security Scope Review check itself, so an author reads why no leg ran without leaving the check.Why it matters
A blocking check whose failure summary names no actionable cause sends the author to the wrong place: they debug the lane or their own diff while the fix is to re-propose the operation without the embedded credential shape. The cause and its remedy now travel to the shared conclusion table, so both the fork and same-repo lanes surface the same explanation and the same remedy in one place instead of a per-lane annotation.
Not a goal
The refusal itself is not relaxed by naming it. The corpus travels to the adjudication legs as a public artifact and must stay byte-faithful for the classifier, so a credential shape in it is still neither published nor rewritten. Naming the cause never softens the verdict — an unmeasured run fails closed exactly as it did while it was unexplained.
What changed (motivation → approach → change)
validaterefusal produced a generic "no leg reported" summary on the Security Scope Review check.rc=corpus-*) was written as a job annotation but never carried into the shared conclusion table, so the table could not name the cause on the check-run.scope_candidates.pygains a_VALIDATE_REFUSALStable mapping eachrc=corpus-*code to a short cause (for the 255-char check-run title) and a full remedy (for the summary).conclude_laneselects the cause by exact key match on a run whose fold isno-report; an unrecognized code falls through to the generic sentence. The fork lane passes the code throughconclude --refusaland prints aremedy=line the publish step reads into the check-run summary.Tests
test_scope_candidates.py: the named refusals surface their own cause and remedy, an unnamed code falls through to the generic sentence, naming a cause never softens the verdict, and a refusal outranks the green nothing-new row.test_ai_review_workflows.py: the fork lane hands the refusal code to the table and interpolates no cause text in shell; and the newly added coupling test pins therc=corpus-*codes the fork lane emits to the exact keys of_VALIDATE_REFUSALS, so a rename on either side fails a test rather than silently regressing to the generic sentence.Manual verification
N/A — unit coverage sufficient. The first-principles reviewer's advisory note (verify on a real fork run that "this run's logs are linked from this check") concerns the check-run's
details_url, which the publish step stamps with the run URL; it is a PASS-verdict observation, not a requested change.Pattern harvest
Rule candidate: review-prompt
Pattern: a stringly-typed code emitted by a workflow and consumed by a lookup table in a script has no test coupling the two, so a rename on either side degrades silently to a fallback. Pin the emitted set against the table's keys, both directions.
Related Issues
Closes #13617