Repository navigation
Conversation
kirodotdev#12351 replaced a `grep | head -n1` verdict capture with one awk pass in the two Security Scope lanes. Its two counted siblings were left: the fork Design Review and fork First Principles lanes read their header the old way, inside a step that carries `set -uo pipefail`. Both statuses that shape produces abort the step under the runner's own `bash -e`, one line ABOVE the `[ -n "$v" ] && verdict="$v"` fallback written for exactly those cases. A review with no header makes grep exit 1 and pipefail promotes it; a long review makes `head -n1` close the pipe and its producer die of SIGPIPE with 141. The lane then writes no verdict output at all, and the comment that would have named the cause is never posted. Reproduced locally on both lanes: rc=1 with a headerless body, rc=141 at 60k headers, and the fallback unreached in all four runs. Each of these steps already uses a here-string twice, each time under a comment saying a long summary must not manufacture a SIGPIPE status under pipefail. The capture between them was the one place the rule was not applied, which is what makes this an oversight rather than a judgement. Convert both to the same awk expression kirodotdev#12351 used, with the anchor left exactly as it was, so which headers count does not change. The same-repo design and first-principles lanes are deliberately untouched: their steps carry no `set -` line, so an unmatched grep's status is masked by the pipeline's last command and they cannot die this way. Two existing pins named `grep` as the program that reads the header. They now assert an ANCHORED reference to the header instead, because a pin keyed to one tool stops finding the capture the day the capture changes tool and then passes while measuring nothing. `_capture_step`'s slice boundary takes both spellings for the same reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: I reviewed the blockers on this PR. The change itself looks right, but everything left is yours — no commit on this branch can move it. The blocker: not one check has ever run. Every workflow on That is the repo's "require approval for all external contributors" setting, which The premise still holds — this is not superseded. I checked current while Decisions I need from you:
What is mechanical once you have ruled, and what I can take back if you want it: the branch is 629 commits behind and has exactly one conflict, in When you've addressed these, the pipeline will re-assess on its next cycle. |
|
Thank you for this fix, @leonlaiyc — the SIGPIPE/grep-exit-1 analysis of the fork verdict captures was spot on, and the test coverage is excellent. Because this PR touches Closing this one in favor of #15845. Thanks again! |
Problem / Motivation
On a fork PR, the Design Review and First Principles lanes can finish their
model call, hold a perfectly good verdict, and post nothing at all — no comment,
no verdict output, and nothing in the job log naming why.
Both lanes capture their verdict with
inside a
run:block that opensset -uo pipefailand, having noshell:key,is executed by the runner as
bash -e. Two ordinary model outcomes make thatassignment's status non-zero, and
-ethen kills the step one line above thefallback written for exactly those outcomes:
grepmatches nothing and exits 1;pipefailpromotes that tothe pipeline's status.
head -n1exits after the first line and closes the pipe,so its producer dies of SIGPIPE, status 141.
Why it matters
The
verdict="UNKNOWN"default and the[ -n "$v" ]test directly beneath thecapture are a complete, fail-closed answer for a review that names no verdict.
Neither line runs. The outcome is not "UNKNOWN" but nothing: the lane writes no
verdict output, its status step reads an empty verdict, and the comment that
would have told a contributor what happened is never posted. A red either way,
but only one of them says why — and on a fork PR the contributor cannot re-run
the lane themselves.
The 141 arm is the worse one: a verdict is already in hand when the read ends
early, so an expression that merely suppressed the status would hand the lane a
verdict read out of a document the step never finished reading.
This is not a hypothetical shape. #12351 fixed precisely it in the two Security
Scope lanes; its First Principles review counted these two as the unconverted
siblings and named both paths. They are still on
mainat1fa7f5ba5.What changed (motivation → approach → change)
Symptom → a fork advisory lane that reviewed fine posts no verdict and no
explanation.
Root cause → the capture is a pipeline in a step that turns a pipeline's
inner failure into the step's failure. Each of these steps already uses a
here-string twice — trimming the summary to the header, and checking for the
[…-REVIEWED]marker — and each time under a comment saying, in the file's ownwords, that a long summary must not manufacture a SIGPIPE status under
pipefail. The capture sitting between them is the one place that rule was notapplied. That is what makes this an oversight rather than a different judgement.
Change → both captures become the one-
awk-pass form #12351 introduced:awkexits 0 whether or not the header is there, so the[ -n ]test belowdecides — as the code was always written to. There is no
exiton first match:the whole summary is read and the FIRST header is printed from
END, so averdict is never taken from a partial read. The character class spells out the
five characters
[[:space:]]matches inside a record, because the runners' awkis mawk.
Three things deliberately do not change:
^…-Verdict:with no leading-whitespace class, soexactly the same set of headers counts as before. Widening it to accept an
indented header is a separate question with its own contract argument (fix(ci): scope review must not abort on a headerless verdict #12351
settled it for the scope lanes, whose contract displays the header indented);
answering it here would mix a behaviour change into an abort fix.
design-review.ymlandfirst-principles-review.yml. Theirpost-summary steps carry no
set -line, so an unmatched grep's status ismasked by the pipeline's last command and the step cannot die this way. I read
both steps to confirm it rather than assuming symmetry. Converting them would
be a change with no defect behind it.
Two existing pins had to move, and that is part of the diff rather than a
side effect.
TestFirstPrinciplesReviewandTestTheOutputDietTightenedAndKeptItsMachineReadLinesasserted the literalgrep -iE '^First-Principles-Verdict:', and_capture_stepused the sameliteral as a slice boundary. All three now key on an anchored reference to the
header rather than on the program that reads it — which is the property those
pins were about, and the reason the repo's own newer selector comments give: "a
pin keyed to one tool silently stops finding the capture the day the capture
changes tool, and then measures nothing." The slice boundary takes both
spellings and cannot match the trim a few lines above it: the trim greps with
-qand its awk opensf ||, where the capture's opens!found &&.Backwards compatibility
Compatible. The capture returns the same value for every input that produced a
value before — pinned, not asserted: the three verdict-carrying cases below
(
plain,lowercase,crlf) pass on pristineorigin/mainas well as onthis branch, so they measure the old expression and the new one to be equal. The
inputs whose behaviour changes are the two that produced no value and killed the
step; they now produce no value and let the step continue to its own fallback.
No workflow trigger, permission, job, step name, check-run name or output changes.
Two open PRs touch the same three files and are worth naming so a reviewer is
not surprised by a textual conflict: #12201 and #12469 both edit the network
allowlist near the top of these workflows (lines ~68-160) and both append to the
end of
test_ai_review_workflows.py. Neither touches a verdict capture -- Igrepped both diffs for
Design-Verdict,First-Principles-Verdict,pipefailand
head -n1and got zero hits -- so this is a rebase, not a competing fix.Tests
test/test_ai_review_workflows.py—TestForkVerdictCaptureNeverAbortsAboveItsOwnFallback, 14 tests over the twolanes.
Each test extracts the step's own expression from the yaml and runs it under
bash -ewith the step's ownset -uo pipefail, then asserts both the exitstatus and the captured value. Asserting the value alone would pass an expression
that returns the right answer and kills the step anyway; asserting the status
alone would pass one that never aborts and has stopped reading verdicts.
The expression is located by shape, and specifically by what the fallback
reads:
_capture_lineparses the variable name out of[ -n "$v" ] && verdict="$v"and finds that assignment. Both steps also trimtheir summary with a second assignment mentioning the same header, and a
first-match-wins selector silently measures that one instead — which it did in my
first draft, and the megabyte-long failure message is how I noticed.
Six review kinds per lane:
no-header""header-shaped-prose""many-headers(60 000)PASSplainCONCERNSlowercaseBLOCKcrlfBLOCKPlus
test_the_step_still_carries_the_fallback_this_pin_is_about, which fails ifverdict="UNKNOWN"or the[ -n "$v" ]test is ever removed — without them thewhole class is measuring something else.
Red-before / green-after, whole file,
-p no:randomly -n 4:1fa7f5ba5test_ai_review_workflows.pyThe red-before was run with the final test file against the two pristine
workflow files, so the 541 passes prove the moved pins are not what turns the
suite green — only the six abort arms are.
Gates on this head:
flake8clean,black --checkclean,scripts/check_black_formatting.pypassed ("nothing in scope is unformattedoutside the baseline"),
scripts/check_brand_name.pyexit 0, and both editedworkflows re-parsed with
yaml.safe_load.Manual verification
The mechanism was reproduced directly on this box before any change, running each
lane's own capture line under
bash -ewithset -uo pipefail:Stated exactly as measured: at 2 000 headers the SIGPIPE arm did not fire on
Git-for-Windows Bash, because a 64 KiB pipe buffer swallows that body whole. It
fires at 60 000, which is why the test fixture uses that size rather than the
2 000 #12351's own suite uses on Linux runners.
What is not verified here:
workflow_runlanes execute the default branch'syaml, so this change is unexercised until after merge. The first fork PR opened
afterwards is the real end-to-end check.
One process note, since this is a fork PR under
.github/**: the ForkWorkflow-Change Guard will flag it by design. Its own header says it is an
advisory check-run and not a merge gate, and that a maintainer who has reviewed
the change clears it with the
allow-fork-workflow-changelabel. The diff addsno job, step, trigger, permission or network allowance — it replaces two
expressions inside one existing step each, and updates the tests that read them.
Related Issues
None. This is the counted, unconverted half of the First Principles CONCERNS on
#12351 (merged), which names both paths: "
fork-design-review.yml:666andfork-first-principles-review.yml:592… so a headerless or many-header reviewaborts them one line above their own
[ -n "$v" ]fallbacks, exactly the defectthis PR names."
Pattern harvest
Rule candidate: review-prompt
Pattern: a step's preamble (
set -euo pipefail) makes a command's exit statusload-bearing, so an expression written to return "nothing found" instead kills the
step above the branch that handles "nothing found".
Both lanes had the handling. Both lanes had already met this exact hazard twice in
the same step and written a here-string with a comment explaining why. The
surviving pipeline is what a reader skips because the fallback beneath it looks
like the answer.
The prompt this rule candidate proposes: when a fix converts a pipeline for
exit-status reasons, grep the sibling lanes for the same expression before
calling the defect closed — and when reviewing any
run:block that opens withpipefail, ask of every$( … | … )whether its "found nothing" status is avalue the next line reads or a signal that kills the step first.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
🤖 Generated with Claude Code