Repository navigation
feat(prepare-pr): tell the loop when a PR's green has expired - #11508
Conversation
|
Intent: Let the loop notice when a PR's CI green has stopped describing the base it will merge against, and rebase only when the base moved in files this PR also touches. |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS
Watch
[DESIGN-REVIEWED] 96daa5a |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsThe regex analysis is deterministic and I've confirmed it by reading the pattern carefully. Let me finalize. For No blocking findings; one advisory. FINDING — src/kiro_crew/builtin_skills/kirocrew-dev/prepare-pr/scripts/green_age.py:462 — a moved base file whose only tie to a changed module is a plain [OPUS-REVIEWED] 96daa5a Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The writer-sweep contract rests on an incident naming writers this tree does not contain, and the loop's new auto-re-sync default rests on unverifiable repo-settings claims. Not justified as shipped
What this change shipsInventory (10 items) — 6 justifiedIntent: let the PR loop notice when a green CI verdict expired because the base moved in overlapping files — ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 6b299f3 |
395e503 to
bf3ee0b
Compare
|
self-added: no
|
|
|
|
|
bf3ee0b to
0c9368c
Compare
|
self-added: no
|
|
self-added: no
|
|
|
|
|
|
/ai-review override first-principles f5c79b8: Maintainer decision: ship the green-expiry fifth readiness criterion and its bounded exit-30 resync without a separate RFC. test_prepare_pr_green_age.py and test_prepare_pr_status.py cover the behavior. This PR records the accepted product choice; do not remove the feature solely for RFC timing. |
Human judgment recorded@chenmingwei23 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
f5c79b8 to
6b299f3
Compare
A pull request's green is a verdict about refs/pull/N/merge at trigger time. Nothing re-derives it when the base branch moves, so a head that passed and then waited for review can merge against a base its tests never saw. Measured: one PR ran CI once, waited about ten hours, and merged with no re-run while two other PRs added tests the merged change makes fail; the red surfaced later on an unrelated PR, because this repo's own main CI is routinely cancel-evicted by the next merge. green_age.py answers the one question that decides whether the wait invalidated the verdict: did origin/main move in files this PR also touches since the commit this head was tested on? Four overlap classes, most direct first -- same-file, same module directory, a moved .py importing a changed module (absolute or relative, resolved against the moved file's own package), and a moved test_<stem>.py naming a changed directory, which is the incident's own shape and shares no path component at all. A directory that only describes layout (test/, .github/) is not a module directory: those are flat and hold thousands of unrelated files, so counting one made every moved test file collide. Exit 0 fresh, 30 stale, 2 could not measure. pr_status.py prints the same line beside the check rollup and carries the summary in advisory.green_age. It is information for the merger, never a gate: no exit code depends on it, and a probe that cannot measure degrades to "unavailable" rather than flipping a verdict. A merge queue or strict up-to-date protection would serialise a repository whose median merge gap is about 1.4 minutes behind a 19-minute CI, so the check stays client-side and costs a rebase only on a PR whose files actually collide. The skill states the loop's own bounds, so a rule that can re-push every cycle has a stated end: a PR goes to the user after three consecutive re-syncs, the no-push-while-runs-are-in-flight rule stays ahead of exit 30, and exit 2 satisfies the readiness criterion no more than it blocks it. The PR template gains a Backwards compatibility section for the same class of failure on the writer side: a diff that tightens a contract owes a Breaking line plus a writer sweep re-run on fresh main before the last push. The section is deliberately absent from the fork description gate's required list, which would fail every body written before this change. prepare-pr's SKILL.md sat 25 bytes under its 68000-byte ceiling, which is a ratchet that may only tighten. The picture-authoring rules therefore move verbatim into references/body-picture.md with a pointer left in their place, which pays for the new instruction and leaves the ceiling untouched.
6b299f3 to
96daa5a
Compare
…tdev#11508) * feat(prepare-pr): tell the loop when a PR's green has expired A pull request's green is a verdict about refs/pull/N/merge at trigger time. Nothing re-derives it when the base branch moves, so a head that passed and then waited for review can merge against a base its tests never saw. Measured: one PR ran CI once, waited about ten hours, and merged with no re-run while two other PRs added tests the merged change makes fail; the red surfaced later on an unrelated PR, because this repo's own main CI is routinely cancel-evicted by the next merge. green_age.py answers the one question that decides whether the wait invalidated the verdict: did origin/main move in files this PR also touches since the commit this head was tested on? Four overlap classes, most direct first -- same-file, same module directory, a moved .py importing a changed module (absolute or relative, resolved against the moved file's own package), and a moved test_<stem>.py naming a changed directory, which is the incident's own shape and shares no path component at all. A directory that only describes layout (test/, .github/) is not a module directory: those are flat and hold thousands of unrelated files, so counting one made every moved test file collide. Exit 0 fresh, 30 stale, 2 could not measure. pr_status.py prints the same line beside the check rollup and carries the summary in advisory.green_age. It is information for the merger, never a gate: no exit code depends on it, and a probe that cannot measure degrades to "unavailable" rather than flipping a verdict. A merge queue or strict up-to-date protection would serialise a repository whose median merge gap is about 1.4 minutes behind a 19-minute CI, so the check stays client-side and costs a rebase only on a PR whose files actually collide. The skill states the loop's own bounds, so a rule that can re-push every cycle has a stated end: a PR goes to the user after three consecutive re-syncs, the no-push-while-runs-are-in-flight rule stays ahead of exit 30, and exit 2 satisfies the readiness criterion no more than it blocks it. The PR template gains a Backwards compatibility section for the same class of failure on the writer side: a diff that tightens a contract owes a Breaking line plus a writer sweep re-run on fresh main before the last push. The section is deliberately absent from the fork description gate's required list, which would fail every body written before this change. prepare-pr's SKILL.md sat 25 bytes under its 68000-byte ceiling, which is a ratchet that may only tighten. The picture-authoring rules therefore move verbatim into references/body-picture.md with a pointer left in their place, which pays for the new instruction and leaves the ceiling untouched. * style: black-format test_prepare_pr_green_age.py
Problem / Motivation
A pull request's green is a verdict about
refs/pull/<N>/mergeat the moment therun was triggered. Nothing re-derives it when
mainmoves, so a head that passedand then waited for review can merge against a base its tests never saw.
That happened. One PR ran CI once, waited about ten hours for review, and merged
with no re-run. Two PRs merged in between had added tests that hand-write payloads
the merged change now validates, so those tests fail on the merged tree. The red
did not surface on either PR:
main's own CI is routinely cancel-evicted by thenext merge, three runs in a row reading
cancelled, so it appeared two hours lateron an unrelated PR and was fixed there.
Why it matters
The loop that drives a PR to review-ready had no way to notice.
prepare-prrebases before a push, never during the review wait, and
babysit'sBEHINDtrigger is dead on this repository:
mergeStateStatusreportsBEHINDonly understrict up-to-date protection, which is off here. So a PR can sit green for hours,
merge, and hand the failure to whoever opens the next PR.
The obvious server-side fixes are refused by the constraint that matters: the
median merge gap here is about 1.4 minutes against a 19-minute CI, so a merge queue
or strict mode would serialise the repository behind its own CI.
What changed (motivation -> approach -> change)
The loop can now tell whether the wait invalidated the verdict, and it costs
nothing on a PR whose files did not collide.
prepare-pr/scripts/green_age.pycompares the filesorigin/maingained since thecommit this head was tested on top of against the PR's own files, and reports an
overlap in one of four classes: the same file, the same module directory, a moved
.pyimporting a module this PR changed, and a movedtest_<stem>.pywhose stemnames a directory this PR changed. The last one is the incident's shape - the new
tests and the changed module shared no path component and no import. A directory
that only describes layout (
test/,.github/) is not a module directory, becausethose are flat and hold thousands of unrelated files. A relative import resolves
against the moved file's own package, since skipping one reports a real overlap as
no overlap - the dangerous direction. Exit 0 fresh, 30 stale, 2 could not measure;
stdlib only, one injected runner for every command, and no flag or parameter without
a caller -
pr_status.pyreads the dict by importingsummarize(), so the scripthas no
--json.prepare-pr/scripts/pr_status.pyprints the same line beside the check rollup andcarries the summary in
advisory.green_age. It is information for the merger,never a gate: nothing is passed to
decide(), no exit code moves, and a probe thatcannot answer prints
unavailableinstead of flipping a verdict. It stays out ofprogress_keytoo, since a base that moves every few minutes would reset a stallstreak forever.
prepare-pr/SKILL.mdruns it each Phase 3 cycle and states the loop's own bounds:a PR goes to the user after three consecutive re-syncs rather than being rebased
forever, the existing no-push-while-runs-are-in-flight rule stays ahead of exit 30,
and exit 2 satisfies the readiness criterion no more than it blocks it - after three
of those the loop reports and hands over.
references/rationale.mdrecords why thecheck is client-side and what it does not close: a human-merged PR gets only the
printed line.
babysit/SKILL.mdwidens its sync trigger and notes whyBEHINDnever fires here.
.github/PULL_REQUEST_TEMPLATE.mdgains a## Backwards compatibilitysection for the writer-side half of the same expiry.The incident's own merge is that half: the PR that waited made a ledger entry's
datavalidate against a registry, so a payload main accepted before now raises -and the writers it broke are the ones the two intervening PRs added, which a sweep
taken when it opened could not have seen. So a tightening diff owes
Breaking:plusa writer sweep re-run on fresh
mainbefore the last push.prepare-pr/SKILL.mdsat 25 bytes under its 68000-byte ceiling, and that ceilingis a ratchet that may only tighten. So the picture-authoring rules move verbatim to
the new
prepare-pr/references/body-picture.mdwith a pointer left in the skill,which pays for the new instruction and leaves the ceiling untouched.
Backwards compatibility
Compatible:
green_age.pyis a new file whose exit code no other gate reads;pr_status.pyadds one printed line and oneadvisorykey while its exit codes,progress_keyand every other field stay as they were; the template gains asection no check requires; the skill's picture rules move to a reference file with
every rule intact and a pointer in their place. Nothing that passes today starts
failing.
Tests
test/test_prepare_pr_green_age.pyruns against two real repositories on disk - anupstreamwhose base branch moves and a clone carrying the feature branch - so theverdict comes from git rather than from a stub. One case per overlap class, one
negative (a base that moved somewhere else stays fresh), one for the flat-directory
rule, one environment error, and the flags:
--ci-basepins the tested commit, anunresolvable one is exit 2,
--prtakes the files from the host, and a failed fetchis exit 2 rather than a green. Each class was mutation-checked: disabling its branch
in
classify_overlapturns its own test red and leaves the rest green.test/test_prepare_pr_status.pyadds four cases: the line prints beside the rollupand lands in
advisorybut notprogress_key; a STALE probe is reported and theexit code stays 0; a probe that raises degrades to
unavailablewith noFRESHclaim; and the probe is asked about the host's own base branch, so a PR against a
release branch is measured against that branch.
test/test_prepare_pr_body_register.pypins the template's two branches, thecontract rule and its Phase 1.5 pointer, and that the section is absent from the
fork gate's required list with the reason recorded in
rationale.md.test/test_prepare_pr_findings.pyaddsgreen_age.pyto the installed-bundlefixtures, since
pr_status.pynow imports it. Three more cases cover the relativeimport (a moved caller writing
from .._sqlite_compat import xis an overlap), theresolution rules themselves (
__init__.py's own package, and climbing past the rootresolving to nothing), and the skill's stated bounds on re-syncs and on exit 2.
Manual verification
Ran
green_age.pyagainst this branch whilemainmoved under it: STALE with theoverlapping paths named when a base commit touched them, and FRESH after the
flat-directory rule stopped every unrelated moved test file from counting - the
first run reported 17 such false overlaps, which is what that rule now removes. The
gate floor's backend surface is green: 1653 scoped tests,
isort,flake8,black,mypyon the new and changed modules, and the diff-scoped scans. The tworemaining
mypy src/kiro_crew/errors are intranscribe.pyandops_mission_control/backend/providers/cloudwatch.py, neither in this diff. Nofrontend file changed, so no rendered surface moved.
Related Issues
no linked issue: the incident was diagnosed while reading merged history, and this
is the client-side half of the fix.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)