Skip to content

ci: let pr readiness accept pre-rename ai review lane names - #16377

Merged
bolichen97 merged 1 commit into
mainfrom
ci/readiness-legacy-lane-names
Oct 4, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
ci/readiness-legacy-lane-names

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a fork PR whose AI review lanes ran before #16238 renamed them reaches its real readiness verdict without a re-push.

#16238 renamed Opus 5 Review to Opus 5.5 Review and GPT 5.6 Review to GPT 6.1 Review. pr-readiness.yml runs from the base branch, so every fork head whose Stage-2 lanes posted before the rename now sits at pending with "readiness check(s) changed since evaluation; re-checking Opus 5.5 Review (not started), GPT 6.1 Review (not started)", even with every check green. Example: #15593 at 634caf55af. Its rows are Opus 5 Review / GPT 5.6 Review, success, external_id opus-pr-15593-37091252134-2 / gpt-pr-15593-37091252134-2. Nothing carries the new names. Open fork PRs stuck this way right now: #15593, #15029, #14969. Others hit the same wall the next time their verdict recomputes to success.

Why it matters

PR Readiness is the only required status, so these PRs cannot merge. The only way out today is a force-push to re-run every lane, which is costly while CI is saturated.

Not a goal

  • Generalising lane identity (for example, dropping the name filter). See "Alternative considered" below.
  • The /ai-review override re-run lookup in ai-review-human-override.yml, which also finds the fork run by its current name (see Residuals).
  • Same-repo PRs. Their readiness keys on workflow files, not check names, so the rename does not strand them.

What changed (motivation → approach → change)

  • Symptom: a fork head reviewed before the rename is scored green by the verdict step, then downgraded to pending by the publish step's pre-success re-check.
  • Root cause: the verdict step binds a Stage-2 fork row by external_id alone, so it accepts the old-name row. The publish step's fork re-check binds by name and external_id, so it finds no Opus 5.5 Review row and reports "(not started)" on every recompute.
  • Change:
    • One LEGACY_LANE_NAMES job-env table, mapping old name to new name for the two renamed lanes only.
    • The fork re-check reads it so that an old-name row bound to this PR and Fast Gate attempt answers for its lane only when no bound row exists under the current name.
  • Fail-closed:
    • The old-name row is read exactly like a current one: red publishes red, a running row holds.
    • The external_id binding still applies, so a stale attempt still reads "(not started)".
    • A malformed table makes jq --argjson fail, which publishes the [read-failed] pending.
  • Other steps checked: the hold step keys on the run id and MONITORED_LANES on workflow files, so neither needs the alias.
  • Removal: delete the table once no open PR has a pre-ci(review): move every AI review lane to Opus 5.5 and GPT 6.1 Sol #16238 head. Tracked in ci: remove the pre-#16238 lane-name aliases from pr readiness #16373.

Alternative considered and rejected: drop the name filter, so the re-check binds by external_id alone like the verdict step (or picks the newest bound row whatever its name). The cleanup review rated this the more general root fix, and that is a fair reading: the verdict step already trusts external_id alone, and future renames would cost nothing. I rejected it for three reasons:

  1. The scope for this fix is explicit: one alias table, renamed lanes only, and a head with new-name rows uses only those.
  2. Dropping the filter changes the binding contract for all seven fork lanes on every fork PR, for a problem that affects two lanes on a closing set of pre-rename heads. That deserves its own change and review, not a hotfix.
  3. The one case where newest-row-wins and current-name-wins disagree needs an old-name row posted after a current-name row with the same external_id. That is effectively unreachable. A current-name row on that attempt means the lane already ran post-rename, and the override re-run locates the lane by its current name, so it re-runs the post-rename run.

The table is temporary and has a named removal issue, so the rejected generalisation can still be proposed on its own.

Backwards compatibility

Compatible: the re-check output for every lane without an alias is byte-identical. The reviewer diffed old and new jq over 400 random check lists. The only new outcome is an old-name bound row answering where "(not started)" was published before.

Removes nothing: the change only adds an accepted row shape to one read. No key, path, record or validator is taken away.

Tests

  • test/test_pr_readiness_publish.py, which runs the publish step's real shell against a gh stub:
    • a pre-rename head with green old-name rows publishes success;
    • a red old-name row publishes failure under the current lane names;
    • a running old-name row holds at pending, and not as "(not started)";
    • an old-name row from a previous Fast Gate attempt is still not current;
    • when both names are bound, the current-name row wins against a newer old-name row, in both directions.
  • The first two of those fail on main's workflow, with pending published instead of the verdict.
  • test/test_ai_review_workflows.py: the table holds exactly the two ci(review): move every AI review lane to Opus 5.5 and GPT 6.1 Sol #16238 renames. Each new name is a live fork lane, no old name is a live lane, and no workflow other than pr-readiness.yml still posts an old name.
  • Run locally: test_pr_readiness_publish.py, test_pr_readiness_evaluate.py, test_pr_readiness_sweep.py, test_pr_readiness_dispositions.py, test_ci_surface_tests.py, test_ai_review_workflows.py: 1489 passed. Also passing: flake8, isort, the black and subprocess-encoding gates, scripts/docs-lint.sh. scripts/local-gate.py backend lane: 2020 passed. Its frontend lane could not run because of a host npm auth error (E401); this diff touches no frontend file. actionlint is not installed on this host.

Manual verification

  • During review, ran the old and new re-check jq against fix(spawn): admit the host operator's cli spawn without a session #15593's real check-runs. Old: |Opus 5.5 Review (not started), GPT 6.1 Review (not started), the live pending. New: |, which publishes success.
  • After this merges, a stuck PR needs only its PR Readiness run re-dispatched, not a re-push. Its pending is lane-derived and carries no new check evidence, so the self-heal sweep will not re-fire it on its own. Re-dispatch with the full 40-character head SHA (a short SHA reads as a stale revision and publishes nothing):
gh workflow run pr-readiness.yml --repo kirodotdev/KiroCrew --ref main -f pr=15593 -f sha=634caf55af3e63ef71756812325163fa569c4709
gh workflow run pr-readiness.yml --repo kirodotdev/KiroCrew --ref main -f pr=15029 -f sha=4b8bd5073b0d7c6aa094f3c1e4c5204b3b5158f3
gh workflow run pr-readiness.yml --repo kirodotdev/KiroCrew --ref main -f pr=14969 -f sha=f8b14fd33fa3a54d4cfdcf0670ea9020513208cb

Related Issues

Part of #16373

Pattern harvest

Rule candidate: review-prompt
Pattern: a check-run rename must be paired with an audit of every reader that binds by name. Here the publish re-check bound by name while the verdict step did not, and the rename stranded heads that already had the old name.

Residuals (out of this PR's goal):

  • ai-review-human-override.yml's fork rerun_reviewer finds the Stage-2 run by the current check name. On a pre-rename fork head, /ai-review override opus|gpt reports no run to re-run. Readiness is not stuck by this, because a red old-name row publishes red. The way out is a manual re-run of the fork run or a push.
  • The workflow_run trigger list matches workflow names. A re-run of a same-repo lane run created before the rename may not fire the in_progress hold. The publish re-check, which matches by workflow file path, and the sweep still cover that window.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed (/code-review max and /simplify); code follows project style guidelines
  • Documentation updated (docs/ci/ci-and-reviews.md)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@bolichen97
bolichen97 requested a review from a team as a code owner October 3, 2026 05:30
@bolichen97
bolichen97 requested a review from iamwhatever October 3, 2026 05:30
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 1cb07f7eab8656cf1392abe6d17548d60b51a996 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

After merge, re-dispatch PR Readiness for #15593, #15029 and #14969 using their full head SHAs. The sweep will not re-fire them on its own.

[DESIGN-REVIEWED] 1cb07f7

Description read

[DESCRIPTION-READ] 21a2c8544b8914b4ae8b345841a4f85cf51f1aad739d396b40b5fb00dee64a90

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 1cb07f7eab8656cf1392abe6d17548d60b51a996 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The LEGACY_LANE_NAMES alias patches one rename. Deleting the re-check's redundant .name == $name filter removes the cause and every future rename's stranding.

Not justified as shipped

  • Item 1 — symptom-level: the verdict step already says the name is redundant: "No name filter on the read: the external_id match below already names the lane (each lane stamps its own prefix), and it is the only binding a fork verdict is trusted on anyway" (pr-readiness.yml:1139-1141). The opus-pr- and gpt-pr- prefixes did not change in ci(review): move every AI review lane to Opus 5.5 and GPT 6.1 Sol #16238. The re-check's select(.name == $name and …) is the one binding the rename broke. The description's own example shows an old-name row collides with a current-name row on the same external_id only in a case it calls "effectively unreachable". Rejection reason 2 ("changes the binding contract for all seven fork lanes") overlooks that the verdict step already applies that contract to every one of them.
    Subtraction: delete .name == $name and from the fork re-check's select. Then delete the LEGACY_LANE_NAMES env, the $old_name fallback, test_the_legacy_lane_name_table_holds_only_the_lanes_16238_renamed, the renamed_rows="both" precedence test, the doc paragraph's alias half, and removal issue ci: remove the pre-#16238 lane-name aliases from pr readiness #16373.
    Clears when: the name filter is removed in place of the table, or a maintainer accepts the alias as the chosen level.

What this change ships

Inventory (5 items) — 4 justified

Intent: let fork PRs reviewed before the lane rename reach their real readiness verdict without a re-push. FIX.

  1. A pre-rename fork head's old-name lane rows now count for the renamed lanes, via a new job-env alias table. Before, they read as "(not started)". — symptom-level
  2. A red or running old-name row now publishes red or holds, rather than reading as "(not started)". — justified
  3. When rows exist under both the old and current names, the current-name row decides. — justified
  4. The CI doc explains the alias and gives a one-time re-dispatch recipe for stuck PRs. — justified
  5. A test pins the table to the two ci(review): move every AI review lane to Opus 5.5 and GPT 6.1 Sol #16238 renames and checks that no other workflow posts an old name. — justified

[FIRST-PRINCIPLES-REVIEWED] 1cb07f7

Description read

[DESCRIPTION-READ] 18a521c3a7b6b915016e0cc6a8ddcd441133b1cfa847402f2c3bdafd49ba2dd3

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 1cb07f7eab8656cf1392abe6d17548d60b51a996 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 1cb07f7

Verdict parsed from the review's SHA-scoped output markers for commit 1cb07f7eab8656cf1392abe6d17548d60b51a996.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 1cb07f7eab8656cf1392abe6d17548d60b51a996: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 1cb07f7eab8656cf1392abe6d17548d60b51a996 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 1cb07f7

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 1cb07f7eab8656cf1392abe6d17548d60b51a996: <one-sentence reason>

@bolichen97
bolichen97 enabled auto-merge (squash) October 3, 2026 20:44
#16238 renamed the Opus and GPT review lanes. The publish step's fork
re-check binds a Stage-2 row by name as well as external_id, so a fork
head reviewed before the rename scored green and then re-checked as
"Opus 5.5 Review (not started), GPT 6.1 Review (not started)" forever,
with only a push able to clear it.

Add one LEGACY_LANE_NAMES table (old name -> new name, renamed lanes
only). An old-name row answers for its lane only when the head has no
bound row under the current name, and it is read exactly like one: red
stays red, running holds, and the external_id binding still applies.
The hold step and MONITORED_LANES key on run ids and workflow files, so
they need no alias. Removal is tracked in #16373.
@bolichen97
bolichen97 force-pushed the ci/readiness-legacy-lane-names branch from 0aef1a0 to 1cb07f7 Compare October 4, 2026 03:42
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 4, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed (read-only check of diff, tests and CI). Security looks fine. The permissions and the pull_request_target trigger were already there, and this PR changes neither. No PR code is checked out, and the alias table is fixed in the workflow. An old-name row only counts if it is bound to this PR, this head SHA and the current Fast Gate attempt. Red and running rows still block. So a PR cannot pass readiness without a real AI review of that SHA. Approving. First Principles suggests dropping the name filter in place of the alias table. That is worth a follow-up, but it does not block this PR. After merge, re-dispatch PR Readiness for #15593, #15029 and #14969 using their full 40-character head SHAs.

I only read: gh pr view, gh pr diff, both workflow files at their head SHAs, the bot review comments, and the check lists. I posted or changed nothing. On both PRs, every check that ran passed and the Opus and GPT reviews report no blocking findings. The @kirocrew-computer MCP server was not available in this session; this review didn't need it.

@bolichen97
bolichen97 merged commit d7933c4 into main Oct 4, 2026
143 of 144 checks passed
@bolichen97
bolichen97 deleted the ci/readiness-legacy-lane-names branch October 4, 2026 06:18
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants