Skip to content

ci(review): move review lanes to Opus 5.5 (Sonnet 5.5) and GPT 6.1 Sol - #16249

Closed
CrysisDeu wants to merge 1 commit into
mainfrom
ci/review-opus-5-5-gpt-6-1
Closed

CrysisDeu wants to merge 1 commit into
mainfrom
ci/review-opus-5-5-gpt-6-1

Conversation

@CrysisDeu

Copy link
Copy Markdown
Contributor

Problem / Motivation

The two blocking code-review lanes run one model generation behind what the fleet can use: the Opus lane (and the GPT lane's Opus adjudicator) on Claude Opus 5 with Opus 4.8 as overload fallback, the GPT lane on GPT 5.6 Sol. Maintainer decision (#16240): move the review lanes to Claude Opus 5.5 with Claude Sonnet 5.5 as fallback, and GPT 6.1 Sol.

Why it matters

Every same-repo and fork PR is gated on these two verdicts, and the GPT lane hands its blocking candidates to the same Opus model for adjudication. The one required check (PR Readiness) is a function of these lanes, so their model is the quality ceiling of the merge gate.

What changed (motivation → approach → change)

Part Change
Opus model (.github/workflows/) every Opus stage — claude-review.yml discovery + validation, fork-opus-review.yml, the adjudication stage in codex-review.yml and fork-gpt-review.yml — runs --model us.anthropic.claude-opus-5-5
Opus fallback the same six stages move --fallback-model from us.anthropic.claude-opus-4-8 to us.anthropic.claude-sonnet-5-5
GPT model codex-review.yml and fork-gpt-review.yml config.toml: model = "openai.gpt-6.1-sol"
Lane names Opus 5 Review → Opus 5.5 Review, GPT 5.6 Review → GPT 6.1 Review wherever the name is live check identity: workflow/job/step names, the fork check-runs, pr-readiness.yml (workflow_run.workflows: and both mapping specs plus the prefix table), the ai-review-human-override.yml rerun table, cross-references in code-review.yml, design-review.yml, ux-review.yml, fork-ux-review.yml
Adjudicator floor ADJ_CLAUDE_CODE_FLOOR stays 2.1.255 (pinned @anthropic-ai/claude-code 2.1.263 satisfies it); its comment now records that the floor was set for Opus 4.8 (#9216) and Opus 5 / Opus 5.5 / Sonnet 5.5 surfaced no higher one
prepare-pr parity profiles/kirocrew.json: opus reviewer model → claude-opus-5.5, model_tier → claude-sonnet-5.5; gpt reviewer model → gpt-6.1-sol, model_tier → gpt-6.x; SKILL.md repair-preference table names the GPT 6.1 lane
Docs and tests docs/ci/ci-and-reviews.md names the lanes and the Sonnet 5.5 fallback; the tests that pin these spellings follow

Same shape as #12988 (Opus 4.8 → Opus 5), which is the precedent for the rename surface.

Deliberately unchanged: the Fable lanes (design / first-principles / ux / security-scope / fix-loop) and their --fallback-model us.anthropic.claude-opus-4-8, the MODEL_IDS lists in issue-summary / issue-triage / ship-report, gui-user-test.yml, the #9216 incident wording ("Opus 4.8's 2.1.255 floor" is a fact), the human-override "Fable 5" log label, release.yml's historical "GPT 5.6 review on 2ce9bce" note, CHANGELOG.md, and every product model-registry file under src/kiro_crew/model_registry*, website/** and their tests.

Merging this PR. Branch protection (protected-branches ruleset) requires only PR Readiness, so no ruleset edit. pr-readiness.yml runs from main, where workflow_run.workflows: still lists the old names, so these two lanes' completion on this PR does not itself re-trigger readiness; readiness still resolves them (same-repo spec is keyed by workflow file) and is recomputed by every other lane's completion and by the 15-minute sweep. #12988 merged green this way. After merge, in-flight PRs show the renamed lanes as missing until their next push; self-healing.

Backwards compatibility

Compatible: no required check is orphaned, the review markers (<!-- claude-ai-review -->, <!-- codex-ai-review -->, [OPUS-REVIEWED], [GPT-REVIEWED], [BLOCK-MERGE]) and the opus / gpt disposition targets are unchanged, and /ai-review override opus|gpt <sha> keeps its external-id lanes.

Tests

  • test/test_ai_review_workflows.py — lane, step and adjudication spellings; the adjudicator pin assertion checks --model us.anthropic.claude-opus-5-5 explicitly; the GPT config assertion checks openai.gpt-6.1-sol.
  • test/test_pr_readiness_evaluate.py, test_pr_readiness_publish.py, test_pr_readiness_sweep.py — readiness log line, check-run names, prefix table.
  • test/test_prepare_pr_profiles.py, test_prepare_pr_local_review.py — profile model assertions; test_opus_profile_model_matches_the_ci_workflow normalizes us.anthropic.claude-opus-5-5 → claude-opus-5.5 and passes against the new --model lines.

Ran locally: the six files above, 1572 passed (-n0). YAML of every changed workflow parses; kirocrew.json parses; scripts/docs-lint.sh passes. black: the three test files that differ are already in the baseline on main (no new unformatted code). The full suite is CI's.

Manual verification

  • git grep -n -E "Opus 5 Review|GPT 5\.6 Review|gpt-5\.6-sol" leaves only release.yml's historical note.
  • Model-id availability is the open fact this PR proves or fails on. kiro-cli chat --list-models serves claude-opus-5.5 but lists neither sonnet-5.5 nor gpt-6.1-sol, and Bedrock was not reachable from the authoring desk. This PR's own Opus 5.5 Review and GPT 6.1 Review runs are the live probe: a same-repo pull_request run executes the head's own workflow file, so a model Bedrock does not serve fails that lane closed here and nothing lands on main. If either lane fails on a model-not-found / access-denied for us.anthropic.claude-sonnet-5-5 or openai.gpt-6.1-sol, the right move is to report it on ci(review): move code-review lanes to Opus 5.5 (fallback Sonnet 5.5) and GPT 6.1 Sol #16240, not to substitute another model. The Sonnet 5.5 fallback and the adjudication stage only exercise when Opus 5.5 overloads / GPT raises a blocking candidate, so a green run proves the primaries, not the fallback id.

Related Issues

Closes #16240

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Every Opus stage in the blocking code-review fleet (claude-review.yml
discovery and validation, fork-opus-review.yml, and the adjudication
stage of codex-review.yml / fork-gpt-review.yml) runs on
us.anthropic.claude-opus-5-5 with us.anthropic.claude-sonnet-5-5 as the
overload fallback. The GPT lanes (codex-review.yml, fork-gpt-review.yml)
run on openai.gpt-6.1-sol.

The lanes are named by their model, so the check, job, step, comment and
readiness spellings move from "Opus 5 Review" / "GPT 5.6 Review" to
"Opus 5.5 Review" / "GPT 6.1 Review": the four lane workflows,
pr-readiness.yml's workflow_run list and both mapping specs, the
human-override rerun table, cross-references in code-review /
design-review / ux-review / fork-ux-review, docs/ci/ci-and-reviews.md and
the tests that pin those spellings. The prepare-pr profile mirrors the
CI pins (opus claude-opus-5.5 / tier claude-sonnet-5.5, gpt gpt-6.1-sol
/ tier gpt-6.x).

ADJ_CLAUDE_CODE_FLOOR stays 2.1.255; the comment records that the new
pins surfaced no higher floor. The Fable lanes and their opus-4-8
fallbacks, MODEL_IDS in issue-summary / issue-triage / ship-report,
gui-user-test.yml, the product model registry and historical mentions
are unchanged.

Closes #16240
@CrysisDeu
CrysisDeu requested a review from a team as a code owner October 2, 2026 20:35
@CrysisDeu
CrysisDeu requested a review from dodgeblaster October 2, 2026 20:35
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 0a504b0eec097b674c440f24a73240003ef9fdb5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The Sonnet 5.5 fallback id ships unverified and no green run will ever prove it — it only exercises during an Opus 5.5 overload, exactly when it must work.

Not justified as shipped

  • Item 2 — inherited (unverified provider claim): the description concedes kiro-cli chat --list-models "lists neither sonnet-5.5 nor gpt-6.1-sol, and Bedrock was not reachable from the authoring desk", and "a green run proves the primaries, not the fallback id" — so --fallback-model us.anthropic.claude-sonnet-5-5 in six blocking stages rests on an availability claim nothing in this PR or its CI run checks, and if wrong, the overload fallback silently becomes a second failure instead of a rescue.
    Clears when: us.anthropic.claude-sonnet-5-5 is confirmed served on Bedrock us-east-1 (a model listing, or one observed fallback invocation), reported on ci(review): move code-review lanes to Opus 5.5 (fallback Sonnet 5.5) and GPT 6.1 Sol #16240.

What this change ships

Inventory (9 items) — 8 justified

Intent: upgrade the two blocking review lanes to the current model generation per maintainer decision #16240 — an ADDITION (maintenance upgrade), description and diff agree.

  1. Both Opus review lanes (same-repo + fork) and the GPT lane's adjudicator now run Opus 5.5 — justified
  2. Overload fallback for all six Opus stages moves from Opus 4.8 to Sonnet 5.5 — inherited: fallback id unverified on the provider and unprovable by a green run
  3. GPT lane (same-repo + fork) now runs GPT 6.1 Sol; primary is self-proved by this PR's own run — justified
  4. Check names rename to "Opus 5.5 Review" / "GPT 6.1 Review" across readiness, override rerun table, fork check-runs — justified
  5. Adjudicator floor stays 2.1.255; its comment now records the floor's provenance — justified
  6. prepare-pr profile pins (model, model_tier) follow the CI models — justified
  7. SKILL.md GPT repair-preference chain re-headed to "GPT 6.1 best available variant" — justified
  8. docs/ci/ci-and-reviews.md lane table and prose follow — justified
  9. Tests re-pin the new spellings and model ids — justified

Rename sweep verified: Opus 5 Review|GPT 5\.6|gpt-5\.6-sol under .github/ leaves only release.yml:304's historical note, as declared. model_tier is a consumed field (local_review.py:1442), so item 6 changes data, not surface. Item 4 is mandated by the documented model-named-check convention in claude-review.yml itself, so the relabel is derived, not a move needing a lost user.

[FIRST-PRINCIPLES-REVIEWED] 0a504b0

Description read

[DESCRIPTION-READ] 7ed8da4a021f8a61fffb393259e0532cf48ff92b730c0c2bfd8788d2f85d696e

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 0a504b0eec097b674c440f24a73240003ef9fdb5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The overload fallback us.anthropic.claude-sonnet-5-5 ships unverified and CI never exercises it, so a bogus id is discovered only mid-incident.

Watch

  • The description concedes "a green run proves the primaries, not the fallback id" and that kiro-cli lists neither sonnet-5.5 nor gpt-6.1-sol. If Bedrock does not serve the Sonnet id, the next Opus 5.5 overload fails both fail-closed Opus stages with no fallback, reddening every open PR until overload clears or a human runs /ai-review override — a reliability regression on the required gate relative to the proven Opus 4.8 fallback, invisible until the exact moment the fallback exists for. Recoverable (one-line revert, override path), hence not a block.
    Clears when: one confirmed Bedrock invocation of us.anthropic.claude-sonnet-5-5 (or an observed fallback engagement) is recorded on ci(review): move code-review lanes to Opus 5.5 (fallback Sonnet 5.5) and GPT 6.1 Sol #16240, or the fallback stays us.anthropic.claude-opus-4-8 until it is.
  • Same root for the floor: the comment's "Opus 5.5 / Sonnet 5.5 surfaced no higher floor" for ADJ_CLAUDE_CODE_FLOOR: 2.1.255 cannot have been established if Bedrock was unreachable from the authoring desk; a too-low floor fails per-PR instead of at configuration time, which is the silence the floor exists to prevent.
    Clears when: the author states how the no-higher-floor claim was checked for the two new ids, on ci(review): move code-review lanes to Opus 5.5 (fallback Sonnet 5.5) and GPT 6.1 Sol #16240 or in the PR.

[DESIGN-REVIEWED] 0a504b0

Description read

[DESCRIPTION-READ] 35d879f13b8b9ade7d3eb419e016ac385ac39c12284a7c154498a51c95789f0f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 0a504b0eec097b674c440f24a73240003ef9fdb5 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0a504b0

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0a504b0eec097b674c440f24a73240003ef9fdb5: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 2, 2026
@CrysisDeu

Copy link
Copy Markdown
Contributor Author

Superseded by #16238 (opened 15 min earlier by @bolichen97), which carries the piece this PR is missing. Root cause of the red Opus 5.5 Review here, verified with a ReadOnly read of the review account: us.anthropic.claude-opus-5-5 and us.anthropic.claude-sonnet-5-5 are both served and AUTHORIZED in us-west-2 and the lane role's IAM covers us.anthropic.*; the failure is the Claude Code floor. The pinned claude-code-action (v1.0.217) bundles Claude Code 2.1.263, and Opus 5.5 needs 2.1.280, Sonnet 5.5 needs 2.1.284 (claude-code CHANGELOG). #16238 moves the action to v1.0.239 (2.1.287), pins .github/review-cli to 2.1.287 and raises ADJ_CLAUDE_CODE_FLOOR to 2.1.284; its own Opus 5.5 and GPT 6.1 lanes are green. Closing without merge.

@CrysisDeu CrysisDeu closed this Oct 2, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 2, 2026
@bolichen97
bolichen97 deleted the ci/review-opus-5-5-gpt-6-1 branch November 6, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(review): move code-review lanes to Opus 5.5 (fallback Sonnet 5.5) and GPT 6.1 Sol

1 participant