Skip to content

fix(claude): name the offered model when an adopted one is refused after a restart - #40424

Merged
buluoray merged 1 commit into
mainfrom
crew/slack/fix-claude-stale-adopted-retry
Nov 21, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/slack/fix-claude-stale-adopted-retry

Conversation

@buluoray

@buluoray buluoray commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A Claude chat whose adopted model is refused after a restart starts on the model the policy now offers, instead of failing to start.

A per-chat Claude chat adopted model A in an earlier start, after the organization policy refused its default and offered A. Later the policy refuses A too and offers B, while the list of models the backend advertised still names A. The Claude process dies and the chat starts again on the same client. The retry asks which model to name, the old list says A is served, and A is what the seed already names, so the client gives up without trying B. The chat cannot start until the cache or the client is reset. Found in review of #40326 (comment 7061547277).

Why it matters

Any Claude chat on a gateway whose model policy tightens while the chat is alive stops starting after its next restart. The user sees a chat that cannot start, with no way to fix it from the chat.

Not a goal

  • Changing how the advertised-model list is refreshed.
  • Changing how many times a start is re-sent (still at most twice).

What changed (motivation → approach → change)

Symptom: the per-chat client returns the refusal and never tries the offered model. Cause: _substitution_retry_model trusts the advertised list even for a model the seed already adopted, and that list can be older than the policy that now refuses it. The shared runtime had its own fix for this one case; the per-chat client had none.

Change: _substitution_retry_model (in client_claude_seed.py) now names the offered model when the pin is the model the seed already adopted. Both callers, the per-chat client and the shared runtime, use this helper, so they take the same branch. The per-chat docstring in client_claude_session.py and the substitution-retry paragraph of docs/system-specs/modules/claude-code-provider.md say so.

Backwards compatibility

Compatible: a start that worked before names the same model; only a start that used to fail at once now tries the offered model.

Removes nothing: the change adds one branch to a helper and keeps every existing path.

Tests

  • test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_model_adopted_earlier_and_refused_now_falls_back_to_the_offered_one: per-chat client with model A adopted earlier. Cases: no cache, cache agrees (lists B only), cache stale (lists A), B refused too, A offered again, A still allowed. Checks each session/new seed: A is never sent again, and B starts.
  • test_claude_shared_runtime.py::test_a_model_adopted_earlier_and_refused_now_falls_back_to_the_offered_one: the same cases on the shared runtime's adopt_substituted_model. That path already took this branch on main; the test pins that both paths agree.
  • Determinism: the 10 new cases ran 3 times in reversed order and 2 times in file order, serially (-n0), all green.

Proof: PROVEN test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_model_adopted_earlier_and_refused_now_falls_back_to_the_offered_one[cache_stale], test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_model_adopted_earlier_and_refused_now_falls_back_to_the_offered_one[offered_refused_too] (head 6a8ed4a)

Manual verification

N/A — unit coverage sufficient: the tests drive the real retry code with stub session/new replies that carry the policy advisory.

Real-machine verification

N/A -- reproducing it needs a live Claude gateway whose organization policy refuses a model mid-session, which no test machine has; the stub tests send the same advisory frames.

Related Issues

Closes #40422

Sibling sweep

Swept: git grep -n "_substitution_retry_model|_adopted_substitute_model" -- src/
Sibling: src/kiro_crew/acp/harness/claude.py -- not affected -- it already fell back to the offered model and now gets the same answer from the shared helper
Sibling: src/kiro_crew/acp/client_claude_session.py -- changed here

Pattern harvest

Not generalizable: the defect is one helper trusting a cached list over a refusal that just happened; no other code adopts a model from a policy advisory.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 21, 2026 14:44
@buluoray
buluoray enabled auto-merge (squash) November 21, 2026 14:44
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 21, 2026
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix sits in the shared _substitution_retry_model helper, so the per-chat client and the shared runtime now give the same answer, and a refused model that was already adopted is never sent again.

[DESIGN-REVIEWED] 6a8ed4a

Description read

[DESCRIPTION-READ] 8fff1ab47f18374a4de024e3048e2505e228a93156553be6da35fa574546f992

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The helper fix is aimed at the cause, and the per-chat client and the shared runtime now use the same code. It leaves a dead copy of the fallback in the shared runtime.

Notes

  • Item 3 — duplicate of src/kiro_crew/acp/harness/claude.py:923-928: the PR description says "The shared runtime had its own fix for this one case". With the new if adopted and adopted in (self._model, pin): return substitute, the helper returns the adopted model only when substitute == adopted. So model = substitute after if model == substitute: return False can no longer run. Subtraction: delete lines 924-928 of adopt_substituted_model (the inner if and the fallback), keep return False under the outer check, and trim the docstring sentence "falls back once to substitute" to match.

What this change ships

Inventory (4 items) — 3 justified

Intent: let a Claude chat start on the model the policy now offers after its earlier adopted model is refused on a restart. FIX.

  1. A per-chat Claude chat whose adopted model is refused after a restart now starts on the offered model instead of failing — justified
  2. The spec's substitution-retry paragraph and the per-chat docstring now describe this case — justified
  3. The shared runtime gets the same answer from the shared helper, which leaves its own fallback branch unreachable — duplicate of harness/claude.py:923-928
  4. Parametrized tests on both paths pin the same cases — justified

[FIRST-PRINCIPLES-REVIEWED] 6a8ed4a

Description read

[DESCRIPTION-READ] bd834ff061f51602b09fdf26482fcc42daa70738ff6f1157b7271644d6901c46

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6a8ed4a

Verdict parsed from the review's SHA-scoped output markers for commit 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6a8ed4a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 6a8ed4aebe29df3f46a71dc06c593ca35afc52dd false-positive: <reason, 20 to 500 characters>

@buluoray
buluoray force-pushed the crew/slack/fix-claude-stale-adopted-retry branch from 5ceb5ae to 6a8ed4a Compare November 21, 2026 14:59
@buluoray

Copy link
Copy Markdown
Contributor Author

span=cbd05e2fa5c4

  • The owning spec still says the adopted-pin refusal is never retried

fixed in 6a8ed4a: the substitution-retry paragraph of docs/system-specs/modules/claude-code-provider.md now says a pin the seed already adopted and the backend refuses is never named again, even when the advertised list still names it, so the retry names the offered model.
Only a refusal of the offered model itself, already named by the seed, is not retried. The paragraph cites both test_a_model_adopted_earlier_and_refused_now_falls_back_to_the_offered_one tests.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 21, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier pending: being reviewed.

The fix in the shared _substitution_retry_model helper makes a restarted per-chat Claude chat name the model the policy now offers when its adopted model is refused. Before, the client trusted a stale list of advertised models and gave up. On the shared runtime the result is unchanged: its own fallback already gave the same answer. Every writer of _adopted_substitute_model also sets _model, so the seed names the adopted model on restart, and a start still sends session/new at most twice. The new per-chat parametrized test (cache_stale and offered_refused_too) would fail on main. CI is green. No regressions found on sibling paths.

Tier: pending: being reviewed.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 6a8ed4aebe · 108 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@bolichen97

Copy link
Copy Markdown
Contributor

Review bot check: no code blocker.

  • _substitution_retry_model now returns the offered model whenever the pin the seed already adopted is the one refused again, so a stale advertised list can no longer send the refused pin a second time. The only case left unretried is a refusal of the offered model itself, which no retry can fix.
  • The parametrized test covers the cache states (none, agrees, stale), the offered model also refused, and the adopted model still allowed, on both the per-chat and shared paths.

One gate item (team real-machine-verify rule): the Real-machine verification section says N/A, but this diff changes what runs at session start. Is there a way to show it on a real gateway? For example, on your own machine with a Claude gateway whose model policy you can change: start a chat that adopts model A, restart, change the policy so A is refused and B is offered, then start the chat again. On main it should re-send A and fail; on this head it should start on B. If that setup is not available to you, say so in the section with what you tried, and a maintainer can decide whether the stub tests are enough.

The Captain held this only for the tier (10%); I'll put it to the maintainer once the section is settled.

AI-assisted, sent by Bolin Review bot

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the whole diff at this head and traced both callers of _substitution_retry_model: the per-chat client's retry and the shared runtime's adopt_substituted_model. The new 3-line guard in client_claude_seed.py sends the offered model when the adopted model is refused. Before this PR the per-chat path re-sent the refused model and gave up. The shared runtime ends with the same result as before. The targeted tests pass at this head. With main's client_claude_seed.py restored, exactly the cache_stale and offered_refused_too cases fail, so the new test pins the bug. The spec paragraph in claude-code-provider.md matches the code.
Done well: the fix lives in the one shared helper, not in each caller, and the tests check the model that each session/new seed actually carried.
Optional:

  • The fallback at src/kiro_crew/acp/harness/claude.py:925-928 can no longer run, because the helper now returns the substitute itself. test_a_refused_pin_falls_back_once_to_the_offered_model reaches it only through a stub. You could shrink the branch to if model == adopted: return False, reword the docstring at 916-918 to match, and drop or retarget that stubbed test.

@buluoray
buluoray merged commit c2b4f1c into main Nov 21, 2026
101 checks passed
@buluoray
buluoray deleted the crew/slack/fix-claude-stale-adopted-retry branch November 21, 2026 16:12
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 21, 2026
buluoray added a commit that referenced this pull request Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude per-chat client re-sends a refused adopted model after a restart instead of the offered one

3 participants