Skip to content

fix(claude): stop pinning the model in settings.local.json - #40326

Merged
buluoray merged 1 commit into
mainfrom
crew/slack/fix-37240-cc-pinned-model
Nov 21, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/slack/fix-37240-cc-pinned-model

Conversation

@buluoray

@buluoray buluoray commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On the Claude Code harness, a Bot or chat with a pinned model sees every model the account serves in the picker, Opus 5 and Opus 5.5 included, and an in-chat /model switch sticks.

A user on the Claude Code harness reported that the model picker has no Opus 5.5 row, and that the Opus 4.6 rows carry an Opus 5.5 description. Standalone Claude Code on the same host lists Opus 5.5.

Why it matters

Anyone on the Claude Code harness with a Bot or chat pinned to a model loses the newer models of that model's family from the picker, so they cannot pick the newest Opus from Kiro Crew at all. The short list is then cached as what the account is served, so every later picker shows it too.

Not a goal

  • Changing how the picker merges Kiro Crew's model registry with the adapter's list.
  • The upstream half of the report: when the user's own ~/.claude/settings.json pins a model, Claude Code itself folds that family the same way. Kiro Crew does not edit that file.
  • Clearing an already-narrowed advertised-model cache by hand. The next session's full list repairs it on its own.

What changed (motivation → approach → change)

Claude Code reads a model key in a settings file as a pin and folds that model's whole family into one alias row. Kiro Crew wrote the session's pinned model as model into <work dir>/.claude/settings.local.json. So a session pinned to Opus 4.8 was offered one opus row and no Opus 5 or Opus 5.5, Kiro Crew cached that short list as what the account is served, and /model <the pinned id> did nothing because the file kept re-pinning it.

The pin already reaches the session a second way: _apply_startup_model pushes it over session/set_config_option, which narrows nothing. So the settings seed (acp/client_claude_seed.py) no longer writes the session's pin. It still writes availableModels when another settings source defines one, and it still writes model for the one case that needs it: the substitute model the backend names in a restricted-model advisory, warm cache or cold.

With no pin in the seed, session/new starts on the user's own default model. When a policy refuses that default ("Model X is restricted ... Using Y instead"), the one retry names the session's pin if the backend advertised it, and the substitute otherwise (_substitution_retry_model). If the backend refuses the pin too, the start falls back once to the model it offers, as before, since a session on that model beats one that never starts. Both the per-chat client and the shared runtime use this, so one start re-sends session/new at most twice. The retry's model also rides the envelope's flag tier (options.settings.model), so a seed another session shares, which is not rewritten, cannot hide it.

A sibling session that shares another session's seed now checks the model it runs (its substitute, else its pin) against that seed's list, since the seed no longer names a pin. An unpinned sibling can now share a pinned owner's seed: the seed no longer starts it on the owner's model.

The Claude Code provider spec, the field table in src/kiro_crew/docs/agent-spec-fields.md and the mirror docstring say the pin is pushed live and never written.

Backwards compatibility

Compatible: a pinned session still runs its pin, now set only over session/set_config_option; an unpinned session, the substitution retry and the permission keys of the seed are unchanged.

Removes nothing: the seed's model key was a duplicate channel for a value the live push already delivers; the substitute case keeps it.

Tests

  • test_acp_session_mcp.py::TestLocalSettingsSeed::test_a_pinned_model_never_reaches_the_settings_file (new, both with and without another source's availableModels): a pinned session's seed has no model key, and the list still names the pin.
  • test_acp_seed_provenance_sibling_sharing.py: an unpinned sibling shares a pinned owner's seed; an unpinned sibling is refused a seed carrying a substitute; a sibling whose pin the seed does not list is still refused.
  • test_acp_client_model_substitution.py and test_claude_shared_runtime.py: on a warm cache a refused inherited default keeps a served pin (substitute equal to the pin or not), a pin the backend does not advertise adopts the substitute, a refused pin falls back once to the offered model, a start never re-sends more than twice, and a sibling sharing the seed carries the retry model on the envelope.
  • Existing seed tests that pinned the old model key now assert its absence; the substitution re-seed tests set the adopted substitute, as both real callers do.
  • Determinism: the five changed test files ran three times under -n 4 --dist load and once serially (-n0), green each time.

Proof: PROVEN test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_served_pin_survives_a_refused_inherited_default[other_model], test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_served_pin_survives_a_refused_inherited_default[the_pin], test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_pin_not_served_adopts_the_substitute, test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_refused_pin_falls_back_once_to_the_offered_model, test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_start_never_retries_more_than_twice (head 079d382)

Manual verification

Drove the Claude Agent SDK that @agentclientprotocol/claude-agent-acp 0.99.1 installs (@anthropic-ai/claude-agent-sdk 0.3.312), with a fresh HOME and CLAUDE_CONFIG_DIR, CLAUDE_CODE_USE_BEDROCK=1 and no credentials (supportedModels() answers offline), settingSources: ["user", "project", "local"] and a project .claude/settings.local.json:

settings.local.json Opus rows supportedModels() lists
{} (the head's seed for a pinned session) Opus 5.5, Opus 5, Opus 4.8, 4.7, 4.6, 4.6 1M, 4.1
{"model": "us.anthropic.claude-opus-4-8"} (main's seed) one opus row: "Opus 4.8 · Custom Opus model"
{"model": "global.anthropic.claude-opus-4-6-v1[1m]"} one opus row plus the pinned id, both "Opus 4.6 (1M context)"

Real-machine verification

Machine: own Linux computer, Amazon Linux 2023, x86_64, 32 vCPU / 247 GB.
Before (main 0a2e92e): a pinned session's seed is {"model": "<pin>", ...}, and the SDK probe above with that file lists one opus row and no Opus 5 or Opus 5.5.
After (head 079d382): python -m pytest -n0 test/test_acp_session_mcp.py -k pinned_model_never passes (it failed on main with assert 'model' not in {...}), the seed carries no model key, and the SDK probe with that file lists every Opus row.
Cleanup: own computer; the probe's scratch HOME and project folder deleted.

Related Issues

Closes #37240

Sibling sweep

Swept: git grep -nE '["model"] *=|"model": *(self|model|pin|resolved)' -- src/kiro_crew/acp src/kiro_crew/providers
Sibling: src/kiro_crew/acp/client_claude_session.py -- not affected -- its inline settings beside a project-owned file are parsed from the same rendered payload
Sibling: src/kiro_crew/acp/harness/claude.py -- not affected -- the shared runtime seeds through the same _write_claude_local_settings

Pattern harvest

Not generalizable: the defect is one adapter's reading of one settings key; no other harness seed writes a model pin into a file the harness re-reads.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix removes the cause: the seed no longer writes the pin that Claude Code reads as a family fold. The retry and fallback items only keep the substitution behaviour working now that the pin is gone.

What this change ships

Inventory (8 items) — 8 justified

Intent: a Claude Code harness user with a pinned model can see and pick every served model, Opus 5.5 included, and an in-chat /model switch sticks. This is a FIX (#37240, SDK probe in the description, new test that fails on base).

  1. A pinned session's settings.local.json no longer carries model, so the picker lists the whole family. — justified
  2. After a policy refuses the inherited default, the retry names the session's pin when the backend advertises it, else the substitute. — justified
  3. If the backend refuses the pin too, the start falls back once more to the model it offers, so up to three session/new sends in total. — justified
  4. The shared runtime loops through the same two-resend cap (_SUBSTITUTION_RESENDS). — justified
  5. The retry's model also rides the envelope's flag tier (options.settings.model), so a sibling sharing the seed gets it. — justified
  6. An unpinned sibling can now share a pinned owner's seed, and the share check compares the pin instead of the seed's model. — justified
  7. A substitute equal to self._model now reaches the client's retry decision instead of being dropped in client_start. — justified
  8. The provider spec, the field table and the mirror docstring now say the pin is pushed live and never written. — justified

[FIRST-PRINCIPLES-REVIEWED] 079d382

Description read

[DESCRIPTION-READ] 3193649f9dd5b91b5528a02dc42b58eec593e8130deeba50f3ee7f85bb459d54

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. The fix removes the duplicate pin channel at its source, and the live session/set_config_option push already carries the pin.

Notes

  • The substitution retry now writes the session's pin as model (_substitution_retry_model → data["model"] = adopted, plus options.settings.model). Before, it wrote only the backend's substitute. On an install whose inherited default is policy-refused, that retry seed folds the pin's family again and can re-cache the short list, which is the bug in Claude Code harness: picker omits Opus 5 / Opus 5.5 that the host Claude Code serves, and in-chat /model is overridden every turn #37240. Consider writing the substitute on the retry and pushing the pin live afterwards.
  • The Concern.MODEL ruling at src/kiro_crew/providers/mirrors/claude_code.py:101 still says "settings.local.json model when the session pinned one". The patch updates only that module's docstring, so the mirror ruling now disagrees with the new behaviour.

[DESIGN-REVIEWED] 079d382

Description read

[DESCRIPTION-READ] 91d0d9546697d814cd755f877736dcd3774301829adcb1c97e81ea7978b7ce22

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 079d382

Verdict parsed from the review's SHA-scoped output markers for commit 079d382df8df7894e116d3f7c1cdeb503ecd3fc4.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 079d382

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 079d382df8df7894e116d3f7c1cdeb503ecd3fc4 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 21, 2026
@buluoray
buluoray force-pushed the crew/slack/fix-37240-cc-pinned-model branch from 5e72eab to bac3368 Compare November 21, 2026 07:44
@buluoray

Copy link
Copy Markdown
Contributor Author

span=69c367528387
self-added: yes
mechanism: substitution retry names the served pin before the substitute

fixed in bac3368: a substitution retry now names the session's pin when the backend advertised it (_substitution_retry_model), else the substitute, on both the per-chat retry and the shared runtime's adopt_substituted_model; a served pin is kept when the user's inherited default is refused.
A pin the backend does not advertise, or a cold cache, adopts the substitute exactly as before.
Pinned by test_a_served_pin_survives_a_refused_inherited_default and test_a_pin_not_served_adopts_the_substitute.

@buluoray

Copy link
Copy Markdown
Contributor Author

span=9131ee4c107f
self-added: yes
mechanism: substitution retry names the served pin before the substitute

fixed in bac3368: the substitute != self._model guard in _new_session_following_substitution is gone, and the shared runtime's substitute == state._model guard too; the retry is skipped only when the seed already names the model it would write (_adopted_substitute_model), so a substitute equal to the unapplied pin still retries.
The retry names the served pin (_substitution_retry_model) instead of overwriting _model with the substitute, on both paths.
Pinned by test_a_served_pin_survives_a_refused_inherited_default[the_pin] and test_a_refusal_of_the_model_the_seed_names_is_not_retried.

@buluoray

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/system-specs/modules/ (acp-client.md, kas-auth.md, security.md and eight more spec docs) since this head went green (8e7f511 -> 438cee9); CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 21, 2026
@buluoray
buluoray force-pushed the crew/slack/fix-37240-cc-pinned-model branch from bac3368 to 83b6299 Compare November 21, 2026 08:11
@buluoray

Copy link
Copy Markdown
Contributor Author

span=9131ee4c107f
self-added: yes
mechanism: a refused served pin falls back once to the offered model

fixed in 83b6299: when the retry names the cached-served pin and the backend refuses it too, the start falls back once to the model the second advisory offers, on the per-chat retry (_retry_after_model_substitution) and the shared runtime (adopt_substituted_model plus _SUBSTITUTION_RESENDS = 2); a stale cache entry costs one extra session/new, never a failed start.
Pinned by test_a_refused_pin_falls_back_once_to_the_offered_model (both paths) and test_a_start_never_retries_more_than_twice.

@buluoray

Copy link
Copy Markdown
Contributor Author

span=d2197c721950
self-added: yes
mechanism: the retry model rides the envelope's flag tier

fixed in 83b6299: _claude_session_meta carries the retry's model as options.settings.model (the flag tier, which outranks the user's settings) whenever _adopted_substitute_model is the session's model, so a seed a sibling shares and Crew does not rewrite cannot hide it; both the per-chat retry and the shared runtime rebuild that envelope before re-sending.
Pinned by test_a_sibling_sharing_the_seed_carries_the_retry_model_inline.

Claude Code reads a settings `model` key as a pin and folds that
model's whole family into one alias row. Crew wrote a session's pinned
model there, so a session pinned to Opus 4.8 was offered one `opus`
row and no Opus 5 or Opus 5.5, Crew cached that short list as what the
account is served, and an in-chat /model to the pinned id did nothing.

The pin already reaches the session over session/set_config_option
(_apply_startup_model), which narrows nothing, so the seed no longer
writes it. The one model the seed still writes is the substitute a
restricted-model advisory names. A sibling sharing a seed checks the
model it runs against the seed's list.

Closes #37240
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 21, 2026
@buluoray
buluoray force-pushed the crew/slack/fix-37240-cc-pinned-model branch from 83b6299 to 079d382 Compare November 21, 2026 09:16
@buluoray

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/system-specs/modules/ (app-kit-platform.md, computer-use.md, crew-mode.md, dashboard-token-auth.md, learn-cron-dashboard.md, shared-crew.md and more sibling spec docs) since this head went green (438cee9 -> 0a2e92e); CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 21, 2026
@buluoray
buluoray enabled auto-merge (squash) November 21, 2026 09:17
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Nov 21, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Nov 21, 2026
@KiroCrewCaptain

KiroCrewCaptain commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver deep review: Approve

Approved on commit 079d382df8: the review says why.

Tier: 2, cleared by a deep review: the decider's hold.

A deep review by the code change approver, not by a person · commit 079d382df8 · 699 changed lines. If it is wrong, reply here: replies are how the approver is evaluated.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by the code change approver's deep review, not by a person.

Approved on head 079d382. I read the seed, both retry paths and the tests. A pinned session's settings.local.json no longer carries model at a normal start (src/⁠kiro_crew/acp/client_claude_seed.py:371-377), and the pin still reaches the session through session/set_config_option on the per-chat client and the shared runtime. Both retry loops are bounded at two re-sends. PR Readiness is green on this head, including the backend test shards and the new-test stability runs on Ubuntu and Windows. I did not run anything myself.

Done well: test_a_pinned_model_never_reaches_the_settings_file asserts the key is absent with and without another source's availableModels, and the per-chat retry tests assert the exact sequence of seeds sent.

Security: the change touches which sibling sessions may share another session's settings seed. I checked _share_sibling_settings_seed (src/⁠kiro_crew/acp/client_claude_seed.py:227-233). A share still requires every key other than availableModels and model to be equal, so permissions.defaultMode, deny rules and ask rules must match exactly. Only the model comparison changed. An unpinned sibling is still refused a seed that names a model. The model name taken from the backend advisory is still redacted twice before it is logged. No permission, sandbox or approval rule is widened.

Optional:

  • On an install where policy refuses the user's inherited default, the retry writes the pin back as model (src/⁠kiro_crew/acp/client_claude_session.py:1075, src/⁠kiro_crew/acp/client_claude_seed.py:375-377). The picker there stays as it is on main, so this is not a regression, but the fix does not reach those installs. Consider naming that case under "Not a goal", changing the spec's "never written as model" to "never written at a normal start", and correcting the Backwards compatibility sentence that says the substitution retry is unchanged.
  • src/⁠kiro_crew/acp/client_claude_session.py:1044-1045 returns the failed response when the chosen model equals the adopted one, even if the backend now offers a different model. The shared runtime falls back to the offered model in the same state (src/⁠kiro_crew/acp/harness/claude.py:923-928). How often this state occurs is unknown.
  • Three texts still describe the removed write: the Concern.MODEL ruling at src/⁠kiro_crew/providers/mirrors/claude_code.py:101-105, item 3 of the _write_claude_local_settings docstring (src/⁠kiro_crew/acp/client_claude_seed.py:420), and the comment at src/⁠kiro_crew/acp/harness/claude.py:345.

Tier: 2, cleared by a deep review: the decider's hold.
Criteria: no merge conflict, no owner block, no change request, the default branch, no security-sensitive path, no design document to agree on, the readiness status, the review lanes, the P0 history, the linked issues' tier, the review lessons, no concern label, the file count, a safe change category, not its own change, the decider.
Design documents ride along with the code (docs/system-specs/modules/claude-code-provider.md): they were not reviewed as a design, and a person who changes what they decide should ask for that review.

To overrule it, request changes or dismiss this review.

@buluoray
buluoray merged commit 6936cc9 into main Nov 21, 2026
101 checks passed
@buluoray
buluoray deleted the crew/slack/fix-37240-cc-pinned-model branch November 21, 2026 10:35
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 21, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

@buluoray a review finding on a later PR points at code this PR added. I checked it on main f80ac95 and it holds.

Trigger. A per-chat Claude client adopted model A in an earlier start (_adopted_substitute_model == A, _model == A). Later the policy restricts A and offers B, while the advertised-model cache still lists A. The process dies, and ensure_ready runs session/new again on the same client.

Per-chat path. _substitution_retry_model(B) returns A (served by the stale cache), and the guard returns the refusal without trying B:

model = self._substitution_retry_model(substitute)
if model == getattr(self, "_adopted_substitute_model", None):
return session_resp

model = self._substitution_retry_model(substitute)
if model == getattr(self, "_adopted_substitute_model", None):
    return session_resp

The fallback to the second advisory at L1046-L1053 only runs when the pin was first named in this same call, so it does not cover a pin adopted in an earlier start.

Shared-harness twin. The same case falls back once to B:

if model == getattr(state, "_adopted_substitute_model", None):
if model == substitute:
return False
# The served pin was refused as well: fall back once to the model the
# backend offers, since a session on it beats one that never starts.
model = substitute

Verdict: confirmed. A local stub run against main shows it. _retry_after_model_substitution re-issues nothing and returns the refusal. ClaudeHarness.adopt_substituted_model returns True and moves to B. That chat then cannot start again until the cache or the client is reset.

Source: the GPT 6.1 Review comment on #40392 (adjudication upheld it as MEDIUM): #40392 (comment)

A likely fix: give the per-chat guard the twin's branch, so an adopted model that is refused again falls back once to the offered substitute.

AI-assisted, posted by Kiro Crew Captain

buluoray added a commit that referenced this pull request Nov 21, 2026
Claude Code reads a settings `model` key as a pin and folds that
model's whole family into one alias row. Crew wrote a session's pinned
model there, so a session pinned to Opus 4.8 was offered one `opus`
row and no Opus 5 or Opus 5.5, Crew cached that short list as what the
account is served, and an in-chat /model to the pinned id did nothing.

The pin already reaches the session over session/set_config_option
(_apply_startup_model), which narrows nothing, so the seed no longer
writes it. The one model the seed still writes is the substitute a
restricted-model advisory names. A sibling sharing a seed checks the
model it runs against the seed's list.

Closes #37240

(cherry picked from commit 6936cc9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude Code harness: picker omits Opus 5 / Opus 5.5 that the host Claude Code serves, and in-chat /model is overridden every turn

2 participants