Skip to content

fix(claude): stop pinning the model in settings.local.json (#40326) - #40403

Merged
buluoray merged 1 commit into
release/0.13.0from
crew/slack/pick-40326-0.13
Nov 21, 2026
Merged

buluoray merged 1 commit into
release/0.13.0from
crew/slack/pick-40326-0.13

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On the Claude Code harness, a Bot or chat with a pinned model sees every model the account serves in the picker, Opus 5 and Opus 5.5 included, and an in-chat /model switch sticks.

A user on the Claude Code harness reported that the model picker has no Opus 5.5 row, and that the Opus 4.6 rows carry an Opus 5.5 description. Standalone Claude Code on the same host lists Opus 5.5.

Why it matters

Anyone on the Claude Code harness with a Bot or chat pinned to a model loses the newer models of that model's family from the picker, so they cannot pick the newest Opus from Kiro Crew at all. The short list is then cached as what the account is served, so every later picker shows it too.

Not a goal

  • Changing how the picker merges Kiro Crew's model registry with the adapter's list.
  • The upstream half of the report: when the user's own ~/.claude/settings.json pins a model, Claude Code itself folds that family the same way. Kiro Crew does not edit that file.
  • Clearing an already-narrowed advertised-model cache by hand. The next session's full list repairs it on its own.

What changed (motivation → approach → change)

This is a pick of #40326 onto release/0.13.0 for the next 0.13 build: one commit, git cherry-pick -x of main's 6936cc9aed45. Every change below comes from that main commit. One doc line in docs/system-specs/modules/claude-code-provider.md conflicted; the release side keeps its (issue #22995) reference and the pick drops the model key claim, as on main.

Claude Code reads a model key in a settings file as a pin and folds that model's whole family into one alias row. Kiro Crew wrote the session's pinned model as model into <work dir>/.claude/settings.local.json. So a session pinned to Opus 4.8 was offered one opus row and no Opus 5 or Opus 5.5, Kiro Crew cached that short list as what the account is served, and /model <the pinned id> did nothing because the file kept re-pinning it.

The pin already reaches the session a second way: _apply_startup_model pushes it over session/set_config_option, which narrows nothing. So the settings seed (acp/client_claude_seed.py) no longer writes the session's pin. It still writes availableModels when another settings source defines one, and it still writes model for the one case that needs it: the substitute model the backend names in a restricted-model advisory, warm cache or cold.

With no pin in the seed, session/new starts on the user's own default model. When a policy refuses that default ("Model X is restricted ... Using Y instead"), the one retry names the session's pin if the backend advertised it, and the substitute otherwise (_substitution_retry_model). If the backend refuses the pin too, the start falls back once to the model it offers, as before, since a session on that model beats one that never starts. Both the per-chat client and the shared runtime use this, so one start re-sends session/new at most twice. The retry's model also rides the envelope's flag tier (options.settings.model), so a seed another session shares, which is not rewritten, cannot hide it.

A sibling session that shares another session's seed now checks the model it runs (its substitute, else its pin) against that seed's list, since the seed no longer names a pin. An unpinned sibling can now share a pinned owner's seed: the seed no longer starts it on the owner's model.

The Claude Code provider spec, the field table in src/kiro_crew/docs/agent-spec-fields.md and the mirror docstring say the pin is pushed live and never written.

Backwards compatibility

Compatible: a pinned session still runs its pin, now set only over session/set_config_option; an unpinned session, the substitution retry and the permission keys of the seed are unchanged.

Removes nothing: the seed's model key was a duplicate channel for a value the live push already delivers; the substitute case keeps it.

Tests

  • test_acp_session_mcp.py::TestLocalSettingsSeed::test_a_pinned_model_never_reaches_the_settings_file (new, both with and without another source's availableModels): a pinned session's seed has no model key, and the list still names the pin.
  • test_acp_seed_provenance_sibling_sharing.py: an unpinned sibling shares a pinned owner's seed; an unpinned sibling is refused a seed carrying a substitute; a sibling whose pin the seed does not list is still refused.
  • test_acp_client_model_substitution.py and test_claude_shared_runtime.py: on a warm cache a refused inherited default keeps a served pin (substitute equal to the pin or not), a pin the backend does not advertise adopts the substitute, a refused pin falls back once to the offered model, a start never re-sends more than twice, and a sibling sharing the seed carries the retry model on the envelope.
  • Existing seed tests that pinned the old model key now assert its absence; the substitution re-seed tests set the adopted substitute, as both real callers do.
  • Determinism: the five changed test files ran three times under -n 4 --dist load and once serially (-n0), green each time.

Proof: PROVEN test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_served_pin_survives_a_refused_inherited_default[other_model], test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_served_pin_survives_a_refused_inherited_default[the_pin], test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_pin_not_served_adopts_the_substitute, test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_refused_pin_falls_back_once_to_the_offered_model, test/test_acp_client_model_substitution.py::TestSubstitutionFollow::test_a_start_never_retries_more_than_twice (head 27328f1)

Manual verification

Drove the Claude Agent SDK that @agentclientprotocol/claude-agent-acp 0.99.1 installs (@anthropic-ai/claude-agent-sdk 0.3.312), with a fresh HOME and CLAUDE_CONFIG_DIR, CLAUDE_CODE_USE_BEDROCK=1 and no credentials (supportedModels() answers offline), settingSources: ["user", "project", "local"] and a project .claude/settings.local.json:

settings.local.json Opus rows supportedModels() lists
{} (the head's seed for a pinned session) Opus 5.5, Opus 5, Opus 4.8, 4.7, 4.6, 4.6 1M, 4.1
{"model": "us.anthropic.claude-opus-4-8"} (main's seed) one opus row: "Opus 4.8 · Custom Opus model"
{"model": "global.anthropic.claude-opus-4-6-v1[1m]"} one opus row plus the pinned id, both "Opus 4.6 (1M context)"

Real-machine verification

Machine: own Linux computer, Amazon Linux 2023, x86_64, 32 vCPU / 247 GB.
Before (main 0a2e92e): a pinned session's seed is {"model": "<pin>", ...}, and the SDK probe above with that file lists one opus row and no Opus 5 or Opus 5.5.
After (head 079d382): python -m pytest -n0 test/test_acp_session_mcp.py -k pinned_model_never passes (it failed on main with assert 'model' not in {...}), the seed carries no model key, and the SDK probe with that file lists every Opus row.
Cleanup: own computer; the probe's scratch HOME and project folder deleted.

Related Issues

Part of #37240

Sibling sweep

Swept: git grep -nE '["model"] *=|"model": *(self|model|pin|resolved)' -- src/kiro_crew/acp src/kiro_crew/providers
Sibling: src/kiro_crew/acp/client_claude_session.py -- not affected -- its inline settings beside a project-owned file are parsed from the same rendered payload
Sibling: src/kiro_crew/acp/harness/claude.py -- not affected -- the shared runtime seeds through the same _write_claude_local_settings

Pattern harvest

Not generalizable: the defect is one adapter's reading of one settings key; no other harness seed writes a model pin into a file the harness re-reads.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Claude Code reads a settings `model` key as a pin and folds that
model's whole family into one alias row. Crew wrote a session's pinned
model there, so a session pinned to Opus 4.8 was offered one `opus`
row and no Opus 5 or Opus 5.5, Crew cached that short list as what the
account is served, and an in-chat /model to the pinned id did nothing.

The pin already reaches the session over session/set_config_option
(_apply_startup_model), which narrows nothing, so the seed no longer
writes it. The one model the seed still writes is the substitute a
restricted-model advisory names. A sibling sharing a seed checks the
model it runs against the seed's list.

Closes #37240

(cherry picked from commit 6936cc9)
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 21, 2026 11:07
@buluoray
buluoray requested review from cruisercohen and removed request for a team November 21, 2026 11:07
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 27328f1b112563e29309ba125fb6b2b80729110b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 27328f1b112563e29309ba125fb6b2b80729110b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. The settings seed stops writing the pin as model, and the pin still reaches the session over session/set_config_option. I found no code outside the patch that reads the seed's model key.

Notes

  • With no pin in the seed, session/new now starts on the user's own default model and switches to the pin only afterwards. So a policy that refuses that default now costs an extra session/new round trip, even for sessions whose pin is served. The bounded retry (_SUBSTITUTION_RESENDS = 2) keeps the cost small, and a revert undoes it.

[DESIGN-REVIEWED] 27328f1

Description read

[DESCRIPTION-READ] 69fbc2f1e0e33adad6e4b5dc09f2c2809f1405632fedad1f9be4d04c4825428c

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 27328f1b112563e29309ba125fb6b2b80729110b — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Every item follows from the measured family-fold defect. The one leftover is that the retry model now reaches the adapter by two routes, the seed file and the envelope.

Notes

  • Item 4 — oversized: the envelope now carries the retry model on every retry, not only for siblings. Your own comment says "the flag tier outranks the user's settings the refused default came from", and both paths build their envelope through _claude_session_meta. That makes the seed's data["model"] = adopted in _render_claude_settings_payload a second spelling of the same value. Subtraction: delete the seed's model write, keep the flag tier alone, and the refused-substitute-seed branch of _share_sibling_settings_seed can then go too.

What this change ships

Inventory (8 items) — 7 justified

Intent: on the Claude Code harness, a pinned Bot or chat sees every model in the picker, and an in-chat /model switch sticks. This is a FIX (a release-branch pick of #40326).

  1. A pinned session's settings file no longer names the model, so the picker lists the whole family — justified
  2. After a policy refusal, the retry names the pin if the backend serves it, otherwise the substitute — justified
  3. If the pin is refused as well, the start falls back once more to the offered model (at most two resends) — justified
  4. The retry model also rides the session envelope's settings — oversized: second channel beside the seed's model
  5. An unpinned sibling can now share a pinned owner's seed — justified
  6. The base client asks for a retry even when the substitute equals the pin — justified
  7. The spec, the field table and the mirror docstring now say the pin is pushed live — justified
  8. An import is collapsed in test_claude_shared_runtime.py — rides along

[FIRST-PRINCIPLES-REVIEWED] 27328f1

Description read

[DESCRIPTION-READ] 4229993a87e988b944823b074972b7a60d3da83f8f406dd939f81881e52b44b9

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 27328f1b112563e29309ba125fb6b2b80729110b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 27328f1

Verdict parsed from the review's SHA-scoped output markers for commit 27328f1b112563e29309ba125fb6b2b80729110b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 27328f1b112563e29309ba125fb6b2b80729110b false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 27328f1b112563e29309ba125fb6b2b80729110b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 27328f1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 27328f1b112563e29309ba125fb6b2b80729110b false-positive: <reason, 20 to 500 characters>

@buluoray
buluoray merged commit 7af22ad into release/0.13.0 Nov 21, 2026
89 of 90 checks passed
@buluoray
buluoray deleted the crew/slack/pick-40326-0.13 branch November 21, 2026 12:02
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant