Repository navigation
feat(models): show backends' own model names in the harness picker - #22985
Conversation
a0b0ea6 to
7a0d295
Compare
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. One note: the session handle's new persist step copies an existing method word for word, and a return value would do the same job. Notes
What this change shipsInventory (10 items) — 10 justifiedIntent: let users pick a Claude/Codex model by the name the backend itself uses, not a stale label or a raw id. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 547270c Description read[DESCRIPTION-READ] 9aa2234b12c2f12a8c73cf730a193bb06f8a160ed22462724e28686679ade455 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concern blocks this. The labels go into a separate sidecar store, so the id list every entitlement check reads is unchanged. The one change to what gets sent to the backend is scoped to aliases the backend itself advertised, and five committed screenshots show the new surface. Notes
[DESIGN-REVIEWED] 547270c Description read[DESCRIPTION-READ] 37036bfbf1aff921c535ad52df295c14d8b22b39fac5941d16976ba9426dc24b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5, fork) — ✅ PASSUX-level review of UX-Verdict: PASS The committed screenshots show every changed control: picker rows with the backend's model name and a monospace id, in the in-chat picker (light/dark) and Selectable Models (Claude light/dark, Codex). No first-time reader has seen them, because a fork PR gets no blind read here. Notes
[UX-REVIEWED] 547270c Description read[DESCRIPTION-READ] aadd8d6e5d508d57942c6a6532207f9044b31eb591cf20ad85cc78b03d0325f9 That sha256 names everything this verdict read: the PR description, plus the 8 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
7a0d295 to
b1a0fc2
Compare
|
Edited Not a goal at the author's explicit request, for two reasons:
This push also fixes the GPT block (picker search now matches |
b1a0fc2 to
3cb26aa
Compare
3cb26aa to
61be4be
Compare
61be4be to
823fd6f
Compare
|
Rebased onto
|
|
The failing checks on
Please re-run only the failed jobs, not Update branch, which adds a merge commit and reruns all of CI. |
cruisercohen
left a comment
There was a problem hiding this comment.
Approving head b7854dd.
Verified by reading the code: live and cached advertised rows go through one admission (live_advertised_model_rows / _admit_label_rows). Ids are capped at 256 chars and 512 rows, and names and descriptions are cut to ADVERTISED_LABEL_MAX_CHARS (400), so the bound bolichen97 accepted at cad083c holds; the model_registry.py patch is unchanged since then and the row merge moved to agents_cc_model_rows.py as-is apart from key_of. Labels render as JSX text in truncating spans (no innerHTML). The cache persist runs off the loop, and kiro create/load gain no call. The to_provider_id passthrough is narrow: claude_code only, the four listed aliases only, and only when the backend advertised that alias. No security default changes. All checks are green on this head after the re-runs, and both conditions of the r124 hold (bolichen97's request cleared, the Design note answered) are met.
Non-blocking:
refresh_advertised_models:None != {}counts as a change, so a backend that sends no labels re-persists the sidecar on every session.(_ADVERTISED_META.get(provider) or {}) != labelsavoids that.- The bare-alias passthrough also reaches the in-chat live switch (
slot_model.py:163); worth naming in the description next to cron, the pool and the configured default. - A small public predicate would be cleaner than reading
_CLAUDE_CODE_PASSTHROUGH_ALIASESfrom the dashboard module.
Not run locally: backend or frontend tests; I relied on CI.
|
HOLD on b7854dd: three AI notes on this head have no reply. The change looks sound. Live and cached labels share one bounded admission, so bolichen97's request (now dismissed) is fixed, the reply on the What clears the hold, a short reply or fix for each:
— crew-pr-reviewer (r128) |
The harness model picker showed stale, versioned registry labels (and raw ids) instead of the names the backend advertises -- so Claude Code's picker read "Opus 4.8 (1M context)" where the backend means "Opus 5.5", and Codex showed only auto because its advertised list was never persisted. Capture each advertised model's native name and description from the ACP session's models payload, persist them in the advertised-model cache beside the ids (under a reserved sidecar key, legacy id-only files still load), and prefer the backend's own label over a colliding registry row when building the picker. The frontend carries the label through a new displayName field so the Selectable Models list and the in-chat dropdown title on it, keeping the id as the wire value, and picker search matches it. Both the direct-spawn and shared-runtime capture paths feed and persist the labels. Two advertised ids that fold to one registry key (fable and claude-fable-5) keep separate rows. On Claude Code, to_provider_id sends a bare family alias (opus, sonnet, haiku, fable) unchanged when the backend advertised that alias, so a cron job, pooled-session switch or configured default runs the model the label names instead of the registry's older versioned id. A cold cache or a versioned-only list keeps the registry fold. Part of kirodotdev#21747, kirodotdev#22318 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A scripted Playwright capture that renders the real ChatPanel against an isolated loopback Vite server with /api/models mocked to each backend's advertised rows, producing the Selectable Models screenshots used in the PR description. Matches the existing website/scripts/capture-*.mjs pattern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Replies to the r128 hold, plus cruisercohen's non-blocking notes. This push also rebases onto
Please don't use GitHub's Update branch button here: it adds a merge commit and reruns all of CI. |
|
Holding on 547270c for one AI note on this head with no reply. The three r128 items are settled: the Unanswered: the First Principles lane on this head raises a new Item 2 note: Checks on this head are green and no change request is active. — crew-pr-reviewer (r131) |
iamwhatever
left a comment
There was a problem hiding this comment.
Approving 547270c8cd6e3be91ca380873bcfd24d79f35fbd. Live and cached labels share one bounded admission, kiro create/load gain no call, the None != {} re-persist is fixed (model_registry.py#L695), and the bare-alias passthrough is limited to claude_code aliases the backend advertised (model_registry.py#L1060-L1092). bolichen97's change request was cleared by their own later approval, and CI is green.
AI findings judged non-blocking:
- First Principles item 2 (persist flag duplicates
client_model_selection.py): a duplication preference; the feed-then-persist pair is correct (session_handle_model_selection.py#L719-L732). - Design note (
to_provider_idreads the cache): answered in your earlier reply; the firstsession/newoverwrites the cache and a cold cache is main's behaviour. - Design note (reversed display-name test): informational; the description states the reversal for #21747.
- UX note (chip shows the id): polish; the row's id hint keeps chip and checked row matching.
— crew-pr-reviewer (r132)
Problem / Motivation
Goal: Show each backend's own model names in the harness model pickers, instead of stale registry labels or raw ids.
The harness model picker titled rows from the model id and a static registry, not the name the backend advertises over ACP. Claude Code's picker read stale versioned labels (e.g. "Opus 4.8 (1M context)") where the backend means "Opus 5.5", and Codex showed only
autobecause its advertised list was never persisted across sessions. With no live session (a cold gateway start) the picker fell back to raw ids likeclaude-sonnet-5.Why it matters
The model picker is how a user chooses which model a chat runs on. Showing a stale or wrong name — or a raw id — makes the user pick blind, and makes the two supported Claude/Codex harnesses look broken relative to their own native CLIs, which show the correct names.
Not a goal
haikurunning Haiku 4.5 on the Claude backend. That is caused by the gateway's seededavailableModelsallowlist, not byclaude-agent-acp, and is tracked separately in claude_code: the seeded availableModels allowlist makeshaikurun Haiku 4.5 #22995.to_provider_idsends a bare family alias (opus,sonnet,haiku,fable) unchanged, but only when the backend itself advertised that alias. Without it, a cron job, a pooled-session switch or a configured default ofopussendsclaude-opus-4-8, so the new "Opus 5.5" label would be false on those paths. With a cold cache, or a backend that advertises only versioned ids (e.g. Bedrockglobal.anthropic.*), the registry fold applies as before.What changed (motivation → approach → change)
The model pickers title each model with the name its backend advertises over ACP; the id a selection stores and sends is unchanged.
Root cause: the advertised-model cache stored ids only, and the frontend had no field for a display label, so rows were titled on the id.
acp/)session/new,session/load(runtime_session_create.py,runtime_session_load.py) and the direct-spawn path (client_model_selection.py) read each advertised model'sname/description;session_handle_model_selection.pypersists them off the event loop. On the shared runtime the cache is fed and persisted only for anACP_BACKENDS_ADVERTISED_MODEL_SELECTIONmember, gated at the create and load call sites asclient_start.pygates its persist, so kiro's create and load gain no call (H13).model_registry.py)__labels__; id-only files still load.advertised_model_rows()serves them when no session is live. A live session's rows go through the same admission (live_advertised_model_rows()): an id over 256 characters is refused and a label's name and description are cut toADVERTISED_LABEL_MAX_CHARS(400), so live and cached rows carry the same bounds.to_provider_idsends a bare Claude Code family alias (CLAUDE_CODE_PASSTHROUGH_ALIASES) unchanged only when the backend advertised it. That reaches everyto_provider_idcaller: cron runs, the session pool's switch, the configured default and the in-chat live switch (dashboard/chat_api/slot_model.py). A backend whose rows carry no labels does not re-persist the sidecar.dashboard/handlers/agents.py)/api/modelscarriesdisplay_name; two advertised ids that fold to one registry key keep separate rows. A versioned id claims the registry row before a bare family alias, which names Claude Code's current model, and an alias with no native label shows its own id._cc_modelshands that row merge tomerge_advertised_rowsin the newdashboard/handlers/agents_cc_model_rows.py, soagents.pystays under the 2,000-line cap now that main has grown it; the facade's_normalize_model_keyis passed in, so a patch of it still reaches the merge.website/src)ModelInfo.displayName(providers/types.ts, mapped inproviders/adapters/acp.ts) titles Selectable Models (pages/settings/ChatPanel.tsx) and the in-chat picker list (components/ModelDropdownList.tsx), and picker search (useFilteredDropdown.ts) matches it. The active row is found by exact id before a normalized match (lib/model.ts), and a grouped effort row drops a single variant's name (useInteractiveModels.ts). Theautorow keeps its catalog label. A kiro-cli row whose printed name differs from its id (already sent asdisplay_name) is titled the same way. In both lists a named row shows the name in the body font and its id after it in small monospace (components/MultiSelect.tsxgains an optionalhintfor the Selectable Models row), so the row still matches the composer chip, the Default Model select and the config file, which show the id; a row without a name keeps its id alone.docs/system-specs/common/model-selection.mddocuments the alias passthrough and how Claude rows are labelled;docs/system-specs/modules/learn-cron-dashboard.mdnames the new row-merge module beside theagents.pyfacade.website/scripts/capture-harness-model-picker.mjsrenders the screenshots below;website/capture/chat-model-picker-native-names.{html,tsx}mounts the real composer and in-chat picker for the in-chat shots..github/black-baseline.txtdropstest/test_model_registry.py, which is black-clean.Backwards compatibility
Compatible: the persisted sidecar adds a new reserved key (
__labels__); an existing id-only advertised-model file still loads and simply has no labels until the next capture. The frontenddisplayNameis optional and falls back to the id. No field is removed, renamed, or made required.Removes nothing: no path, record, key, or public function is hidden, deleted, tightened, or renamed.
Tests
test/test_model_registry.py— label persistence in the sidecar, legacy id-only load, the native-name-wins merge, and the alias passthrough only for an advertised alias.test/test_cc_models_endpoint.py—/api/modelsmapsmodelId/name/description, the backend's label wins over a stale registry version, and two advertised ids sharing a key stay distinct.test_registry_display_name_wins_for_survivorsis deliberately reversed astest_advertised_native_name_wins_for_survivors(claude backend: bare model aliases (sonnet/opus/haiku/fable) resolve to stale versions instead of Claude Code's current model #21747): a backend name that differs from its id now wins over the registry label; a name equal to the id still falls back to the registry.test/test_codex_advertised_models.py— CodexconfigOptionscapture persists labels and survives a cold reload with no live session; a cached label is cut to the label bound, and live and cached rows for one payload are identical. A backend whose rows carry no labels reports no change on a repeat capture, so it does not rewrite the sidecar every session.test/test_cc_models_endpoint.py— an oversized live label is cut to the cache's bound in_advertised_cc_modelsand in the assembled/api/modelsrows, and a live id past the id bound never reaches the picker.test/test_cc_models_endpoint.py— a versioned id claims the registry row before a bare alias, and an unlabelled alias shows its id.test/test_session_handle_bindings.py,test/conftest_resets.py— the session handle's member inventory carries the feed and persist hooks; the label cache is reset between tests.test/test_acp_runtime_load_session.py— a kirocreate_sessionandload_sessionnever feed or persist the advertised-model cache (H13);test/test_acp_codex_harness.py— a codexcreate_sessionfeeds and persists it once.test/test_session_pool.py— on Claude Code, a pooled session on the versioned flagship id switches to an advertised bareopus, and keeps the registry fold (no switch) whenopusis not advertised.website/src/test/useFilteredDropdown.test.tsx,website/src/test/ChatPanel.modelVisibility.test.tsx— search matchesdisplayName; a Selectable Models row renders the native name with its id as a hint while toggling writes the id, and a row without a name carries no hint.website/src/test/MultiSelect.hint.test.tsx— the hint renders only on a row that has one, and search still matches the id; a 400-character label and description stay in truncating elements with the hint shown.website/src/test/AcpAdapter.models.test.ts— theautorow carries no backend display name.website/src/test/ModelDropdownList.displayName.test.tsx— a named row shows the name withoutfont-monoand the id in a monospace hint; an unnamed row, or one whose name equals its id, shows the monospace id and no hint; a 400-character label stays on one truncated line with its id hint.website/src/test/model.displayModel.test.ts,website/src/test/useInteractiveModels.test.ts— withfableandclaude-fable-5both listed, the pinned or inherited id checks its own row; a grouped effort row without a base row carries no variant's display name.Determinism proof (428 tests on the rebased tree: the 38 added or changed backend tests plus
test/test_session_handle_bindings.py): 20/20 runs at-n0; 10/10 at-n 4 --dist loadgroup --max-worker-restart=2; a shuffled order green, seed 28411. Frontend (95 tests in the seven changed test files): 10/10 repeats, and--sequence.shuffle --sequence.seed=40213green.Manual verification
Drove both ACP bridges'
session/newdirectly against a live account: Claude Code advertised "Opus 5.5" / "Sonnet 5.5"; Codex advertised "6.1 Sol" / "6 Astra" / "Daybreak Blue" — each matching that backend's native CLI picker. Rendered the realChatPanelSelectable Models list against mocked/api/modelsfor the screenshots below.Screenshots / video
Selectable Models (Settings → Chat → Model), rows titled on the backend's own names with each id beside them:
Claude Code harness, light theme
In-chat picker (composer, Claude Code harness): the open list titles rows on the backend's names with each id beside it; the chip shows the id
opus, matching the checked row's id.In-chat picker, light theme
Related Issues
Fixes #21747
Fixes #22318
Checklist
docs/system-specs/common/model-selection.mddocuments the alias passthrough