Repository navigation
Conversation
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that [DESIGN-REVIEWED] d2356ce Description read[DESCRIPTION-READ] 967f94fa993d244f2479fed0b4f1251bec6053df07df17d4c40d4c7893e51d23 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: a session's or crew's model and effort pickers offer what its actual harness accepts. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] d2356ce Description read[DESCRIPTION-READ] a135a5b07c024c863e8bcc7fe79a3af1e475f5eea424c6ec6722fbd802cff7cc That sha256 names the PR description this verdict read. Recompute it with |
UX Review (Opus 5.5, fork) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS Harness names reach users as raw ids ("not offered by codex", "runs on codex"), and no first-time reader has seen the new picker states. Watch
Suggestions
[UX-REVIEWED] d2356ce Description read[DESCRIPTION-READ] 058b26999fa292ffd0c8b3d9af0d1d8567a019e43f4cba3042c08c99948349b3 That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed Review detailsOne advisory finding; nothing blocks the merge. FINDING — src/kiro_crew/dashboard/chat_handlers.py:7729 — When a cold member thread is routed to a different harness than [OPUS-REVIEWED] d2356ce |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
ff1373f to
08c9ac8
Compare
|
Rebased onto This PR conflicted with main's
Checks: |
|
Addressed the review findings on The failure notice states the outcome and names the harness (UX blocker). That list keeps no last-good copy, so a failed fetch leaves only Auto: "Couldn't load Claude Code's models, so only “auto” is offered." in the composer and split panes, “Inherited” in both crew editors, and the crew editor's notice gains the Retry the composer already had. All 12 authored catalogs; A stored pin the harness does not offer is marked (UX watch).
A live session there answers only for the BUILD it reports (whether the effort option is advertised, at which levels) — including one of the crew's own, since its The effort-cycle shortcut no longer does nothing silently. It cycles the model's levels where nothing has answered, and on a definite Evidence — three images on this head, replacing the five, listed under Screenshots / video with what is stubbed in each. The gateway is an isolated Backwards compatibility now names the readers of the stricter Not taken, with reasons:
Re-checked on this head: |
ee2454e to
1e104a5
Compare
1e104a5 to
d6c31df
Compare
d6c31df to
2502a25
Compare
GET /api/models answers for the configured backend, agent.acp_backend, and every picker read that one list. A session can run another harness: a crewmate DM thread routes through agent.member_acp_backend, and a live session keeps the harness it started on after the default changes. With the default on kiro-cli and the member route on Claude Code, a crewmate's thread ran claude-opus-5-5[1m] while its picker offered kiro-cli's credit-priced catalog and none of the models Claude Code advertised. The crew editor did the same for a crew whose pin the member route's catalog judges. Effort had the same gap. Until a session reported, the composer judged its effort control by the model name its chip showed, over the levels whichever session had last reported to the gateway. A crew on Claude Code whose chip read auto got no control before its first turn, and a codex session got one its build could not take. The crew editor offered every crew one fixed list. On a live codex session, a build that advertises no reasoning_effort option drops every pick (change_effort skips it), yet a GPT-5.x model still read as effort-capable. GET /api/models?backend=<id> now serves that backend's list (a 400 for an id this build cannot select). selection-capabilities reports models_backend when the slot's harness, cold or live, reads another model-id namespace than the configured backend, and GET /api/agents/resolved-model reports it for the backend that judges the crew's pin (_pin_entitlement_backend). The composer, split panes, the model-cycle shortcut and both crew editors' Model field read that harness's list under its own cache key and degraded flag, which the model chip reads too, so it neither replaces nor reads the configured list's last-good cache. The composer, split panes and both crew editors say when that list fails to load, since without a last-good copy a failure leaves only Auto or Inherited. The template pane keeps the configured list, since other crews run the same template file. kiro-cli and kas share one namespace, so a kas session keeps the configured list. selection-capabilities also answers effort_supported and effort_levels for a cold slot's own harness. Where the level rides the model (kiro-cli, claude), the model the slot will run decides, over the shared vocabulary. Where the harness carries it in a session option (pi) or splits a model pair onto one (codex), a live session on that harness answers, and none means no control. resolved-model reports that harness answer to the crew editor, keeping only the levels a crew pin can store, and the editor takes it wherever it is given and otherwise judges the model. A pair-id harness supports effort only when it advertises its effort option. The composer, split panes, both crew editors and the effort-cycle shortcut read these answers.
2502a25 to
6145867
Compare
When a session's or crew's own harness list failed to load, the
composer, split pane and crew editor said "Couldn't load the model list.
What's shown may be incomplete or out of date." That list keeps no
last-good copy, so what is shown is certainly only Auto or Inherited.
They now say so and name the harness: "Couldn't load Claude Code's
models, so only “auto” is offered" (“Inherited” in the crew editor). The
crew editor's notice gains the Retry the composer and pane already had.
harness_effort reported a pi or codex route with no live session as
effort_supported false, so the crew editor told the user to clear a
valid pin, and the same build said the opposite once any codex session
was live. It now answers null, unknown, in that case. The composer still
offers no effort control on null; the crew editor judges the model, and
says the harness takes no effort only on a definite false. The warning
names the harness ("This crewmate runs on codex, which takes no
reasoning effort, so this setting is ignored. Clear it.") and is in the
tu / ты register in fr and ru. Null means "no session on that harness
has answered yet" everywhere, so the composer, the split panes and the
effort-cycle shortcut judge the model the slot will run, as they did
before a session reported: a crew slot is never eager-spawned, so
reading null as false would lose the pick before its first turn. The
shortcut no longer does nothing silently either: on a definite false it
says the session takes no effort. A live session on that backend, even
one of the crew's own, answers only for the build (whether the effort
option is advertised, at which levels), not for the model it is running,
which a slot pick can make another: a pair-id harness (codex) judges the
pair with the slot's model or the crew's pin, so a non-reasoning pick
cannot tell a crew to clear a pin that works. Which live session speaks
now has one order, newest first after the crew's own, the order
_advertised_evidence_provider already read the pin's spelling in.
A stored pin outside the harness's list sat below that harness's own
rows with nothing saying which row runs. Such a pin is not necessarily
unserved: the gateway folds the kiro-cli spelling claude-opus-5.5 onto
Claude Code's global.anthropic.claude-opus-5-5[1m]. resolved-model now
reports pin_served_as, the id a live session on the pin's harness serves
the stored pin as ("" for none, absent while no session has advertised a
list). When that id is a row the editor already lists, the stored pin
shows as THAT row rather than as a second row for the same model;
otherwise the editor labels it "claude-opus-5.5 (runs as
global.anthropic.claude-opus-5-5[1m])" or "claude-opus-5.5 (not offered
by codex)", and says nothing when no session can tell.
temp-screenshots/model-picker: three images replace the five, covering
the failure notices and Retry in the composer, a split pane and the crew
editor, a crewmate thread's effort control before its first turn, the
Effort field open on a harness's own levels, the effort warning, and the
stored pin folded onto the row its harness serves it as.
6145867 to
d2356ce
Compare
|
Thank you @ash663 for this fix and for the careful evidence behind it. Main has moved a long way since this branch was cut (several of the files it edits were split into owner modules), so we carried your two commits onto current main in a same-repo PR: #20998. Your authorship is kept on both commits. Every hunk was moved to the module that now owns that code; the PR description has a table of where each one went. It also fixes three small review findings from the AI lanes. All checks and PR Readiness are green there. The bug is tracked in #20997. I'm closing this PR in favour of #20998, and your branch stays as it is. Thanks again! |
Problem / Motivation
Goal: A session's model and effort pickers, and a crew's Model and Effort fields, offer what the harness that session or crew actually runs on takes.
GET /api/modelsanswers for the configured backend (agent.acp_backend), and every picker reads that one list. A session can run another harness: a crewmate DM thread routes throughagent.member_acp_backend, and a live session keeps its harness after the default changes. On main, with the default on kiro-cli and the member route on Claude Code:claude-opus-5-5[1m]while its picker offered kiro-cli's credit-priced catalog,autoticked, and none of Claude Code's models. The crew editor's Model field did the same, although the member route's catalog judges the pin (_pin_entitlement_backend).auto).change_effort skipped — adapter build exposes no 'reasoning_effort' option, and Codex's own rollout kept runningmax. A GPT-5.x model still read as effort-capable. The crew editor offered every crew one fixed effort list.Why it matters
A model from the wrong list is an id the harness does not serve, so the pick fails or falls back, and the models the session can run cannot be chosen. An effort control the harness cannot take records a level that never runs; a missing one hides a level it does take.
Not a goal
What changed (motivation → approach → change)
api_modelsreads onlyagent.acp_backend, one model-list query key serves every picker, and cold effort answers came from a gateway-wide list. The server already resolves each slot's harness and each crew pin's judging harness; no picker used them.GET /api/models?backend=<id>serves that backend's list (a 400invalid_backendfor an unselectable id, feat: per-chat backend selection #15978's contract).selection-capabilities(cold and live) andresolved-modelreportmodels_backendwhen the harness reads another model-id namespace; kiro-cli and kas share one.useAvailableModels({ backend })keys['available-models', <provider>, <backend>], with its own degraded flag (which the model chip reads) and no last-good cache. The composer, split panes, model-cycle shortcut and both crew editors read it; the template pane keeps the configured list.resolved-modelreportspin_served_as: the id a live session on the pin's harness serves the crew's stored pin as (""for none, absent while no session has advertised a list). When that id is already a row in the editor's list, the stored pin shows as THAT row rather than as a second row for the same model (the gateway foldsclaude-opus-5.5onto Claude Code'sglobal.anthropic.claude-opus-5-5[1m]); otherwise it is labelledclaude-opus-5.5 (runs as global.anthropic.claude-opus-5-5[1m])orclaude-opus-5.5 (not offered by codex), and the editor says nothing when no session can tell.selection-capabilitiesanswerseffort_supportedandeffort_levelsfor the slot's harness. Where the level rides the model (kiro-cli, claude), the model the slot will run decides. Where the harness carries it in an option (pi) or splits a model pair onto one (codex), a live session on it answers, the crew's own first and otherwise the newest (the order_advertised_evidence_provideralready reads a pin's spelling in), but only for the build it reports (which effort option is advertised, at which levels): its own answer judges the model IT runs, which a slot pick can make another, so a pair-id harness judges this slot's model or the crew's pin itself. With no live session on that harnesseffort_supportedisnull: unknown, not unsupported.resolved-modelgives the crew editor that harness answer (harness_effort), keeping only levels a crew pin can store.false. Onnullthey judge the model the slot will run, as they did before a session reported: a crew slot is never eager-spawned, so readingnullasfalsewould lose the pick before its first turn. The effort-cycle shortcut does the same, and on a definitefalsesays "This session takes no reasoning effort, so there is none to cycle." instead of silently doing nothing. The crew editors judge the model themselves and warn that the backend takes no effort only on a definitefalse, and the warning names it: "This crewmate runs on codex, which takes no reasoning effort, so this setting is ignored. Clear it." Effort stays inside the model picker.change_effortalready makes.model-selection.md.Backwards compatibility
Compatible:
models_backend,pin_served_as,effort_supportedandeffort_levelson a cold answer are new optional fields, andGET /api/modelswithout?backend=answers as before. A cold pi or Codex answer with no live session readseffort_supported: null. Until a session's first capability answer arrives (cached per slot), its picker shows the configured list, as on main.AcpProvider.supports_effortis now false for a pair-id harness whose build advertises no effort option. Reader:change_effort,reapply_live_effort,set_modelandclear_effortinproviders/acp.py, andapi_chat_slot_selection_capabilitiesandapi_chat_slot_reasoning_effortinchat_handlers.py. On such a build each already dropped the level before it reached Codex; now they skip it up front, and the composer hides the control.Tests
Each new test fails without this change.
test/test_session_backend_model_list.pyandwebsite/src/test/sessionBackendModelList.test.tsx:?backend=serves and validates; cold and live slots reportmodels_backend, kas does not; the adapter keeps the configured cache; the composer roster and the degraded chip follow the session's harness;resolved-model'spin_served_asas another id, as none, and absent with no advertised list.ChatPane.sessionModelList.test.tsx: a pane names the harness whose list failed.test/test_session_harness_effort.pyandwebsite/src/test/sessionHarnessEffort.test.tsx: cold claude offers effort for its crew's model; cold codex reads unknown, inselection-capabilitiesand inresolved-model, until a live codex session answers; the crew's own codex session answers over another crew's, in both, and otherwise the newest answers over an older one; a live codex session's own model never answers for the crew's pin, whether the session is the crew's own or another crew's (test_no_codex_sessions_model_answers_for_a_crews_pin); a live codex build without its option readsfalse; the crew editor offers only levels a pin can save; the composer follows the cold answer.CrewEditorDialog.test.tsx,KiroCrewAgentsPage.reasoningEffort.test.tsx,App.test.tsx: each crew editor's Model field and Effort pin follow the pin backend; a failed pin list says only Inherited is left, and its Retry refetches it; a stored pin the harness serves under another id shows as that row, not as a second row for the same model; a pin no row carries says what the harness serves it as; the effort warning names the backend; the effort cycle steps only over the session's levels, judges the model where no session has answered yet, and says the session takes no effort rather than silently doing nothing.sessionHarnessEffort.test.tsxandChatPane.sessionModelList.test.tsx: a cold Codex thread keeps its effort control, in the composer and in a pane.test_chat_slot_reasoning_effort.py,test_dashboard_agents_composition_contract.py,test_acp_deepseek_backend.py.On the rebased head: 336 focused backend tests and 167 focused frontend tests passed; after the final cold-thread correction, all 87 directly affected backend tests passed. Agent-SDK boundary, harness-parity, Black, flake8, TypeScript, ESLint, documentation lint, and the frontend production build passed. Full CI is rerunning.
Manual verification
An isolated
--test-modegateway with a throwaway home, driven through the built SPA with Playwright: kiro-cli default, member route on Claude Code (or Codex, with the OS sandbox on). Main ran the same flow as a control.autotickedclaude-sonnet-5-5[1m], then a turnclaude-sonnet-5-5lowtomaxeffort: "low"maxpin, a Codex session liveThe frames below come from the same kind of gateway on this head, with one Claude Code turn and no kiro-cli turn. Two kinds of state are stubbed in the browser, nowhere else:
GET /api/models?backend=claudeanswers 503 in the failure frames, andresolved-modelanswers as a live Codex session would in the two effort frames (one advertising low, medium and high; one without its effort option, like this host's Codex build). One account-specific row of this host's Claude Code list is dropped from every frame.Screenshots / video
A fork PR cannot attach media, so the PNGs are committed under
temp-screenshots/model-picker/(composer.png,crew-editor-model.png,crew-editor-effort.png).A crewmate thread on Claude Code before its first turn, offering Claude Code's list and an effort control for the crew's model; a chat that started on Claude Code before the default went back to kiro-cli, when Claude Code's list fails to load, in the composer and in a split pane; that chat's effort-cycle shortcut on a harness that takes no effort, stubbed as a live Codex session without its effort option answers, the state the third image shows in the crew editor (where no session has answered yet the shortcut cycles the model's own levels, so there is no notice to show):
The crew editor when Claude Code's list fails to load, the stored pin labelled with the id it runs as; after one Claude Code turn, that pin folded onto Claude Code's own
global.anthropic.claude-opus-5-5[1m]row and ticked there, with no second row for the same model:The Effort field open on a harness that reports low, medium and high; a stored Max on a harness that takes no effort, with the pin marked as not offered by it:
Related Issues
#10453 asks for pickers that follow the session's backend. #15978 adds a per-chat backend pick with the same
?backend=contract; this PR covers the routes that pick does not (crewmate threads, crew pins, live sessions after a default change). #16906 edits the same composer files. Whichever lands later rebases on the others.Pattern harvest
Rule candidate: review-prompt
Pattern: a per-session choice (model, effort) read from a gateway-wide list or setting instead of the session's own harness.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)