Skip to content

fix(models): a session's model and effort pickers follow its harness - #17080

Closed
ash663 wants to merge 2 commits into
kirodotdev:mainfrom
ash663:fix/picker-lists-session-backend-models
Closed

ash663 wants to merge 2 commits into
kirodotdev:mainfrom
ash663:fix/picker-lists-session-backend-models

Conversation

@ash663

@ash663 ash663 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A session's model and effort pickers, and a crew's Model and Effort fields, offer what the harness that session or crew actually runs on takes.

GET /api/models answers for the configured backend (agent.acp_backend), and every picker reads that one list. A session can run another harness: a crewmate DM thread routes through agent.member_acp_backend, and a live session keeps its harness after the default changes. On main, with the default on kiro-cli and the member route on Claude Code:

  • A crewmate's thread ran claude-opus-5-5[1m] while its picker offered kiro-cli's credit-priced catalog, auto ticked, and none of Claude Code's models. The crew editor's Model field did the same, although the member route's catalog judges the pin (_pin_entitlement_backend).
  • Before a session reports, effort was judged by the model name its chip showed, over the levels whichever session last reported. That thread had no effort control before its first turn (its chip read auto).
  • On a Codex route, effort picks never reached Codex. The gateway logs change_effort skipped — adapter build exposes no 'reasoning_effort' option, and Codex's own rollout kept running max. A GPT-5.x model still read as effort-capable. The crew editor offered every crew one fixed effort list.

Why it matters

A model from the wrong list is an id the harness does not serve, so the pick fails or falls back, and the models the session can run cannot be chosen. An effort control the harness cannot take records a level that never runs; a missing one hides a level it does take.

Not a goal

What changed (motivation → approach → change)

  • Cause: api_models reads only agent.acp_backend, one model-list query key serves every picker, and cold effort answers came from a gateway-wide list. The server already resolves each slot's harness and each crew pin's judging harness; no picker used them.
  • Models:
    • GET /api/models?backend=<id> serves that backend's list (a 400 invalid_backend for an unselectable id, feat: per-chat backend selection #15978's contract). selection-capabilities (cold and live) and resolved-model report models_backend when the harness reads another model-id namespace; kiro-cli and kas share one.
    • useAvailableModels({ backend }) keys ['available-models', <provider>, <backend>], with its own degraded flag (which the model chip reads) and no last-good cache. The composer, split panes, model-cycle shortcut and both crew editors read it; the template pane keeps the configured list.
    • When that list fails to load, what is left is certainly only Auto (Inherited in a crew editor), so the composer, split panes and both crew editors say exactly that and name the harness: "Couldn't load Claude Code's models, so only “auto” is offered.", each with a Retry.
    • resolved-model reports pin_served_as: the id a live session on the pin's harness serves the crew's stored pin as ("" for none, absent while no session has advertised a list). When that id is already a row in the editor's list, the stored pin shows as THAT row rather than as a second row for the same model (the gateway folds claude-opus-5.5 onto Claude Code's global.anthropic.claude-opus-5-5[1m]); otherwise it is labelled claude-opus-5.5 (runs as global.anthropic.claude-opus-5-5[1m]) or claude-opus-5.5 (not offered by codex), and the editor says nothing when no session can tell.
  • Effort:
    • Cold selection-capabilities answers effort_supported and effort_levels for the slot's harness. Where the level rides the model (kiro-cli, claude), the model the slot will run decides. Where the harness carries it in an option (pi) or splits a model pair onto one (codex), a live session on it answers, the crew's own first and otherwise the newest (the order _advertised_evidence_provider already reads a pin's spelling in), but only for the build it reports (which effort option is advertised, at which levels): its own answer judges the model IT runs, which a slot pick can make another, so a pair-id harness judges this slot's model or the crew's pin itself. With no live session on that harness effort_supported is null: unknown, not unsupported. resolved-model gives the crew editor that harness answer (harness_effort), keeping only levels a crew pin can store.
    • The composer and split panes offer no effort control on false. On null they judge the model the slot will run, as they did before a session reported: a crew slot is never eager-spawned, so reading null as false would lose the pick before its first turn. The effort-cycle shortcut does the same, and on a definite false says "This session takes no reasoning effort, so there is none to cycle." instead of silently doing nothing. The crew editors judge the model themselves and warn that the backend takes no effort only on a definite false, and the warning names it: "This crewmate runs on codex, which takes no reasoning effort, so this setting is ignored. Clear it." Effort stays inside the model picker.
    • A pair-id harness supports effort only when it advertises its effort option, the check change_effort already makes.
  • Docs: model-selection.md.

Backwards compatibility

Compatible: models_backend, pin_served_as, effort_supported and effort_levels on a cold answer are new optional fields, and GET /api/models without ?backend= answers as before. A cold pi or Codex answer with no live session reads effort_supported: null. Until a session's first capability answer arrives (cached per slot), its picker shows the configured list, as on main.

AcpProvider.supports_effort is now false for a pair-id harness whose build advertises no effort option. Reader: change_effort, reapply_live_effort, set_model and clear_effort in providers/acp.py, and api_chat_slot_selection_capabilities and api_chat_slot_reasoning_effort in chat_handlers.py. On such a build each already dropped the level before it reached Codex; now they skip it up front, and the composer hides the control.

Tests

Each new test fails without this change.

  • test/test_session_backend_model_list.py and website/src/test/sessionBackendModelList.test.tsx: ?backend= serves and validates; cold and live slots report models_backend, kas does not; the adapter keeps the configured cache; the composer roster and the degraded chip follow the session's harness; resolved-model's pin_served_as as another id, as none, and absent with no advertised list. ChatPane.sessionModelList.test.tsx: a pane names the harness whose list failed.
  • test/test_session_harness_effort.py and website/src/test/sessionHarnessEffort.test.tsx: cold claude offers effort for its crew's model; cold codex reads unknown, in selection-capabilities and in resolved-model, until a live codex session answers; the crew's own codex session answers over another crew's, in both, and otherwise the newest answers over an older one; a live codex session's own model never answers for the crew's pin, whether the session is the crew's own or another crew's (test_no_codex_sessions_model_answers_for_a_crews_pin); a live codex build without its option reads false; the crew editor offers only levels a pin can save; the composer follows the cold answer.
  • CrewEditorDialog.test.tsx, KiroCrewAgentsPage.reasoningEffort.test.tsx, App.test.tsx: each crew editor's Model field and Effort pin follow the pin backend; a failed pin list says only Inherited is left, and its Retry refetches it; a stored pin the harness serves under another id shows as that row, not as a second row for the same model; a pin no row carries says what the harness serves it as; the effort warning names the backend; the effort cycle steps only over the session's levels, judges the model where no session has answered yet, and says the session takes no effort rather than silently doing nothing. sessionHarnessEffort.test.tsx and ChatPane.sessionModelList.test.tsx: a cold Codex thread keeps its effort control, in the composer and in a pane.
  • Pins updated: test_chat_slot_reasoning_effort.py, test_dashboard_agents_composition_contract.py, test_acp_deepseek_backend.py.

On the rebased head: 336 focused backend tests and 167 focused frontend tests passed; after the final cold-thread correction, all 87 directly affected backend tests passed. Agent-SDK boundary, harness-parity, Black, flake8, TypeScript, ESLint, documentation lint, and the frontend production build passed. Full CI is rerunning.

Manual verification

An isolated --test-mode gateway with a throwaway home, driven through the built SPA with Playwright: kiro-cli default, member route on Claude Code (or Codex, with the OS sandbox on). Main ran the same flow as a control.

Surface This PR Main
crewmate thread, Claude, after a turn Claude Code's list, the running model ticked kiro-cli catalog, auto ticked
picking claude-sonnet-5-5[1m], then a turn the Claude Code transcript records claude-sonnet-5-5 n/a
that thread's effort control before its first turn low to max none
picking Low, then a turn the transcript records effort: "low" the same once live
a Codex thread, cold and live no effort control no control live; picks could not take
a Codex crew's stored max pin, a Codex session live kept, with the backend's warning kept, no warning

The frames below come from the same kind of gateway on this head, with one Claude Code turn and no kiro-cli turn. Two kinds of state are stubbed in the browser, nowhere else: GET /api/models?backend=claude answers 503 in the failure frames, and resolved-model answers as a live Codex session would in the two effort frames (one advertising low, medium and high; one without its effort option, like this host's Codex build). One account-specific row of this host's Claude Code list is dropped from every frame.

Screenshots / video

A fork PR cannot attach media, so the PNGs are committed under temp-screenshots/model-picker/ (composer.png, crew-editor-model.png, crew-editor-effort.png).

A crewmate thread on Claude Code before its first turn, offering Claude Code's list and an effort control for the crew's model; a chat that started on Claude Code before the default went back to kiro-cli, when Claude Code's list fails to load, in the composer and in a split pane; that chat's effort-cycle shortcut on a harness that takes no effort, stubbed as a live Codex session without its effort option answers, the state the third image shows in the crew editor (where no session has answered yet the shortcut cycles the model's own levels, so there is no notice to show):

Composer and split pane

The crew editor when Claude Code's list fails to load, the stored pin labelled with the id it runs as; after one Claude Code turn, that pin folded onto Claude Code's own global.anthropic.claude-opus-5-5[1m] row and ticked there, with no second row for the same model:

Crew editor Model field

The Effort field open on a harness that reports low, medium and high; a stored Max on a harness that takes no effort, with the pin marked as not offered by it:

Crew editor Effort field

Related Issues

#10453 asks for pickers that follow the session's backend. #15978 adds a per-chat backend pick with the same ?backend= contract; this PR covers the routes that pick does not (crewmate threads, crew pins, live sessions after a default change). #16906 edits the same composer files. Whichever lands later rebases on the others.

Pattern harvest

Rule candidate: review-prompt
Pattern: a per-session choice (model, effort) read from a gateway-wide list or setting instead of the session's own harness.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@ash663
ash663 requested a review from a team as a code owner October 5, 2026 17:28
@ash663
ash663 requested a review from iamwhatever October 5, 2026 17:28
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of d2356ce950edd04d0a0eca262d6102d8cd9e4ed8 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that effort_commit_needs_reset (session_control.py:6227), a reader this PR doesn't list, now correctly skips the session reset when a Codex build has no effort option.

[DESIGN-REVIEWED] d2356ce

Description read

[DESCRIPTION-READ] 967f94fa993d244f2479fed0b4f1251bec6053df07df17d4c40d4c7893e51d23

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of d2356ce950edd04d0a0eca262d6102d8cd9e4ed8 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

_live_on prefers the crew's own live session over the newest one. The only support is an unverified claim that Codex's advertised levels follow the running model.

Not justified as shipped

  • Item 7 — inherited: the _live_on docstring rests the crew-first preference on "The levels a pair-id harness advertises can follow the model a session runs". But harness_effort says the live session is "evidence of the BUILD alone", and the description says it is consulted "only for the build it reports". If the build alone decides, the crew-first preference fixes no harm. The other half of the order, newest first, is copied from core._advertised_evidence_provider, which only supports it by matching another function.
    Subtraction: delete the mine slot scan in handlers/agents.py:_live_on and keep newest-on-backend only. Drop the crew parameter from _live_on and harness_effort, which then passes it only to _live_on.
    Clears when: a Codex or pi build is shown advertising different reasoning_effort levels depending on which model the session runs.

What this change ships

Inventory (10 items) — 9 justified

Intent: a session's or crew's model and effort pickers offer what its actual harness accepts. This is a FIX.

  1. The composer, split-pane and model-cycle pickers list the session's own harness models — justified
  2. Both crew editors' Model fields list the models of the harness that judges the pin — justified
  3. GET /api/models?backend= added, plus a models_backend field on selection-capabilities and resolved-model — justified
  4. When that list fails to load, the pickers name the harness, offer only auto/Inherited, and show Retry — justified
  5. A stored crew pin appears as the row it is served as, or labelled "runs as" / "not offered" — justified
  6. The effort control appears before a session's first turn, judged by the slot's model — justified
  7. A cold pi/Codex effort answer comes from a live session, preferring the crew's own — inherited
  8. A Codex build with no effort option hides the effort control — justified
  9. The crew Effort field offers only the harness's levels, and its warning names the backend — justified
  10. The effort-cycle shortcut steps through the harness's levels, or says the session takes no effort — justified

[FIRST-PRINCIPLES-REVIEWED] d2356ce

Description read

[DESCRIPTION-READ] a135a5b07c024c863e8bcc7fe79a3af1e475f5eea424c6ec6722fbd802cff7cc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — 🟡 CONCERNS

UX-level review of d2356ce950edd04d0a0eca262d6102d8cd9e4ed8 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

Harness names reach users as raw ids ("not offered by codex", "runs on codex"), and no first-time reader has seen the new picker states.

Watch

  • Codex shows as a raw id. acpBackendName maps only kiro-cli, Claude Code and KAS to display names. Every other harness falls through to its raw id, so the new strings pin_not_offered and effort_ignored_by_this_backend show up in attachment-02 as "(not offered by codex)" and "This crewmate runs on codex…". Right next to them, "Claude Code's models" gets a proper name. Users of a non-Kiro, non-Claude harness see this every time they open the crew editor. It causes friction but doesn't break the task. Fix: add display names for the remaining selectable backends (Codex, pi) in acpBackendName, since the new copy depends on them.
  • The saved pin is shown under a different id than the one stored. In attachment-03 (right), the closed Model field reads global.anthropic.claude-opus-5-5[1m] although the crewmate stored claude-opus-5.5. Nothing on the field itself links the two. A user who reopens the editor may think their pin changed or got rewritten. This happens once per editor open on that harness and costs trust, not the task. Fix: label the folded row with both ids, e.g. global.anthropic.claude-opus-5-5[1m] (saved as claude-opus-5.5).

Suggestions

  • effort_ignored_by_this_backend ends with "Clear it." but offers no control that does it. Add an inline "Clear effort" button that sets the field to Inherited, so the user doesn't have to reopen the select.

[UX-REVIEWED] d2356ce

Description read

[DESCRIPTION-READ] 058b26999fa292ffd0c8b3d9af0d1d8567a019e43f4cba3042c08c99948349b3

That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed d2356ce950edd04d0a0eca262d6102d8cd9e4ed8 via the fork AI-review pipeline; updated in place on each push.

Review details

One advisory finding; nothing blocks the merge.

FINDING — src/kiro_crew/dashboard/chat_handlers.py:7729 — When a cold member thread is routed to a different harness than agent.acp_backend (for example member_acp_backend="claude" while the default is kiro), resolve_effective_model(config, slot.agent or None) checks the crew's pin against the configured backend's namespace (capabilities_for(config.agent.acp_backend)). The provider factory checks it against the routed backend's namespace (loader.py:5028). So once the acp advertised cache is warm, a Claude-namespace pin is dropped and the cold answer falls through to ""/agent.model. harness_effort(..., model_answers=True) then returns a definite effort_supported: false. The composer (effortCaps?.effort_supported != null) hides the effort control and the shortcut says "takes no reasoning effort", even though the session that starts runs the effort-capable pin. → Fix: resolve the cold model in backend's namespace, as the factory does (needs a namespace parameter on resolve_effective_model in the untouched loader.py). The fix that fits inside this diff is to return effort_supported: None when model_registry_namespace(backend) differs from the configured backend's namespace.

[OPUS-REVIEWED] d2356ce

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed d2356ce950edd04d0a0eca262d6102d8cd9e4ed8 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d2356ce

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 6, 2026
@bolichen97
bolichen97 force-pushed the fix/picker-lists-session-backend-models branch from ff1373f to 08c9ac8 Compare October 6, 2026 09:38

Copy link
Copy Markdown
Contributor

Rebased onto origin/main @ 5edf2feecb (head ff1373f6a9 → 08c9ac8a16, pushed with --force-with-lease). Nobody had touched this PR in the last 6 hours.

This PR conflicted with main's selectionCapabilitiesFailed change, which treats a 404 slot_not_found as "not known yet" instead of an error. I resolved the conflicts like this:

  • ChatPane.tsx and composerChips.ts: kept this PR's effortCaps harness fallback, but gated it on !selectionCapabilitiesFailed(selectionCapabilitiesQ) instead of !selectionCapabilitiesQ.isError. The selectionCapabilitiesQ type in composerChips.ts now has both error?: unknown (from main) and data? (from this PR).
  • docs/system-specs/common/model-selection.md: merged both paragraphs. The harness-answer text from this PR comes first, followed by main's text on the 404 and real-fault notice.

Checks: py_compile passes. The 5 Python test files this PR touches pass (290 tests). The frontend tests were not run in the rebase sandbox, so please let CI check ChatPane.tsx and composerChips.ts.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 6, 2026
@ash663

ash663 commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review findings on 08c9ac8a16. One commit, same goal, now ee2454eba1.

The failure notice states the outcome and names the harness (UX blocker). That list keeps no last-good copy, so a failed fetch leaves only Auto: "Couldn't load Claude Code's models, so only “auto” is offered." in the composer and split panes, “Inherited” in both crew editors, and the crew editor's notice gains the Retry the composer already had. All 12 authored catalogs; en-XA regenerated. Tests: ChatPane.sessionModelList.test.tsx (a pane names the harness), CrewEditorDialog.test.tsx and KiroCrewAgentsPage.reasoningEffort.test.tsx (the notice and its Retry refetching).

A stored pin the harness does not offer is marked (UX watch). resolved-model now reports pin_served_as, the id a live session on the pin's harness serves the stored pin as. When that id is a row the editor already lists, the pin shows as THAT row — the gateway folds the kiro-cli spelling claude-opus-5.5 onto Claude Code's global.anthropic.claude-opus-5-5[1m], and two rows for one model read as two models. Otherwise it is labelled claude-opus-5.5 (runs as …) or claude-opus-5.5 (not offered by codex), and the editor says nothing while no session can tell. Tests: test_the_crew_editor_learns_what_the_harness_serves_its_pin_as (served as another id / as none / absent), and in both editors the fold test plus the it.each label test.

effort_ignored_by_this_backend names the backend, and fr/ru are in the tu / ты register. "This crewmate runs on codex, which takes no reasoning effort, so this setting is ignored. Clear it." The fr and ru register tests fail on 08c9ac8a16 and pass here; all 45 src/i18n test files pass with I18N_BASE_REF.

harness_effort answers unknown as a third state (design). With no live session on a pi or codex harness, effort_supported is null, not false. null means "nothing has said yet", so every reader falls back to judging the model the slot will run, as they did before a session reported: the crew editor already did this, and the composer, split panes and effort-cycle shortcut now do too. That matters because a crew slot is never eager-spawned, so reading null as false would drop the effort pick on exactly the threads this PR is about. Only a definite false hides the control and warns that the backend ignores the pin.

A live session there answers only for the BUILD it reports (whether the effort option is advertised, at which levels) — including one of the crew's own, since its supports_effort judges the model IT runs, which a slot pick can make another than the pin. So a pair-id harness judges the pair with this slot's model or the crew's pin. Which live session speaks now has one order, newest first after the crew's own, matching _advertised_evidence_provider. Tests: test_a_cold_codex_thread_reads_unknown_until_its_harness_says, test_the_crew_editor_reads_a_cold_codex_route_as_unknown_not_unsupported, test_no_codex_sessions_model_answers_for_a_crews_pin (parametrized over the crew's own session and another crew's), test_a_cold_codex_thread_takes_the_newest_codex_sessions_answer, and on the frontend a cold Codex thread keeping its control in sessionHarnessEffort.test.tsx and ChatPane.sessionModelList.test.tsx.

The effort-cycle shortcut no longer does nothing silently. It cycles the model's levels where nothing has answered, and on a definite false says "This session takes no reasoning effort, so there is none to cycle." Asserted in App.test.tsx.

Evidence — three images on this head, replacing the five, listed under Screenshots / video with what is stubbed in each. The gateway is an isolated --test-mode one with a throwaway home; its frame log shows no kiro-cli prompt. A phone-sized pass (390x844 and 360x780) also confirms the cold-Codex case end to end: the effort control is there before the first turn, the level picked on it reaches the gateway, the shortcut cycles, and the context, model and effort popovers stay inside the viewport with no chip-strip overflow.

Backwards compatibility now names the readers of the stricter supports_effort (design suggestion): change_effort, reapply_live_effort, set_model, clear_effort, and the two chat_handlers endpoints. Each already dropped the level before it reached Codex on such a build.

Not taken, with reasons:

  • pin_served_as is called oversized by one lane now that the fold works. It is here because the previous round asked for it twice: mark a stored pin the harness does not offer, and do not list one model as two rows. Both answers need the id the harness serves the pin as, and nothing else in the response carries it. Happy to drop it if you would rather the editor said nothing about such a pin.
  • acpBackendName shows a harness without a display name by its raw id, so these strings read "codex" rather than a prettier label. Adding display names for the non-baseline harnesses is a separate change to that helper and its catalog entries, and every string here goes through it, so it would follow one.

Re-checked on this head: Backend Tests (3.12, 7)'s test_slots_broadcast_byte_pacing passes merged with main (its red on the previous head was a timing flake, not from this diff), and Backend Lint & Type Check was cancelled by CI rather than failing.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 6, 2026
@kyleseaman
kyleseaman force-pushed the fix/picker-lists-session-backend-models branch from ee2454e to 1e104a5 Compare October 13, 2026 01:41
@kyleseaman
kyleseaman force-pushed the fix/picker-lists-session-backend-models branch from 1e104a5 to d6c31df Compare October 13, 2026 01:47
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention labels Oct 13, 2026
@kyleseaman
kyleseaman force-pushed the fix/picker-lists-session-backend-models branch from d6c31df to 2502a25 Compare October 13, 2026 02:30
GET /api/models answers for the configured backend, agent.acp_backend,
and every picker read that one list. A session can run another harness:
a crewmate DM thread routes through agent.member_acp_backend, and a live
session keeps the harness it started on after the default changes. With
the default on kiro-cli and the member route on Claude Code, a
crewmate's thread ran claude-opus-5-5[1m] while its picker offered
kiro-cli's credit-priced catalog and none of the models Claude Code
advertised. The crew editor did the same for a crew whose pin the member
route's catalog judges.

Effort had the same gap. Until a session reported, the composer judged
its effort control by the model name its chip showed, over the levels
whichever session had last reported to the gateway. A crew on Claude
Code whose chip read auto got no control before its first turn, and a
codex session got one its build could not take. The crew editor offered
every crew one fixed list. On a live codex session, a build that
advertises no reasoning_effort option drops every pick (change_effort
skips it), yet a GPT-5.x model still read as effort-capable.

GET /api/models?backend=<id> now serves that backend's list (a 400 for
an id this build cannot select). selection-capabilities reports
models_backend when the slot's harness, cold or live, reads another
model-id namespace than the configured backend, and GET
/api/agents/resolved-model reports it for the backend that judges the
crew's pin (_pin_entitlement_backend). The composer, split panes, the
model-cycle shortcut and both crew editors' Model field read that
harness's list under its own cache key and degraded flag, which the
model chip reads too, so it neither replaces nor reads the configured
list's last-good cache. The composer, split panes and both crew editors
say when that list fails to load, since without a last-good copy a
failure leaves only Auto or Inherited. The template pane keeps the
configured list, since other crews run the same template file. kiro-cli
and kas share one namespace, so a kas session keeps the configured list.

selection-capabilities also answers effort_supported and effort_levels
for a cold slot's own harness. Where the level rides the model
(kiro-cli, claude), the model the slot will run decides, over the shared
vocabulary. Where the harness carries it in a session option (pi) or
splits a model pair onto one (codex), a live session on that harness
answers, and none means no control. resolved-model reports that harness
answer to the crew editor, keeping only the levels a crew pin can store,
and the editor takes it wherever it is given and otherwise judges the
model. A pair-id harness supports effort only when it advertises its
effort option. The composer, split panes, both crew editors and the
effort-cycle shortcut read these answers.
@kyleseaman
kyleseaman force-pushed the fix/picker-lists-session-backend-models branch from 2502a25 to 6145867 Compare October 13, 2026 03:11
When a session's or crew's own harness list failed to load, the
composer, split pane and crew editor said "Couldn't load the model list.
What's shown may be incomplete or out of date." That list keeps no
last-good copy, so what is shown is certainly only Auto or Inherited.
They now say so and name the harness: "Couldn't load Claude Code's
models, so only “auto” is offered" (“Inherited” in the crew editor). The
crew editor's notice gains the Retry the composer and pane already had.

harness_effort reported a pi or codex route with no live session as
effort_supported false, so the crew editor told the user to clear a
valid pin, and the same build said the opposite once any codex session
was live. It now answers null, unknown, in that case. The composer still
offers no effort control on null; the crew editor judges the model, and
says the harness takes no effort only on a definite false. The warning
names the harness ("This crewmate runs on codex, which takes no
reasoning effort, so this setting is ignored. Clear it.") and is in the
tu / ты register in fr and ru. Null means "no session on that harness
has answered yet" everywhere, so the composer, the split panes and the
effort-cycle shortcut judge the model the slot will run, as they did
before a session reported: a crew slot is never eager-spawned, so
reading null as false would lose the pick before its first turn. The
shortcut no longer does nothing silently either: on a definite false it
says the session takes no effort. A live session on that backend, even
one of the crew's own, answers only for the build (whether the effort
option is advertised, at which levels), not for the model it is running,
which a slot pick can make another: a pair-id harness (codex) judges the
pair with the slot's model or the crew's pin, so a non-reasoning pick
cannot tell a crew to clear a pin that works. Which live session speaks
now has one order, newest first after the crew's own, the order
_advertised_evidence_provider already read the pin's spelling in.

A stored pin outside the harness's list sat below that harness's own
rows with nothing saying which row runs. Such a pin is not necessarily
unserved: the gateway folds the kiro-cli spelling claude-opus-5.5 onto
Claude Code's global.anthropic.claude-opus-5-5[1m]. resolved-model now
reports pin_served_as, the id a live session on the pin's harness serves
the stored pin as ("" for none, absent while no session has advertised a
list). When that id is a row the editor already lists, the stored pin
shows as THAT row rather than as a second row for the same model;
otherwise the editor labels it "claude-opus-5.5 (runs as
global.anthropic.claude-opus-5-5[1m])" or "claude-opus-5.5 (not offered
by codex)", and says nothing when no session can tell.

temp-screenshots/model-picker: three images replace the five, covering
the failure notices and Retry in the composer, a split pane and the crew
editor, a crewmate thread's effort control before its first turn, the
Effort field open on a harness's own levels, the effort warning, and the
stored pin folded onto the row its harness serves it as.
@kyleseaman
kyleseaman force-pushed the fix/picker-lists-session-backend-models branch from 6145867 to d2356ce Compare October 13, 2026 03:13
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Oct 13, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you @ash663 for this fix and for the careful evidence behind it.

Main has moved a long way since this branch was cut (several of the files it edits were split into owner modules), so we carried your two commits onto current main in a same-repo PR: #20998. Your authorship is kept on both commits. Every hunk was moved to the module that now owns that code; the PR description has a table of where each one went. It also fixes three small review findings from the AI lanes. All checks and PR Readiness are green there.

The bug is tracked in #20997. I'm closing this PR in favour of #20998, and your branch stays as it is. Thanks again!

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants