Repository navigation
feat: per-chat backend selection - #15978
billygerhard wants to merge 1 commit into
Conversation
Design Review (Opus 5.5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The backend pick lives in two stores: the metadata line the agent can edit, and the gateway record ( Watch
[DESIGN-REVIEWED] adba45a Description read[DESCRIPTION-READ] 5ed7701a1a69e54251ec8176c6d53569262fe3403a3e15d4433610d45b190b23 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — 🔴 BLOCKPremise-level review of First-Principles-Verdict: BLOCK The pick is stored twice: on the agent-editable metadata line and in a gateway record. About 300 lines of rollback code exist only to keep those two copies in agreement. Not justified as shipped
What this change shipsInventory (10 items) — 7 justifiedIntent: let each chat run on its own AI backend, set from the composer or by an agent, and keep that choice across restarts. ADDITION.
Blockers
[FIRST-PRINCIPLES-REVIEWED] adba45a Description read[DESCRIPTION-READ] 13d39c0dedf16e3ca4bfed69818858aab0cb90dfafbedd59c9ce1b342ab3ed86 That sha256 names the PR description this verdict read. Recompute it with |
UX Review (Opus 5.5, fork) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS The Default row and the Kiro CLI row look like the same choice, and the chip cuts off the one word that tells them apart. Watch
Suggestions
[UX-REVIEWED] adba45a Description read[DESCRIPTION-READ] a3c7a0ffa811d62e85125d213e36818462fbfece22a6d36855d6ac2f2c1fd638 That sha256 names everything this verdict read: the PR description, plus the 19 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings block the merge; one advisory finding. FINDING — src/kiro_crew/dashboard/chat_runner.py:10261 — a chat that gets a backend pick while it is still an unbound channel survivor keeps that pick after [OPUS-REVIEWED] d4d2e56 |
7aee478 to
4a0cf0c
Compare
4a0cf0c to
20de275
Compare
20de275 to
c901da3
Compare
e40cfe6 to
896da52
Compare
22d1894 to
d9a2d07
Compare
a924d85 to
c1a7786
Compare
a953c42 to
f752f66
Compare
f752f66 to
f357675
Compare
|
This PR lets each chat pick its own AI backend between turns, keeps that pick across a gateway restart, and gives agents a tool that makes the same pick through the composer's validator. I read the description, the file list and the AI-lane verdicts on head 69d3770. It is on hold for three reasons. First, GitHub reports mergeable=CONFLICTING (mergeStateStatus DIRTY, label "merge conflict"), so it needs a rebase onto main. Second, it is a big new feature: about 1,300 changed production lines across 30+ modules, including chat_handlers.py (+552), session allocation and lifecycle, and slot persistence. Third, it makes product and UX decisions about where backend choice lives. On the current head the Design, First Principles and UX lanes all say CONCERNS; GPT and Opus report no blocking findings. Please rebase, answer each whole-design CONCERNS item for the new head, and get a maintainer to sign off on the design and UI placement. A feature this size needs a human approver, not an agent approval. — crew-pr-reviewer |
A chat can pick its own AI backend at creation or between turns from a composer picker (locked while a turn runs, like the model picker). Switching resets the session so the next message starts on the new backend; the transcript is kept and the model pin is cleared. The pick is stored on the chat record and restored with it. It is the first tier of the existing selection gate (chat pick, member route, configured default), so a chat without a pick resolves as before. The model list follows the picked backend via GET /api/models?backend=. Agents read and set a peer session's backend with the session_backend MCP tool, which calls the same validation and switch as the composer route behind the session-control owner gates.
Problem / Motivation
Goal: Let each chat choose its own AI backend between turns, instead of every chat running on the one gateway-wide default. The choice must survive a gateway restart (a chat that silently falls back to the default after a restart is running on a backend its user did not pick), and an agent must be able to make the same choice the composer makes, through the same validator and switch, so the feature is not UI-only.
Today the backend is a gateway setting. A user who wants one chat on Kiro and another on Claude Code has to flip the global setting, which moves every chat at once. There is also no way for an agent to put a peer session on a different backend.
Why it matters
Different backends suit different tasks, and a user often wants them side by side. Without a per-chat pick they either live with one backend for everything or keep switching the gateway, which disrupts every open chat. This is the first of the smaller PRs laid out in #13892, which replaces #11863.
Not a goal
spawn_run(backend=...)for subagents. That is a separate follow-up PR.What changed (motivation → approach → change)
The gateway already resolves a session's backend through one selection gate (member route, then configured default). Rather than add a parallel path, the chat's own pick becomes the gate's first tier: a chat with no pick resolves exactly as before, and an unselectable pick degrades through the same gate instead of failing.
The pick is stored on the chat record and restored with it, so it survives a gateway restart, and clearing it stays cleared. Because the chat's metadata line is editable by the agent's own file tools, a restart restores a pick only when it matches a gateway-owned record (
crew-panels/backend_picks/, masked from the sandbox, the same place restored reasoning-effort levels are vouched). The open-tab restore, the recent-session restore and a History resume all apply that check; the History resume reads the record off the event loop before it rebuilds the tab, and a transfer import brings no record, so its metadata line's pick is dropped. Only the owner-gated paths write that record: the switch and fork. An edited metadata line therefore cannot move a chat to another backend across a restart; a mismatch restores no pick, which is the configured default. The switch writes the record before it publishes the new pick: the record first accepts both the prior and the new pick, the metadata line is saved, and only then is the record narrowed to the new pick, so a gateway that dies between the two writes restarts on whichever pick the metadata line holds instead of dropping both. Every refusal narrows the record back to the prior pick (or, if that write fails, removes the record so a restart restores the default rather than the refused pick; a refusal that could do neither answers500 backend_rollback_failed), and a refusal after the session reset also tears down the session a concurrent send started on the refused pick, even mid-turn; only that session, so a send that lands after the rollback keeps its turn on the restored backend. After a committed switch the record is narrowed to the new pick, and success is reported only once it holds the new pick alone. If that write fails the switch is refused and undone: the pick, model pin and pick generation roll back, the metadata line is put back on the prior pick, the record is narrowed back to it where it can be, and a session started meanwhile on the new backend is discarded. The answer is500 backend_persist_failed, or500 backend_rollback_failedif the metadata line cannot be put back. The record is never removed in this case, because a restart with no record restores the default rather than either pick. Re-picking the backend a chat already has rewrites the record and saves the metadata line before it answers success (without resetting the session or clearing the model pin), so a pick whose record an earlier failed rollback removed is repaired by picking it again; a re-pick that cannot be saved answers500 backend_persist_failed. Deleting a chat's transcript deletes its record too. The record is keyed on the transcript's file stem, so the slot'sdashboard:key and thedashboard_stem the recent-session restore and history delete use reach the same record. A forked chat carries its parent's pick, as it already carries the parent's model pin from that backend's catalog.POST /api/chat/slots/{slot}/backendsets it, including on a new chat that has no session yet; chat creation itself takes no backend, since the switch route already covers a new chat. A switch is refused with409 turn_in_flightwhile a turn runs, starts a fresh session on the new backend on the next message, keeps the transcript, and clears the model pin. The caller's authorization is re-checked after every await: after the session reset, before the pick is saved, and once more after the save. A refusal of any type at any of those points rolls back the pick, the model and the pick generation together, and a refusal after the save also rewrites the gateway record to the prior pick, so a restart does not bring the refused one back. The route and thesession_backendtool share one validator, which accepts only the selectable built-in ids (andkiro) and refuses anything else withinvalid_backend.GET /api/models?backend=serves the picked backend's catalog, so the model list follows the pick and is populated on first load.In the composer, a
BackendPickersits beside the model chip. It lists only backends that are startable on this host, names the configured backend on its Default row ("Default (Kiro CLI)"), is disabled while a turn runs ("Stop the current response to switch backend"), shrinks with the composer shelf on a narrow pane so the context readout and model chip stay visible, says on every switch that the next message starts a fresh session (and names a cleared model pin), shows its errors as inline ErrorNotices with the ask-agent hand-off on their own row under the composer shelf, where a narrow pane cannot crush them (a refused switch, such as a turn having started or the save failing; the backend probe failing; the config read failing, which would otherwise leave the Default row unnamed; the permanent 403/404 answers stay silent, as on the Agent Backend tab), and is hidden for crew-bound sessions and for chats linked to a messaging channel (the slot row reportsacp_backend_channel_bound; the channel's own replies allocate the shared session without a pick, so a pick there would run the chat on two backends that keep replacing each other; the switch refuses a pick on such a chat with409 channel_backend_unsupportedbut still lets one be cleared, and the agent verb already refuses a linked target with403 linked_session_target). A chat's own model list carries its own degraded flag, so a failed?backend=fetch keeps polling until it recovers instead of leaving the chat on auto-only, and while it is failing both composers say "Couldn't load the models for this backend" rather than looking like a backend with no models. The model-cycle keyboard shortcut and split-view panes read the same per-pick list. If a pick stops being selectable later (a governance ceiling narrows the set), the gate runs the chat on Kiro; the slot row reports that asacp_backend_degraded, the model list switches to Kiro's (the backend the chat now runs on, so the unselectable pick's list is not polled into a permanent error), and the picker labels the pick "(unavailable, using Kiro CLI)" and swaps its icon for a warning sign that carries the same label, so a truncated chip still says so; it never claims the chat is still on it. For agents, asession_backendMCP tool reads and sets a peer session's pick through the same validator and switch as the composer route, behind the existing session-control owner gates, refuses an old relay chat (a read-only archive) for reads as well as writes with the sharedrelay_archive_read_onlyrefusal, as the composer route does, and is unavailable when session control is off.Where it lands:
src/kiro_crew/members.py(pick as the gate's first tier),src/kiro_crew/config/loader.py(namedbackend_overridefactory argument),src/kiro_crew/session_allocation.py(a picked chat bypasses the warm pool, whose children run on the default backend; a session records the pick it was built on, and a chat turn naming a different pick evicts it and cold-starts rather than reusing it, after parking the evicted session's live queued follow-ups so the replacement adopts them with their attachments),src/kiro_crew/session.pyandsession_lifecycle.py(the compaction restart and the reset successor carry the pick, so they rebuild on it; a hard Stop's respawn names none, and the chat's next turn replaces that session),src/kiro_crew/dashboard/chat_runner.py.src/kiro_crew/dashboard/state.py,chat_persistence.py(the gateway record and restore),slot_persistence/metadata_codec.py(theacp_backendrow: how both saves write the pick and how the open-tab, recent-session and History-resume reads take it back),chat_api/resume.py(the History resume reads the record off the loop and passes it in),slot_projection.py,chat.py,chat_fork.py, andsrc/kiro_crew/history.py(the pick is slot-owned so a cleared pick is not carried forward).src/kiro_crew/dashboard/chat_handlers.py,routes/chat.py,handlers/agents.py(/api/models?backend=),server.py(internal-path list),server_runtime/mcp_routes.py(thesession_backendroute).src/kiro_crew/mcp_dashboard.py,src/kiro_crew/dashboard/session_control.py,handlers/session_control.py,src/kiro_crew/validation.py(tool schema),src/kiro_crew/channel.py(blocked for channel agents, like the model switch),src/kiro_crew/data/mcp_tool_titles.json.website/src/components/BackendPicker.tsx,ChatInput.tsx,chat-input/props.ts,website/src/pages/ChatPage.tsx,pages/chat/page/sessionRosters.ts,website/src/hooks/useAvailableModels.ts,website/src/api/client/chatSlotSettings.ts,website/src/providers/adapters/acp.tsandproviders/types.ts(a picked backend's catalog skips the shared last-good cache and leaves the configured list's degraded flag alone),website/src/types/index.ts,website/src/api/acpBackend.ts(modelsBackendFor: a degraded pick keys the model list on Kiro),components/ChatPane.tsx,website/src/shell/shortcuts/shellKeyboard.ts(the model-cycle shortcuts read the per-pick list key), and the picker's strings inwebsite/src/i18n/locales/.src/kiro_crew/docs/session-control.md,src/kiro_crew/docs/index.md,docs/system-specs/modules/session.md,docs/system-specs/modules/session-control.md,docs/feature-map/README.md.Backwards compatibility
Compatible: the pick is optional everywhere. A chat that never sets one, and every existing chat record, resolves through the unchanged member-route/default chain, and the new request fields are optional.
Tests
test/test_per_chat_backend.py: no pick behaves as before; a pick wins over the member route and the default; an unselectable pick degrades through the gate; the factory honours the pick with the sandbox on and off; the turn passes the pick only when there is one; validation accepts selectable ids andkiroand refuses everything else (including path-shaped values); a failed save restores the pick generation while a committed switch keeps it; authorization is re-checked after the reset and a refused caller's pick is never saved; the slot row'sacp_backend_degradedagrees with the gate; the pick survives a restart, whether made through the switch or by a fork, and a cleared pick is not resurrected; an edited metadata line with no matching gateway record, or a different one, restores no pick, on both restore paths; a gate refusal of either error type after the reset rolls back, and one after the save also restores the prior record durably; on a chat with no session yet, a refusal after the save still tears down a session a send started on the refused pick; thesession_backendverb reads, switches through the shared switch, refuses like the route, keeps the owner gates, and is off with session control off;/api/models?backend=kirois accepted.website/src/test/BackendPicker.test.tsx: offers only startable backends, reports the picked id, locks during a turn, shows a Kiro pick as Kiro, names the configured backend on the Default row once config loads, labels a degraded pick as running on Kiro, reports a failed probe instead of vanishing (but not the permanent 403/404 answers), renders nothing with nothing to choose, keeps showing a pick the list no longer offers, and fetches the picked backend's models on first mount; the pin-cleared notice fires only for a real pin.website/src/test/ChatInput.backendPickerShelf.test.tsx: the picker group can shrink and is capped on compact and tiny shelves.website/src/test/AcpAdapter.models.test.ts: a chat's own backend list failing or succeeding leaves the configured list's degraded flag alone.test/test_per_chat_backend.py(the switch's crash window): the record accepts both picks before the new one is published and is narrowed at the end; a crash before or after the metadata save restarts on the line's pick; a refusal after the reset narrows the record back; deleting the transcript removes the record; a record that cannot be narrowed to the new pick refuses the switch, rolls the pick, model and generation back, and a restart keeps the prior pick. Each was mutation-probed.test/test_per_chat_backend.py(resume and rebuilds): a History resume restores a vouched pick and the next save keeps it; an unvouched or mismatched line restores none; a transfer import carries no record; a session rebuilt without the pick is not reused by the chat's next turn, which cold-starts on its pick and then reuses that session; follow-ups queued on the replaced session move to its replacement with their attachments, while a cancelled one is still discarded with its file; a caller naming no pick reuses the live session; the rebuild identity carries the pick only when there is one. Each was mutation-probed.test/test_per_chat_backend.py(channel chats): a channel-linked chat refuses a pick withchannel_backend_unsupportedbut can clear one, and a pick stored before the chat was linked to a thread is ignored by selection (effective_backend_pick), so the composer never hands the shared channel session a backend its channel replies do not use, its slot row reportsacp_backend_channel_bound(a cron link does not count), and the agent verb refuses a linked target;website/src/test/acpBackend.modelsBackendFor.test.ts(backendPickable): the picker is hidden with no chat, on a peer-bound chat and on a channel-linked one. Mutation-probed.website/src/test/acpBackend.modelsBackendFor.test.ts,BackendPicker.test.tsx,ChatPane.backendModelsError.test.tsx: a degraded pick keys the model list, and its error notice, on Kiro in both composers. Mutation-probed.website/src/test/shellKeyboard.modelsBackend.test.ts: the model-cycle shortcuts read the same per-pick list key as the picker, so a degraded pick cycles Kiro's list instead of finding nothing. Mutation-probed.test/test_manual_compact_gate.py: a cold/compacton a chat picked onto KAS under a Kiro default is answered locally (the gate reads the chat's own backend);test/test_chat_runner_coverage.py: the cold/todoscapability gate does the same, so a chat picked onto Claude under a Kiro default is admitted. Mutation-probed.test/test_eager_spawn.py: the resume prefetch's Tool Search replay check reads the chat's pick, so a Kiro pick under a non-Kiro default spawns nothing it would then refuse, and a non-Kiro pick under a Kiro default still prefetches. Mutation-probed.Local gates on this head: black (py310), isort, flake8, whole-tree mypy, tsc, eslint, i18n check/render/lint with
I18N_BASE_REF=origin/main, the analyze build plus bundle-size gate, feature-map, comment-history, harness-parity, loop-bound-locks and agent-SDK boundary scripts, plus the related backend and frontend test files.Manual verification
Ran on a preview gateway built from this branch, with real turns:
claude-opus-5-5, and the pick stayed after the turn and after a page reload.409 turn_in_flight; the same switch succeeded once the turn ended.Screenshots / video
The composer's backend picker, on the default backend:
The picker open, listing only backends that can start on this host:
Locked while a turn runs:
More: model list follows the backend, reply served by the picked backend
With config loaded, the Default row names the configured backend ("Default (Kiro CLI)"):
Every switch says the next message starts a fresh session on the new backend:
Switching a chat that had a pinned model (
claude-sonnet-5.5on KAS) to Claude Code also names the cleared pin:A refused switch shows inline under the composer shelf with the ask-agent hand-off. This frame was rendered by mocking a
409 turn_in_flightanswer fromPOST /api/chat/slots/{slot}/backendin the browser, since a real refusal needs a turn to start inside the switch window; the message and placement come from the shipped picker code:The config read failing (a mocked 500 from
/api/config/kirocrew) names the problem under the composer instead of leaving the Default row unnamed. The other red notices in this frame are the rest of the app reacting to the same mocked failure:A pick the gateway degraded, on a narrow (600px) composer at
389f9617ca. This frame was rendered by mocking the slot row'sacp_backend: "claude"withacp_backend_degraded: truein the browser, because narrowing the selectable set needs a governance ceiling the pod does not have; the label and icon come from the shipped picker code. The chip's label is truncated to "Claude Code (…", and the warning sign in front of it carries the full "Claude Code (unavailable, using Kiro CLI)" as its accessible name and tooltip:The same degraded pick at
07242f6dbb, with the model menu open. Rendered the same way (the slot row mocked asacp_backend: "claude"withacp_backend_degraded: true; nothing else mocked). The menu lists Kiro's models with their credit multipliers, because the chat now runs on Kiro, and no "Couldn't load the models" notice appears; before this head the composer asked for Claude Code's list, got a permanent error and kept retrying it:A chat linked to a Slack thread, at
a3c194a094. Rendered by mocking the slot row as Slack-linked (linked_session_key: "slack:…",acp_backend_channel_bound: true) in the browser. The composer row keeps its agent, workspace and model chips and offers no backend picker, since the thread's own replies run on the configured backend:The backend probe failing (a mocked 500 from
/api/acp-backends) at389f9617ca: the picker is hidden and "Couldn't load the AI backends" sits on its own row under the composer shelf, with the ask-agent hand-off:A chat picked onto a backend whose own model list fails to load, at
9c5d75adc5. Rendered by mocking the slot row'sacp_backend: "claude"and a 503 from/api/models?backend=claudein the browser. The model chip falls back toauto, and "Couldn't load the models for this backend" sits on the picker's error row under the composer shelf, with the ask-agent hand-off; the list keeps retrying in the background. A split-view pane shows the same message in its own notice stack, without the hand-off because the pane holds an unsent draft:The same failure in a split-view pane (left), at
9c5d75adc5, with the same mocks: the notice sits in the pane's own notice stack above its composer, with no ask-agent hand-off because the pane holds an unsent draft:Related Issues
Part of #13892 (supersedes #11863).
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement