Repository navigation
feat(chat): pick a backend per chat - #19599
billygerhard wants to merge 1 commit into
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: the per-chat pick is opt-in, with no pick every chat behaves as on main, and nothing in the repo already lets you set a backend per chat. What this change shipsInventory (9 items) — 9 justifiedIntent: let an agent move a chat it created to another backend without losing the transcript, and keep that choice across restarts. This is an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] f761870 Description read[DESCRIPTION-READ] e5c9428df1d3c95009233086f2f0d189e654bf9449a29b6bc57b2807d3e01dca That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the pick is a new, defaulted slot field, the swap reuses the queued-reset and sub-agent backend paths, and the creator fence keeps agents off chats a person opened. Notes
[DESIGN-REVIEWED] 86115ad Description read[DESCRIPTION-READ] 144444c3d364d4c977faa528f35bd5e2b3a5154741dd6b0e24ed5170c94d3ca3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — 🔴 changes requested (blocking)Reviewed Adjudication upheld 2 of 2 blocking finding(s); those must be resolved before merging. BLOCKING -- src/kiro_crew/subagent_manager/backend_inheritance.py:25 -- Queued runs lose the parent's backend after a restart
BLOCKING -- src/kiro_crew/dashboard/chat_turn/slash_commands.py:688 -- Compact refuses a backend that the pending reset cannot activate
[BLOCK-MERGE] 86115ad Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 — MEDIUM, and the wrong backend is silent. F2 — LOW, a visible and wrong |
4a7fb63 to
ab3e71d
Compare
ab3e71d to
66e1412
Compare
66e1412 to
2352ca2
Compare
2352ca2 to
e63a8b2
Compare
e63a8b2 to
6850221
Compare
6850221 to
020e9e9
Compare
|
HOLD at 7068483. This PR lets an agent move one dashboard chat to another backend. It adds What blocks approval:
This clears when every AI item has a disposition and a maintainer has reviewed the authorization and tool-posture changes. CI on this head is green. I read the tests but did not run them. — crew-pr-reviewer (r100) |
|
Holding for a human maintainer. I found no definite defect, but this PR adds a cross-session control verb and changes tool posture. Reviewed head:
What would clear the hold:
— crew-pr-reviewer (r103) |
|
Hold at This PR adds a cross-session control verb, Still unanswered on this head:
This clears when each lane ask has a — crew-pr-reviewer (r107) |
|
Holding reviewed head 875bd71 for a human maintainer. I found no definite defect in what I read.
The description now covers the Design question from r107 about the refused-model message, and Phase 2 is tracked on #13892. This clears with a rebase, a — crew-pr-reviewer (r108) |
|
HOLD at 9e23697 for human review (deep check). This head is a rebase of 875bd71 with no source change: a per-file line comparison of the two diffs differs only in two doc context lines and a test hunk that moved with main's test split. The PR adds a cross-session control verb, session_set_backend, and a new internal route. It also derives the side-panel and thread tool posture from the picked backend (side.py#L468). I found no defect: |
|
HOLD at 265a213 for a human maintainer. This PR adds a cross-session control verb, Current-head AI items with no reply:
This clears when each item has a fix or disposition and a maintainer reviews the authorization and tool-posture changes. A fix needs a fresh review. No PR code was run locally. — crew-pr-reviewer (r116) |
Security Scope Review (fork) — ✅ PASSWhich legitimate operations does adjudicated, zero confirmed regressions Scope-Verdict: PASS Reason: This PR adds a feature: a per-chat backend pick and a new TIGHTENING: the channel-agent blocklist and new model-rejection re-checks on model-switch paths -- they refuse a channel agent (or an unauthorized caller) changing which backend another session's next turn runs on, and they stop a model id from being stored for a backend that cannot run it. The blocklist hunk is ADJUDICATED: 12 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: Candidates cover posix, windows and any. The diff makes no new shell assumptions about paths, separators or interpreters; its only platform-sensitive surfaces are which harness executables exist and where chat history lives. The posix rows cover these: UNADJUDICATED:
[SCOPE-REVIEWED] 9597d76Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. |
|
HOLD on b67c3e5. The Opus item from our r116 hold is resolved: What still blocks, and what clears each:
No PR code was run locally. Reviewed head: — crew-pr-reviewer (r123) |
|
Rebased onto main as 3bb5ce3. On the hard-Stop respawn: the respawn-identity change is removed, so a hard-stopped session (including a |
|
Hold at 8dde2e5 for maintainer review. Two of our r123 points are cleared. The branch now merges cleanly with main, and the hard-Stop respawn change is gone, so a stopped What still blocks, and what clears each:
I read the range-diff and CI and did not run the tests. — crew-pr-reviewer (r127) |
|
Disposition for First Principles item 8 (turn-start reset in The turn-start check stays because it is how a backend pick is applied at all, not a patch for the old respawn change:
Chats with no pick, whose session was not started on one, return early and behave exactly as on main. Scoping the respawn identity to dashboard chat keys would bring back the respawn change that review asked to remove, and the check would still be needed for the no-session and cleared-pick cases, so I kept the smaller shape. |
|
@iamwhatever item 2 of the r127 hold is answered: billygerhard posted the First Principles item 8 disposition on 8dde2e5 at 11:50 PDT, about 40 minutes after the hold. The turn-start check is now the mechanism that applies a backend pick at all, including the re-apply after a hard Stop, and not compensation for the removed respawn change. Every check is green on this head. That leaves item 1, the human review by a maintainer and the RFC owner: the cross-session |
|
HOLD at f3e3553, for maintainer review. This head is a rebase of 8dde2e5. The patch is unchanged apart from two doc table rows that moved with main. It merges cleanly, and every check is green. Item 2 of our r127 hold is answered: the First Principles item 8 disposition explains that the turn-start reset is how a pick is applied at all, including after a hard Stop respawns the session on the gateway default. All AI lanes pass with no open findings. What still blocks is the human review that policy requires. The PR adds the cross-session — crew-pr-reviewer (r128) |
CrysisDeu
left a comment
There was a problem hiding this comment.
Maintainer review, comment (not approving yet) at fd8392cd. Reviewed at f3e35538; fd8392cd is a rebase of it (only change: the black-baseline edit dropped, MCP table digest bumped, one pytest marker), so this applies as-is.
Design call for the per-chat / per-member work, so the three PRs stop rebasing in parallel. This PR is the per-CHAT tier and lands first. The per-MEMBER tier is #25485 (kyleseaman's re-port of #11324, scoped to crewmates); #11324 and #16200 close once it is up. Rules both tiers follow: precedence spawn_run(backend=) > chat pick > member pin > member route > default, through ONE resolver (your chat_runner.slot_backend and #25485's crew-aware helper must compose, not duplicate); blank value follows main (#20359: "" / absent = no choice, "kiro" = Kiro), which this PR already does; a pinned backend that is missing or later disallowed fails closed with a visible in-chat notice, never a silent fall-through; crewmate DMs stay refused here (as you have it). Thank you for adopting main's spelling, that saved a round.
Two things to fix before I approve:
- MAJOR
src/kiro_crew/dashboard/chat_handlers.py:1300,members_parts/member_sessions.py:142: a crewmate worker is refused only whenis_crewmate_worker_session(key)is true, and that set lives in memory, filled bynote_worker_sessionwhen the execution record is read. After a gateway restart a worker slot not yet re-read is not in the set; its conductor (creator, so the fence passes) can set a pick, and becausebackend_overridesits above the member/worker route inselect_provider_backend, the next turn starts the worker on a backend outsideACP_BACKENDS_MATE_WORKER_KITand fails withcapability_harness_unsupported. Decide from the durable execution record (or the slot's crew identity), not the in-memory set, and add a restart test. - MAJOR (design)
session_control_parts/turn_control.py:620-700,chat_handlers.py:1348-1361: an unfenced caller, which is the owner's own agent in any other dashboard chat, can move any other chat to a different provider; the next turn replays the whole transcript there, with no human confirmation, no visible backend indicator and no UI to undo until Phase 2. Transcript egress to another provider is a human decision. Apply the creator fence to every caller (an agent may re-pick only sessions it created), and leave moving someone else's chat to the Phase 2 UI where a person clicks.
Also: chat_turn/turn_admission.py:890: when governance later disallows the pick the turn silently runs on Kiro with Kiro's default model, log line only; append a one-time in-chat notice or refuse (rule above). subagent_manager/run_start.py:86: a dedicated child (any model, allowed_tools, bare, AWS profile) gets the gateway default while a shared child inherits the pick; default the dedicated child's backend_override to the parent's pick. handlers/side.py:495, chat_threads.py:956: ... if picked else cfg.agent.acp_backend bypasses the member route; always call slot_backend. Add set_backend to the session-control metric op enum. Nits: English-only switch notice (matches existing), drive-by reformat is now gone.
Checked and fine: apps cannot reach the route (_require_internal), channel agents blocked at the tool layer, fenced callers limited to creator, unknown/uninstalled backend refused up front, hand-edited metadata still gated at allocation. CI green, no BLOCK issued so nothing overridden.
Posted by code-reviewer (Kiro Crew member) on behalf of @CrysisDeu. Full report on file.
|
Thanks @CrysisDeu. All of it is addressed in
Also:
|
|
Follow-up in |
A dashboard chat can carry an optional backend pick, set through the session_set_backend MCP tool (POST /api/session-control/set-backend). The pick is validated by the resolver spawn_run(backend=...) uses, persisted on the slot, and passed to session allocation as backend_override; a change ends the current session so the next turn starts on the new backend. An unpicked chat allocates exactly as before. Implements docs/request-for-change/rfc-chat-backend-selection.md Phase 1.
|
Closing in favour of a fresh, smaller PR: per-session backend selection at session creation (session_create / work_dispatch), built on current main and the single resolver from #25485. Thanks @billygerhard for the groundwork here; the new PR credits you as co-author where it reuses this design. |
Problem / Motivation
Goal: A dashboard chat can be switched to a different backend (for example Claude instead of Kiro) by an agent tool, and keeps that choice across restarts. The dashboard dropdown that lets a person do the same is Phase 2.
Today every dashboard chat runs on the gateway's one global backend. Sub-agents can already pick a backend per run (#18150); a chat cannot. This is Phase 1 of the design in rfc-chat-backend-selection (plan: #13892).
Why it matters
Users who run more than one backend want to choose per conversation: a coding chat on Claude, a cheap triage chat on Kiro. A person gets that choice with the Phase 2 dropdown. This PR serves the agent's half of the job: an agent that opens worker chats with
session_create(a conductor fanning work out, or a cron job) can move one of those chats to another backend when the first backend hits its quota or cannot run the task, without closing it and losing its transcript. The tool goes through the existing session-control gate and its creator fence, so an agent can switch only the sessions it created, never a chat a person opened; switching a person's own chat waits for the dropdown, where the person clicks.Not a goal
session_set_backend("")from another session (a chat's own agent cannot switch the chat it is running in), so until the dropdown ships, undoing a switch takes a second session. This PR adds no field nothing renders; the slot row gainsbackendwith the dropdown that reads it. Until the dropdown, the Side Chat tools indicator (GET /api/chat/side/tools) and the composer's model list (GET /api/models) still describe the gateway backend; the dashboard model routes andsession_set_modelcheck a model against the chat's own backend. Picking a listed gateway model in a chat moved to Claude is refused with the existing... the claude backend does not accept; select a listed model or 'auto'.and the chat keeps its model.backendspelling (governance name, or""for the default), so a saved pick reads back unchanged.docs/request-for-change/rfc-chat-backend-selection.md§2), which said the opposite, and adds no replay cutoff. §4 now names this PR's route.spawn_run(backend=...)does today, even when the pick names the gateway's own backend.What changed (motivation → approach → change)
#18150 already made a session start on a requested backend and keep it through reset and compaction. A chat only lacked a place to store a pick and a way to set it, so this PR adds exactly those and reuses the sub-agent path for everything else. The dashboard route the dropdown will call ships with the dropdown in Phase 2.
_ChatSlot.backend, persisted with the slot's metadata. It stores a backend name such askiroorclaude;""means no pick (gateway default).chat_handlers.switch_slot_backend, the one switch. It validates the pick with the same resolver and refusal codes asspawn_run(backend=...)and takes the model switch's locks. The swap happens between turns: the switch only saves the pick and queues the reset a model pick queues, and the next turn start resets the session onto the pick before acquiring it, so a turn in flight is never cut and finishes on the old backend. That turn start also resets a picked chat's live session that runs another backend (a hard Stop's respawn that started before the pick). A backend change resets the chat's model, and any pending model pick, to the new backend's default (as a gateway provider change does); the new session replays the conversation. A crewmate direct-message session is refused (member_session): it followsagent.member_acp_backend. The caller's authorization is re-run after every await, before anything changes.POST /api/session-control/set-backend, body{"target", "backend"}: the agent's route.session_control.set_backend_target(owned bysession_control_parts/turn_control.py, besidereload_target) runsauthorize_targetbefore the switch and again after each lock await, like every other session-control verb, so the session-control switch, the self/channel-linked/mirrored refusals and the workspace boundary apply, every caller is held to the creator fence, and each refusal is recorded in the SEL. The switch notice names the caller. Registered as an internal-secret route.backend_overrideonly when the slot has a pick, so an unpicked chat allocates with exactly today's arguments. Reset and compaction successors keep the pick throughallocation_identity. A hard Stop respawns on the gateway default, as on main; the next turn start then resets a picked chat onto its pick.chat_runner.slot_backend) where it resolves a slot's backend:session_set_model's model check, the dashboard's model routes (a single chat is judged on its pick; the all-chats route judges unpinned chats together and each chat again under its locks on its current pick, leaving one its backend cannot run unchanged and reporting it underunchanged), the selection-capabilities read, the slash-command and/compactchecks, the app-agent host check, the inherited-model probe, and the eager spawn's resume decision. Thread replies and side-panel turns also start on the pick, and their read-only tool posture is decided by the picked backend through the same helper, so a Kiro gateway's read-only allowance never reaches a chat picked onto Claude. A retained side session is rebound when the pick changes. A backend change also counts as a binding change for an eager handshake already in flight, so its session is torn down instead of kept.spawn_run(backend=...)still wins, and a crewmate's worker keeps the backend its kit needs. The run records that inherited pick (inherited_backend, persisted with the run's state) apart from its ownbackend, so aspawn_continueresumes on the backend the conversation started on even after the parent switches. A continuation never takes the parent's current pick.session_set_backend, besidesession_set_model, posting to the session-control route, and on the channel-agent block list with its siblings.Backwards compatibility
Additive. A chat with no stored pick behaves exactly as before: the metadata field defaults to
""and the allocation keywords are unchanged when it is empty. Older slot metadata without the field decodes to no pick.The all-chats model route (
POST /api/chat/slots/model) keeps its response shape; a pinned chat its backend cannot run is reported underunchanged. The eager-spawn binding tuple (_slot_binding, internal) gains the pick. Removes nothing else: no existing key, route, validator or record is removed or tightened.Tests
test/test_chat_slot_backend.py(new). Switch: a pick is saved and a reset is queued for the next turn, with nothing torn down now, and the replay carries the pre-switch conversation (a no-op switch posts no notice); a turn in flight is left to finish on the old backend; an empty pick returns to the default; re-picking the same backend changes nothing; pinning the backend the chat already runs on keeps its model and pending pick; a backend that cannot start is refused with the shared refusal codes; a switch clears the previous backend's model and drops a pending model pick made for it; a crewmate DM session is refused; a chat with no live session keeps the pick; a caller gate that turns false after the probe changes nothing. Turn start: a picked chat whose live session runs another backend is reset (including a switch made while a hard Stop's respawn was starting), one already on its pick is kept, and an unpicked chat is left alone. Allocation: an unpicked chat adds no allocation keyword; a picked chat's real turn (throughturn_harness.run_turn) allocates its session withbackend_override; the pick round-trips through the slot metadata line. Agent verb: it switches a reachable session, records an allowed SEL event and posts one notice naming the backend and the caller (a refused switch or a re-pick of the current backend posts none); it refuses the caller itself (self_target), a fenced caller's session it did not create (not_creator), and any call with session control off; the switch's own refusal is returned and audited; the tool posts to/api/session-control/set-backendwith the caller's verified key, and accepts""to return to the default.test/test_session_control_set_model.py: a model pick is checked against the chat's own backend. A cold picked chat reports its own backend for selection capabilities and slash commands. A fork keeps the parent's pick. The dashboard's single-chat model route checks a model against the chat's own pick; the all-chats route leaves a pinned chat its backend cannot run unchanged and reports it underunchanged, while the unpinned chats still switch. A chat unpinned while an earlier chat's reset awaits is still checked on its new backend.test/test_config_live.py: a gateway provider switch keeps a picked chat's model. A turn on a pick governance has since disallowed still starts, through the factory's existing fallback to Kiro, and sends no model; an unpicked, a"kiro"and an allowed pick still send theirs; an unpicked, a selectable and a"kiro"pick are left alone. A hard Stop's respawn that registers on the old backend after the turn-start check, or keeps a cleared pick, is reset and reclaimed on the pick. The switch notice loads a lazy transcript off the loop before appending.test/test_chat_threads.pyandtest/test_side.py: a picked chat's thread reply and side turn allocate withbackend_overrideand take their tool posture from the pick (a Claude-picked chat on a Kiro gateway getsREJECT_ALL); a Claude-to-Codex switch rebinds the retained side session. A switch landing during a thread turn does not change the backend that turn allocates on. A pick governance has since disallowed decides no posture (no Claude read-only shape on a session that degrades).Manual verification
N/A — no UI in this PR; the route and tool are covered end to end by the unit tests above. The dropdown PR will carry a live check with screenshots.
Related Issues
Part of #13892. Implements Phase 1 of the design in rfc-chat-backend-selection. Supersedes #15978.
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.