Repository navigation
Conversation
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The kiro-cli spelling correction now also stops for crews that pin nothing, which falls outside the Goal; the description lists it under "Not a goal" while the diff ships it. Not justified as shipped
What this change shipsInventory (8 items) — 6 justifiedIntent: let a single crewmate run on its own ACP harness without changing the install-wide one. This is an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 45a3126 Description read[DESCRIPTION-READ] b2dcd6a9373f0bbbe5347f309528ad0015c975cfe7f4bca0f5df25235e235c34 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS There is one selection gate, but three separate ways of working out which crew a session runs as, so the readouts can drift from the harness the factory actually picks. Watch
[DESIGN-REVIEWED] 45a3126 Description read[DESCRIPTION-READ] aea448e15fd4f5b2659541fec984b48ec8410c21de887290ec40f75b8e3ae3e6 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings that block this merge. FINDING — src/kiro_crew/session_allocation.py:2418 — the new [OPUS-REVIEWED] 45a3126 |
|
Read-only review, holding for a product decision. The pin itself is safe: resolve_selected_backend only accepts registered backend IDs, so it cannot name an arbitrary binary. But #12451 is still needs-human with no maintainer acceptance, and #11324 claims the same agents..acp_backend key with opposite blank-value rules, so maintainers should pick one design first. Before merge: (1) _crew_backend_pin_changed never fires on the retry path because _retry_stale passes the same config as old and new, so a deferred pin edit is lost until restart; add a changed-paths fallback and a test. (2) _pin_entitlement_backend still checks a pinned crew's model against the member/default backend. (3) One live session on a pinned claude crew, or note the gap. |
|
Thanks for the review. #16200 and #11324 now agree on one design: the same key, Asks (1) and (2) are in commit e42d27d:
(3) Live proof on the author's desktop: a preview gateway built from this branch ran two crews, one with Two "Not a goal" lines in the description changed (the model-pin check and pinning kiro-cli), because asks (1)/(2) and the #11324 alignment moved them into scope. |
4b4d327 to
047a253
Compare
The crew record's acp_backend used "" for "pins nothing", but "" is also kiro-cli's own backend id, so a crew could not stay on kiro-cli under a non-kiro member route or default. The field now has the blank rule kirodotdev#11324 gives the same key: null inherits the session default; an empty string selects Kiro explicitly. The selection gate tests the pin with `is not None`; every reader (factory, warm pool, member operating-mode block, dashboard readouts) follows. A full-document save writes null for every crew that pins nothing, so saving pins no crew to kiro-cli, and the config view no longer masks a null pin. A pin edit whose first delivery failed or was deferred was lost until restart: the watcher redelivers it with an `old` that already carries the edit (a retry passes the same snapshot as old and new, a fold-in passes the snapshot that failed), so _crew_backend_pin_changed's value comparison saw no change. It now matches the changed agents.<name>.acp_backend path, like every other factory input in that applier, so a redelivery refreshes even after a refresh that failed once it had adopted the config. A delivery that names only a record or the section is compared against the snapshot the manager has in force. The rule covers both per-crew factory inputs, acp_backend and reasoning_effort, so an effort edit no longer leaves the factory stale either. Adding or removing a crew now costs one warm-pool drain. The warm-pool probe answered "no pin" when it could not read the config or read a torn one, which served a pinned crew from a pool child on the default backend while the factory still held the pin. It now cold-starts in both cases; a wrong answer there only costs a cold start. The pin is kept raw at load and the config view already masks it, but the gate logged it raw, and so did the resolver's refusal warning. The gate and the member operating-mode check now resolve and log a redacted copy; a registered backend id has no credential shape, so no selection changes. A refused crew pin's warning names agents.<name>.acp_backend, not the global agent.acp_backend, so it points at the key to fix; a refused member route names agent.member_acp_backend, and every refusal says it runs on kiro-cli instead. Pooled children were built on the pool agent's crew pin when the pool agent is a pinned crew, and a no-crew session claiming one ran on that harness. The fill now passes pooled=True: the child is still built as that crew (effort, watchdog windows) but on the routed default backend. set_model re-checks the target's session key, agent, agent kind and project before storing a pick: the crew pin it validated against keys on them, and an agent switch could land during its awaits. The member operating-mode check derives the crew from the execution record's member_id, as the factory's readers do (a template selection runs as no crew), so a re-keyed member's stale desk label cannot select the wrong engine's answer. The crews API judged a pinned crew's model against the member route's or the default's catalog, so a crew pinned to claude could not save a claude model. _pin_entitlement_backend now asks the selection gate for a pinned crew's backend; an unpinned crew is judged as before. The kiro-cli spelling correction applies only when that backend reads kiro-cli's model namespace, so a claude-pinned crew is not told that claude-opus-4-8 is "not a model kiro-cli serves". The pool-fill structural test inspects _fill_warm_pool_loop, where main moved the fill.
047a253 to
45a3126
Compare
|
[operator: bolichen97] Maintainer ruling: #11324 is the one path for the per-member harness pin (#12451), so this PR is superseded. Options were (a) keep #11324, (b) keep this PR, or (c) merge parts of both. Both add Next: thank you for the work here, @ash663. This PR stays open until #11324 lands; then a maintainer closes it as superseded. Nothing to change on this branch. |
|
I filed #21910 asking for exactly this (a crewmate-pinned harness that holds across every entry path), and this PR already implements the mechanism I was after — Offer: I run a setup of 11 crewmates across mixed harnesses (some always Claude Code, some always Codex). If this gets a rebase onto One design question worth settling before merge (raised in #21910): when a pinned harness is unavailable/denied, this PR degrades to kiro-cli with a warning. My ask in the issue was the opposite — fail closed with a clear message rather than silently moving the work onto a harness/licence that wasn't chosen for it, since that silent provider-swap is the exact outcome the pin exists to prevent. Not a blocker to validating the happy path, but it's the one behavioural fork a maintainer should rule on. Happy to help get this over the line. |
CrysisDeu
left a comment
There was a problem hiding this comment.
Maintainer comment at 45a31266: closing this in favour of #25485.
Ashish, this was the cleanest backend half of the three and it is the one the re-port borrows from. The 10-16 ruling made #11324 the per-member path and this PR superseded; that path is now #25485 (kyleseaman's re-port onto current main). This branch is 3,887 commits behind with 19 conflicting files in modules main has since split, its blank rule ("" = kiro-cli) disagrees with main's #20359 vocabulary, and a pinned backend that is missing degrades silently to kiro-cli, which the maintainer rule now forbids. So it closes rather than rebases.
Four pieces of yours are to be carried into #25485, named there: the changed-paths hot-reload rule with its ConfigWatch tests, the select_provider_backend( call-site guard, withholding acp_backend from GET /api/agents, and the warm-pool "unreadable config -> cold start" rule. Please review #25485 for those.
On your codex use case: mates can pin only harnesses in ACP_BACKENDS_MATE_WORKER_KIT (today kas, claude), because the worker's per-session MCP kit has to cross the wire and we only have live captures for those two. codex is not excluded by design; it joins the moment a test/fixtures/acp_frames/codex/worker-kit-live.jsonl capture lands, which is a small standalone PR if you want it. kiro-cli cannot join (reads --agent off disk, no per-session channel). Until then a codex reviewer runs as spawn_run(backend=codex) or a per-chat pick (#19599). The agent.sandbox per-backend question is a separate issue; please file it.
Posted by code-reviewer (Kiro Crew member) on behalf of @CrysisDeu. Full report on file.
|
Thanks for this work, and for aligning the key and blank rule with #11324. The maintainer ruling on 2026-10-16 picked #11324 as the single path for the per-member harness pin. This PR now overlaps it, conflicts with main, and has had no new commits since 2026-10-04. I propose closing this PR in favor of #11324. If any piece here is not covered there (for example the entry-path coverage raised in #21910), a small follow-up PR on top of #11324 would be the easiest way to land it. I will leave the close to you or a maintainer. |
|
[operator: bolichen97] Correction to my 2026-10-30 note: the per-mate harness pin landed in #25485 (merged 2026-10-30), not #11324, which is still open. This PR is superseded by #25485, per CrysisDeu's 2026-10-29 review. @ash663, could you check that the four pieces CrysisDeu named made it into #25485: the changed-paths hot-reload with its ConfigWatch tests, the AI-assisted, approved by bolichen97 |
|
@bolichen97 @CrysisDeu I checked the four pieces against the #25485 merge commit (ffff62c; the links pin it) and current main (95abd91). Three are in; hot reload is partial.
The linked code is unchanged on current main, where I can open a small follow-up PR on current main for piece 1: port #16200's |
|
@ash663 thanks for checking #25485 against this PR. Under the team SOP, this is a T3 (it changes how crew members pick a backend, see #12451). A T3 needs an Issue and a 1-2 page RFC/design doc under Most of this PR already landed in #25485. For the part that is still missing (hot reload is partial, per your note), please open an Issue and an RFC for it. Once they are open, Crew Captain will triage it, and the team will look at it by priority. AI-assisted, sent by CommunityCrew |
Problem / Motivation
Goal: Let a Crew Member name the ACP harness its own sessions run on, the way it already names its model and reasoning effort.
One gate,
members.select_provider_backend, picks each session's harness from two install-wide settings:agent.member_acp_backendfor every member DM thread andagent.acp_backendfor everything else. A crew record can pinmodelandreasoning_effortbut not the harness, so two members cannot run on different harnesses, and a member's scheduled and channel turns always run on the install default.Why it matters
Harnesses differ in model catalog, cost and capability (member dispatch, MCP transport, resume). Today, choosing one for a single crewmate means switching it for every session on the install. A per-crew pin puts each crewmate on the harness that suits its work and leaves everything else on the defaults.
Not a goal
config.json(kirocrew config set agents.<name>.acp_backend <id>); a crew-editor field is a follow-up.deepseekstays outsideACP_BACKENDS_MEMBER_DISPATCH, so a member pinned to it runs as plain chat without dispatch tools, asagent.member_acp_backend: deepseekdoes today.ACP_BACKENDS_KNOWN, backend registration or routing, or to subagents that share their parent's runtime and so its harness.modelpin. A crew that pins a backend has itsmodelpin judged against that harness's catalog (_pin_entitlement_backend, through the selection gate). A crew that pins nothing is judged against the same harness as before (the member default, or the install default), and the kiro-cli spelling correction now applies only when that harness reads kiro-cli ids, so an install whose member default is a non-kiro harness no longer sees its own wire ids corrected into refusals.nullinherits the session default and""pins kiro-cli explicitly; no other value is reserved.What changed (motivation → approach → change)
Two commits. The first adds the pin; the second aligns its blank rule with #11324 and addresses the maintainer's asks (1) and (2).
Approach. The pin is a new first input to the existing gate: the crew's pin, then the member-DM route, then the configured default. No second gate (harness-parity H3/H13) and no frozen backend list (H4). The pin goes through
resolve_selected_backend, the gate the persisted field already crosses, so an unknown or policy-denied pin runs the session on kiro-cli with the existing "not selectable in this build" warning. A refused pin does not fall through to a route: that would look like a working pin to anyone checking which harness a thread runs on.One design with #11324. Both PRs now use the same key,
agents.<name>.acp_backend, with the same blank rule and the same schema shape (nullable: true,defaultValue: null): null inherits the session default, an empty string selects kiro-cli explicitly. The label and help text follow #11324's entry, worded to match their siblings ("Crew ACP backend" beside "Crew member ACP backend", "kiro-cli" rather than "Kiro"); whichever PR lands second takes the other's wording. This PR is the backend half only: no dashboard control and noGET /api/agentsfield. #11324's crew-editor selector can sit on top of it unchanged.Where the value lives.
agents.<name>.acp_backendon the crew record inconfig.json, besidemodelandreasoning_effort, rather than in an agent spec or the per-agent model-state sidecar, because:config.json, so no spec carries a key kiro-cli does not know;resolve_crew_identity), so one lookup reaches every surface that starts a session as that crew: its DM thread, a dashboard chat bound to it, cron and channel turns,spawn_run(crew=...);kirocrew config setand hot reload already cover crew records. The sidecar is runtime state with none of that, and a value keyed on the agent template would need a lookup of its own on every surface, whereagentnames a crew on some and a template on others.A session bound to a template runs as no crew and keeps both routes, so a worker dispatched on the
kirocrew-workertemplate does not take its dispatcher's pin; a child started with no agent keeps the member's selection and does. Thekirocrew-workerspec derivation mirrors spec keys only, so it neither copies nor clears a pin.Change, first commit (the pin).
config/sections.py,config/loader.py: the optional field, loaded type-guarded but not normalized, since normalizing would turn a refused pin into a silent kiro-cli pin;crew_acp_backend()besidecrew_pinned_effort().members.select_provider_backend(..., crew_backend=None): the new arm. The provider factory passesself.crew_acp_backend(agent, crew_agent).bypass_backend), because a pooled child was spawned on the default harness;config.jsonedit that changes some crew's pin rebuilds the provider factory (SessionManager._crew_backend_pin_changed; the second commit makes it path-based);set_modelresolve the slot's crew the way the turn path does (chat_utils.session_crew_acp_backend) and pass the pin to the gate;set_modelrefuses withtarget_replaced(409) when the target's agent, its kind, project or session key moved while the model was being checked, since the pin it validated against keys on them;GET /api/config/kirocrewmasks the field with the other agent-writable free-text record fields (it is kept raw at load, so any string can arrive);GET /api/agentswithholds it until a dashboard control reads it.config-baseline.jsonregenerated.Change, second commit (blank rule and asks (1), (2)).
str | None, defaultNone.Nonetakes both existing routes unchanged;""pins kiro-cli over both; any other value goes throughresolve_selected_backend. Every reader testsis not Noneinstead of truthiness: the gate, the factory, the warm-pool probe (a""pin bypasses the pool too, since the pool's default harness need not be kiro-cli), the member operating-mode block, and both dashboard readouts. A non-string value loads asNone, not"", so junk never pins kiro-cli. A full-document save writesnullfor every crew that pins nothing (the first commit wrote"", which under this rule would pin every crew to kiro-cli).GET /api/config/kirocrewpasses anullpin through instead of masking it.pooled=True); pinned crews cold-start, and effort/watchdog inheritance from the pool agent's crew is unchanged. When the probe cannot read the config, or reads a torn one the loader filled with defaults, it now cold-starts instead of serving from the pool. The factory may still hold a pin that read cannot see, and a pool child on the default backend would then take the pinned crew's prompt. A wrong answer there costs only a cold start. The effort twin keeps its "pool as before" answer, since a wrong effort does not change where a prompt goes.redact_external_text), so a credential-shaped value is neither printed by the routing line nor by the resolver's refusal warning. The config view already masks the same field. A registered backend id has no credential shape, so selection is unchanged.resolve_selected_backendtakes an optionalsettingname for its warning, so a refused crew pin saysIgnoring agents.<name>.acp_backend ...(the member-block check names the actual crew) instead of pointing at the globalagent.acp_backend. The default keeps the global and member-route wording unchanged. The schema label is "Crew ACP backend" and its help text says "kiro-cli".oldthat already holds the edit. A retry passes the same snapshot asoldandnew, and a fold-in into the next file change passes the snapshot that failed. The first commit compared values between the two, so both looked unchanged._crew_backend_pin_changednow uses the changed-paths rule asked for: anyagents.<name>.acp_backendleaf among the changed paths refreshes, as every_FACTORY_CONFIG_PATHSprefix already does in that applier. That also covers a refresh that failed afterrefresh_defaultshad adopted the config and factory but before it restarted the pool, where a value comparison against any snapshot sees nothing owed. A delivery naming only a record or the section (a bind-time replay) is compared againstself._cfg, the snapshot in force. The cost: adding or removing a crew moves itsacp_backendleaf and drains the warm pool once. The rule covers both per-crew factory inputs,agents.<name>.acp_backendandagents.<name>.reasoning_effort, so an effort edit invalidates the stale factory the same way._pin_entitlement_backend: for a crew that pins a backend (""included), the crews API asks the selection gate for that crew's harness and judges themodelpin against that harness's catalog, so a crew pinned toclaudecan save a claude model. The kiro-cli spelling correction (which tells the user, for example, thatclaude-opus-4-8is "not a model kiro-cli serves" and names the kiro-cli id) now applies only when the judging harness reads kiro-cli's model namespace: the claude harness servesclaude-opus-4-8as its own wire id. An unpinned crew is judged as before. Create is unchanged: a new crew has no pin yet.Backwards compatibility
Compatible: the field is new and optional. A record without it, or with
null, takes both existing routes unchanged, and nothing main accepts is rejected or renamed.Removes nothing: the change adds a field and a gate input; no path, record, key or validator main has is removed or tightened.
Tests
test/test_crew_acp_backend.py(new):None, or no argument) keeps the member route on a member DM slot and the configured default on a plain slot;""keeps a crew on kiro-cli over a non-kiro member route and over a non-kiro default, in the gate and through the factory;agents.<name>.acp_backendrather than the global key, a credential-shaped pin is refused without its secret reaching the log, and a refused pin does not fall through to the member route;Noneand""stay distinct through a load and through a full-document save, a crew added in memory saves asnull, an unselectable string is kept for the gate to refuse, and non-string values load as inherit;AcpProviderfor a cron turn and a member DM, an unpinned crew keeps both routes, and a template session ignores a same-named crew's pin;""and a pin the gate will refuse) bypasses the pool, and so do an unreadable and a torn config read; no pin does not; a pooled child whose pool agent is a pinned crew is built on the default backend yet still resolves that crew's effort, and the fill marks its children pooled;agentspath; a retry whoseoldisnewsees the pin for a leaf, record or section path; a redelivered pin leaf refreshes even when the pin is already in force, for a retry and a fold-in; a replay of record or section paths for a pin already in force rebuilds nothing; a retry withold is newand areasoning_effortleaf fires, while an unrelated leaf does not;ConfigWatch, a pin edit whose first refresh fails, before or after it adopted the config, is redelivered by the watcher's retry and by folding into the next file change, and each of the four redeliveries rebuilds the factory;""pin withholds the dispatch block, and a credential-shaped pin is not logged).test/test_agent_model_pin_validation.py: ask (2) throughPUT /api/agents/{name}: a crew pinned toclaudesaves a model the claude catalog advertises; the same model on an unpinned crew is still refused by the member route's catalog; a""pin is judged by kiro's catalog even when the member route isclaude; a crew pinned toclaudeis not given the kiro-cli spelling correction forclaude-opus-4-8, while an unpinned crew still is.test/test_config_agent_fields_redaction.py: the free-text redaction ratchet countsstr | Nonerecord fields, so the nullable field stays in the masked set; anullpin is not masked.test/test_session_control_set_model.py: the target slot's pin reaches the gate.test/test_agents_roster_contract.pyclassifies the field as withheld.Every new test was mutation-checked. First commit: 16 mutations, each failing at least one test. Second commit: 30 mutations (dropping the agent kind from
set_model's identity re-check; ignoring a template selection in the member-block crew derivation; deriving the member-block crew from the desk label instead of the record; droppingset_model's identity re-check; labelling a refused member route with the global key; honouring a pin in a pooled child, from the factory or from the pool fill; dropping the pool agent's crew identity from a pooled child; droppingreasoning_effortfrom the path rule; naming the global key in a refused crew pin's warning, from the gate or the member-block check; logging the raw pin in the gate or in the member-block check; pooling, or ignoring a torn read, when the pool probe cannot read the config; dropping the path rule; applying the kiro-cli spelling correction always, or never; truthiness in the gate, the member block, the pool probe and the entitlement check; a""default; junk or null loaded as""; a failed readout answering""; maskingnull; comparing record- or section-level deliveries againstchange.old; dropping the pin from the entitlement check or the crew name from the update path), each failing at least one test. Both new ask tests also fail on the first commit's code.The test files of every module the second commit touches: 8,622 passed. black (baselined gate), isort, flake8, mypy and the subprocess-encoding gate pass.
scripts/run_scoped_tests.py --surface backend, all six batches (3448 related test files): 161,959 passed, 158 failed, 12 errors. Each failing id was checked against this PR's previous head in the same environment: every one also fails there, or passes on both heads when its file runs alone. None fails only on the new head.The branch was then rebased onto main at 59e8b1e as the same two commits. An earlier merge commit was dropped, because PR Hygiene allows at most two commits. The rebased tree is identical to that merge head, 4b4d327. The one conflict was in
config/loader.py: main'sCallable[..., None]gate-callback annotations are taken and thepooledparameter is kept. The pool-fill structural test inspects_fill_warm_pool_loop, the method main moved the fill into. On the rebased head, the touched test files pass (7,381 passed), all 30 mutations are still caught, and lint and mypy pass. The scoped gate counts above are from the pre-rebase head.Manual verification
Live proof on the author's desktop: a preview gateway built from this branch ran two crews, one with
acp_backend: "claude"and a control with no key; the pinned crew's DM loggedrouting to acp_backend='claude' (crew acp_backend='claude')and spawned claude-agent-acp (Claude Code), which answered; the control routedkasviaagent.member_acp_backendon kiro-cli--agent-engine v3with no claude spawn. The run was repeated on the pushed head with the same result.Related Issues
Part of #12451 (per-member backend selection; its MCP half is not addressed here). #11324 implements per-member harness selection with a dashboard selector on the same key; the two PRs now share one design (same key, same blank rule, same nullable schema shape), and this one is the backend-only half.
Follow-ups:
deepseekinACP_BACKENDS_MEMBER_DISPATCH, so a member pinned to it gets dispatch tools.session.pycompareschange.old; the remaining appliers there test paths, which a redelivery carries.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement