Skip to content

feat(crew): let a crew member pin its own ACP backend - #16200

Closed
ash663 wants to merge 2 commits into
kirodotdev:mainfrom
ash663:feat/per-agent-acp-backend
Closed

ash663 wants to merge 2 commits into
kirodotdev:mainfrom
ash663:feat/per-agent-acp-backend

Conversation

@ash663

@ash663 ash663 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Let a Crew Member name the ACP harness its own sessions run on, the way it already names its model and reasoning effort.

One gate, members.select_provider_backend, picks each session's harness from two install-wide settings: agent.member_acp_backend for every member DM thread and agent.acp_backend for everything else. A crew record can pin model and reasoning_effort but not the harness, so two members cannot run on different harnesses, and a member's scheduled and channel turns always run on the install default.

Why it matters

Harnesses differ in model catalog, cost and capability (member dispatch, MCP transport, resume). Today, choosing one for a single crewmate means switching it for every session on the install. A per-crew pin puts each crewmate on the harness that suits its work and leaves everything else on the defaults.

Not a goal

  • A dashboard control. The pin is set in config.json (kirocrew config set agents.<name>.acp_backend <id>); a crew-editor field is a follow-up.
  • Widening a capability set. deepseek stays outside ACP_BACKENDS_MEMBER_DISPATCH, so a member pinned to it runs as plain chat without dispatch tools, as agent.member_acp_backend: deepseek does today.
  • Any change to ACP_BACKENDS_KNOWN, backend registration or routing, or to subagents that share their parent's runtime and so its harness.
  • Changing which harness judges an unpinned crew's model pin. A crew that pins a backend has its model pin judged against that harness's catalog (_pin_entitlement_backend, through the selection gate). A crew that pins nothing is judged against the same harness as before (the member default, or the install default), and the kiro-cli spelling correction now applies only when that harness reads kiro-cli ids, so an install whose member default is a non-kiro harness no longer sees its own wire ids corrected into refusals.
  • Any blank value beyond feat: select a harness and model per crew member #11324's rule. null inherits the session default and "" pins kiro-cli explicitly; no other value is reserved.

What changed (motivation → approach → change)

Two commits. The first adds the pin; the second aligns its blank rule with #11324 and addresses the maintainer's asks (1) and (2).

Approach. The pin is a new first input to the existing gate: the crew's pin, then the member-DM route, then the configured default. No second gate (harness-parity H3/H13) and no frozen backend list (H4). The pin goes through resolve_selected_backend, the gate the persisted field already crosses, so an unknown or policy-denied pin runs the session on kiro-cli with the existing "not selectable in this build" warning. A refused pin does not fall through to a route: that would look like a working pin to anyone checking which harness a thread runs on.

One design with #11324. Both PRs now use the same key, agents.<name>.acp_backend, with the same blank rule and the same schema shape (nullable: true, defaultValue: null): null inherits the session default, an empty string selects kiro-cli explicitly. The label and help text follow #11324's entry, worded to match their siblings ("Crew ACP backend" beside "Crew member ACP backend", "kiro-cli" rather than "Kiro"); whichever PR lands second takes the other's wording. This PR is the backend half only: no dashboard control and no GET /api/agents field. #11324's crew-editor selector can sit on top of it unchanged.

Where the value lives. agents.<name>.acp_backend on the crew record in config.json, beside model and reasoning_effort, rather than in an agent spec or the per-agent model-state sidecar, because:

  • kiro-cli never reads config.json, so no spec carries a key kiro-cli does not know;
  • it is keyed on the crew identity the factory already resolves for the effort pin (resolve_crew_identity), so one lookup reaches every surface that starts a session as that crew: its DM thread, a dashboard chat bound to it, cron and channel turns, spawn_run(crew=...);
  • the loader, schema, kirocrew config set and hot reload already cover crew records. The sidecar is runtime state with none of that, and a value keyed on the agent template would need a lookup of its own on every surface, where agent names a crew on some and a template on others.

A session bound to a template runs as no crew and keeps both routes, so a worker dispatched on the kirocrew-worker template does not take its dispatcher's pin; a child started with no agent keeps the member's selection and does. The kirocrew-worker spec derivation mirrors spec keys only, so it neither copies nor clears a pin.

Change, first commit (the pin).

  • config/sections.py, config/loader.py: the optional field, loaded type-guarded but not normalized, since normalizing would turn a refused pin into a silent kiro-cli pin; crew_acp_backend() beside crew_pinned_effort().
  • members.select_provider_backend(..., crew_backend=None): the new arm. The provider factory passes self.crew_acp_backend(agent, crew_agent).
  • The places that must agree with the factory about which harness a session gets:
    • warm pool: a pinned crew bypasses it (bypass_backend), because a pooled child was spawned on the default harness;
    • hot reload: a config.json edit that changes some crew's pin rebuilds the provider factory (SessionManager._crew_backend_pin_changed; the second commit makes it path-based);
    • dashboard: cold-slot capabilities and set_model resolve the slot's crew the way the turn path does (chat_utils.session_crew_acp_backend) and pass the pin to the gate; set_model refuses with target_replaced (409) when the target's agent, its kind, project or session key moved while the model was being checked, since the pin it validated against keys on them;
    • member operating-mode block: checks the pinned harness's dispatch capability, so a member pinned to a harness that cannot mount the dispatch tools is not taught them.
  • Read surfaces: GET /api/config/kirocrew masks the field with the other agent-writable free-text record fields (it is kept raw at load, so any string can arrive); GET /api/agents withholds it until a dashboard control reads it.
  • Docs: configuration, crew members, agent spec fields, the custom-backend guide, and the crew-mode, session, session-control and ACP-client specs. config-baseline.json regenerated.

Change, second commit (blank rule and asks (1), (2)).

  • Blank rule: the field is str | None, default None. None takes both existing routes unchanged; "" pins kiro-cli over both; any other value goes through resolve_selected_backend. Every reader tests is not None instead of truthiness: the gate, the factory, the warm-pool probe (a "" pin bypasses the pool too, since the pool's default harness need not be kiro-cli), the member operating-mode block, and both dashboard readouts. A non-string value loads as None, not "", so junk never pins kiro-cli. A full-document save writes null for every crew that pins nothing (the first commit wrote "", which under this rule would pin every crew to kiro-cli). GET /api/config/kirocrew passes a null pin through instead of masking it.
  • Warm pool: pooled children are built on the install default harness and never carry a crew's backend pin (the fill passes pooled=True); pinned crews cold-start, and effort/watchdog inheritance from the pool agent's crew is unchanged. When the probe cannot read the config, or reads a torn one the loader filled with defaults, it now cold-starts instead of serving from the pool. The factory may still hold a pin that read cannot see, and a pool child on the default backend would then take the pinned crew's prompt. A wrong answer there costs only a cold start. The effort twin keeps its "pool as before" answer, since a wrong effort does not change where a prompt goes.
  • Logs: the gate and the member operating-mode check resolve and log a redacted copy of the crew pin (redact_external_text), so a credential-shaped value is neither printed by the routing line nor by the resolver's refusal warning. The config view already masks the same field. A registered backend id has no credential shape, so selection is unchanged.
  • Refusal wording: resolve_selected_backend takes an optional setting name for its warning, so a refused crew pin says Ignoring agents.<name>.acp_backend ... (the member-block check names the actual crew) instead of pointing at the global agent.acp_backend. The default keeps the global and member-route wording unchanged. The schema label is "Crew ACP backend" and its help text says "kiro-cli".
  • Ask (1), deferred pin edit lost until restart: the watcher adopts a document before it dispatches it, so every redelivery of an edit the manager did not finish carries an old that already holds the edit. A retry passes the same snapshot as old and new, and a fold-in into the next file change passes the snapshot that failed. The first commit compared values between the two, so both looked unchanged. _crew_backend_pin_changed now uses the changed-paths rule asked for: any agents.<name>.acp_backend leaf among the changed paths refreshes, as every _FACTORY_CONFIG_PATHS prefix already does in that applier. That also covers a refresh that failed after refresh_defaults had adopted the config and factory but before it restarted the pool, where a value comparison against any snapshot sees nothing owed. A delivery naming only a record or the section (a bind-time replay) is compared against self._cfg, the snapshot in force. The cost: adding or removing a crew moves its acp_backend leaf and drains the warm pool once. The rule covers both per-crew factory inputs, agents.<name>.acp_backend and agents.<name>.reasoning_effort, so an effort edit invalidates the stale factory the same way.
  • Ask (2), _pin_entitlement_backend: for a crew that pins a backend ("" included), the crews API asks the selection gate for that crew's harness and judges the model pin against that harness's catalog, so a crew pinned to claude can save a claude model. The kiro-cli spelling correction (which tells the user, for example, that claude-opus-4-8 is "not a model kiro-cli serves" and names the kiro-cli id) now applies only when the judging harness reads kiro-cli's model namespace: the claude harness serves claude-opus-4-8 as its own wire id. An unpinned crew is judged as before. Create is unchanged: a new crew has no pin yet.
  • Two "Not a goal" bullets were rewritten with the owner's approval: the first commit excluded the model-pin check and a kiro-cli pin, and maintainer asks (1) and (2) plus the feat: select a harness and model per crew member #11324 alignment moved both into scope.

Backwards compatibility

Compatible: the field is new and optional. A record without it, or with null, takes both existing routes unchanged, and nothing main accepts is rejected or renamed.

Removes nothing: the change adds a field and a gate input; no path, record, key or validator main has is removed or tightened.

Tests

test/test_crew_acp_backend.py (new):

  • no pin (None, or no argument) keeps the member route on a member DM slot and the configured default on a plain slot;
  • a pin wins over both routes, and "" keeps a crew on kiro-cli over a non-kiro member route and over a non-kiro default, in the gate and through the factory;
  • an unknown pin and a policy-denied pin degrade to kiro-cli with the resolver's warning, which names agents.<name>.acp_backend rather than the global key, a credential-shaped pin is refused without its secret reaching the log, and a refused pin does not fall through to the member route;
  • loader: None and "" stay distinct through a load and through a full-document save, a crew added in memory saves as null, an unselectable string is kept for the gate to refuse, and non-string values load as inherit;
  • factory: the pin reaches AcpProvider for a cron turn and a member DM, an unpinned crew keeps both routes, and a template session ignores a same-named crew's pin;
  • warm pool: a set pin (including "" and a pin the gate will refuse) bypasses the pool, and so do an unreadable and a torn config read; no pin does not; a pooled child whose pool agent is a pinned crew is built on the default backend yet still resolves that crew's effort, and the fill marks its children pooled;
  • hot reload: the predicate fires on a pin set, cleared, pinned to kiro-cli, and a crew added or removed (pinned or not); it does not fire for an unrelated crew edit or a change with no agents path; a retry whose old is new sees the pin for a leaf, record or section path; a redelivered pin leaf refreshes even when the pin is already in force, for a retry and a fold-in; a replay of record or section paths for a pin already in force rebuilds nothing; a retry with old is new and a reasoning_effort leaf fires, while an unrelated leaf does not;
  • ask (1) end to end: through a real ConfigWatch, a pin edit whose first refresh fails, before or after it adopted the config, is redelivered by the watcher's retry and by folding into the next file change, and each of the four redeliveries rebuilds the factory;
  • the dashboard slot readout and cold-slot capabilities, and the member-block capability check (a "" pin withholds the dispatch block, and a credential-shaped pin is not logged).

test/test_agent_model_pin_validation.py: ask (2) through PUT /api/agents/{name}: a crew pinned to claude saves a model the claude catalog advertises; the same model on an unpinned crew is still refused by the member route's catalog; a "" pin is judged by kiro's catalog even when the member route is claude; a crew pinned to claude is not given the kiro-cli spelling correction for claude-opus-4-8, while an unpinned crew still is.

test/test_config_agent_fields_redaction.py: the free-text redaction ratchet counts str | None record fields, so the nullable field stays in the masked set; a null pin is not masked. test/test_session_control_set_model.py: the target slot's pin reaches the gate. test/test_agents_roster_contract.py classifies the field as withheld.

Every new test was mutation-checked. First commit: 16 mutations, each failing at least one test. Second commit: 30 mutations (dropping the agent kind from set_model's identity re-check; ignoring a template selection in the member-block crew derivation; deriving the member-block crew from the desk label instead of the record; dropping set_model's identity re-check; labelling a refused member route with the global key; honouring a pin in a pooled child, from the factory or from the pool fill; dropping the pool agent's crew identity from a pooled child; dropping reasoning_effort from the path rule; naming the global key in a refused crew pin's warning, from the gate or the member-block check; logging the raw pin in the gate or in the member-block check; pooling, or ignoring a torn read, when the pool probe cannot read the config; dropping the path rule; applying the kiro-cli spelling correction always, or never; truthiness in the gate, the member block, the pool probe and the entitlement check; a "" default; junk or null loaded as ""; a failed readout answering ""; masking null; comparing record- or section-level deliveries against change.old; dropping the pin from the entitlement check or the crew name from the update path), each failing at least one test. Both new ask tests also fail on the first commit's code.

The test files of every module the second commit touches: 8,622 passed. black (baselined gate), isort, flake8, mypy and the subprocess-encoding gate pass.

scripts/run_scoped_tests.py --surface backend, all six batches (3448 related test files): 161,959 passed, 158 failed, 12 errors. Each failing id was checked against this PR's previous head in the same environment: every one also fails there, or passes on both heads when its file runs alone. None fails only on the new head.

The branch was then rebased onto main at 59e8b1e as the same two commits. An earlier merge commit was dropped, because PR Hygiene allows at most two commits. The rebased tree is identical to that merge head, 4b4d327. The one conflict was in config/loader.py: main's Callable[..., None] gate-callback annotations are taken and the pooled parameter is kept. The pool-fill structural test inspects _fill_warm_pool_loop, the method main moved the fill into. On the rebased head, the touched test files pass (7,381 passed), all 30 mutations are still caught, and lint and mypy pass. The scoped gate counts above are from the pre-rebase head.

Manual verification

Live proof on the author's desktop: a preview gateway built from this branch ran two crews, one with acp_backend: "claude" and a control with no key; the pinned crew's DM logged routing to acp_backend='claude' (crew acp_backend='claude') and spawned claude-agent-acp (Claude Code), which answered; the control routed kas via agent.member_acp_backend on kiro-cli --agent-engine v3 with no claude spawn. The run was repeated on the pushed head with the same result.

Related Issues

Part of #12451 (per-member backend selection; its MCP half is not addressed here). #11324 implements per-member harness selection with a dashboard selector on the same key; the two PRs now share one design (same key, same blank rule, same nullable schema shape), and this one is the backend-only half.

Follow-ups:

  • A crew-editor control for the pin (feat: select a harness and model per crew member #11324's selector is one).
  • deepseek in ACP_BACKENDS_MEMBER_DISPATCH, so a member pinned to it gets dispatch tools.
  • No other predicate in session.py compares change.old; the remaining appliers there test paths, which a redelivery carries.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 2, 2026
@ash663
ash663 marked this pull request as ready for review October 2, 2026 17:01
@ash663
ash663 requested a review from a team as a code owner October 2, 2026 17:01
@ash663
ash663 requested a review from bolichen97 October 2, 2026 17:01
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 2, 2026
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of 45a31266195774d1740450bf6978f8ba1562c586 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The kiro-cli spelling correction now also stops for crews that pin nothing, which falls outside the Goal; the description lists it under "Not a goal" while the diff ships it.

Not justified as shipped

  • Item 6 — oversized: the Goal is per-crew pinning, yet _kiro_spelling_correction(model, backend) now uses the routed backend for unpinned crews too. The description says "an install whose member default is a non-kiro harness no longer sees its own wire ids corrected". No issue is linked and no test shows the defect on base.
    Subtraction: skip the correction only when crew_acp_backend(...) is not None. Unpinned crews keep base's unconditional model_registry.acp_id_correction(model) in _revalidate_crew_pin and _model_pin_rejected.
    Clears when: the unpinned arm is reverted to base, or a linked issue or a test that fails on base shows the wrong refusal.

What this change ships

Inventory (8 items) — 6 justified

Intent: let a single crewmate run on its own ACP harness without changing the install-wide one. This is an ADDITION.

  1. New agents.<name>.acp_backend crew key (default null; "" pins kiro-cli) that outranks the member route and the default — justified
  2. A pinned crew skips the warm pool and cold-starts; pooled children never carry a pin — justified
  3. Editing a crew's pin, or its reasoning_effort, now rebuilds the provider factory live — undeclared: the effort half fixes stale effort edits on base
  4. Dashboard capabilities and set_model follow the pin; set_model returns 409 if the target moved mid-check — justified
  5. The member dispatch block follows the pinned harness's capability — justified
  6. Unpinned crews on a non-kiro member/default route lose the kiro spelling correction — oversized
  7. A crew's model pin is checked against its pinned harness's catalog — justified
  8. The refusal log names the offending key and says "running on kiro-cli" — justified

[FIRST-PRINCIPLES-REVIEWED] 45a3126

Description read

[DESCRIPTION-READ] b2dcd6a9373f0bbbe5347f309528ad0015c975cfe7f4bca0f5df25235e235c34

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🟡 CONCERNS

Design-level review of 45a31266195774d1740450bf6978f8ba1562c586 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

There is one selection gate, but three separate ways of working out which crew a session runs as, so the readouts can drift from the harness the factory actually picks.

Watch

  • Crew identity is worked out in three places.
    • Cause: the factory uses resolve_crew_identity. chat_utils.session_crew_acp_backend uses resolve_session_agent_bindings plus "template runs as no crew". context.py builds dispatch_crew from desk_member, then member_config_for_id, with its own template check.
    • Mechanism: each surface passes its own idea of the crew into select_provider_backend. When they disagree on a re-keyed member, a legacy member with no id, or a template whose name matches a crew, the readouts no longer match the factory.
    • Consequence: the operating-mode block teaches session_* tools on a harness that can't mount them. Or the dashboard's capabilities and set_model check a model against a harness the turn never runs on.
    • Fix: have all three readers call one crew_for_session(...) helper that the factory also uses.
    • Clears when: the factory, session_crew_acp_backend and _member_backend_can_dispatch's caller all get the crew from one shared function. A test should check that their answers match for the template, re-keyed and legacy-member cases.
  • The take-away check had to stop at CONCERNS. The description is cut off at 8000 bytes before any ## Backwards compatibility section. I found no broken reader: the new target_replaced 409 reuses an existing code, and the spelling-correction narrowing is stated in "Not a goal".
    • Clears when: the description fits under the cap with its ## Backwards compatibility section intact.

[DESIGN-REVIEWED] 45a3126

Description read

[DESCRIPTION-READ] aea448e15fd4f5b2659541fec984b48ec8410c21de887290ec40f75b8e3ae3e6

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 45a31266195774d1740450bf6978f8ba1562c586 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 45a3126

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 45a31266195774d1740450bf6978f8ba1562c586 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings that block this merge.

FINDING — src/kiro_crew/session_allocation.py:2418 — the new pool_decision = "bypass_backend" label is missing from POOL_DECISIONS in session.py:528, so the kirocrew.session.pool.decision metric records every bypass for a pinned crew as outcome="other" (session_pool.py:544). Those bypasses can't be told apart from unknown outcomes → Fix: add "bypass_backend" to POOL_DECISIONS.

[OPUS-REVIEWED] 45a3126

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 2, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Read-only review, holding for a product decision. The pin itself is safe: resolve_selected_backend only accepts registered backend IDs, so it cannot name an arbitrary binary. But #12451 is still needs-human with no maintainer acceptance, and #11324 claims the same agents..acp_backend key with opposite blank-value rules, so maintainers should pick one design first. Before merge: (1) _crew_backend_pin_changed never fires on the retry path because _retry_stale passes the same config as old and new, so a deferred pin edit is lost until restart; add a changed-paths fallback and a test. (2) _pin_entitlement_backend still checks a pinned crew's model against the member/default backend. (3) One live session on a pinned claude crew, or note the gap.

@ash663

ash663 commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. #16200 and #11324 now agree on one design: the same key, agents.<name>.acp_backend, with #11324's blank rule and nullable schema shape (null inherits the session default, an empty string selects kiro-cli explicitly). The label and help text are worded to match their sibling fields ("Crew ACP backend", "kiro-cli"); whichever PR lands second takes the other's wording. This PR stays backend-only, so #11324's crew-editor selector can sit on top of it.

Asks (1) and (2) are in commit e42d27d:

  1. A deferred or failed pin edit now rebuilds the factory when it is redelivered. _crew_backend_pin_changed fires on any changed agents.<name>.acp_backend path (the changed-paths rule), so it no longer depends on old and new differing. That covers the retry (old is new), the fold-in into the next file change, and a refresh that failed after adopting the config. A record- or section-level delivery is compared against the snapshot in force. Tested end to end through ConfigWatch for all four cases. The cost: adding or removing a crew drains the warm pool once. The rule covers both per-crew factory inputs, acp_backend and reasoning_effort.
  2. _pin_entitlement_backend asks the selection gate for a pinned crew's backend, so a crew pinned to claude is judged by the claude catalog. An unpinned crew is judged as before.

(3) Live proof on the author's desktop: a preview gateway built from this branch ran two crews, one with acp_backend: "claude" and a control with no key; the pinned crew's DM logged routing to acp_backend='claude' (crew acp_backend='claude') and spawned claude-agent-acp (Claude Code), which answered; the control routed kas via agent.member_acp_backend on kiro-cli --agent-engine v3 with no claude spawn. The run is being repeated on the pushed head.

Two "Not a goal" lines in the description changed (the model-pin check and pinning kiro-cli), because asks (1)/(2) and the #11324 alignment moved them into scope.

@github-actions github-actions Bot added readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 3, 2026
@ash663
ash663 force-pushed the feat/per-agent-acp-backend branch from 4b4d327 to 047a253 Compare October 3, 2026 23:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 3, 2026
@ash663

ash663 commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

I rewrote the branch history only to satisfy PR Hygiene's two-commit rule. It is now the same two commits rebased onto main at 59e8b1e, with the merge commit dropped.
The tree is identical to the previous head, 4b4d327 (git diff 4b4d327cd is empty), so the reviews on the prior heads still apply.

The crew record's acp_backend used "" for "pins nothing", but "" is also
kiro-cli's own backend id, so a crew could not stay on kiro-cli under a
non-kiro member route or default. The field now has the blank rule
kirodotdev#11324 gives the same key: null inherits the session default; an empty
string selects Kiro explicitly. The selection gate tests the pin with
`is not None`; every reader (factory, warm pool,
member operating-mode block, dashboard readouts) follows. A full-document
save writes null for every crew that pins nothing, so saving pins no
crew to kiro-cli, and the config view no longer masks a null pin.

A pin edit whose first delivery failed or was deferred was lost until
restart: the watcher redelivers it with an `old` that already carries
the edit (a retry passes the same snapshot as old and new, a fold-in
passes the snapshot that failed), so _crew_backend_pin_changed's value
comparison saw no change. It now matches the changed
agents.<name>.acp_backend path, like every other factory input in that
applier, so a redelivery refreshes even after a refresh that failed
once it had adopted the config. A delivery that names only a record or
the section is compared against the snapshot the manager has in force.
The rule covers both per-crew factory inputs, acp_backend and
reasoning_effort, so an effort edit no longer leaves the factory stale
either. Adding or removing a crew now costs one warm-pool drain.

The warm-pool probe answered "no pin" when it could not read the config
or read a torn one, which served a pinned crew from a pool child on the
default backend while the factory still held the pin. It now cold-starts
in both cases; a wrong answer there only costs a cold start.

The pin is kept raw at load and the config view already masks it, but
the gate logged it raw, and so did the resolver's refusal warning. The
gate and the member operating-mode check now resolve and log a redacted
copy; a registered backend id has no credential shape, so no selection
changes. A refused crew pin's warning names agents.<name>.acp_backend,
not the global agent.acp_backend, so it points at the key to fix; a
refused member route names agent.member_acp_backend, and every refusal
says it runs on kiro-cli instead.

Pooled children were built on the pool agent's crew pin when the pool
agent is a pinned crew, and a no-crew session claiming one ran on that
harness. The fill now passes pooled=True: the child is still built as
that crew (effort, watchdog windows) but on the routed default backend.

set_model re-checks the target's session key, agent, agent kind and
project before
storing a pick: the crew pin it validated against keys on them, and an
agent switch could land during its awaits. The member operating-mode
check derives the crew from the execution record's member_id, as the
factory's readers do (a template selection runs as no crew), so a
re-keyed member's stale desk label cannot select the wrong engine's answer.

The crews API judged a pinned crew's model against the member route's
or the default's catalog, so a crew pinned to claude could not save a
claude model. _pin_entitlement_backend now asks the selection gate for
a pinned crew's backend; an unpinned crew is judged as before. The
kiro-cli spelling correction applies only when that backend reads
kiro-cli's model namespace, so a claude-pinned crew is not told that
claude-opus-4-8 is "not a model kiro-cli serves".

The pool-fill structural test inspects _fill_warm_pool_loop, where
main moved the fill.
@ash663
ash663 force-pushed the feat/per-agent-acp-backend branch from 047a253 to 45a3126 Compare October 4, 2026 10:43
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 4, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Status: still holding for the product decision noted earlier (#12451 is needs-human, and #11324 claims the same agents..acp_backend key with different blank-value rules). The head has grown to ~1.4k lines since my last read; I will re-review the code once a maintainer has picked the design.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 4, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Maintainer ruling: #11324 is the one path for the per-member harness pin (#12451), so this PR is superseded.

Options were (a) keep #11324, (b) keep this PR, or (c) merge parts of both. Both add agents.<name>.acp_backend, so only one can land.
Why (a): #11324 also scopes the model and effort to the pinned harness and has the selector; a pin without those is not usable on its own. Two ideas from this PR carry over to #11324: a test that fails on any select_provider_backend( caller that skips the crew pin, and keeping acp_backend out of GET /api/agents.

Next: thank you for the work here, @ash663. This PR stays open until #11324 lands; then a maintainer closes it as superseded. Nothing to change on this branch.

@Mark-McCracken

Copy link
Copy Markdown

I filed #21910 asking for exactly this (a crewmate-pinned harness that holds across every entry path), and this PR already implements the mechanism I was after — agents.<name>.acp_backend as the first arm of select_provider_backend, keyed on the crew identity so it reaches DM, dashboard chat, cron and channel turns, not just the member-DM route.

Offer: I run a setup of 11 crewmates across mixed harnesses (some always Claude Code, some always Codex). If this gets a rebase onto main (it's currently flagged merge conflict), I'm happy to run an insider build against that real 11-crewmate setup and report back on the per-entry-path behaviour — cron wakes, channel turns, and workers dispatched without an explicit backend=.

One design question worth settling before merge (raised in #21910): when a pinned harness is unavailable/denied, this PR degrades to kiro-cli with a warning. My ask in the issue was the opposite — fail closed with a clear message rather than silently moving the work onto a harness/licence that wasn't chosen for it, since that silent provider-swap is the exact outcome the pin exists to prevent. Not a blocker to validating the happy path, but it's the one behavioural fork a maintainer should rule on.

Happy to help get this over the line.

@CrysisDeu CrysisDeu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer comment at 45a31266: closing this in favour of #25485.

Ashish, this was the cleanest backend half of the three and it is the one the re-port borrows from. The 10-16 ruling made #11324 the per-member path and this PR superseded; that path is now #25485 (kyleseaman's re-port onto current main). This branch is 3,887 commits behind with 19 conflicting files in modules main has since split, its blank rule ("" = kiro-cli) disagrees with main's #20359 vocabulary, and a pinned backend that is missing degrades silently to kiro-cli, which the maintainer rule now forbids. So it closes rather than rebases.

Four pieces of yours are to be carried into #25485, named there: the changed-paths hot-reload rule with its ConfigWatch tests, the select_provider_backend( call-site guard, withholding acp_backend from GET /api/agents, and the warm-pool "unreadable config -> cold start" rule. Please review #25485 for those.

On your codex use case: mates can pin only harnesses in ACP_BACKENDS_MATE_WORKER_KIT (today kas, claude), because the worker's per-session MCP kit has to cross the wire and we only have live captures for those two. codex is not excluded by design; it joins the moment a test/fixtures/acp_frames/codex/worker-kit-live.jsonl capture lands, which is a small standalone PR if you want it. kiro-cli cannot join (reads --agent off disk, no per-session channel). Until then a codex reviewer runs as spawn_run(backend=codex) or a per-chat pick (#19599). The agent.sandbox per-backend question is a separate issue; please file it.

Posted by code-reviewer (Kiro Crew member) on behalf of @CrysisDeu. Full report on file.

@bolichen97

Copy link
Copy Markdown
Contributor

Thanks for this work, and for aligning the key and blank rule with #11324.

The maintainer ruling on 2026-10-16 picked #11324 as the single path for the per-member harness pin. This PR now overlaps it, conflicts with main, and has had no new commits since 2026-10-04.

I propose closing this PR in favor of #11324. If any piece here is not covered there (for example the entry-path coverage raised in #21910), a small follow-up PR on top of #11324 would be the easiest way to land it. I will leave the close to you or a maintainer.

@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97]

Correction to my 2026-10-30 note: the per-mate harness pin landed in #25485 (merged 2026-10-30), not #11324, which is still open. This PR is superseded by #25485, per CrysisDeu's 2026-10-29 review.

@ash663, could you check that the four pieces CrysisDeu named made it into #25485: the changed-paths hot-reload with its ConfigWatch tests, the select_provider_backend( call-site guard, withholding acp_backend from GET /api/agents, and the warm-pool "unreadable config -> cold start" path? If one is missing, a small follow-up PR on current main is the easiest way in. Then this PR can be closed.

AI-assisted, approved by bolichen97

@ash663

ash663 commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

@bolichen97 @CrysisDeu I checked the four pieces against the #25485 merge commit (ffff62c; the links pin it) and current main (95abd91). Three are in; hot reload is partial.

  1. Changed-paths hot reload: partial. The path rule is in for agents.<name>.acp_backend (config_apply.py), and feat(crew): let a crew member pin its own ACP backend #16200's four ConfigWatch redelivery cases (retry and fold-in, before and after adoption) pass when run on current main. Missing: those ConfigWatch tests (main has one stubbed test, TestHotReload), and the agents.<name>.reasoning_effort leaf feat(crew): let a crew member pin its own ACP backend #16200's rule also covered: an effort edit made outside the crew editor does not rebuild the provider factory.
  2. select_provider_backend( call-site guard: present. TestEveryCallerPassesThePin fails on any caller that omits crew_backend; dropping it from one caller turns the test red.
  3. acp_backend withheld from GET /api/agents: present. The roster row is an allowlist without it (roster.py); WITHHELD_RECORD_FIELDS lists it, and test_the_roster_leaves_the_pin_out sets a pin and checks every row.
  4. Warm pool "unreadable config -> cold start": present, by another route. Only a mate's DM thread and its own worker take a pin (crewmate_for_session), and the pool bypasses both by session key, before its only config read (get_or_create.py), so an unreadable config cannot put a pinned session on a pooled child.

The linked code is unchanged on current main, where test/test_crewmate_harness.py, test/test_agents_roster_contract.py and TestPoolBypass pass (168 tests).

I can open a small follow-up PR on current main for piece 1: port #16200's ConfigWatch redelivery tests, and add the reasoning_effort leaf to the rule with a test. You can close #16200 once that follow-up lands.

@bolichen97

Copy link
Copy Markdown
Contributor

@ash663 thanks for checking #25485 against this PR.

Under the team SOP, this is a T3 (it changes how crew members pick a backend, see #12451). A T3 needs an Issue and a 1-2 page RFC/design doc under docs/request-for-change/, and at least one teammate has to agree before the PR to main can merge. #12451 has neither yet.

Most of this PR already landed in #25485. For the part that is still missing (hot reload is partial, per your note), please open an Issue and an RFC for it. Once they are open, Crew Captain will triage it, and the team will look at it by priority.

AI-assisted, sent by CommunityCrew

@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this work! It continued in #25485, which has now merged. Closing this one as superseded by #25485.

@iamwhatever iamwhatever closed this Nov 8, 2026
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants