Skip to content

fix(pi): honour an agent spec's zero-tool ban on pi sessions - #22808

Merged
iamwhatever merged 2 commits into
mainfrom
feat/pi-zero-tool-ban
Oct 21, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
feat/pi-zero-tool-ban

Conversation

@kyleseaman

@kyleseaman kyleseaman commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A pi session whose agent spec declares "tools": [] runs no tool at all: Crew refuses every call itself, without asking the user.

Today pi has no mirror, so nothing computes SessionMcpProjection.zero_tools for it, AcpClient._deny_zero_tools never fires, and a zero-tool spec on pi still runs every write, bash or MCP call the user approves. backends_mcp.py said so in as many words ("pi is NOT a member ... nothing computes zero_tools for it").

Why it matters

"tools": [] is how Crew declares a text-only agent, the knowledge/research worker being the main one. On pi that declaration was decorative: native tools stayed reachable behind an approval prompt, and pi was the one gate-enforced harness left out of ACP_BACKENDS_HONOR_ZERO_TOOL_BAN. That kept pi from being fully wired in as a harness.

Not a goal

  • Withholding Crew's MCP mounts from a zero-tool pi session. They still register, but every call to them is refused. Mount composition is fix(pi): confirm the session registered its MCP handoff #22400's area.
  • Adding a pi mirror, i.e. translating the spec's mcpServers for pi.
  • Confining which operator pi extensions load, which is feat(pi): run Side Chat read-only tools under the gate extension #21744's Side Chat work.
  • Saved-spec verification for pi (ACP_BACKENDS_MEMBER_CAPABILITIES). This design does not set _session_agent_spec. The parse it reads zero_tools from in _mirrorless_member_mounts is a SessionMcpProjection, which already carries agent_spec. A follow-up can record it at that same point from the same parse, so mounts, the ban and the consumed spec would all come from one spec version.
  • Any change to the gate extension or its digest.

What changed (motivation → approach → change)

Symptom → root cause. The refusal (_deny_zero_tools) already runs at both permission-answering sites, and pi's gate extension already routes every tool call through session/request_permission. The only missing piece was the fact "this spec bans all tools", because only mirrors computed it.

Approach. Compute that fact for pi from the parse Crew already does, not by adding a mirror. _mirrorless_member_mounts already reads pi's spec for its member mounts. A pi mirror would change what a pi session mounts, overlap #22400, and add nothing to the refusal. No parallel refusal path is built.

Changes:

  • acp/client_mcp_array.py: for a mirror-less host whose array Crew composes (ACP_BACKENDS_SESSION_MCP_ARRAY, today only pi), _mirrorless_member_mounts records zero_tools from the same parse that builds the mounts. kiro-cli isn't in that set and returns before the parse, so its path is unchanged.
  • agent_sdk/backends_mcp.py: pi joins ACP_BACKENDS_HONOR_ZERO_TOOL_BAN. The rule's third fact now reads "something computes zero_tools", which can be a mirror or this parse. The comment cites the live capture.
  • scripts/check_harness_parity.py: X_FEATURE_SETS gains ACP_BACKENDS_HONOR_ZERO_TOOL_BAN: frozenset({"pi"}), so the x-capture gate (test(harness): require a replayed live capture for every X-feature set member #22102) holds pi's membership to a replayed live capture. Removing the citation fails the gate.
  • Knowledge pool (this follows from its "ask the capability" rule): a pool configured for pi now runs on pi. GPT review found it then ignored a pinned knowledge.extraction_model. The same gap already existed for opencode, goose and deepseek, because only kiro-cli reads the spec's own model. The installer's rule (knowledge.extraction_model → agent.model → "auto") is extracted to knowledge_agent_model() and exported through the kiro_crew.agent facade. AcpWorker._spawn_client passes that pin as model= only on the non-AGENT_SPEC branch. The kiro-cli AcpClient(...) call stays byte-identical, with no model= kwarg (H13). The existing startup model application scopes it to the backend. kiro-cli's construction is unchanged.
  • Docstrings for capabilities.py, spec_zero_tools and llm_pool were updated to match, along with knowledge.md and the shipped docs/agent-spec-fields.md, whose zero-tool paragraph now lists opencode, goose, deepseek and pi as closing the gap.
  • _mirrorless_member_mounts keeps its original member-set early return. The array-host clause is an added third condition. An unreadable spec records zero_tools = False, including when the same client re-resolves.
  • test/test_member_mount_pi.py: feat(pi): mount the crew member panel on pi member threads #22813 pinned pi not in ACP_BACKENDS_HONOR_ZERO_TOOL_BAN as one of three reasons pi stays out of ACP_BACKENDS_MEMBER_CAPABILITIES. That pin now asserts the new fact, and that pi is still outside ACP_BACKENDS_MEMBER_CAPABILITIES, because the other two reasons (no consumed spec, a BROKER_ONLY handoff) still hold. The matching third bullet in backends_members.py's comment is now stale. This PR deliberately leaves that file alone because another workstream owns it.

Compatibility with neighbours: #21744 edits harness/pi.py, permission_requests.py and backends.py. #22400 touches client_mcp_array.py only in _republish_pi_mcp_handoff and an import. #12344 touches neither file. None edit these hunks, so a rebase in either order should be mechanical.

Backwards compatibility

Compatible: the change only adds refusals on pi sessions whose spec declares "tools": [], and a model pin on non-kiro pool workers. Every other session and backend behaves as before.

Removes nothing: no path, key, record, validator or public function is removed or narrowed. _install_knowledge_agent writes the same spec; its model rule moved into knowledge_agent_model() unchanged.

Tests

  • test/test_pi_zero_tool_ban.py (26). It replays the three live permission frames from the capture through both answering sites (_deny_zero_tools before the consumer yield, and the auto-approve _handle_permission) and asserts the advertised reject option is sent with a spec_zero_tools audit record. Each case is listed below.
    • Nearby inputs: refused as well — an unknown tool name that reaches the gate, an unmounted MCP server, edit, read, truncated args, and a foreign-nonce frame.
    • Ordinary specs: a spec with no tools key, a populated list, or a non-list leaves the refusal off, and the same frames reach the consumer.
    • Edge cases: an unreadable spec leaves it off, which matches mirrored backends. A respawn on an edited spec reads it again. kiro never records it.
    • Revert-fail: restoring client_mcp_array.py fails 6 tests, and dropping pi from the set fails 19.
  • test/test_agent_sdk_capabilities.py: the membership-rule recompute now reads "mirror OR Crew-composed array". The pool-safe pin includes pi, with its effort channel (thought_level config option) asserted.
  • test/test_llm_pool.py: pi passes the selection gate. A pi, opencode, goose or deepseek worker receives model=<pin>. A kiro worker gets no model kwarg and never reads the pin. These tests fail without the pool fix. _knowledge_spec_model treats "auto", a blank or an unreadable config as no pin.
  • test/fixtures/acp_frames/pi/zero-tools-refused-live.jsonl + .expected.json: replayed by test_acp_frame_replay, clean under check_acp_frame_host_data.py.

Local runs:

  • Gates: HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.py ✓ and --test ✓; black, isort and flake8 ✓; scoped mypy ✓.
  • Focused pytest: about 1650 tests across the pi backend, session MCP, frame replay and host-data, capabilities, backend cards, harness parity, llm_pool, knowledge budget and tool hooks. The full suite was left to CI.

Manual verification

This is a live run of a real AcpClient(acp_backend="pi"). It used pi-acp 0.0.34 over pi 1.2.0, installed into a scratch npm prefix, so the global pi was not used. The model was a scripted one on the loopback interface, plus one stdio MCP probe server, and the frames were recorded by Crew's in-product recorder. The consumer approved every permission event it was handed, and on the zero-tool spec it was handed none.

Input Zero-tool spec Spec with no tools key (control)
write refused by host, failed, file absent user asked, file written
bash (touch) refused by host, failed, file absent user asked, file created
MCP mcp__probe_server__echo refused by host, failed; server logged initialize/tools/list, no tools/call user asked, tools/call ran
unknown tool frobnicate pi fails it itself (Tool frobnicate not found), no permission frame, nothing runs same

Not verifiable on this host: it refuses user namespaces, so no sandbox backend runs and a sandboxed gateway turn isn't reachable here. In that state the routing's refuse-then-mask preflight refuses pi. The capture driver replaced only that one preflight with "no mask". The gate-extension seal, its read-back, the envelope nonce and the permission path ran as shipped. The refusal happens after the spawn and doesn't read the sandbox. I have not observed a pi turn through a full sandboxed dashboard gateway.

Independent fresh-context review (read-only, given only the diff and the invariant) of the refusal path: SOUND.

  • Timing: the flag is set in PiLaunch.resolve_spawn → _prepare_session_mcp before session/new or session/load. _reset_state clears it only together with the process. A warm-pool claim must match the agent.
  • Every site that answers a request was checked. The always-allow grants and the Side Chat READ_ONLY gate only see events that already passed the refusal. The steer and init drains never approve. A late approve_tool after a reject fails. pi-acp offers no "always" option.
  • No behaviour change on other backends.

Residuals it named (pre-existing mechanisms): the pi launcher still loads the operator's own pi extensions, and their / commands or input handlers are not tool calls, so they never raise a permission request. pi-acp also forwards every extension dialog as a permission request, so a zero-tool pi session refuses unrelated dialogs too.

Related Issues

None. Related open PRs: #21744, #22400, #12344.

Pattern harvest

Rule candidate: review-prompt
Pattern: "a capability set whose membership needs a per-backend computed fact (here zero_tools) silently excludes a backend that computes it by a different channel; state the fact, not the channel (mirror), in the rule and its test."

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Sensitive change: none. No file in this diff matches SENSITIVE_GLOBS (checked with fnmatch), and nothing is inherited.

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@kyleseaman
kyleseaman requested review from a team and KiroCrewCaptain as code owners October 21, 2026 15:48
@kyleseaman
kyleseaman requested a review from CrysisDeu October 21, 2026 15:48
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision c8e361a1f053c55c586be18291c0ffb4a6fc417d touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of c8e361a1f053c55c586be18291c0ffb4a6fc417d — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the pi ban reuses the existing refusal and reads zero_tools from the same parse as the mounts, and the knowledge-pool model pin is a documented result of pi joining the pool.

Notes

  • The knowledge-pool model= fix also changes opencode, goose and deepseek workers, so it is separate from the pi goal. It could have been its own commit or PR; the description does explain it.
  • The third bullet in backends_members.py, which says pi is out because it is outside ACP_BACKENDS_HONOR_ZERO_TOOL_BAN, is now wrong. Open a follow-up with that file's owner now, so the next person changing member capabilities doesn't act on a reason that no longer holds.

[DESIGN-REVIEWED] c8e361a

Description read

[DESCRIPTION-READ] 36844f90034c3bb9c459d0e699824037a9f4976769bf544ffbe9af6e993fa84d

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of c8e361a1f053c55c586be18291c0ffb4a6fc417d — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing to check.

What this change ships

Inventory (7 items) — 6 justified

Intent: a pi agent whose spec says "tools": [] actually runs no tools, and Crew refuses every call itself. This is a FIX.

  1. On pi, a zero-tool spec now has every native, MCP or unknown tool call refused, with no approval prompt — justified
  2. pi joins ACP_BACKENDS_HONOR_ZERO_TOOL_BAN, read from the parse that already builds its member mounts, with no new mirror — justified
  3. A knowledge pool set to pi now runs on pi instead of falling back to Kiro — justified
  4. Pool workers on opencode, goose, deepseek and pi now get the pinned knowledge.extraction_model — rides along (declared; tests fail on base)
  5. knowledge_agent_model() is pulled out of the installer and exported through the kiro_crew.agent facade (2 consumers: the installer and llm_pool) — justified
  6. The harness-parity gate now ties pi's membership to a replayed live capture — justified
  7. The test_member_mount_pi.py pin that said "not enforced on pi" is reversed. The description says so and gives the reason — justified

[FIRST-PRINCIPLES-REVIEWED] c8e361a

Description read

[DESCRIPTION-READ] b5331a815bebbe15579af49ce55c4895f8cd51c6084799f1582808a428c3cb5f

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed c8e361a1f053c55c586be18291c0ffb4a6fc417d — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c8e361a

Verdict parsed from the review's SHA-scoped output markers for commit c8e361a1f053c55c586be18291c0ffb4a6fc417d.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c8e361a1f053c55c586be18291c0ffb4a6fc417d: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of c8e361a1f053c55c586be18291c0ffb4a6fc417d and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c8e361a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c8e361a1f053c55c586be18291c0ffb4a6fc417d: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-zero-tool-ban branch from 3f48b94 to 1e2299a Compare October 21, 2026 17:12
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-zero-tool-ban branch from 1e2299a to d860564 Compare October 21, 2026 17:47
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-zero-tool-ban branch from d860564 to d02ff5d Compare October 21, 2026 18:21
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-zero-tool-ban branch from d02ff5d to 2a1ee76 Compare October 21, 2026 18:54
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 21, 2026
Kyle Seaman added 2 commits October 21, 2026 19:28
pi has no mirror, so nothing computed SessionMcpProjection.zero_tools for it
and a spec declaring "tools": [] still ran every pi tool the user approved.
Read the answer beside pi's member mounts from the same spec parse
(_mirrorless_member_mounts, for mirror-less ACP_BACKENDS_SESSION_MCP_ARRAY
hosts) and add pi to ACP_BACKENDS_HONOR_ZERO_TOOL_BAN, so the existing
AcpClient._deny_zero_tools refuses every request the gate extension raises.

Backed by a live capture through AcpClient on pi-acp 0.0.34 over pi 1.2.0
(test/fixtures/acp_frames/pi/zero-tools-refused-live.jsonl), held to it by
the x-capture gate.
…ad no spec

kiro-cli reads kirocrew-knowledge's model from its spec; opencode, goose,
deepseek and now pi read no spec of Crew's, so a pinned
knowledge.extraction_model silently ran on the harness default. Extract the
installer's rule into knowledge_agent_model() (exported through the
kiro_crew.agent facade) and pass it as model= to every non-AGENT_SPEC pool
client; the startup model application scopes it per backend. kiro-cli's
construction is unchanged.
@kyleseaman
kyleseaman force-pushed the feat/pi-zero-tool-ban branch from 2a1ee76 to c8e361a Compare October 21, 2026 19:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) October 21, 2026 21:14
@iamwhatever
iamwhatever merged commit aca1231 into main Oct 21, 2026
113 of 115 checks passed
@iamwhatever
iamwhatever deleted the feat/pi-zero-tool-ban branch October 21, 2026 22:30
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 21, 2026
kyleseaman pushed a commit that referenced this pull request Oct 22, 2026
#22808 gated _mirrorless_member_mounts on two new 'backend in ACP_BACKENDS_SESSION_MCP_ARRAY' tests, which #22489's set-gate ratchet (shrink-only, allowlist 2 for this file) counts, so main reds test_no_new_runtime_site_gates_a_field_backed_line_on_a_set. The question those sites ask is who reads this backend's spec, which registry.PROJECTIONS declares per backend: BROKER_ONLY (pi) means no projection is written and the harness reads no spec of Crew's. Read that record, as this file already does for per_tool_deny, so the file is back to its 2 allowed sites and the zero-tool ban is unchanged.
buluoray pushed a commit that referenced this pull request Oct 22, 2026
…2951)

* test(pi): stand in for the Node probe's sync sandbox wrap in the real-adapter lane

#22883's node-engine guard confines each 'node --version' probe through _pi_probe_wrap, which calls the sync pi_mod.wrap_argv. The lane (#22807) replaced only wrap_argv_async and the preflight, so on a runner without user namespaces every pi_session test errored at setup with SandboxUnavailableError. Replace the sync wrap by identity too, and say so in the module docstring.

* fix(acp): decide pi's zero-tool record from its declared MCP projection

#22808 gated _mirrorless_member_mounts on two new 'backend in ACP_BACKENDS_SESSION_MCP_ARRAY' tests, which #22489's set-gate ratchet (shrink-only, allowlist 2 for this file) counts, so main reds test_no_new_runtime_site_gates_a_field_backed_line_on_a_set. The question those sites ask is who reads this backend's spec, which registry.PROJECTIONS declares per backend: BROKER_ONLY (pi) means no projection is written and the harness reads no spec of Crew's. Read that record, as this file already does for per_tool_deny, so the file is back to its 2 allowed sites and the zero-tool ban is unchanged.

---------

Co-authored-by: Kyle Seaman <kseam@dev-dsk-kseam-1b-55230d27.us-east-1.amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants