Skip to content

test(pi): drive pinned pi-acp + pi in the real-adapter lane - #22807

Merged
iamwhatever merged 2 commits into
mainfrom
feat/pi-real-adapter-lane
Oct 21, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
feat/pi-real-adapter-lane

Conversation

@kyleseaman

@kyleseaman kyleseaman commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a pi or pi-acp release that breaks Kiro Crew's pi harness turns the Real Adapter Contract lane red, the way an opencode or codex-acp release already does.

Kiro Crew's pi harness (acp/harness/pi.py) rests on facts about two third-party packages that nothing in CI measured: test/real_adapters/package.json pinned only codex-acp and opencode, so the lane never installed pi-acp or pi. The gate extension reaches a pi session only if pi-acp runs the launcher PI_ACP_PI_COMMAND names, pi lists the extension's probe command in get_commands, and pi-acp forwards the extension's confirm dialog as session/request_permission. A release that dropped any of those would break pi sessions (refused at the read-back, or tool calls that never ask) with no red check.

Why it matters

pi-acp is a 0.0.x adapter from a third party and pi ships weekly; Dependabot already refreshes test/real_adapters/, but without a pin and a test there was no bump PR and no lane run to catch drift before users hit it.

Not a goal

What changed (motivation → approach → change)

Approach: drive the real pair through Crew's own launch path rather than a hand-rolled spawn, so the lane measures what a session runs.

  • test/real_adapters/package.json + lockfile: pin pi-acp 0.0.34 and @earendil-works/pi-coding-agent 1.3.0 (floor is 0.99.2). Lockfile regenerated with npm install --package-lock-only; every package/version the old lockfile held is still in it.
  • test/real_adapter_gate.py: PI_ACP_PACKAGE, PI_PACKAGE, MEASURED_PI_ACP_VERSION, MEASURED_PI_VERSION, read from the manifest like the other pins.
  • test/test_pi_real_adapter.py (new): one module-scoped drive. PiLaunch.resolve_spawn runs for real against an AcpClient session: both resolvers (honouring PI_ACP_BIN / PI_ACP_PI_COMMAND), the version floor, the digest-verified seal, the encrypted MCP handoff, the gate launcher and the get_commands read-back. apply_spawn_env builds the child env. The driver then speaks ACP to the adapter with Crew's own initialize params and reads every permission frame with Crew's own build_permission_event under the launch's nonce. Stand-ins: the OS sandbox preflight/wrap (identity, so the lane needs no user namespaces), a loopback OpenAI-compatible model in pi's models.json (no credential), and a stdio stub MCP server mounted through _session_mcp_servers. Seven real_adapter tests:
    • pinned pair at or above the floor;
    • handshake answers PROTOCOL_VERSION_PI, agentInfo names the pinned pi-acp;
    • pi's registry lists the probe command sourced from Crew's SEALED copy, and the same registry is refused against the packaged original or any other path;
    • a bash call raises session/request_permission carrying the envelope with the launch nonce, Crew's builder names bash and its command, and the output reaches the model after yes;
    • a no blocks the call (the file it would write is never created; pi reports the gate's denial);
    • the MCP handoff (not the dropped session/new array) mounts the stub: tools/list + tools/call, gated first, title mapped back to mcp__probe-server__echo, ECHO:ping reaches the model;
    • /compact is advertised, calls the model and ends the same prompt turn with Compaction completed.
      Plus two unmarked tests that run on every shard: the pi pin clears PI_MIN_VERSION, and the scripted model only calls tools the request offers.
      The drive puts the lane's own node first on the child's PATH (a symlink in the isolated home's ~/.local/bin, which leads Crew's PATH augmentation). Without it the first lane run refused at the read-back (exit 1: its configuration could not be parsed): pi's bin is #!/usr/bin/env node, and augmented_path puts /usr/local/bin ahead of the inherited PATH, where the runner image's system node is older than pi's >=22.19 engine. A refusal now prints pi's own stderr and the resolved node version.
      The model listener binds through socketserver.TCPServer.server_bind (the repo's no-getfqdn pattern, test_http_server_bind_ratchet).
  • .github/workflows/ci.yml (real-adapter-contract only): assert the installed pi-acp/pi are the pins, pass PI_ACP_BIN / PI_ACP_PI_COMMAND pointing at the lockfile install, collect the new file, raise the executed-contract floor 11 → 18.
  • test/test_real_adapter_gate.py: the manifest/lockfile/no-literal-pin checks cover the two pi packages (the no-literal check is lane-scoped for pi because 1.3.0 also appears as an unrelated build== pin elsewhere in ci.yml).
  • .github/dependency-review-config.yml: one per-package exception, pkg:npm/bowser, the same layer the opencode platform binaries use. bowser's manifest says MIT; GitHub reads its LICENSE as MIT AND MITNFA (MIT plus the "no false attribs" clause). It arrives only through @aws-sdk/core in pi's tree, installed solely by this CI lane; nothing ships it.

Backwards compatibility

Compatible: test and CI-lane additions only; no product code changes.
Removes nothing: the lockfile keeps every package version it held (checked), and no test, step or pin was dropped.

Tests

Mutation proof on the final head (each applied locally to the installed pin, the pi tests re-run, then reverted):

Mutation Result
none (baseline, 3 runs) 9 passed
pi-acp ignores PI_ACP_PI_COMMAND (spawns plain pi) 3 failed: gated bash, refused call, MCP handoff
same, with the pinned ungated pi first on PATH (CI's shape) 3 failed (same three)
pi-acp stops forwarding the confirm dialog (answers yes itself) 3 failed (same three)
pi-acp forwards the dialog without its message (envelope dropped) 2 failed: gated bash, MCP handoff
pi-acp inverts the client's answer 3 failed (same three)
pi drops extension commands from get_commands (probe gone) 6 errors: the launch's read-back refuses the session
pi drops an extension's registerMcpServer 1 failed: MCP handoff

Also run: test/test_real_adapter_gate.py, test/test_real_adapter_fixture_pins.py (pass); HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.py and --test (pass); check_file_line_cap, check_subprocess_encoding, check_sync_io_in_async, check_agent_sdk_boundary (pass); semgrep 1.78 with the repo rules + p/python, p/security-audit, p/secrets, p/owasp-top-ten on the changed files (clean); black, flake8, mypy on the new file (clean).

Manual verification

This host refuses user namespaces, so a full sandboxed turn through the gateway is not reachable here; the tests stand in for the OS sandbox (preflight + argv wrap) and say so in the module docstring. Everything above the sandbox runs for real. No i18n strings touched.

CI evidence: the Real Adapter Contract Tests job ran all 7 pi tests green on head 430abe41ba ("31 real-adapter contract tests executed, none skipped"; installed pins asserted as pi-acp 0.0.34, pi 1.3.0).

Observation, not changed here: the node finding above is product-shaped. Any pi session launched where an old node sits in one of augmented_path's leading directories (/usr/local/bin, /opt/homebrew/bin, ...) runs pi under that node, and the read-back refusal names only a "configuration could not be parsed" class.

Related Issues

N/A

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

Pin pi-acp 0.0.34 and @earendil-works/pi-coding-agent 1.3.0 in
test/real_adapters and add test/test_pi_real_adapter.py: the pair is
launched through PiLaunch.resolve_spawn (seal, MCP handoff, gate
launcher, registry read-back) and driven against a loopback model and a
stub MCP server. The lane now reds when pi-acp stops honouring
PI_ACP_PI_COMMAND, stops forwarding the gate's confirm dialog, or pi
drops the probe command.
@kyleseaman
kyleseaman requested review from a team and KiroCrewCaptain as code owners October 21, 2026 15:47
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 430abe41ba316dfb7066507fca05b581ae3a1c26 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 430abe41ba316dfb7066507fca05b581ae3a1c26 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. This PR adds tests and changes only the real-adapter CI lane. It pins pi-acp and pi the same way the lane already pins codex-acp and opencode, and runs them through Crew's own launch path. No product code changes.

Notes

  • The pkg:npm/bowser license exception covers every version of the package, while the comment's evidence points only at 2.14.1. If a later Dependabot bump changes bowser's license, this gate won't flag it.
  • The tests swap in an identity function for the OS sandbox, so a pi-acp or pi release that breaks only inside the sandbox can still pass this lane green. The PR's description and the test module's docstring both say so.

[DESIGN-REVIEWED] 430abe4

Description read

[DESCRIPTION-READ] 5e13d4a91e6c69cca9e3312507daae6c435015cf9e65384cf0b42a84c8a42c7a

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 430abe41ba316dfb7066507fca05b581ae3a1c26 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Every item is test or CI-lane surface that serves the Goal, and the bowser license exception is needed because pi's dependency tree pulls bowser in.

What this change ships

Inventory (7 items) — 7 justified

Intent: catch a pi or pi-acp release that breaks Kiro Crew's pi harness in the Real Adapter Contract lane, the way opencode and codex-acp releases are already caught. ADDITION.

  1. The Real Adapter lane installs pinned pi-acp 0.0.34 and pi 1.3.0 from the lockfile — justified
  2. The lane checks that the installed pi-acp and pi are the pinned versions before testing — justified
  3. The lane drives a real pi session through Crew's own launch: probe-command read-back, permission dialog for yes and no, MCP handoff, /compact — justified
  4. The lane passes PI_ACP_BIN and PI_ACP_PI_COMMAND so Crew's resolvers find the lockfile install — justified
  5. The lane's required count of executed contract tests rises from 11 to 18 — justified
  6. The existing manifest, lockfile and no-copied-pin gate tests now cover the two pi packages — justified
  7. The dependency-review license gate allows pkg:npm/bowser (MIT + MITNFA, reached only through pi's tree in this CI lane) — justified

[FIRST-PRINCIPLES-REVIEWED] 430abe4

Description read

[DESCRIPTION-READ] 404b7dd6e407511ed6f947a7472961d3619dc2d51e2e3ef67898cabedeed12be

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 430abe41ba316dfb7066507fca05b581ae3a1c26 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 430abe4

Verdict parsed from the review's SHA-scoped output markers for commit 430abe41ba316dfb7066507fca05b581ae3a1c26.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 430abe41ba316dfb7066507fca05b581ae3a1c26: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 430abe41ba316dfb7066507fca05b581ae3a1c26 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 430abe4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 430abe41ba316dfb7066507fca05b581ae3a1c26: <one-sentence reason>

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

The following issues were found:

  • ❌ 1 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ✅ 0 package(s) with unknown licenses.
  • ⚠️ 17 packages with OpenSSF Scorecard issues.

View full job summary

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-real-adapter-lane branch from e4a5c2a to 50fd41f Compare October 21, 2026 16:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-real-adapter-lane branch 2 times, most recently from 5bf3bad to f8ed45a Compare October 21, 2026 17:26
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@kyleseaman
kyleseaman force-pushed the feat/pi-real-adapter-lane branch from f8ed45a to 430abe4 Compare October 21, 2026 18:04
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 21, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) October 21, 2026 21:14
@iamwhatever
iamwhatever merged commit b641597 into main Oct 21, 2026
115 checks passed
@iamwhatever
iamwhatever deleted the feat/pi-real-adapter-lane branch October 21, 2026 22:30
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 21, 2026
buluoray pushed a commit that referenced this pull request Oct 22, 2026
…2951)

* test(pi): stand in for the Node probe's sync sandbox wrap in the real-adapter lane

#22883's node-engine guard confines each 'node --version' probe through _pi_probe_wrap, which calls the sync pi_mod.wrap_argv. The lane (#22807) replaced only wrap_argv_async and the preflight, so on a runner without user namespaces every pi_session test errored at setup with SandboxUnavailableError. Replace the sync wrap by identity too, and say so in the module docstring.

* fix(acp): decide pi's zero-tool record from its declared MCP projection

#22808 gated _mirrorless_member_mounts on two new 'backend in ACP_BACKENDS_SESSION_MCP_ARRAY' tests, which #22489's set-gate ratchet (shrink-only, allowlist 2 for this file) counts, so main reds test_no_new_runtime_site_gates_a_field_backed_line_on_a_set. The question those sites ask is who reads this backend's spec, which registry.PROJECTIONS declares per backend: BROKER_ONLY (pi) means no projection is written and the harness reads no spec of Crew's. Read that record, as this file already does for per_tool_deny, so the file is back to its 2 allowed sites and the zero-tool ban is unchanged.

---------

Co-authored-by: Kyle Seaman <kseam@dev-dsk-kseam-1b-55230d27.us-east-1.amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants