Repository navigation
test(pi): drive pinned pi-acp + pi in the real-adapter lane - #22807
Conversation
Pin pi-acp 0.0.34 and @earendil-works/pi-coding-agent 1.3.0 in test/real_adapters and add test/test_pi_real_adapter.py: the pair is launched through PiLaunch.resolve_spawn (seal, MCP handoff, gate launcher, registry read-back) and driven against a loopback model and a stub MCP server. The lane now reds when pi-acp stops honouring PI_ACP_PI_COMMAND, stops forwarding the gate's confirm dialog, or pi drops the probe command.
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. This PR adds tests and changes only the real-adapter CI lane. It pins pi-acp and pi the same way the lane already pins codex-acp and opencode, and runs them through Crew's own launch path. No product code changes. Notes
[DESIGN-REVIEWED] 430abe4 Description read[DESCRIPTION-READ] 5e13d4a91e6c69cca9e3312507daae6c435015cf9e65384cf0b42a84c8a42c7a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Every item is test or CI-lane surface that serves the Goal, and the bowser license exception is needed because pi's dependency tree pulls bowser in. What this change shipsInventory (7 items) — 7 justifiedIntent: catch a pi or pi-acp release that breaks Kiro Crew's pi harness in the Real Adapter Contract lane, the way opencode and codex-acp releases are already caught. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 430abe4 Description read[DESCRIPTION-READ] 404b7dd6e407511ed6f947a7472961d3619dc2d51e2e3ef67898cabedeed12be That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Dependency ReviewThe following issues were found:
|
e4a5c2a to
50fd41f
Compare
5bf3bad to
f8ed45a
Compare
f8ed45a to
430abe4
Compare
…2951) * test(pi): stand in for the Node probe's sync sandbox wrap in the real-adapter lane #22883's node-engine guard confines each 'node --version' probe through _pi_probe_wrap, which calls the sync pi_mod.wrap_argv. The lane (#22807) replaced only wrap_argv_async and the preflight, so on a runner without user namespaces every pi_session test errored at setup with SandboxUnavailableError. Replace the sync wrap by identity too, and say so in the module docstring. * fix(acp): decide pi's zero-tool record from its declared MCP projection #22808 gated _mirrorless_member_mounts on two new 'backend in ACP_BACKENDS_SESSION_MCP_ARRAY' tests, which #22489's set-gate ratchet (shrink-only, allowlist 2 for this file) counts, so main reds test_no_new_runtime_site_gates_a_field_backed_line_on_a_set. The question those sites ask is who reads this backend's spec, which registry.PROJECTIONS declares per backend: BROKER_ONLY (pi) means no projection is written and the harness reads no spec of Crew's. Read that record, as this file already does for per_tool_deny, so the file is back to its 2 allowed sites and the zero-tool ban is unchanged. --------- Co-authored-by: Kyle Seaman <kseam@dev-dsk-kseam-1b-55230d27.us-east-1.amazon.com>
Problem / Motivation
Goal: a pi or pi-acp release that breaks Kiro Crew's pi harness turns the Real Adapter Contract lane red, the way an opencode or codex-acp release already does.
Kiro Crew's pi harness (
acp/harness/pi.py) rests on facts about two third-party packages that nothing in CI measured:test/real_adapters/package.jsonpinned only codex-acp and opencode, so the lane never installed pi-acp or pi. The gate extension reaches a pi session only if pi-acp runs the launcherPI_ACP_PI_COMMANDnames, pi lists the extension's probe command inget_commands, and pi-acp forwards the extension's confirm dialog assession/request_permission. A release that dropped any of those would break pi sessions (refused at the read-back, or tool calls that never ask) with no red check.Why it matters
pi-acp is a 0.0.x adapter from a third party and pi ships weekly; Dependabot already refreshes
test/real_adapters/, but without a pin and a test there was no bump PR and no lane run to catch drift before users hit it.Not a goal
agent_sdk/gate_extensions/pi/*,acp/harness/pi.py): read-only here.test_real_adapter_fixture_pins.py: the corpus recordsagent_versionas0.0.34 (pi 1.2.0), a compound the pin gate does not read, so that is its own change.What changed (motivation → approach → change)
Approach: drive the real pair through Crew's own launch path rather than a hand-rolled spawn, so the lane measures what a session runs.
test/real_adapters/package.json+ lockfile: pinpi-acp0.0.34 and@earendil-works/pi-coding-agent1.3.0 (floor is 0.99.2). Lockfile regenerated withnpm install --package-lock-only; every package/version the old lockfile held is still in it.test/real_adapter_gate.py:PI_ACP_PACKAGE,PI_PACKAGE,MEASURED_PI_ACP_VERSION,MEASURED_PI_VERSION, read from the manifest like the other pins.test/test_pi_real_adapter.py(new): one module-scoped drive.PiLaunch.resolve_spawnruns for real against anAcpClientsession: both resolvers (honouringPI_ACP_BIN/PI_ACP_PI_COMMAND), the version floor, the digest-verified seal, the encrypted MCP handoff, the gate launcher and theget_commandsread-back.apply_spawn_envbuilds the child env. The driver then speaks ACP to the adapter with Crew's owninitializeparams and reads every permission frame with Crew's ownbuild_permission_eventunder the launch's nonce. Stand-ins: the OS sandbox preflight/wrap (identity, so the lane needs no user namespaces), a loopback OpenAI-compatible model in pi'smodels.json(no credential), and a stdio stub MCP server mounted through_session_mcp_servers. Sevenreal_adaptertests:PROTOCOL_VERSION_PI,agentInfonames the pinned pi-acp;bashcall raisessession/request_permissioncarrying the envelope with the launch nonce, Crew's builder namesbashand its command, and the output reaches the model afteryes;noblocks the call (the file it would write is never created; pi reports the gate's denial);session/newarray) mounts the stub:tools/list+tools/call, gated first, title mapped back tomcp__probe-server__echo,ECHO:pingreaches the model;/compactis advertised, calls the model and ends the same prompt turn withCompaction completed.Plus two unmarked tests that run on every shard: the pi pin clears
PI_MIN_VERSION, and the scripted model only calls tools the request offers.The drive puts the lane's own
nodefirst on the child's PATH (a symlink in the isolated home's~/.local/bin, which leads Crew's PATH augmentation). Without it the first lane run refused at the read-back (exit 1: its configuration could not be parsed): pi's bin is#!/usr/bin/env node, andaugmented_pathputs/usr/local/binahead of the inherited PATH, where the runner image's system node is older than pi's>=22.19engine. A refusal now prints pi's own stderr and the resolved node version.The model listener binds through
socketserver.TCPServer.server_bind(the repo's no-getfqdnpattern,test_http_server_bind_ratchet)..github/workflows/ci.yml(real-adapter-contract only): assert the installed pi-acp/pi are the pins, passPI_ACP_BIN/PI_ACP_PI_COMMANDpointing at the lockfile install, collect the new file, raise the executed-contract floor 11 → 18.test/test_real_adapter_gate.py: the manifest/lockfile/no-literal-pin checks cover the two pi packages (the no-literal check is lane-scoped for pi because1.3.0also appears as an unrelatedbuild==pin elsewhere in ci.yml)..github/dependency-review-config.yml: one per-package exception,pkg:npm/bowser, the same layer the opencode platform binaries use. bowser's manifest says MIT; GitHub reads its LICENSE asMIT AND MITNFA(MIT plus the "no false attribs" clause). It arrives only through@aws-sdk/corein pi's tree, installed solely by this CI lane; nothing ships it.Backwards compatibility
Compatible: test and CI-lane additions only; no product code changes.
Removes nothing: the lockfile keeps every package version it held (checked), and no test, step or pin was dropped.
Tests
Mutation proof on the final head (each applied locally to the installed pin, the pi tests re-run, then reverted):
PI_ACP_PI_COMMAND(spawns plainpi)pifirst on PATH (CI's shape)yesitself)message(envelope dropped)get_commands(probe gone)registerMcpServerAlso run:
test/test_real_adapter_gate.py,test/test_real_adapter_fixture_pins.py(pass);HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.pyand--test(pass);check_file_line_cap,check_subprocess_encoding,check_sync_io_in_async,check_agent_sdk_boundary(pass); semgrep 1.78 with the repo rules + p/python, p/security-audit, p/secrets, p/owasp-top-ten on the changed files (clean); black, flake8, mypy on the new file (clean).Manual verification
This host refuses user namespaces, so a full sandboxed turn through the gateway is not reachable here; the tests stand in for the OS sandbox (preflight + argv wrap) and say so in the module docstring. Everything above the sandbox runs for real. No i18n strings touched.
CI evidence: the Real Adapter Contract Tests job ran all 7 pi tests green on head
430abe41ba("31 real-adapter contract tests executed, none skipped"; installed pins asserted aspi-acp 0.0.34, pi 1.3.0).Observation, not changed here: the node finding above is product-shaped. Any pi session launched where an old
nodesits in one ofaugmented_path's leading directories (/usr/local/bin,/opt/homebrew/bin, ...) runs pi under that node, and the read-back refusal names only a "configuration could not be parsed" class.Related Issues
N/A
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.