Repository navigation
fix(acp): stop retrying a sandbox-init failure and name the failing layer - #12144
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/acp/runtime.py:6605 -- “PRE-INITIALIZE” contradicts the False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the classified error and corroborated switch on a real refusing Linux host — all 55 tests drive synthetic stderr, none a live launcher refusal. [DESIGN-REVIEWED] 30fce44 |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify one drift: the new spec hunk says the latch "is spent when What this change shipsInventory (10 items) — 10 justifiedIntent: stop crash-looping a deterministic OS-sandbox refusal and tell the operator which layer refused — a FIX (Taskei Mesh-2998;
Reuse verified: [FIRST-PRINCIPLES-REVIEWED] 30fce44 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsBoth candidates fail the falsification bar. Candidate 1 (client.py:270 corroboration gate): The claimed defect is that Candidate 2 (sandbox.py:838 governance wording): This is a "add a test that drives the governance path" request — a missing-tests finding, a category this pipeline rejects deterministically. The underlying claim ("governance can clamp No groundable new defect surfaced while falsifying. No findings. [OPUS-REVIEWED] 30fce44 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
a510ac3 to
5ddad3c
Compare
5ddad3c to
3e5e516
Compare
3e5e516 to
d422214
Compare
d422214 to
09888e3
Compare
b8d1a60 to
0543990
Compare
0543990 to
431c3cf
Compare
431c3cf to
8b928e4
Compare
8b928e4 to
e61267a
Compare
…ayer A host whose OS sandbox cannot be built refuses identically on every attempt, but the ACP spawn/init path treated it as a transport fault: the reconnect ladder respawned, the same refusal came back, and the operator saw a generic process-exit card while the cause sat on the child's stderr. Both ACP transports now detect the sandbox-init signature in the spawn/init window and raise AcpSandboxInitFailed -- non-retryable for the same reason AcpAuthRequired and AcpToolGateUnroutable are, with transient fixed False so every retry ladder stops on the first one. The error carries the layer that refused and that layer's own switch, read off the argv the wrap returned rather than guessed from the signature: a harness sandbox nested inside Crew's wrap fails with the harness's wording while the layer to change is Crew's. No automatic downgrade to an unconfined spawn: the fix is classification plus fail-fast plus an actionable remedy.
e61267a to
30fce44
Compare
|
bolichen97
left a comment
There was a problem hiding this comment.
Reviewed against the pr-triage-pipeline gates plus a full-diff security audit of client.py, runtime.py, providers/acp.py and sandbox.py.
fix(acp), 8 files. Classifies a host-level sandbox-init refusal as non-retryable (AcpSandboxInitFailed, transient=False) so ensure_ready stops re-attempting, and names which layer refused (kirocrew vs harness-internal) instead of reporting a generic spawn failure.
This PR hits the security-path denylist (src/kiro_crew/sandbox*) and answers three checklist questions YES (touches_sandbox_or_confinement, modifies_safety_override_or_gate, introduces_external_input_parsing), so it is a security-flagged review, not a routine fix.
[Security: the risk the gate exists to catch is not realized; the change is a net hardening. Enforcement is untouched, only reporting and guidance change. There is NO automatic downgrade to an unconfined spawn, and a sandbox that will not initialize still makes the agent fail fast rather than falling through unconfined. Retry removal is anchored on a sandbox-token signature, so a bare 'Operation not permitted' or 'Failed to spawn child process' does not match, and launcher refusals are classified only for the no_backend kind while readiness-handshake and hardlinked-credential refusals stay retryable. The latch is scoped to the pre-first-session startup window and cleared at _finish_create_session, so a mid-life tool-subprocess refusal cannot permanently mark a later unrelated death as sandbox-broken; a control death still gets its retry. The sandbox-disable guidance text is surfaced only on a corroborated verdict from corroborate_launcher_refusal, which re-runs the real launcher around a trusted no-op, so a planted binary printing a fake refusal cannot talk an operator out of isolation; an uncorroborated verdict names the layer and offers no switch, and the harness layer never gets one. Child stderr reaching the session card, security event log or cron last_error passes through redact_exfiltration_urls and redact_credentials, and the remediation text names only the layer plus at most a pre-existing config key, so no host paths or secrets leak.]
Spec ride-along: docs/system-specs/modules/security.md changed as a minority of the diff (1 file / 63 lines against 7 files / 1610 lines) and was not reviewed as a design decision.
A broken OS sandbox now fails on the first try and tells you which layer to turn off, instead of crash-looping behind a generic "process exited".
What changed
AcpClient.ensure_ready)transientverdict on the errorNone) → wording fallback retried itFalse→ every retry ladder stopsAcpProcessDied/ bareAcpErrorAcpSandboxInitFailed, carryinglayer+remediationDetection and the classified type live in the ACP layer (
src/kiro_crew/acp/client.py,src/kiro_crew/acp/runtime.py); the shared-runtime startup paths that translate a failed spawn are insrc/kiro_crew/providers/acp.pyand read one shared verdict; and the facts a message needs — which layer wrapped a spawn, whether a trusted run agrees, and what each layer's switch is — are answered bysrc/kiro_crew/sandbox.py, which already declares both switches and the corroboration. The per-turn session-provider path is deliberately untouched: the latch is spent by the time it could read one.Signatures matched:
sandbox initialization failed,sandbox_apply,sandbox-exec:, and Crew's Linux launcher refusing after the spawn — the last classified bysandbox.launcher_refusal's ownno_backendverdict rather than by its prefixes, because that function deliberately answers "not a sandbox failure" for launcher lines that are retryable (a failed readiness handshake) or that mean the sandbox worked (the hardlinked-credential guard).Why
A host whose sandbox cannot be built refuses the same way every time. The reconnect budget exists for transport faults, so spending it here buys nothing and hides the cause: the operator saw the agent process exit while the reason sat one line earlier on the child's stderr, and every chat entry crash-looped.
meshclaw diagnosticsreported healthy.The repo already had the right shape for this.
AcpAuthRequiredandAcpToolGateUnroutableare non-retryable because respawning hits the same wall; this is the third member of that family. Reported in Taskei Mesh-2998.Why the layer is not derived from the signature
Identical stderr, opposite remedies. A harness's internal sandbox nested inside Crew's wrap fails with the harness's wording while the layer the operator must change is Crew's — which is how the field reports were actually resolved. So the layer comes from
sandbox.wrapped_by_crew_sandbox(), reading the argv the wrap returned: the wrap's own record of the branch it took. A re-derivation from mode + platform + settings cannot match it, because the delegated branch still falls back to Crew's seatbelt for a masked spawn and the audit-or-deny step can refuse a delegation after it was chosen.Each layer's remedy is read off the module that declares its switch, never spelled at the raise site, so a rename cannot leave a stale path in an operator-facing message.
Why the detector needs a sandbox token
The reported burst is three lines, and only the first is sandbox-specific; the other two are
Failed to spawn child processand a bareOperation not permitted. Those are ordinary output for a missing binary, a bad interpreter or a denied credential file, and matching either alone would latch a permanent verdict onto a class of deaths a respawn legitimately fixes. Every alternative is anchored on a sandbox token, which costs no coverage because whichever layer refused prints one.Why the disable switch needs a trusted run
The signature arrives on the dead child's stderr, and that child is the unverified binary the sandbox exists to contain. Answering it with
kirocrew config set agent.sandbox offwould let a planted binary print one line and have Kiro Crew instruct the operator to remove the isolation it is running under.sandbox.launcher_refusalstates this hazard for its own callers and prescribes the answer, so that is the answer used:corroborate_launcher_refusalre-runs the real launcher around a trusted no-op under the refused spawn's own mode and masks, and only its stderr — text no child wrote — unlocks the switch, and only on itsno_backendverdict: the same function reportstransientfor a failed readiness handshake that self-heals on the next spawn. Off the event loop, as the first-run gate runs it.Uncorroborated, the message names the layer (that comes from our own argv, not from the child), says plainly that Kiro Crew has not confirmed the failure on this host, and stops — naming no switch and no check, on either layer. The harness branch never names its key at all, corroborated or not, and that is the sharper case for the rule rather than the looser one. Two reasons stack: the layer being blamed is the agent's own sandbox, so naming its key would let a planted binary talk the operator into removing the only isolation that child had — and corroboration re-runs Crew's launcher, so its verdict is evidence about Crew's layer and says nothing about the harness's. Letting it unlock the other layer's switch would be a cross-layer inference whose one confirmed outcome is that isolation is gone. And no check is named because corroboration covers Crew's Linux launcher only; on macOS the probe validates an
(allow default)profile while the real wrap applies the strict one, so sending the operator there would false-green exactly the failure they are looking at. That gap closes with the real-wrap self-test, tracked separately.One consequence worth stating, because it is easy to get wrong: the stderr an operator reads and the stderr a classifier matches are different shapes. The shared runtime's retained summary folds the tail onto one line behind a
returncodeprefix, and the launcher's refusal is recognised per line — so the two travel separately. Folding them together would leave corroboration silently never running on the very path the Linux launcher case arrives by.Why the drain is settled twice, in two different places
The latch is written by the
_drain_stderrtask, while the death that brings a caller to the classifier is discovered on the stdout side —_mark_deadfails the pendinginitializesynchronously. Both are runnable at once, which is the ordinary shape of a real refusal: the child writes its signature and closes stdout together. So the read settles the drain first, bounded and swallowing, the same patternAcpClientalready applies at its own EOF.That is necessary but not sufficient on the shared runtime, because its failed-handshake cleanup tears the runtime down and the teardown cancels the drain. A line still in the pipe then is a line nobody ever reads, and the caller's settle finds the task already done and learns nothing. So the cleanup settles before it tears down — the last moment the child's own account of why it could not start is still reachable. Without both, a genuine refusal reports as a generic repeatable crash on whichever fraction of runs loses the schedule.
Why the latch is a startup verdict, and where startup ends
The auth latch beside it is life-long, correctly: a rejected credential does not un-reject itself. This one is not that. The harness's own sandbox can refuse when it spawns a tool subprocess long after the agent started fine, and a life-long latch would let that make the next unrelated death permanently "your sandbox is broken".
Startup ends at the first session handle, not at the
initializehandshake — the handshake is the middle of startup.session/newstarts the child's own MCP servers, and a sandboxed launcher among them can refuse there; a window closing at the handshake would leave every such refusal unclassified on exactly the translation site (create_session) that exists to catch it. So the latch arms until the first handle exists and is cleared there. Clearing rather than merely un-arming is what makes every consumer correct by construction: the three startup translations read it while startup is unfinished, and any path reached after it can only ever seeFalse— which is why the per-turn translation carries no sandbox branch at all.Why no auto-downgrade
Falling back to an unconfined spawn would turn a broken host into a silently unsandboxed agent. The fix is classification, fail-fast, and an actionable remedy — stated with what turning the layer off costs.
Relation to #7396
Complementary, no overlap. #7396 makes the cron retry decision read a type-carried verdict (
AcpProcessDied.resubmit_safe) instead of message wording; this adds a new type that any type-based retry treats as terminal, through thetransient=Falsechannel that already existed onAcpError. Neither touches the other's field or branch —AcpProcessDiedis untouched here, andAcpSandboxInitFailedis a sibling of it, not a subclass — so the two merge in either order. On #7396's own terms this is a death nobody should resubmit at all, which its unclassified-falls-back-to-wording default already routes correctly.Pattern harvest
Rule candidate: none — this is a missing classification, not a mechanical slip a matcher could have caught.
The generalizable lesson is about the shape of a deterministic failure reaching a retry ladder, and the repo already encodes it:
AcpAuthRequiredandAcpToolGateUnroutableexist because respawning re-reads the same answer. Each new deterministic refusal has to be named individually, so there is nothing to grep for.Two patterns are worth naming for reviewers rather than for a linter. A
MagicMockstands in for a collaborator that grew a predicate and answers truthy to whatever nobody stubbed — four test files here needed the new predicate answered explicitly, and a missed one would have silently asserted the new branch instead of the one it is about. And a classifier over untrusted subprocess output is not a verdict: this module already separates the two (launcher_refusalvscorroborate_launcher_refusal), and any new consumer of child stderr has to pick the right side of that line.Tested
55 tests in
test/test_acp_sandbox_init_classification.py: every signature classified and every control tail left retryable (bare errno, auth output, healthy startup, and each launcher line that carries a blocking prefix without being a sandbox failure); exactly one spawn attempt per refused-sandbox case, and a control death still getting its one retry; the switch withheld on an uncorroborated refusal and on a trusted run that clears the host, emitted on one that does not, and no trusted subprocess spent on a signature it could not speak to; the latch surviving a trimmed ring buffer, firing on a restricted session that retains no stderr, refusing a mid-life line, still catching one printed after the handshake, and spent at the first session handle so a live session's death is never a sandbox verdict; both layer attributions through the cold start; the losing schedule of the drain race reproduced deterministically and closed, a wedged drain still unable to hold the failure path open, the cleanup ordering that keeps a cancelled drain from discarding the evidence, and the display-vs-classify shapes staying separate so a folded summary cannot hide the launcher's line; and the shared retry predicate refusing the error on behalf of every consumer.Twenty-two mutations verified caught, one per decision: dropping the sandbox anchor, disabling the fail-fast, always attributing to Crew's layer, leaving
transientunclassified, matching launcher prefixes raw, latching for the runtime's whole life, leaving one startup site unconverted, handing out the switch without corroboration, accepting a transient trusted verdict as corroboration, ignoring the corroboration verdict entirely, never clearing the latch at the handshake, dropping the drain settle, making the settle not wait, making it wait without a bound, settling the drain after the teardown that cancels it instead of before, classifying the display summary instead of the lines, dropping the lines at the shared translation, naming the harness switch on any refusal, letting a Crew-launcher verdict unlock the harness switch, promising an unconfined spawn a governance floor can refuse, closing the startup window at the handshake, and never spending it at all.isort, black, flake8, mypy clean. Neighbouring suites green: 1300 across the ACP client, runtime, provider, session-provider and stderr-latch files, plus the brand, comment-history, harness-parity, agent-sdk-boundary, black-baseline and docs-lint gates. Spec rationale documented under the OS-Level Sandbox section of
docs/system-specs/modules/security.md.no linked issue: reported through Taskei, which has no GitHub issue in this repository.
Doctor real-wrap parity — making diagnostics exercise the strict generated profile rather than the allow-default probe — is deliberately out of scope and tracked separately.