Skip to content

feat(ci): cross-os end-to-end testing framework - #9585

Merged
bolichen97 merged 2 commits into
mainfrom
feat/cross-os-e2e-framework
Sep 12, 2026
Merged

bolichen97 merged 2 commits into
mainfrom
feat/cross-os-e2e-framework

Conversation

@bolichen97

@bolichen97 bolichen97 commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

CI had no job that booted a gateway on Windows or macOS. That is how PR #8117 shipped a Windows startup break with every unit shard green, reverted in 56f67aa43. Since 2026-07-01, 13 more Windows-only fixes landed after the break reached main, 5 of them the same lock-file truncation class and 4 the same junction class. Unit tests were modelling the platform instead of running on it.

This PR adds the end-to-end layers that run the product on all three OSes. Two commits.

Commit 1: feat(pod): add a windows task-scheduler backend

kirocrew pod gains a Windows backend (src/kiro_crew/pod/windows.py) over schtasks.exe, the per-user, no-elevation equivalent of systemd --user and launchd. Linux and macOS dispatch is unchanged. Limits are documented in src/kiro_crew/pod/README.md: no env block on a task (a generated .cmd re-enters kirocrew pod _run), no restart policy, no cgroup ceiling, and pod api refuses on Windows because CPython there has no AF_UNIX. test/test_pod_windows_boot.py boots a real pod on the Windows leg (pod-boot-windows job, pass count grepped), and the gateway child runs under the same Job object resource ceiling the cgroup path applies on Linux, installed on a CREATE_SUSPENDED process before its first instruction.

Commit 2: feat(ci): add a cross-os end-to-end testing framework

Layer What runs Where
Gateway boot matrix test/e2e/test_gateway_boot_matrix.py: real gateway per test against the packaged fake ACP backend; health, one completed prompt turn, both sandbox tiers, clean teardown; pass count grepped new e2e-boot-matrix job on ubuntu-latest, macos-15, windows-latest
Harness testing/harness.py reads READY via a reader thread and tears down via a process-tree kill, so it runs on Windows; test_harness.py is collected there again Windows shards
Windows fail-closed contract test/test_windows_fail_closed_optin.py: on a real win32 host, the official kiro backend delegates with no settings file present, and an unclassified spawn fails closed until sandbox_allow_unsandboxed_exec is set (the exact branch #8117 broke) new backend-test-windows-fail-closed job
Windows installer smoke scripts/smoke-windows-install.ps1: silent install of the built NSIS artifact on a clean runner, boot the installed gateway, health, uninstall build-windows.yml after artifact upload (nightly and dispatch); build.yml's PR-time installer job now installs the real wheel into a python-build-standalone runtime and boots the installed gateway against the fake backend, so -SkipGatewayValidation is gone
macOS full suite backend-test-macos moves from the deprecated macos-14 label and a 51-file glob to macos-15 running the whole suite in 3 shards, with test/macos-expected-failures.txt and test/macos-collect-ignore.txt through the same conftest mechanism as the Windows lists ci.yml
Pod scenario suite test/e2e/scenarios/: settings persistence, cron fire, a tool turn, service-unit rendering, wheel install, against a real kirocrew pod; pods accept the fake backend via KIROCREW_POD_KIRO_BIN new pod-scenarios job in nightly.yml on ubuntu-latest and macos-15, systemd --user via loginctl linger

The OS gap lists changed from skip to strict xfail

Declaring this because it is a semantic change to an existing mechanism, not just a
second copy of it. test/windows-expected-failures.txt and the new
test/macos-expected-failures.txt both call themselves burn-down backlogs whose
entries should be deleted as gaps close — but a skip never executes, so nothing
could ever report that a gap HAD closed and the lists could only grow. Under
xfail(strict=True) a listed test that still fails is green (the gap is known) and
one that starts PASSING fails the job with XPASS, which is the signal to delete the
line, delivered by the run that earned it. The first run under it found 23 entries
whose gaps had already closed.

Two consequences worth stating plainly. Listed tests now RUN, so the Windows shards
do strictly more work than before. And a listed test that HANGS must never be added
to either list, because executing it would hang the shard rather than skip it.

Plain pytest throughout. No BDD or infra-assertion framework: the repo's isolation, timeout and sharding story is pytest-shaped, and a second framework would need a second isolation story.

Tested

Linux, in the consolidated worktree: mypy src/kiro_crew clean; black, isort, flake8, check_black_formatting.py, check_subprocess_encoding.py, check_comment_history.py, docs-lint.sh and the brand gate clean; every changed test file plus the CI-surface pins pass; the 7 boot-matrix tests pass (37 s).

On this PR's own CI: the gateway boot matrix passes on ubuntu-latest, macos-15 and windows-latest; the Windows fail-closed contract job passes; the Windows installer builds a real wheel into python-build-standalone and boots it; the Pod Boot Canary boots a real pod under Task Scheduler on windows-latest and reclaims it; all three macOS full-suite shards finish inside the cap with the terminal handler suite green in a dedicated step; the four Windows shards now also collect test_harness.py and every pod Windows suite.

What booting on macOS and Windows caught

None of these were visible to the unit shards, and every one is fixed in this PR rather than deferred.

Product defects:

  • macOS gateway exited right after KIROCREW_READY: format_dashboard_urls called socket.gethostbyname on the event loop, the mDNS timeout tripped the loop watchdog. The lookup is bounded in a thread and the boot announcement runs off the loop.
  • The embedded terminal's _kill_session closed the controller fd before ending the child; on darwin a parked PTY read never wakes, so the handler hung and left an orphan shell. It now kills the group, waits bounded, joins the reader, then closes.
  • The same teardown could still wait forever after SIGKILL: a PTY child blocked in a kernel tty write cannot die until the controller is read or closed, and the unbounded proc.wait() sat ahead of the close. Every wait in the kill is bounded now.
  • macOS bash 3.2 printed the zsh deprecation banner at the top of every embedded terminal; _pty_child_env sets BASH_SILENCE_DEPRECATION_WARNING=1 (setdefault, a user value wins).
  • apple_speech refused the macOS runner's Xcode because the versioned bundle sits behind a symlink; the trust walk resolves the bundle prefix.
  • dev_fleet/repository.py had no arm for the EPERM a type-change produces on macOS.
  • pod_name_mutex was a no-op wherever fcntl was absent, so a concurrent pod up/pod down on Windows was unguarded; it locks through platform_compat.file_lock on both planes.
  • A pid record that could not be written let stop() delete a live gateway's task and home while reporting success; it is a boot failure now, with the child tree killed.
  • port_owner called the pod's own gateway foreign on Windows: pip's .exe launcher stub is the recorded pid while its child binds the port. The Windows leg accepts a listener among the recorded pid's descendants and nothing else.
  • cli_server._ensure_node spawned bash from inside a pod on Windows, which is WSL's launcher; win32 no longer spawns a POSIX shell there.
  • The pod seed, OS-home build and seed read-back refused every host without O_DIRECTORY/dir_fd; each has a win32 branch built on platform_compat.pin_directory with the residual stated in the docstring, the pod README and the security spec.
  • A pod's private API socket exceeded the AF_UNIX path limit under a deep plane root and the gateway silently fell back to TCP-only.
  • A booted pod could not receive the fake ACP backend because the service manager starts it from a clean environment; KIROCREW_POD_KIRO_BIN flows through the one env selection both backends serialise.
  • codex-review.yml used a BSD-incompatible sed.

Test-suite assumptions that only held on Linux:

  • test/e2e/scenarios/conftest.py shadowed the rootdir conftest for the 93 files that import it (about 120 errors per shard on every OS); both e2e directories are packages now.
  • KIROCREW_KIRO_BIN pointed at a .py, which CreateProcess refuses; the shared harness.fake_acp_backend_launcher writes a .cmd shim on Windows.
  • Tests that fake subprocess.Popen on the Windows pod backend must also pin process_start_time, which shells out to ps on darwin.
  • The 80-column PTY wraps a long probe mid-word on a runner with a long hostname; terminal assertions compare through an unwrap helper.
  • The O_BINARY simulation picked a bit that is O_DIRECTORY on darwin; the recorder strips the simulated bit before the real open.
  • The env-file torn-read test holds the file open across the write, which on Windows is a sharing violation; the win32 contract asserted is fail closed with the old generation intact.
  • The scenario fixture mutated the host before its try, and the boot canary wrote a stub SPA into the checkout; both are inside teardown now, the stub is placed by the workflow.

Research behind the design is archived at workspace/cross-os-e2e-research-2026-09-08/ (audit, framework landscape, harness designs).


Adversarial Windows review — findings fixed in ca09a05a

An adversarial review of this PR's Windows surface (12 units, each required to
measure rather than reason) found that the backend answers "is this pod alive"
and "is this process mine" from data that cannot answer them, and that several of
the new gates could not go red on the thing they exist to catch. GPT 5.6's
blocking F1 turned out to be a third path into the same root cause and is fixed,
not overridden.

Three claims in the section above were contradicted by measurement. Correcting
them here rather than silently editing the list, so the record shows what changed:

  1. "A pod's private API socket exceeded the AF_UNIX path limit under a deep
    plane root and the gateway silently fell back to TCP-only."
    The PRODUCT is
    unchanged: dashboard/server.py's _start_unix_site still wraps the bind in
    one except Exception, logs one warning and degrades to TCP-only, with no
    length pre-check and no refusal. What this PR added is a short plane root in
    the E2E fixture. An operator with a deep KIROCREW_POD_ROOT still gets a pod
    that boots, reports healthy, and whose pod api refuses forever with text that
    never mentions path length. Left as a stated gap rather than quietly claimed.
  2. The win32 seeding residual in security.md. It said a same-UID component
    swap "the re-witness detects but cannot undo". Both halves were false: a swap
    after the pin is refused by the kernel, and the swap that works is undetected,
    because a witness taken before the pin can only be compared against itself.
    Corrected, and the seed branches now pin the anchor before creating anything so
    the window is as small as the platform allows.
  3. The pod-scenario matrix defers Windows until "a Windows backend lands
    (feat/pod-windows-backend)".
    That backend lands in this PR's own first
    commit, so the comment and the doc were stale on arrival. Nothing pins the
    matrix, so it will not self-correct.

pod/README.md's "the ceiling is a Job object — and it IS enforced" is likewise
softened to enforced-when-attachable-and-loud-when-not, which is what
apply_job_limits actually contracts (it returns False on any Win32 failure and
the caller treats that as no ceiling).

Revert granularity (Design Review's Watch)

This PR carries runtime product fixes inside the feat(ci) commit, and the review
is right that the coupling is real. Recording the consequence explicitly, which is
what that Watch asks for: reverting the feat(ci) commit alone reintroduces the
macOS gateway dying right after KIROCREW_READY, the embedded-terminal orphan
shell hang, and the Windows liveness/reclaim defects fixed in ca09a05a.
A
revert must cherry-pick those runtime fixes back first; they are enumerated in the
"What booting on macOS and Windows caught" section above and in ca09a05a's
message.

CI cost of the expected-failures conversion

The gap lists applied pytest.mark.skip, so a listed test never ran and nothing
could report that a gap had CLOSED — on files whose own headers say "fix the test
and DELETE the line". They are strict xfails now. The first run burned 23 stale
entries off the Windows list (203 → 180 entries: those gaps were already fixed).
The cost is that 180 previously-skipped tests now EXECUTE on the Windows shards:
115s single-process locally, which rides the shards' own -n auto. One entry had
to be excluded and skipped at the test instead — a hypothesis property test with
deadline=None whose example budget outruns the per-test --timeout, and on
Windows pytest-timeout kills the process rather than the test, so a hanging entry
in an executed list is a shard-killer.

Pattern harvest

Rule candidate: on Windows, a process's creation-time token answers IDENTITY and
never LIVENESS — the process object outlives the process while any handle to it is
open, so process_start_time(pid) == token keeps matching a corpse. Ask
pid_exists first and let the token only narrow a live pid to the right process.
This PR contained three independent instances of the inversion (supervised_pid,
stop's survivor re-check, and the boot canary's own assertion), which is what
makes it a rule rather than a bug.

Rule candidate: a gate whose pass condition is an unanchored substring match on a
count (grep -q "3 passed") pins nothing, because it also matches every larger
number. Anchor it (grep -qE '(^|[^0-9])3 passed'), and where the count legitimately
moves, compare it numerically against a floor instead of matching a range that a
collapse to one test satisfies.

@bolichen97
bolichen97 requested review from a team and CrysisDeu September 8, 2026 23:41
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of d22ce973d9eb5ee7f0f2083969360b900693a058 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

A measured gap (#8117 plus 13 post-break fixes), closed at the right layer — real boots per OS, with pins that can go red and residuals stated, not claimed closed.

Suggestions

  • The PENDING_VALIDATION windows-latest entry in test/test_pod_scenario_matrix.py clears only after "one green run" that nothing schedules — dispatch that validation run promptly (branch-edited matrix) so the deferral burns down instead of fossilizing behind a well-written reason.
  • Description↔diff: the bullet "codex-review.yml used a BSD-incompatible sed" has no backing hunk in this branch (the file is untouched); delete or re-attribute it so the PR record doesn't claim a fix it doesn't carry.

[DESIGN-REVIEWED] d22ce97

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 28130e68befedb0926020d4bda527384ccbd1cca — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

I've now read the contract, the intent file, and the patch (workflows, scripts, conftest, product code, new pod backend, gap lists, and the modified test pins — the large test diffs in test_denied_commands_security.py etc. are black-baseline reformatting only), and counted consumers for the new surface in the repo at HEAD. Final review:

First-Principles-Verdict: CONCERNS

-MinStartupPycs generalizes for a caller that doesn't exist — one invocation ever, passing 750 — and -SkipGatewayValidation keeps zero callers after this PR.

Not justified as shipped

  • test-windows-installer.ps1 gains a -MinStartupPycs parameter (default 1000) — one consumer, generalized: grepped MinStartupPycs, the sole invocation is build.yml:226 passing 750; no caller ever exercises the 1000 default, which exists for a hypothetical "caller that bundles a narrower payload."

What this change ships

Inventory (10 items) — 9 justified

Intent: make CI run the product for real on all three OSes — boot gateways, install artifacts, run pods — so platform-only startup breaks like #8117 (provenance: revert 56f67aa plus 13 counted follow-up fixes) cannot land green. ADDITION. More than 10 observable differences; the 10 most visible kept.

  1. kirocrew pod now works on Windows via Task Scheduler; pod api refuses there — justified
  2. Every PR now boots a real gateway on ubuntu, macos-15 and windows (e2e-boot-matrix, harness made cross-platform) — justified
  3. The exact fix(security): sandbox verify + UI-stream timeout + mem-info gate #8117 fail-closed branch runs per PR on a real Windows host (backend-test-windows-fail-closed) — justified
  4. A real Windows pod boots per PR under Task Scheduler (pod-boot-windows canary) — justified
  5. The PR-time installer job bundles a real interpreter + wheel and boots the installed gateway — justified
  6. test-windows-installer.ps1 gains -MinStartupPycs (default 1000) — one consumer, generalized: only build.yml:226, always 750
  7. Nightly signed-installer smoke on a clean runner, gated on artifact_uploaded — justified
  8. macOS CI: 51-file glob on deprecated macos-14 → full suite, 3 shards, macos-15, with its own gap lists — justified
  9. OS gap lists execute as strict xfail instead of skip; 23 already-closed entries deleted — justified
  10. ~14 product defects the new runs exposed are fixed in place (bounded mDNS lookup, bounded PTY reaps, Xcode symlink trust, win32 pod mutex/pid/port-owner fixes, seed win32 branch, …) — justified

Watch

  • The framework's Windows scenario leg is a promise, not a run: nightly.yml's pod-scenarios matrix is [ubuntu-latest, macos-15] and the 1650-line Task Scheduler backend's only CI consumer today is the 3-test boot canary. The deferral is mechanized (test_pod_scenario_matrix.py::PENDING_VALIDATION), but nothing forces the validated run to happen. Clears when: one green pod-scenarios run on windows-latest lands with the PENDING_VALIDATION entry deleted and the matrix entry added.

Subtractions

  • Drop the -MinStartupPycs parameter from .github/scripts/test-windows-installer.ps1 and inline the measured 750 floor (1 consumer: build.yml:226); adjust the packaging.test.js pin this PR ships for it.
  • Delete the -SkipGatewayValidation switch and its branch (test-windows-installer.ps1:2, :269) — grepped: 0 invocations pass it after this PR, and test_windows_installer_doc_contract.py already pins that CI must not.

[FIRST-PRINCIPLES-REVIEWED] 28130e6

@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed d22ce973d9eb5ee7f0f2083969360b900693a058 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d22ce97

Verdict parsed from the review's SHA-scoped output markers for commit d22ce973d9eb5ee7f0f2083969360b900693a058.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d22ce973d9eb5ee7f0f2083969360b900693a058: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the feat/cross-os-e2e-framework branch from a83fa88 to c0f7e02 Compare September 9, 2026 00:05
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of d22ce973d9eb5ee7f0f2083969360b900693a058 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- scripts/smoke-windows-install.ps1:355 -- -s -m writes __pycache__ into the installed payload after the ACP spawn strips PYTHONPYCACHEPREFIX, causing the uninstall residue check to fail -> Fix: add -B to the fake-backend invocation.
[GPT-REVIEWED] d22ce97

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d22ce973d9eb5ee7f0f2083969360b900693a058: <one-sentence reason>

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 9, 2026
@bolichen97
bolichen97 force-pushed the feat/cross-os-e2e-framework branch from c0f7e02 to 756400f Compare September 9, 2026 00:50
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 9, 2026
@bolichen97
bolichen97 force-pushed the feat/cross-os-e2e-framework branch from 756400f to 0d76f11 Compare September 9, 2026 00:54
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 9, 2026
@bolichen97
bolichen97 force-pushed the feat/cross-os-e2e-framework branch from 0d76f11 to 787649f Compare September 9, 2026 01:19
@bolichen97
bolichen97 requested a review from a team September 9, 2026 01:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — ✅ PASS

UX-level review of d22ce973d9eb5ee7f0f2083969360b900693a058 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

The PR is entirely backend, CI, and test infrastructure: a Windows Task Scheduler pod backend, an E2E boot matrix, harness changes, and bounded hostname resolution. No website/src change, no dashboard control, label, or state is added or changed — the only rendered-UI-adjacent touch is suppressing macOS's Bash deprecation banner inside the terminal panel's child environment, which removes noise rather than adding an element. The new CLI refusal strings (the pod api Windows refusal, the schtasks policy error) assert what happened, rule out the wrong fix, and name the working path. With no user-visible control in the diff, the absent blind read and screenshots are not evidence gaps, and no lens-13 transition exists.

UX-Verdict: PASS

No user-visible surface changes — CI/backend only; the new CLI refusal copy states cause and next step, and the terminal panel loses noise, not affordance.

[UX-REVIEWED] d22ce97

@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Sep 9, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Sep 9, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Re GPT 5.6 finding windows.py pid record (agent-writable pods_dir → pod down kills an unrelated process) — not fixable at this layer, and the record grants no capability the actor does not already have:

  • The threat requires an agent process that can write into cfg.pods_dir (the plane root, a sibling of the pod's KIROCREW_HOME, not inside it). On Windows there is no OS sandbox layer at all: agent subprocesses are refused unless the operator sets sandbox_allow_unsandboxed_exec, and with that opt-in the agent runs unconfined — it can call taskkill /PID <any> directly. Forging a .winpid to get pod down to do the kill for it adds nothing. On Linux/macOS the pid is not read from a file (systemd/launchctl are asked), so the record exists only on the platform where a sandbox-hidden location cannot exist.
  • What the record IS defended against is the realistic failure, a stale or corrupted record: supervised_pid requires the creation-time token to match the live process, and a live pid whose identity does not match is a refused stop (test_stop_refuses_when_a_recorded_pid_is_alive_but_unattributable), never a kill.

"Store it in sandbox-hidden host state" presumes a sandbox that Windows does not have; the doc in docs/guides/windows-install.md already states the fail-closed contract this rests on.

@bolichen97

bolichen97 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Reasoning kept for the record. The finding this addressed is absent from the lane's output on the current head, so this is commentary rather than a disposition record.

Disposition — GPT 5.6 F1

  • Untrusted PID record makes teardown fail open — the confirmed leg is FIXED in 4f57ce3b2; the second leg is not fixable at this layer, for reasons the codebase states explicitly.

Leg 2 (successor-record failure) — fixed, and it was mine. The adjudicator confirmed this one end to end, including that the code's own remedy was the harm trigger: on an OSError writing the successor's record, supervise_gateway broke, the finally cleared the record, and the successor kept serving under a pid no verb could see — after which the advised pod down would rmtree its isolated HOME. That advice is gone because the situation it advised on is gone: an unrecordable successor is now ENDED, via a tree kill pinned to the creation time read for the adoption, so a pid recycled between the sighting and the kill cannot be signalled. The same applies to a successor whose creation time cannot be read at all — existence is re-probed, and a process that is still there is ended rather than abandoned. Cost is one lost restart, which a boot recovers; pod down is then safe to advise because it acts on a pod that really is gone. Pinned by test_a_successor_that_cannot_be_recorded_is_ended_not_left_serving.

Leg 1 (the record is deleted by something with write access) — no safe fix exists at this layer. The adjudicator did not confirm it ("would require confirming the sandbox does not fence cfg.pods_dir, which I did not open"). Measured: pods_dir resolves under ~/.kiro/crew/, and Windows has no OS sandbox at all — unsandboxed_exec_platform_default() is sys.platform == "win32" precisely because there is no native wrapper to apply. So the premise holds no more for .winpid than for the config, the session store or the memory store on that platform; it is a property of the platform's posture, not of this PR.

The suggested remedy ("refuse teardown when task liveness is unknown") was implemented and then withdrawn, because every available liveness signal is worse than the gap:

  1. The task's Status column is LOCALIZED. Keying on it reports every pod down on a non-English Windows, which is the fail-OPEN direction — is_active documents this as the reason liveness comes from the record.
  2. The task's EXISTENCE is the normal state of a pod that crashed or exited without a down. Refusing there would break every legitimate cleanup, which is a regression, not a hardening.
  3. The PORT is forbidden, in writing. _port_is_free's docstring: "PRIVATE, and it must stay private. instances/run_marker states the rule: no 'is something listening' helper may be offered, because a caller will mistake reachability for identity — and pod's own health probe was held to that rule for exactly this reason. This function is not exempt by being inverted." A busy port does not prove THIS pod is alive, which is the identity question teardown actually needs.

Storing the record outside agent-writable state is the real fix and it is a platform-posture change (where pod state lives, on every platform), not something to land inside a CI-framework PR. Recorded as a residual rather than silently closed.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging 4f57ce3b28076cd19ce9c00523c2aaa371a9ca50. Adjudication produced no [GPT-ADJUDICATED] marker for 4f57ce3: the adjudication stage FAILED TO RUN (its raw output is in the Adjudication details block -- typically an API or setup error), so no proportionality review took place and GPT's blocking verdict stands unreviewed.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/pod/windows.py:1049 -- Restart tracking remains anchored to the original process
successor := _await_successor(gateway_pid_record, proc.pid, spawn_token)
Second restart -> dead intermediate removes the tree bridge to its child -> supervisor clears liveness record -> pod down can delete the live gateway’s home.
Anchor: residual/crash-data-loss-corruption
Fix: advance the reaped PID and creation token after adopting each successor.
[BLOCK-MERGE] 4f57ce3
[GPT-REVIEWED] 4f57ce3
False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4f57ce3b28076cd19ce9c00523c2aaa371a9ca50: <one-sentence reason>

@bolichen97

bolichen97 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Reasoning kept for the record. The finding this addressed is absent from the lane's output on the current head, so this is commentary rather than a disposition record.

Disposition — GPT 5.6, unit-prefix path escape

  • Unit prefix can escape the pod directory and overwrite files — not introduced by this PR, and not fixable on one platform without becoming theatre. Documented at the site instead; the adjudicator reached the same conclusion independently.

Every claim below was checked in code, not taken from the review.

It is pre-existing, and it is not Windows-specific. KIROCREW_POD_UNIT_PREFIX is read with no validation at all — config.py:227 is os.environ.get("KIROCREW_POD_UNIT_PREFIX", DEFAULT_UNIT_PREFIX) — and that line is in main, untouched by this PR. All three backends then splice the value: systemd builds a filesystem path from it at unit.py:123 (~/.config/systemd/user/<prefix>@.service) and a unit name at runtime.py:793, launchd builds its label at launchd.py:138, and Windows builds the sidecar path this finding names. A rooted prefix escapes on Linux in main exactly as it does here, so validating only the Windows write site would move the finding, not close it.

The escape is self-directed and crosses no boundary. It requires the operator running pod up to set their own environment variable to a rooted path, and the file it truncates is one that identity can already write. That is the same trust level the rest of the pod config surface already has by design: KIROCREW_POD_ROOT and KIROCREW_POD_ENV_DIR point pod state anywhere the operator can write, and nothing validates those either. Opus 4.8's adjudication reached this independently and FLAGGED the finding (ADJUDICATION-FENCED … flagged=1); the blocking verdict stands only because security-class findings are withheld from automatic downgrade, not because the reasoning was rejected.

What the finding did catch is a documentation gap, and that is fixed. task_name's docstring explained why splicing name is legal (it has been through validate_name) and said nothing about the prefix, so the prefix's trust was implicit — which is precisely how a reviewer, human or model, ends up re-deriving this every round. _plane_file now states the trust boundary explicitly: which of the two interpolated values is validated, which is not, why not, and where a constraint would have to live to be real.

If the repo does want that surface constrained, the shape is a validator at config load covering unit_prefix, pod_root and pods_dir together, on every platform, with a refusal an operator can act on. That is a coherent change with its own blast radius — it can break a plane whose prefix or root is deliberately unusual — and it does not belong inside a CI-framework PR. Recorded as a residual rather than silently closed.

@bolichen97

bolichen97 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Reasoning kept for the record. The finding this addressed is absent from the lane's output on the current head, so this is commentary rather than a disposition record.

Disposition — GPT 5.6, handoff published after liveness disappears

  • Restart handoff is published after liveness disappears — narrowed to zero intervening statements, which is as small as this liveness contract allows; the remainder needs an atomic pod-state record and is recorded as a residual rather than silently closed.

The finding is correct, and it is a refinement of a fix from two pushes ago. _begin_handoff runs after proc.wait() returns, and liveness is derived from pid_exists on the RECORDED pid — so it has already flipped by the time control returns to Python. Teardown arriving inside that gap observes neither a live record nor a marker.

What changed in response. The marker is now the FIRST statement after the reap: measured, there are zero executable statements between rc = proc.wait() and _begin_handoff(cfg, name), where there was one before. Nothing placed after wait() can be earlier than that.

Why the remainder is not closable at this layer. Publishing the marker BEFORE the wait would make it present for the pod's entire lifetime, which defeats the freshness bound — and that bound is not decoration: a supervisor killed mid-handoff cannot retract its own marker, so without it a dead supervisor would make pod down refuse forever, trading a one-statement window for a permanent inability to reclaim the pod. Making the transition genuinely atomic therefore means liveness and handoff becoming ONE record that flips in a single filesystem operation — i.e. replacing the pid record with a pod-state file, on all three backends, since supervised_pid / is_active / unit_state / stop and the launchd and systemd equivalents all read it. That is a change to the pod's on-disk contract and does not belong inside a CI-framework PR.

Proportion, for the record. Two pushes ago this window was the FULL _await_successor wait — up to SUCCESSOR_ADOPT_TIMEOUT_SECS (30s) — and before the adoption loop existed at all it was unbounded: the finally cleared the record permanently and the successor was never recorded. The sequence is 30s → one statement → zero statements. What is left is a gap between a process dying and its supervisor's next instruction, on a path that also requires a concurrent pod down to land inside it.

The three findings before this one in the same area were fixed outright (the successor that could not be recorded, the anchor that did not advance, the handoff window itself), so this is not a pattern of deferral — it is the point where the next increment costs a contract change.

@bolichen97

bolichen97 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author
  • Task Scheduler executes an agent-writable wrapper (span=738db090c061) — measured on Windows 11, and there is no privilege boundary for the wrapper to cross. schtasks /Create here passes neither /RL nor /RU, so a task registered exactly as this code registers one comes back from schtasks /Query /XML with no <RunLevel> element at all (the default, LeastPrivilege), <UserId> set to the calling user's own SID, and <LogonType>InteractiveToken — no stored credential, no elevation, same integrity level. So the identity that could rewrite the .cmd is the identity the task already runs as: anything able to write that file is already executing as that user and can run code directly, without touching the wrapper.

    The prescribed fix — "register an action whose executable content cannot be modified by the agent" — is not reachable between two identities that are the same one. Pointing the action at an interpreter plus a read-only file moves the problem rather than solving it: the same user owns that file and can restore write access to it. The module states this contract in its own text ("the calling user's own namespace with no elevation") and the gate's refusal message says it to the operator ("Pods are per-user scheduled tasks and never elevate"), so the wrapper is self-directed configuration on the same footing as pod_root, pods_dir and unit_prefix, each of which every backend already splices unvalidated because the whole pod config surface points pod state anywhere the operator's own identity can write.

    What WOULD make this a real escalation is a task registered with /RL HIGHEST or under a different principal via /RU, where a lower-integrity writer feeds a higher-integrity runner. Neither appears anywhere in this backend, and adding one is the change that would need this finding's fix. Constraining the pod config surface itself is a defensible separate change — at config load, for all three backends at once — and does not belong to a CI framework PR.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for ee07885bbfffe1ec344cde8e6db87fea107a00d2; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt ee07885bbfffe1ec344cde8e6db87fea107a00d2: <one-sentence reason>

`kirocrew pod` had two backends (systemd --user on Linux, launchd on
macOS) and refused on Windows. This adds the third so a Windows
developer can pod a worktree, and leaves the Linux and macOS paths
byte-identical: every dispatch site gained an `elif IS_WINDOWS:` beside
the existing `if IS_MACOS:`, and no shared branch changed.

Task Scheduler, not a service. A pod is per-user and disposable, so
`sc.exe create` is wrong twice over: it needs an administrator right and
it installs a machine-wide LocalSystem service, which would stop the pod
running as the user whose data it is isolating. `schtasks.exe` creates a
task in the caller's own namespace with no elevation, matching
`systemd --user` and launchd's `gui/<uid>`. A detached supervised process
was the documented fallback and is not needed.

Five consequences, each stated rather than papered over:

- A task carries no environment block, so the pod plane the CLI resolved
  would be lost. The task's action is a generated `.cmd` that sets the
  plane from the same `config.environment_vars` selection both other
  backends serialise, then re-enters `kirocrew pod _run <name>`. Boot
  logic stays in Python. A plane path holding a double quote or a
  newline is refused at `pod up`, because cmd.exe cannot express it.
- There is no restart policy, so a crashed pod stays down. That removes
  launchd's restart-loop hazard entirely (a terminal refusal keeps its
  honest exit code, no exit-0 translation) but means the crash signal
  `pod up` waits on has to be recorded: the wrapper writes the boot's
  exit code beside the task and `unit_state` reads it.
- `schtasks /Query` reports no pid, and Windows has no `exec` (CPython's
  `os.execve` there spawns and terminates the caller). So `boot`
  supervises the gateway as the wrapper's child and records its pid plus
  its creation-time identity, which restores the invariant `port_owner`
  rests on: the recorded pid IS the process that bound the port. It stays
  independent of the gateway's own PID sidecar.
- `schtasks` output is localized in both its CSV headers and its status
  words, so this backend never parses it. Keying on `Status == "Running"`
  would report every pod down on a non-English host, and that is the
  fail-OPEN direction, where teardown deletes a live pod's HOME.
  `schtasks` is used only where its exit code is the answer.
- There are no cgroups, so the memory and CPU ceiling is NOT enforced.
  Same gap as macOS, and a weaker knob that read as the 4G cap is
  deliberately not emitted.

`port_owner` no longer refuses non-POSIX hosts. That gate was
unsatisfiable rather than strict: `pod up` mints a token and
`mint_token` requires positive proof, so a healthy Windows pod could
never earn its own credential. Both facts the proof needs answer on
win32 now, and the freshness rule alone decides, on every platform.

`pod api` still refuses on Windows, and that is fail-closed rather than a
gap: its authenticated request travels over the pod's private AF_UNIX
dashboard socket with no TCP fallback, and CPython there has no
`AF_UNIX`. Adding a fallback is exactly what would hand a token to
whatever answered the pod's released port.

Docs updated in the same commit: the pod README's Platform section (now
a three-backend capability table), the pod package docstring, the
`cli.md` spec row that claimed Linux-only, the Windows install guide's
per-feature table, and two `platform-compat.md` rows (no `exec` on
Windows, and never parsing a Windows system tool's localized output).

Tests: `test/test_pod_windows.py` covers rendering, the pid record's
start-time binding, the `unit_state` mapping, the pinned `schtasks`
argv, the teardown contract and the runtime dispatch on every platform,
plus four real-`schtasks` tests behind `skipif(not IS_WINDOWS)`. The
file is collected on the win32 CI leg with no mocks of `sys.platform`
there.

Follow-up, deliberately not in this commit: a real-pod win32 boot test
(`pod up --seed minimal` against a scratch KIROCREW_HOME) plus a CI
canary step, and applying a Job object ceiling to the pod's tree.

(cherry picked from commit f0e54e18889b2eced45b30ec60a7a50d8a80327e)
Nothing in CI booted a gateway on Windows or macOS, which is how #8117
shipped a Windows startup break with every unit shard green. This adds
the layers that run the product, not a mock of it, on all three OSes.

Gateway boot matrix. `test/e2e/test_gateway_boot_matrix.py` boots a real
gateway per test against the packaged fake ACP backend on ubuntu-latest,
macos-15 and windows-latest (`e2e-boot-matrix` job), asserts health, one
completed prompt turn, both sandbox tiers and a clean teardown, and greps
the pass count so a skip cannot go green. `testing/harness.py` reads the
READY line through a reader thread and tears down via a process-tree
kill, so it runs on Windows and `test_harness.py` is collected there.

Windows fail-closed contract. `test/test_windows_fail_closed_optin.py`
pins, on a real win32 host with real filesystem state, that the official
kiro backend delegates with no settings file present and that an
unclassified spawn fails closed until `sandbox_allow_unsandboxed_exec` is
set. `backend-test-windows-fail-closed` runs it by node id.

Windows installer smoke. `scripts/smoke-windows-install.ps1` installs the
built NSIS artifact on a clean runner, boots the installed gateway, and
uninstalls; wired into build-windows.yml after the artifact upload.

macOS full suite. `backend-test-macos` moves from the deprecated macos-14
label and a 51-file glob to macos-15 running the whole suite in 3
duration-balanced shards, with `test/macos-expected-failures.txt` and
`test/macos-collect-ignore.txt` applied by the same conftest mechanism as
the Windows lists.

Pod scenario suite. `test/e2e/scenarios/` drives settings persistence,
cron fire, a tool turn, service-unit rendering and a wheel install
against a real `kirocrew pod`, nightly on ubuntu-latest and macos-15
with `systemd --user` via loginctl linger. Pods can now be handed the
fake backend through `KIROCREW_POD_KIRO_BIN`.

Specs and CI docs are updated alongside each layer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants