Repository navigation
fix(windows): fail the probe, not the gateway, on a bad tree scan - #35992
Conversation
On Windows a kiro-cli probe tracks its child's process tree. When a scan of that tree failed after the first one, the tracker cancelled the future the run was waiting on. The run read that as a cancel of itself and raised CancelledError, which the probe's "never raises" guard does not catch, so a gateway that reads the kiro-cli version while it boots exited before it was ready. The tracker now hands its own error to the waiting run. The run ends the leader and every retained handle as before, logs the reason, and returns a failed result marked tree_unverified. A probe reader treats that result as unknown. A cancel of the run itself still propagates. Closes #35849
874a46f to
7c947f3
Compare
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The CI traceback and the red-before tests show the cause, the fix sits at the tracker that turned its own failure into a cancel, and the retry bound is inside the stated Goal. What this change shipsInventory (6 items) — 6 justifiedIntent: keep a Windows gateway booting, with working kiro-cli version checks, when one scan of a probe's process tree fails. FIX.
[FIRST-PRINCIPLES-REVIEWED] 7c947f3 Description read[DESCRIPTION-READ] 37999b1599710c93277c257168aca25af43e45f6773d0c341df91095b15e96a8 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the tracker's error now reaches the run as that error instead of a cancel, which fixes the actual cause, and outside cancels still propagate. Notes
[DESIGN-REVIEWED] 7c947f3 Description read[DESCRIPTION-READ] 76f42ff2abdffa4440e6cb0e99b1f7fd6a72bc9ea4b403edd626edb31eaa253f That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Rebased: main moved in The patch is unchanged: On the old head |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Reviewed the whole diff at 7c947f3, tests included.
On Windows, a failure of the descendant tracker during a kiro-cli probe used to reach the run as a cancellation, which could fail the gateway boot instead of the one probe. Now the tracker hands its own error to the snapshot the run is waiting on, so the run fails with that error. It ends the leader and every retained child handle, closes the handles, logs the reason once, and returns a failed result with tree_unverified set. A real cancel from outside still ends the same teardown and then propagates.
Checks:
- Fail-closed is kept. The leader and the retained children are still ended and no handle leaks, in each of the four failure timings (first or later scan, expected or unexpected error), and also when the leader exited first. Nothing is trusted that was not verified.
- The probe readers treat an unverified run as unknown.
_probe_ranis false for it, so a cut-short run with a zero exit and partial help text is not read as "this binary has no KAS". The cached probes keep an unverified run out of their cache for three runs in a row, then cache it as unknown, as a timed-out probe is. Each retry spawns kiro-cli, so the bound matters, and a test pins it for both probes. - A timeout is still a timeout, and a tracker that fails under a timeout is still logged.
- No regex, no new refusal, no change to what a probe on a healthy tree returns.
The GPT review run cancelled at 19:20 has a passing run at 19:22 on this head, so it is not live red. All other checks are green. Approving without auto, since auto-merge is already set.
Problem / Motivation
Goal: A Windows gateway keeps starting, and keeps its kiro-cli version checks working, when one scan of a kiro-cli probe's process tree fails.
On Windows the gateway sometimes exits while it boots. The log ends with:
The gateway reads the kiro-cli version at boot through
kiro_cli._sandboxed_probe. That function promises that it never raises. On Windows the probe tracks its child's process tree. When a later scan of that tree fails, the tracker cancels the future that the probe run is waiting on. The run reads that as a cancel of itself and raisesCancelledError.CancelledErroris not anException, so the probe's guard does not catch it, and the gateway exits before it is ready.Why it matters
A Windows user whose gateway hits this sees Kiro Crew fail to start. In CI, 8 of the 261
E2E Gateway Boot (windows-latest)legs created after 09:06Z on 2026-11-12 failed this way, counted at about 16:20Z (3.1%). The probe path came in with #35731, which merged at 09:06Z that day. The red legs hit unrelated pull requests (#31604, #34811, #35831, #32983, #34766).Not a goal
descendant_termination_handles_asyncis untouched.kas_relay_help,kas_relay_start_output,kiro_acp_engine_args).What changed (motivation → approach → change)
The tracker's own error now reaches the probe run as that error, not as a cancel. The run ends the process tree as before, logs the reason, and returns a failed result. The gateway keeps booting. The next reader of the version probes again, and after 3 such failed runs in a row on one binary the answer is kept as unknown.
_track_windows_descendants(tracker)_run_process(run)CancelledErrorok=Falsewithtree_unverified=Truekiro_cli._probe_ran(reader)kiro_cli_version_at,chat_agent_engines_at(caches)How the two cancels are told apart. The run does not catch
CancelledErrorat all. The tracker is the only inside source of one, and it now hands over its real error. So anyCancelledErrorthe run still sees came from outside, and it propagates after the same teardown.What happens to the probe's process tree. Lines are in
src/kiro_crew/kiro_prerequisite_process.py. "Before" is8d04c52d2213.The teardown call is the same one in both paths. It terminates the leader and calls
terminate_process_handleon every retained handle. The run never reports success, so containment stays fail-closed.How often a failed read is retried. A run that ended its tree unverified says nothing about the binary, like a sandbox that could not be built. Unlike that case, it did spawn kiro-cli. So
kiro_cli._retry_unverified_probecounts such runs per binary and per probe. The first two stay out of the cache, and the next reader probes again at once. The third is cached as unknown, the way main caches a timed-out probe. There is no clock and no backoff: the count is the bound, three spawns at most.What the gateway does when the version probe fails. It keeps running. While the version is unknown, spec writers leave out the
permissionsblock (agent_materialization/auto_approve.py), each chat keeps its own process (acp/chat_runtime_sharing.py), Crew's MCP servers are not deferred, andkirocrew doctorprints "kiro-cli version unknown". With the retry, one failed scan costs that for one read only.Security (builder-security skill). This PR opens no listener, port, route, storage grant or sign-in path, and it weakens none. The probe still goes through
_prepare_sandboxed_spawnwithfirst_party_fixed_argv; that code is not in the diff. The diff adds no bind or host argument.docs/system-specs/modules/security.mddescribes the tracker hand-over,tree_unverifiedand the retry count.Backwards compatibility
Compatible:
ProcessResult.tree_unverifiedis a new field that defaults toFalse, and a run that used to raiseCancelledErrornow returns a failed result that every reader already treats as a failure.Removes nothing: no path, key, record or public function is hidden, deleted or tightened.
Tests
Proof: PROVEN test/test_kiro_prerequisite_process.py::TestWindowsTrackerFailureDuringARun::test_a_tracker_that_fails_under_a_timeout_is_still_logged (head 7c947f3)
All tests are in
test/test_kiro_prerequisite_process.py.test_an_unexpected_failure_reaches_the_waiting_run_as_its_own_error,test_a_failed_later_snapshot_reaches_the_waiting_run_as_its_own_errortest_a_cancelled_tracker_still_cancels_its_waiterstest_the_run_ends_the_tree_and_returns_a_failed_result(4 cases)test_a_cancel_from_outside_still_cancels_the_runtest_a_run_that_times_out_is_still_a_timeout,test_a_tracker_that_fails_under_a_timeout_is_still_loggedtest_the_version_probe_fails_and_the_spec_gate_still_answers_sandboxed_probereturns a result and the spec gate answerstest_a_run_that_was_cut_short_is_not_read_as_the_childs_answertest_the_next_read_probes_again_and_gets_the_real_version,test_a_tree_that_never_verifies_is_probed_a_bounded_number_of_timesHow the Windows branch runs. On Linux the tests set
platform_compat.IS_WINDOWSand replace only the Windows handle calls (duplicate_asyncio_process_handle,process_handle_active,close_process_handle,terminate_process_handle,descendant_termination_handles_async) with recorded fakes. The real_run_process,_track_windows_descendantsand_run_on_private_loopcode runs. The file is not intest/windows-collect-ignore.txt, so the Windows shards run it too. No test here calls the real Win32 handle functions.Red before the change. On unmodified source the new tests failed with
asyncio.exceptions.CancelledErroratkiro_prerequisite_process.py:645, the line in the CI log. The two retry tests failed withassert None == (2, 23, 0)andassert 1 == 3before the retry was added.Mutations, each run alone, source restored after each.
tree_unverifiedis alwaysFalse_probe_ranignorestree_unverifiedDeterminism. 20 of 20 repeats at
-n0. 10 of 10 repeats at-n 4 --dist loadgroupwithtest_kiro_probe_spawn_off_loop.pyandtest_kiro_prerequisite_supervisor.pybeside it. One shuffled order, seed 25379: 37 passed. No test sleeps or reads a clock; the poll interval is set to 0 and every wait ends on an event or a cancel.Local gate.
scripts/local-gate.pyon this patch before it was rebased (f109c49529d2): 10279 passed, 5 failed. The same 5 fail on the base8d04c52d2213on this host and are host-dependent:test_kas_cli_floor.py::test_a_pinned_link_flipped_after_the_check_never_runs_unsandboxed,test_kas_engine_refusal.py::test_the_probe_reads_the_refusal_from_a_real_start,test_kiro_cli_probe_sandbox.py::test_the_version_probe_child_runs_under_the_sandbox_launcher,test_materialization_restamp_probe.py::test_permissions_removed_for_an_old_cli_is_vouched,test_native_skill_projection.py::test_launch_record_resources_match_the_alias_file_kiro_cli_loads.Manual verification
Not run on a real Windows host. One green
E2E Gateway Boot (windows-latest)leg proves little here: the failure shows in about 3 of 100 legs, so a green leg is the likely result with or without the fix. The deterministic tests above are the evidence.Related Issues
Closes #35849
Sibling sweep
Swept: git grep -nE "_run_process(" -- src/
Sibling: src/kiro_crew/agent_sdk/drivers/acp.py:854 -- not affected -- it has no tracker of its own; it awaits the shared
_run_processthat this PR fixesSwept: git grep -nE "_cache[key] = (version|engines)" -- src/kiro_crew/kiro_cli.py
Sibling: src/kiro_crew/kiro_cli.py:525 -- changed here
Pattern harvest
Rule candidate: review-prompt
Pattern: a helper task cancels the future its caller awaits when the helper itself fails, so the caller reads the helper's failure as its own cancellation
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.