Repository navigation
fix: Windows pod restart adoption and macOS local-model test stall - #16122
Conversation
1f0036a to
6560759
Compare
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the [DESIGN-REVIEWED] 702a971 Description read[DESCRIPTION-READ] cdf977c5dc92da3db9c83f48946f66175b599e3e20e2cd8917ad4dad7a04b4c3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Both fixes rest on platform behaviour the author could not run locally — confirm the Not justified as shipped
What this change shipsInventory (6 items) — 4 justifiedIntent: make the nightly's Windows pod-restart scenario and macOS local-model tests pass for the reasons they were red — a FIX.
[FIRST-PRINCIPLES-REVIEWED] 702a971 Description read[DESCRIPTION-READ] a58f0fcaf2aedc6607b3fa53bec0b719ab4e0c7fd3cd5ab5e338f9bca66e50b9 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsThe liveness short-circuit de-pins the fence's token-identity arm: the test that exists to prove it now returns before reaching it. FINDING — test/test_instance_credential_pairing.py:513 — [OPUS-REVIEWED] 702a971 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
|
…macOS local-model test
6560759 to
702a971
Compare
Problem / Motivation
Goal: The nightly's Windows pod restart scenario and macOS local-model tests pass again, for the reasons they were red.
The Nightly Build has been red for three nights (09-30, 10-01, 10-02). Two legs stay red on every run:
Pod Scenario E2E (windows-latest):test_setting_survives_a_gateway_restartreportspod gateway did not come back after restart (last 0). The successor gateway boots, logs for about 14 seconds, then is never seen again. The next scenario fails withConnectionRefusedErroron the same pod.Platform Tests / Backend Tests (macOS): the three tests intest_decisions_local_runtime_real_spawn.py(new on 10-01) fail withruntime stayed {'state': 'starting', ...}; log: pid=NNN. The stand-in server prints its pid and is alive, but never attests.Why it matters
A red nightly publishes nothing: every
Publish ...andSign + Notarizejob is skipped, so no nightly CLI, Docker image, Windows, Linux or macOS build has gone out since 09-29. On Windows the underlying defect is also a product one: an in-app restart of a pod gateway is killed by its own supervisor.Not a goal
test_context_composition_contract, the third nightly red on 10-02; test: fix two Windows-red tests and write down why #16083 already did.layaorstrands-deciderlaunchers, which serve through uvicorn and do not bind throughhttp.server.What changed (motivation → approach → change)
src/kiro_crew/instances/run_marker.pysrc/kiro_crew/decisions/local_servers/plumb_cpu.pygetfqdnlookuptest/test_decisions_local_runtime_real_spawn.pytest/e2e/scenarios/conftest.pypod status --jsonbeside the logstest/test_instance_credential_pairing.pyWindows pod restart. On Windows
os.execvisCreateProcessplus an exit of the caller, so the restarted gateway is a child with a new pid. The pod supervisor waits up to 30 s for that child to publishrun/gateway-<port>.pid, then drains the Job with everything in it. The successor never published:_record_names_another_live_gatewayread the old record, asked for the old pid's creation time, got the same token back, and declined to overwrite "another live gateway". That token stays readable after the process exits for as long as any handle to it is open, and the supervisor holds one — itsPopento the gateway — for exactly the window the successor boots in. The gate now askspid_existsfirst, the same liveness-before-identity orderpod.windows.supervised_pidalready documents for the same trap. An exited pid is a corpse, not a rival, so the successor publishes and is adopted.One sibling compare is deliberately left as it is:
pod/runtime_attestation.py::_pod_recorded_pidstill answers from the token alone. Both of its readers ask liveness themselves before they act on it --port_owneragrees it againstmain_pid, which issupervised_pid(pid_existsfirst), andpod/windows.py::_restart_successorcheckspid_existson the record before comparing -- and its tests pin synthetic pids on purpose, so a liveness check inside it would change what those tests prove rather than what this restart needs.macOS local-model test.
http.server.HTTPServer.server_bindcallssocket.getfqdn(host)betweenbind()andlisten(). On a macOS runner that resolver call can stall for longer than the test waits, so the stand-in server has a bound socket that refuses every connection. The stand-in and the one real launcher that binds throughhttp.server(plumb_cpu.py) overrideserver_bindto set the name from the address they already have. The gateway attests a server by its secret, never by a name, so nothing read that value.Diagnostics. Both failures were hard to read from the nightly's output alone. The e2e restart failure now prints the pod's own
pod status --jsonverdict next to the logs, and the local-runtime wait prints the port and attest probe state when it gives up.Backwards compatibility
Compatible: the boot-write gate declines in one fewer case — a recorded pid that no longer exists — and that case could never have named a serving gateway.
plumb_cpu.pyserves the same routes on the same address; onlyserver_nameis no longer looked up.Tests
test_instance_credential_pairing.py::TestABootWriteDoesNotOverwriteALiveSuccessor::test_declines_when_the_record_names_another_live_process— the live-rival case still declines, with liveness now mocked true....::test_an_exited_predecessor_is_not_a_rival_even_while_its_token_still_reads— new: an exited pid whose token still matches is written over.test_decisions_local_runtime_real_spawn.py— unchanged assertions; the stand-in binds withoutgetfqdn, and the failure message names the stalled half.test/e2e/scenarios/test_conftest.py— unchanged, still 49 passing.Locally (Linux): the two gate modules plus every module that imports
write_marker(1296 passed), the threetest_decisions_local_*modules (166 passed), isort/flake8/mypy and the backend check scripts from the prepare-pr profile.Manual verification
Windows and macOS cannot run here. This PR carries the
ci:pod-scenariosandci:macoslabels so the hosted Windows pod scenario suite and the macOS backend suite run on this head; their verdict is the verification.Related Issues
no linked issue: found from the nightly runs directly.
Pattern harvest
Rule candidate: review-prompt
Pattern: a Windows process identity (creation
FILETIME) read as liveness — the token stays readable after exit while any handle is held, so every "is this pid still that process" check needspid_existsbefore the token compare.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)