Skip to content

test(windows): wait on signals, not clocks -- test_runtime_cleanup_windows, test_windows_cleanup_capacity - #20014

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
unflaky/wait-migrate-windows
Oct 14, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
unflaky/wait-migrate-windows

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Replace 43 short clock bets in two Windows-specific test files with waits on the real signals the code emits, so the wait-discipline ratchet accepts them and CI stops losing on slow runners.

test/test_runtime_cleanup_windows.py (25 sites) and test/test_windows_cleanup_capacity.py (18 sites) held a combined 43 asyncio.wait_for(coro, N<30) and sleep(N) polling loops. On a slow Windows CI runner, the 5 s / 10 s / 15 s ceilings were not lost-run bounds — they were the race that decided pass versus fail.

Why it matters

Every short fixed wait in these files is a flake waiting to fire on the Windows runner, and these tests already carry Windows-specific marks (pytest.mark.skipif(sys.platform != "win32")): the only real test runs happen on the slower platform.

Not a goal

  • Fixing non-Windows tests (different batch).
  • Changing any test assertion or weakening any check.
  • Migrating other open PRs' files.

What changed (motivation → approach → change)

Each short asyncio.wait_for(coro, N) is a clock bet: the test passes only if the runner is faster than N. The fix follows the pattern in #19248 and subsequent batch PRs (#19325 #19334 #19339 #19331 #19489 #19868 #19928):

  • asyncio.wait_for(event.wait(), N) → await await_until(event) — events are first-class signals; await_until awaits the event object directly under a lost-run bound (half the test's timeout).
  • asyncio.wait_for(coro, N) → await _bounded(coro, what=…) — coroutines are wrapped in asyncio.ensure_future; a module-local _bounded helper (identical to the one in test_acp_runtime.py) awaits the task with await_until and cancels + settles it on timeout.
  • asyncio.sleep(0.02/0.01) in polling loop → await async_wait_until(predicate) — each polling loop is replaced with a predicate that returns the state the test is about to assert; async_wait_until polls under the default lost-run bound.
  • asyncio.timeout(15) context manager → await _bounded(coro, what=…) — the body of the async with asyncio.timeout(15): block was a single await coro, replaced directly.
  • time.sleep(0.02) in sync polling loop (fixture teardown) → wait_until(predicate, timeout=10) — the one synchronous polling loop in _isolate_pending_windows_tree_cleanup's teardown is replaced with wait_until, the synchronous variant, with an explicit 10 s bound.
  • asyncio.sleep(0.01) heartbeat loop → asyncio.sleep(0) — in test_slow_identity_read_leaves_the_event_loop_serving, other_session_work is a background ticker that only needs to yield the event loop; sleep(0) is sufficient and is not counted by the ratchet.

Site counts

File Before After
test/test_runtime_cleanup_windows.py 25 0
test/test_windows_cleanup_capacity.py 18 0

Both entries removed from test/wait_discipline_baseline.txt.

Remaining sites

None. All 43 sites migrated to signal waits.

Tight bounds kept

Two bounds remain, both explicit and documented with # flake-ok::

  1. _owned_tree teardown: await_until(_exit_task, timeout=10, what="worker cooperative exit") — 10 s grace period before the force-kill fallback. The worker subprocess exits in milliseconds once finish is written; the bound is a cleanup deadline that triggers the kill path, not the pass/fail gate. Ratchet does not count await_until(timeout=...).

  2. _isolate_pending_windows_tree_cleanup teardown: wait_until(lambda: not _still_alive(), timeout=10) — 10 s grace for process exit after terminate_process_handle. Same policy: triggers the survivor assertion, not the pass/fail decision. Ratchet does not count wait_until(timeout=...).

Backwards compatibility

Removes nothing: this is a pure test-only change; no production code, public API, or protocol is modified.

Tests

  • All 33 non-Windows tests pass 3× locally; 21 Windows-only tests skip (expected on Linux).
  • test/test_wait_discipline_ratchet.py passes after --write-baseline drops the two now-zero entries.
  • black, isort, flake8 clean on both touched files.

Manual verification

N/A — unit/integration coverage sufficient; the Windows-specific tests will run on the Windows CI runner.

Screenshots / video

N/A — no user-visible UI change.

Related Issues

Part of #20013

Pattern harvest

Not generalizable: each site was already an instance of the documented D1 class; the helpers already existed. The ratchet catches any regression.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 13, 2026 23:49
@CrysisDeu
CrysisDeu requested a review from mbriones98 October 13, 2026 23:49
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 13, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 695838f107184a8218b6b2b48993b09bf6c6fd97 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 695838f107184a8218b6b2b48993b09bf6c6fd97 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 695838f107184a8218b6b2b48993b09bf6c6fd97 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 695838f

Verdict parsed from the review's SHA-scoped output markers for commit 695838f107184a8218b6b2b48993b09bf6c6fd97.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 695838f107184a8218b6b2b48993b09bf6c6fd97: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 695838f107184a8218b6b2b48993b09bf6c6fd97 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 695838f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 695838f107184a8218b6b2b48993b09bf6c6fd97: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 695838f107184a8218b6b2b48993b09bf6c6fd97 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the backend-test-windows shards actually ran both files green at this head. On Linux every changed test is skipped.

Suggestions

  • _bounded now exists as a near-identical copy in test_acp_runtime.py, test_runtime_cleanup_windows.py and test_windows_cleanup_capacity.py, plus variants in test_start_priority.py and test_stt_stream.py. Expose one public bounded(coro, what=) from kiro_crew.testing.wait, which already has a private _bounded, so the cancel-and-settle-on-timeout behaviour can't drift between copies.

[DESIGN-REVIEWED] 695838f

Description read

[DESCRIPTION-READ] 6bde2b579ea40b6a808beba0e4f4a0f8a5ed1d2d561af258d824e5b43d35243b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@CrysisDeu
CrysisDeu force-pushed the unflaky/wait-migrate-windows branch from 5c82241 to 03c5885 Compare October 14, 2026 00:30
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 14, 2026
…ndows, test_windows_cleanup_capacity

Replace 43 short clock bets in two Windows-specific test files with
signal waits using helpers from kiro_crew.testing.wait.

- asyncio.wait_for(event.wait(), N) → await await_until(event)
- asyncio.wait_for(coro, N) → await _bounded(coro, what=...)
- asyncio.sleep(0.02) polling loop → await async_wait_until(predicate)
- asyncio.timeout(N) context → await _bounded(coro, what=...)
- time.sleep(0.02) sync poll (fixture teardown) → wait_until(predicate)
- asyncio.sleep(0.01) heartbeat loop → asyncio.sleep(0) (loop yield)

Two explicitly documented tight bounds remain (both # flake-ok):
- _owned_tree teardown: await_until(task, timeout=10) -- 10 s grace
  before force-kill; worker exits in ms after "finish" is written.
- _isolate_pending_windows_tree_cleanup: wait_until(pred, timeout=10)
  -- 10 s grace for process exit after terminate_process_handle.

Both files drop to 0 sites; their lines removed from wait_discipline_baseline.txt.

Part of #20013
@CrysisDeu
CrysisDeu force-pushed the unflaky/wait-migrate-windows branch from 03c5885 to 695838f Compare October 14, 2026 00:50
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 14, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 14, 2026 02:04

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change that replaces clock-based waits in two Windows cleanup test files with the shared wait helpers, and updates the wait-discipline baseline.

This was referenced Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants