Repository navigation
test: wait on the store-write signal, not a 2-5 s thread wait - #20346
KiroCrewCaptain merged 1 commit into
Conversation
The spec-builder nudge-pause tests and the auto-research watchdog settlement tests waited for an executor-backed durable write with `assert await asyncio.to_thread(<event>.wait, 2 or 5)`. That bound is a bet on the runner's speed (rule D1): the write it waits for is a chain of temp-file creates, fsyncs, a retried rename and a directory sync, which the 4-vCPU Windows runner does not always finish inside it, so the wait returned False and the test failed with a bare `assert False`. The wait also parked itself on the loop's default executor, the same pool the write needs. Each wait now polls the threading.Event from the loop with async_wait_until, which holds no pool thread and fails by name at the lost-run bound. The four tests that flaked run on a one-thread default executor, which makes the old shape fail every time and pins that the wait takes nothing from the pool the code under test uses.
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS The description says pool starvation isn't what fails on CI. Check that the longer lost-run bound alone covers the observed Windows timeouts. [DESIGN-REVIEWED] a819032 Description read[DESCRIPTION-READ] 26c2a60cb9ac4516bcccfa908afa5d2f50c5eb7ef6ae13321937a3d89886e9d0 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
mbriones98
left a comment
There was a problem hiding this comment.
Checks green and readiness passed at a819032; no behavior change.
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces thread-parked waits with async_wait_until and a one-thread executor fixture in two test modules; test files only.
|
Follow-up for the independent review of a819032, which merged before this round could be pushed: #20418 (head cda3ae8).
First-pass check: R1-R8 OK. |
…o_thread waits kirodotdev#20346 moved the waiting side of the spec-builder nudge-pause and auto-research settlement tests off `asyncio.to_thread(<event>.wait, N)`. The held side of the same tests still bet on the runner's speed: a patched write blocked on `assert <release>.wait(5)` while the test drove executor-backed I/O. In test_slow_settlement_cannot_remove_resumed_replacement and test_duplicate_reserves_its_name_before_populating_the_destination that I/O is a store write or a full create, so a slow runner made the held thread give up first and the test failed with a bare assert or a 500. With a one-thread default executor both fail every time at those lines. Every held-side fixed bound in test_auto_research.py and spec_builder/tests/test_routes.py now waits on default_timeout(), the lost-run bound, and a hold the test never releases fails by name at that bound, including one raised inside a request handler or a plain thread. The one-thread executor pin moves into kiro_crew.testing as the pytest fixture one_pool_thread (kiro_crew.testing.fixtures); both test files import it, so the duplicated fixtures are deleted. test/test_wait_discipline_ratchet.py gains a second per-file ratchet, test/to_thread_wait_baseline.txt, for a threading.Event wait parked on an executor under a fixed bound: `asyncio.to_thread(<obj>.wait, N)` (asyncio however imported or aliased) and `loop.run_in_executor(<any>, <obj>.wait, N)`, across test/ and every src/**/tests package. 184 sites in 97 files. It shares the existing gate's baseline machinery, so two burn-downs in different files merge without conflict and moving a site between files fails.
…o_thread waits kirodotdev#20346 moved the waiting side of the spec-builder nudge-pause and auto-research settlement tests off `asyncio.to_thread(<event>.wait, N)`. The held side of the same tests still bet on the runner's speed: a patched write blocked on `assert <release>.wait(5)` while the test drove executor-backed I/O. In test_slow_settlement_cannot_remove_resumed_replacement and test_duplicate_reserves_its_name_before_populating_the_destination that I/O is a store write or a full create, so a slow runner made the held thread give up first and the test failed with a bare assert or a 500. With a one-thread default executor both fail every time at those lines. Every held-side fixed bound in test_auto_research.py and spec_builder/tests/test_routes.py now waits on default_timeout(), the lost-run bound, and a hold the test never releases fails by name at that bound, including one raised inside a request handler or a plain thread. The one-thread executor pin moves into kiro_crew.testing as the pytest fixture one_pool_thread (kiro_crew.testing.fixtures); both test files import it, so the duplicated fixtures are deleted. test/test_wait_discipline_ratchet.py gains a second per-file ratchet, test/to_thread_wait_baseline.txt, for a threading.Event wait parked on an executor under a fixed bound: `asyncio.to_thread(<obj>.wait, N)` (asyncio however imported or aliased) and `loop.run_in_executor(<any>, <obj>.wait, N)`, across test/ and every src/**/tests package. 184 sites in 97 files. It shares the existing gate's baseline machinery, so two burn-downs in different files merge without conflict and moving a site between files fails.
…o_thread waits kirodotdev#20346 moved the waiting side of the spec-builder nudge-pause and auto-research settlement tests off `asyncio.to_thread(<event>.wait, N)`. The held side of the same tests still bet on the runner's speed: a patched write blocked on `assert <release>.wait(5)` while the test drove executor-backed I/O. In test_slow_settlement_cannot_remove_resumed_replacement and test_duplicate_reserves_its_name_before_populating_the_destination that I/O is a store write or a full create, so a slow runner made the held thread give up first and the test failed with a bare assert or a 500. With a one-thread default executor both fail every time at those lines. Every held-side fixed bound in test_auto_research.py and spec_builder/tests/test_routes.py now waits on default_timeout(), the lost-run bound, and a hold the test never releases fails by name at that bound, including one raised inside a request handler or a plain thread. The one-thread executor pin moves into kiro_crew.testing as the pytest fixture one_pool_thread (kiro_crew.testing.fixtures); both test files import it, so the duplicated fixtures are deleted. test/test_wait_discipline_ratchet.py gains a second per-file ratchet, test/to_thread_wait_baseline.txt, for a threading.Event wait parked on an executor under a fixed bound: `asyncio.to_thread(<obj>.wait, N)` (asyncio however imported or aliased) and `loop.run_in_executor(<any>, <obj>.wait, N)`, across test/ and every src/**/tests package. 184 sites in 97 files. It shares the existing gate's baseline machinery, so two burn-downs in different files merge without conflict and moving a site between files fails.
Problem / Motivation
Goal: The spec-builder nudge-pause tests and the auto-research watchdog settlement tests pass on every runner instead of failing on slow Windows shards.
Backend Tests (Windows) (7)fails with a bareassert Falseon pull requests that touch none of these modules. Each failing test waits for an executor-backed durable write withassert await asyncio.to_thread(<event>.wait, 2 or 5). The failure recurred 3 times since 11 Oct, always as a pair on two xdist workers of the same shard (#20345 has the runs).Why it matters
An unrelated red shard costs every open pull request a rerun or a fresh head, and the bare
assert Falsenames neither the write nor the state it reached.Not a goal
asyncio.to_thread(<event>.wait, N)sites in the tree (about 168), or teaching the wait-discipline ratchet to count this shape. Flaky: nudge-pause and watchdog settlement tests on Windows (7) #20345 records both as burn-down candidates.What changed (motivation → approach → change)
Symptom: the wait returns
Falseand the test fails. Cause: the bound is a fixed 2 s or 5 s bet on the runner's speed (rule D1 indocs/system-specs/common/testing-conventions.md), and the write behind it is a chain of temp-file creates, fsyncs, a retried rename and a directory sync. The 4-vCPU Windows runner does not always finish that inside the bound. The wait also parks on the loop's default executor, the pool the write itself needs. That is not what fails on CI (each test loop gets a fresh 8-thread pool, and at most 2 jobs are in flight in these tests). With a one-thread pool, though, the old shape fails every time.Change: every
asyncio.to_thread(<event>.wait, N)in the two files now usesasync_wait_until(<event>.is_set, describe=...)fromkiro_crew.testing.wait: 6 sites intest_routes.pyand 5 intest_auto_research.py. The wait polls from the loop, holds no pool thread, and fails by name at the lost-run bound (half the test's timeout). The four tests that flaked request a newone_pool_threadfixture (one per file), which gives the loop a one-thread default executor.#20345 shows why #20060's
atomic_writechange is not involved: none of these write paths callatomic_write().Backwards compatibility
Compatible: test-only change. No production module is touched.
Removes nothing: the tests keep every assertion they had and wait on the same signals.
Tests
src/kiro_crew/apps/builtins/spec_builder/tests/test_routes.pyandtest/test_auto_research.pyin full: 809 passed with-n 4.-n 4, and passed with-n 8under a 32-process CPU spinner load. The four that flaked, plus their class siblings (13 tests), passed in reverse order with-n0.max_workers=1on the loop, main's versions of the four tests fail at exactly the CI lines.Manual verification
N/A — unit coverage sufficient: test-only change, proven by the one-thread pin and its mutation run.
Related Issues
Closes #20345
#20327 moves the nudge tests into
test_routes_nudge_loops.py. Whichever of the two lands second needs a small rebase.Pattern harvest
Rule candidate: lint
Pattern:
asyncio.to_thread(<threading.Event>.wait, <number under 30>)in a test, a fixed short wait that also takes a default-executor thread from the code under test.test/test_wait_discipline_ratchet.pydoes not count it today.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)