Repository navigation
test(chat): wait on the persist signal in the abnormal-turn usage test - #20152
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] ee11957 Description read[DESCRIPTION-READ] ae04aa2cf1f80371b607fec1a77563dab4ef0106dea600fe247986463e2f5e63 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsI found no defects in the diff, and the discovery pass sent no candidates to check. The change replaces a fixed 5-second No findings. [OPUS-REVIEWED] ee11957 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683). |
test_a_cancel_before_the_executor_runs_still_lands_the_row waited for the turn to reach its offloaded persist with a fixed 5 s asyncio.wait_for on write_scheduled. A fixed short ceiling is a bet on the runner's speed, and test/test_wait_discipline_ratchet.py counts it against the file. Await the event itself with await_until, whose default ceiling is half the running test's timeout and only turns a hang into a named failure. Drop the file's line from the wait-discipline baseline, since it holds no short fixed wait any more.
a6868dc to
ee11957
Compare
|
Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683). |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces a fixed asyncio.wait_for timeout with await_until in one test and drops that file from the wait-discipline baseline; test files only.
Problem / Motivation
Goal: Make the one short fixed wait in
test/test_abnormal_turn_usage_row_15013.pywait for the event the test's own persist stub sets, so the test no longer depends on how fast the runner is.test_a_cancel_before_the_executor_runs_still_lands_the_rowreplacespersist_token_record_asyncwith a stub that sets anasyncio.Event(write_scheduled) when the turn reaches the persist, then waited for it withasyncio.wait_for(write_scheduled.wait(), timeout=5)(line 376 on main). Five seconds is a bet that the runner is as fast as the machine the test was written on.test/test_wait_discipline_ratchet.pycounts that site against the file's line intest/wait_discipline_baseline.txt.Why it matters
A fixed short wait is the commonest cause of a flaky backend test in this repository (the module docstring of
test/test_wait_discipline_ratchet.pysays so), and it fails on slower CI runners first. Removing the site lowers the baseline, and the ratchet then keeps this file from gaining another.Not a goal
_run_chatand its shielded persist).release_write.wait(timeout=5)(test/test_abnormal_turn_usage_row_15013.py:352) runs in the worker thread and holds the append until the test has cancelled the turn.wrote.wait(timeout=5)(test/test_abnormal_turn_usage_row_15013.py:391) waits for that worker's append. Both arethreading.Eventwaits, andawait_untiltakes anasyncio.Eventor a future, so they need a different mechanism and stay as they are.What changed (motivation → approach → change)
The ratchet's rule is "wait on signals, not clocks": a test awaits the event the code emits, under a lost-run bound (half the test's own timeout by default) that only turns a hang into a named failure. The helper for an
asyncio.Eventiskiro_crew.testing.wait.await_until, imported at module level as rule D1 ofdocs/system-specs/common/testing-conventions.mdasks.test/test_abnormal_turn_usage_row_15013.pyimportsawait_until(line 40) and replaces the one site withawait await_until(write_scheduled, what="the turn reaching the shielded persist")(line 377 at this head).await_untilis a coroutine function, so theawaitin front of it stays.test/wait_discipline_baseline.txtloses the line1 test_abnormal_turn_usage_row_15013.py, which is now at zero.Rejected: keeping
wait_forand raising its number. The ratchet counts any ceiling under 30 s, and a larger constant is still a clock bet.Backwards compatibility
Removes nothing: this is a test-only change; no production code, public API or protocol is modified.
Tests
In short: with the baseline line deleted and the test file untouched the ratchet fails; with the site replaced it passes, and the file's own tests are unchanged.
pytest test/test_wait_discipline_ratchet.py -n0 -q:1 failed, 38 passed, with the reporttest/test_abnormal_turn_usage_row_15013.py: 0 -> 1and the sitetest/test_abnormal_turn_usage_row_15013.py:376 wait_for(5s)(main's line numbers).test/test_abnormal_turn_usage_row_15013.pyat-n0, three runs:60 passedeach (39 ratchet tests and 21 in the file; the file had 21 before the change too).TestCancelDuringCompletePersist::test_a_cancel_before_the_executor_runs_still_lands_the_row: 20 runs at-n0(1 passedeach); 10 runs at-n 4 --dist loadgroup --max-worker-restart=2besidetest/test_chat_runner_early_recovery_actor.pyandtest/test_chat_runner_folder_steering.py(15 passedeach); one shuffled order of those three files, seed 25084 (35 passed).test/test_taskrunner_atomic_persistence.py,test/test_taskrunner_execution_capture_async.py,test/test_unattended_slot_guardrails.py,test/test_wheel_apply.py,test/test_chat_runner_early_recovery_actor.pyandtest/test_chat_runner_folder_steering.py, in one process at-n0:147 passed.black --check,isort --check-onlyandflake8are clean on the touched file.Manual verification
N/A: unit coverage is sufficient. The change swaps one test wait for the repository's wait helper and touches no runtime code.
Related Issues
No issue exists; burn-down of the wait-discipline baseline (precedents #19928, #20014, #19527).
Pattern harvest
Not generalizable: this site is one more instance of the class the ratchet already catches (rule D1 of
docs/system-specs/common/testing-conventions.md), so the gate stops the class and no new rule is needed.Checklist