Repository navigation
test: assert the wait or the signal, not seconds, in 23 long bounds - #37906
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change is test-only. Each wall-clock bound becomes the wait or signal it stood for, and the planted-regression table shows every test still fails by name when its regression returns. Notes
[DESIGN-REVIEWED] 9afafc3 Description read[DESCRIPTION-READ] 83eff39d170766efaeb2d14b831f5cc026de39a51cce7aa4a9d16e67b891f9f0 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
986b715 to
da76e9e
Compare
Twenty-three tests bounded a measured duration at 10 to 120 seconds. Under coverage, a busy CPU or a fast clock the tighter ones go red with no bug, and several regressions they exist to catch hang the worker until pytest's timeout instead of failing by name. Each now asserts the claim the bound stood for: the timeout handed to a wait, the signals and kills sent, the timer a bounded call arms, the sleeps a fake child never takes, the drain waits a shutdown makes, a backoff clock that only the code's own waits advance, or interpreter work that grows linearly.
da76e9e to
9afafc3
Compare
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
K8 counted only `assert <span> < <number literal>`, so a measured real-clock span against a name or an expression (`elapsed < budget`), the reversed and age forms (`budget > elapsed`, `stamp > time.monotonic() - 5`), unittest assertLess and assertGreater, and spans held in names like `waited` or `cpu` all landed unseen. The detector, now in test/flake_pattern_k8.py, counts them on real clocks (the stdlib clocks, their aliases, and a running loop's time) and exempts clocks the test drives, remaining-time checks, cutoffs taken before the call and lower bounds. The ratchet stays per file against the merge base, so concurrent merges cannot race it. The real host-speed bounds it finds are converted to assert the wait, the timeout handed to it, or a clock only the code's own waits advance. A kept complexity budget and two bounds that read no host clock carry flake-ok with a reason. test_governance_profiles.py splits a section into a sibling to stay under the line cap. Two follow-ups for tests #37715 and #37906 converted: the graceful terminate_pgid test accepts the EPERM a macOS zombie group answers, and the failed-compaction turn test drives the dispatch loop on a manual clock, so a fast or busy host can no longer let the turn deadline win.
Problem / Motivation
Goal: The last 23 tests that hold their claim to a 10-120 s wall clock stop reading the host's speed, and each still fails, by name and promptly, when the regression it guards comes back.
After #37600, #37630, #37675, #37715, #37782 and #37853 these are the K8 hits left on main: every literal duration bound of 10 s or more, in 18 files. The looser bounds rarely red a healthy run, but under coverage, a busy CPU or a fast clock the tighter ones do, and most of the regressions they exist to catch either pass under them or hang the worker until pytest's timeout.
Why it matters
A bound that a regression sits under or hangs past gives no signal, and one that follows the runner blocks unrelated PRs. Clearing them leaves the K8 ratchet at zero in these files.
Not a goal
src/.elapsed < budget); the K8 detector does not see them yet, and a later PR takes them with the detector.What changed (motivation → approach → change)
Each bound is replaced by what it stood for:
<= _QUEUE_POLL_CEILING_SECS), at most one park after the budget, and the stop reasonbudget - 1e-6 <= spent <= budget + max backoffasyncio.wait(<= _REPORT_DRAIN_TIMEOUT)setitimercalls it arms and disarmssleepshim onPATHis never calledB -> .as one link in the checkout and every previewxcrun/sleeprecord: every hang stopped, no wait past the budget; a stop before the hang counts as an early cancelselect/ reader waits, each within the caller's budget, on a manual clock so a slow lease cannot expire it first_linear_work)kiro_crew.session_allocationfacade;created == wall - 90.0False positives, left unchanged:
test_turn_harness.py:371(record.elapsedis the harness's virtual loop time),test_mcp_quarantine.py:757(arm_at - flag_atare character offsets into source text),test_bench_retrieval_stats.py:762(the sign of a delta between two fixed samples).test_subagent_stop_reason_consistency.pywould have crossed the 2,000-line cap, so its task-runner section moves unchanged totest/test_subagent_stop_reason_taskrunner.py, and the shared_STALL_EVIDENCEtext moves intotest/subagent_stop_reason_helpers.py.Backwards compatibility
Compatible: test files and test helpers only; no product behaviour changes.
Removes nothing: every test keeps its claim; one test section and one constant move to sibling files unchanged.
Tests
Proof: not provable -- test-only change
Planted regressions as in-memory monkeypatches (scratch pytest plugins;
src/untouched), the same plant against main's original test and this branch's test. "Lost run" means pytest's timeout killed the test.IndexError)GIT_TIMEOUT_SdroppedSlow-host repros (real pauses injected in memory before the spawn, signal, poll or lease, up to 6 s) pass on the branch, including the cases the local reviews raised: a helper slow to install its handler, a watchdog SIGTERM before the fake child records its hang, and a >1 s pause after the pool lease.
Forced-condition repeat runs on an 8-core DevSpace, the 23 converted tests (25 ids),
-n0, main (274647852d) against this branch (da76e9e5ad; the later head changed only the source-pin test, outside this set). "Busy" is 8 busy loops; the two trees ran one after the other. These finished after the merge.Main's reds:
test_severs_a_credential_on_2451_pieces_finishes_within_budgetunder load,test_failed_compaction_then_no_response_ends_the_turnwith the clock 6x fast, and with the clock 6x fast plus busy also the oneshot supervisor, apps registry, notarize, tool-input offload and subprocess pool tests. This branch's reds are one test only:test_failed_compaction_then_no_response_ends_the_turn, which still ran on the real clock: the 5 s queue poll waits real seconds while the 30 s turn deadline reads the scaled clock, so the deadline wins. #37946 drives that test's dispatch loop on a manual clock.Changed test files and the new sibling:
-n 4, 901 passed, 14 skipped. isort, flake8, black gate, file line cap, comment-history and brand gates,test_flake_pattern_ratchet.py,test_facade_mirror_guards.py, andsemgrep --config semgrep/(no finding on an added line): clean.Manual verification
N/A — test-only change; the planted regressions and forced-condition runs above are the check.
Related Issues
Follows #37782 and #37853 (no issue).
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.