Repository navigation
test(skills): wait for the coalesced follow-up prune before asserting - #24142
bolichen97 merged 1 commit into
Conversation
test_two_concurrent_releases_each_prune_their_own_view joined only the prune threads it saw right after both releases. Since kirodotdev#23913 an end that arrives while the other end's pass runs records a request and returns, and the running end starts the follow-up pass on a new prune thread just before it exits. The test never joined that thread, so on a loaded runner it asserted while the follow-up pass was still walking and reported whichever view that pass owed as lingering. The test now joins the release threads and then waits, under the bounded wait_until, until no end prune is running or requested for its agents directory. A new test forces the order (one end walks while the other still holds its lease, and the follow-up pass is held until the wait is polling) for each release leading, and fails with the old join-only wait. Closes kirodotdev#23965
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this is a test-only fix for a test that stopped waiting too early. The settle wait reads the same [DESIGN-REVIEWED] 9017c17 Description read[DESCRIPTION-READ] 184e9a88b2a0d8d3515acf51bab6a309fce4848b81dccc50912cb23bf3f91b53 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
bolichen97
left a comment
There was a problem hiding this comment.
Reviewed at 9017c17. Looks good to ship. Test-only, one file, closes a tier:T1 flake issue.
Checked against _coalesced_end_prune on main: a request is only ever recorded while its key is in _END_PRUNE_RUNNING, and the key stays there until the last follow-up pass finds no further request. So waiting until neither set holds a key for this agents directory means every owed pass has run. The key shape matches (key[1] is directory.absolute().as_posix()). The forced-order test holds the follow-up pass until the settle wait is polling, so it fails deterministically under the old join-once wait, and it runs with each release leading. The settle wait uses the bounded wait_until helper, with no sleeps.
Security checklist: no auth, secrets, network exposure, CI trust path, approval logic or untrusted execution is touched. No findings.
Not approving yet: the Backend Tests shards (3.12, 1 to 8) are still queued and readiness is still checking on this head. Approval can follow once readiness reports passed.
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
bolichen97
left a comment
There was a problem hiding this comment.
Approving at head 9017c17. Checked: readiness passed on this head, all checks green, AI review lanes report no blocking findings, no other changes requested, base is main, mergeable, one test file only.
The v0.11.0-rc.2 release run failed two Test Release Candidate shards on tests main has already fixed: - test(skills): wait for the coalesced follow-up prune before asserting (#24142) (cherry picked from commit 5d98031) - test(skills): freeze the lease-scan clock in the legacy-lock test (#24257) (cherry picked from commit 3ff45cb) - test: pin the prune clock in the seam pruning test (#24191) (cherry picked from commit 33fd421) #24191's test seeds a 2026-09-25 history file and prunes at 30 days, so it fails on every run after 2026-10-25. #24142 follows the end-prune coalescing that #23913 brought to this branch.
Problem / Motivation
Goal:
test_two_concurrent_releases_each_prune_their_own_viewstops failing on loaded CI runners, without weakening what it checks.Since #23913 the test fails intermittently on Backend Tests shard 2, on Linux and Windows, on PRs that do not touch skill projection. It reports either
the first release's view lingeredorthe second concurrent release skipped its prune on the busy in-process lock and its view lingered. #23965 collects about 40 occurrence reports.Why it matters
It is the most frequent flake in the repository right now: 51+ PRs failed on it in a day, re-runs often fail again, and every hit costs a shard re-run on an unrelated PR.
Not a goal
_coalesced_end_prune). The product behaviour is correct (see below); only the test's wait is wrong.What changed (motivation → approach → change)
Root cause: a test synchronization gap, not a product race. Since #23913, an end that arrives while another end's prune pass is running only records a request in
_END_PRUNE_REQUESTEDand returns. When the running pass finishes, that end consumes the request and starts the follow-up pass on a newkirocrew-skill-view-prunethread, just before its own thread exits. The test built its join list once, right after the tworelease()calls, so it never joined that third thread. On a loaded runner it asserted while the follow-up pass was still walking. Which view lingered depends on which release took the first pass, which is why CI reports both messages.The product does not leak the view. Every end is still followed by a pass that started after its lease was released, and the forced-order test below checks that the owed view is gone once the follow-up finishes.
_settle_end_prunesjoins the release threads, then waits withkiro_crew.testing.wait.wait_until(the bounded wait the Determinism contract asks for, no sleep) until no end prune is running or requested for the test's agents directory. A pending request always has its key in_END_PRUNE_RUNNINGuntil the last follow-up pass finishes, so an empty state means every owed pass has run.test_two_concurrent_releases_each_prune_their_own_viewuses it in place of the join over a thread list captured once. Its assertions are unchanged.test_a_release_handed_to_the_follow_up_pass_is_waited_for[first|second]forces the failing order. One end's pass walks while the other projection still holds its lease. The other end then records a request, and the follow-up pass is held until the settle wait is polling. It runs once with each release leading.Backwards compatibility
Compatible: test-only change; no product code is touched.
Removes nothing: the diff replaces one test's wait and adds a test; no key, route, record or public function is removed.
Tests
Red/green for the forced order,
test_a_release_handed_to_the_follow_up_pass_is_waited_for, 25 iterations of each parametrization:wait_untilline swapped forreturn)the view handed to the follow-up pass lingeredStress on the original test,
-n 32, with "load" meaning 32 extra CPU-spinning processes on a 32-core host:Proof: not provable -- test-only change
Manual verification
N/A: the change is a test wait, covered by the red/green and stress runs above. Local gates were run on the commit: the scoped backend set (152 passed), the flake-pattern and wait-discipline ratchets (113 passed), black, isort, flake8, comment-history, file-line-cap, sync-io, harness parity, feature map, brand, loop-bound locks and
semgrep scan --config semgrep/1.78.0 (0 findings). The scoped set had to run withTMPDIRoutside~/.kiro/crew. On this host the defaultTMPDIRsits under the crew home, andtest_native_skill_projection.py::test_launch_record_resources_match_the_alias_file_kiro_cli_loadsthen refuses the tmp project as managed state. That test passes with any otherTMPDIR, and its file is not in this diff.Related Issues
Closes #23965
Sibling sweep
Swept: git grep -n "kirocrew-skill-view-prune" -- test/
Sibling: test/test_skill_view_relaunch_harness.py:310 -- not affected -- it waits until no prune thread is alive, and a follow-up thread starts before its parent exits
Sibling: test/test_native_skill_projection_rewrites.py:1063 -- not affected -- a single end, so no coalesced follow-up pass
Pattern harvest
Rule candidate: review-prompt
Pattern: a test that joins a list of worker threads captured once, when the code under test can start more workers from those threads (a hand-off or follow-up pass).
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)