Repository navigation
test: remove crew-log units with the eager folder held in test_crew_log_emit - #16459
Conversation
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] 5666409 Description read[DESCRIPTION-READ] a809e0e1a8a7eea9cc6860e3aa984b1c35df8f02f596e5e1cadda973c8296366 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
1c3e9ba to
e67ff8b
Compare
…og_emit test_a_predecessor_collected_during_the_deferral_costs_no_dropped_write removed a crew-log unit folder with a bare shutil.rmtree while the eager folder could still be folding it on its own thread. A fold that listed the segments before the rmtree recreates the unit's .lock inside the folder, so the final rmdir failed with Directory not empty. Retention removes a unit inside eager.paused(); the three bare removals in this file now drain the folder and hold it the same way. Closes kirodotdev#16458
e67ff8b to
5666409
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
This is a test-only change and it makes the tests stricter. Each of the three removals now runs the way retention does: wait for the eager folder to settle, hold it, then remove. That closes the race where a fold recreates .lock inside the folder being deleted (ENOTEMPTY on POSIX, WinError 32 on Windows). The tests' own assertions are unchanged. The helper adds checks that the folder settled and the hold was granted, so a busy host shows up as a clear failure, not a hidden race. The testing-conventions doc now names the rmtree case too. All checks are green. Approving.
For all three, I read the head SHA's source through gh api, the full diff and every existing review comment. I posted or changed nothing. All required checks pass or are skipped. Each PR has all-PASS AI verdicts and still needs a human review. The @kirocrew-computer MCP server is declared in the agent spec but not configured, so it was unavailable; this review didn't need it.
Problem / Motivation
Goal:
test_a_predecessor_collected_during_the_deferral_costs_no_dropped_writestops failing withDirectory not emptywhen the eager folder is still folding the unit the test removes.The test fails intermittently on
Backend Tests (3.12, 4)withOSError: [Errno 39] Directory not emptyfrom its ownshutil.rmtreeof a crew-log session folder. It has turned shards red on several PRs that touch no crew-log code (#15596, #16387, #16034, #16039, #14457, #15957). Details and the reproduction are in #16458.Why it matters
A red backend shard blocks readiness on whatever unrelated PR it lands on, and Coverage Gate fails closed behind it. Each hit costs a rerun and an investigation.
Not a goal
soleand insideeager.paused(), so the eager folder cannot be mid-batch while the unit's files go. Only the test's bare removal skips that.What changed (motivation → approach → change)
shutil.rmtreeof the unit folder fails withDirectory not emptystore._open_lockrecreates.lockinside the folder the rmtree is emptyingeager.paused()Each committed entry wakes the eager folder, and it folds on its own thread after
emit.flush()returns. The fold opens the unit throughCrewLog.open, which lists the segments and then takes the append lock. Taking the lock creates the folder and.lockif they are missing. A fold that lists the segments before the rmtree and takes the lock during it puts.lockback, and the lastrmdirfails.A new
_remove_as_retention_does()helper intest/test_crew_log_emit.pycallseager.drain()and runs the removal insideeager.paused(), asserting both. It is the pattern #16130 uses. All three bare removals in the file go through it: the failingshutil.rmtree, and twolog.jsonlunlinks intest_a_resume_of_the_same_store_writes_no_previous_edgeandtest_a_store_whose_front_retention_removed_reports_no_edge. Those two race the same thread and hitWinError 32on Windows. The testing-conventions rule from #16130 now namesshutil.rmtreeand the POSIX symptom too.Backwards compatibility
Compatible: test-only change plus one documentation sentence. No product code is touched.
Removes nothing: the diff adds one test helper and routes three existing removals through it.
Tests
Proof that the race is the cause and the hold removes it: a local harness forces the interleaving. It holds the eager thread at its first lock creation for the unit, starts the test's
rmtreeonly once the fold is held, and releases the fold when thermtreereaches the unit folder'srmdir. On main f16fa23 it fails the test with the CI error 3 of 3 runs. On this branch the same harness passes 3 of 3, because the drain makes the held fold finish before the removal starts.Also run on this branch:
pytest test/test_crew_log_emit.pyon the head rebased onto main af1eb61: 316 passedscripts/check_comment_history.pyagainst origin/main andscripts/docs_lint.py: passedManual verification
N/A: test-only change, covered by the runs above.
Related Issues
Closes #16458
Pattern harvest
Rule candidate: review-prompt
Pattern: a test removes crew-log files with a bare
unlink()orshutil.rmtreeinstead of draining the eager folder and holdingeager.paused().Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)