Repository navigation
test(crew-log): wait for the eager seal before sweeping a destroyed log - #36623
Conversation
test_a_destroyed_sessions_log_is_then_collectable_by_the_sweep failed intermittently with (0, 0). The session/closed that destroy writes wakes the eager folder, which seals the unit's savepoint under a shared hold of the unit's lease. When the sweep ran while that seal was still in flight, its sole claim on the lease answered owned and nothing was removed. The test now waits for the eager folder to settle before it ages the close, the same wait the history-delete path takes. The sweep itself is unchanged: it collects only units whose close is older than the retention window, and a unit it finds owned is collected on a later pass. Closes #35553
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change is test-only and waits on the real signal (the eager seal's lease hold) through the existing [DESIGN-REVIEWED] d3d4972 Description read[DESCRIPTION-READ] b8636191ead6f87236eefbdca60a5fcec06afe6f7421a73e71f43bd2e884c5b4 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
buluoray
left a comment
There was a problem hiding this comment.
Approve on d3d4972. This change fixes a flaky retention test. After a session is destroyed, the test now waits for the crew log's background folder to finish sealing the unit before it runs the sweep. The sweep assertions are unchanged: the sweep must still remove exactly one unit and the unit must be gone. The wait is on a real signal. It uses the eager folder's own counters, has a 60 second bound from the shared test helper, and gives a failure message. I reproduced the race on purpose. I held the seal's lease while the sweep claimed it. Without the new wait, the test failed three times out of three with the same (0, 0) seen in CI. With the wait, it passed three times out of three. The same race in production only delays collection: the sweep skips a unit whose lease is held, and the next hourly pass removes it. The history-delete path already waits the same way. Only this one test file changes. Non-blocking: the failure message could include the elapsed time and the folder's queued and settled counts, as the testing conventions ask. Decisive lines: test/test_crew_log_retention.py:1122-1124, test/test_crew_log_retention.py:1144-1145, src/kiro_crew/crew_log/eager.py:1160-1191, src/kiro_crew/crew_log/store_removal.py:103-106, src/kiro_crew/crew_log/store_removal.py:166-199, src/kiro_crew/dashboard/handlers/sessions_parts/slot_teardown.py:383-390
Problem / Motivation
Goal:
test_a_destroyed_sessions_log_is_then_collectable_by_the_sweeppasses on every run, instead of sometimes failing with(0, 0).In Backend Tests the test fails at random with
assert (0, 0) == (1, 0)at its sweep line (test/test_crew_log_retention.py:1141), on main and on pull requests that do not touch the crew log (#35553). It passes when rerun.Why it matters
Every unrelated pull request that draws this red pays a rerun and a flake report, and a sweep test that fails at random trains people to ignore it when it fails for a real reason.
Not a goal
remove_unitor the eager folder. A sweep that meets a held lease skips the unit and a later pass collects it, by design; in production a unit is collected only once its close is older than the retention window, long after the seal its close woke.What changed (motivation → approach → change)
The test now waits for the crew log's background folding to finish before it runs the sweep, so the sweep always finds the closed log free to remove.
destroywritessession/closed, and that entry also wakes the eager folder, which seals the unit's savepoint under a shared hold of the unit's lease. The test waited only for the crew log's writer (assert_drained()), so it could age the close and callsweep_expiredwhile that seal still held the lease. The sweep'ssoleclaim then answeredowned, nothing was removed, and the sweep returned(0, 0). Running the sweep off the loop did not cover this: it waits for the folder only around the unlinks, after the lease claim.After
destroy, the test assertseager.drain(timeout=FALLBACK_TIMEOUT_SECS)before it ages the close: the same wait the history-delete path takes since #35705.Backwards compatibility
Compatible: only one test changes; no product code does.
Removes nothing: the change adds one wait to a test and takes no assertion, record or path away.
Tests
test/test_crew_log_retention.py::test_a_destroyed_sessions_log_is_then_collectable_by_the_sweepwaits for the eager folder afterdestroy, then still asserts the sweep answers(1, 0)and the unit is gone.Determinism proof (testing-conventions, Proving a determinism fix), on Ubuntu 24.04.5 LTS (Linux x86_64), Python 3.12.3, parent =
c22557cd83:c22557cd83-n0, 5 runsassert (0, 0) == (1, 0),remove_unit('session', 'acp-aged') -> 'owned'seal hold seen: True,remove_unit('session', 'acp-aged') -> 'removed'-n0-n 4 --dist loadgroup --max-worker-restart=2withtest/test_crew_log_fold_eager.pyandtest/test_crew_log_lease.py, 10 runstest/test_crew_log_retention.pyshuffled, seed 20848 (62 ids)--durations)The forced condition is a throwaway
-pplugin, not committed. It wraps the lease acquire insidecheckpoint.save(the close-time seal): on the eager worker thread, once the acquire for the unit returns, it sleeps 1.5 s while still holding the lease, and the sweep'sremove_unitfor that unit first waits (bounded) until that hold exists, which is the order CI hit. The plugin logs both steps, so a run where the seal never held the lease cannot count. On this head the seal still held the lease on every run and the test waited it out; the asserts after the sweep ((1, 0), the unit gone) ran each time.The flake-pattern ratchet and
semgrep scan --config semgrep/ --erroragainst the parent report nothing;prove.pyanswersNOTHING_TO_PROVE(test-only change).Manual verification
N/A — test-only change; the forced condition above drives exactly the interleaving CI hit, run on Linux (Ubuntu 24.04).
Related Issues
Closes #35553
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)