Repository navigation
fix(test): cut the wedged-write shutdown inside the wait after the drain - #33704
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only change. It waits on an event that marks the settle branch instead of racing a fixed timer, which removes the flake at its cause. Notes
[DESIGN-REVIEWED] c808844 Description read[DESCRIPTION-READ] 2668aa159ae127df679481ad7cd21dd215645717c604dbaf1a52265f423e32dd That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
The under_graceful_bound case wrapped _shutdown in a 0.3 s wait_for timer. On a loaded Windows runner that timer fired while the drain cut was still waiting out its own window, so _shutdown took its interrupted path, which never reaches settle_admitted_writes, and the deadline warning the test asserts was never logged. The case now delivers the bound as the cancellation it is at expiry, after settle_admitted_writes has started, so it always tests the path it names. The other case keeps _shutdown's own 0.3 s deadline, which has no outer timer to race. Fixes #33609
df3e27e to
c808844
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change; the wedged-write shutdown test now cancels after the wait past the drain cut has started, instead of racing a timer against the cut.
Problem / Motivation
Goal: Make
test_a_wedged_write_past_the_drain_cut_still_ends_shutdown_at_the_graceful_bound[True]test the path it names on every runner, so it stops failing on loaded Windows CI.It failed on Backend Tests (Windows) shard 2/8 on a head that touches no Slack code:
test\test_slack_gateway.py:1178: AssertionError: shutdown ended without naming the admitted write it left pending.Why it matters
A flaky red in a backend shard blocks unrelated PRs and costs a rerun each time it hits.
Not a goal
exit_path.pyor AutoNudge.What changed (motivation → approach → change)
[True]case sometimes finds no "at the graceful shutdown deadline" warning on Windows._shutdown()inasyncio.wait_for(timeout=0.3)._shutdownspends about 0.1 s of that same 0.3 s in the AutoNudge drain cut: thePERSISTENCE_DRAIN_GRACE_SECSbudget, then the drain's own post-cancellation window. On a loaded runner the outer timer fired while the gateway was still inawait asyncio.wait({drain_task})after the cut. That cancellation sends_shutdowndown its interrupted path (src/kiro_crew/slack/gateway_runtime/exit_path.py:243-295on main), which never callssettle_admitted_writes. The removed test comment said settle "is always reached before the graceful deadline, regardless of shard load". The drain phase made that false.[True]case now runs_shutdownas a task and waits untilsettle_admitted_writeshas been entered, recorded through a wrapper. Then it cancels the task. That is what the graceful bound does at expiry, and it now always lands inside the wait after the drain cut. In this case_shutdown's own deadline is the lost-run bound, so only that cancellation ends the wait. The[False]case keeps_shutdown's own 0.3 s deadline. It has no outer timer, so it cannot take the interrupted path. If the drain runs past 0.3 s, settle sees the deadline already passed and still warns. Both cases also assert that settle was entered. The unusedcontextlibimport is dropped.Intended? / Repro
exit_path.py:276-295). There the drain's own warning already names the pending writes ("AutoNudge shutdown stopped waiting on N owned task(s)/write(s) still pending ..."). So the product is right and the test's ordering is wrong.exit_path.py:240 AutoNudge drain still pending after 0.1s; cancelled itandautonudge.py:1072 AutoNudge shutdown stopped waiting on 2 owned task(s)/write(s) still pending 0.1s after cancellation, and no settle warning. So the outer 0.3 s cancellation landed during the drain phase, before settle was reached.Backwards compatibility
Compatible: test-only change.
Removes nothing: only one test's ordering changes; no product code, key or record is touched.
Tests
test_a_wedged_write_past_the_drain_cut_still_ends_shutdown_at_the_graceful_bound[True]: the graceful-bound cancellation landing inside the wait after the drain cut still names the pending write at WARNING.[False]:_shutdown's own deadline ends that wait and names the write.Proof: not provable -- test-only change
Per host rule, no tests were run locally. CI runs them on all platforms. Static gates run: black, flake8, isort, comment-history.
Manual verification
N/A — test-only change, covered by CI.
Related Issues
Fixes #33609
Sibling sweep
Swept: git grep -n "wait_for(.*GRACEFUL_SHUTDOWN_SECS" -- test/
Sibling: test/test_slack_gateway.py:1064 -- not affected -- its bound is FALLBACK_TIMEOUT_SECS since #30653, so it is a safety net and not a race
Pattern harvest
Rule candidate: review-prompt
Pattern: a test that bounds a coroutine with a short fixed timer to reach one inner branch races the earlier phases on a slow runner; deliver the cancellation after an event marks that branch instead
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.