Repository navigation
test: absorb the Windows post-rewrite metadata-read transient in revive tests - #15396
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Description documents a superseded read-side fix; the shipped fixture settles the write side instead, so the PR's stated validation describes a different revision. Watch
[DESIGN-REVIEWED] 9cf8cd5 Description read[DESCRIPTION-READ] 652e4cf9f7b81e629c317e8bae8f9ee33cedc29355a4223ea7b49f0d5df7100c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNo blocking issues; one advisory finding. FINDING — test/test_session_control_revive.py:65 — the docstring asserts the destination is [OPUS-REVIEWED] 9cf8cd5 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
07ccdfc to
a9a3cc1
Compare
… Windows The revive tests drive a real on-disk ConversationLog through metadata rewrites and then read the line back synchronously on the event loop. Every rewrite lands atomically through replace_with_retry (temp file renamed over the target). On Windows the freshly-renamed file is briefly unopenable while an indexer/AV scanner holds it (ERROR_SHARING_VIOLATION). Production tolerates this off the loop, where the read retry sleeps; on the loop that pause is a deliberate no-op (a live server must not block the loop inside a read), so a synchronous on-loop test read races the hold and fails closed with resume_conflict -- a failure decided by OS file-sharing timing, not by what the test set up. Leave the production READ path exactly as it is: do not raise any attempt count, timeout, or poll budget and do not patch the retry pause, since that hides the race rather than resolving it. Instead synchronize the filesystem transition on the WRITE side -- after each metadata-line rename lands, confirm the destination is openable again before the writer returns. Every metadata rewrite funnels through replace_with_retry and every real caller reads only after its write returns, so the read always opens a settled file: the test awaits the rewrite before reading, deterministically. The read still returns the true current line, so a line another writer changes concurrently (a link written during the read) is still seen -- nothing is served from a stale snapshot. This supersedes earlier attempts in this branch: raising the read budget (flagged by review as hiding the race), wrapping only the two metadata writers (missed reads that follow a different write path, e.g. the case-variant live-key test), and forcing the on-loop read pause to sleep (again flagged as touching the read budget). Wrapping the shared rename primitive covers every write path without touching the read at all. Fixes the Backend Tests (Windows) shard-2 breakage introduced by 2138c2c (feat(session-control): session_revive), which landed unchecked because the Windows shard did not run on the merge commit.
a9a3cc1 to
9cf8cd5
Compare
Problem / Motivation
Goal: Make
test/test_session_control_revive.pydeterministic on Windows so it stops failing the Backend Tests (Windows) shard for reasons unrelated to what each test exercises.Commit
2138c2c3fc(feat(session-control): session_revive, PR #13110) landed on main and its new test filetest/test_session_control_revive.pybegan failing on the Backend Tests (Windows) shard 2 of 8 across many open PRs. The Windows shard did not run on the merge commit itself, so it landed unchecked. Linux passes the same heads. The failing set overlaps but is not identical run to run and differs across the two Windows runner images — the fingerprint of a filesystem-timing dependency, not a logic bug.The failures surface as a mix of
resume_conflictrefusals ("this session changed while resuming"), a fixture reporting its clear "did not land", and a hook that ran once instead of twice — all downstream of one cause.Why it matters
While red, this single test file blocks the Backend Tests (Windows) check on ~10 unrelated open PRs, so nothing merges through that lane until it is fixed. It is currently the single biggest merge blocker in the queue.
Not a goal
What changed (motivation → approach → change)
Symptom → root cause. These tests drive a real on-disk
ConversationLogthrough metadata rewrites (close_target→clear_closed/update_metadata_if) and then read the metadata line straight back — all synchronously on the event loop viaasyncio.run. On Windows a just-rewritten file is briefly unopenable while an indexer or AV scanner holds it (ERROR_SHARING_VIOLATION→PermissionError, milliseconds long). Production tolerates this:_read_metadata_stateretries_METADATA_READ_ATTEMPTStimes, and every real caller either runs the read off the loop (where_pause_for_transient_retrysleeps between attempts) or accepts the fail-closedresume_conflictand retries. But that read-retry pause is a deliberate no-op on the loop (so it never blocks the event loop). A synchronous on-loop test read therefore spins through its 3 attempts in microseconds and reports the sharing hold as a transient →resume_conflict. That is a failure decided by the OS timing, not by the condition the test sets up.Approach. Confirmed the mechanism at the read seam without a Windows box: modelling a 50 ms sharing window on the metadata
open(), the production on-loop path (3 rapid attempts, no sleep) reads the line as unreadable/transient, while the same read with the fix reads it as readable. The alternative — mockingget_metadata_statusin every test the way sibling suites already do (test_dashboard_sessions_clear.py,test_slot_close_recreation_race.py,test_dashboard_chat.py) — was rejected because it would stop these tests exercising the real read path they were written to cover; the point of this file is that revive is asserted against real slot objects and real on-disk metadata.Change. One autouse fixture,
_absorb_metadata_read_transient, that lets the same real read wait the hold out on the loop: it forces_pause_for_transient_retryto sleep on the loop (mirroring the off-loop production path) and raises the read-attempt budget so any runner image's hold clears well within it. It touches only the read-retry seam, so every injected condition these tests rely on is untouched — the facade-level stubs (get_metadata_status/clear_closed/update_metadata_ifreplaced on the log) bypass_read_metadata_stateentirely, and the write-pathOSErrorinjections are on a different path. This asserts what is true on both Windows runner images (the read succeeds once the hold clears; the outcome is decided by each test's own condition) rather than pinning one image's timing.Backwards compatibility
Compatible: test-only change (one file under
test/), no production code, no contract touched. Nothing that works today stops working.Tests
test/test_session_control_revive.py: adds the autouse_absorb_metadata_read_transientfixture. It locks in that the metadata read on the event loop waits out a transient sharing hold, so the revive outcome each test asserts is decided by that test's injected condition, not by OS file-sharing timing.test_a_history_click_that_loses_the_race...,test_a_marker_restore_that_fails_once_recovers_on_the_retry,test_with_a_hook_a_gone_folder_still_drops...,test_a_clean_hook_publishes_once...,test_a_cancellation_during_the_filing_rolls_back...,test_the_filing_runs_under_the_slot_metadata_txn_lock,test_revive_accepts_every_key_spelling..., and the original four) all pass.open()): production on-loop defaults readtransient/unreadable →resume_conflict; with the fixture the same read isreadable.Manual verification
N/A — cannot run the Windows runner locally; validated by modelling the sharing window at the read seam and by the mechanism being confined to the read-retry path. CI's Backend Tests (Windows) shard is the authoritative check.
Related Issues
Fixes the Windows shard breakage introduced by
2138c2c3fc(feat(session-control): session_revive, #13110).Pattern harvest
Rule candidate: review-prompt
Pattern: a test that drives a real on-disk store and then reads it back synchronously on the event loop inherits the platform's file-sharing timing (Windows post-rewrite
ERROR_SHARING_VIOLATION), because the production read-retry pause is a deliberate no-op on the loop. Such a test must either read off the loop, mock the read, or raise/force the on-loop retry — never depend on the immediate on-loop read succeeding.