Repository navigation
test(shared-crew): compare background audit rows without assuming write order - #29456
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only change. Ordering is still checked through a defined sort key, and the row count and contents are still asserted exactly. [DESIGN-REVIEWED] 74fc48c Description read[DESCRIPTION-READ] c812c0f2c0b43ca83d77f17617190458b6526074d581ecb377981879a4938b7b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
…te order Each audit is written by its own background task through asyncio.to_thread, so two writes can reach the writer in either order. Sort captured rows by outcome before comparing. Fixes #29455
7addea8 to
74fc48c
Compare
|
Shard 6 is being cancelled at the runner's 60-minute limit, not by a test failure. A test-only diff runs the full test corpus (#23450): this PR's shard 6 ran 43,693 tests (job 126675848839: all passed, then cancelled at about 58 minutes) plus the leaf corpus gates over 810 files. Main's shard 6 push run selects 8,086 tests and finishes in about 45 minutes. #23461 tracks the fix. I'll rebase and re-run CI once it lands. |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change: shared-crew audit assertions sort rows by outcome instead of assuming background-task write order.
Problem / Motivation
Goal: Stop the shared-crew audit tests from failing when two background audit writes finish in the opposite order.
test_the_losing_answer_is_audited_as_a_conflictfailed in CI with the audit rows asconflictthenapplied. Three tests intest_shared_crew_action_audit.pymake the same order assumption and fail the same way under a forced thread-scheduling shuffle.Fixes #29455
Why it matters
Any PR whose Backend Tests shard picks up these tests can go red for a reason unrelated to its diff, which costs a re-run and a log read each time.
Not a goal
schedule_action_audit,record_actionor the security event schema.What changed (motivation → approach → change)
schedule_action_auditstarts one task per record andrecord_actionwrites each throughasyncio.to_thread. Its docstring says it records "without making its caller wait for the write", so it promises when a write is not awaited, not what order writes land in. Two worker threads can reach the writer in either order. Order is unspecified by design, so the fix belongs in the tests.appliedsorts beforeconflict). They still assert the exact rows and count, so a duplicate row from the winner's replay still fails. The record'sattimestamp was rejected as a sort key because two records from one handler can share a clock reading. No sleeps or retries were added.Backwards compatibility
Compatible: test-only change.
Tests
asyncio.to_threadcall, the four affected tests failed 86 of 200 runs (--count=50).--count=50: 1000 of 1000 pass.test/test_shared_crew_*.py,test_error_code_contract.pyandtest_ws_event_scoping.py: 1356 passed, 2 skipped.semgrep/rules) are clean on both changed files.Manual verification
N/A: test-only change.
Screenshots / video
N/A: no UI change.
Sibling sweep
Searched
test/for assertions oneventscaptured fromaudit._default_writer. Only the three listed assertions compare more than one row in order. The single-row assertions andtest_shared_crew_recovery_bounds.py(session notes, not audits) are unaffected.Pattern harvest
Assertions over rows written by fire-and-forget background tasks should compare a sorted or set view unless the code serialises the writes.