Skip to content

test(knowledge): wait on the committed audit instead of a fixed sleep - #13737

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/flake-committed-audit-cancel
Sep 26, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/flake-committed-audit-cancel

Conversation

@jeeshofone

@jeeshofone jeeshofone commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

test_cancelling_the_caller_cannot_drop_a_committed_audit fails intermittently on Windows shard 1 with assert [] == ['item.update'] (#13730). It hit #13538 and #12605, neither of which touches knowledge handlers.

Why it matters

A shared flake on main turns unrelated PRs red and costs each one a fresh head. The assertion also reads like a dropped audit, the exact defect _audited_write exists to prevent, so a reader can mistake the flake for a regression.

What changed (motivation → approach → change)

Gap Change
The test gave the worker thread a fixed 0.4s to finish a 0.2s write and its audit, so a descheduled worker read as a dropped audit The fake write now blocks on an event until after the caller is cancelled, then the test waits on an event the audit itself sets (10s bound)
The old test could not tell whether the cancel landed mid-write The test now asserts no audit exists at the moment of cancellation, which pins the cancel inside the write window

Test-only. src/kiro_crew/dashboard/handlers/knowledge.py is unchanged: it already runs the write and _sel_log in one worker take.

Backwards compatibility

No runtime change.

Tests

  • Head is one commit on current main 694f61983e5e5045326f001940effb726001dc2e.
  • Regression: with a test plugin that stretches the worker's 0.2s timed wait to 0.8s (a descheduled thread), the OLD test fails with the CI assertion assert [] == ['item.update']; the new test passes.
  • Still catches the real defect: with a plugin that restores the pre-fix shape (audit after to_thread returns), the new test fails with the committed audit was dropped.
  • test/test_knowledge_handlers_off_loop.py: 82 passed; TestAuditRidesWithTheWrite 5 consecutive runs green.
  • scripts/check_black_formatting.py passes; isort and flake8 on the changed file pass. mypy scope (src/kiro_crew/) is untouched.

Manual verification

N/A — deterministic test-only change, verified by the fault-injection runs above.

Related Issues

Fixes #13730

Pattern harvest

Rule candidate: review-prompt
Pattern: A test that proves background work finished must wait on a signal that work sets, not sleep a fixed interval and then assert.

Checklist

  • One Conventional Commit on current main
  • Existing applicable tests pass and the regression reproduces on the old test
  • Self-review completed; code follows project style guidelines
  • No runtime behaviour change
  • No secrets, credentials, internal references, or .github changes in the diff

@jeeshofone
jeeshofone requested review from a team and bolichen97 September 25, 2026 07:38
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of a5aa8260526466e2fa4f7e0e1cc0d08a3d2d1157 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

If the mid-write audited == [] assert fails, release never sets and the worker lingers 10s — bounded, failure-path-only; verify that's acceptable.

[DESIGN-REVIEWED] a5aa826

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed a5aa8260526466e2fa4f7e0e1cc0d08a3d2d1157 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a5aa826

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed a5aa8260526466e2fa4f7e0e1cc0d08a3d2d1157 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a5aa826

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@jeeshofone
jeeshofone force-pushed the fix/flake-committed-audit-cancel branch from 3287f94 to 041daee Compare September 26, 2026 00:01
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 26, 2026
@jeeshofone
jeeshofone force-pushed the fix/flake-committed-audit-cancel branch from 041daee to a51663d Compare September 26, 2026 03:19
The cancel test gave the worker thread a fixed 0.4s to finish its 0.2s
write and audit. On a loaded Windows runner the worker is not always
scheduled inside that window, so the assertion read [] before the audit
arrived. Hold the fake write until after the cancellation and wait on an
event the audit sets, with a 10s bound.

Fixes kirodotdev#13730
@jeeshofone
jeeshofone force-pushed the fix/flake-committed-audit-cancel branch from a51663d to a5aa826 Compare September 26, 2026 03:55
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 26, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 26, 2026 06:23
@iamwhatever
iamwhatever merged commit 2e9c0c6 into kirodotdev:main Sep 26, 2026
98 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: committed-audit cancel test races the worker thread with a fixed 0.4s sleep (Windows)

2 participants