Skip to content

test(chat): wait on the persist signal in the abnormal-turn usage test - #20152

Merged
javenciu merged 1 commit into
mainfrom
test/wait-discipline-abnormal-turn-usage
Oct 14, 2026
Merged

javenciu merged 1 commit into
mainfrom
test/wait-discipline-abnormal-turn-usage

Conversation

@javenciu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make the one short fixed wait in test/test_abnormal_turn_usage_row_15013.py wait for the event the test's own persist stub sets, so the test no longer depends on how fast the runner is.

test_a_cancel_before_the_executor_runs_still_lands_the_row replaces persist_token_record_async with a stub that sets an asyncio.Event (write_scheduled) when the turn reaches the persist, then waited for it with asyncio.wait_for(write_scheduled.wait(), timeout=5) (line 376 on main). Five seconds is a bet that the runner is as fast as the machine the test was written on. test/test_wait_discipline_ratchet.py counts that site against the file's line in test/wait_discipline_baseline.txt.

Why it matters

A fixed short wait is the commonest cause of a flaky backend test in this repository (the module docstring of test/test_wait_discipline_ratchet.py says so), and it fails on slower CI runners first. Removing the site lowers the baseline, and the ratchet then keeps this file from gaining another.

Not a goal

  • Other test files' waits: each file has its own baseline line and its own change.
  • Changing what the test asserts or the code it exercises (_run_chat and its shielded persist).
  • Two other timed calls in the same test, which the ratchet does not count. release_write.wait(timeout=5) (test/test_abnormal_turn_usage_row_15013.py:352) runs in the worker thread and holds the append until the test has cancelled the turn. wrote.wait(timeout=5) (test/test_abnormal_turn_usage_row_15013.py:391) waits for that worker's append. Both are threading.Event waits, and await_until takes an asyncio.Event or a future, so they need a different mechanism and stay as they are.

What changed (motivation → approach → change)

The ratchet's rule is "wait on signals, not clocks": a test awaits the event the code emits, under a lost-run bound (half the test's own timeout by default) that only turns a hang into a named failure. The helper for an asyncio.Event is kiro_crew.testing.wait.await_until, imported at module level as rule D1 of docs/system-specs/common/testing-conventions.md asks.

  • test/test_abnormal_turn_usage_row_15013.py imports await_until (line 40) and replaces the one site with await await_until(write_scheduled, what="the turn reaching the shielded persist") (line 377 at this head). await_until is a coroutine function, so the await in front of it stays.
  • test/wait_discipline_baseline.txt loses the line 1 test_abnormal_turn_usage_row_15013.py, which is now at zero.

Rejected: keeping wait_for and raising its number. The ratchet counts any ceiling under 30 s, and a larger constant is still a clock bet.

Backwards compatibility

Removes nothing: this is a test-only change; no production code, public API or protocol is modified.

Tests

In short: with the baseline line deleted and the test file untouched the ratchet fails; with the site replaced it passes, and the file's own tests are unchanged.

  • Red (baseline line deleted, test file untouched), pytest test/test_wait_discipline_ratchet.py -n0 -q: 1 failed, 38 passed, with the report test/test_abnormal_turn_usage_row_15013.py: 0 -> 1 and the site test/test_abnormal_turn_usage_row_15013.py:376 wait_for(5s) (main's line numbers).
  • Green, the ratchet plus test/test_abnormal_turn_usage_row_15013.py at -n0, three runs: 60 passed each (39 ratchet tests and 21 in the file; the file had 21 before the change too).
  • The changed test, TestCancelDuringCompletePersist::test_a_cancel_before_the_executor_runs_still_lands_the_row: 20 runs at -n0 (1 passed each); 10 runs at -n 4 --dist loadgroup --max-worker-restart=2 beside test/test_chat_runner_early_recovery_actor.py and test/test_chat_runner_folder_steering.py (15 passed each); one shuffled order of those three files, seed 25084 (35 passed).
  • The file first, then test/test_taskrunner_atomic_persistence.py, test/test_taskrunner_execution_capture_async.py, test/test_unattended_slot_guardrails.py, test/test_wheel_apply.py, test/test_chat_runner_early_recovery_actor.py and test/test_chat_runner_folder_steering.py, in one process at -n0: 147 passed.
  • black --check, isort --check-only and flake8 are clean on the touched file.

Manual verification

N/A: unit coverage is sufficient. The change swaps one test wait for the repository's wait helper and touches no runtime code.

Related Issues

No issue exists; burn-down of the wait-discipline baseline (precedents #19928, #20014, #19527).

Pattern harvest

Not generalizable: this site is one more instance of the class the ratchet already catches (rule D1 of docs/system-specs/common/testing-conventions.md), so the gate stops the class and no new rule is needed.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 14, 2026 06:17
@javenciu
javenciu requested a review from mbriones98 October 14, 2026 06:17
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision ee119578d1ef2c086ba05c4b8fa14a749661e37a ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision ee119578d1ef2c086ba05c4b8fa14a749661e37a touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of ee119578d1ef2c086ba05c4b8fa14a749661e37a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] ee11957

Description read

[DESCRIPTION-READ] ae04aa2cf1f80371b607fec1a77563dab4ef0106dea600fe247986463e2f5e63

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed ee119578d1ef2c086ba05c4b8fa14a749661e37a — this comment is updated in place on each push.

Review details

I found no defects in the diff, and the discovery pass sent no candidates to check. The change replaces a fixed 5-second asyncio.wait_for with await_until on the same asyncio.Event, which await_until accepts. The event is set from the event loop before the work moves to a thread, so the test sequence behaves as before. Removing the file's line from wait_discipline_baseline.txt matches the one fixed wait the change takes out.

No findings.

[OPUS-REVIEWED] ee11957

Verdict parsed from the review's SHA-scoped output markers for commit ee119578d1ef2c086ba05c4b8fa14a749661e37a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable ee119578d1ef2c086ba05c4b8fa14a749661e37a: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of ee119578d1ef2c086ba05c4b8fa14a749661e37a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] ee11957

False positive or not applicable? A repository writer can comment:
/ai-review override gpt ee119578d1ef2c086ba05c4b8fa14a749661e37a: <one-sentence reason>

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
test_a_cancel_before_the_executor_runs_still_lands_the_row waited for
the turn to reach its offloaded persist with a fixed 5 s
asyncio.wait_for on write_scheduled. A fixed short ceiling is a bet on
the runner's speed, and test/test_wait_discipline_ratchet.py counts it
against the file.

Await the event itself with await_until, whose default ceiling is half
the running test's timeout and only turns a hang into a named failure.
Drop the file's line from the wait-discipline baseline, since it holds
no short fixed wait any more.
@javenciu
javenciu force-pushed the test/wait-discipline-abnormal-turn-usage branch from a6868dc to ee11957 Compare October 14, 2026 11:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@javenciu
javenciu enabled auto-merge (squash) October 14, 2026 13:17

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces a fixed asyncio.wait_for timeout with await_until in one test and drops that file from the wait-discipline baseline; test files only.

@javenciu
javenciu merged commit feb9b3c into main Oct 14, 2026
113 of 175 checks passed
@javenciu
javenciu deleted the test/wait-discipline-abnormal-turn-usage branch October 14, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants