Skip to content

test(acp): replace waits in session_handle_keepalive with await_until - #20150

Merged
javenciu merged 1 commit into
mainfrom
test/wait-discipline-acp-session-handle-keepalive
Oct 14, 2026
Merged

javenciu merged 1 commit into
mainfrom
test/wait-discipline-acp-session-handle-keepalive

Conversation

@javenciu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The keepalive oracle-check test stops using a five-second wall-clock ceiling to decide pass or fail.

test/test_acp_session_handle_keepalive.py::test_a_keepalive_during_the_oracle_check_prevents_the_cancel waits on two things the test itself sets up: the stub oracle's entered event and the drain task that consumes the handle's dispatch loop. Both waits were asyncio.wait_for(..., timeout=5.0) (lines 209 and 213 on origin/main). The rest of the file is deterministic: the handle's clock is a fake that the polled queue advances one exact tick per poll (module docstring), so no assertion reads the wall clock. Only these two waits did.

test/test_wait_discipline_ratchet.py counts a fixed wait_for(..., timeout=N) under 30 s as a bet on the runner's speed and holds each file to its line in test/wait_discipline_baseline.txt. This file's line was 2.

No issue exists; burn-down of the wait-discipline baseline (precedents #19928, #20014, #19527).

Why it matters

A slow runner turns a correct test into a red Backend Tests job on an unrelated pull request, and the failure names a clock instead of the signal that never arrived.

Not a goal

  • No change under src/. The handle's watchdog, the oracle stub and the fake clock are untouched.
  • No change to what the test asserts.
  • No change to the fixed waits in any other test file. They are counted per file in test/wait_discipline_baseline.txt and migrated separately.

What changed (motivation → approach → change)

Symptom: a ceiling that decides pass versus fail around an event and a task the test already controls. Approach: the repository's rule in src/kiro_crew/testing/wait.py (wait on the signal; a ceiling only turns a hang into a named failure).

  • await asyncio.wait_for(entered.wait(), timeout=5.0) becomes await await_until(entered, what="the first oracle consult") (line 210 on this branch). entered is an asyncio.Event.
  • events = await asyncio.wait_for(drain, timeout=5.0) becomes events = await await_until(drain, what="the drain finishing") (line 214). drain is an asyncio.Task, which await_until accepts as a future and whose result it returns.
  • from kiro_crew.testing.wait import await_until is added (line 28), the same import test/test_acp_session_handle_skill_reads.py uses.
  • The file's line is removed from test/wait_discipline_baseline.txt; the file now holds no short fixed wait.

A hang now fails with an AssertionError naming what, after the helper's default lost-run bound (half the running test's timeout, default_timeout()), instead of a TimeoutError after 5 s. await_until leaves the awaited task running on expiry because the test owns it, where wait_for cancelled drain.

Rejected: raising 5.0 to a larger literal (still a clock bet, and the ratchet floor is 30 s).

Backwards compatibility

Compatible: test-only change; no source file, public name or signature changes.

Reader: test/test_wait_discipline_ratchet.py:read_baseline -- CI test run -- the file now counts zero, so its line is removed; test_the_baseline_is_not_stale passes with it gone.

Tests

In short: with the baseline line removed and the old waits in place the ratchet fails (0 -> 2); with the change the ratchet and the file pass in 3 of 3 runs, and the file's own six tests pass before and after.

  • RED, baseline line deleted and the test file untouched: pytest -n0 test/test_wait_discipline_ratchet.py fails (1 failed, 38 passed) with test/test_acp_session_handle_keepalive.py: 0 -> 2, naming lines 209 and 213 as wait_for(5s).
  • GREEN: pytest -n0 test/test_wait_discipline_ratchet.py test/test_acp_session_handle_keepalive.py, three separate processes: 45 passed each (45 passed on origin/main).
  • The changed file alone: 6 passed before and after.
  • Test-order leak check, one process at -n0: the changed file first, then test/test_acp_session_handle_abandoned_turn.py, test/test_acp_session_handle_stream_command.py, test/test_taskrunner_atomic_persistence.py, test/test_taskrunner_execution_capture_async.py, test/test_unattended_slot_guardrails.py, test/test_wheel_apply.py: 147 passed.
  • black --check --target-version py312, isort --check-only and flake8 on the changed file, scripts/check_agent_sdk_boundary.py, and scripts/check_comment_history.py against the merge-base all exit 0.

Not run: -n 4 repeats; mypy (CI type-checks src/kiro_crew/ only).

Manual verification

N/A — test-only change; the RED run above is the check that the ratchet catches the undercount.

Related Issues

No issue exists; burn-down of the wait-discipline baseline (precedents #19928, #20014, #19527).

Pattern harvest

Rule candidate: lint — already enforced: test/test_wait_discipline_ratchet.py counts every short fixed wait per file, which is how this file's two sites were known. This change lowers its count to zero; no new rule is needed.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

The oracle-check test bounded two awaits with
asyncio.wait_for(..., timeout=5.0): the oracle's "entered" event and
the drain task. A five-second ceiling that decides pass or fail is a
bet on the runner's speed, which is the commonest source of flaky
tests in this repository.

Await both with await_until, which waits for the signal under a
lost-run bound and fails by name when it never arrives, and drop the
file's line from the wait-discipline baseline. The assertions are
unchanged: they read the fake clock, not the wall clock.
@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 14, 2026 06:17
@javenciu
javenciu requested a review from mbriones98 October 14, 2026 06:17
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 072442028a9573759a7a2571e61c7329f02a504f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 072442028a9573759a7a2571e61c7329f02a504f ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 072442028a9573759a7a2571e61c7329f02a504f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] 0724420

Description read

[DESCRIPTION-READ] 6e6a5b6163cc39cf8341eb0c5f85e01d6459b341e7dedb82578ac701e2778a64

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 072442028a9573759a7a2571e61c7329f02a504f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0724420

Verdict parsed from the review's SHA-scoped output markers for commit 072442028a9573759a7a2571e61c7329f02a504f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 072442028a9573759a7a2571e61c7329f02a504f: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 072442028a9573759a7a2571e61c7329f02a504f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0724420

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 072442028a9573759a7a2571e61c7329f02a504f: <one-sentence reason>

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@javenciu
javenciu enabled auto-merge (squash) October 14, 2026 09:47

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change; replaces fixed waits with await_until in test_acp_session_handle_keepalive and drops its wait-discipline baseline entry.

@javenciu
javenciu merged commit 8545798 into main Oct 14, 2026
112 of 174 checks passed
@javenciu
javenciu deleted the test/wait-discipline-acp-session-handle-keepalive branch October 14, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants