Skip to content

test(wait-migration): replace fixed waits with signal waits - #21208

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
unflaky/wait-w2-d
Oct 16, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
unflaky/wait-w2-d

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Replace short fixed-wait sites in 13 test files with signal-based waits that key off real completion signals rather than fixed time budgets.

The affected tests use asyncio.wait_for(coro, N) or time.sleep(N) inside loops as a fixed-duration bet that the subject completes before N seconds on every CI runner. Slower runners miss that bet, producing intermittent failures that red unrelated pull requests.

Why it matters

A test that bets on runner speed loses when a CI machine is under load. The betting sites counted by the wait-discipline ratchet are the single commonest cause of flaky backend test failures in this repository. Removing them makes the suite stable across all runner speeds.

Not a goal

  • This PR does not migrate every remaining file in the ratchet baseline; it covers the 13 files assigned to wave-2-d.
  • Latency assertions (timeouts whose value IS the assertion, not a backstop) are kept as asyncio.wait_for or await_until(timeout=N) with # flake-ok: markers; converting them would remove the very property the test exists to catch.

What changed (motivation → approach → change)

Each short fixed-wait site falls into one of these shapes, and is replaced accordingly:

  • asyncio.wait_for(event.wait(), N) → await await_until(event) — awaits the Event directly under the default lost-run ceiling (half the test timeout)
  • asyncio.wait_for(task_or_future, N) → await await_until(task_or_future) — same for any Future
  • asyncio.wait_for(coro(), N) → await await_until(asyncio.ensure_future(coro())) — schedules the coroutine as a Task and awaits it
  • asyncio.gather(...) where the result is a Future — passed directly to await_until
  • time.sleep(N) inside a polling loop → wait_until(predicate, timeout=T) — polls the real condition
  • asyncio.sleep(N) inside an async polling loop → await async_wait_until(predicate)

Latency assertions — sites where the timeout value IS the assertion, kept with explicit bounds and # flake-ok: markers:

  • test_acp_frame_record.py (3 sites): record_frame calls in wedged-writer tests. timeout=0.5 proves record_frame is non-blocking while the writer holds a gate for ~10 s.
  • test_acp_frame_record_provider_safety.py (1 site): _read_one helper. The docstring says "a timeout here IS the failure this module exists to catch."
  • test_acp_runtime_exit_watcher.py (2 sites): await_until(watch_task, timeout=10.0) — the 10 s bound proves the exit watcher fires well before the child holding stdout exits at 60 s. A regression to await process.wait() would take up to 60 s and slip past an unbounded ceiling.

Non-raising retry fix — test_pid_lifecycle_provider_tree.py _reap: the identity-read retry wait_until is wrapped in try/except AssertionError: pass so unreadable pids still fall through to the unconfirmable path (original loop's fall-through behavior preserved).

Outer deadline fix — test_acp_sandbox_init_classification.py settle-stderr test: await_until(..., timeout=5) with except (TimeoutError, AssertionError) to catch both the internal settle timeout and the outer lost-run guard.

Baseline: 13 file entries removed from test/wait_discipline_baseline.txt (ratchet clicks to zero for each).

Backwards compatibility

Compatible: test-only change; no production code paths altered.

Removes nothing: no API, configuration or public function is removed or renamed.

Tests

All 13 modified test files pass locally. The wait-discipline ratchet (test/test_wait_discipline_ratchet.py) passes with all 13 baseline entries removed.

  • test/test_acp_client.py — 6 sites migrated
  • test/test_acp_frame_record.py — 2 sites migrated, 3 flake-ok (latency assertions)
  • test/test_acp_frame_record_provider_safety.py — 4 sites migrated, 1 flake-ok (latency assertion)
  • test/test_acp_refactor_runtime_start.py — 12 sites migrated
  • test/test_acp_refactor_transport_framing.py — 8 sites migrated
  • test/test_acp_runtime_exit_watcher.py — 8 sites migrated, 2 flake-ok (fail-fast bound)
  • test/test_acp_sandbox_init_classification.py — 7 sites migrated (outer deadline fixed)
  • test/test_acp_spawn_offload.py — 8 sites migrated
  • test/test_gatewayd_more_coverage.py — 8 sites migrated
  • test/test_gatewayd_self_exit.py — 10 sites migrated
  • test/test_pid_lifecycle_provider_tree.py — 13 sites migrated (retry fall-through preserved)
  • test/test_runtime_cleanup_regression.py — 13 sites migrated
  • test/test_runtime_ownership_founding.py — 15 sites migrated

Manual verification

N/A — unit coverage sufficient. These are test-file changes only; no user-visible behavior is altered.

Screenshots / video

N/A — no user-visible frontend change.

Related Issues

Part of wave-2 fixed-wait migration.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 16, 2026 11:35
@CrysisDeu
CrysisDeu requested a review from mbriones98 October 16, 2026 11:35
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e8876e0f514aea5ce02895b350f6188434e4d480 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision e8876e0f514aea5ce02895b350f6188434e4d480 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 16, 2026
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e8876e0f514aea5ce02895b350f6188434e4d480 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check the finally-block reaps in test_acp_runtime_exit_watcher.py: proc.wait() now waits up to the default ceiling, not 5 s, so a stuck reap uses half the test budget.

[DESIGN-REVIEWED] e8876e0

Description read

[DESCRIPTION-READ] 05ca4113e228a04d6de1de843158a357a13f37d9b494f5c9ec3cdeeedbcc1335

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e8876e0f514aea5ce02895b350f6188434e4d480 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e8876e0

Verdict parsed from the review's SHA-scoped output markers for commit e8876e0f514aea5ce02895b350f6188434e4d480.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e8876e0f514aea5ce02895b350f6188434e4d480: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e8876e0f514aea5ce02895b350f6188434e4d480 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e8876e0

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e8876e0f514aea5ce02895b350f6188434e4d480: <one-sentence reason>

@CrysisDeu CrysisDeu changed the title test(acp+runtime+gatewayd): replace fixed waits with signal waits test(wait-migration): replace fixed waits with signal waits Oct 16, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@CrysisDeu
CrysisDeu force-pushed the unflaky/wait-w2-d branch 3 times, most recently from ce4e4db to 93d25ff Compare October 16, 2026 12:56
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

  • No sensitive path changed; nothing to check.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 16, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@CrysisDeu
CrysisDeu force-pushed the unflaky/wait-w2-d branch 2 times, most recently from 52feba0 to 3cd6d23 Compare October 16, 2026 13:35
Replace short fixed-wait sites across 13 test files with signal-based
waits from kiro_crew.testing.wait.

Migration patterns:
- asyncio.wait_for(event.wait(), N) -> await_until(event)
- asyncio.wait_for(task/future, N) -> await_until(task/future)
- asyncio.wait_for(coro(), N) -> await_until(asyncio.ensure_future(coro()))
- time.sleep inside polling loops -> wait_until(predicate, timeout=T)
- asyncio.sleep inside async polling loops -> await async_wait_until(predicate)

Latency assertions kept as wait_for or await_until(timeout=N) with
# flake-ok: markers (timeout value IS the assertion, not a backstop):
- test_acp_frame_record.py: 3 record_frame calls in wedged-writer tests
- test_acp_frame_record_provider_safety.py: _read_one helper
- test_acp_runtime_exit_watcher.py: 2 real-subprocess watch_task awaits;
  timeout=10.0 proves watcher fires before child exits at 60 s

Other fixes:
- test_pid_lifecycle_provider_tree.py _reap: wrap identity-read wait_until
  in try/except AssertionError to preserve the unconfirmable fall-through
- test_acp_sandbox_init_classification.py: add timeout=5 to settle_stderr
  await_until; catch (TimeoutError, AssertionError) for both expiry paths
- .github/black-baseline.txt: remove gatewayd files (now black-clean)
- test/test_acp_client.py: fix isort import order

All 13 baseline entries removed from test/wait_discipline_baseline.txt.

Part of wave-2 fixed-wait migration.
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 16, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 16, 2026 15:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (15 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces fixed waits with signal waits in ACP, gatewayd and process-lifecycle tests; deadline loops become wait_until with the same bounds, test files and baselines only.

@KiroCrewCaptain
KiroCrewCaptain merged commit 3f17b61 into main Oct 16, 2026
102 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the unflaky/wait-w2-d branch October 16, 2026 15:07
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants