Skip to content

test(startup): wait on signals, not clocks -- test_mcp_gatewayd_coverage, test_workflows_startup - #19928

Merged
CrysisDeu merged 1 commit into
mainfrom
unflaky/wait-migrate-startup
Oct 13, 2026
Merged

CrysisDeu merged 1 commit into
mainfrom
unflaky/wait-migrate-startup

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test/test_mcp_gatewayd_coverage.py and test/test_workflows_startup.py wait on the signal the code emits, never on a clock, so a slow runner can no longer red them.

Between them the two files held 63 short fixed waits: asyncio.wait_for(task, 5) on tasks and futures, asyncio.wait_for(event.wait(), 5) on asyncio Events, polling loops of asyncio.sleep(0.01) over 200 turns waiting for sweepers to fire, and asyncio.sleep(0.005) inside a composite polling helper. Each bet that CI is as fast as the machine the test was written on. #19248 added the helpers and froze the counts; this burns down all 63 of them.

Why it matters

A lost bet reds a pull request that did not touch these files. The gatewayd sweeper tests (polling loop + wait_for pairs) and the startup initialization tests (event/future/gather waits) are the densest cluster of such bets in the gateway and workflow areas.

Not a goal

  • Any other file's wait-discipline baseline line. Only the two migrated files' lines are deleted.
  • Any product code. Nothing outside test/ and the baselines changes.
  • The other sites in test/wait_discipline_baseline.txt. Sibling batches take them.

What changed (motivation → approach → change)

Every wait now rides on the thing it was waiting for, under the helpers' lost-run bound (half the test's timeout; expiry fails by name, quoting the signal and the last state read).

What the test waited for Was Now
an asyncio.Event the code sets (entered, cancelled, executed, ...) wait_for(event.wait(), 5) await_until(event)
a task, a future, or asyncio.shield(task) wait_for(task, 5..15) await_until(task)
asyncio.gather(task, return_exceptions=True) wait_for(gather(...), 5..15) await_until(gather(...))
a coroutine (drain, zombie diagnostic, sweeper) wait_for(coro, 5) await_until(create_task(coro))
sweeper fired once (poll 200 x sleep(0.01)) for-loop with sleep(0.01) async_wait_until(lambda: condition)
a composite state in _await_restart_write (entered set, startup raised, or initializer settled without the write) while not entered.is_set(): ... await sleep(0.005) with a 60 s clock guard async_wait_until(_settled) -- the async predicate raises the same assertions directly; the default_timeout() lost-run bound replaces the clock guard

The cancellation_delivered() local coroutine in one test (which polled initializer.cancelling() with asyncio.sleep(0)) is folded into async_wait_until(lambda: initializer.cancelling()) -- shorter and carries the lost-run bound.

Black baseline: test_mcp_gatewayd_coverage.py was in .github/black-baseline.txt (unformatted). Running black as part of this migration made it clean, so its line is removed from the baseline.

File-by-file:

  • test_mcp_gatewayd_coverage.py: 32 → 0 (21 wait_for on tasks/coroutines, 9 polling loops with sleep(0.01), 2 async_wait_until replacements for composite polling)
  • test_workflows_startup.py: 31 → 0 (30 wait_for on events/tasks/gathers/coroutines, 1 sleep(0.005) in helper)

Both wait-discipline baseline lines deleted from test/wait_discipline_baseline.txt. One line removed from .github/black-baseline.txt.

Backwards compatibility

Compatible: test-only change. No product code, no public helper's contract. _HANG_GUARD_SECS (a module-level constant private to test_workflows_startup.py) is removed with the polling loop that read it.

Tests

Both files: 212 tests total (158 gatewayd + 54 startup), run 3× at default worker count, all green. test/test_wait_discipline_ratchet.py fully green; neither file appears in the baseline any longer. Black, isort, flake8 on touched files: green. test_flake_pattern_ratchet.py: 53 passed.

Determinism proof (3 × -q runs, all 212 passed):

  • Run 1: 212 passed in 5.56s
  • Run 2: 212 passed in 5.34s
  • Run 3: 212 passed in 5.45s

Manual verification

N/A -- unit coverage sufficient: the change is to the tests' own waits, and the tests prove them.

Related Issues

Part of #19926

Part of the wait-discipline burn-down behind #19248.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 13, 2026 20:30
@CrysisDeu
CrysisDeu requested a review from mbriones98 October 13, 2026 20:30
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 13, 2026
@CrysisDeu
CrysisDeu force-pushed the unflaky/wait-migrate-startup branch from 6aec3df to ce18003 Compare October 13, 2026 20:41
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed ce18003ecfe1d3cd38e66f388e06a019bd1d459b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] ce18003

Verdict parsed from the review's SHA-scoped output markers for commit ce18003ecfe1d3cd38e66f388e06a019bd1d459b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable ce18003ecfe1d3cd38e66f388e06a019bd1d459b: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of ce18003ecfe1d3cd38e66f388e06a019bd1d459b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

await_until never cancels the task it awaits, so a test that used to rely on wait_for's cancel-on-timeout must now clean up that task in its own finally.

[DESIGN-REVIEWED] ce18003

Description read

[DESCRIPTION-READ] 20b43e692f1a2e0547186b2b483720a0a460acb01d777eefffafece7467a8431

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of ce18003ecfe1d3cd38e66f388e06a019bd1d459b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] ce18003

False positive or not applicable? A repository writer can comment:
/ai-review override gpt ce18003ecfe1d3cd38e66f388e06a019bd1d459b: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision ce18003ecfe1d3cd38e66f388e06a019bd1d459b ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision ce18003ecfe1d3cd38e66f388e06a019bd1d459b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Oct 13, 2026
@CrysisDeu
CrysisDeu enabled auto-merge (squash) October 13, 2026 22:10

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (4 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: moves two startup test files from clock sleeps to signal waits and drops them from the black and wait-discipline baselines, test hygiene only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants