Skip to content

test(mcp): replace fixed wait in handlers_mcp_coverage with await_until - #20137

Merged
javenciu merged 1 commit into
mainfrom
test/wait-discipline-handlers-mcp-coverage
Oct 14, 2026
Merged

javenciu merged 1 commit into
mainfrom
test/wait-discipline-handlers-mcp-coverage

Conversation

@javenciu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make the last short fixed wait in test/test_handlers_mcp_coverage.py wait for the event the stubbed code sets, so the test no longer depends on how fast the runner is.

test_starting_a_pass_clears_the_previous_pass_counts replaces the background measure-all body with a stub that sets an asyncio.Event, then waited for it with asyncio.wait_for(started.wait(), timeout=1) (line 3331 on main). One second is a bet that the runner is as fast as the machine the test was written on. test/test_wait_discipline_ratchet.py counts that site against the file's line in test/wait_discipline_baseline.txt.

Why it matters

A fixed short wait is the commonest cause of a flaky backend test in this repository (the module docstring of test/test_wait_discipline_ratchet.py says so), and it fails on slower CI runners first. Removing the site lowers the baseline, and the ratchet then keeps this file from gaining another.

Not a goal

  • Other test files' waits: each file has its own baseline line and its own change.
  • Changing what the test asserts or the handler it exercises (api_mcp_measure_start).
  • Two other timed calls in the same test, which the ratchet does not count. The sleep inside _slow_update (test/test_handlers_mcp_coverage.py:2495) deliberately holds a worker thread so the cancellation lands mid-write. The threading.Event wait passed to asyncio.to_thread in test_a_cancelled_request_still_waits_for_the_write_to_finish (test/test_handlers_mcp_coverage.py:2506) is not a signal await_until accepts, because it takes an asyncio.Event or a future. Both are a different mechanism from this change.

What changed (motivation → approach → change)

The ratchet's rule is "wait on signals, not clocks": a test awaits the event the code emits, under a lost-run bound (half the test's own timeout by default) that only turns a hang into a named failure. The helper for an asyncio.Event is kiro_crew.testing.wait.await_until.

  • test/test_handlers_mcp_coverage.py imports await_until and replaces the one site with await await_until(started, what="the stubbed measure-all body starting") (line 3332 at this head). await_until is a coroutine function, so the await in front of it stays.
  • test/wait_discipline_baseline.txt loses the line 1 test_handlers_mcp_coverage.py, which is now at zero.

Rejected: keeping wait_for and raising its number. The ratchet counts any ceiling under 30 s, and a larger constant is still a clock bet.

Backwards compatibility

Removes nothing: this is a test-only change; no production code, public API or protocol is modified.

Tests

In short: with the baseline line deleted and the test file untouched the ratchet fails; with the site replaced it passes, and the file's own tests are unchanged.

  • Red (baseline line deleted, test file untouched), pytest test/test_wait_discipline_ratchet.py -n0 -q: 1 failed, 38 passed, with the report test/test_handlers_mcp_coverage.py: 0 -> 1 and the site test/test_handlers_mcp_coverage.py:3331 wait_for(1s) (main's line numbers).
  • Green, the ratchet plus test/test_handlers_mcp_coverage.py at -n0, three runs: 205 passed each (39 ratchet tests and 166 in the file; the file had 166 before the change too).
  • pytest test/test_handlers_mcp_coverage.py -n0 -W error::RuntimeWarning -W error::pytest.PytestUnraisableExceptionWarning: 166 passed. With the await in front of await_until removed, the same flags fail that test on coroutine 'await_until' was never awaited, so a dropped await would be caught.
  • The file, then test/test_handlers_memory_coverage.py and test/test_handlers_taskrunner_coverage.py, in one process at -n0: 636 passed.
  • isort --check-only, flake8 and git diff --check are clean on the touched file. The file is listed in .github/black-baseline.txt, and black --diff reports the same hunks before and after this change.

Manual verification

N/A: unit coverage is sufficient. The change swaps one test wait for the repository's wait helper and touches no runtime code.

Related Issues

No issue exists; burn-down of the wait-discipline baseline (precedents #19928, #20014, #19527).

Pattern harvest

Not generalizable: this site is one more instance of the class the ratchet already catches (rule D1 of docs/system-specs/common/testing-conventions.md), so the gate stops the class and no new rule is needed.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 14, 2026 05:23
@javenciu
javenciu requested a review from mbriones98 October 14, 2026 05:23
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 3d10ea3aef46c74be7158f83991063a5f4434704 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 3d10ea3aef46c74be7158f83991063a5f4434704 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 3d10ea3aef46c74be7158f83991063a5f4434704 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] 3d10ea3

Description read

[DESCRIPTION-READ] 3953911262f4620985dc53e69ca2b540803b825ccfc1c09398d06d3172e9b268

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 3d10ea3aef46c74be7158f83991063a5f4434704 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 3d10ea3

Verdict parsed from the review's SHA-scoped output markers for commit 3d10ea3aef46c74be7158f83991063a5f4434704.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 3d10ea3aef46c74be7158f83991063a5f4434704: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 3d10ea3aef46c74be7158f83991063a5f4434704 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 3d10ea3

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 3d10ea3aef46c74be7158f83991063a5f4434704: <one-sentence reason>

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
The measure-start test waited for its stubbed background body with
asyncio.wait_for(started.wait(), timeout=1), a fixed one-second bet
that test_wait_discipline_ratchet.py counts against the file.
Await the event with kiro_crew.testing.wait.await_until, whose
lost-run bound is half the test's own timeout, and delete the
file's line from wait_discipline_baseline.txt.
@javenciu
javenciu force-pushed the test/wait-discipline-handlers-mcp-coverage branch from ebf4275 to 3d10ea3 Compare October 14, 2026 11:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@javenciu
javenciu enabled auto-merge (squash) October 14, 2026 13:17

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces a fixed asyncio.wait_for timeout with await_until in one test and drops that file from the wait-discipline baseline; test files only.

@javenciu
javenciu merged commit 2f33e7a into main Oct 14, 2026
113 of 175 checks passed
@javenciu
javenciu deleted the test/wait-discipline-handlers-mcp-coverage branch October 14, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants