Skip to content

test: wait on the single-flight starts by signal, not a 5 s clock - #23464

Closed
leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/single-flight-start-wait
Closed

leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/single-flight-start-wait

Conversation

@leonlaiyc

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_concurrent_starts_are_single_flight no longer fails just because the first start's status write takes more than 5 s on a loaded runner.

TestTransitionSerialization::test_concurrent_starts_are_single_flight failed with TimeoutError on Backend Tests (Windows) (8) for #20311, a PR that touches nothing in auto-research (#23446). The 5 s asyncio.wait_for at the first await expired while the first start was still inside asyncio.to_thread(lifecycle.update_campaign_status, ...).

Why it matters

A red on an unrelated PR costs its author a rerun, and on a fork that needs a maintainer. The test checks real ordering (the second start must wait for the first transition and get 409), but a slow runner decided the result, not that ordering.

Not a goal

  • Any change to auto_research/handlers.py or to the transition lock. The production behaviour is correct; only the test's waits change.
  • The threading.Event.wait(5) calls in the same file. Those are thread-side waits inside a stubbed worker, not asyncio bounds on the task under test.

What changed (motivation → approach → change)

  • Symptom: TimeoutError from asyncio.wait_for(first, timeout=5) when update_campaign_status runs longer than 5 s.
  • Cause: a fixed wall-clock bound used as the barrier. wait_for also cancels the awaited task when it expires, so the failure surfaces as a CancelledError inside the handler.
  • Change: the three waits in that test (entered, first, second) now use kiro_crew.testing.wait.await_until. That is the repo's replacement for asyncio.wait_for(signal, <short number>): it bounds only a lost run (half the test's timeout), never cancels the task, and names the signal on failure. The single-flight assertions are unchanged.
  • Sibling in the same file: TestDeleteIsFinal::test_a_workflow_poll_in_flight_cannot_resurrect_a_deleted_campaign awaited its poll task with the same asyncio.wait_for(poll, timeout=5), and gets the same wait. After this change the file has no asyncio.wait_for( left.

Backwards compatibility

Compatible: test-only change. No production code is touched.

Removes nothing: no test, assertion or case is removed; only the wait primitive changes.

Tests

Proof: not provable -- test-only change

Manual verification

N/A: test-only change. The reproduction above exercises the exact slow-runner condition from the issue.

Related Issues

Fixes #23446

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

test_concurrent_starts_are_single_flight bounded the first start, and
the wait for it to enter launch preparation, with asyncio.wait_for(...,
timeout=5). The first start's status write runs in a worker thread, so
on a loaded Windows runner it outlived 5 s, wait_for cancelled the
task, and the test failed with TimeoutError while proving nothing.

Await the event and both tasks with kiro_crew.testing.wait.await_until,
which bounds only a lost run and never cancels the awaited task. The
same file's workflow-poll delete test had the same fixed bound on its
poll task and gets the same wait.

Fixes kirodotdev#23446

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leonlaiyc
leonlaiyc requested review from a team and KiroCrewCaptain as code owners October 23, 2026 12:47
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 4f3ad58c9436f58ad8abd7e4c415ed0ec8be7efb ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 4f3ad58c9436f58ad8abd7e4c415ed0ec8be7efb touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 4f3ad58c9436f58ad8abd7e4c415ed0ec8be7efb via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a test-only change that swaps fixed 5 s asyncio.wait_for barriers for the repo's await_until helper, and the ordering assertions stay the same.

[DESIGN-REVIEWED] 4f3ad58

Description read

[DESCRIPTION-READ] 1d4ae7e02b854ecbd87a5165fc4b867b252caa187ef4b7431fe723e3139cd999

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 4f3ad58c9436f58ad8abd7e4c415ed0ec8be7efb via the fork AI-review pipeline; updated in place on each push.

Review details

The diff matches the helper's types: entered at 848 is an asyncio.Event, and first, second and poll (976) are futures. So await_until accepts each of them and returns the same result the old wait_for did.

No findings.

[OPUS-REVIEWED] 4f3ad58

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 4f3ad58c9436f58ad8abd7e4c415ed0ec8be7efb via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4f3ad58

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Oct 23, 2026
@leonlaiyc

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #23376 (merged 2026-10-23 16:14 UTC). It makes the same asyncio.wait_for(..., 5) -> await_until replacements in TestTransitionSerialization and TestDeleteIsFinal, plus the rest of the file, and removes the file's wait_discipline_baseline.txt line. On current main, test/test_auto_research_campaign_contract.py has no asyncio.wait_for( left.

This PR's red Backend Tests shards were caused by this PR, not a flake: test_the_baseline_is_not_stale failed because this PR left the baseline at 5 when the real count was 1. #23376 already includes that change, so nothing here needs to be carried over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_concurrent_starts_are_single_flight times out when the first start's status write takes over 5 s

1 participant