Skip to content

test(auto-improvement): drop the 10 s bound from the coverage suite's watcher waits - #39544

Closed
bolichen97 wants to merge 1 commit into
mainfrom
crew/linuxflackyguard/pr-watchers-exhausted
Closed

bolichen97 wants to merge 1 commit into
mainfrom
crew/linuxflackyguard/pr-watchers-exhausted

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Stop test_the_pass_fix_is_exported_to_the_pr_queue from failing Windows backend shard 5 on unrelated PRs.

Since this morning the test fails on Backend Tests (Windows) (5) with AssertionError: watcher never reached {'exhausted'}, also on rerun. The last snapshot is always status: nudging, nudges: 1, lastNote: pass 1/1: the watcher is still inside its one pass when the test gives up.

Why it matters

The shard is required, so every PR it lands on is red, whatever it changes. #39485 records it on #39301, #39479, #38940 and #39475 within a few hours.

Not a goal

  • No change to the PR watcher, its git safety pins or how many git children a pass starts.
  • No change to the Windows CI worker count.
  • No widening of test/test_wait_discipline_ratchet.py to the in-package app suites (the docs now say it does not scan them).

What changed (motivation → approach → change)

#39545 has since landed the same fix for test_pr_watchers.py itself, so this PR now carries only what that one did not.

Cause, for the record: one watcher pass with an isolated clone starts about 40 git children. That is 0.24 s on Linux and more than 10 s on a Windows shard running 6 xdist workers (#39390), so a wait capped at a fixed 10 s gave up mid-pass.

Change:

  • test_ai_pr_watchers_coverage.py had the same 10 s cap (WAIT_S) on its _await_status helper and on one is_alive wait. Both now use wait_until's default lost-run bound (half the test's --timeout). They still return as soon as the watcher's state is reached.
  • Testing-conventions D1 now says that test/test_wait_discipline_ratchet.py scans test/ only. That is why the in-package watcher suites were never flagged, so wait_until is the default there.

Backwards compatibility

Compatible: one test helper and one docs paragraph; no product code.

Removes nothing: only a test helper's optional timeout parameter, which no caller passed.

Tests

Proof: not provable -- test-only change

  • test_ai_pr_watchers_coverage.py 5/5 at -n 4; with test_pr_watchers.py, 237 passed at -n 4.
  • Before test(auto_improvement): wait on the lost-run bound in test_pr_watchers #39545 landed, this branch carried the same test_pr_watchers.py change. Its forced condition (each git child of the watcher thread delayed 0.3 s) failed 0/5 on main 5036a64 with the CI snapshot and passed 5/5 on the fix. Backend Tests (Windows) (5) passed on both of this PR's earlier heads (85a377b, 3612373).
  • flake8, mypy, test/test_flake_pattern_ratchet.py, diff-scoped semgrep and the comment-history gate pass.

Manual verification

N/A -- unit coverage sufficient; the change is a test wait helper.

Real-machine verification

Machine: cloud desktop, Amazon Linux 2, x86_64, 32 vCPU / 123 GiB. No Windows machine is available, so the Windows evidence is CI.
Before (main 5036a64): the export test 20/20 passed under 40 busy loops on 32 cores (Linux does not reproduce). Forced 0.3 s per git spawn -> 0/5 passed.
After (earlier heads 85a377b and 3612373): forced condition 5/5 passed; Backend Tests (Windows) (5) green on both.
Cleanup: own computer, no machine created.

Related Issues

Part of #39485

Sibling sweep

Swept: git grep -nE "WAIT_S|deadline = time.monotonic() +|PRWatcherRegistry(" -- src/kiro_crew/apps/builtins/auto_improvement/tests test
Sibling: src/kiro_crew/apps/builtins/auto_improvement/tests/test_pr_watchers.py -- not affected -- fixed on main by #39545
Sibling: src/kiro_crew/apps/builtins/auto_improvement/tests/test_ai_pr_watchers_coverage.py -- changed here
Sibling: src/kiro_crew/apps/builtins/auto_improvement/tests/test_reconcile_and_publish.py -- not affected -- it joins a stub thread it owns and starts no watcher pass
Sibling: src/kiro_crew/apps/builtins/auto_improvement/tests/test_dogfood_learnings_credentials.py -- not affected -- its registries never wait on a watcher status

Pattern harvest

Rule candidate: lint
Pattern: a fixed short poll deadline (time.monotonic() + N) in an in-package app suite, which test/test_wait_discipline_ratchet.py does not scan

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 19, 2026 17:34
@bolichen97
bolichen97 enabled auto-merge (squash) November 19, 2026 17:34
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 19, 2026
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 03edaa62da39bacea7829b1d5e2512dcab54c75b ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 03edaa62da39bacea7829b1d5e2512dcab54c75b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 03edaa62da39bacea7829b1d5e2512dcab54c75b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a test-only change. The fixed 10 s wait is replaced by wait_until's lost-run bound, which removes the race on loaded Windows shards. The tests still return as soon as the watcher reaches the state they wait for.

Notes

  • auto-improvement is pinned in apps/adoption-pins.json, so under AGENTS.md "No built-in apps" its fix should land in the app repository first and this PR should name it (App repo: <owner>/<repo>@<sha>). Otherwise the app repo's copy of this suite keeps the 10 s bound.
  • The new D1 sentence asks for wait_until by default in in-package app suites (src/kiro_crew/**/tests/), and those are the suites being moved out under Move every built-in app out to KiroCrewApps and forbid built-in apps #26967. It is better worded as a statement about where the ratchet does not scan than as a lasting convention for those suites.

[DESIGN-REVIEWED] 03edaa6

Description read

[DESCRIPTION-READ] 9d0a545fc080cdc3dcebdd0894bc16d9523e8b0c4277d68365eabdbb27a7a337

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 03edaa62da39bacea7829b1d5e2512dcab54c75b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 03edaa6

Verdict parsed from the review's SHA-scoped output markers for commit 03edaa62da39bacea7829b1d5e2512dcab54c75b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 03edaa62da39bacea7829b1d5e2512dcab54c75b false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 03edaa62da39bacea7829b1d5e2512dcab54c75b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 03edaa6

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 03edaa62da39bacea7829b1d5e2512dcab54c75b false-positive: <reason, 20 to 500 characters>

@bolichen97 bolichen97 changed the title test(auto-improvement): wait for PR watchers on the lost-run bound, not a 10s guess test(auto-improvement): drop the 10 s bound from the coverage suite's watcher waits Nov 19, 2026
@bolichen97
bolichen97 force-pushed the crew/linuxflackyguard/pr-watchers-exhausted branch from 3612373 to 03edaa6 Compare November 19, 2026 19:54
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 19, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Closing: the same fix landed on main in #39545 (test_pr_watchers.py now waits on the wait_until lost-run bound; this branch's copy of that file is identical to main). The only remaining diff was the 10 s cap in test_ai_pr_watchers_coverage.py, whose one status wait reads a READY verdict with no git work and has never failed, plus a docs sentence that #39545's docs change already covers. Nothing here is worth a separate merge.

@bolichen97 bolichen97 closed this Nov 19, 2026
auto-merge was automatically disabled November 19, 2026 20:01

Pull request was closed

@bolichen97
bolichen97 deleted the crew/linuxflackyguard/pr-watchers-exhausted branch November 19, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: checking Automated validation is still running

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant