Skip to content

fix(ci): stop unmonitored check-runs holding the readiness re-fire - #38425

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/37840-readiness-sweep-monitored-running
Nov 17, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/37840-readiness-sweep-monitored-running

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A pending or failed PR Readiness verdict is re-fired once every lane readiness evaluates has finished, without waiting for workflows readiness never reads.

The readiness sweep holds a "check completed later" re-fire while a check-run on the head is still queued or running, so the last lane's completion recomputes the verdict once. It counts every check-run, whatever workflow it belongs to. A queued macOS Tests (on demand) run, which readiness does not evaluate, therefore keeps PR Readiness pending after every monitored lane is done: PR #37795's 05:00 and 05:16 ticks were held that way (#37840). The same count blocks the re-check-hold rescue from #37841, which needs no check-run left running.

Why it matters

PR Readiness is the one required status, so a held re-fire leaves a finished PR showing pending until the next tick that lets it through, a push, or a manual gh workflow run pr-readiness.yml. A macOS Tests (on demand) run can sit in its queue for hours, and for that whole time a frozen re-check hold on the same head cannot be rescued.

Not a goal

What changed (motivation → approach → change)

A queued run of a workflow readiness never reads, such as macOS Tests (on demand), no longer keeps a finished PR's PR Readiness pending: the sweep now waits only for check-runs of the lanes readiness waits on, the workflows in pr-readiness.yml's on.workflow_run.workflows. Before, .github/scripts/readiness_sweep_scan.py counted every non-completed check-run on the head as a lane still running.

The scan now asks GraphQL for each check-run's workflow name (checkSuite { workflowRun { workflow { name } } }, in both rollup queries) and counts a running check-run only when its workflow is in MONITORED_WORKFLOWS, the same 15 names as that trigger list. A check-run with no workflow run (another app's check) is still counted. A test pins the two lists equal. Completions are read as before: a finished check-run of any workflow can still re-fire a recompute, which at worst re-derives the same verdict. The sweep's comments, its re-check-hold log line and docs/ci/ci-and-reviews.md describe the new count.

The nested field adds no rate-limit points (single objects, not connections). Measured on live 25-PR pages of about 1,930 check-runs, it added no time: 8.7 s before and 8.4 s after, then 7.3 s and 7.3 s.

Backwards compatibility

Compatible: the scan's output keys and the sweep's arms are unchanged; a queued check-run of a workflow readiness does not evaluate just stops holding a re-fire.

Removes nothing: no key, path, record or validator is taken away; only the running count is narrower.

Tests

  • test_only_a_monitored_or_non_actions_check_run_counts_as_running: a queued macOS Tests (on demand) check-run is not counted, a queued CI one and one with no workflow run are, a finished unmonitored check-run is still the newest evidence, and both queries ask for the workflow name.
  • test_the_monitored_workflows_are_readinesss_workflow_run_list: MONITORED_WORKFLOWS equals pr-readiness.yml's on.workflow_run.workflows.
  • test_a_hold_on_a_head_with_every_check_completed_is_refired: follows the reworded log line.

Proof: PROVEN test/test_pr_readiness_sweep_recheck_hold.py::test_a_hold_on_a_head_with_every_check_completed_is_refired, test/test_readiness_sweep_scan.py::test_only_a_monitored_or_non_actions_check_run_counts_as_running[CI-1], test/test_readiness_sweep_scan.py::test_only_a_monitored_or_non_actions_check_run_counts_as_running[macOS Tests (on demand)-0], test/test_readiness_sweep_scan.py::test_only_a_monitored_or_non_actions_check_run_counts_as_running[None-1] (head dfd2d6a)

Manual verification

Reproduced and smoke-tested on Linux (Ubuntu 24.04). The sweep's own run: block and scanner replayed a live GraphQL scan of 50 open pull requests, recorded from this repository at 2026-11-17T10:17Z, with gh answered from the recording and the clock pinned to 10:20Z. PR #37459 was on a re-check hold from 09:23:42Z, had a monitored lane complete at 10:17:03Z, and nothing else running except two macOS Tests (on demand) check-runs.

Base (75051817195), held, nothing dispatched:

PR #37459: PR Readiness pending at 687e1ed54ea2ab2a1415b18c036fe9888499f01a has later check evidence, but 2 check(s) are still running; the last completion re-fires it.
--- dispatched
(none)

This branch, re-fired:

PR #37459: PR Readiness pending for 3378s with a check completed later at 2026-11-17T10:17:03Z at 687e1ed54ea2ab2a1415b18c036fe9888499f01a -> re-firing recompute.
--- dispatched
workflow run pr-readiness.yml --repo kirodotdev/KiroCrew -f pr=37459 -f sha=687e1ed54ea2ab2a1415b18c036fe9888499f01a

A head whose monitored lane is still running stays held: PR #38303, with 17 CI check-runs in progress, reads ... has later check evidence, but 20 check(s) are still running on this branch (21 on the base, one of them Sensitive Change Review). Across all 50 recorded pull requests only running_checks changed; every head's newest completion and readiness state were identical before and after.

Screenshots / video

N/A — CI scripts, a workflow comment and docs only; no user-visible frontend surface.

Related Issues

Part of #37840

Sibling sweep

Swept: git grep -nE 'running_checks|status.*!= *"COMPLETED"|select(.status *!= *"completed")' -- .github/ src/kiro_crew/builtin_skills/kirocrew-dev/kirocrew-prepare-pr/scripts/
Sibling: .github/workflows/pr-readiness.yml -- not affected -- each count there runs over one lane's runs or check-runs, already chosen by the lane list
Sibling: .github/scripts/await-fast-gate.sh -- not affected -- counts the jobs of one Fast Gate run only
Sibling: src/kiro_crew/builtin_skills/kirocrew-dev/kirocrew-prepare-pr/scripts/pr_status_verdict.py -- not affected -- the author's status poll reports every check on purpose, since a red macOS Tests (on demand) run is a concern the author answers

Pattern harvest

Not generalizable: one counter in one scan decided which check-runs were lanes; that rule now lives in one constant pinned to readiness's trigger list.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

The readiness sweep waits to re-fire a pending or failure verdict while
another check-run on the head is still queued or running, so the last
lane's completion recomputes it once. The scan counted every check-run,
including workflows readiness never reads. A queued `macOS Tests (on
demand)` run held PR #37795's re-fire for two ticks after every
monitored lane had finished, and it also blocked the re-check hold
rescue, which needs no check-run left running.

The scan now reads each check-run's workflow name and counts a running
one only when that workflow is in pr-readiness.yml's workflow_run list
(MONITORED_WORKFLOWS, pinned to that list by a test) or the check-run
has no workflow run (another app's check). Completions of every
check-run stay evidence, so a stale verdict is still re-fired.

Part of #37840
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This is a scoped fix with a reproduction from replayed live data. The one cost it adds is a second copy of readiness's lane list, and a test keeps that copy equal to the original.

Notes

  • Item 2 — oversized: MONITORED_WORKFLOWS writes out by hand the 15 names in pr-readiness.yml's on.workflow_run.workflows, so the list now has two writers. Subtraction: delete the literal and have the scan read the names from pr-readiness.yml, which already holds them. That makes the drift test unnecessary too. It is only a note because the pin test already stops the two copies drifting apart.

What this change ships

Inventory (5 items) — 4 justified

Intent: let PR Readiness re-fire once every lane it evaluates has finished, without waiting on workflows it never reads. FIX.

  1. A queued or running check-run from a workflow readiness doesn't monitor (macOS Tests (on demand)) no longer holds the pending/failure re-fire — justified
  2. A new hard-coded list of 15 monitored workflow names in the scan, pinned to the readiness trigger list — oversized
  3. A re-check hold whose only running check-runs are unmonitored can now be rescued — justified
  4. Both rollup GraphQL queries now fetch each check-run's workflow name — justified
  5. The log line now reads "every monitored check-run on the head completed", and the comments and ci-and-reviews.md match it — justified

[FIRST-PRINCIPLES-REVIEWED] dfd2d6a

Description read

[DESCRIPTION-READ] 175e6f7374315ced84d6c42ad152d45f85a99b4495d045c2f6ab6cc88b5406b1

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: completions still count as evidence, so the verdict still lands on the final state. The worst case is extra recomputes, and a revert undoes them.

Notes

  • MONITORED_WORKFLOWS is built from the workflow_run trigger list, but the Goal says "every lane readiness evaluates", and readiness evaluates lanes that list leaves out. Sensitive Change Review is one (the reproduction already shows it dropping out of the count, 21 to 20). The fork checkrun: review lanes are others, if their check-suite resolves to their Fork * Review workflow. While one of those is still running, other lanes' completions are no longer held back. On fork PRs this brings back the per-completion re-fires that the debounce was added to stop (ci: Fork PR Description and Sensitive Change Review fail closed on a spent installation rate limit #34120). A re-check hold on such a lane would also be re-fired once per STALE_MINUTES while it stays open. A fix: also count every lane name in the verdict spec list, or pin that set instead of the trigger list.

[DESIGN-REVIEWED] dfd2d6a

Description read

[DESCRIPTION-READ] 095c03d796459e904384b0133ef5ca4a8ebb38f3fd9626c39382228d4369b52f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] dfd2d6a

Verdict parsed from the review's SHA-scoped output markers for commit dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d.

False positive or not applicable? A repository writer can comment:
/ai-review override fable dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] dfd2d6a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt dfd2d6afd18f66d9c3ae83ff69c933b7d0d0542d false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 17, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed head dfd2d6a and the related sweep tests pass on it. The narrower running count only decides whether the sweep sends a pending or failed verdict back for another recompute. The sweep never writes a passing status itself, and the green path never reads this count, so the worst a misclassified check-run can cause is an extra recompute. A check-run with no readable workflow name is still counted as running, and a test keeps the monitored list identical to the readiness trigger list. Nothing blocks.

@KiroCrewCaptain
KiroCrewCaptain merged commit eb37bae into main Nov 17, 2026
97 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/37840-readiness-sweep-monitored-running branch November 17, 2026 13:51
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants