Skip to content

fix: read an empty run listing through check-runs in three readers - #33367

Merged
bolichen97 merged 1 commit into
mainfrom
fix/32348-empty-run-listing-readers
Nov 9, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/32348-empty-run-listing-readers

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A PR Size approval, a Sensitive Change Review author comment and a merge-queue group no longer stall or fail when GitHub's run listing for a head comes back empty although the head's runs exist.

GitHub's actions/runs?head_sha= listing is a filtered search, and it has answered no runs for a head whose own check-runs named 30 of them (#23245). #32383 taught PR Readiness, the AI review override handler and republish_withheld_verdict.py to read that empty answer through the head's check-runs. Three more readers still take it as "no run" (#32348):

  • pr-size-override.yml records the approval, then prints "PR Policy has not run" and never re-runs PR Policy, so PR Size stays red.
  • sensitive-change-review-comment.yml fails with "No Sensitive Change Review run found", so the author's filled template never clears the check.
  • merge-queue-readiness.yml sees no run for any lane and, once its five-minute appear window passes, fails a group whose lanes ran.

Why it matters

Each reader turns a GitHub search glitch into a red check a person has to clear by hand, and its message points at the wrong cause ("has not run", "push to start one"). The size approval and the Sensitive Change Review comment are the documented ways to clear those checks, so when the glitch hits they silently do nothing. Once the merge queue is switched on, the same glitch would eject a green group and hold the groups behind it for a rebuild.

Not a goal

What changed (motivation → approach → change)

When one of these readers gets an empty run listing, it now asks the head's check-runs instead of concluding there is no run. Every Actions job's check-run links to its run, so the reader collects those run ids (every row, filter=all), fetches each run, and keeps only runs of its own workflow and event on that exact head. A run the check-runs name but that cannot be fetched is a failed read, never a missing run. A non-empty listing takes the same path as before.

Reader Empty listing before Empty listing now
pr-size-override.yml "PR Policy has not run", no re-run re-runs the head's newest PR Policy run
sensitive-change-review-comment.yml fails with "No Sensitive Change Review run found" re-runs the head's Sensitive Change Review run
merge-queue-readiness.yml every lane missing; the group fails after 300 s each lane's run is read and judged as usual

A failed fallback read keeps each reader's existing read-failure path: the size handler posts its "could not be read" notice, the comment workflow stops on the failed gh call, and the merge-queue poll marks the lanes errored and retries on the next tick. The check-runs read needs checks: read, so pr-size-override.yml, sensitive-change-review-comment.yml, its caller sensitive-change-review-approval.yml (a called workflow gets no scope its caller lacks) and the merge-queue job gain that one read scope. docs/ci/ci-and-reviews.md names the three readers beside #32383's and the merge-queue job's added scope.

Backwards compatibility

Compatible: the fallback runs only when the listing is empty, so every head whose listing answers takes the same path as before; checks: read adds a read scope and removes none.

Removes nothing: no reader, message or check is taken away; the old "no run" outcomes remain for a head whose check-runs name no run of that workflow either.

Tests

  • test/test_empty_run_listing_readers.py runs the real pr-size-override.yml and sensitive-change-review-comment.yml steps against a stubbed gh whose listings answer no runs: each re-runs the run the check-runs name (the size handler picks the head's newest PR Policy run, skipping another workflow's run and another head's; the comment workflow skips a newer pull_request_target run of another workflow), and a named run that cannot be read is a read failure, not "no run".
  • test/test_ci_fleet_routing_expression_parity.py runs the merge-queue poll with a listing that answers no runs every tick: the group is cleared from the check-runs, and an unreadable named run is logged as a read error, not a missing lane.
  • The permission pins in test_pr_size.py, test_sensitive_change_review.py and the merge-queue test include checks: read; test_workflow_token_scopes.py derives that scope from the new call.

Proof: PROVEN test/test_ci_fleet_routing_expression_parity.py::test_merge_queue_readiness_reads_runs_and_never_reruns_them, test/test_ci_fleet_routing_expression_parity.py::test_an_empty_listing_is_read_through_the_commits_check_runs, test/test_ci_fleet_routing_expression_parity.py::test_a_named_run_that_cannot_be_read_is_an_api_error_not_a_missing_run, test/test_empty_run_listing_readers.py::TestPrSizeOverride::test_an_empty_listing_re_runs_the_pr_policy_run_the_check_runs_name, test/test_empty_run_listing_readers.py::TestPrSizeOverride::test_a_named_run_that_cannot_be_read_is_a_read_failure (head 53df3b7)

Manual verification

Checked live that the fallback finds what it needs: on #32383's head, commits/<head>/check-runs?filter=all lists the Sensitive Change Review (a pull_request_target run) and PR Policy job rows, and each details_url names a run whose event, head_sha and path match what the readers filter on. The empty listing itself cannot be produced on demand, so the stubbed tests above stand in for it.

Screenshots / video

N/A: CI workflows and docs only; nothing a user sees in the app changes.

Related Issues

Closes #32348

Sibling sweep

Swept: git grep -nE 'head_sha[=" ]' -- .github/workflows .github/scripts scripts
Sibling: .github/workflows/pr-size-override.yml -- changed here
Sibling: .github/workflows/sensitive-change-review-comment.yml -- changed here
Sibling: .github/workflows/merge-queue-readiness.yml -- changed here
Sibling: .github/scripts/await-fast-gate.sh -- tracked in #33264
Sibling: .github/workflows/issue-gate-retrigger.yml -- tracked in #33264
Sibling: .github/scripts/fix_loop_metrics.py -- tracked in #33264
Sibling: .github/workflows/pr-readiness.yml -- not affected -- reads through gh_head_runs since #32383
Sibling: .github/workflows/ai-review-human-override.yml -- not affected -- reads through head_runs_from_check_runs since #32383
Sibling: .github/scripts/republish_withheld_verdict.py -- not affected -- applies the same fallback since #32383

Pattern harvest

Rule candidate: review-prompt
Pattern: a workflow reads an actions/runs head_sha listing and treats an empty answer as "no run" without reading the head's check-runs

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 8, 2026 19:30
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • .github/workflows/sensitive-change-review-approval.yml

  • .github/workflows/sensitive-change-review-comment.yml

  • @buluoray (2 approvals checked, none complete): missing the ## Sensitive change review heading

How to clear this check:

  1. The PR author posts the template below as a PR comment, with
    every field filled (N/A needs a reason). Keep the
    Head commit line: it must name the current head SHA.
  2. The check turns green within a minute, with no push.
  3. A new push resets this check: post it again for the new head.

Only the author's comment counts this way. A reviewer can instead
paste the filled template into an "Approve" review on this head.
Merging still needs a reviewer's approval.

## Sensitive change review
- Head commit: 53df3b7c291e70c51c90a6ab5ed5ea300913eb13
- What rule changed:
- Before -> after (what was allowed/blocked, now):
- Worst case if wrong (breaks commands / leaks secret / locks user out):
- Evidence checked (which sandbox result proves it):
- No regression:
  - Tools still work (which tools/commands were run, result):
  - Backward compatible (old config / old data / old callers still work, how checked):
  - Existing tests (which suites passed on this head):
- How to undo:

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change copies #32383's tested fallback (empty listing read through check-runs) into three more readers, only adds a read scope, and takes nothing away.

Notes

[DESIGN-REVIEWED] 53df3b7

Description read

[DESCRIPTION-READ] 6086ae1aa8f7b6c05c0f14fdffefbf78e0565e31306af526d6d84300e42c2e6e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This fixes linked issue #32348, and tests fail on the empty listing without it. Nothing blocks. The one concern is a sixth copy of a check-runs read the repository already has in three places.

Notes

What this change ships

Inventory (5 items) — 5 justified

Intent: when GitHub's run search wrongly comes back empty, a PR Size approval, a Sensitive Change Review comment and a merge-queue group still find their runs and clear. FIX.

  1. A PR Size approval re-runs the head's newest PR Policy run even when the run listing is empty — justified
  2. A Sensitive Change Review author comment re-runs its run even when the run listing is empty — justified
  3. The merge-queue poll reads each lane through the commit's check-runs instead of failing after 300 s — justified
  4. Four workflow tokens gain the checks: read scope — justified
  5. ci-and-reviews.md names the three new readers and the merge-queue job's added scope — justified

[FIRST-PRINCIPLES-REVIEWED] 53df3b7

Description read

[DESCRIPTION-READ] 36a2c31709d7e95a8fb4927bfc4e25fb0f550e1f7da516f5538836a4ca4eb6ca

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 53df3b7

Verdict parsed from the review's SHA-scoped output markers for commit 53df3b7c291e70c51c90a6ab5ed5ea300913eb13.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 53df3b7

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 53df3b7c291e70c51c90a6ab5ed5ea300913eb13 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 8, 2026 19:44
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 8, 2026
@bolichen97
bolichen97 force-pushed the fix/32348-empty-run-listing-readers branch from 115cbd1 to b5f84e3 Compare November 9, 2026 04:24
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/ci/ci-and-reviews.md and test/test_ci_fleet_routing_expression_parity.py since this head's CI ran (2caa69b -> 4a0cb55); CI re-running.

The new base carries #33377, which re-pins the dashboard server's lazy-import count. That stale pin was main's red: it failed test_the_lazy_imports_stay_inside_the_functions_that_need_them in Backend Tests shard 6 (Linux and Windows), and Coverage Gate with it, on this head's merge ref. The test passes on the new base.

The same push makes the Sensitive Change Review comment test's decoy a newer pull_request_target run of another workflow, so the fallback's path filter is pinned, and lists contents: read among the merge-queue job's scopes in docs/ci/ci-and-reviews.md. Head: b5f84e3.
[operator: bolichen97]

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 9, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Three CI readers now read an empty actions/runs head_sha listing through the head's check-runs instead of concluding "no run" (.github/workflows/pr-size-override.yml:165-201, sensitive-change-review-comment.yml:83-106, merge-queue-readiness.yml:197-236). Non-empty listings take the old path, a failed fallback read keeps each reader's existing read-failure path, and the only scope added is checks: read. GPT's block is a test-skip ratchet rule, and Sensitive Change Review still needs the author's template.

@bolichen97

Copy link
Copy Markdown
Contributor Author

Suggestion for the sensitive-change-review-comment.yml part: one more case makes this helper print "No Sensitive Change Review run found", and the empty-listing fallback here does not reach it.

Merged or closed PR. GitHub fills a workflow run's pull_requests only while the PR is open. After merge, the runs listing still returns the head's Sensitive Change Review run, but with pull_requests: []. The unchanged filter below the new block, .pull_requests | any(.number == $pr), then drops that run. A review submitted at or after merge reaches this through sensitive-change-review-approval.yml. The listing is not empty, so the new check-runs fallback never runs. If it did run, its runs would still go through the same clause. Seen on #33425, #33741, #33739 and #33254: each had a successful pull_request_target run on its head, and each helper failure came at or after its merge.

Either of these would cover it:

  • drop the .pull_requests | any(.number == $pr) clause, because head_sha plus event == "pull_request_target" already pin the run; or
  • read .state from the pulls/$PR call the step already makes, and exit 0 with a note when the PR is not open.

Second mode, PR still open. The wait loop gives an in-progress run 150 s (30 x 5 s). When Sensitive Change Review is still running past that, the step fails with "run ... did not finish in time". Seen on #29609 and #33818. Re-running the helper after Sensitive Change Review finishes clears it, so this may only need a note, not a code change.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=3ea2e42fc531 Supply jq instead of skipping the new workflow cases

Rebutted: jq is the interpreter of the subject here, not a tool the harness can stand in for. The new cases run the real poll, size-override and comment steps, and the fallback they pin is written in jq: the jq -rs read of run ids out of each check-run details_url, and the jq -cs rebuild of the runs listing filtered by path, event and head.
A stand-in that answered those programs would reimplement them, and the tests would then check the stand-in instead of the workflow. The supplied-jq case in testing-conventions.md is three fixed calls (-Rs ., -r '.title', -r '.body // ""') around a step whose logic is shell; the same ruling holds on #32383 (span 05fc40eb8d6a).
The skips are the harnesses' own guards, unchanged by this diff: _needs_bash_and_jq in test/test_ci_fleet_routing_expression_parity.py, already on six earlier poll tests, and _fork_child_env in test/ai_review_workflows_helpers.py:215-217. Each checks only the host tool and cannot fire on the subject failing.
Dropping the jq term without a stand-in turns the skip into a red on a jq-less host that says nothing about the subject. CI runners carry jq, so the cases run there.
[operator: bolichen97]

@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] Thanks, both modes check out. #33882 merged at 20:14:57Z, and its approval helper failed one second earlier with "No Sensitive Change Review run found": the listing answered the run, and the .pull_requests | any(.number == $pr) clause dropped it.

Rebutted for this PR: out of its goal. This PR is about an EMPTY listing read as "no run". In the merged case the listing is not empty, and the clause that drops the run is unchanged from main, which is the "A listing that answers some runs but not all" line under Not a goal. The fallback added here runs only on an empty listing, so it neither causes the merged case nor makes it worse. A merged or closed PR's check no longer gates anything, so the cost is a red helper run, not a stuck PR.

Both modes and your two options are in #34113, so they can land as their own fix.

The actions/runs head_sha listing is a filtered search that can answer
no runs for a head whose runs exist. #32383 made PR Readiness, the AI
review override handler and the withheld-verdict republisher read that
empty answer through the head's check-runs. Three readers still took it
as "no run": a PR Size approval was recorded but PR Policy was not
re-run, a Sensitive Change Review author comment failed with "No
Sensitive Change Review run found", and the merge-queue poll would fail
a green group once its five-minute appear window passed.

Each now reads an empty listing through the head's check-runs (every
row, filter=all), fetches the runs their Actions job links name, and
keeps those of its own workflow and event on that exact head. A named
run that cannot be read is a read failure, never an absent run. The
four workflows involved gain checks: read for that read.
@bolichen97
bolichen97 force-pushed the fix/32348-empty-run-listing-readers branch from b5f84e3 to 53df3b7 Compare November 9, 2026 22:27
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/ci/ci-and-reviews.md since this head's CI ran (4a0cb55 -> fd5d9ef); CI re-running.

The new base carries #33695, which re-pins the owner-marked pid reader count to 9. That stale pin was main's red: it failed test_owner_marked_sites_match_the_recorded_count in the Static Ratchet Gate, so Fast Gate failed, and Await Fast Gate and Coverage Gate with it, on this head's merge ref. The test passes on the new base. The patch is unchanged; only the body's Proof line names the new head. Head: 53df3b7.
[operator: bolichen97]

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 9, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 9, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve on 53df3b7. Rebase only: git range-diff of the previously approved b5f84e3 against 53df3b7 is 1: b5f84e35d77 = 1: 53df3b7c291, so the patch is byte-identical. Three CI readers (.github/workflows/pr-size-override.yml:165-201, sensitive-change-review-comment.yml:83-106, merge-queue-readiness.yml:200) fall back to the head check-runs when the actions/runs head_sha listing is empty; the only scope added is checks: read. All AI lanes on this head report no findings. Still needs the owner: Sensitive Change Review for the .github/workflows changes.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 9, 2026
@bolichen97
bolichen97 merged commit 01fcf6d into main Nov 9, 2026
97 of 99 checks passed
@bolichen97
bolichen97 deleted the fix/32348-empty-run-listing-readers branch November 9, 2026 23:23
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three more workflows read an empty head_sha run listing as no run

2 participants