Skip to content

fix(readiness): hold a fork PR into a non-default base for a maintainer - #31446

Merged
bolichen97 merged 1 commit into
mainfrom
fix/18530-fork-release-hold
Nov 6, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/18530-fork-release-hold

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A fork PR into a release branch gets a final readiness verdict that names why it is held, instead of a pending verdict the sweep re-checks forever.

A fork PR whose base is a release branch kept PR Readiness at pending for the life of its head, and the self-heal sweep re-dispatched a recompute every tick. #18441 got 17 identical publications in six and a half hours: [read-failed] 8 readiness check(s) still pending; waiting on Internal Content Scan (not started) (+7 more).

Builds on #25224 by @mbriones98.

Why it matters

An outside contributor backporting a fix to a release branch had no way to reach a final verdict, and each affected PR cost two wasted readiness runs per half hour. The only workaround so far was re-opening the PR from a same-repo branch, which needs push access.

Not a goal

What changed (motivation → approach → change)

A fork PR into a release branch now gets a final, named failure instead of an endless pending. It stays held for a maintainer.

Symptom → cause. fast-gate.yml only runs on PRs to main, and all seven Stage-2 fork lanes fire on Fast Gate completing, so on a fork PR into another base none of them ever runs. pr-readiness.yml still scored them (not started). The supersession gate then found no lane comment at all, and readiness stamped that as a read failure. The sweep re-fires any [read-failed] pending on age, and the recompute re-derived the same pending.

Change. Such a PR gets fork PR into a non-default base needs maintainer review. It never reaches passed, because none of the fork scan or review coverage runs on it while its base is another branch. The sweep re-fires a failure only when a check, review or comment lands after the verdict, never on age, so the loop stops. Nothing in the product clears the hold; a maintainer reviews the PR and decides the merge.

A base change keeps the SHA, so a verdict scored for the old base could land on the status after the base moved. The publish step reads the base right before the status POST and publishes nothing when it moved. When that read fails, a success is published as a [read-failed] pending instead, so it cannot stand for a base it was not scored for.

Situation on a fork PR into a non-default base Before After
Fork lane whose Fast Gate trigger never ran pending (not started) skipped (only runs on PRs to main)
Verdict a lane posted while the base was main (red included) read still read and still counts
Supersession gate cannot evaluate [read-failed] pending still listed; the hold publishes failure first
Overall verdict pending forever failure, named hold
Base changes while a run is publishing old-base verdict lands nothing published; base unreadable: a success becomes a [read-failed] pending

Fork PRs into main and same-repo PRs into any base get the same verdict as before. docs/ci/ci-and-reviews.md names the hold under Fork PRs.

Backwards compatibility

Compatible: only fork PRs into a non-default base change, and they could never reach a verdict before.

Removes nothing: no lane, key or check is removed; lanes that cannot run on this base are listed as skipped, and the PR is held rather than passed.

Tests

test/test_pr_readiness_fork_base.py runs the evaluate step for a fork PR into a release branch: the hold is final, named and free of [read-failed] when no lane ran; the PR does not pass even with every lane green; a red lane posted before a base change still counts. Two controls pin that a fork PR into main and a same-repo PR into a release branch are not held. test/test_pr_readiness_publish.py adds a verdict scored for an older base publishing nothing, and an unreadable base turning a success into a stamped pending.

Proof: PROVEN test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_the_hold_is_terminal_named_and_not_a_read_failure, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_it_does_not_pass_even_with_every_lane_green, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_a_red_lane_posted_before_the_base_changed_is_kept, test/test_pr_readiness_publish.py::test_a_verdict_scored_for_an_older_base_publishes_nothing, test/test_pr_readiness_publish.py::test_an_unreadable_base_publishes_a_stamped_pending (head 1a4363c)

Determinism, on the seven new or changed test ids: 20 runs at -n0, all green; 10 runs at -n 4 --dist loadgroup --max-worker-restart=2 with test/test_pr_readiness_evaluate.py and test/test_pr_readiness_sweep.py as co-tenants, all green; one shuffled order (seed=17805), green.

Manual verification

N/A — the evaluate and publish steps are executed end to end by the test harness with stubbed GitHub reads; nothing was dispatched on a live PR.

Related Issues

Fixes #18530

Sibling sweep

Swept: git grep -n "fast-gate.yml" -- .github/workflows/
Siblings: none -- the fork lane specs are read in one place, the evaluate step of pr-readiness.yml, changed here

Pattern harvest

Rule candidate: review-prompt
Pattern: an "unknown" answer from a gate stamped as a read failure on a population that structurally can never exist, so an age-based retry loops forever

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 6, 2026 14:02
@bolichen97
bolichen97 requested a review from Zedmor November 6, 2026 14:02
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 5b26410b711dcd0f3b24251b110acc8bfe084e51 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 5b26410b711dcd0f3b24251b110acc8bfe084e51 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The hold fixes the cause. The base re-read on publish is a rider that applies to every PR, while the description says only fork PRs into another base are affected.

Notes

  • Item 5 — rides along: the rider covers every PR, which contradicts the description's "Fork PRs into main and same-repo PRs into any base get the same verdict as before". A failed baseRefName read now turns any success into a [read-failed] pending. The SHA race it closes already existed for same-repo stacked PRs. Subtraction: replace the separate base read with one more field on the head read already at pr-readiness.yml:2146 (--json headRefOid,baseRefName). That reuses the existing staleness exit and drops one API call per publish.
  • Item 6 — oversized: "an unreadable base turns a success into a stamped pending" goes beyond the Goal, which is about getting a final verdict for a fork PR into a release branch. Subtraction: delete the failed-read branch and let the existing gh_retry failure behave like the head-SHA read beside it. Clears when: the mechanism is removed, or a human amends the Goal.

What this change ships

Inventory (7 items) — 5 justified

Intent: let an outside contributor's PR into a release branch get a final, named readiness verdict instead of a pending that loops forever. This is a FIX (#18530, repro #18441).

  1. A fork PR into a non-default base gets the terminal failure "fork PR into a non-default base needs maintainer review" instead of an endless pending — justified
  2. Fork lanes whose Fast Gate trigger never ran show as skipped "(only runs on PRs to main)" instead of "(not started)" — justified
  3. A lane verdict posted while the base was main still counts on the held PR — justified
  4. The status description names the hold and counts the other blockers — justified
  5. Any PR whose base changed after scoring publishes nothing — rides along, applies to all PRs
  6. A failed base read downgrades any PR's success to a [read-failed] pending — rides along, oversized
  7. The hold is documented under Fork PRs in ci-and-reviews.md — justified

[FIRST-PRINCIPLES-REVIEWED] 5b26410

Description read

[DESCRIPTION-READ] c65e639b018f1a11a6ca6c5bda507f75834af6c57fa41544da6771aa69229a41

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 5b26410b711dcd0f3b24251b110acc8bfe084e51 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change replaces a pending that could never resolve with a named, final hold, it removes no reader, and a revert walks it back cleanly.

Notes

[DESIGN-REVIEWED] 5b26410

Description read

[DESCRIPTION-READ] 53d69b6dd5d769410009a3d5e37b88387657b747568e8e971724cb3af7a077ef

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 5b26410b711dcd0f3b24251b110acc8bfe084e51 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 5b26410

Verdict parsed from the review's SHA-scoped output markers for commit 5b26410b711dcd0f3b24251b110acc8bfe084e51.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 5b26410b711dcd0f3b24251b110acc8bfe084e51 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 5b26410b711dcd0f3b24251b110acc8bfe084e51 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 5b26410

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 5b26410b711dcd0f3b24251b110acc8bfe084e51 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 6, 2026 14:13
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 6, 2026
Fast Gate only runs on PRs to the default branch, and every Stage-2 fork
lane fires on Fast Gate completing, so a fork PR into a release branch
gets no run of any of them. The evaluate step scored those lanes (not
started) and read the supersession gate's empty answer as a read
failure, so the verdict stayed at a [read-failed] pending the sweep
re-fired a recompute for on every tick.

Such a PR now gets a terminal failure, 'fork PR into a non-default base
needs maintainer review', with no [read-failed] token, so the sweep no
longer re-fires it on age. A fork lane whose trigger never ran for the
head is listed as skipped; a verdict a lane already posted while the
base was the default branch is still read, a red one included.

The verdict is scored for one base as well as one head, and a base
change keeps the SHA. The publish step reads the base right before the
status POST and publishes nothing when it moved; when that read fails,
a success is published as a stamped pending instead of standing for a
base it was not scored for. Every other PR shape is evaluated as before.

Builds on #25224.

Fixes #18530

Co-authored-by: Matthew Briones <50031759+mbriones98@users.noreply.github.com>
@bolichen97
bolichen97 force-pushed the fix/18530-fork-release-hold branch from 1a4363c to 5b26410 Compare November 6, 2026 16:31
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in .github/workflows/ci.yml since this head went green (40e8300 -> 360162b); CI re-running.

[operator: bolichen97]

@bolichen97

bolichen97 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor Author
  • An unreadable base lets a stale hold overwrite current success span=2588de2f2c6c

Rebutted: the trigger needs one unchanged fork head retargeted main -> release -> main, plus all three tries of the final base read failing right after the main-base run published. What it leaves is a visible red on the required check, which the next monitored event or a re-run republishes. Downgrading every verdict the run cannot re-check to a [read-failed] pending is what makes the sweep re-fire it on age, the loop this PR removes. Only a success is downgraded because a success published for the wrong base is the one outcome that can merge. Adjudication already downgraded this as a disproportionate remedy; the code is unchanged and this rebase carries the identical patch.

[operator: bolichen97]

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 6, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by Crew Captain for the on-call: checks green, PR Readiness passed, review lanes clean on this head.

@bolichen97
bolichen97 merged commit 24f8a91 into main Nov 6, 2026
98 checks passed
@bolichen97
bolichen97 deleted the fix/18530-fork-release-hold branch November 6, 2026 17:41
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(readiness): a fork PR into a release branch never resolves, and the sweep re-fires a recompute forever

2 participants