Skip to content

fix(readiness): hold a fork PR into a non-default base for a maintainer - #24936

Closed
mbriones98 wants to merge 2 commits into
mainfrom
fix/fork-release-readiness-18530
Closed

mbriones98 wants to merge 2 commits into
mainfrom
fix/fork-release-readiness-18530

Conversation

@mbriones98

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A fork PR into a release branch gets a final readiness verdict that names why it is held, instead of a pending verdict the sweep re-checks forever.

A fork PR whose base is a release branch kept PR Readiness at pending for the life of its head, and the self-heal sweep re-dispatched a recompute every tick. #18441 got 17 identical publications in six and a half hours: [read-failed] 8 readiness check(s) still pending; waiting on Internal Content Scan (not started) (+7 more).

Why it matters

An outside contributor backporting a fix to a release branch had no way to reach a final verdict, and each affected PR cost two wasted readiness runs per half hour. The only workaround so far was re-opening the PR from a same-repo branch, which needs push access.

Not a goal

  • Running the fork scan and AI reviews on fork PRs into release branches (widening fast-gate.yml). That is a separate decision; this PR changes no fork permission or token scope.
  • Letting these PRs pass on their own. They stay held for a maintainer.

What changed (motivation → approach → change)

This fixes the loop but keeps these PRs held for a maintainer.

Symptom → cause. fast-gate.yml only runs on PRs to main, and all seven Stage-2 fork lanes fire on Fast Gate completing, so on a fork PR into another base none of them ever runs. pr-readiness.yml still scored them (not started). The supersession gate then found no lane comment at all, and readiness stamped that as a read failure. The sweep re-fires any [read-failed] pending on age, and the recompute re-derived the same pending.

Change. A fork PR into a non-default base now gets a final failure: fork PR into a non-default base needs maintainer review. It never reaches passed: none of the fork scan or review coverage runs on it, and nothing retargets it to main later the way a stacked PR is. The sweep re-fires a failure only when a check, review or comment lands after the verdict, never on age, so the loop stops.

Situation on a fork PR into a non-default base Before After
Fork lane whose Fast Gate trigger never ran pending (not started) skipped (only runs on PRs to main)
Verdict a lane posted while the base was main (red included) read still read and still counts
Supersession gate examined zero lanes, no other error [read-failed] pending nothing to evaluate
Supersession history unreadable or timed out [read-failed] pending still a read-failure entry
Overall verdict pending forever failure, named hold

Fork PRs into main and same-repo PRs into any base are unchanged.

Backwards compatibility

Compatible: only fork PRs into a non-default base change, and they could never reach a verdict before.

Removes nothing: no lane, key or check is removed; lanes that cannot run on this base are listed as skipped, and the PR is held rather than passed.

Tests

test/test_pr_readiness_fork_base.py covers the hold being final, named and free of [read-failed]; the PR not passing even with every lane green; a red lane and a lost reviewer block from before a base change still counting; a real supersession read failure still showing; the sweep not re-firing the hold, while it did re-fire the old pending; fork PRs into main and same-repo release PRs being unchanged; and the "no lane examined" text matching the gate's own wording.

Proof: PROVEN test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_the_hold_is_terminal_named_and_not_a_read_failure, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_it_does_not_pass_even_with_every_lane_green, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_a_red_lane_posted_before_the_base_changed_is_kept, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_a_dropped_block_is_kept, test/test_pr_readiness_fork_base.py::TestAForkPullRequestIntoANonDefaultBaseIsHeld::test_a_real_supersession_read_failure_still_surfaces[read-failed] (head 88f964f)

The readiness, sweep, supersession and fork-workflow suites pass, and actionlint is clean.

Manual verification

N/A — the evaluate step is executed end to end by the test harness with stubbed GitHub reads; nothing was dispatched on a live PR.

Related Issues

Fixes #18530

Sibling sweep

Swept: git grep -n "fast-gate.yml" -- .github/workflows/
Siblings: none -- the fork lane specs are read in one place, the evaluate step of pr-readiness.yml, changed here

Pattern harvest

Rule candidate: review-prompt
Pattern: an "unknown" answer from a gate stamped as a read failure on a population that structurally can never exist, so an age-based retry loops forever

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Fast Gate only runs on PRs to the default branch, and every Stage-2 fork
lane fires on Fast Gate completing, so a fork PR into a release branch gets
no run of any of them. The evaluate step scored those lanes (not started)
and read the supersession gate's empty answer as a read failure, so the
verdict stayed at a [read-failed] pending the sweep re-fired a recompute
for on every tick.

Such a PR now gets a terminal failure, 'fork PR into a non-default base
needs maintainer review', with no [read-failed] token, so the sweep no
longer re-fires it on age. It never reaches passed: none of the fork scan
or review coverage runs on it and nothing retargets it to the default
branch later. A fork lane whose trigger never ran for the head is listed
as skipped; a verdict a lane already posted while the base was the default
branch is still read, a red one included. The supersession gate's answer is
set aside only when it examined zero lanes and reported no other error; a
real read failure or timeout still surfaces.

Fixes #18530
@mbriones98
mbriones98 requested review from a team and KiroCrewCaptain as code owners October 27, 2026 18:18
@mbriones98
mbriones98 enabled auto-merge (squash) October 27, 2026 18:18
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of d8133413c530b572367e4fcf9735c3d8d6f26445 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The diff only touches .github/workflows/pr-readiness.yml and a test, so it adds or changes no product UI. The listed screenshots are other PRs' committed images and are not evidence for this one.

[UX-REVIEWED] d813341

Description read

[DESCRIPTION-READ] d76b6d6441b6fef787a443537cf2702af161cf8afa3ab203cb866fc5ff508e91

That sha256 names everything this verdict read: the PR description, plus the 43 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of d8133413c530b572367e4fcf9735c3d8d6f26445 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the hold is a final, named failure, so the sweep stops re-firing it on age. Fork PRs into main and same-repo PRs are not affected.

Notes

  • The description is out of date after d813341. The row "Supersession gate examined zero lanes … → nothing to evaluate" and the test for "the 'no lane examined' text matching the gate's own wording" both describe the carve-out that commit dropped. Now ok=false still adds a superseded verdicts could not be evaluated pending, and the hold outranks it. Correct the table and the Tests paragraph so they describe the current code.
  • The fork-lane comment says a retarget to the default branch makes the recompute "read it as an ordinary fork PR and the hold lifts". That only holds if Fast Gate then runs for that head. If it doesn't, the PR goes back to pending (not started). That gap already existed before this PR.

[DESIGN-REVIEWED] d813341

Description read

[DESCRIPTION-READ] ec31a69f3d29cb0866d7f4878e4380a6a033ee1d33d5da11cea35f2c9f7e1ef1

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of d8133413c530b572367e4fcf9735c3d8d6f26445 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The hold fixes the cause of #18530's loop, at the same layer as the existing stacked carve-out. The one rider and one stale description row are notes, not blockers.

Notes

  • Item 3 — rides along: blockers outrank waiting (if [ "${#blockers[@]}" -gt 0 ] … status_state="failure"), and the hold always adds to failed. So the (not started) lanes could no longer keep the PR pending, and the loop already ends without this item. Subtraction: delete the if [ "$fork_lanes_eligible" = "false" ]; then skipped+=(… only runs on PRs to …); continue; fi branch at the trun lookup.
  • Item 1 — framing: the table says "Supersession gate examined zero lanes … → nothing to evaluate". But HEAD's commit is "drop the supersession no-lanes carve-out", and the false) arm still sets read_failure=true. That row describes code that no longer ships. Subtraction: delete that table row.

What this change ships

Inventory (3 items) — 2 justified

Intent: let an outside contributor's fork PR into a release branch reach a final, named readiness verdict instead of staying pending forever. FIX.

  1. A fork PR into a non-default base now gets a final failure held for a maintainer — justified
  2. The status text names the hold instead of a blocker count — justified
  3. Fork lanes whose trigger never ran are listed as skipped instead of (not started) — rides along; the hold alone ends the loop

[FIRST-PRINCIPLES-REVIEWED] d813341

Description read

[DESCRIPTION-READ] 6eb685bcf3588cab100103f91888f604f685d56ef93033370827081b9a9c5b1c

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed d8133413c530b572367e4fcf9735c3d8d6f26445 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d813341

Verdict parsed from the review's SHA-scoped output markers for commit d8133413c530b572367e4fcf9735c3d8d6f26445.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d8133413c530b572367e4fcf9735c3d8d6f26445 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of d8133413c530b572367e4fcf9735c3d8d6f26445 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d813341

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d8133413c530b572367e4fcf9735c3d8d6f26445 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Oct 27, 2026
The fork-into-non-default-base hold is a blocker, and the blocker branch
publishes `failure` before the pending branch reads read_failure, so the
supersession step's no_lanes output never changed a verdict or the sweep's
outcome. Remove it, its env var, and its tests. A fork PR into a release
branch still ends in the terminal maintainer-hold status with no
[read-failed] token; every other PR shape is evaluated as on main.

State the workflow and test comments in present tense, and say that a PR
GitHub retargets to the default branch is re-read as an ordinary fork PR.
Guard the sweep test class on GNU date, as the sweep suite does.
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 28, 2026
@mbriones98

Copy link
Copy Markdown
Contributor Author

Replaced by #25224, which carries this change as one commit off current main.

@mbriones98 mbriones98 closed this Oct 28, 2026
auto-merge was automatically disabled October 28, 2026 11:56

Pull request was closed

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 28, 2026
@bolichen97
bolichen97 deleted the fix/fork-release-readiness-18530 branch November 6, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(readiness): a fork PR into a release branch never resolves, and the sweep re-fires a recompute forever

1 participant