Skip to content

ci: measure diff-scoped gates from the pr merge commit's first parent - #38678

Merged
bolichen97 merged 1 commit into
mainfrom
crew/CommunityCrew/resolve-base-first-parent-33281
Nov 17, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
crew/CommunityCrew/resolve-base-first-parent-33281

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

In-repo copy of #33281 by @leonlaiyc, rebased onto current main with the author kept. A fork PR that touches .github/ cannot pass the fork workflow gate, and this fix is now blocking many PRs: since the emoji-icon gate landed (#38097), any re-run or stale-base PR fails Source Gates with "ratchet-scope: found no shared history" and is charged with every glyph line main gained (seen on #35435, #37627, #30034). On Linux: pytest test/test_resolve_i18n_base.py 5 passed on this branch.

AI-assisted, sent by Bolin Review bot

Problem / Motivation

Goal: On a pull request, the diff-scoped CI gates measure only the PR's own changes, even on a re-run or when the PR is far behind main.

src/i18n/localeFormatting.test.ts > [vs-base] no file this branch touched gained a host-locale call timed out at 15 s on #31540, a PR that changes one Python comment and no website/ file (#33167, two runs linked there). The reporter traced it in the issue: the run checks out the PR's merge ref built on current main (43ac0cd051), but I18N_BASE_REF is pull_request.base.sha (89a61071c5), which a re-run keeps from the original event. At depth 1 merge-base fails, diffScope() falls back to that old base, and the test re-scans 1238 website/src files main changed in between. As the issue asks, this fixes the scope, not the timeout.

Why it matters

Any PR whose base has fallen behind can fail Frontend Tests on a re-run whatever it changes, and a re-run is exactly what a contributor reaches for after an unrelated red. Every other gate fed by .github/scripts/resolve-i18n-base.sh (the brand, comment-history, focus-cue, feature-map, changelog, decisions, harness-parity and memstore gates in fast-gate.yml, plus the i18n gates in ci.yml) charges the same foreign lines to the PR, so it can also fail on lines it never touched.

Not a goal

  • Raising the 15 s vitest budget, or making diffScope() faster.
  • Changing what a push to main or a merge-group run measures against.
  • Changing the I18N_BASE_REF expressions in ci.yml / fast-gate.yml.

What changed (motivation → approach → change)

  • Symptom: a diff-scoped gate on a stale PR run measures every base-branch commit between the event's base and current main.
  • Root cause: a pull_request run checks out GitHub's test merge commit, while the base it is handed is pull_request.base.sha. The two disagree whenever the merge commit was built on a newer base, and with a depth-1 checkout nothing reconciles them.
  • Change: .github/scripts/resolve-i18n-base.sh, the one resolver every caller already runs, now replaces the base with HEAD's first parent when the event is pull_request and HEAD's second parent is the PR head (pull_request.head.sha from $GITHUB_EVENT_PATH). That is the issue's proposal: the merge commit's first parent IS the base it was built on, so first parent..HEAD is exactly the PR. It reads the parents with git cat-file -p HEAD, because a shallow checkout grafts them away from HEAD^1. The existing fetch then pulls that one commit at depth 1. Everything else is unchanged:
    • a checkout of the PR head itself, even one whose tip is a merge of main into the branch, has a second parent that is not the PR head, so it keeps the base it was given;
    • a push, a merge-group run, or a run with no event file keeps its base;
    • when the first parent already is the given base, nothing changes and no notice is printed.
  • docs/ci/i18n-gates.md describes the rule beside the resolver's existing paragraph.

Backwards compatibility

Compatible: the resolver still prints a commit the callers resolve and fetch the same way; on a pull_request merge checkout it is now the merge's first parent, and every other run gets the value it got before.

Removes nothing: the resolver only picks a different, narrower base commit on one event shape; no gate, input or output is removed.

Tests

test/test_resolve_i18n_base.py, three new cases against real git. The fixture is an origin whose refs/pull/1/merge was built on a main two commits ahead of the event's base, checked out at depth 1 the way actions/checkout does:

  • test_a_pull_request_merge_checkout_measures_from_the_merge_commits_first_parent: resolves to the commit the merge was built on, and git diff --name-only from it to HEAD is exactly the PR's one file.
  • test_a_pr_head_checkout_keeps_the_base_it_was_given: HEAD is a merge whose second parent is not the PR head, so the given base is kept.
  • test_a_push_keeps_the_base_it_was_given: a push event with the same checkout keeps its base.

_git_env also drops the host's GITHUB_EVENT_* variables, so the existing two tests cannot pick up the CI run's own event.

The file skips on Windows. I ran it through Git Bash on this Windows host with the skip lifted: branch 5 passed; with the script reverted to origin/main, the first new test fails (assert <stale base> == <built_on>) and the other 4 pass.

Proof: not provable -- other: the changed tests skip on Windows (prove.py's host); negative control above was run by hand through Git Bash

Manual verification

N/A — unit coverage sufficient: the tests drive the real script against a real shallow merge-ref checkout, which is the CI shape. check_comment_history.py, check_brand_name.py, black/isort/flake8 on the test file and test_flake_pattern_ratchet.py (68 passed) are clean. The run on GitHub's own runners comes from this PR's CI.

Related Issues

Fixes #33167

Sibling sweep

Swept: git grep -n "BASE_REF" -- .github/workflows (every diff-scoped gate that takes a base on a pull_request run)
Sibling: .github/workflows/fast-gate.yml -- changed here -- its eight diff-scoped steps all pass their base through resolve-i18n-base.sh, so they get the same fix
Sibling: .github/workflows/ci.yml -- changed here -- its four resolver calls (frontend-ratchets' i18n and phantom-classes steps, frontend-tests, the e2e render gate) get the same fix

Pattern harvest

Rule candidate: review-prompt
Pattern: a CI diff base taken from github.event.pull_request.base.sha while the job checks out the PR merge ref, whose first parent can be a newer base

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

A pull_request run checks out GitHub's test merge commit but hands the
diff-scoped gates pull_request.base.sha, which a re-run keeps from the
original event while the merge commit is rebuilt on the current base.
At depth 1 there is no merge-base to correct that, so every base-branch
change in between was charged to the PR. On a PR far behind main the
vitest [vs-base] gate then re-scanned over a thousand files and timed
out at 15 s on a diff with no frontend change.

resolve-i18n-base.sh now measures from the merge commit's first parent
when HEAD's second parent is the PR head. A checkout of the PR head
itself, and every push, keeps the base it was given.

Fixes #33167

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 17, 2026 17:52
@bolichen97
bolichen97 requested a review from Zedmor November 17, 2026 17:52
@bolichen97
bolichen97 enabled auto-merge (squash) November 17, 2026 17:52
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix lands in the one resolver every diff-scoped gate already calls, and it only applies when HEAD is the PR's merge commit.

Notes

  • Under ## Sibling sweep, fast-gate.yml and ci.yml are each marked "changed here", but the diff never touches either file. They get the fix only through .github/scripts/resolve-i18n-base.sh, so reword those lines to say "inherits via the resolver".
  • ci.yml:3717 (the metric schema gate) also gets the new base, and the sweep does not mention it.

[DESIGN-REVIEWED] 0e47a27

Description read

[DESCRIPTION-READ] be1686eb2e431ff52e8e52c126e9986719f894f4809a135e3068e39d0e992e65

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 0e47a27

Verdict parsed from the review's SHA-scoped output markers for commit 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0e47a27

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0e47a275bf0f9aedeeb0a67228a208d3e5468cbb false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 17, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates an 8% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier 2: the title's category is not one of fix, test, chore, refactor.

The resolver now replaces the base with the merge commit's first parent, but only on pull_request runs where HEAD's second parent equals pull_request.head.sha from GitHub's event file. All 13 callers pass bare SHAs and use only the printed value. In ratchet_scope.py, resolve_base, added_lines_at and changed_paths_at all scope more exactly with the first-parent base, on both shallow and full clones. The trust condition can't be steered by a PR author, and any failure keeps the original base. Push, merge-group and PR-head checkouts are unchanged. The new real-git tests include one that fails without the script change. No blocking issues.

Tier: 2, held for a person: the title's category is not one of fix, test, chore, refactor.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 0e47a275bf · 115 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review found no blocking issues on this head. On a pull_request run the resolver now measures from the merge commit's first parent. It does that only when HEAD has exactly two parents and the second one is the event's head.sha. Push, merge_group and PR-head checkouts keep their old base. Reviewers checked all 14 workflow call sites: each passes a bare SHA, so the output shape is the same. They also ran test/test_resolve_i18n_base.py at this head. All 5 tests passed, and the new merge-checkout test fails when the resolver is reverted to main's version.

Done well: the fix is in the one shared resolver, not in 14 call sites. The test drives the real script against a real depth-1 checkout of refs/pull/1/merge. It also strips GITHUB_EVENT_* from the test env, so the CI host's own event cannot leak into the older tests.

Optional:

  • Nothing pins the ${#PARENTS[@]} -eq 2 guard. If you delete it, all 5 tests still pass, but a single-parent HEAD on a pull_request run then exits 1 under set -u. One test that checks out refs/pull/1/head under a pull_request event would pin it.
  • At depth 1 the no-op check on line 78 is always true, because the base commit object is not fetched. The notice then reads "X instead of X", which contradicts the body. Comparing "${PARENTS[0]}" != "$REF" directly fixes that.
  • docs/ci/harness-parity-gate.md:49 still says the merge ref was computed against base.sha. One sentence that links to the new i18n-gates.md paragraph would keep the two docs in step. Please close #33281 once this merges.

@bolichen97
bolichen97 merged commit 7fefb7b into main Nov 17, 2026
101 of 102 checks passed
@bolichen97
bolichen97 deleted the crew/CommunityCrew/resolve-base-first-parent-33281 branch November 17, 2026 19:46
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: localeFormatting [vs-base] host-locale diff test times out at 15000ms

3 participants