Skip to content

ci: measure diff-scoped gates from the pr merge commit's first parent - #33281

Closed
leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/locale-formatting-merge-base
Closed

leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/locale-formatting-merge-base

Conversation

@leonlaiyc

@leonlaiyc leonlaiyc commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On a pull request, the diff-scoped CI gates measure only the PR's own changes, even on a re-run or when the PR is far behind main.

src/i18n/localeFormatting.test.ts > [vs-base] no file this branch touched gained a host-locale call timed out at 15 s on #31540, a PR that changes one Python comment and no website/ file (#33167, two runs linked there). The reporter traced it in the issue: the run checks out the PR's merge ref built on current main (43ac0cd051), but I18N_BASE_REF is pull_request.base.sha (89a61071c5), which a re-run keeps from the original event. At depth 1 merge-base fails, diffScope() falls back to that old base, and the test re-scans 1238 website/src files main changed in between. As the issue asks, this fixes the scope, not the timeout.

Why it matters

Any PR whose base has fallen behind can fail Frontend Tests on a re-run whatever it changes, and a re-run is exactly what a contributor reaches for after an unrelated red. Every other gate fed by .github/scripts/resolve-i18n-base.sh (the brand, comment-history, focus-cue, feature-map, changelog, decisions, harness-parity and memstore gates in fast-gate.yml, plus the i18n gates in ci.yml) charges the same foreign lines to the PR, so it can also fail on lines it never touched.

Not a goal

  • Raising the 15 s vitest budget, or making diffScope() faster.
  • Changing what a push to main or a merge-group run measures against.
  • Changing the I18N_BASE_REF expressions in ci.yml / fast-gate.yml.

What changed (motivation → approach → change)

  • Symptom: a diff-scoped gate on a stale PR run measures every base-branch commit between the event's base and current main.
  • Root cause: a pull_request run checks out GitHub's test merge commit, while the base it is handed is pull_request.base.sha. The two disagree whenever the merge commit was built on a newer base, and with a depth-1 checkout nothing reconciles them.
  • Change: .github/scripts/resolve-i18n-base.sh, the one resolver every caller already runs, now replaces the base with HEAD's first parent when the event is pull_request and HEAD's second parent is the PR head (pull_request.head.sha from $GITHUB_EVENT_PATH). That is the issue's proposal: the merge commit's first parent IS the base it was built on, so first parent..HEAD is exactly the PR. It reads the parents with git cat-file -p HEAD, because a shallow checkout grafts them away from HEAD^1. The existing fetch then pulls that one commit at depth 1. Everything else is unchanged:
    • a checkout of the PR head itself, even one whose tip is a merge of main into the branch, has a second parent that is not the PR head, so it keeps the base it was given;
    • a push, a merge-group run, or a run with no event file keeps its base;
    • when the first parent already is the given base, nothing changes and no notice is printed.
  • docs/ci/i18n-gates.md describes the rule beside the resolver's existing paragraph.

Backwards compatibility

Compatible: the resolver still prints a commit the callers resolve and fetch the same way; on a pull_request merge checkout it is now the merge's first parent, and every other run gets the value it got before.

Removes nothing: the resolver only picks a different, narrower base commit on one event shape; no gate, input or output is removed.

Tests

test/test_resolve_i18n_base.py, three new cases against real git. The fixture is an origin whose refs/pull/1/merge was built on a main two commits ahead of the event's base, checked out at depth 1 the way actions/checkout does:

  • test_a_pull_request_merge_checkout_measures_from_the_merge_commits_first_parent: resolves to the commit the merge was built on, and git diff --name-only from it to HEAD is exactly the PR's one file.
  • test_a_pr_head_checkout_keeps_the_base_it_was_given: HEAD is a merge whose second parent is not the PR head, so the given base is kept.
  • test_a_push_keeps_the_base_it_was_given: a push event with the same checkout keeps its base.

_git_env also drops the host's GITHUB_EVENT_* variables, so the existing two tests cannot pick up the CI run's own event.

The file skips on Windows. I ran it through Git Bash on this Windows host with the skip lifted: branch 5 passed; with the script reverted to origin/main, the first new test fails (assert <stale base> == <built_on>) and the other 4 pass.

Proof: not provable -- other: the changed tests skip on Windows (prove.py's host); negative control above was run by hand through Git Bash

Manual verification

N/A — unit coverage sufficient: the tests drive the real script against a real shallow merge-ref checkout, which is the CI shape. check_comment_history.py, check_brand_name.py, black/isort/flake8 on the test file and test_flake_pattern_ratchet.py (68 passed) are clean. The run on GitHub's own runners comes from this PR's CI.

Related Issues

Fixes #33167

Sibling sweep

Swept: git grep -n "BASE_REF" -- .github/workflows (every diff-scoped gate that takes a base on a pull_request run)
Sibling: .github/workflows/fast-gate.yml -- changed here -- its eight diff-scoped steps all pass their base through resolve-i18n-base.sh, so they get the same fix
Sibling: .github/workflows/ci.yml -- changed here -- its four resolver calls (frontend-ratchets' i18n and phantom-classes steps, frontend-tests, the e2e render gate) get the same fix

Pattern harvest

Rule candidate: review-prompt
Pattern: a CI diff base taken from github.event.pull_request.base.sha while the job checks out the PR merge ref, whose first parent can be a newer base

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

A pull_request run checks out GitHub's test merge commit but hands the
diff-scoped gates pull_request.base.sha, which a re-run keeps from the
original event while the merge commit is rebuilt on the current base.
At depth 1 there is no merge-base to correct that, so every base-branch
change in between was charged to the PR. On a PR far behind main the
vitest [vs-base] gate then re-scanned over a thousand files and timed
out at 15 s on a diff with no frontend change.

resolve-i18n-base.sh now measures from the merge commit's first parent
when HEAD's second parent is the PR head. A checkout of the PR head
itself, and every push, keeps the base it was given.

Fixes kirodotdev#33167

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leonlaiyc
leonlaiyc requested review from a team and KiroCrewCaptain as code owners November 8, 2026 16:32
@leonlaiyc
leonlaiyc requested a review from Zedmor November 8, 2026 16:32
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 8, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

Thanks @leonlaiyc. This fix is now urgent: since the emoji-icon gate landed in #38097, every re-run or stale-base PR fails Source Gates on lines it never touched. A fork PR that changes .github/ cannot pass our fork workflow gate, so I opened an in-repo copy with your authorship kept: #38678 (rebased onto current main; the resolver tests pass, 5 of 5, on Linux). I will close this one when #38678 merges.

AI-assisted, sent by Bolin Review bot

@bolichen97

Copy link
Copy Markdown
Contributor

Closing in favour of #38678, which merged with your commit and authorship kept. Thanks @leonlaiyc for the fix.

AI-assisted, sent by Bolin Review bot

@bolichen97 bolichen97 closed this Nov 17, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: localeFormatting [vs-base] host-locale diff test times out at 15000ms

2 participants