Repository navigation
ci: measure diff-scoped gates from the pr merge commit's first parent - #38678
Conversation
A pull_request run checks out GitHub's test merge commit but hands the diff-scoped gates pull_request.base.sha, which a re-run keeps from the original event while the merge commit is rebuilt on the current base. At depth 1 there is no merge-base to correct that, so every base-branch change in between was charged to the PR. On a PR far behind main the vitest [vs-base] gate then re-scanned over a thousand files and timed out at 15 s on a diff with no frontend change. resolve-i18n-base.sh now measures from the merge commit's first parent when HEAD's second parent is the PR head. A checkout of the PR head itself, and every push, keeps the base it was given. Fixes #33167 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the fix lands in the one resolver every diff-scoped gate already calls, and it only applies when HEAD is the PR's merge commit. Notes
[DESIGN-REVIEWED] 0e47a27 Description read[DESCRIPTION-READ] be1686eb2e431ff52e8e52c126e9986719f894f4809a135e3068e39d0e992e65 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: A person should decideWhy: the decider estimates an 8% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier 2: the title's category is not one of fix, test, chore, refactor. The resolver now replaces the base with the merge commit's first parent, but only on pull_request runs where HEAD's second parent equals pull_request.head.sha from GitHub's event file. All 13 callers pass bare SHAs and use only the printed value. In ratchet_scope.py, resolve_base, added_lines_at and changed_paths_at all scope more exactly with the first-parent base, on both shallow and full clones. The trust condition can't be steered by a PR author, and any failure keeps the original base. Push, merge-group and PR-head checkouts are unchanged. The new real-git tests include one that fails without the script change. No blocking issues. Tier: 2, held for a person: the title's category is not one of fix, test, chore, refactor. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
This review found no blocking issues on this head. On a pull_request run the resolver now measures from the merge commit's first parent. It does that only when HEAD has exactly two parents and the second one is the event's head.sha. Push, merge_group and PR-head checkouts keep their old base. Reviewers checked all 14 workflow call sites: each passes a bare SHA, so the output shape is the same. They also ran test/test_resolve_i18n_base.py at this head. All 5 tests passed, and the new merge-checkout test fails when the resolver is reverted to main's version.
Done well: the fix is in the one shared resolver, not in 14 call sites. The test drives the real script against a real depth-1 checkout of refs/pull/1/merge. It also strips GITHUB_EVENT_* from the test env, so the CI host's own event cannot leak into the older tests.
Optional:
- Nothing pins the
${#PARENTS[@]} -eq 2guard. If you delete it, all 5 tests still pass, but a single-parent HEAD on a pull_request run then exits 1 underset -u. One test that checks out refs/pull/1/head under a pull_request event would pin it. - At depth 1 the no-op check on line 78 is always true, because the base commit object is not fetched. The notice then reads "X instead of X", which contradicts the body. Comparing
"${PARENTS[0]}" != "$REF"directly fixes that. - docs/ci/harness-parity-gate.md:49 still says the merge ref was computed against base.sha. One sentence that links to the new i18n-gates.md paragraph would keep the two docs in step. Please close #33281 once this merges.
Problem / Motivation
Goal: On a pull request, the diff-scoped CI gates measure only the PR's own changes, even on a re-run or when the PR is far behind main.
src/i18n/localeFormatting.test.ts > [vs-base] no file this branch touched gained a host-locale calltimed out at 15 s on #31540, a PR that changes one Python comment and nowebsite/file (#33167, two runs linked there). The reporter traced it in the issue: the run checks out the PR's merge ref built on current main (43ac0cd051), butI18N_BASE_REFispull_request.base.sha(89a61071c5), which a re-run keeps from the original event. At depth 1merge-basefails,diffScope()falls back to that old base, and the test re-scans 1238website/srcfiles main changed in between. As the issue asks, this fixes the scope, not the timeout.Why it matters
Any PR whose base has fallen behind can fail Frontend Tests on a re-run whatever it changes, and a re-run is exactly what a contributor reaches for after an unrelated red. Every other gate fed by
.github/scripts/resolve-i18n-base.sh(the brand, comment-history, focus-cue, feature-map, changelog, decisions, harness-parity and memstore gates infast-gate.yml, plus the i18n gates inci.yml) charges the same foreign lines to the PR, so it can also fail on lines it never touched.Not a goal
diffScope()faster.I18N_BASE_REFexpressions inci.yml/fast-gate.yml.What changed (motivation → approach → change)
pull_requestrun checks out GitHub's test merge commit, while the base it is handed ispull_request.base.sha. The two disagree whenever the merge commit was built on a newer base, and with a depth-1 checkout nothing reconciles them..github/scripts/resolve-i18n-base.sh, the one resolver every caller already runs, now replaces the base with HEAD's first parent when the event ispull_requestand HEAD's second parent is the PR head (pull_request.head.shafrom$GITHUB_EVENT_PATH). That is the issue's proposal: the merge commit's first parent IS the base it was built on, sofirst parent..HEADis exactly the PR. It reads the parents withgit cat-file -p HEAD, because a shallow checkout grafts them away fromHEAD^1. The existing fetch then pulls that one commit at depth 1. Everything else is unchanged:docs/ci/i18n-gates.mddescribes the rule beside the resolver's existing paragraph.Backwards compatibility
Compatible: the resolver still prints a commit the callers resolve and fetch the same way; on a pull_request merge checkout it is now the merge's first parent, and every other run gets the value it got before.
Removes nothing: the resolver only picks a different, narrower base commit on one event shape; no gate, input or output is removed.
Tests
test/test_resolve_i18n_base.py, three new cases against real git. The fixture is an origin whoserefs/pull/1/mergewas built on a main two commits ahead of the event's base, checked out at depth 1 the wayactions/checkoutdoes:test_a_pull_request_merge_checkout_measures_from_the_merge_commits_first_parent: resolves to the commit the merge was built on, andgit diff --name-onlyfrom it to HEAD is exactly the PR's one file.test_a_pr_head_checkout_keeps_the_base_it_was_given: HEAD is a merge whose second parent is not the PR head, so the given base is kept.test_a_push_keeps_the_base_it_was_given: apushevent with the same checkout keeps its base._git_envalso drops the host'sGITHUB_EVENT_*variables, so the existing two tests cannot pick up the CI run's own event.The file skips on Windows. I ran it through Git Bash on this Windows host with the skip lifted: branch 5 passed; with the script reverted to
origin/main, the first new test fails (assert <stale base> == <built_on>) and the other 4 pass.Proof: not provable -- other: the changed tests skip on Windows (prove.py's host); negative control above was run by hand through Git Bash
Manual verification
N/A — unit coverage sufficient: the tests drive the real script against a real shallow merge-ref checkout, which is the CI shape.
check_comment_history.py,check_brand_name.py, black/isort/flake8 on the test file andtest_flake_pattern_ratchet.py(68 passed) are clean. The run on GitHub's own runners comes from this PR's CI.Related Issues
Fixes #33167
Sibling sweep
Swept: git grep -n "BASE_REF" -- .github/workflows (every diff-scoped gate that takes a base on a pull_request run)
Sibling: .github/workflows/fast-gate.yml -- changed here -- its eight diff-scoped steps all pass their base through resolve-i18n-base.sh, so they get the same fix
Sibling: .github/workflows/ci.yml -- changed here -- its four resolver calls (frontend-ratchets' i18n and phantom-classes steps, frontend-tests, the e2e render gate) get the same fix
Pattern harvest
Rule candidate: review-prompt
Pattern: a CI diff base taken from github.event.pull_request.base.sha while the job checks out the PR merge ref, whose first parent can be a newer base
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
🤖 Generated with Claude Code