Skip to content

ci: backport blobless checkouts for review lanes to 0.12 (#27513, #29347) - #29387

Merged
buluoray merged 2 commits into
release/0.12.0from
crew/slack/backport-0.12-checkout-blobless
Nov 4, 2026
Merged

buluoray merged 2 commits into
release/0.12.0from
crew/slack/backport-0.12-checkout-blobless

Conversation

@buluoray

@buluoray buluoray commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Pull requests into release/0.12.0 finish their review-lane checkouts inside the job budget instead of being cancelled in actions/checkout.

On 2026-11-03, PR #29291 (base release/0.12.0) had Inclusive Language, Automated Rule Check, PR Hygiene and SAST (Semgrep) cancelled inside their fetch-depth: 0 checkout (10 to 30 minutes each), and Security Scope Review cancelled the same way. On main these lanes were fixed by #27513 (Code Review jobs) and #29347 (Security Scope lanes). release/0.12.0 has neither.

Why it matters

Every PR into the 0.12 release branch hits the same cancellations, so its required PR Readiness cannot go green without repeated reruns. That blocks 0.12 backports and patch releases.

Not a goal

What changed (motivation → approach → change)

Two clean cherry-picks from main, each with its (cherry picked from ...) trailer:

Commit From Effect on 0.12
fix(ci): fetch only what Code Review's jobs read, not every branch (#27513) 300ee84bdc Code Review jobs check out fetch-depth: 2, then .github/scripts/code-review-history.sh fetches the commit history without blobs and the merge-base tree.
ci: blobless full-history checkout for the security scope lanes (#29347) a073c19453 filter: blob:none on every full-history checkout in security-scope-review.yml, fork-security-scope-review.yml and fork-internal-content-scan.yml; 15-minute job budgets raised to 20.

One conflict, in test/test_workflow_bounded_checkout.py: 0.12 lacks #29301, so the new pin test lists only #29347's eight jobs, not internal-content-scan.yml / release.yml. #29291 keeps those two in the bounded-checkout allowlist on 0.12 until it lands.

The fork workflows run from the default branch on workflow_run, so their 0.12 copies change nothing at runtime. They are carried so the branch matches main and the pin test holds.

Backwards compatibility

Compatible: each job reads the same commits and files; only when objects are downloaded changes.

Removes nothing: the diff changes checkout depth and filters and raises timeouts; no path, key, check or record is taken away.

Tests

CI runs no backend tests on PRs into release/*, so the related set ran locally: every test file that reads code-review.yml, security-scope-review.yml, fork-internal-content-scan.yml, code-review-history.sh or the shared AI-review workflow helpers (58 files), in 3 sequential single-process batches.

Batch Files Result
1 20 1632 passed
2 20 1740 passed
3 18 1356 passed

No failure is caused by the picks. One case, test_code_review_history_fetch.py::test_a_checker_runs_the_base_tips_copy_and_a_failed_read_fails_the_step[...proof_line_check.py], hangs on a host whose first python3 on PATH is a version-manager shim: the test sets HOME to a temp dir and keeps PATH, so the shim finds no tools and spins. It hangs the same way on main. With the real interpreter first on PATH, all 6 cases pass, and batch 2 above was run that way.

Proof: not provable -- docs or config only

Manual verification

This PR's own Code Review and Security Scope lanes run on the 0.12 workflows it changes, so their checkout times are the measurement. They are posted in a comment once CI finishes.

Related Issues

Part of #29264. Backports #27513 and #29347 to release/0.12.0.

Sibling sweep

Swept: every actions/checkout with fetch-depth: 0 in .github/workflows/ on release/0.12.0.
Sibling: .github/workflows/code-review.yml -- changed here
Sibling: .github/workflows/security-scope-review.yml -- changed here
Sibling: .github/workflows/fork-security-scope-review.yml -- changed here
Sibling: .github/workflows/fork-internal-content-scan.yml -- changed here
Sibling: .github/workflows/internal-content-scan.yml -- tracked in #29291
Sibling: .github/workflows/release.yml -- tracked in #29291

Pattern harvest

Not generalizable: a branch cut before a CI fix needs that fix backported; the ratchet in test_workflow_bounded_checkout.py already covers the pattern.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

jeeshofone and others added 2 commits November 3, 2026 23:41
…27513)

* fix(ci): fetch only what Code Review's jobs read, not every branch

Every Code Review job checked out with fetch-depth: 0, which downloads
every branch and tag with every blob ever committed (2.87 GiB, 1109
branches). On a slow runner that fetch alone outlasted the job's timeout
and the job was cancelled inside actions/checkout.

The four jobs now check out the merge commit at depth 2 and run one
shared script as their diff-base step. It fetches the commit history of
the base tip and the head without trees or blobs, resolves the
merge-base, fetches the merge-base's files by running the diffs once,
and fails the step if the history is still shallow or any object of the
four commits is still missing, so no later step reads the network. A
retry after an attempt that completed the history asks for no
--unshallow, which git refuses on a complete history.

The script runs from the pull request's own commit, so the jobs hand it
no token. The repository is public, so every fetch is anonymous: the
script turns off any credential helper and the http.extraheader that
actions/checkout writes for github.com. A private repository refuses the
fetch, and the step fails with an error saying the repository must be
public or readable without credentials. PR Hygiene's checker steps now
fail when the base tip's copy of their script cannot be read, and run
the checkout copy only when the base tip has no such file.

Closes #23397

* fix(ci): fetch the history with blob:none, which git 2.56 can deepen

On git 2.56 (the CodeBuild runners), deepening a shallow clone with
--filter=tree:0 aborts in index-pack. blob:none keeps the commit graph
and skips every blob, and passes the suite on git 2.47 and 2.56.

(cherry picked from commit 300ee84)
The Security Scope Review jobs and the fork content scan check out with
fetch-depth: 0, and on the runner fleet that clone alone now outlasts
their 15- and 30-minute budgets: on 2026-11-03 the candidates job and
the publish job were cancelled inside actions/checkout after 30 and 15
minutes.

Apply the pattern from #29301 to every full-history checkout in
security-scope-review.yml, fork-security-scope-review.yml and
fork-internal-content-scan.yml: add filter: blob:none (all commits
and trees arrive, the checked-out tree is complete, older blobs are
fetched on demand) and raise the 15-minute job budgets to 20. The
bounded-checkout test now pins the filter and the budget on each job.

(cherry picked from commit a073c19)

# Conflicts:
#	test/test_workflow_bounded_checkout.py
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 3, 2026 23:48
@buluoray
buluoray requested review from dodgeblaster and removed request for a team November 3, 2026 23:48
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision a99ac9c12744e43c1a5675c257aeb6d0d6855768 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of a99ac9c12744e43c1a5675c257aeb6d0d6855768 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This backports two fixes main already merged. They address cancellations the description reports on PR #29291. The only part with no runtime effect is the fork-workflow copies.

Notes

  • Item 6 — inherited (symmetry): the description says the fork workflows "run from the default branch on workflow_run, so their 0.12 copies change nothing at runtime. They are carried so the branch matches main and the pin test holds." Matching main is the only reason given. Subtraction: drop the edits to fork-security-scope-review.yml and fork-internal-content-scan.yml, and their four rows in test_full_history_jobs_moved_off_a_tight_budget_stay_blobless.

What this change ships

Inventory (7 items) — 5 justified

Intent: let PRs into release/0.12.0 finish their review-lane checkouts within the job time limit, so PR Readiness can go green. FIX.

  1. Code Review's four jobs now check out two commits and fetch the history without blobs via code-review-history.sh. — justified
  2. PR Hygiene's merge-base step now runs first in the job, not partway through. — justified
  3. The Proof-line and Sibling-sweep steps now fail when the base read fails, instead of quietly running the PR's own copy of the checker. — undeclared
  4. The Security Scope lanes' full-history checkouts are now blobless. — justified
  5. Those lanes' time limits go from 15 to 20 minutes. — justified
  6. The fork workflows get the same blobless and time-limit edits, with no effect at runtime. — rides along
  7. A pin test keeps the eight full-history jobs blobless with at least 20 minutes. — justified

[FIRST-PRINCIPLES-REVIEWED] a99ac9c

Description read

[DESCRIPTION-READ] 31de81127ef2cf2d4fa14b0a1d66abbe228c7d41b31eb5d23235bf7519d87328

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of a99ac9c12744e43c1a5675c257aeb6d0d6855768 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This backports two fixes main already carries, and each job still reads the same commits and files. Nothing in it blocks.

Notes

  • The anonymous history fetch in code-review-history.sh relies on the repo being public. That holds here, and when it doesn't, the step fails with a clear error instead of passing silently.

[DESIGN-REVIEWED] a99ac9c

Description read

[DESCRIPTION-READ] 26097ff1d229482576561328a234e13c777e5a08dd7255b645dc665a1ad21c6b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed a99ac9c12744e43c1a5675c257aeb6d0d6855768 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a99ac9c

Verdict parsed from the review's SHA-scoped output markers for commit a99ac9c12744e43c1a5675c257aeb6d0d6855768.

False positive or not applicable? A repository writer can comment:
/ai-review override fable a99ac9c12744e43c1a5675c257aeb6d0d6855768 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does a99ac9c12744e43c1a5675c257aeb6d0d6855768 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 13 classified, 0 newly refused, 13 still allowed.
  • ubuntu-latest (platform posix): 13 classified, 0 newly refused, 13 still allowed.
  • windows-latest (platform windows): 9 classified, 0 newly refused, 9 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: This diff changes CI workflows only. Nothing in hooks.py, the sensitive-command tier, the exfil shapes or the deny-rule catalog changes, so the shell candidates should not flip, and no regression was confirmed. The two guards that did get stricter live in GitHub Actions steps the classifier cannot judge; they are named under UNADJUDICATED, and no gap could be shown with a call path.

TIGHTENING: Two workflow-side guards got stricter. (1) The history fetch is now anonymous: export GIT_TERMINAL_PROMPT=0 GIT_CONFIG_COUNT=2 GIT_CONFIG_KEY_0=credential.helper GIT_CONFIG_VALUE_0='' GIT_CONFIG_KEY_1=http.https://github.com/.extraheader GIT_CONFIG_VALUE_1='' in .github/scripts/code-review-history.sh. (2) The PR Hygiene checker steps no longer fall back to the checkout copy when reading the base tip fails: if ! git show "$BASE_TIP:..." 2>/dev/null became listed="$(git ls-tree --name-only "$BASE_TIP" -- ...)"; if [ -n "$listed" ]; then git show ... > "$script". The one threat they refuse: PR-controlled code (the history script runs from the PR's own commit) getting a runner credential, and a branch swapping in its own weakened checker when the base-tip read fails.

ADJUDICATED: 18 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: I wrote posix, windows and any rows. The posix rows ($VAR quoting, /dev/null, .venv/bin/python) probe the bash spellings the new script and steps assume. The windows rows (%BASE_TIP%, > NUL, %TEMP%\\..., .venv\\Scripts\\python.exe) probe the same git operations in cmd spelling, so a tier change keyed on $ expansion or / separators would show. The any rows (git fetch --filter=blob:none, git merge-base origin/main HEAD, git rev-list --count, git rev-parse --is-shallow-repository) hold no path or variable. The diff's own platform assumption is Linux-only bash on the runner: test_code_review_history_fetch.py skips on win32. No shell command line can stand in for a Windows runner that runs that job, so that platform is left open.

UNADJUDICATED:

  • Guard: the anonymous-fetch environment in .github/scripts/code-review-history.sh. Call path: code-review.yml, job autosde-rules / inclusive-language / sast / pr-hygiene, actions/checkout at fetch-depth: 2, then the "Resolve diff base (merge-base)" step, then bash .github/scripts/code-review-history.sh, then retry fetch_history and the on-demand blob fetches in git diff --binary. Legitimate inputs it would newly refuse:

    • a private fork or mirror of the repository running this workflow;
    • a runner whose git access to github.com works only through a configured credential helper, for example a CodeBuild-hosted runner with an authenticated egress proxy.

    The script names both cases in its error, and they come from the stated design. I could not show either with a call path on this repo, which is public.

  • Guard: the base-tip checker read in the "Require a Proof line inside ## Tests" and "Require a Sibling sweep section" steps. Call path: pr-hygiene job, diff-base step, git ls-tree --name-only "$BASE_TIP" -- <checker>, then git show "$BASE_TIP:<checker>" under bash -e. A legitimate input it would newly refuse: a run where the diff-base step succeeded but a later git show hits a transient object-read failure. The step now fails instead of running the checkout copy, which is the fail-closed behaviour the PR intends.

  • Guard: the filter: blob:none plus timeout-minutes: 20 changes in security-scope-review.yml and fork-security-scope-review.yml. These are not tightenings. Candidate inputs I would have submitted: a scope-review job that runs git show BASE:<path> or git archive on a historical blob after checkout. In a blobless clone that read now needs a lazy network fetch, and it fails on a leg that withholds credentials if the repository is not readable anonymously.

[SCOPE-REVIEWED] a99ac9c

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — 🔴 changes requested (blocking)

GPT 6.1 found at least one blocking issue that must be resolved before merging a99ac9c12744e43c1a5675c257aeb6d0d6855768. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- .github/workflows/security-scope-review.yml:1414 -- Failed lazy fetch executes PR-controlled code in the privileged publisher

filter: blob:none
Trigger: A same-repo PR changes scripts/scope_candidates.py, and fetching its missing base blob fails after the blobless merge checkout. Publish’s staging loop suppresses the read failure and substitutes the already-local head blob.
Outcome: “Post the scope verdict” executes that PR-controlled script with the comment-write credential; the script can select its own verdict. Validate’s staging loop also substitutes checkout code on the same failure.
Anchor: residual/security
Fix: Check the base tree to distinguish genuine absence from a failed read in validate and publish; when the base file exists, fail closed rather than substituting PR code.

[BLOCK-MERGE] a99ac9c
[GPT-REVIEWED] a99ac9c

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 (fenced): FLAG. The finding is real, but reaching it takes a combination of conditions the attacker can't arrange.

  • Conditions:

    • The publish checkout is blobless with persist-credentials: false (.github/workflows/security-scope-review.yml:1414-1415).
    • The base blob is read with git show "$BASE_SHA:scripts/$f" … 2>/dev/null || true (:1551). On an empty result it falls back to the HEAD blob (:1560) and runs it (:1592). Nothing tells a failed lazy fetch apart from a file that is truly missing.
    • So the PR must change scripts/scope_candidates.py and the one on-demand fetch from a public origin must fail at that exact moment. That is a transient GitHub failure the PR author can't cause or time.
  • Visibility and alternatives: the fallback shows on the run as a ::warning::…missing or empty on the base commit annotation (:1559) and a "folder bootstrap" step-summary section (:1571). The trigger is pull_request (:36), so a same-repo author with push access can already change this workflow file directly. This path doesn't give them anything they don't already have.

  • Fix cost: about a two-line git cat-file -e "$BASE_SHA:scripts/$f" check, so it's cheap. The residual is rare and shows up on the run, so a writer could reasonably accept it while the fix ships.

    [ADJUDICATION] a99ac9c total=0 uphold=0 downgrade=0
    [GPT-ADJUDICATED] a99ac9c

    [ADJUDICATION-FENCED] a99ac9c fenced=1 flagged=1
    FLAG F1 .github/workflows/security-scope-review.yml:1414 -- Reaching this needs a transient lazy-fetch failure from a public origin at the exact moment a PR changes scope_candidates.py, which the author can't cause or time; the HEAD-blob fallback raises a ::warning:: and a bootstrap summary (:1559, :1571), and under the pull_request trigger (:36) a same-repo author can already edit this workflow directly, so the residual is acceptable while the two-line cat-file fix lands.
    [GPT-ADJUDICATED-FENCED] a99ac9c

🏷️ Fenced finding(s) machine-flagged as likely edge case

The security fence keeps these findings blocking regardless of adjudication; the only clearance path is a human override. The annotate-only pass judged the condition combination each one requires extreme, and pre-drafted the override rationale below. A repository writer must independently verify a rationale before posting it — it is machine-authored, and a wrong override on a security-class finding ships exactly the class the fence exists to stop.

  • F1 .github/workflows/security-scope-review.yml:1414 — Reaching this needs a transient lazy-fetch failure from a public origin at the exact moment a PR changes scope_candidates.py, which the author can't cause or time; the HEAD-blob fallback raises a ::warning:: and a bootstrap summary (:1559, :1571), and under the pull_request trigger (:36) a same-repo author can already edit this workflow directly, so the residual is acceptable while the two-line cat-file fix lands.

    /ai-review override gpt a99ac9c12744e43c1a5675c257aeb6d0d6855768 <class>: Reaching this needs a transient lazy-fetch failure from a public origin at the exact moment a PR changes scope_candidates.py, which the author can't cause or time; the HEAD-blob fallback raises a ::warning:: and a bootstrap summary (:1559, :1571), and under the pull_request trigger (:36) a same-repo author can already edit this workflow directly, so the residual is acceptable while the two-line cat-file fix lands.
    

Replace <class> with the class that records what you did. These findings are real, so for an accepted risk use owner-decision (link the decision comment in the reason) or deferred #<N> (a tracked follow-up issue); false-positive only if you verified the finding is wrong. Not not-applicable: that class files the override as a rule mismatch in the audit.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt a99ac9c12744e43c1a5675c257aeb6d0d6855768 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 4, 2026
@buluoray
buluoray merged commit 7c13ba0 into release/0.12.0 Nov 4, 2026
29 of 34 checks passed
@buluoray
buluoray deleted the crew/slack/backport-0.12-checkout-blobless branch November 4, 2026 07:34
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants