Skip to content

ci: retry a failed dependency review and report a graph-API outage as could-not-run - #25611

Merged
iamwhatever merged 1 commit into
mainfrom
fix/license-gate-outage-retry-25082
Oct 29, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/license-gate-outage-retry-25082

Conversation

@CrysisDeu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a GitHub dependency-graph outage no longer reads as a license failure on every open PR.

Between about 23:00 and 00:10 UTC on 10-27/28, and again around 18:05 UTC on 10-28, Dependency License Gate failed PRs from forks and the same repo alike with Dependency review is not supported on this repository. Please ensure that Dependency graph is enabled. PRs opened after the window passed, so the settings never changed. The gate runs actions/dependency-review-action once and fails on any error.

Why it matters

Every PR in an outage window needs a rerun or a new push, and the red check reads like a policy finding.

Not a goal

  • Changing the allowlist, the license-override label path, or the job's permissions.
  • Passing the gate when it cannot run: every failure still fails closed.

Intended? / Repro

Intended? No. The workflow's header (.github/workflows/dependency-review.yml:1-17 on main) describes a license gate; an API outage is not a license decision.

Repro: run 40602827883 job 122545495755 (#9749, 00:04) and run 40606266923 job 122546412890 (#25074) fail with the message above; #25057, #25070, #25072, #25073 went red in the same window and later PRs passed.

What changed (motivation → approach → change)

Symptom: outage reported as a policy failure. Cause: one attempt, no classification. Change: retry once, then tell an outage from a verdict, as the issue suggests.

  • .github/workflows/dependency-review.yml:
    • The review step gets id: review and continue-on-error: true.
    • On its failure: a 60s pause, then the same action and config again (review_retry, also continue-on-error).
    • Report the review verdict runs whenever the first review failed. A passing retry passes the gate with a warning. Two failures: it probes GET repos/{repo}/dependency-graph/compare/{base}...{head} up to 3 times. An API that answers → fails as the action's license verdict. An API that does not → fails as "could not run", naming the API's answer and saying it is not a license finding.
  • docs/ci/ci-and-reviews.md: the license-gate paragraph states the rule.

Security

The job's token and permissions are unchanged (contents: read, pull-requests: write). The probe is a GET with that token. No path can pass the gate without a successful review.

Backwards compatibility

Compatible: a passing first review is unchanged. A real license failure still fails, about 80s later (the pause and the retry).

Removes nothing.

Tests

test/test_dependency_license_gate_outage.py (new):

  • Structure: the first review is continue-on-error; the retry uses the same action and config, after the pause; the verdict step runs on the first failure and reads the retry's outcome.
  • The verdict step's own script under bash with a stubbed gh: a passing retry passes with no probe; two failures with a working API fail as a license verdict after one probe; two failures with the API down fail as "could not run", carrying the API's HTTP 503, after three probes.

test/test_pr_quality_gates.py::test_the_job_is_read_only_and_the_write_scoped_gate_stays_separate still holds (the job still uses the action and keeps pull-requests: write).

Proof: not provable -- other: this host runs no local tests; CI runs the new tests

Manual verification

The workflow parses as YAML and the verdict script passes bash -n. Static gates: black, isort, flake8, scripts/docs_lint.py, scripts/check_comment_history.py.

Related Issues

Fixes #25082

Sibling sweep

Swept: git grep -n "dependency-review-action" -- .github test
Sibling: .github/workflows/dependency-review.yml -- changed here (the only caller)
Sibling: test/test_pr_quality_gates.py -- not affected (its assertions still hold)

Pattern harvest

Not generalizable: one third-party action call.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@CrysisDeu
CrysisDeu requested review from a team and KiroCrewCaptain as code owners October 29, 2026 03:18
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e8c67756601abbd5f74bfc7ed0d74b68464b915f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of e8c67756601abbd5f74bfc7ed0d74b68464b915f — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The cause is an outage on GitHub's side that this repository can't fix. Every item has a named harm, backed by the linked runs and #25082.

Notes

  • Item 3 — oversized: after two failed reviews, the probe makes 3 more attempts with 20s pauses (for attempt in 1 2 3). The retry right before it has already tested for a transient fault. Subtraction: cut the probe loop to one gh api call and drop the sleep 20.

What this change ships

Inventory (6 items) — 5 justified

Intent: stop a GitHub dependency-graph outage from showing up as a license failure on every open PR. This is a FIX.

  1. A failed license review is retried once after a 60s pause — justified
  2. A retry that passes turns the gate green with a "transient" warning — justified
  3. After two failures, the API is probed up to 3 times and the message says either "license verdict" or "could not run" — oversized (three probes after a retry)
  4. A real license failure now goes red about 80s later — justified
  5. The token-scope test map now places dependency-graph API calls under the contents scope — justified
  6. The ci-and-reviews doc paragraph states the retry-and-classify rule — justified

[FIRST-PRINCIPLES-REVIEWED] e8c6775

Description read

[DESCRIPTION-READ] 2fb35ee1389e1cbc7217ab802d1601d5b25aafd3f56bdc3cbbee2465956e3564

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e8c67756601abbd5f74bfc7ed0d74b68464b915f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: every path still fails closed, the take-away check finds nothing removed, and the change is a workflow edit that a revert undoes cleanly.

Notes

  • If the API flaps, both reviews can fail during an outage and then the probe succeeds once it recovers. The job then reports "the action's license verdict" when it was really an outage. Only the message is wrong, since the job still fails, so the wording could say "likely" instead of stating it as certain.

[DESIGN-REVIEWED] e8c6775

Description read

[DESCRIPTION-READ] 77e8c969799bf046a4a54ec69912f0e050bcacfaac4e387d7a5c5cb8cac608c0

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e8c67756601abbd5f74bfc7ed0d74b68464b915f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e8c6775

Verdict parsed from the review's SHA-scoped output markers for commit e8c67756601abbd5f74bfc7ed0d74b68464b915f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e8c67756601abbd5f74bfc7ed0d74b68464b915f false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e8c67756601abbd5f74bfc7ed0d74b68464b915f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e8c6775

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e8c67756601abbd5f74bfc7ed0d74b68464b915f false-positive: <reason, 20 to 500 characters>

@CrysisDeu
CrysisDeu force-pushed the fix/license-gate-outage-retry-25082 branch from 3c62e35 to bf9b0c0 Compare October 29, 2026 08:27
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 29, 2026
@CrysisDeu
CrysisDeu force-pushed the fix/license-gate-outage-retry-25082 branch from bf9b0c0 to e8c6775 Compare October 29, 2026 11:26
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 29, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at e8c67756601abbd5f74bfc7ed0d74b68464b915f.

The gate stays fail-closed. A failed review is retried once. If the retry passes, the gate passes with a warning, and every other path of the verdict step ends in exit 1 (dependency-review.yml#L54-L105). The trigger and permissions are unchanged. The token is used only for a GET in a step that runs no PR code, and the override-label path is untouched. CI and every AI lane are green on this head.

AI findings judged non-blocking

  • First Principles Review (Note): three API probes after a retry is oversized. This only adds about 40s on a path that already fails (dependency-review.yml#L93-L101).
  • Design Review (Note): a flapping API can make an outage read as a license verdict. Only the message wording is affected; the job fails either way (dependency-review.yml#L95-L98).

— crew-pr-reviewer (r135)

@iamwhatever
iamwhatever enabled auto-merge (squash) October 29, 2026 16:54
@iamwhatever
iamwhatever merged commit 0169e06 into main Oct 29, 2026
94 checks passed
@iamwhatever
iamwhatever deleted the fix/license-gate-outage-retry-25082 branch October 29, 2026 16:55
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: Dependency License Gate fails with 'Dependency review is not supported' during a GitHub outage

2 participants