Skip to content

fix(ci): re-run an advisory review lane whose withheld verdict never published - #17698

Merged
chenmingwei23 merged 2 commits into
mainfrom
fix/13100-republish-withheld-verdict
Oct 9, 2026
Merged

chenmingwei23 merged 2 commits into
mainfrom
fix/13100-republish-withheld-verdict

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Why this change

The issue was filed by the maintainer (chenmingwei23) after review of #12952/#12957, which deliberately left the recovery for a withheld advisory verdict unimplemented. It affects any PR whose Design/UX/First Principles verdict is computed but lands in an already-occupied comment slot — the PR then reads as unreviewed for its own head and sits at a pending readiness state until a human notices a job-log warning and re-runs the lane by hand. This is CI-infrastructure only: it changes no user-facing default, config, UI or output, and adds no setting; it is a maintainer/reviewer-facing reliability fix for a common steady-state arrangement (a slot occupied by an earlier head's verdict), not a niche need.

Problem / Motivation

Goal: A whole-design review lane that computes a verdict for the current head but cannot publish it is automatically re-run so its verdict reaches the pull request, instead of the PR reading as unreviewed until a human intervenes.

A whole-design advisory lane (Design, UX, First Principles) can finish success, compute a verdict for the current head, and still fail to publish it: the shared comment-write primitive refuses to overwrite a comment slot an earlier head still occupies. The lane reports honestly that it published nothing, the verdict is retained as a workflow artifact, and PR Readiness scores the lane as a named pending — (verdict not published, re-run this lane) — rather than passed. Nothing then re-runs the lane, so the pull request stays stuck as unreviewed for its own head with no automatic recovery.

Why it matters

A PR whose advisory verdict is withheld sits in a pending readiness state indefinitely. The only recovery today is a human noticing and manually re-running the lane. On a busy queue that is slow and easy to miss, and it is exactly the kind of self-healing the readiness sweep already provides for the analogous dropped-signal case — so the gap is both user-visible (a blocked PR) and inconsistent with the existing recovery design.

Not a goal

  • Does not change the comment-write primitive's authority contract (no caller-supplied replace permission): that trust-boundary change was considered and rejected.
  • Does not republish from the retained withheld-verdict artifact: replaying a stored verdict re-opens the same authority question the write primitive exists to answer.
  • Does not touch the required lanes (Opus, GPT, Security Scope): they fail closed on a withheld verdict and are recovered by their own check-run finalize, never read as a stale success.

What changed (motivation → approach → change)

  • Symptom: an advisory lane completes success, its verdict never reaches the comment slot, and readiness parks the PR at a named pending forever.
  • Root cause: the self-heal engine (pr-readiness-sweep.yml) re-fires the readiness aggregator, but a recompute only re-derives the same named pending — nothing re-runs the owing lane, which is the only action that republishes the verdict.
  • Approach (smallest option matching existing patterns): the sweep is already the actions: write, 5-minute, self-terminating re-fire engine. Give the withheld-verdict case a stable signal and let the sweep re-run the lane rather than the aggregator, reusing the one predicate that already decides which lane is owed.
    • pr-readiness.yml stamps a stable leading token [verdict-unpublished] on the pending status description when an advisory lane owes the head a verdict — mirroring the existing [read-failed] token, so readers that already tolerate a leading token are unaffected.
    • pr-readiness-sweep.yml detects that token on the single status-description read it already makes in its no-evidence branch (zero extra REST reads), and for those PRs runs the new .github/scripts/republish_withheld_verdict.py instead of dispatching the aggregator.
    • republish_withheld_verdict.py asks pr_status.py --disposition-gate (the one evaluate_reviewer_markers predicate that both PR Readiness and the local gate answer from) which advisory lanes are owed, locates each lane's run (same-repo by head SHA; fork via its <!-- ai-review-fork-lane run=<id> --> marker, as the human-override handler does) and gh run reruns it. The lane recomputes a fresh verdict and republishes through the same guarded upsert, which already resolves the authority question correctly, so no new trust surface is added.
    • Self-terminating: once a re-run republishes, the lane leaves unpublished, readiness drops the token, and the next sweep does not re-fire. A re-run already in flight shows an in_progress check-run (scored (not started), no token), so the dispatch→republish window re-fires nothing.
    • Bounded re-run (review fix): the sweep places this recovery outside MAX_DISPATCH, so a withhold cause that persists (an unreadable head/slot read, a PATCH that keeps failing) would otherwise re-run a paid model review every STALE_MINUTES forever. Each lane's run now carries its run_attempt (already in the runs-list payload for the same-repo path and in the actions/runs/{id} read the fork path already makes), and the sweep stops re-running a lane once its attempt count reaches MAX_RERUN_ATTEMPTS (3), logging a request for a manual re-run instead. Three attempts clears a transient cause while bounding the fail-open cost; it never reaches GitHub's per-run 50-rerun platform limit.

Backwards compatibility

Compatible: the change only adds a new recovery path and a new leading status-description token. No field is rejected, renamed or removed; the sweep's existing modes and the readiness scoring are unchanged, and the new token mirrors the pre-existing [read-failed] token that status-description readers already tolerate.

Removes nothing: no path is hidden, no record or key is deleted, renamed or validated more strictly — the diff only adds a token prefix, a sweep branch, and a new script.

Tests

  • test/test_republish_withheld_verdict.py — unit tests for the new script: owed-lane parsing from the disposition gate, same-repo run lookup by head SHA, fork run lookup via the lane marker, the gh run rerun invocation, the lane→workflow mapping pinned against the override handler's table, fail-closed no-op when the gate returns nothing or an empty set, the MAX_RERUN_ATTEMPTS bound (same-repo and fork paths stop re-running at the cap and still re-run below it), and the same-repo head_repository.full_name binding that rejects a run in a sibling head repository.
  • test/test_pr_readiness_sweep.py — behavioural: a [verdict-unpublished] pending re-runs the owing lane and does NOT dispatch the aggregator; plus a cross-file pin that the token literal PR Readiness stamps and the token the sweep classifies on stay identical.
  • test/test_readiness_sweep_scan.py — the sweep's sparse-checkout list includes the republisher and the disposition-gate scripts (a dropped entry would be a runtime ImportError only hit in production), and the recovery re-runs the lane via the republisher rather than dispatching the aggregator.

Manual verification

N/A — the sweep and readiness bash paths are exercised end-to-end by the test modules above (they run the actual sweep script against fixtured repository states), and the new script is driven with gh and the disposition gate replaced by stubs. CI's actionlint lane is the authoritative workflow-syntax check.

Related Issues

Fixes #13100

Pattern harvest

Not generalizable: this is a specific missing recovery edge in the readiness self-heal engine (a lane that is success yet owes a verdict), not an instance of a broad defect class — the fix is to add the one recovery branch that was absent, reusing existing predicates and mechanisms.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested a review from a team as a code owner October 7, 2026 10:37
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of f086c7d241683c4424b4ab1a846dd6ed1db3abd2 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check one real sweep tick (same-repo and fork) where a GITHUB_TOKEN POST …/rerun re-executes the lane and replaces the older-head comment, instead of withholding again.

[DESIGN-REVIEWED] f086c7d

Description read

[DESCRIPTION-READ] e0e4e8c09068201fe8f13315225da1f5f87be0f9d6f8617f84949724c19e5f12

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of f086c7d241683c4424b4ab1a846dd6ed1db3abd2 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The new script re-implements the override handler's lane-run lookup, and the fork path has already drifted from it: it drops the handler's per-PR row binding.

Not justified as shipped

  • Item 4 — duplicate of .github/workflows/ai-review-human-override.yml (rerun_fork_lane / rerun_run, lines 225–379): the description says the script locates runs "exactly as the human-override handler does". The handler picks the row whose external_id == "$lane-pr-$PR-$fork_trigger", because a shared head can carry "a sibling pull request's row". _fork_run_id instead takes the "Newest check-run row by id" across every row of that check name. That gives two copies of run location and of the LANES table (2 counted: the handler, republish_withheld_verdict.py), and they already disagree.
    Subtraction: delete the newest-by-id row pick in _fork_run_id and use the handler's <lane>-pr-<PR>-<fast-gate run>-<attempt> row binding, so only one binding rule exists.
    Clears when: the fork lookup selects the same bound row the handler and PR Readiness read, or a test shows a sibling PR's row on a shared head is refused.

What this change ships

Inventory (6 items) — 5 justified

Intent: a PR whose advisory verdict was computed but withheld gets that verdict published without a human re-running the lane. FIX (#13100).

  1. PR Readiness pending description now leads with [verdict-unpublished] when an advisory lane owes a verdict — justified
  2. The readiness sweep re-runs the owing lane instead of re-firing the aggregator for that pending — justified
  3. Automatic re-runs stop after 3 attempts (MAX_RERUN_ATTEMPTS) and the log asks for a manual re-run — justified
  4. New script that finds the lane's run (same-repo by head SHA, fork by marker) and re-runs it — duplicate of ai-review-human-override.yml run lookup, already diverged
  5. The sweep's sparse checkout now also pulls pr_status.py and its two sibling modules — justified
  6. Docs: new ci-and-reviews section, plus a reworded Issue Gate re-run note — justified

[FIRST-PRINCIPLES-REVIEWED] f086c7d

Description read

[DESCRIPTION-READ] d204a97ea2091a7adb3c99d2aa020d88465f2175c4d2797219c630ce0d593435

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of f086c7d241683c4424b4ab1a846dd6ed1db3abd2 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] f086c7d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f086c7d241683c4424b4ab1a846dd6ed1db3abd2: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed f086c7d241683c4424b4ab1a846dd6ed1db3abd2 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] f086c7d

Verdict parsed from the review's SHA-scoped output markers for commit f086c7d241683c4424b4ab1a846dd6ed1db3abd2.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f086c7d241683c4424b4ab1a846dd6ed1db3abd2: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 7, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — request changes (one blocking finding)

This PR adds a recovery for a withheld advisory verdict: PR Readiness stamps [verdict-unpublished], and the self-heal sweep re-runs the owing Design/UX/First Principles lane through republish_withheld_verdict.py. The scope, fork handling and token use look right. The blocking issue is that the re-run has no attempt limit.

Verified

  • Blocking lanes are never re-run. owed_lanes reads pr_status.py --disposition-gate's unpublished list, which is restricted to WHOLE_DESIGN_LANES (pr_status.py:2053-2063), and then filters to LANES (the three advisory lanes). Opus, GPT and Security Scope cannot reach _rerun.
  • It does not overwrite a newer verdict. The re-run lane publishes through its existing guarded upsert. That upsert refuses a superseded-head body (design-review.yml guarded_comment_upsert) and re-confirms the head before any repeat write (confirm_head, rc 4). If the PR moves between the sweep's read and the re-run, the lane stands down. A re-run requested while the run is still queued or in progress is rejected by the API and logged, so it is not doubled.
  • Token permissions and fork safety. The workflow permissions: are unchanged (actions: write already existed for the dispatch). The scripts run from a sparse checkout of the default branch with persist-credentials: false, and no fork code runs. A fork Stage-2 run id read from the check-run marker is re-checked against .github/workflows/fork-*-review.yml before re-running. The re-run reuses the same privileged path the override handler already uses, so there is no new trust surface.
  • The sparse checkout covers everything pr_status.py loads (_review_contract.py, green_age.py).

Mutations (run locally)

Guard removed Test that failed
name in LANES filter in owed_lanes test_owed_lanes_drops_names_this_script_cannot_rerun
fork run path verification test_fork_lane_marker_pointing_at_the_wrong_workflow_is_refused
gate ok fail-closed check test_owed_lanes_fails_closed_when_the_gate_is_not_ok
same-repo head_repository.full_name binding none: only head_branch is exercised

Tests run locally: test/test_republish_withheld_verdict.py (9 passed). test/test_pr_readiness_sweep.py could not run here: it times out per test in this sandbox, identically on origin/main, so I only checked the sweep shell changes by reading the source.

Blocking

The automatic re-run has no attempt bound. The recovery ends only when a re-run publishes. It is fail-open on cost if the cause of the withhold persists. Withhold causes include a slot or head read that stays unreadable after 6 attempts, which is exactly the API-exhaustion case, or a PATCH that keeps failing. In that case the loop is:

  1. The re-run withholds again.
  2. Readiness republishes the pending with the token.
  3. After STALE_MINUTES (15) the sweep re-runs the lane again.

That is a full paid model review roughly every 15-20 minutes, per lane, per PR. The sweep places it deliberately outside MAX_DISPATCH, so the only real bound is GitHub's platform limit of 50 re-runs per workflow run. GitHub's changelog says an attempt past that limit produces a failed check suite. I have not verified whether that would surface as a failed lane, and therefore a BLOCK, in readiness. The shared-head fork mis-binding below makes the same loop reachable with no infrastructure fault at all.

Fix: have _same_repo_run_id / _fork_run_id also return the run's run_attempt. It is already in the runs-list payload, and the fork path already reads actions/runs/{id}. Skip the re-run when run_attempt >= N (for example 3) and log "re-run this lane manually". Add a test that pins the cap and fails when it is removed.

Non-blocking follow-ups

  • pr-readiness-sweep.yml:519: the old short-circuit avoided the status read while the pending was younger than STALE_MINUTES. Move readiness_description back inside if [ "$age" -ge "$stale_seconds" ]; both consumers are age-gated anyway. This removes an extra read per tick on every young pending in the shared REST pool. (Opus finding, confirmed.)
  • _fork_run_id picks the newest check-run by name with no external_id == "<lane>-pr-$PR-<trigger>" binding, unlike ai-review-human-override.yml bind_fork_trigger. On a shared fork head it can re-run a sibling PR's lane. Bind it the same way, or have one implementation call the other. (First Principles finding, confirmed.)
  • Add a test that fails when the same-repo head_repository.full_name binding is removed.
  • The docstring's root-cause sentence describes only the create-path refusal. The replace path does overwrite an older head's comment, so say when a re-run actually helps.
  • republish()'s is_fork/head_repo/head_ref parameters are used only by tests; consider stubbing _resolve_pr_meta instead.
  • _gh_json and owed_lanes call subprocess.run with no timeout. One hung gh call can hold the sweep's concurrency group until the 15-minute job timeout.

Upgrade impact

None for users: this is CI-only with no config or state. The readiness description gains an optional [verdict-unpublished] prefix, placed after [read-failed] so the existing prefix match still holds.

@iamwhatever

Copy link
Copy Markdown
Contributor

Reviewed, not approving (crew-pr-reviewer). This edits the readiness workflows, and @buluoray's change request is open. Left to the author and @buluoray.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review — the verification of the scope, fork handling and non-overwrite guarantees matches the intent. Addressing each point against new head 71ad570135.

Blocking: the re-run has no attempt bound — fixed

Added a bounded re-run in republish_withheld_verdict.py:

  • _same_repo_run_id and _fork_run_id now return (run_id, run_attempt). The attempt count is already in the runs-list payload for the same-repo path, and in the actions/runs/{id} read the fork path already makes for its workflow-verification, so the bound costs no extra REST call on either path.
  • MAX_RERUN_ATTEMPTS = 3. The republish loop skips a lane once run_attempt >= MAX_RERUN_ATTEMPTS and logs ... has reached N attempts without publishing its verdict for <head>; not re-running it. Re-run this lane manually from the Actions tab. — so a persistent withhold cause (an unreadable head/slot read, a PATCH that keeps failing) can no longer re-fire a paid model review every STALE_MINUTES forever. Three attempts clears a transient cause and never approaches GitHub's per-run 50-rerun platform limit.

Tests pin the cap and fail if it is removed: test_same_repo_run_at_the_attempt_cap_is_not_rerun, test_fork_run_at_the_attempt_cap_is_not_rerun, and test_same_repo_run_below_the_cap_is_still_rerun (so the bound stops a stuck lane without stopping the recovery itself). I mutation-checked all three locally — each fails when the guard is removed.

Mutations table — the missing same-repo head_repository.full_name test — added

Added test_same_repo_binding_rejects_a_run_in_a_sibling_head_repo: a run matching the workflow, head SHA and branch but living in a different head repository (the shared-fork-head case) must be rejected. The existing test_same_repo_binding_rejects_a_sibling_prs_run varied both repo and branch, so dropping only the full_name check left it green; the new test isolates the binding and fails when it is removed (mutation-verified).

Non-blocking follow-ups

These are genuinely non-blocking and I am holding them to keep this PR within the PR-Hygiene 2-commit cap (it is now at 2). Taking them each individually:

  1. pr-readiness-sweep.yml:519 — move readiness_description back inside the age gate. Agreed, this is a real per-tick REST saving on young pendings. It touches the sweep workflow (not this fix's script), so I would rather land it as its own small change than squash it into the bounded-retry commit; filing a follow-up.
  2. _fork_run_id lacks the external_id binding bind_fork_trigger uses. Agreed and worth doing — a shared fork head can host a sibling PR's lane. This is a correctness item on the fork path; I will fix it in the follow-up alongside (1) so it gets its own test rather than riding the cap here.
  3. Same-repo head_repository.full_name test — done above (promoted out of the follow-up list since the review named it under Mutations).
  4. Docstring root-cause sentence. The module docstring already states both paths — the create-path refusal ("refuses to overwrite a comment slot an earlier head still occupies") and the replace-path behaviour ("it replaces an occupant left by an OLDER head, and stands down when the slot already holds a verdict for the current head"). If you meant a specific sentence that still reads create-only, point me at it and I will tighten it in the follow-up.
  5. republish()'s is_fork/head_repo/head_ref test-only params → stub _resolve_pr_meta instead. Reasonable; a test-ergonomics refactor with no behaviour change, deferred to the follow-up.
  6. _gh_json/owed_lanes subprocess.run with no timeout. Agreed — one hung gh can hold the sweep's concurrency group to the 15-minute job timeout. A bounded timeout= on both is small and self-contained; deferred to the follow-up with the others.

I will open the follow-up for 1, 2, 4, 5 and 6 and link it here. Happy to fold any of them into this PR instead if you would prefer them here over the 2-commit cap.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Follow-up filed for the five deferred non-blocking items (1, 2, 4, 5, 6 above): #17788.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision f086c7d241683c4424b4ab1a846dd6ed1db3abd2 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
kirocrew-agent added 2 commits October 7, 2026 16:49
…published (#13100)

A whole-design review lane -- Design, UX, First Principles -- can compute a
verdict for the current head and fail to publish it, because the lane's shared
comment-write primitive refuses to overwrite a comment slot an earlier head
still occupies. The lane completes `success` and reports honestly that it
published nothing (#12957), the verdict is retained as a workflow artifact
(#13911), and PR Readiness scores the lane as a named pending rather than passed
(#13419). What was missing is the recovery: nothing re-ran the lane, so every
consumer that keys on the head (`pr_status.py`'s stale-stamp report) read the PR
as unreviewed for its own head until a human noticed the warning in a job log.

This adds the sweep-triggered recovery the issue asks for, in the shape of
`pr-readiness-sweep.yml`'s existing dropped-signal self-heal and reusing #13419's
single predicate, rather than the two rejected alternatives (replaying the
retained artifact reopens the authority question the write primitive's refusal
exists to answer; a caller-supplied replace permission is a trust-boundary change
to the shared primitive).

What changed:

- pr-readiness.yml stamps a stable `[verdict-unpublished]` token first on the
  pending status description whenever an advisory lane is scored
  `(verdict not published, re-run this lane)`, mirroring the `[read-failed]`
  token so a 140-char truncation cannot strip it.

- pr-readiness-sweep.yml, on a stale pending carrying that token, runs the new
  .github/scripts/republish_withheld_verdict.py instead of re-firing the
  aggregator (which would only re-derive the same named pending). The script
  asks pr_status.py's `--disposition-gate` which advisory lanes are owed -- the
  one `evaluate_reviewer_markers` predicate both gates answer from -- locates
  each owing lane's run (same-repo by head; fork via its
  `<!-- ai-review-fork-lane run=<id> -->` check-run marker, as the human-override
  handler does) and `gh run rerun`s it. The re-run recomputes and republishes
  through the lanes' own guarded upsert, whose authority rule is unchanged, so
  the recovery adds no new trust surface.

Scoped to the three advisory lanes by construction: the required reviewer lanes
(Opus, GPT, Security Scope) fail CLOSED on a withheld verdict and are recovered
by their own check-run finalize, never read as a stale `success`.

Self-terminating: once a re-run republishes, the lane leaves the gate's
`unpublished` set, the token clears, and the next sweep does nothing; a re-run in
flight carries an `in_progress` check-run readiness scores `(not started)`, so
the dispatch-to-republish window re-fires nothing.

Tested: unit tests for the republisher (lane location same-repo + fork, the
fork-marker workflow verification, fail-closed on an unreadable gate, the lane
table pinned against the override handler); behavioural sweep tests that a
`[verdict-unpublished]` pending re-runs the lane and does not dispatch the
aggregator; token-agreement and per-site stamp pins across the two workflows;
updated the sparse-checkout pin. Not run on a live forge.

Fixes #13100
 review)

A persistent withhold cause (an unreadable head/slot read, a PATCH that keeps
failing) would make the sweep re-run a paid advisory review lane every
STALE_MINUTES forever, since the recovery is placed outside MAX_DISPATCH on
purpose. _same_repo_run_id and _fork_run_id now also return the run's
run_attempt -- already in the runs-list payload, and in the actions/runs/{id}
read the fork path already makes -- and the sweep stops re-running a lane once
its attempt count reaches MAX_RERUN_ATTEMPTS (3), logging a manual-re-run
request instead.

Also adds the mutation test the review named for the same-repo
head_repository.full_name binding, which no prior test exercised in isolation.
@chenmingwei23
chenmingwei23 force-pushed the fix/13100-republish-withheld-verdict branch from 71ad570 to f086c7d Compare October 7, 2026 16:51
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@chenmingwei23
chenmingwei23 requested a review from buluoray October 7, 2026 22:35
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 8, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Reviewed again at the new head, still not approving (crew-pr-reviewer). It edits the readiness workflows, and @buluoray's change request is still open. Eight checks are red on this head, including Sensitive Change Review.

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. The blocking finding from the previous round is fixed at f086c7d241.

Prior blocking finding: FIXED. republish() now reads run_attempt for the located run and stops at MAX_RERUN_ATTEMPTS = 3 (.github/scripts/republish_withheld_verdict.py:105-115, :359-371). It prints a manual re-run request instead of calling _rerun. The attempt comes from payloads the script already fetches: the runs-list entry on the same-repo path (:197-215) and the actions/runs/{id} verification read on the fork path (:270-276), so the cap adds no REST call. A missing or malformed run_attempt defaults to 1. That means the run gets re-run, the same behaviour as before this PR, but it is limited to the rare case where GitHub omits the field.

What I checked

  • The targeted tests pass at the head.
  • I turned off the cap check, and both test_same_repo_run_at_the_attempt_cap_is_not_rerun and test_fork_run_at_the_attempt_cap_is_not_rerun went red.
  • I forced the same-repo attempt read to 1, and the same-repo cap test went red. Forcing the fork attempt read to 1 turned the fork cap test red.
  • One attempt below the cap still re-runs (test_same_repo_run_below_the_cap_is_still_rerun).
  • I have not exercised this against live Actions. The cap depends on gh run rerun incrementing run_attempt on the same run id, which is GitHub's documented behaviour.

Non-blocking follow-ups

  • docs/ci/ci-and-reviews.md (the "A withheld advisory verdict is re-run and republished" section) and the sweep's inline comment at .github/workflows/pr-readiness-sweep.yml:520-540 still describe the recovery as only "self-terminating". Add a sentence on the 3-attempt cap and the manual re-run it asks for.
  • Carried over: a young pending still costs an extra REST read per tick.
  • Carried over: the fork run lookup verifies the workflow path but does not bind the check-run's external_id.
  • The manual re-run request only appears in the sweep job log. Consider putting it somewhere a human will see, such as the readiness description or a PR comment, so a stuck lane does not stay quietly pending.

Upgrade impact: none. The script and sweep step are new in this PR, and the cap only narrows them. Audience fit: CI-only. The default path (a verdict that publishes normally) never reaches this code.

@chenmingwei23
chenmingwei23 merged commit 0c85be5 into main Oct 9, 2026
149 of 156 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/13100-republish-withheld-verdict branch October 9, 2026 06:16
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: a review lane's completed verdict is never republished after its comment slot is found occupied

3 participants