Repository navigation
fix(prepare-pr): collapse superseded check rows by workflow run, not label - #16510
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that a re-run of failed jobs, which keeps the same run [DESIGN-REVIEWED] bf7a853 Description read[DESCRIPTION-READ] 97740b0229415219fd5c24a3fa9521e6a1715564d4534b4c9daa750b8ce04e18 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The same label-keyed, Not justified as shipped
What this change shipsInventory (7 items) — 6 justifiedIntent: the PR readiness report must never call a board green while one of its checks is still failing. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] bf7a853 Description read[DESCRIPTION-READ] b0b884b9fb8e66821bc8f5f7e202b8c18d72f990c64dfc3d7bf9eda5b3aecc59 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
a7e19eb to
3754676
Compare
93cc484 to
5edeeaf
Compare
|
|
…label
collapse_superseded keyed a CheckRun on ("run", workflowName, name) and kept
the row with the latest startedAt; the displaced row was overwritten, never
added to `undecidable`. Three shapes dropped a LIVE row: two check runs of ONE
workflow run (a workflow publishing under its own job's name), two workflow
files sharing one `name:`, and one file dispatching on `push` and
`pull_request` for the same commit. The run fields the correct rule needs are
not in `gh pr view --json statusCheckRollup` rows (measured: no run id,
definition id or event), so the rollup is now read through GraphQL in its own
call, keeping the stated contract: soft degradation to UNAVAILABLE, the head
re-checked beside the rollup (now on every page, and against the rollup's own
commit), a parity-pinned copy in pr_findings.py.
The collapse adopts the rule the structured monitor's GitHub provider enforces
(_mark_superseded_rows, #11830/#13394): identity is (workflow definition id,
run event, check name); newest is the greatest run id; a row is removed only
when it belongs to an older run that itself concluded CANCELLED and the row is
COMPLETED+CANCELLED; rows of one run, rows without a run identity, a replaced
round's FAILURE rows, a cancelled row inside a live run and a cancelled newest
run all stay. StatusContexts collapse by context on createdAt, ties kept.
Tests: both arrival orders for the same-run pair (the issue's shape, red on
base), one fixture per clause of the rule, the GraphQL selection pinned by
bytes, paging / mid-pagination head move / page cap, and a cross-check that the
tool and the provider answer identically on the same rows. Eleven mutations
each redden at least one named test.
Closes #11832
5edeeaf to
bf7a853
Compare
|
Problem / Motivation
Goal:
pr_status.pynever hides a live check failure behind a re-run of the same check.collapse_supersededkeyed a CheckRun on("run", workflowName, name)and kept the row with the lateststartedAt. The displaced row was overwritten inwinners, never added toundecidable, so the loss was unconditional for any dated pair. Three shapes put two LIVE rows under one key and dropped one of them:Fork PR Description / Fork PR Description, two rows of run35339317192, SUCCESS and FAILURE two seconds apart);name:, each publishing the same check name;pushandpull_requestfor the same commit (four files in this repository declare both,ci.ymlamong them).The fields the correct rule needs -- the run's id, its triggering event, its workflow definition's id, its own conclusion -- are not in the rows
gh pr view --json statusCheckRollupreturns. Measured on #11783 (97 rows), the complete key set is__typename completedAt conclusion context detailsUrl name startedAt state status targetUrl workflowName. There was no selection to extend.Why it matters
pr_status.pyis what the prepare-pr and babysit procedures read to decide whether a board is green. When the discarded row is the failing one, the tool reports a board greener than it is and an agent declares a pull request review-ready over a live failure. The arrival order that produces the harmful case is one nothing in the mechanism prevented.Not a goal
pr_findings.pylists. It never collapsed and still does not: it fetches logs for every failing row it sees, which over-reports a displaced attempt but hides nothing. Pinned bytest_the_findings_collector_lists_every_failing_row_it_fetched.failing_check_identity(theprogress_keylabel): it stays the human-readableworkflow / name, and its docstring now says so.What changed (motivation → approach → change)
Symptom: a live row of a re-run check vanishes from the report. Root cause: the collapse used a display label as identity and a job start time as order, and overwrote rather than kept the row it could not justify dropping. The correct rule already exists and is merged on the monitor side (
_mark_superseded_rows, #11830 and #13394);docs/system-specs/modules/monitor-architecture.mdand the babysit skill both named this tool as the divergent implementation and pointed at #11832.Change, in both scripts (parity-pinned):
fetch_check_rollup(repo, number, expected_head)reads the head commit'sstatusCheckRollupthroughgh api graphqlin its own call, selectingcheckSuite{conclusion workflowRun{databaseId event workflow{databaseId name}}}for CheckRuns andcontext state targetUrlfor StatusContexts. Owner, repository and cursor travel as-fraw strings (gh's typed-Fturns an all-digit value such assomeorg/2048into a JSON integer the host rejects againstString!); only the number is-F. The contract the comment aboveROLLUP_UNAVAILABLE_NOTICEstates is kept: a failed or malformed read degrades to UNAVAILABLE, the pull request's head is re-read beside the rollup -- now on every page, and the rollup's own commitoidis compared too -- and any disagreement discards the whole read as HEAD MOVED. The read pages (100 rows per page,_MAX_ROLLUP_PAGES = 10); a board past the cap reads UNAVAILABLE rather than in part, since a partial read could keep a displaced row whose successor was never fetched.flatten_rollup_rowlifts the run fields out of the check suite beside the row's own fields, so every other reader of a row (classify_check,failing_check_identity, the Actions detection onworkflowName,pr_findings'detailsUrl) is untouched.collapse_supersededadopts the provider's rule: identity is(workflow.databaseId, run.event, name)(superseded_key); newest is the greatest run id; a row is removed only when it belongs to an older run than the newest of its identity AND that run concludedCANCELLEDAND the row isCOMPLETED+CANCELLED(_displaced). Rows of one run never displace each other; a row without a run identity is exempt and takes no part in choosing the newest; a replaced round's FAILURE rows, a cancelled row inside a live run, and a cancelled newest run all stay. StatusContexts are never collapsed: the host reports one status per context and the provider keeps every status row, so the base's label-and-startedAtcollapse of them is removed rather than ported. Input order is preserved.main()s detect the repository before the rollup read (the URL regex each already had) and passrepo,number,head_sha.prepare-pr/references/rationale.mdexplains why the separate call is GraphQL.Guards enumerated. The displacement predicate must cover, and does: (1) same-run two rows -- same run id, never compared, both kept; (2) shared
name:-- different definition ids, different keys; (3)push+pull_request-- different events, different keys; (4) app-created row (workflowRunnull, measured on CodeQL rows of #16497-#16501) -- no key, exempt; (5) a withheld event -- no key, exempt; (6) no run id -- exempt; (7) replaced round with FAILURE rows -- run not CANCELLED, kept; (8) cancelled row in a FAILURE run -- run not CANCELLED, kept; (9) FAILURE row in a CANCELLED run -- row not CANCELLED, kept; (10) cancelled newest run -- it IS the newest, kept; (11) queue inversion ofstartedAt-- run id orders, notstartedAt. Siblings. Three sites collapse rows of astatusCheckRollup:monitoring/github_pull_request.py:_mark_superseded_rowsalready follows the rule;prepare-pr/scripts/pr_status.py:collapse_supersededadopts it here;dashboard/source_providers/github.py:_github_checksis the remaining copy of the defect -- it keys on(kind, workflow label, name)and overwrites bystartedAt, so the dashboard CI glyph and checks panel can still lose a live row in the same three shapes. It is left out of this PR's scope: it feeds a glyph, not the merge decision, and fixing it means moving_github_rollup_readto the same GraphQL read inside the provider's sandboxed runner, which is a larger change than this one. Tracked in #16519 (deferred-finding,needs-triage, assigned, due 2026-10-17). One site that LOOKS like a copy and is not:failing_check_identitykeys on(workflowName, name)too, but it builds aprogress_keylabel and drops nothing, so a label collision there costs at most a stall-detector miss, not a hidden failure.Backwards compatibility
Compatible: both scripts accept the same arguments and print the same sections; the only observable change is that rows the old rule dropped by label and job start time now follow the run-keyed rule (and StatusContexts are no longer collapsed at all), which keeps strictly more live rows and removes only cancelled rows of cancelled older runs. The read needs the same Checks scope as before (a Checks-blind token degrades to the same UNAVAILABLE notice).
Removes nothing:
fetch_check_rollup's signature changed from(pr, expected_head)to(repo, number, expected_head), and its only callers are the twomain()s in this diff;collapse_supersededkeeps its name and return shape.Tests
Targeted, run by path with
-n0(276 passed across the three files):test/test_prepare_pr_status.py-- red on base, green after:test_two_live_rows_of_one_run_both_survive_in_either_arrival_order(the issue's shape, FAILURE first and SUCCESS first),test_two_rows_of_one_run_are_both_reported_not_only_counted(identity-level, both orders),test_same_check_name_in_different_workflows_stays_distinct,test_two_triggers_of_one_workflow_are_concurrent_dispatches_not_attempts,test_rows_without_a_run_identity_are_exempt_not_grouped_together,test_rows_missing_only_the_event_are_exempt_not_merged,test_unordered_duplicates_are_all_kept_fail_closed,test_a_replaced_rounds_failure_rows_stay_live,test_a_cancelled_row_inside_a_live_run_is_not_dropped,test_a_cancelled_rows_verdict_must_also_be_cancelled_to_drop_it,test_a_cancelled_newest_run_is_never_dropped,test_newest_is_the_greatest_run_id_not_the_latest_started_row,test_a_displaced_round_is_dropped_whole_not_row_by_row(the fix(security): quote-aware argv windows for the mint and pkill self-protection floors #16496 shape),test_status_contexts_are_never_collapsed,test_status_contexts_that_cannot_be_ordered_are_all_kept. The read:test_the_rollup_query_selects_every_field_the_collapse_keys_onpins the selection bytes and the-f/-Fsplit of the variables (the fake answers whatever a fixture carries, so a field the query never asked for would otherwise pass every test),test_the_rollup_read_pages_and_re_checks_the_head_on_every_page,test_a_head_that_moves_mid_pagination_discards_the_rollup,test_a_rollup_whose_commit_is_not_the_reported_head_is_discarded,test_a_board_past_the_page_cap_reads_unknown_not_partial,test_a_malformed_rollup_page_reads_unavailable. The three degraded-read tests now fail the GraphQL call and assert the core read never names the rollup.test/test_prepare_pr_findings.py--TestRollupHelperParitynow pinsROLLUP_QUERY,_MAX_ROLLUP_PAGES,flatten_rollup_rowandfetch_check_rollupbyte-identical across the two scripts;test_the_findings_collector_lists_every_failing_row_it_fetchedpins the residual named above.test/test_babysit_guidance_gates.py-- the spec/skill pin flips from "declares the divergence" to "names both enforcers and no divergence", andtest_the_skill_states_the_collapse_rule_the_typed_provider_implementsnow feeds the same four rows tocollapse_supersededand to the provider's_normalize_checksand asserts the same three answers.test/skill_script_helpers.py--rollup_graphql_responsebuilds the host's nested GraphQL shape from flat fixtures;is_rollup_graphql_readroutes the fake.Mutation verification (each applied to the real script, tests run, file restored): keying on the label, dropping the event, ordering by
startedAt, dropping every older-run row, ignoring the run conclusion, ignoring the row verdict, last-writer-wins within a run, grouping unidentified rows under aNonekey, re-introducing acreatedAtcollapse of StatusContexts, passing the string variables as typed-F, skipping the commit-oid head check, and returning the partial board at the page cap -- twelve mutations, each reddening at least one of the named tests.Manual verification
Read-only runs of the modified scripts against live pull requests in this repository:
pr_status.pyon fix(security): quote-aware argv windows for the mint and pkill self-protection floors #16496: 111 raw rows, 106 reported, 5 removed -- every removed row checked on bytes against the predicate (older run37119479820/37119479865concludedCANCELLED, rowCOMPLETED/CANCELLED, newer run37119518007/37119517987of the same(definition, event, name)present).pr_status.pyon fix: cron_add inherits the caller Slack thread automatically #11783: 97 raw rows, 97 reported, 0 removed (no cancelled run on the current head).pr_findings.pyon fix(security): quote-aware argv windows for the mint and pkill self-protection floors #16496: completes, lists failing rows and drills into their logs as before.workflowRun.databaseId,event,workflow.databaseIdon every Actions row andworkflowRun: nullon the CodeQL app row.Screenshots / video
N/A -- no user-visible frontend surface; the change is in two CLI scripts and their docs.
Related Issues
Closes #11832
Pattern harvest
Rule candidate: review-prompt
Pattern: a dedup/collapse keyed on a display label (name, title) and ordered by a per-item timestamp, where the host offers a stable id for the thing being deduplicated; the displaced item is overwritten rather than kept when the order cannot be justified.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)