Skip to content

fix(monitoring): collapse superseded check attempts by workflow run - #11830

Merged
iamwhatever merged 1 commit into
mainfrom
fix/monitor-collapse-superseded
Sep 20, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/monitor-collapse-superseded

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

What is the problem?

A structured monitor_watch on a GitHub pull request reads a check row that a newer
workflow run has already replaced as a live failure, and wakes the session on it.

GitHub leaves a replaced round's completed rows in the rollup beside the round that
replaced them. In src/kiro_crew/monitoring/github_pull_request.py,
_normalize_checks hands every CheckRun row a synthetic per-row group key,
("independent_check_run", str(row_index)), so two rows of one check survive as two
separate GitHubCheck values. A failed one enters the failed bucket,
actionable_fingerprint_facts keeps failed_checks, and the watch reports a blocker
with no live run behind it. The asymmetry is the clearest evidence: a StatusContext
row gets a real group key, ("status_context", context), so its duplicates fold, and
only CheckRun rows opt out of that mechanism.

docs/system-specs/modules/monitor-architecture.md already carried the rule under
"Rules the engine enforces, not the prose", and said in its own text that the two
implementations disagreed on it.

CANCELLED is one instance, not the mechanism. Any completed row of a replaced
round reads as live, whatever its conclusion. Measured on this pull request's own
board: five workflows carried two rounds on one head, and the un-collapsed read
reported failed: 2 naming Code Review / PR Hygiene and GPT 5.6 Review, both
FAILURE, neither CANCELLED, both from a round a newer one had replaced. Across
30 open pull requests and 2292 rollup rows, 96 duplicate identities spanned two runs.

Why this issue matters to the user

A phantom wake spends one of the watch's agent turns, and max_agent_turns is capped
at 8 by the schema, so the budget cannot be raised to absorb the waste. A watch armed
on a pull request whose CI re-ran can spend most of that budget on failures that are
not there, then fall silent before the real verdict arrives. The report is also
unactionable: the row it names has no live run to read or re-run.

Measured on this pull request's own watch, which ran against the unfixed code while
the fix was being reviewed:

Wake What the monitor reported What it was
1 2 failed both rows of a round a newer round replaced
2 6 failed all six CANCELLED from a replaced round
3 8 failed 6 replaced, 2 live -- delivered correctly
4 7 failed all seven from two replaced rounds
5 9 failed 6 replaced, 2 live -- delivered correctly
6 6 failed payload named a head a push had already superseded; not measurable

Three of the six wakes named nothing that existed: replaying wakes 2 and 4 through the
collapse turns failed: 6 and 7 into failed: 0, because every row in them was
CANCELLED. Wake 1's rows had FAILURE conclusions from a replaced round, and under the
cancellation rule those are KEPT, so that phantom is a stated residual rather than a fix. Wakes 3 and 5 were mixed, and
the collapse kept both live failures while dropping six replaced rows each. Wake 6 is
unmeasured rather than counted: its payload named a head the next push superseded, and
that commit's rollup has since changed, so the rows it saw cannot be reconstructed.

So a monitor woke at least three times on the defect it is being changed to fix, and the
change removes exactly those without suppressing a real signal. The abandoned commit is a
negative control for the same rule: every run on it was cancelled with no successor, and
the collapse correctly drops nothing there, reporting all 32 rows.

What changed: how the fix chains from symptom to root cause

The symptom is a failed bucket holding a replaced row. The root cause is that the
rollup selection never asked for the facts that identify one, so the normalizer had
nothing to collapse on and fell back to a per-row key.

_ROLLUP_SELECTION now asks each CheckRun's run for its databaseId and event,
and its workflow's databaseId. _flat_check_row carries all three through, and _collapse_superseded_rows runs before _normalize_checks groups
anything, dropping rows a newer run replaced.

The fields do different jobs. Identity is the workflow DEFINITION's id, the RUN's
triggering event and the check name. Ordering is the monotonic RUN ID.
The display name
is not an identity, since two workflow files may share one name:. The event is in the
identity because one file may declare several triggers and each dispatches its own run.
Ordering may not use a timestamp: a job's startedAt waits on a runner queue, and
createdAt resolves only to the second, which two runs of one workflow on one head
routinely share. The run id settles both and needs no extra field, since it already
groups the rows.

Recency does not license removal. The rollup has no lineage edge saying one run replaced
another, and event is coarser than the action that fired it, so a synchronize run and
an edited run share pull_request. A row is therefore dropped only when its own run was
CANCELLED and a newer run of its identity exists: cancellation by the concurrency group
establishes displacement, and such a row carries no verdict to lose. A row that reached a
conclusion is kept however old it is. In the other direction, the NEWEST run being
cancelled is never a reason to drop it, since that revives the verdict it superseded.

Two rows of ONE run are not a retry and both survive: a workflow can publish a check
run through the Checks API under its own job's display name, so both are live at
once. On one open pull request a description check published FAILURE two seconds
after its own job reported SUCCESS, on a head whose PR Readiness was FAILURE.

One rule covers absent evidence: a row is never collapsed on an id the response
withheld. Both WorkflowRun.databaseId and Workflow.databaseId are nullable Int
even though the objects holding them are not, so either can be absent alone and
neither absence implies the other. Without the definition id or the event two rows
cannot be shown to be one check; without the run id an attempt can neither lose nor
win, since it may be the run that would replace the others. Over-report rather than
hide a live failure.

Three designs were rejected. Worst-wins across attempts folds a replaced attempt and its
successor to failed. Rollup position cannot order anything; it is not chronological.
Timestamp ordering ties, which pr-readiness.yml already records. The added fields cost
nothing: rateLimit.cost is 1 before and after.

What tests we did

test/test_github_pull_request_monitor.py gains the red-first case: a replaced
attempt beside its successor must leave the failed bucket empty, in either rollup
order. It fails on origin/main with failed: ['CI / test'] and passes with the
change. The display-name collision has its own red-first case: two workflow
definitions sharing the name CI, the earlier failing, must both survive. It fails
with assert [] == ['CI / test'] before the identity change.

Nine further cases pin the semantics, and each passes on origin/main, which makes
them guards rather than restatements: two rows of one run stay independent; a run
with no start time is kept; an unidentified run never loses; an unidentified run
never decides which identified run is newest; a row with no workflow definition id is
never collapsed; two runs tied on creation time are both kept; a run missing its
creation time is kept; two triggers of one workflow do not supersede each other; and
the selection asks for every field the collapse reads. That last one exists because the provider tests answer from
canned rows, so without it the selection could drop a field and every behavioural
test would stay green while the collapse silently stopped collapsing.

Two separate hazards sit behind the ordering choice, and only one of them came from a
review lane. The lane's re-run story is refuted by measurement: on re-run commit
e7b493d2b4b094b88aa59a217ca60a4202a4b481, all 16 runs at runAttempt: 2 supply exactly
one row per check name across 36 rows, with zero (run id, check name) pairs repeated,
so no older attempt's row survives beside a new one; and CheckRun exposes no attempt
field at all, while runAttempt is run-level and identical for both hypothetical rows,
so the discriminator that story asks for does not exist. That is 36 rows on one commit,
which shows the premise is not reproducible here rather than impossible.

The second hazard is real and this repository already documented it. pr-readiness.yml
records that createdAt has one-second granularity, that two runs of one workflow on one
head routinely share a second, and that a same-second tie left last picking the OLDEST
run so a lane whose newest run succeeded read as a blocking failure. A tie is observable
on this pull request's own head: Screenshot Evidence has two runs sharing
2026-09-18T18:30:38Z, ids 35380596033 and 35380596705. Ordering moved to the run id
for that reason, not for the refuted one.

Two properties became structural rather than conditional, so they carry no mutation: two
rows of one run share one id and are therefore always both kept, and two distinct runs
cannot tie. The tie test and the missing-timestamp test were deleted with the branches
they guarded.

On the trigger case specifically: four workflow files here declare both push and
pull_request (build.yml, ci.yml, fast-gate.yml,
internal-content-scan-gate.yml), and all four filter push to branches: [main], so
a pull-request head never sees a push run and zero push runs appear on either measured
head. internal-content-scan-gate.yml also declares merge_group beside
pull_request. So the trigger in the identity closes a case the schema and the workflow
syntax permit, and that one file already pairs two events, rather than a collision
observed on this pull request.

test/test_babysit_guidance_gates.py asserts the rule against the real normalizer, and
that the spec names the identity, the function, and BOTH halves of the divergence from
the sibling tool. docs/system-specs/modules/monitor-architecture.md declares that
sibling's identity key AND its startedAt ordering, plus that it overwrites the row it
displaces instead of retaining it, so a reader is not sent to reconcile only half.
src/kiro_crew/builtin_skills/kirocrew-dev/babysit/SKILL.md told agents the typed
provider did not collapse and to resolve the newest run by hand; it states the enforced
rule instead, and says plainly what the BUNDLED pr_status.py does today, so an agent
reading that tool's output knows it can drop a live failure where the provider keeps
one.

Fourteen mutations, each anchored on an exact source string with a marker assertion
so an unapplied mutation is a failed run rather than a silent pass: five deletions (the
collapse call and each of the four fields) and nine wrapping mutations, including
ordering by the job's start instead of the run's creation, dropping the event from the
identity, keying identity on the display name, falling back to the label when the
definition id is absent, newest-wins ignoring the run, dropping the unorderable keep,
letting an unidentified run lose, letting one win, and breaking a tie. All fourteen
redden; none survives. The surviving-mutant pass is what found the untested half of the
unidentified-run rule.

prove.py returns PROVEN. The scoped backend selection for this diff, 22 files,
passes at 1351 tests, and both touched files pass at 174. Nineteen applicable
repository gates pass. mypy src/kiro_crew/ reports 4 errors, all in
src/kiro_crew/transcribe.py and the Ops Mission Control CloudWatch provider, both
byte-identical to origin/main and outside this diff.

One red on this head is not this diff's. Backend Tests (Windows) (2) fails
test_ci_fleet_routing_expression_parity.py::test_every_ci_yml_resolver_consumer_reads_the_resolver_not_a_literal.
main fails the same test on its own run 35383665208 (sha 0da53b3aaa), and this diff
changes no .github/workflows/ file. d0ff8107da (#11699) gave backend-test a
resolver-dependent runs-on without adding it to _EXPECTED_RESOLVER_CONSUMER_JOBS.
Tracked in #11883, fixed on main. That commit is already in this base, so a rebase does
not clear it and a re-run will not either.

Pattern harvest

Rule candidate: review-prompt
Pattern: a test fixture supplies a field the request under test never asked for, so the field selection is unverified and the test stays green after the selection drops it

Found by mutation, not by reading. Two selection mutations survived the first harness
run: removing a field from _ROLLUP_SELECTION changed nothing, because _provider
answers from canned rows carrying those fields regardless of the query. A fixture
more complete than its request makes a whole class of adapter bug invisible while
reading as coverage. The check is: for any adapter tested against canned payloads,
one test must assert the request asks for every field the parsing path reads.
test_the_rollup_selection_asks_for_every_field_the_collapse_reads is that test here.

Any other suggestions on the work

Two residuals are ACCEPTED here rather than fixed, and
#11890 tracks both.

Gating removal on cancellation does not cover the phantom this PR opened on. Head
d1f08548c4b7 carried FAILURE rows from a replaced round, and a row that reached a
conclusion is kept, so that wake still happens. This is the body's own point turned on
the fix: cancellation is one instance of displacement, not the mechanism, and the
rollup carries no lineage edge for the mechanism itself.

_MAX_CHECK_ROWS truncates BEFORE the collapse runs. _checks slices
rows[member.raw][:_MAX_CHECK_ROWS] at src/kiro_crew/monitoring/github_pull_request.py:490,
and _normalize_checks calls _collapse_superseded_rows at :839. On a board wider
than the cap the truncation can drop a successor row and keep the row it replaced, which
is the same phantom reached another way.

#11890 also records the shape a later fix should take: declassify a superseded failure
in the wake decision instead of deleting its row. That needs
classify_pull_request_facts in src/kiro_crew/monitoring/pull_request.py, the
classifier four providers share, which is why it is not in this diff.

Why this pushed over green boards twice. Two heads were abandoned while passing, and
the second one had more than a green board. 5bcba53863 carried Design PASS, First
Principles PASS, Opus PASS, and a consumed human override on the GPT lane. Pushing voided
all four, and they have to be re-earned here.

Both pushes were still right, for the same reason. A passing lane is one sample of a
non-deterministic reviewer, not a proof -- the GPT lane returned both a fenced block and
a clean pass on the identical tree d1f08548c4 -- and in each case the code was wrong by
construction. The first push fixed an identity keyed on a display name, inside a diff
whose own argument is that a label is not an identity. The second fixed an ordering this
repository has already documented as unsafe in the gate that decides readiness, inside the
function whose whole purpose is this collapse. A verdict obtained over either one would
have been a verdict about code we knew to be wrong. Neither push was made to refresh
verdicts; each replaced a mechanism, and the body moved with the code so the cost was one
re-roll rather than two.

Why removal is gated on cancellation rather than moved out of the provider. The
alternative considered was to keep every row and make a superseded failure non-actionable
in the wake decision instead. That is the better shape and it is deliberately not here:
classify_pull_request_facts in src/kiro_crew/monitoring/pull_request.py is the
cross-provider classifier that four providers share, and making a retained row
non-actionable means changing it or the canonical checks schema. That is a second
subsystem, and this diff's argument is the GitHub provider. Gating on cancellation keeps
the change provider-local while making the whole class of identity and ordering mistakes
unable to remove a verdict: only a row with no verdict can ever be dropped.

collapse_superseded in the prepare-pr status tool has the same label-identity
defect and is deliberately not in this diff; it is tracked in #11832. Its key
omits the run id as well, so on a same-run pair it keeps only the later row and
discards the other. Its consumer, tests and output contract are separate, so fixing
it here would double the review surface of a change whose argument is the structured
provider.

Two runs of one workflow from the SAME trigger on one commit are treated as an
attempt and its replacement, so a deliberate second dispatch of the same event
collapses to the later run. The provider exposes no replacement-lineage edge, so
definition plus trigger plus commit is the closest available approximation, and it
matches what a human reading the board does.

One hazard this does not fix, found while driving the watch: a probe can capture state
just before a push lands, so a wake's payload names a head that is already superseded.
Acting on the reported head instead of re-reading the live one means repairing something
already repaired. Re-reading the head on every wake is the only thing that catches it,
and it is not something the payload can signal.

The row cap is applied before the collapse: _normalize_checks receives
rows[:_MAX_CHECK_ROWS], so a truncated page can drop a successor and leave the row
it replaced. That is an over-report, the safe direction, but a very wide board can
still show a phantom.

This is independent of the entry-model migration in #11821 and shares no code with it.

@chenmingwei23
chenmingwei23 requested review from a team and krishdhasmana September 18, 2026 16:37
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Intent: Stop a structured pull-request watch from reporting a check attempt that a newer workflow run already replaced as a live failure, by asking the rollup for each row's startedAt and its run's databaseId and dropping superseded rows before the state fold. Supersession is keyed on the workflow run, so two rows of one run stay independent and no live failure can be erased.
Not a goal: Changing collapse_superseded in the prepare-pr status tool, altering the row cap that is applied before the collapse, replacing the ISO-8601 string comparison, or touching the entry-model migration.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 18, 2026
@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 608e2a0d369fb157d386ca1c181d437580788bd4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the stated residual — completed replaced rounds still wake as phantoms (the PR's own wake 1) — gets a tracked follow-up alongside #11832.

Suggestions

  • The classifier-level "better shape" (superseded-but-retained rows made non-actionable in classify_pull_request_facts) is named in the description but untracked; file it so the residual has an owner.

[DESIGN-REVIEWED] 608e2a0

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 608e2a0d369fb157d386ca1c181d437580788bd4 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The sibling collapse in pr_status.py keeps the label-and-startedAt rule this diff proves wrong, deferred to an issue only the description names.

Not justified as shipped

What this change ships

Inventory (8 items) — 7 justified

Intent: stop a PR watch spending its capped agent turns waking on check rows a newer workflow run already replaced — a FIX.

  1. A watch no longer wakes on a cancelled attempt a newer run of the same workflow and trigger replaced — justified
  2. A completed row of a replaced round still wakes the watch — justified (declared residual, kept to never hide a live failure)
  3. The rollup query now fetches run id, trigger event, run conclusion and workflow-definition id per check row — rides along, declared; consumers counted: _superseded_key, _check_run_of, _run_was_cancelled
  4. Rows missing any of those fields are exempt from the collapse and always kept — justified
  5. Two check rows of one run both stay live (Checks-API publisher case) — justified
  6. Monitor spec section rewritten to state the rule the provider now enforces — justified (AGENTS.md same-commit mandate)
  7. Bundled pr_status.py collapse left on display-name + startedAt, now documented as diverging — symptom-level: 1 unfixed sibling, deferral record unverifiable
  8. Tests pinning "same-labelled rows stay independent" replaced by collapse pins — justified (the pin's own message anticipated this reversal; base spec pinned the collapse rule)

[FIRST-PRINCIPLES-REVIEWED] 608e2a0

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 608e2a0d369fb157d386ca1c181d437580788bd4 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/monitoring/github_pull_request.py:75 -- current Actions run IDs exceed GraphQL Int, so "databaseId" yields unusable IDs and prevents collapsing -> Fix: select fullDatabaseId and parse its BigInt value. (origin: validation)
[GPT-REVIEWED] 608e2a0

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 608e2a0d369fb157d386ca1c181d437580788bd4: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 608e2a0d369fb157d386ca1c181d437580788bd4 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 608e2a0

Verdict parsed from the review's SHA-scoped output markers for commit 608e2a0d369fb157d386ca1c181d437580788bd4.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 608e2a0d369fb157d386ca1c181d437580788bd4: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 18, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from d1f0854 to d7e1dab Compare September 18, 2026 17:28
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition: fixed -- F1 src/kiro_crew/monitoring/github_pull_request.py (was line 737), "Distinct workflows sharing display names are collapsed". Head d7e1dabba551d95bf378e00f1cf8e2ea8bbdbdac.

Legitimate and reachable as stated. GitHub permits two workflow files to carry one name:, so (workflowName, name) grouped two independent workflows and the earlier run's row was dropped as superseded. That hides a live failure, which the function's own docstring forbids.
_ROLLUP_SELECTION now also asks for workflow{databaseId}; _flat_check_row carries it as workflowDefinitionId; _superseded_key keys identity on (workflow_definition_id, name). The display name no longer participates in the key.
A row whose definition id the response withheld is exempt from the collapse rather than falling back to the label, which would reinstate this exact collision. WorkflowRun.workflow is NON_NULL but WorkflowRun.databaseId and Workflow.databaseId are both nullable Int, so either id can be absent alone and neither exemption subsumes the other.
Red-first: test_two_workflows_sharing_a_display_name_do_not_supersede_each_other fails before the key change with assert [] == ['CI / test'] and passes after. test_a_row_whose_workflow_definition_was_not_identified_is_never_collapsed covers the exemption.
Mutation-verified: wrap-key-identity-on-the-display-name and wrap-fall-back-to-the-label-when-the-definition-id-is-absent each redden their test; 11 marker-asserted mutations, zero survivors.

self-added: yes
mechanism: collapse identity keyed on a structural workflow-definition id instead of a display label, with absence of either id exempting the row

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Sep 18, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from d7e1dab to e5fc1e7 Compare September 18, 2026 17:49
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 18, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition: fixed -- F1 src/kiro_crew/monitoring/github_pull_request.py (was line 787), "Check start times can invert workflow-run ordering". Head e5fc1e71295d12557685b897943ab0c2a199d9cb.

Legitimate, and the triggering condition is ordinary rather than extreme, as the adjudication says. A check row's startedAt is when its JOB got a runner, so an older run whose job queued can outrank a newer run, and max(dated) then drops the newer run's rows including a live failure.
Ordering moves to the RUN's own clock: _ROLLUP_SELECTION asks for workflowRun{createdAt}, _flat_check_row carries it as workflowRunCreatedAt, and _run_created_at replaces the job-start reader. startedAt is no longer selected or read, so no job clock can reorder the collapse.
WorkflowRun.createdAt is NON_NULL(DateTime) in the schema, so ordering evidence arrives with the run rather than independently. A response that omits it anyway still degrades to keeping the row rather than guessing.
Red-first: test_a_queued_job_does_not_make_its_older_run_the_newer_one builds the inversion (run 100 created 10:00 with its job starting 10:20, run 200 created 10:05 with its job starting 10:06, run 200 failing) and fails with assert [] == ['CI / test'] before the change.
Mutation-verified: wrap-order-by-the-job-start-instead-of-the-run reddens exactly that test. Twelve marker-asserted mutations, zero survivors. 175 tests and 19 repo gates pass.

self-added: yes
mechanism: supersession ordered by the run-level creation time, never by a job start time a runner queue can reorder

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 18, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from e5fc1e7 to f7a9c73 Compare September 18, 2026 18:20
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition: fixed -- F1 src/kiro_crew/monitoring/github_pull_request.py (was line 747), "independent workflow runs are treated as replacements". Head f7a9c7378d756637841019f92d84ba1d3bef2896.

Legitimate, and the adjudication is right that the trigger is a common pattern. One workflow file may declare several events, so a file on push and pull_request dispatches twice for one commit; those runs share definition id and check name while being concurrent, and keying identity on (definition, name) dropped the earlier one's live failure.
Identity is now (workflow_definition_id, run_event, check_name): _ROLLUP_SELECTION asks for workflowRun{event}, _flat_check_row carries it as workflowRunEvent, and _superseded_key requires it. Only a later run of the SAME trigger may replace an earlier one.
WorkflowRun.event is NON_NULL(String), so a row missing it means a truncated response, and such a row is exempt from the collapse rather than keyed on the display name.
Measured on this pull request's own board: 28 runs on the head commit, and every workflow definition with more than one run has a single event, so the fix changes no verdict here while closing the case the schema permits.
Red-first: test_two_triggers_of_one_workflow_do_not_supersede_each_other (push run failing, pull_request run passing) fails before the change. Mutation-verified: wrap-drop-the-event-from-the-identity and delete-event-from-selection each redden their test. Fourteen marker-asserted mutations, zero survivors; 176 tests and 19 gates pass.
Accepted residual, stated in the body: two runs of one workflow from the same trigger on one commit are treated as attempt and replacement, so a deliberate second dispatch of the same event collapses to the later run. The provider exposes no replacement-lineage edge, so definition plus trigger plus commit is the closest approximation available.

self-added: yes
mechanism: collapse identity carries the run's triggering event, so concurrent dispatches of one workflow file never read as attempts of each other

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 18, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from f7a9c73 to 5bcba53 Compare September 18, 2026 18:30
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition: accepted-and-deferred -- target=first-principles, Item 1 (pr_status.py collapse_superseded duplicate), head 5bcba538632758321c813b3900931202a4ed9ab6.

Both halves of the contradiction this PR created are fixed here, and they were the real finding. The spec now declares BOTH halves of the divergence: the sibling's identity key AND its startedAt ordering, plus that it overwrites the row it displaces instead of retaining it. The babysit skill now says what the BUNDLED pr_status.py actually does today, so its guidance no longer contradicts the tool this repository ships.
The sibling's CODE fix is deferred to #11832, not declined. Two subsystems in one PR doubles the review surface of a change whose argument is the structured provider, and this lane is advisory.
Its Clears when cannot be satisfied as written. It asks the sibling to key on the workflow-definition id and order by the run's createdAt. The GPT lane blocked exactly that key on this PR at e5fc1e71295d, as too coarse: one workflow file may declare several triggers, so (definition, name) merges two concurrent dispatches. The identity that survived review is (definition, event, name). Copying the earlier key into a second reader would propagate a design a blocking lane already rejected.
#11832 therefore carries the reviewed rule, not the one in the clears-when: identity (definition_id, run_event, check_name), ordering by the run's createdAt, and retaining rather than overwriting a displaced row.

self-added: no
mechanism: the spec declares both halves of the divergence and the skill describes the shipped tool, so the contradiction is closed without copying a rejected key into a second implementation

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
cixuuz
cixuuz previously approved these changes Sep 19, 2026

@cixuuz cixuuz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Careful, well-evidenced fix for a measured problem (96 duplicate check identities across 30 PRs in the PR's own measurement) causing phantom monitor wakes. The core rule -- displacement is proven only by the RUN's own cancellation, never inferred from a row's own CANCELLED conclusion, which also occurs inside live runs (fail-fast, failed needs, manual single-job cancel) -- is correct and consistently implemented. Field extraction in _flat_check_row populates exactly the keys the collapse functions read. Honestly documents the sibling tool's (pr_status.py) unfixed divergence via tracked issue #11832 rather than silently leaving it inconsistent.

@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from 6ec6b57 to b5bbdb7 Compare September 19, 2026 19:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Sep 19, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition of every advisory finding on head 6ec6b57c5d056af3e5d56db90160837f07160658, all three carried forward to b5bbdb7707405b28c3ba24d978286679aebdeb62.

GPT, test/test_github_pull_request_monitor.py:109 -- flat workflowRunConclusion leaves the suite extraction unexercised: fixed. This was right, and I measured it before fixing. _wire_check_row nests five flat fixture keys into the shape GitHub actually returns but did not nest workflowRunConclusion, so every fixture handed the normalizer a top-level field the host never sends and _flat_check_row's suite.get("conclusion") read was never reached. Deleting that read outright left all 202 tests green. The key now joins the flattened set and lands on checkSuite.conclusion; with the same deletion the suite goes red (2 failed), so the extraction is pinned. The helper's docstring records why the key is listed, because the next field added to the collapse will hit the same trap.

First Principles, verify on live GitHub data that checkSuite.conclusion is the row's own run's conclusion: verified, evidence below. The premise is load-bearing exactly as the review says, so it was checked against the real API rather than reasoned about. WorkflowRun exposes no conclusion field at all (schema introspection: checkSuite createdAt databaseId deploymentReviews displayTitle event file id pendingDeploymentRequests resourcePath runAttempt runNumber updatedAt url workflow), and CheckSuite.workflowRun is the inverse of the edge this selection already follows, so the suite is that run's own status container rather than an aggregate over several runs. Measured on this PR's own earlier rounds:

check suite suite conclusion row conclusions
95849195015 (superseded round) cancelled cancelled x4
95849194625 (superseded GPT round) cancelled cancelled x1
95849194861 (CI, three real test failures) failure failure x3, skipped x6, success x30

The two genuinely displaced runs conclude cancelled; the run holding real failures concludes failure. That is the discrimination the row conclusion could not make, in the false-CANCELLED direction the review named as the only one that can hide a live row.

Design, confirm the accepted residual is tolerable, and fold into #11890 when it lands: accepted as stated, no code change. A replaced round whose rows COMPLETED with a real verdict still reads as live and still wakes the session. That is deliberate and unchanged by this PR: dropping a row that reached a verdict is what would hide a live failure, and this change is the conservative direction of that trade. The residual is written into docs/system-specs/modules/monitor-architecture.md as a consequence rather than left implicit. On the suggestion: agreed, and it is the right sequencing rather than something to pre-empt here -- #11890's declassify-in-wake-decision operates on the classifier, this operates on the provider's rollup fold, and collapsing the two before #11890 exists would mean designing against an unlanded shape. Recorded for whoever picks up #11890.

No /ai-review override was used on this PR by me. The ai-review-human-override comment still visible above is from 2026-09-18 and is pinned to head 5bcba538632758321c813b3900931202a4ed9ab6; three heads have replaced it, and GPT has since produced its own unoverridden verdict on 6ec6b57c5d05, so nothing in the current board rests on it.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 19, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from b5bbdb7 to 2fe304e Compare September 19, 2026 20:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/monitor-collapse-superseded branch from 2fe304e to 7558da6 Compare September 19, 2026 21:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition of every advisory item raised on head ac1790b365dd3a356822a4d1778d748e82b9bc28. All four lanes returned PASS / no blocking findings on that head; these were the advisory remainder. Now on head cf2094bd44c6b83d2bb11fe4fd06defd354d8cde.

fixed -- target=gpt, the only finding: docs/system-specs/modules/monitor-architecture.md, "An absent run conclusion exempts it" contradicting _collapse_superseded_rows.

The finding is correct and the spec was the wrong side. The winner loop consults only the row's key and its integer run id, never the run conclusion, so a row whose CheckSuite.conclusion the host withheld still competes for newest and can displace an older cancelled row. Only that row's own removal needs displacement proof, because _run_was_cancelled returns false without the conclusion. The spec's "exempts it the same way" claimed the absence also removed the row from the comparison, which the code never did. The sentence now says absence exempts the row from removal but not from that comparison, and names the consequence.

The claim is also pinned now, not just corrected: test_a_newest_run_without_a_conclusion_still_displaces_an_older_cancelled_row asserts an older cancelled row is dropped by a newest row carrying no run conclusion. Mutation-verified. Adding a run-conclusion requirement to the winner loop fails it by its own message, plus two siblings; the unmutated tree is green at 203.

accepted, deferred -- target=design, the suggestion to run _collapse_superseded_rows before the [:_MAX_CHECK_ROWS] slice in _checks.

The reading is right. The slice at the call site truncates before _normalize_checks collapses, so a cap that cuts a successor while keeping the row it replaced leaves that row reporting live. Not folded into this PR: it moves the collapse out of _normalize_checks to its call site, which changes that function's contract and needs its own truncation-boundary tests. This PR has converged on review and the residual is disclosed rather than hidden, so it belongs in a follow-up rather than a late structural change here.

confirmed -- target=design, that the accepted residual is tolerable interim.

It is, and it is narrower than the general case: a completed FAILURE row of a replaced round still wakes a phantom (#11890), because only a cancelled row inside a cancelled run carries displacement proof. Budget waste is reduced, not removed, which is what the spec and the PR body both say.

verified -- target=first-principles, both checks.

CheckSuite.conclusion standing in for the run's conclusion was checked against live API data on this repository: the two superseded rounds returned suites concluding CANCELLED (4 rows and 1 row), while the CI run holding real failures returned a suite concluding FAILURE with 3 failure, 6 skipped and 30 success rows. So the suite conclusion discriminates supersession from a live run carrying cancelled rows, which is the property the collapse depends on. Issue #11832 exists and tracks pr_status.py's remaining divergence.

A structured watch counted a cancelled attempt a newer run had already replaced, so it woke the session on a failure that no longer existed. The rollup selection now carries each check run's startedAt and its run's databaseId, and rows a newer run replaced are dropped before the state fold. Two rows of one run stay: a workflow can publish a check run under its own job's name, and both are live.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants