Skip to content

fix(dashboard): collapse check rows by workflow run, not label - #17570

Merged
iamwhatever merged 2 commits into
mainfrom
fix/16519-dashboard-check-collapse
Oct 7, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
fix/16519-dashboard-check-collapse

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The dashboard's CI glyph, sidebar chip and checks panel never read green while a live check on the pull request is failing.

_github_checks (the projection behind the dashboard's checks panel, CI glyph and sidebar chip) kept "the latest run per check", keyed on (kind, workflow label, name) and ordered by the row's startedAt; the later row overwrote the earlier one. That loses a live row in three shapes: two check runs of one workflow run (measured on #11783: Fork PR Description / Fork PR Description, SUCCESS and FAILURE two seconds apart in one run), two workflow files sharing one name:, and one workflow file running on both push and pull_request for the same commit. When the overwritten row is the failing one, the glyph reads green over a live failure.

Why it matters

The CI glyph and the sidebar chip are what a person glances at before opening a pull request. A green glyph over a live failure sends them toward merge with the wrong picture, and nothing in the old mechanism prevented the arrival order that produces it.

Not a goal

  • Changing how buckets roll up into a glyph (_rollup_ci) or the GitLab projection.
  • Changing the structured monitor's or prepare-pr's collapse rules; this PR adopts the monitor's rule as-is.

What changed (motivation → approach → change)

  • Root cause: the gh pr view --json statusCheckRollup payload carries no workflow-run identity, so the projection could only key on the display label and order by job start time, which cannot tell "a newer run replaced this one" from "this is a different live row".
  • Approach: use the rule the other two sites already share (monitoring/github_pull_request.py:_mark_superseded_rows, prepare-pr/scripts/pr_status.py:collapse_superseded). Identity is (workflow definition id, run event, check name); newest is the greatest run id; a row is dropped only when its older run concluded CANCELLED and the row itself is COMPLETED+CANCELLED. Rows of one run, rows without a run identity, and status contexts are kept. The dashboard calls the monitor's _rollup_page (page parse and row flattening) and _mark_superseded_rows directly rather than carrying a third copy; they are imported at call time so loading the gateway with monitors off does not load the monitor's provider runtime.
  • Change:
    • _github_rollup_read now reads the head commit's statusCheckRollup through gh api graphql (same sandboxed provider runner, same _CHECKS_OUTPUT_BYTES cap, still its own call so a Checks-blind token costs only the checks section). It pages 100 rows at a time up to 10 pages, parses each page with _rollup_page, joins the pages before collapsing, requires every page to name the same head on both the pull request and the rollup's commit, and raises (degrading the section to partial) on a malformed page or a board past the cap rather than rendering part of it.
    • _github_checks now projects the flat rows through _mark_superseded_rows and drops only the flagged ones, in input order.
    • _github_check falls back to a status context's createdAt for startedAt (the GraphQL shape names it differently from gh pr view).
    • _github_check_identity and _github_check_rank are removed; nothing else called them.
  • The chip refresh path (chip_refresh.py) and the polled checks endpoint (_fetch_github_checks) consume _github_rollup_read and inherit the fix unchanged.

Backwards compatibility

Compatible: the payload shape the dashboard and plugins receive (checks rows with name, workflow, status, conclusion, bucket, url, startedAt, completedAt) is unchanged; only which rows survive the collapse changes, and only toward keeping more.

Reader: src/kiro_crew/dashboard/handlers/source_providers.py:_OWNED_NAMES["github"] -- entry: crew page -- the two removed helper names are dropped from the re-export list; no caller on main reads them (grep of src/ and test/ finds only github.py and the facade list).
Reader: test/test_source_providers_refactor_facade.py:_HISTORICAL_SURFACE -- entry: release -- names removed from the pinned surface alongside the export list; test_the_handler_and_its_owners_split_the_surface_exactly passes.
Reader: src/kiro_crew/dashboard/source_providers/chip_refresh.py:_fetch_check_status -- entry: crew page -- still receives (checks, head) from _github_rollup_read; test_github_check_status_collapses_superseded_runs.
Reader: src/kiro_crew/dashboard/source_providers/cache.py:_fetch_pull_request_checks_uncached -- entry: crew page -- still receives the checks list from _fetch_github_checks; test_fetch_github_checks_uses_one_call_without_rewriting_cache.

Tests

In test/test_source_providers.py:

  • test_github_checks_drop_a_row_its_cancelled_run_left_behind: a concurrency-cancelled older run is dropped and the glyph reads passed.
  • test_github_checks_keep_a_failure_from_an_older_run: an older run that FAILED is kept (a newer run id alone proves no replacement).
  • test_github_checks_keep_both_rows_of_one_run (both arrival orders) and test_fetch_github_checks_keeps_both_rows_of_one_run (end to end through the GraphQL shape): the fix: cron_add inherits the caller Slack thread automatically #11783 same-run SUCCESS/FAILURE pair keeps both rows and rolls up failed.
  • test_github_checks_do_not_group_two_workflow_files_sharing_one_name, test_github_checks_do_not_group_runs_of_different_events: identity is definition id + event, not the label.
  • test_github_checks_keep_a_cancelled_row_of_a_live_run, test_github_checks_keep_a_cancelled_newest_run, test_github_checks_keep_rows_without_a_run_identity, test_github_checks_queued_rerun_keeps_the_glyph_pending, test_github_checks_preserve_input_order.
  • test_github_rollup_read_follows_pages_and_refuses_a_moved_head: pages are joined before the collapse; a page naming another head discards the read.
  • Fetch and chip tests now feed the GraphQL reply shape; test_fetch_github_reads_rollup_outside_the_core_field_set and test_fetch_github_checks_uses_one_call_without_rewriting_cache pin the new command and its output cap.

Run locally: test/test_source_providers.py, test/test_source_providers_refactor_facade.py, test/test_source_providers_refactor_handlers.py, test/test_source_provider_plugin.py, test/test_public_repo_chip_status.py, and test/test_slack_gateway.py::TestInitAutonudge::test_disabled_gateway_import_does_not_load_monitor_runtime (all pass). mypy, flake8 and black clean on the touched source files.

Manual verification

N/A — unit coverage sufficient: the change is a provider read and a pure projection over its rows, both exercised through the provider runner seam with the exact GraphQL reply shape.

Related Issues

Fixes #16519

Supersedes #17350

Rebased onto current main: SAST (Semgrep) baselines at the PR's merge-base, so on the older base it reported two findings in test/conftest.py and test/test_testing_determinism_helpers.py that came from main (#17265, #17434), not from this diff; the rebase moves the baseline past them.

Related: #11832, #16510 (same defect class, fixed in the prepare-pr status tool).

Pattern harvest

Rule candidate: review-prompt
Pattern: collapsing CI check rows by display label and timestamp instead of workflow definition id, run event and run id (third copy of the same defect after #11832 and #16510).

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@chenmingwei23
chenmingwei23 requested a review from a team as a code owner October 7, 2026 00:30
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

One thing to confirm before merge: when a newer run of the same workflow and event passes, an older FAILURE now stays shown and turns the glyph red. That reverses the deleted test_github_checks_keep_only_the_latest_run_per_check.

What this change ships

Inventory (8 items) — 8 justified

Intent: stop the dashboard's CI glyph, sidebar chip and checks panel from reading green while a live check on the pull request is failing. This is a FIX.

  1. Two check rows from one workflow run (the SUCCESS/FAILURE pair measured on fix: cron_add inherits the caller Slack thread automatically #11783) both stay shown, so the glyph reads failed. — justified
  2. Two workflow files that share one name:, or one workflow running on both push and pull_request, are no longer merged into one row. — justified
  3. When a newer run of the same workflow and event passes, an older run's FAILURE still shows and keeps the glyph red. — justified
  4. A row left behind by an older run that a concurrency group cancelled is still hidden. — justified
  5. Checks are now read through paginated gh api graphql (up to 10 pages) instead of gh pr view. A board past that cap degrades the checks section. — justified
  6. If the head moves between pages, or a page describes another commit, the checks section degrades instead of showing a mixed board. — justified
  7. Status contexts take startedAt from createdAt. — justified
  8. _github_check_identity and _github_check_rank are removed from the facade's export list and from the pinned surface. — justified

[FIRST-PRINCIPLES-REVIEWED] b363a83

Description read

[DESCRIPTION-READ] 5235324f140193c934940345144d35d0fe16fb6c9338ec80341ff798555943f3

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The glyph now stays red after a failed run is replaced by a passing re-dispatch on the same head. The old test pinned the opposite on purpose, and the PR reverses it without saying so.

Watch

  • Red over a re-dispatch that passed. The deleted test_github_checks_keep_only_the_latest_run_per_check turned CANCELLED, FAILURE, SUCCESS into SUCCESS alone. The new test_github_checks_keep_a_failure_from_an_older_run pins the opposite: ["FAILURE", "SUCCESS"] rolls up to failed. Here is how that plays out. A pull_request workflow re-runs on edited with the same event and definition but a newer run id. _mark_superseded_rows flags an older row only when that run was CANCELLED. So after an author fixes a failing description check by editing the PR, the glyph and chip stay red until the next push. GitHub's merge box shows green in that case. The description says compatibility moves "only toward keeping more", but it never names this regression. That leaves the trade from the Goal ("never read green") to a false red undecided.
    Clears when: the description names this behaviour change and a maintainer accepts it, or the dashboard rule also drops an older non-cancelled run when a newer COMPLETED run with the same (definition, event, name) exists.

Suggestions

  • The dashboard now calls the monitor's private _rollup_page and _mark_superseded_rows through a lazy import. Move them into a small public module that both sides import, so a monitor-only edit can't silently change what the dashboard shows.

[DESIGN-REVIEWED] b363a83

Description read

[DESCRIPTION-READ] aad8720d7f959a07375759ea926c18ca536a62238092aeb85f973cb57ad632df

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] b363a83

Verdict parsed from the review's SHA-scoped output markers for commit b363a8316db3bc68dd2f0cd3b7da1f80e34073c8.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b363a8316db3bc68dd2f0cd3b7da1f80e34073c8: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b363a83

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b363a8316db3bc68dd2f0cd3b7da1f80e34073c8: <one-sentence reason>

kirocrew-agent added 2 commits October 7, 2026 01:48
…artedAt

The dashboard's check projection (_github_checks) kept "the latest run per
check" keyed on (kind, workflow label, name) and ordered by the row's
startedAt, overwriting the loser. That loses a live row when two check runs
belong to one workflow run, when two workflow files share one name:, or when
one file runs on both push and pull_request for a commit. When the dropped row
is the failing one, the CI glyph and sidebar chip read green over a failure.

The rollup read now goes through gh api graphql so each check run carries its
workflow-run identity, and the collapse calls the structured monitor's own
rule (_mark_superseded_rows): identity is the workflow definition id, the run
event and the check name; newest is the greatest run id; a row is dropped only
when its older run concluded CANCELLED and the row itself is
COMPLETED+CANCELLED. Rows of one run, rows without a run identity, and status
contexts are kept. Pages are joined before the collapse, every page must name
the same head, and a malformed page or a board past the page cap degrades the
checks section instead of rendering part of it.

The superseded _github_check_identity and _github_check_rank helpers are
removed.

Fixes #16519
The paged rollup read now parses each GraphQL page with the monitor's
_rollup_page instead of a second hand-written walk (the query gains
totalCount, which that parser requires), so the dashboard and the monitor
share one parser, one row flattener and one collapse rule.

The monitor helpers are imported at call time: loading the gateway with
monitors disabled must not load the monitor's GitHub provider runtime
(test_disabled_gateway_import_does_not_load_monitor_runtime).
@chenmingwei23
chenmingwei23 force-pushed the fix/16519-dashboard-check-collapse branch from 39c7bab to b363a83 Compare October 7, 2026 01:51
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 7, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

What it brings: the dashboard's CI glyph, sidebar chip and checks panel can no longer turn green while a live failing check exists. Rows are now collapsed by workflow run identity, so a same-run FAILURE (the #11783 shape) or a same-named check from a second workflow file can't be overwritten by a later SUCCESS.

Reviewed at head b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 (merge-base 5b2aee24e457). CI on that head: 85 success, 13 skipped, 0 failed, 0 pending. Auto-merge is off and there are no human reviews yet. Sensitive Change Review is green.

Focus question: can a red check be hidden behind a green one?

No. _github_checks (src/kiro_crew/dashboard/source_providers/github.py:46-74) drops only the rows that monitoring/github_pull_request.py:_mark_superseded_rows flags (:1155-1208). A row is flagged only when all of these hold at once:

  • It has a full identity key (workflowDefinitionId, workflowRunEvent, name) (_superseded_key, :1077). Status contexts, app check runs without a workflow run, and rows with a missing id or event are exempt and always kept.
  • Its run id is below the greatest run id for that key.
  • The row itself is COMPLETED+CANCELLED, and its run's suite conclusion is CANCELLED (_run_was_cancelled).

So a dropped row is always a cancellation inside a cancelled run. Its same-key successor row (the winner) is never flagged, so it is always shown. FAILURE, TIMED_OUT, ERROR and ACTION_REQUIRED rows are never dropped. Two distinct checks can't share a key: name, workflow definition id and event are all part of it. Display labels play no part in the key any more.

The read path is fail-closed. Pages are joined before the collapse, so a cancelled row's successor on a later page is seen. Each of these raises SourceProviderError, which degrades the section rather than rendering part of the board:

  • a GraphQL errors member or a malformed page
  • the head moving between pages, or the rollup commit differing from headRefOid
  • more than 10 pages

Blocking

None.

Upgrade impact

  • Intended behaviour change, not stated in the PR body. The glyph can now stay red after a passing re-dispatch on the same head. I checked this against live data. statusCheckRollup on open PRs here carries both runs of one (definition, event, name), both COMPLETED and neither cancelled. Example: head 021e482d… has PR Hygiene from runs 37563799630 and 37565606180, both pull_request. With an older FAILURE and a newer SUCCESS, main's label/startedAt collapse showed green. This PR keeps both rows and rolls up to failed until the next push. The common flow is: a description gate fails, the author edits the body, the edited run passes, and the glyph stays red. This is fail-closed (over-reports red, never hides it). It is also deliberate:

    • test_github_checks_keep_a_failure_from_an_older_run pins it.
    • It matches the structured monitor's rule, already on main.
    • The reverse rule would hide a real failure behind a trivial later run. For example, a labeled run whose jobs succeed vacuously would cover the synchronize run's real FAILURE, since both share the pull_request event. That is exactly the red-under-green this PR exists to prevent.

    So it doesn't block. But the "Backwards compatibility: Compatible … only toward keeping more" line should name this visible effect. That is also the Design Review's "clears when" condition.

  • Payload shape (checks rows) is unchanged. No config or state files are involved. The two removed helpers have no callers outside the facade list. I grepped src/ and test/ and found only github.py and handlers/source_providers.py:_OWNED_NAMES.

  • Same token permission surface as before. gh pr view --json statusCheckRollup already walks checkSuite.workflowRun.workflow to fill workflowName, so no new scope is needed. A PR with more than 100 rollup rows now costs one GraphQL call per 100 rows, up to 10 calls, instead of one call.

Non-blocking follow-ups

  1. Amend the PR body's Backwards compatibility section to state the sticky-red-after-re-dispatch effect above, and have a maintainer accept it.

  2. Three new guards in _github_rollup_read have no test. Removing any of them leaves every test green. Each one is a fail-open hazard if lost, because it would render a partial board that could be missing a red row:

    • the data.get("errors") refusal
    • the revision != head refusal
    • the past-the-page-cap raise

    Add one test for each.

  3. The createdAt fallback for status contexts in _github_check (github.py:37) also has no test that reddens.

  4. The spec is slightly stale. docs/system-specs/modules/learn-cron-dashboard.md:3524 still calls the checks endpoint "a lightweight one-call GitHub … path". It is now one call per 100 rollup rows (max 10). The PR changes no docs.

  5. Design Review suggestion (agree, non-gating): the dashboard now lazy-imports the monitor's private _rollup_page and _mark_superseded_rows. Moving them into a small shared public module would stop a monitor-only edit from silently changing the dashboard.

Bot dispositions

  • Design Review (CONCERNS): checked and confirmed on live rollup data. It is the intended fail-closed trade-off described above, so it doesn't gate. It is recorded as follow-up 1.
  • First Principles: PASS. Opus 5.5 and GPT 6.1: no blocking findings.

What I verified

  • I read the full source diff and the monitor helpers at the head SHA. I traced every drop condition (above). I checked that no website code changed, so no vitest file is related and none was run.
  • Related tests at head: test/test_source_providers.py, test/test_source_providers_refactor_facade.py, test/test_public_repo_chip_status.py. Result: 760 passed. They ran in an isolated KIROCREW_HOME/HOME under the scratch dir.
  • Live GitHub probes (read-only GraphQL and REST):
    • On three open PR heads, the rollup carries multiple completed runs per (definition, event, name). This confirms the upgrade-impact item.
    • On sampled runs with run_attempt > 1, the rollup holds only latest-attempt rows. So a "re-run failed jobs" attempt doesn't leave a stale FAILURE beside its SUCCESS.
  • Mutations to github.py, each restored afterwards, with the worktree confirmed clean:
    • M1, _github_checks keeps flagged rows. 5 tests go red, including test_github_checks_drop_a_row_its_cancelled_run_left_behind, test_fetch_github_checks_collapses_superseded_runs and test_github_check_status_collapses_superseded_runs.
    • M2, the head-moved guard is removed. test_github_rollup_read_follows_pages_and_refuses_a_moved_head goes red.
    • M3 (revision≠head guard removed), M4 (GraphQL errors check removed), M5 (past-cap returns a partial board): no test goes red. See follow-up 2.
    • M6, the createdAt fallback is removed: no test goes red. See follow-up 3.
    • M7, _github_checks is replaced with main's label collapse. 11 tests go red, including test_github_checks_keep_both_rows_of_one_run[True/False], test_fetch_github_checks_keeps_both_rows_of_one_run, test_github_checks_keep_a_failure_from_an_older_run, test_github_checks_do_not_group_two_workflow_files_sharing_one_name and test_github_checks_do_not_group_runs_of_different_events.
  • Not verified: the full suite (not run, per the brief) and any manual dashboard rendering.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving (crew-pr-reviewer). The collapse calls the monitor's own _mark_superseded_rows, so the glyph and the monitor cannot disagree. A row is dropped only when its older run concluded CANCELLED and the row is COMPLETED+CANCELLED. Rows that are failures, from different workflow definitions or events, from one run, or without a run identity are all kept. Tests pin each of those. A page with a mismatched head, or a board past the page cap, makes the checks section degrade instead of rendering a partial board. CI is green and the base is main.

@iamwhatever
iamwhatever enabled auto-merge (squash) October 7, 2026 06:46
@iamwhatever
iamwhatever merged commit c20a7f8 into main Oct 7, 2026
103 checks passed
@iamwhatever
iamwhatever deleted the fix/16519-dashboard-check-collapse branch October 7, 2026 06:46
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dashboard check projection: _github_checks collapses live check rows by workflow label and startedAt

3 participants