Repository navigation
fix(dashboard): collapse check rows by workflow run, not label - #17570
Conversation
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS One thing to confirm before merge: when a newer run of the same workflow and event passes, an older FAILURE now stays shown and turns the glyph red. That reverses the deleted What this change shipsInventory (8 items) — 8 justifiedIntent: stop the dashboard's CI glyph, sidebar chip and checks panel from reading green while a live check on the pull request is failing. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] b363a83 Description read[DESCRIPTION-READ] 5235324f140193c934940345144d35d0fe16fb6c9338ec80341ff798555943f3 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The glyph now stays red after a failed run is replaced by a passing re-dispatch on the same head. The old test pinned the opposite on purpose, and the PR reverses it without saying so. Watch
Suggestions
[DESIGN-REVIEWED] b363a83 Description read[DESCRIPTION-READ] aad8720d7f959a07375759ea926c18ca536a62238092aeb85f973cb57ad632df That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
…artedAt The dashboard's check projection (_github_checks) kept "the latest run per check" keyed on (kind, workflow label, name) and ordered by the row's startedAt, overwriting the loser. That loses a live row when two check runs belong to one workflow run, when two workflow files share one name:, or when one file runs on both push and pull_request for a commit. When the dropped row is the failing one, the CI glyph and sidebar chip read green over a failure. The rollup read now goes through gh api graphql so each check run carries its workflow-run identity, and the collapse calls the structured monitor's own rule (_mark_superseded_rows): identity is the workflow definition id, the run event and the check name; newest is the greatest run id; a row is dropped only when its older run concluded CANCELLED and the row itself is COMPLETED+CANCELLED. Rows of one run, rows without a run identity, and status contexts are kept. Pages are joined before the collapse, every page must name the same head, and a malformed page or a board past the page cap degrades the checks section instead of rendering part of it. The superseded _github_check_identity and _github_check_rank helpers are removed. Fixes #16519
The paged rollup read now parses each GraphQL page with the monitor's _rollup_page instead of a second hand-written walk (the query gains totalCount, which that parser requires), so the dashboard and the monitor share one parser, one row flattener and one collapse rule. The monitor helpers are imported at call time: loading the gateway with monitors disabled must not load the monitor's GitHub provider runtime (test_disabled_gateway_import_does_not_load_monitor_runtime).
39c7bab to
b363a83
Compare
buluoray
left a comment
There was a problem hiding this comment.
Verdict: APPROVE
What it brings: the dashboard's CI glyph, sidebar chip and checks panel can no longer turn green while a live failing check exists. Rows are now collapsed by workflow run identity, so a same-run FAILURE (the #11783 shape) or a same-named check from a second workflow file can't be overwritten by a later SUCCESS.
Reviewed at head b363a8316db3bc68dd2f0cd3b7da1f80e34073c8 (merge-base 5b2aee24e457). CI on that head: 85 success, 13 skipped, 0 failed, 0 pending. Auto-merge is off and there are no human reviews yet. Sensitive Change Review is green.
Focus question: can a red check be hidden behind a green one?
No. _github_checks (src/kiro_crew/dashboard/source_providers/github.py:46-74) drops only the rows that monitoring/github_pull_request.py:_mark_superseded_rows flags (:1155-1208). A row is flagged only when all of these hold at once:
- It has a full identity key
(workflowDefinitionId, workflowRunEvent, name)(_superseded_key,:1077). Status contexts, app check runs without a workflow run, and rows with a missing id or event are exempt and always kept. - Its run id is below the greatest run id for that key.
- The row itself is
COMPLETED+CANCELLED, and its run's suite conclusion isCANCELLED(_run_was_cancelled).
So a dropped row is always a cancellation inside a cancelled run. Its same-key successor row (the winner) is never flagged, so it is always shown. FAILURE, TIMED_OUT, ERROR and ACTION_REQUIRED rows are never dropped. Two distinct checks can't share a key: name, workflow definition id and event are all part of it. Display labels play no part in the key any more.
The read path is fail-closed. Pages are joined before the collapse, so a cancelled row's successor on a later page is seen. Each of these raises SourceProviderError, which degrades the section rather than rendering part of the board:
- a GraphQL
errorsmember or a malformed page - the head moving between pages, or the rollup commit differing from
headRefOid - more than 10 pages
Blocking
None.
Upgrade impact
-
Intended behaviour change, not stated in the PR body. The glyph can now stay red after a passing re-dispatch on the same head. I checked this against live data.
statusCheckRollupon open PRs here carries both runs of one(definition, event, name), both COMPLETED and neither cancelled. Example: head021e482d…hasPR Hygienefrom runs 37563799630 and 37565606180, bothpull_request. With an older FAILURE and a newer SUCCESS, main's label/startedAtcollapse showed green. This PR keeps both rows and rolls up tofaileduntil the next push. The common flow is: a description gate fails, the author edits the body, theeditedrun passes, and the glyph stays red. This is fail-closed (over-reports red, never hides it). It is also deliberate:test_github_checks_keep_a_failure_from_an_older_runpins it.- It matches the structured monitor's rule, already on main.
- The reverse rule would hide a real failure behind a trivial later run. For example, a
labeledrun whose jobs succeed vacuously would cover thesynchronizerun's real FAILURE, since both share thepull_requestevent. That is exactly the red-under-green this PR exists to prevent.
So it doesn't block. But the "Backwards compatibility: Compatible … only toward keeping more" line should name this visible effect. That is also the Design Review's "clears when" condition.
-
Payload shape (
checksrows) is unchanged. No config or state files are involved. The two removed helpers have no callers outside the facade list. I greppedsrc/andtest/and found onlygithub.pyandhandlers/source_providers.py:_OWNED_NAMES. -
Same token permission surface as before.
gh pr view --json statusCheckRollupalready walkscheckSuite.workflowRun.workflowto fillworkflowName, so no new scope is needed. A PR with more than 100 rollup rows now costs one GraphQL call per 100 rows, up to 10 calls, instead of one call.
Non-blocking follow-ups
-
Amend the PR body's Backwards compatibility section to state the sticky-red-after-re-dispatch effect above, and have a maintainer accept it.
-
Three new guards in
_github_rollup_readhave no test. Removing any of them leaves every test green. Each one is a fail-open hazard if lost, because it would render a partial board that could be missing a red row:- the
data.get("errors")refusal - the
revision != headrefusal - the past-the-page-cap
raise
Add one test for each.
- the
-
The
createdAtfallback for status contexts in_github_check(github.py:37) also has no test that reddens. -
The spec is slightly stale.
docs/system-specs/modules/learn-cron-dashboard.md:3524still calls the checks endpoint "a lightweight one-call GitHub … path". It is now one call per 100 rollup rows (max 10). The PR changes no docs. -
Design Review suggestion (agree, non-gating): the dashboard now lazy-imports the monitor's private
_rollup_pageand_mark_superseded_rows. Moving them into a small shared public module would stop a monitor-only edit from silently changing the dashboard.
Bot dispositions
- Design Review (CONCERNS): checked and confirmed on live rollup data. It is the intended fail-closed trade-off described above, so it doesn't gate. It is recorded as follow-up 1.
- First Principles: PASS. Opus 5.5 and GPT 6.1: no blocking findings.
What I verified
- I read the full source diff and the monitor helpers at the head SHA. I traced every drop condition (above). I checked that no website code changed, so no vitest file is related and none was run.
- Related tests at head:
test/test_source_providers.py,test/test_source_providers_refactor_facade.py,test/test_public_repo_chip_status.py. Result: 760 passed. They ran in an isolatedKIROCREW_HOME/HOMEunder the scratch dir. - Live GitHub probes (read-only GraphQL and REST):
- On three open PR heads, the rollup carries multiple completed runs per
(definition, event, name). This confirms the upgrade-impact item. - On sampled runs with
run_attempt > 1, the rollup holds only latest-attempt rows. So a "re-run failed jobs" attempt doesn't leave a stale FAILURE beside its SUCCESS.
- On three open PR heads, the rollup carries multiple completed runs per
- Mutations to
github.py, each restored afterwards, with the worktree confirmed clean:- M1,
_github_checkskeeps flagged rows. 5 tests go red, includingtest_github_checks_drop_a_row_its_cancelled_run_left_behind,test_fetch_github_checks_collapses_superseded_runsandtest_github_check_status_collapses_superseded_runs. - M2, the head-moved guard is removed.
test_github_rollup_read_follows_pages_and_refuses_a_moved_headgoes red. - M3 (revision≠head guard removed), M4 (GraphQL
errorscheck removed), M5 (past-cap returns a partial board): no test goes red. See follow-up 2. - M6, the
createdAtfallback is removed: no test goes red. See follow-up 3. - M7,
_github_checksis replaced with main's label collapse. 11 tests go red, includingtest_github_checks_keep_both_rows_of_one_run[True/False],test_fetch_github_checks_keeps_both_rows_of_one_run,test_github_checks_keep_a_failure_from_an_older_run,test_github_checks_do_not_group_two_workflow_files_sharing_one_nameandtest_github_checks_do_not_group_runs_of_different_events.
- M1,
- Not verified: the full suite (not run, per the brief) and any manual dashboard rendering.
iamwhatever
left a comment
There was a problem hiding this comment.
Approving (crew-pr-reviewer). The collapse calls the monitor's own _mark_superseded_rows, so the glyph and the monitor cannot disagree. A row is dropped only when its older run concluded CANCELLED and the row is COMPLETED+CANCELLED. Rows that are failures, from different workflow definitions or events, from one run, or without a run identity are all kept. Tests pin each of those. A page with a mismatched head, or a board past the page cap, makes the checks section degrade instead of rendering a partial board. CI is green and the base is main.
Problem / Motivation
Goal: The dashboard's CI glyph, sidebar chip and checks panel never read green while a live check on the pull request is failing.
_github_checks(the projection behind the dashboard's checks panel, CI glyph and sidebar chip) kept "the latest run per check", keyed on(kind, workflow label, name)and ordered by the row'sstartedAt; the later row overwrote the earlier one. That loses a live row in three shapes: two check runs of one workflow run (measured on #11783:Fork PR Description / Fork PR Description, SUCCESS and FAILURE two seconds apart in one run), two workflow files sharing onename:, and one workflow file running on bothpushandpull_requestfor the same commit. When the overwritten row is the failing one, the glyph reads green over a live failure.Why it matters
The CI glyph and the sidebar chip are what a person glances at before opening a pull request. A green glyph over a live failure sends them toward merge with the wrong picture, and nothing in the old mechanism prevented the arrival order that produces it.
Not a goal
_rollup_ci) or the GitLab projection.What changed (motivation → approach → change)
gh pr view --json statusCheckRolluppayload carries no workflow-run identity, so the projection could only key on the display label and order by job start time, which cannot tell "a newer run replaced this one" from "this is a different live row".monitoring/github_pull_request.py:_mark_superseded_rows,prepare-pr/scripts/pr_status.py:collapse_superseded). Identity is(workflow definition id, run event, check name); newest is the greatest run id; a row is dropped only when its older run concludedCANCELLEDand the row itself isCOMPLETED+CANCELLED. Rows of one run, rows without a run identity, and status contexts are kept. The dashboard calls the monitor's_rollup_page(page parse and row flattening) and_mark_superseded_rowsdirectly rather than carrying a third copy; they are imported at call time so loading the gateway with monitors off does not load the monitor's provider runtime._github_rollup_readnow reads the head commit'sstatusCheckRollupthroughgh api graphql(same sandboxed provider runner, same_CHECKS_OUTPUT_BYTEScap, still its own call so a Checks-blind token costs only thecheckssection). It pages 100 rows at a time up to 10 pages, parses each page with_rollup_page, joins the pages before collapsing, requires every page to name the same head on both the pull request and the rollup's commit, and raises (degrading the section to partial) on a malformed page or a board past the cap rather than rendering part of it._github_checksnow projects the flat rows through_mark_superseded_rowsand drops only the flagged ones, in input order._github_checkfalls back to a status context'screatedAtforstartedAt(the GraphQL shape names it differently fromgh pr view)._github_check_identityand_github_check_rankare removed; nothing else called them.chip_refresh.py) and the polled checks endpoint (_fetch_github_checks) consume_github_rollup_readand inherit the fix unchanged.Backwards compatibility
Compatible: the payload shape the dashboard and plugins receive (
checksrows withname,workflow,status,conclusion,bucket,url,startedAt,completedAt) is unchanged; only which rows survive the collapse changes, and only toward keeping more.Reader: src/kiro_crew/dashboard/handlers/source_providers.py:_OWNED_NAMES["github"] -- entry: crew page -- the two removed helper names are dropped from the re-export list; no caller on main reads them (grep of src/ and test/ finds only github.py and the facade list).
Reader: test/test_source_providers_refactor_facade.py:_HISTORICAL_SURFACE -- entry: release -- names removed from the pinned surface alongside the export list;
test_the_handler_and_its_owners_split_the_surface_exactlypasses.Reader: src/kiro_crew/dashboard/source_providers/chip_refresh.py:_fetch_check_status -- entry: crew page -- still receives
(checks, head)from_github_rollup_read;test_github_check_status_collapses_superseded_runs.Reader: src/kiro_crew/dashboard/source_providers/cache.py:_fetch_pull_request_checks_uncached -- entry: crew page -- still receives the checks list from
_fetch_github_checks;test_fetch_github_checks_uses_one_call_without_rewriting_cache.Tests
In
test/test_source_providers.py:test_github_checks_drop_a_row_its_cancelled_run_left_behind: a concurrency-cancelled older run is dropped and the glyph reads passed.test_github_checks_keep_a_failure_from_an_older_run: an older run that FAILED is kept (a newer run id alone proves no replacement).test_github_checks_keep_both_rows_of_one_run(both arrival orders) andtest_fetch_github_checks_keeps_both_rows_of_one_run(end to end through the GraphQL shape): the fix: cron_add inherits the caller Slack thread automatically #11783 same-run SUCCESS/FAILURE pair keeps both rows and rolls up failed.test_github_checks_do_not_group_two_workflow_files_sharing_one_name,test_github_checks_do_not_group_runs_of_different_events: identity is definition id + event, not the label.test_github_checks_keep_a_cancelled_row_of_a_live_run,test_github_checks_keep_a_cancelled_newest_run,test_github_checks_keep_rows_without_a_run_identity,test_github_checks_queued_rerun_keeps_the_glyph_pending,test_github_checks_preserve_input_order.test_github_rollup_read_follows_pages_and_refuses_a_moved_head: pages are joined before the collapse; a page naming another head discards the read.test_fetch_github_reads_rollup_outside_the_core_field_setandtest_fetch_github_checks_uses_one_call_without_rewriting_cachepin the new command and its output cap.Run locally:
test/test_source_providers.py,test/test_source_providers_refactor_facade.py,test/test_source_providers_refactor_handlers.py,test/test_source_provider_plugin.py,test/test_public_repo_chip_status.py, andtest/test_slack_gateway.py::TestInitAutonudge::test_disabled_gateway_import_does_not_load_monitor_runtime(all pass). mypy, flake8 and black clean on the touched source files.Manual verification
N/A — unit coverage sufficient: the change is a provider read and a pure projection over its rows, both exercised through the provider runner seam with the exact GraphQL reply shape.
Related Issues
Fixes #16519
Supersedes #17350
Rebased onto current main: SAST (Semgrep) baselines at the PR's merge-base, so on the older base it reported two findings in
test/conftest.pyandtest/test_testing_determinism_helpers.pythat came from main (#17265, #17434), not from this diff; the rebase moves the baseline past them.Related: #11832, #16510 (same defect class, fixed in the prepare-pr status tool).
Pattern harvest
Rule candidate: review-prompt
Pattern: collapsing CI check rows by display label and timestamp instead of workflow definition id, run event and run id (third copy of the same defect after #11832 and #16510).
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.