Skip to content

fix(dashboard): stop counting day-old failed workflows as blocked - #18291

Merged
KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
Premshay:fix/command-center-stale-failed-workflows
Nov 5, 2026
Merged

KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
Premshay:fix/command-center-stale-failed-workflows

Conversation

@Premshay

@Premshay Premshay commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Command Center's Blocked tile stops counting failed workflow runs that ended more than a day ago.

buildCommandCenter maps status === 'failed' straight to blocked. Workflow runs are kept on disk, so a session's Blocked count includes failures from weeks earlier and never returns to zero. Seen: three runs that failed on 2026-09-17 still counted as Blocked on 2026-10-09.

Why it matters

Blocked is meant to say "something here needs you now". Once old failures accumulate it is always non-zero, so users learn to ignore it and miss a genuinely new failure.

Not a goal

What changed (motivation → approach → change)

  • Symptom → cause: the panel cannot tell an old failure from a fresh one, because the run list carries no end time.
  • Backend (src/kiro_crew/workflows/registry.py): RunHandle.snapshot adds ended_at: the ts of the run's terminal event (run_finished / run_failed / run_cancelled). An ended run with no terminal event (interrupted, restored after a restart) falls back to its last event's ts. It is null while the run is active. The field is typed in website/src/types/index.ts.
  • Panel (website/src/pages/chat/command-center/model.ts):
    • A failed run counts as blocked for FAILED_WORKFLOW_BLOCKED_MS (24 h) after ended_at, then rests as stopped, the existing resting state a cancelled run uses. A missing or unparseable ended_at keeps blocked.
    • buildCommandCenter takes an optional now so the rule is testable.
  • Why a time window rather than a dismiss:
    • It needs no new state or route, and clears on its own.
    • A dismiss can still be added on top.

Backwards compatibility

Compatible: ended_at is a new optional field. A client that ignores it is unaffected, and a run without it (from an older gateway) keeps the previous blocked behaviour.

Reader: website/src/pages/chat/command-center/model.ts:buildCommandCenter -- app -- a failed run past the window now reads stopped (resting) instead of blocked; commandCenterModel.test.ts covers fresh, old and unknown-end-time runs.
Reader: website/src/pages/chat/command-center/SessionDashboardsPage.tsx:blocked -- app -- reads the node state above, so the same window applies to the session list badge.

Tests

  • test/test_workflows_registry.py::test_list_reports_when_a_run_ended: the list row's ended_at equals the terminal event's ts. It fails without the change.

  • website/src/test/commandCenterModel.test.ts, "counts a failed workflow as blocked only within the window after it ended":

    • a fresh failure is blocked;
    • a failure past the window is stopped;
    • a failure with no ended_at stays blocked;
    • the Blocked count is 2.

    It fails without the change.

  • Ran: test_workflows_registry.py (11 passed) and commandCenterModel.test.ts (52 passed).

Manual verification

Still required: open a session that has a failed workflow older than 24 hours and check that the dock's Blocked tile no longer counts it. I could not sign a headless browser into the dashboard in my environment, so this has not been done in a browser.

Screenshots / video

Why no screenshot: no component or style changes; only which runs feed the existing Blocked count, covered by the model test.

Related Issues

Closes #18290

Pattern harvest

Rule candidate: review-prompt
Pattern: a status-to-attention mapping over persisted records with no age bound turns an "act now" signal into a permanent count.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@Premshay
Premshay requested review from a team and KiroCrewCaptain as code owners October 9, 2026 00:56
@Premshay
Premshay requested a review from chenmingwei23 October 9, 2026 00:56
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 9, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 25895705271a5d365132d301da1464de1b5d63aa via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the 24-hour window is additive, needs no new state, and a gateway that sends no ended_at keeps the old blocked behaviour.

Notes

  • ended_at is stamped by the gateway's clock, but now comes from the browser's Date.now(). If the two clocks disagree, a failure can drop out of Blocked early or late, by the size of the gap. This is reversible and needs no change here.

[DESIGN-REVIEWED] 2589570

Description read

[DESCRIPTION-READ] b2d9319260161e44da74d3a89fce23dba755ad0c0809842e047d5cc194d55b1a

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ✅ PASS

UX-level review of 25895705271a5d365132d301da1464de1b5d63aa via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check: the diff changes no component, string or style, only which workflow runs count toward the existing Blocked tile, so no screenshot is needed.

Notes

  • After 24 hours, an old failed run shows the same "stopped" label as a run the user cancelled (failedWorkflowState → 'stopped'). The failure is now told apart only by the ErrorNotice that SessionDashboardsPage.tsx:60 renders from n.error. If an older failed run has no error text, it will look exactly like a cancelled run.

[UX-REVIEWED] 2589570

Description read

[DESCRIPTION-READ] 97362a1ae253f9a38d8540e574f1432cc3088dcfec8dbd4eebcc17d2985dbddd

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings (all downgraded on adjudication)

Reviewed 25895705271a5d365132d301da1464de1b5d63aa via the fork AI-review pipeline; updated in place on each push.

Opus 5.5 adjudication downgraded every blocking finding. Downgraded: the remedy each one needs is disproportionate to the harm it prevents. Adjudication downgraded all 1 blocking finding(s): the remedy each one requires is disproportionate to the harm it prevents.

Review details

BLOCKING -- website/src/pages/chat/command-center/model.ts:177 -- Failed workflows do not expire while All Dashboards stays open

export function buildCommandCenter(source: CommandCenterSources, now = Date.now()) {
Trigger: Open All Dashboards with a failed run just under 24 hours old and leave the idle page open across the cutoff. SessionDashboardsPage calls useCommandCenter in fleet mode, where useMemo caches this projection without a clock dependency and work-board polling is disabled.
Outcome: The session retains its Blocked badge and blocked-first priority after expiry, until an unrelated model dependency changes or the page remounts.
Anchor: defect
Fix: Schedule a projection refresh at the next failure-expiry boundary in useCommandCenter, passing reactive time into buildCommandCenter.

[BLOCK-MERGE-DOWNGRADED] 2589570
[GPT-REVIEWED] 2589570

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1: LOW harm. The failure needs three things at once, all confirmed in the code. The page has to run in fleet mode, where work-board polling is off (useCommandCenter.ts:198, enabled: canRead && !fleet). The useMemo dependency list has no clock (useCommandCenter.ts:213). And nothing can change while the page sits idle past the run's 24 h mark, because the sources only refresh from frames, reconnects or a focus re-read after an error (useCommandCenter.ts:84-86). When it happens the result is visible and goes away on its own: the stale Blocked badge clears on the next slot, workflow or approval frame, or on a remount. It also never does worse than the base, which showed every failure as blocked forever (model.ts:220). Shipping the fix means adding a timer at the expiry boundary and a reactive clock to the hook. Every future reader would carry that, and it would serve only the idle edge of a cosmetic window, so the fix costs more than the harm it prevents.

[ADJUDICATION] 25895705271a5d365132d301da1464de1b5d63aa total=1 uphold=0 downgrade=1
DOWNGRADE F1 website/src/pages/chat/command-center/model.ts:177 reason=disproportionate-remedy
[GPT-ADJUDICATED] 25895705271a5d365132d301da1464de1b5d63aa

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 25895705271a5d365132d301da1464de1b5d63aa via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: the fix is at the mechanism (the status-to-attention mapping), it is fed by the one field the list view lacked, and that field has a real consumer.

Notes

  • Item 3 — inherited: the 24 h value has nothing behind it beyond "FAILED_WORKFLOW_BLOCKED_MS = 24 * 60 * 60 * 1000". The issue (Command Center: a failed workflow counts as Blocked forever #18290) gives a three-week-old failure, which any window would clear. Subtraction: none smaller. A window still needs a value, and this one is reversible.
  • Item 4 — sibling left unfixed: model.ts:241 maps a subagent's a.status === 'error' to 'blocked' with no age bound. That is 1 sibling found by grepping blocked in model.ts, and ## Not a goal excludes it on purpose ("Changing how subagents … are classified"). Deferred, not demanded.

What this change ships

Inventory (6 items) — 4 justified

Intent: make the Blocked tile mean "a failure needs you now" again, not "a failure ever happened". FIX.

  1. Blocked tile no longer counts failed workflow runs that ended more than 24 h ago — justified
  2. Workflow list rows now carry ended_at (terminal event ts, last event as fallback, null while active) — justified
  3. A 24 h window constant, FAILED_WORKFLOW_BLOCKED_MS, exported — inherited: the number has no stated source
  4. An expired failure shows as stopped in the session list badge too — justified
  5. A failure with a missing, invalid or future end time stays blocked — justified
  6. buildCommandCenter gains an optional now parameter — rides along: only tests pass it

[FIRST-PRINCIPLES-REVIEWED] 2589570

Description read

[DESCRIPTION-READ] c311caeea26164ee1c22ec206cfbff5ef8ec090d97ef22fb90525f5424236e87

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 25895705271a5d365132d301da1464de1b5d63aa via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 2589570

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 9, 2026
@dodgeblaster
dodgeblaster force-pushed the fix/command-center-stale-failed-workflows branch from 2a4b3bc to 4cb08ba Compare October 9, 2026 19:50
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 9, 2026
@Premshay

Copy link
Copy Markdown
Contributor Author

Automated verification report — kirocrew-steward.

The workflow repair remains unpublished while a required lint gate is red on main.

isort --check-only src/kiro_crew test fails on test/test_crew_log_types.py in the local PR worktree based on main 24aab85bfbcadc601af45620d674f08cf06cbfac. Checking that file directly from the same clean-main commit with the pinned isort environment and repository settings reproduces exit 1:

ERROR: Imports are incorrectly sorted and/or formatted.

The new from kiro_crew.crew_log import lease import must be folded into the existing grouped import. This file is absent from the PR's diff; it was changed by main's #18984.

No unrelated source change, gate override, or push was made. Next: rebase onto a main commit with the import ordering repaired, then rerun the unchanged lint gate and remaining prepare-pr checks.

@Premshay

Premshay commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author
  • Freshness versus acknowledgment (span=961a3a657840)

needs-a-decision: Maintainers, accept #18290's fixed24-hour ended-at window as operational freshness for Blocked, including unseen failures, or require explicit acknowledgment/dismissal semantics? The issue proposes the window and is tier:T1 but contains no human acceptance of that policy. Run history remains available; expiration does not prove the user saw the failure. The local repair preserves failed versus cancelled classification and does not change the window.

AI-authored by kirocrew-steward (Codex).

@Premshay

Premshay commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author
  • Unseen failures still need a policy ruling (span=2daecd440ab9)

needs-a-decision: Maintainers, should Blocked mean a failure ended within24hours, or a failure the user has not acknowledged? This head uses ended_at only; it records no seen/dismissed state. We will not represent expiration as acknowledgment. Accept the issue18290 proposal explicitly or request a scoped policy change; no clearance is claimed.

AI-authored by kirocrew-steward (Codex).

@Premshay
Premshay force-pushed the fix/command-center-stale-failed-workflows branch from 4cb08ba to 6d25f15 Compare October 15, 2026 19:16
@Premshay

Copy link
Copy Markdown
Contributor Author

Updated to 6d25f15f0, rebased on current main (absorbs #18580's focus-ring fix, which was this PR's earlier CI red). Posted by the kirocrew-steward agent on behalf of @Premshay.

New commit fix(workflows): keep expired workflow failures distinct from cancellation. Gary Jennings's formatting commit is kept as is.

  • Interrupted or restored failures fall back to their last event time.
  • A failure older than the 24-hour window rests as failed, not as cancelled.
  • A missing or invalid end time stays blocked.
  • Specs in workflows.md and artifacts.md are updated. The tile-list test is dropped, because main deleted TileList.tsx in feat: root session side panel shows its own Dynamic Dashboard #18105; commandCenterModel.test.ts covers the same behaviour.
  • The new driver-crash test waits with await_until, so the wait-discipline ratchet is unchanged.

Local evidence:

  • 62 backend and 58 frontend tests pass; tsc is clean.
  • Independent GPT review (gpt-6.1-sol, CI-extracted contract): no findings.
  • Gate failures also fail on clean main on this host: test_dashboard_server_composition_contract, 9 ArtifactDetailPage/SchedulePage frontend tests in files this PR doesn't touch, and the Electron module setup.

The open policy question remains for maintainers: accepting the unchanged 24-hour freshness window for unseen failures.

@Premshay
Premshay force-pushed the fix/command-center-stale-failed-workflows branch from 6d25f15 to 992a8f2 Compare October 16, 2026 08:36
@Premshay

Copy link
Copy Markdown
Contributor Author

Updated to 992a8f2af: one commit on current main. The tree matches the previous head 6d25f15f0 before the rebase. PR Hygiene allows at most 2 commits, so Gary Jennings's black-format commit is folded in and credited as Co-authored-by: Gary Jennings. The rebase also picks up main's fix for test_the_lazy_imports_stay_inside_the_functions_that_need_them (84 == 83). The description now matches the code: an expired failure rests as failed, distinct from a cancelled run's stopped; a missing end time stays blocked; and an interrupted or restored failure falls back to its last event time. Posted by the kirocrew-steward agent on behalf of @Premshay.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 17, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Oct 17, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 992a8f2.

The code does what the description says: ended_at comes from the terminal event (or the last event for an interrupted run), and the panel stops counting a failure as Blocked 24 hours after it ended. Tests cover the boundary, invalid and missing end times, and CI is green.

What blocks approval:

  1. Current-head AI concerns have no current-head reply. UX Review (CONCERNS) Watch 1, unattended failures dropping out of Blocked unseen, and Watch 2, an expired failure showing the Idle badge (SessionDashboardsPage.tsx#L46-L52). First Principles (CONCERNS): the new 'failed' state is read only by RESTING (model.ts#L277). The existing design/ux dispositions target head 4cb08ba, and there is no first-principles disposition.
  2. The product policy is open. Your needs-a-decision asks whether Blocked means "failed within 24h" or "not yet acknowledged". Command Center: a failed workflow counts as Blocked forever #18290 has no maintainer ruling yet.

To clear: a maintainer rules on the 24h window in #18290; then post one disposition per current-head item (ux x2, first-principles x1) with head=992a8f2…. Either give failed a reader that renders it (for example a muted Failed badge) or return stopped as First Principles suggests. The manual browser check from the description is still outstanding.

— crew-pr-reviewer (r110)

@Premshay

Copy link
Copy Markdown
Contributor Author

Agent update — needs-a-decision.

  • Unattended failures leaving Blocked

Outcome: needs-a-decision. Maintainers: should Blocked mean failed within 24 hours of the terminal event, or unacknowledged failure until the user sees it? The latter needs acknowledgement state and changes the proposed policy. Issue #18290 has no maintainer ruling; this PR remains held. No code change or policy acceptance is claimed.

@Premshay

Copy link
Copy Markdown
Contributor Author

Agent update — needs-a-decision.

  • Expired failures display Idle

Outcome: needs-a-decision. Maintainers: once the freshness policy is settled, should an expired failure retain a muted Failed badge, or intentionally use the existing stopped/resting presentation? The current badge falls back to Idle, as this finding states. I recommend a distinct muted Failed badge if the failed state is retained; no UI change is claimed yet.

@Premshay

Copy link
Copy Markdown
Contributor Author

Agent update — needs-a-decision.

  • Failed state has no distinct rendering reader

Outcome: needs-a-decision. Maintainers: choose between a real muted Failed badge consumer and returning the existing stopped state. Keeping an indistinguishable enum variant is not justified by the current reader set. This depends on the open 24-hour policy ruling in #18290; the implementation and manual browser check remain on hold rather than silently changing that policy.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 992a8f2.

The code still does what the description says, and CI is green. Thanks for answering all three current-head items. Each reply is needs-a-decision, so the PR now waits on a maintainer.

To clear this:

  1. A maintainer rules in Command Center: a failed workflow counts as Blocked forever #18290 on whether Blocked means "failed within 24 h of ending" or "failed and not yet acknowledged". The issue has no ruling yet.
  2. After that ruling, settle the 'failed' state. Either give it a reader, such as a muted Failed badge in place of Idle (SessionDashboardsPage.tsx#L46-L52), or return stopped (model.ts#L277).
  3. The Tests section says an expired failure is stopped, but the code returns failed. Fix whichever is wrong.
  4. Do the manual browser check the description lists.

— crew-pr-reviewer (r111)

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 19, 2026
@Premshay
Premshay force-pushed the fix/command-center-stale-failed-workflows branch from 992a8f2 to 9604657 Compare October 26, 2026 21:22
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision labels Oct 26, 2026
@Premshay
Premshay force-pushed the fix/command-center-stale-failed-workflows branch from 9604657 to 46389e8 Compare October 27, 2026 09:50
A failed workflow run counted as Blocked in the Command Center dock for
as long as its record existed, and runs are kept on disk, so the tile
kept counting failures from weeks earlier.

The run snapshot now reports ended_at (the terminal event's ts), and the
panel counts a failed run as blocked for 24 hours after it ended, then as
stopped. A run with no ended_at keeps the previous behaviour.

Closes kirodotdev#18290

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Restored interruptions and driver exceptions can lack a terminal event, so
their snapshot ended_at falls back to the last recorded event, which stays
stable across gateway restarts. Active runs and runs without events still
report None. Past the 24-hour blocked window, a failed workflow now rests
in the command-center model as `stopped`, the same state as a cancelled
run, keeping its error and last log.

Co-authored-by: Gary Jennings <garysjennings@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Agent: kirocrew-steward
@Premshay
Premshay force-pushed the fix/command-center-stale-failed-workflows branch from 46389e8 to 2589570 Compare October 27, 2026 21:17
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 5, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 5, 2026 01:05

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (7 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: command center counts a failed workflow as blocked only for 24 hours after it ended, with tests. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/artifacts.md, docs/system-specs/modules/workflows.md.

@KiroCrewCaptain
KiroCrewCaptain merged commit c00f742 into kirodotdev:main Nov 5, 2026
106 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 5, 2026
@Premshay
Premshay deleted the fix/command-center-stale-failed-workflows branch November 13, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Command Center: a failed workflow counts as Blocked forever

3 participants