Skip to content

fix(dashboard): gate /todos on the live session's harness, not config - #13876

Closed
leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/b18-todos-live-harness-20260925
Closed

leonlaiyc wants to merge 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/b18-todos-live-harness-20260925

Conversation

@leonlaiyc

@leonlaiyc leonlaiyc commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Ensure harness-specific slash-command decisions use the authority that will actually execute the command, so stale session or config hints cannot incorrectly permit or refuse it.

/todos is valid on the Claude harness and invalid on Kiro. The old pre-dispatch gate read config or a registry row before session allocation, but get_or_create is the operation that probes liveness. It can evict a dead row and instantiate the opposite harness from current config.

Why it matters

A stale Claude row could allow /todos, then be replaced by Kiro and deliver an unsupported command. The inverse could refuse a command that the acquired Claude provider supports. The gate and execution must answer from the same authority.

Not a goal

  • Redesigning slash-command dispatch generally.

  • Adding native /todos support to harnesses that do not already support it.

  • Changing unrelated session-allocation behavior.

What changed (motivation → approach → change)

| Command decision | Authority and behavior |

|---|---|

| /todos refusal | A positively live provider's native-todos capability, otherwise the member-aware backend allocation selector |

| /todos execution | The acquired provider confirms native-todos support before dispatch |

| /todos autocomplete | The configured default backend's capability; this endpoint has no session or member identity |

| Slack OPTIONS | Local refusals preserve controls; accepted turns expire them once, including early failures |

A stale provider row can describe a harness that will never execute the command. The dashboard runner checks liveness before its local refusal and confirms capability again on the provider returned by acquisition. Unsupported commands create no session or wait on a busy turn. A first-claim refusal preserves member bookkeeping and the existing replay lease for the next ordinary prompt, including speculative resume.

The agent_sdk/capabilities.py declarations and backend cards expose supports_native_todos through acp_backends.py. dashboard/chat_runner.py uses that answer for dispatch; dashboard/handlers/agents.py uses the configured global backend for suggestions. Other suggestions retain their behavior. Deferred OPTIONS cleanup belongs to the existing protected turn finally, so memory preparation, session claim, and spec-hook failures still expire controls, and cancellation cannot skip session release.

The harness-parity spec records the capability boundary. The dashboard spec names the liveness-qualified, member-aware select_provider_backend(session_key, member_acp_backend, acp_backend) fallback for /compact.

Backwards compatibility

Compatible: Kiro still refuses unsupported /todos; Claude ACP accepts it. Universal blocked commands remain local refusals. The public provider axis remains ACP-only, and a dormant claude_code label cannot select a harness. Backend identity ratchets retain their existing exemptions. /compact keeps the current live-provider or member-aware successor gate.

Tests

Review response (@iamwhatever, CHANGES_REQUESTED on f24e16da2): the refusal now runs before acquisition, and the discriminating case the review asked for is pinned in both directions. Each one fails behaviorally against a config-only pre-acquisition gate (negative control: swapping select_provider_backend(...) for agent.acp_backend turns 6 of 7 rows red; the live-provider row stays green by design):

  • test_todos_member_override_selects_claude_over_default_kiro (alive = dead / unknown / absent): the global default is kiro, the member override selects claude, the preflight must ALLOW /todos, and the acquired claude provider confirms. The old config-only read refused here. The behavioral assertions now come before the selector spy, so a regression fails as a refusal, not only as a missed call.
  • test_todos_member_kiro_refuses_without_acquisition_over_default_claude (new, the mirror): the global default is claude and the member routes to kiro. /todos is refused locally with get_or_create never awaited, no prompt or command streamed, and Slack OPTIONS not expired. The old config-only read admitted this and cold-started kiro-cli only to refuse after the claim.
  • Still covered: the default-kiro refusal without acquisition, the busy live kiro refusal that does not wait on the turn semaphore, stale-row eviction to the acquired successor, and re-projected claims.

On this head (rebased onto a60fda266; the only conflict was the learn-cron-dashboard.md chat bullet, resolved by keeping main's new chat_handlers.py and ws.py text and this PR's /todos paragraph):

  • test_chat_runner_coverage.py, test_manual_compact_gate.py, test_api_slash_commands.py, test_agent_sdk_capabilities.py, test_backend_cards.py, test_harness_parity.py, test_chat_runner_composition_contract.py, test_acp_capability_sets_leaf.py, test_acp_session_handle_capabilities.py: 793 passed locally on Windows. One harness-parity test failed only on this box's cp950 locale; it fails the same way on a main-based tree and passes with PYTHONUTF8=1.
  • The same runner, compact and slash suites passed (383) on a throwaway merge of this head with main 38d5688d9. That is the shape CI tests, and it now includes main's new member-DM identity refusal.
  • black (changed files), flake8, and the harness-parity added-line gate are clean.

Manual verification

N/A — deterministic runner and endpoint tests cover this backend dispatch change, including tracked OPTIONS records and Slack edits. No live harness or Slack message was sent locally.

Related Issues

Follow-up to #13805.

Closes #8939

Pattern harvest

Not generalizable: This PR aligns one harness-specific slash command with the repository's existing capability and allocation authorities; it does not introduce a new reusable defect pattern beyond those existing boundaries.

Checklist

  • One logical commit, rebased onto upstream main a60fda266

  • Deterministic stale/dead and exceptional OPTIONS coverage

  • Capability/identity ratchet preserved

  • Documentation updated

  • No secrets or credentials in the diff

Contribution License Agreement

🤖 Generated with Claude Code

@leonlaiyc
leonlaiyc requested review from a team and smeyffret September 25, 2026 16:42
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 42964745145a9678c4efb5499d6963064aaa3ec7 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4296474

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🟡 CONCERNS

Design-level review of 42964745145a9678c4efb5499d6963064aaa3ec7 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Deferring the OPTIONS expiry for /todos makes that one command's turn lifecycle a three-site state machine inside _run_chat, separate from every other turn's.

Watch

  • Deferred OPTIONS expiry is split across three sites. For /todos only, the patch takes away the single "expire OPTIONS at turn start" boundary. In its place is _deferred_options_expiry, which is armed before the try, cleared on post-acquisition refusal, drained after spec hooks, and drained again in the finally. A post-acquisition refusal also re-arms the replay lease with mark_provider_switch_replay. Consequence: any future edit that adds an early return or an await between acquisition and the drain has to keep all three sites in step. If one is missed, Slack controls go stale or a turn's session-start context is lost, with no failure to show it. The pre-acquisition gate already uses the same select_provider_backend selector that allocation uses. So the post-acquisition branch only covers a race in which the provider is replaced between the probe and the claim.
    Clears when: the race is handled by a refusal that does nothing else (no OPTIONS deferral and no replay re-arm, accepting the rare cost), or the PR records a reason the race needs full first-claim bookkeeping.

Suggestions

  • Drop the is_claude_backend_name import kept under # noqa: F401. A leftover identity import in chat_runner.py invites the negative-identity branch that H6 forbids to return.

[DESIGN-REVIEWED] 4296474

Description read

[DESCRIPTION-READ] 55c470490dcd832342dbaf718187db2e7ef10b90a001fb756e7a95c97a5ff4d1

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of 42964745145a9678c4efb5499d6963064aaa3ec7 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The second /todos check after acquisition (replay-lease re-arm, deferred OPTIONS expiry at three sites) guards a race the member-aware pre-check already closes.

Not justified as shipped

  • Item 4 — oversized: the description says "acquisition still confirms support, because the provider can be replaced between this probe and the claim". The pre-acquisition gate already calls select_provider_backend(...), which is the selector get_or_create uses. The only test that tells the two checks apart (("claude", "claude", "", True)) gets there by mocking an acquisition that returns a different backend. No issue or reproduction shows that happening.
    Subtraction: delete _refused_native_todos, the mark_provider_switch_replay re-arm, _deferred_options_expiry and its three drain sites, and keep the pre-acquisition gate only.
    Clears when: a non-mocked path is shown where get_or_create returns a backend other than select_provider_backend's answer for the same key.
  • Item 5 — rides along: "/compact keeps the current live-provider or member-aware successor gate". The diff actually replaces the agent.acp_backend read with a liveness probe plus select_provider_backend.
  • Item 8 — zero consumers: is_claude_backend_name # noqa: F401 has 0 remaining uses in chat_runner, and grepping chat_runner.is_claude_backend_name across the repo finds 0 patch sites.
    Subtraction: delete the import.

What this change ships

Inventory (8 items) — 5 justified

Intent: stop /todos being forwarded to, or refused by, a harness other than the one that will actually run it. This is a FIX (#8939).

  1. Kiro /todos refused before any session is created or waited on — justified
  2. A member DM routed to Claude accepts /todos while the global default is Kiro — justified
  3. The legacy claude_code provider label no longer unlocks /todos — justified
  4. /todos re-checked on the acquired provider, with replay-lease re-arm and deferred Slack OPTIONS expiry — oversized
  5. The /compact gate ignores dead rows and uses the member-aware backend — rides along
  6. Autocomplete shows /todos when the default backend is Claude — justified
  7. New supports_native_todos capability and ACP_BACKENDS_NATIVE_TODOS set — justified
  8. Dead is_claude_backend_name import kept under noqa — zero consumers

[FIRST-PRINCIPLES-REVIEWED] 4296474

Description read

[DESCRIPTION-READ] deabac8c049471ba406c2a72ff1b29d091ffa717414fae3cf181bcc368069fc6

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 42964745145a9678c4efb5499d6963064aaa3ec7 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings block this merge.

FINDING — src/kiro_crew/dashboard/handlers/agents.py:3080 — GET /api/slash-commands drops /todos from the blocked set whenever capabilities_for(cfg.agent.acp_backend).supports_native_todos is true, but it never checks the member backend. agent.member_acp_backend defaults to "kas", so with acp_backend="claude" every member-DM composer suggests /todos. The gate in _run_chat routes the same command through select_provider_backend(...) to KAS and refuses it with the "not available" warning. This breaks the documented rule that autocomplete never suggests a command the dashboard rejects. → Fix: keep /todos blocked unless both capabilities_for(acp_backend) and capabilities_for(resolve_selected_backend(member_acp_backend)) support it, or have the endpoint take the slot's session key. The second option needs a frontend change outside this diff.

[OPUS-REVIEWED] 4296474

@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 8205070 to 9e03593 Compare September 27, 2026 08:08
@leonlaiyc
leonlaiyc requested a review from a team as a code owner September 27, 2026 08:08
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 9e03593 to 21b3289 Compare September 27, 2026 14:43
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 21b3289 to 6f63165 Compare September 27, 2026 15:11
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 27, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from f24e16d to 3a9af80 Compare October 1, 2026 06:17
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 3a9af80 to a58a4b9 Compare October 1, 2026 09:00
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from a58a4b9 to 0baca19 Compare October 1, 2026 14:44
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 0baca19 to 0df44c7 Compare October 3, 2026 04:32
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 3, 2026
@leonlaiyc
leonlaiyc force-pushed the claude/b18-todos-live-harness-20260925 branch from 0df44c7 to 2a327f1 Compare October 3, 2026 10:44
Refuse unsupported /todos before acquiring a session, using a live
provider or the member-aware backend selector. Confirm support on the
acquired provider and preserve OPTIONS and first-turn replay on refusal.

Pin the public ACP-only allocation contract: a dormant provider label
does not select a harness. Keep migrated consumers on capability answers
and liveness-qualify the existing manual compact gate.
@leonlaiyc

Copy link
Copy Markdown
Contributor Author

@iamwhatever thanks, ready for another look at 4296474.

  • Refusal before acquisition: with no positively live provider, /todos is decided by the member-aware select_provider_backend(session_key, member_acp_backend, acp_backend) before get_or_create. An unsupported backend is refused locally: no kiro-cli cold start, no wait on _wait_if_busy, no OPTIONS expiry. The acquired provider's supports_native_todos is still checked after the claim and stays the final authority.
  • Discriminating test, both directions:
    • test_todos_member_override_selects_claude_over_default_kiro: default kiro, member override claude. /todos must be allowed, and the acquired claude confirms.
    • test_todos_member_kiro_refuses_without_acquisition_over_default_claude (new): default claude, member routes to kiro. /todos is refused, and get_or_create is never awaited.
  • Negative control: with the preflight swapped back to a config-only agent.acp_backend read, all six dead/absent rows fail on behavior (a refusal, or an acquisition), not just on the selector spy.

Also rebased onto a60fda266 to clear the conflict. Only the learn-cron-dashboard.md doc bullet collided. The /compact liveness item stays advisory, as you noted, and is untouched.

@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this fix, @leonlaiyc. The /todos live-and-allocated capability gate has been re-authored onto main's current layout (the gate moved from chat_runner.py to chat_utils.py, and SessionCapabilities grew new fields since this PR was cut) and opened as #17971, which is now green. Closing this in favour of that replacement; your authorship is preserved via a Co-authored-by trailer. Fixes the capability half of #8939.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(dashboard): scope the /todos slash-command block per harness instead of globally

2 participants