Skip to content

fix(dashboard): don't tell a productive episode it returned nothing - #20764

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/13254-empty-turn-rung2-episode
Oct 15, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/13254-empty-turn-rung2-episode

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Every automatic continuation of a chat turn that already did work keeps saying it continues from what already ran, instead of telling the user and the model that the model returned nothing.

With session.empty_response_max_continues set to 2 or more, a turn that ran a tool but ended without a closing reply is continued automatically. When that continuation itself streams nothing, the next recovery posts ℹ️ The model returned nothing — auto-continuing (recovery 2 of 2). and queues the empty-response continuation, whose text says the model returned an empty response twice. The first turn of that same episode already ran a tool.

Why it matters

The model reads both lines back from the transcript. Being told its work produced no output invites it to redo tool calls whose side effects already landed: a second message sent, a second write, a second PR. The give-up card already keeps the "completed steps will not re-run" wording for the same episode, so one episode tells two different stories. Anyone who raised the continue budget above the default of 1 is exposed.

Not a goal

  • Changing the wording of any notice or continuation message.
  • Changing the give-up rung, the recovery counter or the continue budget.
  • Moving the empty-response tests out of test/test_dashboard_chat.py; the open split series does that.

What changed (motivation → approach → change)

A productive episode's later continuation was told the model returned nothing. Rung 2 of the empty-turn ladder in src/kiro_crew/dashboard/chat_runner.py chose its words from the current turn alone (_empty_activity.productive). The give-up rung reads slot._empty_episode_productive too: the flag that remembers an earlier turn of the same episode did work.

Rung 2's wording branch reads that same flag, so every continuation of a productive episode gets the "continuing from what already ran" notice and _ACTIVITY_NO_REPLY_CONTINUE_MSG:

Turn, budget 2 Before After
1. ran a tool, no reply continuing from what already ran (recovery 1 of 2) + _ACTIVITY_NO_REPLY_CONTINUE_MSG same
2. continuation, empty The model returned nothing — auto-continuing (recovery 2 of 2) + _EMPTY_AUTO_CONTINUE_MSG continuing from what already ran (recovery 2 of 2) + _ACTIVITY_NO_REPLY_CONTINUE_MSG
3. continuation, empty give-up: completed steps will not re-run same

The counter branch keeps its per-turn test and moves exactly as before: the flag is only ever set when the counter is already 2 or more, where the productive step max(n + 1, 2) and the plain step n + 1 agree. The session spec (docs/system-specs/modules/session.md) records that rung 2 splits per episode, like rung 3.

Backwards compatibility

Compatible: only the notice and continuation text that a productive episode's later continuations get changes; no config key, API, stored record or default moves.

Removes nothing: both continuation messages and both notices stay; one path switches from one existing pair to the other.

Tests

  • test/test_dashboard_chat_empty_response_ladder.py::TestEmptyResponseRetryLadder::test_productive_episode_survives_a_second_continuation already ran this exact episode (budget 2, a tool turn, two empty continuations) but checked only the give-up card. It checks the second continuation as well: no notice says "returned nothing", one notice reads from what already ran (recovery 2 of 2), _EMPTY_AUTO_CONTINUE_MSG is never queued and _ACTIVITY_NO_REPLY_CONTINUE_MSG is queued twice. Its docstring describes that flow.
  • Red before: on main 5b7b29f9ad with the product code unchanged, it fails with AssertionError: a continuation inside a productive episode was told the model returned nothing, which invites a redo of tools that already ran.
  • Green after: it passes on this branch.
  • Determinism: 20 of 20 runs at -n0; 10 of 10 at -n 2 --dist loadgroup --max-worker-restart=2 beside test/test_dashboard_chat_empty_response.py and test/test_promise_only_recovery.py (-n 2 rather than 4 because the shared host was loaded); test/test_dashboard_chat_empty_response_ladder.py in a shuffled order (seed 18061, 21 ids) green.
  • Scoped runs: test/test_dashboard_chat_empty_response_ladder.py, test/test_dashboard_chat_empty_response.py, test/test_dashboard_chat.py, test/test_chat_runner_composition_contract.py and test/test_promise_only_recovery.py together (453 passed); test/test_flake_pattern_ratchet.py (56 passed). mypy, flake8, isort, the black, comment-history, brand-name and line-cap gates, scripts/docs-lint.sh and the repo's semgrep rules on the diff all exit 0.

Manual verification

N/A — unit coverage sufficient: the test drives the real _run_chat and its queue drain through a whole three-turn episode, and the change is two predicates on that path.

Related Issues

Fixes #13254

Pattern harvest

Rule candidate: review-prompt
Pattern: one rung of a recovery ladder decides its wording from the current turn while a sibling rung reads episode-scoped state for the same decision.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 15, 2026 09:35
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] 9f6af58

Description read

[DESCRIPTION-READ] 6af90444db349269040e44cae107624ff7706586402319d3e86eaca66e6059df

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Nothing to check.

What this change ships

Inventory (3 items) — 3 justified

Intent: when a turn already ran tools, every automatic continuation in that episode keeps saying it continues from the work already done, never "the model returned nothing", so the model doesn't redo side effects. FIX (#13254).

  1. With a continue budget of 2 or more, the second automatic continuation now shows "continuing from what already ran (recovery 2 of 2)" and queues _ACTIVITY_NO_REPLY_CONTINUE_MSG, instead of "returned nothing" and _EMPTY_AUTO_CONTINUE_MSG — justified
  2. The session spec says the rung-2 wording is split per episode, like rung 3 — justified
  3. The existing test now checks the second continuation's notice and queued body, and fails on base — justified

[FIRST-PRINCIPLES-REVIEWED] 9f6af58

Description read

[DESCRIPTION-READ] 5c12c9a7cecb94917a25ffd6e74b1bb13d5b4cc7846e7c058589f0a87dece8cb

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9f6af58

Verdict parsed from the review's SHA-scoped output markers for commit 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Red-before / green-after evidence for test/test_dashboard_chat.py::TestEmptyResponseRetry::test_productive_episode_survives_a_second_continuation.

Red before - main 213339f671, this test change applied, product code unchanged:

$ python -m pytest -n0 -p no:cacheprovider -q --color=no "test/test_dashboard_chat.py::TestEmptyResponseRetry::test_productive_episode_survives_a_second_continuation"
E       AssertionError: a continuation inside a productive episode was told the model returned nothing, which invites a redo of tools that already ran
E       assert not True
FAILED test/test_dashboard_chat.py::TestEmptyResponseRetry::test_productive_episode_survives_a_second_continuation
========================= 1 failed, 1 warning in 2.16s =========================

Green after - this head (2fee768aa1), same command:

========================= 1 passed, 1 warning in 2.11s =========================

Per-hunk mutation on this head: the wording predicate reverted alone fails the test; the counter predicate reverted alone keeps TestEmptyResponseRetry green (34 passed).

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9f6af58

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9f6af58fc3ad8b50577b400c762d28d8cc49c7ad: <one-sentence reason>

@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@iamwhatever
iamwhatever force-pushed the fix/13254-empty-turn-rung2-episode branch from 2fee768 to de63a98 Compare October 15, 2026 10:57
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased onto main 5b7b29f9ad to clear the conflict with #20696, which moved TestEmptyResponseRetry's ladder tests into test/test_dashboard_chat_empty_response_ladder.py. The changed test moved with them: its hunk is line-for-line the one reviewed on 2fee768aa1, and the product and spec hunks are unchanged. Red-before on 5b7b29f9ad (product code at main's form) fails with the same AssertionError; green-after passes. CI re-running on de63a98410.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 15, 2026
Rung 2 of the empty-turn ladder chose its notice and continuation body
from the current turn alone. With empty_response_max_continues >= 2, a
continuation that streamed nothing after a turn that ran tools got the
"model returned nothing" notice and _EMPTY_AUTO_CONTINUE_MSG, which the
model reads back as an invitation to redo work that already landed.

Rung 2's wording branch reads the same episode flag the give-up rung
reads, so every continuation of a productive episode keeps the
"continuing from what already ran" notice and
_ACTIVITY_NO_REPLY_CONTINUE_MSG. The counter branch keeps its per-turn
test: the flag is only ever set with the counter at 2 or more, where
the productive jump and a plain step agree.

Fixes #13254
@iamwhatever
iamwhatever force-pushed the fix/13254-empty-turn-rung2-episode branch from de63a98 to 9f6af58 Compare October 15, 2026 11:25
@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • Item 3 rides along: the counter-step branch reads the episode flag span=4de715bceedd

fixed in 9f6af58: chat_runner.py:17742 is back to if _empty_activity.productive: with main's comment on 17744.
The remaining product diff is the wording predicate and its comment; the PR body describes only that hunk and drops the counter-predicate mutation line.

@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • A rider whose fix is already complete without it span=0a5f4c56ff27

fixed in 9f6af58: the counter-step if at chat_runner.py:17742 and its comment on 17744 are back to main's text.
Only the wording-branch predicate at chat_runner.py:17754 reads slot._empty_episode_productive; that predicate alone is the fix.
Evidence: the changed test fails on main 5b7b29f9ad and passes on this head, and git diff origin/main -- src/kiro_crew/dashboard/chat_runner.py is the one wording hunk.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 15, 2026 13:04

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: the empty-response continue rung keeps the productive notice and body for the whole episode, with an updated ladder test. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/session.md.

@KiroCrewCaptain
KiroCrewCaptain merged commit e660192 into main Oct 15, 2026
103 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/13254-empty-turn-rung2-episode branch October 15, 2026 13:28
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Empty-turn rung 2 still splits per-turn, so a productive episode's later continuation is told the model returned nothing twice

2 participants