Skip to content

fix(dashboard): keep the reply and its footer when a turn ends abnormally - #19984

Merged
bolichen97 merged 1 commit into
mainfrom
fix/abnormal-turn-footer-19855
Oct 14, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/abnormal-turn-footer-19855

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A reply keeps its saved text and its footer (model, credits, elapsed, and its Copy/Fork actions) when its turn ends on an error, a mid-reply 5xx, the time limit, a stall recovery or a prompt timeout.

The line under a reply that names the model, the credits and the elapsed time is missing whenever the turn does not finish normally. In the stall, lost-connection and prompt-timeout cases the text that streamed before the card is not saved as a reply at all, so it has no actions and is gone after a reload.

Why it matters

These ends are common on a busy backend: every capacity blip, stall recovery or timeout leaves a reply with no cost, no model and no actions, and some of them lose the reply text on reload. The credits are still billed and written to the usage row, so the footer and the usage page disagree.

Not a goal

  • The frontend half of the issue: a completed reply's footer hidden when an auto-nudge, sub-agent or cron reply follows it (ChatPage.tsx / messageRenderers.tsx). Left for a follow-up; the issue stays open.
  • The transport-deadline (stop_reason == "timeout") arm, which another open PR adds.
  • Any change to how the usage row is billed or labelled.

What changed (motivation → approach → change)

Turn end Before After
Stop press footer footer
involuntary cancel, terminal error usage row, no footer footer
post-token 5xx retry usage row, no footer turn_stats saved; shown once the frontend half lands
stall / tool stall / compaction failed / connection lost text left as chunk rows, never saved saved reply with turn_stats; footer shown once the frontend half lands
shared-runtime prompt timeout (except Exception) text never saved, no usage row saved reply, footer, usage row
cancel after EVENT_COMPLETE (turn ceiling, late Stop) no footer the completion's footer
cancel after the end-of-turn flush reply saved twice saved once

Root cause: _record_abnormal_turn_usage in dashboard/chat_runner.py, the seam every abnormal end shares, attached turn_stats only for a Stop press. The recovery arms and the generic exception arm returned without calling _persist_partial_reply, and the seam's once-per-turn guard returned before any footer when EVENT_COMPLETE had already written the row.

The seam now attaches a footer on every end with a turn boundary: the completion's own figures (first-token time included) when EVENT_COMPLETE already measured the turn, otherwise the one shared provider read. Both the end-of-turn attach and this seam write through _attach_owned_turn_stats in chat_turn/turn_stats.py, one scan narrowed to reply rows this runner appended (slot._turn_reply_mids), so a workflow result that lands in the window mid-turn never takes the turn's figures. The end-of-turn attach keeps the plain boundary scan only for a turn that recorded no reply row (a command's own confirmation). .github/black-baseline.txt drops test/test_turn_stats.py, which is black-clean now.

On the retry ends a continuation reply follows the saved partial, and today's footer rule in ChatPage.tsx hides a footer whenever another assistant row follows before the next user row. Those rows carry the data now; they show it once the frontend half of #19855 lands. The four recovery arms and the generic exception arm call _persist_partial_reply. A _reply_flushed flag keeps the text to one saved row. The turn-stats-footer spec describes the new rule.

Backwards compatibility

Compatible: rows gain meta.turn_stats and saved replies where they had none; no field, key or caller changes shape.

Removes nothing: every existing row, footer and usage row is still written; this only adds the ones that were missing.

Tests

test/test_abnormal_turn_usage_row_15013.py, driving the real _run_chat:

  • TestStoppedTurnFooter::test_an_involuntary_cancel_gets_the_footer_too replaces the old pin that an involuntary cancel stays bare.
  • TestAbnormalEndFooter pins the footer after a terminal error, a post-token transient, stale_recover and tool stall (the text saved once, no chunk rows left), a prompt timeout (plus its usage row), a cancel after completion, a late cancel that keeps the end-of-turn footer whole, a late cancel that leaves another writer's row without the footer, and one saved row when a recovery arm then fails.
  • test/test_turn_stats.py::TestAttachTurnStats::test_reply_mids_skips_a_row_another_writer_appended pins the narrowed scan, and test_a_completed_turn_leaves_a_row_another_writer_appended_alone pins it on the end-of-turn attach.

The new or changed cases are red without the production change and green on this head. Determinism: the file passed repeat runs and 3 shuffled orders (seeds 1-3), -n 2 and -n0. The backend related set (run_scoped_tests.py --surface backend) passed on this head: 20825 passed, 6 skipped.

Manual verification

N/A — unit coverage sufficient: each end is driven through the real _run_chat with a fake stream; the frontend reads the same meta.turn_stats a completed turn writes.

Related Issues

Refs #19855

Pattern harvest

Rule candidate: review-prompt
Pattern: an early return from a turn's end-handling arm skips the end-of-turn persist/attach; every exit path of a turn must go through the one shared save helper.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners October 13, 2026 22:32
@bolichen97
bolichen97 requested a review from mbriones98 October 13, 2026 22:32
@bolichen97
bolichen97 enabled auto-merge (squash) October 13, 2026 22:32
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 13, 2026
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e6711d1ef46d5decbdfee958b391c154ed9050e7 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of e6711d1ef46d5decbdfee958b391c154ed9050e7 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

On stall, compaction and retry ends, the new turn_stats stays hidden until #19855's frontend half lands. Only the saved reply text shows now.

What this change ships

Inventory (7 items) — 6 justified

Intent: when a turn ends on an error, a timeout, a stall or a cancel, the user keeps the reply they watched stream, along with its cost, model and actions. This is a FIX (Refs #19855; it widens #17669's Stop-only scope, whose test the description says it replaces).

  1. Replies cut off by an involuntary cancel or a terminal error now show the footer — justified
  2. Text streamed before a stall, tool-stall, compaction or connection-lost card is now saved as a reply and survives reload — justified
  3. A shared-runtime prompt timeout now saves the partial reply, its footer and a usage row — justified
  4. A cancel that lands after completion (turn ceiling, late Stop) keeps the completed turn's footer — justified
  5. A cancel after the end-of-turn flush no longer saves the reply twice (_reply_flushed) — justified
  6. On completed turns, the footer now skips assistant rows another writer appended mid-turn. It reuses _turn_reply_mids, the same list _flush_file_changes already scans by — rides along
  7. test/test_turn_stats.py dropped from the black baseline — justified

[FIRST-PRINCIPLES-REVIEWED] e6711d1

Description read

[DESCRIPTION-READ] 3d985190d0558d5d85affbe4455086ec77f50c138ac7c6df71663df0cda03021

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e6711d1ef46d5decbdfee958b391c154ed9050e7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the recovery continuations (pipe-death requeue, stale/tool-stall) don't regenerate the answer from scratch beneath the partial reply that is now saved permanently.

[DESIGN-REVIEWED] e6711d1

Description read

[DESCRIPTION-READ] db8d8a36a84bae86f2ceb9f3328a1a09e7b210a4515c763a273b10263217090f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e6711d1ef46d5decbdfee958b391c154ed9050e7 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e6711d1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e6711d1ef46d5decbdfee958b391c154ed9050e7: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e6711d1ef46d5decbdfee958b391c154ed9050e7 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e6711d1

Verdict parsed from the review's SHA-scoped output markers for commit e6711d1ef46d5decbdfee958b391c154ed9050e7.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e6711d1ef46d5decbdfee958b391c154ed9050e7: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the fix/abnormal-turn-footer-19855 branch from 9116574 to 6292fd3 Compare October 13, 2026 23:30
@bolichen97

bolichen97 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: abnormal-end footer attach chose the last assistant row after the boundary

  • span=db37e66c9e96 Late cancellation assigns chat usage to an unrelated workflow result

fixed in 6292fd3: _attach_abnormal_turn_stats writes turn_stats only onto a row in slot._turn_reply_mids (rows this runner appended via _note_reply_row), the same ownership test _flush_file_changes uses.
A workflow or sub-agent row appended into the window mid-turn is skipped, so it never takes this turn's figures.
Test: test_a_late_cancel_leaves_a_row_another_writer_appended_alone is red on 9116574 and green on 6292fd3.

@bolichen97

Copy link
Copy Markdown
Contributor Author

rebutted: #17669 recorded involuntary cancels and error ends as out of scope for a first pass, not as a product decision; #19855 is the tracked report asking for the footer on exactly those ends.
The old test comment called it a "Scope pin"; the test is replaced by test_an_involuntary_cancel_gets_the_footer_too, and turn-stats-footer.md states the new rule.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • A cancel after EVENT_COMPLETE shows the completion's figures, not a fresh provider read

intended: that is the same footer the end-of-turn attach would have written for that turn, and the same figures EVENT_COMPLETE wrote to the usage row, so footer and usage page agree.
A fresh provider_last_turn_usage read after completion could see a consumed or zeroed total; the figures are only used once _turn_stats_captured is set.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 13, 2026
@bolichen97
bolichen97 force-pushed the fix/abnormal-turn-footer-19855 branch from 6292fd3 to c501ce5 Compare October 14, 2026 00:24
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=56ac07262d35 duplicate of _attach_turn_stats's scan

fixed in c501ce5: one scan serves both paths. _attach_owned_turn_stats in chat_turn/turn_stats.py holds it, _attach_turn_stats delegates to it with reply_mids=None (its pinned base signature unchanged), and the abnormal-end seam calls it with slot._turn_reply_mids.
The copy in chat_runner.py is gone; test_reply_mids_skips_a_row_another_writer_appended pins the narrowed scan.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=73cf8da78cd6 No visible footer after a retry

accepted: true as described. On the retry ends a continuation reply follows the saved partial, and the showFooter walk in ChatPage.tsx stops at the next assistant row, so the footer data is saved but not shown yet.
That walk is the frontend half of #19855, listed under Not a goal and kept open (Refs #19855). The body's table and What changed now say the retry ends show their footer once that half lands (c501ce5).

@bolichen97
bolichen97 force-pushed the fix/abnormal-turn-footer-19855 branch from c501ce5 to e6711d1 Compare October 14, 2026 01:00
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=994b38e05d01 the end-of-turn attach keeps the unnarrowed scan

fixed in e6711d1: the end-of-turn attach now calls _attach_owned_turn_stats with slot._turn_reply_mids, the same narrowed scan the abnormal-end seam uses; it keeps the plain boundary scan only for a turn that recorded no reply row (a command's own confirmation).
test_a_completed_turn_leaves_a_row_another_writer_appended_alone is red on c501ce5 and green on e6711d1.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=a9b88e77a978 the Goal names retry ends whose footer is stored, not shown

rebutted: the Goal line is frozen once the PR opens, and the retry ends are scoped in the body instead: the What changed table says those rows get turn_stats saved and show it once the frontend half of #19855 lands, which stays open (Refs #19855).
On the error, Stop, cancel and prompt-timeout ends no continuation follows, so their footer is visible with this PR alone.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=a5f1a222f426 recovery arms save the partial before a retry

intended, and the continuation is unchanged: build_stale_recovery_prompt() and build_tool_stall_recovery_prompt(...) read no transcript rows, and the pipe-death requeue comes from build_recovery_requeue(message, ...).
A history replay after a failed resume sees the saved partial as the assistant text the backend already streamed; turn-stats-footer.md records this in e6711d1.

@bolichen97

Copy link
Copy Markdown
Contributor Author

rebutted: #17669 named involuntary cancels and error ends as out of scope for a first pass, not as a rule; #19855 is the follow-up report for exactly those ends, triaged as a T1 bug and assigned to a maintainer (bolichen97), who dispatched this fix.
The flipped test and turn-stats-footer.md record the new rule.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@hickoxd

hickoxd commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

i had a fix for #19855 in review when this opened, so i ran my backend tests against this branch and your test files against mine. mine is now up as #20128 and i think it should be the one taken: it covers everything here (your test_abnormal_turn_usage_row_15013.py passes on it with _attach_owned_turn_stats renamed to _attach_turn_stats; the one test_turn_stats.py case that passes reply_mids= explicitly doesn't apply since mine reads slot._turn_reply_mids itself, and has its own test for that row) plus the parts this one leaves out. against this branch these still fail:

  • the shutdown save runs before the footer goes on. _persist_partial_reply saves at shutdown right after the append, and the attach runs later in _record_abnormal_turn_usage, so a restart mid-turn brings the reply back without its footer. when the text was already flushed at a tool call there's no shutdown save at all, so the footer is never written.
  • the seam isn't best-effort, and the stall arms call it now. if anything in it raises (i forced _redact_segment), the stall ends as a chat error and its recovery is never queued. a CancelledError on the shutdown save does the same: the arm loses its card and its recovery.
  • the footer doesn't reach the open page during a recovery. an end that queues one hands the slot over without chat_done, so the cut reply's footer only shows after the recovery finishes. fix(chat): keep a reply's footer however its turn ends #20128 sends it on a chat_message_update patch keyed on the row's mid; the client already merges meta.
  • _attach_owned_turn_stats scans slot.messages[turn_boundary:], which is empty once the window is front-trimmed at _MAX_SLOT_MESSAGES, so a turn in a full window gets no footer. _turn_rows with the turn's start mid handles it.
  • when the completion reports no billing but the provider still holds credits, the footer takes the completion's figures and the usage row takes the live read, so they disagree. one read for both fixes it.
  • a failed compaction appends its notice after the text; the position scan here stamps the notice.
  • the frontend half is deferred here, so the auto-nudge case in Reply loses its footer (model, credits, Copy/Fork) when its turn ends early or an auto-nudge follows #19855 stays broken, and once a cut-short synthesis answer carries turn_stats, groupDisplayItems reads its first half as the answer's end and folds the error card, the recovery and the resumed half behind the next wave. fix(chat): keep a reply's footer however its turn ends #20128 has the shared footer rule for both surfaces and the grouping change.

happy to split #20128 into the backend and frontend halves if you'd rather land it that way, but i'd take it over this one as it stands.

@bolichen97
bolichen97 merged commit 31c5926 into main Oct 14, 2026
101 of 102 checks passed
@bolichen97
bolichen97 deleted the fix/abnormal-turn-footer-19855 branch October 14, 2026 05:50

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after a maintainer read the review notes on this head. The open notes are follow-up polish, not blockers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants