Repository navigation
Conversation
be5851a to
748cada
Compare
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the fail-open behaviour and its low hit rate are recorded in the description and were accepted in an earlier maintainer review. A revert undoes all of it. Notes
[DESIGN-REVIEWED] 782c64e Description read[DESCRIPTION-READ] 9819bafd7a013945c10fac80d0233f024299e7f96d6cec4dbc28c2314ef340ee That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The goal is in scope and the ACP-layer move is declared and deferred (#18970). Most of the size goes to hiding the banner, and the author's own replay says that hiding fires rarely. Notes
What this change shipsInventory (9 items) — 6 justifiedIntent: stop a leaked
[FIRST-PRINCIPLES-REVIEWED] 782c64e Description read[DESCRIPTION-READ] 1ac82d28e13fce49440c4450f771b1c2f02cf0f2521eb262cfe7931adb8b0b33 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review (fork) — 🔴 changes requested (blocking)Reviewed Adjudication upheld 1 of 2 blocking finding(s); those must be resolved before merging. BLOCKING -- src/kiro_crew/dashboard/chat_utils_turn_text.py:1361 -- Tool-call inputs are missing from echo evidence
BLOCKING -- src/kiro_crew/dashboard/chat_utils_turn_text.py:1064 -- Equivalent numeric spellings cause legitimate translations to be suppressed
[BLOCK-MERGE] 782c64e Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 — rung MEDIUM, and the harm is silent: a reply the user asked for gets deleted with nothing shown. The tool row stores the full redacted F2 — rung LOW. All of these must hold: the banner shape matches ( |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
748cada to
c7e954e
Compare
c7e954e to
e00588a
Compare
|
Disposition for an earlier head, kept for the record (no longer a ledger record):
|
|
Maintainer rerun requested for the current-head GPT 5.6 Review ( |
|
Disposition for an earlier head, kept for the record (no longer a ledger record):
|
|
Disposition for an earlier head, kept for the record (no longer a ledger record):
|
|
Disposition for an earlier head, kept for the record (no longer a ledger record):
|
|
|
|
|
|
|
|
@bolichen97 the round on There are two outcomes, and I'll build whichever you pick:
The disposition comments above name this call for each open finding. Nothing is built until you answer. |
|
Rebased onto main Checked on this head: the body's validation set (500 modules: 19,727 passed, 25 skipped, 4 xfailed, with |
|
|
|
Rebase onto main #22914 makes Nothing else changed while the scope question to bolichen97 (comment 6332872266) is open. The gate dispositions 6342072133 and 6342072496 were written for Checked on |
|
[operator: bolichen97] @rubencu re your question in #7626 (comment): Thanks for waiting. Option (a): keep the echo evidence in this PR. #18970 will follow after this lands and only add the ACP-layer detection and the Slack test. AI-assisted, posted on behalf of bolichen97 |
|
Maintainer decision on scope: this PR is accepted as an interim fix for leaked token budget banners. #18970 is an open issue with no PR yet, and waiting on it would block a user-visible fix indefinitely. Two conditions:
On the GPT 6.1 lane: its findings at f8d0f5c were downgraded and no longer block. Both describe a case where a legitimate, requested sentence that looks like a banner could be stripped (an earlier expanded |
|
Addressed the Opus 5.5 findings on 5295103 in 30713f7 (rebased on latest main, one commit).
Also fixed the Black 26.10.1 formatting in |
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes at f0a9148. The description still names #18970 as the follow-up that moves recognition into the ACP layer, which is the first of the two conditions this was accepted on. The second condition is that the filter hides only the exact provider banner text, with a test for it. Two upheld GPT 6.1 findings show real reply text that only repeats the banner wording still being hidden, so that condition does not hold yet.
1. Blocking (GPT 6.1, upheld): an earlier expanded prompt counts as complete evidence. In chat_utils_turn_text.py, TurnEchoEvidence.complete returns true even though _expand_prompt_mention_off_loop keeps only a loading chip for an earlier @reference prompt, not its text. So when a later turn asks for a translation of a banner-like sentence that came from that earlier prompt, _classify_turn_banner reads the evidence as whole and hides the translation the user asked for. Please mark evidence incomplete when a prompt's expanded text was not kept, so the filter falls back to showing the reply.
2. Blocking (GPT 6.1, upheld): embedded text resources are dropped without marking the evidence incomplete. In acp/_dispatch_parts/tool_result_events.py, a completed tool result with a text block plus a wrapped resource keeps only the text block and leaves tool_output_truncated false. If the banner-like sentence was inside the resource, the turn reads as having seen the whole output and the reply is hidden. Please set tool_output_truncated (or the equivalent) whenever a resource or other block is omitted.
A test for each case, where reply text repeats the banner wording and must stay visible, would cover the condition.
3. Blocking (UX Review): the give-up notice names a button that is not there. The notice says to press Continue, but the composer's recovery button reads Resume, and on a turn that already showed output no recovery button appears at all. Please make the notice match what the user can actually do.
4. Blocking (First Principles): _prompt_names_an_inlinable_image re-implements the image-path screen from acp/prompt_blocks.py and already differs from it (no user-span limit, no suffix check), and it runs a file probe through asyncio.to_thread on every turn, although the spec says a turn with no banner probes no file. Please reuse the builder's screen and run the probe only when a banner is present.
CI: two backend tests fail on both Linux and Windows: test_chat_runner_composition_contract.py::test_no_owner_captures_a_name_tests_rebind_on_the_runner (stop_recovery) and test_leaked_toolcall_notice.py::TestTheBoundaryScanRunsBeforeTheReset::test_every_mid_segment_text_reset_clears_the_raw_copy. Both touch modules this PR changes (stop_recovery.py, segment_flush.py). Frontend Tests (3) fails hiStyle.test.ts ("addresses the reader as तुम, never आप"), so please check the new Hindi strings. Coverage fails because of those shards.
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes at f401b5f. Besides the points in my last review, this repository now asks for structured matching instead of regex when deciding which text to hide or flag, and this head recognizes the provider banner with new patterns:
dashboard/chat_utils_turn_text.py:_PROVIDER_BUDGET_BANNER_RE,_PROVIDER_BUDGET_TOPIC_RE,_DIGIT_GROUPING_RE,_COUNT_RE,_JSON_UNICODE_ESCAPE_RE, and a per-count pattern built withre.escape(count).acp/_dispatch_parts/tool_result_events.py:_RUN_STATUS_CODE.
These patterns decide whether reply text is hidden from the user, and the upheld GPT 6.1 findings already show real reply text that matches them. Requested:
- Recognize the banner by exact comparison: keep the provider's banner strings as fixed templates, split a candidate line into its fixed parts and its numeric fields, compare the fixed parts exactly, and check each numeric field with
str.isdigit()(allowing only the grouping characters you list, explicitly). - Read the run status code from the tool result's typed fields, if the result carries one, instead of matching text. If it does not, compare the known prefixes (
exit status,exit code) exactly and parse the number withint(). - Better still, and in line with the follow-up named in the description (#18970), recognize the banner in the ACP layer from the provider's own event type, so no reply text has to be matched at all.
The two evidence-completeness points from my last review still apply. First Principles is red on this head, and CI is still running, so I have not read those results yet.
|
Fix round on
GPT 6.1 and Opus 5.5 pass on this head. First Principles still blocks on scope ("Mechanism cost far exceeds the measured effect"). Its clear condition is a human accepting the ≤2/73 coverage, which matches the maintainer decision of 2026-11-02. I have not posted an override. If a maintainer agrees, this is the text to post:
AI-assisted, posted on behalf of iamwhatever |
AI-assisted, posted on behalf of iamwhatever |
|
@rubencu, every check and all AI reviews (GPT 6.1, Opus 5.5, Design, UX, First Principles) now pass on Could you add a span to its title bullet?
Once it is added, anyone can re-dispatch readiness on AI-assisted, posted on behalf of iamwhatever |
|
@iamwhatever Thanks for tracking that down. I neutralized comment 6183890234 instead of adding a span. It now starts with "Disposition for an earlier head, kept for the record (no longer a ledger record):", the same edit the other legacy records on this PR got on 2026-10-29, so neither the readiness gate nor the adjudication ledger reads it as a record. Main's I left out The head conflicts with main again ( |
|
Two things on a58a733, to save a round trip:
We will review it again once both are done and CI is green. |
|
@iamwhatever Are you taking this round for bolichen97's request above, the rebase plus the exact-comparison matcher from their change request? We'll stay off the branch until you answer, so our pushes don't collide with yours. If you'd rather hand it back, say so and we'll do it in one push. |
|
@rubencu yes, we took this round. It is pushed now at @bolichen97 both asks are done:
AI-assisted, posted on behalf of iamwhatever |
|
Fix round on
AI-assisted, posted on behalf of iamwhatever |
Some model backends emit their private "You have N weighted tokens left" reminder as answer text. Observed transcripts show it as a whole message or fused to the reply's first word with no separator, and it was persisted, mirrored to channels and spoken as the answer. Recognize the exact leading banner at a line break, end of text, or glued to an uppercase letter, then classify authorship separately. The checks that keep the text visible run first, on every turn kind: a typed transient retry, an explicit capacity request from the session's own human, and a banner whose phrase appears in a tool result delivered this turn. Only then does a typed provider recovery strip the prefix and keep the answer, and an ordinary turn strip a banner-only tail after visible output; quoted, embedded and suffixed banners stay visible. A banner-only turn recovers once on the same conversation with a continuation tagged RecoveryProvenance.PROVIDER_BUDGET_ARTIFACT, queued only while tool approval is interactive and re-checked at drain whenever such an entry is queued; the post-token transient retry carries TRANSIENT_RETRY for the same text, so ownership never comes from text equality. The capacity gate reads only what the session's own human wrote: the turn's own text unless it is runner- or app-authored, plus the steers that human typed and the turn consumed. A steer's user-origin record is kept past the steer RPC's acknowledgement, which arrives before the consumption echo, and is released when the steer settles: by the consumption settle, or by the turn-end requeue for a steer the turn never consumed. A banner-shaped answer may echo text the turn can see, so the strip runs only when the evidence it reads is whole. A tool result is not whole when the display bound cut it, when a Json item's stdout-only display left out another output field (stderr, a nested object or list, a number), or when it held an image block, bare or inside an MCP envelope. A prompt image is unreadable evidence on the turn that sent it and on every later turn, since the model's conversation keeps it. What the model read as input this turn is count evidence whoever wrote it: the prompt as sent (an @prompt body, an injected payload) and every steer the turn consumed, a peer's included; only the topic gate stays limited to the session's own human. The dispatch builder's display of a pure-text MCP envelope drops structuredContent and any other output-bearing sibling, so such a result is not whole, and a status field beside stdout is set aside only while it reads as a run status. An earlier turn's skill load sent its whole body to the model, so the bodies its row recorded are echo evidence like the row's own text, and a skill-load row that kept no readable body leaves the evidence incomplete. The prompt-image check joins the link-screen baseline as a refusing site: it screens each candidate the way build_prompt_blocks does and skips every link or junction it cannot vet, and what follows is only an is_file() probe, so it holds no chain. The echo evidence has one owner, chat_utils_turn_text.TurnEchoEvidence, which the settle phase builds only once the banner's shape matched, so a turn with no banner walks no rows and probes no file. The prompt as sent, the consumed steers and the human's own words are kept within a count and size bound there; a text left out marks the evidence incomplete and the banner stays visible. When a typed recovery strips the banner and keeps the answer after it, the banner is cut off the raw segment copy too, so the flush saves the answer instead of the withheld notice.
|
Update on
Still blocked on a maintainer: First Principles now BLOCKs on the premise (recognise the banner as an ACP
AI-assisted, posted on behalf of iamwhatever |
|
@rubencu handing this one back: we are off the branch, so your pushes will not collide with ours. Where it stands at 782c64e: bolichen97's change request is addressed (no regex left in the banner path; exact template, digit check, exit-status prefixes compared exactly), and the Opus BLOCK is fixed. The GPT 6.1 review on that head raised two new findings in the same echo-evidence matcher (#7626 (comment)): tool-call inputs (meta.input) are not part of the echo evidence, and a zero-padded or Indian-grouped count is not recognised as the same number. We stop after one fix round per area, so those two and the rebase (it now conflicts with main) are yours. Backend Tests (3.12, 4) and Coverage Gate were also red on that head. |
|
@bolichen97 @iamwhatever thanks for handing this back. One design question before the next push. The open GPT 6.1 finding (F1 in the review on 782c64e) is one more case of a wider class. The matcher deletes a banner-only reply when it cannot find the count in the evidence it reads, but Kiro Crew never sees everything the model read: the prompt's prefix context is not stored with the user row, and kiro-cli adds context of its own. More evidence narrows the risk but cannot remove it, and this PR has had many rounds of that. Proposal: never delete a banner-only reply. Show it as a collapsed notice row instead. It would not be shown or stored as the answer and not delivered to channels, and the one continuation still runs. A legitimate reply that happens to match the banner then stays readable. The evidence code (count folding, escape decoding, the ACP truncation flags, the prompt-image probe) can then be removed, which makes this PR much smaller. Does a collapsed notice row meet the goal ("not shown, stored or delivered as the assistant's answer")? If yes, the next push is the rebase plus that change. If not, I'll make the narrower fix instead: every recorded tool field counts as evidence, and a truncated field makes the evidence incomplete. |
Problem / Motivation
Goal: Stop a provider's private
You have N weighted tokens leftbanner from being shown, stored or delivered as the assistant's answer, without hiding a legitimate answer or echo.The kiro-cli backend behind this installation's dashboard sessions emits the exact
You have N weighted tokens leftcontrol banner to interactive chat consumers as if it were the assistant's answer (73 assistant rows across 8 of this host's dashboard transcripts between 2026-09-01 and 2026-09-17 start with it, 52 of them banner-only; verbatim excerpts are in this PR's discussion): as a whole message, glued to the reply's first word, or on its own line before the reply. Dashboard chat persisted it, mirrored it to linked channels and treated it as the turn's answer. Text alone cannot tell that artifact from an answer or echo the user asked for.Why it matters
A banner-only turn ends with no answer while looking finished, so the user's request silently stalls. Stripping too much is the opposite failure: a false positive deletes content the user asked for. Recovery therefore needs structural provenance, not text matching, and the capacity-request fail-open has to see what the user actually said in the turn, including a steer they typed mid-turn.
Not a goal
control_noticeis tracked as Recognise the provider token-budget banner in the ACP layer beside control_notice #18970.$skillbody or another prepend, counts too, and the slot keeps that for every later turn until the gateway restarts), and no result that was cut, held an image or left out a field. Replayed on this head over this host's 2026-09-01 to 2026-09-17 transcripts, it strips 1 of the 8 banner-only tails that end a turn after visible output on the stored rows, and at most 2 of 8 once the new writer has marked the rows: 4 sit in one 21,713-row session past the live window and 2 follow an earlier prompt screenshot. Those banners stay visible; that is the cost of failing open, recorded here at @bolichen97's request (review of 2026-10-17). The rounds since add evidence sources (the prompt as sent, every consumed steer, a peer's included, count digits a tool result carries as JSON escapes, and the bodies an earlier skill load recorded) and not-whole marks (an MCP sibling field, a free-textstatus, an image file the prompt as sent names, a skill-load row with no readable body); each can only keep more of those tails visible, so 1 and 2 of 8 stay upper bounds.What changed (motivation → approach → change)
Symptom → cause. The runner took any leading text as the answer, so a provider-authored budget banner became a transcript row, a channel reply and the turn's result.
Grammar (
chat_utils.strip_provider_budget_banner). One leading banner with a plain or comma-grouped count, ending at a line break, at the end of the text, or directly before an uppercase letter. Recognition proves the shape, not who wrote the bytes.Authority (
chat_utils.classify_provider_budget_banner, called once in the settle phase before answer detection, persistence, Stop hooks and delivery). The checks that keep the text visible run first, on every turn kind: a typed transient retry keeps a repeated answer, any message from the session's own human that names tokens, a budget, capacity, a quota or the context window keeps the bytes visible (chat_utils.mentions_provider_budgetis a topic test, not a wording list: no verb, subject or question shape is required and whitespace is folded first; seven review rounds of ever-new remaining-token wordings showed a list of question shapes cannot be complete, and failing open on a turn that mentions tokens costs one visible leaked banner, the behaviour before this change on exactly those turns, while a missed wording deletes an answer the person asked for, so a message about an API token or a project budget fails open too), and a banner whose count appears in the human's own words this turn (the message and each steer they typed, in any language: a translation request holding the figure names no topic word the gate knows), in what the model read as input this turn, whoever wrote it (the prompt as sent, so an@promptbody or an injected payload, and every steer the turn consumed, a peer's included; count evidence only, never the topic gate), in a tool result delivered this turn or in any row of an earlier turn, whoever wrote it (its text, or a tool row's recorded output; the same number, however the digits are grouped, in any case or wording: the text may hold the banner sentence or only the figure) stays visible as a possible echo, reformatting or repeat of text the turn can see; a count that appears nowhere the turn can see is the provider's own figure. Only then may a typed provider-budget recovery strip the prefix and keep the answer, and an ordinary turn strip a banner-only final tail after it already showed visible output and only while the echo evidence is whole (Evidence completeness and What the capacity gate reads, under Change detail).Owning specs:
docs/system-specs/modules/session.md(Provider-budget banner suppression) anddocs/system-specs/modules/providers.md(transient retry provenance).Backwards compatibility
Compatible: suppression applies only to the recognized leading banner under the authority rule above; the queue drain drops only entries this PR tags; every other reply, row and queue entry is unchanged.
Reader: src/kiro_crew/dashboard/chat_turn/segment_flush.py:_flush_segment -- chat -- a banner-only segment persists no assistant row and the continuation's answer is persisted instead (test_banner_only_final_segment_is_suppressed_and_continued_once, test_banner_only_file_changes_use_current_turn_placeholder)
Reader: src/kiro_crew/dashboard/chat_turn/segment_flush.py:_turn_settle_answer -- chat -- Stop hooks and cross-surface delivery read the classified
assistant_text, which is the stripped answer, or empty on a recovering turn (Stop-hook order on that turn: test_banner_recovery_precedes_stop_hook_without_replacing_visible_variant)Reader: src/kiro_crew/dashboard/chat_turn/steer_queue.py:_requeue_unconsumed_steers -- chat -- an acknowledged steer the turn never consumed requeues with the origin its caller reported and releases the record (test_acked_steer_the_turn_never_consumed_settles_at_the_turn_end)
Tests
Rebased on main
32a43272d, two commits, author kept. On this head: the 288 test modules that touch the changed modules (12,990 passed, 16 skipped), thesrc/i18nfrontend tests (1,163 passed,I18N_BASE_REF=origin/main), isort, Flake8 and mypy on the changed files. Earlier validation, on5295103662: Main split_run_chatinto phase owners underdashboard/chat_turn/, so this change's runner hunks moved with the code they edit: the banner classification and the banner-only recovery arm are_classify_turn_bannerand_settle_provider_budget_bannerinchat_turn/segment_flush.py, called from_turn_settle_answer; the steer-consumed evidence is inchat_turn/steer_queue._on_steer_consumed; theprovenancearguments sit on the recovery sites inacp_recovery.py,stop_recovery.py,turn_failures.pyandmodel_fallback.py; the not-landed guards instop_recovery.pyandturn_stats.py; the Stop-hook insert index inturn_spec_hooks.py. Main already holds a steer's origin records until it settles, so this change's edits tochat_delivery.py,steer_queue._settle_consumed_steersand their tests were dropped as landed. Main has since movedsteering_outcomeandsteering_advertisedintoacp/steering_wire.pyitself, so this change's own copy (acp/steering_answers.py) was dropped on rebase. Checked on this head: the PR's test modules, everytest_chat*,test_acp*,test_dashboard_chat*,test_provider*,test_steer*,test_session_*,test_tool_result*,test_discord*,test_state_parts*,test_link_screen*and facade module (594 modules: 23,708 passed, 17 skipped); 7 cases fail the same way on main on this host (agent-env and launch goldens, private-group CLI name, two composition guards fixed here). isort, Flake8, mypy, comment-history, brand-name, line-cap, docs-lint and the flake-pattern ratchet pass.test/test_provider_budget_banner_evidence_owner.py(17 tests): both bounds by count and size, each unreadable source making the evidence incomplete, rows read only when asked, an overflowed record keeping a banner-only tail visible, and a provider-recovery turn that streams the banner then an answer (line-separated and glued) saving the answer. Without the raw-copy cut both end-to-end cases save the withheld notice.test/test_provider_budget_banner.py(189 tests): grammar table, capacity-gate, classifier and tool-result echo tables, end-to-end turns for preservation, suppression, the one-shot continuation, the continuation under YOLO and session trust, the stop-generation snapshot, the pre-stream trust race, the loop bound, the transient owner, regeneration and the Stop-hook order.test_capacity_gate_reads_a_steer_consumed_after_its_ack(7 cases) drives the real delivery order:steer_into_running_turngets the RPC acknowledgement, then the stream echoessteering_consumed. The origin record survives the acknowledgement, a consumed user capacity request keeps the exact answer visible, a non-capacity user steer still suppresses, a peer's steer that carries the banner's count keeps the answer as an echo, a peer naming tokens without the count still suppresses, and the record is released afterwards.test_acked_steer_the_turn_never_consumed_settles_at_the_turn_endcovers the turn-end settle.test_capacity_request_steered_into_a_provider_recovery_keeps_the_banner(3 cases): a capacity request the human steers into a running provider recovery keeps the exact text; with no human request, or with the recovery's own continuation text matching the topic grammar, the banner is still stripped.test_remaining_token_questions_keep_a_banner_only_tail_after_visible_output(26 cases): with prior visible output, where a banner-only tail is otherwise metadata, every remaining-token wording the review rounds produced (subject or none, modifiers, the ability form, a qualifier beforetokens, the inverted auxiliary, a quantity phrase, the yes/no form, wrapped or double-spaced) and the other senses of the words (how many tokens did the prompt use,API tokens,project budget) keep the answer; only a message that names none of the topic words (finish the remaining tasks,how many files are left,count the lines in this file) still strips.test_remaining_token_question_keeps_its_answer_after_narration_and_a_read(8 cases) drives the end-to-end turn: narration, ausage.jsonread whose result holds only a number, then the banner-shaped answer, which stays visible with no continuation queued.test_banner_tail_after_narration_reads_this_turns_tool_results(4 cases) covers the echo evidence end to end, including a file whose line groups the digits (8,154) and one in lower case, with a request to repeat the line capitalized and without commas: the reply carries the file's number and stays visible.test_repeating_an_earlier_answer_after_a_read_stays_visible: an earlier turn's banner-shaped answer, thenCheck README, then repeat your previous answer exactlywith a read that holds no figure; the repeat stays visible because the count is in an earlier assistant row.test_repeating_a_line_from_any_earlier_row_stays_visible(2 cases): the line came from the human's earlier message, or from an earlier tool's output, and this turn prints it after an edit; both stay visible.test_a_cut_tool_result_keeps_an_ordinary_banner_only_tail_visible(4 cases): with prior visible output and a result whose visible head lacks the phrase, whole evidence strips and a cut result keeps the tail; typed recovery strips either way.test_banner_tail_after_a_cut_read_stays_visibledrives the turn end to end: narration, a read whose result the parser cut past the phrase, the banner-shaped echo stays visible, no continuation.test_banner_echoed_from_stderr_stays_visibledoes the same for a shell result whose banner sits instderr, with the event built by the real dispatch builder and provider translation of the envelope.test_recovery_turn_keeps_a_banner_shaped_echo_when_its_evidence_is_not_wholeruns that envelope on a provider-budget recovery turn: the typed strip stands down and the echo stays.test_an_earlier_cut_tool_row_keeps_a_later_echo_visible: an earlier turn's cut result, marked on its row, makes a later turn's evidence incomplete and its banner-shaped repeat stays visible.test_the_count_in_the_humans_own_words_is_echo_evidence(5 cases): the figure in the human's own request, in French, Spanish or English, keeps the reply; words without it, or another figure, do not.test_capacity_gate_reads_a_steer_consumed_after_its_ackgaineduser-translation-countandpeer-translation-count: a French translation steer, consumed after flushed narration, keeps the banner-shaped answer whoever typed it (a peer's too, since this round).Evidence that is not whole, from the GPT round on
2e68fbf15c:test_an_earlier_prompt_image_keeps_a_later_transcription_visible(2 cases: an earlier user row attached the image, or typed its path) keeps a later banner-shaped transcription visible;test_a_container_or_number_beside_stdout_is_omitted_outputandtest_every_parser_marks_a_stdout_display_that_leaves_out_a_containermark a stdout-onlyJsondisplay that leaves out a nested object, a list or a number (a boolean, a null or an empty container is not output);test_every_parser_marks_a_json_wrapped_mcp_imageandtest_a_raw_output_mcp_image_is_marked_by_both_streaming_parsers(2 cases) mark an MCP image block inside aJsonitem, a barerawOutputenvelope or one underresult. Each is red without its fix.From the GPT round on
a19a4f949d. F1:test_a_turn_input_carrying_the_count_is_echo_evidence_and_never_the_topic_gate(4 cases) andtest_an_expanded_prompt_carrying_the_count_keeps_its_answer_visible(2 cases: an@promptbody that carries the count keeps the answer, one without it leaves the strip), with the peer cases of the steer test above. F2:test_a_status_field_is_set_aside_only_while_it_reads_as_a_run_status(12 cases),test_every_parser_marks_a_free_text_status_beside_stdout,test_the_dispatch_builder_marks_an_mcp_field_its_text_display_dropsandtest_a_result_wrapped_mcp_text_display_marks_what_it_drops(3 cases). Mutation: droppingturn_inputsfrom the runner's call turnspeer-capacity-text,peer-translation-countandprompt-carries-countred; dropping the MCP sibling mark turns the dispatch-builder test red.From the GPT round on
5b7b891a25. F1 (test/test_provider_budget_banner_sent_prompt_image.py): a$skillbody that names a real screenshot keeps the later transcription visible on every turn, an image an@promptbody sent stays evidence on a later turn, and a restored skill-load row keeps its image as evidence; a skill whose example path names no file still lets the tail be stripped, and only a path a regular file backs counts. F2 (test/test_tool_result_escaped_digit_evidence.py): count digits a tool result carries as JSON escapes (grouped, double-escaped, astral, after a no-break space) are evidence through the dispatch builder, the streaming parser and the session-file read-back, and the requested translation stays visible end to end; longer, decimal and lone numbers stay negative. Each fails without its fix.From the GPT round on
79d63418e0. F1 (test/test_provider_budget_banner_skill_body_evidence.py): an earlier$referencebody that carries the sentence keeps the next turn's repeat visible, and a skill-load row with no body leaves the evidence incomplete; a body without the figure still lets the tail be stripped. The first two fail without the fix. The same round's CI found two repository guards this change had not updated: thechat_utilsfacade's manifest now names the two JSON-escape helpers it forwards, and the link-screen baseline records the prompt-image check as a refusing site.test/test_nothing_to_do_directive.py:test_a_banner_only_tail_after_the_quiet_end_keeps_the_quiet_enddrives the real applier: narration,nothing_to_do, then a banner-only tail; the banner stays out of the transcript and the quiet end stands (no provider-artifact stop reason, no continuation, no notice, the turn is not read as interrupted).test_post_token_transient_retry_mints_its_owner_and_keeps_the_tailproves the transient owner from the production path: output, a transient 5xx after it, the one-shot CONTINUE queued underRecoveryProvenance.TRANSIENT_RETRY, the replayed row carrying that owner, and the owned retry keeping a banner-shaped tail an ordinary turn strips (red if the producer drops the tag or the consumer ignores it).test/test_tool_result_truncation_evidence.py(43 tests):providers.acp_turns._to_llm_eventforwards the flag (the production seam between parser and runner;test_acp_provider.py's field-parity test pins it too); each of the three parsers records the cut only when a bound removed text (under, at and over the bound; the read-back's per-part cut on text, stdout and dumped JSON parts; its join cut), and the exact shape the runner must not misread (a banner line as the last line past the bound).test_discord_resumed_busy.py:test_an_acked_but_unconsumed_channel_steer_requeues_as_a_channel_turn(a Discord steer the client accepted and the turn never consumed requeues with both marks and commands off) and the accepted-steer reconciliation test now pins both records held;test_chat_runner_coverage_pending.py::TestSteerLifecycle::test_a_settled_steer_releases_both_origin_records.test_banner_tail_after_narration_reads_this_turns_tool_results(2 cases): narration is flushed at a read's tool boundary, then the model repeats the file. The banner tail stays visible when that tool result carries its phrase, and is stripped and continued once when it does not.test_banner_phrase_in_a_tool_result_is_an_echo_candidate(6 rows) is the classifier table for the same rule.TestPostTokenRecoveryPurge(5 tests,test/test_chat_runner_coverage_queue.py): drain-time purge on a queued correction and a resolved Stop; under auto-approve both owners of the shared continuation text keep their queue place and dispatch in order.test_unattended_slot_guardrails.py(36 tests) passes.test_discord_resumed_busy.pyandtest_session_broadcast.pynow assert the origin record is held for the pending steer.No frontend file changed, so no frontend run.
Mutation: putting the provider-recovery strip back before the capacity check turns
test_capacity_request_steered_into_a_provider_recovery_keeps_the_banner[consumed-capacity-steer]red, with three classifier rows; counting the turn's own runner-authored text as human authority again turns its[recovery-text-names-capacity]case red.Mutation: deleting the tool-result check turns
test_banner_tail_after_narration_reads_this_turns_tool_results[file-echo]red, with three rows of the echo table.Mutation: disabling the banner grammar match turns 46 of the 113 banner tests red, among them
test_strip_is_narrow,test_banner_only_final_segment_is_suppressed_and_continued_onceandtest_capacity_gate_reads_a_steer_consumed_after_its_ack.Mutation: deleting the origin record on the acknowledgement again turns all five real-sequence authority tests red, including
test_capacity_request_steered_into_a_provider_recovery_keeps_the_banner[consumed-capacity-steer], which asserts only the visible answer.Provider transcript:
provider-banner-transcript.mdholds verbatim excerpts of real dashboard transcripts, 73 assistant rows across 8 sessions from 2026-09-01 to 2026-09-17 that start with the banner: 52 banner-only (45 of them after visible output earlier in the same turn), 5 line-separated, 15 glued to an uppercase letter, and 1 glued to a Markdown heading, which the grammar does not recognize. A gate-by-gate replay on this head runsclassify_provider_budget_bannerover those turns with every argument computed from the stored rows as the runner computes it (a turn starts at a user, nudge, inject or subagent row; 500-row live window). 8 banner-only rows end their turn after visible output (the earlier count was 7; the eighth is in the rotated archive copy of one session); the rest are segments followed by more output in the same turn, which the runner keeps visible. On the stored rows, which predate theoutputTruncatedmark, 1 of the 8 strips: each of the other 7 has an unmarked earlier tool row in the live window, so the evidence reads as incomplete and the banner stays visible. With the marks the new writer records, reconstructed from each stored result's length (a result'sstderris not recoverable from the stored row, so this is an upper bound), 2 of the 8 strip: 4 are in one 21,713-row session, where the restored prefix outside the live window keeps the banner visible, and 2 follow an earlier prompt screenshot in the live window. No other gate fired on any of the 8: no token or budget topic in the human's words, no count in this turn's tool results or in any earlier row of the live window, and no result over 8,000 characters.Black, isort, Flake8, mypy on the changed modules, docs lint, comment-history, brand-name, subprocess-encoding and
git diff --checkpassed.From @bolichen97's review of 2026-11-03. Ask 1:
test/test_provider_budget_banner_prompt_chip_evidence.pykeeps a banner-shaped reply visible after an earlier@promptchip (tagged and legacy untagged), with a control where no chip is present and the tail is still stripped, and pins theprompt_loadtag on the chip row. Ask 2:test/test_tool_result_truncation_evidence.pymarks a droppedresource,resource_linkor audio block in both streaming parsers, with a control where an empty text block stays whole. Ask 4:test_a_turn_with_no_banner_probes_no_fileshows dispatch records the path and probes nothing until a banner arrives;test_the_screen_is_the_prompt_builders_ownshows a path written as code is not an image. The exact-text condition stays covered bytest_ordinary_read_only_near_miss_is_preserved.Change detail
Evidence owner. Per the maintainer's call on 2026-11-01 (option (a): keep the echo evidence in this PR), the evidence has one owner,
chat_utils_turn_text.TurnEchoEvidence. The settle phase builds it only once the banner's shape matched, so a turn with no banner walks no rows and probes no file. Which prompt paths count as images is the prompt builder's own scan and screen, moved out ofbuild_prompt_blocksinto the leaf modulekiro_crew.image_path_screenso the dashboard reuses it without importing the ACP layer. Dispatch records those paths lexically (remember_sent_image_refs, at most 64 per slot); they are probed on a worker thread only once a reply opens with the banner (_probe_sent_prompt_images). A file removed after it was sent reads as no image. This is an interim fix: the filter's docstring points to #18970 for the ACP-layer recognition. The prompt as sent, the consumed steers and the human's own words are bounded there (retain_turn_evidence,extend_user_authority:TURN_EVIDENCE_MAX_ITEMStexts,TURN_EVIDENCE_MAX_CHARScharacters); a text left out marks the evidence incomplete, so the banner stays visible. When a typed recovery strips the banner and keeps the answer,_drop_segment_headcuts the banner off the slot's raw segment copy too, so the flush saves the answer instead of the withheld notice.src/kiro_crew/providers/acp_turns.pyforwards the parsers'tool_output_truncatedflag onto the provider event the runner reads.Evidence completeness. The three tool-result parsers (the dispatch terminal-frame builder, the client's streaming update parser, the kiro-cli session-file read-back, which also bounds each part) now record on the event (
AcpEvent.tool_output_truncated) when their display bound cut the text, when aJsonitem's stdout-only display left out another output field the tool returned (stderr, or astatusthat does not read as a run status; an exit code or a status word is not output), or when the dispatch builder's display of a pure-text MCP envelope left outstructuredContentor another output-bearing field, the ACP provider forwards it, and every new completed tool row records explicit completeness asmeta.outputTruncated: falseortrue; a missing mark on a legacy row means unknown rather than whole. Any content block the parsers render no text for (an image, an embeddedresource, aresource_link, audio, an unknown type) marks the result not whole too (tool_call_content_omits_text); only an empty text block omits nothing. Prompt images, an earlier@promptexpansion (the slot keeps only its "Loaded prompt" chip, now taggedmeta.kind: prompt_load; an older untagged chip is matched by its text) and a restored frozen prefix outside the 500-row live window also make echo evidence incomplete. When any source is incomplete, a banner-only tail stays visible because the number may sit in unavailable text or pixels, and the typed-recovery strip stands down for the same reason.What the capacity gate reads. The capacity gate reads the turn's own text unless it is runner- or app-authored (a recovery's continuation text never counts), plus every steer the turn consumed that this session's human typed. A message another session injected (
session_send) or a scheduled job delivered is read like typed text: an injected capacity phrase keeps a banner visible, the pre-suppression behaviour, and loses nothing. The user-origin flag is not the gate because a queue entry restored across a restart carries no such flag (a documented security property of the durable queue), so a human request replayed from it would lose its answer. The tool-result text is read off this turn's own tool rows (meta.output: the redacted result, at most its first 8,000 characters).Consumed-steer authority. The steer RPC acknowledges before kiro-cli echoes
steering_consumed, andsteer_into_running_turndeleted the steer's_steer_user_originrecord on that acknowledgement, so the consumption echo could never read it. The record is now held while the steer is pending, read by the turn at the consumption echo, and released by_settle_consumed_steers; a steer the turn never consumed is released by the turn-end requeue, which reads the same record. The home is unchanged (_ChatSlot._steer_user_origin); only its release moved from the acknowledgement to the settle. The channel mark (_steer_channel_origin) is held and released in the same lockstep: the requeue reads the two as a pair, so a channel human's acknowledged, unconsumed steer re-enters the queue as a channel turn (commands off, channel authority), never as a dashboard human's turn.Recovery provenance. A banner-only turn sets the stop reason
provider_budget_artifact, persists no row, clears the streamed banner with an empty assistant frame, and queues one continuation taggedRecoveryProvenance.PROVIDER_BUDGET_ARTIFACTon the same conversation. The approval mode does not gate it: under YOLO, session trust and scoped unattended trust the continuation is queued and runs exactly as the post-token transient retry does, because it re-drives the same request that retry re-drives and no harm separates a banner-triggered continuation from a 5xx-triggered one. The entry arms the continuation replay record, so Stop, rebind and user input purge it like the promise-only and compaction continuations. The post-token transient retry carriesRecoveryProvenance.TRANSIENT_RETRYfor the same text. Requeues within one recovery episode carry the tag forward; a Stop-hook continuation queues behind the recovery. A regeneration whose prior answers are pending still persists its selector row, so those answers survive. An applied terminal directive (nothing_to_do) outranks the recovery: a banner arriving after it is activity after the sanctioned quiet end, so the banner stays out of the transcript but no stop reason, continuation or notice follows and the quiet end stands as a landed turn (docs/decisions/2026-10-03-every-agent-turn-ends-with-text-or-nothing-to-do.md).Scope amendment (review round 16). Two parts of the earlier cut are gone: the auto-approve authority gate for the banner recovery (
_provider_recovery_is_authorized,_drop_unauthorized_provider_recovery, its drain call and the two notices), because no threat separates a banner-triggered continuation under auto-approve from the 5xx-triggered one that already runs there; and the remaining-token wording list in the capacity gate, replaced by the topic test above after seven review rounds each found a wording it missed.Manual verification
N/A — deterministic tests drive
_run_chatandsteer_into_running_turnwith scripted provider streams for every banner shape and recovery path. The one UI change is the give-up notice row, which now carries a notice token and renders localized copy.Give-up notice row (
provider_budget_artifact_give_up), drawn by the real transcript renderer onwebsite/capture/gateway-notice.html(dark theme). It names no button: on a turn that already showed output no recovery button appears, so it asks the person to send the request again. English, then German:Related Issues
Follow-up #18970: recognise the banner in the ACP layer beside
control_noticeso every consumer of the event stream gets the same classification. Maintainer decision (bolichen97, 2026-11-01, option (a)): the echo evidence stays in this PR and its cost is accepted; #18970 follows after this lands and adds only the ACP-layer detection and the Slack test.Sibling sweep
Sibling: src/kiro_crew/slack -- tracked in #18970
The same provider stream also reaches cron dispatch, workflows, side chat and apps; #18970's recognition beside
control_noticecovers every consumer.Pattern harvest
Rule candidate: review-prompt
Pattern: recovery entries that share text must carry a typed owner; never key stripping or queue policy on text equality.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement