Repository navigation
fix(chat): re-send a throttled turn with one copy of the prompt in history - #21907
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the 6/12/24 s throttle wait should apply on non-KAS harnesses. Its only stated reason is KAS's own ~6 s retry inside each attempt. [DESIGN-REVIEWED] 713f5c7 Description read[DESCRIPTION-READ] 23a06ec53c165c8fb9600a4b35a6e686130cc7e7e822ec2e5469262c1a0ce1a9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The new throttle classifier repeats one the same ladder already runs, and Not justified as shipped
What this change shipsInventory (7 items) — 5 justifiedIntent: stop a throttled chat retry from piling copies of the prompt into KAS history, and make throttle retries visible. FIX.
[FIRST-PRINCIPLES-REVIEWED] 713f5c7 Description read[DESCRIPTION-READ] d56b4b3ca0e348a4ca0b9281e3582ca0ea9a3f9f9fe23c5e9a2844d20e3f5755 That sha256 names the PR description this verdict read. Recompute it with |
|
|
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Not ready. Two repo invariant tests fail on this head, so readiness cannot pass.
Head reviewed: e7779a7.
What I checked:
- The rewind gate in
discard_unanswered_prompt: one shot, only when the turn is done, only when no later prompt started (_prompt_startsordinal), cleared by a steer, and only with the advertisedcheckpointscapability. Both call sites sit on thenot _turn_emittedarms and run before the Stop re-read, so a Stop during the rewind still drops the re-send. _turn_retrieslifecycle: reset at the start of a top-level non-recovery turn, read intoturn_statsbefore the landed-turn reset, emitted asrecoveredon an ok turn and asexhaustedon the terminal no-requeue arm.- The new tests patch the backoff (
_recovery_delay,transient_retry_delay) and usetmp_path, so they do not sleep for the 6 s curve. - The design and first-principles dispositions on this head. I agree with both answers.
Failing on this head (from the CI logs):
test/test_acp_bindings.py::test_the_member_set_is_the_frozen_inventory(Linux shard 5, Windows shard 5):assert ['discard_unanswered_prompt'] == [].test/test_harness_parity.py::test_every_runtime_path_identity_test_is_declared(Windows shard 6):session_handle.py:4051 in discard_unanswered_prompt() compares ACP_BACKEND_KAS.
Inline comments give a fix for each. No other findings.
e7779a7 to
350864c
Compare
|
|
…story When the provider throttles a chat turn before any output, KAS has already stored the user row, so the dashboard's verbatim re-send left one copy of the message per attempt in the model's history. The session handle now keeps the row id KAS reports for each prompt (user_message_id_assigned), and before each re-send of a throttled turn the runner asks the provider to rewind it with _kiro/checkpoint/revertMultiple. Nothing is sent off KAS, without the advertised checkpoints capability, after a later turn or a steer, or twice. A throttle (AcpError.throttle, tagged from the raw frame) waits on its own curve from 6 s, about one KAS attempt, instead of 2 s. Re-sends are counted per mechanism on the slot: the landed row's turn_stats records retries, and kirocrew.watchdog.recovery.outcome gains throttle and transient_5xx points. Refs #21876
350864c to
713f5c7
Compare
|
|
Both earlier findings are fixed: discard_unanswered_prompt is now in the frozen _MEMBERS inventory, and the backend identity check no longer needs a declared test entry. Head 713f5c7 is the same patch rebased onto main. Approval follows once readiness passes.
bolichen97
left a comment
There was a problem hiding this comment.
Both earlier findings are fixed on this head and readiness passed. Approving.
…ext turn (#21937) Follow-up to #21907. A Stop that ended a throttle episode reset the transient budget (or, for a Stop still resolving, nothing) but kept _turn_retries. A later Continue runs as a recovery turn, which keeps the running count, so it reported the stopped episode's retries in turn_stats and emitted a "recovered" outcome for it. Each of the four Stop arms on the ladder (a Stop still resolving, before the backoff, after it, and during the model-fallback wait) now clears the count. Refs #21876 Co-authored-by: Joe Guo <zejiangg@amazon.com>
Problem / Motivation
Goal: When the model provider throttles a chat turn before it answers, the retry leaves one copy of the message in the model's history, waits about one KAS attempt before re-sending, and records how many retries the turn took.
In a matched latency run (opus-5.5, medium effort, 15 warm turns on KAS), 4 turns were throttled. Each retry re-sent the same message into the same KAS session, and KAS kept every failed copy: one turn ended with four copies of its prompt, and the model's next reply began "There are several queued requests…". The ladder re-sent 2.4 s after a failure that KAS had already retried for about 6 s inside the attempt. Nothing in
turn_statsor telemetry said a turn was retried.Why it matters
Throttled turns cost +7.6 to +30.8 s each in that run, and one got no answer. The extra copies stay in the session for good: the model treats them as more requests, and a copy of a first message carries the whole session context (128k chars). Without a retry count, a latency report cannot split a turn's model time into the first attempt and the retries.
Not a goal
stream_and_collectpath (cron, sub-agents, workflows) and the dedicatedAcpClientbackends.retries.What changed (motivation → approach → change)
turn_stats.retrieson the landed rowkirocrew.watchdog.recovery.outcomethrottle/transient_5xx,recoveredorexhaustedKAS stores the user row before it asks the model, so a failed attempt's row stays, and a plain re-send adds another. KAS can rewind a session in place:
_kiro/checkpoint/revertMultiple {sessionId, messageId}hides that row and everything after it, and every prompt reports its row id as auser_message_id_assignedsession_info_update. A live probe on kiro-cli 2.32.0 confirmed the model's next answer leaves the reverted prompt out.The KAS session handle keeps the row id of its latest prompt. Before each verbatim re-send of a throttled turn (the same-model retry and the model-fallback retry), the chat runner calls the provider's new
discard_unanswered_prompt. The handle sends the verb only with_meta.kiro.checkpointsadvertised and a row recorded (only KAS's update branch records one), only when no later turn or steer landed after that prompt, and once; any other case, or a failed call, sends nothing and the retry goes out with the copy kept. The rewind runs after the backoff and before the existing Stop re-read, so a Stop during the backoff rewinds nothing and a Stop during the rewind still drops the re-send.AcpError.throttleis tagged from the raw frame likeconnection_failure, and a throttle waits onthrottle_retry_delay, the same doubling curve from 6 s. The slot counts re-sends per mechanism (_turn_retries) for the landed row'sturn_statsand the recovery-outcome counter; the specs for providers, the turn-stats footer and metrics say so.acp/session_handle.py,acp/kas_wire.py,acp/types.pydiscard_unanswered_prompt)providers/base.py,providers/acp.py,acp/session_provider.pyacp/transport_errors.py,llm_helpers.pydashboard/chat_runner.py,chat_turn/recovery.py,chat_turn/turn_stats.py,state_parts/slot_ladder.py,state_parts/chat_slot.pyBackwards compatibility
Compatible: every new method, field and tag is additive; a backend without the rewind keeps the failed copy, and non-throttle errors retry on the same schedule.
Removes nothing: the change adds a step before a throttle re-send, a longer throttle wait, and two recorded counts.
Tests
test/test_dashboard_chat_transient_throttle.py: a fake provider models KAS history (each attempt stores the prompt; the rewind removes the unanswered one). Two throttles then success: one copy,turn_stats.retries == 2, waits 6–7.5 s and 12–15 s, onerecoveredtelemetry point. The same on a crewmate DM slot. A full ladder of throttles: one copy and anexhaustedpoint. The model-fallback re-send rewinds too. A Stop that lands during the rewind still drops the re-send, on the same-model arm and the fallback arm; with the rewind moved after the Stop re-read, both of those tests fail by assertion. Nearby inputs keep the plain retry: a non-throttle 5xx (no rewind, three copies, 2 s and 4 s waits, retries still recorded), a throttle after partial output (no rewind, continuation), and a Stop during the backoff (no re-send, no rewind).test/test_kas_unanswered_prompt_rewind.py: the handle sends_kiro/checkpoint/revertMultiplewith the row id fromuser_message_id_assigned; it sends nothing off KAS, withoutcheckpoints, after a later turn, after a steer, with no row id, or on a second call; asuccess: falseanswer reads False; both providers forward, and a client without the method reads False.test/test_crew_log_edge_throttle.py: also zeroes the throttle curve, since its "throttled" error now takes it.test/test_state_parts_facade.py: the new slot default.test/test_acp_bindings.py:AcpProvider.discard_unanswered_promptjoins the frozen member inventory.Red before the fix:
prove.py --base origin/mainreverts every production hunk in a throwaway worktree and re-runs the changed tests: PROVEN, an assertion fails with the bug reintroduced (on main the fake history holds three copies after two throttles, and noretrieslands).Determinism: 20 runs at
-n0of the two new files (16 passed each), 10 runs at-n 2 --dist loadgroup --max-worker-restart=2withtest/test_dashboard_chat_transient_stream.pyandtest/test_kas_manual_compact.pyas co-tenants (53 passed each), and one shuffled order, seed 27034 (16 passed).Local gates: isort, flake8, black (changed files), mypy (
src/kiro_crew/, no issues), the backend static scripts from the profile, 42 related test files (2,063 passed), and 13 chat retry files on this head (337 passed). semgrep could not install on this aarch64 host; CI's SAST job runs it.Manual verification
A stdio probe against kiro-cli 2.32.0 (
kiro-cli acp --agent-engine v3 --auth-method cli, a throwaway session, deleted after): prompt 1 named codeword ALPHA, prompt 2 named BRAVO, then_kiro/checkpoint/revertMultipleon prompt 2'suser_message_id_assignedid answered{"success": true, "affectedFiles": [], "totalFiles": 0}. Asked for every codeword, the model answeredALPHAonly, andmessages.jsonlheld atombstonerow (checkpoint_revert). A second revert of the same id answeredsuccess: false("not found"). A real provider throttle was not forced.Related Issues
Refs #21876
Pattern harvest
Rule candidate: review-prompt
Pattern: a retry that re-sends a prompt into a stateful session leaves the failed attempt in that session's history unless it removes it first.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)