Repository navigation
feat(acp): read a claude turn the agent starts between prompts - #18285
Conversation
|
Replay script behind Manual verification. It feeds a stdout log recorded by the Phase 0 probe (the script in #18258 (comment), one JSON line per frame: Run from a checkout of this branch: replay.py"""Replay a recorded claude-agent-acp stdout log through AcpClient's idle reader."""
import asyncio, json, sys
from unittest.mock import AsyncMock, MagicMock
from kiro_crew.acp.client import AcpClient
from kiro_crew.acp.types import EVENT_COMPLETE, EVENT_TEXT_CHUNK, EVENT_TOOL_CALL
def kind(m):
if m.get("method") == "session/update":
u = m["params"]["update"]
o = (u.get("_meta") or {}).get("_claude/origin")
return u["sessionUpdate"] + (f" origin={o['kind']}" if o else "")
return m.get("method") or f"response id={m.get('id')}"
async def main(path):
rows = [json.loads(l) for l in open(path)]
prompt_id = next(r["msg"]["id"] for r in rows if r["dir"] == "out" and r["msg"].get("method") == "session/prompt")
i_resp = next(i for i, r in enumerate(rows) if r["dir"] == "in" and r["msg"].get("id") == prompt_id and "result" in r["msg"])
i_prompt = next(i for i, r in enumerate(rows) if r["dir"] == "out" and r["msg"].get("id") == prompt_id)
user_turn = [r["msg"] for r in rows[i_prompt + 1 : i_resp + 1] if r["dir"] == "in"]
after = [r for r in rows[i_resp + 1 :] if r["dir"] == "in"]
client = AcpClient(acp_backend="claude")
reader = asyncio.StreamReader()
proc = MagicMock(); proc.stdout = reader; proc.stdin = MagicMock(); proc.stdin.drain = AsyncMock(); proc.returncode = None
client._process = proc
client._session_id = user_turn[0]["params"]["sessionId"] if user_turn[0].get("params") else "s"
handed = []
client.on_agent_turn = lambda c: handed.append(len(fed))
fed = []
for m in user_turn:
reader.feed_data((json.dumps(m) + "\n").encode())
client._turn_done.clear()
async for e in client._dispatch_events(prompt_id, 60):
if e.kind == EVENT_COMPLETE:
print(f"user turn: complete stop_reason={e.stop_reason}")
break
await asyncio.sleep(0.05) # let the read loop's finalizer release stdout
print("idle reader running after the user turn:", client._idle_reader is not None and not client._idle_reader.done())
for r in after:
m = r["msg"]
fed.append(m)
reader.feed_data((json.dumps(m) + "\n").encode())
await asyncio.sleep(0.02)
print(f" t={r['t']:7.2f} fed {kind(m):45s} handed_off={bool(handed)} pending={client.agent_turn_pending()}")
if handed:
break
if not handed:
print("no hand-off"); return
print(f"hand-off after frame #{handed[0]} of {len(after)} between-turn frames")
for r in after[len(fed):]:
reader.feed_data((json.dumps(r["msg"]) + "\n").encode())
texts, tools, end = [], [], None
async for e in client.stream_unsolicited(timeout=30):
if e.kind == EVENT_TEXT_CHUNK: texts.append(e.text)
elif e.kind == EVENT_TOOL_CALL: tools.append(e.title)
elif e.kind == EVENT_COMPLETE: end = e
print("agent-started turn tools:", tools)
print("agent-started turn text:", "".join(texts)[:200])
print("ended:", end and end.stop_reason, "| stdin writes:", proc.stdin.write.call_count)
await asyncio.sleep(0.05)
left = []
while client._buffer: left.append(kind(client._buffer.popleft().__dict__))
print("left in _buffer:", left)
if client._idle_reader: client._cancel_idle_reader()
asyncio.run(main(sys.argv[1])) |
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: accepted RFC Notes
What this change shipsInventory (7 items) — 5 justifiedIntent: let a claude session's client notice and read a turn the agent starts between prompts, with no visible change yet. ADDITION (Phase 1 of a FIX for #17566).
[FIRST-PRINCIPLES-REVIEWED] 6200a2b Description read[DESCRIPTION-READ] 895e6b8874601ddd7dc4d7b87247bb1b636d05af8696e2deb10d00d1fe9a53ae That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the reader stays dormant until something registers Notes
[DESIGN-REVIEWED] 6200a2b Description read[DESCRIPTION-READ] 664898181c18bb4d7789eb13603ef506feaead66b75ca5960db0f29b86a91890 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
95debab to
b983f42
Compare
|
|
|
|
|
b983f42 to
dd5d344
Compare
|
|
HOLD at 5a59e45. The transport design follows the accepted RFC. The claim is bound only on claude clients, Kiro read paths stay main's, and CI is green on this head. What blocks approval:
To clear: remove the four re-exports or post a first-principles disposition at — crew-pr-reviewer (r110) |
5a59e45 to
9616715
Compare
|
|
bolichen97
left a comment
There was a problem hiding this comment.
Thanks for the update. Head reviewed: 9616715.
Both items from my last review are fixed.
- Response to a newer request.
_read_turnnow records_next_idwhen the turn starts. A response with an id at or above that value goes back to the front of_buffer, and the turn ends with a synthesizedcomplete.test_a_response_to_a_request_sent_during_the_turn_is_left_for_its_readerpins it. - Shared cancel and done state. The four prompt entry points now wait on
_agent_turn_gatebefore they reset_cancelledor_turn_done, and count themselves in_prompts_in_flight.stream_unsolicitedholds the gate for the whole read and checks the count under it.test_a_prompt_waits_behind_the_agent_started_turn_and_keeps_its_own_stateshows that a Stop on the agent-started turn ends that turn ascancelled, and thathas_active_turn()stays True for the queued prompt until its own reply.
I also checked the edges of the gate. A prompt cancelled while it waits never increments the count. The finally in stream_unsolicited waits for _turn_lock before it releases the gate. The spec now says not to await a prompt inside an open stream_unsolicited() loop, which is the one way to deadlock it. The changes outside this feature between the two heads come from the rebase onto main.
On the First Principles concern about the gate: I asked for it in my last review. The dashboard slot.running queue covers only dashboard prompts. The gate keeps the contract inside the client, so a future caller cannot reopen the shared-state window by mistake, and it costs one uncontended lock per prompt. I think it should stay. A disposition reply from you on that item for this head will close it.
I did not run the tests locally on this head. CI on this head is green and readiness is passed.
🔴 0 must-fix · 🟡 0 should-fix.
|
HOLD at 9616715. Our r110 items and @bolichen97's two should-fix threads are fixed on this head. The facade re-exports are gone, the spec notes the instance-binding caveat, a response to a newer request stays in What clears the hold, two replies for this head and no code change required:
I read the source and the test list. I did not run the tests. — crew-pr-reviewer (r111) |
|
|
On the Design punchline for A live run on this head against claude-agent-acp 0.87.0, with no callback registered:
In the tests, |
iamwhatever
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES at 9616715. The code needs no change; three follow-ups need issues.
Both replies clear the remaining lane items. The First Principles disposition explains why the gate belongs in the client (comment). The no-callback run, plus test_no_idle_reader_when_nothing_could_take_the_turn[no_callback] (test), shows a claude session without on_agent_turn never starts the reader. The prompt wrappers then take the gate only to bump the counter (client_transport.py L430-L455). CI is green and @bolichen97 has no open items.
What changed lists three limits that "stay for the dashboard PR". Neither #17566 nor the RFC's Phase 2 exit criteria name them, so nothing tracks them yet. Please open one issue per limit and link each in the description:
- A request wait,
wait_for_compactionor_drain_post_compaction_metadatain the hand-off window files the turn's startsession/updatein_mcp_notifications. - A
_bufferfull of state-only frames stops the reader, so a later agent-started turn waits for the next prompt. - A steer or late steering answer can end an agent-started turn early through
_dispatch_events.
Our earlier rounds missed this; it is raised now on the unchanged head. I read the source and the test. I did not run the tests.
— crew-pr-reviewer (r112)
|
@iamwhatever, on r112: I opened one issue per limit and linked each one from the limits paragraph in What changed:
Each issue links the code at |
|
@iamwhatever, a reminder on r112: the three limit issues are open and linked from What changed (comment): #22262, #22263 and #22264. The head is still |
|
Heads-up from #23248. That PR adds a turn-end notice that says no follow-up is scheduled when a turn promises to report back and nothing will wake the session. Once this PR makes claude agent-started turns visible, a Bash |
claude-agent-acp streams a turn Claude Code starts on its own (a background command finished) while no prompt is in flight. AcpClient reads stdout only while a request waits, so the next prompt read that turn as its own. This is the transport half of the agent-started turns RFC. On the backends in ACP_BACKENDS_AGENT_STARTED_TURNS (claude), an idle reader watches stdout between turns; every other stdout reader claims stdout first, which stops it. On a frame that shows the agent acting, the reader leaves the frames in _buffer and calls on_agent_turn. stream_unsolicited() reads that turn without sending a prompt and ends it on the adapter's autonomous-origin usage_update. A response to a request sent after the turn began ends it too, but stays in _buffer for that request's reader. A prompt entry point waits behind the open turn before it resets the turn state or writes, so a Stop on the agent-started turn is kept and has_active_turn() reads the prompt's own state. The claude adapter binds the claim, the turn end, the prompt queue and the reader's teardown on the session it launches, so a Kiro client's constructor and read methods stay as they are. Nothing registers on_agent_turn yet, so the reader never starts and behaviour matches main. Part of kirodotdev#17566
iamwhatever
left a comment
There was a problem hiding this comment.
Approve at 6200a2b.
Our r112 request is met. The three limits the description leaves for the dashboard PR are now tracked in #22262, #22263 and #22264, which are linked from What changed. Each issue describes its limit with code links. This head is a rebase of 9616715. The reader, the stdout claim, the prompt gate and the turn-end rules are unchanged from the version we reviewed (client_transport.py#L395-L476), and the claim-coverage test still checks every _read_message caller.
The AI lane notes on this head repeat items already answered: the prompt gate was kept at @bolichen97's request, and a no-callback live run showed claude sessions behave as on main. @bolichen97 has no open items, every check is green, and the branch merges cleanly with main. I read CI and the test source and did not run the tests.
— crew-pr-reviewer (r127)
Problem / Motivation
Goal: Let
AcpClientnotice and read a turn Claude Code starts on its own between prompts, with no user-visible change yet.On the
claudebackend, arun_in_backgroundcommand that finishes after the dashboard turn has ended makes Claude Code answer on its own. claude-agent-acp writes that turn to stdout, butAcpClientreads stdout only while a request waits, so the next user message reads the old reply as its own (#17566). The accepted RFC splits the fix into a transport PR and a dashboard PR; this is the transport one.Why it matters
The dashboard turn in the next PR is built on this API, and the reader is the risky part: an
asyncio.StreamReaderallows one waitingreadline(), so a reader that runs between turns must step aside for every other stdout reader without losing or reordering a frame. Landing it with no caller lets that contract be reviewed and tested on its own.Not a goal
on_agent_turn, theagent.agent_started_turnskill switch and the origin label. Those are the RFC's Phase 2.claude. Phase 3 stays blocked.What changed (motivation → approach → change)
Users see no change from this PR. It adds the plumbing the next PR needs to show a reply Claude Code sends on its own between prompts as its own message.
Nothing reads stdout between turns today, so the change adds a reader for that gap, a rule that every other reader stops it first, a way to read the turn it finds without sending a prompt, and a queue that holds a prompt until that turn ends.
The reader and its claim live in
acp/client_transport.py, the owner of_read_message;stream_unsolicitedis inacp/client_turn.py, andacp/client_class.pybinds both ontoAcpClient. The backend set is declared inagent_sdk/backends_turns.py, with its disposition inagent_sdk/backend_cards.py, anddocs/system-specs/modules/acp-client.mdgains a section on the reader and its claim rule.ACP_BACKENDS_AGENT_STARTED_TURNS(claude) with a callback registered, a task reads stdout whenever no other reader holds it. A frame that shows the agent acting (agent_message_chunk,agent_thought_chunk,tool_call,tool_call_update,plan, or a request from the agent) makes it put every frame it read back at the front of_buffer, record that frame as the turn's start and callon_agent_turn(client). A frame that only reports state (usage_update,session_info_update) goes back and starts nothing. It stops once_bufferis full (itsmaxlen, 100) and leaves the rest in the pipe.ClaudeLaunch.resolve_spawn) calls_install_idle_reader()on the session it launches when its backend is in that set, once per client. On that instance it wrapsensure_ready,_initialize_session,_wait_for_response,_drain_notifications,wait_for_compaction,_drain_post_compaction_metadata,_read_steer_answersand_prompt_loopin a claim, and makes_kill_processand_reset_statedrop the reader first. A claim cancels the reader and waits for it to exit; the last release restarts it. A cancel insidereadline()loses no bytes, because the stream consumes a line only once its newline has arrived. The kiro-cli arm of_spawnreaches no adapter, so a Kiro client's constructor and read methods are main's.stream_unsolicited()_dispatch_events, under request id-1, and sends nothing. After the turn's first activity frame, the bound_prompt_loopends it on theusage_updatewhose_meta["_claude/origin"].kindis in the adapter's autonomous set; a user turn's{"kind": "human"}ends nothing, and a marker left between turns by an earlier cycle does not end this one. A response to a request sent before the turn began ends it too: nothing is sent under-1, so that response answers a wait that already ended, usually a prompt whose turn ended here, and it keeps its ownstopReason. A response to a request sent after the turn began (an id at or above the_next_idit started with) ends it but stays at the front of_bufferfor that request's reader. Process death, the stale-turn gate and the prompt timeout end it as they end a user turn. It yields nothing when a prompt is in flight or has already read those frames.send_message,send_message_stream,stream_eventsandstream_command. Each waits on_agent_turn_gate, whichstream_unsolicited()holds until the turn's_prompt_loophas released_turn_lock, before it resets_cancelledand_turn_doneor writes, and counts itself in_prompts_in_flightuntil it returns. So a Stop pressed on the agent-started turn ends that turn, andhas_active_turn()reads the prompt's own state (RFC decision 3).Four details differ from the RFC's wording. The RFC has every read site call
_claim_stdout(); here the claude adapter binds the claim on the session it launches, the seam its MCP array and settings seed already use, so a Kiro client's construction and prompt paths gain no step (harness-parity H13). The turn also ends on a response, which the RFC's end list does not name yet. The reader does not hold_turn_lock: the claim count keeps it off stdout while any reader is active, and it hands off and calls back in one step with noawaitbetween. And it reads with the cancel-grace check off, because_cancelledbelongs to the turn that ended; otherwise a background result that lands after the user pressed Stop would again wait for the next message.Three limits stay for the dashboard PR, the first caller that registers
on_agent_turnand so the first that opens any of these windows. Each has an issue:session/updatein_mcp_notificationsas main does, and so dowait_for_compactionand_drain_post_compaction_metadata. Keeping them for the turn belongs at the claim, for all three readers at once._bufferfills with frames that only report state, the reader stops, and a restart reads nothing until a turn or a request wait takes those frames. A turn the agent starts after that many state frames then waits for the next prompt, as it does on main; the fix scans what is already queued.stream_unsolicited()reads through_dispatch_events, which takes_session/steeringsteers for the request it reads, so a steer sent into an agent-started turn, or a steering answer that arrives late, can end the turn early under the response rule. Whether an agent-started turn takes steers at all is decided there.Backwards compatibility
Compatible: nothing registers
on_agent_turn, so the reader never starts, and a client of any backend outsideACP_BACKENDS_AGENT_STARTED_TURNSnever binds it, so its constructor, read methods and prompt entry points are unchanged. The one shared signature change is_read_message's keyword-onlycheck_cancel, which defaults to today's check and is passed only by the idle reader.Removes nothing: the change adds a reader, a set and a method, and keeps every existing read path.
Tests
test/test_acp_client_agent_started_turns.pydrivesAcpClientover a realasyncio.StreamReaderand covers the RFC's Phase 1 exit criteria: a cancel insidereadline()on a partial line loses no bytes; each of the five read sites, and a warmensure_readyas a prompt starts, stops a parked reader (a second waitingreadline()would raise) and the last release restarts it; a check fails if a new_read_messagecaller is neither wrapped nor called only from wrapped methods;stream_unsolicited()ends on the autonomous origin, on process death and on the timeout gate, and ahumanorigin ends nothing;session_info_updateand a lone originusage_updatestart no turn; each documented activity kind starts a turn and each autonomous origin kind ends one. Further tests pin the claude adapter binding the reader once across respawns, a Kiro client keeping its own methods and never reading between turns, frame order ahead of what was already buffered, the reader stopping at the buffer bound, a request wait that reads the start frame ending the hand-off and one that puts a permission request back keeping it, the reader running after a stopped turn, an earlier cycle's end marker not ending the turn, a late prompt's tail ending on its response, a prompt in flight or one that starts first keeping the frames as its own, a response to a request sent during the turn staying in_buffer, a prompt sent during the turn waiting behind it with the Stop kept andhas_active_turn()true until it ends, a prompt past the gate keepingstream_unsolicited()out, a cancelled claim restarting the reader, and a reset dropping what the reader held.Of 22 mutations of the production code, one per rule above (for example the claude adapter not binding the reader, binding twice on a respawn, leaving
ensure_readyor_wait_for_responseunclaimed, appending instead of prepending, or ending on a marker before any activity), 21 turn at least one of these tests red. The one that survives is_cancel_idle_readerforgetting the start frame, which only a kill with no reset after it would show. Six more for the response rule and the queue (dropping or loosening the newer-response check, dropping the in-flight check, the prompt wrappers, the_turn_lockwait or the in-flight decrement) each turn a test red.Determinism: the new file passed 20 runs at
-n0, 10 at-n 4 --dist loadgroup --max-worker-restart=2besidetest_acp_client_shutdown_reset.pyandtest_acp_client_capabilities.py, and one shuffled run of all 48 ids (seed 21408). It waits on observed state or the task it started (async_wait_until,await_until, a parkedStreamReader), never on a sleep or a short fixedwait_for.test/test_client_bindings.py, theAcpClientmember inventory added by #20470, gains rows for this PR's 18 new members, and the_read_messagerow's signature hash changes for the keyword-onlycheck_cancel. The AST check intest_acp_client_agent_started_turns.pynow scansclient.pyand every owner module in_CLIENT_OWNERS, so a stdout reader moved into an owner module is still caught.scripts/check_harness_parity.pypasses: the reader dispatches no frame, so it adds no enforcement role.Manual verification
Replayed the four stdout logs Phase 0 recorded from claude-agent-acp 0.87.0 through
AcpClientover aStreamReader, with a callback registered, on the earlier headdd5d344642a442d227c0048d428928af5ae9dff0. In the background-command, Stop and prompt-during logs, the reader was running once the user turn ended, left thesession_info_updateand the firstusage_updatein_buffer, and handed off on the firsttool_calloragent_message_chunk.stream_unsolicited()then read the agent's reply and ended on thetask-notificationoriginusage_update, with nothing written to stdin. In the background-command log that reply is aRead Filecall and "The background probe finished with exit code 0 and printedprobe-done." In the background-subagent log, the lone origin-markedusage_updateafter the held turn and thesession_info_updatestarted no turn. The replay script is in a comment on this PR. The logs were not kept, so this head was not replayed; it moves the claim and the end rule into per-client bindings without changing what they do on those logs, and adds the response rule and the prompt queue, which those logs do not exercise; the tests above pin all four.A live run on
cb31adc2fd84ae712ab4590afab44bd31fb5f22a, this patch one rebase onto main earlier, against claude-agent-acp 0.87.0, with a callback registered: the session started with the bindings installed, answered a prompt, accepted a steer sent from another task mid-turn (steer()returned True andsteer_consumedarrived) and ran/compacttoend_turn. Then arun_in_backgroundcommand (sleep 6; echo probe-done) firedon_agent_turnafter the user turn ended,stream_unsolicited()read "The background command finished with exit code 0 and printedprobe-done." and ended onend_turn, and asend_messagestarted during that read waited for it, then answered "after", withhas_active_turn()False afterwards. The dashboard PR carries the user-visible repro.Related Issues
Part of #17566
Phase 1 of
docs/request-for-change/rfc-agent-started-turns.md(written in #18051, accepted in #18191; the Phase 0 verdicts it relies on are in #18258).Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement