Repository navigation
fix(subagent): flag sub-agent runs with no tool calls in completions - #18633
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, check that What this change shipsInventory (6 items) — 6 justifiedIntent: let a parent agent know when a sub-agent's reported disk work can't have happened, so it checks before telling the user the work is done. FIX.
[FIRST-PRINCIPLES-REVIEWED] d9bf567 Description read[DESCRIPTION-READ] d291cda34bb608ace7cfa11b289966a785e5f38e8a0615b49b74dd958561580a That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that every selectable ACP backend emits a [DESIGN-REVIEWED] d9bf567 Description read[DESCRIPTION-READ] 54cd94abc1c7ae05c41880cefe84cd20b49b0a0ed8cd6a3e31a04c89e4d22794 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
defe32b to
51c03d0
Compare
51c03d0 to
9b5de75
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
This PR tries to stop a parent agent from reporting a zero-tool sub-agent's made-up work as done. It covers only half of the case it targets. I read the full diff and the lane verdicts on head 9b5de75. Design CONCERNS is real and has no disposition. Per docs/system-specs/common/injected-messages.md, a lone sub-agent or a single wave digest normally gets no synthesis turn, so the new claim-check in SUBAGENT_SYNTHESIS_PROMPT (src/kiro_crew/dashboard/state.py:2014) never runs there. The only warning left is the dispatch-reply line added at src/kiro_crew/mcp_tools/spawn.py:1044, which is several turns before the [Subagent completion event] where the made-up narration arrives. That is the common way #13012 reproduces. Suggested fix: reuse _declares_no_tools in the completion-envelope builder (subagent_manager terminal/monitoring path) and append NO_TOOLS_AGENT_NOTE to a zero-tool agent's completion event. Add a test that runs one zero-tool spawn_run to completion and asserts the note is in the envelope. Also answer the two advisory items: the GPT finding that KAS reads only user-level specs, so the work_dir lookup at src/kiro_crew/agent_sdk/drivers/acp.py:234 can flag an agent that does have tools, and the First Principles item that the predicate duplicates AcpClient.authored_spec_declares_zero_tools. Note that auto-merge is armed while these concerns have no answer. — crew-pr-reviewer
9b5de75 to
475d7d8
Compare
b3e5832 to
f953977
Compare
6bff554 to
8387081
Compare
8387081 to
ecc9e54
Compare
ecc9e54 to
3ceb01d
Compare
A sub-agent run that made no tool call wrote no file, ran no command and made no commit, yet a reasoning-only agent handed write work narrates it as done. A lone run or a single wave digest gets no synthesis turn, so the claim note rides on the completion event and each digest entry the parent reads the narration from. The signal is the run's own observed tool calls (SubagentInfo.tool_count), so it holds on every backend without reading specs; spawn_sub_agents reads the same count from the spawn status. Fixes #13012
3ceb01d to
d9bf567
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
This PR warns a parent, inside the [Subagent completion event] it reads, that a completed sub-agent made no tool calls and so cannot have written files, run commands or made commits. That closes the #13012 gap where a zero-tool agent's invented narration got reported as done. Our CHANGES_REQUESTED on 9b5de75 is met. The note now rides on the completion event and on each wave-digest entry (slack/gateway.py _inject_with_retry), so lone and single-wave runs that get no synthesis turn are covered, and the dispatch-reply line is gone. The spec-based predicate, along with its KAS scope and duplicate-predicate problems, is replaced by SubagentInfo.made_no_tool_calls (tool_count == 0 and turns == 0). I checked that both counters come from the shared run loop: every normalized EVENT_TOOL_CALL and approved child escalation bumps tool_count, and every own permission request bumps turns. The replay gates already trust this pair as "no side effect", so the Design/First Principles backend worry applies equally to existing safety code and is no new risk. I tried to make it lie or fire wrongly. Stopped and failed runs and flush-only records get no note, GET /api/spawn/{id} sends the field only for outcome == "completed", and spawn_sub_agents attaches unverified only on an exact True. The change is purely advisory text, so the worst case is a false warning, never a refused dispatch. Checks are green and the base is main. — crew-pr-reviewer (deep re-check)
Problem / Motivation
Goal: A parent agent is told, in the message it reads a sub-agent's result from, when that run cannot have done on-disk work, so it checks claimed changes on disk instead of reporting a sub-agent's self-report as done.
A
spawn_runthat hands file-writing, git or build work to a reasoning-only agent type (kirocrew-lite,kirocrew-knowledge, whose specs declare"tools": []) gets a completion event that narrates the work, sometimes with invented command output and commit SHAs, while nothing changed on disk. The parent then reports that work to the user as done.Why it matters
Delegating anything that must land on disk becomes untrustworthy: the user reads a success report for files, commits or PRs that were never made, and only finds out by checking by hand. The issue reports it reproduced across several waves (worktrees left pristine, 22 MDX rewrites reported done with an empty
git status).Not a goal
What changed (motivation → approach → change)
Symptom → cause: nothing tells the parent, where it reads the narration, that the run made no tool call at all, so the narration reads like a report. A lone sub-agent or a single wave digest gets no synthesis turn (
docs/system-specs/common/injected-messages.md), so the note has to ride on the completion itself rather than on the synthesis prompt.Options considered:
spawn_rundispatch reply. Rejected: the dispatch reply is several turns before the completion event the narration arrives in.tools: [], so it needs that whole resolution chain, says nothing on backends that do not enforce the ban, and misses an agent with tools that only narrated.SubagentInfo.made_no_tool_callsis true when the run emitted notool_callframe (tool_count) and raised no permission request (turns), the same pair the transient-retry and cancel-respawn replay gates already read as "no side effect happened". Counting permission requests too means a backend that surfaces a gated tool only as a permission request still counts as having called a tool. Such a run wrote nothing, on every backend, and the check reads no spec or config.Change:
subagent.py:SubagentInfo.made_no_tool_calls, the one predicate.slack/gateway.py(the single completion consumer): a completed run for which it is true ends its[Subagent completion event]with⚠ Agent `<id>` made no tool calls in this run, so it did not itself write a file, run a command or make a commit. Treat any such change it reports as a claim: check it on disk before relying on it.; in a wave digest the same line sits under that member's entry. Failed and stopped runs get no note (their text already says the output is not a finished result).dashboard/messaging_api/run_views.py:GET /api/spawn/{id}reportsmade_no_tool_callsfor a completed live run (not for a stopped or failed one).mcp_tools/spawn.py:spawn_sub_agentsadds anunverifiedfield carrying the note to a completed record whose status reportsmade_no_tool_calls: true; a status read without it (a run recovered from disk, a stopped or failed run) attaches nothing. Thespawn_rundispatch reply carries no note.constants.py:NO_TOOL_CALLS_NOTE, shared by the gateway and the tool.subagent.mdandinjected-messages.mddescribe the note on each path.Review findings and disposition
spawn_sub_agentsanswer one question.tools: []and agents with tools that only narrated.spawn_sub_agentsas completed and get the note): fixed. The status read sendsmade_no_tool_callsonly for a completed run; tests pin that a stopped or failed run carries none, in the status read and in the completion event.tools: []predicate; an export with no outside caller): resolved by the same change. No spec is read for the note, so there is no lookup scope to get wrong and no second predicate.spawn_sub_agentsstatus read carries no agent name): resolved. The tool now keys on the status read'smade_no_tool_calls, whichGET /api/spawn/{id}reports for a completed run.Backwards compatibility
Compatible: no dispatch is refused and no field is removed; a completed run with no tool call gains one advisory line in its completion event and digest entry, a
spawn_sub_agentsrecord from one gains one extra key, andGET /api/spawn/{id}gains one field for a completed run. Runs that called tools see no change.Removes nothing that callers depend on: the advisory line that was on the unreleased dispatch reply moves to the completion event.
Tests
test/test_subagent_no_tool_calls_note.py:spawn_sub_agentsaddsunverifiedonly for a status reportingmade_no_tool_calls: true, not for false, absent or a non-bool value.GET /api/spawn/{id}reportsmade_no_tool_callsfor a completed run (true with no activity, false after a tool call or a permission request) and omits it for a stopped or failed run.spawn_runstill dispatches and its reply carries no note.The lone, single-wave and status tests fail on the previous head and pass with this change. The sub-agent, spawn, gateway, messaging handler and dashboard chat suites pass.
test/test_approval_decide_owner_bound.pyfails onorigin/mainwithout this change and is unrelated.Manual verification
N/A — unit coverage sufficient: the change is a count check plus text appended to a completion event, a digest entry and a tool record, and the tests drive the real gateway completion consumer, status handler and tool handlers.
Related Issues
Fixes #13012
Related: #17307
Pattern harvest
Not generalizable: the gap is one specific missing signal on the delegation path, not a recurring code shape.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.