Repository navigation
feat(cli): kirocrew chat --headless for unattended runs - #37768
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. The gateway marker is owner-only and audited, and every caller's typed copy is defanged. Skipping the plan gate gives no more than a person typing "just do it" already gets. Notes
[DESIGN-REVIEWED] bf049b3 Description read[DESCRIPTION-READ] 50afbbba9726e93440277ec54dd343bf95905cbf5245908c86e9cd61703be2d4 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS This adds a new headless path next to unchanged Notes
What this change shipsInventory (9 items) — 8 justifiedIntent: let an eval run one Crew task (chat or conductor mode) with nobody at the terminal and get back an exit code and a JSON summary. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] bf049b3 Description read[DESCRIPTION-READ] 3d8ddfbe02ecf0b0d0ea80e72ade77d060d9d970bef1bc1cf8bafa546de51e86 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
58dc554 to
4fa44eb
Compare
|
Findings on 58dc554, all addressed in 4fa44eb (one amended commit):
|
4fa44eb to
ecb34d6
Compare
|
Findings on 4fa44eb, addressed in ecb34d6 (one amended commit), each with a test in test/test_cli_chat_headless.py:
|
ecb34d6 to
1ac5c00
Compare
|
Round 3 on ecb34d6, addressed in 1ac5c00:
|
1ac5c00 to
7e6c242
Compare
|
Opus finding on 1ac5c00 (conductor seed hits the plan gate), addressed in 7e6c242 per the repository owner's decision of 2026-11-16 (relayed by the conductor): no forged
|
7e6c242 to
fcbb555
Compare
|
Round on 7e6c242, addressed in fcbb555:
|
fcbb555 to
1ade30f
Compare
|
Round on fcbb555, addressed in 1ade30f (rebased on main):
|
|
Real Adapter Contract Tests, evidence:
|
1ade30f to
8ccd8fb
Compare
|
GPT F1 on 1ade30f (background subagents outlive the run), addressed in 8ccd8fb: a row with |
30904b3 to
df6ee34
Compare
|
GPT 6.1 findings on 8ccd8fb: all three fixed in df6ee34 (rebased on 31cc202).
Also from the First Principles note (item 5): the second "nobody typed" list, now in Local: touched tests plus every goal-conductor test (979 passed), headless e2e (2 passed), isort/black/flake8/mypy, line cap, comment history, docs lint, flake ratchet. |
|
span=39e5f7f0b1ae
|
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 20% chance that this change introduces a defect, above the 12% limit for approving on its own. This PR adds Tier: 2, held for a person: design-doc-gate: D. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
This adds a new concept, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here. The review found no blocking defect on this head. It checked that every gateway route and slot field the new CLI uses exists, that The decision stays with the owner and a security reviewer, because the new
For the security review:
Optional:
|
0a9ba09 to
bbadd82
Compare
bbadd82 to
09401c0
Compare
|
span=39e5f7f0b1ae
|
One unattended run on the local gateway with a result: --mode conductor opens the chat in conductor mode on the agent the gateway's crew_mode_agent resolves, --approval-mode trust puts the chat this run opens in Trust through the audited POST /api/chat/mode, --until-done stays through patrol and worker wakes until every work-ledger item is terminal, plus --cwd, --timeout, --budget, --idle-grace, --message-file and --json (asks, wall, credits, completed, transcript, resolved agent and its source). One exit code per outcome. Any ask of a person stops the run with exit 3, and every exit stops the run's chats and their automation. The gateway opens an owner's unattended send with its own [headless run cli:<id>] marker (audited, owner only, typed copies defanged), which the goal-conductor skill lists as a nobody-typed opener. rfc-builtin-eval.md: status accepted; sections 5 and 12 describe what ships.
09401c0 to
bf049b3
Compare
|
span=39e5f7f0b1ae
|
bolichen97
left a comment
There was a problem hiding this comment.
Approving bf049b3a0a. One deep pass over all 14 files, focused on unattended safety.
What I checked
- No silent auto-approval. Without
--approval-mode trustnothing is pre-approved: a pending approval,needs_input, anask_questioncard, a ledger item awaiting the person, or an[OPTIONS:]turn on any run chat ends the run asasked(exit 3) and stops every chat (cli_chat_headless.pyasks_now/_poll).trustgoes through the normalPOST /api/chat/modehandler, so it stays per-slot and under theapproval_modesgovernance scope; a refused mode raises and the run stops before the seed is sent. YOLO is never sent, and_stopno longer resets tonormal, so a process-wide YOLO is not cleared by accident. - Identity matches interactive. The run is the dashboard owner through the
kirocrew tokenpath (1-minute owner token over the peer-verified loopback transport).headless_marker.applyadmitsunattendedonly for the owner (app, internal, member and teammate get 403), checksrun_idagainst a strict word shape, writes a SEL row, and defangs a typed[headless run(anyPsopener, case-folded) for every caller. Skipping the plan gate gives nothing a person typing "go ahead" does not already get. - Exit codes and timeouts. Codes 0-9 and 130 are one table (
STATUS_EXIT), and--jsonalways prints one object. Every exit after the chat exists goes through_stop(automation deleted, turn stopped and forced if declined,spawn/stop-all, chats closed root last, unclosed chats named). - Secrets. Output is status, counts, slot, transcript path and error text only. The token is never printed.
--message-filegoes throughsafe_read_file, so a credential path is refused.
Optional, not blocking
--timeoutdefaults to 0 (no limit). A root turn that never goes idle keeps the CLI waiting forever. That matches interactivechat, but an eval runner should always pass--timeout; worth one line incli.md.- The defang runs only on
POST /api/chat; channel and queued inputs can still carry a typed opener. Design Review already rated this harmless, and I agree.
Local results (worktree on bf049b3a0a)
test_cli_chat_headless.py,test_conductor_agent.py,test_e2e_smoke.py,test_headless_marker.py: 164 passed, 14 skipped (e2e gated). WithKIROCREW_E2E=1, both headless e2e tests passed.- flake8 clean on changed sources; mypy: no issues.
- CI: 79 pass, 18 skipped, none failing.
Design note: the Captain's sign-off ask is about the RFC's status moving to accepted. Joe asked the core group to review this PR, and the code matches RFC §12 as written.
AI-assisted, sent by Bolin Review bot
|
🤖 Kiro Crew Auto-Pipeline [operator: buluoray]: Reviewed at bf049b3. This merged before the review was posted, so the findings are in #38170 instead of a review. None of them blocks; the main ones are an agent shell starting a Trust run on hosts where the local mint is open, and no default wall cap. |
|
Maintainer design sign-off from @bolichen97, recorded after the merge because this PR merged before it was posted (my bot approved it without waiting for this step, sorry about that). On the two points Crew Captain left for the owner:
The AI-assisted, sent by Bolin Review bot on behalf of @bolichen97 |
Problem / Motivation
Goal: Let one Crew task run unattended from the command line, in chat or conductor mode, and end with a result code and a JSON summary, so evals can drive Crew the way they drive Claude Code and Codex through their CLIs.
kirocrew chat -manswers one turn in-process, waits on the terminal fory/n/oat every tool prompt, and prints no structured result. It cannot run conductor mode, cannot outlast the first turn through the ledger patrol and worker wakes, and cannot say whether the run asked a person. That is the first missing piece ofdocs/request-for-change/rfc-builtin-eval.md§12 (merged in #37669).Why it matters
The built-in eval compares Crew chat, Crew conductor mode, kiro-cli V3, Claude Code and Codex, where any ask of a person fails the run. Claude Code (
claude -p) and Codex (codex exec) have headless CLIs; without one, Crew can only be driven through private gateway calls in an out-of-tree bench, which is not a like-for-like comparison and nobody else can rerun.Not a goal
--on-ask continue,--effortand--resume(listed as later in §12.2).What changed (motivation → approach → change)
kirocrew chatwith--headlessrather than add a command:kirocrew runis already theTASK.mdrunner, andchatalready owns agent resolution.kirocrew tokenpath), open one chat slot, optionally set Trust on that slot (POST /api/chat/mode, audited by the existing handler), send the seed and poll. Owner reads used:/api/chat/slots?crew=all,/api/chat/slots/{slot},/api/ask-question/pending,/api/agents/catalog,/api/crew-board?conductor=,/api/autonudge/slot/{slot}./api/chatsend is such a turn, so the conductor arm could never run unattended.POST /api/chatwith"unattended": trueand arun_id, from the dashboard owner only, now opens the turn with the gateway's own honest marker[headless run cli:<run_id>]and writes achat_headless_runSEL row; the goal-conductor skill lists[headless runwith its other nobody-typed openers. Every caller's typed copy of the marker is defanged to[quoted: headless run(the rule a forged[sent by sessionalready gets). Non-owner callers get 403, a badrun_id400, and an unattended seed that opens with/or@400unattended_command_seed(the marker opens the turn, so a command or saved prompt would not run; the CLI refuses it first, exit 2). No other route, caller rule or approval policy changes.src/kiro_crew/cli_chat_headless.py(new): options, conductor agent resolution, gateway client, the run loop, the--jsonresult and exit codes.src/kiro_crew/dashboard/headless_marker.py(new) and its hook inchat_api/chat_send.py;[headless runinbuiltin_skills/goal-conductor/SKILL.md"Skip the gate".--mode conductor: the agent is the gateway'screw_mode_agent(catalog), so a fallback the gateway applied (set_unusable) is honoured;--agentis refused with it;--jsonrecords the agent and its source.--approval-mode trust(the one choice): Trust on the chat this run opens; workers inherit it atsession_create.ask_questioncard for one of them, a ledger item the board reads as awaiting the person, or an idle root turn whose[OPTIONS:]parses to choices. The run stops at the first one (exit 3).doneincluded, deletes the run's automation loops and stops its turns (a declined stop is pressed again withforce), root first, cancels each chat's background agents (POST /api/spawn/stop-all), then re-reads the slot list and stops any chat created since the last poll, then closes the chats; a chat the gateway refuses to close is named in the result'sunclosed. Running or queued background subagents keep the run busy. Trust is left on the stopped chats: the owner route'snormalwould also end a process-wide YOLO.--until-done,--idle-grace,--cwd,--timeout(exit 4),--budgetin credits (exit 5; spend includes workers closed between polls, found from their transcripts'created_by, and finished background agents the run's chats started, read fromGET /api/spawn; a turn or agent counts when it ends),--message-file,--json. Anerroror gatewaysystemrefusal row ends as error (sign-in exit 6, sandbox exit 7). No gateway is exit 8, a conductor that goes quiet with items still open is exit 9 (incomplete), interrupt 130.--cwdsets the run's chat project (POST /api/chat/slots/{slot}/project), so no folder is left behind. A background agent that finished in the last 60 s (the gateway's report wait) keeps the run open, so its result reaches the parent before the chat is closed. Whilework_ledger.remote_dispatchis on, a run exits 1 before opening a chat: a remote crew's worker could not be stopped by the CLI.cli_parser_agent.pyflags,cli.pydispatch (a headless-only flag without--headlessexits 2),docs/system-specs/modules/cli.md.docs/request-for-change/rfc-builtin-eval.md: statusaccepted(owner decision); §5 and §12 describe what ships, including the marker and its boundary.Backwards compatibility
Compatible:
kirocrew chatwithout--headlessis unchanged; every new flag is optional and refused without--headless;POST /api/chatwithoutunattendedbehaves as before except that a typed[headless runopener is rewritten to[quoted: headless run.Reader: src/kiro_crew/dashboard/chat_api/chat_send.py:_api_chat_send -- chat -- a typed
[headless run…opener in any send is rewritten before dispatch; no existing caller sends that text (test/test_headless_marker.py::test_a_typed_marker_is_defanged_for_every_caller,test/test_e2e_smoke.py::test_a_typed_headless_marker_is_defanged)Tests
test/test_cli_chat_headless.py: every flag and bad flag set; conductor agent from the gateway (default, config, fallback); the run loop for done, trust only on its own slot,--cwd, conductor slot checks,--until-donewaiting for items and surviving a wake, busy and asking workers (only listed withcrew=all), chat-created children, ledger asks, options parsing, question cards, timeout, budget (incl. workers closed between polls), error and refusal rows, sign-in failures, interrupts and failed polls stopping the run's chats, automation disarm and forced stop on every exit, children made after the last poll, queued subagents, a non-UTF-8 or sensitive-path--message-file, the empty-turn notice as an error, spend counted again after the chats close; the gateway client; the board read;mainexit codes and--json.test/test_headless_marker.py: owner gets the marker and an audit row; no flag, no marker; typed copies defanged (fullwidth, zero-width, case); app, internal and non-owner refused; bad run ids; command seeds refused; the skill lists the marker as nobody-typed.test/test_e2e_smoke.py: a real gateway with the fake ACP backend:--mode conductor --approval-mode trust --until-done --jsonopens a trusted Crew Mode chat, the seed row opens with[headless run cli:, the run keeps polling while a ledger item is open and exits 0 once it is accepted; a person's typed copy of the marker reads[quoted: headless run.Proof: not provable -- other: new feature, no fix to revert; the new tests fail on main because the module and flags do not exist
Manual verification
Ran locally under the shared test lock: the new suites, the conductor-skill and work-ledger suites, the chat CLI suites and both headless e2e tests; flake ratchet, subprocess-encoding gate, comment-history gate,
scripts/docs_lint.py, mypy. The plan-gate skip itself is model behaviour on the skill text: pinned by the skill-text test and the marker e2e, not by a signed-in model run.Related Issues
Implements the first part of §12 of
docs/request-for-change/rfc-builtin-eval.md(#37669).Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.