Repository navigation
feat(acp): answer MCP elicitation on both ACP wires - #23271
Conversation
Sensitive Change ReviewSensitive files changed:
|
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: a real tool-call failure, fixed fail-closed in one place that both drivers use, with rendered evidence of the card attached. Notes
[DESIGN-REVIEWED] 6b94f5b Description read[DESCRIPTION-READ] 83ab13113c6ce1af898049a55c253997b4ef3af9c3d2780959d4a92597896188 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing blocks. The blind reader understood Submit, Decline and "I've finished" and what each one does. Every card state (asking, answered, declined, not answered, missing field, refused value) appears in light and dark screenshots. Notes
[UX-REVIEWED] 6b94f5b Description read[DESCRIPTION-READ] e0de75b4f3c7cee161b5c859a863ad3585351c60c743698a249d169cea5a4f2c That sha256 names everything this verdict read: the PR description, plus the 17 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Every item comes from the -32601→cancel defect probed live in #23210, and the capability declaration comes back under the condition the deleted pin set for it. Notes
What this change shipsInventory (10 items) — 9 justifiedIntent: a person in the dashboard answers an MCP server's mid-tool-call confirmation or form, so the tool no longer gets a silent cancel. ADDITION (restores a broken capability).
[FIRST-PRINCIPLES-REVIEWED] 6b94f5b Description read[DESCRIPTION-READ] 0e92e31daa85cea647821063ed4504a72f23f7aa66972c9478664aabebfcf8a8 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
4563f35 to
d851b4c
Compare
|
|
|
|
d851b4c to
383d5de
Compare
|
Rebased: main moved in docs/feature-map/README.md, docs/system-specs/modules/learn-cron-dashboard.md and src/kiro_crew/security_posture.py since this head went green (2d5bb2f -> 241e39c); CI re-running on 383d5de, which also folds in the review fixes (approval-registry guard, display redaction, row-instance stamping, urlsplit refusal, ErrorNotice, agent_sdk import surface, codex wording). |
Sensitive change review
|
|
@buluoray thanks — the blocking item is the same race GPT flagged, and it is addressed on
I'll ping you when that head is green. |
buluoray
left a comment
There was a problem hiding this comment.
Reviewed at c9aff11688c277b4e6e2cc90c622d2ff1fbb6c81.
My blocking item from 2dbc7b2160 is fixed. slotRefresh.ts now retires a pending elicitation card only when no live frame reduced into the view after the fetch was dispatched (liveFrameSeq === liveFrameSeqAtDispatch). That reuses the guard the refresh walk already trusts, so a card delivered over the stream during the round trip stays actionable. ChatSliceCoverageSecondPass.test.tsx pins both the retired case and the raced case. Locally: that file and ElicitationCard.test.tsx pass (130), and tsc is clean.
I am not approving yet, because both review lanes block this head with fresh findings that need your answer (fix or rebut on the PR):
- GPT 6.1 (2 of 2 upheld):
slotRefresh.ts:601, a backgroundwarmSlotCachesnapshot keeps onlyresolvedand drops the submitted answer values; andtranscript_rows.py:488with the frontendCLIENT_ONLY_ROLES, where an answered elicitation row shifts the history cursor because the two sides disagree on whether the role is persisted. - Opus 5.5: several, including a decision path with no SEL record and no
elicitation_resolvedbroadcast (the card stays actionable and a later permission reusing that id gets 409), a blocking file write on the gateway loop where the twin module offloads it, a Stop path that answersdeclinewhere the module's own vocabulary sayscancel, and asks that used to reach Slack/Discord assession/request_permissionnow being declined there.
My request-changes stays in place until those are answered.
|
Push note: head CI on @buluoray's review on Opus on GPT on
Local: backend related set 571 passed (+ |
Sensitive change review
|
|
Push note: head GPT on Local: |
Sensitive change review
|
|
Push note: head GPT on Local: elicitation, dashboard, replay and posture tests 204 passed; mypy, black, isort, flake8, comment-history, line-cap, agent-sdk boundary, docs-lint clean. |
Sensitive change review
|
|
Push note: head CI on GPT and Opus were clean on Local: |
Sensitive change review
|
|
Rebased: One conflict, in
Local: runner plumbing, composition, elicitation and posture tests 469 passed; |
Sensitive change review
|
Disposition: not changed in this PR; the label is unchanged since the UX lane passed it on every earlier head.
"Send answer" is a reasonable wording follow-up, along with the three Notes (the required hint shown at the field, the bound named in the above-maximum error, and what "Not answered" means for the action). It is not a defect in what this PR ships. |
|
Push note: head Opus (span=37a38476072a, GPT (span=3efda54743d5, UX (span=8d01bfbf49b1): unchanged, see the disposition above. Local: elicitation, dashboard, replay tests 151 passed; card + reducer + ownership tests 31 passed; mypy, tsc, eslint, restyle ratchet, comment-history, line-cap clean. |
Sensitive change review
|
An MCP server that gates a tool on a human answer sends its agent an
elicitation; the harness turns it around as a server-to-client request
that only a client with a person can answer. Kiro Crew answered every
such request -32601, which the KAS engine reports to the server as a
cancel, so every elicitation-gated tool failed silently on the default
backend.
acp/elicitation.py owns the wire shape on both methods: KAS's
_kiro/mcp/elicitation (fields nested under params.elicitation) and the
standard elicitation/create (fields flat), read fail-closed into one
request and validated on answer. Both drivers classify the request and
answer it themselves with decline unless the consumer declared it can
put the ask to a person; otherwise the event is yielded, answered
through answer_elicitation, and whatever is still pending is cancelled
at the turn boundary. No trust tier answers an elicitation: it is the
server's requirement, not Crew's tool gate.
The dashboard chat runner renders the ask as a card (a form from the
requested schema, or a URL to visit) and answers it through
POST /api/chat/slots/{slot}/elicitation, dashboard owner only. The
elicitation client capability is declared again, pinned to the handler.
|
Push note: head
The card-reuse question (ChangeCard shell + QuestionCard controls) stays open as a follow-up, as you noted. Local: elicitation, dashboard and bindings tests 625 passed; mypy, black, isort, flake8, comment-history, line-cap, docs-lint, wait/flake ratchets clean. |
Sensitive change review
|
|
@buluoray |
|
HOLD at
All AI lanes pass at this head, and CI is green. — crew-pr-reviewer (r124) |
buluoray
left a comment
There was a problem hiding this comment.
Approving at 6b94f5bf445fbb08fc5026fcc6941395ae5a1222. Everything I left open at 2dbc7b2160 and c9aff11688 is answered at this head. I checked each item in source and backed every guard with a mutation that turns a named test red:
slotRefresh.tsretire-stale race: a pending card is retired only when no live frame reduced since dispatch andpageOldest <= ts <= pageNewest. Dropping either condition turnsChatSliceElicitation.test.tsx› "a PENDING elicitation card … is retired as stale" red.- GPT, warm path:
warmSlotCachenow keepsmeta.answerbesidemeta.resolved. Keeping onlyresolvedturns "a background warm keeps the answer an elicitation card sent" red. - GPT, history cursor:
CLIENT_ONLY_ROLESintranscript.tsnow includeselicitation, matching_TRANSIENT_ROLESintranscript_rows.py:488. Removing it turnstranscript.clientOnlyRoles.test.tsred. - Opus, unsettled decision path: the fallback decline and the SEL write in
elicitation_prompt.pyare each guarded, so the row stamp andelicitation_resolvedalways run. Unguarding either one turnstest_a_failed_audit_or_fallback_write_still_settles_the_cardred. - Opus, blocking write:
client_elicitation.pyresolvessel()and writes inside one bounded executor call. Inlining it turnstest_the_clients_audit_resolves_sel_off_the_event_loopred. - Opus, Stop vocabulary: a Stop now answers
cancel. Reverting turnstest_a_stop_rejection_clears_the_stopped_mark_and_audits_as_stoppedred. - Opus, channel declines: I accept the rebuttal. Only the dashboard turn opts in (
turn_admission.py:1244, reset inturn_entry.py). No trust tier answers an elicitation, which closes the old path where a tier could approve a URL ask nobody opened. - Also pinned: the per-turn cancel now runs before
_turn_done.clear(); moving it back turnstest_a_turn_cancelled_while_answering_the_left_over_ask_leaves_the_handle_idlered. The owner-only check on the answer route: removing it turnstest_a_non_owner_dashboard_session_cannot_answer_the_owners_askred.
Run locally on this head: test_acp_elicitation*.py, test_dashboard_elicitation.py and test_chat_runner_turn_plumbing.py (128 passed); ElicitationCard.test.tsx, ChatSliceCoverageSecondPass.test.tsx, ChatSliceElicitation.test.tsx and transcript.clientOnlyRoles.test.ts (133 passed); tsc is clean. Every AI lane passes on this head and CI has no failures.
Follow-ups (non-blocking):
clientCapabilities.elicitationgoes to every backend, but only codex is measured. The Design lane raises the same point. Declaring it per harness, or measuring one more harness, would settle whether another backend moves approvals offsession/request_permission. That would get them declined on Slack, Discord and cron.- In
elicitation_prompt.py, a turn cancellation that lands while thefinallyblock is awaitinganswer_elicitationskips the row stamp and the frame. I found this from source and did not reproduce it. The id is already unregistered, so it cannot cause a 409, but the card would stay actionable until a refresh. - Please open one issue each for the channel card and the ChangeCard/QuestionCard reuse, and link them in the description.
- The UX lane's three notes: the number bound in the error text, the required-field hint style, and what "Not answered" means for the action.
Problem / Motivation
Goal: An MCP server that needs the person to confirm or fill something in mid tool call gets its answer from the person in the dashboard, instead of a silent cancel.
An MCP server may gate a tool on a human answer (MCP elicitation: a confirmation, a short form, a link to visit). The agent cannot answer it, so the harness turns it around as a server→client ACP request. Kiro Crew answered every such request
-32601 method not found. On the default KAS backend the engine reports that to the MCP server ascancel, so every elicitation-gated tool failed with the server's refusal text and the agent could do nothing about it (#891 recorded the symptom; #23210 recorded that the engine now routes the request). Live probe in #23210: KAS sends_kiro/mcp/elicitation; a-32601becomes{"action":"cancel"}at the server.Why it matters
Servers adopt elicitation for exactly the operations a person must own (publish-to-public, delete, compliance gates). Until Crew answers it, every one of those tools is unusable from Crew, on the backend Crew itself owns, with no way for the user to say yes.
Not a goal
elicitation/completebeyond classifying it as a notification, or the request-scoped (requestId) elicitation outside a session.v1/v2engine route elicitations: it answers the MCP server-32601itself and nothing reaches Crew.What changed (motivation → approach → change)
Goal: answer the ask with a human, never automatically, never leaving it hanging.
Approach: one module owns the wire shape, both drivers reach it, and a consumer must opt in before an event is yielded — so a consumer that predates the event kind can never leave the tool call open. Chosen over adding a branch to every event consumer (eight of them) because fail-closed-by-default is the property that matters.
acp/elicitation.py:parse_elicitation_requestreads KAS's_kiro/mcp/elicitation(fields nested underparams.elicitation) and the standardelicitation/create(fields flat) into oneElicitationRequest, fail-closed (unsupported property type, non-string enum, >32 fields, non-http(s) URL →ElicitationParseError).validate_form_contentchecks a person's answers against the requested fields.elicitation_responsebuilds the identical answer for both wires.acp/types.py:METHOD_ELICITATION_CREATE,METHOD_KAS_MCP_ELICITATION,EVENT_ELICITATION_REQUEST,AcpEvent.elicitation;clientCapabilities.elicitation = {form, url}declared again, now pinned to the handler. The key is what makes a standard harness forward a server's elicitation at all; it moves no tool approvals: codex-acp (@agentclientprotocol/codex-acp,CodexElicitationHandler.ts:createMcpElicitationContext/shouldUseAcpElicitation/canUsePermissionFallback) recognises a tool approval as a message-only form and routes it throughsession/request_permissionwhatever the client declared; only a form WITH fields and a URL ask consultclientCapabilities.elicitation._dispatch.classify_notification,client_turn._process_message) returnelicitationfor a request on either method.acp/session_handle_elicitation.py+acp/client_elicitation.py(new owners, same contract): unrenderable →decline; consumer withoutaccepts_elicitation→decline; otherwise record pending, yield the event,answer_elicitationvalidates and sends;cancel_pending_elicitationsat the per-turn reset, onsession/canceland at teardown. Providers forwardaccepts_elicitation/answer_elicitationlikechild_fidelity_aware.chat_turn/elicitation_prompt.pyrenders anelicitationrow (transient likepermission), registers it in the slot's approval registry (Board Blocked lane, Continue 409, Stop all see it), waits the permission window, sends the answer. Only the elicitation route's ownacceptaccepts: every permission-vocabulary resolution (a reject declines; a Stop, the generic approve route and the trust/yolo sweep cancel) is mapped so the server never hearsacceptfor a confirmation nobody gave, the approve route refuses an elicitation id with 409 and the sweep skips those futures. Server-authored display text (message, URL, field titles/descriptions, option labels) is redacted; a request whose field name, option value or default would be rewritten is declined.minItems/maxItemson a multi-select are kept and enforced on both sides. The runner stamps the row it appended and theelicitation_resolvedframe carries itsmid, which the frontend matches strictly (no fallback to another row) and which display redaction keeps on the row, so a reused ACP request id cannot mark an earlier card. Anelicitationrow is transient but carved out of the unowed-window set and counted as answered like apermissionrow, so a bounded transcript read hands the open ask back.handlers/chat_elicitation.pyservesPOST /api/chat/slots/{slot}/elicitation: dashboard owner only (internal-secret and app callers 403), bound to the row the person read (request_midmust equalslot.approval_instance(request_id, row), so a stale card cannot answer a later ask that reused the ACP id), validates form content before resolving, writes the validated answer onto the row. The approve route's bare-approve fallback leaves elicitation futures out of its sole-pending pick, and its 409 is checked on the resolved owner slot after the session-sharing scan.agent_sdk/elicitation.pyis the import surface application code reads the vocabulary from.ElicitationCard.tsxdraws string/number/integer/boolean/enum/multi-select fields or the URL on thebg-card ring-1 ring-bordernotice surface; once resolved it retitles and shows the answer that was sent (accepted / declined / cancelled); a refused answer renders through the sharedErrorNotice, a required-field hint as status text. Strings in all 12 catalogs.mcp_elicitation(outcome = action, reason = who decided) in both drivers and the dashboard runner; the pre-turn drain answers a stranded elicitation requestcancellike a stranded permission instead of dropping it.AcpProvider._to_llm_eventforwardsAcpEvent.elicitation(the field-parity test caught it being dropped, which would have handed the dashboard an empty card); the replay corpus gains one synthesized fixture per wire (kas/mcp-elicitation.jsonl,codex/mcp-elicitation.jsonl) andreplay_framesanelicitationbranch.[[ELICIT:form|url]]raises the request on the launch shape's method and echoes the host's answer.Backwards compatibility
Compatible: every harness that never sends an elicitation is untouched; a harness that now sees the capability and sends
elicitation/creategetsdecline(consumer not opted in) or a real answer, never the-32601it got before. A newelicitationrow role is transient and read by no existing reader.Removes nothing: the change adds a method handler, a row role, a route and a card; no key, kind or path is renamed, hidden or removed.
Tests
test/test_acp_elicitation.py: both wires parse to one shape; the MCP default mode; URL mode, non-web URL refusal and a URLurlsplititself rejects; unhashablemode/type/method values refused rather than raising; every unrenderable property shape refused (a bound past the float range and an over-long offered choice included); bounds;as_dict/from_dictround trip; content validation (type, range, length, enum, unknown field); response carries content only on accept;ftp://example.com/xrefused as a non-web scheme; handle declines without a consumer, declines unrenderable even with one, yields and sends the validated answer, keeps a bad answer pending, cancels what is left; the same decline-without-consumer, text-only-path decline, yield and relay onAcpClient; and every exit that answers an open askcancel, pinned on BOTH drivers:AcpSessionHandle.cancelandAcpClient.cancel_session(session/cancel first, then the open request), the per-turn reset in_run_turn/_dispatch_events(answered before the next prompt goes out), andAcpSessionHandle.destroy.test/test_acp_elicitation_runtime.py: the REAL runtime (reader loop → session queue → classifier → handle) against the fake backend as a child, on both launch shapes: a form answer reaches the engine onelicitation/createand on_kiro/mcp/elicitation; a URL ask accepts without content; a consumer that cannot ask a person yields no event and the engine hearsdecline. Ahandle.cancel()issued instead of an answer reaches the engine as{"action":"cancel"}for the open ask.test/test_dashboard_elicitation.py: the route through an aiohttp TestClient: valid form answer validated, written to the row, resolves; refused content is a 400 and the ask stays open; decline needs no content and a second answer is 404; URL accept; a permission id cannot be answered through this route; internal-secret and app callers 403; bad bodies 400;read_elicitation_answermaps every permission-vocabulary approval to cancel; the tool approve route refuses an elicitation id (409) and leaves the future pending, a bare approve with no id never picks the sole pending elicitation, and the owner scan refuses an elicitation held by a shared slot; switching the slot totrustoryoloresolves a sibling tool approval and leaves the pending elicitation future, its row and its registry entry untouched; an answer withoutrequest_midis 400 and a stale card'srequest_midis 404 while the live card's resolves; the app token's/api/approvals/{id}/{action}route reads an elicitation id as not found while the same grant still settles a tool approval; display redaction redacts labels and taints values and the server's opaqueelicitation_idnever reaches the row; the row in hand is stamped, not the oldest row with the id; the runner's SEL record readsresolvedfor a person's answer,window_expiredfor an unanswered window andstoppedfor a Stop (notfut.done()afterwait_forcancelled it), a Stop's_approval_stoppedmark is consumed, the asking server and tool are stamped from the turn's trusted identity, and an accepted answer rides theelicitation_resolvedframe.test/test_acp_client_capabilities.py: the capability is declared AND both classifiers route both methods (replaces the "not advertised without a handler" pin).test/test_codex_session_mcp_approval.py::test_real_codex_acp_command_approval_stays_on_request_permission_with_elicitation_declared(Real Adapter lane): the pinned codex-acp 2.2.2, handshaken withCodexHarness().client_capabilities(theelicitationkey declared), forwards a command approval assession/request_permission, sends noelicitation/create, and runs the command onallow_once; the frames are the live fixturecodex/command-approval-elicitation-declared-live.jsonl.test_chat_persistence_inline_attachments.py: every_TRANSIENT_ROLESmember is skipped by the persistence builder.test/test_acp_runtime_pre_turn_drain.py: a stranded elicitation request is answeredcancelby the drain.test/test_acp_frame_replay.py: both wires replay into the committed snapshots.test/test_acp_provider.pyfield parity now coverselicitation.website/src/test/ElicitationCard.test.tsx: draft seeding and shaping (includingminItems/maxItems; an untouched optional multi-select is omitted, a required empty one is sent and bounds-checked); the post carriesrequest_mid; an enum choice whose value is''submits while an unset enum is still caught; adate-timedefault renders as the text that will be sent;ChatSliceElicitation.test.tsx:resolveByApprovalIdwrites the frame's answer onto the row themidnames and no other; the card names the asking server and tool and the link's host, refuses an integer past the safe range and a fraction in an integer field with distinct hints, and hands an unset enum a value no option carries so''stays a real choice;ChatSliceElicitation.test.tsx: an answered elicitation card survives a refresh whose page omits it, and a still-pending one the page no longer lists is retired as stale -- unless a live frame landed after the fetch was dispatched or the card is newer than the page's last row;chatRolesParity.contract.test.ts/appSdkEventScope.test.ts: the elicitation renderer lives in the sharedcreateTranscriptRenderersandelicitation_resolvedis in the SDK event-scope table; posts a validated answer; a required-field hint is status text, never an alert; declines; a refused answer renders throughErrorNoticeand keeps the form; URL, resolved, declined and cancelled states with the answer summary; claimed by the default renderer registry (slot-less, disabled).test_session_handle_bindings,test_client_bindings,test_acp_bindings, chat runner composition, config-dir async guard, redaction sink registry).Manual verification
agentclientprotocol/codex-acp; the pinned 2.2.2 dist): command approvals gohandleCommandExecution→requestPermission→session/request_permissionand never enter the elicitation handler;shouldUseAcpElicitationreturns false for a message-only form (an MCP tool approval) and consultsclientCapabilities.elicitationonly for a form with fields or a URL ask. pi:test_real_pi_gated_bash_call_asks_through_request_permissionalready runs under Crew's own handshake with the key declared and is green on this PR. opencode 1.19.1: its ACP server routes tool permissions throughrequestPermissionand never calls the SDK'sunstable_createElicitation(one occurrence in the binary, its definition). claude-agent-acp and goose carry no elicitation code. So with the key declared, none of pi, opencode, claude-agent-acp or goose changes what it sends today; codex-acp sendselicitation/createonly for an MCP server's form-with-fields or URL ask, and any harness that does send one is served by the sameelicitation/createhandler the fake-backend runtime tests drive (the kiro-cli launch shape).-32601answer becomes{"action":"cancel"}at the server; kiro-cliv2never routes the request. Logged in #23210.AcpRuntimeagainst the real KAS engine spawned and ran a session, but the probe MCP server could not be mounted under the Crew-bound agent (its tool allowlist), so the end-to-end "real engine + dashboard card" path is covered by the fake-backend runtime tests above rather than a live run.i18n:check, docs lint, comment-history, brand, harness-parity, flake ratchet, acp-frame host-data — all green locally.local-gate.py's related set is green apart fromtest_acp_spawn_offload.pyand theauto_improvementapp tests, which fail identically onmainon this host.Screenshots / video
Form ask, URL ask (naming the asking server · tool and the link's host), answered / declined / not-answered rows, the required-field hint and a refused answer, each in dark and light. Captured from the isolated entry
website/capture/elicitation-card.htmlviascripts/capture-elicitation-card.mjs.More states
Related Issues
Fixes #23210. Follow-up to #891.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.