Repository navigation
fix(kas): match spec hook tool matchers against KAS's tool id - #14065
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Hand-pinned KAS tool-id table has no runtime drift signal: a kiro-cli rename silently re-creates the exact never-fires bug this PR fixes. Watch
[DESIGN-REVIEWED] 8ccd4a5 |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify KAS's Not justified as shipped
What this change shipsInventory (10 items) — 8 justifiedIntent: make an agent spec's tool-scoped hooks actually fire on KAS, where matching against the call's title meant they never ran — a FIX (#13926 item 3; live probe; the new turn-loop test fails on base per the mutation check).
[FIRST-PRINCIPLES-REVIEWED] 8ccd4a5 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: On a KAS PreToolUse permission request, a spec hook's tool matcher is no longer compared with the call's title but only with the harness-stated tool id and its kiro-cli translations, and the id itself must survive a new validator — ADJUDICATED: 14 candidates, 0 confirmed newly-refused handed to this review. The tightening does not touch the composite the classifier reads — PLATFORM COVERAGE: posix (6 rows: the sanctioned UNADJUDICATED: The whole tightening lives outside what the classifier can settle — a shell command line cannot express any of it, because the refusal depends on the session's spec and the wire frame, not on argv. The honest verdict is therefore CONCERNS, not PASS, despite the empty confirmed set.
[SCOPE-REVIEWED] 8ccd4a5 |
|
Addressed in f9545e4:
|
cbc073b to
cb477da
Compare
cb477da to
77c2db6
Compare
|
Addressed in 77c2db6 (squashed to one commit):
|
77c2db6 to
856d190
Compare
|
Advisory items on 856d190:
|
856d190 to
5df56f6
Compare
|
Addressed in 5df56f6, from the Backend Tests reds and the Opus 5 findings on 856d190:
|
|
Addressed in 5cf3d4c:
|
5cf3d4c to
f508561
Compare
|
Addressed in f508561:
|
f508561 to
7f211a0
Compare
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
7f211a0 to
290baf0
Compare
A spec hook's matcher names kiro-cli tools (execute_bash, fs_write). On KAS the turn loop compared it with the call's title, which for a shell call is the command line, so a tool-scoped spec PreToolUse hook never fired there. Read the tool id KAS states on a permission request (_meta.kiro.toolId) into AcpEvent.harness_tool_id, and on a PreToolUse match the spec's own hooks against that id plus the kiro-cli names it stands for, from one table (kas_permissions.KAS_TOOL_IDS_BY_KIRO_TOOL, with the shell/read/write aliases). Every id in the table was stated by a live KAS session. Hooks-page hooks, PostToolUse and kiro-cli sessions are unchanged. Nothing is dropped and nothing widens: a matcher the table does not know is kept, meets a KAS id it matches as written, and is warned about when the spec is read. A call KAS names no tool for keeps title matching. The spec hook's stdin reports the tool's kiro-cli name as tool_name. Refs #13926
290baf0 to
8ccd4a5
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Read the full diff. The tool-id read is validated to an identifier (no glob metacharacter, no whitespace, length-capped) and comes from the engine-written _meta.kiro.toolId, not the model's arguments; the no-overlapping-rows test is the right invariant. Required gate green. Approving.
Summary
Item 3 of #13926 ("Matcher vocabulary across harnesses"). Refs #13926; the issue stays open for its other items.
A spec hook's
matchernames kiro-cli tools (execute_bash,fs_write). On KAS, Crew's turn loop compared it with the call's title. For a KAS shell call the title is the command line (touch /tmp/x), so a tool-scoped specpreToolUsehook never fired. KAS's tool names also differ: its shell tool isrun_command.What changed
acp/_dispatch.py:build_permission_eventreads the id KAS states on a permission request,_meta.kiro.toolId, into a newAcpEvent.harness_tool_id(acp/types.py). Only an identifier survives (word characters,.,-and the/,@,:separators; no glob characters or whitespace), and it is not read under a gate envelope.providers/acp.pyforwards it through_to_llm_event, so the turn loop sees it.acp/kas_permissions.py:KAS_TOOL_IDS_BY_KIRO_TOOLis the one kiro-cli name -> KAS ids table (execute_bash->run_command,fs_write->fs_write/fs_append/str_replace/delete_file,fs_read->read_file/list_directory,grep,glob,web_fetch,web_search->remote_web_search,use_subagent->invoke_sub_agent). Every id in it was stated by a live KAS session on a permission request (see Testing).KIRO_TOOL_ALIASESkeeps kiro-cli's legacyshellkey and the shortread/writenames working. The public surface isKAS_TOOL_IDS_BY_KIRO_TOOL,KIRO_TOOL_ALIASES,KAS_TOOL_MATCH_VOCABULARYandkas_tool_match_names(tool_id), so other callers can import it instead of adding a second table. There was no name table to reuse:CAPABILITY_BY_TOOLmaps names to capabilities, not to KAS tool ids.hooks.py:ScriptHookStore.fire(extra_hooks_tool_names=...): when given, an extra (spec) hook's tool matcher is compared with those names instead oftool_name. Globs keep working, because the tool id is translated (KAS id plus the kiro-cli names it stands for) and the matcher is left as written. A spec hook's stdin reports the tool's kiro-cli name (execute_bash) astool_name, so a script that branches on it reads the same vocabulary as its matcher; a Hooks-page hook in the same fire still gets the title.dashboard/chat_runner.py:_fire(tool_id=...)passesagent_sdk.spec_hooks.spec_hook_tool_names(event.harness_tool_id)(theagent_sdkseam ontokas_tool_match_names) only when the session has spec hooks, which only acrew_fires_spec_hooksbackend (KAS) gets. A call KAS names no tool for getsNone, so its spec hooks are matched on the title as before rather than skipped. The four PreToolUse call sites each gain onetool_id=event.harness_tool_idargument.agent_sdk/spec_hooks.py: nothing is dropped and nothing widens. A PreToolUse matcher the table does not know (use_aws,disclose_context, an MCP id) is kept, meets a KAS id it matches as written, and is warned about when the spec is read. A tool-scoped PostToolUse matcher is warned about too, since it still meets only the title.docs/agent-spec-fields.md: thehooksrow says how a tool matcher is read on KAS.Unchanged: Hooks-page hooks still match
tool_name. kiro-cli sessions get no spec hooks from Crew. PostToolUse spec hooks still see the title, because KAS'stool_callframes carry no tool id. That gap is recorded on #13926.Testing
test/test_spec_hook_kas_tool_names.py(new) covers the table, the wire read (missing, malformed and trailing-newline ids read as empty), conversion (every matcher kept; one the table does not know is warned about, and still meets the KAS id it names), and store matching (the title is never matched). Through_run_chat: on KAS, anexecute_bashspecpreToolUseexit 2 rejects therun_commandcall; anfs_writematcher does not fire and the call is approved; a kiro-cli session fires no spec hook._firepassing no translated names, the KAS block test fails.test/test_crew_fired_spec_hooks.pywas updated to kiro-cli matcher names (execute_bash,exec*). The related suites pass:test_kas_permissions,test_script_hooks,test_dashboard_approval,test_hook_matcher_modes,test_agent_spec_fields_doc(439 passed).Live KAS probe (kiro-cli 2.24.1,
acp --agent-engine v3). It uses Crew's ownbuild_permission_event,spec_script_hooksandScriptHookStore.fire, and a spec hook that exits 2:The same probe, with every call allowed, asked KAS to use each built-in tool once. These are the
toolIdvalues KAS stated on the permission requests, which is where every table row comes from:Pattern harvest
Rule candidate: review-prompt
Pattern: a matcher or allowlist written in one harness's tool vocabulary is compared with another harness's identity (or with a display title) without a translation, so it silently never matches.