Repository navigation
feat(decisions): let Jev flag risky tool calls on the tool card - #12253
Conversation
UX Review (Fable 5) — ✅ human override accepted@iamwhatever overrode this lane for |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Description contradicts HEAD's consent design: it says tool-argument egress rides existing consent with widening deferred, but commit 2 added a separate consent gate. WatchThe PR body's "Known, and deliberately not in this PR" section and the "New routes / knobs: none" row describe commit 1; HEAD adds [DESIGN-REVIEWED] a8fe2f0 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 7 justifiedIntent: let a person whose session auto-approves its tool calls see which calls Jev flagged as risky, without changing any permission — an ADDITION, off by default behind two consents.
[FIRST-PRINCIPLES-REVIEWED] a8fe2f0 |
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @iamwhatever overrides the GPT 5.6 finding for This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
[operator: iamwhatever] Screenshot evidence for the tool risk badge. Captured by Three tool cards in one shot, because the badge's claim is comparative: a read the seam answered |
|
[operator: iamwhatever] Disposition — GPT 5.6 Review on No rebuttals this round. Each one named something real, and two of them named something I had written down as a guarantee. 1. Correct, and the module disagreed with itself about it. Fixed as the gate does it for the same write on the same loop, rather than with a bare written = await asyncio.wait_for(asyncio.to_thread(_log.append, row), LOG_BUDGET_SECS)
2. Both clauses are right, and this is the finding I am most glad to have. "The call is neither delayed" — false. The harness sends the "A flagged call still runs" — false. A host gate, a Corrected in five places, not one: the point's module docstring, And the corrected sentence is now checkable rather than merely rewritten. 3. Right on the mechanism: with membership plus inequality, a third point keeps both assertions true and the registry grows with nobody asked. I had read the test by its name and docstring ("is a point the backend ships") and missed that its shape was doing a second job — guarding the list that decides what may send conversation text off the machine. Now Still open, and named in the PR body rather than fixed here: the consent card's Not mine, unchanged: |
3e46831 to
aa0e60c
Compare
Human judgment recorded@iamwhatever marked the ux AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
…badge claiming the call ran
Settles the two blockers the review lanes upheld.
1. TOOL-ARGUMENT CONSENT. `skills.select` sends a message excerpt and candidate
descriptions; `tool.risk` adds the tool's name and its arguments. An owner who
consented before this point existed consented to the former, so reading their
keystone as permission for the latter widens egress with no new choice -- the
argument that already gave `history_budget_chars` a keystone ceiling instead of
a config value. It now takes a SECOND consent, built on that same pattern:
- `consent.STATE_KEY_TOOL_ARGS` / `consented_tool_args`, where only a literal
`true` consents and ABSENT reads as not consented, so every record written
before the key existed keeps meaning what its owner agreed to.
- `save_enabled(tool_args=...)` writes it on enable and clears it on disable;
`KEEP_TOOL_ARGS` preserves a recorded scope and is resolved inside the same
lock the ceiling uses, so an enabling PUT cannot restore a scope a concurrent
revoking PUT already cleared.
- `gate._tool_args_scoped` refuses any point in `POINTS_NEEDING_TOOL_ARGS`,
checked inside `_consented_for` -- the chokepoint every `decide` and
`is_enabled` path funnels its one keystone read through -- so the scope costs
no second read and is not enforceable in the preflight alone. Warned once per
point, because an owner whose feature does nothing must be able to tell that
from a broken build.
- `PUT /api/decisions/consent` takes `tool_args`, absent PRESERVING the recorded
value and a non-boolean refused with 400 rather than coerced; the GET reports
it. No new endpoint.
- The card grows a second switch under the main one, visible only while it is
on. The main switch deliberately sends no `tool_args`, so flipping it cannot
grant or erase a scope the owner did not touch.
The consequence is the point of it: an already-consented install is INERT for
`tool.risk` until its owner opts in.
2. THE TOOLTIP no longer claims the call ran. A `PreToolUse` hook or a host gate
can refuse a call that already carries a note, so "It still ran" was false in
the case the badge is most likely to appear in -- and the feature-map row cited
that tooltip as carrying the correction, which made the pair worse than the
plain overclaim. It now says flagging changed nothing about whether the call was
allowed and that the call may still have been refused.
Tests: `test_decisions_tool_args_scope.py` covers the reader's default, the
writer's round trip (preserve, revoke, clear on disable, non-boolean refused), the
gate refusing the point while leaving `skills.select` untouched, `decide` sending
nothing, the once-per-point warning, and the route's preserve/revoke/sentinel
behaviour. The scope check is revert-verified: neutering it fails three of them.
The end-to-end now writes a REAL keystone instead of stubbing `_consented_for`, so
it covers the scope rather than asserting around it, and a sibling drives the
revoked state through `_run_chat` to show nothing is sent and no row is written.
fed1e43 to
a8fe2f0
Compare
|
[operator: iamwhatever] Disposition for the GPT 5.6 finding on the rebased head This head is a pure rebase of
Local verification on this head: 297 backend decisions/consent/gate tests, 130 frontend decision tests, |
Human judgment recorded@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Maintainer readiness-sweep record (agent-assisted, on behalf of @bolichen97), head a8fe2f0: verified |
…badge claiming the call ran (kirodotdev#12253) Settles the two blockers the review lanes upheld. 1. TOOL-ARGUMENT CONSENT. `skills.select` sends a message excerpt and candidate descriptions; `tool.risk` adds the tool's name and its arguments. An owner who consented before this point existed consented to the former, so reading their keystone as permission for the latter widens egress with no new choice -- the argument that already gave `history_budget_chars` a keystone ceiling instead of a config value. It now takes a SECOND consent, built on that same pattern: - `consent.STATE_KEY_TOOL_ARGS` / `consented_tool_args`, where only a literal `true` consents and ABSENT reads as not consented, so every record written before the key existed keeps meaning what its owner agreed to. - `save_enabled(tool_args=...)` writes it on enable and clears it on disable; `KEEP_TOOL_ARGS` preserves a recorded scope and is resolved inside the same lock the ceiling uses, so an enabling PUT cannot restore a scope a concurrent revoking PUT already cleared. - `gate._tool_args_scoped` refuses any point in `POINTS_NEEDING_TOOL_ARGS`, checked inside `_consented_for` -- the chokepoint every `decide` and `is_enabled` path funnels its one keystone read through -- so the scope costs no second read and is not enforceable in the preflight alone. Warned once per point, because an owner whose feature does nothing must be able to tell that from a broken build. - `PUT /api/decisions/consent` takes `tool_args`, absent PRESERVING the recorded value and a non-boolean refused with 400 rather than coerced; the GET reports it. No new endpoint. - The card grows a second switch under the main one, visible only while it is on. The main switch deliberately sends no `tool_args`, so flipping it cannot grant or erase a scope the owner did not touch. The consequence is the point of it: an already-consented install is INERT for `tool.risk` until its owner opts in. 2. THE TOOLTIP no longer claims the call ran. A `PreToolUse` hook or a host gate can refuse a call that already carries a note, so "It still ran" was false in the case the badge is most likely to appear in -- and the feature-map row cited that tooltip as carrying the correction, which made the pair worse than the plain overclaim. It now says flagging changed nothing about whether the call was allowed and that the call may still have been refused. Tests: `test_decisions_tool_args_scope.py` covers the reader's default, the writer's round trip (preserve, revoke, clear on disable, non-boolean refused), the gate refusing the point while leaving `skills.select` untouched, `decide` sending nothing, the once-per-point warning, and the route's preserve/revoke/sentinel behaviour. The scope check is revert-verified: neutering it fails three of them. The end-to-end now writes a REAL keystone instead of stubbing `_consented_for`, so it covers the scope rather than asserting around it, and a sibling drives the revoked state through `_run_chat` to show nothing is sent and no row is written. Co-authored-by: Joe Guo <zejiangg@amazon.com>






What this adds
A second point on the shipped Jev decision seam,
tool.risk, that puts a small note on a tool card — Jev: risky (0.88) — and decides nothing.In a session that approves its own tool calls (trust or YOLO) nothing stops to describe what is about to run. That is where the note lives. A session that prompts is never annotated: the person reading the call is already the annotation, and a second opinion beside a control they are operating is noise.
Screenshots
Three tool cards in one transcript, because the badge's claim is comparative: a read the seam answered
safefor — which stamps no record, so that card is exactly what every tool card looks like without this feature — then a risky tree clear and a caution config rewrite.The two badges up close, each with the pill it annotates:
riskycautionProduced by
website/scripts/capture-tool-risk-badge.mjs(new here, the sibling ofcapture-decision-strip.mjs): the real built SPA behind the shared transcript harness, every/api/**call answered from fixtures. No gateway, no agent, no Jev — soreadToolRiskRecord, the transcript virtualizer,ToolCallLineand the badge render exactly as in production, and the record arrives throughmeta.decisions_tool_riskon a tool row, the same door a scrollback read goes through.Permission behaviour is byte-identical, and that is a test
The hook sits in the
tool_callevent branch ofchat_runner— the one funnel that appends the tool card row, and a region of the file containing no approval code at all. It is deliberately not in the permission branch: anawaitbesideapprove_toolwould make "nothing about approval changed" a claim about the ordering of two adjacent statements instead of a claim about two disjoint regions.test_decisions_tool_risk_card.py::TestPermissionBehaviourIsUnchangedruns one scripted turn twice — with the point answeringriskyand with it silent — and holds theapprove_tool/reject_toolcall lists and the SEL audit rows equal. A sibling test asserts the request really was answered in both runs, so the equality cannot hold by measuring nothing.Two things this does NOT claim, because neither is true, and both are stated in the docs rather than implied away. It is not free of timing: the
tool_callframe arrives before the permission request, so awaiting the point delays when the turn's next event is read, by up to its own wait budget — bounded, sampled, paid only where the seam is on. And a badge is not a verdict on whether the call ran: a host gate, aPreToolUsehook or a batch cascade can refuse a call that already carries one, whichtest_a_badged_call_can_still_be_refuseddrives through a real turn.The shape
decisions/points/tool_risk.py, oneChoiceoversafe/caution/riskywith a one-sentence rubric per tiercautionandrisky;safeis recorded in the log and draws nothingskills.selectuses; the row is written off the loop underLOG_BUDGET_SECSside: "jev",POST /api/decisions/feedbackTwo design points worth reading, because both are the opposite of what the sibling point does:
Redaction, not refusal. The gate refuses a state carrying a credential, which is right for a message excerpt — a secret there is a finding. A credential in a tool argument is ordinary (an
awscommand, a curl header), so refusing on it would mean the seam never annotates the calls most worth annotating. The point replaces it with the shipped placeholder, and the gate then scans the placeholder. Pinned in both directions: the redacted state passesgate.scrub_reason, the raw one is refused asscrubbed:credential. This is also whytool_risk.pyis a registeredredaction_pathssink while its siblingdecisions/gate.pyis allowlisted as a non-egress detector — the gate emits nothing on a hit, this module's scrubbed bytes are what leaves the machine.safeis recorded and not badged. The log is the observation, so a log that only kept the alarming answers could not say how often the seam is wrong. A badge on every card of a sampled session would cost the flag its meaning. So the note's presence is the flag, andtoolRiskRecord.tsrefuses thesafetier too — a producer that started stampingsaferecords could not quietly badge every card.Why
policyand not a per-call verdictThe row carries
policy— the grant that answers the call (yolo/trust/trust_scope, from the shipped_auto_approve_reason) — rather than allowed/blocked/asked. It cannot carry the latter: the harness sends thetool_callnotification before it asks for permission (acp/_dispatch: "the precedingtool_callnotification"), so the card this record rides exists before any verdict about it does. A later refusal — a host gate, aPreToolUsehook, a batch cascade — is already recorded by the SEL tool-invocation audit. Guessing at it here would give the log two descriptions of one call and make the weaker one look authoritative.No new registry entry
decisions.outcomesholds one outcome per session andconsumepops it — right for a strip describing one turn, wrong for an annotation describing one of several calls inside one turn. Producer and consumer here are the same line of code, so the record is handed straight to theslot.appendthat carries it — before the append, so it reaches both doors (the livechat_messageframe and the persisted transcript line) from one write.Frontend
pages/chat/toolRiskRecord.ts— the pure reader; validates the record, refusessafe, an unknown tier, a missingturn_idand a probability outside 0..1.pages/chat/ToolRiskBadge.tsx— the row under the pill.riskyuses the warning colour, never the danger colour this transcript uses for a call that was blocked, and the tooltip says in words that the call ran anyway and that flagging it changed nothing.pages/chat/DecisionVerdictThumbs.tsx— extracted out ofDecisionStrip.tsxso both surfaces post one implementation of the verdict. The log's summary is keyed onside, and a second copy would be a second chance to send the wrong one.ToolCallLine.tsxreads the record off its ownmessage, so no host wiring changes.Tested
test/test_decisions_tool_risk.py(36) — the record, the request, every refusal, both budgets, the wait clamp held equal toskills.select's, and a source check that the point imports no dashboard or permission code.test/test_decisions_tool_risk_card.py(14) — the ride-along through the real_run_chaton both doors, the tool row rather than the reply, one record per card across two calls in one turn, the absence case, and the permission mutation check.test/test_decisions_tool_risk_end_to_end.py(2) — the whole chain with nothing stubbed between the links: real_run_chat, real gate, real scrub, real day-file, real verdict route; plus the credential case through the real scrub.website/src/test/ToolRiskBadge.test.tsx(22) andToolRiskBadgeContract.test.ts(7) — the latter parses the record fixture out of spec § 9 and asserts the reader accepts it field for field, the same waydecisionStripContract.test.tsdoes for the strip.Gates green locally: black, mypy, comment-history, subprocess-encoding, sync-io-in-async, brand, focus-cue, changelog-history, harness-parity, feature-map,
tsc,eslint, and all 19 i18n checks including the eleven per-locale style suites.Two doc sentences corrected
src/kiro_crew/docs/decisions.mdsaid "Nothing in this version displays that record" and "This version has no button for that verdict". Both were already false onmain— the strip and its thumbs shipped — and the new section directly contradicts them, so they are corrected rather than left standing beside it. The page's title and its two index rows widen from "Jev skill selection" to "Jev decisions", since the page now covers two things.Known, and deliberately not in this PR
The consent card's egress sentence does not yet mention tool arguments.
decisions_egresssays "the text of your messages and the names and descriptions of your skills leave this machine"; with this point on, a trusting session also sends a tool name and its (redacted) arguments. The spec's own rule is that consent is recorded against the text the owner reviewed, so that sentence should widen. It is one changed value across 13 locales and each one re-enters the per-locale style gates, so it is held out of this push rather than bundled into a red-clearing round — happy to land it here on request, otherwise it follows immediately.Backend Tests (Windows) (1)is main-side. Fourtest/test_acp_runtime.pyfailures ([returncode=-15]vs[returncode=<not reaped>]). That file andsrc/kiro_crew/acp/runtime.pywere both last touched by this PR's own base commit, #12005, which landed at 01:17 UTC; the Windows shard's last green run onmainpredates it. PRs #12251 and #12252 fail the same four tests with unrelated diffs. This PR touches noacp/orruntimefile.