Skip to content

fix(messaging): steer the in-band host-deny notice before every reject - #15083

Merged
bolichen97 merged 1 commit into
mainfrom
fix/deny-notice-messaging-5403
Sep 30, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/deny-notice-messaging-5403

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On the messaging surfaces -- the native Slack handler, the channel agent stream and the channel-neutral messaging.TurnDriver -- a host block on a tool call tells the model in the same turn that the HOST refused it, and why, instead of the bare "User denied tool execution".

Bug: issue #5403 lists the reject_tool( clusters outside the dashboard chat runner; slice 1 (#14141) covered llm_helpers.py. The messaging surfaces named next in the issue still answer most host denies with reject_tool alone: the Slack handler's PreToolUse hook deny (slack/handler.py), the channel agent's blocked messaging tool, PreToolUse gate deny, over-long approval card and expired card (channel.py), and the TurnDriver's deny-every-tool switch and gate deny (messaging/driver.py). A person watching the Slack thread or the channel sees the reply silently give up, because the model read a refusal nobody made. Only the approval-timeout arms on Slack and in the TurnDriver already steer the shared notice from #10537.

Why it matters

These are the surfaces where a human is watching: a Slack thread or a channel conversation. A model that reads a host block as the person's Deny stops, apologises to the reader for a cancellation they never made, or asks whether to retry. With the notice it learns which rule (or which surface constraint) refused the call and can post to the channel instead, pick an allowed alternative, or split an over-long request.

Not a goal

  • The remaining reject_tool( clusters listed in In-band deny notice: host denies on surfaces other than the chat runner still say "user denied" #5403 (cli_chat.py, task_executor.py, task_planner.py, subagent.py, eval/*, dashboard/handlers/*, agent_runner.py) -- later slices; the issue stays open.
  • Steering a genuine user rejection: the Slack Deny click in handle_interaction, the channel reader's Deny on the approval card, and a human's Deny through the TurnDriver's decider stay bare; a person said no and kiro-cli's wording is correct there.
  • Steering teardown cleanup: _reject_orphaned_tool and the cancellation-time orphan rejects answer a wire whose own reject was skipped; they are not decisions of their own.
  • Any change to what is approved or denied; only what the model is told about a deny changes.
  • Threading a reason through the Discord transport's inline _tool_gate copy (it stays a plain callable and gets the notice naming the gate).

What changed (motivation → approach → change)

Symptom: a provider double with supports_refusal_steer=True driven through each host-deny branch of slack.handler.handle_message, channel._stream_task and TurnDriver.run on main records ['reject'] and no steer (the new test file is red on base: 31 failed / 13 passed).

Root cause: each site answers the deny straight on the wire; the shared in-band notice helper is never called there, and on Slack and in the TurnDriver the SEL audit row was written after the reject.

Change:

  • A thin _steer_host_deny(..., reason, *, cause, audited) on Slack (module-level in slack/handler.py) and the TurnDriver (a method in messaging/driver.py) redacts the possibly agent-authored reason and forwards to kiro_crew.deny_notice.steer_refusal_notice; channel.py reuses llm_helpers._steer_host_deny (which gains an optional title keyword for the channel's event.text rendering) rather than a fourth copy. It builds no text and sends nothing itself, so the "two spellings of the bounded steer" pin in test_approval_timeout_cause.py still holds. cause is a required keyword and every site names it. The Slack approval-timeout arm and the TurnDriver's _steer_deny_cause now go through the same helper instead of inlining the cancellation handling.
  • Every reject_tool( on the three surfaces has a verdict, enforced by the new source guard:
surface site verdict cause reason steered
slack handle_message: hook TOOL_DENY host POLICY tool_result.reason
slack _request_approval: prompt expired host APPROVAL_TIMEOUT unanswered for Ns (unchanged)
slack _reject_orphaned_tool cleanup -- not steered
slack handle_interaction: Deny click user -- not steered
channel blocked direct-to-user messaging tool host SURFACE_POLICY channel agents reach people through channel posts
channel PreToolUse gate deny host POLICY the gate's own reason
channel approval card over _APPROVAL_FIELD_MAX_CHARS host new APPROVAL_OVERSIZE N characters vs the limit
channel approval card: expired / reader's Deny mixed APPROVAL_TIMEOUT under the timeout flag only card unanswered for 3600s; a Deny is not steered
driver deny_all_tools (untrusted sender) host SURFACE_POLICY this channel denies every tool for a non-operator sender
driver tool_gate deny host POLICY the gate's last_deny_reason, else "blocked by the PreToolUse security gate"
driver decider said no mixed APPROVAL_TIMEOUT via _steer_deny_cause (unchanged) a human's Deny is not steered
  • New cause DENY_CAUSE_APPROVAL_OVERSIZE (constants.py; wording in deny_notice._DENY_CAUSE_TEXT; not re-exported by dashboard/state.py -- nothing on the dashboard reads it): a channel approval card refused because the request is too long to show in full. Nothing about the action was judged and the fix is in the model's hands (split the request into steps that fit), so it is kept apart from APPROVAL_UNDELIVERABLE, whose guidance is to state the permission needed, and from POLICY, whose remediation would name a sanctioned alternative to an action nobody refused.
  • The channel approval wait names its timeout (_APPROVAL_TIMEOUT_SECS = 3600) and records _approval_timed_out beside the collapsed decision = "rejected", so the one reject line can steer the host's timeout and stay bare for the reader's Deny.
  • messaging.dispatch.build_tool_gate leaves the hook's reason on the gate as last_deny_reason (set on every call, "" on allow) -- the same attribute-on-a-callable shape as ApprovalDecider.last_deny_cause. The TurnDriver reads it with getattr, so a plain callable (tests, the Discord inline copy) still works and gets a notice naming the gate.
  • Audit-first at every newly covered site: the SEL row is written BEFORE the steer and the reject (Slack hook deny, both TurnDriver sites; channel.py already audited first), matching the chat runner and llm_helpers.
  • Cancel-safe: a CancelledError out of the steer schedules the reject as a strongly referenced task and re-raises. Slack and the TurnDriver keep their existing shielded-await shape inside the helper, and that orphan reject audits ONLY when audited=False (the caller audits after the wire: the Slack approval-timeout arm, the driver's decider path); an audit-first site (audited=True) already has its SEL row, and the ledger is append-only, so a second row for one decision would be a duplicate nothing reconciles. channel.py uses llm_helpers' arm (re-raise at once, _orphan_reject_done retires the task and reads its outcome; never audits, every channel site is audit-first).
  • docs/system-specs/common/injected-messages.md: the messaging surfaces' per-site causes, the approval_oversize cause, the mixed sites, and the per-surface source guard.

A backend without supports_refusal_steer behaves byte-identically to before.

Backwards compatibility

Compatible: no verdict changes; a deny is still a deny and a user rejection is still bare. The only new observables are one extra in-band steer on steer-capable backends immediately before an existing reject_tool, and a last_deny_reason attribute on the gate build_tool_gate returns (read with getattr, never required). DENY_CAUSE_APPROVAL_OVERSIZE is an added constant; no existing cause is renamed or removed. The TurnDriver's cancellation-time SEL row now names the cause (reason=<cause>_cancelled_mid_steer) instead of hardcoding approval_timeout.

Tests

test/test_messaging_deny_notice.py (51 tests; 31 red on base, the 13 green ones are the user-rejection, no-steer and already-shipped assertions):

  • Source-level guard per surface: every await <x>.reject_tool( in slack/handler.py, channel.py and messaging/driver.py must match exactly one enumerated fingerprint carrying a verdict (host / user / cleanup / mixed); the scan is cross-checked against a textual count (the non-awaited orphan spellings are pinned to one occurrence each); every host deny has the surface's steer within the window; user rejections and cleanup have none; a mixed site's steer sits directly under its host guard (if _approval_timed_out:; the decider path pins _steer_deny_cause's cause gate); every steered site with audit-first audits before the steer; every steer names cause=DENY_CAUSE_* or forwards cause=cause; each helper only forwards to steer_refusal_notice with a default-less cause.
  • Behavioural, one per host-deny reason, driving the real handle_message, _stream_task and TurnDriver.run with a provider double recording steer/reject order and checking the notice names "User denied tool execution", "NOT a user action", the site's reason and the cause wording (surface-policy, oversize and timeout notices carry no "allowed alternative" guidance; policy notices carry the policy clause).
  • The channel reader's Deny and a human's Deny through the decider record ['reject'] only; supports_refusal_steer=False records ['reject'] only on all three surfaces; an AWS-key-shaped secret in a hook reason does not reach the model.
  • Audit order: audit, steer, reject on Slack, channel and driver.
  • Cancellation: a steer() that never returns is cancelled and the reject still reaches the wire on Slack, channel and driver, with the orphan set emptied; an audited site writes NO second SEL row, an after-wire site writes exactly one; the source guard pins audited= per site against the measured audit order.
  • build_tool_gate sets last_deny_reason on deny and clears it on allow.
  • The new cause is in the shared table, keeps the invariant half, tells the model to split, and offers no remediation.

Mutation-verified locally: dropping the channel gate steer fails the guard and the behavioural test; steering the reader's Deny too fails the mixed-site guard and the human-deny test.

Sibling files on the same paths, run by CI: test_refusal_inband_notice.py (its _DENY_CAUSE_TEXT iteration covers the new cause), test_approval_timeout_cause.py, test_deny_audit_first.py, test_llm_helpers_deny_notice.py, test_messaging_driver.py, test_messaging_dispatch.py, test_messaging_import_purity.py, test_channel_blocked_tools.py, test_channel_approval_meta.py, test_slack_handler_more_coverage.py, test_transport_permission_floor.py, test_discord.py.

Manual verification

N/A -- unit coverage sufficient: the behaviour is the order and content of two awaits on a provider double; the source guard pins every site on the three surfaces and the behavioural tests drive the real handlers.

Screenshots / video

N/A -- backend only, no user-visible frontend surface touched.

Related Issues

Fixes the messaging-surface slice of #5403 (slice 2; the issue stays open for the remaining clusters listed under Not a goal). Follows #14141 (slice 1) and builds on #10537 (deny_notice.steer_refusal_notice).

Pattern harvest

Rule candidate: review-prompt
Pattern: a single reject_tool line that answers both a host decline (a timed-out prompt collapsed into decision = "rejected") and a human's Deny hides a per-site verdict; record the provenance beside the collapsed decision and gate the notice on it, and have the source guard pin the steer under that gate.
Rule candidate: review-prompt
Pattern: when a gate/decider callable returns only a verdict string and the caller needs the reason, carry it as an attribute on the callable (last_deny_reason, like last_deny_cause) read with getattr, so plain callables in tests and sibling transports keep working.
Not generalizable: the per-site reason strings; they name these surfaces' constraints.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@CrysisDeu
CrysisDeu requested a review from a team as a code owner September 29, 2026 18:25
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of b8ac657080e3402f73e998e356a20ab58426ba30 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] b8ac657

Description read

[DESCRIPTION-READ] 1ab4b25785c968a1f7c71f27995ad11fbaf3c421f534931a9723694232610d8f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of b8ac657080e3402f73e998e356a20ab58426ba30 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the base-red claim ("31 failed / 13 passed") by running test/test_messaging_deny_notice.py on base — the description's reproduction, unrunnable here.

Not justified as shipped

  • Item 9 — rides along: the reader's oversize refusal now quotes len(sanitized_name) instead of len(_card_name) — a byproduct correction (the guard at channel.py:1460 measures the redacted title), not the steer fix itself.

What this change ships

Inventory (10 items) — 9 justified

Intent: FIX — issue #5403 slice: host denies on the messaging surfaces tell the model the host refused, not the person.

  1. Slack hook deny steers the hook's reason before rejecting — justified
  2. Channel's blocked direct-messaging tool steers a surface-policy notice — justified
  3. Channel PreToolUse gate deny steers the gate's reason — justified
  4. Over-long channel approval card steers a new "split the request" notice — justified
  5. Expired channel card now distinguished from a reader's Deny and steered as a host timeout — justified
  6. TurnDriver deny-all and gate denies steered; gate carries last_deny_reason (1 consumer counted: driver.py:814, grepped last_deny_reason) — justified
  7. New approval_oversize deny cause with its wording (1 constructing consumer: channel.py:1500) — justified
  8. SEL row now written before steer and reject on Slack hook deny and both TurnDriver sites — justified
  9. Reader's oversize message quotes the measured (redacted) length — rides along: byproduct fix declared only in a diff comment
  10. Cancellation orphan-reject audits only where the caller audits after the wire (audit/audited flags) — justified

[FIRST-PRINCIPLES-REVIEWED] b8ac657

Description read

[DESCRIPTION-READ] 744a9912c988512c83b9edfeda8a3f39d5b01c937bc4c0e434fcad7f329c7475

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of b8ac657080e3402f73e998e356a20ab58426ba30 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b8ac657

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b8ac657080e3402f73e998e356a20ab58426ba30: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed b8ac657080e3402f73e998e356a20ab58426ba30 — this comment is updated in place on each push.

Review details

Nothing blocks; two advisory findings on the new deny-notice wording.

FINDING — src/kiro_crew/channel.py:1498 — the oversize refusal reports len(sanitized_name) (the card name, which for a shell call is _APPROVAL_SHELL_TITLE_PREFIX/"Shell command (exact text unverified): " + the command) while naming the subject this {_what} = "this command", and DENY_CAUSE_APPROVAL_OVERSIZE's guidance then tells the model to "split the request into shorter steps that each fit it", so a 495-char command is reported as 504 characters against a stated limit of 500 and an obedient 500-char resubmission is refused identically (same in the reader-facing post at channel.py:1482) → Fix: report the length of the subject named — compare/quote len(_safe_cmd) against _APPROVAL_FIELD_MAX_CHARS - len(prefix), or keep measuring the card name and word both notices as "this approval card".

FINDING — src/kiro_crew/channel.py:1250 — the blocked-messaging-tool deny reuses DENY_CAUSE_SURFACE_POLICY, whose guidance asserts "the reason above says what this surface permits -- nothing at all, or only calls provably read-only", which is false at this site: _blocked_tool_named refuses one tool class only and every other tool still reaches the gate and the approval card below, so the host injects an untrue statement about the surface's tool policy into the running turn → Fix: give this site its own cause in constants.py/deny_notice.py (both already touched) whose guidance says only direct-to-user messaging tools are refused and a channel post is the way forward.

[OPUS-REVIEWED] b8ac657

Verdict parsed from the review's SHA-scoped output markers for commit b8ac657080e3402f73e998e356a20ab58426ba30.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b8ac657080e3402f73e998e356a20ab58426ba30: <one-sentence reason>

@CrysisDeu
CrysisDeu force-pushed the fix/deny-notice-messaging-5403 branch from d4ac903 to e6d9d07 Compare September 29, 2026 19:26
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@CrysisDeu
CrysisDeu force-pushed the fix/deny-notice-messaging-5403 branch from e6d9d07 to 866ccb8 Compare September 29, 2026 20:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 29, 2026
#5403 slice 2)

Every host deny on the three messaging surfaces -- the native Slack
handler, the channel agent stream and the channel-neutral TurnDriver --
now awaits the shared deny_notice.steer_refusal_notice through a thin
_steer_host_deny (which redacts the reason; Slack and the driver carry
their own because their cancellation arms differ, the channel reuses
llm_helpers' with the rendered tool name as title) immediately
before reject_tool, so a steer-capable backend learns in the same turn
that the HOST blocked the call instead of reading kiro-cli's "User
denied tool execution". supports_refusal_steer False is byte-identical
to before.

Per site, with a verdict for every reject_tool on the three surfaces:

- slack/handler.py: the PreToolUse hook's deny on the message path is
  DENY_CAUSE_POLICY with the hook's reason; the expired approval prompt
  keeps DENY_CAUSE_APPROVAL_TIMEOUT and now goes through the same helper.
  A Deny click in handle_interaction is a genuine user rejection and the
  teardown-only _reject_orphaned_tool is cleanup: both stay bare.
- channel.py: the containment boundary refusing a direct-to-user
  messaging tool is DENY_CAUSE_SURFACE_POLICY (the model's way forward is
  a channel post); the PreToolUse gate's deny is DENY_CAUSE_POLICY with
  the gate's reason; a card the channel cannot show in full is the new
  DENY_CAUSE_APPROVAL_OVERSIZE, whose notice tells the model to split
  the request -- kept apart from approval_undeliverable, whose guidance
  is to state the permission needed; a card that expires unanswered is
  DENY_CAUSE_APPROVAL_TIMEOUT. The reader's Deny shares that reject
  line and sends no notice: the steer sits under the timeout flag alone.
- messaging/driver.py: the deny-every-tool switch for a sender other
  than the operator is DENY_CAUSE_SURFACE_POLICY; the gate's deny is
  DENY_CAUSE_POLICY. build_tool_gate leaves the hook's reason on the gate
  as last_deny_reason (the ApprovalDecider.last_deny_cause shape) so the
  notice names the rule; a plain callable gets a notice naming the gate.
  The decider path is unchanged: a human's Deny stays bare.

The SEL audit row precedes the steer and the reject at every newly
covered site (audit-first, as the chat runner and llm_helpers do).
Cancellation mid-steer schedules a strongly referenced reject_tool so a
stranded permission cannot wedge the backend; that orphan reject audits
only where the caller audits after the wire (audited=False on the Slack
timeout arm and the driver's decider path), so an audit-first site never
gains a second SEL row for one decision.

Tests: a source-level guard per surface that enumerates every
reject_tool( with its verdict (host / user / cleanup / mixed) and fails
on a missing site, a bare host deny, a steered user rejection, or a
mixed site whose steer is not under its host guard; one behavioural
test per deny reason with a provider double recording steer/reject
order; audit-order, capability-gate, redaction and cancellation tests.

Fixes the messaging-surface slice of #5403.
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 30, 2026
@CrysisDeu
CrysisDeu force-pushed the fix/deny-notice-messaging-5403 branch from 866ccb8 to b8ac657 Compare September 30, 2026 03:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 30, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on behalf of the maintainers (sweep 2026-09-30): readiness passed, required checks green on b8ac657, no blocking review threads, security posture unchanged. Approving and merging.

@bolichen97
bolichen97 merged commit 9295deb into main Sep 30, 2026
97 checks passed
@bolichen97
bolichen97 deleted the fix/deny-notice-messaging-5403 branch September 30, 2026 07:28
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants