Skip to content

fix: refuse person-only directives on task runner turns - #36792

Merged
bolichen97 merged 1 commit into
mainfrom
fix/34756-task-runner-directive-refusal
Nov 17, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/34756-task-runner-directive-refusal

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A person-only directive (set_project, reset_conversation, chat_tag, suggest_followup) that a task runner turn calls is refused out loud and audited, the way cron, workflow, app and completion turns already refuse it.

Why it matters

A task runner step, its self-review, its decomposition turn and its lesson turn can call set_project today and get only the tool's own "requested" text back: nothing is changed, nothing says so, and no SEL denied row records the call. Someone reading the task result or the audit log sees a call that looks like it worked.

Not a goal

  • Applying any of these directives on a task runner turn: a task runner turn is not a person's turn.
  • Telling the agent mid-turn that its call was refused (chat_tag: report a user-origin gate refusal to the agent #36233 asks for that on session turns).
  • Changing the refusal wording, the set of person-only directives, or any other kind of turn.

What changed (motivation → approach → change)

A person-only directive call on a task runner turn (any of USER_ORIGIN_DIRECTIVES: set_project, reset_conversation, chat_tag, chat_rename, suggest_followup) is refused and recorded instead of silently doing nothing: the step's result starts with the refusal, and the SEL log gets a denied row on that turn's key. Cron, workflow, app and completion turns already behave this way.

Symptom: such a call ends with only the tool's "requested" text, and the SEL log has no denied row. Cause: no session applier reads a task runner turn, and unlike the other headless turns, none of the four task runner turns passed its stream to the existing person_only_directive_refuser. Each one does now, on its own session key.

Turn Where What the refusal does
Step (execute_task) task_executor.py Every attempt's refusals lead task.result (joined with a blank line), plus a SEL denied row on taskrunner:{id}:task{N}
Self-review (self_review) task_executor.py SEL denied row on taskrunner:{id}:review; the JSON text is left alone
Lesson (_call_llm_for_lesson) taskrunner_parts/run_reporting.py SEL denied row on the lesson key; the JSON text is left alone
Decomposition (decompose) task_planner.py SEL denied row on taskrunner:{id}:decompose; the JSON text is left alone

The three JSON turns leave their text alone because a refusal line ahead of the JSON would break the parse. Their refusal lives only in the SEL row, as it does on a workflow schema= step. To reach the self-review and lesson turns, stream_and_collect_json (src/kiro_crew/llm_helpers_parts/turn_stream.py) now accepts an on_event keyword (default None) and passes it to stream_and_collect. Existing callers are unchanged. The refuser is imported inside each function, as at every other call site. There is no new rule, wording, directive set or config. docs/system-specs/modules/taskrunner.md gets one paragraph under Tool Approval.

Backwards compatibility

Compatible: the on_event keyword defaults to None. A turn that calls no person-only directive produces a byte-identical result and writes no new SEL row (smoke happy path below).
Removes nothing: the only things added are a refusal line on the step result and a denied audit row, and only when such a call happens. Nothing is hidden, deleted, renamed or tightened.

Tests

  • test/test_taskrunner_deny_notice.py::test_a_step_turns_person_only_directive_leads_the_result runs the real execute_task with a provider that completes a set_project call. task.result starts with the user-origin refusal (naming the turn's key) and ends with the turn's text. A synchronous SEL under tmp_path holds exactly one mcp-directive denied row on the step's key.
  • test/test_taskrunner_deny_notice.py::test_a_json_turns_person_only_directive_is_audited_and_parses[self_review|lesson|decompose] runs the real JSON turns. Each one's JSON still parses to the same verdict, lesson or plan, and exactly one denied row lands on that turn's own key.
  • test/test_taskrunner_v2_scenarios.py: two stream_and_collect_json fakes now accept keywords.

Proof: PROVEN test/test_taskrunner_deny_notice.py::test_a_step_turns_person_only_directive_leads_the_result, test/test_taskrunner_deny_notice.py::test_a_json_turns_person_only_directive_is_audited_and_parses[self_review], test/test_taskrunner_deny_notice.py::test_a_json_turns_person_only_directive_is_audited_and_parses[lesson], test/test_taskrunner_deny_notice.py::test_a_json_turns_person_only_directive_is_audited_and_parses[decompose] (head 15df7b4)

Determinism (testing-conventions D12): the new tests passed 20/20 at -n0, then 10/10 at -n 4 --dist loadgroup --max-worker-restart=2 beside test_task_planner_decompose_gate.py and test_taskrunner_v2_scenarios.py. The whole file also passed in a shuffled order (seed 17641, 28 passed).

Manual verification

How it was verified, on Linux (Ubuntu 24.04.5 LTS, x86_64). This host has no signed-in agent backend, so the exact code path was driven instead. The script calls the real execute_task, self_review, _call_llm_for_lesson and task_planner.decompose under an isolated KIROCREW_HOME=$(mktemp -d). Each gets a scripted provider whose stream() yields the frames a real set_project call produces: a tool call on the core MCP server, its final result, text, then completion. The script then reads the real security_events.jsonl written under that home. Only the session manager, the prompt builder, the context check and the config load are doubles.

Before (base, fix not applied):

"execute_task": {"passed": true, "task.result": "Step done."}
"sel_mcp_directive_rows": []

After (this branch):

"execute_task": {
  "passed": true,
  "task.result": "Error: set_project only works from a user-facing session (dashboard or a messaging channel); headless callers such as cron jobs and sub-agents are refused (this turn is 'taskrunner:repro:task1'). Nothing was changed.\n\nStep done."
},
"self_review": {"passed": true, "task.error": ""},
"lesson": {"rule": "r"},
"decompose": [[1, "A"]],
"sel_mcp_directive_rows": [
  ["taskrunner:repro:task1", "set_project", "denied"],
  ["taskrunner:repro:review", "set_project", "denied"],
  ["taskrunner:repro:runtime:lesson", "set_project", "denied"],
  ["taskrunner:repro:decompose", "set_project", "denied"]
]

Happy path, the same script with no directive call: task.result is "Step done." both before and after, the JSON turns parse the same, and the SEL log has no mcp-directive rows (0 rows in total).

Related Issues

Closes #34756

Sibling sweep

Swept: git grep -n "stream_and_collect|.stream(|continue_timed_out_calls" -- src/kiro_crew/task_executor.py src/kiro_crew/task_planner.py src/kiro_crew/taskrunner_parts/ src/kiro_crew/taskrunner.py
Swept: git grep -ln "taskrunner" -- src/kiro_crew | xargs grep -ln "stream_and_collect|run_bg_oneliner|.stream("
Sibling: src/kiro_crew/task_executor.py:961 -- changed here
Sibling: src/kiro_crew/taskrunner_parts/run_reporting.py:236 -- changed here
Sibling: src/kiro_crew/task_planner.py:350 -- changed here
task_executor.py holds two of the four sites: the step turn (line 961) and self_review (line 1741).

Sibling: src/kiro_crew/dashboard/handlers/taskrunner.py -- not affected -- _run_refine is the spec-writer turn a person starts from the dashboard, not a headless task runner turn, so the headless refusal this fix adds does not describe it

Pattern harvest

Rule candidate: review-prompt
Pattern: "a headless turn that streams a provider itself (client.stream, stream_and_collect[_json]) without person_only_directive_refuser, where no session applier reads the stream"

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision b0cce125e88844cda0d99df7f8b579f8efb47dd5 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of b0cce125e88844cda0d99df7f8b579f8efb47dd5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This reuses the existing person_only_directive_refuser on all four task runner turns, with a linked issue (#34756) and tests that fail on base. Nothing blocks.

Notes

  • Item 6 — symptom-level: each headless stream must remember to opt in to the refuser. With this PR it is wired at 10 call sites (grep person_only_directive_refuser). Grepping stream_and_collect_json( turns up one more headless JSON turn that is still not wired: history_consolidation_pass.py:1639. The PR's "Not a goal" excludes "any other kind of turn", so this sibling is deferred, not a demand.

What this change ships

Inventory (7 items) — 7 justified

Intent: when a task runner turn calls a person-only directive, refuse it visibly and audit it, the same way other headless turns already do. FIX.

  1. A step result now starts with the refusal text when the step called a person-only directive — justified
  2. A SEL denied row is written on the step's key — justified
  3. A SEL denied row is written on the self-review key, and the JSON is left untouched — justified
  4. A SEL denied row is written on the lesson key, and the JSON is left untouched — justified
  5. A SEL denied row is written on the decompose key, and the JSON is left untouched — justified
  6. stream_and_collect_json accepts an on_event keyword that defaults to None — justified
  7. The taskrunner spec gets one paragraph describing the refusal — justified

[FIRST-PRINCIPLES-REVIEWED] b0cce12

Description read

[DESCRIPTION-READ] 48190ed6856aa66952e5bec675adea961270509e52562f920b5e293fbb77b8fb

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of b0cce125e88844cda0d99df7f8b579f8efb47dd5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix wires the existing person_only_directive_refuser into the four task runner turns that skipped it. It adds no new rule, wording or surface, and on_event is purely additive.

Notes

  • The description's sibling sweep only covers task runner streams. The "Pattern harvest" rule candidate states the general class. Turning it into a source-level guard would catch the next headless client.stream caller that skips the refuser, instead of waiting for the next issue.

[DESIGN-REVIEWED] b0cce12

Description read

[DESCRIPTION-READ] d90d5137547c38f853c17f60afacaaec6907dafcb58fdf6b4cd73ec74a3936f4

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b0cce125e88844cda0d99df7f8b579f8efb47dd5 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b0cce12

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b0cce125e88844cda0d99df7f8b579f8efb47dd5 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b0cce125e88844cda0d99df7f8b579f8efb47dd5 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] b0cce12

Verdict parsed from the review's SHA-scoped output markers for commit b0cce125e88844cda0d99df7f8b579f8efb47dd5.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b0cce125e88844cda0d99df7f8b579f8efb47dd5 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 13, 2026 18:20
@KiroCrewCaptain

KiroCrewCaptain commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 7% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier pending: being reviewed.

This PR wires the existing person_only_directive_refuser into all four task runner turns: the step, self-review, lesson and decomposition turns. Each uses its own session key. The step turn puts the refusal text ahead of task.result, while the three JSON turns record the refusal only in a SEL denied row, so their JSON still parses. The new on_event keyword on stream_and_collect_json defaults to None and is passed straight to stream_and_collect, which already accepts it. SEL write failures are logged and ignored in _audit, and no code parses the start of task.result. Every existing fake of stream_and_collect_json in the tests still works with the new keyword. The new tests run the real functions, assert the exact SEL rows and would fail on the base branch. CI is green. No regression or trust-boundary problem was found.

Tier: pending: being reviewed.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit b0cce125e8 · 170 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Nov 13, 2026
No session applier reads a task runner turn, so a person-only directive
(a USER_ORIGIN_DIRECTIVES call such as set_project or chat_tag) made by
a step, its self-review, the decomposition turn or the lesson turn was
neither applied nor refused, and no SEL denied row recorded it. The
task result could only echo the tool's own "requested" text.

Each of those turns now observes its stream with
person_only_directive_refuser on its own session key, as the cron,
workflow, app and completion turns already do. The step turn leads
task.result with every attempt's refusals; the JSON turns keep their
text for the parse, so the refusal lives in its SEL row.
stream_and_collect_json gains a pass-through on_event keyword for that.

Fixes #34756

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/system-specs/modules/taskrunner.md, src/kiro_crew/task_executor.py, src/kiro_crew/taskrunner_parts/run_reporting.py and test/test_taskrunner_deny_notice.py since this head's CI ran (c22557c -> 226c9d6); CI re-running.

Why a re-sync at all: the only red on the old head that was not infrastructure was Coverage Gate, on src/kiro_crew/agent_ipc_client.py (75.7%, 87/115 lines), a file this PR does not touch. It sits under the 80% per-file floor on the base this PR was cut from. Its fix (#36804, tests only) merged into main after that base, and main's own CI is green on a later main commit (c212328). This rebase picks it up.

Main touched four of this PR's files, all outside its hunks: #36361 and #35183 edited other paragraphs of the task runner spec, #31863 swapped a fixture token in an unrelated test of the deny-notice file, #37578 and #35011 changed lines of task_executor.py away from this PR's lines, and #37218 added a memory-provider branch to _extract_lesson, above the lesson turn this PR changes. The code patch is unchanged apart from hunk offsets. One spec sentence changed: main's #33736 added chat_rename to USER_ORIGIN_DIRECTIVES after this PR was written, so the paragraph this PR adds to the task runner spec now names that set instead of listing four tools. The shared refuser already refuses every member of the set, chat_rename included. The body's line pointers moved with the rebase: self_review 1738 -> 1741, the lesson turn 214 -> 236.

The other three reds on the old head were infrastructure: Analyze (actions) and Publish readiness signal both ended on API rate limit exceeded for installation, and GPT 6.1 Review ended with "the pass-2 verdict was not published because both calls are required". All three re-run on this head.

[operator: bolichen97]

@bolichen97
bolichen97 force-pushed the fix/34756-task-runner-directive-refusal branch from 15df7b4 to b0cce12 Compare November 17, 2026 17:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 17, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. On head b0cce12, the review confirmed four things. The shared person_only_directive_refuser is now wired into all four task runner turns, each with its own session key: the step turn, decompose, self review and the lesson call. The refusal list is created once, before the retry loop, and leads task.result. A turn with no directive call keeps its result byte-identical. The JSON turns still parse to the same verdict, lesson or plan. CI and PR Readiness are green on this head.

Done well: the fix reuses the one existing refuser instead of copying it. The tests drive the real turn functions against a real synchronous security event log, and each one pins the exact (key, tool, outcome) row.

Optional:

  • task_executor.py:967: the every-50-chunks refresh of task.result leaves out refusals. A step that gives up after repeated process deaths keeps the audit row, but its result has no refusal line. Building the refresh from the same "\n\n".join([*refusals, result_prefix + result_text]) expression would keep both writers in step.
  • No test drives a retry. A case where the first attempt calls set_project and then retries, with an assertion that the refusal still leads task.result, would pin the "every attempt" promise in the taskrunner spec.
  • The Tests section's Proof line names head 15df7b4, not b0cce12. The production and test hunks are the same, so this is wording only.

@bolichen97
bolichen97 merged commit 7409dba into main Nov 17, 2026
100 checks passed
@bolichen97
bolichen97 deleted the fix/34756-task-runner-directive-refusal branch November 17, 2026 19:07
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Person-only directives are not refused on task runner turns

2 participants