Skip to content

feat: name program and target tokens in self-kill refusal - #15055

Closed
SaRedfiche wants to merge 1 commit into
kirodotdev:mainfrom
SaRedfiche:pipe/issue-14827-selfkill-diag-offsets
Closed

SaRedfiche wants to merge 1 commit into
kirodotdev:mainfrom
SaRedfiche:pipe/issue-14827-selfkill-diag-offsets

Conversation

@SaRedfiche

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make the self-kill refusal say which token it read as the kill program and which as the target, so a reader can tell a real kill from a false positive without running the classifier.

When the self-protection floor blocks a command, its refusal diagnostic reports only a whole-command span=0..N. That is enough to see the floor fired, but not why — a reader cannot tell whether the block was a genuine pkill kirocrew or a false positive where an innocent argument (a glob path whose basename is *, next to a path carrying the product name) happened to look like a by-name kill of this gateway.

Why it matters

The self-protection floor is meant to stop the agent killing its own supervisor, but a false positive silently blocks ordinary work with no way to diagnose it short of reading the classifier source. Naming the two token offsets turns "this was refused, and here is exactly which two words tripped it" into something a reader can act on.

Not a goal

  • No other floor's diagnostic changes — this is self-protection-kill only.
  • The floor's decision (what it blocks) is untouched; only the refusal text gains fields.
  • No command bytes are exposed — the new fields carry offsets and a char-class census, never a substring of the command.

What changed (motivation → approach → change)

The self-kill refusal reported only a whole-command span, so a false positive gave a reader no way to see which two words the floor keyed on. The fix adds two optional fields to the refusal diagnostic — program=<a>..<b> and target=<c>..<d> — each with the same char-class census the existing span= already carries.

Locating those offsets is the work. A new _raw_word_spans() splitter walks the shared quote/escape state machine to break the command into shell words at their real source offsets, keeping quoted separators inside their word and an abutting operator as its own span. _self_kill_token_spans() then aligns those raw spans to the resolved frame's tokens by index — sound because both derive from the same word split, before per-word variable and tilde resolution — so a variable-resolved token like "$C"/gateway/* is bracketed at its true position even though the product name entered it by $C expansion.

Piece What it does
argv_floor._raw_word_spans splits the command into words with source offsets
argv_floor._self_kill_token_spans returns the program/target offsets the by-name leg keyed on, or None to fall back
diagnostics.RefusalTokenSpan + refusal_diagnostic() render the two additive fields; span= and the first two refusal lines are byte-identical when they are omitted
security/__init__.py the floor loop computes the spans for self-protection-kill only and threads them through

A leg with no single-token target (bare kill $(...)) returns None and keeps the whole-command span. docs/system-specs/modules/security.md documents the token-level exception in the same change.

Backwards compatibility

Compatible: the fields are additive and omitted for every floor except self-protection-kill; the span= field and the first two refusal lines are byte-identical when they are absent, which the tests pin.

Tests

A new TestSelfProtectionKillDiagnosticSpans class (10 tests) covers all shapes: pkill -f kirocrew and killall kirocrew bracket the program and target tokens; the variable-resolved glob command brackets its real source offsets; a quoted ERE pattern brackets its raw region; kill $(pgrep -f kirocrew) and a non-self-kill fall back with no fields; every other floor's line stays byte-identical; and _refusal_diagnostic_id still reads the rule id. The denied-command count pin and the keystone test_sandbox_governance_mask continue to hold.

Manual verification

N/A — unit coverage is sufficient; the change is a diagnostic string and its offset computation, both fully exercised by the new tests against real is_denied output.

Related Issues

Fixes #14827

Pattern harvest

Not generalizable: this is a targeted diagnostic enrichment for one floor whose decision is already token-level; the offset-alignment approach is specific to the self-kill by-name leg and does not describe a defect class worth a lint or review rule.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@SaRedfiche
SaRedfiche requested a review from a team as a code owner September 29, 2026 16:58
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of 49c9558bef46ad59e3e6ef9c594ef79df2bf8e12 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Mirrored copy of _is_self_kill's by-name leg can drift silently, making the diagnostic bracket tokens the floor no longer keys on.

Watch

_self_kill_token_spans duplicates the by-name loop of _is_self_kill (trigger check, _data_consumer_exempt, substitution-depth walk, _ends_argv boundary) instead of sharing it; the PR's own claim "mirrors the by-name leg … so the two never disagree" holds only until someone edits one copy — a new exemption in the classifier that isn't mirrored yields a diagnostic pointing at tokens the floor didn't key on, misleading the exact false-positive triage this feature exists for, and no pinned test detects it because both are only exercised end-to-end on today's shapes.
Clears when: the per-token match loop is extracted into one helper returning (program_idx, target_idx) | None that both the boolean leg and the span function call (decision behavior pinned by the existing classifier suite), or a parity test asserts the mirror and the leg agree across the classifier's existing self-kill corpus.

[DESIGN-REVIEWED] 49c9558

Description read

[DESCRIPTION-READ] 5f37ab90880dd1996ddd103a639e6f7697fd4136021c4e70272234b460314db7

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 49c9558bef46ad59e3e6ef9c594ef79df2bf8e12 via the fork AI-review pipeline; updated in place on each push.

Review details

FINDING -- src/kiro_crew/security/init.py:1443 -- length-expanding Unicode in lower shifts token offsets for X=İ pkill -f kirocrew -> Fix: omit token spans when lowercasing changes length.
FINDING -- src/kiro_crew/security/argv_floor.py:1052 -- deleting len(raw_spans) != len(tokens) leaves tests green and mislabels true;pkill -f kirocrew -> Fix: pin mismatched counts to omit token fields.
[GPT-REVIEWED] 49c9558

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of 49c9558bef46ad59e3e6ef9c594ef79df2bf8e12 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

_self_kill_token_spans re-implements _is_self_kill's by-name walk line for line; the single shared walk that cannot diverge was never considered.

Not justified as shipped

  • Item 4 — duplicate of src/kiro_crew/security/argv_floor.py:858-895: the new span walk copies the by-name leg's exact sequence (_argv_programs, lazy disqualified, _data_consumer_exempt, the _SELF_NAME_RE/_debracket/_normalize_operand pair, the depth/_ends_argv window — ~30 lines, six helper calls, counted against _is_self_kill). The description's guarantee — "mirrors the by-name leg … so the two never disagree" — rests on manual lockstep: the next exemption added to _is_self_kill silently makes the diagnostic bracket tokens the floor no longer keyed on, which is the exact misleading reading the Goal exists to remove.
    Subtraction: replace both copies with one index-returning walk (the by-name leg yields (i, j); _is_self_kill truth-tests it, _self_kill_token_spans shrinks to raw-span mapping), deleting the mirrored block.
    Clears when: the mirrored walk is deleted in favour of a shared index-returning walk, or a maintainer records why two walks must stay separate.

What this change ships

Inventory (8 items) — 7 justified

Intent: let a reader of a self-kill refusal see which two tokens tripped the floor, distinguishing real kills from false positives (#14827). ADDITION.

  1. self-kill refusals now name program/target token offsets with a census — justified
  2. shapes with no provable token pair keep the old whole-command line — justified
  3. every other floor's refusal line stays byte-identical — justified
  4. a second copy of the by-name matching walk locates the offsets — duplicate of argv_floor.py:858-895 (see above)
  5. new raw-offset word splitter _raw_word_spans — justified
  6. RefusalTokenSpan becomes an importable facade name — justified
  7. package line budget re-pinned 27,942→28,146 with written reason — undeclared; mandated by the ratchet's own process
  8. security.md documents the token-level exception in the same commit — justified

[FIRST-PRINCIPLES-REVIEWED] 49c9558

Description read

[DESCRIPTION-READ] d834592009f8f51499182631352340d7d3bdc9c395fe0751dcc5835bf0b794cc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 49c9558bef46ad59e3e6ef9c594ef79df2bf8e12 via the fork AI-review pipeline; updated in place on each push.

Review details

No blocking findings — the new program=/target= offsets can name the wrong region of the command in two independent ways.

FINDING — src/kiro_crew/security/argv_floor.py:1052 — len(raw_spans) != len(tokens) infers index ALIGNMENT from equal word COUNT, but the two partitions can differ while the counts agree: for pkill -f x& kirocrew "q r the unbalanced quote makes _shell_tokens fall into its except ValueError: [t.strip(...) for t in cmd.split()] branch (splitting "q r into two tokens) while _raw_word_spans splits x& into x + & — both lists reach 6, the guard passes, and return raw_spans[i], raw_spans[j] (line 1068) emits target=10..11, the &, with shape=len=1,…,symbol=1, instead of the kirocrew token at 12..20 the floor actually keyed on → Fix: in the loop body, before returning, require the resolved token to be present in its claimed raw region (_debracket/_normalize_operand of tokens[j] occurring in text_lower[raw_spans[j][0]:raw_spans[j][1]], same for i) and continue/return None when it is not.

FINDING — src/kiro_crew/security/init.py:1444 — spans = _argv._self_kill_token_spans(lower) yields offsets into lower which _reason then applies to tool_name as the diagnostic subject, and the adjacent comment's premise "lower-casing never changes length" is false ('İ'.lower() is two code points), so pkill -f /home/İvan kirocrew other returns target=(21,29) against a 34-char tool_name whose kirocrew lives at 20..28 — the line reports irocrew (shape=len=8,…,space=1) as the target token → Fix: guard the forward at this call site — if len(lower) != len(tool_name): spans = None — so a length-changing fold degrades to the historical whole-command span rather than a shifted region.

[OPUS-REVIEWED] 49c9558

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review (fork) — ✅ PASS

Which legitimate operations does 49c9558bef46ad59e3e6ef9c594ef79df2bf8e12 newly refuse? Model proposes, scripts/deny_diff.py decides — updated in place on each push. A script-confirmed regression blocks PR readiness.

adjudicated, zero confirmed regressions

Scope-Verdict: PASS

TIGHTENING: The self-protection-kill refusal diagnostic in is_denied — the argv floor that refuses killing the gateway's own process by name (pkill -f kirocrew / killall kirocrew), per the hunk if rule_id == "self-protection-kill": spans = _argv._self_kill_token_spans(lower). No matcher got stricter: the change runs strictly after _emit_deny_event(tool_name, pattern, lower) on a denial already decided, and the new argv_floor._raw_word_spans / _self_kill_token_spans functions are called only from that refusal-text path — _is_self_kill and every deciding predicate are byte-identical at base and head. The additive program=/target= fields render only when supplied (for name, span in (("program", self.program), ("target", self.target)): if span is not None), so every other floor's refusal line is unchanged. The deny decision surface is therefore the same set at both refs; the candidates exist to let the classifier confirm that rather than take my word for it.

ADJUDICATED: 15 candidates, 0 confirmed newly-refused
No confirmed rows were handed back for this review; the deterministic classification at f10a74d and 49c9558 gates on the harness's own verdicts regardless of this section.

PLATFORM COVERAGE: posix (10 rows), windows (3 rows), any (2 rows). The diff's only platform-shaped assumption is _raw_word_spans, which hardcodes bash's word-break set (;&|()<> plus unquoted whitespace) and the shared bash quote/escape state machine — posix rows probe that neighborhood directly, including a quoted-target kill (pkill -f 'flask run'), the read-only pgrep/ps | grep forms, the bare-PID and signal-0 kills, and the maintainers' greps and test invocation over the touched files; windows rows cover the same operations in the other tokenizer's spelling (taskkill /PID, taskkill /F /IM node.exe, a quoted tasklist /FI filter carrying the product name), since a cmd-style argv never resembles the bash shapes the new splitter assumes. No assumed platform was left without a candidate. Per-leg verdicts are the deterministic fold's to report, by leg name, after this review is captured.

[SCOPE-REVIEWED] 49c9558

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • ubuntu-latest (platform posix): 12 classified, 0 newly refused, 12 still allowed.
  • windows-latest (platform windows): 5 classified, 0 newly refused, 5 still allowed.

No candidate flipped from allowed to refused.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 29, 2026
@SaRedfiche
SaRedfiche force-pushed the pipe/issue-14827-selfkill-diag-offsets branch from f9d5476 to 58b337b Compare September 29, 2026 19:39
@SaRedfiche

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in the security module (argv_floor.py, diagnostics.py, security/init.py, _exports.py) and docs/system-specs/modules/security.md since this head went green (d7330c1 -> df0ea79); CI re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 29, 2026
The self-protection-kill refusal diagnostic now carries additive
program and target offset fields with the same char-class census, so a
reader can tell a real by-name kill from a glob-argument false positive
without running the classifier. A new raw-word-span splitter aligns the
resolved frame tokens to their real source offsets by index, so even a
shell-variable-resolved glob token is bracketed at its true position.
The span field and the first two refusal lines are byte-identical when
the fields are omitted, and no field carries a byte of the command.
@SaRedfiche
SaRedfiche force-pushed the pipe/issue-14827-selfkill-diag-offsets branch from 58b337b to 49c9558 Compare September 30, 2026 12:15
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 30, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 30, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thanks for this, @SaRedfiche! This PR had readiness: passed but fell into conflict with main. Since fork PRs can't consume /ai-review overrides, I've opened an in-repo replacement that rebases your commit onto current main: #16333. Your commit is preserved as the author (with a Co-authored-by: @SaRedfiche trailer) so a Rebase-and-merge lands your work with full credit. Closing this in favour of #16333.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security diagnostics: a self-protection-kill refusal reports the whole command as its span, so a false positive reads the same as a real kill

2 participants