Skip to content

fix(security): read only the last word for an rm floor fd prefix - #38898

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/37216-rm-floor-fd-prefix-linear
Nov 20, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/37216-rm-floor-fd-prefix-linear

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A shell command line full of << no longer holds the tool-call decision for seconds.

The recursive-force rm floor's run time grows with the square of the number of << operators on one line. "((1" + "<<1" * N + ")); rm -f scratch" is a short, valid command, but deny_gate.is_denied takes 4.4 s on it at N = 6000, and doubling N roughly quadruples the time.

Why it matters

Every shell tool call goes through this check before it runs. One line built this way blocks the gate worker for seconds, and the delay grows quadratically with the line, so a single command can stall the decision for as long as its author likes.

Not a goal

What changed (motivation → approach → change)

At every live redirection, the rm floor checks whether the text just before it is a file-descriptor prefix (2>, {fd}>). That check joined the whole command read so far and ran an end-anchored regex over it, so each check cost the length of the line and N redirections cost O(N²). At N = 3000 under cProfile, 0.93 s of 1.8 s went to re.Pattern.search, called 6000 times from rm_floor._pull_trailing_fd_prefix.

A prefix only counts as a whole word (the 2 in /tmp/2>x is part of the path), and neither prefix form holds a blank, so the regex could only ever match the line's last blank-delimited word. The helper now walks back from the end to the last blank or \x00, without joining, checks that word with the regex's own character classes (str.isdecimal; isalnum or _, with an ASCII letter or _ first), and on a hit trims only the tail. Each redirection the floor appends starts with a blank, so the next walk stops there and the total work is linear. _FD_PREFIX_RE had no other reader and is removed. A row in test/security_line_budget/ records the 14 lines this adds to the security package.

Backwards compatibility

Compatible: the helper returns the same prefix and leaves the same joined segment for every input. A throwaway equivalence check compared the old join + regex with the new walk on every code point in six shapes and on 200,000 random multi-element segments: 0 mismatches.
Reader: src/kiro_crew/security/rm_floor.py:_FD_PREFIX_RE -- entry: chat|cron|subagent (every shell tool call) -- its only reader was _pull_trailing_fd_prefix, now rewritten; git grep _FD_PREFIX_RE -- src test finds no other reader. Pinned by the existing fd-prefix rows in TestRecursiveForceRmFloor and test/test_shell_commands_reader.py, unchanged and passing.

Tests

Proof: PROVEN test/test_denied_commands_security_rows.py::TestRecursiveForceRmFloor::test_an_fd_prefix_check_reads_only_the_segment_tail[1000], test/test_denied_commands_security_rows.py::TestRecursiveForceRmFloor::test_an_fd_prefix_check_reads_only_the_segment_tail[2000] (head d83c8f66605b)

prove.py ran on a Linux test host on d83c8f66605b: this same patch on main 36881767809c (git range-diff against this head: =).

  • TestRecursiveForceRmFloor::test_an_fd_prefix_check_reads_only_the_segment_tail[1000|2000] passes the helper a MutableSequence that counts element reads, in the issue's segment shape. It must return "" after at most 8 reads at both sizes (3 measured), with no clock, like the neighbouring test_nested_open_expansions_are_read_a_bounded_number_of_times. On the base, "".join reads every element: 2004 and 4004 reads.
  • Determinism: the new node passed 20/20 at -n0, and the whole file passed 384/384 in a seeded shuffled order (seed 37216).
  • Targeted: test_denied_commands_security_rows.py, test_shell_commands_reader.py, test_shell_heredoc_reader.py and test_security_gate_liveness.py (the line budget, with the new row), 536 passed. The existing fd-prefix rows are unchanged.

Manual verification

Run on a Linux test host (Ubuntu 24.04, Python 3.12) against main cf61d1dce0a8, before and after this commit. Thread CPU time (time.thread_time, min of 3) of deny_gate.is_denied("((1" + "<<1" * N + ")); rm -f scratch"):

N before after
1,500 0.422 s 0.149 s
3,000 1.432 s (x3.4) 0.313 s (x2.1)
6,000 4.969 s (x3.5) 0.677 s (x2.2)
12,000 n/a 1.590 s (x2.3)

N = 6,000 is about 18 KB, close to the 20 KiB command cap. A cProfile at N = 3,000 shows re.Pattern.search at 6,315 calls / 0.953 s before (6,000 of them from _pull_trailing_fd_prefix) and 315 calls / 0.023 s after; the helper itself is 0.022 s cumulative after.

The PreToolUse gate itself (HookManager.on_tool_call("bash", command=..., is_shell=True), isolated KIROCREW_HOME) takes 4.999 s before and 0.764 s after on the issue's command at N = 6,000.

Same answers before and after, from is_denied and from the PreToolUse gate:

command before after
2>/dev/null rm -r -f / refused refused
X=1 2>x rm -r -f ~ refused refused
{fd}>x rm -rf / refused refused
2>&1 rm -rf ~ refused refused
rm -fr / 2>/dev/null refused refused
rm -fr /tmp/2>/dev/null allowed allowed
2>&1 rm -fr /tmp/x allowed allowed
git status allowed allowed
the issue's command, N = 50 allowed allowed
rm -f scratch allowed allowed

The rest of the growth (about x2.1 to x2.3 per doubling) is mostly _rm_wipe_targets stepping over each leading redirection with words = words[1:], which copies the word list once per redirection. It is small up to the command cap and is left out of this change.

Real-machine verification

Machine: EC2 m7i.2xlarge, Ubuntu 24.04.5 LTS x86_64, 8 vCPU / 30 GiB, Python 3.12.3
Before (main cf61d1d): python3 -m pytest -n0 "test/test_denied_commands_security_rows.py::TestRecursiveForceRmFloor::test_an_fd_prefix_check_reads_only_the_segment_tail", with this PR's test file on the base source -> 2 failed, assert 2004 <= 8 and assert 4004 <= 8; the issue's command at N = 6,000 takes 4.969 s of thread CPU in deny_gate.is_denied
After (head e8c0470, run on efa0bc375a60, the same patch on main cf61d1d): the same command -> 2 passed, 20 of 20 runs at -n0; the issue's command at N = 6,000 takes 0.677 s
Cleanup: instance terminated (describe-instances and describe-volumes for its id return nothing); it had no key pair and no inbound rules

Related Issues

Closes #37216

Sibling sweep

Swept: git grep -n '"".join(seg)' -- src/kiro_crew/security (its other hit, the join that ends each command in rm_floor._segments, runs once per segment: linear)
Swept: git grep -nE '\Z' -- src/kiro_crew/security, then each call site of an end-anchored pattern that runs once per step
Sibling: src/kiro_crew/security/rm_floor.py:48 -- changed here
Sibling: src/kiro_crew/security/shell_normalizer.py:715 -- not affected -- _PLAIN_REDIRECTION_RE matches one token at a time, never a growing buffer
Sibling: src/kiro_crew/security/argv_floor_self.py:483 -- not affected -- _RAW_ASSIGNMENT_RE matches one word at a time
Sibling: src/kiro_crew/security/stream_aws_anchor.py:76 -- not affected -- the search is windowed to at most _KEY_NAME_MAX characters before the end
Sibling: src/kiro_crew/security/inline_payload.py:496 -- tracked in #37482
Sibling: src/kiro_crew/security/argv_floor_ssh.py:656 -- tracked in #37373

Pattern harvest

Rule candidate: review-prompt
Pattern: a per-step helper that joins or regex-searches the whole growing buffer

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/rm_floor.py

  • @bolichen97 posted a complete reasoning template for e8c0470.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does e8c047056416a8f2a9216556b8fa50da53eac47a newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 13 classified, 0 newly refused, 12 still allowed.
  • ubuntu-latest (platform posix): 13 classified, 0 newly refused, 12 still allowed.
  • windows-latest (platform windows): 4 classified, 0 newly refused, 4 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: This change rewrites _pull_trailing_fd_prefix so it runs in linear time. It does not add a new deny pattern. The new code walks back to the segment's last blank or NUL instead of running the regex over the joined segment. For every input I traced, it accepts the same fd words as the old (?:^|(?<=[\s\x00]))(?:\{[A-Za-z_]\w*\}|\d+)\Z (isdecimal≈\d, isalnum/_≈\w, isspace≈\s). The caller joins the segment afterwards anyway, so its output should be the same. No confirmed regression was handed back, and I wrote 17 boundary rows to check that claim.

TIGHTENING: rm-floor fd-prefix recognition in _segments (sensitive-bash/deny-rules reach via rm_floor). It reads only the tail of the segment, replacing - joined = "".join(seg) / - match = _FD_PREFIX_RE.search(joined) with + while end: ... if cut: head = text[cut:]; break. The one threat it refuses is quadratic work on a line with many << redirections (seg = _Counted(["(", "(", *(["1", " <<"] * n), "1"]), assert seg.reads <= 8). It is a cost fix, not a broader refusal set.

ADJUDICATED: 17 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE:

  • posix (13 rows): the diff assumes POSIX/bash redirection spelling: a bare-number fd prefix (2>, 3>&1), bash's named fd {lock}>, and blank/NUL word boundaries. The rows cover each of these:
    • fd-2 silences right after an rm target, under /tmp, home and $HOME;
    • the docstring's own /tmp/2> path case, and a numbered path ending in a digit;
    • a named-fd lock;
    • a redirection after a subshell paren;
    • a nested bash -c;
    • the {} brace word from find -exec;
    • fd swaps;
    • the many-heredoc shape;
    • the touched test run.
  • windows (4 rows): the floor's _RM_WORDS includes rm.exe. The rows probe rm.exe … 2>NUL, a drive-letter \ path with %USERNAME%, PowerShell $env:TEMP … 2>$null, and .venv\Scripts\python.exe … 2>&1. Together they cover the cmd and PowerShell stderr-sink spellings that sit in the fd-prefix position.
  • any: no rows. Every candidate carries a path or an executable name.

[SCOPE-REVIEWED] e8c0470

@bolichen97
bolichen97 force-pushed the fix/37216-rm-floor-fd-prefix-linear branch from 72bc726 to f411085 Compare November 19, 2026 14:59
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of e8c047056416a8f2a9216556b8fa50da53eac47a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e8c0470

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e8c047056416a8f2a9216556b8fa50da53eac47a false-positive: <reason, 20 to 500 characters>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in src/kiro_crew/security/rm_floor.py and test/security_line_budget/ since this head went green (4a3697d -> 8301e68); CI re-running.

[operator: bolichen97]

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 19, 2026
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision e8c047056416a8f2a9216556b8fa50da53eac47a touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of e8c047056416a8f2a9216556b8fa50da53eac47a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix removes the quadratic cost where it starts, the per-step join plus end-anchored regex, and it keeps the same answers.

Notes

  • The linear bound rests on one claim: every redirection the floor appends starts with a blank. A segment with no blank in it would make each walk run back to the start again. Asserting that claim where the floor appends a redirection, or pinning it in a test, would keep a later tokenizer change from bringing the quadratic cost back without anyone noticing.

[DESIGN-REVIEWED] e8c0470

Description read

[DESCRIPTION-READ] ebb3aa0a919a65ee93bb09d302c3ccd25f3cb61354ed08a037bc53854c217798

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of e8c047056416a8f2a9216556b8fa50da53eac47a — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. This is a linear rewrite of one helper, at the mechanism level. The defect is tracked as #37216 and pinned by a read-count test that fails on base, and the helper still gives the same answers.

What this change ships

Inventory (4 items) — 4 justified

Intent: stop one shell command line packed with << from stalling the tool-call deny decision for seconds. FIX.

  1. A command line with many << redirections now gets its rm-floor decision in linear time instead of quadratic time — justified
  2. _FD_PREFIX_RE regex removed; it had no other reader — justified
  3. Security line-budget row recording the 14 added lines, as test_security_gate_liveness.py requires — justified
  4. Read-count test that bounds the helper's element reads at 1,000 and 2,000 redirections — justified

[FIRST-PRINCIPLES-REVIEWED] e8c0470

Description read

[DESCRIPTION-READ] e5728f663c9841e0cf942145079c00213b872debae24dc16f0e7adb7ece2c552

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed e8c047056416a8f2a9216556b8fa50da53eac47a — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e8c0470

Verdict parsed from the review's SHA-scoped output markers for commit e8c047056416a8f2a9216556b8fa50da53eac47a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e8c047056416a8f2a9216556b8fa50da53eac47a false-positive: <reason, 20 to 500 characters>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in no file this PR changes; rebased onto current main for the test-floor fix that landed there (8301e68 -> bdbec31); CI re-running.

[operator: bolichen97]

@bolichen97
bolichen97 force-pushed the fix/37216-rm-floor-fd-prefix-linear branch from f411085 to 283f13e Compare November 19, 2026 17:54
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 19, 2026
The rm floor keeps a redirection's fd prefix (2>, {fd}>) on the
redirection word. At every live redirection it joined the whole segment
read so far and searched it with an end-anchored regex, so a line with N
redirections cost O(N^2): a short command full of << held the tool-call
decision for seconds (4.4 s at N = 6000).

The prefix can only be the segment's last blank-delimited word, because
no character of either prefix form is a blank. The helper now walks back
from the end of the segment to the last blank, checks that word with the
regex's own character classes, and trims only the tail elements. Each
redirection piece starts with a blank, so the total work is linear. The
answers are unchanged; _FD_PREFIX_RE had no other reader and is removed.

A test pins the cost by counting element reads, without a clock. A
security line-budget row records the 14 lines this adds to the package.

Closes #37216
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in no file this PR changes; rebased onto current main for the Windows test-shard fix that landed there (bdbec31 -> be1ce64); CI re-running.

[operator: bolichen97]

@bolichen97
bolichen97 force-pushed the fix/37216-rm-floor-fd-prefix-linear branch from 283f13e to e8c0470 Compare November 19, 2026 20:20
@bolichen97

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: e8c0470
  • What rule changed: no rule or pattern was added, removed or edited in what it refuses. The recursive-force rm floor's fd-prefix helper (rm_floor._pull_trailing_fd_prefix) finds a 2> / {fd}> prefix by reading only the segment's last blank-delimited word instead of joining the whole segment and searching it with an end-anchored regex at every redirection; the regex it replaces had no other reader and is removed.
  • Before -> after (what was allowed/blocked, now): the same commands are refused and allowed before and after; only the cost changes. On the issue's command ("((1" + "<<1" * N + ")); rm -f scratch") at N = 6,000 the PreToolUse gate takes 4.602 s of thread CPU before and 0.676 s after; deny_gate.is_denied grows x3.3-x3.6 per doubling of N before and x2.1-x2.3 after.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a misread fd prefix makes the text-level rm floor refuse an ordinary rm with a redirection, or miss a recursive-force rm of / or ~ spelled with an fd-prefixed redirection; the OS sandbox stays the enforcement point either way. No secret path, sandbox mask, seal or credential is touched.
  • Evidence checked (which sandbox result proves it): on a disposable Linux host (Ubuntu 24.04.5 LTS, x86_64, Python 3.12.3), deny_gate.is_denied on 10 commands (5 fd-prefixed recursive-force rm of / or ~, rm -fr /tmp/2>/dev/null, 2>&1 rm -fr /tmp/x, git status, the issue's command at N = 50, rm -f scratch) and the real PreToolUse gate (HookManager.on_tool_call with is_shell=True, isolated data home) on 8 commands gave identical answers on main 36881767809c and on d83c8f66605b (this head's patch on that main; git range-diff against this head: =; this head's tree e3a9c2167898 is the tree that host's git merge-tree of d83c8f66605b and main be1ce645a66e printed). The diff changes no sandbox surface (sandbox.py and its masks, seals and dispositions are untouched); no live agent session ran.
  • No regression:
    • Tools still work (which tools/commands were run, result): git status, rm -f scratch, rm -fr /tmp/2>/dev/null, 2>&1 rm -fr /tmp/x and the issue's command stay allowed; rm -rf /, 2>/dev/null rm -r -f /, X=1 2>x rm -r -f ~, {fd}>x rm -rf /, 2>&1 rm -rf ~ and rm -fr / 2>/dev/null stay refused (before and after identical).
    • Backward compatible (old config / old data / old callers still work, how checked): no config key, data format or public function changed; the helper's only caller in src is rm_floor._segments (the other two callers are tests: the oracle in test/test_shell_commands_reader.py and the new test); git grep _FD_PREFIX_RE -- src test finds no other reader.
    • Existing tests (which suites passed on this head): on d83c8f66605b (same patch), black, subprocess-encoding, isort, flake8, mypy, file line cap, comment history, brand, harness parity, testpaths, docs-lint, semgrep (0 findings), the flake-pattern ratchet (75 passed), the targeted security files (536 passed), scripts/local-gate.py (backend related 644 passed; frontend related 15,736 tests passed) and prove.py (PROVEN); the new test 20/20 at -n0, 10/10 at -n 4 with a co-tenant, the whole file 384/384 in a seeded shuffled order; the new test fails on main (2004 and 4004 reads against a bound of 8). CI on e8c0470 is green on every check except this review and the PR Readiness status that folds it: Backend Tests (3.12) shards 1-8, Backend Tests (Windows) shards 1-8, Backend Tests (IPv6, ubuntu-latest and windows-latest), Backend Tests (namespace sandbox), Backend Tests (crew container), Windows Fail-Closed Sandbox Contract, Denial Differential (ubuntu-latest, macos-latest, windows-latest), Security Scope Review, E2E (stub ACP backend, offline), Integration (in-process gateway) and Coverage Gate.
  • How to undo: revert the single commit e8c0470; no data or config migration is involved.

[operator: bolichen97]

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 20, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed head e8c0470. The rewritten fd-prefix helper in the rm floor is a linear walk back to the segment's last blank, and I checked that it is equivalent to the regex it replaces. A differential probe over hundreds of thousands of random segments and every code point in several word shapes found no difference in the returned prefix or the remaining joined segment. The only caller joins the segment right away, so the refuse and allow answers do not change. The new read-count test fails on the base and passes here, and the related security and shell-reader suites pass on this head. Nothing blocks.

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 20, 2026 00:55
@KiroCrewCaptain
KiroCrewCaptain merged commit ab3ca54 into main Nov 20, 2026
104 of 106 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/37216-rm-floor-fd-prefix-linear branch November 20, 2026 00:55
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rm floor time grows quadratically with the number of << operators on one line

2 participants