Repository navigation
fix(shell-walk): read a heredoc body as data only when every command just reads it - #19986
mbriones98 wants to merge 1 commit into
Conversation
…just reads it The shared shell walk folded every heredoc body into the frame as commands, so a message or file body that only names a command was read as that command. Set the body aside, from both the frame's tokens and its source, only when every command of every frame is a listed read-only data consumer (or git/gh with a message subcommand directly after the program). The boundary is read from the raw opener line as the shell reads it: one unquoted, unescaped operator, a delimiter that ends at the first unquoted blank or metacharacter, and a closing line with no backslash-newline before it. Every other shape keeps the body. Measure the walk's descent bound with the home path discounted, so a nested payload is read whatever the length of $HOME. Fixes #19588
Sensitive change review
|
|
Review history carried forward from #19892 and #19951 (same change, recut as one commit). Each item was raised by the review lanes there and is addressed in this commit; plain notes, not disposition records.
|
Sensitive Change ReviewSensitive files changed:
|
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (5 items) — 4 justifiedIntent: stop the shared shell reader refusing commit messages and heredoc files that only name a flagged command, and stop its descent depending on
[FIRST-PRINCIPLES-REVIEWED] 717bba8 Description read[DESCRIPTION-READ] ddae983566a5053c31c5ad4d1fb9eebc22bba28725ea849ee126185b845c80d3 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the argv-floor callers at [DESIGN-REVIEWED] 717bba8 Description read[DESCRIPTION-READ] f5558ac4ff9fc68907f633202032f95a123e8d05ca8d9191855d5f6398aad0a7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — 🔴 changes requested (blocking)Reviewed A quoted string that spans lines can contain a fake heredoc opener, which strips real commands from the shell walk and lets BLOCKING — src/kiro_crew/security/shell_normalizer.py:3661 [BLOCK-MERGE] 717bba8 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ⛔ reviewer refused this diffThe provider declined to review This comment is updated in place on each push. The refusal text is in the GPT 6.1 Review job logs. It is caused by the diff's content: re-runs have not been observed to clear this class of refusal. False positive or not applicable? A repository writer can comment: |
|
Closing in favour of a design pass in #20006. |
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: the shared shell-payload walk's descent bound ( ADJUDICATED: 17 candidates, 0 confirmed newly-refused (no confirmed rows have reached this review; the deterministic fold reports the per-leg results) PLATFORM COVERAGE:
UNADJUDICATED:
[SCOPE-REVIEWED] 717bba8 |
Problem / Motivation
Goal: Make the shared shell reader treat a heredoc body the way bash does and bound its descent on structure, so every argv floor reads the same, correct command shape.
security/shell_normalizer._shell_payload_walkis the one reader every argv floor goes through. It has two readings that do not match what the shell runs:;, so it cannot tellbash <<EOF(runs the body) fromcat <<EOForgit commit -F - <<EOF(reads it as data). A commit message, PR body or spec bullet that only names a flagged command is then read as that command.$HOMEexpansion is not read. The walk descends a payload only while it is shorter than its parent's text, but$HOMEis expanded per token before that comparison, so on a host with a long home path a payload exceeds the line and is skipped.Why it matters
The first reading produces false refusals of ordinary data (a commit message, a file written through a heredoc). The second makes a correctness bound depend on a value outside the command, so the same command reads differently on two hosts. Both live in the one reader, so every floor inherits them.
Not a goal
What changed (motivation → approach → change)
The reader (
security/shell_normalizer.py) folds a heredoc body into the frame as;-separated commands, which is why a data consumer's body is scanned as if it ran. The fix attributes a body to its consumer with a positive allow-list over the whole command: the body is set aside as data only when, with it removed, every command of every frame is a listed data consumer -- a barecat/tee/head/grep-style program, orgit/ghfollowed directly by a message subcommand (git commit/tag/notes,gh pr/issue/release). Any other program anywhere keeps every body: a shell or evaluator,xargs, a pipe into an unlisted program, a path such as./f, a leading assignment, an operator glued inside a word, a process substitution, orgit/ghwith an option before the subcommand. The boundary is read from the raw line as the shell reads it: the one heredoc operator must be unquoted and unescaped (a<<in quotes or behind a backslash is text), its delimiter ends at the first unquoted blank or metacharacter (<<eof>outis the tageofand a redirect), and some later line must close the body with no backslash-newline before it (in an unquoted-delimiter body a backslash-newline joins two lines before the shell compares them to the delimiter). The opener line must also end its command at the newline and carry no comment, substitution, subshell or ANSI-C string. Any other shape keeps the body. The frame's source drops the body together with its tokens, so token readers and source readers see the same command.The descent bound is measured with the home path discounted, so a payload that lives inside one token of its parent stays shorter than the parent whatever the home path's length. The bound stays structural and no longer varies with
$HOME.The module spec entry for the reader is updated to describe both, and states the heredoc attribution as the one reading the reader sets aside, with why it lives in the reader. The package line budget rises for the added helpers.
This unblocks the parked
chmodfloor work in #19563, which can drop its local heredoc workaround and rely on the reader once this lands. #14993 rewrites the same walk for performance and will need a rebase against whichever lands first.Backwards compatibility
Compatible: no command that is refused today becomes allowed unless every command in it is a listed data consumer; the only behaviour that changes is that a provably-inert data body is no longer read as commands, which removes false refusals. The frames the walk returns for every runner shape are the same.
Removes nothing: no reader, record, key or public function is removed; the walk's return type and callers are unchanged.
Tests
test/test_shell_walk_heredoc_and_home_bound.pylocks the walk's behaviour on neutral commands: a data-consumer body is set aside (and absent from the frame source); a body any unlisted program, path, assignment, glued operator, process substitution, unlistedgit/ghsubcommand or-coption could run is read; a backslash-continued opener line, multi-heredoc shapes, a quoted or escaped<<, a body no line closes, and a body line ending in a backslash keep their body; a delimiter with a glued redirect or parenthesis ends where the shell ends it; a substitution inside a data body is still its own frame; and a nested payload is descended for both a short and a long home path, with the walk terminating on deep nesting.test/test_security_gate_liveness.pyraises the package line budget to match the added helpers.Manual verification
N/A — unit coverage sufficient; the change is in a pure text reader exercised directly by the new tests and the existing shell-normalizer / argv-floor / denied-command suites.
Related Issues
Fixes #19588
Supersedes #19951
Pattern harvest
Rule candidate: review-prompt
Pattern: a text reader that folds a data region into executable structure refuses data that merely names a flagged command; attribute the region to its consumer through a positive allow-list over the whole command, so an unrecognised program keeps the region as commands.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)