Repository navigation
fix(shell-walk): read a heredoc body as data only when every command just reads it - #19951
mbriones98 wants to merge 2 commits into
Conversation
…just reads it The shared shell walk folded every heredoc body into the frame as commands, so a message or file body that only names a command was read as that command. Set the body aside, from both the frame's tokens and its source, only when every command of every frame is a listed read-only data consumer (or git/gh with a message subcommand directly after the program) and the opener line ends at its newline; every other shape keeps the body. Measure the walk's descent bound with the home path discounted, so a nested payload is read whatever the length of $HOME. Fixes #19588
Sensitive change review
|
|
Review history carried forward from #19892 (same change, recut as one commit). Each item below was raised by the review lanes on #19892 and is already addressed in this commit; plain notes, not disposition records.
|
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. [UX-REVIEWED] aa0e7ad Description read[DESCRIPTION-READ] 374f81013fb947f63b55055f92ffb3e9d5bd6f4d1b78a573dd4b8edb4f664c25 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the existing denied-command and argv-floor suites pass unchanged in CI. This PR narrows the reader that every floor shares, and the spec previously said narrowing never happens there. [DESIGN-REVIEWED] aa0e7ad Description read[DESCRIPTION-READ] 6b86aaf622d5d770fd4436a37fe91d81ad1a2ac5110b079222f511a144d4fa91 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The fix targets the right cause, but its new heredoc program list is a second copy of an existing set, and one of its parameters is never read. Not justified as shipped
What this change shipsInventory (8 items) — 6 justifiedIntent: stop every argv floor refusing commit messages and files written through a heredoc, and make nested-payload reading independent of
[FIRST-PRINCIPLES-REVIEWED] aa0e7ad Description read[DESCRIPTION-READ] 2162e8eed6c39261f1df48e0d9bad4c28ac5fa906f130ed06a7907c100882803 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — 🔴 changes requested (blocking)Reviewed Blocks merge: a heredoc delimiter split by a backslash-newline hides a protected-branch BLOCKING — src/kiro_crew/security/shell_normalizer.py:3652 [BLOCK-MERGE] aa0e7ad Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
|
GPT 6.1 Review — 🔴 changes requested (blocking)GPT 6.1 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- src/kiro_crew/security/shell_normalizer.py:3609 -- Misparsed heredoc boundaries bypass protected-branch denial
[BLOCK-MERGE] a4ad2a1 Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 (fenced, unbounded harm: the vetter wrongly allows a protected-branch push). I confirmed both conditions in the code:
False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS TIGHTENING: The shared shell payload walk's descent bound ( ADJUDICATED: 16 candidates, 0 confirmed newly-refused (the harness handed this review no confirmed rows; the per-leg verdicts come from the deterministic fold) PLATFORM COVERAGE: The diff's platform assumption is the spelling and length of the home path that
No platform the diff assumes is left without a candidate. One note for the Windows legs: [SCOPE-REVIEWED] aa0e7ad |
Find the heredoc operator in the frame's raw source with a quote- and escape-aware scan, so a quoted or escaped `<<` is text, not an opener. Read the delimiter as the shell does, ended by an unquoted blank or metacharacter, so a glued redirect or parenthesis is not part of it. Keep the body whenever no line closes it, and skip attribution when the opener line carries a comment, substitution, subshell or ANSI-C string.
|
|
|
Sensitive change review
|
|
Superseded by #19986 (same change plus one more boundary case, recut as a single commit on current main). |
Pull request was closed
Problem / Motivation
Goal: Make the shared shell reader treat a heredoc body the way bash does and bound its descent on structure, so every argv floor reads the same, correct command shape.
security/shell_normalizer._shell_payload_walkis the one reader every argv floor goes through. It has two readings that do not match what the shell runs:;, so it cannot tellbash <<EOF(runs the body) fromcat <<EOForgit commit -F - <<EOF(reads it as data). A commit message, PR body or spec bullet that only names a flagged command is then read as that command.$HOMEexpansion is not read. The walk descends a payload only while it is shorter than its parent's text, but$HOMEis expanded per token before that comparison, so on a host with a long home path a payload exceeds the line and is skipped.Why it matters
The first reading produces false refusals of ordinary data (a commit message, a file written through a heredoc). The second makes a correctness bound depend on a value outside the command, so the same command reads differently on two hosts. Both live in the one reader, so every floor inherits them.
Not a goal
What changed (motivation → approach → change)
The reader (
security/shell_normalizer.py) folds a heredoc body into the frame as;-separated commands, which is why a data consumer's body is scanned as if it ran. The fix attributes a body to its consumer with a positive allow-list over the whole command: the body is set aside as data only when, with it removed, every command of every frame is a listed data consumer -- a barecat/tee/head/grep-style program, orgit/ghfollowed directly by a message subcommand (git commit/tag/notes,gh pr/issue/release). Any other program anywhere keeps every body: a shell or evaluator,xargs, a pipe into an unlisted program, a path such as./f, a leading assignment, an operator glued inside a word, a process substitution, orgit/ghwith an option before the subcommand. The boundary is read from the raw line as the shell reads it: the one heredoc operator must be unquoted and unescaped (a<<in quotes or behind a backslash is text), its delimiter ends at the first unquoted blank or metacharacter (<<eof>outis the tageofand a redirect), and some later line must close the body. The opener line must also end its command at the newline and carry no comment, substitution, subshell or ANSI-C string. Any other shape keeps the body. The frame's source drops the body together with its tokens, so token readers and source readers see the same command.The descent bound is measured with the home path discounted, so a payload that lives inside one token of its parent stays shorter than the parent whatever the home path's length. The bound stays structural and no longer varies with
$HOME.The module spec entry for the reader is updated to describe both, and states the heredoc attribution as the one reading the reader sets aside, with why it lives in the reader. The package line budget rises for the added helpers.
This unblocks the parked
chmodfloor work in #19563, which can drop its local heredoc workaround and rely on the reader once this lands. #14993 rewrites the same walk for performance and will need a rebase against whichever lands first.Backwards compatibility
Compatible: no command that is refused today becomes allowed unless every command in it is a listed data consumer; the only behaviour that changes is that a provably-inert data body is no longer read as commands, which removes false refusals. The frames the walk returns for every runner shape are the same.
Removes nothing: no reader, record, key or public function is removed; the walk's return type and callers are unchanged.
Tests
test/test_shell_walk_heredoc_and_home_bound.pylocks the walk's behaviour on neutral commands: a data-consumer body is set aside (and absent from the frame source); a body any unlisted program, path, assignment, glued operator, process substitution, unlistedgit/ghsubcommand or-coption could run is read; a backslash-continued opener line, multi-heredoc shapes, a quoted or escaped<<, and a body no line closes keep their body; a delimiter with a glued redirect or parenthesis ends where the shell ends it; a substitution inside a data body is still its own frame; and a nested payload is descended for both a short and a long home path, with the walk terminating on deep nesting.test/test_security_gate_liveness.pyraises the package line budget to match the added helpers.Manual verification
N/A — unit coverage sufficient; the change is in a pure text reader exercised directly by the new tests and the existing shell-normalizer / argv-floor / denied-command suites.
Related Issues
Fixes #19588
Supersedes #19892
Pattern harvest
Rule candidate: review-prompt
Pattern: a text reader that folds a data region into executable structure refuses data that merely names a flagged command; attribute the region to its consumer through a positive allow-list over the whole command, so an unrecognised program keeps the region as commands.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)