Repository navigation
fix: model parameter expansion and comments in shell char walk - #9241
aniruddhaadak80 wants to merge 1 commit into
Conversation
68fd3a5 to
accef9a
Compare
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of I've now verified the base state machine ( First-Principles-Verdict: CONCERNS Harvest claims "not generalizable," but heredoc bodies are a third data-interior construct the walker still truncates on — same root cause, left uncounted. What this change shipsIntent: stop Inventory (4 items)
Watch
[FIRST-PRINCIPLES-REVIEWED] 7173f92 |
GPT 5.6 Review (fork) —
|
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of I have what I need. I traced the new Design-Verdict: CONCERNS The Watch
Suggestions
[DESIGN-REVIEWED] 7173f92 |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed Review detailsI've completed my falsification analysis. Let me summarize the key verification I did before finalizing. The candidate's core premise — that bash closes But the candidate's claimed observable wrong outcome (an allow-direction miss) does not hold up. I traced every consumer of
So the divergence at most causes over-blocking (the diff's stated, safe "scan MORE" direction) and is backstopped everywhere against an actual allow. I could not re-derive a concrete allow-direction failure at 80+ without a contrived, unverifiable combination — which the falsification bar forbids. No new grounded defect surfaced in the diff either. No findings. [OPUS-REVIEWED] 7173f92 |
|
@aniruddhaadak80 Thanks for staying on this. Notes from a repository-wide audit of open PRs, audited at 68fd3a5. Your branch has since moved to accef9a, so please treat anything already fixed as resolved. Already on main: merged #9183 deleted Still missing on main: Two defects we found in 68fd3a5, please confirm they are addressed in the new revision:
One description fix: #8830 is the Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong. |
|
All blocking findings addressed in the latest push (rebased onto current main, 2 commits total):
|
|
🤖 Kiro Crew [operator: dwu96#8c08bee0]: This PR has been inactive for 7+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll send fixes directly to this branch as a co-author. Assessment: Exactly two red checks on Scope note: I will not change the design direction. Design's further suggestion — replacing the local interior tracker with recursive descent through the shared machinery — is an architecture change I'll disposition rather than perform, and I will not weaken any refusal this walk feeds. If you'd prefer I don't touch this PR, add the |
Add regression tests for `${...}` parameter expansion and `#` comment
handling already present in the shell-char walk via `_iter_shell_chars`
and `_matching_close_paren` / `_opens_comment`.
## What these tests pin
**test_security.py (7 new methods in TestBuiltinDenyPatterns /
TestSubstitutionCloserReadsCommandGrammar):**
- `test_expansion_close_ignores_quoted_braces` — a quoted `}` inside
`${...}` is literal per POSIX 2.6.2 and must not close the expansion.
- `test_expansion_interior_reports_its_own_quote_state` — interior chars
report their own quote state so a single-quoted `$(` reads as data.
- `test_expansion_nests_only_on_dollar_brace` — a bare `{` does not open
an expansion; only `${` does.
- `test_ansi_c_hidden_self_kill_is_denied` — a self-kill hidden behind an
ANSI-C `$'\''` inside `${...}` is still DENIED.
- `test_comment_inside_nested_substitution_is_skipped_by_gate` — a `#`
comment inside a nested `$(...)` must not make `)` or `}` reach the
outer walk as structural closers. Three shapes bash 5.2 runs where the
gate must still deny: `${v:-$(echo # )\necho y)}`-style and the
bare-paren-default `: ${v:-(}` shapes.
- `test_a_hash_right_after_an_empty_substitution_is_a_word_not_a_comment`
— a `#` right after `$()` opens a comment only if a word break precedes.
- `test_an_interior_quote_opener_is_syntax_not_a_quoted_run` — an interior
quote delimiter is syntax for the frame it opens from, not quoted data.
**test_security_gate_liveness.py (4 new functions):**
- `test_expansion_opener_flood_is_fast` / `test_self_kill_expansion_flood_is_fast`
— a flood of unclosed `${` openers must not make the gate quadratic via
the `is_sensitive_bash_command` entry point.
- `test_is_denied_single_trailing_brace_flood_is_fast` /
`test_is_denied_self_kill_brace_flood_is_fast` — same flood through
`is_denied` (no size ceiling; Opus finding #1 & #3 from #9241).
## What was NOT kept
Three tests from the earlier PR (#9241 / wip/shell-char-walk-expansion)
reference private functions (`_expansion_span`, `_skip_nested_substitution`)
that exist only on the wip branch; those are follow-up work.
## Related
Closes #9181
Supersedes #9241
Co-authored-by: cixuuz <26782336+cixuuz@users.noreply.github.com>
Add regression tests for `${...}` parameter expansion and `#` comment
handling already present in the shell-char walk via `_iter_shell_chars`
and `_matching_close_paren` / `_opens_comment`.
## What these tests pin
**test_security.py (7 new methods in TestBuiltinDenyPatterns /
TestSubstitutionCloserReadsCommandGrammar):**
- `test_expansion_close_ignores_quoted_braces` — a quoted `}` inside
`${...}` is literal per POSIX 2.6.2 and must not close the expansion.
- `test_expansion_interior_reports_its_own_quote_state` — interior chars
report their own quote state so a single-quoted `$(` reads as data.
- `test_expansion_nests_only_on_dollar_brace` — a bare `{` does not open
an expansion; only `${` does.
- `test_ansi_c_hidden_self_kill_is_denied` — a self-kill hidden behind an
ANSI-C `$'\''` inside `${...}` is still DENIED.
- `test_comment_inside_nested_substitution_is_skipped_by_gate` — a `#`
comment inside a nested `$(...)` must not make `)` or `}` reach the
outer walk as structural closers. Three shapes bash 5.2 runs where the
gate must still deny: `${v:-$(echo # )\necho y)}`-style and the
bare-paren-default `: ${v:-(}` shapes.
- `test_a_hash_right_after_an_empty_substitution_is_a_word_not_a_comment`
— a `#` right after `$()` opens a comment only if a word break precedes.
- `test_an_interior_quote_opener_is_syntax_not_a_quoted_run` — an interior
quote delimiter is syntax for the frame it opens from, not quoted data.
**test_security_gate_liveness.py (4 new functions):**
- `test_expansion_opener_flood_is_fast` / `test_self_kill_expansion_flood_is_fast`
— a flood of unclosed `${` openers must not make the gate quadratic via
the `is_sensitive_bash_command` entry point.
- `test_is_denied_single_trailing_brace_flood_is_fast` /
`test_is_denied_self_kill_brace_flood_is_fast` — same flood through
`is_denied` (no size ceiling; Opus finding #1 & #3 from #9241).
## What was NOT kept
Three tests from the earlier PR (#9241 / wip/shell-char-walk-expansion)
reference private functions (`_expansion_span`, `_skip_nested_substitution`)
that exist only on the wip branch; those are follow-up work.
## Related
Closes #9181
Supersedes #9241
Co-authored-by: cixuuz <26782336+cixuuz@users.noreply.github.com>
|
Open-PR triage (2026-10-05), audited at head Status: DISTINCT and still needed. Change requested: before merge, fix one fail-open that this head adds. Already on main: merged PR #8579 ( What breaks: I ran
In bash, both bodies run the second command ( Fix: when matching Other notes:
|
bolichen97
left a comment
There was a problem hiding this comment.
[P0]
Class: Shell-normalizer span truncation: a security floor fails open.
Mechanism: The new ${...} interior loop in _iter_shell_chars closes the expansion at a } that sits inside a nested command substitution. The following ) is then treated as active and the outer substitution body is cut short, so later commands are hidden from the argv-floor rules. At least two command shapes that main denies (self-protection and token-exfiltration floors) return no denial on this head and on the merge result with main.
User impact: Agent commands that main blocks at the self-protection floor would be allowed after merge.
Location: src/kiro_crew/security/shell_normalizer.py:1594 @ 7173f920e16e743b53d3b3cec72e1abe00185811
Minimum fix: When scanning ${...}, skip nested $(...) and backtick bodies before counting { / } (e.g. reuse _matching_close_paren), and pin main-denied shapes as regression rows.
How to verify: Regression rows in test_security asserting is_denied is non-None for nested-substitution-inside-${...} shapes. They pass on main, fail on this head, and must pass after the fix along with this PR's own new case.
Exploit details are left out on purpose. They are available to maintainers through the repo's private security reporting.
Independently re-verified on head 7173f920e1 with a focused local reproduction.
| qstate = 0 | ||
| elif qstate == 0 and c == "{": | ||
| depth += 1 | ||
| elif qstate == 0 and c == "}": |
There was a problem hiding this comment.
[P0]
cruisercohen
left a comment
There was a problem hiding this comment.
Reviewed locally at head 7173f92. Current main already handles the #-comment half of the issue (_opens_comment in _matching_close_paren). The ${v:-)} half is still real, and this PR fixes it. But the new ${...} interior loop counts a } inside a nested $(...) or backtick as the closer, so a parameter expansion whose default holds a command substitution that contains } is now cut short at that inner brace. Main extracts the full body for that shape, both on this head and when the branch is merged with main. Requested changes:
- When scanning for the matching
}, skip enclosed command substitutions, for example by reusing the existing paren matcher for nested$(and backticks. - Add regression cases that assert the deny result is unchanged for nested-substitution-inside-
${...}shapes. - Rebase onto main's comment handling and drop the second comment model, so there is one comment rule. Then re-check
TestSubstitutionCloserReadsCommandGrammar. Backend Lint & Type Check (3.12)is red on this head. flake8 and black are clean locally, so this is likely mypy.
…on closer
A ) inside a parameter expansion such as ${v:-)} is the default word,
not a closer, but the span walk closed the substitution body there and
the commands after it were never scanned. _matching_close_paren now
jumps past the matching }; quoting runs through the span, nested {
opens a level, and a } inside a nested $(...) or backtick body belongs
to that command. An unclosed expansion fails closed.
Fixes kirodotdev#9181
Co-authored-by: ANIRUDDHA ADAK <aniruddhaadak80@users.noreply.github.com>
7173f92 to
732edb4
Compare
|
Thanks @aniruddhaadak80 for this fix. Main has moved a lot since this opened: the shell reader now lives in A fork PR cannot pass the Sensitive Change Review gate for files under |
Problem / Motivation
Goal: Make the shell-character walk expand parameters and skip comments the way a shell does, so a deny check sees the command that would actually run.
_iter_shell_chars(src/kiro_crew/security/shell_normalizer.py) models shell quote andescape handling, but did not handle two additional bash syntax categories:
${...}parameter expansion#commentsA literal
)inside$(echo ${v:-})or after a#comment arrived withactive=True, so_matching_close_parencounted it as the substitutioncloser and truncated the extracted body. The nested-payload extractor then
handed every downstream scan a fragment while bash runs the whole body, so a
nested publish hides below the truncation point --
is_deniedallows a commandbash executes (same shape as the documented quoted-paren truncation behind
_matching_close_paren).Why it matters
The
_iter_shell_charsgenerator is the single source of truth for shellquote/escape state walked by every consumer: the boundary walk, the git-publish
border walk, the nested-payload extractor, and
is_denied/is_sensitive_bash_command.When the machine did not know about
${...}or#, those constructs wereinvisible to it, and a
)inside them was treated as a bare)-- afalse-allow of the class security reviews have caught before.
Not a goal
What changed (motivation → approach → change)
Rebuilt on current main, where the shell reader now lives in
shell_normalizer_walk.pyand the#comment half is already fixed (_opens_commentin_matching_close_paren). Only the${...}half was still open: on main,$(: ${v:-)}; printf X)still ends the body at: ${v:-.Main handles comments inside
_matching_close_paren, not in_iter_shell_chars, so this fix does the same for${...}. When the walk meets an unpaired${, it jumps to the matching}(_matching_close_brace). Quotes count inside the span, a nested{adds a level, and a}inside a nested$(...)or backtick body belongs to that command. That last rule answers both review findings: the old head closed${v:-$(echo })}at the inner}. An unclosed expansion fails closed and returns the whole remainder._iter_shell_charsitself is unchanged.The package line budget in
test_security_gate_liveness.pygoes up by 90, with a note.Backwards compatibility
Compatible: substitution bodies only get longer, never shorter, so nothing allowed today becomes refused unless the hidden part of its body holds a refused command.
Removes nothing: no rule, key or function is removed or renamed.
Tests
test/test_security_shell_grammar.py::TestParameterExpansionHoldsItsOwnParens:${v:-)}, double- and single-quoted}, nested${, bare nested{,}inside$(...)and backticks,$${(PID, no expansion) and\$${(escaped first dollar);${v:-)}is now refused;}inside a nested substitution cannot close the expansion early, and the command stays refused;Every body shape was checked in bash. The 94 security, shell, argv-floor, push-gate and rm-floor test files pass locally (9,620 tests).
Proof: PROVEN test/test_security_shell_grammar.py::TestParameterExpansionHoldsItsOwnParens (head 732edb4)
Manual verification
N/A — unit coverage sufficient: the change is a pure text walk, and each case was checked against bash.
Related Issues
Fixes #9181. Original work by @aniruddhaadak80, kept as co-author.
Sibling sweep
Swept: git grep -n "_matching_close_paren(" -- src/
Sibling: src/kiro_crew/security/shell_normalizer_walk.py:279 -- changed here (via
_matching_close_paren)Sibling: src/kiro_crew/security/argv_floor_self.py:589 -- changed here (calls
_matching_close_paren, gets the fix)Sibling: src/kiro_crew/security/rm_floor.py:96 -- changed here (calls
_matching_close_paren, gets the fix)Pattern harvest
Not generalizable: one missing shell construct in a single span walk; the comment and case rules beside it already cover the other cases.
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.