Skip to content

fix(security): read text after a heredoc's closing line as unquoted, like bash - #40394

Open
buluoray wants to merge 1 commit into
mainfrom
crew/slack/self-floor-heredoc-case
Open

buluoray wants to merge 1 commit into
mainfrom
crew/slack/self-floor-heredoc-case

Conversation

@buluoray

@buluoray buluoray commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The argv floors read the text after a heredoc's closing line with no quote open, as bash does, so a quote character inside a heredoc body no longer changes how the following command lines are read.

bash ends a heredoc body at its closing line and starts the next line with no open quote, whatever quote characters the body held. The argv floors (self-protection-* subcommand floors and sandbox-escape-ssh-self) walked the whole command text with one quote state. A lone ' or " in a data body therefore turned the following command lines into quoted text, and those lines were not judged as commands.

Why it matters

The floors decide whether a self-management command or an ssh to this host is refused. When their reading of a multi-line command differs from bash's, a command line bash runs is judged as text. Agents write heredocs with quotes in the body all the time (Markdown, scripts, prose), so the floors must read those lines the way bash does.

Not a goal

What changed (motivation → approach → change)

Symptom → cause → change. The argv floors read a command through one shared quote/escape walk, shell_normalizer_walk._iter_shell_chars. That walk did not understand heredocs, so a quote character inside a heredoc body carried its quote state past the body's closing line, and the following command lines were read as quoted text. Nesting made it worse: a heredoc inside a $(...) left the substitution close-finder (which runs on the same walk) open, so it swallowed the tail and a later command line was never read as a command.

  • _iter_shell_chars now reads heredocs itself: a << / <<- outside quotes records its tag; at that line's end it finds each body's closing line; and the walk resets to unquoted where the closing line ends. Body characters change no quote state. The body steps are still yielded, so a shell fed a body is judged as before.
  • Every consumer inherits this from the one walk with no call-site change: the $(...) and backtick close-finders (_close_paren_walk / _matching_close_paren, _matching_close_backtick), read_commands, the word and segment splitters, _heredoc_line_pieces, and the git, rm, s3 and cron readers.
  • A new leaf module, shell_heredoc_lines.py, reads each heredoc's tag and closing line from the raw lines only. It skips a <<< here-string and a << inside an unclosed ((, imports nothing at load time, and never calls a close-finder, so the body search does not depend on the substitution parsing it feeds. read_commands now takes its tag and closing line from it, and its own copies are removed. It decodes an ANSI-C $'...' tag with the shared escapes helper before matching a closing line, as bash does.
  • The walk stays one forward pass: each walk reads each body once, so the cost is linear. argv_floor.py lists the new facade name; security.md describes the reading.

Backwards compatibility

Breaking: the argv floors now read the lines after a heredoc's closing line as commands even when the body held an unmatched quote, and read a $(...) that contains a heredoc to its real ). Main read those lines as quoted text, so main allowed four commands this change refuses: an ssh to this host, or kirocrew restart, hidden after a heredoc whose body holds a quote, in a $(...), a backtick, or an x=$(...) assignment. Each is the floor doing its job once the line is read as bash reads it. Writer sweep: no in-tree caller writes these shapes.

Removes nothing: no deny pattern, rule id, catalog row or operator toggle is removed or renamed. heredoc_closing_lines and _heredoc_tag are replaced by shell_heredoc_lines, an internal helper with no caller outside security/.

scripts/deny_diff.py --base origin/main on golden-paths.json plus 23 heredoc rows: 0 newly allowed; 4 newly refused, all self/ssh commands a heredoc body was hiding; 45 golden paths allowed at both refs, no regression.

Tests

  • test/test_denied_commands_security_heredoc_data.py: the quote-after-closing-line rows from the first approach, plus nesting rows where a self-subcommand or an ssh line follows a heredoc inside $(...), a backtick, <( ), and an x=$(...) assignment. The nesting rows fail first: 4 fail on origin/main and 2 on the PR's earlier head.
  • Related set (every test file naming _heredoc_data_view, argv_floor_self, argv_floor, denied_commands_security, shell_commands or shell_normalizer, plus facade mirror, facade mirror guards, split-patch reach, listing budget and gate liveness): 6,869 passed, 1 skipped. mypy, black, isort, flake8, subprocess encoding, line cap, comment history, brand and the flake ratchet pass.
  • Determinism: the rows judge strings only: no clock, process, file or network. test_heredoc_pieces_walk_the_text_once reads a character-counting string over 400 nested bodies and holds the walk to at most 3x the text length.

Proof: PROVEN test/test_denied_commands_security_heredoc_data.py::test_line_after_closing_line_starts_unquoted[cat <<'EOF'\nit's\nEOF\nkirocrew restart\n# '-self-protection-restart], test/test_denied_commands_security_heredoc_data.py::test_line_after_closing_line_starts_unquoted[cat <<EOF\nit's\nEOF\nkirocrew restart\n# '-self-protection-restart], test/test_denied_commands_security_heredoc_data.py::test_line_after_closing_line_starts_unquoted[cat <<-'EOF'\n\tit's\n\tEOF\nkirocrew restart\n# '-self-protection-restart], test/test_denied_commands_security_heredoc_data.py::test_line_after_closing_line_starts_unquoted[tee out.txt <<'EOF'\nit's\nEOF\nkirocrew restart\n# '-self-protection-restart], test/test_denied_commands_security_heredoc_data.py::test_line_after_closing_line_starts_unquoted[cat <<'A'\nit's\nA\ncat <<'B'\nok\nB\nkirocrew restart\n# '-self-protection-restart] (head 8e79a24)

Manual verification

N/A — the floors are pure functions of the command text, and the unit rows call them through is_denied.

Real-machine verification

N/A — the diff changes only how the argv floors read command text; it runs nothing on a machine.

Screenshots / video

N/A — no UI change.

Related Issues

Related: #39056 (the ssh-to-self floor's heredoc data reading).

Sibling sweep

Swept: git grep -n "heredocs=True|_iter_shell_chars" -- src/kiro_crew/security/
Sibling: src/kiro_crew/security/shell_normalizer_walk.py -- changed here
Sibling: src/kiro_crew/security/shell_heredoc_lines.py -- changed here
Sibling: src/kiro_crew/security/shell_normalizer.py -- changed here
Sibling: src/kiro_crew/security/shell_commands.py -- changed here
Sibling: src/kiro_crew/security/argv_floor_ssh.py -- changed here
Sibling: src/kiro_crew/security/rm_floor.py -- not affected -- it reads the same walk, so it inherits the fix
Sibling: src/kiro_crew/security/s3_direction.py -- not affected -- it reads bodies only to tell an upload from a download
Sibling: src/kiro_crew/security/argv_rules.py -- not affected -- its command reader already ends a body at its closing line

Pattern harvest

Not generalizable: the other heredoc readers already end a body at its closing line; the argv floors' frame reader was the one that did not.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 21, 2026 10:24
@buluoray
buluoray enabled auto-merge (squash) November 21, 2026 10:24
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/argv_floor.py

  • src/kiro_crew/security/argv_floor_ssh.py

  • src/kiro_crew/security/shell_commands.py

  • src/kiro_crew/security/shell_heredoc_lines.py

  • src/kiro_crew/security/shell_normalizer.py

  • src/kiro_crew/security/shell_normalizer_walk.py

  • No reasoning template for 8e79a24 yet.

How to clear this check:

  1. The PR author posts the template below as a PR comment, with
    every field filled (N/A needs a reason). Keep the
    Head commit line: it must name the current head SHA.
  2. The check turns green within a minute, with no push.
  3. A new push resets this check: post it again for the new head.

Only the author's comment counts this way. A reviewer can instead
paste the filled template into an "Approve" review on this head.
Merging still needs a reviewer's approval.

## Sensitive change review
- Head commit: 8e79a247451cbe14a36e340225351bd27f216917
- What rule changed:
- Before -> after (what was allowed/blocked, now):
- Worst case if wrong (breaks commands / leaks secret / locks user out):
- Evidence checked (which sandbox result proves it):
- No regression:
  - Tools still work (which tools/commands were run, result):
  - Backward compatible (old config / old data / old callers still work, how checked):
  - Existing tests (which suites passed on this head):
- How to undo:

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 21, 2026
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 8e79a247451cbe14a36e340225351bd27f216917 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 8e79a247451cbe14a36e340225351bd27f216917 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the heredoc reading goes into the one shared quote walk, which fixes the cause, and every new verdict refuses more rather than allows more.

Notes

  • The PR deletes the allow row cat > f.sh <<'EOF'\n…\nEOF\n# ssh %s id from test_a_quote_inside_a_substitution_rows_that_stay_allowed and adds the same command as a refused row. bash runs nothing after #, so main allowed it correctly, but only because of the old quote bug. This newly refused case is not among the four the ## Backwards compatibility section counts. Name it there as an accepted false denial.
  • _iter_shell_chars takes a start/state that lets a caller resume mid-text. A walk resumed after a << line never sees the opener, so it reads that body with the old quote state. Say in security.md that resumed walks are outside this fix, or add a test that pins it.

[DESIGN-REVIEWED] 8e79a24

Description read

[DESCRIPTION-READ] 0629a858658176b1c5073b228f0282ceb8a666dc0d0bd1cfb439eb9cd1bef21b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 8e79a247451cbe14a36e340225351bd27f216917 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix sits at the cause: it is one quote walk that every reader inherits. Older heredoc copies that the new reader could replace are still in the tree.

Notes

  • Item 3 — symptom-level: the walk now understands heredocs inside $(...), yet shell_commands._double_quoted_substitution_spans still blanks the bodies and re-reads them itself, and its docstring says that stays "quadratic". argv_rules._without_heredoc_bodies also still finds closing lines with its own _HEREDOC_OP regex. That is 2 copies, found by grepping def \w*heredoc\w* under src/kiro_crew. Subtraction: replace both with shell_heredoc_lines.heredoc_body_spans / heredoc_closing_line, or defer.
  • Item 5 — undeclared: the diff deletes the allow row "cat > f.sh <<'EOF'\n…\nEOF\n# ssh %s id" from test_denied_commands_security_ssh_self.py and puts it in the deny set. That row was in the list "Heredoc text keeps its old reading", but its line sits after EOF, so the pin contradicted its own wording. The description's "4 newly refused" count leaves this fifth refusal (a commented-out ssh line) out.

What this change ships

Inventory (6 items) — 4 justified

Intent: stop a quote character inside a heredoc body from hiding the self-protection and ssh-to-self command lines that follow it — FIX.

  1. A command line after a heredoc's closing line is judged as a command even when the body held a quote — justified
  2. A $(...) or backtick holding a heredoc now closes at its real ) or backtick — justified
  3. Heredoc tags and closing lines are read by one new shared module, and the reader's own copies are removed — symptom-level: 2 older heredoc readers remain
  4. A $'…' heredoc tag closes on its decoded line — justified
  5. A # ssh <self> comment line after a heredoc is now refused — undeclared
  6. Shorter docstrings for the walk and the backtick closer — justified

[FIRST-PRINCIPLES-REVIEWED] 8e79a24

Description read

[DESCRIPTION-READ] 323b22f17e39e6a38ae4a9c96d2f4b72152c6aff46fc01a8905de568e24208a1

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — 🔴 changes requested (blocking)

Reviewed 8e79a247451cbe14a36e340225351bd27f216917 — this comment is updated in place on each push.

Self-protection regression: the new heredoc logic in the shared quote walk can hide a live kirocrew restart that the gate denied before this PR.

BLOCKING — src/kiro_crew/security/shell_normalizer_walk.py:605

elif state == 0 and not ends and ch == "<" and (tag := heredoc_opener(text, i)):

Trigger: the agent submits : # <<X\necho '\nX\n' ; kirocrew restart ; echo \'. _iter_shell_chars does not model # comments, so it treats the commented <<X as an opener. heredoc_bodies then closes a fake body at line X, and line 597 resets state to 0. This text reaches the walk through _is_self_restart, then _self_token_frames, _shell_payload_walk, _self_tokens and _heredoc_line_pieces. _heredoc_data_view returns the text unchanged, because it holds several commands.
Outcome: in bash, line 1 is a comment, the ' on line 2 is closed by the leading ' on line 4, and kirocrew restart runs. The walk restarts unquoted after X, so line 4's ' opens a quote and the unquoted \' closes it. The pieces then give shlex ' ; kirocrew restart ; echo \' as one token, quotes balanced in both readings, and the self-protection-restart floor allows the command. Before this PR the walk paired these quotes the way bash does and the floor denied it.
Anchor: residual/security
Fix: have heredoc_opener reject a << that sits after a live # comment opener on its line (_opens_comment), or fail closed when the walk's state at a body's end is not 0, instead of resetting it.

BLOCKING — src/kiro_crew/security/shell_normalizer.py:323

pieces = _heredoc_line_pieces(command) if "\n" in command else [command]

Trigger: the agent submits cat <<EOF\n'\neof\n'\nEOF\nkirocrew restart\n# '. _self_tokens receives the lowercased text, so the new body search in heredoc_closing_line compares lines against tag eof and closes the body at the line eof, not at EOF.
Outcome: bash ends the body at EOF, runs kirocrew restart, and reads # ' as a comment. The walk restarts unquoted after the early eof, and the ' on line 4 pairs with the ' in # '. That turns kirocrew restart into a single quoted token, so self-protection-restart (and the ssh-self mask that uses heredoc_body_spans) allows the command. Before this PR the walk had no case-sensitive body logic and the command was denied.
Anchor: residual/security
Fix: find the body spans on the original-case text and map the offsets onto the lowercased copy before _heredoc_line_pieces and _mask_quoted_separators cut or reset on them.

[BLOCK-MERGE] 8e79a24
[OPUS-REVIEWED] 8e79a24

Verdict parsed from the review's SHA-scoped output markers for commit 8e79a247451cbe14a36e340225351bd27f216917.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 8e79a247451cbe14a36e340225351bd27f216917 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — 🔴 changes requested (blocking)

GPT 6.1 found at least one blocking issue that must be resolved before merging 8e79a247451cbe14a36e340225351bd27f216917. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/security/shell_normalizer_walk.py:605 -- Non-heredoc << resets quoting and bypasses the SSH-to-self floor

elif state == 0 and not ends and ch == "<" and (tag := heredoc_opener(text, i)):
Trigger: Submit : $((\n1 << 2\n))\n: '\n2\n'\nssh -p 22 localhost id\n# ' (\n denotes newlines). is_denied → _reaches_self_over_ssh → _mask_quoted_separators → heredoc_body_spans → this walk mistakes the multiline arithmetic shift for a heredoc. Replacing the arithmetic prefix with # <<2 reproduces the same defect.
Outcome: The fake closing line resets quoting inside the literal argument, hiding the subsequent executable SSH command. HEAD returns None for both inputs; the base refuses both, and Bash executes the SSH invocation. On a host permitting authenticated local SSH, this bypasses the agent sandbox through sshd.
Anchor: residual/security
Fix: Queue only real heredoc redirections: exclude comment text and track arithmetic nesting across newlines instead of counting parentheses only on the current line.

[BLOCK-MERGE] 8e79a24
[GPT-REVIEWED] 8e79a24

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

The one fenced finding (F1) still blocks. The input had no adjudicable findings.

F1 is UNBOUNDED harm: it bypasses a deny floor. The quote walk treats any << as a heredoc opener once it is unquoted, no heredoc is pending, and no closing line is ahead (src/kiro_crew/security/shell_normalizer_walk.py:605). It skips comment text only when the opener check (heredoc_opener) rejects the <<, and that check never looks for a # comment. For arithmetic it only compares (( and )) counts on the current line (src/kiro_crew/security/shell_heredoc_lines.py:26-28), so # <<2 or a << shift inside a multiline $((…)) opens a fake heredoc. Its closing line then resets quoting to unquoted (shell_normalizer_walk.py:595-596), and the SSH command hidden inside the literal argument goes unseen. No condition here is extreme: the agent writes the command text itself, and that text can be steered by prompt injection.

[ADJUDICATION] 8e79a247451cbe14a36e340225351bd27f216917 total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] 8e79a247451cbe14a36e340225351bd27f216917

[ADJUDICATION-FENCED] 8e79a247451cbe14a36e340225351bd27f216917 fenced=1 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/security/shell_normalizer_walk.py:605 -- heredoc_opener (shell_heredoc_lines.py:26-28) never excludes comment text and counts arithmetic parens only on the current line, so an agent-authored `# <<2` or multiline `$((1 << 2))` opens a fake heredoc whose closing line resets quoting, which is an easy input to craft that hides a self-SSH command from the deny floor.
[GPT-ADJUDICATED-FENCED] 8e79a247451cbe14a36e340225351bd27f216917

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 8e79a247451cbe14a36e340225351bd27f216917 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- 🔴 BLOCK (blocking)

Which legitimate operations does 8e79a247451cbe14a36e340225351bd27f216917 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: a script-confirmed newly-refused operation.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 21 classified, 1 newly refused, 20 still allowed.
  • ubuntu-latest (platform posix): 21 classified, 1 newly refused, 20 still allowed.
  • windows-latest (platform windows): 3 classified, 0 newly refused, 3 still allowed.

Newly refused (confirmed by the real classifier at both refs)

  • cat > notes.md <<'EOF' it's done EOF ssh deploy@build.example.com uptime (posix, leg macos-latest)
    • Who loses it: an operator records a note then checks a remote build host's uptime (not this machine)
    • Refused by: deny-rules -- Blocked by security policy: (?:\A|[;&|\n]|\$\()[\s"'(]*(?:[\w.:/\\-]*[/\\])?(?:ssh|scp|sftp|rsync)(?:\.exe)?\s+(?:\S*@)?(?:localhost|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|\[?::1\]?|\$\(\s*hostname|\s*hostname|${?hostname}?)(?=[\s:'");&|#]|\Z)
      Matched structurally on the command's argv, not by the pat
  • cat > notes.md <<'EOF' it's done EOF ssh deploy@build.example.com uptime (posix, leg ubuntu-latest)
    • Who loses it: an operator records a note then checks a remote build host's uptime (not this machine)
    • Refused by: deny-rules -- Blocked by security policy: (?:\A|[;&|\n]|\$\()[\s"'(]*(?:[\w.:/\\-]*[/\\])?(?:ssh|scp|sftp|rsync)(?:\.exe)?\s+(?:\S*@)?(?:localhost|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|\[?::1\]?|\$\(\s*hostname|\s*hostname|${?hostname}?)(?=[\s:'");&|#]|\Z)
      Matched structurally on the command's argv, not by the pat

Each row above is an operation that the base ref ALLOWS and this change REFUSES. Narrow the rule at the tier named. Withdrawing a row is not the way to green: that is its own pull request, on its own merits.

Scope-Verdict: PASS

Reason: The change only affects how the shared shell quote walk reads heredocs. Text after a body's closing line now starts unquoted, so a quote character in the body no longer hides the commands that follow it. Every effect of that is a shell command line, so the classifier can rule on all of it; I have no confirmed regression to report. Separately, the diff deletes the allowed test row cat > f.sh <<'EOF'…EOF\n# ssh <self> id and now expects that comment line to be refused. It is a self-ssh comment, close to the threat shape, so I did not submit it as a row. Whether to keep it allowed is for the authors to decide.

TIGHTENING: shell_normalizer_walk._iter_shell_chars, which every tier reads, plus _self_tokens pieces and the ssh-floor separator mask -- it refuses a sensitive command hidden behind a quote that bash closed at the heredoc boundary: "while ends and i >= ends[0]: state, ansi, dollar_run = 0, False, 0" and "if i == body_tail: quote, body_tail = None, -1".

ADJUDICATED: 24 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: 21 posix rows and 3 windows rows; no any rows.

  • posix: these rows cover the shapes the diff assumes bash reads:
    • quoted, unquoted, <<- and $'…' tags
    • two bodies in a row
    • bodies inside $(…), including the commit and PR-body recipes and a variable capture
    • here-strings
    • arithmetic << in $((…)), ((…<<=…)) and legacy $[…]
    • ordinary daily commands after a body that holds a quote: git push, kirocrew status, kirocrew cron list, push_guard, ssh to a remote host
    • a body that only holds data and mentions ssh after an apostrophe line, which probes the ssh mask now keeping the body's line breaks
  • windows: the same heredoc shapes as Git Bash runs them, with .venv\Scripts\python.exe and \ paths.

The diff assumes nothing about paths, executable names or environment variables beyond bash heredoc grammar. cmd and PowerShell have no << heredoc for this change to touch, so I wrote no rows for them.

[SCOPE-REVIEWED] 8e79a24

The confirmed rows, in the committed corpus's own shape
{
  "golden_paths": [
    {
      "kind": "shell",
      "command_or_flow": "cat > notes.md <<'EOF'\nit's done\nEOF\nssh deploy@build.example.com uptime",
      "platform": "posix",
      "reason": "an operator records a note then checks a remote build host's uptime (not this machine)"
    }
  ]
}

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 21, 2026
@buluoray
buluoray force-pushed the crew/slack/self-floor-heredoc-case branch from d4de175 to 29d1a2d Compare November 21, 2026 11:38
@buluoray

Copy link
Copy Markdown
Contributor Author

resolution: one forward walk; no re-walk from an earlier offset

  • span=c65544f87915 _heredoc_line_pieces re-walks from at, so the gate is quadratic in the number of heredocs that open inside a quote

fixed in 29d1a2d: _heredoc_line_pieces now walks the text once, carrying the quote state forward, and never re-walks command[at:start]. test_heredoc_pieces_walk_the_text_once feeds a character-counting string and asserts at most 3x the length is read; it fails on d4de175 (2,657,090 reads for the 400-body case) and passes now. Wall time for 1,600 heredocs that open inside a quote: 2.6-4.3s, matching origin/main's 2.2-3.4s, down from 15-26s.

@buluoray

Copy link
Copy Markdown
Contributor Author

resolution: body is its own piece; the command line is read without the body's quote

  • span=5e784cc06287 Heredoc splitting leaves an unmatched quote in the command-line piece, so a quote-spliced executable is read raw

fixed in 29d1a2d: _heredoc_line_pieces now cuts three pieces around each heredoc, so the body is its own piece and its quote state reaches neither the opener line before it nor the lines after it. The opener line is tokenized on its own, so s's'h normalizes to ssh and kiro'crew' to kirocrew. Both of your commands (s's'h -p 22 localhost id and kiro'crew' restart, each with a quoted heredoc body and a trailing # ') are refused at 29d1a2d, as on origin/main; they were allowed on d4de175. Rows in test_line_before_body_is_read_without_the_body_quote.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 21, 2026
@buluoray
buluoray disabled auto-merge November 21, 2026 12:41
@buluoray
buluoray marked this pull request as draft November 21, 2026 12:41
@buluoray
buluoray force-pushed the crew/slack/self-floor-heredoc-case branch from 29d1a2d to d98133e Compare November 21, 2026 13:18
@buluoray
buluoray marked this pull request as ready for review November 21, 2026 13:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 21, 2026
…like bash

bash reads a heredoc body line by line, up to its closing line, and reads the
next line with no quote open, whatever quote characters the body held. It does
the same for a heredoc inside a $(...) substitution or a backtick. The shared
shell quote walk read the whole text with one quote state, so a lone quote in
a body turned the lines after it into quoted text, and inside a substitution it
hid the ) that closes the substitution.

The quote walk (_iter_shell_chars) now reads heredocs itself: a << outside
quotes opens a body on the next line, and the walk starts unquoted where the
body's closing line ends. The body's steps are still yielded, so a shell fed
the body is judged as before. Every reader built on the walk inherits this,
including the $(...) and backtick close-finders, the word and segment
splitters, and read_commands, with no change at their call sites.

A new module, shell_heredoc_lines, reads the tag after << and each body's
closing line from raw lines. It never looks for the close of a substitution.
It decodes the escapes of a $'...' tag with the shared ANSI-C decoder, as bash
does before it compares lines with the tag. read_commands reads its tags and
closing lines from it, which removes its own copies. _self_tokens tokenizes the
text in pieces cut around each body, in the same walk (_heredoc_line_pieces),
and the ssh floor's separator mask starts fresh after a body's closing line
(heredoc_body_spans). Each walk reads each body once, so the cost stays linear
in the text.
@buluoray
buluoray force-pushed the crew/slack/self-floor-heredoc-case branch from d98133e to 8e79a24 Compare November 21, 2026 16:20
@buluoray

Copy link
Copy Markdown
Contributor Author

resolution: the tag is decoded before a closing line is matched

  • span=b5911f931fe6 Undecoded ANSI-C heredoc delimiters read the tag literally

fixed in 8e79a24: heredoc_tag in shell_heredoc_lines.py now decodes an ANSI-C $'...' tag with the shared escapes helper (shell_normalizer_escapes._decode_ansi_c_body, the same decoder main's old reader used through _shell_tokens) before it matches a closing line, so <<$'e\x6ff' closes on eof as bash does. A plain, single- or double-quoted tag keeps its handling.

Your command (echo "$(cat <<$'e\x6ff' … eof … ssh … )") and the octal and \x65 siblings are refused at 8e79a24, as on origin/main; each was allowed on d98133e. Rows in test_ansi_c_tag_closes_on_its_decoded_line.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant