Skip to content

perf(security): read each mint-verb suffix once per gate call - #39929

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/39617-mint-reader-unclosed-subst
Nov 20, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/39617-mint-reader-unclosed-subst

Conversation

@JLargent42

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Cut the credential-mint reader's work on a command of never-closing command substitutions from cubic to quadratic in the command length, with no verdict changed.

is_denied runs the mint reader on the synchronous PreToolUse gate. A command like a=$(: #); a=$(: #); ...; <program> doctor has substitutions whose ) is swallowed by the # comment, so none closes. The payload walk turns each opener into a frame holding the whole remainder (N frames). Every frame then resolves its N assignments, and each one reads its own suffix for the mint verb. That is N frames x N suffixes x an O(N) read. The issue measured 10.4 s at about 2 KiB and more than 180 s at about 8 KiB.

Why it matters

One agent-typed command under the 20 KiB cap can hold the gate's event loop long enough to trip the loop-stall watchdog and hard-exit the gateway. No verdict is wrong; the harm is the stall.

Not a goal

  • Making the rest of the per-frame work linear. After this change the remaining cost is the N frames each being lexed and walked by the floors (payload walk, self-subcommand scan, git-publish scan), which is quadratic. Cutting the frames themselves is the issue's candidate A1 and needs the unterminated-versus-unprovable split it describes.
  • Touching PR fix(security): refuse a mint whose verb comes from a quoted substitution #37077's hunks. This diff changes only _mint_verb_in_substitution and adds one helper above _resolve_local_assignments; it does not edit _resolve_local_assignments, _shell_payload_walk_uncached or shell_normalizer_walk.py.
  • Changing any closer, quote walk or refusal rule.

What changed (motivation → approach → change)

The suffix read, _mint_verb_in_substitution(tokens, idx), joins tokens[idx:] and looks for a mint verb in every substitution body of that text. It is a pure function of the joined text. The frames of a never-closing command are suffixes of one another, so they share almost all of their assignment suffixes, and each frame was reading them again.

The body moved into _mint_verb_in_text(joined), marked once_per_call (the existing per-gate-call memo in call_views.py, dropped when is_denied returns). _mint_verb_in_substitution now calls it with the joined suffix. Outside a gate call the memo is off and the read is the same code as before.

The memo holds one entry per distinct suffix of the command for the length of the call, so a command near the 20 KiB cap holds about 10 KiB x its assignment count in keys, then drops them.

Measured with the new test's counter (characters handed to _substitution_bodies):

repeats length main this PR
30 315 61,870 10,395
60 615 425,225 38,775
200 2,015 not run 409,215
800 8,015 not run 6,436,815

Wall clock on the ARM desk host (is_denied, one call): 200 repeats 1.7 s (the issue measured 10.4 s on main), 800 repeats 27.8 s (the issue measured more than 180 s). The 800 case still stalls; see Not a goal.

Backwards compatibility

Compatible: the memoized function is the old loop moved unchanged and keyed by its only input, so every command gets the answer it got before.

Removes nothing: no refusal, rule, flag or public name is removed or narrowed; _mint_verb_in_substitution keeps its signature.

Tests

test/test_security_mint_reader_cost.py:

  • test_never_closing_substitutions_are_read_in_quadratic_work[30|60] counts the characters handed to _substitution_bodies and bounds them by a quarter of the length squared. It fails on main (61,870 and 425,225 against bounds of 24,806 and 94,556) and passes here. It counts work, not time, so a loaded host does not move it.
  • test_the_memo_leaves_the_verdict_where_it_was pins the verdict main gave nine commands, including the issue's three multi-line and lettered-glob reproducers (allowed on main), the mint verb behind an unclosed substitution (refused), and plain commands.
  • test_the_memo_answers_every_suffix_as_the_unmemoized_reader_does compares the memoized answer for every suffix start of the same commands with the old loop, inside and outside a gate call.

Proof: PROVEN test/test_security_mint_reader_cost.py::test_never_closing_substitutions_are_read_in_quadratic_work[30], test/test_security_mint_reader_cost.py::test_never_closing_substitutions_are_read_in_quadratic_work[60] (head ba23b92)

Local gate: scripts/local-gate.py ran 7187 passed, 1 failed. The failure, test_security_conductor_verify_fix.py::TestTheFenceClassifiedAgainstIsTheWorktrees::test_a_worktree_without_the_package_does_not_borrow_the_installed_fence, also fails on origin/main on this host (the child process cannot import kiro_crew from this environment), so it is unrelated.

Manual verification

N/A — the unit tests drive is_denied itself.

Real-machine verification

Machine: own computer (dev desk host), Linux, aarch64, 64 vCPU
Before (main c251020): python -m pytest -n 0 test/test_security_mint_reader_cost.py -> 2 failed (30 repeats: 61870 characters, 60 repeats: 425225), 18 passed.
After (head ba23b92): same command -> 20 passed.
Cleanup: own computer; no machine to delete.

Related Issues

Fixes #39617

Related: #37077 touches the same reader and is independent of this change.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

A command of N assignments whose substitutions never close becomes N
nested frames, and each frame read every suffix of its tokens for the
mint verb again, so the reader's work grew with the cube of the length.
The read is a pure function of the suffix text, so it is kept per call.
No verdict changes.
@JLargent42
JLargent42 requested review from a team and KiroCrewCaptain as code owners November 20, 2026 10:07
@JLargent42 JLargent42 self-assigned this Nov 20, 2026
@JLargent42
JLargent42 requested a review from CrysisDeu November 20, 2026 10:07
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

Head: ba23b92. Written by the PR author's session; it needs a human reviewer's approval before merge.

  • What rule changed: No rule changed. The credential-mint reader's per-suffix read (_mint_verb_in_substitution in src/kiro_crew/security/shell_normalizer.py) moved into a pure helper, _mint_verb_in_text, which is kept per gate call through the existing once_per_call memo.
  • Before -> after: Before, a command of N never-closing substitutions made the reader read each assignment suffix once per frame (61,870 characters at 30 repeats, 425,225 at 60). After, each distinct suffix is read once per call (10,395 and 38,775). Every command is refused or allowed as before.
  • Worst case if wrong: A stale memo answer could let a mint verb inside a substitution go unread, which would allow a credential mint the gate refuses today. The key is the full suffix text, the helper reads nothing else, and the memo is dropped when is_denied returns.
  • Evidence checked: test_the_memo_answers_every_suffix_as_the_unmemoized_reader_does compares the answer for every suffix start of nine commands with the old loop, inside and outside a gate call. test_the_memo_leaves_the_verdict_where_it_was pins the verdicts main gave those commands, including the three multi-line and glob-self-program reproducers from the issue. test_deny_gate_call_sharing.py and the local gate (7187 passed) cover the memo's other readers.
  • Tools still work: Yes. Ordinary commands go through the same reader with the same text; only a repeat read of identical text is skipped.
  • Backward compatible: Yes. The function keeps its name and signature, and no refusal or rule is added or removed.
  • Existing tests: The local gate passed except test_a_worktree_without_the_package_does_not_borrow_the_installed_fence, which also fails on origin/main on this host because the child process cannot import kiro_crew.
  • How to undo: Revert the commit; _mint_verb_in_substitution returns to the inline loop and the line-budget row goes with it.

@github-actions

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/shell_normalizer.py

  • @JLargent42 posted a complete reasoning template for ba23b92.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. It memoizes one pure reader through the existing per-call view seam, nothing is taken away, and the description says plainly that the frames are still quadratic.

Notes

[DESIGN-REVIEWED] ba23b92

Description read

[DESCRIPTION-READ] 7a9ab7f6fc06f85ac1490a209035a1ef327544d7d00cdb38eb77590a664dc562

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix reuses the existing once_per_call memo on a pure reader. The description states that the per-frame lexing, which keeps the cost quadratic, is deferred to the issue's candidate A1.

What this change ships

Inventory (4 items) — 4 justified

Intent: stop one agent-typed command from stalling the synchronous PreToolUse gate long enough to trip the watchdog. This is a FIX (#39617).

  1. The credential-mint suffix read is computed once per distinct suffix per gate call, not once per frame, using the existing call_views.once_per_call. Cost goes from cubic to quadratic. — justified
  2. The security spec's list of memoized readers now names _mint_verb_in_text. — justified
  3. A line-budget row for the security reader growth, following the existing test/security_line_budget/ convention (31 sibling rows). — justified
  4. New tests: a work-count bound that fails on base, verdict pins, and a check that the memoized and unmemoized readers agree. — justified

[FIRST-PRINCIPLES-REVIEWED] ba23b92

Description read

[DESCRIPTION-READ] 7da0a985e53b9ea8f878b582d34cd429c310bfdd4e43896d6c82f43fba42e10c

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] ba23b92

False positive or not applicable? A repository writer can comment:
/ai-review override gpt ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] ba23b92

Verdict parsed from the review's SHA-scoped output markers for commit ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does ba23b92d5acdf70c8f548b781b02a6fe92cbc2c9 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 12 classified, 0 newly refused, 11 still allowed.
  • ubuntu-latest (platform posix): 12 classified, 0 newly refused, 11 still allowed.
  • windows-latest (platform windows): 4 classified, 0 newly refused, 3 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: This change adds no tightening. It only memoizes a pure reader that already existed: _mint_verb_in_substitution now passes its joined suffix to _mint_verb_in_text, which is marked @once_per_call(). The decision logic itself (_substitution_bodies plus _is_mint_verb) is unchanged. The cache is keyed on (fn, text) and lives for one gate call, so the same text gets the same verdict. I wrote no candidate shaped like the attack, and I have no script-confirmed regression.

TIGHTENING: none. The patch is a cost change. _mint_verb_in_text caches, for each suffix text, whether that suffix contains the credential-mint verb. That answer is what makes the gate bind $T to token for the threat T=$(printf <verb>); <name> $T. The hunk is - joined = " ".join(tokens[idx:]) / + return _mint_verb_in_text(" ".join(tokens[idx:])) / +@once_per_call() / +def _mint_verb_in_text(joined: str) -> bool:. The loop body underneath it has not changed.

ADJUDICATED: 16 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: The diff makes no assumption about any platform. It adds no path, separator, interpreter, executable name or environment variable, and it only re-routes a text reader. Posix (12 rows): bash $(...) assignments followed by ordinary git, gh, pytest and kirocrew commands. These include several assignments in one command, a comment inside a substitution that is closed normally, a substitution body that contains the bare verb word but feeds only echo, and the attack's T=$(...); kirocrew ... shape with a non-mint verb. Those are the inputs whose suffixes the memo now shares. Windows (4 rows): a PowerShell $var = ... capture, a cmd.exe for /f capture, and kirocrew.exe. They check that the same operations, spelled for the other tokenizer, still reach the reader the same way. No platform was assumed that I could not cover.

[SCOPE-REVIEWED] ba23b92

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 20, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 20, 2026 11:41

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review checked head ba23b92. The body of _mint_verb_in_substitution moved unchanged into _mint_verb_in_text, now under the existing once_per_call memo, and its only caller and signature are the same as before. On this head, the new test file, the deny gate call sharing tests, the gate liveness tests (including the package line budget) and the denied commands and shell grammar suites all passed. When the @once_per_call() decorator is removed, both cost tests fail with exactly the numbers the PR quotes, and all 9 verdict rows also pass on main. One thing done well: the cost test counts the characters handed to _substitution_bodies instead of timing a call, so it gives the same result on a loaded runner.

Security: no rule, refusal or trust boundary changes. The memoized function depends only on its text argument. It reads no file, environment, clock or config. The memo is a per-call ContextVar opened by sharing_views on is_denied. It keeps no module-level state, does not store a raised answer and computes afresh outside a gate call. So a stale answer, or one carried over from another call, cannot let a mint through.

Optional:

  • Fixes #39617 will close the issue on merge. By the PR's own table, the 800-repeat case still takes 27.8 s, which is longer than the default 25 s loop-stall exit_after. Changing it to Part of #39617, or filing a follow-up for candidate A1 and the 8 KiB pin, would keep that work tracked. This is a body edit only.
  • test_the_memo_answers_every_suffix_as_the_unmemoized_reader_does never returns a cached answer, because every suffix is a different string. If it called _every_start twice inside one sharing_views scope, it would also cover the cache-hit path.
  • The suffix join and memo key still cost O(N) per read, at C speed. The quadratic claim holds for the body scan the counter measures.

@KiroCrewCaptain
KiroCrewCaptain merged commit 98f989e into main Nov 20, 2026
103 of 105 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/39617-mint-reader-unclosed-subst branch November 20, 2026 11:41
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Credential-mint reader is O(n^3) on never-closing command substitutions (synchronous PreToolUse gate stall)

2 participants