Repository navigation
perf(security): read each mint-verb suffix once per gate call - #39929
Conversation
A command of N assignments whose substitutions never close becomes N nested frames, and each frame read every suffix of its tokens for the mint verb again, so the reader's work grew with the cube of the length. The read is a pure function of the suffix text, so it is kept per call. No verdict changes.
Sensitive change reviewHead: ba23b92. Written by the PR author's session; it needs a human reviewer's approval before merge.
|
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns. It memoizes one pure reader through the existing per-call view seam, nothing is taken away, and the description says plainly that the frames are still quadratic. Notes
[DESIGN-REVIEWED] ba23b92 Description read[DESCRIPTION-READ] 7a9ab7f6fc06f85ac1490a209035a1ef327544d7d00cdb38eb77590a664dc562 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The fix reuses the existing What this change shipsInventory (4 items) — 4 justifiedIntent: stop one agent-typed command from stalling the synchronous PreToolUse gate long enough to trip the watchdog. This is a FIX (#39617).
[FIRST-PRINCIPLES-REVIEWED] ba23b92 Description read[DESCRIPTION-READ] 7da0a985e53b9ea8f878b582d34cd429c310bfdd4e43896d6c82f43fba42e10c That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: This change adds no tightening. It only memoizes a pure reader that already existed: TIGHTENING: none. The patch is a cost change. ADJUDICATED: 16 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: The diff makes no assumption about any platform. It adds no path, separator, interpreter, executable name or environment variable, and it only re-routes a text reader. Posix (12 rows): bash [SCOPE-REVIEWED] ba23b92 |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
This review checked head ba23b92. The body of _mint_verb_in_substitution moved unchanged into _mint_verb_in_text, now under the existing once_per_call memo, and its only caller and signature are the same as before. On this head, the new test file, the deny gate call sharing tests, the gate liveness tests (including the package line budget) and the denied commands and shell grammar suites all passed. When the @once_per_call() decorator is removed, both cost tests fail with exactly the numbers the PR quotes, and all 9 verdict rows also pass on main. One thing done well: the cost test counts the characters handed to _substitution_bodies instead of timing a call, so it gives the same result on a loaded runner.
Security: no rule, refusal or trust boundary changes. The memoized function depends only on its text argument. It reads no file, environment, clock or config. The memo is a per-call ContextVar opened by sharing_views on is_denied. It keeps no module-level state, does not store a raised answer and computes afresh outside a gate call. So a stale answer, or one carried over from another call, cannot let a mint through.
Optional:
Fixes #39617will close the issue on merge. By the PR's own table, the 800-repeat case still takes 27.8 s, which is longer than the default 25 s loop-stallexit_after. Changing it toPart of #39617, or filing a follow-up for candidate A1 and the 8 KiB pin, would keep that work tracked. This is a body edit only.test_the_memo_answers_every_suffix_as_the_unmemoized_reader_doesnever returns a cached answer, because every suffix is a different string. If it called_every_starttwice inside onesharing_viewsscope, it would also cover the cache-hit path.- The suffix join and memo key still cost O(N) per read, at C speed. The quadratic claim holds for the body scan the counter measures.
Problem / Motivation
Goal: Cut the credential-mint reader's work on a command of never-closing command substitutions from cubic to quadratic in the command length, with no verdict changed.
is_deniedruns the mint reader on the synchronous PreToolUse gate. A command likea=$(: #); a=$(: #); ...; <program> doctorhas substitutions whose)is swallowed by the#comment, so none closes. The payload walk turns each opener into a frame holding the whole remainder (N frames). Every frame then resolves its N assignments, and each one reads its own suffix for the mint verb. That is N frames x N suffixes x an O(N) read. The issue measured 10.4 s at about 2 KiB and more than 180 s at about 8 KiB.Why it matters
One agent-typed command under the 20 KiB cap can hold the gate's event loop long enough to trip the loop-stall watchdog and hard-exit the gateway. No verdict is wrong; the harm is the stall.
Not a goal
_mint_verb_in_substitutionand adds one helper above_resolve_local_assignments; it does not edit_resolve_local_assignments,_shell_payload_walk_uncachedorshell_normalizer_walk.py.What changed (motivation → approach → change)
The suffix read,
_mint_verb_in_substitution(tokens, idx), joinstokens[idx:]and looks for a mint verb in every substitution body of that text. It is a pure function of the joined text. The frames of a never-closing command are suffixes of one another, so they share almost all of their assignment suffixes, and each frame was reading them again.The body moved into
_mint_verb_in_text(joined), markedonce_per_call(the existing per-gate-call memo incall_views.py, dropped whenis_deniedreturns)._mint_verb_in_substitutionnow calls it with the joined suffix. Outside a gate call the memo is off and the read is the same code as before.The memo holds one entry per distinct suffix of the command for the length of the call, so a command near the 20 KiB cap holds about 10 KiB x its assignment count in keys, then drops them.
Measured with the new test's counter (characters handed to
_substitution_bodies):Wall clock on the ARM desk host (
is_denied, one call): 200 repeats 1.7 s (the issue measured 10.4 s on main), 800 repeats 27.8 s (the issue measured more than 180 s). The 800 case still stalls; see Not a goal.Backwards compatibility
Compatible: the memoized function is the old loop moved unchanged and keyed by its only input, so every command gets the answer it got before.
Removes nothing: no refusal, rule, flag or public name is removed or narrowed;
_mint_verb_in_substitutionkeeps its signature.Tests
test/test_security_mint_reader_cost.py:test_never_closing_substitutions_are_read_in_quadratic_work[30|60]counts the characters handed to_substitution_bodiesand bounds them by a quarter of the length squared. It fails on main (61,870 and 425,225 against bounds of 24,806 and 94,556) and passes here. It counts work, not time, so a loaded host does not move it.test_the_memo_leaves_the_verdict_where_it_waspins the verdict main gave nine commands, including the issue's three multi-line and lettered-glob reproducers (allowed on main), the mint verb behind an unclosed substitution (refused), and plain commands.test_the_memo_answers_every_suffix_as_the_unmemoized_reader_doescompares the memoized answer for every suffix start of the same commands with the old loop, inside and outside a gate call.Proof: PROVEN test/test_security_mint_reader_cost.py::test_never_closing_substitutions_are_read_in_quadratic_work[30], test/test_security_mint_reader_cost.py::test_never_closing_substitutions_are_read_in_quadratic_work[60] (head ba23b92)
Local gate:
scripts/local-gate.pyran 7187 passed, 1 failed. The failure,test_security_conductor_verify_fix.py::TestTheFenceClassifiedAgainstIsTheWorktrees::test_a_worktree_without_the_package_does_not_borrow_the_installed_fence, also fails on origin/main on this host (the child process cannot importkiro_crewfrom this environment), so it is unrelated.Manual verification
N/A — the unit tests drive
is_denieditself.Real-machine verification
Machine: own computer (dev desk host), Linux, aarch64, 64 vCPU
Before (main c251020):
python -m pytest -n 0 test/test_security_mint_reader_cost.py-> 2 failed (30 repeats: 61870 characters, 60 repeats: 425225), 18 passed.After (head ba23b92): same command -> 20 passed.
Cleanup: own computer; no machine to delete.
Related Issues
Fixes #39617
Related: #37077 touches the same reader and is independent of this change.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.