Repository navigation
test: keep the near-ceiling shell title test off the slow regex path - #25952
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only input change; the size boundary under test is unchanged, and the slow regex behind it is tracked separately in #25954. Notes
[DESIGN-REVIEWED] 8f9d22d Description read[DESCRIPTION-READ] ea2b3aefabbf4553b054e3f72f9f224049a2a70d4653385f74802f3f275dae21 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
The only red on head 34c4d03 is 🤖 AI-assisted comment 🤖 |
The test fed one 20 KiB word to the deny rules. Two rules with a \w+ run are quadratic on such a word, so the call took about 14s on Linux and passed the 120s timeout on a Windows runner, crashing the worker. Use short words of the same total length: the size boundary under test is unchanged and the call takes about 1s. Closes #23346
34c4d03 to
8f9d22d
Compare
|
Review-ready on head 8f9d22d: PR Readiness passed (label 🤖 AI-assisted comment 🤖 |
chenmingwei23
left a comment
There was a problem hiding this comment.
Approved as part of a bulk pass over readiness: passed PRs.
Problem / Motivation
Goal:
test_a_command_just_under_the_size_ceiling_is_not_pushed_over_itstops crashing its xdist worker on Windows.On Windows shard 1/8 of PR #23334 the test ran 224s and its worker died (
[gw1] node down: Not properly terminated). On Linux the same test takes about 14s, by far the slowest in its file.Why it matters
A crashed worker turns an unrelated PR's Windows shard red. Every author then has to rerun or explain it.
Not a goal
What changed (motivation → approach → change)
The test built its command from one 20 KiB word (
"a" * N). A profile showed 13.7s of the run insidere.Pattern.search, all called fromdenied_rules._DenyMatcher.match. Two rules with a\w+run (self-protection-kill-interpreter,credential-exfil-kirocrew-token-argv) cost about 0.84s each per pass on that word, and the gate makes several passes. A slower Windows runner multiplied that past the 120s timeout.The test only cares about length: a command that fits the ceiling must not be refused because the
Running:prefix pushes the title over it. So the filler is now short words ("a a a ...") of the same total length. The deny rules then run in about 0.03s per pass and the test takes about 1s. A new assert pins the command at exactlyMAX_SCANNABLE_COMMAND_CHARS - 5, as before.Backwards compatibility
Compatible: test-only change; no product code is touched.
Removes nothing: the test keeps the same length and the same assertions, plus one new length assert.
Tests
test/test_llm_helpers_tool_input_offload.py::TestTruncatedShellTitleTier::test_a_command_just_under_the_size_ceiling_is_not_pushed_over_itstill checks that a command just under the ceiling is approved and nothing is rejected.Proof: not provable -- test-only change
Determinism proof (the forced condition is a slow runner, modelled as a 5s per-test timeout):
origin/main),--timeout=5: FAILED,Timeout >5.0s.--timeout=5, 20 runs at-n0: 20/20 passed (call 1.07s).-n 4 --dist loadgroup,--timeout=5: 53 passed.--timeout=5: 53 passed each time.test/test_flake_pattern_ratchet.py: 64 passed.Manual verification
N/A — unit coverage sufficient; the change is one test's input.
Related Issues
Closes #23346
Related: #25954 (the two slow deny rules; not changed here)
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)