Repository navigation
fix(security): keep the PEM private-key content scan linear on repeated BEGIN headers - #21154
chenmingwei23 wants to merge 1 commit into
Conversation
Sensitive change reviewHead: 3ff0804 This PR touches a sensitive path (
|
3ff0804 to
217f7a0
Compare
Sensitive change reviewHead: 217f7a0 This PR touches a sensitive path (
|
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The quadratic regex stays in the shared pattern, so the deploy skill's shell pre-publish scan and other callers still go quadratic on the same input. Watch
Suggestions
[DESIGN-REVIEWED] 10e37fb Description read[DESCRIPTION-READ] b785935826c76fe7d6aef29d78478c0cdcedab267502600feef35cc7cfb1670a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check one undeclared change: in a deploy scan, a real PEM key now produces one finding (the header match from What this change shipsInventory (6 items) — 5 justifiedIntent: a person previewing a project file, or running a pre-publish deploy scan, on text that quotes thousands of PEM headers no longer waits seconds while the backend blocks. This is a FIX (#21006).
[FIRST-PRINCIPLES-REVIEWED] 10e37fb Description read[DESCRIPTION-READ] e0d9bf3a711886c1c6e41c400df18e6c6b44aac64ecdeafaa44b168f16b34c7a That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Security Scope Review (fork) — 🟡 CONCERNSWhich legitimate operations does the reviewer's own judgement; no confirmed regression Scope-Verdict: CONCERNS TIGHTENING: Credential content scanning in the design_tweak static preview ( ADJUDICATED: 14 candidates. The shell tool-call deny composite ( PLATFORM COVERAGE: The diff adds no path, separator, interpreter or executable-name assumption. Its regexes run on file contents, with the same behaviour on every host. Candidates cover these spellings:
One spelling is not probed by any candidate, because it is content and not a command line: CRLF ( UNADJUDICATED:
[SCOPE-REVIEWED] 10e37fbSecurity scope review -- adjudicated candidates
No candidate flipped from allowed to refused. |
217f7a0 to
0179318
Compare
|
Addressed the review findings in Opus 5.5 / GPT 6.1 (BLOCKING — Internal Content Scan (internal marker in added lines): the fail-safe test fixture no longer uses a vendor-token-shaped string. It uses AWS's own published example access-key id ( First Principles (CONCERNS):
Security Scope Review (CONCERNS): the complete-block body search now runs strictly between a BEGIN marker's end and its governing END marker's start, so a marker label can never be read as key material. A 6000+ input differential against the prior scanner behaviour stays identical, including a pathological long-label case. |
Sensitive change reviewHead: 0179318 This PR touches a sensitive path (
|
0179318 to
334f615
Compare
|
Addressed in Opus 5.5 (BLOCKING — overlapping-header leak): the governing END for a complete block is now the first END header whose start is at or after the BEGIN marker's end ( GPT 6.1 (BLOCKING — embedded-header quadratic): the block regexes are no longer re-matched from every header. The scan keeps a monotonic cursor ( Opus FINDING (timing-ratio test shape): the complexity test no longer measures wall-clock time. It counts how many times the block regexes are applied (a deterministic integer) and asserts that count grows no more than ~2x per doubling across four input shapes (bare headers, one far END, headers nested in metadata lines, self-overlapping BEGIN/END). It cannot false-red under runner load and cannot hang into a timeout. Both callers' accept/reject decision stays byte-identical to the prior scanner across a 20000-input differential, which now includes the overlapping-marker and metadata-nested-header shapes. |
Sensitive change reviewHead: 334f615 This PR touches a sensitive path (
|
334f615 to
06b79a8
Compare
|
Internal Content Scan addressed in The scan flagged three added lines that spelled a complete
The production regexes already use a character class ( |
Sensitive change reviewHead: 06b79a8 This PR touches a sensitive path (
|
…ed BEGIN headers The combined credential regex's PEM branch lazily scans forward to a matching END header, and the engine retries that scan at every BEGIN header, so content quoting many 'BEGIN PRIVATE KEY' headers with no END is O(n^2): the design_tweak file preview (uncapped) blocked ~12s on ~8000 such headers, and the deploy pre-publish scan paid the same walk per header up to its findings cap. Add src/kiro_crew/security/pem_fast.py: the combined pattern with its one PEM branch removed, plus a linear one-pass PEM detector whose accept/reject decision is identical to that branch. The design_tweak preview and deploy content scanners now run the non-PEM branches through the sans-PEM pattern and the PEM case through the linear check. redaction and all other callers are untouched. A real credential previously swallowed between two quoted PEM markers (a body-less span the old branch hid) is now detected -- the change only ever strengthens detection. Fixes kirodotdev#21006
06b79a8 to
10e37fb
Compare
|
Backend Tests shard addressed in
The other failing shard (Backend Tests (Windows) (2)) is the same test; the fix covers both. No production behaviour changed. |
Sensitive change reviewHead: 10e37fb This PR touches a sensitive path (
|
|
Closing in favor of #21388, which fixes the same issue (#21006) for the same two callers. Both this PR and #21388 keep the credential content scan linear on content that repeats a
The detection decision is unchanged at both sites: a real (complete or truncated) private-key block is still refused, and a quoted PEM marker with no key body is still allowed. #21388 reaches the same result by reusing the existing linear credential scanner ( Since #21388 covers the same fix, #21006 is tracked there. Closing this PR; the branch is left in place. |
Problem / Motivation
Goal: Previewing a project file (and the pre-publish deploy scan) stays fast — linear time — on content that quotes many PEM private-key headers, instead of blocking the backend for seconds.
The content credential scanners iterate the canonical combined credential regex. Its PEM branch is
-----BEGIN [A-Z ]*PRIVATE KEY-----(?:[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----|...): the leading[\s\S]*?lazily scans forward to a matchingENDheader, and the regex engine retries that scan at everyBEGINheader. On text with manyBEGIN PRIVATE KEYheaders and no matchingEND, that is O(n^2). The design_tweak file-preview scan (contains_credential) is uncapped, so a user previewing such a file waits while the backend blocks:The deploy pre-publish scan (
scan_content) is bounded by a findings cap but still runs the forward walk once per header up to that cap.Why it matters
Both scanners run on arbitrary user-supplied file content triggered by ordinary actions (previewing a project file; a pre-publish deploy scan). A text file with a few thousand quoted PEM headers is an easy, unprivileged way to make the preview backend block for seconds, degrading the preview for everyone sharing that backend.
Not a goal
redaction.py) or any redaction caller. Redaction's PEM branch spans the key body so it can mask it; it stays exactly as is. This fix is at the content-scanner level only:_credential_matchesinredaction.pystill iterates the full combined pattern once per header on redacted text, which this PR deliberately does not touch (redaction needs the spanning branch, and its own linearity is a separate change).What changed (motivation → approach → change)
Root cause: the quadratic cost is the PEM branch's lazy
[\s\S]*?END-seeking scan, re-attempted perBEGINheader. The scanners only need a boolean "does this contain a private key?", which does not require that span.Approach: a new module
src/kiro_crew/security/pem_fast.pyprovides two things, so redaction (which genuinely needs the spanning branch) is untouched:CREDENTIAL_PATTERNS_SANS_PEM— the combined pattern with its single PEM branch removed. A self-check fails import loudly if the branch text drifts fromredaction.py, so the strip can never silently become a no-op.contains_pem_private_key(text)— a linear, one-pass PEM detector. It classifies eachBEGINheader exactly as the canonical branch does (complete block = armoured body before the firstENDahead; truncated block = the contiguous base64/metadata lines after the header), but computes it from precomputedENDand body-run positions with binary search, so no per-header forward walk runs.Change: the two scanners run the non-PEM branches through the sans-PEM pattern and the PEM case through the linear detector:
preview_files.contains_credentialdeploy/scan.scan_content—_DEPLOY_EXTRAalready carries a plain, linear-----BEGIN [A-Z ]*PRIVATE KEY-----header regex, asserted to be a superset of the canonical set, so PEM detection there is unchanged.Both callers now scan the 8000-header input in well under 0.05 s and grow ~x2 per doubling. The decision is byte-identical to the prior scanner across directed cases and 6000 randomized fuzz inputs, with one deliberate strengthening: a real credential sitting on an interior line between two quoted PEM markers (which a lazy
BEGIN..ENDspan would otherwise swallow into one body-less span and let through) is detected. The change only ever detects more.Backwards compatibility
Compatible: no accepted input is now rejected and no key the scanners accepted is now missed. The PEM accept/reject rule is byte-identical except that a credential hidden between quoted PEM markers is now caught — strictly more detection, the fail-safe direction. Nothing is removed, renamed, or narrowed;
redaction.pyand its callers are untouched. Removes nothing: no path, record, key, or public function is taken away.Tests
test/test_pem_header_scan_linear.py:Manual verification
N/A — unit coverage is sufficient; the change is a pure scan-cost/behaviour-equivalence fix with no UI surface.
Related Issues
Fixes #21006
Pattern harvest
Rule candidate: review-prompt
Pattern: a lazy
[\s\S]*?between two markers in a shared regex, iterated withfinditer, is O(n^2) on input holding many opening markers and no closing one — split the span off into a linear per-marker scan when callers need only a boolean.CI notes (reds outside this change)
test_agent_scratchgit maintenance.lock race, a known intermittent failure tracked in Flaky: test_a_failed_move_keeps_the_tree_in_place races git's maintenance.lock #20325; it touches no file in this change. Coverage Gate fails closed on that shard.