Repository navigation
fix(security): redact both of two credentials written back to back - #23326
JLargent42 wants to merge 1 commit into
Conversation
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS The leak is real and stays inside pass 1 of the batch redactor. Callers that don't pass Notes
[DESIGN-REVIEWED] 8f989f3 Description read[DESCRIPTION-READ] 3e78da8c570f3c3191add11d110b5e7f1c3aa1c7dfc009ee3aa3f373e3581870 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The glued-pair defect is pinned by a test that fails on base: the legacy oracle keeps Notes
What this change shipsInventory (7 items) — 6 justifiedIntent: when two credentials are written back to back, both are fully redacted in replies, logs and saved chat. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] 8f989f3 Description read[DESCRIPTION-READ] 1db40ec9d7c73fb5beb8ce678a0a6f5279db2f6d484bb6a8ad3dec18829c08cb That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: The stricter piece is the output-redaction scan in pass 1 ( TIGHTENING: Pass-1 credential redaction, ADJUDICATED: 19 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: I wrote posix, windows and any rows. The diff assumes nothing about paths, separators, interpreters or executable names. Its readers work on literal text (
No platform the diff assumes is left without a row. UNADJUDICATED:
[SCOPE-REVIEWED] 8f989f3 |
d17a3c9 to
d148b41
Compare
Sensitive change review
|
|
|
Sensitive change review
|
d148b41 to
6833d9b
Compare
6833d9b to
660c991
Compare
Sensitive change review
|
|
|
Sensitive change review
|
|
Opus 5.5 finding on 50a0b6b (
The change is ready locally and is not pushed, so the head stays 50a0b6b. |
Sensitive change review
|
|
|
HOLD for human security review at This PR changes what the credential redactor removes. Pass 1 now also redacts a credential glued to the one before it, clipping each overlap so spans stay disjoint (redaction.py#L1081-L1099), and the glued scan fails toward redacting the rest of a run once its probe budget is spent (redaction_patterns.py#L639-L762). This head also applies the owner's option (a): the spaced-separator AWS clause is gone, and main's #21573 stream test is relaxed so batch may redact one word the stream keeps. Our review policy reserves changes like this for a maintainer. I found no concrete defect. Callers that do not pass AI findings judged non-blocking:
— crew-pr-reviewer (r146) |
Sensitive change review
|
|
CI note for head 87201f4: |
Sensitive change review
|
|
CI note for head f74ea1d (also applies to 87201f4): |
Sensitive change review
|
|
@iamwhatever the three points in your change request at 13061b1 are addressed on head 7498c33:
Could you re-review? |
Sensitive change review
|
Sensitive change review
|
Sensitive change review
|
Code change approver recommendation: A person should decideAutomated, not a person, and not a review: a maintainer still decides.
Why: the decider approves but is not confident enough to auto-approve (p_defect 0.15 > 0.12). Fixes the batch redactor so that when two credentials are written back to back, both are redacted whole. The glued scan is opt-in (glued=True) and only pass 1 of If this recommendation is wrong, reply here; the replies are how the approver is evaluated. |
Sensitive change review
|
Sensitive change review
|
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 20% chance that this change introduces a defect, above the 12% limit for approving on its own. The PR makes the batch redactor (pass 1) scan with An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
Sensitive change review
|
|
Design Review notes at b63bb3d (both non-blocking; the lane passed):
|
|
@iamwhatever your three points from the r138 change request at 13061b1, checked again on head b63bb3d:
CI is green on this head with |
bolichen97
left a comment
There was a problem hiding this comment.
Request changes at b63bb3d for one point.
The goal is right, and it isn't added strictness. Two credentials written back to back should both be redacted, not just the first. Redacting more of a secret is data safety, not a refusal, and callers that don't pass glued=True keep the old scan.
Please don't solve it with new regex. This repo doesn't take new regex in product code, and security/redaction_patterns.py adds 13 new compiled patterns that main doesn't have:
_CREDENTIAL_PATTERNS_GLUED,_CREDENTIAL_PATTERNS_GLUED_SANS_JWTand_CREDENTIAL_PATTERNS_GLUED_SANS_TELEGRAM_AWS_LABEL_PREFIX_REand_AWS_VALUE_RUN_RE_BEARER_PREFIX_RE,_SPACED_START_RE,_URI_START_REand_AUTHORITY_END_RE_PEM_END_RE,_RUN_REST_RE,_TOKEN_CHAR_REand_GUARD_CHAR_RE
_TELEGRAM_DIGITS_RE already exists on main, so moving it is fine. Most of the new ones are character-class or literal checks that plain code covers:
_TOKEN_CHAR_REand_GUARD_CHAR_REare single-character tests. Use afrozensetof allowed characters andch in ALLOWED._RUN_REST_RE,_AWS_VALUE_RUN_REand_AUTHORITY_END_REfind the end of a run. Use a small loop that scans until a character in a stop set (whitespace,",',,,}or/)._PEM_END_REcompilespem_scan.END, the PEM end marker. Find that marker as a plain string withtext.find(...), and check the label after it by hand._BEARER_PREFIX_RE,_AWS_LABEL_PREFIX_RE,_SPACED_START_REand_URI_START_REmatch known labels and schemes. Lower-case a bounded window and compare it withstartswithagainst a tuple of literals, then skip the optional quote,:or=, and whitespace by hand.- The three
_CREDENTIAL_PATTERNS_GLUED*variants: run the existing credential patterns as they are, and do the glued split in code at each match boundary, rather than compiling new alternations.
Existing patterns the PR doesn't add are out of scope. The tests in test_credential_redaction_glued.py should keep passing as written once the scanning is plain code.
This is a separate point from iamwhatever's open request (the follow-up issue for the frontend mirror). All 109 checks are green.
Sensitive change review
|
Sensitive change review
|
Sensitive change review
|
|
@bolichen97 your review 5883359647 at b63bb3d is addressed on head 164a5d6 (rebased onto main 51349e1).
|
|
Following up on 164a5d6. The regex point from my request-changes is fixed: the 13 compiled patterns are gone, and every pattern still used is one main already has. Two of the lane findings hold up in the code, and one of them comes from the string rewrite I asked for, so here is how I'd fix each without regex. 1. GPT 6.1: quadratic scan after the probe budget runs out. It holds, and the rewrite caused it.
2. Opus 5.5: stream and batch disagree, and the equality test was loosened. It holds, but the rewrite didn't cause it.
3. Security Scope Review: no finding. Its posted verdict is PASS, with 0 candidates newly refused on any platform. It names the redaction surface as unadjudicated, which is expected for an output-redaction change. I see nothing to fix there. No approval from me yet. I'll re-review once 1 and 2 are in and the lanes are green. |
Sensitive change review
|
|
@bolichen97 both points from issuecomment-6850467595 are in head 8f989f3. It's the same base (main 51349e1), with no rebase since the PR isn't DIRTY. 1. Quadratic after the probe budget (GPT 6.1). I took the windowed option. Once the probes are spent, 2. Stream and batch agree again (Opus 5.5). Budget: 36,525 (main 35,901 + 624). SCR for this head: issuecomment-6851012153. |
Problem / Motivation
Goal: When two credentials are written back to back with nothing between them, both are redacted whole.
The credential redactor kept most of the second of two glued credentials. A branch's body class can hold the leading characters of the next credential's prefix: a GitHub token body ends on the next token's
ghp, and ansk-key body ends on the next JWT's header or a PEM block's-----BEGIN. The first match then stops short inside the second credential, and the scan resumed after it, past the second one's start.Why it matters
A reply, tool output or log line that prints two credentials back to back kept one of them nearly whole. Each credential alone, or with a space between, was already fully redacted.
Not a goal
dashboard/chat_runner.py; that waits for Credential redactor keeps a link token that follows a '.' #21550 as well..case (Credential redactor keeps a link token that follows a '.' #21550).website/src/utils/sanitize.ts); this fix covers the backend redactor only. Its glued gap was tracked in Frontend credential sanitizer keeps most of a second token glued to the first #28366 and fixed by fix(security): redact both of two credentials written back to back in the frontend sanitizer #28488.What changed (motivation → approach → change)
Pass 1 of the batch redactor now scans with
_credential_matches(text, glued=True). After each match,_glued_matchesprobes every credential start inside it, nested ones included, and yields any match that runs past its end; the splice in_credential_redaction_planredacts only the part past the previous span, so each credential gets its own tag.@is found as the plain scan finds iteyJstart is a candidate on its own and is read whole, however long its claimsEach match runs at most 8 probes that extend nothing; after that only PEM, Bearer and AWS field starts are probed, still one window at a time, and the rest of a non-space run is redacted whole, so the bound fails closed. Starts are searched one window at a time, so the scan stays linear. Callers that do not pass
glued=True(deploy scan, design-tweak preview, stream cut points) see exactly the matches they saw before.The glued scan adds no regex (review 5883359647).
src/kiro_crew/security/redaction_patterns.pyruns the existing_CREDENTIAL_PATTERNSand reads everything else in plain code: field names, URI schemes and the PEM END marker as string literals withstr.find/startswith, separators and value runs by a small loop, token and guard characters asfrozensets, andAuthorizationthrough the pre-filter's existing case-folding anchors. A Discord or link-token start glued behind a token character is matched on a slice of the text (at most five segment runs) where those branches' start lookbehinds see nothing, in place of the three glued pattern copies. The security spec (docs/system-specs/modules/security.md) describes the glued pass.Backwards compatibility
Compatible: text without two glued credentials redacts as before; glued credentials now redact fully. One case can redact more than main: once a match spends its 8 probes with its run of non-space text still going, the rest of that run is redacted whole. That errs toward redacting too much, never too little. A Discord or link token that starts exactly where a fixed-length key ends is now redacted as well.
One behaviour from #21573 changes, by the owner's decision (largjaco, 2026-11-04): an AWS field name glued onto the end of an AWS value, with its separator after a space (
aws_access_key_id=AKIA...aws_session_token = <value>), is now probed as a field of its own, so<value>is redacted. Main keeps it. Text with the same shape that only looks like such a field (SessionToken=<long value>AccessKeyId : visible text) loses the word after the colon. The streaming redactor reads it the same way (src/kiro_crew/security/text_redaction.py,_glued_key_tail): when an AWS key name ends the batch match it sits in, the stream commits the credential before it, which redacts on its own, and holds the name in the existing AWS discard. The next chunk keeps the name as a label if its separator follows, and otherwise drops it with the value it ended. Streamed and stored output therefore stay equal, astest_stream_redactor_aws_keyed.pyasserts again.Removes nothing: no key, function, record or route is removed.
Tests
test/test_credential_redaction_glued.py(new): same-rule, key-then-JWT and different-rule pairs; three in a row; a credential behind nested matches and past the probe budget; a glued AWS label value (with or without spaces around its separator), Bearer header, JWT with long claims, link password (including one whose@lies past the window) and PEM block (short, long, and after nine nested tokens); a label value the previous match swallowed stays visible; an AWS field inside a Bearer header glued after a credential, with a value starting in%2F, is redacted; and linear-time checks for each adversarial shape a review raised, each under a timeout that the quadratic form exceeds.test/test_credential_redaction_glued.pyalso pins a glued link token followed by.(probed and abutting paths) and a glued AWS field name whose separator follows a space (" = "," ="," : "); each fails on the code before its fix.test/test_stream_redactor_aws_keyed.py: the fix(security): hold a streamed AWS key-value pair past the 512 floor #21573 case keeps main'sout == redact(...)equality, now over five tails (three separators, two without one) and a short and a past-the-ceiling value; the three separator tails fail on the stream without_glued_key_tail.test/test_credential_redaction_glued.pyalso pins a label repeated 64,000 times after the probe budget is spent (aws_secret_access_key,SessionToken, case-foldedAUTHORIZATION); each times out at 60 s when the spaced-start search reads to the span's end.test/test_credential_prefilter.py: the gluedghp_pair moves from the legacy-equivalent corpus toLEGACY_DIVERGENT_SHAPES(legacy keeps the second body, the live pass removes it), and the by-span reference oracle models the glued splice.test/test_chat_runner_coverage.py: the cases that used glued tokens to trigger the save-time withhold now assert that no 12-character window of any token body, the sample key, or a password survives in the saved segment or its link records; the forged-placeholder case keeps its withhold assertion as its own test.test_each_removed_value_is_checked_on_its_own(added on main by fix(security): redact a streamed link cut inside its query as one link #20805) used a gluedghp_pair to reach the withhold; the whole-text pass now redacts that pair as two credentials, so it asserts the saved text carries one tag per token and none of the three tokens.test/test_credential_glued_plain_readers.py(new): holds every plain-code reader to the regex spelling it replaced, kept only in the test as an oracle, over a seeded random corpus; checks_glued_matchagainst the old glued pattern copies (and that the corpus reaches the sliced path and finds Discord starts); and checks over every code point thatstr.lowerandre.IGNORECASEfold the same characters onto the letters ofBearer.test/test_security_regex_linearity.pyand the security package line budget intest/test_security_gate_liveness.pyfollow the new code.Revert proof: restoring only the two production files turns 9 of the new file's tests red, and the no-END-marker linearity test times out without the shared marker lookup. Determinism: the new file passes repeated and shuffled runs.
Manual verification
N/A — unit coverage sufficient: the change is in the pure-text redaction pass, driven directly by the tests above.
Related Issues
Closes #21595
Sibling sweep
Swept: git grep -n "_credential_matches(" -- src/
Sibling: src/kiro_crew/deploy/scan.py -- not affected -- reports each match it finds, a glued second credential's prefix is already flagged by the first
Sibling: src/kiro_crew/apps/builtins/design_tweak/backend/preview_files.py -- not affected -- answers only whether any credential is present
Sibling: src/kiro_crew/security/text_redaction.py -- not affected -- uses match spans as cut points; the batch pass it hands text to is the one fixed here
Pattern harvest
Not generalizable: the defect is specific to resuming a leftmost-longest scan after a match whose body class overlaps another branch's prefix; the fix sits in the one shared scanner.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)