Skip to content

fix(security): redact credentials by span in every pass - #11252

Merged
iamwhatever merged 2 commits into
mainfrom
fix/redact-by-span-8049
Sep 16, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
fix/redact-by-span-8049

Conversation

@CrysisDeu

@CrysisDeu CrysisDeu commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

What

redact_credentials (src/kiro_crew/security/redaction.py) now positions every redaction by span in all three passes. Each pass records (start, end, tag) against the immutable input; the string is spliced once, left to right, at the end. No str.replace on a matched value remains in the function.

Why

Passes 2 (base64-encoded credential) and 3 (bare 40-char key) still substituted by matched value: result.replace(chunk, tag, 1) and result.replace(run, tag, 1). str.replace(x, tag, 1) rewrites the first textual occurrence of x, not the span the scan selected. When a redactable value also occurs as a substring of an earlier, longer run that is itself not redactable, the tag lands inside that host and the real standalone credential survives in plaintext:

  • pass 2: chunk C decodes to a credential; Q + C is a 57-char run that cannot decode (length % 4 == 1) and is skipped, so replace(C, …) hits the copy inside Q…C and emits standalone C verbatim;
  • pass 3: a 40-char four-slash key is a bare secret standing alone, but / + key is a 41-char run every window of which exceeds the fragment slash ceiling, so the host is declined and replace(run, …) hits it instead of the standalone key. The run not in result guard checks presence, not position, so it does not close this.

Pass 1 was fixed the same way in #7941; this closes the class.

How

  • _uncovered(start, end, taken) subtracts the already-claimed spans (sorted, disjoint) from a candidate span with one bisect and a forward walk, returning the gaps still in plaintext. _splice applies the final sorted span list in one pass.
  • Precedence is positional and unchanged: pass 1 > pass 2 > pass 3. A later pass only redacts the parts of its span an earlier pass has not claimed, so no character is redacted twice and no character a pass flagged is left standing.
  • Warning contract preserved: pass 1 warns per match; pass 2 warns for every chunk that decodes to a credential (as before, including one pass 1 already covered in full); pass 3 warns only when it splices something (the positional equivalent of the old run not in result skip). All pass-2 warnings still precede all pass-3 warnings.
  • Behaviour change beyond the leak fix, deliberate: a run partly claimed by pass 1 (e.g. a glued key whose tail is the aws of a following aws_secret_access_key= label, or sk-proj-…Z<key> where pass 1 consumed the head) previously fell through run not in result and left the unclaimed portion of a key-shaped run in plaintext. The remainder is now redacted. The fuzz differential surfaced both shapes on the unmodified base.

Tests

test/test_credential_prefilter.py, two commits:

  1. The reference oracle _reference_redact_credentials is rewritten in the same by-span form with none of the live shortcuts (no pre-filter gate, separate _BARE_SECRET_RUN_RE scan, generic _decode_b64_safe, linear bisect-free _reference_uncovered). New direct pins: test_hosted_encoded_credential_is_redacted_at_its_own_span (pass 2), test_hosted_bare_secret_is_redacted_at_its_own_span (pass 3), test_partly_claimed_run_keeps_no_plaintext, test_redact_credentials_never_substitutes_by_value (AST pin: no .replace( call in redact_credentials), plus corpus entries for the hosting, partly-claimed and same-value-twice shapes.
  2. Design Review and First Principles both asked that the "byte-identical to the old code on every pre-existing shape" and "red on base" claims be checked rather than reported. The shipped by-value passes 2 and 3 are now kept verbatim in the test module as _legacy_redact_credentials, and two tests carry those claims in CI: test_by_span_agrees_with_legacy_wherever_legacy_landed_right (110 shapes, text and warning order identical) and test_legacy_diverges_on_every_by_span_shape (the 4 divergent shapes, each paired with the plaintext the legacy output keeps and the by-span output removes). The by-span oracle shares its algorithm with the live function, so it could not carry either claim alone.
  • test/test_security_regex_linearity.py::test_credential_pattern_module_still_compiles_one_alternation pinned pass 1's spelling (_CREDENTIAL_PATTERNS.sub(, _might_contain_credential(result)); it now pins the invariant it documents -- the shared compiled pattern scans the same string the pre-filter gated, and no local re.compile( -- in the span-recording spelling.
  • Red on base: 13 failed / 361 passed with the final test file against main (git archive main into a scratch dir). Green on this branch: 374 passed. Also green: test/test_credential_patterns.py, test/test_security.py, test/test_log_redaction.py, test/test_credential_redaction_notice.py.
  • Local gates: isort, flake8, mypy, scripts/check_black_formatting.py, scripts/check_comment_history.py, scripts/check_subprocess_encoding.py all pass.

Other

Backend-only change to a pure string function; no UI surface is touched, so the no-screenshots label is applied. bisect(..., key=) requires Python 3.10+; the package requires 3.12.

Pattern harvest

Rule candidate: semgrep
Pattern: a str.replace(<value>, <tag>, 1) whose first argument is the text of a regex match (m.group(), or a variable bound from it) inside a redaction/scrub function — flag as redact-by-value; the redaction must splice m.start()/m.end() instead.

Closes #8049

@CrysisDeu
CrysisDeu requested review from a team and dwu96 September 16, 2026 08:03
@CrysisDeu CrysisDeu added the no-screenshots PR has no visual delta; screenshot gate exempt label Sep 16, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 09d6df5de6f663807b7df47000a4fc696875c8ae — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the deliberate extra redaction of partly-claimed runs (documented "Behaviour change beyond the leak fix") is an acceptable false-positive trade — it errs toward redaction and is pinned by test_partly_claimed_run_keeps_no_plaintext.

[DESIGN-REVIEWED] 09d6df5

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 09d6df5de6f663807b7df47000a4fc696875c8ae and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 09d6df5

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 09d6df5de6f663807b7df47000a4fc696875c8ae: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 09d6df5de6f663807b7df47000a4fc696875c8ae — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Confirm linked issue #8049 describes this leak; every other claim in the description is carried in CI by the verbatim legacy oracle.

What this change ships

Inventory (9 items) — 9 justified

Intent: stop a real credential surviving redaction when its text also occurs inside an earlier, non-redactable run — a FIX (linked issue #8049; the leak is reproduced in CI against the retired code kept verbatim).

  1. Encoded credential hosted in a longer non-decodable run no longer survives in plaintext — justified
  2. Bare 40-char key hosted in a longer declined run no longer survives — justified
  3. Unclaimed remainder of a partly-claimed key-shaped run is now redacted, not left standing — justified
  4. Output spliced once from spans; private _uncovered/_splice helpers (3 call sites, no existing span mechanism — grepped _splice|_uncovered|bisect across src/) — justified
  5. Pass-3 skip is positional: a value's second occurrence judged on its own span — justified
  6. New AST pin: no .replace( call may return to redact_credentials — justified
  7. Legacy by-value passes kept verbatim in tests; 110-shape equivalence + 4-shape divergence pins — justified
  8. Reference oracle rewritten by-span, retiring its own declared "out of scope, NOT fixed" deferral — justified
  9. Linearity pin re-spelled to finditer(text), adding a no-local-re.compile clause — justified

Root-cause check: the class (replace(match_text, tag, 1) in a redaction path) has zero remaining siblings — grepped \.replace\([^,]+,[^,]+, ?1\) and replace(.*redact/TAG) across src/; exfil.py:1092 replaces all occurrences (matched span always rewritten, no survival), and cron_script.py:659 / policy_distribution.py:2007 replace known secret values everywhere. Cause-level, class closed and pinned.

[FIRST-PRINCIPLES-REVIEWED] 09d6df5

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 09d6df5de6f663807b7df47000a4fc696875c8ae — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 09d6df5

Verdict parsed from the review's SHA-scoped output markers for commit 09d6df5de6f663807b7df47000a4fc696875c8ae.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 09d6df5de6f663807b7df47000a4fc696875c8ae: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- 🟡 CONCERNS

Which legitimate operations does 09d6df5de6f663807b7df47000a4fc696875c8ae newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: the reviewer's own judgement; no confirmed regression.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 6 classified, 0 newly refused, 5 still allowed.
  • ubuntu-latest (platform posix): 6 classified, 0 newly refused, 5 still allowed.
  • windows-latest (platform windows): 5 classified, 0 newly refused, 5 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: CONCERNS

TIGHTENING: redact_credentials in src/kiro_crew/security/redaction.py rewritten from by-value substitution to by-span splicing — every pass now records (start, end, tag) against the immutable input and the string is spliced once, with pass 3 additionally redacting the still-plaintext remainder of a partly-claimed run. The ONE threat it refuses: a credential surviving redaction in plaintext because result.replace(matched, tag, 1) landed the tag on an earlier innocent lookalike instead of the span that matched — "when a later match also occurs as a substring of an earlier, longer run that is NOT itself redacted, the tag lands inside the innocent host and the real standalone credential survives in plaintext" (the new docstring hunk), pinned by LEGACY_DIVERGENT_SHAPES in test/test_credential_prefilter.py.

ADJUDICATED: 10 candidates, 0 confirmed newly-refused

The candidates cannot probe the tightening itself, only its one coupling to the adjudicated surface: the diff touches none of the four tiers the classifier runs (is_sensitive_path, is_sensitive_bash_command, audit_bash_exfiltration, is_denied — verify_fix.py TIERS), and none of the predicates the exfiltration tier shares with this module (_contains_bare_secret, _text_contains_bare_secret, _contains_fixed_credential at src/kiro_crew/security/exfil.py:37 are all unchanged). The remaining coupling is import health — the tiers and the rewritten function live in the same kiro_crew.security package, and the diff adds import bisect plus a bisect_right(..., key=...) call (Python ≥3.10; the repo floor is 3.12 per pyproject.toml, so sound) — which the import-smoke and daily-operation rows exercise, alongside sensitive-path-adjacent read-only operations and a 40-char-hex-SHA row one token off the bare-secret shape pass 3 catches.

PLATFORM COVERAGE: posix, windows, and any. The diff itself is a pure-Python string algorithm — it adds no path, separator, interpreter, executable-name, or environment-variable assumption, so no platform spelling is assumed by the change. Candidates still carry both spellings wherever the row itself carries one: the test invocation (.venv/bin/python / .venv\Scripts\python.exe), the security-tree content search (grep -rn / Select-String), the sensitive pathspec in read-only git argv (/ and \ separators), and the import smoke check; the SHA-range row is platform-neutral and tagged any. No assumed platform was left without a candidate. Per-leg verdicts are the deterministic fold's to report, not this section's.

UNADJUDICATED: The tightening lives in the output redactor, not in the tool-call deny composite, so no candidate row can obtain a verdict on it. The guard: redact_credentials (src/kiro_crew/security/redaction.py:940, plus the new _uncovered/_splice helpers). The call path an input takes: output text on every persist/display/forward path — slack/handler.py:4330 (final reply text), slack/renderer.py:125, history_consolidation.py:1881/2033, workflows/runner.py:162, workflows/store.py:88, mcp_core.py:1581, session_summary.py:392, among ~80 call sites — is passed through the function before it is persisted or shown; nothing on the shell-command classification path (hooks.py tool gate, verify_fix.py tiers) calls it. The candidate legitimate inputs I would have submitted, in prose:

  • A 40-char hex commit SHA or hex digest inside ordinary output text — the exact length pass 3 windows over; it must continue to pass through unredacted (the deciding predicate _contains_bare_secret is unchanged, but the splice path around it is new).
  • An innocent long base64 run that does not decode, adjacent to a decodable chunk of the same text (the hosted shape): the rewrite now leaves the innocent host fully intact where the legacy code destroyed its middle — the delta here is less over-redaction, not more, pinned by test_hosted_encoded_credential_is_redacted_at_its_own_span.
  • The one direction that redacts more: a partly-claimed run's non-secret glue characters (e.g. the ZGHUT8 glue in test_partly_claimed_run_keeps_no_plaintext) are now removed along with the key, and a second bare occurrence of an already-labelled value is now judged on its own span. Both deltas only fire adjacent to genuine key material; I can argue no legitimate-text loss but cannot confirm it through any classifier, which is why the verdict is CONCERNS rather than PASS.
  • A consumer-side input: slack/handler.py:4392 counts CREDENTIAL_REDACTION_TAGS occurrences in redacted text to classify a reply as credential-only; on multi-occurrence texts the tag count and pass-3 warning emission differ from legacy, so that classification is an input the change can shift and nothing here could adjudicate.

[SCOPE-REVIEWED] 09d6df5

`redact_credentials` passes 2 and 3 substituted by matched VALUE
(`result.replace(matched, tag, 1)`), which rewrites the first textual
occurrence of the value rather than the span the scan selected. A decodable
base64 chunk, or a bare key, that also occurs inside an earlier, longer run
which is itself not redactable had its tag land inside that host while the
real standalone credential stayed in plaintext.

All three passes now record `(start, end, tag)` spans against the immutable
input and the string is spliced once, left to right. Precedence is
positional (pass 1 > pass 2 > pass 3): a later pass redacts only the part of
its span an earlier pass has not claimed, so nothing is redacted twice and a
run partly consumed by a labelled match keeps no plaintext.

The differential oracle in test_credential_prefilter.py is rewritten in the
same by-span form without any of the live shortcuts, and the corpus gains
the hosting shapes for both passes, the partly claimed runs, and an AST pin
that no `.replace(` call remains in `redact_credentials`.

Closes #8049
@CrysisDeu
CrysisDeu force-pushed the fix/redact-by-span-8049 branch from 16213dd to 2801b3f Compare September 16, 2026 08:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 16, 2026
… one

Keep the shipped by-value passes 2 and 3 verbatim in the test module as a
second oracle, so both claims the rewrite makes are checked in CI instead
of asserted off-CI: the by-span redactor is byte-identical to it (text and
warning order) on every corpus shape where redacting by value landed on the
right span, and on each of the four divergent shapes the legacy output keeps
plaintext the by-span output removes. The by-span oracle shares its
algorithm with the live function, so it cannot carry either claim alone.

The linearity guard in test_security_regex_linearity.py pins its invariant
(pass 1 scans with the shared compiled pattern, gated by the pre-filter on
the same string) in the span-recording spelling instead of the retired
`.sub(` one.
@CrysisDeu
CrysisDeu force-pushed the fix/redact-by-span-8049 branch from 2801b3f to 09d6df5 Compare September 16, 2026 08:57
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 16, 2026
@CrysisDeu

Copy link
Copy Markdown
Contributor Author

Security Scope Review (advisory CONCERNS, 0 newly refused) — mapping each candidate input it could not adjudicate to the test that pins it on 09d6df5de6f663807b7df47000a4fc696875c8ae:

  • 40-char hex SHA / hex digest in ordinary output. Corpus entries 3f786850… (64-hex), the 40 lowercase and 40 uppercase {i:064x} digests, and the _B64_RUN_40 residue family are all in LEGACY_EQUIVALENT_CORPUS; test_by_span_agrees_with_legacy_wherever_legacy_landed_right asserts the by-span output is byte-identical to the retired implementation on every one of them (text and warnings), and test_output_is_byte_identical_to_reference pins the same against the unoptimised oracle. The deciding predicate _contains_bare_secret is untouched; the splice path adds no redaction where that predicate says no.
  • Innocent non-decodable host beside a decodable chunk. test_hosted_encoded_credential_is_redacted_at_its_own_span asserts the exact output f"{host} {tag}": the host is left intact and only the standalone chunk is replaced. Pass-3 mirror: test_hosted_bare_secret_is_redacted_at_its_own_span.
  • The one direction that redacts more. test_legacy_diverges_on_every_by_span_shape enumerates every shape on which the by-span output differs from the retired one — four, including the two partly-claimed runs — and names the plaintext each removes (ZGHUT8 glue; the 26-char key tail after sk-proj-…Z). Both deltas sit inside a run _contains_bare_secret already judged a key. A second bare occurrence of an already-labelled value is in the corpus in both orders (aws_secret_access_key=K K and K aws_secret_access_key=K) and is in the legacy-equivalent set: the by-span redactor agrees with the retired one there. The 4000-string fuzz in test_prefilter_soundness_under_fuzz compares live to the by-span oracle on random text with real and near-miss credentials inserted.
  • Consumer-side tag count (slack/handler.py credential-only classification). Measured on this head: on all 110 legacy-equivalent corpus shapes the CREDENTIAL_REDACTION_TAGS count is identical to the retired implementation. On the four divergent shapes the counts are 1→1, 1→1, 2→2 and 1→2; the only shift is the sk-proj-…Z<key> shape, where the retired code emitted one tag and left the key tail in plaintext and the by-span code emits two tags — more credential material classified as credential, which is the safe direction for that consumer.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after a full read of the diff on the current head: the change matches the linked issue and body, every load-bearing claim was spot-checked against main, tests pin the behaviour, all review lanes are clean and nothing is left undisposed on this head.

@iamwhatever
iamwhatever merged commit 834aae9 into main Sep 16, 2026
87 checks passed
@iamwhatever
iamwhatever deleted the fix/redact-by-span-8049 branch September 16, 2026 19:38
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 16, 2026
amadsalmon pushed a commit to amadsalmon/KiroCrew that referenced this pull request Sep 22, 2026
…1252)

* fix(security): redact credentials by span in every pass

`redact_credentials` passes 2 and 3 substituted by matched VALUE
(`result.replace(matched, tag, 1)`), which rewrites the first textual
occurrence of the value rather than the span the scan selected. A decodable
base64 chunk, or a bare key, that also occurs inside an earlier, longer run
which is itself not redactable had its tag land inside that host while the
real standalone credential stayed in plaintext.

All three passes now record `(start, end, tag)` spans against the immutable
input and the string is spliced once, left to right. Precedence is
positional (pass 1 > pass 2 > pass 3): a later pass redacts only the part of
its span an earlier pass has not claimed, so nothing is redacted twice and a
run partly consumed by a labelled match keeps no plaintext.

The differential oracle in test_credential_prefilter.py is rewritten in the
same by-span form without any of the live shortcuts, and the corpus gains
the hosting shapes for both passes, the partly claimed runs, and an AST pin
that no `.replace(` call remains in `redact_credentials`.

Closes kirodotdev#8049

* test(security): pin the by-span redactor against the retired by-value one

Keep the shipped by-value passes 2 and 3 verbatim in the test module as a
second oracle, so both claims the rewrite makes are checked in CI instead
of asserted off-CI: the by-span redactor is byte-identical to it (text and
warning order) on every corpus shape where redacting by value landed on the
right span, and on each of the four divergent shapes the legacy output keeps
plaintext the by-span output removes. The by-span oracle shares its
algorithm with the live function, so it cannot carry either claim alone.

The linearity guard in test_security_regex_linearity.py pins its invariant
(pass 1 scans with the shared compiled pattern, gated by the pre-filter on
the same string) in the span-recording spelling instead of the retired
`.sub(` one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-screenshots PR has no visual delta; screenshot gate exempt

Projects

None yet

Development

Successfully merging this pull request may close these issues.

redact_credentials passes 2 and 3 still redact by value, not by span

2 participants