Skip to content

perf(security): cut the per-run decode and scan work in redaction passes 2 and 3 - #7067

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
rnoack1:perf/redact-b64-loop-bodies
Sep 1, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
rnoack1:perf/redact-b64-loop-bodies

Conversation

@rnoack1

@rnoack1 rnoack1 commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

redact_credentials runs three passes. #7056 cut most of the scanning cost by
gating pass 1 behind a cheap superset pre-filter and feeding passes 2 and 3 from
one shared base64 scan. What it did not touch is the per-run work inside the
pass 2 and pass 3 loop bodies
, which is now the largest remaining term on
content carrying many base64-alphabet runs.

Measured on this branch's base (#7056's head, 0235df78), pass 2 + pass 3 loop
bodies with the shared scan excluded:

corpus size loop bodies share of redact_credentials
real session text 981.5 KB 1.01 ms 1.9% of 53.4 ms
hash-heavy (commit ids, digests) 98.3 KB 11.0 ms 68% of 16.1 ms
blob-heavy (encoded attachments) 313.8 KB 35.8 ms 78% of 47.1 ms

Two costs drive that:

  1. _decodes_to_printable_text counted printable bytes in a Python loop. It
    runs once per base64 run and again per 40-char window as gate 7 of
    _looks_like_secret_key, and its cost scaled with the decoded byte count
    rather than with the 40-char window. On a 2 KB encoded blob it was 86.3 us of
    the 88.2 us _contains_bare_secret spent on that run — 98%.
  2. Pass 2 ran the full 23-branch alternation over every decoded blob. Real
    decoded blobs are almost never credentials: 0 of 18 in the session corpus and
    1 of 849 in the hash-heavy corpus.

Why it matters

This is the redaction boundary; a regression writes live credentials into
persisted chat history. So the constraint is byte-identical (text, warnings)
output, and the value has to be stated honestly:

  • On ordinary session text this is a no-op (53.4 ms -> 53.8 ms, inside noise),
    because 981 KB of real text contains only 116 qualifying runs. The remaining
    cost there is pass 1 and the shared scan, which perf(security): gate the credential scan behind a cheap superset pre-filter #7056 owns.
  • The win is on hash-heavy and blob-heavy content — build logs, git output,
    commit lists, encoded attachments — where the loop bodies were 68–78% of the
    function.

A digest-keyed cache for redact_credentials was in flight concurrently
(branch perf/redact-credentials-cache), so these figures are miss-path cost.
That is deliberate: a cache cannot help a miss, and this change targets exactly
the term a cache leaves behind. I checked for a cache at the function entry first
and found none — redact_credentials carries no memo, and the only sha256 cache
nearby is _entry_cache in chat_persistence.py, keyed on the whole message
dict
in front of the entry builder, which misses on every new message. Do not
read the numbers above as a whole-workload improvement a landed cache would absorb.

Stacking: #7056 is still open, so this branches off its head rather than
main, and the diff against main includes its commit. I expect #7056 to merge
first
, then the cache PR and this one in either order — the cache lands at the
function entry and this lands in the pass 2/3 loop bodies.

What changed in code

Two changes in src/kiro_crew/security.py, three executable lines.

1. _decodes_to_printable_text counts printable bytes with bytes.translate.
translate(None, _PRINTABLE_BYTES) returns exactly the bytes not in the set, so
len(raw) - len(...) is the member count. An integer identity, so the ratio and
the comparison are bit-identical to the previous per-byte sum. Measured 14.6x at
48 bytes rising to 69x at 1500.

2. Pass 2's decoded-blob scan is gated behind the pre-filter, length-gated.
_might_contain_credential is a proven superset of _CREDENTIAL_PATTERNS, so the
verdict cannot move. It is not unconditionally cheaper, though: its ~590 ns
floor is fixed while the alternation scales with length, so the crossover was
measured rather than assumed — 8 chars 684 vs 494 ns (alternation wins), 12 chars
746 vs 898 (pre-filter wins), 3.4x by 256. Decoded blobs (measured 12–31 chars)
genuinely straddle that line.

_PREFILTER_MIN_LEN is held at 16, above the measured crossover of 12, on
purpose. The gate is verdict-neutral — the pre-filter is a proven superset, so
either route reaches the same answer — which makes a conservative threshold cost at
most one alternation scan on a 12–15 character blob, and makes it robust to the
crossover drifting as the pre-filter's own cost changes. It has already drifted
once: this PR was rebased onto a revision of #7056 that replaced the .lower()
substring check with a case-insensitive Authorization anchor, moving the crossover
from 16 to 12. The figures above are re-measured against that implementation.

A third optimisation was written, then removed as unsound — read this before re-adding it

An earlier revision short-circuited pass 2's decode when len(chunk) % 4 was
non-zero, reasoning that validate=True rejects such a length. That is
interpreter dependent and is a redaction bypass on the versions CI builds.

binascii.a2b_base64's padding leniency changed with strict_mode:

chunk shape 3.10 3.11 3.12
40 data chars + = (len 41, %4 = 1) decodes decodes raises
40 data chars + == (len 42, %4 = 2) decodes decodes raises
43 data chars + == (len 45, %4 = 1) decodes raises raises

So on 3.10 the guard skipped a chunk that does decode, leaving a base64-encoded
credential unredacted. No version-invariant form of the predicate exists either:
43 data chars + "==" decodes on 3.10 while failing both a total-length and a
stripped-length test. The guard is gone, the reasoning is recorded at the call
site, and test_a_decode_length_precondition_would_be_version_dependent plus
test_a_length_short_circuit_would_leak_an_encoded_credential pin it.

Removing it cost essentially nothing measurable — it was the least valuable of the
three changes, and the benchmark below is after its removal.

Also deliberately not done

  • Passes 2 and 3 remain separate sequential loops. Fusing them would change the
    order of warnings, which occurrence each str.replace(..., 1) lands on, and
    whether pass 3's run not in result guard sees pass 2's edits. Computation is
    shared; control flow is not.
  • The decode is not shared between the passes. Pass 2 decodes the chunk as
    matched with validate=True; pass 3 reaches _decodes_to_printable_text, which
    strips =, re-pads and decodes with validate=False. Those inputs coincide only
    when the chunk already carries minimal padding — measured 867 same, 249
    differing
    — so they are not one value. Pinned by
    test_chunk_and_repadded_run_are_not_interchangeable_decode_inputs.
  • _decode_b64_safe is left unoptimised on purpose. It is the independent
    oracle for _decode_b64_chunk; applying the same gates would make both sides of
    that comparison share the change and it would stop detecting anything.
  • _shannon_entropy (4.8 us, the costliest single window gate) and
    _vowel_ratio are untouched.
    Entropy is worth revisiting, but a
    precomputed-term rewrite must be proven bit-identical or it can flip a verdict at
    the >= 4.3 boundary, and that belongs in its own change.

Tests

test/test_credential_prefilter.py, the differential test #7056 added, extended
from 157 to 213 passing. It pins the pre-optimisation implementation as a
reference oracle and asserts byte-identical (text, warnings).

Verified on Python 3.10 AND 3.12 — 213 passed on each. The removed guard is
exactly why: it passes on 3.12 and fails on 3.10, so a single-version run cannot
see it.

Corpus extended toward this change's blast radius: 40 hex digests in one string
(both cases), runs at every length residue 40–44, a long non-decodable run beside a
decodable one, padding variants per residue, blobs decoding to printable text / to
high-entropy garbage / to the empty string once invalid UTF-8 is dropped, encoded
credentials on both sides of the length gate, and an encoded credential in the
misaligned padding shapes whose decodability differs by interpreter.

New independent oracles. _decodes_to_printable_text is reached through
_contains_bare_secret, which the reference oracle also calls — so the
byte-identity assertion shares that helper and structurally cannot detect a change
inside it. Two tests close that gap against a verbatim copy of the pre-rewrite
body: all 256 single-byte values exhaustively (a complete proof for the membership
question, since translate decides each byte independently) plus 4000 fuzzed
multi-byte inputs, and 3000 fuzzed tokens across every residue and padding form.

Negative controls. Each sabotage was applied to the real source, the guarding
test observed, and the file restored and verified by sha256:

sabotage on 3.10 on 3.12
printable byte set narrowed (drops tab/LF/CR) CAUGHT CAUGHT
decoded-blob gate too narrow (anchors only on ghp_) CAUGHT CAUGHT
the removed len(chunk) % 4 guard put back CAUGHT not caught

The third row is the point: the same defect is caught on 3.10 and invisible on
3.12. The blob-gate sabotage fails test_output_is_byte_identical_to_reference,
i.e. an encoded credential slips through and the differential test notices.

Suite results. test_credential_prefilter.py 213 passed on 3.10 and on 3.12;
with test_security.py, 1208 passed / 1 skipped / 0 failed on 3.12. All 21
redaction-related test files ran clean against a baseline worktree at the same
commit, the delta being the new tests. scripts/check_black_formatting.py passes,
and isort --check-only src/kiro_crew test conftest.py xdist_budget.py is clean —
note that the isort failure was pre-existing at #7056's head
(_REDACTED_CREDENTIAL_TAG misplaced in the import block), so #7056 will hit the
same gate on its own; this PR fixes it in passing.

Benchmark, baseline and optimised interleaved back to back in two passes so
neither is a stale capture, same session files and RNG seed on both sides:

corpus size loop bodies before -> after whole function before -> after
real-session 981.5 KB 1.01 -> 0.90 ms 53.4 -> 53.8 ms (noise)
hash-heavy 98.3 KB 11.0 -> 9.05 ms (-17%) 16.1 -> 13.6 ms (-16%)
blob-heavy 313.8 KB 35.8 -> 6.62 ms (-82%) 47.1 -> 15.4 ms (-67%)

@rnoack1
rnoack1 requested review from a team and chenmingwei23 August 30, 2026 17:47
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 30, 2026
@rnoack1
rnoack1 force-pushed the perf/redact-b64-loop-bodies branch from 31a51b6 to fdc8284 Compare August 30, 2026 18:34
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 30, 2026
@rnoack1
rnoack1 force-pushed the perf/redact-b64-loop-bodies branch from fdc8284 to 9fa64de Compare August 30, 2026 19:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 30, 2026
@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 9f61023abf6957f6a37644dae94b15af95e61dd0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9f61023

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of 9f61023abf6957f6a37644dae94b15af95e61dd0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All callers verified: _decode_b64_chunk has exactly one caller (pass 2 of redact_credentials), and _decode_b64_safe stays ungated as the oracle for _contains_fixed_credential and the differential tests. The pass-2 gate leans on the superset property that #7056's tests already pin from multiple angles (fuzz, per-branch mutation, case-folding equivalence), so the verdict-neutrality claim is grounded, not assumed. The removed %4 short-circuit is documented at the call site and pinned by cross-version tests — the one genuinely dangerous idea in this change was caught and killed before landing. No contract, schema, or public-surface changes; trivially reversible.

Design-Verdict: PASS

Verdict-neutral gates on an already-pinned superset invariant, cross-version differential tests, and rejected alternatives each pinned by a test — sound and proportionate.

[DESIGN-REVIEWED] 9f61023

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of 9f61023abf6957f6a37644dae94b15af95e61dd0 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All claims verified against the base tree: redact_credentials has 20+ real call sites (chat persistence, dashboard, workflows, subagents) with no existing cache or memo at its entry; the gate reuses the existing _might_contain_credential pre-filter rather than adding a second spelling; and the new literal-pinning test covers a gap the existing span-equality test's docstring-admitted corpus dependence leaves open. Consumer counts: _PREFILTER_MIN_LEN and _PRINTABLE_BYTES each have exactly one consumer, both satisfying the documented named-limit convention.

First-Principles-Verdict: PASS

Every item removes a measured cost on the redaction hot path or pins a named bypass risk, with byte-identical output enforced by an independent oracle.

What this change ships

Intent: make credential redaction affordable on hash-heavy and blob-heavy content without moving any verdict — a FIX (measured perf defect, 68–78% of function time in two loop bodies).

  1. Redacting digest/blob-heavy text gets much faster; output stays byte-identical — justified (measured, differential-tested)
  2. Pass 2 skips the 23-branch alternation on non-credential decoded blobs — justified; reuses existing _might_contain_credential, no second spelling
  3. New internal threshold _PREFILTER_MIN_LEN=16 — one consumer, but named-limit convention is documented; crossover measured, margin declared
  4. New internal byte-set constant _PRINTABLE_BYTES — one consumer; module level is what buys the once-only C-path
  5. A tempting %4 decode short-circuit is recorded as a 3.10/3.11 redaction bypass and test-pinned against re-adding — justified (interpreter-version constraint, verified on both)
  6. The two base64 run patterns are now literally pinned against silent one-sided widening — rides along (guards a hazard the base commit created), justified by the redaction boundary
  7. Differential corpus grows 157→213 toward this change's blast radius — justified
  8. _decode_b64_safe marked as the deliberately-unoptimized oracle — justified; keeps the differential test able to fail

Watch

Item 6 hardens the base commit's pass-3 split rather than anything this diff changes — fine here, but it belongs with #7056 if that PR is still amendable.

[FIRST-PRINCIPLES-REVIEWED] 9f61023

@github-actions

github-actions Bot commented Aug 30, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review (fork) — ✅ no blocking findings

Reviewed 9f61023abf6957f6a37644dae94b15af95e61dd0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9f61023

@rnoack1
rnoack1 force-pushed the perf/redact-b64-loop-bodies branch from 9fa64de to 8453625 Compare August 30, 2026 20:19
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Aug 30, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 1, 2026
…ses 2 and 3

Counts printable bytes with bytes.translate instead of a Python per-byte loop,
and gates the decoded-blob scan behind the existing pre-filter. Byte-identical.
@rnoack1
rnoack1 force-pushed the perf/redact-b64-loop-bodies branch from 8453625 to 9f61023 Compare September 1, 2026 01:07
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Sep 1, 2026
@iamwhatever
iamwhatever merged commit 6c3c565 into kirodotdev:main Sep 1, 2026
72 checks passed
@rnoack1
rnoack1 deleted the perf/redact-b64-loop-bodies branch September 1, 2026 20:51
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 1, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

Open PR relationship audit

This is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion.

Relationship findings

  • PR #7031 is PARTIALLY_COVERED relative to this PR. Coverage is explicitly incomplete; this finding is not a completion or closure claim. Recommended action for PR #7031: REBASE. The root-cause work the PR defers to has landed, so the 60.1%/41-168s figures that carry its entire cost-benefit argument were measured against a slower main than exists now. The memo is not implemented anywhere on main, so nothing is duplicated — but the numbers should be re-taken on current main before merge, since a much smaller contended latency changes how much new staleness the 3s window is worth buying. Files: src/kiro_crew/security.py.

No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants