Skip to content

fix(chat): key the display-redaction cache on the prefix set pass 4 judged under - #22455

Closed
cruisercohen wants to merge 2 commits into
mainfrom
fix/display-redaction-prefix-snapshot-22388
Closed

cruisercohen wants to merge 2 commits into
mainfrom
fix/display-redaction-prefix-snapshot-22388

Conversation

@cruisercohen

@cruisercohen cruisercohen commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A chat render never shows a companion approval link's ?token= value in plaintext, or as [REDACTED], because the display cache stored output judged under a different token-parameter prefix set than the one its key names.

_redact_for_display (dashboard/chat_utils_display.py) memoizes the display battery, keyed on the text, the exempt-host set and the pass-4 token-parameter prefix set (CredentialPolicy.token_param_exempt_url_prefixes). It reads the prefix set twice: once in _display_redaction_cache_key and again inside redact_credentials (pass 4, security/redaction.py). The platform read can move between the two (a companion loads, a policy tightens) and degrades every failure to the empty set. When the key read sees the empty set and the pass-4 read sees the prefix, the approval link is kept and filed as a clean verdict under the empty-set key, so every later render under the empty set shows the token value. The reverse order serves [REDACTED] for an exempt link.

Confirmed on main 754421e3b5: both regression tests below fail there.

Why it matters

The display cache fronts every chat render and feeds the persistent verdict store, so one read that moves mid-call poisons a hot entry. In one direction a ?token= value the settled policy redacts is shown; in the other an exempt approval link the user has to click is hidden.

Not a goal

  • The exempt-host set has the same double read; fix(chat): key the display-redaction cache on the host set the battery judged under #16544 (open) fixes that input. This PR touches only the prefix set. The two edit neighbouring lines of _redact_for_display, so whichever merges second takes a small rebase.
  • _redacted_preview and wire_render._context_digest keep their own key reads (the None default).
  • redact_credentials_with_records and the streaming path are unchanged.
  • Any change to which prefixes are exempt or what an exemption relaxes.

What changed (motivation → approach → change)

Root cause: two independent reads of a set that can move between them, feeding a key and a battery that must agree. Change: ONE snapshot feeds both.

  • _redact_for_display reads _token_param_exempt_prefixes() once and hands that value to _display_redaction_cache_key(..., token_param_prefixes=...) and to display_battery(..., token_param_prefixes=...). It is the value the key digested before, so every key and stored verdict tag is byte-identical to main for the same set.
  • _display_redaction_cache_key gains a keyword-only token_param_prefixes; None reads the set as before.
  • display_battery and redact_credentials gain an optional keyword token_param_prefixes. When supplied it replaces pass 4's lazy live read, and the credential pass hands it on to the nested scan of a PlantUML diagram's source (_document_link_spans -> _plantuml_verdict -> redact_credentials(source, ...)), so a diagram that carries an approval link is judged under the same set as the text around it. A snapshot goes through the same shape gate as the live read (_well_formed_token_param_prefixes, now shared by both), so it cannot exempt a URL the live read could not, and the streaming-window suspension still pins the set empty over a snapshot. None, the default for every other caller, is the live read.

No redaction rule is widened or narrowed: pass 4 judges under the set it would read itself, read once instead of twice. display_battery.py and redaction.py are rule sources, so the rules version moves once and stored verdicts are recomputed once.

Backwards compatibility

Compatible: only optional keywords whose None default keeps the live read; nothing is renamed or removed.

Removes nothing: the diff adds optional parameters and one private helper, and every existing call keeps its behaviour.

Tests

New test/test_display_redaction_prefix_snapshot.py (11 tests):

  • test_an_entry_is_keyed_on_the_prefix_set_the_battery_judged_under: first read ALLOWED, later reads empty, then settles at ALLOWED; the warm render must equal the battery output under ALLOWED. RED on main (served [REDACTED: credential]), green here.
  • test_a_tightened_set_mid_call_never_serves_plaintext: the reverse order; RED on main (served the token value), green here.
  • test_the_prefix_set_is_read_exactly_once_per_call: 1 read cold, 1 read warm (2 cold on main).
  • test_a_diagram_entry_is_keyed_on_the_set_its_scan_judged_under and test_a_diagram_scan_takes_the_snapshot_and_performs_no_live_read: the same race through a PlantUML link whose source holds the approval link; red without the diagram-scan threading (served the diagram in plaintext after the set emptied), green here.
  • test_the_key_digests_the_snapshot_it_is_handed, test_the_battery_takes_the_snapshot_and_performs_no_live_read, test_a_snapshot_passes_the_live_reads_shape_gate (with an ungated control showing it is the gate that drops the member), test_the_stream_suspension_still_wins_over_a_snapshot, test_the_default_path_still_reads_live, test_the_fixture_turns_on_the_prefix_set.

Adjusted: the redact_credentials wrappers in test_chat_utils.py and test_display_verdicts.py, and the _display_redaction_cache_key wrapper in test_slot_render_context_key.py, forward keyword arguments. test_security_gate_liveness.py re-pins _PACKAGE_LINE_BUDGET 31,509 -> 31,548 with the reason beside it.

Runs: the new file 11 passed. 71 related files (redaction, exfil, PlantUML links, display, chat_utils, facade mirrors, security gate liveness, token-param deferral, slot render memo): 3488 passed, 1 skipped; 19 more files that call the display path (slot render key, slot detail, chat history and pagination, decisions gate): 1218 passed. black, isort, flake8, mypy (3 source files), check_comment_history.py (base origin/main) and check_file_line_cap.py clean.

The race tests on main 754421e (no fix)
E       AssertionError: a cold call reads the prefix set once, for key and battery
E       assert 2 == 1
E       AssertionError: the cache served output computed under a different prefix set
E         - uth?token=Zq7vK2mP9xR4tL8nW3cY6bH1 and then reply.
E         + uth?token=[REDACTED: credential] and then reply.
E       AssertionError: the cache served a token the settled prefix set redacts
E         - uth?token=[REDACTED: credential] and then reply.
E         + uth?token=Zq7vK2mP9xR4tL8nW3cY6bH1 and then reply.
FAILED test/test_display_redaction_prefix_snapshot.py::test_the_prefix_set_is_read_exactly_once_per_call
FAILED test/test_display_redaction_prefix_snapshot.py::test_an_entry_is_keyed_on_the_prefix_set_the_battery_judged_under
FAILED test/test_display_redaction_prefix_snapshot.py::test_a_tightened_set_mid_call_never_serves_plaintext
3 failed, 1 passed

Manual verification

N/A — unit coverage sufficient: backend-only change with no user-visible dashboard state of its own; the race is pinned by the unit tests above.

Screenshots / video

N/A — no user-visible frontend surface is touched.

Related Issues

Fixes #22388

Pattern harvest

Rule candidate: review-prompt
Pattern: a memo keyed on mutable state read by the key function AND re-read by the memoized function; the two reads can disagree, so pass the key's snapshot into the function.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/redaction.py

  • @cruisercohen posted a complete reasoning template for 29258e0.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: one prefix-set snapshot now feeds both the cache key and pass 4, which fixes the root cause, the double read. Nothing is taken away, and every None default keeps the live read.

Notes

[DESIGN-REVIEWED] 29258e0

Description read

[DESCRIPTION-READ] 545a3662c64350449df7f911010c18818a92cb9a045e945644e75f06ccd593f6

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix is at the cause: both the cache key and pass 4 now use one read of the prefix set. It follows the snapshot the same function already takes for the exempt hosts. Two red-on-base tests back it, and issue #22388 is linked.

Notes

  • Item 4 — inherited: the only caller passing token_param_prefixes (_redact_for_display) hands over _token_param_exempt_prefixes(), and that value has already been through _well_formed_token_param_prefixes. So the elif token_param_prefixes is not None: exempt_prefixes = _well_formed_token_param_prefixes(...) re-gate inside redact_credentials checks the set a second time and changes nothing for the 1 caller today. Subtraction: use the snapshot as given (exempt_prefixes = token_param_prefixes) and keep the helper only for the live read.

What this change ships

Inventory (6 items) — 5 justified

Intent: a chat render never caches an approval-link verdict that was judged under a different token-prefix set than its key names. This is a FIX.

  1. The display cache key and the pass-4 redaction use the same single read of the prefix set — justified
  2. The token check inside a PlantUML diagram's source is judged under that same snapshot — justified
  3. redact_credentials, display_battery and _display_redaction_cache_key each gain an optional token_param_prefixes keyword — justified
  4. The snapshot is checked again by the shape gate inside pass 4 — rides along; checks a value the 1 caller has already checked
  5. The rules version moves, so stored display verdicts are recomputed once — justified
  6. The package line budget in the security-gate liveness test is re-pinned — justified

[FIRST-PRINCIPLES-REVIEWED] 29258e0

Description read

[DESCRIPTION-READ] 761658c96e953395f0257c5bfa75f4e428f0b300d967da0eedd615bb92dc58a4

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 29258e0

Verdict parsed from the review's SHA-scoped output markers for commit 29258e0ab72cc34924e48e2f8cfbc2dd01c12308.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 29258e0

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 false-positive: <reason, 20 to 500 characters>

@cruisercohen

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 723963a
  • What rule changed: no redaction rule changed. redact_credentials gains an optional token_param_prefixes snapshot that replaces pass 4's live read of CredentialPolicy.token_param_exempt_url_prefixes() for one call. A snapshot passes the same shape gate as the live read (_well_formed_token_param_prefixes, shared by both), and the streaming-window suspension still pins the set empty over it. Only the display cache (_redact_for_display) passes it.
  • Before -> after (what was allowed/blocked, now): before, the display cache read the prefix set for its key and pass 4 read it again; if the two reads differed, the cache stored one set's output under the other set's key (a ?token= value the settled set redacts shown in plaintext, or an exempt approval link shown [REDACTED], until eviction). After, one read feeds both, so a cached entry always matches the set its key names. Every other caller keeps the live read (None default).
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a display render judged under a stale prefix set, i.e. the defect this fixes. A snapshot cannot exempt more than the live read could (test_a_snapshot_passes_the_live_reads_shape_gate, with an ungated control). No shell, file or network gate reads this code.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result: no sandbox or command floor reads redact_credentials' pass-4 prefix set. test/test_display_redaction_prefix_snapshot.py: the three race tests are red on main 754421e (wrong-key output, plaintext token served, two reads per cold call) and green on this head.
  • No regression:
    • Tools still work (which tools/commands were run, result): no tool path passes the snapshot keyword; test_display_redaction_source_guard.py, test_redaction_facade.py, test_redaction_mirror_parity.py, test_token_param_stream_deferral.py, test_security_gate_liveness.py pass.
    • Backward compatible (old config / old data / old callers still work, how checked): optional keywords with a live-read default; cache keys and stored verdict tags are byte-identical to main for the same set (test_the_key_digests_the_snapshot_it_is_handed); test_the_default_path_still_reads_live.
    • Existing tests (which suites passed on this head): locally 66 related files (redaction, display, chat_utils, facade mirrors, token-param deferral, slot render memo): 3383 passed, 1 skipped; CI backend suites running on this head.
  • How to undo: revert the squash commit; no data or config migration (the rules version moves once, so stored display verdicts are recomputed once).

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does 29258e0ab72cc34924e48e2f8cfbc2dd01c12308 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 7 classified, 0 newly refused, 7 still allowed.
  • ubuntu-latest (platform posix): 7 classified, 0 newly refused, 7 still allowed.
  • windows-latest (platform windows): 4 classified, 0 newly refused, 4 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: No regression is confirmed. The only thing that got stricter is a display-redaction cache, and the shell classifier can't check that. Every shell-tier caller of redact_credentials (for example exfil.py:532, :540, :648) passes no snapshot and keeps the live read, and the refactored shape gate _well_formed_token_param_prefixes filters exactly as the old inline code did. The display guard itself has no verdict and is named under UNADJUDICATED.

TIGHTENING: Pass 4 of display redaction (the token-parameter exemption) now runs under the same prefix-set snapshot as the cache key -- it refuses a cache entry computed under one prefix set and stored under another set's key, which would let a token stay in plain text after the set is emptied: "prefixes = _token_param_exempt_prefixes() / key, input_bytes = _display_redaction_cache_key(text, base, hosts, token_param_prefixes=prefixes) ... redacted = display_battery(text, exempt_hosts=hosts, token_param_prefixes=prefixes)" in chat_utils_display.py, plus in redaction.py "elif token_param_prefixes is not None: exempt_prefixes = _well_formed_token_param_prefixes(token_param_prefixes)".

ADJUDICATED: 11 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: Candidates cover posix and windows. The diff assumes nothing about paths, separators, interpreters or executable names; its only new input is the prefix tuple, which passes the platform-neutral _TOKEN_PARAM_EXEMPT_PREFIX_RE gate. The rows check the shell tiers next to the change on each host's spelling:

  • a PlantUML diagram fetch (curl / curl.exe), because the diagram scan now receives the snapshot;
  • a check that the approval host is reachable, with no token parameter;
  • a search for the token_param_exempt_url_prefixes identifier (grep / findstr);
  • the pytest runs this PR touches, with / and \ separators.

No platform the diff assumes is left without a candidate.

UNADJUDICATED: The guard is pass 4 of redact_credentials, through the display battery and its cache. Call path: a transcript row is rendered → _redact_for_display(text) (chat_utils_display.py) → _token_param_exempt_prefixes() reads the set once → _display_redaction_cache_key(..., token_param_prefixes=prefixes) → on a miss, display_battery(text, exempt_hosts=hosts, token_param_prefixes=prefixes) → redact_credentials(out, token_param_prefixes=...) → _credential_redaction_plan. Pass 4 there uses _well_formed_token_param_prefixes(snapshot), and PlantUML links go through _document_link_spans → _plantuml_verdict → a nested redact_credentials. The shell classifier never reaches this path. These are the legitimate inputs I would have submitted, and none got a verdict:

  • (a) an assistant message holding a companion approval link: an exempt https prefix followed by a token query parameter, while the policy is stable. It must still render unredacted on cold and warm cache.
  • (b) the same link inside a markdown ](...) opener and inside backticks.
  • (c) a PlantUML link whose encoded source contains that approval link, under a stable non-empty set.
  • (d) a render while the stream is suspended mid-window, where the empty set is meant to win.
  • (e) a prefix member that the live read accepted before this change. It must still be accepted when it arrives as a snapshot, since both now go through the same fullmatch gate.

I can argue that (a)–(c) keep working: with the prefix set unchanged, one read gives the same tuple the old two reads gave. That is reasoning, not a confirmed result.

[SCOPE-REVIEWED] 29258e0

@cruisercohen
cruisercohen force-pushed the fix/display-redaction-prefix-snapshot-22388 branch from 723963a to 5480f6f Compare October 20, 2026 08:51
@cruisercohen

Copy link
Copy Markdown
Contributor Author

Review dispositions for 5480f6f

  • target=gpt head=723963a8a4a05797ada001aec1e819d5d4dcd4f2: recursive PlantUML redaction ignores the prefix snapshot — fixed in 5480f6f

Reproduced first: test_a_diagram_entry_is_keyed_on_the_set_its_scan_judged_under and test_a_diagram_scan_takes_the_snapshot_and_performs_no_live_read were red on 723963a (a diagram holding an approval link was served in plaintext after the set emptied). The credential pass now hands token_param_prefixes to _document_link_spans, which hands it to _plantuml_verdict and its nested redact_credentials(source, ...); both tests are green here. The exfil pass's own nested redact_credentials calls (_credential_clean, _blocked_link_safe_domain) shape only the blocked-link records, which the display battery discards, never the cleaned text, so they stay on the live read.

  • target=design / first-principles / opus head=723963a8a4a05797ada001aec1e819d5d4dcd4f2: PASS — no action.
  • target=semgrep head=c47b8575342cfa2a74cc70a8dff767a7e143fcfd: contextvar-token-reset in the new test — fixed: the suspended-window case runs in a copied context, so no token reset is used.

@cruisercohen

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 5480f6f
  • What rule changed: no redaction rule changed. redact_credentials gains an optional token_param_prefixes snapshot that replaces pass 4's live read of CredentialPolicy.token_param_exempt_url_prefixes() for one call, and hands it on to the nested scan of a PlantUML diagram's source (_document_link_spans -> _plantuml_verdict). A snapshot passes the same shape gate as the live read (_well_formed_token_param_prefixes, shared by both), and the streaming-window suspension still pins the set empty over it. Only the display cache (_redact_for_display) passes it.
  • Before -> after (what was allowed/blocked, now): before, the display cache read the prefix set for its key and pass 4 (and a diagram's nested scan) read it again; if the reads differed, the cache stored one set's output under the other set's key (a ?token= value the settled set redacts shown in plaintext, or an exempt approval link shown [REDACTED], until eviction). After, one read feeds the key, pass 4 and the diagram scan, so a cached entry always matches the set its key names. Every other caller keeps the live read (None default).
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a display render judged under a stale prefix set, i.e. the defect this fixes. A snapshot cannot exempt more than the live read could (test_a_snapshot_passes_the_live_reads_shape_gate, with an ungated control). No shell, file or network gate reads this code.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result: no sandbox or command floor reads pass 4's prefix set. test/test_display_redaction_prefix_snapshot.py: the race tests are red on main 754421e (wrong-key output, plaintext token served, two reads per cold call), the two diagram tests red on 723963a, and all 11 green on this head.
  • No regression:
    • Tools still work (which tools/commands were run, result): no tool path passes the snapshot keyword; test_display_redaction_source_guard.py, test_plantuml_link_redaction.py, test_redaction_facade.py, test_redaction_mirror_parity.py, test_token_param_stream_deferral.py, test_security_gate_liveness.py pass.
    • Backward compatible (old config / old data / old callers still work, how checked): optional parameters with a live-read default; cache keys and stored verdict tags are byte-identical to main for the same set (test_the_key_digests_the_snapshot_it_is_handed); test_the_default_path_still_reads_live.
    • Existing tests (which suites passed on this head): locally 71 related files (redaction, exfil, PlantUML links, display, chat_utils, facade mirrors, token-param deferral, slot render memo): 3488 passed, 1 skipped; CI backend suites running on this head.
  • How to undo: revert the squash commit; no data or config migration (the rules version moves once, so stored display verdicts are recomputed once).

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 20, 2026
@cruisercohen

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 8159db7
  • What rule changed: no redaction rule changed. redact_credentials gains an optional token_param_prefixes snapshot that replaces pass 4's live read of CredentialPolicy.token_param_exempt_url_prefixes() for one call, and hands it on to the nested scan of a PlantUML diagram's source (_document_link_spans -> _plantuml_verdict). A snapshot passes the same shape gate as the live read (_well_formed_token_param_prefixes, shared by both), and the streaming-window suspension still pins the set empty over it. Only the display cache (_redact_for_display) passes it.
  • Before -> after (what was allowed/blocked, now): before, the display cache read the prefix set for its key and pass 4 (and a diagram's nested scan) read it again; if the reads differed, the cache stored one set's output under the other set's key (a ?token= value the settled set redacts shown in plaintext, or an exempt approval link shown [REDACTED], until eviction). After, one read feeds the key, pass 4 and the diagram scan, so a cached entry always matches the set its key names. Every other caller keeps the live read (None default).
  • Worst case if wrong (breaks commands / leaks secret / locks user out): a display render judged under a stale prefix set, i.e. the defect this fixes. A snapshot cannot exempt more than the live read could (test_a_snapshot_passes_the_live_reads_shape_gate, with an ungated control). No shell, file or network gate reads this code.
  • Evidence checked (which sandbox result proves it): N/A for a sandbox result: no sandbox or command floor reads pass 4's prefix set. test/test_display_redaction_prefix_snapshot.py: the race tests are red on main 754421e (wrong-key output, plaintext token served, two reads per cold call), the two diagram tests red on 723963a, and all 11 green on this head.
  • No regression:
    • Tools still work (which tools/commands were run, result): no tool path passes the snapshot keyword; test_display_redaction_source_guard.py, test_plantuml_link_redaction.py, test_redaction_facade.py, test_redaction_mirror_parity.py, test_token_param_stream_deferral.py, test_security_gate_liveness.py pass.
    • Backward compatible (old config / old data / old callers still work, how checked): optional parameters with a live-read default; cache keys and stored verdict tags are byte-identical to main for the same set (test_the_key_digests_the_snapshot_it_is_handed); test_the_default_path_still_reads_live.
    • Existing tests (which suites passed on this head): locally 71 related files (redaction, exfil, PlantUML links, display, chat_utils, facade mirrors, token-param deferral, slot render memo): 3488 passed, 1 skipped, plus 19 files on the display path 1218 passed; on 5480f6f every CI backend shard passed but one test whose wrapper did not forward the new keyword (test_slot_render_context_key.py), fixed on this head.
  • How to undo: revert the squash commit; no data or config migration (the rules version moves once, so stored display verdicts are recomputed once).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 20, 2026
@cruisercohen
cruisercohen marked this pull request as ready for review October 20, 2026 10:28
@cruisercohen
cruisercohen requested review from a team and KiroCrewCaptain as code owners October 20, 2026 10:28
@cruisercohen
cruisercohen enabled auto-merge (squash) October 20, 2026 10:28
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold at 8159db70c753a0c6ce8c495bd0f9bb2f021db4b1 for human review.

This PR changes which token-parameter prefix set decides whether a ?token= value is redacted on the display path: redact_credentials pass 4 and the nested PlantUML scan in security/redaction.py can now take a caller snapshot instead of reading the set live (redaction.py#L1185). Redaction-behaviour changes are reserved for a human reviewer, so I am not approving this one.

I found no defect. The cache key and the battery now take one read of the set (chat_utils_display.py#L488-L518). A snapshot goes through the same shape gate as the live read, stream suspension still wins, and every other caller keeps the live read. Checks on this head are green.

Also open: the Security Scope Review on this head is CONCERNS, with five display-path inputs it could not adjudicate, and there is no target=scope reply yet. Mapping those inputs to the tests in test/test_display_redaction_prefix_snapshot.py, or adding the missing cases (markdown/backtick openers, trailing-slash prefix), would clear it. Auto-merge is armed, so the human approval is the remaining gate.

— crew-pr-reviewer (r115)

@cruisercohen

Copy link
Copy Markdown
Contributor Author
  • CONCERNS: five display-path inputs the shell classifier could not adjudicate

Outcome: answered, no change. The change reads the pass-4 token-parameter prefix set once per _redact_for_display call and hands that one snapshot to the cache key, to the display battery's redact_credentials call, and to the nested PlantUML scan. A snapshot passes the same shape gate as the live read (_well_formed_token_param_prefixes, the old inline filter moved verbatim), the stream suspension still pins the set empty, and every caller that passes no snapshot keeps the live read. Which links are exempt, and how pass 4 decides it, is unchanged; what changes is that the key and the output now come from the same read of the set. The five inputs, mapped to tests that pass at this head:

  1. The approval link stays visible on a cold and a warm render: test_the_fixture_turns_on_the_prefix_set and test_an_entry_is_keyed_on_the_prefix_set_the_battery_judged_under (the second render, keyed on the same set, returns the kept text) in test_display_redaction_prefix_snapshot.py. test_the_prefix_set_is_read_exactly_once_per_call pins one read per call, cold and warm.
  2. The link inside a markdown ](...) opener and inside backticks: the opener scan (_TokenParamExemptScan, _TOKEN_PARAM_EXEMPT_OPENERS) and the _token_param_exempt call are untouched by this diff, and the snapshot only sets the prefix tuple that call receives. TestTokenParamExemptUrlPrefixes::test_exempt_prefix_keeps_value in test_security.py covers both openers on the live read. This PR adds no snapshot-path case for them.
  3. A PlantUML svg/ link whose source holds the approval link: test_a_diagram_scan_takes_the_snapshot_and_performs_no_live_read (kept under the set, masked without it, no live read) and test_a_diagram_entry_is_keyed_on_the_set_its_scan_judged_under (through _redact_for_display).
  4. A message with no ?token= parameter: TestRedactForDisplay in test_chat_utils.py (test_content_cache_reuses_byte_identical_real_redactions and test_long_session_renders_warm_with_zero_redactions: a warm render runs zero redactors and matches the cold output). test_the_key_digests_the_snapshot_it_is_handed shows a key built from a snapshot equals the key built from the same live read, so a stable set keys exactly as before.
  5. A prefix with a trailing slash or a path segment: _well_formed_token_param_prefixes keeps the old filter's isinstance check and _TOKEN_PARAM_EXEMPT_PREFIX_RE fullmatch unchanged, so it drops exactly what the old gate dropped. Under that regex a trailing slash or a deeper path is well formed and kept, before and after. test_malformed_or_failing_seam_degrades_to_redaction and test_a_snapshot_passes_the_live_reads_shape_gate cover dropped shapes; no test pins a trailing-slash prefix.
    Run at this head: test_display_redaction_prefix_snapshot.py, TestTokenParamExemptUrlPrefixes and TestRedactForDisplay, 100 passed.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 21, 2026
@cruisercohen
cruisercohen force-pushed the fix/display-redaction-prefix-snapshot-22388 branch from 8159db7 to 07bed9e Compare October 21, 2026 04:55
@github-actions github-actions Bot removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision labels Oct 21, 2026
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 21, 2026
@cruisercohen
cruisercohen force-pushed the fix/display-redaction-prefix-snapshot-22388 branch from 11c85d2 to 483c838 Compare October 22, 2026 16:24
@github-actions github-actions Bot added readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 22, 2026
@cruisercohen

Copy link
Copy Markdown
Contributor Author

@iamwhatever two short questions on your r116 hold (human review of the redaction behaviour):

  1. Which maintainer will do the human review?
  2. Does anything else block besides that review and a rebase?

Since r116 on 07bed9e the head moved once, to 483c838 (10-22), a rebase onto main's exempt-host snapshot. git range-diff shows conflict resolution only: the display cache key and the battery now take both snapshots (the exempt hosts and the pass-4 prefix set) from the same single read. The pass-4 change itself is the same. The branch conflicts with main again; I will rebase once the hold lifts, or sooner if the reviewer would rather read a clean head.

…udged under

_redact_for_display read the pass-4 token-parameter prefix set once for the cache key and again inside redact_credentials, including the nested credential scan of a PlantUML diagram's source. When the platform read moved between the reads, output computed under one set was cached under the other set's key. Read the set once and hand the same snapshot to the key, to pass 4 and to the diagram scan.

Fixes #22388
@cruisercohen
cruisercohen force-pushed the fix/display-redaction-prefix-snapshot-22388 branch from 483c838 to 29258e0 Compare November 2, 2026 21:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Nov 2, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold at 29258e0 for human review of redaction behaviour, as in r115 and r116.

redact_credentials pass 4 and the nested PlantUML scan can take a caller snapshot of the token-parameter prefix set instead of a live read (

elif token_param_prefixes is not None:
). Changes to how redaction is decided are reserved for a human, so I am not approving.

I found no defect. _redact_for_display now reads the exempt hosts and the prefix set once, side by side, and hands the same values to the key and the battery (

hosts = _exempt_exact_hosts() | current_scoped_exempt_hosts()
prefixes = _token_param_exempt_prefixes()
key, input_bytes = _display_redaction_cache_key(
text, base, hosts, token_param_prefixes=prefixes
)
), which also settles the Design note about the two snapshots. The snapshot passes the same shape gate as the live read, so it cannot exempt more; the First Principles note that this re-gate is redundant for today's one caller does not block. All checks on this head are green and every AI lane is PASS or has no findings.

On @cruisercohen's two questions: a repository maintainer with write access needs to review it, and I cannot pick one. Nothing else blocks. The branch is mergeable, and auto-merge is armed, so the human approval is the only remaining gate.

— crew-pr-reviewer (r143)

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 3, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #32252; thanks for the start.

KiroCrewCaptain pushed a commit that referenced this pull request Nov 7, 2026
A chat message shown again from the display cache could carry a
different redaction than a fresh render under the settings in force at
that moment. The cached and the fresh result now always agree.

Builds on #22455.

Closes #22388
buluoray added a commit that referenced this pull request Nov 8, 2026
A chat message shown again from the display cache could carry a
different redaction than a fresh render under the settings in force at
that moment. The cached and the fresh result now always agree.

Builds on #22455.

Closes #22388

(cherry picked from commit bab2387)

Original author: Bolin Chen (#32252). Hand-resolved, tests only: test/test_security_gate_liveness.py raises release's own package line budget by this commit's 39 lines (34,548 -> 34,587) with its comment; main's separate +10 re-measure is not carried.
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you for this work! As noted in the thread, it continued in #32252, which has now merged. Closing this one as superseded by #32252.

@iamwhatever iamwhatever closed this Nov 8, 2026
auto-merge was automatically disabled November 8, 2026 09:35

Pull request was closed

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chat: display-redaction cache reads the token-parameter prefix set twice

3 participants