Repository navigation
perf(dashboard): cache clean redaction verdicts for slot detail - #19624
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The output-cache cap raise to 40,000 rests on "35.6k changed strings", which contradicts the description's "110 of 708k" redacted. If only ~110 strings change, the old 8,000 cap is enough. Not justified as shipped
What this change shipsInventory (4 items) — 3 justifiedIntent: reopening a long chat session renders fast because text already checked isn't redacted again. FIX (perf).
[FIRST-PRINCIPLES-REVIEWED] f77384a Description read[DESCRIPTION-READ] 1ae70a24c867da19d824f69f5e97cdd70c93aa60e7aeb66a9b96ff5cabb562dc That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] f77384a Description read[DESCRIPTION-READ] 7d5fb1e2894daa7f7958feef830087c46468838761c702c38b7d744209dd9502 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
c63cb29 to
10c1a42
Compare
|
GPT 6.1 finding at chat_utils.py:2406 (clean verdict drops input length): confirmed and fixed in 10c1a42. The verdict key is now |
10c1a42 to
4a310ac
Compare
…ons render warm The display-redaction cache stored every OUTPUT, so its 8,000-entry / 16 MiB caps held a few thousand strings while one slot-detail render of a long session redacts hundreds of thousands: each render evicted the last and nothing hit. Nearly every transcript string comes back unchanged (110 of 708k on a real corpus), and for those the input is the output. They are now remembered as a 128-bit slice of the same keyed digest (which already covers the exempt-host set) plus the input byte length, in two bounded generations of 200k ints (~34 MB ceiling); strings the battery changes still use the output cache, whose entry cap rises to 40k with the 16 MiB byte cap unchanged. Content that changes is a new key. The verdict key keeps the input length because the MAC input text + NUL + hosts does not mark where the text ends: a text ending in NUL + host under a narrower host set would otherwise reuse a clean verdict and show a no-longer-exempt URL. Replay of 39 real dashboard transcripts (the set the gateway bench views): repeat pass 34-37 s -> 6.7-7.3 s, 196,927 -> 0 battery calls, ~19 MB held. Byte-for-byte oracle vs the previous code: 0 diffs over 773 transcripts, cold and warm (1,546 renders). test_long_session_renders_warm_with_zero_redactions and test_clean_verdict_does_not_carry_over_a_narrowed_host_set fail on the previous code. test_the_import_prefix_is_streamed_not_joined starts the verdict store empty: the import prefix write runs display redaction, and a store other tests filled can double its table inside the traced window (4.4 MB peak at 75k verdicts), which is the cache's growth, not a copy of the prefix.
4a310ac to
f77384a
Compare
#19624 caches clean display-redaction verdicts in memory, so a long session renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of security.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Fail closed: any read error, bad MAC, version mismatch or symlink (O_NOFOLLOW) means "not scanned"; the string goes through the battery. - Bounds: one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it.
#19624 caches clean display-redaction verdicts in memory, so a long session renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of security.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Clean verdicts are stored under the second half of the 32-byte MAC and redacted ones under the first, so a tag copied between the files names nothing. - Fail closed: any read error, bad MAC, version mismatch, symlink (O_NOFOLLOW) or FIFO (O_NONBLOCK) means "not scanned"; the string goes through the battery. Storage stays off where there is no uid (Windows). - Bounds: the save-to-worker queue holds at most 256 batches and 32 Mi characters of rows; one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it.
#19624 caches clean display-redaction verdicts in memory, so a long session renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of dashboard.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Clean verdicts are stored under the second half of the 32-byte MAC and redacted ones under the first, so a tag copied between the files names nothing. - Fail closed: any read error, bad MAC, version mismatch, symlink (O_NOFOLLOW) or FIFO (O_NONBLOCK) means "not scanned"; the string goes through the battery. Storage stays off where there is no uid (Windows). - Bounds: the save-to-worker queue holds at most 256 batches and 32 Mi characters of rows; one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it.
#19624 caches clean display-redaction verdicts in memory, so a long session renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of dashboard.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Clean verdicts are stored under the second half of the 32-byte MAC and redacted ones under the first, so a tag copied between the files names nothing. - Fail closed: any read error, bad MAC, version mismatch, symlink (O_NOFOLLOW) or FIFO (O_NONBLOCK) means "not scanned"; the string goes through the battery. Storage stays off where there is no uid (Windows). - Bounds: the save-to-worker queue holds at most 256 batches and 32 Mi characters of rows; one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it.
#19624 caches clean display-redaction verdicts in memory, so a long session renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of dashboard.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Clean verdicts are stored under the second half of the 32-byte MAC and redacted ones under the first, so a tag copied between the files names nothing. - Fail closed: any read error, bad MAC, version mismatch, symlink (O_NOFOLLOW) or FIFO (O_NONBLOCK) means "not scanned"; the string goes through the battery. Storage stays off where there is no uid (Windows). - Bounds: the save-to-worker queue holds at most 256 batches and 32 Mi characters of rows; one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it.
Problem / Motivation
Goal: Reopening a long chat session renders fast because already-checked text is not redacted again.
slot_detailruns every transcript string through the display-redaction battery (exfiltration URLs, then credentials) before it is shown. A cache already existed, but it stored every OUTPUT under an 8,000-entry / 16 MiB cap. One render of a long session redacts hundreds of thousands of strings, so each render evicted the previous one and the cache effectively never hit. Reopening a long session redid all the work every time.Why it matters
Opening or switching to a long session is one of the most common dashboard actions. Under load,
slot_detailp95 was 724-808 ms and it competed with tool calls for the same CPU; a repeat pass over 39 real transcripts took 34-37 s.Not a goal
What changed (motivation → approach → change)
Symptom: repeat renders of long sessions are as slow as cold ones. Root cause: nearly every transcript string comes back from the battery unchanged (110 of 708k on a real corpus), yet each was stored as a full output entry, so the byte/entry caps were exhausted by a few thousand rows.
Change, all in
src/kiro_crew/dashboard/chat_utils.py:inttogether with the input byte length. The length is required: the MAC inputtext + NUL + hostsdoes not mark where the text ends, and the output cache already relies on its length component to tell such inputs apart.Why this over a bigger output cache: an output cache pays for the output bytes, which is exactly what made it thrash. A verdict costs a fixed small amount per string.
Measured
Gateway benchmark at a fixed offered request rate (so both sides do the same work), plus an offline replay of 39 real dashboard transcripts:
slot_detailp95Byte-for-byte oracle against the previous code: 0 diffs over 773 transcripts, rendered cold and warm (1,546 renders).
Security notes
This touches display redaction, so the risk is a secret shown unredacted.
NUL + host.Backwards compatibility
Compatible: same function, same outputs (0-diff oracle); only internal caching changed.
Removes nothing: no key, field, function or record is removed; the cache only adds a verdict store.
Tests
In
test/test_chat_utils.py:test_long_session_renders_warm_with_zero_redactions: a 12,000-row session rendered twice must run zero redactors on the second pass, give identical output, and still hide the secret in the rows that leak one; editing a row to add a secret must redact again. Fails on the previous code (checked by restoring main'schat_utils.py: "a repeat render of the session must not redact again").test_clean_verdicts_are_bounded_by_two_generations: with a small generation size, the verdict store never exceeds two generations.test_clean_verdict_does_not_carry_over_a_narrowed_host_set: a clean text under hosts{a, b}, then the same text +NUL a.example.comunder{b}, must still be redacted. Fails if the verdict key drops the input length (checked by stashing the fix).test/test_session_transfer.py::test_the_import_prefix_is_streamed_not_joinednow starts the verdict store empty. The import prefix write runs display redaction, and a store that earlier tests filled can double its hash table inside the traced window (4.4 MB peak with 75k verdicts held, against a 2.1 MB limit; reproduced locally). That is the bounded cache growing, not a copy of the prefix, which is what the test measures.Local runs: every test file that references
chat_utils/ display redaction (2,467 passed, 25 skipped), black gate, flake8, isort, mypy on the changed module.Manual verification
N/A — behavior is covered by unit tests and the byte-for-byte oracle above; no UI change.
Related Issues
Part of the gateway performance work. A follow-up PR (redact at write time) is built on top of this one and should merge after it.
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.