Repository navigation
perf(dashboard): store display-redaction verdicts on write - #19753
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that no battery path reads an operator file at run time: the trace test sees executed [DESIGN-REVIEWED] 9289dac Description read[DESCRIPTION-READ] 789e85989c16654dafb1ef52e750884609c170a8b4d085d1ea416b842112d41d That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check one thing before merge: the battery reads no runtime input (config, env var, operator pattern) that the tag and rules version leave out. The trace test only sees code. What this change shipsInventory (9 items) — 9 justifiedIntent: reopening a long session after a gateway restart renders fast instead of re-scanning every string for secrets. This is an ADDITION (a persistent cache) aimed at a measured cost: 785,748 battery calls taking about 84 s, down to 3,843.
[FIRST-PRINCIPLES-REVIEWED] 9289dac Description read[DESCRIPTION-READ] 819bc1fa32855071c84d362db7214020cb4dc290a74412944f55faf3562b2f78 That sha256 names the PR description this verdict read. Recompute it with |
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: Fail-closed admission guards on the new persistent display-verdict store ( ADJUDICATED: 14 candidates, 0 confirmed newly-refused. The diff does not touch PLATFORM COVERAGE:
UNADJUDICATED:
[SCOPE-REVIEWED] aef8d96 |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
f37a011 to
b1860db
Compare
Sensitive change review
|
|
Addressed on
|
b1860db to
aef8d96
Compare
|
Addressed on
Re-measured on this head (replay, 5 interleaved runs, median): restart open of 799 sessions 12.19 s (variant reading every sidecar first) vs 12.60 s (+3.4%); first open of the 40 longest 43.34 s (main) vs 43.74 s (+0.9%); oracle 20,375 renders vs main, 0 diffs. |
Sensitive change review
|
aef8d96 to
d367194
Compare
7ece57c to
91c1430
Compare
Sensitive change review
|
91c1430 to
09eede8
Compare
Sensitive change review
|
renders warm only until the gateway restarts. This stores each row's verdict (clean, or the redacted text) in a per-session sidecar under <data>/display-verdicts/ once the transcript save commits, and reads it back on the next open. The transcript itself is never rewritten. - Tag: HMAC over (rules version, text length, text, exempt hosts, companion token-parameter prefixes) under a subkey of token_signing.key; a verdict whose tag does not match is ignored. While the owner-view switch bypasses the credential pass, nothing is stored or read. - Rules version: sha256 of dashboard.display_battery.RULE_SOURCES (the display battery and the files its output depends on), the native gate binary when one is loaded, and the Python and Unicode versions. A release that leaves the redaction rules alone keeps every stored verdict; a process whose rule files changed on disk since import stores and reads nothing. - Clean verdicts are stored under the second half of the 32-byte MAC and redacted ones under the first, so a tag copied between the files names nothing. - Fail closed: any read error, bad MAC, version mismatch, symlink (O_NOFOLLOW) or FIFO (O_NONBLOCK) means "not scanned"; the string goes through the battery. Storage stays off where there is no uid (Windows). - Bounds: the save-to-worker queue holds at most 256 batches and 32 Mi characters of rows; one sidecar is cleared past 16 MiB, the directory past 512 MiB. - A render reads the sidecar after 4 misses on a cold open (more misses than hits), or when the sidecar is at most 1 KiB per miss so far; a live render with a few new rows scans them rather than parse a long file. Verdicts the sidecar already holds are not appended again, so reopening does not grow it. - The first save of a transcript after a restart offers its whole window; it reads the stored set first, so judged rows are neither scanned nor appended again and restarts do not grow the sidecar.
… harness test_display_verdicts covers invalidation on a rules-version, allowed-host or exempt-host change, fail-closed reads, when a render reads the sidecar, and a reopen that must not grow it, and a text ending in a revoked host that must not reuse a stored verdict, a tag copied between the files, a FIFO sidecar and a batch past the queue budget. test_display_battery_rules_version traces the battery in a fresh interpreter and fails on any package file outside RULE_SOURCES, and checks that a non-rule edit keeps the digest while any rule-file edit moves it. test/perf/redact_on_write_replay.py is the replay the PR's numbers come from.
09eede8 to
9289dac
Compare
Sensitive change review
|
mbriones98
left a comment
There was a problem hiding this comment.
Checks green and readiness passed at 9289dac; no behavior change.
Problem / Motivation
Goal: Reopening a long chat session after a gateway restart renders fast, because text already checked for secrets is not checked again.
#19624 remembers clean display-redaction verdicts in memory, so a second open in the same process is cheap. A gateway restart empties that memory: the first open of every session after it runs the whole display battery (
redact_exfiltration_urls, thenredact_credentials) over every displayed string again. On 799 real sessions that is ~87 s of CPU for the first open after each restart.Why it matters
Restarts are routine (updates, crashes, the desktop app relaunching), and the first thing a user does afterwards is open the session they were in. Long sessions are the ones that pay most: before this change the longest session in the corpus (~12k rows) took 4-5 s to open after every restart.
Not a goal
What changed (motivation → approach → change)
Symptom: after a restart, every session open re-runs the display battery. Cause: verdicts only live in process memory. Change: once a transcript save commits, a background worker projects the new rows the way
slot_detaildoes and stores each displayed string's verdict (clean, or the redacted text) in a per-session sidecar under<data>/display-verdicts/. A render reads the sidecar instead of scanning.dashboard/display_verdicts.py(new): the sidecar store, tags, read rule and bounds.dashboard/display_battery.py(new): the display battery as one function, plusRULE_SOURCES, the files its output depends on. Its docstring states that any battery input read at run time must go into the tag or the version.dashboard/chat_utils.py:_redact_for_displayconsults the sidecar after the in-memory cache misses and records new verdicts.transcript_merge.py,chat_persistence.py,chat_handlers.py): hand rows to the worker after the write commits. A save that fails records nothing.server_runtime/maintenance.py,server.py): enable the store off the event loop after bind.docs/system-specs/modules/security.md: a "Stored display verdicts" entry.Security design
The risk is showing a secret unredacted. Every rule below fails toward scanning.
HMAC(K, "tag" ‖ 0 ‖ rules_version ‖ len(text) ‖ text ‖ 0 ‖ exempt hosts)[:16].Kis a purpose-labelled subkey oftoken_signing.key, which the agent sandbox masks and the file tools fence, so a same-user agent that can write the sidecar cannot mint a "clean" tag for text it chose. A redacted record also carriesHMAC(K, "out" ‖ 0 ‖ tag ‖ output)[:16], checked on load. No usable key: nothing is stored or read.sha256(RULE_SOURCES bytes ‖ loaded native-gate binary ‖ Python version ‖ unicodedata version ‖ darwin user-dir id).RULE_SOURCESis the battery,security/redaction.py,security/redaction_switch.py,security/exfil.py,credential_patterns.py,slack_manifest.pyandslack-manifest.yaml. Any change to them changes every tag, so old verdicts are never found; a release that leaves them alone keeps its verdicts. A process whose on-disk rule files differ from the ones it imported stores and reads nothing..dvrinto.dvcyields nothing.Tunder hosts{H, Z}andT + NUL + Hunder{Z}get different tags.platform_compat.effective_uid()isNone).O_NOFOLLOW | O_NONBLOCKunder a directory fd, so a planted FIFO cannot block a render; the directory (created 0700) is openedO_NOFOLLOW | O_DIRECTORYtoo, and a non-regular sidecar is ignored. A read error, malformed or torn record, bad MAC, or another version in the file header reads as "not scanned".token_signing.keycan already mint dashboard login tokens; this does not widen that.When a render reads the sidecar
Parsing costs ~10-20 ns per sidecar byte and a scan ~40 µs per string. A render reads the session's sidecar once, after 4 misses on a cold open (misses exceed hits), or when the sidecar is at most 1 KiB per miss so far. A live session (older rows hit memory, a few new rows miss) scans up to 64 strings before it reads a long file. Verdicts already in the sidecar are not appended again.
Measured
Offline replay of real transcripts (
test/perf/redact_on_write_replay.py), one fresh process per run, all trees on the same interpreter, variants interleaved, 5 runs each, median. "Before" for the first open isorigin/main. Render time only (transcript read excluded).The restart open was also compared against the earlier in-tree variant that read every sidecar up front (no 64-miss rule): 11.88 s against 12.25 s, +3.1%, 5 interleaved runs each.
Byte-for-byte oracle against
origin/main: 20,375 renders over the 799 sessions (cold, restart, and first-open runs), 0 diffs.Not measured: a full gateway benchmark on this head. The gateway harness used during development is not part of this PR, so no gateway number is quoted.
Backwards compatibility
Compatible: transcripts are unchanged and gain no field; code that predates this never reads
display-verdicts/, and this code treats a missing or foreign sidecar as "not scanned".Removes nothing: no key, field, function or record is removed; the sidecar is a new, optional cache.
Tests
test/test_display_verdicts.py(new): a restart renders from stored verdicts; a text ending inNUL + hostafter that host is revoked does not reuse the stored verdict of the shorter text (fails without the length in the tag); a changed rules version, a revoked allowed host and a changed platform exempt host each invalidate stored verdicts (each was checked to fail with its invalidation removed: plaintext shown); forged, torn and symlinked sidecars are never served; owner view neither reads nor writes; a save that never committed leaves rows unjudged; cold-open and small-sidecar read rules; a live render does not read the sidecar; reopening does not grow it (fails on the code without the dedupe); a tag copied from.dvrinto.dvcis not served as clean, a FIFO sidecar does not block the read, a batch past the queue budget is dropped, and storage stays off without a uid (each of the first three fails with its fix removed).test/test_display_verdicts.pyis skipped on Windows, where storage is off by design.test/test_display_battery_rules_version.py(new): traces the battery in a fresh interpreter and fails on any executed package file outsideRULE_SOURCES; a non-rule edit keeps the digest, any rule-file edit moves it. Checked against mutants: droppingexfil.pyfrom the list, hashing names instead of bytes, and leaving the rule digest out of the version all fail it.test/test_display_redaction_source_guard.py,test/test_chat_utils.py,test/test_dashboard_server_composition_contract.py: updated for the battery's new home and the new boot step.test/test_stop_hook_continuation.py::TestRealHookProcess, which fail identically on unmodifiedorigin/mainin this environment.src/kiro_crew/), docs lint, brand-name, comment-history and feature-map gates pass locally.Manual verification
N/A — the replay harness above drives the real render path over real transcripts; no UI changes.
Related Issues
Follows #19624.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.