Repository navigation
feat(security): turn on the structured credential detectors, with owner engine and prompt-and-logs switches - #29423
Conversation
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ✅ human override accepted@bolichen97 overrode this lane for |
Sensitive change review
|
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the default flip ships with its rollback selector as the RFC requires, the seams scope fails closed when unpublished, and screenshots of the new toggles are attached. Notes
[DESIGN-REVIEWED] 2043597 Description read[DESCRIPTION-READ] fe0899e73efa55f4e891b4da0f20d9a2e847d9c61ffbb0ba1bd7abc28a53223b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ human override accepted@bolichen97 overrode this lane for |
GPT 6.1 Review — 🔴 changes requested (blocking)GPT 6.1 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 (fenced): UNBOUNDED, because it involves a literal credential reaching the dashboard or stream output.
[ADJUDICATION] 2043597 total=0 uphold=0 downgrade=0 [ADJUDICATION-FENCED] 2043597 fenced=1 flagged=0 False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — 🔴 changes requested (blocking)Reviewed BLOCKING: an engine change made while a dashboard tab is offline never clears that tab's cached file bodies. The reconnect heal only compares BLOCKING — website/src/hooks/websocket/serverState.ts:61 [BLOCK-MERGE] 2043597 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: The PR doesn't change anything in the shell deny composite. TIGHTENING: The structured credential engine is now on by default on every redaction path, including every admission gate that refuses when ADJUDICATED: 20 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: Candidates cover posix, windows and any.
The diff's own platform assumptions are the macOS temp-path shape ( UNADJUDICATED:
[SCOPE-REVIEWED] 2043597 |
|
Hold at 263561e. This PR targets Before approval:
The Design and First Principles notes about — crew-pr-reviewer (r145) |
bolichen97
left a comment
There was a problem hiding this comment.
[operator: bolichen97] Approving at head 263561e.
Reviewed with a focus on the switch-off path:
display_verdicts.engine_mark()is empty with the switch off, so every display cache key and stored verdict tag is the key it was before. The new engine-change checks compare two empty marks and never fire._frame_record._scrub_dict_keyonly adds a branch behindcompat.use_rule_table().- In
text_redaction.StreamRedactor, the_NO_TABLEstubs match what the base branch's table stream hooks already return with the switch off (read_bufferempty,redact_window=redact(text), discard regexes and carry unchanged for legacy kinds). A discard the table opened (table-*kinds) still uses the table module after the switch moves.
Non-blocking suggestions:
_block_tablesis keyed byid(table)with an identity check; that is safe, but it keeps a reference to every table ever loaded. A single-entry cache would be enough if tables are only reloaded on config change.
This PR targets a stacked base branch, so I have not armed auto-merge.
AI-assisted, approved by bolichen97
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes at 263561e. This replaces my approval from 11-04, which came before our review rule on regex. Sorry for the change of course.
- This PR edits the old
redaction_rules/table. #29369 now deletes that table and moves detection toredaction_structured/, so this change no longer fits the stack. - It adds regex in product code: new patterns in
redaction_rules/rules.yaml,redaction_rules/detectors.py,redaction_rules/stream.pyandredaction_rules/compat.py. We can no longer take new regex in product code.
@mbriones98, is this PR superseded by #29369? If so, would you close it? If something here still needs to land, could you rebuild it on #29369's structured table instead?
263561e to
f97f436
Compare
|
/ai-review override first-principles 1fb944e reissue: Same ruling as the earlier record #29423 (comment) on 8215d7a. bolichen97 signed the structured-engine default on RFC 34048; the RFC status line on main still reads draft. The only change since is the rebase onto main after #31863 merged: the patch touches the same 131 files with the same line counts as be654e9, except test/test_security_gate_liveness.py, which changes one more line. |
Human judgment recorded@mbriones98 recorded a reissue release (class
This decision applies only to this commit. A new push requires a new judgment. |
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 30% chance that this change introduces a defect, above the 12% limit for approving on its own. Turns on the structured credential detectors by default and adds two owner settings under Settings > Security > Redaction: an engine selector (structured or legacy) and a prompt-and-logs switch. I checked the state, lifecycle and trust boundaries. The record has a single owner-gated writer; each setter keeps the other fields under the store lock and reads fail toward redacting. Engine changes are audit-or-refuse and published under the lock. The seams verdict is published only by the gateway, so other processes keep redacting, and a policy refresh re-publishes it. Refusal gates, the exfil-URL pass and vault literals stay on, and the live log stream redacts every line whatever the switch says. Governance has its own capability row, default off. Stream boundary releases keep pass-1 matches and undecided units held. Tests are extensive. What remains is process (First Principles override still needed for this head, sensitive-change template, pending CI) and accepted, deliberately-crafted false-negative gaps that the description and docs acknowledge. None is a concrete defect I would block on. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
State of this PR after the conflict resolution and review fixes (head 95d9dfd)The branch is one commit on main 12a03ac (author Matt Briones kept, rebased, no merge commits). Content is the PR's own: structured detectors on, engine selector, prompt-and-logs switch. Nothing was disabled, skipped or weakened; the only test-side change is one raised line budget, with its usual note. What was fixed, per finding
Checks on this headGreen: GPT 6.1 Review, Opus 5.5 Review, Design Review, UX Review, Security scope review, lint and type checks, ratchets, and the rest of the matrix except as listed below. Red or pending, and why:
What the maintainer should doEvery push resets the overrides and the template, so I am making no further pushes. On this head: record the First Principles override, have |
The credential plan now reads the structured detectors first by default.
The owner can choose main's regex passes instead (the engine selector
below), and can turn credential redaction off for the agent's prompt
and the logs (the output-seams switch below).
Plan:
- Secret hits are tagged; Identifier values (AWS key IDs, documentation
example keys, git object names, bitmap image data-URI payloads,
commit permalink SHAs, request ids, the macOS per-user temp
directory) are written as a private-use placeholder while main's
passes read the text, then restored. An SVG or other non-bitmap
data-URI payload is read by main's passes.
- ARNs, public OAuth parameter values and placeholders are reported
but not masked, so main's passes still judge them.
- Pass 1 skips the branches the detectors cover (private keys, JWTs,
the AWS secret and session-token labels), the bare AWS key ID and a
labelled AccessKeyId whose value is protected. A JWT match is
dropped only when its first segment decodes to no JSON object; a
signed token the detectors refuse (no alg, as in a Flask session
cookie, alg none, glued on its left) is still removed by pass 1. A
labelled AWS secret or session-token match is dropped only when its
value is the detectors' tag or a placeholder, so a value they never
read, or refuse for its shape (an S3-compatible secret that is not
40 characters, a short session token), is still removed. Every other
branch, including the URL userinfo branch, and passes 2-4 run
unchanged.
- An Identifier mask is dropped before the passes read the text when
the base64 run under it decodes to a credential, when it overlaps a
pass-1 credential match pass 1 still reports (a hex Bearer token, a
64-character hex value after an AWS secret label), or when it is a
bitmap payload whose bytes do not open with that format's fixed
header (a PNG IHDR, a JPEG marker, a GIF version, WEBP, a BMP header
size) and that reads as a bare secret, however long it is padded or
whatever file signature is typed in front of it.
- The rest of an Authorization: Bearer value a detector hit starts (an
SD-JWT's disclosures) is claimed with the hit, as main's single match
claims the whole value.
- A token= value a detector hit touches is claimed before the passes
read the text, its uncovered parts redacted with the token
parameter pass's own tag, warning and record.
- A JWT or JWE may follow a sentence's dot; the decoded header check
decides.
- Pass-1 warnings give the length on the original text.
Stream:
- StreamRedactor's detector cut moves back to a word start and out of
any pass-1 match, and the fixed tail hold ends at the last blank.
Past the holdback cap the floor reads the detectors' own cut, so a
long unbroken run is released and the buffer stays bounded.
- release_at_boundary() commits what only the detectors' tail hold
kept back; the dashboard calls it at a tool call or prompt cut, and
Slack at a tool card, a wait and the turn's end. The released text
is redacted like any committed window.
- Dropped AWS runs are kept in the look-behind; flush() no longer
clears the discard counter.
Engine selector:
- compat.use_rule_table() reads the owner's engine selector: "engine"
in the credential-redaction keystone, structured (default) or legacy
(main's regex passes alone). A missing field, any other value or an
unreadable record selects structured. It is read once per process;
the owner-gated PUT /api/security/credential-redaction with
{"engine": ...} audits the change, writes it without moving the
file-view switch, and republishes it to the gateway at once.
- Settings -> Security -> Redaction shows it as its own "Redaction
engine" section, apart from the file-view switch.
Owner switches:
- The engine selector (above) and a second owner switch,
seams_enabled (default on), share the credential-redaction keystone
with the file-view switch; each writer keeps the others' fields, and
each change has its own time. seams_enabled turns off the credential
pass at the output seams listed in redaction_seams.py, and only
there: the conversation replayed into the agent's prompt, and the
shape passes over log lines. The exfiltration-URL pass, the literal
vault-secret pass over log lines, every request-blocking gate and
every refusal gate stay on. Chat, transcripts, the live chat stream
and channel egress are not seams. A record without the field reads
it on, so an earlier file-view OFF keeps its meaning.
- Every setter returns the switches in force under the governance
rule, so a file-view write on a governed host never publishes a
stored seams OFF. The engine choice is published under the same
store lock as its write, so overlapping PUTs publish in write order.
A seams change, by the owner or by a policy refresh, is pushed to
owner tabs as credential_redaction_settings_changed, which re-reads
the settings and purges no file body.
- Settings -> Security -> Redaction shows three toggles under three
headings: the file-view switch, the prompt-and-logs switch, and the
redaction engine.
Display:
- display_battery.RULE_SOURCES lists every redaction_structured file
and pass_compose.py, so a detector change gives stored verdicts a
new version. The process caches and stored verdicts carry the
engine's mark. A recorded frame keeps a field-name key under an
authorization header legible, decided by a plain name check.
Tests: the fixture swaps to synthetic credentials are in the base
branch (#31863). Here, tests whose subject is a value the design keeps
visible expect it visible. Tests that pin main's pass-1 internals run
with the switch off and say why. Three Slack golden digests follow the
stream's tail hold; the final text is unchanged. An unsigned image
data-URI payload and a link after an AWS secret label are judged as
with the switch off, and their tests expect that again. Files whose
expectations are the structured engine's output pin it with an autouse
fixture, so the suite also passes with the legacy engine selected. The
security package's line budget is raised by 201, re-measured.
Not generalizable: the fixture swaps follow from Identifier values
being kept visible by design.
Review fixes:
- The output-seams switch answers to its own governance row,
capabilities.credential_redaction_seams_off. The Files-view row
(capabilities.credential_redaction_off) no longer lets the owner turn
redaction off for the agent's prompt and the logs; each switch is read
against its own row in read_state and in the PUT handler.
- A bitmap payload with its base64 padding left off no longer keeps its
mask over a credential in its metadata: the encoded-credential checks
pad the run as the bitmap check does.
- The stream's cut walk (_outside_pass1_matches) is one pass back over
the pass-1 matches instead of a rescan after each move, which was
quadratic in the matches a model's output can chain.
- redact_log_via_context is not an output seam: its callers clip the
result and send it to sinks that stay redacted, so it redacts before
the clip with the seams switch off too. The logs seam is the log
record filter alone.
- The "Keep IDs readable" switch reads the same in its label, its failed-save
error and its help (a sample ID, and that Off brings back the earlier
redaction); the seams switch has its own failed-save line, so the page's
three redaction switches no longer share "Redaction is still on."
- A labelled AWS secret or key ID that starts with a protected documentation
example and has more glued on (a '.' and a real secret) is no longer
ruled on; only a protected run alone, or followed by a value-ending
character such as '&', is.
- The prompt-and-logs switch's help starts with what turning it off is for,
and the Keep IDs readable notice says it is the earlier redaction, not a
lower protection. The capture script gains the engine and seams
save-failed scenes.
- The help for the prompt-and-logs switch and its off notice say whose logs
they mean (the app's own log files on this computer); the Keep IDs
readable help says what Off does (key IDs and commit hashes hidden like
secrets) and that an example key ID is not a secret. French spacing
before double punctuation uses U+202F.
- A labelled AWS secret or session token a detector hit covers only the start
of (the first base64 run, then %2F and more) is claimed whole, as a Bearer
value is, so its tail no longer survives; and a value a protected example
only starts is no longer ruled on.
- Bengali and Hindi help text addresses the reader as tum, as those style
guides require; the listing import ceilings for mcp-cron and mcp-mate are
raised by one with the reason.
- A presigned URL's X-Amz-Signature is claimed like a detector hit (its
key ID stays readable), so the link no longer stays usable: the
signature would otherwise read as a git object name and be kept.
- Finding a presigned URL's signature against the hits and masks uses
sorted sweeps, so a listing of thousands of URLs stays linear.
- A presigned URL's signature inside a credential value (token=,
aws_secret_access_key=, SecretAccessKey, SessionToken,
aws_session_token) no longer cuts the enclosing value short, in batch
or in the stream.
- A base64 run under an Identifier mask whose decode holds a structured
Secret (a Google API key in a PNG's text chunk) is claimed whole.
- Known limits, accepted (security.md): a complete bitmap keeps its mask
when its base64 text spells a bare secret, and a 38-character name
directly under a macOS temp directory stays visible.
|
/ai-review override first-principles 2043597 reissue: Same ruling as #29423 (comment) (the premise, structured detectors on by default, was signed by bolichen97 on RFC 34048). Since 1fb944e the PR added fixes for GPT and Opus review findings, UX copy and rebases onto main; the premise is unchanged. |
Human judgment recorded@bolichen97 recorded a reissue release (class
This decision applies only to this commit. A new push requires a new judgment. |
bolichen97
left a comment
There was a problem hiding this comment.
Sensitive change review
- Head commit: 2043597
- What rule changed: Credential redaction runs the structured detectors by default, chosen by the owner's
enginefield (structuredorlegacy) in the keystonecredential_redaction.json. Their Secret hits are removed and their Identifier values are protected from main's regex passes, which still run after them. A second owner switch,seams_enabled(default on), turns the credential pass off for the prompt replay and the log record filter only; it has its own governance row (capabilities.credential_redaction_seams_off,owner_may_disable_seams), so a Files-view opt-in no longer lets the owner switch the seams off.redact_log_via_contextis no longer a seam: it redacts before the callers' clips whatever the switch says. Chat, transcripts, the live stream and channel egress stay redacted either way. - Before -> after (what was allowed/blocked, now): Before, every AWS access key ID, git object name, request id, bitmap data-URI payload, commit permalink SHA and macOS per-user temp directory was removed. After, those stay visible, and secrets are still removed by the detectors or by pass 1. Findings from the review lanes since the first head are fixed, each with a regression test that fails without the fix: a bitmap with its base64 padding left off, a bitmap whose decoded text chunk holds a structured secret, a signature inside an enclosing credential value, a labelled secret or session token that kept its tail, a presigned URL signature, the stream's quadratic cut walk, and the quadratic URL scans.
- Worst case if wrong (breaks commands / leaks secret / locks user out): Leak of a secret the detectors classify as an identifier or in a shape no pass reads. Two limits are accepted and recorded in
docs/system-specs/modules/security.mdand in the PR body ("Known gaps, accepted"): a secret whose plain text is spelled by the base64 of an image the exemption keeps, and a 38-character name after the macOS temp directory'sT/, which cannot be told from a file name. A detector error replaces the whole text with the fail-closed placeholder. A switch or engine change that cannot write its SEL record is refused (503). - Evidence checked (which sandbox result proves it): A matrix run on a DevSpace at head f41b929 (258 items, 142 synthetic plus the repo's 116 cases, through 11 surfaces and all 8 engine x seams x Files-view combinations, with a real loopback gateway writing the settings through the real PUT handler) found no item that stays visible under the structured engine but is redacted under legacy other than identifiers, documentation examples, placeholders and a clean bitmap; the unit matrix was re-run on the next head with 0 differing cells. CI on e7d5067, this head before its rebase, was green on every check except the two human gates, including GPT 6.1, Opus 5.5, UX, Design, Security Scope and all backend, Windows and macOS shards. This head is that commit rebased onto main 46f62de: the only conflicts were the generated ui-index (regenerated) and the security line budget (now one row,
test/security_line_budget/structured-detectors-on.txt, +230). After the rebase, 350 tests of the ui-index, line-budget, settings-registry, wiring and switch files pass locally; the full matrix runs on this head. - No regression:
- Tools still work (which tools/commands were run, result): No tool or command handling changed; the dashboard and Slack stream paths are covered by their redaction tests and by the gateway run in the matrix.
- Backward compatible (old config / old data / old callers still work, how checked): The record gains optional fields; a record without them reads as both switches on and the structured engine.
PUT /api/security/credential-redactionstill accepts{"enabled": bool}and returns 400 on every malformed body (matrix run). The settings survived 8 of 8 gateway restarts. - Existing tests (which suites passed on this head): see Evidence; the full CI matrix on the pre-rebase head, and the touched gates after the rebase.
- How to undo: Owner: Settings > Security > Redaction, turn off "Keep IDs readable" (main's regex passes alone, no release). Code: revert this PR. No schema or data migration is involved.
Written by Claude Code (AI-assisted) on behalf of bolichen97, who asked for this review to be recorded on the final head.
|
Maintainer decision on the UX Review BLOCK for the "AI model and server logs" help text (recorded by Claude Code, AI-assisted, on behalf of bolichen97, who ordered it). The finding is about wording: the help line packs the purpose, two effects and three untouched surfaces into one block. It is a copy suggestion, not a defect in what the switch does; the security behaviour was reviewed by the GPT, Opus and Security Scope lanes and measured in the redaction matrix. The Settings > Security > Redaction page is being replaced in the maintainer's follow-up PR by a single on/off toggle with short copy, so rewording these strings again here would only be thrown away. Accepted for this PR; the notes (split switches, grey versus yellow notice, shorter "Keep IDs readable" text) go to the follow-up PR. |
|
/ai-review override ux 2043597 owner-decision: Copy-only UX note on the AI-model-and-logs help text, accepted by the maintainer; the page is replaced by a single toggle in the follow-up PR. Decision: #29423 (comment) |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
Merging with admin rights, on the maintainer's order (recorded by Claude Code, AI-assisted, on behalf of bolichen97). Why not wait for the rest of the matrix: this head is the commit that was fully green on e7d5067 (every check except the two human gates; GPT 6.1, Opus 5.5, UX, Design, Security Scope and every backend, Windows and macOS shard passed), rebased onto main 46f62de. The only conflicts were the generated ui-index (regenerated) and the security line budget (now the row Recorded on this head: First Principles reissue override, the Sensitive change review as an approving review, and an owner-decision override for a copy-only UX note (the Settings page is replaced by a single toggle in the follow-up PR). Accepted limits are listed under "Known gaps, accepted" in the PR body and in security.md. |
Summary
This turns on the structured credential detectors that #29369 added behind an internal switch. With them on, AWS access key IDs, git object names, request ids and image payloads stay readable, and secrets are still removed. Two owner settings ship with the flip, both under Settings > Security > Redaction and both stored in the existing owner-only keystone:
The head also fixes the credential-exposure findings from the GPT 6.1 and Opus 5.5 reviews (see "Security fixes on this head").
The fixture swaps to synthetic credentials are in #31863, this PR's base, so the diff here is the behaviour change and the tests that describe it.
RFC. The RFC for a default-on structured engine is merged as #34048 (
rfc-redaction-structured-engine-default-on.md, stillstatus: drafton main). Its maintainer conditions, from bolichen97's review on #34048:How this PR meets them: the selector is in this same commit as the flip. Choosing
legacyputs back main's regex-only plan (compat.use_rule_table()answers False; a test shows an AWS key ID is removed again). Both findings were fixed before this head, with regression tests that fail with the fix removed (see the table below).Problem / Motivation
Goal: credential redaction uses the structured detectors by default, so identifiers such as AWS access key IDs stay readable while secrets are still removed, and the owner can put back main's engine without a release.
#29369 added the structured detectors behind an internal switch that is off. With it off, redaction runs only main's regex passes, which remove an AWS access key ID, a git object name or a request id as if it were a secret, and miss credentials whose shape they do not know.
Why it matters
Over-redaction hides values people need to read in chat, logs and tool output (key IDs in IAM errors, commit SHAs, temp paths), and every false removal teaches people to ignore the redaction tags. The detectors read published formats and labels, so they remove fewer harmless values and judge secrets by their real shape. The selector is the way back from a bad detector verdict that does not need a code revert.
Not a goal
What changed (motivation → approach → change)
redaction_structured/compat.py,protection.py(new),identifier_values.py,table.py,security/redaction.pytoken=value, and the rest of anAuthorization: Bearervalue, that a detector hit starts is claimed with the hit.security/text_redaction.py,slack/handler_runtime/stream.py,dashboard/chat_turn/turn_reply_seams.pyStreamRedactor's detector cut moves back to a word start and out of any pass-1 match; past the holdback cap the floor is lowered only for a unit the detectors are still deciding.release_at_boundary()commits what only the detectors' tail hold kept back; the dashboard calls it at a tool call or prompt cut, and Slack at a tool card (before main's sentence-end release), a wait and the turn's end.security/redaction_switch.py,dashboard/handlers/credential_redaction.py,redaction_structured/compat.pycredential_redaction.json, now holds three independent settings: the file-view switch (enabled), the output-seams switch (seams_enabled, default on) and the engine selector (engine:structured, the default, orlegacy). Each has its own change time, and each writer keeps the others' fields. A missing field reads as on (switches) or structured (engine), so an earlier file-view OFF keeps its meaning with no migration. The owner-gatedPUT /api/security/credential-redactiontakes exactly one ofenabled,seams_enabledorengineper write and writes an SEL record first; turning a switch off, or choosinglegacy, does not happen when that record cannot be written (503). A governed host needs the policy's opt-in to turn either switch off; the engine is not governed, because either engine removes every secret main removes. Every writer returns the switches in force under the governance rule, the engine choice is published under the same store lock as its write, and a seams change (by the owner or a policy refresh) is pushed to owner tabs ascredential_redaction_settings_changed, which re-reads the settings without purging file bodies.compat.use_rule_table()reads the engine selector once per process; the PUT republishes it to the gateway at once. Other processes use a change when they next start. The merge steps after the detectors (_token_param_gaps,_bearer_gaps,_joined) find the hits a value overlaps by bisect over sorted spans and test membership in a set, so their work grows linearly with the number of hits and values (pinned by operation counts intest/test_redaction_structured_scaling.py).redaction_seams.py(new),kiro_crew/context.py,context_assembly/session_start.py,log_redaction.py,platform/context.py,security_posture.py,dashboard/handlers/updates_streams.py,security_posture_sinks_delivery.pyseams_enabledis off. The gateway publishes the verdict at boot, on every policy refresh and on each change, so no seam reads a file on a hot path. The exfiltration-URL pass, the literal vault-secret pass over logs, every request-blocking gate and every refusal gate stay on. The dashboard's live log stream and its ring replay reach non-owner sockets, so they redact every line whatever the switch says; the sink is registered insecurity_posture_sinks_delivery.py.website/(pages/settings,api/client/security.ts,hooks/websocket/serverState.ts,commandPalette/settingsManual.ts,i18n/locales/en.manual.jsonand the eleven other catalogs)credential_redaction_settings_changed, and the command palette lists the two new settings.dashboard/display_battery.py,display_verdicts.py,chat_utils.py,chat_utils_display.py,acp/_frame_record.pyRULE_SOURCESlists everyredaction_structuredfile; caches, stored verdicts anddisplay_redaction_stampcarry the engine's mark. A field-name key under an authorization header in a recorded frame stays readable.docs/system-specs/modules/security.md,acp-client.md,docs/settings-registry.generated.json,docs/ui-index.generated.json,.github/black-baseline.txtAWS access key IDs stay visible. An access key ID names a key; it does not authenticate without its secret, and AWS prints it in errors, policies and CLI output. The secret key and the session token are still removed.
Known non-credential values intentionally not redacted. These are kept visible by design; a credential padded to their shape is shown too, and the
legacyengine removes them.x-amz-id-2: a genuine value is base64 of random bytes, so a padded secret cannot be told apart from it, and scanning it again would re-redact 98.7% of genuine values (measured).HostId: the same value asx-amz-id-2under a JSON or XML label, so it has the same cost.SHA256:fingerprint: 43 random base64 characters and public by design; scanning it again would re-redact 87.7% of genuine values (measured).Known gaps, accepted.
token=value that a detector hit touches gets different redaction records and warnings than with the switch off. The text is fully redacted either way. Tracked in Detect labelled credential values in the structured engine #31692.x-amz-id-2,HostIdorSHA256:slot keeps its mask on a value shaped like a real ID: a 40-character key followed by 16 or more characters that together are canonical base64, or a 43-character value that happens to end on a zero padding bit pair, stays readable.legacyengine removes such payloads. Documented indocs/system-specs/modules/security.md./var/folders/<xx>/<id>/T/<name>) stays visible: with theT/it is 40 characters, and a genuine 38-character file name cannot be told apart from a secret borrowing the directory-kind character. Pinned bytest_a_38_char_name_after_the_temp_directory_stays_visible; thelegacyengine removes it. Documented indocs/system-specs/modules/security.md.Backwards compatibility
Compatible: no config key, API or stored format is rejected or renamed. The credential-redaction record gains optional
seams_enabled,seams_changed_at,engineandengine_changed_atfields; a record written before them reads as both switches on (or as stored) and the structured engine, and the file-view switch keeps its meaning. The PUT still accepts{"enabled": bool}exactly as before. Stored display verdicts carry the engine's mark, so a verdict computed under the other engine is recomputed, not reused.Removes nothing: no path, record, key or public function is removed; main's regex passes still run after the detectors.
Tests
test/test_redaction_structured_wiring.pyandtest/test_redaction_structured_protect.pypin the switch-on plan.test/test_redaction_engine_selector.py(15 tests): structured by default, legacy when chosen (an AWS key ID removed again), the choice round-trips and keeps both switches and their times, an old record with no field loads as structured, the GET and PUT, the audit-or-refuse rule.test/test_redaction_output_seams.py(21 tests) and the switch tests intest/test_credential_redaction_switch.py: each seam on and off, the vault-secret and exfiltration-URL passes staying on, the refusal gates still refusing, the stored-OFF migration, the PUT forseams_enabled.website/src/pages/settings/SecurityPanel.redactionEngine.test.tsx(4) andSecurityPanel.credentialRedaction.test.tsx(19, including the prompt-and-logs switch): defaults, notices, each toggle goes to its own writer, and the three toggles have different headings and labels.main(619d46aaab). This run was on an earlier build of this commit, before the engine pins and the owner-setting fixes below.redaction_structured_cases.structured_engine), so they pass whichever engine is selected. Over 263 redaction, credential, secret, security, seam, liveness and PR-changed test files on this head: 19,422 passed with the structured engine and 19,422 passed with the legacy engine selected, none failed.test_a_files_view_on_after_a_withdrawn_opt_in_keeps_the_seams_on,test_every_setter_returns_the_switch_in_force); the engine is published under the store lock (test_the_engine_is_published_inside_the_write_transaction); a seams change reaches owner tabs (test_put_seams_off_writes_publishes_and_audits,test_a_policy_move_of_the_seams_alone_reaches_owner_tabs, and thecredential_redaction_settings_changedcase inuseWebSocket.credentialRedactionChanged.test.ts).test/test_mcp_dashboard_session_queue.py(two excerpt tests) andtest/test_mcp_server_wait.py(test_a_name_in_the_hint_is_read_back_scrubbed), from main, used an AWS key ID as the credential; they use a synthetic GitHub token so they test a credential on both engines.test/test_project_hooks_consent.py::test_notice_command_rows_redact_before_they_cutchecks a credential cut at the notice limit; it used an AWS key ID, which the structured engine keeps readable, and uses a synthetic GitHub token so it tests a credential on both engines.test/test_secrets_usage.py(test_an_agent_written_grant_key_and_job_name_are_redacted) andtest/test_app_backend_sdk.py(test_scrub_outbound_is_the_context_scrub), from main, used an AWS key ID as the credential; they use a synthetic GitHub token so they test a credential on both engines.test/test_stream_tail_release.py, from main, pins two of main's regex patterns: an AWS key ID split across two chunks, and a two-parteyJrun glued to a released word. Both assertions are kept and run with the legacy engine selected; the structured engine keeps the key ID readable and reads that run, with or without the word, as no credential.test_a_secret_split_across_two_chunks_is_masked_wholechecks the same dashboard path with a synthetic GitHub token on both engines.npm run i18n:check,npm run typecheck, eslint on the changed files, and 2,954 tests across Settings, the command palette, i18n, UI locations and the redaction frame pass.scripts/docs-lint.shpass. Semgrep in baseline mode, as CI runs it, reports nothing. The security package's line budget is raised by 201, from 35,804 to 36,005; the package measures 36,003 lines.46eab0d264, before the fixes below; every fix only adds redactions, so recall can rise and precision and break rate can move. The corpora are not on the host this head was built on.Proof: not provable -- other: the change is the switch flip and its owner settings; the wiring, protection, selector and seam tests fail on the base
Security fixes on this head
All are in
redaction_structured/; each has tests that fail with the fix removed and pass with it, in batch and streamed.residual_matchesdrops a labelled AWS match only when its value is the detectors' tag or a placeholder; an identifier mask under such a value is dropped, except the published AWS example secret.test_a_labelled_secret_the_detectors_do_not_read_is_still_removed,test_a_labelled_value_the_detectors_refuse_for_its_shape_is_removedalg),alg: none, a JWT glued after_or-(GPT 6.1, Opus 5.5)._jose_refusal_ruledsuppresses a JWT match only when its first segment decodes to no JSON object.test_a_signed_token_the_detectors_refuse_is_redactedtEXtchunk, anx-amz-id-2value that decodes to a labelled secret) from the encoded-credential pass (GPT 6.1)._unmask_encoded_credentialsdrops a mask over a base64 run that decodes to a credential.test_an_image_payload_holding_an_encoded_secret_is_removed,test_a_request_id_holding_an_encoded_secret_is_removed_unmask_credential_matchesdrops a mask that overlaps a pass-1 match pass 1 still reports.test_a_hex_credential_shaped_like_a_git_object_name_is_removediVBORw0KGgo,/9j/,R0lGOD), or after a real image's first 24 bytes (GPT 6.1).data_uriandimage.base64) is kept only when the whole payload decodes to one complete file of its format (protection.well_formed_bitmap_head): a PNG's chunks fromIHDRthroughIDATtoIEND, each with its CRC and nothing after; a JPEG's marker segments, each length landing on the next marker, through a frame and a scan toEOI; a GIF's blocks and sub-blocks through an image to its trailer; a WebP's RIFF chunks tiling the file around a VP8 or VP8L frame; an uncompressed BMP's pixel rows filling the file. Otherwise a payload that reads as a bare secret loses its mask. Of the repo's 1,639 images, 1,638 validate as complete (the same 1,638 before and after the structure walk); all 373 within the row's 65,536-character bound stay visible as data URIs, the same as before. Pillow's baseline, progressive and restart-marker JPEG, still and animated GIF, lossy, lossless, alpha and animated WebP, 1-, 8- and 24-bit BMP and PNG output all validate.test_a_valid_image_header_does_not_shield_an_appended_secret,test_a_forged_file_signature_does_not_make_a_secret_an_image,test_a_well_formed_bitmap_payload_stays_visible,test_a_bare_complete_image_payload_stays_visible,test_a_secret_inside_a_malformed_image_is_removed(a zero-length JPEG APP0, a GIF with no image, a WebP with no VP8 frame, a BMP whose rows do not fit, a PNG with noIDAT; fails on the signature-and-trailer check),test_gpt_review_jpeg_with_a_zero_length_app0_is_not_an_imageAuthorization: Bearerkept its disclosures in clear (main's test).compat._bearer_gaps).test_a_bearer_value_a_detector_hit_starts_is_redacted_whole,test_agent_stream_tail.py/var/folders/<b>/<key>/T) kept its Identifier mask and was shown (Opus 5.5).macos.temp_pathmask is dropped when its<id>reads as a bare secret; a genuine id is 28-30 characters of[a-z0-9_]and never does.test_a_bare_secret_typed_as_a_macos_temp_dir_id_is_removedlog-scoped apps) carried unredacted lines (Opus 5.5).updates_streams._stream_saferedacts every line in a fresh context before it is buffered or sent, whatever the switch says (a record built while the switch was off can reach the stream after it is back on, GPT 6.1); about 24 µs per ordinary log line; registered as a redaction sink and listed inredaction_seams.ALWAYS_REDACTED.TestLiveLogStreamintest_redaction_output_seams.py, includingtest_a_record_made_while_the_seams_were_off_is_redacted_after_they_come_back(fails on the switch-gated sink)alg: none, a Flask cookie) in anx-amz-id-2orHostIdslot kept its Identifier mask and was shown (GPT 6.1)._unmask_credential_matchesdrops a mask over ajwtmatch pass 1 keeps.test_a_refused_token_in_an_identifier_slot_is_redacted; 0 of 60,000 genuine-shapedx-amz-id-2/HostIdvalues (header, JSON, XML) are redacted, the same as without the fixseed_pushed_switchandpush_switch_moved_by_policyread and publish underredaction_switch.transaction().TestPublishUnderTheStoreLock,TestRefresherOwnerWriteRace(fails on the unlocked handler)compat.use_rule_table()once and hands that answer to pass 1 (_main_planand_pattern_passtake it;_pass1_matchesreads the selector only for the stream's holds).test_redaction_engine_read_once.py: a selector that answers one engine then the other, no threads or sleeps; fails on the old planx-amz-id-2:,"HostId":,<HostId>orSHA256:kept that row's Identifier mask, so the later passes never read it (GPT 6.1).x-amz-id-2andHostId, the canonical base64 of 32 bytes for a fingerprint. Another value that reads as a bare secret loses its mask (compat._issued_id).test_a_bare_secret_in_an_aws_id_slot_is_redacted(12 cases),test_a_secret_glued_into_a_sha256_fingerprint_slot_is_redacted;test_a_genuine_shaped_aws_id_stays_readableandtest_a_genuine_sha256_fingerprint_stays_readablepin real shapesOSErrorleft the security event log saying the engine changed, with no failure row (Opus 5.5)._put_enginewrites awrite_failedaudit row and re-raises, as the switch writers do.test_an_engine_write_that_fails_is_audited_as_failed_stream_saferedacts one line per thread at a time; a line logged while it is redacting is replaced whole with the unavailable tag.test_a_line_the_redactor_logs_while_redacting_does_not_recurse(hung past 120 s before the guard)compat._passes_view), so every placeholder the passes see is a mask this plan wrote.test_a_placeholder_character_in_the_input_does_not_exempt_a_labelled_value(six labels; before, after and inside the value; 12 cases fail on the old head)legacystill showed AWS key IDs there (Opus 5.5).display_verdicts.engine_mark(), with or without a store, so the render memo's context digest changes with the engine.test_an_engine_change_re_renders_without_the_verdict_store(fails on the old head)legacyand back while one display render ran left the engine mark as it was, so the legacy answer was kept under the structured key in the string, preview and render caches (GPT 6.1).compat.engine_generation()counts every engine publish; each display cache write, and the render memo, keeps an answer only when the count is the same before and after the work.test_display_engine_round_trip.py(string, preview, render memo; the battery does the round trip itself, no threads or sleeps; all three fail on the old head)compat._fail_closedlogs and counts once per failure on a thread; a failure nested inside that report still replaces its text.test_a_failed_redaction_is_reported_once_with_the_stream_attached(fails on the old head with three counts)<id>of a macOS temp path kept its Identifier mask: the encoded-credential walk reads the run from/varwith the id inside it, which decodes to nothing (Opus 5.5).compat._temp_id_is_credentialalso decodes the id on its own and drops the mask when it decodes to a credential.test_an_encoded_secret_as_the_temp_dir_id_is_removed(both prefixes; fails on the old head); the genuine-id assertions in the temp-dir tests still pass_cut_segmentsettles the held run only against a non-empty release.test_a_word_split_by_a_boundary_stays_put_through_the_next_one(both boundary kinds; fails on the old head)frStyle.test.ts[changed-values](fails on the old string withI18N_BASE_REFset)Sensitive change review
Posted for head
be654e928f21: #29423 (comment)Manual verification
The Settings page was rendered for the screenshots above with fixture answers, not against a live gateway. Still to check by hand on a running gateway: each of the three toggles on and off, and an agent prompt and a log line with the prompt-and-logs switch off and on.
Screenshots / video
Settings > Security > Redaction, rendered from the built Settings page with fixture answers (
website/scripts/capture-credential-redaction.mjs, run at the final head), attached to this PR (nothing is committed).Before (
main): the file-view switch and allowed link hosts only.After, defaults: the "AI model and server logs" and "Redaction engine" (Keep IDs readable) sections are added, all on.
After, everything off: each card shows its notice; with IDs no longer kept readable, the Keep IDs readable card says AWS key IDs are hidden too and that this is the earlier redaction.
Write failed, engine: the write is rejected; the card names the switch (Could not change 'Keep IDs readable') and says IDs are still kept readable, with a link to ask the agent.
Write failed, AI model and server logs: the write is rejected; the card says credentials are still hidden from the AI model and server logs.
Related Issues
Part of #22552. Part of #31895 (the agent's prompt and the logs; chat and transcripts get their own design). RFC: #34048. Follow-ups: #31690 (retire the shape-only passes), #31692 (labelled credential values; the URL userinfo records gap).
Stack: #31863 (fixtures, base
main) → this PR (base #31863's branch). Review #31863 first.Pattern harvest
Not generalizable: the expectation changes follow from identifier values being kept visible by design.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.
Written by the Crew Tasks Bot on behalf of mbriones98.