Skip to content

feat(security): turn on the structured credential detectors, with owner engine and prompt-and-logs switches - #29423

Merged
bolichen97 merged 1 commit into
mainfrom
mdbrio/redaction-pr2
Nov 14, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
mdbrio/redaction-pr2

Conversation

@mbriones98

@mbriones98 mbriones98 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This turns on the structured credential detectors that #29369 added behind an internal switch. With them on, AWS access key IDs, git object names, request ids and image payloads stay readable, and secrets are still removed. Two owner settings ship with the flip, both under Settings > Security > Redaction and both stored in the existing owner-only keystone:

  • Redaction engine picks which engine redacts wherever redaction runs: the structured detectors (the default) or main's regex passes alone.
  • AI model and server logs ("Hide credentials from the AI model and server logs", part of Add one owner setting that turns credential redaction off on every path #31895) turns the credential pass off at two output seams only: the conversation replayed into the agent's prompt, and the shape passes over log lines. Chat, transcripts, the live chat stream and channel egress are not covered and stay redacted.

The head also fixes the credential-exposure findings from the GPT 6.1 and Opus 5.5 reviews (see "Security fixes on this head").

The fixture swaps to synthetic credentials are in #31863, this PR's base, so the diff here is the behaviour change and the tests that describe it.

RFC. The RFC for a default-on structured engine is merged as #34048 (rfc-redaction-structured-engine-default-on.md, still status: draft on main). Its maintainer conditions, from bolichen97's review on #34048:

The owner engine selector (the rollback switch) ships in the same PR as the default-on flip (#29423), not as a later PR. This answers open question 2 in the RFC.
The two blocking findings on #29423 are fixed first, each with a regression test: legacy pass-1 matches dropped for covered rules even when the detectors did not handle the value, and identifier masks applied before pass 2 so an encoded secret inside a masked span is never read.

How this PR meets them: the selector is in this same commit as the flip. Choosing legacy puts back main's regex-only plan (compat.use_rule_table() answers False; a test shows an AWS key ID is removed again). Both findings were fixed before this head, with regression tests that fail with the fix removed (see the table below).

Problem / Motivation

Goal: credential redaction uses the structured detectors by default, so identifiers such as AWS access key IDs stay readable while secrets are still removed, and the owner can put back main's engine without a release.

#29369 added the structured detectors behind an internal switch that is off. With it off, redaction runs only main's regex passes, which remove an AWS access key ID, a git object name or a request id as if it were a secret, and miss credentials whose shape they do not know.

Why it matters

Over-redaction hides values people need to read in chat, logs and tool output (key IDs in IAM errors, commit SHAs, temp paths), and every false removal teaches people to ignore the redaction tags. The detectors read published formats and labels, so they remove fewer harmless values and judge secrets by their real shape. The selector is the way back from a bad detector verdict that does not need a code revert.

Not a goal

What changed (motivation → approach → change)

Area Change
redaction_structured/compat.py, protection.py (new), identifier_values.py, table.py, security/redaction.py Secret hits are tagged. Identifier values (AWS access key IDs, documentation example keys, git object names, bitmap image payloads, commit permalink SHAs, request ids, the macOS per-user temp directory) are held as a placeholder while main's passes read the text, then restored. ARNs, public OAuth parameter values and placeholders are reported but not masked. Pass 1 skips the branches the detectors cover, except a JWT whose first segment is a JSON object and a labelled AWS value that is not the detectors' tag or a placeholder. A token= value, and the rest of an Authorization: Bearer value, that a detector hit starts is claimed with the hit.
security/text_redaction.py, slack/handler_runtime/stream.py, dashboard/chat_turn/turn_reply_seams.py StreamRedactor's detector cut moves back to a word start and out of any pass-1 match; past the holdback cap the floor is lowered only for a unit the detectors are still deciding. release_at_boundary() commits what only the detectors' tail hold kept back; the dashboard calls it at a tool call or prompt cut, and Slack at a tool card (before main's sentence-end release), a wait and the turn's end.
security/redaction_switch.py, dashboard/handlers/credential_redaction.py, redaction_structured/compat.py One keystone record, credential_redaction.json, now holds three independent settings: the file-view switch (enabled), the output-seams switch (seams_enabled, default on) and the engine selector (engine: structured, the default, or legacy). Each has its own change time, and each writer keeps the others' fields. A missing field reads as on (switches) or structured (engine), so an earlier file-view OFF keeps its meaning with no migration. The owner-gated PUT /api/security/credential-redaction takes exactly one of enabled, seams_enabled or engine per write and writes an SEL record first; turning a switch off, or choosing legacy, does not happen when that record cannot be written (503). A governed host needs the policy's opt-in to turn either switch off; the engine is not governed, because either engine removes every secret main removes. Every writer returns the switches in force under the governance rule, the engine choice is published under the same store lock as its write, and a seams change (by the owner or a policy refresh) is pushed to owner tabs as credential_redaction_settings_changed, which re-reads the settings without purging file bodies. compat.use_rule_table() reads the engine selector once per process; the PUT republishes it to the gateway at once. Other processes use a change when they next start. The merge steps after the detectors (_token_param_gaps, _bearer_gaps, _joined) find the hits a value overlaps by bisect over sorted spans and test membership in a set, so their work grows linearly with the number of hits and values (pinned by operation counts in test/test_redaction_structured_scaling.py).
redaction_seams.py (new), kiro_crew/context.py, context_assembly/session_start.py, log_redaction.py, platform/context.py, security_posture.py, dashboard/handlers/updates_streams.py, security_posture_sinks_delivery.py The output seams: the prompt replay sites and the log redaction stand down when seams_enabled is off. The gateway publishes the verdict at boot, on every policy refresh and on each change, so no seam reads a file on a hot path. The exfiltration-URL pass, the literal vault-secret pass over logs, every request-blocking gate and every refusal gate stay on. The dashboard's live log stream and its ring replay reach non-owner sockets, so they redact every line whatever the switch says; the sink is registered in security_posture_sinks_delivery.py.
website/ (pages/settings, api/client/security.ts, hooks/websocket/serverState.ts, commandPalette/settingsManual.ts, i18n/locales/en.manual.json and the eleven other catalogs) Settings > Security > Redaction shows three toggles under three headings: "Credential redaction in file views", "AI model and server logs" ("Hide credentials from the AI model and server logs"; one line says chats, Slack and values on the Secrets page stay protected either way), and "Redaction engine" ("Keep IDs readable"; one line: "On: keeps IDs such as AWS key IDs and commit SHAs readable. Off: hides them too. This setting doesn't change whether secrets are hidden."). A failed write shows an inline error on its own card and leaves the toggle as it was. Strings in all twelve languages; the settings registry and UI index are regenerated. The API client gains the engine and seams writers, the socket hook re-reads the settings on credential_redaction_settings_changed, and the command palette lists the two new settings.
dashboard/display_battery.py, display_verdicts.py, chat_utils.py, chat_utils_display.py, acp/_frame_record.py RULE_SOURCES lists every redaction_structured file; caches, stored verdicts and display_redaction_stamp carry the engine's mark. A field-name key under an authorization header in a recorded frame stays readable.
docs/system-specs/modules/security.md, acp-client.md, docs/settings-registry.generated.json, docs/ui-index.generated.json, .github/black-baseline.txt Describe the above; the settings registry and UI index are regenerated, and the black baseline drops files this PR formats.

AWS access key IDs stay visible. An access key ID names a key; it does not authenticate without its secret, and AWS prints it in errors, policies and CLI output. The secret key and the session token are still removed.

Known non-credential values intentionally not redacted. These are kept visible by design; a credential padded to their shape is shown too, and the legacy engine removes them.

  • x-amz-id-2: a genuine value is base64 of random bytes, so a padded secret cannot be told apart from it, and scanning it again would re-redact 98.7% of genuine values (measured).
  • HostId: the same value as x-amz-id-2 under a JSON or XML label, so it has the same cost.
  • SSH SHA256: fingerprint: 43 random base64 characters and public by design; scanning it again would re-redact 87.7% of genuine values (measured).

Known gaps, accepted.

  • URL userinfo inside a token= value that a detector hit touches gets different redaction records and warnings than with the switch off. The text is fully redacted either way. Tracked in Detect labelled credential values in the structured engine #31692.
  • The image check walks each format's structure, not its compressed or raw pixel data: bytes spliced inside a JPEG scan, a GIF image block, a WebP frame or BMP pixel rows (sized to fit) still read as a complete image. A PNG's CRCs catch a splice into an existing chunk, but not a crafted extra chunk with its own valid CRC.
  • An x-amz-id-2, HostId or SHA256: slot keeps its mask on a value shaped like a real ID: a 40-character key followed by 16 or more characters that together are canonical base64, or a 43-character value that happens to end on a zero padding bit pair, stays readable.
  • A bitmap that decodes to one complete file keeps its mask even when its base64 text happens to spell a 40-character bare secret. An agent that already holds a secret can encode it elsewhere, and redacting every image payload that holds a secret-shaped run would redact ordinary images again, which the structured engine exists to stop. The legacy engine removes such payloads. Documented in docs/system-specs/modules/security.md.
  • A 38-character name directly under a macOS per-user temp directory (/var/folders/<xx>/<id>/T/<name>) stays visible: with the T/ it is 40 characters, and a genuine 38-character file name cannot be told apart from a secret borrowing the directory-kind character. Pinned by test_a_38_char_name_after_the_temp_directory_stays_visible; the legacy engine removes it. Documented in docs/system-specs/modules/security.md.

Backwards compatibility

Compatible: no config key, API or stored format is rejected or renamed. The credential-redaction record gains optional seams_enabled, seams_changed_at, engine and engine_changed_at fields; a record written before them reads as both switches on (or as stored) and the structured engine, and the file-view switch keeps its meaning. The PUT still accepts {"enabled": bool} exactly as before. Stored display verdicts carry the engine's mark, so a verdict computed under the other engine is recomputed, not reused.

Removes nothing: no path, record, key or public function is removed; main's regex passes still run after the detectors.

Tests

  • test/test_redaction_structured_wiring.py and test/test_redaction_structured_protect.py pin the switch-on plan.
  • test/test_redaction_engine_selector.py (15 tests): structured by default, legacy when chosen (an AWS key ID removed again), the choice round-trips and keeps both switches and their times, an old record with no field loads as structured, the GET and PUT, the audit-or-refuse rule.
  • test/test_redaction_output_seams.py (21 tests) and the switch tests in test/test_credential_redaction_switch.py: each seam on and off, the vault-secret and exfiltration-URL passes staying on, the refusal gates still refusing, the stored-OFF migration, the PUT for seams_enabled.
  • website/src/pages/settings/SecurityPanel.redactionEngine.test.tsx (4) and SecurityPanel.credentialRedaction.test.tsx (19, including the prompt-and-logs switch): defaults, notices, each toggle goes to its own writer, and the three toggles have different headings and labels.
  • Run with the structured engine (the default), 834 files (every file changed here or in test: use synthetic credentials in redaction test fixtures #31863, every redaction, credential, secret, security, liveness, seam, log-redaction and session-start test, and every file still holding the AWS example key): 50,513 passed, 137 skipped, 39 failed; all 39 fail the same way on untouched main (619d46aaab). This run was on an earlier build of this commit, before the engine pins and the owner-setting fixes below.
  • The 22 test files that assert the structured engine's output pin it with an autouse fixture (redaction_structured_cases.structured_engine), so they pass whichever engine is selected. Over 263 redaction, credential, secret, security, seam, liveness and PR-changed test files on this head: 19,422 passed with the structured engine and 19,422 passed with the legacy engine selected, none failed.
  • Owner settings, each test written first and failing before its fix: a Files-view write on a governed host keeps the seams on (test_a_files_view_on_after_a_withdrawn_opt_in_keeps_the_seams_on, test_every_setter_returns_the_switch_in_force); the engine is published under the store lock (test_the_engine_is_published_inside_the_write_transaction); a seams change reaches owner tabs (test_put_seams_off_writes_publishes_and_audits, test_a_policy_move_of_the_seams_alone_reaches_owner_tabs, and the credential_redaction_settings_changed case in useWebSocket.credentialRedactionChanged.test.ts).
  • test/test_mcp_dashboard_session_queue.py (two excerpt tests) and test/test_mcp_server_wait.py (test_a_name_in_the_hint_is_read_back_scrubbed), from main, used an AWS key ID as the credential; they use a synthetic GitHub token so they test a credential on both engines.
  • test/test_project_hooks_consent.py::test_notice_command_rows_redact_before_they_cut checks a credential cut at the notice limit; it used an AWS key ID, which the structured engine keeps readable, and uses a synthetic GitHub token so it tests a credential on both engines.
  • test/test_secrets_usage.py (test_an_agent_written_grant_key_and_job_name_are_redacted) and test/test_app_backend_sdk.py (test_scrub_outbound_is_the_context_scrub), from main, used an AWS key ID as the credential; they use a synthetic GitHub token so they test a credential on both engines.
  • test/test_stream_tail_release.py, from main, pins two of main's regex patterns: an AWS key ID split across two chunks, and a two-part eyJ run glued to a released word. Both assertions are kept and run with the legacy engine selected; the structured engine keeps the key ID readable and reads that run, with or without the word, as no credential. test_a_secret_split_across_two_chunks_is_masked_whole checks the same dashboard path with a synthetic GitHub token on both engines.
  • Frontend on this head: npm run i18n:check, npm run typecheck, eslint on the changed files, and 2,954 tests across Settings, the command palette, i18n, UI locations and the redaction frame pass.
  • flake8, isort, the black gate, mypy and scripts/docs-lint.sh pass. Semgrep in baseline mode, as CI runs it, reports nothing. The security package's line budget is raised by 201, from 35,804 to 36,005; the package measures 36,003 lines.
  • Not re-scored: recall, precision and break rate on the factorial and r2mix corpora were last scored on 46eab0d264, before the fixes below; every fix only adds redactions, so recall can rise and precision and break rate can move. The corpora are not on the host this head was built on.

Proof: not provable -- other: the change is the switch flip and its owner settings; the wiring, protection, selector and seam tests fail on the base

Security fixes on this head

All are in redaction_structured/; each has tests that fail with the fix removed and pass with it, in batch and streamed.

Finding Fix Tests
A labelled AWS secret or session-token value the detectors never read, or refused for its shape (an S3-compatible secret that is not 40 characters; a session token under 100), was dropped from pass 1 (GPT 6.1, Opus 5.5). residual_matches drops a labelled AWS match only when its value is the detectors' tag or a placeholder; an identifier mask under such a value is dropped, except the published AWS example secret. test_a_labelled_secret_the_detectors_do_not_read_is_still_removed, test_a_labelled_value_the_detectors_refuse_for_its_shape_is_removed
A signed token the detectors refuse was suppressed: a Flask session cookie (no alg), alg: none, a JWT glued after _ or - (GPT 6.1, Opus 5.5). _jose_refusal_ruled suppresses a JWT match only when its first segment decodes to no JSON object. test_a_signed_token_the_detectors_refuse_is_redacted
Identifier masks hid encoded credentials (a PNG tEXt chunk, an x-amz-id-2 value that decodes to a labelled secret) from the encoded-credential pass (GPT 6.1). _unmask_encoded_credentials drops a mask over a base64 run that decodes to a credential. test_an_image_payload_holding_an_encoded_secret_is_removed, test_a_request_id_holding_an_encoded_secret_is_removed
A hex Bearer token or a vendor token's hex body was masked as a git object name (Opus 5.5). _unmask_credential_matches drops a mask that overlaps a pass-1 match pass 1 still reports. test_a_hex_credential_shaped_like_a_git_object_name_is_removed
A bare AWS secret behind an image media type kept its image mask: padded past 64 characters, behind a typed file signature (iVBORw0KGgo, /9j/, R0lGOD), or after a real image's first 24 bytes (GPT 6.1). A bitmap mask (data_uri and image.base64) is kept only when the whole payload decodes to one complete file of its format (protection.well_formed_bitmap_head): a PNG's chunks from IHDR through IDAT to IEND, each with its CRC and nothing after; a JPEG's marker segments, each length landing on the next marker, through a frame and a scan to EOI; a GIF's blocks and sub-blocks through an image to its trailer; a WebP's RIFF chunks tiling the file around a VP8 or VP8L frame; an uncompressed BMP's pixel rows filling the file. Otherwise a payload that reads as a bare secret loses its mask. Of the repo's 1,639 images, 1,638 validate as complete (the same 1,638 before and after the structure walk); all 373 within the row's 65,536-character bound stay visible as data URIs, the same as before. Pillow's baseline, progressive and restart-marker JPEG, still and animated GIF, lossy, lossless, alpha and animated WebP, 1-, 8- and 24-bit BMP and PNG output all validate. test_a_valid_image_header_does_not_shield_an_appended_secret, test_a_forged_file_signature_does_not_make_a_secret_an_image, test_a_well_formed_bitmap_payload_stays_visible, test_a_bare_complete_image_payload_stays_visible, test_a_secret_inside_a_malformed_image_is_removed (a zero-length JPEG APP0, a GIF with no image, a WebP with no VP8 frame, a BMP whose rows do not fit, a PNG with no IDAT; fails on the signature-and-trailer check), test_gpt_review_jpeg_with_a_zero_length_app0_is_not_an_image
An SD-JWT after Authorization: Bearer kept its disclosures in clear (main's test). The rest of a Bearer value a detector hit starts is claimed with the hit (compat._bearer_gaps). test_a_bearer_value_a_detector_hit_starts_is_redacted_whole, test_agent_stream_tail.py
A 40-character AWS secret typed in place of a macOS per-user temp id (/var/folders/<b>/<key>/T) kept its Identifier mask and was shown (Opus 5.5). A macos.temp_path mask is dropped when its <id> reads as a bare secret; a genuine id is 28-30 characters of [a-z0-9_] and never does. test_a_bare_secret_typed_as_a_macos_temp_dir_id_is_removed
With the prompt-and-logs switch off, the dashboard's live log stream and its ring replay (which reach non-owner dashboard sockets and log-scoped apps) carried unredacted lines (Opus 5.5). updates_streams._stream_safe redacts every line in a fresh context before it is buffered or sent, whatever the switch says (a record built while the switch was off can reach the stream after it is back on, GPT 6.1); about 24 µs per ordinary log line; registered as a redaction sink and listed in redaction_seams.ALWAYS_REDACTED. TestLiveLogStream in test_redaction_output_seams.py, including test_a_record_made_while_the_seams_were_off_is_redacted_after_they_come_back (fails on the switch-gated sink)
A signed token the detectors refuse (alg: none, a Flask cookie) in an x-amz-id-2 or HostId slot kept its Identifier mask and was shown (GPT 6.1). _unmask_credential_matches drops a mask over a jwt match pass 1 keeps. test_a_refused_token_in_an_identifier_slot_is_redacted; 0 of 60,000 genuine-shaped x-amz-id-2/HostId values (header, JSON, XML) are redacted, the same as without the fix
A policy refresh could publish a stale seams OFF over an owner's ON (GPT 6.1). seed_pushed_switch and push_switch_moved_by_policy read and publish under redaction_switch.transaction(). TestPublishUnderTheStoreLock, TestRefresherOwnerWriteRace (fails on the unlocked handler)
An engine change published part way through one credential plan ran the legacy passes with pass 1 narrowed to what the structured detectors leave, though they never ran, so a private-key block's short last line stayed visible (GPT 6.1). The plan reads compat.use_rule_table() once and hands that answer to pass 1 (_main_plan and _pattern_pass take it; _pass1_matches reads the selector only for the stream's holds). test_redaction_engine_read_once.py: a selector that answers one engine then the other, no threads or sleeps; fails on the old plan
A 40-character AWS secret after x-amz-id-2:, "HostId":, <HostId> or SHA256: kept that row's Identifier mask, so the later passes never read it (GPT 6.1). The mask stays only on a value shaped like a real ID: canonical whole-byte base64 of at least 56 characters for x-amz-id-2 and HostId, the canonical base64 of 32 bytes for a fingerprint. Another value that reads as a bare secret loses its mask (compat._issued_id). test_a_bare_secret_in_an_aws_id_slot_is_redacted (12 cases), test_a_secret_glued_into_a_sha256_fingerprint_slot_is_redacted; test_a_genuine_shaped_aws_id_stays_readable and test_a_genuine_sha256_fingerprint_stays_readable pin real shapes
An engine PUT whose record write raised OSError left the security event log saying the engine changed, with no failure row (Opus 5.5). _put_engine writes a write_failed audit row and re-raises, as the switch writers do. test_an_engine_write_that_fails_is_audited_as_failed
With the detectors broken, the fail-closed warning reached the live log stream, whose redaction failed and logged again without end; on Windows the worker crashed (Backend Tests Windows 7). _stream_safe redacts one line per thread at a time; a line logged while it is redacting is replaced whole with the unavailable tag. test_a_line_the_redactor_logs_while_redacting_does_not_recurse (hung past 120 s before the guard)
Under `/private/var/folders///<T C 0>` the temp-dir id was read from the wrong segment, so a 40-character key in the id's place kept its Identifier mask and was shown (GPT 6.1).
The mask placeholder is a character input can carry: one typed beside a labelled AWS secret or access-key value made pass 1 read the value as a protected mask and skip it (GPT 6.1). The plan reads an input-borne placeholder as a different private-use character (compat._passes_view), so every placeholder the passes see is a mask this plan wrote. test_a_placeholder_character_in_the_input_does_not_exempt_a_labelled_value (six labels; before, after and inside the value; 12 cases fail on the old head)
With no display-verdict store (always on Windows, and before the store is enabled), an engine change did not reach chat rows the render memo already held, so legacy still showed AWS key IDs there (Opus 5.5). Every display cache key folds in display_verdicts.engine_mark(), with or without a store, so the render memo's context digest changes with the engine. test_an_engine_change_re_renders_without_the_verdict_store (fails on the old head)
An engine change to legacy and back while one display render ran left the engine mark as it was, so the legacy answer was kept under the structured key in the string, preview and render caches (GPT 6.1). compat.engine_generation() counts every engine publish; each display cache write, and the render memo, keeps an answer only when the count is the same before and after the work. test_display_engine_round_trip.py (string, preview, render memo; the battery does the round trip itself, no threads or sleeps; all three fail on the old head)
With the live log stream attached, one detector failure was logged and counted three times: the stream redacts the fail-closed warning, and that redaction failed and reported again (Backend Tests shard 7). compat._fail_closed logs and counts once per failure on a thread; a failure nested inside that report still replaces its text. test_a_failed_redaction_is_reported_once_with_the_stream_attached (fails on the old head with three counts)
A base64-encoded credential typed as the <id> of a macOS temp path kept its Identifier mask: the encoded-credential walk reads the run from /var with the id inside it, which decodes to nothing (Opus 5.5). compat._temp_id_is_credential also decodes the id on its own and drops the mask when it decodes to a credential. test_an_encoded_secret_as_the_temp_dir_id_is_removed (both prefixes; fails on the old head); the genuine-id assertions in the temp-dir tests still pass
At a tool call or approval prompt whose boundary release was empty, the run an earlier cut held was moved to the current row, so a word split by the first boundary moved across it (GPT 6.1). _cut_segment settles the held run only against a non-empty release. test_a_word_split_by_a_boundary_stays_put_through_the_next_one (both boundary kinds; fails on the old head)
French copy in the engine card used a plain space before its two colons. U+202F before both. frStyle.test.ts [changed-values] (fails on the old string with I18N_BASE_REF set)

Sensitive change review

Posted for head be654e928f21: #29423 (comment)

Manual verification

The Settings page was rendered for the screenshots above with fixture answers, not against a live gateway. Still to check by hand on a running gateway: each of the three toggles on and off, and an agent prompt and a log line with the prompt-and-logs switch off and on.

Screenshots / video

Settings > Security > Redaction, rendered from the built Settings page with fixture answers (website/scripts/capture-credential-redaction.mjs, run at the final head), attached to this PR (nothing is committed).

Before (main): the file-view switch and allowed link hosts only.

Before: Settings > Security > Redaction on main

After, defaults: the "AI model and server logs" and "Redaction engine" (Keep IDs readable) sections are added, all on.

After: defaults, every toggle on

After, everything off: each card shows its notice; with IDs no longer kept readable, the Keep IDs readable card says AWS key IDs are hidden too and that this is the earlier redaction.

After: both switches off and the legacy engine

Write failed, engine: the write is rejected; the card names the switch (Could not change 'Keep IDs readable') and says IDs are still kept readable, with a link to ask the agent.

Write failed: Keep IDs readable card

Write failed, AI model and server logs: the write is rejected; the card says credentials are still hidden from the AI model and server logs.

Write failed: AI model and server logs card

Related Issues

Part of #22552. Part of #31895 (the agent's prompt and the logs; chat and transcripts get their own design). RFC: #34048. Follow-ups: #31690 (retire the shape-only passes), #31692 (labelled credential values; the URL userinfo records gap).

Stack: #31863 (fixtures, base main) → this PR (base #31863's branch). Review #31863 first.

Pattern harvest

Not generalizable: the expectation changes follow from identifier values being kept visible by design.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

Written by the Crew Tasks Bot on behalf of mbriones98.

@mbriones98
mbriones98 requested review from a team and KiroCrewCaptain as code owners November 4, 2026 01:16
@mbriones98
mbriones98 requested review from bolichen97 and dodgeblaster and removed request for a team November 4, 2026 01:16
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/dashboard/handlers/credential_redaction.py

  • src/kiro_crew/log_redaction.py

  • src/kiro_crew/security/redaction.py

  • src/kiro_crew/security/redaction_structured/__init__.py

  • src/kiro_crew/security/redaction_structured/compat.py

  • src/kiro_crew/security/redaction_structured/identifier_values.py

  • src/kiro_crew/security/redaction_structured/protection.py

  • src/kiro_crew/security/redaction_structured/table.py

  • src/kiro_crew/security/redaction_switch.py

  • src/kiro_crew/security/text_redaction.py

  • src/kiro_crew/security_posture.py

  • src/kiro_crew/security_posture_sinks_delivery.py

  • @bolichen97 approved 2043597 with a complete reasoning template.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ human override accepted

@bolichen97 overrode this lane for 20435979907a9f10b93ec18102de843b2c8262db via /ai-review override (this commit only).

@mbriones98

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 263561e
  • What rule changed: No redaction rule changes for any user. Every change is behind the credential rule table's internal switch (redaction_rules.compat.use_rule_table()), which is off and has no config key, CLI flag or API route. With the switch on, the table's rows and detectors, StreamRedactor's hold logic, the display caches and recorded-frame key handling are corrected so the table matches the fixed-pattern pass where it should (see the PR description's table).
  • Before -> after (what was allowed/blocked, now): Switch off: unchanged for every surface; display-cache keys are byte-identical to the base, and recorded frames take the base path. Switch on: ordinary text the table wrongly redacted (agent names, max_token, secret:// references, permission values, <hidden ...> placeholders, a bare session outside a cookie) is kept; values it wrongly kept (a link token after a dot, a URL password holding ', an AWS field split across lines, a token on the line after a bare Bearer, a password equal to a scheme word) are redacted; streamed output equals the batch result for every chunking of the covered shapes.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Nothing reachable while the switch is off. With it on, a stream/batch mismatch would show a value the batch pass hides, and a slow reader would stall the gateway loop; both are covered by tests below.
  • Evidence checked (which sandbox result proves it): Three new test files turn the switch on: 104 of their cases fail on the base and pass here. The stream file compares stream and batch output across fixed chunk sizes and every two-chunk cut, and counts newline-index work to show name/value reading is linear (8,000 unquoted pairs: 0.21 s, from 51.75 s before the last fix). Local GPT 6.1 and Opus 5.5 reviews of this head report no blocking findings after the repairs.
  • No regression:
    • Tools still work (which tools/commands were run, result): No tool or command handling changed. Recorded frames keep the base output with the switch off (88 key and ancestor combinations compared against the base).
    • Backward compatible (old config / old data / old callers still work, how checked): No config key, schema or stored data added. With the switch off the engine mark is empty, so stored display verdicts and cache keys are unchanged.
    • Existing tests (which suites passed on this head): Switch off, 979 passed across the redaction, stream-parity, compat, display-battery, display, chat-utils, frame-record, security-liveness and new suites; switch on, 898 passed. black, isort, flake8, mypy, the file line cap and Semgrep 1.78.0 with CI's packs pass.
  • How to undo: Revert this single commit. No config, schema or data migration is involved.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 20435979907a9f10b93ec18102de843b2c8262db — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the default flip ships with its rollback selector as the RFC requires, the seams scope fails closed when unpublished, and screenshots of the new toggles are attached.

Notes

  • The "AI model and server logs" switch (seams_enabled, part of Add one owner setting that turns credential redaction off on every path #31895) is a separate change from the engine flip, and the RFC only asked for the engine selector to ship with it. Splitting it out would make each change easier to revert on its own.
  • "compat.use_rule_table() reads the engine selector once per process … Other processes use a change when they next start." So choosing legacy does not reach MCP servers, cron or CLI processes until they restart. The rollback settings text should say so, or those processes should re-read the setting.
  • The description is truncated before ## Backwards compatibility, so the reader list was not checked against it.

[DESIGN-REVIEWED] 2043597

Description read

[DESCRIPTION-READ] fe0899e73efa55f4e891b4da0f20d9a2e847d9c61ffbb0ba1bd7abc28a53223b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ human override accepted

@bolichen97 overrode this lane for 20435979907a9f10b93ec18102de843b2c8262db via /ai-review override (this commit only).

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — 🔴 changes requested (blocking)

GPT 6.1 found at least one blocking issue that must be resolved before merging 20435979907a9f10b93ec18102de843b2c8262db. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/security/redaction_structured/compat.py:580 -- Valid bitmap payloads shield literal credentials
"and not well_formed_bitmap_head(text, s, e)"
Trigger: An agent returns a valid 10×1 BMP data URI containing a 54-byte header, the base64-decoded bytes of a 40-character AWS secret, and two padding bytes. Encoding preserves the secret verbatim; display_battery → redact_credentials → _unmask_encoded_credentials retains the mask because the bitmap is valid.
Outcome: Dashboard output contains the original credential unchanged. Streamed redaction also exposes it, while the legacy engine removes it.
Anchor: backend-security-controls
Fix: Apply the plaintext bare-secret check to image payloads regardless of bitmap validity, dropping matching masks before the remaining credential passes.
Sites: Valid PNG private-chunk payloads reproduce this through both data_uri and image.base64 masks; batch redaction, dashboard display, and StreamRedactor share the bypass.
[BLOCK-MERGE] 2043597
[GPT-REVIEWED] 2043597

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 (fenced): UNBOUNDED, because it involves a literal credential reaching the dashboard or stream output.

  • What I confirmed: at compat.py:577-581, an image mask is dropped only if the payload is not a well-formed bitmap. At protection.py:125-126, a BMP counts as well-formed when its DIB size is valid and its rows fill the file; the pixel bytes are not checked, and the docstring at protection.py:100-101 says so. A 54-byte header lines up with base64's 3-byte groups, so a 40-character secret placed in the pixel data comes out verbatim in the encoded payload. The decode walk at compat.py:589-598 decodes the whole run, which is binary, so it does not find the secret and the mask stays.
  • Why not FLAG: the trigger is one crafted agent output. It needs no conflicting timing, no input the system's own writer cannot produce, and no legacy artifact, so the rarity bar for FLAG is not met.

[ADJUDICATION] 2043597 total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] 2043597

[ADJUDICATION-FENCED] 2043597 fenced=1 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/security/redaction_structured/compat.py:580 -- A single crafted agent output reaches this: a valid BMP with a 3-byte-aligned secret in its pixel data keeps its mask, because only the bitmap's structure is checked (protection.py:100-101,125-126), so the credential is shown unredacted.
[GPT-ADJUDICATED-FENCED] 2043597

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 20435979907a9f10b93ec18102de843b2c8262db false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — 🔴 changes requested (blocking)

Reviewed 20435979907a9f10b93ec18102de843b2c8262db — this comment is updated in place on each push.

BLOCKING: an engine change made while a dashboard tab is offline never clears that tab's cached file bodies. The reconnect heal only compares enabled, so the tab keeps showing files redacted under the engine the owner just turned off.

BLOCKING — website/src/hooks/websocket/serverState.ts:61
const moved = before !== undefined && after.enabled !== before.enabled
Trigger: The owner changes the engine (PUT {"engine": "legacy"} → _put_engine → _push_to_owners(purge=True) → credential_redaction_changed) from tab A while tab B's socket is down, for example while the laptop sleeps. The frame is not replayed. When tab B reconnects, healRedactionSwitchAfterReconnect re-reads the record, sees enabled unchanged, and skips purgeDocumentBodiesForRedactionChange. handleCredentialRedactionChanged also stores only {enabled, changed_at}, so before never holds an engine to compare against.
Outcome: Tab B keeps its open file bodies and diff tabs as they were redacted under the old engine. For example, AWS key IDs and commit hashes stay readable after the owner chose "legacy" (the "Keep IDs readable" toggle turned off). A connected tab gets these bodies purged; tab B keeps them until a reload.
Anchor: defect
Fix: Seed engine from the frame payload in handleCredentialRedactionChanged, and in the heal treat after.engine !== before.engine (or a moved engine_changed_at) as moved.

[BLOCK-MERGE] 2043597
[OPUS-REVIEWED] 2043597

Verdict parsed from the review's SHA-scoped output markers for commit 20435979907a9f10b93ec18102de843b2c8262db.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 20435979907a9f10b93ec18102de843b2c8262db false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does 20435979907a9f10b93ec18102de843b2c8262db newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 14 classified, 0 newly refused, 14 still allowed.
  • ubuntu-latest (platform posix): 14 classified, 0 newly refused, 14 still allowed.
  • windows-latest (platform windows): 7 classified, 0 newly refused, 7 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: The PR doesn't change anything in the shell deny composite. hooks.py SHELL_DENY_TIERS, the sensitive-bash tier, exfil_bash.audit_bash_exfiltration and the deny_gate rule catalog are all untouched, and none of them gets its verdict from redact_credentials (deny_gate uses redact_and_truncate only for audit metadata). So no shell row should flip. The real tightening is in output redaction and in refusal gates that compare redact(x) with x. The classifier can't adjudicate either of those; both are named under UNADJUDICATED.

TIGHTENING: The structured credential engine is now on by default on every redaction path, including every admission gate that refuses when redact(x) != x. Hunk in security/redaction_structured/__init__.py: -:mod:\.compat` is the seam the credential plan reads; it is off by default./+... it is on by default.. That brings new claims with it, such as compat._bearer_gaps("The rest of anAuthorization: Bearervalue a detector hit starts (an SD-JWT's disclosures) is claimed with the hit") andcompat._token_param_gaps. The PUT body validator is also stricter: exactly one key, a JSON boolean, or a 400. The threat it refuses is a credential whose value the legacy passes only partly removed (the trailing part of a Bearer value or a token=` value) reaching an output or a third party.

ADJUDICATED: 20 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: Candidates cover posix, windows and any.

  • posix: pytest runs, ~/.kiro/crew/gateway.log reads and greps, a seam-call-site census, git log and diff, aws sts/sns/s3api reads, ssh-keygen -lf, and a macOS /var/folders/*/*/T/ listing.
  • windows: .venv\Scripts\python.exe, %USERPROFILE% and $env:USERPROFILE paths, findstr with backslash separators, and the aws/ssh-keygen rows.
  • any: one gh pr diff row.

The diff's own platform assumptions are the macOS temp-path shape (/var/folders/<xx>/<id>/T/), the gateway log under the config directory, and the Windows render-memo path its cache comment names. The rows above probe the first two. The render memo and the macOS temp-path Identifier rule work on rendered text, not on shell commands, so no shell row can probe them.

UNADJUDICATED:

  • Refusal gates comparing redact(x) with x under the now-default structured engine. These are _gate_upload_file, a non-owner's outbox download, and the .crewmate reader's needs_redaction. Call path: file or bundle text → redact_credentials → _credential_redaction_plan → compat.credential_plan_via_table (now default) → _bearer_gaps / _token_param_gaps / residual pass 1 → a non-empty span refuses. Inputs I would have submitted:
    • a Slack upload of a markdown runbook containing an Authorization: Bearer header example followed by prose on the same line;
    • a file containing an approval link whose ?token= value runs into a URL fragment;
    • a .crewmate bundle whose cron command includes an S3-compatible secret label with a non-40-character placeholder value;
    • a data URI with an svg+xml payload, which the hunk says is never masked;
    • a bitmap whose base64 does not decode to one complete file.
  • The PUT validator in dashboard/handlers/credential_redaction.py. Call path: PUT /api/security/credential-redaction → keys = set(body) & _SWITCH_KEYS → anything other than exactly one key is a 400. Possible narrowing: a client that sends {"enabled": true, "seams_enabled": true} in one body, which used to be accepted on enabled, is now refused. _put_engine also refuses {"engine": ..., "enabled": ...}. The shipped UI sends one key per request, and I could not confirm that any caller sends two.
  • acp/_frame_record._scrub_dict_key. This is a loosening under the rule table (field-name keys under Authorization-type ancestors are skipped), so it can't newly refuse anything and no verdict is needed.

[SCOPE-REVIEWED] 2043597

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 4, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold at 263561e.

This PR targets mdbrio/redaction-cutover-v2 (stacked on #29369), not main, so it cannot be approved for merge here. It also changes what the credential redactor removes, which the user's rules reserve for a human reviewer. That holds even though the new behaviour sits behind use_rule_table(), because text_redaction.py swaps in hand-written switch-off hooks for code that runs today (text_redaction.py#L105). My reading of those hooks found no switch-off difference, and CI is green.

Before approval:

The Design and First Principles notes about _NO_TABLE duplicating stream.py's off gate, and about cache flushing having no runtime producer yet, are cleanup suggestions and do not block.

— crew-pr-reviewer (r145)

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[operator: bolichen97] Approving at head 263561e.

Reviewed with a focus on the switch-off path:

  • display_verdicts.engine_mark() is empty with the switch off, so every display cache key and stored verdict tag is the key it was before. The new engine-change checks compare two empty marks and never fire.
  • _frame_record._scrub_dict_key only adds a branch behind compat.use_rule_table().
  • In text_redaction.StreamRedactor, the _NO_TABLE stubs match what the base branch's table stream hooks already return with the switch off (read_buffer empty, redact_window = redact(text), discard regexes and carry unchanged for legacy kinds). A discard the table opened (table-* kinds) still uses the table module after the switch moves.

Non-blocking suggestions:

  • _block_tables is keyed by id(table) with an identity check; that is safe, but it keeps a reference to every table ever loaded. A single-entry cache would be enough if tables are only reloaded on config change.

This PR targets a stacked base branch, so I have not armed auto-merge.

AI-assisted, approved by bolichen97

@mbriones98
mbriones98 marked this pull request as draft November 4, 2026 17:07
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 4, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 5, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes at 263561e. This replaces my approval from 11-04, which came before our review rule on regex. Sorry for the change of course.

  1. This PR edits the old redaction_rules/ table. #29369 now deletes that table and moves detection to redaction_structured/, so this change no longer fits the stack.
  2. It adds regex in product code: new patterns in redaction_rules/rules.yaml, redaction_rules/detectors.py, redaction_rules/stream.py and redaction_rules/compat.py. We can no longer take new regex in product code.

@mbriones98, is this PR superseded by #29369? If so, would you close it? If something here still needs to land, could you rebuild it on #29369's structured table instead?

@mbriones98

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 1fb944e reissue: Same ruling as the earlier record #29423 (comment) on 8215d7a. bolichen97 signed the structured-engine default on RFC 34048; the RFC status line on main still reads draft. The only change since is the rebase onto main after #31863 merged: the patch touches the same 131 files with the same line counts as be654e9, except test/test_security_gate_liveness.py, which changes one more line.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@mbriones98 recorded a reissue release (class reissue) for the first-principles AI finding on 1fb944eed370961204704f03b03909fbabac92ac.

Same ruling as the earlier record #29423 (comment) on 8215d7a. bolichen97 signed the structured-engine default on RFC 34048; the RFC status line on main still reads draft. The only change since is the rebase onto main after #31863 merged: the patch touches the same 131 files with the same line counts as be654e9, except test/test_security_gate_liveness.py, which changes one more line.

This decision applies only to this commit. A new push requires a new judgment.

@KiroCrewCaptain

KiroCrewCaptain commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider thinks the change is probably fine but estimates a 30% chance that this change introduces a defect, above the 12% limit for approving on its own.

Turns on the structured credential detectors by default and adds two owner settings under Settings > Security > Redaction: an engine selector (structured or legacy) and a prompt-and-logs switch. I checked the state, lifecycle and trust boundaries. The record has a single owner-gated writer; each setter keeps the other fields under the store lock and reads fail toward redacting. Engine changes are audit-or-refuse and published under the lock. The seams verdict is published only by the gateway, so other processes keep redacting, and a policy refresh re-publishes it. Refusal gates, the exfil-URL pass and vault literals stay on, and the live log stream redacts every line whatever the switch says. Governance has its own capability row, default off. Stream boundary releases keep pass-1 matches and undecided units held. Tests are extensive. What remains is process (First Principles override still needed for this head, sensitive-change template, pending CI) and accepted, deliberately-crafted false-negative gaps that the description and docs acknowledge. None is a concrete defect I would block on.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit e7d5067c00 · 8331 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@bolichen97

Copy link
Copy Markdown
Contributor

State of this PR after the conflict resolution and review fixes (head 95d9dfd)

The branch is one commit on main 12a03ac (author Matt Briones kept, rebased, no merge commits). Content is the PR's own: structured detectors on, engine selector, prompt-and-logs switch. Nothing was disabled, skipped or weakened; the only test-side change is one raised line budget, with its usual note.

What was fixed, per finding

  • Backend Tests (3.12, 6), Backend Tests (Windows) (6), Coverage Gate (first head): not this PR. test_search_docs "how do I run a task in the background" failed on the old base 92d3216 as well (reproduced); main fixed it in fix(guide): rank a page holding every rare query word above a name match #37063, which the rebase picked up. Coverage Gate only fails closed when a backend shard fails.
  • GPT 6.1 Review, round 1 (real): a bitmap payload with its base64 padding left off kept its mask over a token in the PNG tEXt chunk. The encoded-credential checks now pad the run the way the bitmap check does. Regression cases in test_redaction_structured_wiring.py (8 failed before the fix).
  • Opus 5.5 Review, round 1 (real): the prompt-and-logs switch shared the Files-view governance row, so an administrator's Files-view opt-in also let the owner turn redaction off for the agent prompt and the logs. It now has its own row, capabilities.credential_redaction_seams_off (owner_may_disable_seams), each switch is checked against its own row in read_state and in the PUT handler, and governance.md and security.md say so. Regression tests in test_credential_redaction_switch.py.
  • GPT 6.1 Review, round 2 (real): the stream's cut walk (_outside_pass1_matches) rescanned after every move, which is quadratic in chained matches (measured 1.2 s for 6,000). It is now one pass back over the matches. Tests count comparisons instead of reading a clock.
  • Opus 5.5 Review, round 2 (real): redact_log_via_context was opened as the logs seam, so with the seams switch off its callers' clips ([:120], [:200]) could cut a raw credential into a fragment that later passes cannot match. It is no longer a seam; the logs seam is the log record filter alone. Regression tests fail on the old code.
  • GPT 6.1 Review, round 3 and Flake Pattern Ratchet: a wall-clock bound in one of my new tests. Replaced by a deterministic count.
  • test_security_gate_liveness line budget: raised by 31 for the new governance row and the padding helper, on top of main's own raise.

Checks on this head

Green: GPT 6.1 Review, Opus 5.5 Review, Design Review, UX Review, Security scope review, lint and type checks, ratchets, and the rest of the matrix except as listed below.

Red or pending, and why:

  • First Principles Review: BLOCK on the "structured detectors on by default" premise. Earlier heads were released by a human override from the PR author (mbriones98); an override applies to one commit only, so it must be recorded again on the final head. Needs a human.
  • Sensitive Change Review: needs the PR author's filled "Sensitive change review" template as a PR comment naming the final head SHA (or a reviewer's Approve review carrying it). Needs a human.
  • Review decision is still CHANGES_REQUESTED (bolichen97). Needs a human.
  • Backend Tests (3.12, 2) and Coverage Gate: two tests in test/tracing/test_tracing_otlp.py fail in that shard (an extra kirocrew.step span; a count of 5 against 4), twice. They are not in code this PR touches, they come from the tracing change that landed on main just before the last rebase (feat(tracing): steps inside a turn: derived model calls, compaction, approvals, steers (#1643) #37005), and they pass here when run alone on both main and this branch (and after our redaction tests). I have not proven that main fails the same way in that shard, so please treat this as unconfirmed until someone checks main.
  • Backend Tests (Windows) (8) failed once on test_dashboard_save_tool_older_rule::test_two_first_saves_at_once_bind_one_package (a race; passes alone) and was re-run.
  • macOS shard 2: a git checkout network reset. Publish readiness signal: "unsupported event". Neither is about the code.
  • A burst of runs failed on GitHub's installation API rate limit (Await Fast Gate, Dependency License Gate, CodeQL, PR Policy); those were re-run and are green.

What the maintainer should do

Every push resets the overrides and the template, so I am making no further pushes. On this head: record the First Principles override, have mbriones98 post the sensitive-change template for 95d9dfdc3969a1cc898ae3b675a170b0263abace, re-run the red backend shard once main's tracing tests are known good, and re-review. If anything must change in the code first, do the overrides after the last push.

The credential plan now reads the structured detectors first by default.
The owner can choose main's regex passes instead (the engine selector
below), and can turn credential redaction off for the agent's prompt
and the logs (the output-seams switch below).

Plan:
- Secret hits are tagged; Identifier values (AWS key IDs, documentation
  example keys, git object names, bitmap image data-URI payloads,
  commit permalink SHAs, request ids, the macOS per-user temp
  directory) are written as a private-use placeholder while main's
  passes read the text, then restored. An SVG or other non-bitmap
  data-URI payload is read by main's passes.
- ARNs, public OAuth parameter values and placeholders are reported
  but not masked, so main's passes still judge them.
- Pass 1 skips the branches the detectors cover (private keys, JWTs,
  the AWS secret and session-token labels), the bare AWS key ID and a
  labelled AccessKeyId whose value is protected. A JWT match is
  dropped only when its first segment decodes to no JSON object; a
  signed token the detectors refuse (no alg, as in a Flask session
  cookie, alg none, glued on its left) is still removed by pass 1. A
  labelled AWS secret or session-token match is dropped only when its
  value is the detectors' tag or a placeholder, so a value they never
  read, or refuse for its shape (an S3-compatible secret that is not
  40 characters, a short session token), is still removed. Every other
  branch, including the URL userinfo branch, and passes 2-4 run
  unchanged.
- An Identifier mask is dropped before the passes read the text when
  the base64 run under it decodes to a credential, when it overlaps a
  pass-1 credential match pass 1 still reports (a hex Bearer token, a
  64-character hex value after an AWS secret label), or when it is a
  bitmap payload whose bytes do not open with that format's fixed
  header (a PNG IHDR, a JPEG marker, a GIF version, WEBP, a BMP header
  size) and that reads as a bare secret, however long it is padded or
  whatever file signature is typed in front of it.
- The rest of an Authorization: Bearer value a detector hit starts (an
  SD-JWT's disclosures) is claimed with the hit, as main's single match
  claims the whole value.
- A token= value a detector hit touches is claimed before the passes
  read the text, its uncovered parts redacted with the token
  parameter pass's own tag, warning and record.
- A JWT or JWE may follow a sentence's dot; the decoded header check
  decides.
- Pass-1 warnings give the length on the original text.

Stream:
- StreamRedactor's detector cut moves back to a word start and out of
  any pass-1 match, and the fixed tail hold ends at the last blank.
  Past the holdback cap the floor reads the detectors' own cut, so a
  long unbroken run is released and the buffer stays bounded.
- release_at_boundary() commits what only the detectors' tail hold
  kept back; the dashboard calls it at a tool call or prompt cut, and
  Slack at a tool card, a wait and the turn's end. The released text
  is redacted like any committed window.
- Dropped AWS runs are kept in the look-behind; flush() no longer
  clears the discard counter.

Engine selector:
- compat.use_rule_table() reads the owner's engine selector: "engine"
  in the credential-redaction keystone, structured (default) or legacy
  (main's regex passes alone). A missing field, any other value or an
  unreadable record selects structured. It is read once per process;
  the owner-gated PUT /api/security/credential-redaction with
  {"engine": ...} audits the change, writes it without moving the
  file-view switch, and republishes it to the gateway at once.
- Settings -> Security -> Redaction shows it as its own "Redaction
  engine" section, apart from the file-view switch.

Owner switches:
- The engine selector (above) and a second owner switch,
  seams_enabled (default on), share the credential-redaction keystone
  with the file-view switch; each writer keeps the others' fields, and
  each change has its own time. seams_enabled turns off the credential
  pass at the output seams listed in redaction_seams.py, and only
  there: the conversation replayed into the agent's prompt, and the
  shape passes over log lines. The exfiltration-URL pass, the literal
  vault-secret pass over log lines, every request-blocking gate and
  every refusal gate stay on. Chat, transcripts, the live chat stream
  and channel egress are not seams. A record without the field reads
  it on, so an earlier file-view OFF keeps its meaning.
- Every setter returns the switches in force under the governance
  rule, so a file-view write on a governed host never publishes a
  stored seams OFF. The engine choice is published under the same
  store lock as its write, so overlapping PUTs publish in write order.
  A seams change, by the owner or by a policy refresh, is pushed to
  owner tabs as credential_redaction_settings_changed, which re-reads
  the settings and purges no file body.
- Settings -> Security -> Redaction shows three toggles under three
  headings: the file-view switch, the prompt-and-logs switch, and the
  redaction engine.

Display:
- display_battery.RULE_SOURCES lists every redaction_structured file
  and pass_compose.py, so a detector change gives stored verdicts a
  new version. The process caches and stored verdicts carry the
  engine's mark. A recorded frame keeps a field-name key under an
  authorization header legible, decided by a plain name check.

Tests: the fixture swaps to synthetic credentials are in the base
branch (#31863). Here, tests whose subject is a value the design keeps
visible expect it visible. Tests that pin main's pass-1 internals run
with the switch off and say why. Three Slack golden digests follow the
stream's tail hold; the final text is unchanged. An unsigned image
data-URI payload and a link after an AWS secret label are judged as
with the switch off, and their tests expect that again. Files whose
expectations are the structured engine's output pin it with an autouse
fixture, so the suite also passes with the legacy engine selected. The
security package's line budget is raised by 201, re-measured.

Not generalizable: the fixture swaps follow from Identifier values
being kept visible by design.

Review fixes:
- The output-seams switch answers to its own governance row,
  capabilities.credential_redaction_seams_off. The Files-view row
  (capabilities.credential_redaction_off) no longer lets the owner turn
  redaction off for the agent's prompt and the logs; each switch is read
  against its own row in read_state and in the PUT handler.
- A bitmap payload with its base64 padding left off no longer keeps its
  mask over a credential in its metadata: the encoded-credential checks
  pad the run as the bitmap check does.

- The stream's cut walk (_outside_pass1_matches) is one pass back over
  the pass-1 matches instead of a rescan after each move, which was
  quadratic in the matches a model's output can chain.

- redact_log_via_context is not an output seam: its callers clip the
  result and send it to sinks that stay redacted, so it redacts before
  the clip with the seams switch off too. The logs seam is the log
  record filter alone.

- The "Keep IDs readable" switch reads the same in its label, its failed-save
  error and its help (a sample ID, and that Off brings back the earlier
  redaction); the seams switch has its own failed-save line, so the page's
  three redaction switches no longer share "Redaction is still on."

- A labelled AWS secret or key ID that starts with a protected documentation
  example and has more glued on (a '.' and a real secret) is no longer
  ruled on; only a protected run alone, or followed by a value-ending
  character such as '&', is.

- The prompt-and-logs switch's help starts with what turning it off is for,
  and the Keep IDs readable notice says it is the earlier redaction, not a
  lower protection. The capture script gains the engine and seams
  save-failed scenes.

- The help for the prompt-and-logs switch and its off notice say whose logs
  they mean (the app's own log files on this computer); the Keep IDs
  readable help says what Off does (key IDs and commit hashes hidden like
  secrets) and that an example key ID is not a secret. French spacing
  before double punctuation uses U+202F.

- A labelled AWS secret or session token a detector hit covers only the start
  of (the first base64 run, then %2F and more) is claimed whole, as a Bearer
  value is, so its tail no longer survives; and a value a protected example
  only starts is no longer ruled on.
- Bengali and Hindi help text addresses the reader as tum, as those style
  guides require; the listing import ceilings for mcp-cron and mcp-mate are
  raised by one with the reason.

- A presigned URL's X-Amz-Signature is claimed like a detector hit (its
  key ID stays readable), so the link no longer stays usable: the
  signature would otherwise read as a git object name and be kept.

- Finding a presigned URL's signature against the hits and masks uses
  sorted sweeps, so a listing of thousands of URLs stays linear.

- A presigned URL's signature inside a credential value (token=,
  aws_secret_access_key=, SecretAccessKey, SessionToken,
  aws_session_token) no longer cuts the enclosing value short, in batch
  or in the stream.
- A base64 run under an Identifier mask whose decode holds a structured
  Secret (a Google API key in a PNG's text chunk) is claimed whole.
- Known limits, accepted (security.md): a complete bitmap keeps its mask
  when its base64 text spells a bare secret, and a 38-character name
  directly under a macOS temp directory stays visible.
@bolichen97

Copy link
Copy Markdown
Contributor

/ai-review override first-principles 2043597 reissue: Same ruling as #29423 (comment) (the premise, structured detectors on by default, was signed by bolichen97 on RFC 34048). Since 1fb944e the PR added fixes for GPT and Opus review findings, UX copy and rebases onto main; the premise is unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 recorded a reissue release (class reissue) for the first-principles AI finding on 20435979907a9f10b93ec18102de843b2c8262db.

Same ruling as #29423 (comment) (the premise, structured detectors on by default, was signed by bolichen97 on RFC 34048). Since 1fb944e the PR added fixes for GPT and Opus review findings, UX copy and rebases onto main; the premise is unchanged.

This decision applies only to this commit. A new push requires a new judgment.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sensitive change review

  • Head commit: 2043597
  • What rule changed: Credential redaction runs the structured detectors by default, chosen by the owner's engine field (structured or legacy) in the keystone credential_redaction.json. Their Secret hits are removed and their Identifier values are protected from main's regex passes, which still run after them. A second owner switch, seams_enabled (default on), turns the credential pass off for the prompt replay and the log record filter only; it has its own governance row (capabilities.credential_redaction_seams_off, owner_may_disable_seams), so a Files-view opt-in no longer lets the owner switch the seams off. redact_log_via_context is no longer a seam: it redacts before the callers' clips whatever the switch says. Chat, transcripts, the live stream and channel egress stay redacted either way.
  • Before -> after (what was allowed/blocked, now): Before, every AWS access key ID, git object name, request id, bitmap data-URI payload, commit permalink SHA and macOS per-user temp directory was removed. After, those stay visible, and secrets are still removed by the detectors or by pass 1. Findings from the review lanes since the first head are fixed, each with a regression test that fails without the fix: a bitmap with its base64 padding left off, a bitmap whose decoded text chunk holds a structured secret, a signature inside an enclosing credential value, a labelled secret or session token that kept its tail, a presigned URL signature, the stream's quadratic cut walk, and the quadratic URL scans.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Leak of a secret the detectors classify as an identifier or in a shape no pass reads. Two limits are accepted and recorded in docs/system-specs/modules/security.md and in the PR body ("Known gaps, accepted"): a secret whose plain text is spelled by the base64 of an image the exemption keeps, and a 38-character name after the macOS temp directory's T/, which cannot be told from a file name. A detector error replaces the whole text with the fail-closed placeholder. A switch or engine change that cannot write its SEL record is refused (503).
  • Evidence checked (which sandbox result proves it): A matrix run on a DevSpace at head f41b929 (258 items, 142 synthetic plus the repo's 116 cases, through 11 surfaces and all 8 engine x seams x Files-view combinations, with a real loopback gateway writing the settings through the real PUT handler) found no item that stays visible under the structured engine but is redacted under legacy other than identifiers, documentation examples, placeholders and a clean bitmap; the unit matrix was re-run on the next head with 0 differing cells. CI on e7d5067, this head before its rebase, was green on every check except the two human gates, including GPT 6.1, Opus 5.5, UX, Design, Security Scope and all backend, Windows and macOS shards. This head is that commit rebased onto main 46f62de: the only conflicts were the generated ui-index (regenerated) and the security line budget (now one row, test/security_line_budget/structured-detectors-on.txt, +230). After the rebase, 350 tests of the ui-index, line-budget, settings-registry, wiring and switch files pass locally; the full matrix runs on this head.
  • No regression:
    • Tools still work (which tools/commands were run, result): No tool or command handling changed; the dashboard and Slack stream paths are covered by their redaction tests and by the gateway run in the matrix.
    • Backward compatible (old config / old data / old callers still work, how checked): The record gains optional fields; a record without them reads as both switches on and the structured engine. PUT /api/security/credential-redaction still accepts {"enabled": bool} and returns 400 on every malformed body (matrix run). The settings survived 8 of 8 gateway restarts.
    • Existing tests (which suites passed on this head): see Evidence; the full CI matrix on the pre-rebase head, and the touched gates after the rebase.
  • How to undo: Owner: Settings > Security > Redaction, turn off "Keep IDs readable" (main's regex passes alone, no release). Code: revert this PR. No schema or data migration is involved.

Written by Claude Code (AI-assisted) on behalf of bolichen97, who asked for this review to be recorded on the final head.

@bolichen97

Copy link
Copy Markdown
Contributor

Maintainer decision on the UX Review BLOCK for the "AI model and server logs" help text (recorded by Claude Code, AI-assisted, on behalf of bolichen97, who ordered it).

The finding is about wording: the help line packs the purpose, two effects and three untouched surfaces into one block. It is a copy suggestion, not a defect in what the switch does; the security behaviour was reviewed by the GPT, Opus and Security Scope lanes and measured in the redaction matrix. The Settings > Security > Redaction page is being replaced in the maintainer's follow-up PR by a single on/off toggle with short copy, so rewording these strings again here would only be thrown away. Accepted for this PR; the notes (split switches, grey versus yellow notice, shorter "Keep IDs readable" text) go to the follow-up PR.

@bolichen97

Copy link
Copy Markdown
Contributor

/ai-review override ux 2043597 owner-decision: Copy-only UX note on the AI-model-and-logs help text, accepted by the maintainer; the page is replaced by a single toggle in the follow-up PR. Decision: #29423 (comment)

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 recorded a owner-decision release (class owner-decision) for the ux AI finding on 20435979907a9f10b93ec18102de843b2c8262db.

Copy-only UX note on the AI-model-and-logs help text, accepted by the maintainer; the page is replaced by a single toggle in the follow-up PR. Decision: #29423 (comment)

This decision applies only to this commit. A new push requires a new judgment.

@bolichen97

Copy link
Copy Markdown
Contributor

Merging with admin rights, on the maintainer's order (recorded by Claude Code, AI-assisted, on behalf of bolichen97).

Why not wait for the rest of the matrix: this head is the commit that was fully green on e7d5067 (every check except the two human gates; GPT 6.1, Opus 5.5, UX, Design, Security Scope and every backend, Windows and macOS shard passed), rebased onto main 46f62de. The only conflicts were the generated ui-index (regenerated) and the security line budget (now the row test/security_line_budget/structured-detectors-on.txt, +230). On this head Await Fast Gate and the lint, type and gate checks pass; the backend shards were still running. Main changes about 300 files an hour, so waiting an hour for the shards would mean another conflicting rebase and another reset of the overrides and the review. Main's CI on the merge commit will run the full matrix.

Recorded on this head: First Principles reissue override, the Sensitive change review as an approving review, and an owner-decision override for a copy-only UX note (the Settings page is replaced by a single toggle in the follow-up PR). Accepted limits are listed under "Known gaps, accepted" in the PR body and in security.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants