Repository navigation
fix: keep the key when redacting an authorization bearer header - #32367
Conversation
Sensitive Change ReviewSensitive files changed:
How to clear this check:
Only the author's comment counts this way. A reviewer can instead |
d3af236 to
2f5c14e
Compare
|
Rebased: main moved in test/test_security_gate_liveness.py and docs/system-specs/modules/security.md since this head's CI ran (f58c3cf -> 26d3928); CI re-running. The new head also fixes the red [operator: bolichen97] |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Issue #13873 is the evidence for the fix. The stream hold exists only because of the kept prefix, and the new parametrized case fails without it. What this change shipsInventory (5 items) — 4 justifiedIntent: a config, log or chat text holding an
[FIRST-PRINCIPLES-REVIEWED] ac8ec52 Description read[DESCRIPTION-READ] c18c8ff2d83b1dd2622010cc4e1c6a5f725cae3eb04ab463da40c7ba99be1647 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change only shrinks the redacted span to the token itself. Every derived alternation compiles its own copy of Notes
[DESIGN-REVIEWED] ac8ec52 Description read[DESCRIPTION-READ] 12936477b0a6b74fb3f3594ca8a13cd4b948f45d5208423f60697e824984f719 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: The shell deny composite cannot newly refuse anything because of this change. Adding the named group TIGHTENING: The subagent output SSE tail now holds more of its output -- it holds back the whole poll when a credential match crosses the line cut, in addition to the existing "apart != together" test: ADJUDICATED: 11 candidates, 0 confirmed newly-refused (the harness gave no confirmed rows) PLATFORM COVERAGE: posix and windows. The diff adds no assumptions about paths, interpreters or environment variables, since the redaction regex and the SSE hold work on text only. Even so, the Bearer-header curl call, the source search and the targeted pytest runs were each written in both spellings: UNADJUDICATED: Guard: the hold predicate in [SCOPE-REVIEWED] ac8ec52 |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
[operator: bolichen97] The remaining CI reds on 2f5c14e are main's, not this change's, so this PR waits instead of pushing:
Once those are fixed on main, this branch rebases onto main and CI re-runs. It still needs a human review and a Sensitive Change sign-off. |
2f5c14e to
e97b354
Compare
|
[operator: bolichen97] Rebased onto main 0b75594 to clear the conflict; head e97b354. The only CI reds on this head are not this change's:
Local GPT and Opus reviews of the rebased commit found nothing. AI-assisted, posted on behalf of bolichen97 |
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
e97b354 to
ac8ec52
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approving. On head ac8ec52 the review confirmed these behaviours. The Bearer branch gains one named bearer_prefix group, and pass 1 starts its span where that group ends. Every other branch has no such group, so it falls back to m.start() and is redacted whole, as before. Raw, JSON, curl, JWT and newline Bearer inputs all lose the token, and the JSON form now parses. Mutation runs show the tests catch this change: reverting the span start turns 35 tests red, and deleting the new stream hold in core.py turns the SD-JWT case red.
Security: the token is still removed in every shape tested, and a second redaction pass changes nothing. Rules are still classified from the whole match, so no tier moves. The new capture group has no consumer that uses group indices, findall or split. Keeping the label opened one new leak: in the dashboard SSE stream, an SD-JWT disclosure could stream on the next poll. The new hold in core.py closes it. The hold is bounded by _AGENT_STREAM_HOLD_MAX and never fires on the last poll. The other match consumers still use the whole match, which removes more text than needed but exposes no token.
Done well: the fix stays at the root (one group, one span-start line), it follows the recorded "label kept, value replaced" decision, and the prefilter oracle was split honestly instead of loosened.
Optional:
- This PR and #32401 break
test_a_credential_spanning_a_key_and_its_value_still_sends_valid_json(test/test_cron_script_sessions.py:350) once both are on main, whichever lands first. Itspytest.raises(json.JSONDecodeError)step stops holding, and merge-tree will not show it. Name the landing order in one body. The PR that lands second should rewrite that precondition. - A separate StreamRedactor leak exists on main and is unchanged here: if a chunk ends on the closing quote of
"Bearer <token>", the token streams unredacted. It deserves its own issue with that reproduction. - When this lands, #21488 can be closed as superseded, with credit to dodgeblaster.
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
The bearer branch of the credential patterns matched the header name,
separator, quote and scheme together with the token, and pass 1
replaced the whole match. A JSON document holding
{"Authorization": "Bearer <tok>"} collapsed to a bare tag, so the
dashboard Files view reported a valid file as Invalid JSON.
The branch now names its non-secret prefix in a bearer_prefix group
and pass 1 starts the redacted span at its end, so only the token is
replaced and the document still parses.
The sub-agent output stream sends a poll up to its last line only
when both halves redact the same as the whole. With the label kept, a
token cut from it on the next line could pass that check and lose its
anchor, so the stream also holds while a credential match spans the
cut.
Builds on #21488 by @dodgeblaster.
Closes #13873
Co-authored-by: Gary Jennings <garysjennings@gmail.com>
(cherry picked from commit 3a5c452)
Original author: @bolichen97 (picked by hand: conflict only in docs/system-specs/modules/security.md test/test_security_gate_liveness.py)
Since kirodotdev#32367 the redactor keeps the Authorization header name and redacts only the token, so the split-bearer expectations now include the header. The Codex native card tests from kirodotdev#23094 now pass the card_output_len and streams this PR makes required, and count only titled tool rows, since a call's result row now reaches the card too.
Since kirodotdev#32367 the redactor keeps the Authorization header name and redacts only the token, so the split-bearer expectations now include the header. The Codex native card tests from kirodotdev#23094 now pass the card_output_len and streams this PR makes required, and count only titled tool rows, since a call's result row now reaches the card too.
Builds on #21488 by @dodgeblaster.
Problem / Motivation
Goal: Redacting an
Authorization: Bearerheader replaces only the token, so a JSON or header document that holds one still parses after redaction.The dashboard Files view shows
Invalid JSON:for a valid file such as~/.kiro/settings/mcp.jsonwhen a remote MCP server carriesheaders.Authorization. Redaction turns{"Authorization": "Bearer <tok>"}into{"[REDACTED: credential]"}, which is not JSON. The secret was removed correctly; the shape around it was destroyed.Why it matters
Anyone whose config, log or chat text carries a Bearer header sees a corrupted-looking file in the Files view and loses the field name in every redacted copy (chat, transcripts, logs). It reads like a broken config when the file on disk is fine.
Not a goal
sanitize.tsmirror, which has no Bearer branch.What changed (motivation → approach → change)
A redacted Bearer header keeps its name and scheme:
{"Authorization": "Bearer <tok>"}becomes{"Authorization": "Bearer [REDACTED: credential]"}, and the Files view renders it as a tree.The Bearer branch in
redaction_patterns.pystarted its match at the header name, and pass 1 of_credential_redaction_planinredaction.pyreplaced every match whole. That branch now names its non-secret prefix (header name, separator, optional quote,Bearerscheme) in abearer_prefixgroup, and pass 1 starts the redacted span where that group ends. Every other branch has no such group and is redacted whole, as before. The rule is still classified from the whole match, and a record's removed value is now exactly the token.The dashboard's sub-agent output stream (
handlers/core.py) holds a poll's last line while it lands inside a credential match, so a token on the line after its kept label is never sent apart from it and its tail stays redacted.The output is the issue's expected shape with the
Bearerscheme word kept, which is not secret.security.mdrecords what survives and the stream's hold rule, and the package line budget intest_security_gate_liveness.pymoves to the measured size.Backwards compatibility
Compatible: every input that was redacted is still redacted and its token is still gone; only the non-secret
Authorization: Bearerprefix now stays in the output.Removes nothing: the redacted output gains the header prefix and loses no record, key or capability.
Tests
test_security.py: a raw Bearer header redacts to exactlyAuthorization: Bearer [REDACTED: credential]; the JSON formjson.loadsback with the key kept and re-redacting it changes nothing; a Bearer JWT is one match with the prefix kept.test_credential_prefilter.py: every non-Bearer corpus shape still equals the by-value legacy oracle exactly; each Bearer shape equals it exactly once its kept prefix is folded back into the tag.test_coder_session_browse.py,test_stream_redactor_aws_keyed.py,test_cron_script_sessions.py,test_agent_stream_tail.py: expectations that encoded the old whole-match output now state the kept prefix; the cron example of a key-spanning credential moves to an AWS key.test_agent_stream_tail.py: an SD-JWT cut by a poll inside its signature, on the line after its label, streams as one redacted frame with none of the disclosure; it fails without the stream's hold.Determinism:
test_a_value_cut_between_two_reads_is_redacted_wholegreen 20 times at-n0and 10 times at-n 4 --dist loadgroup --max-worker-restart=2besidetest_core_handlers_facade.pyandtest_dashboard_handlers_core_coverage_agent_settings.py;test_agent_stream_tail.pygreen in a shuffled order (seed 31038).Proof: PROVEN test/test_agent_stream_tail.py::test_a_value_cut_between_two_reads_is_redacted_whole[bearer-token-cut-after-its-label], test/test_agent_stream_tail.py::test_a_value_cut_between_two_reads_is_redacted_whole[bearer-token-on-the-line-after-its-label], test/test_agent_stream_tail.py::test_a_value_cut_between_two_reads_is_redacted_whole[bearer-token-read-after-its-label-line-ends], test/test_agent_stream_tail.py::test_a_value_cut_between_two_reads_is_redacted_whole[bearer-token-cut-in-a-line-too-long-to-hold], test/test_agent_stream_tail.py::test_a_value_cut_between_two_reads_is_redacted_whole[bearer-token-cut-from-its-kept-label-waits-whole] (head ac8ec52)
Manual verification
Ran the issue's headless repro:
redact_credentialson the samplemcp.jsonreturns{"headers": {"Authorization": "Bearer [REDACTED: credential]"}}, andjson.loadsaccepts it.Related Issues
Closes #13873
Sibling sweep
Swept: git grep -nF '[:=]\s*' -- src/kiro_crew/security/redaction_patterns.py
Sibling: src/kiro_crew/security/redaction_patterns.py:73 -- tracked in #22576
Sibling: src/kiro_crew/security/text_redaction.py -- not affected -- its fail-closed ceiling drops the whole over-long value on purpose
Pattern harvest
Not generalizable: the Bearer branch is the only key-anchored credential branch outside the AWS key-value ones, which #22576 tracks.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)