Skip to content

refactor(security): compose the redaction pass pair through one helper - #11501

Closed
DeryFerd wants to merge 1 commit into
kirodotdev:mainfrom
DeryFerd:fix/redaction-pair-order-and-helper
Closed

DeryFerd wants to merge 1 commit into
kirodotdev:mainfrom
DeryFerd:fix/redaction-pair-order-and-helper

Conversation

@DeryFerd

Copy link
Copy Markdown
Contributor

Problem / Motivation

Two redaction passes have a fixed order, and a call site that writes them by hand can get it backwards without anything failing loudly. kiro_crew.security.redact composes them correctly, and its docstring states why: redact_exfiltration_urls matches whole token-bearing URLs, so a credential replaced ahead of it leaves a placeholder sitting inside the URL and the URL pass stops recognising the shape it exists to catch. Call sites have been writing the pair out anyway. Across src/kiro_crew there are 452 pairs composed on the same variable, and 84 of them use the reversed order in 48 files. The dashboard package holds 37 of those.

The credential itself is removed either way, which is what makes the reversed order read as harmless. What it loses is the other half: the destination. Reducing the input to the minimum, see https://evil.example.com/collect?token=AKIAIOSFODNN7EXAMPLE&x=1 now, the canonical order produces see [REDACTED: suspicious URL to evil.example.com] now, while the reversed order produces see https://evil.example.com/collect?token=[REDACTED: credential]&x=1 now. The host survives as a live, clickable endpoint in a line a reader is meant to trust as cleaned.

Why it matters

Redaction is the boundary between what the operator may see and what must not travel. Every one of these sites is an egress: a chat transcript, a dashboard WebSocket broadcast, a SEL audit record, a channel delivery, an exception message shown in a UI. When the order is reversed, the text arrives at that surface looking redacted while still naming the endpoint an attacker chose. That is worse than not redacting at all, because the tag signals "this was handled" and a reader stops looking.

This is a bug class the repository has already paid for once. Issue #9014 measured five egress sites composing the passes in reverse; PR #9033 fixed exactly those five and added no mechanism to stop the sixth. Nothing prevents the pattern from being written again, which is why the 84 current sites exist. This change fixes one package and adds the missing mechanism, so the count can only go down from here.

What changed (motivation → approach → change)

The two passes already have a single correct home. kiro_crew.security.redact composes URL-then-credential, and redact_with_findings does the same work while also returning the two warning lists. Both live beside the passes they sequence, so the ordering lives in exactly one place instead of being re-spelled at hundreds of call sites.

The dashboard package's 37 reversed sites now compose through those helpers. redact_with_findings covers the sites that capture the warning lists as a signal rather than merely counting them, because that path returns both lists in the same order the hand-written pair did. The swap is otherwise a pure composition change: I checked redact() against the manual pairing over a corpus of 15 inputs (empty string, plain prose, a bare AWS key, a slash-bearing secret key, a GitHub token, a JWT, a JSON blob, a URL with the credential in its query, and a base64-encoded key) and got byte-identical output on every one. So the only behavioural delta a reviewer has to reason about is the order itself, which is the fix.

I deliberately did not migrate all 84 sites in one go. The remaining 47 pairs in 36 files are pinned in the baseline instead. A sweep that large mixes the ordering fix with a lot of unrelated review surface, and the sites that are left sit in packages where the pinned list is easy to work through separately. Pinning them means the rule is enforced from today rather than after a long sweep lands.

Two alternatives I passed on. Wrapping just the URL pass so the order becomes unstateable at the call site would have been a smaller diff, but it hides the sequencing inside a helper whose name says nothing about ordering, and the next person writing the pair by hand still gets it backwards. Doing the whole tree at once would leave the ratchet with an empty baseline, which sounds cleaner but makes the PR un-reviewable and couples a mechanical sweep to a security fix.

Tests

test/test_redaction_pair_order.py (19 tests, new) carries both halves: the behaviour the order decides, and the mechanism that keeps the order from being rewritten.

The behavioural half asserts the canonical order suppresses the destination, the reversed order leaves a live URL standing, the two orders genuinely disagree on this input, and they agree when no URL carries the credential. That last one is a negative control, without which "the orders differ" could be satisfied by any divergence and the rule would say nothing about when it matters. A separate test pins redact() against the manual composition across the corpus, so a third variant cannot appear unnoticed.

The mechanism half scans src/kiro_crew and compares both spellings against a pinned baseline. It covers a two-statement pair and a single nested expression, and the nesting direction matters: the innermost call runs first, so redact_credentials(redact_exfiltration_urls(x)) is the correct order and redact_exfiltration_urls(redact_credentials(x)) is the reversal. The scan parses the AST with ast, so a comment or a string literal quoting the pattern can never count as a violation.

Twelve probes keep the scan honest: same variable, a different target variable fed from the first, nested in a branch, nested in an except handler, the subscript form, the correct order, unrelated redactions, both nesting directions, and a comment quoting the pattern. One further test asserts the walk actually reaches real files and the security package, because a scan that silently found nothing would pass forever.

The baseline is shrink-only. A pinned entry that has been paid off must be pruned, or the comparison fails, so the list cannot quietly stop measuring anything.

The ratchet is proven to catch a regression rather than merely observed going green. I planted a reversed pair in session_summary.py, watched the gate fail and name that file, then removed it and watched the gate pass.

Gates run locally:

  • black --target-version py310 on the changed files, isort, flake8 — clean
  • mypy — Success: no issues found in 12 source files
  • scripts/check_black_formatting.py — 14 changed files in scope, nothing unformatted outside the baseline
  • scripts/check_subprocess_encoding.py — passed
  • scripts/check_comment_history.py with COMMENT_HISTORY_BASE_REF=upstream/main — passed
  • scripts/docs_lint.py — All documentation checks passed
  • test/test_redaction_pair_order.py test/test_security.py test/test_workflows_agent_pool.py — 1044 passed, 19 skipped, 3 xfailed
  • test/test_artifacts.py test/test_artifacts_handlers.py test/test_artifacts_handlers_coverage.py test/test_chat_runner_coverage.py test/test_dashboard_sessions_search.py — 841 passed, 10 skipped, 6 xfailed

Manual verification

N/A — unit coverage is sufficient. The change moves which helper composes two pure text functions; it touches no HTTP surface, no rendering, and no user-facing string, and the corpus check above proves the composition is byte-identical to the manual pairing it replaces. The one observable difference is the order itself, and that is asserted directly against the helper.

Related Issues

Relates to #9014, closed by #9033, which fixed the five egress sites that issue identified and added no mechanism to catch a recurrence. This change migrates one package and adds that mechanism, so the class is gated rather than re-fixed per instance. There is no dedicated issue for the remaining 47 sites.

Pattern harvest

Rule candidate: test

Pattern: a required composition order re-spelled by hand at call sites is a silent downgrade, so gate the tree against the reversed spelling and pin what is not migrated yet in a shrink-only baseline.

Checklist

  • One commit, Conventional Commits title
  • Existing tests pass and a new gate test locks the change
  • Self-review completed; the change follows the existing security.redact seam rather than inventing one
  • Documentation updated — docs/system-specs/modules/security.md records the rule, the gate, and the pinned remainder
  • No secrets, credentials, or internal references in the diff

@DeryFerd
DeryFerd requested review from a team and dwu96 September 17, 2026 09:34
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of 48d4c73866433c881f03e1a9ca712ddcf11d8f21 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The demonstrated leak stays live at 47 pinned egress sites (channel, voice, discord, ACP) with no tracked follow-up — the ratchet only stops growth.

Watch

The PR's own test proves the reversed order leaves "a live, clickable exfiltration endpoint" standing, and the pinned baseline includes outbound-message paths (channel.py, voice_reply.py, discord/transport_dispatch.py, acp/client.py) where that text reaches external readers; the description concedes "There is no dedicated issue for the remaining 47 sites," so a shrink-only baseline that nothing forces to shrink can carry those leaks indefinitely.
Clears when: a filed issue or follow-up PR(s) covering the 36 pinned files exists and is referenced from the baseline docstring or spec.

Suggestions

  • The scan recognises only bare-name calls (_call_name requires ast.Name), so a new reversed pair spelled security.redact_credentials(...) or via an import alias passes silently despite the gate's "either spelling" claim — extend _call_name to attribute calls and add a probe.

[DESIGN-REVIEWED] 48d4c73

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 48d4c73866433c881f03e1a9ca712ddcf11d8f21 via the fork AI-review pipeline; updated in place on each push.

Review details

FINDING -- test/test_redaction_pair_order.py:233 -- restricting _call_name to ast.Name misses the qualified reversed pair in apps/builtins/md_notebook/server.py, leaving the ratchet incomplete -> Fix: recognize ast.Attribute via .attr.
[GPT-REVIEWED] 48d4c73

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of 48d4c73866433c881f03e1a9ca712ddcf11d8f21 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

The gate's two-statement scan links calls only through bare names — a reversed pair on a subscript (the entry["path"] form this PR removes) escapes it.

Not justified as shipped

  • Item 9 — rides along: test_tokenize_is_available_for_a_literal_safe_scan asserts stdlib tokenize behavior; it "records the tool the AST scan replaces" and protects nothing in this repo.

What this change ships

Inventory (9 items) — 8 justified

Intent: FIX — stop the reversed redaction order leaving live exfiltration URLs at dashboard egress surfaces, and gate the tree against new reversed pairs (documented invariant at src/kiro_crew/security/__init__.py:785-791 names this exact hazard).

  1. ~40 dashboard egress surfaces now suppress a token-bearing URL's destination (URL pass first) — justified
  2. Prompt-detail endpoints get warnings via redact_with_findings (return order verified against the unpacking) — justified
  3. New shrink-only ratchet gate over src/kiro_crew pinning 47 unmigrated pairs — justified
  4. websocket_hub broadcast keeps a hand-written pair, order fixed — justified (redactors are injected providers, websocket_hub.py:49-50; the helper would drop companion patterns)
  5. files.py error strings now route through the context-aware redact_via_context shim — justified (direction the spec mandates, security/__init__.py:798-802)
  6. test_cse_2026_08_07_fixes.py pin rewritten to the new mechanism; the substantive sink assertions are unchanged — justified
  7. Heartbeat test monkeypatch retargeted to the composed redact — justified
  8. security.md updated in the same commit — justified (AGENTS.md same-commit rule)
  9. A probe test asserting stdlib tokenize works — rides along: harm-free documentation-as-test, guards nothing in-repo

Verified against base: no existing gate covers pair order (grepped test/ for reversed-pair scans — none); the tree-scan-baseline idiom is established (20 files use the tree_scan xdist-group convention); baseline spot-checks (session_summary.py:392, metrics/schema.py:145) match their pinned entries and the schema.py boolean-only claim holds.

[FIRST-PRINCIPLES-REVIEWED] 48d4c73

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review (fork) — ✅ no blocking findings

Reviewed 48d4c73866433c881f03e1a9ca712ddcf11d8f21 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 48d4c73

@DeryFerd
DeryFerd force-pushed the fix/redaction-pair-order-and-helper branch from 49e1397 to d0ad586 Compare September 17, 2026 12:53
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@DeryFerd

Copy link
Copy Markdown
Contributor Author

The remaining Backend Tests (3.12, 3) and Coverage Gate failures are inherited from main, not from this change:

Coverage Gate is failing closed because that shard is red (backend-test=cancelled). This PR touches neither pod/ nor session_pid.py; its diff is limited to dashboard/ redaction call sites, one spec doc, and a new gate test.

@DeryFerd
DeryFerd force-pushed the fix/redaction-pair-order-and-helper branch from d0ad586 to 746a2ab Compare September 17, 2026 17:41
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@DeryFerd
DeryFerd force-pushed the fix/redaction-pair-order-and-helper branch from 746a2ab to 48d4c73 Compare September 18, 2026 09:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 18, 2026
@bolichen97
bolichen97 force-pushed the fix/redaction-pair-order-and-helper branch from 48d4c73 to 8f1c508 Compare September 19, 2026 05:29
@bolichen97

Copy link
Copy Markdown
Contributor

Rebased onto main cff7de5416 by a maintainer to resolve merge conflicts. Old head 48d4c73866, new head 8f1c508e4b.

Conflicts resolved (only this file was hand-merged; no other manual edits):

  • src/kiro_crew/dashboard/handlers/files.py -- Kept main’s is_sensitive_resolved_path import for path completion, while preserving the PR’s removal of the two raw redaction-pass imports in favor of the composed helper.

Same single commit, same author, same message. Apart from changes inherited from main, everything outside the conflict hunk was preserved without manual edits. The original 16-file PR change set is unchanged. All 15 changed Python files compile; the focused redaction-order suite passes (19 tests); diff whitespace and conflict-marker checks pass. The full suite was not run.

Any prior approval was dismissed by the push and needs to be re-given after CI re-runs.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Sep 19, 2026
The credential and exfiltration-URL passes must run in a fixed order, and
`kiro_crew.security.redact` already composes them that way. Its docstring
gives the reason: the URL pass matches whole token-bearing URLs, so a
credential replaced ahead of it leaves a placeholder inside the URL and
the URL pass fails to recognise the shape it exists to catch. The
credential goes either way; what the reversed order loses is suppression
of the destination, which is the half a reader acts on.

Call sites hand-sequenced the pair anyway: 452 same-variable pairs across
the tree, 84 of them in the reversed order across 48 files. The dashboard
package's 37 now compose through `redact` (or `redact_with_findings` where
the warning lists are used as a signal, which preserves both lists in the
same order). The remaining 47 pairs in 36 files are pinned rather than
migrated, so the rule holds from today and the sweep reviews separately.

`test_redaction_pair_order.py` pins the behaviour the order decides, the
pinned baseline (shrink-only: a paid-off entry that is not pruned fails),
and the scan's own soundness. The scan resolves every spelling the tree
uses — a bare name, a qualified chain, and an import alias that resolves
to a module re-exporting the passes — while refusing a same-named method
on an unrelated object. It reads the AST, so a comment or string literal
quoting the pattern cannot count, and the target/argument comparison is
by source, so a `entry["path"]` target counts too.

Two existing tests assert the seams this change moves, so both follow the
migrated call: `files.py` now composes through the context-aware `redact`
shim and no longer binds either raw pass, and `state._log_task_exception`
calls `redact`, which is the symbol a failing-redactor test patches.
@DeryFerd
DeryFerd force-pushed the fix/redaction-pair-order-and-helper branch from 8f1c508 to 31da7ea Compare September 19, 2026 08:16
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@DeryFerd

Copy link
Copy Markdown
Contributor Author

@bolichen97 thanks for rebasing this onto cff7de5416 and for preserving the single-commit shape — I checked the result and your conflict resolution is intact (the is_sensitive_resolved_path import is kept in files.py alongside the removal of the two raw redaction-pass imports). I also addressed the three findings the review lanes raised against the previous head, plus one you did not see because the lanes went quiet afterwards.

What I changed since your rebase (8f1c508e4b → 31da7ead0d)

Both reviewers flagged the same real defect in the new gate, and they were right — my scan only resolved calls through a bare name (ast.Name), so a reversed pair spelled security.redact_credentials(...) passed silently while the test's own docstring claimed "either spelling".

  • _call_name now resolves three spellings: a bare name, a qualified chain, and an import alias that resolves to a module re-exporting the passes. The alias path matters in this tree: dashboard/handlers/{members,sessions}.py reach the passes as _h.<pass> via import kiro_crew.dashboard.handlers as _h, and I verified by identity that _h.redact_credentials is security.redact_credentials. It deliberately does not credit a same-named method on an unrelated object (vendor_sdk.redact_credentials), with a probe pinning that direction.
  • This immediately caught one site the earlier version had missed: apps/builtins/md_notebook/server.py:1349. It is now pinned in the baseline like the rest.
  • The target/argument comparison is now by source (ast.unparse) rather than by identifier, so a dict-entry target counts. That shape matters because this PR's own migration rewrote entry["path"], _ = redact_credentials(entry["path"]), which an identifier-only reader sees no target on at all.
  • Removed the test_tokenize_is_available_for_a_literal_safe_scan probe that First Principles correctly called unjustified: it asserted stdlib behaviour and guarded nothing in this repo.

The suite is now 25 tests (up from 19). I proved the tightened scan catches a regression rather than merely passing: I planted an attribute-spelled reversed pair in session_summary.py and the gate failed naming that file, then restored it.

I also fixed a performance problem I had introduced. Two independent walks of src/ (~1600 files, ~35s per 300) were pushing the xdist worker past its budget, which pytest reports as worker crashed rather than an assertion failure. Both scanners now share one traversal and release each tree as they go; on this machine that went from ~480s with crashes to ~80s passing. Worth knowing for anyone running the file locally: this box reports 1 of 4 workers (3.2 GiB free, 11 GiB installed), so a worker crashed here is memory pressure, not the test.

On the review lanes

The four model lanes are currently reporting review incomplete for this head — GPT 5.6 Review, Opus 4.8 Review, First Principles Review and Design Review. That is the workflow's own fallback when no complete verdict is produced, and it started at your rebase head (8f1c508e4b), not with my last push. Security Scope Review and UX Review both completed normally on the same commit. For contrast, #11759 currently shows GPT 5.6 Review — no blocking findings, so the pipeline looks healthy in general rather than down.

I have not re-pushed to try to shake it loose, because a re-run is unlikely to help if the cause is transient, and each push dismisses whatever approval state exists. Happy to force-push a fresh SHA if you would prefer the lanes to re-run against the current head.

Status of the checks I can act on

75 checks pass, including Internal Content Scan, PR Hygiene, PR Scope, Comment History Gate, Docs Lint, Backend Lint & Type Check, all eight Windows shards, E2E and the frontend lanes. There are no failing test jobs. The rebased head is 2 commits behind main (1a8b9f775); I checked that neither of those commits touches any file in this diff, so I have left it rather than churn another force-push — say the word if you would rather I rebase again.

One thing still open from Design Review, which I deliberately did not act on unilaterally: the 47 sites pinned in the baseline include outbound paths (channel.py, voice_reply.py, discord/transport_dispatch.py, acp/client.py), and a shrink-only baseline does not force anyone to shrink it. If you'd like, I can open a follow-up issue listing the 36 files so the baseline docstring can reference it, or fold the migration into this PR. Your call.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 19, 2026
@dwu96 dwu96 added the needs-pr-triage PR scanner: awaiting automated triage label Sep 23, 2026
@NicholasRBowers NicholasRBowers added drive-to-green PR claimed by drive-to-green pipeline and removed needs-pr-triage PR scanner: awaiting automated triage labels Sep 23, 2026
@NicholasRBowers

Copy link
Copy Markdown
Contributor

🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: This PR has been inactive for 7+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll push fixes directly to this branch as a co-author.

Assessment: Nothing is wrong with the change itself — every CI job is green, First Principles is PASS, and the two findings the lanes raised are already fixed on your current head. Three things are left:

  1. One conflicting file. A trial merge against current main (500 commits ahead of the merge base) shows src/kiro_crew/dashboard/handlers/discover.py as the only conflict; all fifteen other files auto-merge. Both hunks are the same semantic change on each side, differing only in line-wrapping — main reached the identical redact()-then-URL-pass composition independently and its side also carries the explanatory "Canonical composition" comment. I'll take main's side on both hunks.

  2. All four AI review lanes failed closed on the current head. GPT 5.6, Opus 4.8, Design and First Principles each reported "review incomplete" for 31da7ead0 inside the same minute (2026-09-19T08:22Z) while their comment bodies still stamp the older 48d4c738 — the model-service blip pattern, not a finding. GPT and Opus are the two blocking readiness items. A rebase mints a new head and re-fires all of them.

  3. Design Review's advisory CONCERNS needs an answer, not a code change. It notes the 47 pinned sites keep the leak live with no tracked follow-up, and clears when a filed issue or follow-up PR covering the pinned files is referenced from the baseline docstring or spec. I'll file that tracking issue and reference it, then disposition the item.

For the record, so it isn't re-litigated: GPT's _call_name-only-matches-ast.Name finding and Design's matching suggestion are already resolved on 31da7ead0 — the gate now carries _alias_targets, _resolves_to_security and _SECURITY_QUALIFIERS, so qualified and aliased spellings are caught. I also confirmed this is not superseded: main carries only test_redaction_mirror_parity.py, so your tree-wide test_redaction_pair_order.py ratchet is genuinely new. I'll keep First Principles' one subtraction (dropping the stdlib-tokenize test that protects nothing in this repo) and will not widen the diff beyond that.

If you'd prefer I don't touch this PR, add the pr-no-autofix label.

@DeryFerd DeryFerd closed this Sep 24, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 24, 2026
@dwu96 dwu96 removed the drive-to-green PR claimed by drive-to-green pipeline label Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants