redaction: mark AI coding sessions as redacted whenever the scan runs, not only when secrets were replaced #3578

Problem

The crafter sets chainloop.material.redacted=true on an AI coding session only when betterleaks replaced at least one secret (annotateRedaction returns early on !report.Changed() in pkg/attestation/crafter/materials/chainloop_ai_coding_session.go). When betterleaks scans the session and finds nothing, the material carries no redaction annotation at all.

So a session that betterleaks scanned and found clean looks the same as one that was never scanned (CLI older than v1.107.0). The ai-config-no-secrets policy can't tell them apart. It falls back to its own regex scan and overrules the betterleaks verdict with a much cruder pattern.

Example

chainloop-dev/platform#6816. The session (ai-coding-session-3ca59f, run a17f94ad-464b-4157-bc85-23c078c3261a) read the lab Dex config, which has the dummy client secret secret: ZXhh…mV0 (base64 of example-app-secret).

• betterleaks (CLI 1.7.0, default rules) doesn't flag it, which is correct.
• The uploaded session has no [CHAINLOOP_TRACE_REDACTED:…] placeholders and no chainloop.material.redaction.* annotations.
• ai-config-no-secrets fails anyway: Potential secret (Quoted API key/password) found in session content [turn=12, source=tool_result, line=74, value=secret: ...mV0"]. That message comes from the fallback regex branch.

PFM-6881 is the same symptom on another PR.

Proposal

chainloop.material.redacted=true should mean "this content went through redaction", not "secrets were found". Whenever the redaction scan runs, set it, even when nothing was replaced:

• chainloop.material.redacted=true
• chainloop.material.redaction.count=0 (keep redaction.rules empty or omit it)

Then the policy needs just one condition: when the annotation is present, trust the redaction and report only [CHAINLOOP_TRACE_REDACTED:…] placeholders. A clean scan reports nothing. The regex fallback only runs for materials with no redaction annotation (old CLIs). redaction.skipped=true keeps its current meaning.

Things to check in the change

• Fail-closed check. Attestation_Material.GetEvaluableContent returns ErrRedactedContentRequired when the material is marked redacted and no content is supplied. Today redact() returns nil content for a clean session, so setting the annotation alone would break policy evaluation. Pass the scanned bytes through as the policy input in the clean case too.
• Digest. Keep the current behavior: only use withContentOverride when something was replaced, so a clean session's digest still matches its source file.
• Policy side (platform, ai-config-no-secrets-coding-session.rego). Make content_redacted depend on material_redacted alone, not also on finding a placeholder (that second condition was added in PFM-7073 as a guard against a crafter handing over the original file). With the fail-closed check above, a material marked redacted can only be evaluated against the content that was scanned, so the guard isn't needed. This needs a follow-up PR in platform after this ships.
• policy devel eval (app/cli/internal/policydevel/eval.go) mirrors the crafter annotations. Make sure it follows the new meaning.

Acceptance criteria

• A session scanned with zero findings is annotated redacted=true, redaction.count=0.
• Policy evaluation for that session gets the scanned content and doesn't fail closed.
• Stored digest for a clean session is unchanged.
• Tests cover: secrets redacted, clean scan, --skip-secret-redaction.
• Follow-up platform issue/PR to simplify content_redacted in ai-config-no-secrets.

Related

• PFM-7073: send redacted content to policies (introduced the annotation semantics)
• PFM-6881: ai-config-no-secrets false positives on test fixtures
• PFM-7478: review AI policies
• feat(crafter): redact secrets from AI coding sessions before CAS upload #3368: redact secrets from AI coding sessions before CAS upload

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri