feat(materials): add trace_sessions to AI security context schema by matiasinsaurralde · Pull Request #3411 · chainloop-dev/chainloop

matiasinsaurralde · 2026-09-07T19:23:03Z

Summary

Allows the CHAINLOOP_AI_SECURITY_CONTEXT material to carry Chainloop Trace session IDs (session_external_id) declared on a commit via the Chainloop-Trace-Sessions git trailer.

The optional trace_sessions array is added to both the fingerprint (fix commit) and commit_ref (introduced_by entries) definitions of the embedded JSON schema, so a consumer can correlate a security-relevant change to the AI coding session(s) that produced or introduced it. The matching field is added to the Go wire types for symmetry.

Allow the CHAINLOOP_AI_SECURITY_CONTEXT material to carry Chainloop Trace session IDs (session_external_id) declared on a commit via the Chainloop-Trace-Sessions git trailer. The optional trace_sessions array is added to both the fingerprint (fix commit) and commit_ref (introduced_by entries) definitions so a consumer can correlate a security-relevant change to the AI coding session(s) that produced or introduced it. Chainloop-Trace-Sessions: 3f365023-61a3-4af3-bbe9-03688f152834, 7cf89fb4-1736-425d-bafb-bfb8b59155a4, df945bde-e982-472c-bb81-50055df403bb Signed-off-by: Matías Insaurralde <matias@chainloop.dev>

chainloop-platform · 2026-09-07T19:23:11Z

AI Session Checks — 🟡 83% · ✅ 0 failing

Avg score	Sessions	Failing policies	Attribution	Files	Lines	Total Duration
🟡 83%	3	✅ 0	92% AI / 8% Human	15	+390 / -12	21h48m58s

🟢 86% — 100% AI — ✅ All policies passing

Sep 7, 2026 19:10 UTC · 20m8s · $11.02 · 196 in / 63.3k out · claude-code 2.1.263 (claude-opus-4-8)

View session details ↗

Change Summary

Adds trace_sessions to the AI security context schema for both fix and introducing commits.
Updates Go wire types and JSON fixtures to carry the new field.
Adds validator coverage for trace_sessions, including the later maxLength follow-up case.

AI Session Overall Score

🟢 86% — Well-scoped schema change with strong verification; only final user confirmation is missing.

AI Session Analysis Breakdown

🟢 96% · scope-discipline

🟢 AI staged only the five schema-related files despite unrelated dirty files. · High Impact

🟢 91% · solution-quality

No notes.

🟢 90% · alignment

No notes.

🟢 89% · user-trust-signal

No notes.

🟢 88% · context-and-planning

🟢 AI researched repo touchpoints and wrote strata_ai_coding_oss.md before editing code. · High Impact

🟡 72% · verification

🟢 AI built affected packages and ran focused plus full validator tests. · High Impact

🟠 Tests and builds ran, but the user never explicitly confirmed the final recommit's behavior. · Medium Severity

💡 Before closing an interactive session, get a brief confirmation or show final proof for the last change.

File Attribution

████████████████████ 100% AI / 0% Human

Status	Attribution	File	Lines
modified	ai	`internal/schemavalidators/schemavalidators_test.go`	+84 / -0
modified	ai	`internal/schemavalidators/internal_schemas/aisecuritycontext/ai-security-context-0.1.schema.json`	+18 / -0
modified	ai	`pkg/attestation/crafter/materials/aisecuritycontext/aisecuritycontext.go`	+12 / -0
modified	ai	`internal/schemavalidators/testdata/ai_security_context_valid.json`	+8 / -2
modified	ai	`pkg/attestation/crafter/materials/testdata/ai-security-context.json`	+8 / -2

Policies (4)

Status	Policy	Material	Messages
✅ Passed	`ai-config-ai-agents-allowed`	`ai-coding-session-7cf89f`	-
✅ Passed	`ai-config-no-dangerous-commands`	`ai-coding-session-7cf89f`	-
✅ Passed	`ai-config-no-secrets`	`ai-coding-session-7cf89f`	-
✅ Passed	`ai-config-mcp-servers-allowed`	`ai-coding-session-7cf89f`	-

🟡 84% — 76% AI — ✅ All policies passing

Aug 26, 2026 21:49 UTC · 43m5s · $16.63 · 328 in / 138.2k out · claude-code 2.1.245 (claude-opus-5)

View session details ↗

Change Summary

Centralizes the AI security context L0 taxonomy behind a shared schema definition and adds guard tests.
Publishes AI security context tool metadata through the shared material-tool annotations.
Enables AI security context auto-detection and updates the related validation tests.

AI Session Overall Score

🟡 84% — Strong execution, but planning and final user confirmation were lighter than the task warranted.

AI Session Analysis Breakdown

🟢 92% · alignment

🟢 The AI respected the evaluation-only phase before switching back to implementation. · High Impact

🟢 91% · solution-quality

🟢 The fixes addressed root causes instead of silencing symptoms or checks. · High Impact

🟢 90% · scope-discipline

No notes.

🟢 86% · user-trust-signal

No notes.

🟡 74% · verification

🟢 Test-first loops caught failures before each substantive fix landed. · High Impact

🟠 Tests and builds ran broadly, but the active user never explicitly confirmed the final shipped behavior. · Medium Severity

💡 Before closing an interactive session, ask for or capture one explicit confirmation after the last verification run.

🟡 72% · context-and-planning

🟠 The last two-item implementation phase crossed several files and packages without a visible written plan. · Medium Severity

💡 For multi-file follow-up work, write a short shared plan first so boundaries and checkpoints stay explicit.

File Attribution

███████████████░░░░░ 76% AI / 24% Human

Status	Attribution	File	Lines
modified	ai	`internal/schemavalidators/schemavalidators_test.go`	+84 / -0
modified	ai	`internal/schemavalidators/internal_schemas/aisecuritycontext/ai-security-context-0.1.schema.json`	+18 / -0
modified	human	`pkg/attestation/crafter/materials/aisecuritycontext/aisecuritycontext.go`	+12 / -0
modified	human	`internal/schemavalidators/testdata/ai_security_context_valid.json`	+8 / -2
modified	human	`pkg/attestation/crafter/materials/testdata/ai-security-context.json`	+8 / -2

Policies (4)

Status	Policy	Material	Messages
✅ Passed	`ai-config-ai-agents-allowed`	`ai-coding-session-3f3650`	-
✅ Passed	`ai-config-no-dangerous-commands`	`ai-coding-session-3f3650`	-
✅ Passed	`ai-config-no-secrets`	`ai-coding-session-3f3650`	-
✅ Passed	`ai-config-mcp-servers-allowed`	`ai-coding-session-3f3650`	-

🟡 79% — 100% AI — ✅ All policies passing

Aug 26, 2026 00:21 UTC · 20h45m45s · $62.15 · 692 in / 374.5k out · claude-code 2.1.245 (claude-opus-5)

View session details ↗

Change Summary

Adds CHAINLOOP_AI_SECURITY_CONTEXT material support, including schema wiring and crafter logic.
Updates workflow contract enum, generated artifacts, and CLI/docs output for the new kind.
Adds validator/crafter fixtures and tests, then scrubs fixture repo references in follow-up commits.

AI Session Overall Score

🟡 79% — Well-planned and well-tested work, but the scrub discussion introduced avoidable drift.

AI Session Analysis Breakdown

🟢 93% · verification

🟢 AI ran targeted and package tests, then reran them after the scrub. · High Impact

🟢 91% · scope-discipline

🟢 The follow-up scrub commits stayed confined to the requested fixture changes. · High Impact

🟢 88% · context-and-planning

🟢 User supplied a concrete plan in pfm7010.md before implementation began. · High Impact

🟢 87% · user-trust-signal

No notes.

🟡 68% · alignment

🟠 AI initially resisted the requested full-fixture scrub with claims it later retracted after checking the actual test surface. · Medium Severity

💡 When pushing back on a requested change, check current tests and code paths before treating hypothetical future checks as decisive.

🟡 62% · solution-quality

🟠 The final scrub rewrote fixture repo metadata without regenerating matching provenance, a user-approved shortcut for testdata convenience. · Medium Severity

💡 For provenance-rich fixtures, prefer regeneration or an explicit note when metadata is deliberately synthetic.

File Attribution

████████████████████ 100% AI / 0% Human

Status	Attribution	File	Lines
modified	ai	`internal/schemavalidators/schemavalidators_test.go`	+84 / -0
modified	ai	`internal/schemavalidators/internal_schemas/aisecuritycontext/ai-security-context-0.1.schema.json`	+18 / -0
modified	ai	`pkg/attestation/crafter/materials/aisecuritycontext/aisecuritycontext.go`	+12 / -0
modified	ai	`internal/schemavalidators/testdata/ai_security_context_valid.json`	+8 / -2
modified	ai	`pkg/attestation/crafter/materials/testdata/ai-security-context.json`	+8 / -2

Policies (4)

Status	Policy	Material	Messages
✅ Passed	`ai-config-ai-agents-allowed`	`ai-coding-session-df945b`	-
✅ Passed	`ai-config-no-dangerous-commands`	`ai-coding-session-df945b`	-
✅ Passed	`ai-config-no-secrets`	`ai-coding-session-df945b`	-
✅ Passed	`ai-config-mcp-servers-allowed`	`ai-coding-session-df945b`	-

Security Checks — ✅ 8 passing

✅ `secret-scan`

Status	Policy	Messages
✅ Passed	`secrets-detection`	-

✅ `sast-scan`

Status	Policy	Messages
✅ Passed	`owasp-top10-2025`	-
✅ Passed	`sast`	-
✅ Passed	`cwe-top25`	-
✅ Passed	`cwe-top26-40-cusp`	-

✅ `iac-scan`

Status	Policy	Messages
✅ Passed	`iac-misconfiguration`	-

PR info

Status	Policy	Messages
✅ Passed	`pr-description-required`	-
✅ Passed	`pr-user-story-linked`	-

⏭️ 2 scans not applied

Scan	Reason
`vulnerability-scan`	no manifest/lockfile changed
`github-actions-scan`	no workflow files changed

View attestation ↗

Powered by Chainloop and Chainloop Trace

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

The TestValidateSecurityContextTraceSessions doc comment claimed per-item maxLength coverage, but the rejected cases only hit maxItems, minLength, and the array type. Add a >256-char session-id case on both commit roles so the maxLength bound is actually exercised. Assisted-by: Claude Code Signed-off-by: Matías Insaurralde <matias@chainloop.dev> Chainloop-Trace-Sessions: 3f365023-61a3-4af3-bbe9-03688f152834, 7cf89fb4-1736-425d-bafb-bfb8b59155a4, df945bde-e982-472c-bb81-50055df403bb

…ainloop-dev#3411) Signed-off-by: Matías Insaurralde <matias@chainloop.dev> Signed-off-by: Yunare Maia <yunare@gmail.com>

matiasinsaurralde requested a review from a team September 7, 2026 19:23

migmartri previously approved these changes Sep 7, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Sep 7, 2026

View reviewed changes

Comment thread internal/schemavalidators/schemavalidators_test.go

matiasinsaurralde dismissed migmartri’s stale review via d71029e September 7, 2026 19:32

matiasinsaurralde requested a review from migmartri September 7, 2026 19:41

migmartri approved these changes Sep 7, 2026

View reviewed changes

matiasinsaurralde merged commit 1957011 into main Sep 7, 2026
17 checks passed

matiasinsaurralde deleted the feat/security-context-trace-sessions branch September 7, 2026 20:05

Conversation

matiasinsaurralde commented Sep 7, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

chainloop-platform Bot commented Sep 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Session Checks — 🟡 83% · ✅ 0 failing

🟢 86% — 100% AI — ✅ All policies passing

Change Summary

AI Session Overall Score

AI Session Analysis Breakdown

🟢 96% · scope-discipline

🟢 91% · solution-quality

🟢 90% · alignment

🟢 89% · user-trust-signal

🟢 88% · context-and-planning

🟡 72% · verification

File Attribution

Policies (4)

🟡 84% — 76% AI — ✅ All policies passing

Change Summary

AI Session Overall Score

AI Session Analysis Breakdown

🟢 92% · alignment

🟢 91% · solution-quality

🟢 90% · scope-discipline

🟢 86% · user-trust-signal

🟡 74% · verification

🟡 72% · context-and-planning

File Attribution

Policies (4)

🟡 79% — 100% AI — ✅ All policies passing

Change Summary

AI Session Overall Score

AI Session Analysis Breakdown

🟢 93% · verification

🟢 91% · scope-discipline

🟢 88% · context-and-planning

🟢 87% · user-trust-signal

🟡 68% · alignment

🟡 62% · solution-quality

File Attribution

Policies (4)

Security Checks — ✅ 8 passing

✅ secret-scan

✅ sast-scan

✅ iac-scan

PR info

⏭️ 2 scans not applied

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matiasinsaurralde commented Sep 7, 2026 •

edited by cubic-dev-ai Bot

Loading

chainloop-platform Bot commented Sep 7, 2026 •

edited

Loading

✅ `secret-scan`

✅ `sast-scan`

✅ `iac-scan`

cubic-dev-ai Bot left a comment •

edited

Loading