Skip to content

docs(trace): add a guide to implement a new trace provider - #3681

Merged
migmartri merged 4 commits into
chainloop-dev:mainfrom
jiparis:docs/trace-provider-guide
Oct 21, 2026
Merged

migmartri merged 4 commits into
chainloop-dev:mainfrom
jiparis:docs/trace-provider-guide

Conversation

@jiparis

@jiparis jiparis commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Adds app/cli/internal/trace/PROVIDERS.md, a guide to add a new Chainloop Trace provider (harness), and links it from the Trace README.

Refs #3657: the guide describes what a Codex provider must implement.

The guide covers:

  • The session lifecycle, and which provider method and shared handler run at each step (trace init, agent hooks, commit, push, uninstall, trace run).
  • The trace.Provider methods by group: hook installation, hook input, code attribution (snapshot or reverse edits), session copy and parsing, and the user and model channels. It uses Claude Code, Cursor, and OpenCode as examples.
  • The evidence mapping: session metadata, models, token usage, and cost.
  • The optional interfaces: SkillTracker, SpecCopyReplacer, and PastedImageFinder.
  • The CLI wiring: registry, init wizard, --<agent> flags, and trace hook <agent> subcommands.
  • The required end-to-end harness and compatibility matrix updates.
  • A checklist for new providers.

AI disclosure: written with the assistance of Claude Code.

View guided diff

Describe the provider interfaces, the session lifecycle, CLI wiring,
evidence mapping (models, token usage, cost), the required end-to-end
harness and compatibility matrix updates, and a checklist for new
providers.

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 5297a53e-f2e8-4920-9552-4f0e046aad70
@chainloop-platform

chainloop-platform Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

PR validation — ✅ 3 passing

Status Policy Material Messages
✅ Passed pr-min-approvals pr-info -
✅ Passed pr-description-required pr-info -
✅ Passed pr-user-story-linked pr-info -

View attestation ↗

AI Session Checks — 🟡 60% · ✅ 0 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟡 60% 1 ✅ 0 100% AI / 0% Human 3 +322 / -1 36m20s

🟡 60% — 100% AI — ✅ All policies passing

Oct 21, 2026 09:49 UTC · 36m20s · $8.83 · 250 in / 104.5k out · claude-code 2.1.306 (claude-opus-5-5)

View session details ↗

Change Summary

  • Adds app/cli/internal/trace/PROVIDERS.md, a detailed provider-implementation guide.
  • Links that guide from app/cli/internal/trace/README.md.
  • Adds .claude/skills/trace-provider/SKILL.md so the guide is auto-discoverable.

AI Session Overall Score

🟡 60% — Strong scope and trust, but repeated documentation inaccuracies kept alignment red.

AI Session Analysis Breakdown

🟢 95% · user-trust-signal

🟢 The user expanded the work without corrective pushback or resets. · _ Impact_

🟢 94% · scope-discipline

🟢 The changed files stayed on the guide, README link, and reviewer-requested skill. · _ Impact_

🟡 72% · context-and-planning

🟠 The AI drafted a wide guide without a visible outline or plan. · Medium Severity

💡 For multi-section guides, write a short outline first so later checks can trace against it.

🔴 36% · alignment

🟢 After review, the AI checked each finding against code before editing. · _ Impact_

🔴 Six confirmed review findings show the first guide draft misstated implementation details. · High Severity

💡 Before opening a docs PR, re-check every implementation claim against the code paths it describes.

abstained · solution-quality

🟡 solution-quality not assessed — reviewer should confirm coverage · Low Severity

abstained · verification

🟡 verification not assessed — reviewer should confirm coverage · Low Severity


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
created ai app/cli/internal/trace/PROVIDERS.md +303 / -0
created ai .claude/skills/trace-provider/SKILL.md +18 / -0
modified ai app/cli/internal/trace/README.md +1 / -1

Policies (4)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-5297a5 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-5297a5 -
✅ Passed ai-config-no-secrets ai-coding-session-5297a5 -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-5297a5 -

Security Checks — ✅ 5 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -
Scans not applied (3)
Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗

Security context

✅ Nothing this change touches has a recorded security-fix history.

View in Chainloop ↗ · How this works ↗


Powered by Chainloop and Chainloop Trace

@jiparis
jiparis requested a review from a team October 21, 2026 10:00

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread app/cli/internal/trace/PROVIDERS.md Outdated
Comment thread app/cli/internal/trace/PROVIDERS.md Outdated
Comment thread app/cli/internal/trace/PROVIDERS.md Outdated
Comment thread app/cli/internal/trace/PROVIDERS.md Outdated
Comment thread app/cli/internal/trace/PROVIDERS.md
Comment thread app/cli/internal/trace/PROVIDERS.md Outdated
Separate git hook installation from InstallHooks, describe the init and
trace run backup lifecycles, state that Cursor sets no usage, clarify
skipActionOptsInit, note plugin reinstall after a payload change, and
list the matrix cells that the end-to-end suite does not check.

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 5297a53e-f2e8-4920-9552-4f0e046aad70
Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 5297a53e-f2e8-4920-9552-4f0e046aad70

@migmartri migmartri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiparis can you add a skill that points to this guide so it gets autodiscovered?

Assisted-by: Claude Code
Signed-off-by: Jose I. Paris <jiparis@chainloop.dev>

Chainloop-Trace-Sessions: 5297a53e-f2e8-4920-9552-4f0e046aad70
@jiparis

jiparis commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

@migmartri Added .claude/skills/trace-provider/SKILL.md in d29a9af. It triggers on adding or changing a Trace provider and points to PROVIDERS.md and its checklist.

@migmartri
migmartri merged commit 9ea459a into chainloop-dev:main Oct 21, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants