Repository navigation
feat(tracing): OTLP span export to Langfuse and CloudWatch (phase 2, #1643) - #35792
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. Span export goes through the same Notes
[DESIGN-REVIEWED] 915477c Description read[DESCRIPTION-READ] 6f31ebaccc88d3f9ed000ff9275fa5a863202e8707c5a3b4816f78b4f3402edd That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Every item traces to the phase 2 row of Notes
What this change shipsInventory (10 items) — 8 justifiedIntent: let an operator see Kiro Crew's per-turn traces in Langfuse, CloudWatch or any OTLP collector. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 915477c Description read[DESCRIPTION-READ] 84a2365b88f52b6a9b462e8a8bc906cc4c49ea10a59ea5e3c32bfdff2d4801cc That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ human override acceptedHuman judgment by @kyleseaman overrides the GPT 6.1 finding for [GPT-OVERRIDE] 915477c This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
ed0f853 to
c61c97e
Compare
|
span=178890badd65
|
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 15% chance that this change introduces a defect, above the 12% limit for approving on its own. Adds opt-in OTLP span export through the trust path metrics already uses. Checked in the code:
An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
/ai-review override gpt c61c97e false-positive: documented limitation (collector echoing credentials into local urllib3 WARNING logs); the only fix is a process-wide log filter, which would be a redaction sink requiring Sensitive Change Review. Tracked as a known limitation in the spec and guide. |
Human judgment recorded@kyleseaman recorded a false-positive release (class
This decision applies only to this commit. A new push requires a new judgment. |
c61c97e to
915477c
Compare
|
Rebased: main moved in docs/system-specs/modules/metrics.md since this head went green (386875e -> 5e7b587); CI re-running. The rebase resolved one docs-only conflict in the Traces paragraph (main #35651 reworded the |
|
/ai-review override gpt 915477c false-positive: documented limitation (collector echoing credentials into local urllib3 WARNING logs); the only fix is a process-wide log filter, which would be a redaction sink requiring Sensitive Change Review. Tracked as a known limitation in the spec and guide. |
Human judgment recorded@kyleseaman recorded a false-positive release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
This adds a new opt-in path that sends trace data off the machine, and #1643 is a T3 item that waits on a design decision, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here. The review found no blocking defect on this head. It checked these points in the diff:
The only design acceptance on record is the RFC author's own comment on #1643, and #35443 is not merged yet. Security: the PR author overrode the one security lane finding as a false positive. That finding is that a collector which echoes a vault-backed Authorization value in a malformed response header can get that value logged by urllib3. The rebuttal reads as sound: the trigger needs the operator's own collector, and the guide documents the risk. But it is a real limitation that has been accepted, so a maintainer should record that release under the class that fits it. Done well: the PR parses Optional:
|
bolichen97
left a comment
There was a problem hiding this comment.
Export phase direction signed by maintainer (bolichen97): opt-in, default off; whole-RFC acceptance still tracked in #35443.
I read the full diff: config, edition_locks, otlp_governance, metrics/provider, tracing/otlp.py, tracing/provider.py and the tests.
- Nothing leaves without a collector.
resolvereturnsNoneunlesstraces_enabledis truly on and a config orOTEL_EXPORTER_OTLP_*endpoint is named. Without one, no OTLP processor or thread is added, and the test counts only the local sink's thread. Switching off clears the exporter, so spans a still-open turn ends go nowhere. An LTS build ignores all three endpoint sources. - What leaves.
RemoteSpansstrips everykirocrew.local.*key from spans and links before export. The wire test shows the raw session key only in the local sink. - Headers. Values are read only from Settings -> Secrets entries or named variables. A literal value in config is refused. A config endpoint never gets the process-wide OTel header variables, and no warning quotes a value or a vault reference.
- Transport. No redirect is followed,
.netrcis never read, andrequestserrors and reason phrases are rewritten so a value cannot reach a log. - Endpoint rules. https is required unless the host is loopback, with no user:password or query. The host is held to
otlp_hostsat build and again on every export, with SEL audit. - No regex in product code. The character checks are sets with length bounds.
The checks the feature adds apply only to its own new inputs; no path main has is tightened. All checks green on this head. LGTM.
Problem / Motivation
Goal: Kiro Crew can send its agent-turn traces to Langfuse, AWS CloudWatch or any OTLP collector the operator names, with auth headers kept in the vault.
Phases 1a (#31460) and 1b (#35077) record one trace per agent turn, sub-agents included, but only to local JSONL files under
~/.kiro/crew/traces. The customer ask in #1643 is to see those traces, with model, credits, tokens and tool calls, in their own trace backend. Today nothing leaves the host, so a Langfuse or CloudWatch user sees nothing.Why it matters
This is the phase of the native tracing RFC (#31433,
docs/request-for-change/rfc-otel-tracing.md, phase 2 row of §10) that a customer actually sees. Without it the traces recorded since 1a are only useful to someone reading JSONL on the machine.Not a goal
config.jsonand the vault.What changed (motivation → approach → change)
Goal → approach. Reuse what metrics already trusts for egress rather than adding a second path: the same
otlpextra, thecapabilities.telemetryotlp_hostspolicy with its per-export recheck and audit row, the LTS lock, and main's no-redirect session.What you can now do. With
telemetry.traces_enabledon, settelemetry.traces_otlp_endpointto a collector's/v1/tracesURL, store the auth header in Settings → Secrets, and map it intraces_otlp_headers_secret. Each turn then reaches that collector. Langfuse shows eachchatspan as a generation with its Kiro credits; CloudWatch accepts an X-Ray API key as a bearer header.Rules that keep it safe.
OTEL_EXPORTER_OTLP_*variables set for another tool send nothing.https://unless the host is loopback, and carry nouser:passwordor query.requestswould refuse, and request errors are replaced with fixed text.~/.netrc, so the collector receives only the headers you configured.kirocrew.local.*keys never reach the wire.Known limitations.
OTEL_EXPORTER_OTLP_*CERTIFICATE,CLIENT_KEYandCOMPRESSIONvariables still apply to a configured endpoint, because the SDK exporter reads them for any argument left empty. No secret is sent, but a client certificate set for another collector is presented to this one.A new guide,
docs/guides/telemetry-otlp-traces.md, walks through Langfuse, CloudWatch (API key and collector) and a credits query.Where.
src/kiro_crew/tracing/otlp.py(new): where spans may go, header reading, the export session, the wire filter for local keys.src/kiro_crew/tracing/provider.py: adds the OTLP batch processor on the first collector, swaps it on a config write, pins batch limits;tracing/__init__.pynames the module.src/kiro_crew/metrics/provider.pyandmetrics/otlp_governance.py: the policy check and per-export gate, reused for spans.src/kiro_crew/config/(sections_telemetry.py,loader_builders.py,loader.py) andconfig-baseline.json: the three settings;src/kiro_crew/edition_locks.py: their LTS lock.Backwards compatibility
Compatible: every new setting is off or empty by default, and with tracing off nothing changes.
gate_exportergains an optionalrefusedargument; existing callers pass nothing and keep the metric result.Removes nothing: the change only adds settings, a module and a guide; the metrics
/v1/traceswarning still fires and now points at the traces setting.Tests
test/tracing/test_tracing_otlp.py, against an in-process receiver bound to127.0.0.1:OTEL_SERVICE_NAME, and nokirocrew.local.*key; the local file keeps them;.netrcentry for the collector's host is never sent, with or without a vault header;OTEL_BSP_*says, and the exit flush sends queued spans.Determinism: the file passed 5 repeats and a reversed order; it binds only an ephemeral loopback port and waits on no wall-clock delay.
Manual verification
Still required, needs AWS: one real Claude turn against CloudWatch Transaction Search, confirming that model and token fields fill (RFC phase 2 exit criterion), and the guide's Logs Insights credits query.
Related Issues
Part of #1643
RFC: #31433. Builds on #31460 (phase 1a) and #35077 (phase 1b). Whole-RFC acceptance: #35443.
Deferred: the live CloudWatch check above (needs AWS), phase 2b native SigV4, phase 7 Settings UI.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)