Repository navigation
test(tracing): expect the kirocrew.step span in the OTLP wire tests (#1643) - #37188
kyleseaman wants to merge 1 commit into
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This only updates stale tests to match the span tree the specs already define, and the exact-set assertion is stricter than the old count. [DESIGN-REVIEWED] 4a9cca0 Description read[DESCRIPTION-READ] 9b5056ec3d15d34cf57a0a3782291e1cc08bbf55d0a46d20cdbadaad1a11dfc8 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Closing: main's red was already fixed by #37171, which landed first with the same change. |
Problem / Motivation
Goal: Turn main's Backend Tests green again by updating two OTLP wire tests to the span tree a traced turn now exports.
Main is red. CI run 43580582401 on
16e11a1870fails Backend Tests shard 2 on Linux and Windows:test/tracing/test_tracing_otlp.py::test_a_traced_turn_reaches_a_loopback_receiver_with_its_vault_header: the exported span-name set has an extrakirocrew.step.test/tracing/test_tracing_otlp.py::test_the_exit_flush_sends_spans_still_queued:assert 5 == 4.Why it matters
Every PR's Backend Tests shard 2 inherits this red until main is fixed, so unrelated PRs cannot reach a clean readiness verdict.
Not a goal
What changed (motivation → approach → change)
Two merges crossed. #37005 (tracing phase 4a) added a derived
kirocrew.stepspan to every traced turn, so one turn now exports five spans. #36721 installed theotlpextra in CI, which is what makes theserequires_otlpwire tests run at all. #37005's CI ran on a base from before #36721, so it never ran them; main has both.The new span is intended (the documented tree in
docs/system-specs/modules/metrics.mdandtest_tracing_turns.pyalready expect it), so the tests were stale, not the product. Both tests now compare against one named set,_TURN_SPANS, that listskirocrew.stepexplicitly. The exit-flush test asserts the exact set instead of a count of 4, so it still fails if a span is lost or added.Backwards compatibility
Compatible: test-only change; no product code moves.
Removes nothing: the asserts compare exact sets, as before.
Tests
Updated
test/tracing/test_tracing_otlp.py(the two tests above).Proof: not provable -- test-only change
Run locally in a Python 3.13 venv with the
otlpextra installed (as CI installs it):origin/main(205d08d) both tests fail exactly as in CI; on the same main with feat(tracing): steps inside a turn: derived model calls, compaction, approvals, steers (#1643) #37005 reverted both pass. That pins the cause.test/tracing+test/metrics, 2157 passed.-n0, 10/10 at-n 4 --dist loadgroup --max-worker-restart=2withtest_tracing_steps.pyandtest_tracing_turns.pyas co-tenants, and all 93 tests oftest_tracing_otlp.pygreen in a shuffled order (seed 29862).Manual verification
N/A — the change is two test assertions; the local runs above cover them.
Related Issues
Part of #1643
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)