Repository navigation
test: expect the kirocrew.step span in the otlp wire tests - #37171
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change only updates two test expectations to match the [DESIGN-REVIEWED] b9c1377 Description read[DESCRIPTION-READ] 5a91036f37fd4dd1a3544703915d716c66ce4474384156fdf3d38056a8342851 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Verified on head b9c1377. This changes test expectations only, in one file, test/tracing/test_tracing_otlp.py. The traced-turn test now expects kirocrew.step in its span set. The exit-flush test now expects 5 spans instead of 4. Both values match the step span that #37005 added in tracing/turns.py. They also match the five-span turn already pinned in test_tracing_turns.py and test_tracing_runs.py. No other assertion in the file still expects the old four-span set. The vault-header, redaction and loopback assertions are unchanged. The change adds no sleep, clock, port or ordering dependency.
Done well: the test still asserts the exact set of span names instead of a subset. So any span added later without a test update will still fail here.
#37005 adds one kirocrew.step span per derived model call to a traced turn. The two wire-level OTLP tests still expected the four spans from before, so they fail wherever the otlp extra is installed, which CI does since #36721. Expect the step span and a count of five. Closes #37169 (cherry picked from commit 6de8528)
…#37539 Two backend tests on release/0.13.0 were red for reasons main has already fixed in test-only commits: - test/tracing/test_tracing_otlp.py expected four spans, but #37005 (already on the branch) adds kirocrew.step. Cherry-picked #37171 unchanged. - TestSelfModuleIndexIsLinear failed three runs in a row on the 0.5 s budget (0.522 s, 0.511 s, 0.509 s). Took main's end state of test/test_denied_commands_security_cost.py and test/denied_commands_security_helpers.py, which carries #37451 (min of five readings), #37530 (tracer paused) and #37539 (count work, not CPU time), plus the #37539 hunk of docs/system-specs/common/testing-conventions.md. The picks conflict one by one because the file also carries #28293, a test-only change. No src/ changes. (cherry picked from commit 6de8528)
…#37539 Two backend tests on release/0.13.0 were red for reasons main has already fixed in test-only commits: - test/tracing/test_tracing_otlp.py expected four spans, but #37005 (already on the branch) adds kirocrew.step. Cherry-picked #37171 unchanged. - TestSelfModuleIndexIsLinear failed three runs in a row on the 0.5 s budget (0.522 s, 0.511 s, 0.509 s). Took main's end state of test/test_denied_commands_security_cost.py and test/denied_commands_security_helpers.py, which carries #37451 (min of five readings), #37530 (tracer paused) and #37539 (count work, not CPU time), plus the #37539 hunk of docs/system-specs/common/testing-conventions.md. The picks conflict one by one because the file also carries #28293, a test-only change. No src/ changes. (cherry picked from commit 6de8528)
Problem / Motivation
Goal: Main's backend test shard 2 goes green again by making the two wire-level OTLP tests expect the
kirocrew.stepspan that #37005 added.Every pull request that contains #37005 fails
Backend Tests (3.12, 2)andBackend Tests (Windows) (2)ontest/tracing/test_tracing_otlp.py: one test sees an extrakirocrew.stepspan, the other counts 5 spans where it expects 4.Why it matters
Main is red for every open PR rebased after 08:30Z. Authors either rerun a test that cannot pass or read the red as their own bug.
Not a goal
kirocrew.stepspan; this PR only updates the tests to that.otlpextra is installed in CI (ci: install the otlp extra in backend test lanes so wire-level OTLP tests run #36721).What changed (motivation → approach → change)
#37005 adds one
kirocrew.stepspan per derived model call. The two OTLP wire tests still listed the four spans from before. They are markedrequires_otlp, so they skip on a machine without theotlpextra, which is why the change passed locally; CI installs the extra since #36721 and runs them. The fix addskirocrew.stepto the expected span set intest_a_traced_turn_reaches_a_loopback_receiver_with_its_vault_headerand makes the count 5 intest_the_exit_flush_sends_spans_still_queued.Backwards compatibility
Compatible: only two test expectations change.
Removes nothing: no product code changes.
Tests
test/tracing/test_tracing_otlp.py: the two tests now expect the step span.Proof: not provable -- test-only change
Run with
opentelemetry-exporter-otlp-proto-http==1.44.0installed (theotlpextra):Determinism: this change touches only the expected values; the three repeat runs above passed.
Manual verification
N/A — the change is two expected values in tests, shown red on main and green here with the extra installed.
Related Issues
Closes #37169
Follows #37005 (the step span) and #36721 (the
otlpextra in CI).Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement