Repository navigation
fix(dashboard): skip a usage or telemetry line nested past the recursion limit - #29058
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns: catching Notes
[DESIGN-REVIEWED] 877a494 Description read[DESCRIPTION-READ] 748362fbafc27a3b6c6b3183b736949f71a15128c284c330406ff9b2acda97b9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS This is a narrow fix with a test that fails on main for each reader. The only gap is that it hand-writes again what Notes
What this change shipsInventory (2 items) — 1 justifiedIntent: one deeply nested JSONL line should cost only that line, not the whole Usage or Telemetry panel or the note-durability probe. FIX.
[FIRST-PRINCIPLES-REVIEWED] 877a494 Description read[DESCRIPTION-READ] 6b9a0befe102c4feae94373f693087a0e582c2fadb1f22fa7845cc5499d44cbc That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Thanks. Catching RecursionError beside ValueError in each line reader is the right size of fix, and the new test covers every reader you touched. One real red blocks this head:
SAST (Semgrep) reports 2 findings in the new test, rule semgrep.kirocrew.test-naive-local-time, in test/test_dashboard_readers_nested_line.py. In the rule's words: "A naive local-time read depends on the host's zone and on when the test runs: it passes in UTC and fails where local midnight falls between two reads. Read the zone the product stamps with the instant frozen (ManualClock), or pin the zone with the local_tz fixture (D5)."
Both hits look like the two datetime.now().astimezone() reads: one in _row() for ts, and one in the shard fixture for today's file name. A run that crosses local midnight between them would put the row in the wrong day's shard. Please freeze the instant with ManualClock, or pin the zone with the local_tz fixture, and use one reading for both.
The other reds are cancelled duplicate runs. E2E, Integration and Build Desktop are still running. I will look again on the next head.
…ion limit The usage spend, cost, context, history and series readers, the telemetry export-cycle reader and the note-committed probe parse each JSONL line inside except ValueError. json.loads raises RecursionError on a line nested past the interpreter limit, which is not a ValueError, so one such line raised out of the whole read. Each now catches both and skips the line, as the readers that already do (pipeline_fold) and the history readers in the companion fix.
25d18c6 to
877a494
Compare
|
Thanks, fixed on 877a494:
The other reds you saw were cancelled duplicate runs. |
bolichen97
left a comment
There was a problem hiding this comment.
Approving 877a494. This fixes my earlier change request. The shard fixture now pins the zone with local_tz("UTC") and takes one datetime.now(timezone.utc) reading. _row(now) gets that same reading, so the row's ts and the shard's file name can no longer land on different days. SAST (Semgrep) passes on this head.
The reader change is the same as before: each line reader catches RecursionError beside ValueError, and the test covers every reader touched. All checks pass, with only PR Readiness still running, and each cancelled check has a newer pass. Opus, GPT, First Principles and Design report no blocking findings; UX was skipped.
Problem / Motivation
Goal: One usage, telemetry or transcript line nested past the recursion limit costs that line, not the whole Usage, Telemetry or note read.
The usage spend, cost, context, history and series readers, the telemetry export-cycle reader and the note-committed probe parse each JSONL line inside
except ValueError.json.loadsraisesRecursionErroron a line nested past the interpreter limit, and it is not aValueError, so one such line raised out of the whole read.Why it matters
These readers back the Usage and Telemetry tabs and the deferred-note durability probe. They already treat a torn or garbled line as "skip it and keep reading"; a deeply nested line is the same unusable input and blanked the whole panel instead.
Not a goal
What changed (motivation → approach → change)
Symptom: a shard with one line like
[[[[...]]]]past the decoder depth makesslot_spend()raise. Cause:RecursionErroris not aValueError. Change: each of the nine sites catches(ValueError, RecursionError)and skips the line, the same shape the repo readers that already handle it use (for exampleapps/builtins/issue_radar/backend/pipeline_fold.py), and the history readers in #29020.slot_spend,daily_credits,slot_turn_usagedashboard/handlers/usage_spend.pycontext_occupancydashboard/handlers/usage_context.pycost_breakdowndashboard/handlers/usage_cost.py_parse_token_historydashboard/handlers/usage_history.py_shard_rowsdashboard/handlers/usage_series.py_iter_export_cyclesdashboard/handlers/telemetry.py_note_row_committeddashboard/slot_buffers.pyBackwards compatibility
Compatible: every line that parsed before parses the same; only a line that raised is now skipped.
Removes nothing: no row, route or setting is removed.
Tests
test/test_dashboard_readers_nested_line.pywrites a nested line beside one ordinary row for each reader and asserts the reader does not raise and still returns the ordinary row. All nine fail on main. The usage shard fixture pins the zone to UTC withlocal_tzand reads the instant once, so the shard day and the rowtsalways share a day.Proof: PROVEN test/test_dashboard_readers_nested_line.py::test_slot_spend_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_daily_credits_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_slot_turn_usage_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_context_occupancy_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_cost_breakdown_skips_a_nested_line (head 877a494)
Manual verification
N/A. Each test drives the real reader against a temp shard.
Related Issues
Part of #29018 (the dashboard readers; the diag, stub-fallback and spool readers are in their own PR, and
usage_sessions.pywaits for open PR #26951, which rewrites those lines).Sibling sweep
Swept: git grep -nA2 "json.loads(" -- src/kiro_crew/dashboard/handlers/usage_*.py src/kiro_crew/dashboard/handlers/telemetry.py src/kiro_crew/dashboard/slot_buffers.py
Sibling: src/kiro_crew/dashboard/handlers/usage_spend.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_context.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_cost.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_history.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_series.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/telemetry.py -- changed here
Sibling: src/kiro_crew/dashboard/slot_buffers.py:229 -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_sessions.py -- tracked in #29018
Sibling: src/kiro_crew/dashboard/slot_buffers.py:879 -- not affected -- parses a permission card the server itself wrote with json.dumps, which cannot emit text nested past the decoder limit
Pattern harvest
Rule candidate: semgrep
Pattern: json.loads(...) inside except ValueError (or JSONDecodeError) with no RecursionError, in a line reader that promises to skip bad lines
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the Apache-2.0 license of this project.