Skip to content

fix(dashboard): skip a usage or telemetry line nested past the recursion limit - #29058

Merged
bolichen97 merged 1 commit into
mainfrom
crew/conductor/nested-line-dashboard-r80a
Nov 3, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
crew/conductor/nested-line-dashboard-r80a

Conversation

@RohanK6

@RohanK6 RohanK6 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: One usage, telemetry or transcript line nested past the recursion limit costs that line, not the whole Usage, Telemetry or note read.

The usage spend, cost, context, history and series readers, the telemetry export-cycle reader and the note-committed probe parse each JSONL line inside except ValueError. json.loads raises RecursionError on a line nested past the interpreter limit, and it is not a ValueError, so one such line raised out of the whole read.

Why it matters

These readers back the Usage and Telemetry tabs and the deferred-note durability probe. They already treat a torn or garbled line as "skip it and keep reading"; a deeply nested line is the same unusable input and blanked the whole panel instead.

Not a goal

  • Not changing how any parseable line is read.
  • Not changing the bounded-record cap or the outer file-level error handling.

What changed (motivation → approach → change)

Symptom: a shard with one line like [[[[...]]]] past the decoder depth makes slot_spend() raise. Cause: RecursionError is not a ValueError. Change: each of the nine sites catches (ValueError, RecursionError) and skips the line, the same shape the repo readers that already handle it use (for example apps/builtins/issue_radar/backend/pipeline_fold.py), and the history readers in #29020.

reader file
slot_spend, daily_credits, slot_turn_usage dashboard/handlers/usage_spend.py
context_occupancy dashboard/handlers/usage_context.py
cost_breakdown dashboard/handlers/usage_cost.py
_parse_token_history dashboard/handlers/usage_history.py
_shard_rows dashboard/handlers/usage_series.py
_iter_export_cycles dashboard/handlers/telemetry.py
_note_row_committed dashboard/slot_buffers.py

Backwards compatibility

Compatible: every line that parsed before parses the same; only a line that raised is now skipped.
Removes nothing: no row, route or setting is removed.

Tests

test/test_dashboard_readers_nested_line.py writes a nested line beside one ordinary row for each reader and asserts the reader does not raise and still returns the ordinary row. All nine fail on main. The usage shard fixture pins the zone to UTC with local_tz and reads the instant once, so the shard day and the row ts always share a day.

Proof: PROVEN test/test_dashboard_readers_nested_line.py::test_slot_spend_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_daily_credits_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_slot_turn_usage_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_context_occupancy_skips_a_nested_line, test/test_dashboard_readers_nested_line.py::test_cost_breakdown_skips_a_nested_line (head 877a494)

Manual verification

N/A. Each test drives the real reader against a temp shard.

Related Issues

Part of #29018 (the dashboard readers; the diag, stub-fallback and spool readers are in their own PR, and usage_sessions.py waits for open PR #26951, which rewrites those lines).

Sibling sweep

Swept: git grep -nA2 "json.loads(" -- src/kiro_crew/dashboard/handlers/usage_*.py src/kiro_crew/dashboard/handlers/telemetry.py src/kiro_crew/dashboard/slot_buffers.py
Sibling: src/kiro_crew/dashboard/handlers/usage_spend.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_context.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_cost.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_history.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_series.py -- changed here
Sibling: src/kiro_crew/dashboard/handlers/telemetry.py -- changed here
Sibling: src/kiro_crew/dashboard/slot_buffers.py:229 -- changed here
Sibling: src/kiro_crew/dashboard/handlers/usage_sessions.py -- tracked in #29018
Sibling: src/kiro_crew/dashboard/slot_buffers.py:879 -- not affected -- parses a permission card the server itself wrote with json.dumps, which cannot emit text nested past the decoder limit

Pattern harvest

Rule candidate: semgrep
Pattern: json.loads(...) inside except ValueError (or JSONDecodeError) with no RecursionError, in a line reader that promises to skip bad lines

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the Apache-2.0 license of this project.

@RohanK6
RohanK6 requested review from a team and KiroCrewCaptain as code owners November 3, 2026 09:53
@RohanK6
RohanK6 requested a review from bolichen97 November 3, 2026 09:53
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 877a494c4229892b5445abb840cfb3680c9fddb1 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 877a494c4229892b5445abb840cfb3680c9fddb1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 877a494

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 877a494c4229892b5445abb840cfb3680c9fddb1 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 877a494c4229892b5445abb840cfb3680c9fddb1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: catching RecursionError next to ValueError makes a too-deep line skipped like any other unparseable line, and parseable lines read as before.

Notes

[DESIGN-REVIEWED] 877a494

Description read

[DESCRIPTION-READ] 748362fbafc27a3b6c6b3183b736949f71a15128c284c330406ff9b2acda97b9

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 877a494c4229892b5445abb840cfb3680c9fddb1 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This is a narrow fix with a test that fails on main for each reader. The only gap is that it hand-writes again what kiro_crew.json_line.parse_json_object_line already does.

Notes

  • Item 1 — inherited: the PR adds an inline except (ValueError, RecursionError) at 9 sites. src/kiro_crew/json_line.py:27 parse_json_object_line already does the same thing (catches both errors and returns None for a non-object line), and 30 modules import it. The trailing isinstance(obj, dict) check at 8 of the 9 sites is that helper's contract too. Subtraction: delete the try/except plus the isinstance check at the 8 bounded_records dict readers and call parse_json_object_line(line) instead. Leave _iter_export_cycles alone, because it yields non-dict values.
  • Item 1 — symptom-level, accepted-and-deferred: under dashboard/, grepping json.loads(...)\n except (ValueError|json.JSONDecodeError): still finds 26 sites in 20 files, for example slot_persistence/transcript_merge.py (5) and stt_stream.py (3). The description scopes this PR as "Part of JSONL readers outside history catch ValueError only and raise on a line nested past the recursion limit #29018" and names only usage_sessions.py as left over, so the other 19 files are uncounted siblings.

What this change ships

Inventory (2 items) — 1 justified

Intent: one deeply nested JSONL line should cost only that line, not the whole Usage or Telemetry panel or the note-durability probe. FIX.

  1. A usage, telemetry or transcript line nested too deep to parse is now skipped instead of breaking the whole read — inherited (second spelling of parse_json_object_line)
  2. A new test file pins the skip for all 9 readers — justified

[FIRST-PRINCIPLES-REVIEWED] 877a494

Description read

[DESCRIPTION-READ] 6b9a0befe102c4feae94373f693087a0e582c2fadb1f22fa7845cc5499d44cbc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 877a494c4229892b5445abb840cfb3680c9fddb1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 877a494

Verdict parsed from the review's SHA-scoped output markers for commit 877a494c4229892b5445abb840cfb3680c9fddb1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 877a494c4229892b5445abb840cfb3680c9fddb1 false-positive: <reason, 20 to 500 characters>

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Catching RecursionError beside ValueError in each line reader is the right size of fix, and the new test covers every reader you touched. One real red blocks this head:

SAST (Semgrep) reports 2 findings in the new test, rule semgrep.kirocrew.test-naive-local-time, in test/test_dashboard_readers_nested_line.py. In the rule's words: "A naive local-time read depends on the host's zone and on when the test runs: it passes in UTC and fails where local midnight falls between two reads. Read the zone the product stamps with the instant frozen (ManualClock), or pin the zone with the local_tz fixture (D5)."

Both hits look like the two datetime.now().astimezone() reads: one in _row() for ts, and one in the shard fixture for today's file name. A run that crosses local midnight between them would put the row in the wrong day's shard. Please freeze the instant with ManualClock, or pin the zone with the local_tz fixture, and use one reading for both.

The other reds are cancelled duplicate runs. E2E, Integration and Build Desktop are still running. I will look again on the next head.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 3, 2026
…ion limit

The usage spend, cost, context, history and series readers, the
telemetry export-cycle reader and the note-committed probe parse each
JSONL line inside except ValueError. json.loads raises RecursionError on
a line nested past the interpreter limit, which is not a ValueError, so
one such line raised out of the whole read. Each now catches both and
skips the line, as the readers that already do (pipeline_fold) and the
history readers in the companion fix.
@RohanK6
RohanK6 force-pushed the crew/conductor/nested-line-dashboard-r80a branch from 25d18c6 to 877a494 Compare November 3, 2026 10:27
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 3, 2026
@RohanK6

RohanK6 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, fixed on 877a494:

  1. SAST test-naive-local-time (2 hits in test/test_dashboard_readers_nested_line.py): both datetime.now().astimezone() reads are gone. The shard fixture pins the zone with local_tz("UTC") and takes ONE aware reading, datetime.now(timezone.utc), which names the shard file and is passed to _row(now) for ts, so the day name and the row can no longer straddle midnight. The file has no datetime.now() call left; the 9 tests pass and still fail on main.
  2. Deferral check (rule 25): the one site of JSONL readers outside history catch ValueError only and raise on a line nested past the recursion limit #29018 not in this PR, dashboard/handlers/usage_sessions.py, is held because open fix(usage): count KAS sessions on the Usage page #26951 rewrites those lines; it is named under Related Issues and in the sibling list as tracked in JSONL readers outside history catch ValueError only and raise on a line nested past the recursion limit #29018, which stays open for it.

The other reds you saw were cancelled duplicate runs.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 3, 2026
@RohanK6
RohanK6 requested a review from bolichen97 November 3, 2026 10:28

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 877a494. This fixes my earlier change request. The shard fixture now pins the zone with local_tz("UTC") and takes one datetime.now(timezone.utc) reading. _row(now) gets that same reading, so the row's ts and the shard's file name can no longer land on different days. SAST (Semgrep) passes on this head.

The reader change is the same as before: each line reader catches RecursionError beside ValueError, and the test covers every reader touched. All checks pass, with only PR Readiness still running, and each cancelled check has a newer pass. Opus, GPT, First Principles and Design report no blocking findings; UX was skipped.

@bolichen97
bolichen97 enabled auto-merge (squash) November 3, 2026 11:10
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 3, 2026
@bolichen97
bolichen97 merged commit 859b4ce into main Nov 3, 2026
99 of 106 checks passed
@bolichen97
bolichen97 deleted the crew/conductor/nested-line-dashboard-r80a branch November 3, 2026 11:36
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants