Skip to content

fix(apps): treat an undecodable or non-object dependency ledger as empty - #24425

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
IngridMorstrad:fix/dependency-ledger-undecodable
Oct 26, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
IngridMorstrad:fix/dependency-ledger-undecodable

Conversation

@IngridMorstrad

@IngridMorstrad IngridMorstrad commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: App install, uninstall and the uninstall preview keep working when the dependency ledger file holds non-UTF-8 bytes or valid JSON that is not an object.

_read_ledger_unlocked() in src/kiro_crew/apps/dependency_ledger.py is the one reader behind every ledger operation (record_install, record_uninstall, get_entry, list_by_app, classify_for_uninstall, classify_and_clean_for_uninstall). It already treats malformed JSON and an unreadable file as an empty ledger. A file with a bad byte raises UnicodeDecodeError, which is not caught. A file that parses to [], a string, a number or null is returned as-is, and the caller's ledger.get(...) / ledger.items() raises AttributeError or TypeError.

Why it matters

A hand-edited, truncated-then-restored or wrongly-encoded dependency-ledger.json makes every app install and uninstall fail until someone repairs the file by hand. The reader already has a contract for this case (warn and treat as empty), and these two faults bypass it.

Not a goal

  • Changing the ledger format, the locking, or how it is written.
  • Validating individual entries inside an object-shaped ledger.

What changed (motivation → approach → change)

  • Symptom: get_entry/record_install/record_uninstall raise UnicodeDecodeError, AttributeError: 'list' object has no attribute 'get' or TypeError: argument of type 'int' is not iterable on a bad ledger file.
  • Root cause: _read_ledger_unlocked() catches only (JSONDecodeError, OSError) and never checks the parsed shape.
  • Change: add UnicodeDecodeError to the caught errors, and when the parsed value is not a dict, log the existing Failed to read dependency ledger warning and return {}. src/kiro_crew/apps/dependency_ledger.py, one hunk.

Backwards compatibility

Compatible: every object-shaped ledger reads exactly as before. Only the shapes that raised before now warn and read as empty, which is what a malformed ledger already does. As with malformed JSON today, the next record_install rewrites the file as a well-formed object.

Removes nothing.

Tests

New test/test_dependency_ledger_decode.py (not test/test_dependency_ledger.py, which open #24269 edits), parametrized over latin-1 bytes, [], a string, a number, null, and a {bad malformed-JSON control:

  • test_readers_treat_bad_ledger_as_empty: get_entry returns None and logs the warning, list_by_app is empty, and the uninstall preview lists the dependency as user-installed.
  • test_uninstall_on_bad_ledger_finds_nothing_to_remove: record_uninstall and classify_and_clean_for_uninstall do not raise and remove nothing.
  • test_install_rewrites_bad_ledger_as_an_object: record_install succeeds and rewrites the file as a one-entry object, and the uninstall then removes it.

Any exception escaping a ledger call is turned into a pytest.fail so the failure is an assertion, not a crash.

Red on unfixed origin/main @ a88d228 (test file applied alone): 15 failed, 3 passed. Each failure is <fn> raised on a bad ledger: UnicodeDecodeError(...), AttributeError(...) or TypeError(...); the 3 passes are the malformed-JSON control. Green with the fix: test/test_dependency_ledger_decode.py, test/test_dependency_ledger.py, test/test_dependencies.py, test/test_apps_backend_coverage_dependency_install.py, test/test_capability_ledger.py: 169 passed.

Proof: PROVEN test/test_dependency_ledger_decode.py::test_readers_treat_bad_ledger_as_empty[latin1-bytes], test/test_dependency_ledger_decode.py::test_readers_treat_bad_ledger_as_empty[list], test/test_dependency_ledger_decode.py::test_readers_treat_bad_ledger_as_empty[string], test/test_dependency_ledger_decode.py::test_readers_treat_bad_ledger_as_empty[number], test/test_dependency_ledger_decode.py::test_readers_treat_bad_ledger_as_empty[null] (head 6c74077)

prove.py --per-hunk also reports PROVEN for the single production hunk.

Local gates: Semgrep with the repo rules (semgrep/, 1.78.0), black gate, subprocess-encoding gate, brand gate (diff-scoped), comment-history gate (base origin/main), isort, flake8 and mypy src/kiro_crew/apps/dependency_ledger.py all pass, and check_file_line_cap reports 0 failures. scripts/local-gate.py backend related set: 284 passed. Its frontend cross-surface set could not run cleanly here: npx cannot reach the npm registry (E401), and a run against another checkout's node_modules gave 16 failures in files unrelated to this change (DiffBlock, MarkdownRenderer, ArtifactDetailPage, SchedulePage). This diff touches no website/ file, so that set is CI's to judge.

Manual verification

N/A — unit coverage sufficient: the defect and the fix are confined to _read_ledger_unlocked(), and the tests drive the public ledger functions against the real on-disk file under an isolated KIROCREW_HOME.

Screenshots / video

N/A — backend only, no UI change.

Related Issues

Closes #24424

#18366 (assigned) lists the lock touch() at dependency_ledger.py:187, in _locked_ledger(). This PR does not touch that function or line.

Sibling sweep

Swept: git grep -nE "json\.loads?\(" -- src/kiro_crew/apps/dependency_ledger.py src/kiro_crew/apps/dependencies.py
Sibling: src/kiro_crew/apps/dependency_ledger.py:202 -- changed here
Sibling: src/kiro_crew/apps/dependencies.py -- not affected -- it has no JSON reader of its own and reads the ledger only through the functions fixed here

The same narrow except (json.JSONDecodeError, OSError) also appears at these sites under src/kiro_crew/apps/ (from git grep -nE "except \(json\.JSONDecodeError, OSError\)" -- src/kiro_crew/apps/). They are not checked or fixed here:

  • src/kiro_crew/apps/bridges_agents.py:814
  • src/kiro_crew/apps/bridges_mcp_config.py:147
  • src/kiro_crew/apps/bridges_register.py:110
  • src/kiro_crew/apps/builtins/code_review_sage/sage_lib/report.py:749,797,809
  • src/kiro_crew/apps/builtins/meetings/backend/providers/tasks.py:167
  • src/kiro_crew/apps/builtins/ops_mission_control/backend/http_routes/actions.py:525,597
  • src/kiro_crew/apps/builtins/ops_mission_control/backend/http_routes/configuration.py:191 (open feat(ops-mission-control): show the incident.io on-call roster #13428 edits this file)
  • src/kiro_crew/apps/builtins/ops_mission_control/backend/http_routes/lifecycle.py:85,224,263
  • src/kiro_crew/apps/manager_records.py:351
  • src/kiro_crew/apps/registry_pipeline/caches.py:168,712
  • src/kiro_crew/apps/registry_pipeline/sources.py:789
  • src/kiro_crew/apps/routes_install.py:377
  • src/kiro_crew/apps/routes_listing.py:395

Each of those needs its own check of whether it decodes text and what its fallback contract is, so they are left to separate changes rather than folded in here.

Pattern harvest

Rule candidate: semgrep
Pattern: a function documented to degrade to an empty dict on a bad file that catches JSONDecodeError but not UnicodeDecodeError around read_text(encoding="utf-8"), or returns the json.loads result without an isinstance(..., dict) check

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 6c74077a49ffd6186cd8379f1f9d0226dd1d8eef touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 6c74077a49ffd6186cd8379f1f9d0226dd1d8eef via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: a fix at the single shared reader, applying the corrupt-ledger contract it already has to two shapes that slipped past it.

Notes

  • The next record_install rewrites a corrupt ledger with only its own entry, so the record of which app installed each earlier dependency is lost. After that, uninstalling one of those apps classifies its dependencies as user-installed and keeps them. That fails safe, and malformed JSON already behaves this way, but the warning could name the file so the operator can recover it before it is overwritten.

[DESIGN-REVIEWED] 6c74077

Description read

[DESCRIPTION-READ] cea8ba400430d63132bee176a6998910780432e4e4d117be8bf8639e0f0e2e3c

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 6c74077a49ffd6186cd8379f1f9d0226dd1d8eef via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Both items route two failing inputs into the reader's existing "warn and treat as empty" contract. The test file fails on base (15 failed), and the author lists the unfixed siblings and defers them.

What this change ships

Inventory (3 items) — 3 justified

Intent: keep app install, uninstall and uninstall preview working when dependency-ledger.json holds non-UTF-8 bytes or valid JSON that is not an object. This is a FIX.

  1. A ledger file with non-UTF-8 bytes now logs a warning and reads as empty, where it used to raise UnicodeDecodeError. — justified
  2. A ledger that parses to a list, string, number or null now logs a warning and reads as empty, where it used to raise AttributeError or TypeError. — justified
  3. On either kind of bad file, the next install rewrites the ledger as a well-formed object, the same as malformed JSON today. — justified

[FIRST-PRINCIPLES-REVIEWED] 6c74077

Description read

[DESCRIPTION-READ] b59a0a9ec4209aaf51b1cbd927c5eb6b2b0b89cf9edc239a00dc11c109d9a140

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 6c74077a49ffd6186cd8379f1f9d0226dd1d8eef via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6c74077

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 6c74077a49ffd6186cd8379f1f9d0226dd1d8eef via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6c74077

_read_ledger_unlocked already returns an empty ledger for malformed JSON
or an unreadable file. A ledger with non-UTF-8 bytes raised
UnicodeDecodeError, and one holding valid JSON of another shape ([],
a string, a number, null) was returned as-is, so every caller's
ledger.get()/.items() raised. Either fault failed app install,
uninstall and the uninstall preview.

Catch UnicodeDecodeError with the existing decode errors, and return
an empty ledger with the same warning when the parsed value is not a
dict.
@IngridMorstrad
IngridMorstrad force-pushed the fix/dependency-ledger-undecodable branch from 67da933 to 6c74077 Compare October 26, 2026 09:18

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 6c74077.

The change is one hunk in _read_ledger_unlocked(). It adds UnicodeDecodeError to the caught errors and returns {} with the existing warning when the parsed value is not a dict. Every object-shaped ledger reads as before.

What I checked:

  • The new tests cover all six ledger entry points against a real file under an isolated home. The decode and shape cases fail on main because the call raises, so the tests guard the fix.
  • An empty read is the safe direction for uninstall. _classify_deps puts an untracked dependency in userInstalled, so a bad ledger never causes a dependency to be removed.
  • The next write replaces a bad file with a well-formed object. This matches what malformed JSON already does on main. For a file with one bad byte, the old entries are lost and those dependencies are kept on uninstall instead of removed. That is the same trade-off the reader already makes, so I do not ask for a change here.
  • Two files changed, no security-sensitive path, no shared file with another open PR, and main has not changed this function.

This looks good to ship. I am not approving yet because the PR readiness signal is still pending on this head. I will approve once it passes on 6c74077.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 26, 2026
@IngridMorstrad

Copy link
Copy Markdown
Contributor Author

Re the Design Review note (name the file in the warning so an operator can recover it before the next install overwrites it): deferring. The warning text and the overwrite-on-next-install behaviour are unchanged from main, where malformed JSON already takes the same path. This PR only routes two more bad shapes into it. Naming the path, or saving a .corrupt copy, would be a separate change to that existing contract.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 6c74077.

Gates checked at this head: readiness passed, every required check green (the cancelled runs were superseded by passing reruns of the same checks), AI review lanes report no blocking findings, base is main, no merge conflict, two changed files, no security sensitive path.

The fix keeps the existing contract: an undecodable or non-object ledger is warned about and read as empty, and the new tests cover every ledger entry point.

@bolichen97
bolichen97 merged commit de83227 into kirodotdev:main Oct 26, 2026
109 of 115 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dependency ledger with non-UTF-8 bytes or a non-object value fails app install and uninstall

2 participants