Skip to content

fix(deploy): degrade a profile registry json.loads refuses outright - #29252

Closed
dimwael wants to merge 1 commit into
kirodotdev:mainfrom
dimwael:fix/deploy-registry-refuses
Closed

dimwael wants to merge 1 commit into
kirodotdev:mainfrom
dimwael:fix/deploy-registry-refuses

Conversation

@dimwael

@dimwael dimwael commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A profiles.json that json.loads refuses outright costs one registry read, not every deploy route and every profile save.

load_registry (src/kiro_crew/deploy/profiles.py) promises that one bad profiles.json cannot 500 every route that reads the registry, and degrades to the legacy fallbacks and then an empty registry. Its three reads caught json.JSONDecodeError and UnicodeDecodeError only. json.loads raises RecursionError on a file nested past the interpreter limit, and a plain ValueError on a number past the integer digit limit; neither is a JSONDecodeError, so either escaped out of every reader as an HTTP 500. The locked writer _load_registry_for_update caught ValueError but not RecursionError, so a nested file raised out of a profile save instead of being set aside.

Why it matters

The registry is agent-writable and operator-editable. Every deploy route, the AWS Control account resolver and the consent card read it, so one such file blanks the whole deploy surface, and a profile add or edit on top of it fails instead of moving the file aside the way it already does for truncated JSON.

Not a goal

  • Not changing how any decodable registry, or any wrong-shaped one, is read or written.
  • Not adding a warning to the reader path: load_registry degrades silently for every unreadable shape today, and the locked writer's _set_aside already logs the move.
  • Not touching the other json.loads sites under deploy/, which parse AWS CLI output or a request body rather than the registry file.

What changed (motivation → approach → change)

Symptom: a profiles.json holding [[[[...]]]] past the decoder depth, or {"default": 999...9} with 5000 digits, makes load_registry() raise, and locked_registry() raises on the nested one. Cause: the except tuples name json.JSONDecodeError and UnicodeDecodeError, and RecursionError is not a ValueError at all. Change: each of the three reads catches ValueError (which covers both named exceptions and the digit-limit one) and RecursionError, the same (ValueError, RecursionError) # past the nesting limit shape deploy/pending.py and the readers in #29058 and #29059 use, so such a file takes the fallback ladder the function already implements. The locked writer catches the same pair, so the file is moved to <name>.corrupt-<utc> before the save. The load_registry docstring names the two exceptions.

Backwards compatibility

Compatible: every file that read or saved before reads and saves the same; only a file that raised now degrades or is set aside.
Removes nothing: no route, key, setting or public function is removed.

Tests

test/test_deploy_registry_shape.py gains two parametrized tests over the two refused shapes (nested past the decoder, over the digit limit): all three registry files refused still end at the empty default without raising, and a refused primary still falls through to the legacy registry. test/test_deploy_registry_unreadable_write.py adds the same two shapes to the moved-aside parametrization: the write lands, the original bytes sit in <name>.corrupt-<utc>, and the warning names it. The nested text comes from the shared stray_line_helpers.too_deep_to_decode() probe; the writer's over-digit case already passed and stays as a control.

Proof: PROVEN test/test_deploy_registry_shape.py::test_a_registry_json_loads_refuses_reads_as_empty_at_every_step[nested-past-the-decoder], test/test_deploy_registry_shape.py::test_a_registry_json_loads_refuses_reads_as_empty_at_every_step[over-digit-limit] (head 15f3e45)

Determinism: the two test files pass 20/20 at -n0, 10/10 at -n 4 --dist loadgroup --max-worker-restart=2 beside test_deploy_web_profiles.py and test_deploy_round37_fixes.py, and in a shuffled order (seed 29089, 27 node ids). With the production hunk reverted, 5 of the 27 fail (both reader tests for both shapes, and the writer's nested case).

Manual verification

N/A — unit coverage sufficient: each test drives the real load_registry / locked_registry against a temp file; no UI or external service is involved.

Related Issues

Closes #29089

Sibling sweep

Swept: git grep -nA3 "json.loads(" -- src/kiro_crew/deploy/
Sibling: src/kiro_crew/deploy/profiles.py -- changed here
Sibling: src/kiro_crew/deploy/pending.py -- not affected -- both reads already catch (ValueError, RecursionError), and the writer's sets the file aside
Sibling: src/kiro_crew/deploy/handlers.py:828 -- not affected -- parses the request's own body, not a shared file; a refused body fails that one request, not every route
Sibling: src/kiro_crew/deploy/engine.py:481 -- not affected -- parses AWS CLI output the CLI itself serialized, not an operator-editable file
Sibling: src/kiro_crew/deploy/iam.py:844 -- not affected -- parses AWS CLI output the CLI itself serialized, not an operator-editable file

Pattern harvest

Rule candidate: semgrep
Pattern: json.loads(...) of an operator-editable file inside except JSONDecodeError (or ValueError) with no RecursionError, in a reader that promises to degrade an unreadable file

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the Apache-2.0 license of this project.

load_registry promises that one bad profiles.json cannot 500 every route
that reads the registry, and degrades to the legacy fallbacks and then an
empty registry. Its three reads caught json.JSONDecodeError and
UnicodeDecodeError only. json.loads raises RecursionError on a file
nested past the interpreter limit, and a plain ValueError on a number
past the integer digit limit; neither is a JSONDecodeError, so either
escaped every reader. The locked writer caught ValueError but not
RecursionError, so a nested file raised out of a profile save instead
of being set aside.

Each read now catches ValueError (which covers JSONDecodeError and
UnicodeDecodeError) and RecursionError, so such a file takes the same
fallback ladder as truncated JSON, and the locked writer moves it to
<name>.corrupt-<utc> before the save, as it does for truncated JSON.

Closes kirodotdev#29089
@dimwael
dimwael requested review from a team and KiroCrewCaptain as code owners November 3, 2026 16:51
@dimwael
dimwael requested a review from buluoray November 3, 2026 16:51
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 3, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #32454; thanks for the start.

bolichen97 added a commit that referenced this pull request Nov 7, 2026
…32454)

load_registry promises that one bad profiles.json cannot 500 every route
that reads the registry. Its three reads caught json.JSONDecodeError and
UnicodeDecodeError only, so a file nested past the interpreter's
recursion limit (RecursionError) or holding a number past the integer
digit limit (a plain ValueError) escaped every reader. The locked writer
caught ValueError but not RecursionError, so a nested file raised out of
a profile save instead of being moved aside.

Each read now catches ValueError and RecursionError, and the locked
writer sets a nested file aside to <name>.corrupt-<utc> before the save,
as it already does for truncated JSON.

Builds on #29252.

Closes #29089
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 7, 2026
buluoray pushed a commit that referenced this pull request Nov 8, 2026
…32454)

load_registry promises that one bad profiles.json cannot 500 every route
that reads the registry. Its three reads caught json.JSONDecodeError and
UnicodeDecodeError only, so a file nested past the interpreter's
recursion limit (RecursionError) or holding a number past the integer
digit limit (a plain ValueError) escaped every reader. The locked writer
caught ValueError but not RecursionError, so a nested file raised out of
a profile save instead of being moved aside.

Each read now catches ValueError and RecursionError, and the locked
writer sets a nested file aside to <name>.corrupt-<utc> before the save,
as it already does for truncated JSON.

Builds on #29252.

Closes #29089

(cherry picked from commit 32ce0d8)
buluoray pushed a commit that referenced this pull request Nov 8, 2026
…32454)

load_registry promises that one bad profiles.json cannot 500 every route
that reads the registry. Its three reads caught json.JSONDecodeError and
UnicodeDecodeError only, so a file nested past the interpreter's
recursion limit (RecursionError) or holding a number past the integer
digit limit (a plain ValueError) escaped every reader. The locked writer
caught ValueError but not RecursionError, so a nested file raised out of
a profile save instead of being moved aside.

Each read now catches ValueError and RecursionError, and the locked
writer sets a nested file aside to <name>.corrupt-<utc> before the save,
as it already does for truncated JSON.

Builds on #29252.

Closes #29089

(cherry picked from commit 32ce0d8)
@iamwhatever

Copy link
Copy Markdown
Contributor

Thanks for this work! The same change has since landed on main in #32454 ("fix(deploy): degrade a profile registry json.loads refuses outright"), which closed the linked issue #29089. Closing this one as superseded. Please reopen if something here is still missing from main.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(deploy): a profile registry json.loads refuses still 500s every deploy route

3 participants