Skip to content

fix(core): pin utf-8 on the diagnostics and env subprocess calls - #20147

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/subprocess-encoding-diagnostics-env
Oct 29, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/subprocess-encoding-diagnostics-env

Conversation

@javenciu

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The support-bundle kiro-cli version probe and the mise environment activation decode their child's output as UTF-8 on every platform instead of through the host code page.

Two text-mode subprocess.run calls pass text=True and no encoding=: _kiro_cli_version in src/kiro_crew/diagnostics.py (the call is at src/kiro_crew/diagnostics.py:976 on 70d2fa6) and activate_mise in src/kiro_crew/env.py (the call is at src/kiro_crew/env.py:1615 on 70d2fa6). With no encoding the pipes are decoded with locale.getpreferredencoding(False): UTF-8 on POSIX, the legacy ANSI code page on Windows. #5249 added scripts/check_subprocess_encoding.py so this class stops growing; these two calls are among the sites still grandfathered in .github/subprocess-encoding-baseline.txt (1 src/kiro_crew/diagnostics.py and 1 src/kiro_crew/env.py).

The children are kiro-cli --version and mise env --json. The mise output is JSON and carries PATH plus any tool-provided variable, so a non-ASCII character in a home directory is decoded through the host code page: mojibake on Windows, or a UnicodeDecodeError under a code page that rejects the bytes. A UnicodeDecodeError is a ValueError; the except (OSError, subprocess.SubprocessError) clause around each call does not catch it.

Why it matters

activate_mise runs once at gateway start, and its docstring promises that a failing mise never stops the gateway. The call at src/kiro_crew/cli_server.py:2972 on 70d2fa6 has no try around it, so a decode error that escapes activate_mise propagates out of the start-up function instead of becoming a skipped activation, and a PATH entry decoded as mojibake names a directory that does not exist. _kiro_cli_version is called by _versions_text (src/kiro_crew/diagnostics.py:1053 on 70d2fa6, the kiro_cli_version: line of versions.txt) and by _issue_url (src/kiro_crew/diagnostics.py:1140 on 70d2fa6, the kiro-cli: line of the pre-filled issue). Neither call has a try, although the function's docstring says diagnostics must degrade like a failed spawn rather than raise.

Not a goal

  • No change to any other subprocess call and no other baseline entry. src/kiro_crew/diagnostics.py has this one subprocess call. src/kiro_crew/env.py has two others: the bash call at src/kiro_crew/env.py:792 captures bytes (no text mode), and the git helper _run (src/kiro_crew/env.py:834) already splats **UTF8_TEXT. Neither is a sibling of this bug.
  • No change to the argv, timeouts, cwd or env of either call, to how the mise binary is found, or to what the version line or the mise variables contain.
  • No new test file: the subprocess-encoding ratchet is the regression test for the diagnostics call, and the existing mise test is updated to pin the new keywords (below).

What changed (motivation → approach → change)

  • Symptom: the output of kiro-cli --version and mise env --json is decoded with the host code page, so non-ASCII text is garbled on Windows or raises.
  • Root cause: both calls pass text=True and no encoding=, which defers to the locale.
  • Change in src/kiro_crew/diagnostics.py: add from kiro_crew.subprocess_utf8 import UTF8_TEXT, and the call's text=True becomes **UTF8_TEXT. The mapping is {"text": True, "encoding": "utf-8", "errors": "replace"}, so the explicit text=True goes (passing both is a duplicate keyword) and a malformed byte costs one U+FFFD instead of raising. The same child's other probe, kiro_cli_version_at (src/kiro_crew/kiro_cli.py:270 on 70d2fa6), already runs [binary, "--version"] with **UTF8_TEXT.
  • Change in src/kiro_crew/env.py: UTF8_TEXT is already imported (src/kiro_crew/env.py:22 on 70d2fa6) and used by the git helper, so the call's text=True becomes **UTF8_TEXT with no new import.
  • Test: test_activation_uses_homebrew_binary_from_bare_path in test/test_env_mise.py asserts the exact keyword arguments of the mise call (text=True among them). It now asserts **UTF8_TEXT, so it pins the encoding instead of only the mode.
  • Ratchet: delete 1 src/kiro_crew/diagnostics.py and 1 src/kiro_crew/env.py from .github/subprocess-encoding-baseline.txt, so an unpinned call in either file now fails the gate instead of being grandfathered.
  • Rejected: a literal encoding="utf-8" on each call. src/ pins children whose output encoding is known through the one shared mapping in src/kiro_crew/subprocess_utf8.py, which keeps the errors= policy in one place.

Backwards compatibility

Compatible: argv, timeout, cwd and env of both calls are unchanged and stdout / stderr are still str. On a UTF-8 locale valid output decodes to the same text (the control run below prints the same strings before and after); the one difference is that an undecodable byte becomes U+FFFD instead of raising UnicodeDecodeError.

Removes nothing: the explicit text=True is carried by the mapping, and the two deleted baseline lines are the files' grandfathering, which is the point of the change.

Tests

In short: the subprocess-encoding ratchet goes red when either baseline line is deleted from an unfixed file and green with the fix; the mise test now pins the encoding (red against the old source, green with the fix); the eight test files that exercise these two functions or the gate keep the same pass count.

  • Baseline lines deleted, calls untouched, on origin/main (RATCHET_SCOPE_WHOLE_TREE=1 python3 scripts/check_subprocess_encoding.py, exit 1; the variable is needed because the default scope is the committed diff, which names no changed file on an unmodified checkout). Deleting both lines:

    ::error file=src/kiro_crew/diagnostics.py::text-mode subprocess call without encoding= (decodes with the Windows ANSI code page). Pin it with encoding="utf-8" / **UTF8_TEXT (src/kiro_crew/subprocess_utf8.py), or mark deliberate locale decoding with `# subprocess-encoding: locale`.
      src/kiro_crew/diagnostics.py:976
    ::error file=src/kiro_crew/env.py::text-mode subprocess call without encoding= (decodes with the Windows ANSI code page). Pin it with encoding="utf-8" / **UTF8_TEXT (src/kiro_crew/subprocess_utf8.py), or mark deliberate locale decoding with `# subprocess-encoding: locale`.
      src/kiro_crew/env.py:1615
    
    subprocess-encoding gate FAILED: 2 new offender(s), 0 grown count(s), 0 file(s) with new calls on added lines, 0 entr(y/ies) to prune.
    

    Deleting only the diagnostics line reports 1 new offender(s) at src/kiro_crew/diagnostics.py:976; deleting only the env line reports 1 new offender(s) at src/kiro_crew/env.py:1615.

  • On this branch: python3 scripts/check_subprocess_encoding.py --test exit 0, python3 scripts/check_subprocess_encoding.py exit 0 (scope: the 4 changed files), and the whole-tree run exit 0. --update-baseline on a copy of the baseline leaves it byte-identical, so no further prune is owed.

  • test/test_env_mise.py, with the source fixed and the test not yet updated: test_activation_uses_homebrew_binary_from_bare_path[arm] and [intel] fail with AssertionError: expected call not found. (the old text=True expectation). With the test updated, the file passes (15 passed, the same count as on origin/main). Mutation: env.py restored to its origin/main bytes with the updated test gives 2 failed, 13 passed, so the assertion now pins the encoding.

  • pytest -n0 over test/test_diagnostics.py, test/test_kiro_cli_pin.py, test/test_env_mise.py, test/test_env.py, test/test_stderr_redact_then_tail.py, test/test_spawn_audit.py, test/test_subprocess_encoding_gate.py and test/test_ci_surface_tests.py: 381 passed on origin/main and 381 passed on this branch. The diagnostics tests replace subprocess.run with callables that take **kw, so they accept the new keywords and do not exercise the decode; the next section does.

  • flake8, isort --check-only and black --check on the three touched Python files, GITHUB_BASE_REF=main python3 scripts/check_black_formatting.py, and scripts/check_comment_history.py: all pass.

Manual verification

A stand-in kiro-cli and a stand-in mise print UTF-8 bytes for café. The snippet below runs them through the real _kiro_cli_version and activate_mise and the real subprocess.run, replacing only pin_kiro_cli and _mise_bin (the two lookups the tests already patch). Run from the repo root as LC_ALL=C PYTHONCOERCECLOCALE=0 PYTHONUTF8=0 PYTHONPATH=src python repro.py; the ASCII locale stands in for a code page that cannot decode the bytes and makes the failure deterministic on a POSIX host. Three runs on each checkout:

  • origin/main: kiro-cli version raised UnicodeDecodeError and mise raised UnicodeDecodeError, both escaping the function.
  • this branch: kiro-cli version: 'kiro-cli caf\xe9 1.0' and mise: ['PROBE'] 'caf\xe9'.
  • Control: with a UTF-8 locale (LC_ALL=C.UTF-8) both checkouts print the same two lines as this branch, so the difference comes from the locale and not from the stand-ins.

No Windows host was used, and the real kiro-cli and mise binaries were not run; the stand-ins exercise the same calls and the same decode.

repro.py
import os
import tempfile
from pathlib import Path

from kiro_crew import diagnostics, env

tmp = Path(tempfile.mkdtemp())
kiro = tmp / "fake-kiro-cli"
kiro.write_text("#!/bin/sh\nprintf 'kiro-cli caf\\303\\251 1.0\\n'\n", encoding="utf-8")
mise = tmp / "fake-mise"
mise.write_text("#!/bin/sh\nprintf '{\"PROBE\": \"caf\\303\\251\"}\\n'\n", encoding="utf-8")
for script in (kiro, mise):
    script.chmod(0o755)

diagnostics.pin_kiro_cli = lambda: (str(kiro), False)
env._mise_bin = lambda: str(mise)

try:
    print("kiro-cli version:", ascii(diagnostics._kiro_cli_version()))
except Exception as exc:
    print("kiro-cli version raised", type(exc).__name__)

target = {"PATH": os.environ["PATH"]}
try:
    print("mise:", env.activate_mise(target), ascii(target.get("PROBE")))
except Exception as exc:
    print("mise raised", type(exc).__name__)

Screenshots / video

N/A — no user-visible UI change.

Related Issues

No issue exists; burn-down of the subprocess-encoding baseline (precedents: the gate's own PR #5378 for #5249, and the merged per-site pins #14648 and #14664).

Pattern harvest

Rule candidate: lint (already in place: scripts/check_subprocess_encoding.py, #5378)
Pattern: a text-mode subprocess call without encoding= defers to the host code page; the gate stops new ones, and this PR retires two grandfathered entries from its baseline.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 14, 2026 05:48
@javenciu
javenciu requested a review from mbriones98 October 14, 2026 05:48
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@javenciu
javenciu force-pushed the fix/subprocess-encoding-diagnostics-env branch from a3f9c2e to cec257e Compare October 14, 2026 11:09
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision b9c69777b4be664bbd97bd62aabf9418d8aab1f9 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of b9c69777b4be664bbd97bd62aabf9418d8aab1f9 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. This removes the two grandfathered sites the check_subprocess_encoding.py ratchet already names, using the shared UTF8_TEXT mapping that kiro_cli.py:281 already uses for the same --version child.

What this change ships

Inventory (4 items) — 4 justified

Intent: the support bundle's kiro-cli version line and the mise PATH activation at gateway start read their child's output as UTF-8 on every OS, so non-ASCII text is never garbled or raised on. This is a FIX.

  1. The support bundle's kiro_cli_version line and the issue link decode kiro-cli --version as UTF-8, and a bad byte becomes U+FFFD instead of raising — justified
  2. Mise activation at gateway start decodes mise env --json as UTF-8, and a bad byte can no longer escape activation as a decode error — justified
  3. An unpinned text-mode call in diagnostics.py or env.py now fails the encoding gate instead of being grandfathered — justified
  4. The mise test pins the encoding keywords, not just text mode — justified

[FIRST-PRINCIPLES-REVIEWED] b9c6977

Description read

[DESCRIPTION-READ] db7c25a88bff22f9701c13bc69c45fa36b47074e46b776a5c1f3e14dfee9c705

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of b9c69777b4be664bbd97bd62aabf9418d8aab1f9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix routes both calls through the shared UTF8_TEXT mapping that sibling callers already use, and only removes these two files' baseline lines.

[DESIGN-REVIEWED] b9c6977

Description read

[DESCRIPTION-READ] 0ae57d5972cc7ccb63679709a79387ff126fc4e4f48f764e9e3008ad08d9bcf0

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b9c69777b4be664bbd97bd62aabf9418d8aab1f9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] b9c6977

Verdict parsed from the review's SHA-scoped output markers for commit b9c69777b4be664bbd97bd62aabf9418d8aab1f9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b9c69777b4be664bbd97bd62aabf9418d8aab1f9 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b9c69777b4be664bbd97bd62aabf9418d8aab1f9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b9c6977

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b9c69777b4be664bbd97bd62aabf9418d8aab1f9 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@javenciu
javenciu force-pushed the fix/subprocess-encoding-diagnostics-env branch from cec257e to 1b7d181 Compare October 16, 2026 02:15
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 16, 2026
@javenciu javenciu reopened this Oct 16, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 16, 2026
@javenciu
javenciu enabled auto-merge (squash) October 16, 2026 11:41
@javenciu

javenciu commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author
  1. Design PASS. Nothing to check: this PR changes only how two subprocess calls decode their output (src/kiro_crew/diagnostics.py:977, src/kiro_crew/env.py:1615) and has no UI surface. The encoding self-test and gate pass, RC=0.

  2. First Principles PASS / the rest of the baseline. src/kiro_crew/cli_server.py, which holds the gateway start function, is not in .github/subprocess-encoding-baseline.txt, so the encoding gate already checks every subprocess call it makes directly. One helper it calls at start lives in a baselined file: the node probe _node_ok (src/kiro_crew/cli.py:322, called at src/kiro_crew/cli_server.py:3091), whose subprocess.run decodes node -v with the locale encoding (src/kiro_crew/cli.py:334-348). Neither failure this PR fixes can happen there. The version line it reads is ASCII, which decodes the same under every Windows code page, and the call sits inside except Exception: return False (src/kiro_crew/cli.py:355-356), so a decode error cannot escape gateway start. Non-ASCII text on the call's stderr would make the probe answer False, and the start would then try to repair node; it would not crash. I left the probe unchanged because it is outside this PR's two calls. I did not trace every other baselined file for a call reachable from gateway start, so this is not a claim that none exists.

This was referenced Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants