Repository navigation
fix(core): pin utf-8 on the diagnostics and env subprocess calls - #20147
Conversation
|
Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683). |
a3f9c2e to
cec257e
Compare
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. This removes the two grandfathered sites the What this change shipsInventory (4 items) — 4 justifiedIntent: the support bundle's kiro-cli version line and the mise PATH activation at gateway start read their child's output as UTF-8 on every OS, so non-ASCII text is never garbled or raised on. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] b9c6977 Description read[DESCRIPTION-READ] db7c25a88bff22f9701c13bc69c45fa36b47074e46b776a5c1f3e14dfee9c705 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the fix routes both calls through the shared [DESIGN-REVIEWED] b9c6977 Description read[DESCRIPTION-READ] 0ae57d5972cc7ccb63679709a79387ff126fc4e4f48f764e9e3008ad08d9bcf0 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683). |
cec257e to
1b7d181
Compare
|
Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683). |
|
Problem / Motivation
Goal: The support-bundle kiro-cli version probe and the mise environment activation decode their child's output as UTF-8 on every platform instead of through the host code page.
Two text-mode
subprocess.runcalls passtext=Trueand noencoding=:_kiro_cli_versioninsrc/kiro_crew/diagnostics.py(the call is atsrc/kiro_crew/diagnostics.py:976on 70d2fa6) andactivate_miseinsrc/kiro_crew/env.py(the call is atsrc/kiro_crew/env.py:1615on 70d2fa6). With no encoding the pipes are decoded withlocale.getpreferredencoding(False): UTF-8 on POSIX, the legacy ANSI code page on Windows. #5249 addedscripts/check_subprocess_encoding.pyso this class stops growing; these two calls are among the sites still grandfathered in.github/subprocess-encoding-baseline.txt(1 src/kiro_crew/diagnostics.pyand1 src/kiro_crew/env.py).The children are
kiro-cli --versionandmise env --json. The mise output is JSON and carriesPATHplus any tool-provided variable, so a non-ASCII character in a home directory is decoded through the host code page: mojibake on Windows, or aUnicodeDecodeErrorunder a code page that rejects the bytes. AUnicodeDecodeErroris aValueError; theexcept (OSError, subprocess.SubprocessError)clause around each call does not catch it.Why it matters
activate_miseruns once at gateway start, and its docstring promises that a failing mise never stops the gateway. The call atsrc/kiro_crew/cli_server.py:2972on 70d2fa6 has notryaround it, so a decode error that escapesactivate_misepropagates out of the start-up function instead of becoming a skipped activation, and aPATHentry decoded as mojibake names a directory that does not exist._kiro_cli_versionis called by_versions_text(src/kiro_crew/diagnostics.py:1053on 70d2fa6, thekiro_cli_version:line ofversions.txt) and by_issue_url(src/kiro_crew/diagnostics.py:1140on 70d2fa6, thekiro-cli:line of the pre-filled issue). Neither call has atry, although the function's docstring says diagnostics must degrade like a failed spawn rather than raise.Not a goal
src/kiro_crew/diagnostics.pyhas this onesubprocesscall.src/kiro_crew/env.pyhas two others: thebashcall atsrc/kiro_crew/env.py:792captures bytes (no text mode), and the git helper_run(src/kiro_crew/env.py:834) already splats**UTF8_TEXT. Neither is a sibling of this bug.cwdorenvof either call, to how the mise binary is found, or to what the version line or the mise variables contain.What changed (motivation → approach → change)
kiro-cli --versionandmise env --jsonis decoded with the host code page, so non-ASCII text is garbled on Windows or raises.text=Trueand noencoding=, which defers to the locale.src/kiro_crew/diagnostics.py: addfrom kiro_crew.subprocess_utf8 import UTF8_TEXT, and the call'stext=Truebecomes**UTF8_TEXT. The mapping is{"text": True, "encoding": "utf-8", "errors": "replace"}, so the explicittext=Truegoes (passing both is a duplicate keyword) and a malformed byte costs one U+FFFD instead of raising. The same child's other probe,kiro_cli_version_at(src/kiro_crew/kiro_cli.py:270on 70d2fa6), already runs[binary, "--version"]with**UTF8_TEXT.src/kiro_crew/env.py:UTF8_TEXTis already imported (src/kiro_crew/env.py:22on 70d2fa6) and used by the git helper, so the call'stext=Truebecomes**UTF8_TEXTwith no new import.test_activation_uses_homebrew_binary_from_bare_pathintest/test_env_mise.pyasserts the exact keyword arguments of the mise call (text=Trueamong them). It now asserts**UTF8_TEXT, so it pins the encoding instead of only the mode.1 src/kiro_crew/diagnostics.pyand1 src/kiro_crew/env.pyfrom.github/subprocess-encoding-baseline.txt, so an unpinned call in either file now fails the gate instead of being grandfathered.encoding="utf-8"on each call.src/pins children whose output encoding is known through the one shared mapping insrc/kiro_crew/subprocess_utf8.py, which keeps theerrors=policy in one place.Backwards compatibility
Compatible: argv, timeout,
cwdandenvof both calls are unchanged andstdout/stderrare stillstr. On a UTF-8 locale valid output decodes to the same text (the control run below prints the same strings before and after); the one difference is that an undecodable byte becomes U+FFFD instead of raisingUnicodeDecodeError.Removes nothing: the explicit
text=Trueis carried by the mapping, and the two deleted baseline lines are the files' grandfathering, which is the point of the change.Tests
In short: the subprocess-encoding ratchet goes red when either baseline line is deleted from an unfixed file and green with the fix; the mise test now pins the encoding (red against the old source, green with the fix); the eight test files that exercise these two functions or the gate keep the same pass count.
Baseline lines deleted, calls untouched, on
origin/main(RATCHET_SCOPE_WHOLE_TREE=1 python3 scripts/check_subprocess_encoding.py, exit 1; the variable is needed because the default scope is the committed diff, which names no changed file on an unmodified checkout). Deleting both lines:Deleting only the diagnostics line reports
1 new offender(s)atsrc/kiro_crew/diagnostics.py:976; deleting only the env line reports1 new offender(s)atsrc/kiro_crew/env.py:1615.On this branch:
python3 scripts/check_subprocess_encoding.py --testexit 0,python3 scripts/check_subprocess_encoding.pyexit 0 (scope: the 4 changed files), and the whole-tree run exit 0.--update-baselineon a copy of the baseline leaves it byte-identical, so no further prune is owed.test/test_env_mise.py, with the source fixed and the test not yet updated:test_activation_uses_homebrew_binary_from_bare_path[arm]and[intel]fail withAssertionError: expected call not found.(the oldtext=Trueexpectation). With the test updated, the file passes (15 passed, the same count as onorigin/main). Mutation:env.pyrestored to itsorigin/mainbytes with the updated test gives 2 failed, 13 passed, so the assertion now pins the encoding.pytest -n0overtest/test_diagnostics.py,test/test_kiro_cli_pin.py,test/test_env_mise.py,test/test_env.py,test/test_stderr_redact_then_tail.py,test/test_spawn_audit.py,test/test_subprocess_encoding_gate.pyandtest/test_ci_surface_tests.py: 381 passed onorigin/mainand 381 passed on this branch. The diagnostics tests replacesubprocess.runwith callables that take**kw, so they accept the new keywords and do not exercise the decode; the next section does.flake8,
isort --check-onlyandblack --checkon the three touched Python files,GITHUB_BASE_REF=main python3 scripts/check_black_formatting.py, andscripts/check_comment_history.py: all pass.Manual verification
A stand-in
kiro-cliand a stand-inmiseprint UTF-8 bytes forcafé. The snippet below runs them through the real_kiro_cli_versionandactivate_miseand the realsubprocess.run, replacing onlypin_kiro_cliand_mise_bin(the two lookups the tests already patch). Run from the repo root asLC_ALL=C PYTHONCOERCECLOCALE=0 PYTHONUTF8=0 PYTHONPATH=src python repro.py; the ASCII locale stands in for a code page that cannot decode the bytes and makes the failure deterministic on a POSIX host. Three runs on each checkout:origin/main:kiro-cli version raised UnicodeDecodeErrorandmise raised UnicodeDecodeError, both escaping the function.kiro-cli version: 'kiro-cli caf\xe9 1.0'andmise: ['PROBE'] 'caf\xe9'.LC_ALL=C.UTF-8) both checkouts print the same two lines as this branch, so the difference comes from the locale and not from the stand-ins.No Windows host was used, and the real
kiro-cliandmisebinaries were not run; the stand-ins exercise the same calls and the same decode.repro.py
Screenshots / video
N/A — no user-visible UI change.
Related Issues
No issue exists; burn-down of the subprocess-encoding baseline (precedents: the gate's own PR #5378 for #5249, and the merged per-site pins #14648 and #14664).
Pattern harvest
Rule candidate: lint (already in place:
scripts/check_subprocess_encoding.py, #5378)Pattern: a text-mode subprocess call without
encoding=defers to the host code page; the gate stops new ones, and this PR retires two grandfathered entries from its baseline.Checklist