Repository navigation
Conversation
…ost codepage `_git_status` runs `git status --porcelain=1 -z`. The `-z` form is the one porcelain shape that does NOT C-quote a path: verified against real git, the default form returns `?? "caf\303\251-\346\227\245....txt"` (pure ASCII whatever the filename is) while `-z` returns the same path verbatim as raw UTF-8 bytes. Both git calls there passed `text=True` with no `encoding=`, so those bytes were decoded with `locale.getpreferredencoding(False)`. On a Windows console codepage that is cp950 / cp932 / cp1252, not UTF-8, and one file with a non-ASCII name took out the whole repository's status: `UnicodeDecodeError` is neither `OSError` nor `TimeoutExpired`, so it escaped both `except` clauses in `_git_status` and the endpoint 500ed. Measured on a live cp950 host -- `'cp950' codec can't decode byte 0xe6`. Where the codepage happens to be lenient the failure is quieter and worse: cp1252 maps every byte to something, so the path is silently mojibaked and never matches its tree entry. `_search_rg` has the same defect with a different producer: ripgrep's `--json` stream is UTF-8 by definition and carries both the matched path and the matched line, so non-ASCII is the norm there rather than the exception. Its `except` clause is also `OSError` / `TimeoutExpired` only, so a decode failure does not degrade to the `_search_python` fallback the way a spawn failure does -- it 500s the search. Name the encoding on all three spawns, which is what `platform/update_capability.py` already does on its own git call for this reason. Refnames are UTF-8 too, so the branch call gets the same treatment. That takes this file off the subprocess-encoding baseline entirely (3 -> 0). The regressions drive the real functions with git's `-z` bytes and ripgrep's own JSON through a `run_limited` stand-in that decodes the way `subprocess` documents -- the `encoding=` keyword when given, the locale codec otherwise -- so the interesting codepage is a parameter rather than a property of the runner. cp950 and ascii cover the raising case from both directions, cp1252 covers the silent-mojibake case (which is why `errors=` would not be a fix), and an ASCII path pins that the ordinary case is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
d74b265 to
7332bed
Compare
Open PR relationship auditThis is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion. Relationship findings
No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit. |
|
🤖 Kiro Crew [operator: NicholasRBowers#a942f9ca]: This PR has been inactive for 7+ days. I reviewed the blockers but they require your input:
When you've addressed these, the pipeline will re-assess on its next cycle. |
|
Closing as superseded.
The branch has been in merge conflict and idle 8+ days, and the only residual delta is the regression test plus a If you want to keep the coverage, refile just the regression test rebased onto |
Problem / Motivation
file_explorer's_git_statusrunswith
text=Trueand noencoding=, so git's bytes are decoded withlocale.getpreferredencoding(False).-zis what makes that unsafe, and it is worth being exact about. Measured againstreal git on a repo containing one untracked
café-日本.txt:The default form is ASCII whatever the filename is.
-zdeliberately turns thatoff. So on any host whose preferred encoding is not UTF-8 — which is every Windows
console codepage — the handler decodes UTF-8 bytes with cp950 / cp932 / cp1252.
The same call is made for
rev-parse --abbrev-ref HEAD, and git refnames are UTF-8too.
Why it matters
One file with a non-ASCII name takes out the whole repository's status.
UnicodeDecodeErroris neither anOSErrornor asubprocess.TimeoutExpired, soit escapes both
exceptclauses in_git_statusand the endpoint 500s — there isno
status_errorfallback for it. Measured on a live cp950 host:Where the codepage is lenient the failure is quieter and worse. cp1252 maps
every byte to something, so nothing raises: the path comes back mojibaked
(
café-日本.txt→café-日本.txt), the endpoint returns 200, and that keysimply never matches its tree entry. The file silently shows no git status at all.
This is not a hypothetical class either — the repository ships a CI gate for it,
scripts/check_subprocess_encoding.py, withfile_explorer/server.pyalready onits baseline at 3 calls.
What changed (motivation → approach → change)
Symptom: non-ASCII paths crash or corrupt git status. Root cause: git's UTF-8 output
decoded with the host codepage. Fix: name the encoding, on both git calls in
_git_status.run_limitedforwards every keyword tosubprocess.rununtouched, so this needs nonew plumbing. It is also not a new convention:
platform/update_capability.pyalready pins
encoding="utf-8"on its own git call, with a comment naming non-ASCIIpaths as the reason.
errors=is deliberately not added.errors="replace"would convert the crashinto the silent-mojibake case, which the tests below treat as equally wrong; UTF-8 is
what git actually writes, so the right move is to say so.
The subprocess-encoding baseline goes from
3to0for this file, the way thegate asks (
--update-baseline), so its row is pruned rather than lowered.The third spawn in this file is the same defect, and is fixed here too.
_search_rgruns ripgrep with--json, a stream that is UTF-8 by definition andthat carries both the matched path and the matched line — so non-ASCII is the norm
there, not the exception. Its
exceptclause is alsoOSError/TimeoutExpiredonly, which means a decode failure does not degrade to the
_search_pythonfallbackthe way a spawn failure does: it 500s the search.
Contention was checked before folding it in rather than assumed. Open PR #4650
rewrites
_search(itsserver.pyhunk is anchored there and adds ~170 lines) andleaves
def _search_rgas unchanged context, so the one line changed here is not aline that PR touches.
With all three pinned, this file leaves
.github/subprocess-encoding-baseline.txtentirely — the diff to that file is the removal of its row, not a lowered count.
Tests
New file
test/test_file_explorer_git_status_encoding.py— deliberately a new filerather than an addition to
test_file_explorer_app.py, which #4650 also edits.The harness feeds
_git_statusgit's real-zbytes through arun_limitedstand-in that decodes the way
subprocessdocuments: theencoding=keyword whenone is given,
locale.getpreferredencoding(False)otherwise. That makes theinteresting codepage a test parameter instead of a property of the runner, so these
are deterministic on a UTF-8 Linux CI box and on a cp950 laptop alike — no console
codepage to switch, no timing, no skip.
..._a_non_ascii_path_survives_a_non_utf8_host_codepage[cp950]/[ascii]LC_ALL=Cone. Also asserts nostatus_error/branch_errorwas set and that both git calls ran, so a regression cannot pass as an empty-but-successful result..._a_locale_that_silently_mojibakes_is_also_refusederrors="replace"would leave in place, and asserting on the path itself catches both failure modes with one assertion..._an_ascii_path_is_unaffected..._a_ripgrep_match_survives_a_non_utf8_host_codepage--jsonrecord. Asserts the decoded path and the matched line survive exactly, so a lenient codepage's mojibake fails it as loudly as a raising oneAn autouse fixture stubs
wrap_argvandcgroup_scope_argvto identity. Neithertouches decoding;
wrap_argvrefuses outright on a host with no OS sandbox backend(every Windows runner), so stubbing them keeps the test measuring one thing and keeps
it portable. The same idiom is already used in
test_file_explorer_app.py.Red-before, with
server.pyrestored to currentorigin/main(7308a8b30) andnothing else changed — 3 failed, 1 passed, and the three failures are the three
distinct mechanisms:
The one that passed is the ASCII over-fix guard, which must pass on both sides.
The ripgrep case has its own red-before against the same tree:
Green after: the new file plus
test/test_file_explorer_app.py— 100 passed,2 skipped.
Gates on the touched files:
scripts/check_subprocess_encoding.pypassed(239 known calls in 117 files remain, down from 242 in 118),
scripts/check_black_formatting.pypassed in scope,
flake8andisort --check-onlyclean,git diff --checkclean, and
mypy src/kiro_crew/apps/builtins/file_explorer/server.py— Success: noissues found in 1 source file. No repository-wide suite; server CI owns that.
Manual verification
N/A — unit coverage sufficient: the defect is a decoding choice, and the tests drive
the real
_git_statuswith the exact bytes git emits under-z, which is closer tothe failure than opening the panel on one machine would be.
Related Issues
None. Found by auditing spawns that ask a tool for raw bytes and then decode with the
host locale, against
scripts/check_subprocess_encoding.py's own baseline.Pattern harvest
Rule candidate: review-prompt
Pattern: a flag that turns OFF a tool's ASCII-escaping (
-z,--null,core.quotePath=false,--porcelainvariants) added to a call that decodes with thehost locale.
The generalizable question is narrower and more useful than "does this call pass
encoding=": did someone ask the tool for raw bytes, and is anything downstreamstill assuming ASCII?
-zandcore.quotePath=falseexist precisely to remove theescaping that was making the locale codec harmless, so adding one silently converts a
latent issue into a live one — and the failure lands only on hosts whose codepage is
not UTF-8, which is nobody on the CI matrix.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement