Repository navigation
test: read setup.cfg as utf-8 in the ci-surface testpaths guard - #14593
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] 3962c9d |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNothing blocks; one advisory finding. FINDING — test/test_ci_surface_tests.py:162 — [OPUS-REVIEWED] 3962c9d Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
Disposition for
|
`ConfigParser.read` opens with no encoding, so it decodes with
`locale.getpreferredencoding()` -- UTF-8 on POSIX, the legacy ANSI code
page on Windows. `setup.cfg` carries 48 bytes of non-ASCII (em dashes in
its comments), so on a Windows host whose code page is not UTF-8 the read
raises before the test reaches its own assertion:
UnicodeDecodeError: 'cp950' codec can't decode byte 0xe2 in
position 514: illegal multibyte sequence
Two effects. The suite cannot run clean on such a host, and -- worse --
`test_backend_roots_cover_every_configured_testpath` is the guard that
every `setup.cfg` testpath is a scanned selector root, i.e. the guard
against tests silently never running. On the affected hosts it is itself
silently not running.
Same failure class `scripts/check_subprocess_encoding.py` was added for
(#3219, #3669, #5249), reached through a file read rather than a
subprocess pipe, which that gate does not cover.
Pins the encoding at the read, and adds a test asserting the two facts
that make it load-bearing -- setup.cfg really carries non-ASCII, and those
bytes really are undecodable under a legacy code page -- because a UTF-8
CI runner passes either way and cannot exercise the bug itself.
Deliberately not a repo-wide missing-encoding rule: cross-platform.yml
documents why it ships none, and an AST version means editing
.github/workflows/**. The fixture string at lines 280-282 that spells the
same call is left byte-identical; it is written to a temp file for the
surface selector to match, never executed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
b026fb0 to
3962c9d
Compare
|
Rebased: main moved (50018eb -> cd783d2) and #14578 dropped |
|
Disposition for
|
Pull Request is not mergeable
bolichen97
left a comment
There was a problem hiding this comment.
One-line encoding pin plus a premise-guarding test (non-ASCII present, cp950 raises, utf-8 reads). 67 passed locally. LGTM.
Problem / Motivation
Goal: Make the CI-surface testpaths guard run on every host, so it cannot fail silently on a Windows machine whose code page is not UTF-8.
One test reads
setup.cfgwithout naming an encoding.setup.cfgholds a few non-ASCII bytes (em dashes in its comments). On a host whose default code page is not UTF-8, that read crashes before the test checks anything.Why it matters
That test is the guard that every
testpathsentry insetup.cfgis a scanned selector root -- the thing that catches tests silently never running. On the affected hosts the guard is itself silently not running.Not a goal
encoding=" rule. That belongs to a workflow gate, not this one read.What changed (motivation → approach → change)
A
ConfigParser.readwith no encoding decodes with the host's default code page. On Windows that can be a legacy code page, andsetup.cfg's em dashes are undecodable there, so the read raisesUnicodeDecodeErrorand the guard never reaches its own assertion.The fix pins
encoding="utf-8"on that one read.A new test locks in why the encoding is needed, because a UTF-8 CI runner passes either way and cannot show the bug: it asserts
setup.cfgreally carries non-ASCII bytes, that those bytes really do raise under a legacy code page (cp950), and that the pinned UTF-8 read works.Backwards compatibility
Compatible: adds an explicit encoding to a read that already assumed UTF-8 on the CI hosts; nothing that passes today changes.
Tests
test_setup_cfg_carries_non_ascii_so_its_read_must_pin_utf8(new): asserts the premise (setup.cfghas non-ASCII), that the bytes are undecodable undercp950, and that the UTF-8 read succeeds and yields testpaths.test_backend_roots_cover_every_configured_testpath(updated): now readssetup.cfgas UTF-8.Manual verification
N/A — unit coverage is sufficient; the added test exercises the exact failure and fix on every host.
Related Issues
no linked issue: this supersedes #9243 (fork PR by @leonlaiyc); it tracks no separate issue.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Supersedes #9243. Co-authored with @leonlaiyc (original author).