Repository navigation
Use UTF-8 for JSON command-line file I/O - #238
Merged
stefankoegl merged 1 commit intoOct 24, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused changes cover all intended file I/O paths with targeted regression tests and no identified blocking issues.
No open issues 🎉
What changed in this PR
Makes JSON command-line file I/O consistently use UTF-8, preventing locale-dependent Unicode corruption while preserving existing standard input and output encodings.
Changes:
- Explicitly uses UTF-8 for file arguments and both in-place output paths.
- Adds regression tests for UTF-8 inputs and Unicode output with Python’s UTF-8 mode disabled.
- Documents the file-encoding behavior.
| File | Description |
|---|---|
| tests.py | Adds encoding regression coverage and updates subprocess and output-mocking helpers. |
| jsonpatch_cli.py | Sets UTF-8 explicitly for JSON file reads and in-place writes. |
| doc/commandline.rst | Documents UTF-8 file input and in-place output. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On non-UTF-8 locales,
jsondiffandjsonpatchread UTF-8 JSON files usingthe system encoding, which can silently change Unicode values. In-place
jsonpatch -i -ucan also raiseUnicodeEncodeErrorwhen writing charactersoutside that encoding, even if the input files contain only ASCII escapes.
Explicitly use UTF-8 for JSON file arguments and both in-place output paths,
and document the file encoding. Standard input and standard output continue
to use their existing stream encodings.
The regression tests disable Python's UTF-8 mode and locale coercion. They
cover both programs' UTF-8 inputs and Unicode in-place output through the
atomic and fallback paths. All four cases fail against the original source
on a GBK Windows system.
Validation: Python 3.10 and 3.13 full unit suites each pass 217 tests with one
POSIX-only skip; all 20 Python 3.13 property tests pass. Critical Flake8
checks and
git diff --checkpass.AI assistance: OpenAI Codex prepared the change, regression tests, and this description.