hygiene: add an escapes mode — LaTeX eaten by Python string escapes - #245
Merged
Conversation
Turns the sweep built for the 41-file raw-stringing task into a standing
Hygiene check, so the debt is caught continuously instead of re-derived.
The mode reports TWO independent classes, which is the whole point:
warned `\s`, `\l`, `\[` — escapes Python does NOT recognise. It keeps them
literal but warns on every compile, and they are slated to become a
SyntaxError.
silent `\t` in `\theta`, `\f` in `\frac`, `\r` in `\rm`, `\b` in `\beta` —
escapes it DOES recognise. The value is corrupted and NOTHING is
emitted. A warning-only sweep reports a clean repo over a corrupted
one, so files with ONLY silent damage are marked explicitly.
Two interpreter traps are handled, both of which have already produced a false
"clean" in this codebase:
1. Invalid escapes are a SyntaxWarning only on Python 3.12+; on 3.11 they are a
DeprecationWarning. Both categories are collected, so the scan is
interpreter-independent rather than silently returning zero on 3.11.
2. compileall needs -f or __pycache__ suppresses recompilation and the counts
drop. This helper compiles from source text and never consults a cache.
Shape follows the existing helper-backed modes exactly: a stdlib-only,
read-only `_hygiene_escapes.py` with --summary/--json-row/human output, wired
into MODE_ORDER, the delegate and kind maps, prescan dispatch, the arg parser,
the json-row path, the human body and the default ranked worklist. Delegates to
/refactor; Hygiene still never edits source.
Validated against a known answer: run over the six workspace repos immediately
after the raw-stringing merges, it reports exactly the one documented residue
(autolens_workspace scripts/group/likelihood_function.py, 2 warned + 1 silent)
and nothing else. A synthetic corpus confirms it flags a silent-only file,
flags a warned-only file, and correctly ignores both an already-raw docstring
and a deliberate `print("\nreal newline")`.
CI caught the registry drift guard: tests/test_hygiene_conductor.py pins the
exact mode set, and adding `escapes` without updating MODES failed it. That
guard is doing its job — a new mode must be declared, not just wired.
Adds `escapes` to MODES and its kind assertion, plus five tests for the mode
itself, matching the fixture style of the other helper-backed modes:
- warned and silent classes reported separately, per file
- an already-raw `r\"\"\"` docstring and a deliberate `print("\nreal newline")`
are NOT flagged — raw-ifying that newline would be the regression
- the summary marks files a warning-only sweep would miss (the whole point of
the mode)
- human output names the silent-only file and states the fix is the `r` prefix
- a clean repo reports status=clean, count=0
One expectation was wrong on the first run and the tool was right: `silent`
counts corrupted LITERALS, not corrupted macros, so a docstring carrying both
`\theta` and `\frac` is one silent hit. Corrected to the real semantics and
documented in the test.
Full suite: 375 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns the sweep built for the 41-file raw-stringing task (PyAutoLabs/autolens_workspace#491) into a standing Hygiene check, so this debt is caught continuously instead of re-derived each time someone notices warning noise.
Why a mode, not a CI guard
The task's own follow-up suggested a
-W error::SyntaxWarningcompile guard in workspace CI. That would have been the weaker fix twice over: it catches only one of the two damage classes, and it returns a vacuous pass on Python 3.11. Both limitations are handled here.The two classes
\s,\l,\[— escapes Python does not recogniseSyntaxError\tin\theta,\fin\frac,\rin\rm,\bin\beta\theta_EbecomesTAB+heta_E)A warning-only sweep reports a clean repo over a corrupted one, so the two are counted separately and files with only silent damage are marked explicitly — those are exactly the ones such a sweep misses.
Two interpreter traps, both of which have already produced a false "clean" here
SyntaxWarningonly on Python 3.12+; on 3.11 they are aDeprecationWarning. ASyntaxWarning-only sweep returns zero on 3.11, indistinguishable from "already fixed". Both categories are collected, so the scan is interpreter-independent.compileallneeds-f, or__pycache__suppresses recompilation and the counts silently drop. This helper compiles from source text and never consults a cache.Shape
Follows the existing helper-backed modes exactly: a stdlib-only, read-only
_hygiene_escapes.pywith--summary/--json-row/ human output, wired intoMODE_ORDER, the delegate and kind maps,prescandispatch, the arg parser, the json-row path, the human body, and the default ranked worklist. Delegates to/refactor; Hygiene still never edits source.The human output states the fix explicitly — prefix
ron the enclosing docstring, not doubling the backslashes, which would leak into the rendered notebook prose.Validation
Against a known answer, not just eyeballed. Run over the six workspace repos immediately after the raw-stringing merges, it reports exactly the one residue documented in #491 —
autolens_workspace/scripts/group/likelihood_function.py, 2 warned + 1 silent — and nothing else.A synthetic corpus confirms the discrimination:
bash -nclean, and the default worklist still renders all 14 modes.🤖 Generated with Claude Code
https://claude.ai/code/session_01MTjtx5mdituitiyQYFGn2E
Generated by Claude Code