diff --git a/agents/conductors/hygiene/AGENTS.md b/agents/conductors/hygiene/AGENTS.md index a260ac4..fd6d9da 100644 --- a/agents/conductors/hygiene/AGENTS.md +++ b/agents/conductors/hygiene/AGENTS.md @@ -26,7 +26,7 @@ kinds, which is what makes its count comparable (or not): - **debris** — finds directly-removable items; a real, rankable count (`tidy`). - **finding** — confirms a source-quality defect; a real, rankable count - (`docstrings`, `refs`, `optdeps`, `extras`). + (`docstrings`, `escapes`, `refs`, `optdeps`, `extras`). - **timing** — measures import cost; a real, rankable count of *slow* imports (`perf`). - **surface** — only *sizes* the audit; the real problems emerge when the delegated skill runs, so the count is **not** a problem count (`deps`, `docs`). @@ -63,7 +63,8 @@ than its category — because category alone gets it wrong. The config layer is | ships a distribution | has a `pyproject.toml` | `deps` | | ships api docs | has a `docs/api/` tree | `docs` | -The helper-backed modes (`docstrings`, `refs`, `optdeps`, `extras`, `config`) +The helper-backed modes (`docstrings`, `escapes`, `refs`, `optdeps`, `extras`, +`config`) are **not** on this list: they discover their own targets by walking the scan root for workspace-shaped directories, so they can find material the map never names — and they keep reporting even when the repo-array modes are `unscanned`. @@ -78,13 +79,14 @@ names — and they keep reporting even when the repo-array modes are `unscanned` | `docs` | `docs/api/*.rst` + `currentmodule` counts across every managed repo shipping a `docs/api/` tree (**surface**) | `/audit_docs` (Heart, imports) | | `crlf` | executable scripts (`.sh` + shebang-`755` `.py`) with CRLF — the shebang breaks on Linux/HPC (**debris**, the ranked count); plain `.py` CRLF is reported separately as *cosmetic* (Python reads it fine — don't mass-normalise) | `/refactor` + `.gitattributes eol=lf` | | `docstrings` | consecutive module-level triple-quoted expressions separated only by whitespace in user-facing `*_workspace` and `HowTo*` root `*.py` entry scripts and `scripts/**/*.py` files (**finding**) | `/refactor` (mechanically merge each confirmed boundary) | +| `escapes` | LaTeX in non-raw docstrings eaten by Python's escape handling in user-facing `*_workspace` and `HowTo*` `scripts/**/*.py`, in **two** classes (**finding**). `\s`/`\l`/`\[` are escapes Python does not recognise: it keeps them literal but warns. `\t` in `\theta`, `\f` in `\frac`, `\r` in `\rm`, `\b` in `\beta` are escapes it **does** recognise: the value is silently corrupted (`\theta_E` becomes TAB + `heta_E`) with **no diagnostic of any kind**, so a warning-only sweep reports a clean repo over a corrupted one. Both classes are counted separately and files with **only** silent damage are marked, because those are exactly the ones a warning sweep misses. The scan collects `DeprecationWarning` as well as `SyntaxWarning`: invalid escapes are only the latter on Python 3.12+, so a `SyntaxWarning`-only sweep returns a vacuous zero on 3.11 that is indistinguishable from "already fixed" | `/refactor` (prefix `r` on the enclosing docstring — **not** doubling the backslashes, which would leak into the rendered notebook prose) | | `refs` | file/folder references in user-facing `*_workspace` and `HowTo*` prose (`scripts/**/*.py` docstrings + comments, every `scripts/**/README.md` and `config/**/README.md`, and the top-level README) whose target no longer exists — restructure debt no health sweep can see, since the scripts still run (**finding**). Covers the README idioms a `scripts/`-anchored matcher cannot see: structure-list bullets (``- `slam_pipeline`: ``), slash-less relative folder paths (`data_preparation/imaging`), and config YAML names. Scans the **inverse direction** too: a folder that exists but whose own parent README never names it — a package can ship fully working, with every reference resolving, and still be invisible to a reader browsing the folder list (`interferometer/features/datacube` sat unlisted for three months) | `/refactor` (re-point each dead reference; add an entry for each undocumented folder, sourced from that folder's own README or a script docstring) | | `optdeps` | smoke-listed workspace scripts that construct an optional-dependency-gated API (`TransformerNUFFT` → `nufftax`) without the house `find_spec` skip guard, so they hard-fail the CI matrices that omit the extras (**finding**). AST-confirmed — prose mentions don't count; scripts outside `smoke_tests.txt` are never flagged | `/refactor` (add the skip guard) | | `extras` | the complement of `optdeps`: an optional dependency a library **declares** (in the `[optional]` extra `mode=release` installs) that the `workspace-validation.yml` **`mode=smoke`** leg never installs (**finding**). The extras chain only reaches each library's own `[jax]`, never a sibling's `[optional]`, so those need hand-adding and silently drift — the symptom is a script red in smoke and **green in release** | `/bug` (add the install; fix the install set, **never** the script) | | `config` | library `config/*.yaml` keys missing from the matching workspace config — recursive diff (**surface**) | `/refactor` (mirror keys) | | `artifacts` | tracked files that look like leaked run outputs / stray data (under `output/`, or data-ext outside fixtures) (**debris**) | `/repo_cleanup` (gitignore + `git rm --cached`) | | `packaging` | ignored, fully-untracked top-level `*.egg-info/` and `build/` directories in the managed code repos (**debris**) | preview then run `PyAutoBrain/bin/clean_slate.sh --packaging`; repo-set, exact-name, root-depth and tracked-file guards apply | -| *(default)* | all of the above (**perf timing deferred** — it spawns real imports) | a ranked `HygieneDecision` worklist — recommends the highest-count direct mode (`tidy`/`crlf`/`docstrings`/`refs`/`artifacts`/`packaging`), then `hygiene perf`, then the periodic surface audits | +| *(default)* | all of the above (**perf timing deferred** — it spawns real imports) | a ranked `HygieneDecision` worklist — recommends the highest-count direct mode (`tidy`/`crlf`/`docstrings`/`escapes`/`refs`/`artifacts`/`packaging`), then `hygiene perf`, then the periodic surface audits | ``` pyauto-brain hygiene # pre-scan across modes → ranked worklist @@ -97,6 +99,7 @@ pyauto-brain hygiene deps # dependency-cap surface → /dep_audit pyauto-brain hygiene docs # API-docs surface → /audit_docs pyauto-brain hygiene crlf # CRLF .py files → /refactor pyauto-brain hygiene docstrings # adjacent top-level documentation → /refactor +pyauto-brain hygiene escapes # LaTeX eaten by string escapes (warned + SILENT) → /refactor pyauto-brain hygiene refs # folder-list drift in workspace prose → /refactor pyauto-brain hygiene optdeps # smoke-listed scripts missing an optional-dep skip guard → /refactor pyauto-brain hygiene extras # optional deps the smoke CI leg never installs → /bug diff --git a/agents/conductors/hygiene/_hygiene_escapes.py b/agents/conductors/hygiene/_hygiene_escapes.py new file mode 100755 index 0000000..3dffe01 --- /dev/null +++ b/agents/conductors/hygiene/_hygiene_escapes.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +"""Read-only scanner for LaTeX corrupted by Python's string-escape handling. + +Workspace tutorial prose carries LaTeX in module-level docstrings. Unless the +docstring is raw, Python's escape handling damages it in two INDEPENDENT ways, +and a scan that looks for only the first reports a clean sweep over a corpus +full of the second: + +warned ``\\s``, ``\\l``, ``\\[`` -- escapes Python does NOT recognise. It keeps + them literal but warns on every compile, and they are slated to become a + SyntaxError. +silent ``\\t`` in ``\\theta``, ``\\f`` in ``\\frac``, ``\\r`` in ``\\rm``, + ``\\b`` in ``\\beta``. Escapes Python DOES recognise: the value is + corrupted and there is NO diagnostic of any kind. ``\\theta_E`` becomes + a TAB followed by ``heta_E``. + +TWO INTERPRETER TRAPS, both of which have already produced a false "clean": + +1. Invalid escapes are a ``SyntaxWarning`` only on Python 3.12+; on 3.11 and + earlier they are a ``DeprecationWarning``. A ``SyntaxWarning``-only sweep + returns zero on 3.11 and is indistinguishable from "already fixed". Both + categories are collected here, so the scan is interpreter-independent. +2. ``compileall`` needs ``-f``, or ``__pycache__`` suppresses recompilation and + the counts silently drop. This module compiles from source text and never + consults a cache. + +Read-only, stdlib-only, and consistent with the rest of the Hygiene Agent: it +reports and delegates to /refactor; it never edits a file. +""" + +from __future__ import annotations + +import argparse +import ast +import json +import warnings +from dataclasses import asdict, dataclass +from pathlib import Path + +SKIP_PARTS = {".git", "__pycache__", "build", "dist", ".ipynb_checkpoints"} +# A newline in a string value is ordinary text, not evidence of a mangled macro. +CONTROL_EXCEPT_NEWLINE = "\n" + + +@dataclass(frozen=True) +class Finding: + repo: str + file: str + warned: int + silent: int + + +@dataclass(frozen=True) +class ParseError: + repo: str + file: str + message: str + + +def repository_paths(root: Path) -> list[Path]: + """Return user-facing ``*_workspace`` and ``HowTo*`` repositories.""" + candidates = [*root.glob("*_workspace"), *root.glob("HowTo*")] + return sorted( + {path.resolve() for path in candidates if (path / "scripts").is_dir()}, + key=lambda path: path.name.lower(), + ) + + +def warned_count(source: str, path: Path) -> int: + """Escapes Python does not recognise, on ANY interpreter version.""" + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + try: + compile(source, str(path), "exec") + except SyntaxError: + return 0 + return sum( + 1 + for item in caught + # Both categories: SyntaxWarning on 3.12+, DeprecationWarning on <=3.11. + if issubclass(item.category, (SyntaxWarning, DeprecationWarning)) + and "invalid escape sequence" in str(item.message) + ) + + +def silent_count(source: str) -> int: + """Escapes Python DOES recognise, which corrupt the value with no warning. + + A non-raw literal whose SOURCE carries a backslash but whose VALUE carries a + control character has had a macro eaten -- ``\\theta`` became TAB + ``heta``. + """ + try: + tree = ast.parse(source) + except SyntaxError: + return 0 + total = 0 + for node in ast.walk(tree): + if not (isinstance(node, ast.Constant) and isinstance(node.value, str)): + continue + segment = ast.get_source_segment(source, node) or "" + if "\\" not in segment: + continue + if segment[:1] in "rR" or segment[:2].lower() in ("br", "rb", "fr", "rf"): + continue # already raw: the escape never happened + if any( + ord(char) < 32 and char != CONTROL_EXCEPT_NEWLINE for char in node.value + ): + total += 1 + return total + + +def scan(root: Path) -> tuple[list[Finding], list[ParseError], int]: + findings: list[Finding] = [] + errors: list[ParseError] = [] + repositories = repository_paths(root) + for repository in repositories: + for script in sorted((repository / "scripts").rglob("*.py")): + if SKIP_PARTS & set(script.parts): + continue + try: + source = script.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError) as error: + errors.append( + ParseError(repository.name, str(script.relative_to(repository)), str(error)) + ) + continue + warned, silent = warned_count(source, script), silent_count(source) + if warned or silent: + findings.append( + Finding( + repository.name, + str(script.relative_to(repository)), + warned, + silent, + ) + ) + return findings, errors, len(repositories) + + +def summary_for(findings: list[Finding], errors: list[ParseError], repos: int) -> str: + warned = sum(finding.warned for finding in findings) + silent = sum(finding.silent for finding in findings) + silent_only = sum(1 for finding in findings if finding.silent and not finding.warned) + return ( + f"{len(findings)} script(s) across {repos} repo(s) with LaTeX damaged by " + f"string escapes: {warned} warned, {silent} silent " + f"({silent_only} file(s) have ONLY silent damage, which a warning-only " + f"sweep would miss); {len(errors)} read error(s)" + ) + + +def row_for(root: Path) -> dict: + findings, errors, repository_count = scan(root) + if errors: + status = "partial" + elif findings: + status = "finding" + else: + status = "clean" + return { + "mode": "escapes", + "kind": "finding", + "status": status, + "count": len(findings), + "summary": summary_for(findings, errors, repository_count), + "delegate": "/refactor", + "findings": [asdict(finding) for finding in findings], + "parse_errors": [asdict(error) for error in errors], + } + + +def render_human(row: dict) -> None: + print(row["summary"]) + for finding in row["findings"]: + marker = " <- silent only" if finding["silent"] and not finding["warned"] else "" + print( + f" {finding['repo']}/{finding['file']}: " + f"{finding['warned']} warned, {finding['silent']} silent{marker}" + ) + if row["parse_errors"]: + print("Read errors (scan incomplete):") + for error in row["parse_errors"]: + print(f" {error['repo']}/{error['file']}: {error['message']}") + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--root", type=Path, required=True) + output = parser.add_mutually_exclusive_group() + output.add_argument("--json-row", action="store_true") + output.add_argument("--summary", action="store_true") + args = parser.parse_args() + + row = row_for(args.root.resolve()) + if args.json_row: + print(json.dumps(row, sort_keys=True)) + elif args.summary: + print(f"{row['count']}|{row['summary']}") + else: + render_human(row) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/agents/conductors/hygiene/hygiene.sh b/agents/conductors/hygiene/hygiene.sh index 3cd8939..141b7e0 100755 --- a/agents/conductors/hygiene/hygiene.sh +++ b/agents/conductors/hygiene/hygiene.sh @@ -16,6 +16,8 @@ # noise -> /cli_noise_clean (Heart) deps -> /dep_audit (Heart) # docs -> /audit_docs (Heart) packaging -> clean_slate.sh (Brain) # docstrings -> /refactor (exact findings; Hygiene remains read-only) +# escapes -> /refactor (LaTeX eaten by string escapes; the SILENT class has no +# diagnostic at all, so a warning-only sweep reports a false clean) # refs -> /refactor (folder-list drift in workspace prose: dead refs + undocumented folders) # optdeps -> /refactor (smoke-listed scripts missing an optional-dep skip guard) # extras -> /bug (optional deps a library declares that the smoke CI leg never installs) @@ -34,6 +36,7 @@ # hygiene.sh docs # API-docs pre-scan -> /audit_docs # hygiene.sh crlf # executable scripts w/ CRLF break on HPC (+ cosmetic .py) -> /refactor # hygiene.sh docstrings # adjacent top-level script documentation -> /refactor +# hygiene.sh escapes # LaTeX damaged by string escapes (warned + SILENT) -> /refactor # hygiene.sh refs # folder-list drift in workspace prose -> /refactor # hygiene.sh optdeps # smoke-listed scripts w/ a gated API but no skip guard -> /refactor # hygiene.sh extras # optional deps declared by a library but missing from the smoke CI install -> /bug @@ -143,12 +146,13 @@ PERF_PY="${HYGIENE_PYTHON:-python3}" PERF_THRESHOLD="${HYGIENE_PERF_THRESHOLD:-3.0}" read -r -a PERF_LIBS <<< "${HYGIENE_PERF_LIBS:-autoconf autofit autoarray autogalaxy autolens}" -MODE_ORDER=(perf tidy crlf docstrings refs optdeps extras artifacts packaging noise deps docs config) +MODE_ORDER=(perf tidy crlf docstrings escapes refs optdeps extras artifacts packaging noise deps docs config) declare -A MODE_DELEGATE=( [perf]="/refactor" [tidy]="condemn → condemned.md (async; 'hygiene sweep' voids)" [crlf]="/refactor" [docstrings]="/refactor" + [escapes]="/refactor" [refs]="/refactor" [optdeps]="/refactor" [extras]="/bug" @@ -168,7 +172,7 @@ declare -A MODE_DELEGATE=( # 'finding', and 'timing' counts drive the ranking. declare -A MODE_KIND=( [perf]="timing" [tidy]="debris" [crlf]="debris" [artifacts]="debris" [packaging]="debris" - [docstrings]="finding" [refs]="finding" [optdeps]="finding" [extras]="finding" + [docstrings]="finding" [escapes]="finding" [refs]="finding" [optdeps]="finding" [extras]="finding" [deps]="surface" [docs]="surface" [config]="surface" [noise]="advisory" ) @@ -269,6 +273,17 @@ prescan_docstrings() { python3 "$HERE/_hygiene_docstrings.py" --root "$ROOT" --summary } +# escapes: LaTeX in non-raw docstrings eaten by Python's escape handling, in +# TWO classes. `\s`/`\l` are escapes Python does not recognise -- it warns. +# `\t` in `\theta`, `\f` in `\frac`, `\r` in `\rm` are escapes it DOES +# recognise: the value is corrupted and NOTHING is emitted, so a warning-only +# sweep reports a clean repo. The helper also collects DeprecationWarning as +# well as SyntaxWarning -- invalid escapes are only the latter on 3.12+, so a +# SyntaxWarning-only sweep returns a vacuous zero on 3.11. +prescan_escapes() { + python3 "$HERE/_hygiene_escapes.py" --root "$ROOT" --summary +} + # optdeps: smoke-listed workspace scripts that construct an optional-dependency # gated API (e.g. TransformerNUFFT -> nufftax) without the house skip guard. # Those scripts hard-fail the CI matrices that omit the optional extras, where a @@ -417,6 +432,7 @@ prescan() { perf) prescan_perf ;; tidy) prescan_tidy ;; deps) prescan_deps ;; docs) prescan_docs ;; noise) prescan_noise ;; crlf) prescan_crlf ;; docstrings) prescan_docstrings ;; refs) prescan_refs ;; + escapes) prescan_escapes ;; optdeps) prescan_optdeps ;; extras) prescan_extras ;; artifacts) prescan_artifacts ;; packaging) prescan_packaging ;; config) prescan_config ;; @@ -429,7 +445,7 @@ mode="default"; json=0; profile_script=""; expect_script=0 for arg in "$@"; do if [[ "$expect_script" -eq 1 ]]; then profile_script="$arg"; expect_script=0; continue; fi case "$arg" in - perf|tidy|sweep|noise|deps|docs|crlf|docstrings|refs|optdeps|extras|config|artifacts|packaging) mode="$arg" ;; + perf|tidy|sweep|noise|deps|docs|crlf|docstrings|escapes|refs|optdeps|extras|config|artifacts|packaging) mode="$arg" ;; default) mode="default" ;; --json) json=1 ;; --profile) mode="perf"; expect_script=1 ;; @@ -615,6 +631,10 @@ emit_json_row() { # mode python3 "$HERE/_hygiene_docstrings.py" --root "$ROOT" --json-row return fi + if [[ "$m" == "escapes" ]]; then + python3 "$HERE/_hygiene_escapes.py" --root "$ROOT" --json-row + return + fi if [[ "$m" == "refs" ]]; then python3 "$HERE/_hygiene_refs.py" --root "$ROOT" --json-row return @@ -709,7 +729,14 @@ render_row() { # mode render_delegate_line "$m" } -if [[ "$mode" == "docstrings" ]]; then +if [[ "$mode" == "escapes" ]]; then + echo "LaTeX damaged by Python string escapes (read-only scan):" + python3 "$HERE/_hygiene_escapes.py" --root "$ROOT" + echo + echo "→ the fix is an r-prefix on the enclosing docstring, NOT doubling the" + echo " backslashes (which would leak into the rendered notebook prose)." + echo "→ route to /refactor; Hygiene never edits source." +elif [[ "$mode" == "docstrings" ]]; then echo "Confirmed adjacent top-level documentation blocks (read-only scan):" python3 "$HERE/_hygiene_docstrings.py" --root "$ROOT" echo @@ -744,7 +771,7 @@ elif [[ "$mode" == "default" ]]; then # timing is deferred here — too slow for the fast scan). Rank across them and # recommend the mode with the largest confirmed workload. best=""; best_n=0 - for m in tidy crlf docstrings refs optdeps extras artifacts packaging; do + for m in tidy crlf docstrings escapes refs optdeps extras artifacts packaging; do local_n="$(prescan "$m")"; local_n="${local_n%%|*}" if [[ "$local_n" -gt "$best_n" ]]; then best_n="$local_n"; best="$m"; fi done diff --git a/skills/hygiene/hygiene.md b/skills/hygiene/hygiene.md index 1c1e761..94935a4 100644 --- a/skills/hygiene/hygiene.md +++ b/skills/hygiene/hygiene.md @@ -10,7 +10,7 @@ Shared routing context: `PyAutoBrain/skills/COMMANDS.md`. ## Do 1. Run `bin/pyauto-brain hygiene [perf | tidy | noise | deps | docs | crlf | - docstrings | refs | optdeps | extras | config | artifacts | packaging]` (no arg = pre-scan across modes → a ranked worklist; + docstrings | escapes | refs | optdeps | extras | config | artifacts | packaging]` (no arg = pre-scan across modes → a ranked worklist; perf's import timing is deferred there). This is a **dry run** — each mode does a cheap read-only pre-scan and emits a `HygieneDecision` naming the skill to run for the full audit. Nothing is executed or mutated. (`crlf` = @@ -20,7 +20,13 @@ Shared routing context: `PyAutoBrain/skills/COMMANDS.md`. `*.egg-info/` and `build/` directories in managed library repositories; `docstrings` = exact adjacent module-level triple-quoted documentation boundaries in user-facing workspace and HowTo root entry scripts and - `scripts/**/*.py` files; `refs` = dead internal references — file/folder + `scripts/**/*.py` files; `escapes` = LaTeX in non-raw docstrings eaten by + Python's escape handling, counted in TWO classes — the `\s`/`\l` ones that + warn, and the `\t`-in-`\theta` / `\f`-in-`\frac` ones that corrupt the + value with NO diagnostic at all, so a warning-only sweep reports a clean + repo (files with only silent damage are marked, and both SyntaxWarning and + DeprecationWarning are collected since invalid escapes are only the former + on 3.12+); `refs` = dead internal references — file/folder paths quoted in workspace and HowTo prose whose target no longer exists after a restructure, across `scripts/**/*.py`, every nested `scripts/**/README.md` and `config/**/README.md`, and the top-level README, @@ -34,7 +40,7 @@ Shared routing context: `PyAutoBrain/skills/COMMANDS.md`. or for `perf` route slow imports/functions to `/refactor` / `/bug` (JAX-adapt is a judgement call, never automatic). For `packaging`, preview with `DRY_RUN=1 PyAutoBrain/bin/clean_slate.sh --packaging`, then run it without `DRY_RUN` to - remove only the reported generated directories. For `docstrings` and `refs`, route the + remove only the reported generated directories. For `docstrings`, `escapes` and `refs`, route the exact reported findings to `/refactor`; the Hygiene scan remains read-only. A `refs` finding is the reference **as written** — judge the intended target (a moved file, a file that became a directory, a reference meant for a diff --git a/tests/test_hygiene_conductor.py b/tests/test_hygiene_conductor.py index 0a3c20b..909a527 100644 --- a/tests/test_hygiene_conductor.py +++ b/tests/test_hygiene_conductor.py @@ -15,7 +15,7 @@ BRAIN = BRAIN_HOME / "bin" / "pyauto-brain" MODES = { "perf", "tidy", "noise", "deps", "docs", "crlf", "config", "artifacts", - "packaging", "docstrings", "refs", "optdeps", "extras", + "packaging", "docstrings", "escapes", "refs", "optdeps", "extras", } _PROFILE_TARGET = """ @@ -63,6 +63,7 @@ def test_default_json_is_a_hygiene_decision_with_all_modes(tmp_path): assert kinds["crlf"] == "debris" and kinds["artifacts"] == "debris" assert kinds["packaging"] == "debris" assert kinds["docstrings"] == "finding" + assert kinds["escapes"] == "finding" assert kinds["refs"] == "finding" assert kinds["optdeps"] == "finding" assert kinds["extras"] == "finding" @@ -1119,3 +1120,101 @@ def test_an_explicit_body_map_override_is_authoritative(tmp_path): assert result.returncode == 3 assert names == [] + + +# --- escapes: LaTeX eaten by Python's string-escape handling ------------------ + + +def _write_escapes_fixture(tmp_path): + """Four scripts covering both damage classes and both non-findings. + + `silent_only` is the important one: it emits NO diagnostic of any kind, so + a warning-driven scan reports it clean while its docstring value is already + corrupted. + """ + scripts = tmp_path / "demo_workspace" / "scripts" + scripts.mkdir(parents=True) + + # `\t` in `\theta` and `\f` in `\frac` are escapes Python RECOGNISES: the + # value silently becomes TAB + "heta_E". Zero warnings. + (scripts / "silent_only.py").write_text( + '"""\nEinstein radius $\\theta_E$ and $\\frac{a}{b}$.\n"""\nx = 1\n' + ) + # `\s` and `\l` are NOT recognised: kept literal, but warned about. + (scripts / "warned_only.py").write_text( + '"""\nDispersion $\\sigma$ and $\\lambda$.\n"""\nx = 1\n' + ) + # Already raw: the escape never happened. + (scripts / "clean_raw.py").write_text( + 'r"""\nEinstein radius $\\theta_E$.\n"""\nx = 1\n' + ) + # A DELIBERATE newline, not a mangled macro — must never be flagged. + (scripts / "deliberate.py").write_text('print("\\nreal newline")\n') + return scripts + + +def test_escapes_reports_warned_and_silent_classes_separately(tmp_path): + _write_escapes_fixture(tmp_path) + + result = _run(["escapes", "--json"], tmp_path) + + assert result.returncode == 0, result.stderr + row = json.loads(result.stdout)["row"] + assert row["kind"] == "finding" + assert row["status"] == "finding" + assert row["delegate"] == "/refactor" + assert row["parse_errors"] == [] + + # `warned` counts warnings; `silent` counts corrupted LITERALS, so one + # docstring carrying both `\theta` and `\frac` is a single silent hit. + found = {f["file"]: (f["warned"], f["silent"]) for f in row["findings"]} + assert found == { + "scripts/silent_only.py": (0, 1), + "scripts/warned_only.py": (1, 0), + } + + +def test_escapes_ignores_raw_docstrings_and_deliberate_escapes(tmp_path): + _write_escapes_fixture(tmp_path) + + row = json.loads(_run(["escapes", "--json"], tmp_path).stdout)["row"] + + flagged = {finding["file"] for finding in row["findings"]} + # An `r"""` docstring is already correct, and `print("\nreal newline")` + # wants its newline — raw-ifying that one would be the regression. + assert "scripts/clean_raw.py" not in flagged + assert "scripts/deliberate.py" not in flagged + + +def test_escapes_summary_marks_files_a_warning_only_sweep_would_miss(tmp_path): + _write_escapes_fixture(tmp_path) + + row = json.loads(_run(["escapes", "--json"], tmp_path).stdout)["row"] + + # The silent-only count is the whole reason this mode exists: a sweep + # driven by warnings alone reports `silent_only.py` as clean. + assert "1 file(s) have ONLY silent damage" in row["summary"] + + +def test_escapes_human_output_names_the_silent_only_file(tmp_path): + _write_escapes_fixture(tmp_path) + + result = _run(["escapes"], tmp_path) + + assert result.returncode == 0, result.stderr + assert "scripts/silent_only.py: 0 warned, 1 silent <- silent only" in result.stdout + # The fix is the `r` prefix, NOT doubling backslashes (which would leak + # into the rendered notebook prose). + assert "r-prefix" in result.stdout + + +def test_escapes_clean_repo_reports_no_findings(tmp_path): + scripts = tmp_path / "demo_workspace" / "scripts" + scripts.mkdir(parents=True) + (scripts / "fine.py").write_text('r"""\nAll LaTeX is raw: $\\theta_E$.\n"""\nx = 1\n') + + row = json.loads(_run(["escapes", "--json"], tmp_path).stdout)["row"] + + assert row["status"] == "clean" + assert row["count"] == 0 + assert row["findings"] == []