Skip a companion candidate that is not a ranges document

CHANGELOG.md

@@ -71,7 +71,7 @@ releases may contain breaking changes.
   file, entry, and line it concerns. RELAX NG layer with two documented
   departures from strict validation (invalid `file://` hrefs downgraded to
   `uri-not-rfc` warnings; legal interleaving not falsely flagged); vendored
-  ranges schema over companions; and eleven semantic checks, one `Problem`
+  ranges schema over companions; and twelve semantic checks, one `Problem`
   code each (with missing-id opt-in via `require_ids`). Every code is
   described in `docs/en/guides/validate.md`.
 - Canonical sort: `Lexicon.sort()` / `RangesFile.sort()` (entries by

docs/en/guides/folder-media.md

@@ -31,6 +31,8 @@ Several candidates are tried, and every distinct file among them is loaded.
 
 Names that differ only in case or Unicode normalization still match — `Dict.LIFT` finds `Dict.lift-ranges` — unless several files match one name, which loads none of them and is reported as [`ambiguous-ranges-file`](validate.md#problem-codes).
 
+A candidate that cannot be read as a `<lift-ranges>` document (e.g., a zero-byte file from an interrupted export) is skipped rather than failing the load, and is reported as [`unreadable-ranges-file`](validate.md#problem-codes). Loading one directly with `RangesFile.load()` raises instead.
+
 Pass `resolve_ranges=False` to `load()` to skip companion discovery.
 
 ## Media

docs/en/guides/validate.md

@@ -24,11 +24,11 @@ Each `Problem` carries `level` (`"error"`/`"warning"`), a stable `code`, `messag
 
 1. **RELAX NG** against the LIFT 0.13 grammar (vendored from lift-standard — a byte-identical copy committed into this package).
 2. **Ranges schema** — this project's `lift-ranges-0.13.rng` — over every tracked `.lift-ranges` companion, addressed to the companion rather than the `.lift`.
-3. **Semantic checks** the grammar cannot express — ten of them, one code each.
+3. **Semantic checks** the grammar cannot express — twelve of them, one code each.
 
 ## Problem codes
 
-Every finding carries one of these, whichever layer produced it — `schema` and `uri-not-rfc` come from the schema layers, the other eleven are semantic checks. The strings are a supported interface; `--strict` promotes every warning to an error.
+Every finding carries one of these, whichever layer produced it — `schema` and `uri-not-rfc` come from the schema layers, the other twelve are semantic checks. The strings are a supported interface; `--strict` promotes every warning to an error.
 
 | code                     | level   | what it flags                                                              |
 | ------------------------ | ------- | -------------------------------------------------------------------------- |
@@ -44,12 +44,15 @@ Every finding carries one of these, whichever layer produced it — `schema` and
 | `range-parent`           | error   | a `range-element/@parent` no sibling id defines                            |
 | `schema`                 | error   | a RELAX NG grammar violation, in the `.lift` or in a companion             |
 | `undefined-range-value`  | warning | a grammatical-info or range-keyed trait value the range does not list      |
+| `unreadable-ranges-file` | warning | a companion candidate that exists but is not a readable ranges document    |
 | `uri-not-rfc`            | warning | an href that is not a valid URI — FLEx's `file://C:/...`                   |
 
 All three layers work from the document serialized as it stands, so one that cannot be serialized at all is reported as a single `lone-surrogate` error instead — see [Fidelity guarantees](../fidelity.md#content-xml-cannot-represent). Validation is read-only: it reports the document as it stands, before the `dateModified` stamping a save does. Nothing generated is ever a finding, so validate-then-save is sound.
 
 A companion name matching several files loads none of them: the ranges they define go absent until all but one is renamed or removed.
 
+The three companion-folder codes (`ambiguous-ranges-file`, `dangling-ranges-href`, and `unreadable-ranges-file`) are reported only when companion discovery ran. Loading with `resolve_ranges=False` puts companions out of scope, so none of them is reported; attaching one afterwards with `add_ranges_file()` does not bring them back, since it never reads the folder these codes report on. `missing-media` is unaffected: media is never resolved into the model, so nothing was opted out of.
+
 ## Real-world FieldWorks (FLEx) output
 
 FieldWorks systematically writes some content that strict tooling rejects. Here is sil-lift's policy, so that real lexicons validate usefully:

src/sil_lift/_model.py

@@ -15,8 +15,9 @@
 from dataclasses import dataclass, field
 from datetime import date, datetime
 from pathlib import Path, PurePosixPath, PureWindowsPath
-from typing import TYPE_CHECKING
+from typing import TYPE_CHECKING, NamedTuple
 
+from ._errors import LiftError
 from ._extras import Extras
 from ._header import Header, Range
 from ._text import Annotation, Form, Multitext, Text, Trait
@@ -540,27 +541,41 @@ def _existing_file(candidate: Path, listings: dict[Path, dict[str, list[Path]]])
     return matches[0] if len(matches) == 1 else None
 
 
-def _ranges_candidates(lift_path: Path, ranges: Iterable[Range]) -> list[Path]:
+class _Candidate(NamedTuple):
+    """A place a companion may be found, and what sent :meth:`Lexicon.load` there."""
+
+    path: Path
+    #: The header ``range`` id and ``href`` that named it, or None for the
+    #: conventional sibling, which no part of the document asserts.
+    source: tuple[str, str] | None
+
+
+def _ranges_candidates(lift_path: Path, ranges: Iterable[Range]) -> list[_Candidate]:
     """Where a companion may be found, in the order :meth:`Lexicon.load` tries.
 
     Each path once: an href shared by several ranges, or agreeing with the
-    sibling, is one candidate however many times it is written.
+    sibling, is one candidate however many times it is written. The first
+    spelling wins, so a candidate's ``source`` is the route load took to it.
     """
     base = lift_path.parent
     # with_name and with_suffix agree on every name that has an extension, but
     # with_suffix would raise on a name that has none — which parse_document
     # accepts, since it never inspects the extension.
-    candidates = [lift_path.with_name(lift_path.name + "-ranges")]
+    candidates = [_Candidate(lift_path.with_name(lift_path.name + "-ranges"), None)]
     for range_ in ranges:
         if range_.href is None:
             continue
+        source = (range_.id, range_.href)
         relative = _normalize_href(range_.href)
         if relative is not None:
-            candidates.append(base / relative)
+            candidates.append(_Candidate(base / relative, source))
         basename = range_.href.replace("\\", "/").rpartition("/")[2]
         if basename:
-            candidates.append(base / basename)
-    return list(dict.fromkeys(candidates))
+            candidates.append(_Candidate(base / basename, source))
+    seen: dict[Path, _Candidate] = {}
+    for candidate in candidates:
+        seen.setdefault(candidate.path, candidate)
+    return list(seen.values())
 
 
 def _same_file(left: Path, right: Path) -> bool:
@@ -582,6 +597,17 @@ def _same_file(left: Path, right: Path) -> bool:
         return False
 
 
+def _reason(path: Path, exc: Exception) -> str:
+    """``exc``'s message without the path it repeats.
+
+    Every rejection is reported against the file it concerns, so a parse
+    error's own ``<path>: `` prefix would print the path twice.
+    """
+    text = str(exc)
+    prefix = f"{path}: "
+    return text[len(prefix) :] if text.startswith(prefix) else text
+
+
 def _same_dir(left: Path, right: Path | None) -> bool:
     """Whether two paths denote the same directory, spelling aside.
 
@@ -601,6 +627,7 @@ class Lexicon:
     """The root handle: a parsed ``.lift`` document and its folder companions."""
 
     __slots__ = (
+        "_rejected_ranges",
         "_source",
         "_stamps",
         "_tempdir",
@@ -630,6 +657,9 @@ def __init__(
         self._source: _SourceInfo | None = None  # set by the reader
         self._stamps: dict[int, _EntryRecord] = {}  # stamping baselines (see stamp_entries)
         self._tempdir: tempfile.TemporaryDirectory[str] | None = None  # zip extraction, if any
+        # None until companion discovery runs, which is what lets validation
+        # tell "nothing was rejected" from "nothing was ever looked at".
+        self._rejected_ranges: dict[Path, str] | None = None
 
     @classmethod
     def load(cls, path: str | os.PathLike[str], *, resolve_ranges: bool = True) -> Lexicon:
@@ -650,6 +680,13 @@ def load(cls, path: str | os.PathLike[str], *, resolve_ranges: bool = True) -> L
         ambiguous and resolves to none of them, which validation reports as
         ``ambiguous-ranges-file``; the ``.lift`` is never its own companion.
 
+        Companion discovery never raises. A candidate that exists but cannot be
+        read as a ``<lift-ranges>`` document -- a zero-byte or truncated
+        sidecar, another ``.lift``, an unrelated file an href names -- is
+        skipped, so its entries still load, and validation reports it as
+        ``unreadable-ranges-file``. :meth:`RangesFile.load` is unchanged and
+        still raises.
+
         A ``.zip`` path is treated as a packaged LIFT folder: it is extracted
         to a temporary directory (kept alive for the returned lexicon's
         lifetime) and the single contained ``.lift`` is loaded.
@@ -667,11 +704,13 @@ def load(cls, path: str | os.PathLike[str], *, resolve_ranges: bool = True) -> L
         return lexicon
 
     def _resolve_ranges(self) -> None:
+        rejected: dict[Path, str] = {}
+        self._rejected_ranges = rejected
         if self.path is None:
             return
         listings: dict[Path, dict[str, list[Path]]] = {}
         for candidate in _ranges_candidates(self.path, self.header.ranges):
-            found = _existing_file(candidate, listings)
+            found = _existing_file(candidate.path, listings)
             if found is None:
                 continue
             try:
@@ -681,12 +720,20 @@ def _resolve_ranges(self) -> None:
             # Identity, not content, settles the .lift: it is the document being
             # loaded, so a header href folding onto it names no companion. Two
             # spellings of one companion, which resolve() leaves distinct on
-            # macOS, would otherwise load and write it twice.
-            if resolved in self.ranges_files or any(
-                _same_file(resolved, other) for other in (self.path, *self.ranges_files)
+            # macOS, would otherwise load and write it twice -- and a rejected
+            # one, parse and report it twice.
+            if resolved in self.ranges_files or resolved in rejected:
+                continue
+            if any(
+                _same_file(resolved, other) for other in (self.path, *self.ranges_files, *rejected)
             ):
                 continue
-            self.ranges_files[resolved] = RangesFile.load(found)
+            try:
+                self.ranges_files[resolved] = RangesFile.load(found)
+            except (LiftError, OSError) as exc:
+                # One unusable file must not cost the entries: skipped like
+                # the .lift, and recorded for unreadable-ranges-file to report.
+                rejected[resolved] = _reason(found, exc)
 
     def _apply_stamps(self, stamp: bool, when: datetime | None) -> _StampUndo:
         """The stamping step shared by :meth:`save` and :meth:`save_zip`.
@@ -1007,6 +1054,15 @@ def iter_problems(self, *, require_ids: bool = False) -> Iterator[Problem]:
         With ``require_ids``, entries missing a ``guid`` and senses missing an
         ``id`` are reported as ``missing-id`` errors — stricter than LIFT (both
         are optional there), for workflows that re-import by a stable id.
+
+        The companion-folder checks (``ambiguous-ranges-file``,
+        ``dangling-ranges-href``, and ``unreadable-ranges-file``) are reported
+        only when companion discovery ran: a lexicon loaded with
+        ``resolve_ranges=False`` put companions out of scope, so none is
+        reported for it, and attaching one afterwards with
+        :meth:`add_ranges_file` does not reinstate them. ``missing-media`` is
+        unaffected: media is never resolved into the model, so nothing was
+        opted out of.
         """
         from ._validate import iter_lexicon_problems
 
@@ -1040,6 +1096,10 @@ def add_ranges_file(self, ranges_file: RangesFile | None = None, *, href: str) -
         (or call again to reference ranges added later).
 
         Returns the attached (or newly created) :class:`RangesFile`.
+
+        Attaching a companion does not read the folder, so it does not put one
+        back in scope for the companion-folder checks: under
+        ``resolve_ranges=False`` those stay silent (see :meth:`iter_problems`).
         """
         if ranges_file is None:
             ranges_file = RangesFile()

src/sil_lift/_validate.py

@@ -527,7 +527,15 @@ def range_named(name: str) -> str | None:
     # here, not inside the file check below, which a lexicon with no path skips.
     header_ranges = {range_.id: range_named(range_.id) for range_ in lexicon.header.ranges}
 
-    if lexicon.path is not None:
+    # Only what companion discovery actually went looking for. With
+    # resolve_ranges=False the caller put companions out of scope for this
+    # load, so a finding about one is unwanted whether or not it is accurate.
+    # add_ranges_file attaches a companion without reading the folder, so it
+    # does not reinstate these: the checks below re-derive from a folder this
+    # load never looked at, where a sibling that supplies a header range went
+    # unloaded and so reads as a dangling href.
+    rejected = lexicon._rejected_ranges
+    if lexicon.path is not None and rejected is not None:
         base = lexicon.path.parent
         listings: dict[Path, dict[str, list[Path]]] = {}
 
@@ -536,14 +544,14 @@ def range_named(name: str) -> str | None:
         # several candidate names can fold onto the same one.
         reported: set[tuple[Path, tuple[str, ...]]] = set()
         for candidate in _ranges_candidates(lexicon.path, lexicon.header.ranges):
-            matches = _folded_matches(candidate, listings)
+            matches = _folded_matches(candidate.path, listings)
             if len(matches) < 2:
                 continue
             # One of the colliding files loaded, named exactly by another candidate.
             if any(_same_file(path, loaded) for path in matches for loaded in lexicon.ranges_files):
                 continue
             names = sorted(path.name for path in matches)
-            key = (candidate.parent, tuple(names))
+            key = (candidate.path.parent, tuple(names))
             if key in reported:
                 continue
             reported.add(key)
@@ -552,7 +560,7 @@ def range_named(name: str) -> str | None:
             yield Problem(
                 "warning",
                 "ambiguous-ranges-file",
-                f"companion {candidate.name!a} matches {spellings}; they differ only "
+                f"companion {candidate.path.name!a} matches {spellings}; they differ only "
                 "in case or Unicode normalization, so none of them is loaded",
                 file=lexicon.path,
             )
@@ -582,6 +590,47 @@ def range_named(name: str) -> str | None:
                     file=lexicon.path,
                 )
 
+        # Candidates that exist but could not be read as ranges documents.
+        # Walked rather than read straight off the record: the header and the
+        # folder can both have moved since discovery, and a rejection whose
+        # file is gone, or that nothing names any more, would contradict the
+        # checks above. Only the parser's reason is necessarily historical.
+        seen: set[Path] = set()
+        for candidate in _ranges_candidates(lexicon.path, lexicon.header.ranges):
+            found = _existing_file(candidate.path, listings)
+            if found is None:
+                continue
+            try:
+                resolved = found.resolve()
+            except OSError:
+                continue
+            # Exact spelling, then identity, the way _resolve_ranges keys
+            # these: a candidate reaching a rejected file under another
+            # spelling yields a path no == will match -- see _same_file.
+            # Dedup on the key that matched, so two spellings report once.
+            recorded = (
+                resolved
+                if resolved in rejected
+                else next((other for other in rejected if _same_file(resolved, other)), None)
+            )
+            if recorded is None or recorded in seen:
+                continue
+            seen.add(recorded)
+            reason = rejected[recorded]
+            # A broken sidecar is fixed in the file; an href pointing at an
+            # unrelated file is fixed in the header. Naming the route says which.
+            if candidate.source is None:
+                where = f"the conventional companion beside {lexicon.path.name!a}"
+            else:
+                named_by, href = candidate.source
+                where = f"this file, named by header range {named_by!r} href {href!r},"
+            yield Problem(
+                "warning",
+                "unreadable-ranges-file",
+                f"{where} could not be read as a ranges document, so it is not loaded: {reason}",
+                file=resolved,
+            )
+
     # Undefined range values: every grammatical-info and every trait whose
     # name matches a known range, anywhere in the entry. Empty values are
     # skipped -- FLEx writes them.