Skip to content

docs: consistency audit — 41 findings across the standard, get-started, and mirrored root docs #79

Description

@maehr

Systematic consistency audit of every documentation file against the implementation, ahead of the v0.1.0 tag (#68, #4).

Baseline: staging @ 37faf37 — content-identical to what https://textrefs.org currently serves.
Ground truth used: standard/schema/*.ts, scripts/source-schema.ts, scripts/compile.ts, public/contexts/v1.jsonld, api/openapi.yaml, decisions/ADR-*.md, and the compiled output of npm run compile:data against the pinned data/ submodule.

Every item below was verified by reading both sides. Findings that turned out to be correct on inspection are not listed; notable clean results are summarised at the end. Line numbers are against the baseline commit.

Numbering is stable so items can be ticked off individually — this is deliberately one issue rather than 40, but it is meant to be worked in the sections below, and several sections are independent.


A. Normative contradictions in standard/

A1. Spec §13's flagship worked example is invalid against the schema.
specification.md:319-327 — the @graph Work is {id, key, type, preferred_label, status, created, modified}. standard/schema/work.ts:28 declares preferred_citation_system_key: FlatKey (non-optional), the spec's own §6 (specification.md:131) lists it as required, and api/openapi.yaml:203-204 agrees. The document's headline example would fail Work.safeParse at scripts/compile.ts:514. The class diagram at specification.md:67-72 has the same omission — it lists optional creators but not the required key.

A2. "Tombstone" is defined two ways, and json-ld.md states the Work projection wrongly as a result.
specification.md:279 defines a tombstone as status deprecated, withdrawn, or blocked. json-ld.md:35 then says the projection is "derived from their non-tombstoned mapping assertions", which under that definition would exclude deprecated. The compiler excludes only two: scripts/compile.ts:633 const TOMBSTONE_STATUSES = new Set(['withdrawn', 'blocked']), applied at :491. specification.md:140 and versioning.md:109 both state the narrow sense correctly. As written, json-ld.md is wrong for deprecated mappings, which the compiler does project.

A3. The advertised alternate-link media type contradicts the OpenAPI contract.
specification.md:281 (a normative SHOULD): <link rel="alternate" type="application/ld+json" href="…json">. api/openapi.yaml:13: <link rel="alternate" type="application/json" href="…json">. The implementation emits application/ld+json (src/layouts/RecordPage.astro:25), so api/openapi.yaml:13 is the incorrect side.

A4. §14's SPDX validation rule contradicts §9's published-record format.
specification.md:388 (validation requirement 10) requires a validator to check "SPDX syntax of license when present". But specification.md:233 and scripts/compile.ts:289 say published records carry the SPDX IRI (https://spdx.org/licenses/{id}). A validator applying requirement 10 to published records would reject every conforming record. The SPDX check exists only on the authoring side (scripts/source-schema.ts:48-54); standard/schema/canonical-reference.ts:10 is an unconstrained z.string().optional().

A5. One example, two license encodings.
specification.md:352 has "license": "https://spdx.org/licenses/CC-BY-4.0" and :360 has "license": "CC0-1.0" — two entries of the same resolver_targets array in the same JSON-LD document ("@context" at :317), i.e. published output, where per A4 only the IRI form occurs.

A6. §14's BCP 47 requirement is enforced nowhere.
specification.md:388 requires a validator to check "BCP 47 syntax of language and its presence for language-specific entries". standard/schema/canonical-reference.ts:6, scripts/source-schema.ts:42 and api/openapi.yaml:251 all have only minLength: 2. Neither the tag syntax nor the presence condition is checked anywhere in the pipeline the spec calls "the implementation source of truth" (specification.md:395). Either implement it or soften the requirement.

A7. license is a SHOULD in prose but a hard build failure in practice, and "current" overstates the accepted set.
specification.md:233: "license SHOULD be authored as a current SPDX license identifier". scripts/source-schema.ts:16-19,48-54: the accepted set is spdx-license-ids plus spdx-license-ids/deprecated, and a non-SPDX value throws in parseSource (:222). Deprecated ids pass; non-SPDX ids fail the build. That is a MUST, and not restricted to current ids.

A8. versioning.md misplaces one of the two "parse-time" checks.
versioning.md:115: "Two further checks run when a source file is parsed, before any record is built: a work MUST NOT declare the same citation system twice, and a locator MUST NOT contain /." Only the first is parse-time (scripts/source-schema.ts:178-190). The / check is assertValidLocator in scripts/compile.ts:199-203, called per reference during emission at :366.

B. ADR-0006 leftovers

The relation-vocabulary sweep was thorough — no exactMatch/closeMatch prose survives anywhere in the docs (the only repo occurrence is the deliberate out-of-enum rejection fixture at scripts/compile.test.ts:360). Four attribution leftovers remain:

B9. The normative-references table still credits SKOS with mapping relations.
specification.md:418: "| Concepts and mapping relations | SKOS |". After ADR-0006 the relations are prov:alternateOf / dcterms:isReferencedBy (public/contexts/v1.jsonld:41-49, standard/schema/mapping-assertion.ts:22); SKOS's remaining role is skos:prefLabel / skos:inScheme only.

B10. The §16 table has no PROV-O row at all (specification.md:405-422), although prov: is a published namespace and specification.md:140 cites prov:alternateOf normatively.

B11. Two vocabulary lists omit PROV-O.
standard/index.md:33 — "the context mapping TextRefs records onto SKOS, Dublin Core, and schema.org". get-started/index.md:20 — "published as JSON-LD against SKOS, Dublin Core and schema.org". Both omit the vocabulary that now carries the primary mapping relation (json-ld.md:9, v1.jsonld:6).

B12. MappingAssertion is still framed as "equivalence" although half the vocabulary is not.
specification.md:59 ("A curated equivalence between a Work and an external identifier"), :240, and versioning.md:90 ("reserved for genuine work-level equivalence claims"). But specification.md:264 and ADR-0006 define isReferencedBy as "a document or page about the work" — explicitly not an equivalence. The type's own summary line now excludes half its enum.

C. Examples that do not match the registry or would fail the build

C13. how-it-works.md "compiled registry record" blocks show the authoring license form.
how-it-works.md:54 "license": "CC-BY-SA-3.0", :81 "license": "CC-BY-4.0". Introduced as "The compiled registry record … is:", but real records carry the IRI (compile.ts:289, specification.md:233).

C14. The same blocks claim status: "active"; every record in data/ is draft.
how-it-works.md:59,85,103 vs data/works/plato.republic.yaml:5 and data/works/new-testament.yaml:5. The modified dates are stale too (:61 2026-06-03, :87 2026-05-31, :105 2026-06-02 vs compiled 2026-08-12/2026-08-11). Under ADR-0004 active carries the permanence promise, so this misstates the lifecycle state of live records on the page that introduces the lifecycle.

C15. The John.3.16 example is missing a resolver target and shows an unescaped URL.
how-it-works.md:68-88 shows one resolver_targets entry with …version=SBLG|reference=John.3.16 and no last_checked. data/works/new-testament.yaml:33 has %7C escaped and last_checked: '2026-06-03', and the compiled record for b6438d55-… has two targets (STEP Bible + Deutsche Bibelgesellschaft). The omission undercuts the point the passage is making — that many locations share one reference identity.

C16. Wrong Wikidata QID for the Dhammapada, in two files.
authoring.md:39 and mappings-and-resolver-targets.md:43 both show …/entity/Q220114. data/works/dhammapada.yaml:13 registers …/entity/Q748878 (mapping 7b3679c7-aafd-5db2-9fbc-98621d9aeb87). Both are presented as the work's registered mapping.

C17. Dhammapada provider/platform counts are wrong in three places.
authoring.md:24 "four different reading platforms", how-it-works.md:126 "(four providers, two languages, 423 references)", get-started/index.md:37 "three real providers". data/works/dhammapada.yaml:35,47,61 has three resolver entries across two providers (SuttaCentral ×2, ancient-buddhist-texts.net), and :56-59 records that no Wikisource resolver is wired up. The 423 references and two languages are correct.

C18. authoring.md's additional_systems example would fail the build.
authoring.md:343 uses citation_system: book-chapter; no data/systems/book-chapter.yaml exists and scripts/compile.ts:582-585 throws references unknown citation_system. authoring.md:334 also shows status: active for plato.republic, which is draft (data/works/plato.republic.yaml:5).

C19. authoring.md documents an examples key the schema rejects.
authoring.md:20 ("declares the locator regex and examples") and :417 ("regex, examples, references"). scripts/source-schema.ts:194-207 — SystemSource is a strictObject with no examples field, so an unknown key is an authoring error. No data/systems/*.yaml has one, and src/pages/id/system/[key]/index.astro:89-108 renders none.

D. Docs vs implementation drift

D20. related-systems.md tells implementers to read a mappings array that no record has.
related-systems.md:39: "Read external identifiers and resolver targets from the mappings array". Published Works carry alternateOf / isReferencedBy (standard/schema/work.ts:30-32); resolver targets live on CanonicalReference (canonical-reference.ts:27) and never were in mappings. This is the page's integration advice.

D21. use-cases.md attributes two nonexistent fields to CitationSystem.
use-cases.md:32: "a documented locator regex, a normalisation rule, and a list of valid reference types". standard/schema/citation-system.ts:5-11 and specification.md:162: id, key, type, preferred_label, description, locator_regex + admin metadata. Normalisation lives in prose inside description; "valid reference types" has no counterpart at all.

D22. Work.creators is missing from the OpenAPI schema.
Documented at specification.md:131,136, implemented at standard/schema/work.ts:29, in the context at v1.jsonld:18-22 — but api/openapi.yaml:205-231 lists only id, key, type, preferred_label, preferred_citation_system_key, alternateOf, isReferencedBy.

D23. url_by misses are described as silent; the compiler warns.
authoring.md:176: "References for chapters not in the map are silently skipped for this provider". scripts/compile.ts:263-272,371-374: a url_by miss returns null and increments warnings exactly like a missing template variable. This also contradicts authoring.md:158 on the same mechanism.

D24. The json-ld.md dcterms row omits two terms the context uses.
json-ld.md:23 lists created, modified, source, language, license, conformsTo, isReferencedBy — missing dcterms:description (v1.jsonld:17) and dcterms:isReplacedBy (v1.jsonld:80-83).

D25. A /reg/ route that does not exist.
authoring.md:420 "browse paginated reference lists from work/system pages"; url-layout.md:13 "from those record pages". src/pages/reg/ contains only index.astro and work/[key]/refs/[page].astro; src/pages/id/system/[key]/index.astro:93-108 lists works with counts only. There is no per-system reference list.

D26. verify and verify:fast are described as not running tests, but both do.
package.json:21-22 — both chains include npm test. Contradicted in three places: README.md:82 ("Prettier + astro check + production build — the CI gate"), AGENTS.md:12-13, and CONTRIBUTING.md:69 plus its site mirror.

E. Internal contradictions in get-started/

E27. mappings-and-resolver-targets.md contradicts itself and the spec on edition-level identifiers.
:120 — "If an identifier points to a whole edition, scan, or digital object …, it is not a MappingAssertion target at all". Same file :32 lists "DOIs, Handles, ARKs, PURLs, or URN:NBNs for editions or digital objects" as common mapping targets, and specification.md:262 normatively allows an edition as a target. The edge-case bullet forbids what both the same page and the spec permit.

E28. The TEI row claims passage-level mapping targets the spec forbids.
related-systems.md:26: "TextRefs MappingAssertions can point at a specific TEI anchor in a published edition". specification.md:261: per-passage external identifiers "MUST NOT be stored as separate MappingAssertion records"; standard/schema/mapping-assertion.ts:6-11 requires a Work IRI as subject.

E29. The PURL row inverts the /id/ vs /cite/ distinction.
related-systems.md:21: "TextRefs IDs are themselves HTTP URIs that redirect". url-layout.md:12 says /id/ is "the canonical, persistent URL of every record"; only /cite/ redirects (src/pages/cite/[...alias].astro:21). This is the opposite of url-layout.md's "/id/ is the registry. … /cite/ shortcuts to it."

E30. Prose calls reading URLs "mappings".
get-started/index.md:14 lists "the reading URL of the archive that holds the text" among "curated mappings", against mappings-and-resolver-targets.md:22 ("A MappingAssertion is about work-level equivalence. A resolver_targets entry is about dereferencing one passage."). A Stephanus locator, also in that list, is not a mapping either.

E31. "Two sources" introduces a five-item list.
authoring.md:127 — "Variables are drawn from two sources:" followed by items 1–5 at :129-134. All five are real (compile.ts:141-187, :232-252); only the count is wrong.

F. Mirror-pair divergence

AGENTS.md:50 and ROADMAP.md:41 require these pairs be edited in the same commit.

F32. The published contributing page gives the wrong branch. ⚠️ Most user-facing item in this issue.
CONTRIBUTING.md:110,114 — "Branch from staging for content/docs/blog; branch from main for infra …", "Open the PR against the branch you started from". src/content/docs/community/contributing.md:102,106 — "Branch from main." / "Open the PR against main."
The site version also omits the entire ## Branching model section (CONTRIBUTING.md:95-107, 13 lines) with no counterpart anywhere. And the guidance conflicts with the actual infrastructure: .github/dependabot.yml sets target-branch: staging, and data/.github/workflows/validate.yml:4-10,29 checks out parent/staging because "staging (not main) is the integration branch".

F33. The mirrored release checklist diverges, 9 steps vs 7, and the site's step 2 is a documented footgun.
CONTRIBUTING.md:125-133 vs src/content/docs/community/contributing.md:118-124. The published page prescribes bare npm run changelog, which the canonical file's step 3 explicitly warns "would file the commits under ## [Unreleased]" (it says to pass --tag explicitly). The site version also drops the CITATION.cff step and the Pages-dispatch step — the latter being the one that explains why a release PR stays blocked.

F34. AGENTS.md's list of mirrored files omits ROADMAP.md.
AGENTS.md:50 names CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md. ROADMAP.md:41 declares itself a mirror pair too. An agent following AGENTS.md alone would not know — plausibly how F32/F33 arose.

F35. The mirrored roadmap page links to itself.
ROADMAP.md:3 "mirrored on the docs site at https://textrefs.org/community/roadmap/" is carried verbatim into src/content/docs/community/roadmap.md:8, where it is a self-link. It should point at ROADMAP.md in the repo. (Otherwise the pair is byte-identical apart from frontmatter — the maintenance rule holds.)

F36. English association pages drop the legal-reservation notice the binding German originals carry.
de/association/statutes.md:9 (and governance.md:9, expenses.md:9, mission.md:9): "Dies ist eine Arbeitsfassung … Sie steht unter rechtlichem Vorbehalt bis zur Annahme durch die Gründungsversammlung." The English pages (association/statutes.md:8-10 etc.) carry only the :::caution[Non-binding translation] aside. An English reader learns the translation is non-binding but not that the underlying document is an unadopted draft. Otherwise all four pairs match exactly (Art. 1–22, all figures).

G. Stale or unreachable

G37. README.md:58 lists docs-internal/, which does not exist (and is not gitignored). The same block omits src/pages/, src/layouts/ and src/lib/, all documented in AGENTS.md:26-28.

G38. The conformance-failure issue template requires test IDs from a suite that does not exist.
.github/ISSUE_TEMPLATE/conformance_failure.yml:26-30 has a validations: required: true field asking to "List the conformance test IDs that fail". ROADMAP.md:35 lists the conformance suite as "planned".

G39. ROADMAP.md:21 "in progress: Replace remaining community placeholder pages" is stale — all seven files in src/content/docs/community/ carry substantive prose (203–1218 words); the only occurrence of "placeholder" in that directory is the roadmap line itself.

G40. versioning.md:34 attributes the -draft suffix to the wrong artefact.
"0.x releases stay working-draft regardless of any -draft suffix on the tag." The tag is v0.1.0, with no suffix; -draft is on the spec document version (specification.md:9). The spec-version-vs-release-tag distinction is deliberate (#68 §4) but is currently explained nowhere on the site — versioning.md's three-train table (:11-15) is where it belongs. Without it, a reader citing release v0.1.0 and landing on a spec headed 0.1.0-draft has nothing to reconcile the two.

G41. standard/index.md:38 — "A frozen, citable release will be tagged once the core stabilises" — reads oddly next to package.json/CITATION.cff at 0.1.0 and CHANGELOG.md dated 2026-08-12. Related to, but distinct from, the release-state claims already tracked in #68 (ROADMAP.md:27, CITATION.cff date vs the absent tag); not restating those here.


Checked and clean

Worth recording, so this ground is not re-audited:

  • All internal anchors resolve. All 27 /standard/…#… cross-references point at headings that exist; no get-started/ page uses a cross-file anchor.
  • Both example UUIDs recompute from their documented v5 seeds and namespaces (dc799d4b-…, b6438d55-…).
  • All reference counts are right: Iliad 15,693 / Analects 517 / Dhammapada 423 / Genesis 1,533 / Republic 1,475.
  • ADR-0006 sweep is otherwise complete — no exactMatch/closeMatch/skos: mapping prose survives in any doc; data/AGENTS.md:14 is correct.
  • Draft-lifecycle, tombstone, identifier-persistence and alias-minting rules agree across specification.md, identifier-syntax.md, versioning.md, ADR-0004 and the compiler. The six compiler invariants at versioning.md:108-113 match compile.ts:658-737 one for one.
  • Status / access / relation enums, the flat-key regex, UUID namespaces and seeds are consistent across docs, Zod, context and OpenAPI.
  • Bare-vs-qualified alias rules and the asymmetric reference_status default match compile.ts:405-408,595-596 and ADR-0005.
  • SECURITY.md and CODE_OF_CONDUCT.md are byte-identical to their mirrors; every npm run … command named in any doc exists in package.json; all root-relative links resolve except G37.

Suggested handling before the tag

A3, A4, D22 touch the published API contract and A1 is the spec's headline example — those are the ones that are cheapest to fix now and most expensive to have frozen into a citable baseline. F32/F33 mislead contributors today. The rest is safe to land after v0.1.0.


Audit performed with Claude Code.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    addressed-in-v0.1.0Addressed in the v0.1.0 release (PR #4)documentationImprovements or additions to documentationstandardThe published specification and schemas

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions