The build graph, generated for Ninja

.claude/skills/dgs-editor/SKILL.md

@@ -40,7 +40,7 @@ a reference key and `sk:` is a language — so neither gets one.
 `POST /api/evaluate` does the work, in-process and writing nothing. It reads the **buffers** out
 of the request rather than the files, because the whole point is to answer for the text on
 screen, which is usually unsaved; and it shares `build_render_context` and `render_twice` with
-`core/builder/renderer.py`, so it cannot answer differently from what `make` would write —
+`core/builder/renderer.py`, so it cannot answer differently from what the build would write —
 including whether an `eq:` display carries its `{#eq:…}` label, which depends on the open tab
 through the module's `equation_numbering`. A test asserts that agreement rather than trusting it.
 
@@ -55,12 +55,12 @@ in the pane and the rest are map lookups; a keystroke in either pane, a differen
 different problem throws the lot away.
 
 **It degrades rather than failing.** The three output tabs are the pipeline's three stages --
-Rendered Markdown is Jinja and needs only make, TeX adds pandoc, the PDF adds all of TeX Live and
-the fonts -- and `tools/editor/capabilities.py` probes for each at startup. A stage this machine
+Rendered Markdown is Jinja and needs only Ninja, TeX adds pandoc, the PDF adds all of TeX Live
+and the fonts -- and `tools/editor/capabilities.py` probes for each at startup. A stage this machine
 cannot do is greyed, and the pane behind it says what is missing and the command that fixes it, so
 a translator with no TeX Live can still check their own rendering. `install.md` has the tiers;
 `uv run python tools/editor/capabilities.py` prints the same report. `uv` is not required --
-without it the app calls `make` directly with its own interpreter's directory on `PATH`.
+without it the app calls `ninja` directly with its own interpreter's directory on `PATH`.
 
 It must run from the root, and it `chdir`s there itself: several `core` modules open
 their data by a repository-relative path, so `core/i18n` fails to import from anywhere

.github/workflows/python-app.yml

@@ -33,11 +33,19 @@ jobs:
     # `poppler-utils` is `pdftocairo`, which converts every picture for the web, and
     # `gnuplot-nox` draws the plots. Both were absent, so `test_picture_svg.py` skipped itself
     # and CI had never once exercised the picture pipeline.
+    #
+    # `mupdf-tools` is `mutool`, and it is dvisvgm's PDF reader now. Ghostscript dropped the
+    # interpreter dvisvgm drove in **10.01.0**, so on a runner carrying 10.05 every `--pdf`
+    # run ends in `To process PDF files, either Ghostscript < 10.01.0 or mutool is required` --
+    # which is the whole of the HTML maths, and it surfaced the moment those tests stopped
+    # skipping. dvisvgm picks mutool up by itself when it finds it; `DVISVGM_PDF_PROC` forces
+    # the choice. Ghostscript stays -- `pdfcrop` is Ghostscript, and dvisvgm still wants it for
+    # PostScript specials.
     - name: "Install utilities"
       run: |
         sudo apt-get update -qq
         sudo apt-get install -y --no-install-recommends \
-          librsvg2-bin lcdf-typetools dvisvgm ghostscript poppler-utils gnuplot-nox
+          librsvg2-bin lcdf-typetools dvisvgm ghostscript mupdf-tools poppler-utils gnuplot-nox
     # MathJax is the other HTML maths engine, and it is node's. A hosted runner has node
     # whether or not anyone installed the packages, which is exactly how CI reported
     # `Cannot find module '@mathjax/mathjax-pagella-font/package.json'` from a test that should
@@ -122,7 +130,7 @@ jobs:
     - name: "Verify the HTML output's prerequisites"
       run: |
         missing=""
-        for tool in dvisvgm pdftocairo gnuplot rsvg-convert; do
+        for tool in dvisvgm mutool pdftocairo gnuplot rsvg-convert; do
           command -v "$tool" >/dev/null 2>&1 || missing="$missing $tool"
         done
         for package in mathjax @mathjax/mathjax-pagella-font; do
@@ -135,12 +143,35 @@ jobs:
     - name: Test with pytest
       run: |
         uv run python -m pytest -s -v
+    # `source/seminar/test`, which is where `modules/seminar/editor.yaml` says this repository
+    # goes. It was cloned to `.../testing` while nothing read it; a scope is named after the
+    # directory, so the two had to agree before anything could be built by name.
     - name: "Clone testing repository"
       uses: actions/checkout@master
       with:
         repository: TrojstenSK/dgs-seminar-testing
-        path: source/seminar/testing
-    - name: "Build testing/01/1/1"
+        path: source/seminar/test
+
+    # The graph before the build, as its own step. It reads the whole source tree and writes
+    # every edge, so a generator that crashes or emits two different edges for one output fails
+    # here -- named, and in under two seconds -- rather than as a confusing Ninja parse error in
+    # the step below.
+    - name: "Generate the build graph"
+      run: |
+        uv run python -m tools.ninja.generate --scope all
+
+    # One real round, end to end: Jinja, pandoc, the pictures and two xelatex passes. The scope
+    # is read out of the graph rather than written here, because it is this repository's own
+    # layout and a path typed into CI is a second copy of it that goes stale silently -- which
+    # is exactly what the commented-out line this replaces had done.
+    - name: "Build a seminar round"
       run: |
-        pwd
-        # pipenv run make output/seminar/test/01/1/1
+        set -euo pipefail
+        scope=$(./dgs-build -t targets all \
+                | sed -n 's|^output/seminar/\(.*\)/all: phony.*|\1|p' | head -1)
+        if [ -z "$scope" ]; then
+          echo "::error::no seminar round in the graph; is source/seminar/test populated?"
+          exit 1
+        fi
+        echo "building output/seminar/$scope/all"
+        ./dgs-build -k 0 "output/seminar/$scope/all"

.gitignore

@@ -13,6 +13,12 @@
 *.dblite
 .idea/
 build/
+
+# The Ninja spike (`tools/ninja/generate.py`). The graph is generated, never written by hand,
+# and `.ninja_log` / `.ninja_deps` are Ninja's own state beside it.
+build.ninja
+.ninja_log
+.ninja_deps
 render/
 output/
 source/

CLAUDE.md

@@ -47,7 +47,7 @@ stale entry is still worth catching in the sources rather than in a PDF. Both di
 are checked (`unit-unlisted`, `listed-missing`). The page can sort alphabetically instead,
 and keeps showing the competition number when it does. A pass over the whole repository is a fifth of a second, so
 nothing is cached — except the two checks that cannot be answered by reading files, which land in
-`build/.audit/` with a fingerprint of the sources: the build checks, which shell out to make, and
+`build/.audit/` with a fingerprint of the sources: the build checks, which shell out to the build, and
 the macro sweep.
 
 **There is a third direction, and it is the one make cannot fix: `build-orphan`.** Rename a
@@ -58,12 +58,38 @@ because `\protectedInput` finds the stale `.tex` and the booklet goes on printin
 problem instead of the red box it is owed; volume 19's `onion` was exactly that. So
 `listed-missing` is the error and this is the warning, and together they say what happened.
 
-`make stale` lists them and **`make prune` removes them** — two targets on purpose, since one
+`./dgs-build stale` lists them and **`./dgs-build prune` removes them** (`make stale` and
+`make prune` are the same two targets) — two targets on purpose, since one
 word between asking and deleting is not a safe thing to offer. `core/audit/stale.py` is the whole
 of it, and it is the one check that reads outside `source/`: a directory listing, no file opened,
 and a tree that has never been built reports nothing.
 
-**It looks at unit directories and nothing coarser**, which is the only reason it is usable.
+**It looks at two kinds of directory and nothing coarser**, which is the only reason it is
+usable. A **unit** directory, above; and a **language** directory, which the unit sweep cannot
+see, because `languages/` is not a place units live. That half is self-limiting rather than a
+rule about Náboj: a module with no `source/<scope>/languages/` has no notion of one and yields
+nothing, which is how seminar and scholar pass through.
+
+It is worth having for the same reason as the first half, only louder. The language documents in
+`modules/naboj/module.mk` are pattern rules, so `%` matches any string and the rest of each
+prerequisite list is `$(wildcard …)`, which yields *nothing* for a language that is not there
+rather than failing. A stale `build/<volume>/languages/<lang>/` therefore supplies everything
+they ask for: `make output/naboj/phys/29/languages/uk/booklet.pdf` built a **40-page Ukrainian
+problems booklet** out of intermediates whose sources had been removed, for a volume that has
+`en`, `es` and `sk`. Clearing `build/` was the only thing that turned it back into an error.
+
+Two changes came out of that, and they catch the two halves. Every language document now takes
+**`source/naboj/$*/meta.yaml`** as a prerequisite — the one thing in those lists not derived from
+something else — so a cold build of a language that does not exist fails at once and names the
+missing file. And this check finds the warm one, where the stale `build/` is already there. The
+aggregates never had the problem: `output/naboj/%/languages` and `booklets` wildcard
+`source/naboj/$*/languages/*` and always did.
+
+`fks-naboj`'s ten language directories have no meta and are unbuildable by name as a result.
+They already were — it spells languages `slovak` rather than `sk`, so nothing in
+`SUPPORTED_LANGUAGES` ever matched them.
+
+The rest of the narrow rule, and why it is narrow:
 `build/` is full of directories that never had a source — `answers/`, `solutions/`, `build/core/`,
 every aggregate a template produces — so the obvious rule, *is there a source directory of this
 name*, calls **567** of them stale. What makes a unit different is that the audit already knows
@@ -556,7 +582,39 @@ terminal counts in, and a script that wants its own canvas writes
 
     set terminal @gpterm size 432*gppt, 108*gppt
 
-A script that wants the default writes nothing at all, which is 39 of the 42.
+A script that wants the default writes nothing at all, which is 30 of the 42. Of the 12 that do
+not, nine are scholar's and were written this way; the other three are naboj's and were not --
+they still opened `set terminal pdf size 6, 1.5` when the web path was added, and so did
+`FKS/42/1/1/03`'s `blocks:` preamble, which is a thirteenth script that no `.gp` on disk carries.
+All four say `@gpterm` now.
+
+> **Open: scholar's nine say `@gpterm` and then set a font of their own, and none of them
+> draws.** Every one of them -- the `monte-carlo.gp` of `TA1/2018`, `2019`, `2022`, `2024` and
+> `2025`, the `voigt.gp` of `2018`, `2019` and `2022`, plus the copy inside `TA1/2025/2025/` --
+> opens
+>
+>     set terminal @gpterm size 510.236*gppt, 311.811*gppt font "Minion Pro, 10"
+>
+> and `@gpterm` already carries the font, so pdfcairo refuses the line outright: *Duplicated or
+> contradicting arguments in cairo terminal options*. They have been broken since `@gpterm`
+> landed and nothing said so, because nothing built scholar until the Ninja graph did. It is
+> one failing edge today, `build/scholar/TA1/2025/homework/03/monte-carlo.pdf`, and it is the
+> only thing between `TA1/2025` and a clean build.
+>
+> **Deliberately not fixed, and the reason is that the fix is a choice.** Deleting the trailing
+> `font` makes all nine build at once, at the 6pt the build supplies rather than the 10pt their
+> authors asked for on a 510pt canvas -- visibly smaller labels on somebody else's drawings.
+> Letting `@gpterm` take a size instead is a change here rather than in the content, and it is
+> the only way a script can ask for larger type without re-specifying the face. Neither is
+> urgent: scholar is not being run, and Náboj is.
+
+**`set terminal pdf` left in a script is not a print defect and is a web one**, which is what
+made it survive. Re-issued over a `pdfcairo` that is already current it keeps the font *and* the
+`fontscale`, so the printed figure is unchanged -- all six of those plots are pixel-identical
+across the conversion, and so is volume 29's whole booklet. On the `svg` terminal it is simply a
+different terminal, so gnuplot writes a PDF into a file named `.svg` and the page shows a blank
+figure. Hence the sniff below, and hence the rule that the terminal is the build's to supply
+even where leaving it in the script appears to cost nothing.
 
 Three things about that, each of which cost a wrong picture to find:
 

Containerfile

@@ -112,6 +112,12 @@ FROM common AS tex
 # --no-install-recommends silently drops dvisvgm and ghostscript, which is why
 # both are named: Makefile:208 runs pdfcrop (ghostscript) and :226 dvisvgm.
 #
+# mupdf-tools is mutool, which is dvisvgm's PDF reader now: Ghostscript dropped
+# the interpreter dvisvgm drove in 10.01.0 and trixie ships 10.05, so without it
+# every `dvisvgm --pdf` in the image dies on "either Ghostscript < 10.01.0 or
+# mutool is required" -- i.e. all of the HTML maths. Ghostscript stays anyway,
+# for pdfcrop and for PostScript specials.
+#
 # texlive-full is 5-6 GB and drags in CJK, ConTeXt, asymptote and the doc tree.
 # This list is about 2.4 GB. texlive-fonts-extra is 1.7 GB of that and is not
 # optional: MnSymbol (dgs.cls:93), esvect (:100), ifsym and skull (fonts.tex:51)
@@ -125,7 +131,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
         texlive-lang-german texlive-lang-french texlive-lang-polish \
         texlive-lang-spanish texlive-lang-portuguese texlive-lang-cyrillic \
         lcdf-typetools fontconfig \
-        ghostscript dvisvgm librsvg2-bin poppler-utils gnuplot-nox \
+        ghostscript dvisvgm mupdf-tools librsvg2-bin poppler-utils gnuplot-nox \
         pdftk-java zip rsync \
     && rm -rf /var/lib/apt/lists/*
 
@@ -135,7 +141,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 RUN missing=""; \
     for tool in tex fontinst vptovf pltotf otfinfo otftotfm cfftot1 kpsewhich \
                 mktexlsr updmap-sys perl \
-                xelatex texfot pdfcrop pdfjam pdftk pdftoppm dvisvgm \
+                xelatex texfot pdfcrop pdfjam pdftk pdftoppm dvisvgm mutool \
                 rsvg-convert gnuplot zip rsync fc-cache; do \
         command -v "$tool" >/dev/null || missing="$missing $tool"; \
     done; \

core/audit/__init__.py

@@ -3,7 +3,8 @@
 
 Two layers. This one is source-only: it reads every file once and runs a registry of checks over
 the result, which costs about a fifth of a second for the whole repository, so nothing needs to be
-cached or made incremental. `core.audit.build` is the other one, and it shells out to make.
+cached or made incremental. `core.audit.build` is the other one, and it shells out to the
+build -- to whatever runner its caller injects, which is the editor's Ninja one.
 
     from core.audit import audit
     report = audit(Path('source/naboj'), 'naboj', 'phys/28', units)

core/audit/build.py

@@ -2,8 +2,8 @@
 The audit checks that cost minutes rather than milliseconds: does it render, does it compile.
 
 Kept apart from `core.audit.checks` because everything there is a file read and a regex, and this
-shells out to make. Run one scope at a time, when asked, and cache the answer -- the source-only
-layer can afford to recompute on every request and this cannot.
+shells out to the build. Run one scope at a time, when asked, and cache the answer -- the
+source-only layer can afford to recompute on every request and this cannot.
 """
 import json
 import re
@@ -56,14 +56,15 @@ def run_targets(repo_root: Path, module: str, scope: str, languages, targets, *,
     """
     Build every target of every language, collecting what came back.
 
-    `run` is injected rather than called directly so the Flask app can pass its own make runner --
-    the one that holds `BUILD_LOCK` and strips ANSI -- and so this is testable without a toolchain.
+    `run` is injected rather than called directly so the Flask app can pass its own build
+    runner -- the one that holds `BUILD_LOCK` and strips ANSI -- and so this is testable without
+    a toolchain.
     """
     results = []
     for language in languages:
         for target in targets:
-            make_target = f"output/{module}/{scope}/languages/{language}/{target}.pdf"
-            outcome = run(make_target, timeout=timeout)
+            build_target = f"output/{module}/{scope}/languages/{language}/{target}.pdf"
+            outcome = run(build_target, timeout=timeout)
             log = RE_ANSI.sub('', outcome.get('log', ''))
             overfulls = sorted(
                 ({'box': m['box'], 'size': float(m['size'])} for m in RE_OVERFULL.finditer(log)),

core/audit/checks.py

@@ -825,11 +825,17 @@ def build_orphan(sources):
     without its sweep.
     """
     for orphan in stale.orphans(sources):
+        # The tail differs because the consequence does. A stale unit is pulled in by
+        # `\protectedInput` and prints the problem that was deleted; a stale language is a whole
+        # booklet that still builds for a translation nobody has any more.
+        consequence = (f"if {sources.scope}/meta.yaml still names it, the booklet is printing "
+                       f"this copy" if orphan.kind == 'unit' else
+                       "the language documents are pattern rules, so this is enough to build a "
+                       "whole booklet from")
         yield Finding('build-orphan', 'warning',
                       f"`{orphan}` has no source directory left -- {orphan.files} "
                       f"file{'' if orphan.files == 1 else 's'} from a rename or a deletion. "
-                      f"`make prune` removes it; if {sources.scope}/meta.yaml still names it, "
-                      f"the booklet is printing this copy")
+                      f"`dgs-build prune` removes it; {consequence}")
 
 
 @check('translation-set', 'warning',