Repository navigation
The build graph, generated for Ninja - #44
Merged
Merged
Conversation
`uv lock --upgrade`, not mine: twelve packages move, numpy 2.5.3 to 2.5.4, pint 0.26.1 to 0.26.2, flask 3.1.8 to 3.1.9, ruff 0.16.9 to 0.17.3 and the rest of the dev tools with them. `pyproject.toml` is unchanged, so nothing about what is required moved -- only what was resolved for it. On master rather than on `kvik/ninja`, which is where it was sitting, because it has nothing to do with that branch and everything else does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The upgrade changes how pint formats three units for siunitx, and this
repository declares its own macros for all three, so both halves of the
booklet's temperature typography went with them.
Up to 0.26.1 a Celsius quantity formatted as `\degree_Celsius`, which is not
valid TeX, so `PhysicsQuantity.PINT_TO_SIUNITX` caught it by name through
`_UNDERSCORE_MACRO` and wrote `\celsius`. 0.26.2 writes `\degreeCelsius`,
which *is* valid TeX and so never reaches that mapping at all: siunitx's own
`\degreeCelsius` sets a math ring operator with a space before the C, so
`25 °C` became `25 ∘ C` in the 146 source files that declare one.
`siunitx.tex` now declares `\degreeCelsius` as `\text{°}C` beside the
`\celsius` it already had, and the page is what it was.
Fahrenheit is worse and is a repair rather than a renaming. 0.26.2 writes
`\degree F` -- a macro and a bare letter -- and this document sets
`forbid-literal-units`, so siunitx raises *Literal units disabled* and the
unit disappears: `phys/23/yankees` printed `25` with nothing after it, in
five languages. `PINT_LITERALS` puts it back to `\fahrenheit`.
`\dcelsius` is collateral: pint now spells a temperature *difference*
`\degreeCelsius` too, so the rendered TeX no longer distinguishes it from an
absolute. Both set `°C`, so no page moves, and the macro stays for anything
written by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`make output/naboj/phys/29/languages/uk/booklet.pdf` built a 40-page Ukrainian problems booklet for a volume that has `en`, `es` and `sk`. The language documents are pattern rules, so `%` takes any string, and the rest of each prerequisite list is `$(wildcard …)`, which yields nothing for a language that is not there rather than failing. A target with no prerequisites whose file exists is up to date, so make reported success over whatever an older run had left in `build/`. Clearing `build/` was the only thing that turned it back into an error. Two halves, because there are two states. Cold: every language document now takes `source/naboj/$*/meta.yaml`, the one prerequisite in those lists not derived from something else, so a language that does not exist fails at once and names the missing file. Warm, where the stale `build/` is already there: `stale.py` splits into `unit_orphans` and `language_orphans` over a shared `_sweep`, and `build-orphan` says which it found, since the consequences differ -- a stale unit prints a deleted problem, a stale language is a whole booklet for a translation nobody has. The language half is self-limiting rather than a rule about Náboj: a module with no `source/<scope>/languages/` yields nothing, which is how seminar and scholar pass through. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every recipe in this build shells out to `python -m something`, and 92 % of what that costs is the interpreter coming up: 445 ms of wall and 1.10 s of CPU to import `core.builder.jinja` and the pint/numpy stack under it, against 53 ms to do the render once those are in memory. A volume booklet runs about 320 such steps. `server.py` holds the imports and forks a child per job; `client.py` is what the build invokes, and imports four stdlib modules so that it starts in 14 ms. Measured end to end: 0.457 s per render becomes 0.067 s. Fork, not a reused worker. The expensive state -- the pint registry at `core/builder/jinja.py` module level, i18n's thirteen parsed and validated locales, Jinja's caches -- is shared copy-on-write and read only, while anything a job mutates dies with the child. That is what separates this from a Bazel-style persistent worker, where state leaking between jobs is the standard failure. The fork costs about 2 ms of the 53. Three things it has to get right, each of which cost a round to find: - The length prefix rides in the same `sendmsg` as the descriptors. Ancillary data attaches to one byte of the stream, so a plain `recv` of a separate header can swallow that byte and the descriptors are closed without a word. - An acknowledgement byte divides "never started" from "failed". Before it the client falls back and runs the job itself; after it, a lost connection is a failure, because falling back would run half a job twice. - `OPENBLAS_NUM_THREADS=1` before the first import. `import numpy` spawns a pool at 630 % CPU for a build that does no linear algebra, and forking a process whose threads hold locks is how children deadlock. The CPU it saves is incidental. The daemon fingerprints every `.py`, `.yaml` and `.lua` under `core/`, `modules/` and `tools/` before each fork -- 176 files, 1.2 ms -- and retires rather than answer with code that is no longer on disk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`tools/ninja/generate.py` writes every edge of the build from a walk of `source/`, which is possible because no `$(wildcard …)` in any of the four makefiles reads `build/`: every `build/` prerequisite is `$(subst source/,build/,…)` of a `source/` glob, so the graph is a pure function of the source tree. 35 155 edges for the whole repository, 1.9 s to generate, 0.13 s for Ninja to decide there is nothing to do. What it is for is not speed. Make decides with pattern rules whose prerequisites come from wildcards, and a wildcard that matches nothing yields nothing rather than failing, so a rule silently does not apply and an existing target is declared current. That has hidden three defects on record -- `Makefile:20-23`, `Makefile:146`, and the Ukrainian booklet of the previous commit. Ninja has no pattern rules and no implicit search: a target either has an edge or it is `unknown target`, and it says which real one you probably meant. Checked against make rather than assumed: a volume's 142 `.tex` come out byte-identical and the 44-page booklet differs in one line, the colophon's build clock. `ink.py` carries the Makefile's four colours onto Ninja's `description` lines, so the build still says what it is doing in the words it used to. One departure: make's single `jinja` macro called everything a "Markdown template", which is right for a volume's 161 fragments and wrong for its 30 pictures, so `$kind` now names the format. A rule block may not carry a default for such a variable -- that is a parse error, not something Ninja ignores -- so every edge binds it. Nothing in `Makefile` or `modules/*/module.mk` is touched. The two drive the same trees, which is what lets them be diffed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`run_make` becomes `run_build`, the capability probe looks for `ninja` instead of GNU make, and the launcher's argv follows. `ensure_graph` generates `build.ninja` when it is absent, which is the one state Ninja cannot recover from itself -- it keeps the graph current through the `generator` edge, but a first run after a clone has none, and `ninja: error: loading 'build.ninja'` names a file nobody has heard of. The probe gets simpler rather than merely renamed. `probe_make` had to reject a make that was not GNU, because BSD make accepts the invocation and then misparses `.SECONDEXPANSION`, which every rule in this repository leans on. There is one Ninja and no dialects, so finding it is the whole test. This is also what lets `/api/compile` work again: its target is `modules/naboj/editor.yaml`'s `preview:`, `output/…/standalone.pdf`, which was missing from the graph until the previous commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Starting the editor the documented way -- `python tools/editor/app.py` from
the root, no activated venv -- reported **markdown, tex and pdf all
unusable** on a machine that builds all three. `_which('ninja')` searched
`PATH` and nothing else, and Ninja is a project dependency, so the one the
build runs is `.venv/bin/ninja`, which is on `PATH` only inside an activated
venv. A tier is never usable when one below it is missing, so that single
lookup took the whole pipeline down.
The launcher already knew better: `probe_python_launcher` prepends exactly
that directory when uv is absent, and returns `uv run ninja` when it is not.
The probe and the thing it probes for disagreed.
`_which` now falls back to the interpreter's own directory, which is the
same directory and the same reason, and `probe_ninja` runs the path it found
rather than the bare name -- the two are different questions once the venv
is not activated.
Verified against a running editor: all four tiers report true, `/api/render`
and `/api/tex` build, `/api/compile` writes `standalone.pdf` in 3.9 s, and
the audit's build checks run 15 of 15 targets across en, es and sk in 61 s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`run_make` became `run_build` when the editor switched over and the rest of
the names did not, so the module read half in one vocabulary and half in the
other: `make_env`, `make_result`, twenty-odd `make_target` locals, and the
docstrings around them. They are `build_env`, `build_result` and
`build_target` now, and `core/audit/build.py`'s injected runner with them.
Two of those were not cosmetic. `build_result` and `refusal` both wrote
`"command": f"make {target}"` into the payload the editor displays, so every
build the page reported named a command nobody had run -- and `app.js`
printed "make reported success but wrote no fragment" in the preview pane
for the same reason. Both say `ninja` now.
The two mentions of make that remain are deliberate: `tex_target` explains
what the Makefile's fall-through rule used to print for a `.gp.tex`, and
`missing_meta_result` quotes `No rule to make target` beside Ninja's
`unknown target`, because both name the PDF rather than the meta.yaml that
is actually missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every Markdown file was rendered once and converted once, to TeX. The second
writer was missing, and with it everything downstream of it: the HTML
fragments, the pictures in the format a browser wants, and the pages those
two assemble into. `ninja output/naboj/phys/29/problems/chopper/sk/index.html`
answered `unknown target`, which is the honest version of what make said --
its own `output/%.html` has been an XFAIL fall-through since the per-module
rules landed.
Four rules and one helper each side of them:
- `pandoc_html` hangs off the same `render/…md` as `pandoc_tex`. One Jinja
stage, two writers, which is the point: a second route to the web would
be a second chance for the page and the booklet to disagree about what
the problem says.
- `pdf_svg` and `gnuplot_svg` give every drawing its web twin. There is no
separate web picture -- a TikZ drawing and a PDF copied in are converted
from the artefact the booklet prints, and a plot is drawn a second time
by gnuplot, which is one step shorter and keeps the PDF parser out of the
chain for the files that provoked the conversion bug. A hand-drawn `.svg`
is copied.
- `page` runs `core.html.page`, which is the `output/` step: fragments in,
one `index.html` with its pictures beside it and its maths typeset.
`_page` takes the module rather than inferring it from the path, because
where a fragment lives is the module's to say and `page.py` reads the same
fact out of `modules/<module>/editor.yaml`: Náboj nests a directory per
language inside a problem, the other two do not. `_beside` matches the unit's
directory exactly rather than by prefix -- scholar's issue directory contains
its problems, and a prefix would hand an issue's page every drawing below it.
New names: `output/naboj/<comp>/<vol>/languages/<lang>/pages`, and `pages`
over all of a volume's languages; `output/seminar/<round>/pages`. Not folded
into `all`, which is the Makefile's reading too -- the documents are what a
volume is run from and the pages are a second output of the same sources.
Scholar gets the fragments and no pages, and that asymmetry is deliberate:
`modules/scholar/editor.yaml` says `targets: [text]` where 28 of its issues
hold a `content.md`, so `page.py` would find nothing to assemble and the edge
could only ever fail. The fragments are still right whatever the descriptor
calls them, and that is what the editor's HTML tab shows.
Measured on `phys/29`: 1774 edges against 1507, 118 of 120 pages in 64 s
warm, and the no-op decision still 0.006 s. The two that do not build are
`chopper`, whose `freq.gp` still opens with `set terminal pdf size 6, 1.5` --
the pre-`@gpterm` spelling, which overrides the terminal the build supplies
and makes gnuplot write a PDF into a file named `.svg`. The sniff in
`gnuplot_svg` is what says so rather than leaving a blank figure on the page.
Thirteen scripts across the repository are in that state, `FKS/42/1/1/03`'s
`blocks:` preamble among them; each wants
`set terminal @gpterm size W*gppt, H*gppt` and each changes the printed
figure's typeface when it gets it, so none is touched here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`UnitKind.html` derived `output/<module>/…/<target>.html` from the render target, and said so in its docstring: "the root Makefile has one `output/%.html: render/%.md` for every module". That rule became an XFAIL fall-through when the per-module `.html` rules landed beside the `.tex` ones, and this did not follow them -- so `/api/html` has been asking the build for a target nothing makes. Under make that is `Incorrect fall-through rule called`; under Ninja it is `unknown target`, which is how it was found. A fragment is an intermediate and belongs in `build/`, for the same reason a `.tex` does: no `<head>`, nothing to typeset its `\(…\)`, and `<img>` pointing at files that are not beside it. So `html` is now `tex` with the other extension, which is also what every `module.mk` writes. That moves where the preview pane's pictures resolve, and the fragment's own directory is the wrong answer for Náboj: a fragment sits inside the language directory and its drawings beside the problem above it. The server names the directory instead -- `html_base`, the unit's own `build/` directory, which is where the generator puts a picture in all three modules -- and `/api/build/` serves it, path-checked exactly as `/api/output/` is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The note said "a script that wants the default writes nothing at all, which is 39 of the 42", and 30 of the 42 do: nine of scholar's plots size their own canvas and were written with `@gpterm`, and naboj's three were not. Those three and `FKS/42/1/1/03`'s `blocks:` preamble still opened `set terminal pdf size 6, 1.5`; all four are converted now, in their own repositories. The reason they lasted is worth writing down, because it is the opposite of what the section implies. Re-issued over a `pdfcairo` that is already current, `set terminal pdf` keeps both the font and the `fontscale` it finds, so the printed figure is unchanged -- the six plots and volume 29's whole booklet are pixel-identical across the conversion. Only the `svg` terminal notices, because there it is a different terminal rather than a reset one, and gnuplot then writes a PDF into a file named `.svg`. Nothing was wrong until the web path was built, and then three problems' pages were blank figures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`build/seminar/FKS/.static/` was in no edge, so the first round document Ninja reached died on `File 'build/seminar/FKS/.static/format-override.tex' not found`. make had it as `copy-static`, one `cp -r` of the whole tree -- and a rule whose own target it never touched, so it had no prerequisites, re-ran on every build and told the documents nothing about what they read. Five files here, one copy edge each, named in the PDFs' implicit deps. FKS is the only competition with a `.static`; a module without one yields nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Its one block put a bare `\hline` between the restated problem and its solution. `\hline` is a tabular command, so in ordinary body text it is `Misplaced \noalign` every time -- not for this round, for every round of every seminar since the template was written. Nothing caught it because nothing builds that document: `output/seminar/%/pdf` asks for `problems` and `solutions` and leaves the third alone. A rule across the measure is vertical-mode material, so `\hrule`, with the `\par` that gets there and a `\medskip` on each side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An optician writes `+5 D` and never `+5 m^{-1}`, so the unit wants declaring
rather than spelling out. `forbid-literal-units` is on, which is what made
`FKS/42/1/3/05` fail -- the eye-model problem that asked for it -- and the
message it fails with, `Literal units disabled`, says nothing about which
unit or where.
`core/filters/siunitx.py` reads this file, so the web side needs nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The generator edge takes every source directory the graph was read from as an
input, which is how adding a problem or a translation re-runs it. Removing one
was the other half and did not work: an input with no rule that has gone
missing is a hard error --
ninja: error: rebuilding 'build.ninja':
'source/naboj/phys/29/problems/amusement-speed/es', needed by 'build.ninja',
missing and no known rule to make it
-- and an unrecoverable one, because the edge that would have noticed the
removal is the edge Ninja is refusing to run. Deleting a language left the
whole tree unbuildable until somebody ran the generator by hand, which is
exactly the state a `generator` rule exists to prevent.
A `phony` for each of those directories fixes it: a phony whose output does not
exist is simply dirty, so `build.ninja` is out of date, the generator reruns,
and the directory leaves the graph. Same trick CMake uses for its own manifest
dependencies.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`build.ninja` is generated and gitignored, so a fresh clone has none, and Ninja
has no bootstrap of its own -- it does not find a manifest the way make finds a
Makefile. What a new machine got was
ninja: error: loading 'build.ninja': No such file or directory
naming a file nobody has heard of and saying nothing about a generator.
The editor already handled this in `run_build`. That rule moves to
`tools/ninja/bootstrap.py` so there is one of it, and `./dgs-build` is the same
thing for a terminal: generate the graph if it is absent, then become Ninja.
How to invoke Ninja is `capabilities.probe_python_launcher`'s answer rather
than a second one of our own, so the editor and the command line cannot drift.
`exec` rather than a subprocess, so Ninja owns the terminal and Ctrl-C reaches
it. The shell half chooses an interpreter, which is the one thing that cannot
be done from Python; the venv goes before `uv run` because it is a process
cheaper and reaches the same place.
Not in `install.sh`, which is where one would first look. The graph is a
function of `source/`, cloned in step 2 and added to whenever another content
repository is, so a graph written during the install would be wrong by the end
of it; and `install.sh` is the PDF tier, which a translator never runs and
still needs a build. `ninja` itself arrives with `uv sync`, as a project
dependency, and `capabilities.py` already says so when it is missing.
`install.md` gets the section, including the three changes the generator edge
does not notice -- a new volume, a new template, a different scope set -- and
loses three references to make that the editor stopped making a while ago.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A full cold rebuild of the last volume of every content repository -- phys/29, chem/04, test/01, FKS/42, TA1/2025, STA/2023 -- turned these up. They are independent but they are one file and one sitting, and they share a shape: the generator deciding what counts as content. **A venue is set in its own language.** `output/naboj/phys/29/all` failed 34 documents at once on `File 'build/core/i18n/cs.tex' not found`. The venue edges took the *volume's* languages, and a venue need not be in one of them: 29 has `en` and `sk` and runs in Praha, Olomouc, Budapest, Gdańsk, Lisboa, Madrid, Toruń, Wien, Wrocław and Uzhhorod. Under make nothing showed, because `i18n.stamp` built every locale whether anything asked for one or not. Each venue's documents now take their own locale, and that locale joins the set `core_edges` emits -- which also fixes the narrow case, where `--scope phys/29` emitted two locales and the Czech venue had no rule to reach a third. The language is read out of the venue meta, the first time this generator reads a file rather than a directory listing, so the meta joins the generator edge's inputs -- otherwise changing `language:` would leave the graph pointing at the locale the file used to name. A line scan, not `yaml.safe_load`, because `tools/ninja/bootstrap.py`'s last-resort path runs this module under whatever `python3` is on PATH. 69 of the 144 venue metas name no language at all (`phys/02`, `20`-`25`, several of them empty); they get no dependency and no guess, and they cannot build today for a reason that is not the graph's: `contexts/buildable.py:66` subscripts `i18n.languages[venue.language]` outright. **A dot directory is not a unit.** `pathlib.glob` is not the `glob` module -- `*` matches a leading dot -- so `seminar/FKS/.template` became six scopes and 48 problems of scaffolding, and `scholar/STA/2023/homework/02/.priors`, a draft kept beside the real problems, was rendered and failed on a meta the schema refuses. 118 scopes become 112 and 1893 problems 1845. **Scholar's `.static/` is copied.** The same gap seminar's had: every document of a course `\input`s `source/scholar/<course>/.static/format-override.tex` and nothing built it. The comment here used to say the fix could not be checked because scholar built under neither system; it builds now, and `STA/2023` goes from five failures to none. While in there, `A and B or C` in the scholar scope discovery gets the brackets it meant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The picture list accumulated across every issue of a year and every document took all of it, so one drawing that would not render stopped the issues after it as well: `TA1/2025/homework/03/monte-carlo.gp` fails on its own line 1, and `04` -- which names no drawing of 03's -- never got built at all. It is collected per issue now and added to the year's total afterwards, so the count the generator prints is unchanged. An issue is a document; what it depends on is what sits under it. The other two modules were already right for their own shape: a naboj volume's documents really do take every problem's drawings, and seminar collects per round, which is its scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recorded rather than fixed. All nine of scholar's gnuplot scripts write `set terminal @gpterm size … font "Minion Pro, 10"`, and `@gpterm` carries the font, so pdfcairo refuses the line: *Duplicated or contradicting arguments in cairo terminal options*. They have been broken since `@gpterm` landed and nothing reported it, because nothing built scholar until the Ninja graph did. The fix is a choice rather than a repair -- drop the font and the labels come out at 6pt instead of the 10pt somebody chose for a 510pt canvas, or teach `@gpterm` a size, which is a change in dgs. Both wait; scholar is not being run and Náboj is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`modules/<module>/editor.yaml` names `output/<unit>/standalone.pdf` as its
`preview:` and `/api/compile` builds that and nothing else, so a module whose
standalone has no edge has an editor that renders Markdown, converts TeX and
then cannot show the page. The graph had 3523 of naboj's and none of the other
two, while make builds all three -- the one place where the spike had become a
regression rather than an addition, because `capabilities.py` and `run_build`
were switched to Ninja a while ago and the Makefile was not consulted again.
A second rule, because the two builders take the unit as one path where
naboj's takes the competition, volume, problem and language apart; `-l`
defaults to `source/<module>/`, so what they are given is the path below that
and not the one the graph uses. Scholar gets one for both shapes
`editor.yaml` declares -- the sheet with its `text.md`, and a problem below it.
The edge names only the unit's own sources, which keeps the property both
builders were written for: each reads its meta through a `read_yaml` that
answers `{}` to anything unparseable, so a preview compiles for the problem in
front of you even when the round around it does not.
Checked by building four of them -- two seminar problems, a scholar sheet and
a scholar problem -- one page each, the figure of `FKS/42/1/3/03` among them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of them had no edge, so a build command that worked yesterday answered `unknown target` today -- which is the right answer for a volume with no Ukrainian and the wrong one for a target nobody retired. `instagram.pdf` is built now. The round builder already wrote `instagram.tex` and the graph stopped there; it stays out of `all`, which is `modules/seminar/module.mk:270`'s own reading -- its line in the aggregate is commented out, and what a round is run for is its problems and its solutions. `output/.../instagram` splits that into one PNG per page, as make did. How many there are depends on how long the document came out, so the outputs cannot be declared: a stamp carries the date, the way `burst` already does, and a phony carries the name people type. Make had it as a file target whose recipe never created it, so it re-ran on every build. `pdf`, `problems` and `solutions` are phonies over what they were there, and the last two reach the HTML fragments as well, which is what `html-problems` and `html-solutions` were for. **`output/*/copy` is deliberately not here.** Those two are deployment -- naboj's rsyncs to `ago:/home/balaz/public_html/`, seminar's shells out to `dgs-copy.py` with a `$(user)` the Makefile never defines, so it has been passing an empty argument. A build graph that uploads to a host when you name a target is a surprise worth not building; they want to be a script, and until they are, `make` still has them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`./dgs-container ninja <target>` alongside `make`. It runs `./dgs-build` rather than `ninja`, because a fresh mount has no `build.ninja` and no image can carry one -- the graph is a function of `source/`, which is bind-mounted from the host and deliberately never baked. No apt package was needed: `ninja` is a project dependency, so `uv sync` has already put it in `/opt/venv/bin`, which is first on PATH. CI gets the two steps its commented-out `make` line was standing in for. The generator runs on its own, so a crash or a pair of edges claiming one output is named in under two seconds instead of arriving as a Ninja parse error. Then one real round is built end to end -- Jinja, pandoc, the pictures, two xelatex passes -- which is the first time this repository has compiled anything in CI. The round is **read out of the graph**, not typed into the workflow: a path written there is a second copy of this repository's layout and goes stale in silence, which is what the line being replaced had already done. The testing repository is also cloned to `source/seminar/test` now, where `modules/seminar/editor.yaml` says it goes, rather than `.../testing`; nothing read it before, and a scope is named after its directory. The discovery line is tested here against `source/seminar/FKS`, where it finds `FKS/30/1/1` and the target resolves. The workflow itself is unverified until it runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`instagram.jtex` sets each problem on its own page and ends it with a
`\clearpage`, so the document's page count is the round's problem count --
a fact the generator already holds. The split is therefore not `burst`,
whose count genuinely is unknowable, and the stamp it borrowed is gone:
each PNG is a declared output of its own edge, named by `-f`/`-l` and
`-singlefile` rather than by pdftoppm's digit-padding rule.
The count is also checked, once per document, and every page hangs off the
check. Without it a document that came out longer would be cut into images
silently misnumbered -- `instagram-02.png` holding the tail of the first
problem rather than the second -- which is worse than no image at all.
pdftoppm does not treat a page number past the end as an error, it writes
nothing, so the per-page edges end in `test -f` as well.
**No round satisfies the invariant today**, which is what the check is for:
FKS/42/1/1 is 12 pages for 7 problems, 42/1/2 is 10, 42/1/3 is 12 and
41/1/1 is 9. The cause is the problems, not the template -- 1800 characters
at 12pt do not fit a 160mm square, and `\vspace*{\fill}` then spends a whole
page on the glue alone. `output/<round>/instagram` is red everywhere until
that is settled; it is in no aggregate, so nothing else notices.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`output/naboj/%/copy` rsynced a volume to `ago:/home/balaz/public_html/` and `output/seminar/%/copy` shelled out to `dgs-copy.py`, and neither was ever a build: they were a temporary workaround that stuck. A graph that uploads to a host because you named a target is a surprise worth not having, and it is the one kind of edge that cannot be undone by `rm -rf build/`. Both are gone from make and neither was ported to the Ninja graph, so deployment now lives nowhere on it. `dgs-copy.py` stays as the script it always was, to be run by hand. The seminar one had never worked anyway: `$(user)` is defined nowhere in the Makefile, so the recipe passed an empty fifth argument for as long as it has existed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ninja's default, unsaid, is every root of the graph -- here 33 929 outputs, the whole repository down to each unit's preview PDF. A bare `make` touched a stamp and stopped, so the two spellings of "I typed the command to see what happens" differed by about four hours of xelatex. `default core` now, where `core` is the i18n tables: core's own, needed by every other edge, and a second or two. `Writer.default` is the one line of manifest syntax that was missing. Along the way, `probe_advisories` learns the three tools that take a finished PDF apart -- `pdftk` for a volume's online burst, `pdftoppm` for a round's instagram images, `pdfjam` for a semester's imposition. None was listed, which is how this machine reports the PDF tier green, has no `pdftk`, and fails exactly one edge out of 59 355 with nothing having said why. They are advisories and not a gate, because most documents want none of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three sentences that named make for something the graph now does: the audit's build layer takes an injected runner and the editor's is Ninja's, `build-orphan`'s message told the reader to run `make prune`, and CLAUDE.md said the same. `make` still has both targets and still works, so this is accuracy rather than removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every formula on an HTML page goes through `dvisvgm --pdf`, and Ghostscript
dropped the interpreter dvisvgm drove in 10.01.0. So on anything that new --
Debian 13 ships 10.05.1, and so does the runner -- the conversion does not
degrade, it refuses:
ERROR: To process PDF files, either Ghostscript < 10.01.0 or mutool is
required. The installed Ghostscript version 10.05.1 isn't supported.
`mutool` is the answer dvisvgm itself names, and it picks it up by itself when
it finds it; `DVISVGM_PDF_PROC` forces the choice either way, which is what
this was verified with here, where Ghostscript is still 10.00. So the package
goes into the three places that install the toolchain -- CI, the container and
`install.sh` -- and `mutool` joins the two lists that fail by name rather than
letting a test skip itself: CI's prerequisite check and the image's.
The editor's advisories name it too, and deliberately without asking
Ghostscript's version: a machine where the old interpreter is still there is
one upgrade away from this, and there is no reading of a version that makes it
safe to be missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`canvas()` read a gnuplot PDF's `/MediaBox` with a regex over the raw bytes, which works only while cairo writes the page dictionary as a plain object. From 1.17 it writes it into a *compressed object stream*, so on a newer machine the regex finds nothing and the test dies as `NoneType has no attribute groups` -- with nothing to say that the PDF was perfectly good and the reader was wrong. It passed here on cairo 1.16 and failed in CI, which is the worst shape for a test to be in. `pdfinfo` answers the question instead. It is poppler, the same package as the `pdftocairo` this module already requires, so it costs no new dependency, and it keeps working whatever cairo does with its object layout next. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TomasVoros
approved these changes
Nov 4, 2026
TomasVoros
left a comment
TomasVoros
left a comment
Collaborator
There was a problem hiding this comment.
It works faster than before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The build moves from 1623 lines of GNU make to a Ninja graph generated from the source tree
by
tools/ninja/generate.py. The Makefile is not deleted and still works — this adds asecond way to build, proves it is better, and leaves the migration's last step to a separate
pass.
The failure this ends
Every recipe in the Makefile shells out to Python, pandoc or xelatex; the 1623 lines are path
algebra and dependency declaration, which is what make is worst at. The concrete cost is that
a stale file in
build/lets a rule silently not apply, recorded twice in the Makefile'sown comments and met again while writing this:
Makefile:20-23$(lang)was never set, so every picture rule passed an empty argument and died oninvalid choice— invisible, "because a stale intermediate inbuild/lets make skip the rule"make …/languages/uk/booklet.pdfbuilt a 40-page Ukrainian booklet for a volume that hasen,es,sk, out of intermediates whose sources were goneOne mechanism each time: a pattern rule whose prerequisites come from
$(wildcard …), whichyields nothing rather than failing, so the rule either does not apply — and an existing
target is declared current — or applies against leftovers.
Ninja has no globbing, no pattern rules and no implicit rule search. A target either has an
edge or it does not:
Ninja also records the command line per output and rebuilds when it changes, which make does
not. That alone would have caught
Makefile:20-23four years earlier.The numbers
.texbyte-identical; the 44-page booklet differs only in the colophon's build clockMost of the win is the warm interpreter, not the scheduler: one
python -mrendererinvocation costs 0.498 s of which 0.459 s is interpreter and import startup, and one
booklet is 162 of them.
tools/runner/is a fork server that pays that once.What is new
tools/ninja/generate.pywrites every edge from a directory walk. The whole dependencygraph is a pure function of
source/— no$(wildcard …)in the Makefile ever readbuild/— which is what makes this feasible at all.tools/ninja/fls.pyturns xelatex's-recorder.flsinto a depfile. It is the onlyhonest answer to
\protectedInput, whose include set is decided at run time against a pathbuilt from a TeX macro: touching
core/latex/symbols.tex, which is declared nowhere,correctly rebuilds the booklet and nothing else.
tools/runner/, the warm-interpreter fork server behind$run../dgs-build— generatesbuild.ninjawhen it is missing and then becomes Ninja, so afresh clone has nothing to remember. Verified on a worktree with no
source/, no.venvand no graph: it creates the venv, writes the graph and builds.
phonyper listed directory. The generator edge depends on the directories the graphwas read from, so adding or removing a problem re-runs it mid-build; the phonies are what
make a removal recoverable rather than an unrecoverable
missing and no known rule.What changed around it
core/audit/build.pygo through Ninja;probe_makebecameprobe_ninja,and the advisories now name
pdftk,pdftoppmandpdfjam../dgs-container ninja <target>; no apt package needed, sinceuv syncalready puts Ninjain the venv.
makeline it had was commented out.It now generates the graph as its own step and then builds a complete seminar round, with
the scope read out of
ninja -t targetsrather than typed into the workflow.default core— Ninja's unsaid default is every root of the graph, 33 929 outputs here,where a bare
maketouched a stamp and stopped.Defects found by building everything
Nothing had ever built scholar, and seminar's
solutions-fullhad never compiled at all:solutions-full.jtexused\hline, a tabular command, so everysolutions-full.pdfeverasked for died on
Misplaced \noalign;cs.texwhatever language the venue is in;title:where 73 metas saidname:;blocks:preamble still openedset terminal pdf,which silently wrote PDFs into files named
.svg;core/audit/stale.pygrew the language half of its sweep, which is what the 40-pageUkrainian booklet needed.
Deliberately not in this PR
output/naboj/%/copyrsynced to a host andoutput/seminar/%/copyshelled out todgs-copy.pywith a$(user)the Makefile neverdefined, so it had never worked. Neither was ported; a graph that uploads because you named
a target is a surprise worth not having.
dgs-copy.pyremains as a script.output/<round>/instagramis red on every round. The images are now one declared outputper problem, with the page count checked, and no round satisfies it — 42/1/1 is 12 pages for
7 problems, 41/1/1 is 9. The cause is the problems, not the graph: 1800 characters at 12 pt
do not fit a 160 mm square. It is in no aggregate, so nothing else notices.
instructions.pdf,naboj/test'smissing
tearoff.instruction, scholar's nine gnuplot fonts (recorded in CLAUDE.md).Verified
uv run pytest— 1793 passed, 10 skipped; 1613 passed in a worktree with no contentrepositories and no
build/, which is the rulecore/tests/is held to. A seminar round,a unit preview, an HTML page and the 44-page phys/29 booklet all build end to end.
Regeneration fires on a directory added and removed.
makestill plans the same booklet.The CI workflow itself is unverified until it runs here for the first time.
🤖 Generated with Claude Code