Skip to content

DM_NVN_SHADER_ALT: an in-run A/B of any translated shader variant, and fsbench, an offline benchmark of one pixel shader - #2315

Open
dpadr wants to merge 12 commits into
seedds:mainfrom
dpadr:oot/shader-alt-tools
Open

dpadr wants to merge 12 commits into
seedds:mainfrom
dpadr:oot/shader-alt-tools

Conversation

@dpadr

@dpadr dpadr commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Two diagnostic tools for measuring what a translation choice costs in one hot pixel shader, offline and then in game. Both are off by default; nothing changes for a player.

DM_NVN_SHADER_ALT=<dir> (host, nvn_mtl.m):

  • A shader whose <key>_b.metallib is in that folder draws with that library instead of the game's. The library is a variant built the way dmsh_build.py builds the game's, with the same side libraries.
  • How it works. The variant gets its own shader object and its own program twin (prog_alt), so its pipelines and archive entries are separate: pso_key holds the shader objects, and lib_name tags the library alt/<file>. Variant pipelines are never written to the prewarm lists or the user's record.
  • Refusals. A variant whose side-library set (_bx, _bf, _bp, _bv), entry points or argument-buffer layout differ from the game's is refused with one [RED] line, and the key keeps the game's library.
  • A/B. DM_NVN_SHADER_ALT_AB=ab with DM_AB=<frames> draws the variants only in the odd phases, so one run compares them with the game's library in the neighbouring phases. The [ab] line of each phase is read as for any other DM_AB switch.
  • Switch unset: prog_get is prog_alt(prog_get_base(cp)), and prog_alt returns its argument after a dispatch_once check.

fsbench (dm/gfx/tools/fsbench/, a tool only):

  • It times one translated fragment shader and source variants of it on a full-screen draw, rebuilt from a DM_NVN_PASSDUMP_ALL pass dump and DM_NVN_CBDUMP uniform dumps.
  • For each variant it checks the output against the first library and prints Apple's compiler statistics (registers, spill, instruction mix).
  • The source variants are regex transforms of the translated MSL: border emulation off (nobs), no NaN scrub of coordinates, read-only storage buffers through constant, the pool read mark off, and others.
  • The scene format is in its README, because scenes are made from a game's dumps and stay out of the repository.

The method these tools were made for, used on Ocarina of Time (0400486024DEC000, v131072):

  1. Map which shaders cover the picture: DM_NVN_HILITE stills.
  2. Rank them by GPU time: DM_NVN_GPUPROF joined with DM_NVN_PASSDUMP.
  3. Bench the top ones offline with fsbench.
  4. Confirm in game with DM_NVN_SHADER_ALT and DM_AB.

The first result, on a MacBook Air M2 (16 GB):

The coverage and ranking scripts used for steps 1 and 2 are not in this PR: they are tied to one game's scripted route and saves.

Checked

  • Tests (job log in the work folder):
    • make -C dm/gfx/tests GAME=0400486024DEC000 check_t_shader_alt: the variant is drawn in its phases only; the twin is made once; archive tags; the pipeline key differs; nothing is recorded for the prewarm; a missing side library, a missing entry point, or no variant leaves the game's library.
    • check_t_pso_archive: OK. check_t_pso_bundle: 71 checks, 0 failed. check_t_pso_warm: 123 checks, 0 failed. check_t_nvn: 298 checks, 0 failed.
    • t_shader_alt: 15 checks, 0 failed (with the twin's reference count, with it and after it).
    • dmhost builds for 0400486024DEC000 with no warning on any line this branch changes.
    • python3 -m unittest dm/tools/tests/test_docs.py: OK.
    • dm/tools/env_switches.py --check reports only what it reports on main: 4 removed switches still quoted, DM_NVN_VCONV_KEEP without a row.
    • fsbench.py tool builds the binary on main, and fsbench.py compile compiles a fixture. The same source ran fsbench run and stats on five of Ocarina of Time's shaders (below).
  • Games run, with title ID and version: Ocarina of Time 0400486024DEC000 v131072.
    • The switch was used on cut6 (chain cd098f1631bd): 4 runs of 12,000 frames on a scripted route, the variants loaded in every run, and no [RED] line from the switch.
    • fsbench was used on five of its shaders, with every variant bit-exact. One scene bug was found on the way and fixed: the dump's target= is Metal's numbering, so a 2D array was read as a cube, and compare samplers had no compare function.
    • The branch on current main was compiled and tested, not played.
  • Devices and OS: MacBook Air M2 (10-core GPU, 16 GB), macOS 27.0.
  • Not run: scripts/regress.py check --games. Both switches are off by default, and with them unset the only change on the draw path is the prog_alt check, which returns at once. Not checked on an iPhone (it is a Mac diagnostic, but nothing in it is Mac-only).

Docs

  • docs/env_switches.tsv: rows for DM_NVN_SHADER_ALT and DM_NVN_SHADER_ALT_AB.
  • docs/development.md, GPU measurement: a paragraph on the in-run A/B of one shader, with a link to fsbench.
  • dm/gfx/tools/fsbench/README.md: commands, inputs, the scene format, and how to read the numbers.
  • No game files, keys or data extracted from a game in the change or the attachments
  • A chain tool changed: not applicable (the chain is unchanged; fsbench only reads its flags)

🤖 Generated with Claude Code

seedds and others added 6 commits November 11, 2026 00:30
…ternated with the DM_AB phases (diagnostic, off by default)

DM_NVN_SHADER_ALT=<dir> gives a shader key with <key>.metallib in <dir> a variant: a shader object of its own,
loaded with the game's reflection, its side libraries (_bx, _bf, _bp, _bv) from <dir>. prog_get returns the
program's twin with the variants in the phases DM_NVN_SHADER_ALT_AB (default on; ab, ab:<k>/<n>) selects,
so a draw uses one variant for all its state. Pipelines are other pso_key entries (the key holds the shader
objects) and other archive entries (lib_name: alt/<file>). A variant whose side libraries are not the game's
set, or whose entry points or argument buffer differ, is refused with a red line; the key stays the game's.
Combined programs (bind_stages) take the game's shaders, never a twin's. Layer entry points get no variant.

t_shader_alt: the twin in the switch's phases only, made once, its own pipeline key and archive name; no
variant, side-library mismatch and entry-point mismatch keep the game's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rm lists or the user's record

pso_skey names shaders by key, so a pipeline recorded from a variant would replay as the game's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… vertex inputs), which shader_load requires; stop when they do not load instead of dereferencing nil

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ce variants of it, on a full-screen draw rebuilt from a pass dump and its uniform dumps (time, output against the first library, Apple's compiler statistics); the scene format and its use are in its README, and development.md's GPU measurement section points to it and to DM_NVN_SHADER_ALT

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…signment, not memcpy, so it holds its own references: the copy left the twin's strong slots unretained, and replacing one then released the game's shader once too often (a program freed later would over-release its shaders). t_shader_alt checks the shared shader's retain count with the twin and after it. The ignored __unsafe_unretained on the program getters' return types is gone (clang's -Wignored-qualifiers), and a mixed && / || has its parentheses

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pool around its use ends, since the program getters' results are autoreleased

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
seedds and others added 5 commits November 11, 2026 04:23
…l's and a mesh stage's function; distinct pipelines)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion, and crun's multiset compare

On top of the PR's fsbench (which carried the work up to compare samplers and declared texture types), three-way merged from the fsbench series (base 619dddd4, head 04187b8a):
- variants mark1 (the pool read mark once a SIMD group when every active lane reads the same ID) and guard (seedds#2254's loop guard)
- scene: float textures get finite values below 1; the light-cluster grid filled as {count, list offset} pairs; a missing uniform block falls back to the same slot of another draw
- mstats: a mesh pipeline's statistics; every statistics dictionary with its key path (print_stages; FSBENCH_STATSKEYS=1 prints the top-level keys)
- a compute mode: cscene builds a dispatch's scene from a pass dump (uniform blocks from DM_NVN_CBDUMP, storage buffers by size, the dispatch's groups and the reflection's threadgroup size); cstats gives a kernel's statistics, threadgroup memory and threads; crun times a dispatch of each library against a null kernel and compares every written resource with the first library's from the same seeded inputs
- stats, run, mstats and cstats make functions with newFunctionWithName:constantValues: (none set): cut libraries declare function constants (dm_border_off) and Metal refuses a plain function from them
- crun also compares each written resource as a multiset of its 32-bit words (sorted, merge walk): Ocarina of Time's foliage kernels 510c344f and e855420a differ from copies of themselves in 8-17 words, as many by value as in place (racing writes or atomic accumulation), so a variant is judged against the first library's own difference with itself
- README: the new commands and the two notes

Checked: fsbench.py compiles, fsbench.m passes clang -fsyntax-only; the series' tools ran in OoT's measurements (cut7 stats of 06750e79 / 2a712d1a / d729234e, the cbdev crun timings, crunself3b's multiset counts, chainBstats2's mstats)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zed locals in translated MSL (open; zero-initialization proposed)

Ocarina of Time's full build (chain ecd9cb97f72c) logs 1,907 "variable '_N' is used uninitialized whenever 'if' condition is false" warnings, all of one shape: a switch on a small index structured with goto flags reads one flag's value after the join where only one path set it. In SPIR-V that is a phi with an OpUndef incoming (defined); SPIRV-Cross writes it as a local without an initializer, which MSL (C++) leaves undefined to read, and clang lowers the short-circuit to a branch on it. SPIRV-Cross --force-zero-initialized-variables in dmsh_build.py's spirv-cross arguments removed the warnings on the two measured shaders (d731a682, 58f47abf: 2 -> 0, same line count, both compile). Docs only: the chain change, its regress run and an fsbench exactness check are not done. Symptom index regenerated; OoT's note links the entry. test_docs passes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (open; not a chain rule)

A compute kernel that rereads many constant-buffer words hits Apple's 256 uniform registers: the compiler merges reads of `constant` memory and keeps the words in uniform registers, and the overflow spills (Ocarina of Time's foliage kernel cee08c63: 292 words read 973 times, 219 live merged, 1,168 bytes of spill). Reading those slots as `const device` removed the spill (0.32 -> 0.04 ms a dispatch, bit-exact) but costs below the limit (5-18 % slower at 219 uniform registers), and in game it gave about 0.14 ms of compute; a general rule would need a calibrated static count of merged words. Symptom index regenerated; OoT's note links the entry. test_docs passes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, mstats)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dpadr

dpadr commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Four commits added (24b9391):

  • fsbench:
    • compute and mesh modes;
    • cstats and mstats;
    • function-constant specialization (needed for libraries that declare constants, such as dm_border_off);
    • crun's multiset compare, so kernels whose outputs vary run to run (atomics) can still be checked.
  • Two reference entries, both open findings, not chain rules:
    • uninitialized locals in translated MSL: a goto flag read before it's set, with zero-initialization proposed;
    • constant buffers that overflow the uniform registers.

Checked: t_shader_alt 22/22, t_pso_archive, t_nvn 298/298, test_docs, and the fsbench tool builds (M2 Air, macOS 27.0).

…it has none

The generated vertex stage writes the fragment's interpolants, which stats read from <lib>.metal beside the library and, when that file was missing, silently
from an empty source: the vertex stage then wrote none and AGX refused every fragment with user inputs ("Fragment input(s) user(locnN) ... not written by
vertex shader"), which read as a stats failure of the shader. Now --msl <file> names the source, and stats stops with a message when neither is there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dpadr

dpadr commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

fsbench stats now finds the fragment's source explicitly (9745b04):

  • stats <lib> [fn] [format] --msl <file> names the MSL that the generated vertex stage takes the fragment's interpolants from. Without --msl it still reads <lib>.metal beside the library.
  • If neither is there, it stops with stats: no MSL source at … (exit 1). Before, it silently used an empty source: the vertex stage wrote no interpolants, and AGX refused any fragment with user inputs ("Fragment input(s) user(locnN) … not written by vertex shader"). That looked like a failure of the shader itself. An OoT run lost all 52 of its statistics to it.
  • Tested (iMac build): missing source → exit 1 with the message; --msl before or after [fn], and the beside-the-library form, give the same statistics for the same library.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants