Skip to content

Share HiROM contracts and evidence while preserving execution banks - #199

Merged
mstan merged 4 commits into
mainfrom
fix/hirom-alias-contracts-193
Oct 27, 2026
Merged

mstan merged 4 commits into
mainfrom
fix/hirom-alias-contracts-193

Conversation

@mstan

@mstan mstan commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Standard HiROM contracts authored in C0 did not reliably reach physical aliases, and inherited near-dispatch targets could make low-bank execution call a full-bank compiled body. Share physical contracts while analyzing each execution bank separately.

  • Expand cfg contracts into valid physical aliases, preserving explicit local entry/exit overrides and friendly-symbol ownership.
  • Resolve near JMP/JSR dispatch targets in the executing bank in both decoders. Preserve absolute long targets and null slots.
  • Extend guarded XOR-80 dispatch to standard full-bank HiROM pairs without adding fallback checks to the established low-bank path.
  • Group physical coverage while retaining execution PCs, M/X context and replay qualification. Add standard HiROM layout/export and physical authority overlap checks; SA-1/SDD1 banking is unchanged.

Rebased onto current main 246c0d4; head 8a409ac. Range-diff confirms the three implementation commits are unchanged from qualified head 0dae9c8; the additional commit records the reporter's qualification.

Validation:

  • The inherited-dispatch regression reproduced the wrong-bank call before repair. All 12 HiROM integration cases pass.
  • Implementation validation: 115 Rust tests, rustfmt and Clippy with warnings denied pass. All three GitHub workflows are green on rebased head 8a409ac, including Linux/Windows/macOS analyzer jobs.
  • Current-main combined Share HiROM contracts and evidence while preserving execution banks #199/Clear the index high bytes when the X flag is set #214/Draw BG modes 4-6 with the compatibility renderer under NewRenderer #215 integration: 860 Python tests passed, 2 platform skips, codegen width lint, all six production PPU tests, and the runtime dispatch fixture pass. Additional emitted SEP/REP/XCE and 256/342/512/800-pixel framebuffer-bound checks pass. Final branch review/open-prs-final-20261026 differs from the tested tree only in this qualification documentation.
  • The reporter's updated Secret of Mana evidence qualifies 0dae9c8: boot, field/flight, cutscene and shop replays match the hand-alias baseline in WRAM, VRAM, CGRAM, OAM, framebuffer, master clock and current-build state checkpoints, in both compiled and interpreter-only runs.
  • Matched warmed Release timing pairs show no consistent runtime regression; startup and peak working set are unchanged within the reported measurements. Larger generation output and longer generation time are accepted by the owner.

The reporter-owned qualification is complete, and the earlier owner approval for #199 remains accepted. The new current-main #199/#214/#215 bundle at b19edfb also passes matched muted gameplay/current-build save-load checks in Mega Man X, A Link to the Past, Super Mario World and Super Metroid, with exact engine/title/UI/ROM/executable provenance retained. Brief diagnostic runtime observations show no consistent introduced regression; Super Metroid's initial slow sample did not reproduce in opposite-order repeats. SMW is paced near 60fps in Yoshi's House (wide-stage margins unverified); SM uses the established original Zebes loader and bypasses Ceres. The owner explicitly authorized merging #199/#214/#215 after confirming the completed combined regression tests. All three are merged; final main 503063e has exactly the tested b19edfb tree. No additional visual check was performed or claimed. This does not claim to fix SD3 intro issue #80.

Fixes #193. Tracked in beads-qc3k. Contract: docs/HIROM_ALIASES.md.

@pogoflow

Copy link
Copy Markdown
Contributor

Replayed this on Secret of Mana (USA), as asked in #193. Short version: it works, and we can drop our hand-written bank aliases. Two observations below.

Setup

  • Our fork state (cycle-parity work on top of f7bf13b) merged with upstream 4e959d3, then 5bd5792 cherry-picked. Both applied cleanly; the only conflict was in our own cache-key change.
  • Build A: without Share HiROM contracts and evidence while preserving execution banks #199, with our previous workaround. Our authority export wrote a copy of every instruction, entry, data region and dispatch from C0–C2 into 00–02.
  • Build B: with Share HiROM contracts and evidence while preserving execution banks #199, authority for C0–FF only, no hand aliases.
  • tests/test_hirom_aliases.py and the authority/layout/coverage tests pass on that merge.
  • Each script runs twice per build, once normally and once as an interpreter-only reference with every AOT node denied. Checkpoints compare WRAM, VRAM, CGRAM, OAM, framebuffer, master clock and the savestate blob.

Results

Script Frames Checkpoints A vs B, interpreter-only A vs B, compiled
boot, no input 3600 361 identical identical incl. WRAM and clock
title-screen B presses ~3500 177 identical identical incl. WRAM and clock
six save-game scripts (load, field, ring menu, shop, cutscene, flight) 2800–4600 281–392 each identical everywhere VRAM, CGRAM, OAM and frame identical at every checkpoint

In the compiled runs of the save-game scripts, the master clock differs by at most ±50 master clocks from about frame 1460 on. A few stack bytes ($01DE–$01F4) differ as well. This starts after the point (frame 1280) where our fork's compiled and interpreted tiers already disagree in both builds. The cause there is a known cycle-accounting gap in our fork: dispatch table bytes are not charged. Shifting code between tiers moves that gap around, so I don't attribute these offsets to #199.

Observation 1: expansion into every alias bank

Every authored entry in C0–FF is copied into 40–7D, 80–BF and the 00–3F upper halves, including banks the game never executes from:

A (aliases 00–02 only) B (#199)
roots 3,134 6,870 (2,470 in 40–7D, 961 in 80–BF)
exact variants 3,727 7,924
emitted banks 35 128
emit time 64 s 307 s
binary 27 MB 51 MB

The C0–FF part is unchanged. Would you consider restricting expansion to alias banks that are actually entered, for example from vectors, observed coverage or an explicit cfg list? Making the wholesale expansion opt-in would also work.

Observation 2: dispatch targets keep the authored bank

Copied indirect_dispatch sites keep their target list. For example, the site copied from C1:9542 to 01:9542 still lists C1:9566, C1:956D and so on. The generated switch at 01:9542 therefore calls the C1 bodies directly, but a JMP (abs,X) there stays in bank 01 on hardware.

Here this did not change anything visible, because the generated bodies read PB at runtime. It is, however, the full-bank-for-low-bank body substitution that docs/HIROM_ALIASES.md rules out. Our hand aliases rewrote such targets to 01:xxxx. With #199, 18 low-bank variants at six dispatch sites in 01/02 disappear for this reason.

Suggestion: when a target is in the site's own bank, move it into the alias bank together with the site.

mstan commented Oct 24, 2026 •

Copy link
Copy Markdown
Member Author

@pogoflow The objective review fixes are pushed in 0dae9c8, rebased onto current main. Inherited near-dispatch targets now resolve in the executing bank in both decoders; absolute long targets remain unchanged. A ROM-free regression reproduces the bank-00 -> C0 wrong-body call before the fix and passes afterward.

The owner accepts longer generation time and larger output for broader native coverage, so no demand-driven/opt-in expansion redesign is requested. I also isolated full-bank mirror fallback checks from the established low-bank path after a dispatch microbenchmark exposed overhead there.

The corrected five-PR bundle passes 856 Python tests (2 skips), 115 Rust tests, Clippy and the production dispatch fixture. The MMX/SMW/ALttP/Super Metroid gameplay and current-build save/load checks also pass, with brief paced runtime observations. These are LoROM titles; they cannot qualify your HiROM Secret of Mana project. The synthetic dispatch benchmark cannot establish its built runtime performance either.

Please rerun a representative existing gameplay/save-load replay on this updated head, checking the state/video checkpoints that previously qualified the aliases. Please also provide a brief matched, warmed Release runtime comparison against your hand-alias baseline, with audio output muted: build identities, frame count, raw host times/frame times and AOT/interpreter share. Note startup or memory regressions if observed. Keep it proportionate; expand or repeat only if a failure or noisy result warrants it. We need updated title correctness and built runtime evidence, not a generation-time reduction.

@pogoflow

pogoflow commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Reran this on 0dae9c8 with Secret of Mana (USA). Short version: no correctness difference against the workaround I used before, and runtime is the same within noise.

Builds

A B
roots / exact variants 3,595 / 4,443 8,122 / 9,877
emitted banks 35 128
emit time 88 s 278 s
binary 29.7 MB 58.7 MB

Correctness

Four scripts: boot without input (3600 frames), and three save-game scripts with load, field, ring menu, flight, shop and a cutscene with dialogue (3700–4600 frames). Each runs normally and as an interpreter-only reference with every AOT node denied. Checkpoints every 10 frames compare WRAM, VRAM, CGRAM, OAM, framebuffer, master clock and the savestate blob.

Script Checkpoints A vs B, interpreter-only A vs B, compiled
boot 361 identical identical
field + flight 371 identical identical
cutscene 461 identical identical
shop 391 identical identical

This time that includes WRAM and the master clock in the compiled runs too. The ±50 clock offsets from my last report were the dispatch-table charging gap in my fork, which is fixed now, so they no longer show up. Within each build, compiled and interpreter-only runs are also identical at every checkpoint.

B executes about 0.5–0.8% more compiled blocks over the same scripts. The low-bank dispatch fix works: every 00–02 variant that A has also exists in B (previously 18 were missing), plus 280 more.

Runtime

Uncapped (frame delay off), headless SDL video, audio enabled with volume 0. Whole-process wall time, peak working set and per-frame host time (run_frame + draw, measured in my game host around the two calls). One warm-up pair, then 5 pairs with alternating order (AB, BA, …). Same three save-game scripts, no dumps.

Script (frames) A frame ms, mean (median of 5) B paired Δ, median [range] p99 frame ms A / B peak WS A / B
field + flight (3700) 4.43 4.37 +0.5% [−3.1 … +19.7] 10.7 / 10.6 71 / 71 MB
cutscene (4600) 4.20 4.17 −0.4% [−4.2 … +1.4] 9.7 / 9.3 71 / 71 MB
shop (3900) 4.15 4.17 +0.5% [−3.0 … +2.3] 9.4 / 9.4 71 / 71 MB

Startup plus shutdown is about 2.1 s on both sides. Peak working set did not grow even though the binary doubled. Peak commit is within ±1%.

Interpreter share (separate TIER2 capture run, interpreted guest cycles over total master clocks): 9.39% → 9.38%, 10.34% → 10.33%, 10.10% → 10.09%. Most of that is wait loops and RAM-resident overlay code, which #199 isn't meant to change.

So from my side #199 is good to go.

@mstan
mstan force-pushed the fix/hirom-alias-contracts-193 branch from 0dae9c8 to 8a409ac Compare October 26, 2026 22:51
@mstan
mstan marked this pull request as ready for review October 26, 2026 22:51
@mstan
mstan merged commit be68fcb into main Oct 27, 2026
8 checks passed
mstan added a commit that referenced this pull request Oct 27, 2026
Clear the index high bytes when the X flag is set.

Qualified in the combined #199/#214/#215 regression batch at b19edfb: 860 Python tests, native checks, and four representative matched gameplay/save-load/runtime checks. Owner explicitly approved merge after regression confirmation.
mstan added a commit that referenced this pull request Oct 27, 2026
Draw BG modes 4-6 with the compatibility renderer under NewRenderer.

Qualified in the combined #199/#214/#215 regression batch at b19edfb: 860 Python tests, native checks, and four representative matched gameplay/save-load/runtime checks. Owner explicitly approved merge after regression confirmation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HiROM: bank mirrors are not folded in analyzer, emitter, runtime or coverage (eg. Secret of Mana)

2 participants