Skip to content

Guest files: write-combined NVN pools stay anonymous (DM_GUEST_FILE_POOL_ANON, off by default) - #2815

Draft
dpadr wants to merge 2 commits into
seedds:mainfrom
dpadr:pepper/fixb-anon-pools
Draft

dpadr wants to merge 2 commits into
seedds:mainfrom
dpadr:pepper/fixb-anon-pools

Conversation

@dpadr

@dpadr dpadr commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

With the guest's memory in files (DM_GUEST_FILE=1), the memory the CPU rewrites for the GPU every frame lives in the shared file mapping. Every page of it is dirty again within a frame, so the system's periodic flush writes it back over and over.

On Ocarina of Time (cut9, a 25-minute route at cap 30 on a 16 GB M2 Air), the game thread has a 45–111 ms frame on an exact 30.0 s wall-clock grid:

  • With guest files on: 2.26 such episodes a minute, 80 % of them on the grid.
  • The same route with DM_GUEST_FILE=0: 0.47 a minute, none on the grid.

Turning guest files off is not the fix: that run read 1.3 GB back from swap.

This PR keeps just the write-combined pools out of the files.

Changes

  • Pool census (hle_nvn.c): counters at MemoryPoolInitialize / Finalize. One [nvn] pool census log line, written with the pool tables line when the counts change, gives:

    • live pools and bytes by flags;
    • the CPU-written, GPU-read share;
    • its peak.

    t_pool_notes checks the count, bytes, share and peak.

  • DM_GUEST_FILE_POOL_ANON=1 (test switch, off by default; hle_nvn.c + hle_svc.c):

    • Which pools: at MemoryPoolInitialize, before the no-copy buffer wraps it, a write-combined pool the GPU reads gets anonymous memory with the same bytes. That means CPU_UNCACHED with GPU_UNCACHED or GPU_CACHED, not shader code, not virtual, at most DM_GUEST_FILE_POOL_ANON_MAX (default 256M).
    • How (dm_guest_anon): copy the bytes, then mach_vm_remap with VM_FLAGS_OVERWRITE, so the range is never unmapped. Partial host pages at either end stay file pages.
    • Finalize writes the bytes back to the file and maps the file again. guest_release does the same for a pool freed without Finalize, before main's hole punching runs, so the punch still sees file pages.
    • CPU_CACHED pools stay in the files. Ocarina of Time keeps its assets there (census: 3.6 GB of flags 0x24, against 0.24 GB of CPU_UNCACHED pools in ~800 pools).
    • The census line counts the anonymous pools.
  • docs/env_switches.tsv: the two switches.

Tests

t_svc_memory's new anon_pools case runs in file mode. It checks that:

  • the bytes are kept;
  • the range is private, and the partial first page is still shared;
  • msync reaches no file;
  • Finalize writes the file and maps it back;
  • a release without Finalize does the same.

Results (iMac-built arm64, run on an M2 Air):

  • t_svc_memory / _unlimited / _16k / _dirty, each with DM_GUEST_FILE=0 and =1 (65 guest files in file mode): all 8 pass
  • t_pool_notes (Metal debug layer): 93 checks, 0 failed

Not measured yet: whether the switch removes the 30-s hitch. An earlier form, on a standing scene that had no flush to remove, moved 229 MB out of the files for +0.09 GiB footprint and no time change. An A/B on the route that shows the grid (cut9 with this commit, POOL_ANON 0 vs 1, A B A B, ~12 grid periods a run) is queued. This PR stays a draft, with the switch off, until it reports.

🤖 Generated with Claude Code

seedds and others added 2 commits November 15, 2026 18:30
…PU-read share

Counters at MemoryPoolInitialize / Finalize, one log line with the pool tables line when they change. Sizes what keeping the CPU-written pools anonymous under guest files would cost (notes/2026-11-11-salt-hotset-plan.md). t_pool_notes checks the count, bytes, share and peak.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_GUEST_FILE_POOL_ANON)

With the guest in files, the system's periodic flush writes the memory the CPU rewrites for the GPU every frame: a stall every ~30 s and steady SSD writes. At MemoryPoolInitialize, before the no-copy buffer wraps it, a write-combined pool the GPU reads (CPU_UNCACHED with GPU_UNCACHED or GPU_CACHED, not shader code or virtual, at most DM_GUEST_FILE_POOL_ANON_MAX) gets anonymous memory with the same bytes (dm_guest_anon: copy, then mach_vm_remap with VM_FLAGS_OVERWRITE, so the range is never unmapped; partial host pages at the ends stay file pages). MemoryPoolFinalize writes the bytes back to the file and maps it again; guest_release does so for a pool freed without Finalize. CPU_CACHED pools stay in the files: Ocarina of Time keeps its assets there (census: 3.6 GB of 0x24 against 0.24 GB of CPU_UNCACHED pools). The pool census line counts the anonymous pools. Off by default (test switch).

t_svc_memory anon_pools (file mode): bytes kept, the range private and the partial first page shared, msync reaches no file, Finalize writes the file and maps it back, a release without Finalize too. Passes on an M2 Air (iMac-built arm64): t_svc_memory, _unlimited, _16k and _dirty, each with DM_GUEST_FILE=0 and =1; t_pool_notes (Metal debug layer) 93 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added host The Horizon host: SVCs, services, threads, memory (dm/host) nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation labels Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation host The Horizon host: SVCs, services, threads, memory (dm/host) nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants