Report real free address space and attribute page boxes by holder

[athei]

Symptoms

The address-space watch's "free" figure is not free address space. In a GTA IV run (GTAIV-24284.log, 15 debug lines from line 216 to 22375), from line 7249 on the largest free block is larger than the free total, which cannot happen for real free space: 2030 MiB largest against 1836 MiB free at line 7249, and address space: 903 MiB free, largest free block 1182 MiB on the last line. The threshold warnings (1536, 1024, 768, 512, 256, 128 MiB) are driven by that figure, so they fire on resident growth, and an earlier run reported 255 MiB free while the 32-bit space was fine.

The same last line lists page boxes 1042 MiB while the holders it names account for about 430 MiB of it (texture staging 295, vertex/index backing 3, retained 2, and 126 parked in the recycle pool per the perf summary), leaving about 610 MiB with no owner in the log.

The periodic line is gated on mtld3d::d3d9 at debug, which also turns on every per-pass shader log of the layer and distorts frame times.

Cause

avail_virtual_mib read GlobalMemoryStatusEx. Wine computes ullAvailVirtual as ullTotalVirtual minus the process working set (dlls/kernelbase/memory.c, "approximate"), and on macOS the working set is the mach resident size of the whole process (dlls/ntdll/unix/process.c), the 64-bit host side included. The figure therefore falls as the process grows resident, whatever the 32-bit space holds.

The page-box gap has parts the line did not see. Texture staging was summed at the levels' requested lengths, while every level is its own page box rounded up to 16 KiB, so a chain of small levels holds several times what it asks for; cube face staging was not counted at all. Surfaces are not in the texture registry the watch walks, so the backing of system-memory offscreen plain surfaces, the staging of lockable render targets and back-buffer read-back pages were counted only in the total. The "retained" figure was the device's shared retention gauge, which the native encoder also charges with pages it allocates on the host side, outside the 32-bit space. Which part holds the 610 MiB in GTA IV is conjecture until the next run logs the new line; the rounding of small mip levels is the likely one.

Changes

The free figure is now the sum of the free regions of a VirtualQuery walk, each trimmed to the 64 KiB allocation granularity so a sliver no reservation can use counts nothing, from the same walk that finds the largest free block (crash::free_space, one walk, allocation-free so the exception handler can call it too). The tally is pure, in mtld3d_core::address_space::FreeSpace. GlobalMemoryStatusEx is no longer called: the working-set figure it gives under Wine says nothing about the address space and is easy to read as if it did. The crash lines (FATAL, fault outside d3d9.dll) carry free_mib and largest_free_mib from the same walk in place of avail_virtual_mib.

Two threshold latches watch the samples: the free total (1536 down to 128 MiB, as before) and the largest free block (512, 256, 128 MiB), so a fragmented space with plenty free in total still warns before a large allocation fails. A sample that crosses thresholds logs one warning per latch naming the lowest one crossed (threshold_step, pure and tested), then one address space map: line. The map used to log once, and only when the largest block fell below 512 MiB. The first sample only arms the latches: a process that starts below some thresholds, such as one without large-address-aware and its 2 GiB, logs one info line and reports only the lower ones, rather than warning about every threshold at its first present. The latches advance by compare-exchange, so two devices cannot report one crossing twice.

The page boxes are reported by holder, in padded bytes on the PE side so they add up to the total: texture staging (every level not dropped and every cube face level, split by pool and usage, with the requested total beside it), surfaces, vertex/index backing, encoder leases, the recycle pool, and other. A new HeldPages wrapper in mtld3d-core (held_pages) charges a page box to its holder's gauge while held, with the charged length stored so a replacement through DerefMut cannot skew the gauge. Surfaces hold their system-memory backing, lockable staging and read-back pages through it, and GuestOwnedPageLease holds its renamed backing or upload snapshot through it until the encoder's acknowledgment. The shared retention gauge is no longer in the line, since it counts host pages too. The warnings carry the same breakdown.

The watch logs on its own target, mtld3d::d3d9::mem_watch: the periodic breakdown at debug, the threshold warnings and the map at warn, so RUST_LOG=mtld3d::d3d9::mem_watch=debug turns on the breakdown alone. Nothing in the tree, the tests or the docs read these lines on the old target.

The walk now runs on every sample instead of only for the debug line. It runs after Present's stall timer has stopped, so it is not charged to the present block, and the debug line reports how many regions it walked and how long it took, in microseconds of the PE side's calibrated counter, so the next game run measures its real cost. In a test process it costs about 0.75 microseconds a region under Wine (150 to 240 microseconds for 225 to 255 regions, one outlier at 2.9 ms while four tests ran at once); no game has been measured yet. The sample interval goes from 120 to 600 presents, about five seconds at 120 presents a second, with the debug line every second sample, which keeps its old ten-second cadence. A 64-bit build skips the walk and the thresholds, since its space cannot run out and Wine took 9 to 19 ms to walk it in a test process, and keeps the breakdown, which still says what the layer holds in memory.

The new debug line, from an i686 test process:

address space: 4028 MiB free, largest free block 2048 MiB, walked 225 regions in 164 us; mtld3d holds 0 textures with 0 MiB of mip data; page boxes 0 MiB: texture staging 0, surfaces 0, vertex/index backing 0, encoder leases 0, pool parked 0, other 0; texture staging split default static 0 / default dynamic 0 / other 0, 0 MiB before page rounding; vertex/index backing split writeonly static 0 / dynamic 0 / other 0; locks on static default textures 0

docs/ARCHITECTURE.md gets a section on the watch and a row for the new target.

Alternatives considered

Keeping the GlobalMemoryStatusEx figure relabelled as resident memory. It is the resident size of the whole process, host side included, which Activity Monitor already shows, and it is the number that misled the earlier reading, so it is dropped rather than kept under a new name.

Tagging every PageBox with its holder and keeping one counter per holder in the page-box module. That attributes every byte by construction, but it changes every constructor and the pool's park and pop paths for a diagnostic. The texture walk the watch already does, plus one gauge each for surfaces and owned encoder leases, covers the holders the log could not see, and other keeps whatever is left visible.

Scaling the thresholds to the process's total user space (2 GiB without large-address-aware, about 4 GiB with it). The thresholds are headroom figures, and 128 MiB left is as close to failure in a 2 GiB process as in a 4 GiB one, so scaling would move the low ones the wrong way. Arming the latches at the first sample keeps the absolute values and only drops the ones a process starts below.

Walking the address space a slice at a time across presents to keep the sample interval at 120. The thresholds are a diagnostic whose point is the trail before a failure, and a five-second interval keeps that trail; the slicing would add state for no other gain.

Verification

• make fmt, then make check (formatting, Clippy on both PE arches and the Unix workspace, make audit, make doc): green.
• make test-unit: 1995 and 755 host tests passed, including the new address_space::tests (free-region sums, granule trimming of slivers, unaligned and aligned regions and the top of the space, the largest block never above the total, the first-sample arming, crossings once and after a recovery, the largest-block latch on a fragmented space, the microsecond conversion, the holder split and its text) and held_pages::tests (the surface gauge charges padded bytes while held, returns them on drop and on into_page, and returns the charged length after a DerefMut replacement).
• make test ISOLATED=1: the host tests as above, and the end-to-end suite on both legs with 0 failed (i686: 1072 passed, 13 ignored; x86_64: 1070 passed, 13 ignored; 6 processes each). The suite reads none of the watch's lines.
• RUST_LOG='mtld3d=warn,mtld3d::d3d9::mem_watch=debug' make test-e2e-i686 ISOLATED=1 FILTER='e2e::device::' produced the debug line above, 16 of them, each with the walk's region count and time.
• A reviewer should see, in a 32-bit game log with RUST_LOG=mtld3d::d3d9::mem_watch=debug, a free figure never below the largest free block, page-box holders that add up to the total, and the walk's cost in that game.

Rules check

Checked against CONTRIBUTING.md, docs/CONVENTIONS.md and docs/ARCHITECTURE.md. New statics holding mutable state, each with its argument in its doc block: SURFACE_BYTES and ENCODER_LEASE_BYTES in mtld3d_core::held_pages (the resource is process-wide, since the watch reads them against the one 32-bit address space and the process-wide page-box total; a surface can outlive its device and a lease retires after the frame that recorded it, so neither has a device record to give its bytes back to), and NEXT_LARGEST_THRESHOLD in mem_watch.rs (the largest free block is a property of the one address space). NEXT_THRESHOLD was already process-wide and is renamed NEXT_FREE_THRESHOLD; the MAP_LOGGED latch is gone. The test module held_pages/tests.rs has a GAUGE lock static, with the argument that the gauge it serialises is process-wide. Derives: ThresholdStep and ThresholdReport derive Debug, PartialEq and Eq for the tests' assert_eq!; no Clone or Copy, so the derive inventory is unchanged. FreeSpace derives nothing and has a hand-written Default that calls new, which Clippy's new_without_default requires. No config key, environment variable, wire field, dependency or lint suppression. The new log target follows the per-module LOG_TARGET pattern and has its row in the target table. The pure parts (the free-space tally, the threshold latch, the microsecond conversion, the holder split and its text, the held-pages gauge) live in mtld3d-core with their tests in <stem>/tests.rs; the VirtualQuery walk stays in windows/d3d9 and has one unsafe block with its SAFETY comment, shared by the free-space tally and the region map. make check enforces the doc-block shape, the tests-in-own-files layout, the derive inventory and the suppression ban, and ran green.