Add native hardware video decoding, HDR output, and audio continuity

[nocell]

ARM64 guest applications spend substantial CPU time decoding video, while the existing eight-bit display path loses HDR output. This adds host VideoToolbox decoding, a paired ten-bit HDR presentation path, consistent SDR interface brightness, and a fix for reproduced HDA audio dropouts.

Closes #167. CPU/RAM settings remain separate in #156.

Demonstration

The two owner-supplied recordings show the same Tron 4K60 HDR source. The accelerated setup uses a 120 Hz guest display; this is not a claim of 120 FPS source playback. The recordings are resized to 1920×1242 for upload, retain their original timing and audio, and preserve ten-bit BT.2020/PQ capture metadata.

Before — hardware video decoding and guest HDR disabled (42.64 seconds):

try-omarchy-without-acceleration-and-hdr.mp4

After — hardware video decoding and native HDR enabled (60.52 seconds):

try-omarchy-4k60-hdr-demo.mp4

These are visual demonstrations, not an isolated decoder benchmark: both acceleration and HDR differ, and the window layouts differ. Both files use the Mac's HDR screen-recording format, including the recording of the SDR guest. Playback appearance depends on the viewer's browser and display. Neither recording measures physical panel luminance or proves frame delivery at 120 FPS.

Implementation

• Hardware-required VideoToolbox sessions support HEVC Main/Main10, AV1 Main and VP9 profiles 0/2. A supervised VA-API broker/helper exchanges bounded messages; private Mach capabilities pass decoder IOSurfaces to QEMU/ANGLE for GPU copies into VirGL textures. A bounded memory path supports CPU readback. Firefox retains its RDD seccomp sandbox.
• The paired virtio-gpu module exposes ten-bit scanout and HDR metadata. Hyprland composites BT.2020/PQ; QEMU presents through an RGBA16Float Metal/EDR layer. Shared GPU events and a bounded three-surface pool avoid CPU readback and completion waits. Drawable acquisition runs off QEMU's main loop; an activity token prevents App Nap throttling while allowing normal system sleep.
• The guest enables HDR only after capability negotiation. Scoped Mesa/mpv builds handle P010 and Wayland color metadata; Vivaldi enables Wayland color management. Older guest drivers keep the SDR path.
• The active host display preset supplies an SDR-white hint, with a 250-nit fallback. A Hyprland patch advertises the same white level to color-managed clients, correcting dim Chromium interfaces beside brighter wallpaper. Chromium can also adapt HDR midtones to this white level; mpv's explicitly targeted PQ output remains unchanged in the tested comparison.
• SDL's audio timer changes from 10 ms to 1 ms, preventing the reproduced full HDA-ring drops. PipeWire's existing quantum stays at 4096.
• Sources, patches, binary hashes and corresponding-source notices are included in the native guest packages and build integration.

Validation

Development machine: M5 Pro, 18 vCPUs and 12 GiB guest RAM. Measurements are specific to the tested streams and machine.

• Final make test passed with Python 3.12: 219 Swift tests in 48 suites, 87 guest unit tests, guest contracts, build/cache tests and macOS shell/storage suites. Native SDR-white and client-white policy tests passed separately.
• Rebuilt and verified the QEMU runtime, native-video/HDR packages and patched Hyprland. The actual Metal presenter test passed SDR → HDR → SDR transitions, 100/1000-nit signal values, primaries, orientation, GPU synchronization and bounded submission under a stalled presentation queue.
• FFmpeg: all 480 frames across four HEVC/AV1/VP9 streams matched software decoding exactly, including timestamps. GPU surface reuse and CPU readback were tested separately. Three alternating decode-only repetitions produced the medians below; guest CPU time excludes the host helper and is not total system power.

Decode-only stream	Hardware FPS	Software FPS	Hardware guest CPU s	Software guest CPU s
HEVC 4K, 10-bit	56.68	43.89	0.794	14.318
AV1 4K, 10-bit	115.16	17.34	0.403	103.364

Hardware decoding reduces guest CPU work but is not always faster for every codec. The AV1 software result is below real-time 60 FPS; this benchmark uses a different fixture from the Tron recordings.

• Foreground playback: the complete 126.56-second 4K60 YouTube test had zero drops across 7,529 measured frames. mpv's complete 4K AV1 test had one presentation drop and zero decoder drops. Subsequent HDR checks covered YouTube VP9 Profile 2 4K60, local AV1 10-bit 4K60 and HEVC Main10 4K30, with media time checked against wall time.
• Digital scanout checks confirmed Chromium white moving from approximately 205 to 597 nits for a 600-nit host hint, HDR highlights near 1000 nits, and unchanged explicitly targeted mpv PQ pixels across the SDR-white change. These are signal checks, not colorimeter readings.
• Audio A/B during muted 4K60 YouTube playback: the 45-second stereo tone changed from 37 discontinuities/channel and 43.3787 seconds retained audio to zero discontinuities and 45.0000 seconds. QEMU and SDL callback captures agreed.
• A clean factory image was assembled from an unprovisioned verified base, current overlays and verified packages, finalized/repacked with project scripts, and booted on a new user disk through graphical setup. No previous user state was imported. Source changes and added commit history were checked for private workspace references and common credential patterns before publication.

Remaining validation and tradeoffs

This remains a draft for architecture/implementation review. A full from-source Docker guest build is still blocked because pinned Rust 1:1.98.0-1 disappeared from the current Arch ARM repository; the clean-image validation reused the verified existing ttfx package without changing that pin. It does not establish a complete from-source rebuild.

HDR uses a private paired virtio extension and an exact 7.2.2-2-aarch64-ARCH kernel module. Optional private CoreDisplay APIs supply only a display capability hint and fall back safely when unavailable. The HDR surface pool adds about 190 MiB at 4K, excluding Metal drawables. Dolby Vision dynamic metadata is not transported. Other Mac generations, external displays, Bluetooth and physical acoustic output need separate validation. The 1 ms audio timer increases requested wakeups while audio is active.

See native video and native HDR for contracts, reproduction commands and detailed results.

[Fail-Safe]

@nocell && @themartiano:

Tested revision 38f7f2b7011002eb72d781147f9eb95960efba62 on an M3 Max with an isolated preview app and VM. There are some encouraging results here: the hardware decoder bridge works across the tested codecs, the HDR guest/presenter path initializes, and native Wayland presentation feedback reaches approximately 120 fps.

These results are from a local integration with build adjustments and a repaired guest dependency set, rather than a clean build of the unmodified PR.

Build fixes

1. Require the DRM VA-API backend in the private FFmpeg build

Our initial FFmpeg build completed, but hardware decoding failed with No VA display found. The configure link check could not resolve the guest libva-drm library’s transitive libdrm dependency, so FFmpeg was built without HAVE_VAAPI_DRM.

Adding the following build-time linker option to the generated compiler wrapper allowed that check to succeed:

-Wl,-rpath-link=$root/usr/lib

We also added a post-configure assertion so a missing backend fails the build instead of producing an unusable package:

grep -q '^#define HAVE_VAAPI_DRM 1$' config.h ||
  fail "FFmpeg DRM VA-API backend is required"

After rebuilding and installing the package, the decoder tests passed through the normal installed library paths. The -rpath-link adjustment only affects build-time dependency resolution; it does not add a runtime search path.

2. Build the mpv target explicitly

The unrestricted Ninja invocation attempted an auxiliary protocol-list generation step that executed a guest-linked binary in the builder and failed to load libass.

Building the target explicitly completed the executable used by the package:

ninja -C "$build/mpv-build" -j"$jobs" mpv

The full project test suite, app assembly, and strict signature validation passed with these adjustments.

Runtime results

Area	Observed result
Hardware video decoding	HEVC 8-bit, HEVC 10-bit, VP9 8-bit, and AV1 10-bit each produced the expected 120 frames: 480 frames total.
Guest CPU consumption	Hardware decoding used approximately 83–88% less guest CPU time than software decoding in the short test clips. Software decoding had higher maximum throughput; this does not establish total host CPU or power savings.
Installed mpv playback	Confirmed VA-API hardware decoding and playback to completion for 10-bit HEVC, but recorded 7 dropped frames during the four-second clip.
Multisampling	Both the default guest Mesa and the private HDR Mesa exposed four samples and passed actual multisample allocation, rendering/clear, resolve, and readback checks.
Native Wayland presentation	Approximately 119–120 fps from presentation feedback, including a modest rendering workload. This measures the virtual presentation path, not physical panel scanout.
HDR	Guest 10-bit output and BT.2020/PQ metadata handoff were observed; direct host presenter tests passed. End-to-end color accuracy remains unverified.
Chromium	WebGL 1 and WebGL 2 passed with explicit ANGLE/GLES selection, using VirGL rather than software rendering.

Remaining validation

• Audio continuity.
• Firefox.
• A controlled 60 Hz run. One attempted run was restored to 120 Hz by display synchronization, so it has not been counted as 60 Hz coverage.
• Longer playback and soak tests, including dropped-frame measurements.
• Visual HDR correctness.

Separate build reproducibility finding

Initial guest assembly encountered an Aquamarine/hyprtoolkit ABI mismatch in the available package set. We used a compatible local pair to continue testing. I would track that separately unless it can be tied directly to this change.

[Fail-Safe]

Follow-up to my earlier M3 Max testing: we now have a useful SDR-white calibration result and a repeatable HDR-highlight discrepancy to investigate separately.

These observations are from the local integration of this PR, subsequently updated to guest kernel 7.2.5-1-aarch64-ARCH, with temporary diagnostic instrumentation. They are not qualification of an unmodified checkout of the current PR head (38f7f2b7011002eb72d781147f9eb95960efba62).

SDR-white calibration

With Mac display brightness held at 50%, setting Hyprland's sdr_max_luminance to 105 brought ordinary white in the guest into close agreement with ordinary white on the host:

Comparison	Phone light-meter reading
Host ordinary white	14.2 fc
Guest ordinary white, SDR white 105	14.2–14.3 fc

The host/guest difference remained approximately 0.1 fc when rechecked after HDR video playback. These are relative phone light-meter measurements, not calibrated panel luminance or colorimetry. 105 is a useful calibration for this tested display/setup at 50% brightness, not a proposed universal default or a claim of 105 physical panel nits. Other brightness levels, display presets, and displays still need separate checks.

HDR highlights remain a separate issue

Keeping SDR white at 105 preserves visibly brighter HDR output, but the same HDR-white test video in host and guest Vivaldi produced:

HDR video comparison at 50% host brightness	Reading
Guest Vivaldi	63.7 fc
Host Vivaldi	90.1 fc

That is approximately 29.3% lower guest output, or 0.50 stops, while ordinary white remains matched. An earlier run was similarly around 64 versus 89 fc.

We corrected an initial resolution mismatch before the final readings. The final statistics showed 3840×2160 at approximately 59 fps and PQ/BT.2020 on both sides. A remaining confounder is that the host received AV1, YouTube variant 701, while the guest received VP9, variant 337. This is therefore not an identical-bitstream comparison, and we have not isolated decoding, guest tone mapping/compositing, or Mac presentation as the cause. Raising SDR white to compensate would discard the successful desktop calibration without resolving that question.

Digital signal checks

A temporary diagnostic sampled the GL framebuffer immediately before the existing PQ-to-Metal conversion. In controlled, focused white-page captures, requested SDR-white levels produced approximately:

Requested SDR white	Decoded red PQ signal luminance for near-neutral white samples
140	136.4–137.7
170	166.5–168.1
250	244.2–246.5

This confirms that changing the setting changes the guest's emitted white signal. Those values are digital signal luminance, not measured panel nits. Later neutral bright samples reached roughly 490–512 during the video investigation, but were not tied to a specific frame and should only be treated as a lead. The readback instrumentation can stall rendering, so these diagnostic runs are not performance evidence.

Persistence and next steps

The calibrated 105 setting is now saved in the local guest's monitor configuration and display-sync override. Live configuration reload and the display-change reapply path retained 10-bit HDR (XRGB2101010) and 105, with no configuration errors. Saving only a one-off hyprctl adjustment is insufficient: startup or display synchronization can replace it with the EDID-derived default. This is a local personalization, not an upstream patch; restart validation is still pending.

The useful next investigation is an equivalent, controlled HDR source with known signal values, traced through guest scanout and Mac presentation while retaining the SDR-white control. Current results establish a practical desktop-white match and HDR output above ordinary white, but do not yet establish host/guest HDR-highlight equivalence or end-to-end color accuracy.