diff --git a/quest/m1/README.md b/quest/m1/README.md index b518c5d3455..3491c2697fc 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -53,6 +53,7 @@ moves what remains of them once their claims merge. - [Vendor and trim the WebRTC stack](/quest/m1/webrtc-vendor.md) - str0m, dimpl, `is`, and sctp-proto live in moq-webrtc with unused code removed, so moq-rtc can take rind - [MoQ on rind](/quest/m1/rind/README.md) - every MoQ crate takes its crypto from rind, a pure-Rust graviola fork, so no build compiles C crypto - [VAAPI encode and decode](/quest/m1/video-vaapi.md) - H.265 encode and decode, pre-generated bindings, and pooled resize surfaces, including the moq-dev/vaapi release that carries them +- [VA-API encode is correct on radeonsi](/quest/m1/vaapi-radeonsi.md) - AMD's VA-API H.264 encode decodes, or the node is refused for OpenH264 - [Tests run on a mocked clock](/quest/m1/mock-clock-tests/README.md) - about 300 tests stop proving nothing happened by waiting, sleeping to settle, or asserting on elapsed time - [moq-mux reads H.265 SPS without scuffle](/quest/m1/h265-sps-reader.md) - moq-mux's own SPS reader replaces scuffle-h265 and scuffle-expgolomb, and the fuzz regressions still pass diff --git a/quest/m1/vaapi-radeonsi.md b/quest/m1/vaapi-radeonsi.md new file mode 100644 index 00000000000..3762d34aaaa --- /dev/null +++ b/quest/m1/vaapi-radeonsi.md @@ -0,0 +1,58 @@ +# [M] VA-API encode is correct on radeonsi + +## Goal + +On an AMD RX 9070 (radeonsi), moq-video's VA-API H.264 encoder either +produces a stream that ffmpeg decodes, or refuses the node, so `Kind::Auto` +falls to the next H.264 encoder (OpenH264 in a build that has it; a build +without one gets `NoEncoder` naming the refused driver). Its VPP NV12 conversion carries no chroma offset, and its +rate control stays near the target bitrate. The ignored VA-API encode tests +in `rs/moq-video/src/encode/backend/vaapi.rs` pass on that card with +`MOQ_VAAPI_DEVICE` naming its node, or skip with a reason that names the +refused driver. + +## Plan + +Found 2026-11-06 in [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013)'s +hardware run on an RX 9070 (Mesa 26.0.8, `renderD129`): 6 of 19 VA-API tests +failed, all of them H.264 encode. A 320x240 dump gives ffprobe +`Invalid data found when processing input`, and its first NAL unit has type +0. This also reproduces on main's moq-vaapi. H.265 encode and both decoders +pass there. [moq-dev/vaapi#10](https://github.com/moq-dev/vaapi/pull/10) +also noted a chroma offset in radeonsi's VPP NV12 output and a bitrate +overshoot. + +Decided 2026-11-06 (maintainer, /quest-unblock): ✅ keep #8013 queued, which +puts `vaapi` on by default in moq-cli and moq-transcode, and fix this in m1. +Until it lands, an AMD-only host encodes H.264 through VA-API and publishes +a stream nobody can decode; moq-c and moq-ffi already did. Not chosen: +pulling #8013 to refuse radeonsi first, or keeping `vaapi` opt-in. Also +decided: ✅ find the root cause in m1. Not chosen: m3 beside +[Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md), or only a note +there. + +- Find whether the H.264 bytes are ours. NAL type 0 at the start points at + how moq-vaapi reads the coded buffer (a segment's `bit_offset`, or more + than one segment) or at packed headers radeonsi does not take for H.264. + [moq-dev/vaapi#10](https://github.com/moq-dev/vaapi/pull/10) had to + require packed H.265 parameter sets for the same reason. Compare against + `ffmpeg -c:v h264_vaapi` on the same node. +- If it is moq-vaapi's, fix it there with a test, release the crate, and bump + `moq-vaapi` here. If it is the driver's, refuse a radeonsi node for H.264 + encode, using the driver's vendor string, so the candidate fails to open and + OpenH264 takes over. Then link the Mesa issue. +- Fix the chroma offset and the bitrate overshoot if they are ours. If the + chroma offset is the driver's VPP, skip VPP on radeonsi and convert to NV12 + before upload. If the overshoot is the driver's, refuse a radeonsi node for + that encode, as for the H.264 bytes, so `Kind::Auto` moves on. Link the + Mesa issue either way. +- Vulkan Video still replaces VA-API on AMD later; this quest does not wait + on it. + +Public API: none. Wire: none. + +## Related + +- [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - AMD's long-term encode path +- [VA-API renderer tests](/quest/m4/vaapi-render-tests.md) - runs the VA-API tests on each render node +- [moq-vaapi vendor sync](/quest/m3/vendor-sync/vaapi.md) - the crate's upstream record diff --git a/quest/m3/README.md b/quest/m3/README.md index 4026818f465..b2672cd1395 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -61,6 +61,7 @@ federation) land. - [Plan: watch worker](/quest/m3/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides - [Watch worker](/quest/m3/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio - [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070 +- [H.265 encodes respect the encoder's minimum size](/quest/m3/h265-min-width.md) - an H.265 encode below AMD's 384-wide minimum is skipped by `Kind::Auto`, and the error names it - [VA-API encodes an external Vulkan image](/quest/m3/vaapi-vulkan-import.md) - an explicit-sync DMA-BUF reaches Intel's encoder, proven on Arrow Lake - [C++ FFI reuse](/quest/m3/cpp-ffi-reuse.md) - C++ packages link the shared default-feature FFI build once it is proven compatible - [OBS native codecs](/quest/m3/obs-moq-video/README.md) - replace FFmpeg video and audio decoding with moq-video and moq-audio, deliver GPU frames, and use native audio/video encoders diff --git a/quest/m3/h265-min-width.md b/quest/m3/h265-min-width.md new file mode 100644 index 00000000000..20f547e334f --- /dev/null +++ b/quest/m3/h265-min-width.md @@ -0,0 +1,47 @@ +# [S] H.265 encodes respect the encoder's minimum size + +## Goal + +An H.265 encode narrower than the hardware encoder's minimum never fails +with an opaque driver error. moq-video's encoder probe reports the driver's +minimum width and height, `Kind::Auto` passes over a hardware candidate +whose minimum the frame is below, and with no candidate left the error names +the minimum. The caller today is `moq import capture --codec h265` +(`CaptureArgs::codec`, `rs/moq-cli/src/publish.rs`). + +## Plan + +Found 2026-11-06 in [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013). +AMD's VCN 5 (radeonsi, Mesa `si_get.c`) does not encode H.265 narrower than +384. moq-video has no software H.265 encoder, so on an AMD-only host an +H.265 encode below that fails to open. + +moq-transcode cannot hit this yet: its `Config` has no output codec, and +`catalog::rung_entry` (`rs/moq-transcode/src/catalog.rs`) and `Rung::encode` +(`rs/moq-transcode/src/rung.rs`) leave `encode::Config` at its H.264 +default. Adding an H.265 output to the ladder is out of this quest's scope. +Whoever adds it applies the probe's minimum per rung (the default 240p rung +is 320 wide for a 4:3 source), and a failed rung probe must fail only that +rung: today `Pipeline::new` propagates it and aborts the whole transcode. + +Decided 2026-11-06 (maintainer, /quest-unblock): ✅ an S quest in m3. Not +chosen: folding it into [the radeonsi quest](/quest/m1/vaapi-radeonsi.md), +or dropping it. + +Decided 2026-11-07 (maintainer, /quest-complete): ✅ narrowed to the +capture encode (`moq import capture --codec h265`), an S in m3, with H.265 +transcode output out of scope. Not chosen: making H.265 transcode output a +prerequisite inside this quest. + +- Learn the minimum rather than hard-coding 384. libva has no minimum + picture-size attribute (only `VAConfigAttribMaxPictureWidth`/`Height`), so + it comes from the failed open or a per-driver table. Check what NVENC and + VideoToolbox report for the same case. + +Public API: maybe a minimum size on the encoder probe in moq-video. Wire: +none. + +## Related + +- [VA-API encode is correct on radeonsi](/quest/m1/vaapi-radeonsi.md) - the same AMD hardware run +- [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - may report a different minimum diff --git a/quest/m4/vaapi-render-tests.md b/quest/m4/vaapi-render-tests.md index 428637ada0f..407e8a29cff 100644 --- a/quest/m4/vaapi-render-tests.md +++ b/quest/m4/vaapi-render-tests.md @@ -41,6 +41,21 @@ which covers only NVIDIA. recipe's filter selects them. - No hosted runner has these GPUs, so this stays a local recipe until a GPU runner exists ([GPU CI](/quest/m3/gpu-ci.md) owns that decision). +- Added 2026-11-06 (maintainer, /quest-unblock) from + [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013)'s follow-ups: + - The VA-API tests open the first suitable render node, so a second GPU + is covered only when `MOQ_VAAPI_DEVICE` names it. `just rs gpu` runs the + VA-API tests once for each AMD or Intel render node, with + `MOQ_VAAPI_DEVICE` set; a node of either vendor without its VA-API + driver fails, and NVIDIA nodes are left to GPU CI. The renderer + fixtures allocate their NV12 and I420 buffers on that node and pick the + Vulkan adapter whose DRM render node matches it + (`VK_EXT_physical_device_drm`), so each run stays on one GPU. + - The H.265 decode tests gain an open-GOP stream with RASL pictures, and + a Main 10 stream holding 8-bit samples that decodes. They also gain a + 10-bit stream, which is refused, since moq-vaapi has no P010 output + ([moq-dev/vaapi#11](https://github.com/moq-dev/vaapi/pull/11)). + - Not chosen: dropping either gap. ## Related