From f591d9a93b7aed888489d6c026a75e5f90f9a3d6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 16:10:01 -0800 Subject: [PATCH 1/3] quest: plan #8013's VA-API follow-ups radeonsi H.264 encode in m1, the H.265 minimum rung width in m3, and the per-node and H.265 decode test gaps folded into vaapi-render-tests. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 + quest/m1/vaapi-radeonsi.md | 55 ++++++++++++++++++++++++++++++++++ quest/m3/README.md | 1 + quest/m3/h265-min-width.md | 35 ++++++++++++++++++++++ quest/m4/vaapi-render-tests.md | 10 +++++++ 5 files changed, 102 insertions(+) create mode 100644 quest/m1/vaapi-radeonsi.md create mode 100644 quest/m3/h265-min-width.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 66c3452e9c0..4f8d4cecb3c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -55,6 +55,7 @@ moves what remains of them once their claims merge. - [MoQ on rind](/quest/m1/rind/README.md) - every MoQ crate takes its crypto from rind, a pure-Rust graviola fork, so no build compiles C crypto - [VAAPI encode and decode](/quest/m1/video-vaapi.md) - H.265 encode and decode, pre-generated bindings, and pooled resize surfaces, including the moq-dev/vaapi release that carries them - [Read a texture back without an encoder](/quest/m1/video-readback.md) - moq-video copies a wgpu texture into packed RGBA with no encoder probe, for the studio's Linux virtual camera +- [VA-API encode is correct on radeonsi](/quest/m1/vaapi-radeonsi.md) - AMD's VA-API H.264 encode decodes, or the node is refused for OpenH264 - [The over-limit PUBLISH refusal invites a retry](/quest/m1/retry-interval-overload.md) - `EXCESSIVE_LOAD` sends the may-pass retry interval, not 0 - [A relay forwards an upstream refusal's retry interval](/quest/m1/retry-interval-forward.md) - a forwarded SUBSCRIBE refusal keeps the upstream's retry interval - [Tests run on a mocked clock](/quest/m1/mock-clock-tests/README.md) - about 300 tests stop proving nothing happened by waiting, sleeping to settle, or asserting on elapsed time diff --git a/quest/m1/vaapi-radeonsi.md b/quest/m1/vaapi-radeonsi.md new file mode 100644 index 00000000000..7e440d8a19d --- /dev/null +++ b/quest/m1/vaapi-radeonsi.md @@ -0,0 +1,55 @@ +# [M] VA-API encode is correct on radeonsi + +## Goal + +On an AMD RX 9070 (radeonsi), moq-video's VA-API H.264 encoder either +produces a stream that ffmpeg decodes, or refuses the node, so `Kind::Auto` +falls to OpenH264. Its VPP NV12 conversion carries no chroma offset, and its +rate control stays near the target bitrate. The ignored VA-API encode tests +in `rs/moq-video/src/encode/backend/vaapi.rs` pass on that card with +`MOQ_VAAPI_DEVICE` naming its node, or skip with a reason that names the +refused driver. + +## Plan + +Found 2026-11-06 in [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013)'s +hardware run on an RX 9070 (Mesa 26.0.8, `renderD129`): 6 of 19 VA-API tests +failed, all of them H.264 encode. A 320x240 dump gives ffprobe +`Invalid data found when processing input`, and its first NAL unit has type +0. This also reproduces on main's moq-vaapi. H.265 encode and both decoders +pass there. [moq-dev/vaapi#10](https://github.com/moq-dev/vaapi/pull/10) +also noted a chroma offset in radeonsi's VPP NV12 output and a bitrate +overshoot. + +Decided 2026-11-06 (maintainer, /quest-unblock): ✅ keep #8013 queued, which +puts `vaapi` on by default in moq-cli and moq-transcode, and fix this in m1. +Until it lands, an AMD-only host encodes H.264 through VA-API and publishes +a stream nobody can decode; moq-c and moq-ffi already did. Not chosen: +pulling #8013 to refuse radeonsi first, or keeping `vaapi` opt-in. Also +decided: ✅ find the root cause in m1. Not chosen: m3 beside +[Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md), or only a note +there. + +- Find whether the H.264 bytes are ours. NAL type 0 at the start points at + how moq-vaapi reads the coded buffer (a segment's `bit_offset`, or more + than one segment) or at packed headers radeonsi does not take for H.264. + [moq-dev/vaapi#10](https://github.com/moq-dev/vaapi/pull/10) had to + require packed H.265 parameter sets for the same reason. Compare against + `ffmpeg -c:v h264_vaapi` on the same node. +- If it is moq-vaapi's, fix it there with a test, release the crate, and bump + `moq-vaapi` here. If it is the driver's, refuse a radeonsi node for H.264 + encode, using the driver's vendor string, so the candidate fails to open and + OpenH264 takes over. Then link the Mesa issue. +- Treat the chroma offset and the bitrate overshoot the same way: fix them if + they are ours, and if they are the driver's, record them with their Mesa + issue. +- Vulkan Video still replaces VA-API on AMD later; this quest does not wait + on it. + +Public API: none. Wire: none. + +## Related + +- [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - AMD's long-term encode path +- [VA-API renderer tests](/quest/m4/vaapi-render-tests.md) - runs the VA-API tests on each render node +- [moq-vaapi vendor sync](/quest/m3/vendor-sync/vaapi.md) - the crate's upstream record diff --git a/quest/m3/README.md b/quest/m3/README.md index 6f087585f18..146e8c8fe2d 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -61,6 +61,7 @@ federation) land. - [Plan: watch worker](/quest/m3/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides - [Watch worker](/quest/m3/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio - [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070 +- [H.265 rungs respect the encoder's minimum size](/quest/m3/h265-min-width.md) - a rung below AMD's 384-wide H.265 minimum is widened or dropped, not failed - [VA-API encodes an external Vulkan image](/quest/m3/vaapi-vulkan-import.md) - an explicit-sync DMA-BUF reaches Intel's encoder, proven on Arrow Lake - [C++ FFI reuse](/quest/m3/cpp-ffi-reuse.md) - C++ packages link the shared default-feature FFI build once it is proven compatible - [OBS native codecs](/quest/m3/obs-moq-video/README.md) - replace FFmpeg video and audio decoding with moq-video and moq-audio, deliver GPU frames, and use native audio/video encoders diff --git a/quest/m3/h265-min-width.md b/quest/m3/h265-min-width.md new file mode 100644 index 00000000000..dec204fc1bd --- /dev/null +++ b/quest/m3/h265-min-width.md @@ -0,0 +1,35 @@ +# [S] H.265 rungs respect the encoder's minimum size + +## Goal + +A moq-transcode ladder whose H.265 rung is narrower than the hardware +encoder's minimum never fails that rendition. The encoder probe reports the +driver's minimum width and height, and the ladder widens or drops the rung +below it instead of failing to open an encoder. + +## Plan + +Found 2026-11-06 in [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013). +AMD's VCN 5 (radeonsi, Mesa `si_get.c`) does not encode H.265 narrower than +384. moq-video has no software H.265 encoder, so on an AMD-only host a +rendition below that fails to open. The default ladder's 240p rung is 320 +wide for a 4:3 source. + +Decided 2026-11-06 (maintainer, /quest-unblock): ✅ an S quest in m3. Not +chosen: folding it into [the radeonsi quest](/quest/m1/vaapi-radeonsi.md), +or dropping it. + +- Read the minimum from the driver rather than hard-coding 384, such as + through VA-API's `VAConfigAttribMaxPictureWidth` and minimum picture + attributes, or the encoder's open error. Check what NVENC and VideoToolbox + report for the same case. +- Choose between widening the rung to the minimum and dropping it, and + record why. A dropped rung must stay out of the catalog. + +Public API: maybe a minimum size on the encoder probe in moq-video. Wire: +none. + +## Related + +- [VA-API encode is correct on radeonsi](/quest/m1/vaapi-radeonsi.md) - the same AMD hardware run +- [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - may report a different minimum diff --git a/quest/m4/vaapi-render-tests.md b/quest/m4/vaapi-render-tests.md index 428637ada0f..7544a22ddbd 100644 --- a/quest/m4/vaapi-render-tests.md +++ b/quest/m4/vaapi-render-tests.md @@ -41,6 +41,16 @@ which covers only NVIDIA. recipe's filter selects them. - No hosted runner has these GPUs, so this stays a local recipe until a GPU runner exists ([GPU CI](/quest/m3/gpu-ci.md) owns that decision). +- Added 2026-11-06 (maintainer, /quest-unblock) from + [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013)'s follow-ups: + - The VA-API tests open the first suitable render node, so a second GPU + is covered only when `MOQ_VAAPI_DEVICE` names it. `just rs gpu` runs the + VA-API tests once for each render node, with `MOQ_VAAPI_DEVICE` set. + - The H.265 decode tests gain an open-GOP stream with RASL pictures, and + a Main 10 stream holding 8-bit samples that decodes. They also gain a + 10-bit stream, which is refused, since moq-vaapi has no P010 output + ([moq-dev/vaapi#11](https://github.com/moq-dev/vaapi/pull/11)). + - Not chosen: dropping either gap. ## Related From 5667134ac6c33918132ae79e3a5450421ed1e41b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 17:59:47 -0800 Subject: [PATCH 2/3] quest: address #8098 review threads (H.265 caller, radeonsi fallback, per-node VA-API runs) Co-Authored-By: Claude Opus 5.5 --- quest/m1/vaapi-radeonsi.md | 3 ++- quest/m3/h265-min-width.md | 33 ++++++++++++++++++++------------- quest/m4/vaapi-render-tests.md | 7 ++++++- 3 files changed, 28 insertions(+), 15 deletions(-) diff --git a/quest/m1/vaapi-radeonsi.md b/quest/m1/vaapi-radeonsi.md index 7e440d8a19d..b324c37b231 100644 --- a/quest/m1/vaapi-radeonsi.md +++ b/quest/m1/vaapi-radeonsi.md @@ -4,7 +4,8 @@ On an AMD RX 9070 (radeonsi), moq-video's VA-API H.264 encoder either produces a stream that ffmpeg decodes, or refuses the node, so `Kind::Auto` -falls to OpenH264. Its VPP NV12 conversion carries no chroma offset, and its +falls to the next H.264 encoder (OpenH264 in a build that has it; a build +without one gets `NoEncoder` naming the refused driver). Its VPP NV12 conversion carries no chroma offset, and its rate control stays near the target bitrate. The ignored VA-API encode tests in `rs/moq-video/src/encode/backend/vaapi.rs` pass on that card with `MOQ_VAAPI_DEVICE` naming its node, or skip with a reason that names the diff --git a/quest/m3/h265-min-width.md b/quest/m3/h265-min-width.md index dec204fc1bd..961f7bed15c 100644 --- a/quest/m3/h265-min-width.md +++ b/quest/m3/h265-min-width.md @@ -2,29 +2,36 @@ ## Goal -A moq-transcode ladder whose H.265 rung is narrower than the hardware -encoder's minimum never fails that rendition. The encoder probe reports the -driver's minimum width and height, and the ladder widens or drops the rung -below it instead of failing to open an encoder. +An H.265 encode narrower than the hardware encoder's minimum never fails +with an opaque driver error. moq-video's encoder probe reports the driver's +minimum width and height, `Kind::Auto` passes over a hardware candidate +whose minimum the frame is below, and with no candidate left the error names +the minimum. The caller today is `moq-cli publish --video-codec h265` +(`rs/moq-cli/src/publish.rs`). ## Plan Found 2026-11-06 in [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013). AMD's VCN 5 (radeonsi, Mesa `si_get.c`) does not encode H.265 narrower than -384. moq-video has no software H.265 encoder, so on an AMD-only host a -rendition below that fails to open. The default ladder's 240p rung is 320 -wide for a 4:3 source. +384. moq-video has no software H.265 encoder, so on an AMD-only host an +H.265 encode below that fails to open. + +moq-transcode cannot hit this yet: its `Config` has no output codec, and +`catalog::rung_entry` (`rs/moq-transcode/src/catalog.rs`) and `Rung::encode` +(`rs/moq-transcode/src/rung.rs`) leave `encode::Config` at its H.264 +default. Adding an H.265 output to the ladder is out of this quest's scope. +Whoever adds it applies the probe's minimum per rung (the default 240p rung +is 320 wide for a 4:3 source), and a failed rung probe must fail only that +rung: today `Pipeline::new` propagates it and aborts the whole transcode. Decided 2026-11-06 (maintainer, /quest-unblock): ✅ an S quest in m3. Not chosen: folding it into [the radeonsi quest](/quest/m1/vaapi-radeonsi.md), or dropping it. -- Read the minimum from the driver rather than hard-coding 384, such as - through VA-API's `VAConfigAttribMaxPictureWidth` and minimum picture - attributes, or the encoder's open error. Check what NVENC and VideoToolbox - report for the same case. -- Choose between widening the rung to the minimum and dropping it, and - record why. A dropped rung must stay out of the catalog. +- Learn the minimum rather than hard-coding 384. libva has no minimum + picture-size attribute (only `VAConfigAttribMaxPictureWidth`/`Height`), so + it comes from the failed open or a per-driver table. Check what NVENC and + VideoToolbox report for the same case. Public API: maybe a minimum size on the encoder probe in moq-video. Wire: none. diff --git a/quest/m4/vaapi-render-tests.md b/quest/m4/vaapi-render-tests.md index 7544a22ddbd..407e8a29cff 100644 --- a/quest/m4/vaapi-render-tests.md +++ b/quest/m4/vaapi-render-tests.md @@ -45,7 +45,12 @@ which covers only NVIDIA. [moq-dev/moq#8013](https://github.com/moq-dev/moq/pull/8013)'s follow-ups: - The VA-API tests open the first suitable render node, so a second GPU is covered only when `MOQ_VAAPI_DEVICE` names it. `just rs gpu` runs the - VA-API tests once for each render node, with `MOQ_VAAPI_DEVICE` set. + VA-API tests once for each AMD or Intel render node, with + `MOQ_VAAPI_DEVICE` set; a node of either vendor without its VA-API + driver fails, and NVIDIA nodes are left to GPU CI. The renderer + fixtures allocate their NV12 and I420 buffers on that node and pick the + Vulkan adapter whose DRM render node matches it + (`VK_EXT_physical_device_drm`), so each run stays on one GPU. - The H.265 decode tests gain an open-GOP stream with RASL pictures, and a Main 10 stream holding 8-bit samples that decodes. They also gain a 10-bit stream, which is refused, since moq-vaapi has no P010 output From ab8b569ab0ddc11952df11041a3ca4d672ad40bc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 18:19:36 -0800 Subject: [PATCH 3/3] quest: fix pullfrog's three threads on #8098 Co-Authored-By: Claude Opus 5.5 --- quest/m1/vaapi-radeonsi.md | 8 +++++--- quest/m3/README.md | 2 +- quest/m3/h265-min-width.md | 11 ++++++++--- 3 files changed, 14 insertions(+), 7 deletions(-) diff --git a/quest/m1/vaapi-radeonsi.md b/quest/m1/vaapi-radeonsi.md index b324c37b231..3762d34aaaa 100644 --- a/quest/m1/vaapi-radeonsi.md +++ b/quest/m1/vaapi-radeonsi.md @@ -41,9 +41,11 @@ there. `moq-vaapi` here. If it is the driver's, refuse a radeonsi node for H.264 encode, using the driver's vendor string, so the candidate fails to open and OpenH264 takes over. Then link the Mesa issue. -- Treat the chroma offset and the bitrate overshoot the same way: fix them if - they are ours, and if they are the driver's, record them with their Mesa - issue. +- Fix the chroma offset and the bitrate overshoot if they are ours. If the + chroma offset is the driver's VPP, skip VPP on radeonsi and convert to NV12 + before upload. If the overshoot is the driver's, refuse a radeonsi node for + that encode, as for the H.264 bytes, so `Kind::Auto` moves on. Link the + Mesa issue either way. - Vulkan Video still replaces VA-API on AMD later; this quest does not wait on it. diff --git a/quest/m3/README.md b/quest/m3/README.md index 146e8c8fe2d..adc4fa6039c 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -61,7 +61,7 @@ federation) land. - [Plan: watch worker](/quest/m3/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides - [Watch worker](/quest/m3/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio - [Vulkan Video encode on AMD](/quest/m3/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070 -- [H.265 rungs respect the encoder's minimum size](/quest/m3/h265-min-width.md) - a rung below AMD's 384-wide H.265 minimum is widened or dropped, not failed +- [H.265 encodes respect the encoder's minimum size](/quest/m3/h265-min-width.md) - an H.265 encode below AMD's 384-wide minimum is skipped by `Kind::Auto`, and the error names it - [VA-API encodes an external Vulkan image](/quest/m3/vaapi-vulkan-import.md) - an explicit-sync DMA-BUF reaches Intel's encoder, proven on Arrow Lake - [C++ FFI reuse](/quest/m3/cpp-ffi-reuse.md) - C++ packages link the shared default-feature FFI build once it is proven compatible - [OBS native codecs](/quest/m3/obs-moq-video/README.md) - replace FFmpeg video and audio decoding with moq-video and moq-audio, deliver GPU frames, and use native audio/video encoders diff --git a/quest/m3/h265-min-width.md b/quest/m3/h265-min-width.md index 961f7bed15c..20f547e334f 100644 --- a/quest/m3/h265-min-width.md +++ b/quest/m3/h265-min-width.md @@ -1,4 +1,4 @@ -# [S] H.265 rungs respect the encoder's minimum size +# [S] H.265 encodes respect the encoder's minimum size ## Goal @@ -6,8 +6,8 @@ An H.265 encode narrower than the hardware encoder's minimum never fails with an opaque driver error. moq-video's encoder probe reports the driver's minimum width and height, `Kind::Auto` passes over a hardware candidate whose minimum the frame is below, and with no candidate left the error names -the minimum. The caller today is `moq-cli publish --video-codec h265` -(`rs/moq-cli/src/publish.rs`). +the minimum. The caller today is `moq import capture --codec h265` +(`CaptureArgs::codec`, `rs/moq-cli/src/publish.rs`). ## Plan @@ -28,6 +28,11 @@ Decided 2026-11-06 (maintainer, /quest-unblock): ✅ an S quest in m3. Not chosen: folding it into [the radeonsi quest](/quest/m1/vaapi-radeonsi.md), or dropping it. +Decided 2026-11-07 (maintainer, /quest-complete): ✅ narrowed to the +capture encode (`moq import capture --codec h265`), an S in m3, with H.265 +transcode output out of scope. Not chosen: making H.265 transcode output a +prerequisite inside this quest. + - Learn the minimum rather than hard-coding 384. libva has no minimum picture-size attribute (only `VAConfigAttribMaxPictureWidth`/`Height`), so it comes from the failed open or a per-driver table. Check what NVENC and