diff --git a/Cargo.toml b/Cargo.toml index d3c1442ab25..3345cd8b5a4 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -98,10 +98,11 @@ moq-srt = { version = "0.2.3", path = "rs/moq-srt" } moq-stats = { version = "0.1.4", path = "rs/moq-stats" } moq-token = { version = "0.7", path = "rs/moq-token" } moq-token-cli = { version = "0.5", path = "rs/moq-token-cli" } -# default-features off (the hardware codecs) on moq-transcode and moq-video so -# each consumer opts in: binaries (libmoq, moq-boy, moq-cli) enable them, but a -# self-compiler can leave them off to drop the CUDA / libva deps. Both crates -# still default them on when depended on directly. +# default-features off (the NVIDIA hardware codecs) on moq-transcode and +# moq-video so each consumer opts in. Binaries enable them, but a self-compiler +# can leave them off to drop the CUDA dependencies. Both crates still default +# them on when depended on directly. VAAPI is opt-in everywhere, since that +# backend has never been validated on real hardware. moq-transcode = { version = "0.0.7", path = "rs/moq-transcode", default-features = false } # Standalone crate (moq-dev/vaapi); vendored from cros-libva + cros-codecs. # dlopen's libva at runtime (no libva-dev at build, no NEEDED libva in the binary). diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 5b5e30d205b..939d2ad8ea3 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -240,10 +240,12 @@ moq --client-connect https://relay.example.com/anon --broadcast screen.hang \ import capture --display --system-audio ``` -On Linux the NVENC (NVIDIA) and VAAPI (Intel/AMD) encoders and the PipeWire -screen capture are compiled in by default and link the CUDA / libva / libpipewire -system libraries. To build `capture` without them (software openh264 + V4L2 -camera capture only, no system library dependency), drop the default features: +On Linux the NVENC (NVIDIA) encoder and the PipeWire screen capture are compiled +in by default. VAAPI (Intel/AMD) is behind the off-by-default `vaapi` feature, +since that backend has never been validated on real hardware. To build `capture` +without any of them (software openh264 + V4L2 camera capture only), drop the +default features. `capture` itself still needs libclang and the V4L2 headers for +the camera, and ALSA for the microphone: ```bash cargo build --release -p moq-cli --no-default-features \ @@ -260,15 +262,15 @@ On macOS these capture at the logical resolution, i.e. what the screen looks lik to its owner rather than its native pixels: a 2x retina display shares as 1710x1106, not 3420x2214, which keeps the derived bitrate sane. Pass `--width`/`--height` to capture native pixels instead. -The Linux screen backend links libpipewire and -is behind the default-on `pipewire` feature; drop it (like the codecs above) for -a build without the dependency. The codec is chosen with `--codec` -(`h264` default, or `h265`). For H.264 it picks a hardware encoder -(VideoToolbox on macOS, NVENC on Linux NVIDIA, VAAPI on Linux Intel/AMD) when one -is present, falling back to the built-in software encoder (openh264); force either +The Linux screen backend links libpipewire and is behind the default-on +`pipewire` feature; drop it (like the codecs above) for a build without the +dependency. The codec is chosen with `--codec` (`h264` default, or `h265`). For +H.264 it picks a hardware encoder (VideoToolbox on macOS, NVENC on Linux NVIDIA, +or VAAPI on Linux Intel/AMD when built with the `vaapi` feature) when one is +present, falling back to the built-in software encoder (openh264); force either with `--hardware` / `--software`. H.265 is hardware-only (VideoToolbox on macOS, -Media Foundation on Windows). `--camera` takes a bare integer as a device index, otherwise a -device path (Linux) or name (a friendly-name substring on Windows, the +Media Foundation on Windows). `--camera` takes a bare integer as a device index, +otherwise a device path (Linux) or name (a friendly-name substring on Windows, the AVFoundation `uniqueID` on macOS). Microphone capture uses cpal (CoreAudio / WASAPI / ALSA) and encodes Opus. `--system-audio` is macOS-only: there is no loopback input device, so it goes through ScreenCaptureKit (the same API as diff --git a/doc/lib/rs/crate/moq-video.md b/doc/lib/rs/crate/moq-video.md index 19a924890e0..7a10316bd2e 100644 --- a/doc/lib/rs/crate/moq-video.md +++ b/doc/lib/rs/crate/moq-video.md @@ -52,9 +52,10 @@ slow path. AV1 is decode-only, via NVDEC. cargo add moq-video ``` -Hardware codecs are on by default. Rendering and PipeWire screen capture are not, -since they pull in a graphics stack and a `libpipewire` build dependency -respectively: +NVIDIA hardware codecs are on by default. VAAPI is not, since that backend has +never been validated on real hardware, and neither are rendering and PipeWire +screen capture, which pull in a graphics stack and a `libpipewire` build +dependency respectively: ```bash cargo add moq-video --features render,pipewire @@ -63,11 +64,11 @@ cargo add moq-video --features render,pipewire | Feature | Default | Pulls in | | --- | --- | --- | | `nvenc` / `nvdec` | yes | NVIDIA encode/decode on Linux (`cudarc`, `moq-nvenc`) | -| `vaapi` | yes | Intel/AMD encode on Linux (`moq-vaapi`) | +| `vaapi` | no | Intel/AMD encode on Linux (`moq-vaapi`), unvalidated on hardware | | `render` | no | `wgpu` and the GPU renderer | | `pipewire` | no | Wayland/X11 screen capture via xdg-desktop-portal | -The three default features are Linux-only in effect (their dependencies are), and +The two default features are Linux-only in effect (their dependencies are), and `--no-default-features` gives a slim build that still captures V4L2 and encodes with openh264. A relay never needs any of them. diff --git a/flake.nix b/flake.nix index c1604a0747f..609a1b5885e 100644 --- a/flake.nix +++ b/flake.nix @@ -128,11 +128,6 @@ # PulseAudio rather than the `pipewire` PCM below. Loaded at runtime # only, via `alsaPlugins`. pkgs.alsa-plugins - # moq-video's VAAPI backend (always-on for Linux): moq-vaapi dlopen's - # libva at runtime, so it isn't needed to build. This is here only to - # actually run vaapi in the devShell; a libva-less host loads and falls - # back cleanly. macOS has no VAAPI. - pkgs.libva # moq-video's `pipewire` screen-capture feature: the pipewire crate # links libpipewire-0.3 via pkg-config and generates bindings at build # time (bindgenHook above provides libclang). Linux-only; macOS uses diff --git a/rs/CLAUDE.md b/rs/CLAUDE.md index 001620c42bd..865c5a045cc 100644 --- a/rs/CLAUDE.md +++ b/rs/CLAUDE.md @@ -170,7 +170,7 @@ Then `Config::load()?` (initializes tracing), build clients/servers via `.init() - Windows (moq-video's Media Foundation and D3D11 backends): `just rs windows`, which must run ON Windows. You can't reproduce it elsewhere, since cross-compiling dies in openh264-sys2's vendored C++. - macOS (moq-video's VideoToolbox and ScreenCaptureKit, moq-audio's system audio): `just rs macos`, which must run ON macOS. Scoped to moq-video + moq-audio, and needs `--all-features` because moq-audio's capture backend is off by default. - - Linux: covered. `just rs ci` already runs `--all-features` in a dev shell carrying pipewire/libva/alsa, so nvenc/nvdec/vaapi/pipewire all compile. + - Linux: covered. `just rs ci` already runs `--all-features` in a dev shell carrying PipeWire and ALSA. VAAPI loads libva dynamically, so nvenc/nvdec/vaapi/pipewire all compile without libva installed. What still compiles these automatically, and when: diff --git a/rs/justfile b/rs/justfile index 180fcca7e44..a8d160d5654 100644 --- a/rs/justfile +++ b/rs/justfile @@ -127,8 +127,8 @@ fix-changed $FILES: # someone runs this by hand on a Windows host. # # Default features rather than `--all-features`: jemalloc and the quiche -# backend don't build on MSVC, while moq-video's Linux-only nvenc/vaapi/nvdec -# defaults are already no-ops off Linux. moq-gst is excluded because it links +# backend don't build on MSVC, while moq-video's Linux-only nvenc/nvdec defaults +# are already no-ops off Linux. moq-gst is excluded because it links # GStreamer via pkg-config, which the Windows runner doesn't have. # Compile the whole workspace on a Windows host. Must run ON Windows. diff --git a/rs/libmoq/Cargo.toml b/rs/libmoq/Cargo.toml index 3e65b411a9b..a3d7179b059 100644 --- a/rs/libmoq/Cargo.toml +++ b/rs/libmoq/Cargo.toml @@ -25,8 +25,8 @@ moq-json = { workspace = true } moq-mux = { workspace = true } moq-native = { workspace = true, default-features = true } moq-net = { workspace = true } -# Enable the NVENC / VAAPI / NVDEC hardware codecs (default-off at the workspace level). -moq-video = { workspace = true, features = ["nvenc", "vaapi", "nvdec"] } +# Enable the NVIDIA hardware codecs (default-off at the workspace level). +moq-video = { workspace = true, features = ["nvenc", "nvdec"] } # Blocking on `encode::Sink` from the synchronous C entry points; see # `video::block_on` for why not a tokio helper. pollster = "1.0" diff --git a/rs/libmoq/native-libs/linux.txt b/rs/libmoq/native-libs/linux.txt index 7092a410229..53a76ba55a9 100644 --- a/rs/libmoq/native-libs/linux.txt +++ b/rs/libmoq/native-libs/linux.txt @@ -8,7 +8,3 @@ pthread # openh264 (the software H.264 fallback) is C++ stdc++ - -# moq-video: the vaapi feature links libva for hardware encode and decode -va -va-drm diff --git a/rs/libmoq/src/video.rs b/rs/libmoq/src/video.rs index 8610c97fd6a..e748e5bdac0 100644 --- a/rs/libmoq/src/video.rs +++ b/rs/libmoq/src/video.rs @@ -3,7 +3,7 @@ //! The video counterpart to [`audio`](crate::audio): publish raw pictures as an //! encoded video track, and subscribe to one and hand back decoded raw frames, //! with the codec running inside the FFI boundary (VideoToolbox on macOS, Media -//! Foundation on Windows, NVENC/VAAPI/NVDEC on Linux, openh264 as the software +//! Foundation on Windows, NVENC/NVDEC on Linux, openh264 as the software //! fallback; no ffmpeg). Siblings to `moq_publish_media_*` / //! `moq_consume_video`, which carry already-encoded frames for a caller that //! brings its own codec. @@ -100,9 +100,8 @@ pub struct moq_video_encoder_output { pub gop: u32, /// `moq_video_encoder_kind` discriminant. pub kind: u32, - /// Backend name, UTF-8, e.g. `"videotoolbox"`, `"nvenc"`, `"vaapi"`, - /// `"mediafoundation"`, `"openh264"`. Read only when `kind` is - /// `MOQ_VIDEO_ENCODER_KIND_NAMED`. + /// Backend name, UTF-8, e.g. `"videotoolbox"`, `"nvenc"`, `"mediafoundation"`, + /// `"openh264"`. Read only when `kind` is `MOQ_VIDEO_ENCODER_KIND_NAMED`. pub encoder: *const c_char, pub encoder_len: usize, } diff --git a/rs/moq-boy/Cargo.toml b/rs/moq-boy/Cargo.toml index 2cf1e3f0dd4..92b124b2f53 100644 --- a/rs/moq-boy/Cargo.toml +++ b/rs/moq-boy/Cargo.toml @@ -29,8 +29,8 @@ moq-json = { workspace = true } moq-mux = { workspace = true } moq-native = { workspace = true, default-features = false, features = ["aws-lc-rs"] } moq-net = { workspace = true } -# Enable the NVENC / VAAPI hardware encoders (default-off at the workspace level). -moq-video = { workspace = true, features = ["nvenc", "vaapi"] } +# Enable the NVENC hardware encoder (default-off at the workspace level). +moq-video = { workspace = true, features = ["nvenc"] } serde = { workspace = true } tokio = { workspace = true, features = ["full"] } tracing = "0.1" diff --git a/rs/moq-cli/Cargo.toml b/rs/moq-cli/Cargo.toml index e86124fdbcd..889b1edafe7 100644 --- a/rs/moq-cli/Cargo.toml +++ b/rs/moq-cli/Cargo.toml @@ -18,7 +18,7 @@ name = "moq" path = "src/main.rs" [features] -default = ["iroh", "quinn", "websocket", "nvenc", "vaapi", "nvdec", "pipewire"] +default = ["iroh", "quinn", "websocket", "nvenc", "nvdec", "pipewire"] iroh = ["moq-native/iroh"] jemalloc = ["moq-native/jemalloc"] noq = ["moq-native/noq"] @@ -36,15 +36,17 @@ websocket = ["moq-native/websocket"] # extra install. H.264 is vendored static (openh264), so no system ffmpeg/libav. # Enable with `cargo build -p moq-cli --features capture`. capture = ["dep:moq-video", "dep:moq-audio", "moq-audio/capture"] -# Just-in-time transcoding (`moq ... transcode`). Off by default for the same -# reason as `capture`: it pulls in moq-video, whose default-on `vaapi` feature -# links libva on Linux. Enable with `cargo build -p moq-cli --features transcode`. +# Just-in-time transcoding (`moq ... transcode`). Off by default because it pulls +# in moq-video and its codec dependencies. Enable with +# `cargo build -p moq-cli --features transcode`. transcode = ["dep:moq-transcode", "dep:moq-video"] -# NVENC / VAAPI / NVDEC hardware codecs for `capture` (Linux), on by default. Each -# just turns on the matching moq-video feature, and only bites when `capture` pulls -# in moq-video. For a `capture` build with no CUDA / libva system deps, drop them, e.g. +# NVIDIA hardware codecs for `capture` (Linux), on by default. Each just turns on +# the matching moq-video feature, and only bites when `capture` pulls in moq-video. +# For a `capture` build with no CUDA dependencies, drop them, e.g. # cargo build -p moq-cli --no-default-features --features "iroh quinn websocket capture" nvenc = ["moq-video?/nvenc", "moq-transcode?/nvenc"] +# Intel/AMD VAAPI hardware encoding (Linux). Off by default because the backend is +# unvalidated on hardware; enable it explicitly alongside `capture` or `transcode`. vaapi = ["moq-video?/vaapi", "moq-transcode?/vaapi"] nvdec = ["moq-video?/nvdec", "moq-transcode?/nvdec"] # Screen capture for `capture --display` on Linux (xdg-desktop-portal + PipeWire), diff --git a/rs/moq-ffi/Cargo.toml b/rs/moq-ffi/Cargo.toml index acc0485847b..a57dd907efb 100644 --- a/rs/moq-ffi/Cargo.toml +++ b/rs/moq-ffi/Cargo.toml @@ -31,7 +31,7 @@ moq-mux = { workspace = true } moq-native = { workspace = true, default-features = true } moq-net = { workspace = true } # Default features only: the Linux hardware codecs stay off so the wrappers' -# cross-compiled targets (Android, iOS) don't pull a CUDA / libva graph. +# cross-compiled targets (Android, iOS) don't pull a CUDA graph. moq-video = { workspace = true } # Blocking on `encode::Sink` from the synchronous producer exports; see # `video::block_on` for why not a tokio helper. diff --git a/rs/moq-ffi/src/video.rs b/rs/moq-ffi/src/video.rs index 7247e233b25..d2008fa9ad2 100644 --- a/rs/moq-ffi/src/video.rs +++ b/rs/moq-ffi/src/video.rs @@ -49,8 +49,8 @@ impl From for moq_video::encode::Codec { /// Which encoder implementation to use. /// /// These bindings compile VideoToolbox (macOS), Media Foundation (Windows), and -/// openh264 (software, everywhere). The Linux hardware codecs (NVENC, VAAPI) -/// are a libmoq-only build option, so Linux here is software-only. +/// openh264 (software, everywhere). NVENC/NVDEC are a libmoq-only build option +/// and VAAPI is opt-in everywhere, so Linux here is software-only. #[derive(Clone, uniffi::Enum)] pub enum MoqVideoEncoderKind { /// Prefer a platform hardware encoder, falling back to software. On Linux diff --git a/rs/moq-transcode/Cargo.toml b/rs/moq-transcode/Cargo.toml index be3dc9905ec..795a6a6730d 100644 --- a/rs/moq-transcode/Cargo.toml +++ b/rs/moq-transcode/Cargo.toml @@ -13,12 +13,14 @@ keywords = ["quic", "http3", "webtransport", "media", "video"] categories = ["multimedia", "multimedia::video", "multimedia::encoding"] [features] -# Hardware codecs on by default, mirroring moq-video: NVDEC -> NVENC is the whole -# point of a GPU transcode fleet. Disable for a slimmer self-compiled Linux build. -default = ["nvenc", "vaapi", "nvdec"] +# NVIDIA hardware codecs are on by default, mirroring moq-video: NVDEC -> NVENC +# is the whole point of a GPU transcode fleet. Disable for a slimmer +# self-compiled Linux build. VAAPI remains available as an opt-in feature. +default = ["nvenc", "nvdec"] # NVIDIA NVENC hardware encoder (Linux, dlopen'd at runtime; a no-op elsewhere). nvenc = ["moq-video/nvenc"] -# Intel/AMD VAAPI hardware encoder (Linux; libva dlopen'd at runtime; a no-op elsewhere). +# Intel/AMD VAAPI hardware encoder (Linux; opt-in because the backend is +# unvalidated on hardware; libva dlopen'd at runtime). vaapi = ["moq-video/vaapi"] # NVIDIA NVDEC hardware decoder (Linux, dlopen'd at runtime; a no-op elsewhere). # With `nvenc` this is the zero-copy GPU pipeline: NVDEC decodes and scales in diff --git a/rs/moq-video/Cargo.toml b/rs/moq-video/Cargo.toml index 1ea35b75fec..be8ce26d738 100644 --- a/rs/moq-video/Cargo.toml +++ b/rs/moq-video/Cargo.toml @@ -13,11 +13,11 @@ keywords = ["quic", "http3", "webtransport", "media", "video"] categories = ["multimedia", "multimedia::video", "multimedia::encoding"] [features] -# Hardware codecs are enabled by default. Disable them (e.g. `--no-default-features`) -# for a slimmer self-compiled Linux build that drops the CUDA / libva deps: openh264 -# stays as the always-compiled software fallback and V4L2 capture is unaffected. All -# three features are no-ops off Linux (their deps are Linux-only). -default = ["nvenc", "vaapi", "nvdec"] +# NVIDIA hardware codecs are enabled by default. Disable them (e.g. +# `--no-default-features`) for a slimmer self-compiled Linux build that drops the +# CUDA dependencies. Openh264 stays as the always-compiled software fallback and +# V4L2 capture is unaffected. The default features are no-ops off Linux. +default = ["nvenc", "nvdec"] # NVIDIA NVENC hardware encoder (Linux). dlopen'd at runtime (cudarc + moq-nvenc, # which loads libnvidia-encode), with a libloading probe for the driver libs, so # an nvenc build still links GPU-less and starts driverless, falling back to the @@ -28,10 +28,11 @@ nvenc = ["dep:cudarc", "dep:moq-nvenc", "dep:libloading"] # links GPU-less and starts driverless, falling back to the next decoder. Decoded # frames stay in CUDA memory and feed NVENC zero-copy (the transcode path). nvdec = ["dep:cudarc", "dep:moq-nvenc", "dep:libloading"] -# Intel/AMD VAAPI hardware encoder (Linux). moq-vaapi dlopen's libva at runtime, so -# an enabled-vaapi build needs no libva at build time and still starts on a -# libva-less host; automatic backend selection then falls back to the next encoder, -# like the NVENC backend. +# Intel/AMD VAAPI hardware encoder (Linux). Off by default because the backend has +# never been validated on real hardware, not because of anything it costs the +# build: moq-vaapi dlopen's libva, so an enabled-vaapi build needs no libva at +# build time and still starts on a libva-less host, where automatic backend +# selection falls back to the next encoder like the NVENC backend does. vaapi = ["dep:moq-vaapi"] # GPU rendering of decoded frames (the `render` module): a wgpu pipeline that # converts a frame to RGBA and hands back a texture the caller presents. Off by @@ -106,7 +107,7 @@ cudarc = { version = "0.19", optional = true, default-features = false, features # absent. Lets a GPU-less host fall back to the next encoder instead of crashing. libloading = { version = "0.9", optional = true } moq-nvenc = { workspace = true, optional = true } -# Intel/AMD VAAPI hardware encoder, behind the default-on `vaapi` feature. As of +# Intel/AMD VAAPI hardware encoder, behind the opt-in `vaapi` feature. As of # moq-vaapi 0.0.3 libva is dlopen'd at runtime, so the binary carries no NEEDED # libva: the build needs no libva-dev, and a libva-less host still loads. Automatic # backend selection then falls back to the next encoder instead of failing at load, diff --git a/rs/moq-video/DESIGN-native-codecs.md b/rs/moq-video/DESIGN-native-codecs.md index b1ba7c28598..bb46dc4e194 100644 --- a/rs/moq-video/DESIGN-native-codecs.md +++ b/rs/moq-video/DESIGN-native-codecs.md @@ -404,15 +404,11 @@ actively-maintained fork (Discord ships it for Go Live) that bumps to cros-libva **dlopen, like NVENC.** `dlopen` makes cros-libva load libva at runtime (no `cargo:rustc-link-lib`, no `DT_NEEDED libva.so.2`), so a `--features vaapi` binary links on a libva-less builder and loads on a libva-less machine, falling back to -software (see `backend::open`). The build still needs libva *headers* for -cros-libva's bindgen, so `libva` is in the nix devShell. - -> **Status (moq-vaapi 0.0.2): not yet realized.** The published `moq-vaapi` crate -> we depend on today *links* libva via `pkg_config` (its `build.rs` probes `libva` -> and `libva-drm`), so the binary carries `NEEDED libva.so.2` / `libva-drm.so.2` -> and needs libva at both build and run time; a libva-less host fails to load -> rather than falling back. Restoring the `dlopen` path in `moq-vaapi` (so this -> section holds) is tracked in #1837. +software (see `backend::open`). + +> **Status: realized in moq-vaapi 0.0.3** (#1837, bumped in #2465). It dlopens +> libva and vendors the libva headers its bindgen reads, so VAAPI costs the build +> nothing: no libva-dev, no `NEEDED libva.so.2`, no entry in the nix devShell. **Input is an NV12 surface upload, not zero-copy dmabuf.** The encoder wants an NV12 VA surface, but UVC webcams deliver YUYV/MJPEG (decoded to CPU I420); they diff --git a/rs/moq-video/src/encode/backend/vaapi.rs b/rs/moq-video/src/encode/backend/vaapi.rs index 33e7ad7a387..dc0c8646666 100644 --- a/rs/moq-video/src/encode/backend/vaapi.rs +++ b/rs/moq-video/src/encode/backend/vaapi.rs @@ -1,4 +1,4 @@ -//! Intel/AMD VAAPI hardware backend via the `moq-vaapi` crate (Linux, always-on). +//! Intel/AMD VAAPI hardware backend via the opt-in `moq-vaapi` crate on Linux. //! //! `moq-vaapi` is a focused VA-API H.264 encoder vendored and trimmed from //! cros-libva + discord/cros-codecs. It takes tightly-packed NV12 and emits an diff --git a/test/smoke/clients/c/subscribe.c b/test/smoke/clients/c/subscribe.c index bd21c060333..381e7b5bb92 100644 --- a/test/smoke/clients/c/subscribe.c +++ b/test/smoke/clients/c/subscribe.c @@ -160,7 +160,7 @@ int main(int argc, char **argv) { if (got) { fprintf(stderr, "received a frame from %s\n", broadcast); // The data path succeeded, which is all this smoke client verifies. - // libmoq statically bundles moq-video (openh264/vaapi/cuda), whose + // libmoq statically bundles moq-video (openh264/cuda), whose // worker threads use priority-protected mutexes; tearing them down at // normal exit can trip a glibc pthread priority assertion and abort. // Skip that atexit teardown with _exit now that we have our result.