Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,11 @@ moq-srt = { version = "0.2.3", path = "rs/moq-srt" }
moq-stats = { version = "0.1.4", path = "rs/moq-stats" }
moq-token = { version = "0.7", path = "rs/moq-token" }
moq-token-cli = { version = "0.5", path = "rs/moq-token-cli" }
# default-features off (the hardware codecs) on moq-transcode and moq-video so
# each consumer opts in: binaries (libmoq, moq-boy, moq-cli) enable them, but a
# self-compiler can leave them off to drop the CUDA / libva deps. Both crates
# still default them on when depended on directly.
# default-features off (the NVIDIA hardware codecs) on moq-transcode and
# moq-video so each consumer opts in. Binaries enable them, but a self-compiler
# can leave them off to drop the CUDA dependencies. Both crates still default
# them on when depended on directly. VAAPI is opt-in everywhere, since that
# backend has never been validated on real hardware.
moq-transcode = { version = "0.0.7", path = "rs/moq-transcode", default-features = false }
# Standalone crate (moq-dev/vaapi); vendored from cros-libva + cros-codecs.
# dlopen's libva at runtime (no libva-dev at build, no NEEDED libva in the binary).
Expand Down
26 changes: 14 additions & 12 deletions doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,12 @@ moq --client-connect https://relay.example.com/anon --broadcast screen.hang \
import capture --display --system-audio
```

On Linux the NVENC (NVIDIA) and VAAPI (Intel/AMD) encoders and the PipeWire
screen capture are compiled in by default and link the CUDA / libva / libpipewire
system libraries. To build `capture` without them (software openh264 + V4L2
camera capture only, no system library dependency), drop the default features:
On Linux the NVENC (NVIDIA) encoder and the PipeWire screen capture are compiled
in by default. VAAPI (Intel/AMD) is behind the off-by-default `vaapi` feature,
since that backend has never been validated on real hardware. To build `capture`
without any of them (software openh264 + V4L2 camera capture only), drop the
default features. `capture` itself still needs libclang and the V4L2 headers for
the camera, and ALSA for the microphone:

```bash
cargo build --release -p moq-cli --no-default-features \
Expand All @@ -260,15 +262,15 @@ On macOS these capture at the logical resolution, i.e. what the screen looks lik
to its owner rather than its native pixels: a 2x retina display shares as
1710x1106, not 3420x2214, which keeps the derived bitrate sane. Pass
`--width`/`--height` to capture native pixels instead.
The Linux screen backend links libpipewire and
is behind the default-on `pipewire` feature; drop it (like the codecs above) for
a build without the dependency. The codec is chosen with `--codec`
(`h264` default, or `h265`). For H.264 it picks a hardware encoder
(VideoToolbox on macOS, NVENC on Linux NVIDIA, VAAPI on Linux Intel/AMD) when one
is present, falling back to the built-in software encoder (openh264); force either
The Linux screen backend links libpipewire and is behind the default-on
`pipewire` feature; drop it (like the codecs above) for a build without the
dependency. The codec is chosen with `--codec` (`h264` default, or `h265`). For
H.264 it picks a hardware encoder (VideoToolbox on macOS, NVENC on Linux NVIDIA,
or VAAPI on Linux Intel/AMD when built with the `vaapi` feature) when one is
present, falling back to the built-in software encoder (openh264); force either
with `--hardware` / `--software`. H.265 is hardware-only (VideoToolbox on macOS,
Media Foundation on Windows). `--camera` takes a bare integer as a device index, otherwise a
device path (Linux) or name (a friendly-name substring on Windows, the
Media Foundation on Windows). `--camera` takes a bare integer as a device index,
otherwise a device path (Linux) or name (a friendly-name substring on Windows, the
AVFoundation `uniqueID` on macOS). Microphone capture uses cpal (CoreAudio /
WASAPI / ALSA) and encodes Opus. `--system-audio` is macOS-only: there is no
loopback input device, so it goes through ScreenCaptureKit (the same API as
Expand Down
11 changes: 6 additions & 5 deletions doc/lib/rs/crate/moq-video.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,10 @@ slow path. AV1 is decode-only, via NVDEC.
cargo add moq-video
```

Hardware codecs are on by default. Rendering and PipeWire screen capture are not,
since they pull in a graphics stack and a `libpipewire` build dependency
respectively:
NVIDIA hardware codecs are on by default. VAAPI is not, since that backend has
never been validated on real hardware, and neither are rendering and PipeWire
screen capture, which pull in a graphics stack and a `libpipewire` build
dependency respectively:

```bash
cargo add moq-video --features render,pipewire
Expand All @@ -63,11 +64,11 @@ cargo add moq-video --features render,pipewire
| Feature | Default | Pulls in |
| --- | --- | --- |
| `nvenc` / `nvdec` | yes | NVIDIA encode/decode on Linux (`cudarc`, `moq-nvenc`) |
| `vaapi` | yes | Intel/AMD encode on Linux (`moq-vaapi`) |
| `vaapi` | no | Intel/AMD encode on Linux (`moq-vaapi`), unvalidated on hardware |
| `render` | no | `wgpu` and the GPU renderer |
| `pipewire` | no | Wayland/X11 screen capture via xdg-desktop-portal |

The three default features are Linux-only in effect (their dependencies are), and
The two default features are Linux-only in effect (their dependencies are), and
`--no-default-features` gives a slim build that still captures V4L2 and encodes
with openh264. A relay never needs any of them.

Expand Down
5 changes: 0 additions & 5 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,6 @@
# PulseAudio rather than the `pipewire` PCM below. Loaded at runtime
# only, via `alsaPlugins`.
pkgs.alsa-plugins
# moq-video's VAAPI backend (always-on for Linux): moq-vaapi dlopen's
# libva at runtime, so it isn't needed to build. This is here only to
# actually run vaapi in the devShell; a libva-less host loads and falls
# back cleanly. macOS has no VAAPI.
pkgs.libva
# moq-video's `pipewire` screen-capture feature: the pipewire crate
# links libpipewire-0.3 via pkg-config and generates bindings at build
# time (bindgenHook above provides libclang). Linux-only; macOS uses
Expand Down
2 changes: 1 addition & 1 deletion rs/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Then `Config::load()?` (initializes tracing), build clients/servers via `.init()

- Windows (moq-video's Media Foundation and D3D11 backends): `just rs windows`, which must run ON Windows. You can't reproduce it elsewhere, since cross-compiling dies in openh264-sys2's vendored C++.
- macOS (moq-video's VideoToolbox and ScreenCaptureKit, moq-audio's system audio): `just rs macos`, which must run ON macOS. Scoped to moq-video + moq-audio, and needs `--all-features` because moq-audio's capture backend is off by default.
- Linux: covered. `just rs ci` already runs `--all-features` in a dev shell carrying pipewire/libva/alsa, so nvenc/nvdec/vaapi/pipewire all compile.
- Linux: covered. `just rs ci` already runs `--all-features` in a dev shell carrying PipeWire and ALSA. VAAPI loads libva dynamically, so nvenc/nvdec/vaapi/pipewire all compile without libva installed.

What still compiles these automatically, and when:

Expand Down
4 changes: 2 additions & 2 deletions rs/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@ fix-changed $FILES:
# someone runs this by hand on a Windows host.
#
# Default features rather than `--all-features`: jemalloc and the quiche
# backend don't build on MSVC, while moq-video's Linux-only nvenc/vaapi/nvdec
# defaults are already no-ops off Linux. moq-gst is excluded because it links
# backend don't build on MSVC, while moq-video's Linux-only nvenc/nvdec defaults
# are already no-ops off Linux. moq-gst is excluded because it links
# GStreamer via pkg-config, which the Windows runner doesn't have.

# Compile the whole workspace on a Windows host. Must run ON Windows.
Expand Down
4 changes: 2 additions & 2 deletions rs/libmoq/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ moq-json = { workspace = true }
moq-mux = { workspace = true }
moq-native = { workspace = true, default-features = true }
moq-net = { workspace = true }
# Enable the NVENC / VAAPI / NVDEC hardware codecs (default-off at the workspace level).
moq-video = { workspace = true, features = ["nvenc", "vaapi", "nvdec"] }
# Enable the NVIDIA hardware codecs (default-off at the workspace level).
moq-video = { workspace = true, features = ["nvenc", "nvdec"] }
# Blocking on `encode::Sink` from the synchronous C entry points; see
# `video::block_on` for why not a tokio helper.
pollster = "1.0"
Expand Down
4 changes: 0 additions & 4 deletions rs/libmoq/native-libs/linux.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,3 @@ pthread

# openh264 (the software H.264 fallback) is C++
stdc++

# moq-video: the vaapi feature links libva for hardware encode and decode
va
va-drm
7 changes: 3 additions & 4 deletions rs/libmoq/src/video.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
//! The video counterpart to [`audio`](crate::audio): publish raw pictures as an
//! encoded video track, and subscribe to one and hand back decoded raw frames,
//! with the codec running inside the FFI boundary (VideoToolbox on macOS, Media
//! Foundation on Windows, NVENC/VAAPI/NVDEC on Linux, openh264 as the software
//! Foundation on Windows, NVENC/NVDEC on Linux, openh264 as the software
//! fallback; no ffmpeg). Siblings to `moq_publish_media_*` /
//! `moq_consume_video`, which carry already-encoded frames for a caller that
//! brings its own codec.
Expand Down Expand Up @@ -100,9 +100,8 @@ pub struct moq_video_encoder_output {
pub gop: u32,
/// `moq_video_encoder_kind` discriminant.
pub kind: u32,
/// Backend name, UTF-8, e.g. `"videotoolbox"`, `"nvenc"`, `"vaapi"`,
/// `"mediafoundation"`, `"openh264"`. Read only when `kind` is
/// `MOQ_VIDEO_ENCODER_KIND_NAMED`.
/// Backend name, UTF-8, e.g. `"videotoolbox"`, `"nvenc"`, `"mediafoundation"`,
/// `"openh264"`. Read only when `kind` is `MOQ_VIDEO_ENCODER_KIND_NAMED`.
pub encoder: *const c_char,
pub encoder_len: usize,
}
Expand Down
4 changes: 2 additions & 2 deletions rs/moq-boy/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ moq-json = { workspace = true }
moq-mux = { workspace = true }
moq-native = { workspace = true, default-features = false, features = ["aws-lc-rs"] }
moq-net = { workspace = true }
# Enable the NVENC / VAAPI hardware encoders (default-off at the workspace level).
moq-video = { workspace = true, features = ["nvenc", "vaapi"] }
# Enable the NVENC hardware encoder (default-off at the workspace level).
moq-video = { workspace = true, features = ["nvenc"] }
serde = { workspace = true }
tokio = { workspace = true, features = ["full"] }
tracing = "0.1"
16 changes: 9 additions & 7 deletions rs/moq-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ name = "moq"
path = "src/main.rs"

[features]
default = ["iroh", "quinn", "websocket", "nvenc", "vaapi", "nvdec", "pipewire"]
default = ["iroh", "quinn", "websocket", "nvenc", "nvdec", "pipewire"]
iroh = ["moq-native/iroh"]
jemalloc = ["moq-native/jemalloc"]
noq = ["moq-native/noq"]
Expand All @@ -36,15 +36,17 @@ websocket = ["moq-native/websocket"]
# extra install. H.264 is vendored static (openh264), so no system ffmpeg/libav.
# Enable with `cargo build -p moq-cli --features capture`.
capture = ["dep:moq-video", "dep:moq-audio", "moq-audio/capture"]
# Just-in-time transcoding (`moq ... transcode`). Off by default for the same
# reason as `capture`: it pulls in moq-video, whose default-on `vaapi` feature
# links libva on Linux. Enable with `cargo build -p moq-cli --features transcode`.
# Just-in-time transcoding (`moq ... transcode`). Off by default because it pulls
# in moq-video and its codec dependencies. Enable with
# `cargo build -p moq-cli --features transcode`.
transcode = ["dep:moq-transcode", "dep:moq-video"]
# NVENC / VAAPI / NVDEC hardware codecs for `capture` (Linux), on by default. Each
# just turns on the matching moq-video feature, and only bites when `capture` pulls
# in moq-video. For a `capture` build with no CUDA / libva system deps, drop them, e.g.
# NVIDIA hardware codecs for `capture` (Linux), on by default. Each just turns on
# the matching moq-video feature, and only bites when `capture` pulls in moq-video.
# For a `capture` build with no CUDA dependencies, drop them, e.g.
# cargo build -p moq-cli --no-default-features --features "iroh quinn websocket capture"
nvenc = ["moq-video?/nvenc", "moq-transcode?/nvenc"]
# Intel/AMD VAAPI hardware encoding (Linux). Off by default because the backend is
# unvalidated on hardware; enable it explicitly alongside `capture` or `transcode`.
vaapi = ["moq-video?/vaapi", "moq-transcode?/vaapi"]
nvdec = ["moq-video?/nvdec", "moq-transcode?/nvdec"]
# Screen capture for `capture --display` on Linux (xdg-desktop-portal + PipeWire),
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ moq-mux = { workspace = true }
moq-native = { workspace = true, default-features = true }
moq-net = { workspace = true }
# Default features only: the Linux hardware codecs stay off so the wrappers'
# cross-compiled targets (Android, iOS) don't pull a CUDA / libva graph.
# cross-compiled targets (Android, iOS) don't pull a CUDA graph.
moq-video = { workspace = true }
# Blocking on `encode::Sink` from the synchronous producer exports; see
# `video::block_on` for why not a tokio helper.
Expand Down
4 changes: 2 additions & 2 deletions rs/moq-ffi/src/video.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ impl From<MoqVideoCodec> for moq_video::encode::Codec {
/// Which encoder implementation to use.
///
/// These bindings compile VideoToolbox (macOS), Media Foundation (Windows), and
/// openh264 (software, everywhere). The Linux hardware codecs (NVENC, VAAPI)
/// are a libmoq-only build option, so Linux here is software-only.
/// openh264 (software, everywhere). NVENC/NVDEC are a libmoq-only build option
/// and VAAPI is opt-in everywhere, so Linux here is software-only.
#[derive(Clone, uniffi::Enum)]
pub enum MoqVideoEncoderKind {
/// Prefer a platform hardware encoder, falling back to software. On Linux
Expand Down
10 changes: 6 additions & 4 deletions rs/moq-transcode/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,14 @@ keywords = ["quic", "http3", "webtransport", "media", "video"]
categories = ["multimedia", "multimedia::video", "multimedia::encoding"]

[features]
# Hardware codecs on by default, mirroring moq-video: NVDEC -> NVENC is the whole
# point of a GPU transcode fleet. Disable for a slimmer self-compiled Linux build.
default = ["nvenc", "vaapi", "nvdec"]
# NVIDIA hardware codecs are on by default, mirroring moq-video: NVDEC -> NVENC
# is the whole point of a GPU transcode fleet. Disable for a slimmer
# self-compiled Linux build. VAAPI remains available as an opt-in feature.
default = ["nvenc", "nvdec"]
# NVIDIA NVENC hardware encoder (Linux, dlopen'd at runtime; a no-op elsewhere).
nvenc = ["moq-video/nvenc"]
# Intel/AMD VAAPI hardware encoder (Linux; libva dlopen'd at runtime; a no-op elsewhere).
# Intel/AMD VAAPI hardware encoder (Linux; opt-in because the backend is
# unvalidated on hardware; libva dlopen'd at runtime).
vaapi = ["moq-video/vaapi"]
# NVIDIA NVDEC hardware decoder (Linux, dlopen'd at runtime; a no-op elsewhere).
# With `nvenc` this is the zero-copy GPU pipeline: NVDEC decodes and scales in
Expand Down
21 changes: 11 additions & 10 deletions rs/moq-video/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ keywords = ["quic", "http3", "webtransport", "media", "video"]
categories = ["multimedia", "multimedia::video", "multimedia::encoding"]

[features]
# Hardware codecs are enabled by default. Disable them (e.g. `--no-default-features`)
# for a slimmer self-compiled Linux build that drops the CUDA / libva deps: openh264
# stays as the always-compiled software fallback and V4L2 capture is unaffected. All
# three features are no-ops off Linux (their deps are Linux-only).
default = ["nvenc", "vaapi", "nvdec"]
# NVIDIA hardware codecs are enabled by default. Disable them (e.g.
# `--no-default-features`) for a slimmer self-compiled Linux build that drops the
# CUDA dependencies. Openh264 stays as the always-compiled software fallback and
# V4L2 capture is unaffected. The default features are no-ops off Linux.
default = ["nvenc", "nvdec"]
# NVIDIA NVENC hardware encoder (Linux). dlopen'd at runtime (cudarc + moq-nvenc,
# which loads libnvidia-encode), with a libloading probe for the driver libs, so
# an nvenc build still links GPU-less and starts driverless, falling back to the
Expand All @@ -28,10 +28,11 @@ nvenc = ["dep:cudarc", "dep:moq-nvenc", "dep:libloading"]
# links GPU-less and starts driverless, falling back to the next decoder. Decoded
# frames stay in CUDA memory and feed NVENC zero-copy (the transcode path).
nvdec = ["dep:cudarc", "dep:moq-nvenc", "dep:libloading"]
# Intel/AMD VAAPI hardware encoder (Linux). moq-vaapi dlopen's libva at runtime, so
# an enabled-vaapi build needs no libva at build time and still starts on a
# libva-less host; automatic backend selection then falls back to the next encoder,
# like the NVENC backend.
# Intel/AMD VAAPI hardware encoder (Linux). Off by default because the backend has
# never been validated on real hardware, not because of anything it costs the
# build: moq-vaapi dlopen's libva, so an enabled-vaapi build needs no libva at
# build time and still starts on a libva-less host, where automatic backend
# selection falls back to the next encoder like the NVENC backend does.
vaapi = ["dep:moq-vaapi"]
# GPU rendering of decoded frames (the `render` module): a wgpu pipeline that
# converts a frame to RGBA and hands back a texture the caller presents. Off by
Expand Down Expand Up @@ -106,7 +107,7 @@ cudarc = { version = "0.19", optional = true, default-features = false, features
# absent. Lets a GPU-less host fall back to the next encoder instead of crashing.
libloading = { version = "0.9", optional = true }
moq-nvenc = { workspace = true, optional = true }
# Intel/AMD VAAPI hardware encoder, behind the default-on `vaapi` feature. As of
# Intel/AMD VAAPI hardware encoder, behind the opt-in `vaapi` feature. As of
# moq-vaapi 0.0.3 libva is dlopen'd at runtime, so the binary carries no NEEDED
# libva: the build needs no libva-dev, and a libva-less host still loads. Automatic
# backend selection then falls back to the next encoder instead of failing at load,
Expand Down
14 changes: 5 additions & 9 deletions rs/moq-video/DESIGN-native-codecs.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,15 +404,11 @@ actively-maintained fork (Discord ships it for Go Live) that bumps to cros-libva
**dlopen, like NVENC.** `dlopen` makes cros-libva load libva at runtime (no
`cargo:rustc-link-lib`, no `DT_NEEDED libva.so.2`), so a `--features vaapi` binary
links on a libva-less builder and loads on a libva-less machine, falling back to
software (see `backend::open`). The build still needs libva *headers* for
cros-libva's bindgen, so `libva` is in the nix devShell.

> **Status (moq-vaapi 0.0.2): not yet realized.** The published `moq-vaapi` crate
> we depend on today *links* libva via `pkg_config` (its `build.rs` probes `libva`
> and `libva-drm`), so the binary carries `NEEDED libva.so.2` / `libva-drm.so.2`
> and needs libva at both build and run time; a libva-less host fails to load
> rather than falling back. Restoring the `dlopen` path in `moq-vaapi` (so this
> section holds) is tracked in #1837.
software (see `backend::open`).

> **Status: realized in moq-vaapi 0.0.3** (#1837, bumped in #2465). It dlopens
> libva and vendors the libva headers its bindgen reads, so VAAPI costs the build
> nothing: no libva-dev, no `NEEDED libva.so.2`, no entry in the nix devShell.

**Input is an NV12 surface upload, not zero-copy dmabuf.** The encoder wants an
NV12 VA surface, but UVC webcams deliver YUYV/MJPEG (decoded to CPU I420); they
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-video/src/encode/backend/vaapi.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! Intel/AMD VAAPI hardware backend via the `moq-vaapi` crate (Linux, always-on).
//! Intel/AMD VAAPI hardware backend via the opt-in `moq-vaapi` crate on Linux.
//!
//! `moq-vaapi` is a focused VA-API H.264 encoder vendored and trimmed from
//! cros-libva + discord/cros-codecs. It takes tightly-packed NV12 and emits an
Expand Down
2 changes: 1 addition & 1 deletion test/smoke/clients/c/subscribe.c
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ int main(int argc, char **argv) {
if (got) {
fprintf(stderr, "received a frame from %s\n", broadcast);
// The data path succeeded, which is all this smoke client verifies.
// libmoq statically bundles moq-video (openh264/vaapi/cuda), whose
// libmoq statically bundles moq-video (openh264/cuda), whose
// worker threads use priority-protected mutexes; tearing them down at
// normal exit can trip a glibc pthread priority assertion and abort.
// Skip that atexit teardown with _exit now that we have our result.
Expand Down
Loading