Skip to content

feat(spf): add an audio codec preference to the publish encoder ladder - #37

Merged
mmcc merged 1 commit into
mmcc/moq-publisherfrom
mmcc/moq-audio-codec
Aug 18, 2026
Merged

mmcc merged 1 commit into
mmcc/moq-publisherfrom
mmcc/moq-audio-codec

Conversation

@mmcc

@mmcc mmcc commented Aug 18, 2026 •

Copy link
Copy Markdown
Owner

Closes #36

Summary

probeEncoderSupport hardcoded Opus as the only audio codec, so every browser publish was Opus unconditionally — and RTMP/FLV re-publish and LL-HLS egress (both structurally unable to carry Opus) went video-only. This adds the audio.codec seam the issue asks for, as a preference walk with an Opus fallback.

Changes

  • MoqPublishEngineConfig.audio (and ProbeEncoderSupportConfig.audio) gains a codec field — a new exported AudioEncodeTuning type — with the same semantics as the existing camera/screen codec seam: the preferred WebCodecs codec string prepends to the Opus default ladder rather than replacing it, so audio: { codec: 'mp4a.40.2' } publishes AAC where the browser can encode it and degrades to Opus where it can't (e.g. headless Chromium).
  • The audio ladder is now a codec-major cross product over the existing sample-rate ladder — the preferred codec exhausts its sample rates before Opus gets a turn, since the preference exists for egress compatibility, which a rate tweak can't buy back.
  • No catalog or packaging changes needed, as the issue predicted: deriveCatalog already gates mp4a.* on its AudioSpecificConfig init data and only special-cases Opus's 48 kHz sample-rate rewrite, and the audio encoder actor already rides decoderConfig.description on every frame in the LOC Audio Config property.

Testing

  • New tests: the codec-major preference walk (mocked AudioEncoder.isConfigSupported — headless Chromium ships no AAC encoder), the Opus fallback when the preference is unsupported, and ladder dedup when the preference is the default.
  • pnpm -F @videojs/spf test src/publish — 247 passed; spf build + full pnpm typecheck clean.
  • Codex CLI review of the branch diff: no findings.

🤖 Generated with Claude Code


Summary by cubic

Adds an audio codec preference to the publish encoder ladder so browsers that can encode AAC publish AAC; others fall back to Opus. Previously, audio was always Opus, causing RTMP/FLV re-publish and LL‑HLS egress to drop audio.

Review and rollout

  • Adds and exports AudioEncodeTuning; MoqPublishEngineConfig.audio and ProbeEncoderSupportConfig.audio now accept { bitrate?, codec? }.
  • Builds the audio ladder codec-major: the preferred audio.codec is tried across sample rates before Opus; unsupported preferences degrade to Opus.
  • No catalog or packaging changes; existing behavior is unchanged when no audio.codec is set.
  • Tests cover the preference walk, Opus fallback, and ladder dedup when the preference is Opus.
  • Migration: set audio: { codec: 'mp4a.40.2' } to publish AAC where supported; no changes required otherwise.

Written for commit 7f056cb. Summary will update on new commits.

Review in cubic

probeEncoderSupport built its audio ladder from a hardcoded Opus codec,
so every publish session was Opus unconditionally — making RTMP/FLV and
LL-HLS egress video-only for every browser publish even where the
browser ships an AAC encoder.

Mirror the camera/screen codec seam: `audio.codec` on the engine config
prepends a preferred WebCodecs codec string to the Opus default rather
than replacing it, codec-major across the sample-rate ladder, so
`audio: { codec: 'mp4a.40.2' }` publishes AAC where the browser can
encode it and degrades to Opus where it can't. The catalog side already
describes whatever the encoder emits (mp4a init-data gating, non-Opus
sample-rate passthrough), so this is encoder-ladder work only.

Closes #36
@github-actions

Copy link
Copy Markdown

📦 Bundle Size Report

🎨 @videojs/html — 1 small size change
Path Base initial PR initial Initial diff Lazy diff
/media/moq-publish-video 23.91 kB 23.89 kB -13 B (-0.1%) —
Presets (7)
Entry Initial Lazy
/video (default) 58.84 kB 68.34 kB
/video (default + hls) 197.39 kB 68.34 kB
/video (minimal) 58.60 kB 68.34 kB
/video (minimal + hls) 197.08 kB 68.34 kB
/audio (default) 49.20 kB 68.34 kB
/audio (minimal) 45.30 kB 68.34 kB
/background 5.66 kB 68.34 kB
Media (17)
Entry Initial Lazy
/media/mux-video 141.61 kB —
/media/mux-audio 141.39 kB —
/media/background-video 1.14 kB —
/media/container 3.12 kB 68.34 kB
/media/dash-video 211.89 kB —
/media/google-cast 6.46 kB —
/media/hls-audio 21.86 kB —
/media/hls-background-video 15.69 kB —
/media/hls-video 27.39 kB —
/media/hlsjs-video 140.52 kB —
/media/moq-publish-video 23.89 kB —
/media/mux-background-video 15.72 kB —
/media/mux-data 26.71 kB —
/media/native-hls-video 7.51 kB —
/media/simple-moq-video 28.11 kB —
/media/vimeo-video 12.97 kB —
/media/youtube-video 6.75 kB —
Players (6)
Entry Initial Lazy
/video/player 10.14 kB 68.34 kB
/audio/player 7.05 kB 68.34 kB
/background/player 5.38 kB 68.34 kB
/live-video/player 9.54 kB 68.34 kB
/live-audio/player 7.08 kB 68.34 kB
/publisher/player 7.85 kB 68.34 kB
Skins (32)
Entry Type Initial Lazy
/video/minimal-skin.css css 6.30 kB —
/video/skin.css css 6.18 kB —
/video/minimal-skin js 58.59 kB 68.34 kB
/video/minimal-skin.tailwind js 59.33 kB 68.34 kB
/video/skin js 58.83 kB 68.34 kB
/video/skin.tailwind js 59.57 kB 68.34 kB
/audio/minimal-skin.css css 4.62 kB —
/audio/skin.css css 4.56 kB —
/audio/minimal-skin js 45.30 kB 68.34 kB
/audio/minimal-skin.tailwind js 45.85 kB 68.34 kB
/audio/skin js 49.23 kB 68.34 kB
/audio/skin.tailwind js 49.77 kB 68.34 kB
/background/skin.css css 145 B —
/background/skin js 1.14 kB —
/live-video/minimal-skin.css css 6.30 kB —
/live-video/skin.css css 6.18 kB —
/live-video/minimal-skin js 52.69 kB 68.34 kB
/live-video/minimal-skin.tailwind js 53.00 kB 68.34 kB
/live-video/skin js 55.20 kB 68.34 kB
/live-video/skin.tailwind js 55.76 kB 68.34 kB
/live-audio/minimal-skin.css css 4.62 kB —
/live-audio/skin.css css 4.56 kB —
/live-audio/minimal-skin js 39.40 kB 68.34 kB
/live-audio/minimal-skin.tailwind js 38.81 kB 68.34 kB
/live-audio/skin js 42.95 kB 68.34 kB
/live-audio/skin.tailwind js 42.53 kB 68.34 kB
/publisher/skin.css css 4.84 kB —
/publisher/skin js 42.82 kB 68.34 kB
/publisher/skin.tailwind js 43.07 kB 68.34 kB
/global.css css 196 B —
/shared.css css 99 B —
/skin-element js 1.47 kB —
UI Components (52)
Entry Initial Lazy
/ui/airplay-button 11.69 kB 68.34 kB
/ui/alert-dialog 4.22 kB —
/ui/alert-dialog-close 1.95 kB —
/ui/alert-dialog-description 1.63 kB —
/ui/alert-dialog-title 1.65 kB —
/ui/audio-track-radio-group 11.83 kB 68.34 kB
/ui/buffering-indicator 8.88 kB —
/ui/camera-button 11.84 kB 68.34 kB
/ui/camera-radio-group 12.00 kB 68.34 kB
/ui/captions-button 11.94 kB 68.34 kB
/ui/captions-radio-group 11.93 kB 68.34 kB
/ui/capture-placeholder 10.12 kB 68.34 kB
/ui/cast-button 11.63 kB 68.34 kB
/ui/compounds 35.49 kB 68.34 kB
/ui/connection-indicator 9.99 kB 68.34 kB
/ui/controls 9.04 kB 68.34 kB
/ui/enable-devices-button 11.89 kB 68.34 kB
/ui/error-dialog 12.16 kB 68.34 kB
/ui/fullscreen-button 11.63 kB 68.34 kB
/ui/hotkey 10.18 kB —
/ui/menu 19.07 kB 68.34 kB
/ui/mic-button 11.83 kB 68.34 kB
/ui/mic-radio-group 12.00 kB 68.34 kB
/ui/mute-button 11.63 kB 68.34 kB
/ui/pip-button 11.62 kB 68.34 kB
/ui/play-button 11.61 kB 68.34 kB
/ui/playback-rate-button 11.82 kB 68.34 kB
/ui/playback-rate-radio-group 11.85 kB 68.34 kB
/ui/popover 8.11 kB —
/ui/poster 9.28 kB 68.34 kB
/ui/publish-badge 10.07 kB 68.34 kB
/ui/publish-button 10.47 kB 68.34 kB
/ui/publish-timer 10.67 kB 68.34 kB
/ui/quality-radio-group 12.44 kB 68.34 kB
/ui/screen-share-button 11.85 kB 68.34 kB
/ui/seek-button 11.65 kB 68.34 kB
/ui/seek-indicator 12.86 kB —
/ui/seek-indicator-value 1.03 kB —
/ui/slider 12.56 kB 68.34 kB
/ui/status-announcer 11.05 kB 68.34 kB
/ui/status-indicator 13.41 kB 68.34 kB
/ui/status-indicator-value 1.02 kB —
/ui/thumbnail 10.05 kB —
/ui/time 11.41 kB 68.34 kB
/ui/time-slider 15.48 kB 68.34 kB
/ui/time-slider-chapter-title 9.07 kB —
/ui/time-slider-chapters 10.05 kB —
/ui/tooltip 9.44 kB 68.34 kB
/ui/volume-indicator 13.43 kB 68.34 kB
/ui/volume-indicator-fill 1.03 kB —
/ui/volume-indicator-value 1.02 kB —
/ui/volume-slider 13.21 kB 68.34 kB
⚛️ @videojs/react — 1 small size change
Path Base initial PR initial Initial diff Lazy diff
/media/moq-publish-video 22.07 kB 22.08 kB +15 B (+0.1%) —
Presets (7)
Entry Initial Lazy
/video (default) 45.86 kB 68.34 kB
/video (default + hls) 183.05 kB 68.34 kB
/video (minimal) 45.94 kB 68.34 kB
/video (minimal + hls) 183.28 kB 68.34 kB
/audio (default) 35.47 kB 68.34 kB
/audio (minimal) 35.49 kB 68.34 kB
/background 589 B —
Media (19)
Entry Initial
/media/mux-video/hls-js 139.96 kB
/media/mux-video/spf 26.70 kB
/media/mux-audio/hls-js 139.85 kB
/media/mux-audio/spf 20.98 kB
/media/background-video 394 B
/media/dash-video 210.24 kB
/media/google-cast 5.33 kB
/media/hls-audio 20.16 kB
/media/hls-background-video 15.24 kB
/media/hls-video 25.69 kB
/media/hlsjs-video 138.86 kB
/media/moq-publish-video 22.08 kB
/media/mux-audio 139.75 kB
/media/mux-background-video 15.27 kB
/media/mux-data 25.46 kB
/media/mux-video 139.97 kB
/media/native-hls-video 5.74 kB
/media/vimeo-video 11.14 kB
/media/youtube-video 4.85 kB
Players (5)
Entry Initial
/video/player 7.15 kB
/audio/player 4.01 kB
/background/player 2.28 kB
/live-video/player 6.53 kB
/live-audio/player 4.03 kB
Skins (29)
Entry Type Initial Lazy
/video/minimal-skin.css css 6.18 kB —
/video/skin.css css 6.05 kB —
/video/minimal-skin js 45.83 kB 68.34 kB
/video/minimal-skin.tailwind js 52.46 kB 68.34 kB
/video/skin js 45.80 kB 68.34 kB
/video/skin.tailwind js 52.34 kB 68.34 kB
/audio/minimal-skin.css css 4.47 kB —
/audio/skin.css css 4.40 kB —
/audio/minimal-skin js 35.41 kB 68.34 kB
/audio/minimal-skin.tailwind js 40.30 kB 68.34 kB
/audio/skin js 35.40 kB 68.34 kB
/audio/skin.tailwind js 40.29 kB 68.34 kB
/background/skin.css css 90 B —
/background/skin js 273 B —
/live-video/minimal-skin.css css 6.18 kB —
/live-video/skin.css css 6.05 kB —
/live-video/minimal-skin js 37.98 kB 68.34 kB
/live-video/minimal-skin.tailwind js 44.33 kB 68.34 kB
/live-video/skin js 38.07 kB 68.34 kB
/live-video/skin.tailwind js 44.36 kB 68.34 kB
/live-audio/minimal-skin.css css 4.47 kB —
/live-audio/skin.css css 4.40 kB —
/live-audio/minimal-skin js 27.88 kB 68.34 kB
/live-audio/minimal-skin.tailwind js 31.66 kB 68.34 kB
/live-audio/skin js 27.96 kB 68.34 kB
/live-audio/skin.tailwind js 31.79 kB 68.34 kB
/publisher/skin.css css 4.67 kB —
/publisher/skin js 28.78 kB 68.34 kB
/publisher/skin.tailwind js 33.44 kB 68.34 kB
UI Components (47)
Entry Initial Lazy
/ui/airplay-button 10.88 kB 68.34 kB
/ui/alert-dialog 3.32 kB —
/ui/audio-track 8.50 kB 68.34 kB
/ui/audio-track-radio-group 9.65 kB 68.34 kB
/ui/buffering-indicator 8.07 kB —
/ui/camera-button 11.04 kB 68.34 kB
/ui/camera-radio-group 8.50 kB 68.34 kB
/ui/captions-button 10.87 kB 68.34 kB
/ui/captions-radio-group 9.77 kB 68.34 kB
/ui/capture-placeholder 9.06 kB 68.34 kB
/ui/cast-button 10.87 kB 68.34 kB
/ui/connection-indicator 8.99 kB 68.34 kB
/ui/controls 7.81 kB —
/ui/enable-devices-button 11.11 kB 68.34 kB
/ui/error-dialog 10.92 kB 68.34 kB
/ui/fullscreen-button 10.84 kB 68.34 kB
/ui/gesture 8.17 kB —
/ui/hotkey 8.65 kB —
/ui/live-button 9.28 kB 68.34 kB
/ui/menu 18.21 kB —
/ui/mic-button 11.04 kB 68.34 kB
/ui/mic-radio-group 8.51 kB 68.34 kB
/ui/mute-button 10.83 kB 68.34 kB
/ui/pip-button 10.84 kB 68.34 kB
/ui/play-button 10.81 kB 68.34 kB
/ui/playback-rate 8.56 kB 68.34 kB
/ui/playback-rate-button 10.80 kB 68.34 kB
/ui/playback-rate-radio-group 9.70 kB 68.34 kB
/ui/popover 7.81 kB —
/ui/poster 7.69 kB —
/ui/publish-badge 9.04 kB 68.34 kB
/ui/publish-button 9.54 kB 68.34 kB
/ui/publish-timer 9.61 kB 68.34 kB
/ui/quality 8.99 kB 68.34 kB
/ui/quality-radio-group 10.15 kB 68.34 kB
/ui/screen-share-button 11.04 kB 68.34 kB
/ui/seek-button 10.85 kB 68.34 kB
/ui/seek-indicator 12.14 kB —
/ui/slider 12.95 kB 68.34 kB
/ui/status-announcer 10.59 kB 68.34 kB
/ui/status-indicator 12.66 kB 68.34 kB
/ui/thumbnail 8.97 kB —
/ui/time 10.33 kB 68.34 kB
/ui/time-slider 14.53 kB 68.34 kB
/ui/tooltip 8.54 kB —
/ui/volume-indicator 12.76 kB 68.34 kB
/ui/volume-slider 12.36 kB 68.34 kB
🧩 @videojs/core — no changes
Entries (77)
Entry Initial Lazy
. 14.09 kB —
/dom 20.94 kB —
/components 927 B —
/i18n 3.41 kB 68.34 kB
/i18n/locales/all 35.47 kB —
/i18n/locales/ar 1.45 kB —
/i18n/locales/az 1.31 kB —
/i18n/locales/bg 1.53 kB —
/i18n/locales/bn 1.52 kB —
/i18n/locales/bs 1.23 kB —
/i18n/locales/ca 1.29 kB —
/i18n/locales/cs 1.29 kB —
/i18n/locales/cy 1.24 kB —
/i18n/locales/da 1.20 kB —
/i18n/locales/de 1.31 kB —
/i18n/locales/el 1.74 kB —
/i18n/locales/en 993 B —
/i18n/locales/es 1.21 kB —
/i18n/locales/et 1.27 kB —
/i18n/locales/eu 1.24 kB —
/i18n/locales/fa 1.45 kB —
/i18n/locales/fi 1.26 kB —
/i18n/locales/fr 1.29 kB —
/i18n/locales/gd 1.32 kB —
/i18n/locales/gl 1.21 kB —
/i18n/locales/he 1.37 kB —
/i18n/locales/hi 1.54 kB —
/i18n/locales/hr 1.25 kB —
/i18n/locales/hu 1.31 kB —
/i18n/locales/id 1.11 kB —
/i18n/locales/it 1.22 kB —
/i18n/locales/ja 1.41 kB —
/i18n/locales/ko 1.36 kB —
/i18n/locales/lt 1.24 kB —
/i18n/locales/lv 1.30 kB —
/i18n/locales/mr 1.54 kB —
/i18n/locales/nb 1.18 kB —
/i18n/locales/ne 1.52 kB —
/i18n/locales/nl 1.22 kB —
/i18n/locales/nn 1.18 kB —
/i18n/locales/oc 1.29 kB —
/i18n/locales/pl 1.36 kB —
/i18n/locales/pt 1.25 kB —
/i18n/locales/pt-BR 1.25 kB —
/i18n/locales/pt-PT 1.23 kB —
/i18n/locales/ro 1.28 kB —
/i18n/locales/ru 1.63 kB —
/i18n/locales/sk 1.32 kB —
/i18n/locales/sl 1.26 kB —
/i18n/locales/sr 1.29 kB —
/i18n/locales/sv 1.22 kB —
/i18n/locales/te 1.55 kB —
/i18n/locales/th 1.53 kB —
/i18n/locales/tr 1.30 kB —
/i18n/locales/uk 1.64 kB —
/i18n/locales/vi 1.32 kB —
/i18n/locales/zh 1.23 kB —
/i18n/locales/zh-CN 1.23 kB —
/i18n/locales/zh-TW 1.24 kB —
/i18n/text/airplay 101 B —
/i18n/text/buttons 137 B —
/i18n/text/captions 93 B —
/i18n/text/cast 114 B —
/i18n/text/common 90 B —
/i18n/text/container 75 B —
/i18n/text/errors 285 B —
/i18n/text/fullscreen 98 B —
/i18n/text/live 126 B —
/i18n/text/menu 249 B —
/i18n/text/pip 101 B —
/i18n/text/playback 80 B —
/i18n/text/publish 451 B —
/i18n/text/seek 105 B —
/i18n/text/slider 65 B —
/i18n/text/status 222 B —
/i18n/text/time 280 B —
/i18n/text/volume 133 B —
🏷️ @videojs/element — no changes
Entries (2)
Entry Initial
. 996 B
/context 943 B
📦 @videojs/store — no changes
Entries (3)
Entry Initial
. 1.72 kB
/html 703 B
/react 365 B
🔧 @videojs/utils — no changes
Entries (13)
Entry Initial
/array 271 B
/dom 4.49 kB
/events 320 B
/function 369 B
/i18n 44 B
/jwt 176 B
/object 535 B
/predicate 297 B
/percent 281 B
/string 239 B
/style 188 B
/time 1.01 kB
/number 199 B
📦 @videojs/jsx — no changes
Entries (3)
Entry Initial
. 393 B
/jsx-runtime 371 B
/jsx-dev-runtime 375 B
📦 @videojs/media — no changes
Entries (15)
Entry Initial
. 1.15 kB
/dom 121 B
/dom/audio-host 1.10 kB
/dom/custom-media-element 2.09 kB
/dom/dash 209.84 kB
/dom/google-cast 4.06 kB
/dom/hls-js 138.53 kB
/dom/media-host 1.20 kB
/dom/media-played-ranges 576 B
/dom/mux 161.78 kB
/dom/native-hls 5.26 kB
/dom/video-host 1.39 kB
/dom/vimeo 10.54 kB
/dom/youtube 4.23 kB
/media-tracks 2.05 kB
📦 @videojs/spf — 2 small size changes
Path Base initial PR initial Initial diff Lazy diff
/moq-publish 20.34 kB 20.35 kB +12 B (+0.1%) —
/moq-publish-video 21.57 kB 21.58 kB +10 B (+0.0%) —
Entries (13)
Entry Initial
. 4.53 kB
/dom 6.58 kB
/hls 21.06 kB
/media-tracks 604 B
/moq 26.59 kB
/moq-publish 20.35 kB
/moq-publish-video 21.58 kB
/hls-background-video 14.76 kB
/hls-audio 19.67 kB
/hls-video 25.23 kB
/mux-audio 20.49 kB
/mux-background-video 14.76 kB
/mux-video 26.12 kB

ℹ️ How to interpret

Each entry is independently bundled, minified, and brotli-compressed. Initial size includes its static import graph; lazy dynamic chunks are reported separately.

Entries are not additive because their dependency graphs overlap. Preset rows represent realistic combined bundles.

Changes of 300 B or less across initial, lazy, and total size are collapsed, not discarded.

Run pnpm size locally to check current initial sizes.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Re-trigger cubic

@mmcc
mmcc merged commit 61284e6 into mmcc/moq-publisher Aug 18, 2026
25 checks passed
@mmcc
mmcc deleted the mmcc/moq-audio-codec branch August 18, 2026 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant