Repository navigation
feat: adaptive bitrate (ABR) over MoQ — qumo abr transcoder and playground ABR player - #602
Merged
Merged
Conversation
Add `qumo abr`, a relay client that, for every broadcast announced beneath ABR_PREFIX, publishes an ABR broadcast at the same path with ABR_SUFFIX appended (/live/cam1 -> /live/cam1.abr). It carries the source's tracks as they are plus one rendition per ladder rung shorter than the source video, encoded by ffmpeg. - Every video track is in one MSF alternate group (altGroup 1) with its width, height, bitrate, codec and initRef; the source's bitrate is measured from its first groups. - Every rendition numbers its groups as the source does and opens each with a keyframe at the same frame (-force_key_frames source), so a player can switch tracks at any group boundary. - Rendition frames keep the exact LOC timestamp of the source frame they were encoded from: outputs are matched back to source frames by order, confirmed by timestamp. - One ffmpeg decodes the source once for all renditions. Frames go in as fragmented MP4 (one fragment per frame) and come out as FLV (one pipe per rendition): both are read and written frame by frame, so neither adds a frame of delay. Encoding uses a microsecond time base, as the default frame-rate time base snaps live timestamps to a grid. - The source track is forwarded without re-encoding and never waits on the encoder; a lagging encoder drops frames up to the next group. Tested by unit tests (ladder, FLV reader, matcher, track buffer, bitrate meter), an encoder test against real ffmpeg with off-grid timestamps, and an integration test: ffmpeg -> RTMP ingest -> relay -> abr -> relay -> viewer, checking the catalog and group-by-group alignment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
The playground viewer plays an MSF broadcast whose video tracks share an alternate group (as `qumo abr` publishes) adaptively, switching between renditions at group boundaries, and lets the user pick one by hand. - Renditions come from the catalog: the first video track and the video tracks in its altGroup, tallest first, each with its decoder config. - A SwitchingTrack merges the renditions' subscriptions into the one track the frame reader consumes. Renditions number their groups alike, so the new rendition takes over at its first group past those handed out; when it joins at a group already played, the next group is reserved for it so the old rendition cannot race it. A step up keeps the old subscription until it has sent every group before the cutoff (make before break); a step down ends it at once to free the link. - The decoder is configured for each group's rendition at its keyframe. - An AbrController steps down when frames fall behind their media time (or a group is lost), to the best rendition the receive rate since they fell behind carries with headroom; it steps up one rendition after a calm stretch, doubling the wait whenever a step up does not hold, and not past what a subscription's opening burst measured. - The subscribe board gains an Auto/rendition picker and ABR stats in the overlay. fix(relay): a subscriber stopping one group keeps its subscription deliverGroup treated any WriteFrame failure as the subscriber leaving and ended its egress, so a subscriber that stopped one group's stream (STOP_SENDING), as a player gives up a late group or one of a rendition it is leaving, lost the whole subscription. A failure while the subscription is still open now abandons only that group. Verified in headless Chromium against playground + rtmp + abr: playback climbs 360p -> 720p -> 1080p; a 3 Mbit/s limit on the relay-to-browser link drops it to 360p within ~2.5 s, and it climbs back once lifted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
gosec G204: ffmpeg's path is the operator's ABR_FFMPEG and its arguments are built from the ladder, so the launch is marked as intended. staticcheck SA4023: encoder.read returns only once its stream ends, so the nil check on its error was always true. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
…aliases - Rebase source timestamps on the first frame before handing them to ffmpeg: its FLV output carries 32-bit milliseconds, which a source on a large clock (microseconds since 1970) overflowed, so no rendition frame matched its source frame. The encoder test now uses such timestamps. - Close the encoders already started when a later one fails to start, so their ffmpeg is waited for rather than left a zombie (Windows starts one per rendition). - Do not transcode HLS opt-in announcements (/.hls/...): they name a broadcast already transcoded at its own path. - Treat group sequence 0 as a group: readTrack no longer drops it, and the track buffer tracks its position with flags, not a 0 sentinel. - Integration test: no require from the reader goroutines. - Playground SwitchingTrack: watch each stop signal once, not with a new handler on the long-lived signal per acceptGroup call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
…efix A broadcast whose catalog subscription is redirected to another is an alias of it, as an HLS opt-in is: qumo abr no longer transcodes it, and transcodes the broadcast it names, announced at its own path. This replaces the hard-coded token.HLSDirectory check with what makes those announcements aliases, so any other alias is covered too. The integration test announces an HLS opt-in beside the source and checks it is not transcoded; with the redirect check disabled it fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
…ge cases Go unit tests (internal/abr coverage 37% -> 52%): - outputBroadcast: init data listed under the track's name and replaced, not duplicated; TRACK_INFO passes the source's properties on; bitrate. - renditions: the catalog entry a configuration adds (altGroup, codec, size, bitrate, initRef); frames dropped before configuration and before a group opens; groups numbered as the source's, with its LOC and wire timestamps; a group opening on a non-keyframe continues the last one. - initData, isAVC, outputTrack, AliasError. - startEncoders stops the ffmpeg it started when a later one fails. Integration (TestABR_RenditionsBesideSource): audio is forwarded, the source's measured bitrate reaches the catalog, and the ABR broadcast is withdrawn when its source stops. Playground: the controller does not step below the lowest rendition, forgets a burst measurement after 30 s, steps up to a rendition of unknown bitrate, follows catalog updates, and falls to the lowest when nothing fits; SwitchingTrack lets go of a quiet old rendition after 5 s and closing ends every subscription and cancels queued groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
Resolves the CHANGELOG [Unreleased] conflict with #603 by keeping both sides' entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
This was referenced Nov 17, 2026
Merged
The merge of main filed them under the release heading it brought in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJNAzsnsiqRPREa2JqY
… fail - TestStartEncodersClosesStartedOnFailure passed with the cleanup it names removed. Its ffmpeg is now the test binary (TestMain), which marks when its input is closed, so the test fails unless the started encoder is closed and waited for. It no longer needs ffmpeg on PATH. - renditions.configured needs no mutex: each element is touched only by its rendition's goroutine (renditionSink). - Remove flvTagAudio (unused) and encoder.close's nil-cmd check (close is only called on started encoders). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJNAzsnsiqRPREa2JqY
- AbrController's seven tuning options were set by no caller: they are module constants beside WINDOW_MS. Only the clock stays an option (tests). - decoderConfigs built a video config no one read (video is configured per rendition): it is now audioConfig. Rendition.width was never read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJNAzsnsiqRPREa2JqY
…anges f5729d5 changed playground/src without rebuilding the committed dist, so Web UI dist freshness failed. Rebuilt with deno 2.8.1, as CI pins it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB
Collaborator
Author
|
Integration failed on
What I think is happening, the same pattern as the
I'm re-running the failed job once. The ordering fix in the relay (settle and register before answering the grant, or have the test wait for the slot) is a separate change. Generated by Claude Code |
okdaichi
added a commit
that referenced
this pull request
Nov 17, 2026
…o-abr-optin Merged with -s ours, then #606 and #607's changes applied on top: this branch already holds #602, whose squash on main would otherwise conflict with every line this branch changed in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KPWMJNAzsnsiqRPREa2JqY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds end-to-end adaptive bitrate streaming over MoQ, to demonstrate a MoQ advantage: a player switches rendition at any group boundary just by changing which track it subscribes to. There are no manifests and no segments.
qumo abr(server side) is a relay client. For every broadcast announced beneathABR_PREFIX, it publishes an ABR broadcast beside it (/live/cam1→/live/cam1.abr). That broadcast carries the source's tracks unchanged, plus ffmpeg-encoded renditions of a ladder (default 720p/360p), all in one MSFaltGroup.Related Issue
None.
Changes
qumo abr(internal/abr)altGroup: 1, width, height, bitrate, codec andinitRef. The source's bitrate is measured from its first few groups.-force_key_frames source). That makes any group boundary a valid switch point.-enc_time_base 1:1000000). ffmpeg's default time base would snap live timestamps to a frame-rate grid./.hls/) is recognized by that redirect; the broadcast it points to is transcoded at its own path. No path prefix is hard-coded.docs/site/content/en/docs/cli/abr.md, the README, the CLI index andCHANGELOG.md.Playground player (
playground/src/player)catalog.ts:videoRenditions()returns the first video track and itsaltGroupalternates, tallest first.abr/switching_track.tsmerges the renditions' subscriptions into the one track the existing frame reader consumes:abr/controller.tsdecides when to switch. A live stream is sent only as fast as it's made, so the receive rate can't show spare capacity. The controller uses lateness instead:viewer.ts: the decoder is configured for each group's rendition at its keyframe. A newsetRendition()method andonAbrcallback support manual selection and stats.SubscribeBoard.tsxadds the rendition picker andabr/rx/burst/lagstats. The committeddist/is rebuilt with Deno 2.8.1, andscripts/check-webui-dist.shpasses.Relay (
internal/relay/handler.go)deliverGrouptreated anyWriteFramefailure as the subscriber leaving. A failure while the subscription is still open now returnsgroupStopped, which abandons only that group. This also affects the existing player, which already cancels late groups.Testing
internal/abr: ladder, FLV reader, frame matcher, track buffer (including group 0), bitrate meter (-race).deno checkis clean.-tags=integration):TestABR_RenditionsBesideSource: ffmpeg → RTMP ingest → relay →abr→ relay → viewer. It checks the catalog and that every shared group has the same timestamps and frame count, and opens with a keyframe. It also announces an HLS opt-in beside the source and checks it is not transcoded (fails with the redirect check disabled).TestRelay_StoppedGroupKeepsSubscriptionfails without the relay fix.qumo playground+qumo rtmp+qumo abr, 1080p source):tchtb filtered by the browser's port) drops it straight to 360p in about 2.5 s. It climbs back to 1080p once the limit is lifted.Demo:
Checklist
🤖 Generated with Claude Code
https://claude.ai/code/session_01KPWMJL7PH3rPhK5UTGavaB