Skip to content

feat: encoder rate control, off by default - #95

Draft
Frando wants to merge 8 commits into
mainfrom
Frando/encoder-rate-control
Draft

Frando wants to merge 8 commits into
mainfrom
Frando/encoder-rate-control

Conversation

@Frando

@Frando Frando commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Stacked on #94 (base Frando/cleanup-auth). Review that first.

Adds encoder rate control: a publisher lowers its encoders' bitrate when its links carry less and raises it again when they carry more, without restarting an encoder or forcing a keyframe. A ladder pauses its upper renditions before it starves the lowest one. It is off by default: with NodeConfig::rate_control unset, every link reports the congestion window over the RTT as before, and every encoder runs at its configured bitrate.

Design and limits are written up in docs/architecture/rate-control.md, a page of the book under Architecture (listed in docs/SUMMARY.md, linked from the how-to/tune.md guide, and its Rust block compiles with the guide doctests). The measurements and the retune that followed the first version are in plans/rate-control-retune.md, and the full plan in plans/encoder-rate-control-impl.md.

What it does

Everything above the transport is upstream moq: moq_net::bandwidth::Allocator divides the estimate among reservations, moq_mux::rate::Grant turns a slice into encoder targets, and Sink::set_bitrate retunes the encoder. iroh-live adds:

  • An application-level send-rate estimator in moq-transport-iroh (RateConfig, SendEstimate::{Window, App, Controller}, Transport::with_rate). It reads the selected path's PathStats every 50 ms, treats a sustained rise of the smoothed RTT over its 10 s minimum as a queue and falls to 0.85x the delivered rate, lowers on loss above 10%, and otherwise climbs toward the session's probe target. A flat RTT after a back-off is taken as a longer path, and the fall is undone. Controller (the default under rate control) reports BBR3's model, capped by the estimator once the path has shown congestion, since noq's BBR3 never lowers its model while app-limited.
  • The publisher follows the grant. LocalBroadcast::set_bandwidth reserves each video rendition's configured bitrate on the allocator and follows its Grant. A rendition above a lower one pauses below its band boundary (max + 2 * lower) / 3 (drain, mark the catalog entry disabled, close the encoder), and resumes 5% above it with a fresh encoder whose first frame is a keyframe. Backends that cannot retune still pause and resume. Audio reserves its bitrate and keeps it. EncodeStats reports grant and target, and RenditionState::Paused a paused rendition.
  • One threshold for publisher and player. A broadcast with an allocator adds { "rate": { "probe": true } } to its catalog as a root section. A player that sees it fits each rung at the boundary the publisher pauses it at (moq_ext_adaptive::set_boundaries) instead of fit_ratio, so the two loops stop fighting over the same link. Leaving a rung the publisher paused counts as a step down for the backoff. Without the section the player behaves as before.
  • Node wiring. RateControl on NodeConfig, set with NodeConfig::with_rate_control(RateControl::default()). Like the other config structs, RateControl and the transport's RateConfig are #[non_exhaustive] with public fields and with_* builders (RateControl::with_fan_out, with_settle; RateConfig::with_settle). It comes with FanOut::Shared: one allocator registry shared by every session, dialed and accepted, fed with the mean estimate of the links carrying media. Node::bandwidth hands it out. Calls::new sets it on its LocalBroadcast automatically; any other broadcast, including one from LiveLayout::publish, takes it with set_bandwidth, since LiveLayout now lives in iroh-live-node, which knows no media types.

Not in this PR

  • Padding probes. The estimator's climb is the probe; the session's probe target is the sum of reservations. No DATAGRAM padding prober yet.
  • Other fan-out policies (Lowest, Uplinks, Quantile). Only Shared.
  • Pre-encoded sources like rpicam declare no bitrate, so they reserve nothing and keep their rate.

moq dependency (not for merge)

Sharing the allocator with accepted sessions uses moq_net::Server::with_allocator, the counterpart of Client::with_allocator, which landed upstream in moq-dev/moq#8926 and is in the moq rev main pins (4a6f52248). No moq patch is needed.

Test evidence

Patchbay lab, IROH_LIVE_PATCHBAY_RATE=controller for "on". The adaptation scenarios use high at 800 kbit/s over low at 200 kbit/s. "Before" is the first version of the estimator, "after" is this branch, three runs each with --retries 0.

Scenario Rate control Before After
Marginal cap, switches (limit 3) on 7, 7, 5 (fails) 3, 3, 3
Marginal cap, switches off 3 3
Rate limit, up-switch after the cap lifts on 15.0 s (17 to 19 s earlier) 9.3, 9.6, 10.6 s
Rate limit, up-switch off 3.3 s 3.2 s
Risen baseline RTT, switches on 1 (fails) 0, 0, 0
Risen baseline RTT, switches off 0 0

Full patchbay suite with --run-ignored all: 10 of 10 pass on the first try with rate control on and off. a_switch_lands_while_the_link_stays_capped stays ignored and fails either way for the starvation documented in its ignore reason; it now always runs without rate control, since an encoder that follows the cap drains the queue the test relies on. cargo nextest run --workspace --all-features: 683 passed.

New scenarios: the_encoder_follows_a_cap_without_stalling, the_encoder_recovers_when_the_cap_lifts, a_paused_rendition_resumes_when_the_cap_lifts.

Known limits

  • Up-switch takes ~10 s after a cap lifts, against ~3 s without rate control. Without padding, the estimate climbs blind from the floor the cap left it at to what the paused rung needs beside the lower one.
  • The estimate is per hop. The rate catalog section travels with the broadcast, but a player behind an iroh-live-relay gets the relay's window estimate, which reads high, and holds a rung down to its boundary on it. Relay links over moq_tokio don't feed the node's estimate, and the relay doesn't run the estimator toward its viewers.
  • RTT steps cost one false fall. The smoothed RTT can't tell a longer route on the same QUIC path from a queue. The estimator falls once and undoes it about 2 s later; a rung whose share drops under its boundary in that window still pauses. An iroh path change is a new path and holds through the settle window instead.
  • The player compares the whole estimate against a rung's boundary while the publisher compares the rung's grant, so the publisher pauses first and the catalog moves the player.

Current state: rebased onto main after the moq bump (#103). The moq fork patch is gone. LiveLayoutExt::publish_media and RoomExt::publish_media hand the node's allocator to layout and room publishes, as Calls does. Checked with the workspace check, clippy, fmt, doctests and the rate-related lib tests; the patchbay rate scenarios were not re-run on this rebase.

🤖 Generated with Claude Code

@Frando
Frando force-pushed the Frando/cleanup-auth branch from ca11cd1 to fa7c55f Compare November 14, 2026 00:15
@Frando
Frando force-pushed the Frando/encoder-rate-control branch 2 times, most recently from 0922a7c to d64bde1 Compare November 16, 2026 11:12
@Frando
Frando changed the base branch from Frando/cleanup-auth to main November 16, 2026 14:04
@Frando
Frando force-pushed the Frando/encoder-rate-control branch from d64bde1 to 8a85520 Compare November 16, 2026 15:25
Frando and others added 8 commits November 18, 2026 11:08
…fault

Transport::with_rate takes a RateConfig whose SendEstimate picks the
estimate moq-net reads. Window, the default, keeps the congestion
window over the round trip. App and Controller run an estimator over
the selected path's statistics: a queue in the smoothed round trip
lowers it to 0.85 times the delivered rate, loss over 10% by half the
loss rate, and a clean path lets it climb toward 1.5 times the
session's probe target. Controller reports BBR3's model capped by it.
Such a transport takes the session's probe target, so reservations
count as probed. It sends no padding: the climb is the probe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LocalBroadcast::set_bandwidth reserves each video rendition's
configured bitrate on an allocator and follows its grant through
moq_mux::rate::Grant with Sink::set_bitrate. A rendition above a lower
one pauses below its band boundary, shown as RenditionState::Paused
and as a disabled catalog entry, and resumes when the grant clears it.
An encoder that cannot retune still pauses. Audio reserves its bitrate
on the same allocator. EncodeStats reports each rendition's grant and
target. Without an allocator nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With NodeConfig::rate_control set, the node builds every direct and
iroh relay transport with the configured SendEstimate, and shares one
allocator registry with all its sessions. FanOut::Shared divides the
mean estimate of the links that carry media among the publications'
reservations. Node::bandwidth hands that allocator out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calls::new sets the node's allocator on its LocalBroadcast when the node
has rate control; other broadcasts take it with set_bandwidth. Describes rate control,
its estimate and its limits in docs/architecture/rate-control.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three patchbay scenarios: the encoder follows a cap down without
stalling, climbs back once it lifts, and a paused rung resumes.
IROH_LIVE_PATCHBAY_RATE runs every scenario with the publisher's rate
control on, and a trace prints the bitrate's course.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A round trip that rose on the same QUIC path read as a queue. The
estimate fell, and fell again every two seconds while the round trip
stayed flat, until the ten-second minimum rolled over. A queue drains
once the sender backs off, and a longer path does not. So a round trip
that stays flat for a second after a back-off, with no loss, now becomes
the baseline, and the fall is undone.

After a cap lifted, the estimate took 15 s or more to clear what a
paused rendition needs. One burst of loss counted on every reading of
its two-second window, and drove the estimate to its floor. Each fall
of that one congestion also doubled the hold. A loss fall now needs loss
since the last fall, and only a fall that ends a climb doubles the hold.

The climb now runs at 25% a second up to the rate the path last
congested at, since the path carried that rate. A long climb beyond the
near band speeds up to about doubling each second. An encoder still
follows the estimate no faster than its own ramp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A publisher with rate control pauses a rendition below its band
boundary, (max + 2 * lower) / 3. Its players left the rendition at 1.25
times its bitrate. For 800 kbit/s above 200 kbit/s that is 1 Mbit/s
against 400 kbit/s, so a player left a rendition the publisher still
funded.

A broadcast with an allocator now adds a root `rate` section to its
catalog. A player that reads it fits each rung at the boundary the
publisher pauses it at, computed from the catalog bitrates. A step up
needs 5% more, as the publisher needs to resume. Without the section the
player keeps its fit ratio, so nothing changes with rate control off.

Leaving a rung the publisher paused now counts as a step down, so the
backoff spaces out the tries under a marginal cap. The estimate now
climbs to the probe target, not 1.5 times it, since no player needs more.

The capped switch test always runs without rate control. Its premise is
a link that stays saturated, and an encoder that follows the cap drains it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`LiveLayout` and `Room` live in crates that know no media types, so their
generic `publish` cannot give a `LocalBroadcast` the node's allocator, and
every caller had to remember a manual `set_bandwidth`. The sealed
`media::LiveLayoutExt` and `media::RoomExt` traits add `publish_media`,
which publishes a `LocalBroadcast` and hands it `Node::bandwidth` when the
node has rate control. Calls use the same helper for their own broadcast.

`Room::node` returns the node a member publishes through, which the room
extension needs. The examples, the CLI, the Android and Pi Zero demos, the
patchbay fixture and the docs move to `publish_media`, and the
`Node::bandwidth` doc no longer claims that `LiveLayout::publish` hands the
allocator over itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Frando
Frando force-pushed the Frando/encoder-rate-control branch from 8a85520 to aa0140c Compare November 18, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant