Repository navigation
Conversation
This was referenced Sep 2, 2026
moq accepted iroh-live's native media stack and shipped it (moq-dev/moq#2481, closed). Everything we upstreamed now exists in `moq-video` and `moq-audio`, so the local copy is dead weight: this deletes `rusty-codecs`, `rusty-capture`, and most of `moq-media`, and rebuilds what is left on the released crates. About 50,000 lines go. What survives in `moq-media` is the layer upstream has no counterpart for. One upstream producer publishes one rendition and owns its device, so simulcast is ours: the source is opened once and its frames fan out to an encoder per rendition. `moq_mux::select` fixes a rendition at construction, so subscriber-side adaptation is ours. Neither crate has a playout clock, so that is ours too. The catalog extension carrying chat and publisher identity stays, since it is built on the `CatalogExt` seam moq designed for it. Rooms leave `iroh-live` for a new `iroh-rooms` crate, media-free. They are cut rather than ported: the redesign onto moq's announce bus is planned separately, and what this branch owes it is working primitives, not a faithful copy of the gossip-and-KV design it replaces. `iroh-moq` is rewritten against moq-net 0.2. Content now enters an origin only through `create_broadcast`, so `Moq` owns one node origin and publishing is node-wide; the actor's republish-to-every-session bookkeeping is gone, and `Call` moves from the fixed name "call" to a per-peer path, which also lets two calls coexist. Sessions hand back a `Driver` that has to be polled, so it joins the actor's existing `JoinSet`. The single hardcoded ALPN is replaced by the full `moq_net::ALPNS` list on both the dial and the router, which fixes an interop bug that would only have surfaced once the two sides drifted. Five moq changes this depends on are not released yet, so the root manifest patches the moq family to a local checkout. Deleting that block is the whole revert once they land; until then CI cannot pass, and the workflow says so. - `Live::publish(path)` is synchronous and returns a `LocalBroadcast`; `Live::publish_raw(path)` returns a bare `broadcast::Producer`. `LocalBroadcast::new()` and `Live::publish_broadcast_producer` are gone. - `MoqSession::publish` is gone. Publishing is node-wide; use `Moq::publish`. `Call` broadcasts live at `Call::path(endpoint_id)`. - `iroh_live::rooms` moves to the `iroh-rooms` crate, and `RoomEvent::BroadcastSubscribed` carries a `broadcast::Consumer` rather than a `RemoteBroadcast`. - `moq_media::{codec, format, traits, capture, render, transport, audio_backend, pipeline, test_util}` are gone. A frame is `moq_video::Frame`, a capture config is `moq_video::capture::Config`, and the audio device engine is `moq_audio::playback::Engine` behind `moq_media::playback`. - Codec feature flags are gone, because every codec compiles upstream. The features are now `capture`, `playback`, `aec`, `pipewire`, `render`, `vaapi`, `nvidia`, `rpicam`, and `test-source`. - VAAPI decode and the V4L2 M2M codecs have no replacement upstream yet. Software H.264 decode covers correctness on those machines. - `irl call`, `room`, `record`, `run`, and `relay` are removed, along with the `headless` and `opengl` demos and `moq-media-dioxus`. All are recoverable from `main`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two adversarial reviews of the restructure, one on Rust and API quality and one on concurrency and lifecycles. Neither found data loss or a deadlock; between them they found eleven real defects. The correctness ones: Every media track was created with `track::Info::default()`, which is milliseconds. moq-lite-05 delta-encodes frame timestamps at the track's own timescale, so each rendition quantized its timestamps and ran with moq-net's 5s latency window rather than hang's 30s. The source-ended path published the encoder's tail and then dropped the producer, so subscribers saw `Error::Dropped` on the one path the code called a clean end. The encode-error path did the same and now aborts with the real cause. Nothing cancelled a broadcast's shutdown token when its last handle dropped, so iroh-live's stats recorder held a `Connection` and woke every 200ms forever, one per subscription. `FrameReceiver::recv` registered for its wakeup after checking for closure rather than before, so a sender dropping in that window left the receiver parked with nothing to wake it. A rendition switch that failed to open left its request set, and the adaptation loop reads an outstanding request as a switch in flight, so the first failed downgrade under congestion disabled adaptation for the session. Opening the replacement also awaited inside a `select!` arm, so the incumbent stopped flowing for exactly the window the overlap exists to hide. Replacing a video source spawned the new publish task before the old one's track producers had dropped, so `create_track` returned `Duplicate`; the CLI's source picker took that path on every change and the only symptom was a log line. The upgrade probe was inert: `StartProbe` was treated as a plain switch, so the probe timers and the abort threshold did nothing. `fps` recorded the constant 1.0. The accept half of the handshake did not validate the negotiated ALPN. `Live::shutdown` skipped closing the endpoint whenever the router teardown failed, which its own doc denied. Two peers dialing each other simultaneously still end up with two connections, and the second is now kept rather than closed. Closing either one breaks the peer: sessions are handed to callers before the collision becomes visible, so a subscribe already in flight dies with it, and a deterministic tie-break does not help for the same reason. A generation counter stops the loser's task from evicting the live session, which was the half that actually caused harm. The rest is honesty about what the code does: an audio setter whose `Result` could never be an error, a playout-clock setter repurposed without its docs following, a blocking `wait` and its condvar with no callers left, and doc comments in `call.rs`, `sync.rs`, and `playout.rs` describing designs this branch deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Moq::connect` defined a typed `Error` with variants for every way a dial can fail and then laundered all of them through `AnyError`, so a caller could not tell a peer that speaks no version we know from a dial that timed out. That is exactly the case the ALPN work was about. `Call` now carries the same typed error through. `IncomingSession` claimed an application could inspect a peer and accept or reject it, but by the time the value exists the handshake is complete, the session is in the map, its driver is running, and every other subscriber holds a clone. Neither it nor `reject` had a caller. `incoming_sessions` now yields the session directly and says plainly that it reports what happened rather than gating it. Room broadcasts were created at their bare name on the node origin, so a peer in two rooms that published "cam" in each had the second rejected as a duplicate. They live under `rooms/<topic>/<name>` now, which both sides derive from the ticket they already share. Also removes build configuration for code that no longer exists: the Pi cross-test task built the hardware-gated codec tests that went upstream with the codecs, and `iroh-live` still pulled eframe, wgpu, winit and five more dev-dependencies for the four examples this branch deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`VideoTrack` derived `Clone` over a latest-wins slot whose `take` removes the value, so two clones would split the frame stream and neither would see a coherent picture. Nothing cloned it; the derive is gone and the doc says why. `--bitrate` was applied verbatim to every rung of a simulcast ladder, so 180p and 1080p advertised the same figure and a subscriber comparing its estimate against `bitrate_bps` could not tell them apart, which is the whole input to the bandwidth half of adaptation. It now scales by pixel count against the largest rung, with a floor. `<w>x<h>` also rounds both dimensions up to even, which the `<h>p` shorthand already did and the comment already claimed: I420 chroma is subsampled 2x2 and an odd dimension is rejected further down. `LagTracker`, `EncodeOpts`, `DecodeStats`, `RenditionMode`, and `FrameReceiver::new_sender` were all public with no caller, each of them serving a piece of the pipeline module this branch deleted. `new_sender`'s tests are replaced by one that covers the close race the last commit fixed. `enable_chat` and `set_user` took `&mut self` where every other mutator on `LocalBroadcast` takes `&self` through a lock, which a UI loop holding the broadcast cannot satisfy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round two reviewed the round-one fixes hardest, on the reasoning that a fix commit is where fresh, less-tested code lives. Two of those fixes had introduced defects of their own. The generation counter that was supposed to stop a stale session task from evicting a live session tagged the duplicate connection with the *incumbent's* generation, because `spawn_driver` read the global counter rather than the one assigned to the session it was driving. The idle duplicate closing first then evicted the live session from the dedup map, which is exactly the harm the counter was added to prevent. The replacement decoder's open was moved onto its own task and its handle dropped to cancel it, but dropping a `JoinHandle` detaches rather than aborting. A superseded open ran to completion, holding a track subscription and a broadcast clone for as long as the peer took to answer. Three more in the same supervisor. The incumbent's reader ending discarded a replacement that was already open, so a downgrade racing a failing incumbent stopped video permanently; the replacement is promoted now. Re-requesting the rendition already playing did not cancel a pending swap, so un-pinning still swapped. And a superseded open labelled the overlay with a rendition that never played. The publish slot guard covered `run` but not the encoder tasks, which is where the track producers actually live, so on the abort path it was released before those futures dropped and a replacement could still hit a live track name. The reviewer reproduced that ordering; the guard is shared with each encoder now. A probe whose switch never landed still armed the abort path, which then stepped down from the rung the viewer never left. `AudioTrack` did not hold the broadcast, so an audio-only subscription fell silent when its `RemoteBroadcast` went out of scope, where `VideoTrack` holds one through its control block. The relay's pull dialed a single ALPN and assumed raw QUIC, the bug the transport crate had just fixed for itself, so a publisher on an older moq release could not be pulled. Both now go through `iroh_moq::dial`. Pulled sessions also gave up their connection only when the remote closed, so a relay held one per ticket ever pulled; they now stop once the connection has carried nothing for a minute. Android's `disconnect` held a `std::sync::Mutex` across `block_on(live.shutdown())`, blocking every other JNI entry point, including the status line the UI thread reads, for the length of an endpoint close. The rest is docs telling the truth: `RoomEvent::PeerLeft` says it is derived from media liveness rather than membership, `Room::publish` says not to call it from the task that drains `recv`, and three intra-doc links no longer point at a method this branch deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round three was pointed at round two's fixes, and found that two of them were broken. That is the second round in a row where the fix commit was where the defects were. `Reader::spent()` gave the video supervisor somewhere to park while a replacement decoder opened, and no way out. If the publisher or the connection went away during a switch, the incumbent's reader closed, the replacement's open then failed against the closed broadcast, and all four select arms pended forever: nothing playing, nothing pending, nothing opening, and a shutdown token nobody had cancelled. The supervisor never returned, so its frame sender was never dropped, so `VideoTrack::recv` never returned `None` and the Pi demo's watch loop sat on a frozen picture instead of exiting. The reader is an `Option` now, and every path that leaves nothing to decode returns. The relay's idle timer could never fire. `bytes_received` is a transport counter, and iroh sends keep-alives every five seconds, so a connection nobody is reading still moves it. The timer is gone and the module doc says plainly that the connection outlives the broadcast, why, and what would actually answer: the cluster's subscriber count, which `moq_relay::Cluster` does not expose. Three smaller ones. Promoting a replacement on incumbent-end updated the watchables but not the overlay labels, which is the bug the last commit moved those labels to fix. An encoder parked on `demand.used()` never observed the source ending, so `fan_out` waited on it forever holding every track producer; it selects against the frame channel's closure now. And a probe took its congestion baseline when the switch was requested rather than when it landed, charging congestion on the rung it had not left yet against the probe, so a slow switch could be aborted on its first evaluated tick. The rest is documentation: fifteen rustdoc warnings, including three links this branch broke while fixing others, a `wait_async` described as "the async counterpart of `wait_async`", and a relay module doc that said no idle timer was needed directly above one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dial A duplicate connection was driven but unreferenced: nothing could reach it, and when the session that did hold the peer's slot ended, the slot was cleared and the next `connect` dialed a third connection, which the peer then treated as another duplicate. Sessions are per-peer lists now, oldest first. `connect` hands out the first, both are driven, and a task's exit removes its own entry by generation, so whichever remains starts serving. The generation still exists for the reason it was added: a task landing after a newer session arrived must not evict the newer one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `[patch.crates-io]` block pointed at a local `../moq` checkout, so CI could not run at all. It points at `Frando/moq@iroh-live` now, the same five commits one per prospective PR, with `Cargo.lock` pinning the revision so a clean clone and CI build what a local checkout builds. Pushing to `moq-dev/moq` needs write access this account does not have; the fork serves until the commits land. Running the checks the way CI would then turned up four breaks: - `demos/android/rust` did not compile. `AudioPublisher::set` became infallible in an earlier review round and the Android crate kept the `?`. Nothing caught it because the crate only builds under `--target aarch64-linux-android`, which `cargo check --workspace` never does. `cargo ndk -t arm64-v8a check -p iroh-live-android` does. - `--no-default-features` did not build. The `publish` example opens a camera, so it takes `required-features = ["capture"]`. The CLI is `devices` and `publish` plus a viewer, and neither of the first two compiles without capture, so it asks for it on the dependency edge rather than pretending it is optional. `pi-zero-demo`'s poll interval is used only by the windowed renderer and is gated with it. - Nothing on this branch had been formatted with the repository's own rustfmt config, only with the default one. Seventeen files move. - The nextest config still reserved a serial test group for the patchbay suite and cargo-make still had a task to run it, both deleted along with the suite. The devtools doc claimed a dev-dependency that is also already gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps the moq pin from `dc3d9945` to `ac8a95c8`, which carries a VAAPI H.264
decoder, a VPP re-tile, and a dead-code fix. `--features vaapi` now reaches a
hardware decoder on Intel and AMD Linux instead of falling through to openh264
in software, which closes the regression the v2 summary called the one most
likely to be noticed.
The decoder is split the way the encoder already was: `moq_vaapi::decode`
drives libva, and moq-video's backend is a thin adapter. It is verified
byte-for-byte against ffmpeg's software decoder on constrained baseline, main
with B-frames, high 720p, a cropped non-macroblock-aligned size, and a
mid-stream resolution change.
`moq-vaapi` needs a patch entry of its own here. A `[patch]` block only takes
effect in the top-level workspace, so moq patching it for its own builds does
nothing for a workspace that depends on moq, and `--features vaapi` failed to
resolve `moq_vaapi::{decode, vpp}` until the entry was repeated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Subscribes to a broadcast and writes it to a file, headless. The muxing is
`moq_mux::container::{fmp4,mkv}::Export`, which is what `moq export fmp4`
drives on the other side of the same library, so the only thing this adds is
the iroh subscription, the output file, and the flags.
That also retires the AVCC-to-Annex-B conversion the pre-v2 recorder carried.
The exporter builds each track's decoder configuration from the catalog and
transcodes an `avc3` track with inline parameter sets into the `avc1` shape a
container wants, so nothing in the CLI has to understand H.264 framing.
The fMP4 exporter is given an explicit fragment cap, which is a workaround
rather than a preference: with none set it rolls a fragment only on a video
keyframe, so on a broadcast carrying both media the audio buffers in memory
and never reaches the file. The Matroska exporter has no such problem.
The container follows `--output`'s extension, `--format` overrides it, and
both formats are fragmented: an interrupted recording is complete up to the
last chunk written rather than a file with no header.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One endpoint publishing several broadcasts, subscribing to several others, and recording any of them, which a pile of `irl publish` and `irl watch` processes cannot do between them: each of those binds an endpoint of its own. The schema keeps the old one's shape and its `secret_key_name`, `[[send]]`, and `[[recv]]` blocks. What changed follows the media stack: - A `[[send]]` block's keys are now the `irl publish` capture flags under the flags' own names, so `video_source` is `video`, `audio_source` is `audio`, and `video_codec` splits into `codec` and `encoder` the way the flags did. - `video_preset` is gone with the presets themselves. `renditions` takes the simulcast ladder instead, alongside `bitrate`, `width`, `height`, and `fps`. - Every key but `name` is optional and defaults to its flag's default, where the old schema required all of them. An unknown key is rejected rather than silently dropped, which is what the old required-field list bought. - `audio_bitrate` and `audio_output` are wired rather than parsed and ignored. - `recv.record` names one output file, whose extension picks the container, where it used to be a stem for a pile of raw elementary streams. A `[[recv]]` block also takes `rendition`, to record one rung of a ladder. `secret_key_name` binds the endpoint with the stored key instead of writing `IROH_SECRET` into the process environment, so nothing reaches for `set_var`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pull mode dialled the publisher and then held the connection until the publisher went away, so a relay accumulated one QUIC connection per ticket ever pulled, whether or not anyone was still watching. Nothing in the cluster owns that connection: the mirrored broadcast is announced for as long as the session feeds it, not the other way round, so the session has to be retired deliberately. Two signals decide when, because either one alone is wrong. Every local session that named the ticket now holds a `PullGuard` for its lifetime, which covers a browser that has connected but not subscribed yet and so has no demand to show for itself. The mirrored broadcast's `moq_net::broadcast::Demand` covers the opposite case, a subscriber that reached the broadcast over some other session and holds no guard. Once both have been quiet for the linger window the session is dropped, which closes the transport, and the ticket's entry is retired so the next pull dials afresh. The claim and the retirement are taken under one lock, which is what makes the re-dial path safe. A pull claimed while the holder task was waiting out the linger keeps running, and a pull the holder retired can no longer be claimed, so a second browser arriving as the first leaves either joins the live session or starts a new one, never a session that is about to close. The dial itself runs in its own task and every caller waits on its outcome, so a caller that gives up (a timeout around a now public `pull`) withdraws its interest without stranding the ticket on an entry nobody will ever connect. The idle timer that was tried and removed stays removed. Every counter `moq_net::Session::stats` reports is a QUIC counter, and iroh sets a five second keep-alive on every connection, so a connection nobody reads moves them exactly like a busy one does. `pull` is public so the tests can drive it: the leak is a lifecycle property of the relay's own state, and the accept loop that owns it needs a real HTTP listener and data directory to run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`irl record` set a 500 ms `fragment_duration` because `fmp4::Export` left unset rolls a fragment only on a video keyframe, so an audio track alongside video accumulated in memory and reached the file as one fragment at the end. That is fixed upstream in `17abe77c6`: a video keyframe now rolls every audio track alongside it, which is what `mkv::Export` already did. So the cap goes, and a recording gets its GOP written whole again rather than cut every half second. Verified against a live publisher: 8 seconds of 1280x720 H.264 and stereo Opus, 49 video and 165 audio packets, decoding clean through `ffmpeg -f null -`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nothing anywhere impaired a link any more. The adaptive path shipped untested end to end: the algorithm has unit tests and `e2e.rs` drives a switch, but it does so by writing `NetworkSignals` by hand, which covers the decision and none of the four stages that feed it. moq-net has no equivalent, since its session tests run over in-memory transport doubles. Four tests, rewritten against the new pipeline rather than ported. A publisher and a subscriber sit in separate network namespaces with a router between them, and netem impairs both links, because the router only forwards and impairing one leg would leave acknowledgements a clear path back. Two tests hold delivery to account across a latency ramp and a loss spike. `adaptation_follows_a_real_link` runs the whole chain, from dropped packets through QUIC's loss detection and the path stats the signal producer samples to a downgrade, and back up once the loss clears. `a_switch_does_not_blank_the_picture` holds the decode supervisor to its overlap, that the replacement takes over on its own first frame. The impairment is loss rather than a rate limit, which took a rewrite to arrive at. Both signals are read off the path the subscriber sends on, and a subscriber sends little but acknowledgements, so its congestion window measures how little it sends rather than what the downlink could carry. A rate-limited downlink registers in neither, and an application-limited congestion window takes about a minute to recover afterwards, which is what the upgrade would have been waiting on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite asserts on frame timing and on switches driven by measured transport stats, so it needs the machine to itself: a single-threaded nextest group, and one retry for the moment netem takes to settle after it is reconfigured. `cargo make test-patchbay` runs it on its own, including ignored tests. The devtools guide said the suite was gone along with its dev-dependency, its task and its group. Corrected, and what is genuinely still missing named instead: the A/V sync measurements, whose timestamping audio backend has no counterpart in the new pipeline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`irl publish --preview` and `irl watch` each built their own wgpu `NativeOptions`, and each carried its own copy of the frame-pump that turns a capture tap or a decoded track into something egui can draw. The windows still to come, one remote for a call and a grid of them for a room, want the same three pieces, so they move into `ui`: - `native_options`, the wgpu configuration `moq-media-egui`'s renderer needs, which is not optional: a frame arrives as a `wgpu::Texture` and nothing draws one through glow. - `LocalPreview`, the publisher's own picture, read fresh from the broadcast each frame so a source switch cannot orphan the tap. - `RemoteView`, one remote broadcast on screen: the decoded picture, the audio track that keeps it playing, the stats overlay, and the rendition and volume controls that go with them. `irl watch` loses its private window state to `RemoteView` and picks up its `--rendition` handling there, which moves the choice out of the connect path and into the window that can change it anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Call::dial` and `Call::accept` took a `LocalBroadcast` by value, stored it, and handed it back through `Call::local`. Its own documentation said the parameter did nothing: "the local broadcast was created on the node origin before the call started, so it is already announced." Doing nothing is not free. It meant the broadcast could not outlive an attempt, so a failed dial, a refused answer, or a timeout took the open capture devices with it, and the only way back was to publish at the same path again. Republishing under a live session is announce churn that subscribers see as the broadcast ending, so a plain `irl watch` against a node waiting for a call had its media torn down the moment that node gave up on answering it. Publishing is node-wide. A broadcast created once is announced on every session, including a call's own, and outlives any number of calls, so the call has no business owning it. `dial` and `accept` lose the parameter, `local` goes, and the caller keeps its own broadcast where it already keeps its capture. The android demo did the same shuttling through an `or_else` that reached into the call for the broadcast; it now just holds it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both peers publish their own camera at `calls/<their endpoint id>` and subscribe to the other's, so once the session is up there is no caller and no callee: the difference is only who dialed. `iroh_live::Call` does that part; the window is what was missing. It opens on a waiting screen showing this node's ticket, a box to paste the peer's into, and the local camera. A ticket on the command line is dialed straight away, so a call is `irl call` on one machine and `irl call <TICKET>` on the other. Connected, the peer fills the window with the local camera in a corner, and moving the pointer brings up the ticket bar, the stats overlay, and the rendition and volume controls. Hanging up on either side returns both windows to the waiting screen. Two things the old command had are gone. The device selectors that switched camera and microphone mid-call went with the UI they lived in; `--video` and `--audio` pick the devices, and `irl publish --preview` is where switching sources is exercised. The decoder and audio-device flags went too: `moq-media` chooses a decoder itself now, and no CLI command has an output-device flag left. The state machine runs in `App::logic` rather than `App::ui`, and a heartbeat wakes the window ten times a second. eframe runs a pass only when something asks it to, and a window that is unfocused or occluded stops asking, which would leave a call unanswered for as long as nobody was looking at it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rooms left `iroh-live` for the `iroh-rooms` crate in the v2 rewrite, and the command that drew them went with the old UI. This is the same idea against the crate's own API: join a gossip topic, publish one broadcast called `cam` into it, and turn every broadcast the room reports into a tile in a grid, with the local camera as the first tile. Chat rides on the same broadcast as the video. `LocalBroadcast::enable_chat` creates the track and advertises it in the catalog, and the room finds it by the well-known name it was created under, so a peer subscribed for the picture gets the messages without a second subscription. Joining and leaving appear in the panel as they happen. `iroh-rooms` reports a subscription as a session and a broadcast consumer side by side, which is what `Live::subscribe` assembles into a `Subscription` and then wires the stats recorder and signal producer onto. Rather than repeat that wiring here, `Subscription::new` becomes public; the CLI wraps the consumer in a `RemoteBroadcast` and hands both halves over. Left out of the old UI: the grid and picture-in-picture self-view toggle, which was two layouts for one small window, and the per-tile rendition picker, which is noise in a grid where every tile follows the downlink anyway. Membership is what the crate offers and no more: a tile appears when a peer's broadcast opens and disappears when it closes or its session drops, so a participant that joined the topic and published nothing is never drawn. That is the `PeerLeft` derivation the crate documents as pending the announce redesign, not something to paper over here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`spawn_signal_producer` derived its bandwidth estimate from `cwnd * 8 / rtt`. That congestion window belongs to the direction the endpoint sends in, and a subscriber sends little but acknowledgements, so the figure measured how little it sends rather than what the downlink could carry. A rate limit on the way in was invisible to it, and after one lifted the estimate crawled back over tens of seconds. The stats recorder forty lines below has computed real received throughput for the UI all along, so the picture on screen was honest while the loop deciding renditions was not. `NetworkSignals::available_bps` becomes `goodput_bps: Option<u64>`, taken from `udp_rx.bytes` over a one-second window. Goodput is not capacity: it is bounded by what the publisher chose to send, which makes it a lower bound on the path and enough to show a rendition failing to arrive in full, never enough to show room above the rate already flowing. It reads `None` while too little is arriving to be media, so a publisher going quiet is an absence of evidence rather than a link that collapsed. Two findings from the patchbay lab shaped what the loop does with it. A 200 kbit/s cap on a 316 kbit/s stream leaves `loss_rate` at exactly zero throughout, because nothing this endpoint sent was dropped; loss on a subscriber is loss among its own acknowledgements and stands in for path loss only as far as both directions are impaired alike, which is now what it says. And a rendition's catalog bitrate is a ceiling rather than a promise: openh264 sends about 40% of what it is given, so a goodput shortfall measured against it says nothing on its own. What the cap does show is a queue, taking the round trip from 1ms to 330. So a downgrade now needs the shortfall and the queue together, against a new `min_rtt` baseline, and the upgrade gate loses its bandwidth precondition entirely: no receiver-side measurement can show room above the rate already flowing, and finding it is what the probe is for. `adaptation_follows_a_rate_limit` drives the whole chain through an impairment that drops nothing, and asserts loss never reached the threshold that could have explained the downgrade. It lifts the cap before the switch completes, because a cap that starves the top rung cannot carry both rungs during the overlap the seamless switch needs, and under that overlap the replacement's subscription does not get set up at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rate limit test asserted only that the signals eventually showed the cap, which the congestion-window estimate it was written to catch also manages: not because the window learned anything, but because the queue in front of the bottleneck inflates the round trip it is divided by. Run against the old estimate the assertion passed after 43 seconds, against about 4.8 with the goodput window. So the bound is now part of the claim. A bandwidth figure that arrives at the right answer a minute later has not measured the link, it has been dragged along by it, and a loop acting on it is acting on the link the subscriber had rather than the one it has. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The landed rate-limit test lifts the cap before awaiting the switch, because a switch could not complete under one. That was not a property of the cap: moq-net opened control streams at the transport's default send order of 0, so on a link the media saturates, the request for the replacement track was never transmitted. Instrumented, it arrived at the publisher in 0.7 ms uphill and the answer sat in quinn's send buffer for 56 seconds while group streams on the same connection kept flowing. Fixed upstream in moq `23b4258fb`, which gives control streams `u8::MAX`. This test holds the cap across the whole switch and drives it with an explicit `set_rendition` rather than adaptation, so a failure is about the handover and not the decision. Against the old moq it times out at 60 seconds; against the new one it passes in 7.7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bumps the moq pin to `d2b63c263`, which carries what a review round found across the work that landed tonight: - Control streams no longer sit at the transport's default send order, so a saturated link stops starving announces, subscribe responses and track info. That is what `a_switch_lands_while_the_link_stays_capped` tests. - Seven V4L2 fixes, the largest being a real drain command on the encoder: `flush()` runs at every group boundary and could only wait, so any encoder that pipelines would have failed at the first one. - Five fMP4 fixes, including a stack overflow. The export poll restarted itself by recursing once per buffered sample, so a single long GOP aborted the process; a 10 second GOP at 60 fps is enough. - H.264 conformance decoding, the first tests anywhere in the stack that decode a bitstream this workspace did not encode. Verified after: 8 seconds recorded from a live publisher, 34 video and 136 audio packets, decoding clean through `ffmpeg -f null -`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was a shortfall against a rendition's declared bitrate and a queueing path, conjoined. A declared bitrate is a ceiling handed to the encoder and openh264 spends some 40% of it, so the shortfall held on any clear link and the conjunction reduced to the queueing test alone. That test had its own problem: `min_rtt` was a running minimum over every path the connection had ever selected and it never reset, so a fallback from a direct path to a relay left both halves of it true for good. Together they walked the ladder to its bottom rung and kept it there. Measured through the lab, 27 rendition changes in 40 seconds on a path whose only fault was that it got longer. The shortfall is now measured against a reference the subscriber establishes for itself: the peak goodput this rung delivered while the path was healthy. It decays, so a scene that quietened for good stops being the reference, and it freezes while the path queues, so the capped rate never becomes the normal one. The minimum is taken over about thirty seconds as two tumbling halves and starts again when the selected path changes, which iroh reports through `Path::id`. The queueing half needed one more thing. QUIC takes a round trip sample only from a packet that asks to be acknowledged, a subscriber sends few of those, and the same figure is then handed out across many evaluation ticks, so `downgrade_hold` was satisfied by a single reading and one scheduler hiccup on a millisecond path was a downgrade. `rtt_samples` counts the distinct readings and the rule now wants two of them. The upgrade probe gains the two failure conditions a receiver can observe. Loss and congestion both describe the direction a subscriber barely sends in, so on the asymmetric case a probe is most likely to meet, a downlink bottleneck behind a healthy uplink, neither can fail it and the ladder climbs whatever the link says. A queue measured after the step up fails it now, and so does a higher rung delivering less than the one below it was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The app could watch a broadcast or publish one, and the native side already had `dial`, so what was missing was the half of a call that waits to be called and the screens either side of it. `answer` publishes this node's side of the call and returns at once, with the ticket already readable, so the code goes on screen before any peer exists. A task on the handle then takes the first inbound session that turns out to be a caller and installs its tracks. Sessions this node dialed are skipped, and a session that never publishes the call path is an ordinary subscriber rather than an error, so the wait keeps going for somebody who does. The screen is one surface in two states. While nobody has dialed it draws this node's own camera with the code over it; when a peer arrives the code goes and the same surface carries their picture, because the frame source is swapped under the render loop rather than the loop being restarted. The loop does restart once, to drop the sensor rotation: this node's camera needs it and the peer's pictures do not. The ticket card moves out of the publish screen and into `Qr.kt`, since both screens now put a code up for another device's camera and a code that rendered differently in the two places would be a code that read differently. Two build changes ride along. The Gradle heap goes up: the dex merge holds every class in the app and its dependencies at once, and Compose plus CameraX plus zxing overruns the 512m default. It does not fail cleanly, giving a `DexArchiveMergerException` with no mention of memory, a heap dump left in the project directory, and often a success on the retry, which reads as flakiness rather than a limit. And the release workflow drops the macOS x86_64 build: an Intel Mac is not a target we support, and it was a second cross compile off the same runner on every release for nobody.
The guide described the bridge's entry points and the call path as if dialling were the only half of a call that existed. Adds the mode list, and says why `answer` cannot block the way `dial` does: the code has to be on screen before any peer exists, so it returns once this node's side is published and leaves a task waiting.
The task waiting for somebody to call held an `Arc` to the session handle it was going to write into, and the handle is what owns the endpoint, the camera and the broadcast. So the two kept each other alive: `disconnect` dropped the handle Kotlin held, the task's reference kept the rest open, and the task itself runs until a peer arrives. Backing out of the call screen before anybody called therefore left a camera and an endpoint running for the rest of the process. The task now takes a `Weak` and the handle owns the task, so dropping the handle aborts it. A call that lands in the gap, after the screen is gone but before the abort, is closed rather than dropped, which tells the peer instead of leaving them to time out.
Replacing the colour gradient with the diagnostic pattern moved what the browser tests were looking for, and both of their pixel probes had been written against the old picture. The liveness probe sampled the centre pixel and waited for it to be non-black. The new pattern's middle band is the sweep band, black except when a bar a few pixels wide crosses it once every two seconds, so the probe was really waiting for that bar to pass under one point and timed out when it did not. It samples a grid now, which is what "the canvas has a picture on it" means regardless of what the picture is. The marker probe looked for yellow at the centre. The pattern does have a flashing yellow marker, and nothing else in it is yellow, but the marker is the bottom band. Both tests sample across that band rather than one point in it, because the sweep bar crosses the band too and would occlude a fixed sample every time it passed. Also makes the bitrate metric test deterministic. It slept 33ms four times and asserted a figure derived from how long those sleeps actually took, which a loaded macOS runner does not honour, so it failed there and nowhere else. `record_at` takes the clock from the caller, and the assertion tightens from a 300-to-1200 range to the 606 kbit/s that 2500 bytes every 33ms actually is.
Shipped earlier tonight in "downgrade from a rung that was never seen arriving cleanly", and wrong. The problem it addressed is real: a subscriber whose link was already congested when it joined records no clear-path goodput reading and never can, so it holds the top rung on a lossless bottleneck forever. The fix is not. An encoder sends well under what it advertises by design. The lab measures openh264 at about a quarter of the top rung's declared bitrate, and the threshold this used was half of it, so the shortfall it looked for is true of a perfectly healthy stream. Any path that merely looks queueing, a relay fallback or a Wi-Fi to cellular handoff, then steps the ladder down for no reason. `a_risen_baseline_round_trip_does_not_downgrade` exists for exactly this and caught it, but only under parallel load, because whether a clean reading lands before the round trip rises is a race. Reproduced deterministically afterwards: no measured reference, a stream at a quarter of its advertised rate, a round trip risen from 1ms to 35ms and holding, and the loop returns Downgrade. The behaviour is pinned by a test that says in its own doc comment why the obvious fix is worse, so the next person to notice the gap finds this first. Both directions are covered now: the rung with no reference holds, and loss still moves it, because loss stands alone.
A subscriber cannot draw anything until the next keyframe, so the interval is how long joining takes, how long a rendition switch waits, and how much media a skipped group throws away. It was upstream's default of two seconds, and nothing downstream could reach it: `VideoRendition` had size, bitrate, codec and kind, and `encode_config` never set `gop`. `VideoRendition::with_keyframe_interval` takes a duration and `irl publish --keyframe-interval` takes seconds, defaulting to one. Two seconds is a broadcast figure and this is a tool for pointing a camera at something and having somebody else see it, usually after scanning a code. What halving it costs is picture quality rather than bitrate, because the encoder is given a target rate and keeps to it. Recording the same source at one second and at two, the difference in bytes on the wire was inside the run-to-run variation. The first guess was that it costs bitrate; the measurement said otherwise, so no figure is quoted for it.
…ling The panel tests asserted the module count, the scale, and that the code stays on the panel, but not the white border around it, which is the constraint that actually binds. It is not spare room: it is the quiet zone a decoder needs to find the code, the standard asks for four modules of it, and the current layout has 3.7. A payload one QR version longer would keep the same three pixels per module, grow the code, and eat the border instead, which leaves a panel that still looks right and no longer scans. The second test records why three pixels is a ceiling rather than a compromise. The narrow axis is 122 px and a ticket carrying a 32 byte endpoint id needs 33 modules, so a fourth pixel each would want 132 px and does not fit even with no border at all.
…itself
`irl watch --scan` on a Raspberry Pi opened `/dev/video0`, which is the Unicam
sensor node: it accepts any geometry you ask for and hands back raw Bayer, so
it opened cleanly at 1280x720 and never delivered a frame. The screen was black
with no error, indistinguishable from a lens cap.
`--scan-camera <SPEC>` takes the grammar `--video` already takes, restricted to
sources that hand over pixels: `cam`, `cam:<id>` for one `irl devices` lists on
any platform, or `rpicam`, which here always means the camera's raw pictures
since the H.264 it can produce instead is nothing a QR decoder can read. A
display, a file or a test pattern parse and are refused at the flag. Omitted,
the scanner takes the Pi camera where the build can drive `rpicam-vid` and the
default camera otherwise.
A camera that opens and then sends nothing for five seconds is reported on the
screen with the flag to pass, because the fix is a flag and the reader is
looking at a black rectangle.
Verified on the Pi 4: with no flag the scanner starts rpicam-vid at 1280x720 and
the no-frames watchdog stays quiet, so pictures flow. Decoding itself is the
same code path the round-trip unit tests cover.
This replaces a first cut that added a separate `ScanCameraArg { Auto, Cam,
Rpicam }` enum: Linux-specific on a cross-platform flag, no device selection,
and a second grammar beside the one that already existed.
…gs it The requested arm has a guard that keeps a rebuild alive when the user pins the rendition already playing: a rebuild is an open for that same rendition under a new decoder, and cancelling it would drop the decoder change with nothing left to retry it. The reopen arm that spawns rebuilds built its `Opening` with `rebuild: false`, so the guard written for it never fired. Pick Decoder = vaapi, then pin the current rendition while vaapi was still coming up, and the picker showed `vaapi (openh264)` for good. `Opening::switch` and `Opening::rebuild` are the only ways to build one now, and `survives_repin` is the question the guard asks. The arm that spawns a rebuild cannot tag it as a switch, which is the whole fix; the tests pin the two answers. Closes B7.
`transport::subscribe` waits without limit after its ten-second warning, and that is right for the terminal: it has a keyboard, it printed advice, Ctrl+C works. The window went through the same path, so on the touchscreen `--scan` was built for a code for a publisher that is not running left a spinner that only the Cancel button could end. The window's dial now has a twenty-second deadline and fails like any other failed attempt: back to the scan screen, where the ticket that just failed is held off and a different code connects at once. Twenty covers a hole-punch that falls back to a relay and still ends inside the time somebody will hold a code up. The terminal path is untouched. Closes what was left of B1 after the Cancel button.
…econds Yesterday's change defaulted `--keyframe-interval` to one second on the reasoning that two is a broadcast figure and this is not a broadcast tool. It is a broadcast tool. Upstream's default stands, the flag stays, and the doc on the constant says when to lower it: a call, or a demo where somebody is holding a code up to a camera waiting for a first picture.
`demos/pi-zero-minimal` was thirty-five lines and no flags: the shortest program that puts a Pi camera on the network. That is an example of the library, not a demo of the product, and keeping it as a workspace member cost five `--exclude` clauses in CI for a crate that had nothing to check. It is `iroh-live/examples/publish-pi.rs` now, gated on the `rpicam` feature so every other example builds on a machine without a Pi. Cross-built for aarch64 and run on the Pi 4.
…roduces
Franz could not scan the Pi Zero's e-paper from his laptop. The panel is small,
so a webcam has to get close to resolve the modules, and close is out of focus;
the picture was soft in a way that looked almost readable and was not.
A harness makes that reproducible: a fixed ticket drawn at a given number of
pixels per module in the middle of a scan-sized frame, blurred with a Gaussian
whose sigma is a fraction of a module, and the largest fraction each decoder
still reads through found by bisection. In a release build on that harness:
px/module 3 4 5 6 8 10
rqrr raw 0.2 0.2 0.3 0.2 0.3 0.3
rxing raw 0.4 0.4 0.5 0.5 0.5 0.5
shipped 0.6 0.6 0.6 0.6 0.6 0.6
The shipped pipeline sharpens the plane once with an unsharp mask whose radius
scales with the frame, asks rxing, the zxing port, and falls back to rqrr on the
same sharpened picture, since the two fail on different pictures and rqrr is
cheap. Two to three times the blur, flat across module sizes. The floor test
asserts 0.5, one bisection step under what was measured. One decode of a 720p
frame costs 75ms in release on this machine and is logged at debug with its
duration, so the Pi's figure can be read off rather than guessed.
rxing is pulled in QR-only: the default feature set brings image formats, serde
and a URI parser for pixels already held in a buffer.
…ests Three lints CI's `-D warnings` on the test target would have stopped on. The rxing hints were built with `Default::default()` and then a field assigned, which clippy asks to be a struct update instead. `blur_ceiling` wrapped the bisection for the shipped decoder; the floor test became a single point per module size, so nothing called it. And the harness's decoder function type was spelled out at each use, which clippy calls complex; it is a `Decoder` alias.
The two-machine measurement, publisher here and player on a Pi 4, read 900ms
glass to glass and could not say where. Both ends in one process share a wall
clock, so this stamps every frame as it is handed to the publisher and reads
the stamp back after decode: encode, mux, a real QUIC connection over loopback,
demux, decode and the playout policy, and nothing else.
Two runs, one per policy, so the playout hold is read off as a difference:
unmanaged: join 210ms; median 84ms, p90 94ms over 60 frames
synced (default): join 459ms; median 203ms, p90 204ms
The pipeline proper is under 100ms. The default policy adds its 100ms jitter
buffer plus about 20ms, which is what the latency budget predicted for that
stage. Everything else in the 900ms is outside this: the Pi's hardware decoder,
its compositor, and the screenshot that read the glass. The assertions are
sanity bounds that will not flake; the figures print with --nocapture.
The doc-test census was 4 passing and 1 ignored across the workspace, with nothing at all on iroh-moq's entry points. `Moq::new` now shows the publish side end to end, router and every ALPN included, and `Moq::connect` the subscribe side. moq-media-egui's example was `ignore`d over a link that only breaks under `--no-default-features`, which is a doc build, not a doc test; it compiles as `no_run` under the features the tests use. Seven doc tests pass, none ignored.
…hannel The rust.md sweep. Four `tracing::` path-qualified macro calls in moq-media become imports and bare calls; a `use` inside a function in call.rs moves to the top of the file; and a test in audio_file.rs used `std::sync::mpsc::channel` for a single completion signal, which is unbounded on principle even where it cannot grow. Nothing here changes behaviour.
Decoding a 720p frame costs about 175ms on a Raspberry Pi 4, and the capture loop that ran it is also the loop that feeds the preview. Every look froze the picture for that long, three times a second: a preview that stutters for a third of the time on the device the scan screen was built for. The capture loop hands a luma plane to a worker and carries on reading; the worker decodes, parses, applies the hold-off for a ticket that just failed, and reports. One plane in flight at a time, since a second look before the first has finished would read the same code out of a later frame. The found ticket comes back over a channel the loop selects on beside the camera. Measured on the Pi 4 with a checksummed binary: decode cost unchanged at a median 180ms, and the preview delivering 14.1 frames a second against the 15 the camera runs at, where it used to lose about a third. The rate is logged every hundred frames at debug so the next reading is a number, not an impression.
…its own `spawn_signal_producer` was the longest function in the crate, one 134-line loop carrying six pieces of state between ticks. The arithmetic that turns a path reading into a `NetworkSignals` is a `Sampler` now: the previous counters for the loss delta, the round trip minimum and the path it came from, the distinct-sample count, and the goodput window. The task that drives it is twenty lines. The point is what that makes testable. Four unit tests now pin, with no connection under them, that loss is measured between readings rather than from the connection's totals, that a repeated round trip is one sample, that goodput is nothing until the readings span the window, and that the minimum starts over on a new path. The patchbay suite passes on the refactor end to end.
macOS CI checks `--all-targets --all-features`, which turns `rpicam` on, and the `moq_media::rpicam` module the example imports exists only on Linux. The example compiles to a stub with a message off Linux; `required-features` alone could not express that, since the feature is on and the module still is not there.
…r reads through Measured with a clock drawn into the picture against the screenshot's time, same Pi 4, same publisher, a minute apart: `--decoder v4l2`, which `auto` picks there, showed a frame about a second after it was drawn, and `--decoder openh264` about 360ms, both holding 30fps. The hardware decoder keeps roughly 700ms of pictures in its own queue. It is there to spare the CPU, which a Pi Zero needs and a Pi 4 at 720p does not, so the CLI reference and the Pi guide now say to name openh264 on a Pi 4 when delay matters. The media-stack page stops calling the V4L2 codecs unproven: both halves run on a Pi 4. Also documents what the scanner does with a soft picture, and that its decode no longer holds the preview.
`to_bytes` and `from_bytes` serialised a ticket as postcard alongside the URI form, and nothing in the workspace called either: every ticket that exists is a string, printed, scanned or pasted. Two public functions with no caller are API surface to explain and keep compatible for nobody. `new` takes `impl Into<String>` for the name rather than `impl ToString`, which is the idiom and stops a caller paying for a format of a value that was already a string. Two Android call sites passed a `&String` and now pass a `&str`. BREAKING CHANGE: `LiveTicket::to_bytes` and `LiveTicket::from_bytes` are gone; use `to_string` and `parse`, which are what every caller already used.
…hares A diff of each subcommand's --help against its section of the reference found publish's --fullscreen undocumented, and call's note about which publish flags apply naming only some of them: --keyframe-interval and --test-source are capture flags too.
Frando
marked this pull request as ready for review
September 4, 2026 07:00
The publish example built a ticket from `endpoint().addr()`, which is not convertible to the endpoint id the constructor takes, so it did not compile. The preview banner said hardware decode on Linux was missing, which VAAPI and V4L2 decode have not been for a while, and that Windows was untested, which undersells a CI job that builds and ships an x86-64 binary. And the patch note counted five upstream changes where there are eleven open pull requests to moq and one to moq-vaapi.
The adaptation loop judged a rung by goodput against a clear-path peak and by the round trip against a windowed minimum. Both need a baseline the subscriber cannot always take: a link already at its limit when the subscriber joins never yields a clear-path reading (D1), and a bottleneck that outlasts the window becomes the baseline (D2). moq-net has carried a figure with neither problem all along, the publisher's own estimate of the path in its PROBE message every 100 ms, and we dropped it. Three things, because none stands alone: - Every endpoint runs BBR3 (`iroh_live::util::transport_config`, installed by `Live::from_env` and `irl`). On iroh the estimate is `cwnd / rtt`, and under CUBIC a publisher's window says nothing about the link: in the lab a 100 kbit/s cap read as 1.9 Mbit/s. Under BBR3 the same cap reads 108 to 164 kbit/s, a clear loopback path tens of Mbit/s, and a path with its round trip raised to 50 ms but no cap 1.5 to 1.9 Mbit/s. - `NetworkSignals::delivery_bps` carries it. `spawn_signal_producer` takes the `MoqSession` rather than the connection, since the estimate lives on the session (`recv_bandwidth()`), and logs it beside goodput. - `evaluate` decides on it where it is present: a rung is left when the estimate covers less than half its advertised bitrate, and the next rung is taken outright, as `Decision::Upgrade`, when it covers that one and a half times. No probe is needed to find room the publisher already reports. The goodput and round trip rules and the probe remain as the fallback for a publisher on a MoQ version without PROBE, or a rendition that advertises no bitrate. Tested in both directions this time, in the unit tests and in the patchbay lab: the rate-limit test now downgrades on the estimate and returns with an upgrade rather than a probe, the risen-baseline test holds, and the loss tests exercise BBR3's loss path. What would tighten the figure further is upstream in order: iroh exposing the controller's pacing rate (1.1.0 keeps the noq connection private and `PathStats` carries none), then `web-transport-iroh` preferring it as `web-transport-quinn` does since moq-dev/web-transport#385. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… the estimate Tested on a Pi 4 over Wi-Fi, the branch dropped the player to its lowest rung three times in 45 s on loss readings of 0.2, 0.33 and 0.5, each followed by an upgrade ten seconds later. The readings were one lost acknowledgement among the three to five packets a subscriber sends in a 200 ms tick: a fraction with a denominator of three, which the emergency rule read as a fifth of everything lost. `v2` computed the same fraction. Loss is now measured across a two-second window and reads zero while the window holds fewer than twenty packets, which is what keeps one loss under the downgrade threshold. The same run then held `high` for 45 s with loss between 2 and 5 percent, which showed the second fault: `loss_good` (0.02) against acknowledgement loss would have reset the upgrade hold on half the ticks. With an estimate present the step up is gated on the estimate's room, which already carries the loss on the direction the media flows in; loss at `loss_downgrade` still blocks it, and `loss_good` is the fallback's gate alone. Two more things the run wanted. The video task logs the frame rate it decoded once every five seconds at debug, which is what told a stalled picture (a debug-build publisher encoding 11 fps at 720p, starving the latency skip after every switch) from a fine one against the same goodput figure. And the signal trace names the path it read. Against a release publisher the Pi 4 decoded 30 fps for two minutes: one loss-driven downgrade with the picture at 28.7 fps through it, an upgrade eight seconds later, back to 30 fps within ten, no errors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
align to moq.
see commits and moq-dev/moq#2481, moq-dev/moq#3267
Claude description:
What this branch is
iroh-live becomes an application over moq's media stack rather than a second
media stack beside it. On
main, codecs and capture are two in-house crates(
rusty-codecs,rusty-capture, about 40,000 lines) behind apipelinemodule and a set of decoder traits. At
HEADthat code is gone:moq-videoand
moq-audiodo the work, and everything they lacked for our platforms ismaintained upstream, submitted as the pull requests listed at the end. Against
mainthe diff is 299 files, +28.5k / -64k lines: a net deletion of about36,000 lines, most of it code we maintained becoming code maintained upstream.
Three things had to keep working and do, verified on hardware:
irl publishand
irl watchon Linux, macOS and the Pi; the Android app; the Raspberry Pidemos, on a Pi Zero 2 W and a Pi 4.
What is different from
mainCodec and capture.
moq_videoandmoq_audioreplace the in-house crates.moq-mediakeeps only what upstream has no reason to carry: the catalogextension, the publish and subscribe sides, the playout clock, and three
sources of our own (rpicam, audio files, generated test patterns). Decoder
selection is a runtime
Kindrather than a type parameter on the subscribe API.Transport. One
origin::Producerper node instead of one per session:Moq::publishis synchronous and every session serves whatever the originholds. A session carries its own driver, so an unpolled session cannot exist.
Two peers dialing each other at once both keep a working session.
Rooms move out of
iroh-liveintoiroh-rooms, so consumers of thelibrary do not compile gossip unless they want it.
Playback resilience. A refused access unit is a bad picture, not the end of
the track; a run of failures or a window in which most access units fail is.
Rendition switches open the replacement decoder beside the incumbent and swap
on its first frame. A decoder rebuild survives the user pinning the rendition
already playing.
The Pi as a viewer.
irl watchdraws through wgpu on a Pi 4, requesting theadapter's own limits, and wakes the window when a frame arrives rather than
polling.
Adaptation decides on the publisher's estimate. Every endpoint runs BBR3,
so the send-rate estimate moq's publisher already sends every 100 ms tracks
the link rather than a loss-based window;
NetworkSignals::delivery_bpscarries it, and a subscriber leaves a rendition when the estimate covers less
than half its advertised bitrate and steps up, with no probe, when it covers
the next rung one and a half times.
main's goodput-against-a-clear-path andround-trip-against-a-minimum rules remain as the fallback for a publisher
that sends no estimate. Loss is measured over two seconds with a
twenty-packet minimum rather than per tick, where one lost acknowledgement
among three read as a third of everything lost.
Choices that are flags now.
--encoderand--decoder(with a live comboin the window),
--latency realtime|balanced|smoothfor the playout hold,--fpsand per-rungname:WxH@fps,--keyframe-interval(default thebroadcast two seconds),
--video rpicamandrpicam:rawfor the Pi camera,--scan-camerafor the QR scanner.irl deviceslists cameras with theirmodes and rates, and the Pi camera through
rpicam-vid.Tickets and scanning. A ticket is 58 characters instead of 184: the
endpoint id and the broadcast name, with addresses left to discovery
(
iroh-mdns-peer-lookupis always on). On the Pi Zero's e-paper that takesthe QR from 1 to 3 px per module.
irl watch --scanandirl callread aticket off a camera. The scanner sharpens the frame and tries two decoders,
reading through 0.6 of a module of blur, and decodes off the capture thread
so the preview keeps moving; on a Pi it picks the Pi camera by itself.
Measurement.
irl publish --test-sourcedraws a sweep bar, a ruler, stripepitches, a frame counter, a millisecond clock and an audio-synced marker. The
frame-rate figure is a count over a window rather than a smoothed reciprocal.
The adaptation loop logs its inputs and decision every tick, and the video
task logs the frame rate it decoded every five seconds. An in-process test
measures pipeline latency with both ends on one clock: 84 ms with no playout
hold, 203 ms with the default policy. On a Pi 4,
--decoder v4l2(whatautopicks there) shows a frame about 1.05 s after it was drawn and
--decoder openh264about 360 ms, both at 30 fps: the hardware decoder holds roughly690 ms in its own queue, so on a Pi 4 name
openh264when delay matters.Android. Watch, publish, and call (scan a code to dial, or show one and
wait). Hardware H.264 through MediaCodec, zero-copy EGL rendering.
CI. A rolling release with binaries for Linux x86-64 and aarch64, macOS on
Apple silicon, Windows x86-64, and an arm64 Android APK; the Linux builds carry
nvidia,vaapi,v4l2,rpicam,pipewire,sound-serverandplayback. The netns network-simulation suite, the browser suite, 230 Rusttests and 7 doc tests run on every push.
What
mainhas thatHEADdoes notRaw libcamera capture (
rpicam-vidis the only Pi camera source); AV1 encodeand decode; our own DMA-BUF and Metal zero-copy import into wgpu, along with
the VAAPI VPP re-tile (moq-video keeps decoded frames on the GPU its own way,
which
irl watchuses; no target hardware needs the re-tile); themoq-media-dioxuscrate; theheadlessandopengldemos; thepi-zero-minimaldemo, which is
iroh-live/examples/publish-pi.rsnow. 422 Rust tests onmainare 230 here; most of the difference left with the crates whose tests they
were, and that coverage exists upstream against the code it tests.
plans/v2/summary-new.mdhas the full accounting.Known limitations
acknowledgements, measured over two seconds. On Wi-Fi it trips the
downgrade threshold about once every two minutes with nothing wrong with
the picture, and recovers in about eight seconds. A loss figure for the
direction the media flows in would replace it.
cwnd / rtt, which under BBR3tracks the link (a 100 kbit/s cap reads as 108 to 164) but over-reads by
BBR's gain. iroh exposes no controller metrics, so mirroring
fix(web-transport-quinn): prefer the controller's pacing rate for estimated_send_rate moq-dev/web-transport#385 for the iroh backend is an iroh PR first.
--decoder autoprefers the hardware decoder on a Pi 4, where it costs about700 ms of latency. Changing what
autoprefers wants a Pi Zero measurementfirst, since that is the device the hardware path is for.
Frando/moq@iroh-livethrough a[patch]blockuntil the pull requests below land. Deleting the block is the whole revert.
Upstream pull requests
Merged:
Open:
publish_capturecarry a catalog extension (breaking)devwants