Repository navigation
Conversation
Quest-Claim: 2fab16df5cb8d985
…e every frame of a higher group A keyframe at or before the newest frame written continues the open group (a TS content join), and a later frame presenting at or before an earlier group's newest frame (a leading picture after a splice) is dropped and counted as skipped. The Rust producer, the fMP4 passthrough, and the JS legacy producer keep it; the hang draft states it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… what decodes from a dropped keyframe The fMP4 passthrough drops only the samples of a fragment that present with an earlier group, rewriting it as the edit list does, instead of the whole fragment. A keyframe dropped with no group open takes the frames decoding from it along until a keyframe opens a group, in Rust and JS, rather than refusing them with MissingKeyframe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itten Passthrough::plan advances the track's planned group state (open, start, key, newest, ceiling) per traf, so a moof with several runs of one track trims each against the group it actually enters. start() only executes the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-overlap # Conflicts: # doc/bin/hls.md
…ns a group Two audio runs of one track in a segment's first moof opened two groups, since the segment state advanced only while writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…against the group's ceiling The plan trimmed a fragment against the newest sample before deciding, from its key, that it continued the open group, so after a seek a leading picture the group could hold was dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 9 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
ℹ️ No critical issues. Two places where the fMP4 passthrough drifts from the unwrapped path; comments inline.
Approach: Sound. Keeping a ceiling at each producer lets an overlapping keyframe continue its group and drops frames that would present with an earlier group, which fixes the cause at the producer rather than shifting timestamps. The passthrough now decides where each fragment lands before writing it, which keeps the two fMP4 paths aligned.
Reviewed changes
This review covers all 7 commits at 4ea1ee0: the Rust and JS producers, the fMP4 passthrough planner, the TS and fMP4 tests, and the draft, doc and quest edits.
container::Producer::prepare: a keyframe at or beforenewestcontinues the open group. With no group open, it is dropped andskippingbegins. A delta at or beforeceilingis dropped and counted as skipped. The oldoverlapscut is gone, which is safe because an opening keyframe is always pastprevious_timestamp.- Streamed
Payload::Dropped:open/commitdiscard a dropped frame's bytes without taking the group, and still check its size. Passthrough::plan: runs atbegin_mdatand lands each fragment asOpensorContinues. It trims samples at or before the ceiling throughkept(), and mirrorsstart/key/newest/ceilingahead of the writes.- JS
Producer.encode: mirrors the Rust rules with#ceilingand#skipping. - hang draft: the group rule becomes the strong rule. The removed MUSTs follow from it, and the splice guidance and the hang-04 changelog are updated to match.
- Readers/HLS/quests: the overlap tests and notes are deleted, and no reference to the deleted quest remains.
claude-opus-5-5 | 𝕏
| // A group may not start below the one before it: a restart, refused before anything of it | ||
| // publishes, as an unwrapped track does. | ||
| if opens | ||
| && let Some(floor) = self.start | ||
| && earliest < floor | ||
| { | ||
| return Err(crate::container::TimestampRewind { | ||
| timestamp: earliest, | ||
| floor, | ||
| } | ||
| .into()); | ||
| } |
There was a problem hiding this comment.
This restart check compares the fragment's earliest sample with self.start before plan decides whether the fragment continues the open group or is dropped. So a splice fragment that would end up continuing (sync sample at or before newest) fails the import with TimestampRewind if one of its leading pictures sits below the open group's start. The unwrapped container::Producer accepts the same input: it checks the keyframe against start and the deltas against floor/ceiling.
Technical details
# Passthrough restart check runs before the continuation decision
## Affected sites
- rs/moq-mux/src/container/fmp4/import.rs:1652-1663: `if opens && let Some(floor) = self.start && earliest < floor` runs while `opens` is still the tentative `!self.open || (new_segment && keyframe)`.
## Example
G1 starts at 100. G2 opens at 200 with ceiling 180, and newest is 250. A new-segment fragment has a sync sample at 240 and then a leading picture at 190.
- Unwrapped: 240 >= start 200, so it continues. 190 >= floor 100 and > ceiling 180, so it is kept.
- Passthrough: `earliest` = 190 < `start` 200, so it is refused and the import ends.
## Required outcome
- Apply the restart refusal only to a fragment that really opens a group: after the `at <= newest` and key-based continuation decisions. A continuing fragment's below-ceiling samples are already trimmed.
- A test like `video_groups_never_overlap`, with a continuing fragment whose leading picture sits below the open group's start, should give the same frames on both paths.| if let Land::Opens { .. } = land { | ||
| track.segment = Some(self.segment); | ||
| } |
There was a problem hiding this comment.
track.segment is set here only for Land::Opens. When a sync fragment at a new segment continues the open group (a splice), the segment never counts as rolled. In a stream without styp, video_rolled() then stays false, so the next keyframe moof doesn't move to a new segment, and audio keeps its group across that video keyframe. The unwrapped path marks the segment as soon as it sees a sync sample (open_sample returns opened = keyframe), so after a splice the two paths group audio differently. CMAF input with a styp per segment isn't affected, since pending_cut forces the boundary.
Technical details
# Passthrough continuation at a segment boundary doesn't mark the segment as rolled
## Affected sites
- rs/moq-mux/src/container/fmp4/import.rs:1184-1186: segment marked only on `Land::Opens`
- rs/moq-mux/src/container/fmp4/import.rs:1112-1117 (`video_rolled`) and 1088 (`starts_segment` returns false until video rolls)
- rs/moq-mux/src/container/fmp4/import.rs:1850 (unwrapped `open_sample` reports `opened = keyframe` even when the producer continues or drops)
## Required outcome
- When a sync-led fragment at a new segment continues the group, passthrough marks the segment the same way unwrapped does, so the next video keyframe starts a segment and audio rolls with it.
- A test with both an audio and a video track, no `styp`, and a continuing keyframe should give the same audio groups on both paths.
Finishes quest/m2/groups-never-overlap.md.
Problem
Groups could go back in time. Since moq-dev/moq#4543,
container::Produceropened a group for any keyframe above the open group's start, even one below its newest frame, so a TS content join published an IDR below the old content's last frames, and a later frame could present anywhere above the previous group's start. A reader then can't bound a group that isn't whole by a higher group's lowest timestamp, and HLS edges could list one segment differently (moq-dev/moq#8874). lite-07 publishes the strong rule as a publisher MUST, so first-party producers have to keep it first.Approach
Every frame of a group presents strictly after every frame of every lower group; frames within a group still reorder.
container::Producer(rs/moq-mux/src/container/producer.rs) tracks aceiling, the newest frame of the groups before the open one.skipped, as a leading picture after a splice would be. So is a keyframe at or before the newest frame with no group open, such as after the TS importer cuts on lost bytes, and with it the frames after it until a keyframe opens a group (skipping).TimestampRewind(a restart).open/commit(the streamed path fMP4 and MKV use) carry a dropped frame asPayload::Dropped, which discards the bytes.fmp4/import.rs), which writes whole CMAF fragments without the producer, now decides where each fragment lands while the moof is planned (Passthrough::plan), andstart()only executes that decision.kept(), as edit-list drops are, so a fragment mixing a leading picture with valid pictures keeps the valid ones.js/hanglegacyProducermirrors the Rust producer.rs/moq-hls/src/export/grid.rs,doc/bin/hls.md) is deleted, since a group whose frame passes a line now always contains it.Sources surveyed: TS, fMP4 (unwrapped and passthrough), FLV/RTMP, MKV, SRT, HLS import, the codec importers, gst, moq-ffi/libmoq media, and WebRTC all go through
container::Producer, except the fMP4 passthrough, which this PR handles.moq-transcodeandmoq-archivecopy their source's groups, so they keep whatever their source keeps. The JS publishers never reorder (their encoders refuse B-frames), and JS has no importer.Choices made without asking, for review:
Impact
moq-muxcontainer::Producer::write, the fMP4 importer,@moq/hanglegacyProducer.encode): an overlapping keyframe continues its group instead of opening one, and frames that would present with an earlier group are dropped and counted as skipped. A keyframe overlapping a closed group is dropped, not refused (TimestampRewind). Today's MIGRATE line ("refuses a group that does not start strictly after the previous group's start") stays true. The release cut can add the continuation and the drop.Validation
-bf 0clip offset so its IDR (pts 486000) lands one frame below the old content's last P-frame (489000), concatenated on one clock.moq import tsinto a localmoq-relayexits 0, both with the continuity counters fixed (tsp -P continuity --fix) and without.container::Consumer, the fixed file publishes all 240 frames in 7 groups. The join group holds 60 frames (two GOPs) and every group's frames all follow the previous group's. Without the fix, the cut on lost bytes drops the new content's first GOP: 209 frames, no overlap.TimestampRewind, which is why that case drops.groups_never_overlap_on_random_streams, 256 random streams with reordering, splices, and cuts) checks that no group overlaps a lower one, and that a frame is dropped only when it presents at or before a frame already published. Disabling the ceiling drop makes it fail.video_groups_never_overlap): unwrapped and passthrough both keep the rule, including a mixed fragment, a resume afterrejoin()at the newest sample, and one moof of two runs of the track.audio_runs_in_one_moof_share_a_group): a segment's first moof of two audio runs of one track opens one group, unwrapped and passthrough. It failed on the passthrough (four groups instead of two) before the plan loop advanced the segment.video_continuing_a_seeked_group_keeps_its_leading_pictures): after a seek, a sync sample whose key doesn't pass the seek's start continues the group and keeps its leading picture, unwrapped and passthrough. The passthrough dropped it before its plan decided the continuation ahead of trimming (found by the review's shadow in round 5).main(conflict indoc/bin/hls.md: main's overrun text kept, the overlap sentences dropped),cargo test -p moq-mux(2139 passed) andjust check(4949 tests) pass.just rs testpasses except moq-tokio's QUIC socket tests, which timed out under the full run's load and pass alone (599/599).bun testpasses injs/hang/src/container, andjust checkpasses.Alternatives
Caused by
Follow-ups
None added. Tracked in moq-dev/moq#9263 (quest/m2/time-addressing/group-times.md), whose plan this PR updates:
container::Producerstill finishes a group whole at the next keyframe's timestamp, above that keyframe's open-GOP leading pictures. That's valid under today's End rule (no greater than the next start). Under group times' spans, measured from the lowest frame, the two spans overlap. Which End to write belongs to that quest.quest/m3/fmp4-group-start-tfdt.mdloses its exact-repeat case, which a continuing keyframe now covers. Its near-repeat case remains.Decisions
track.segmentadvanced only while writing), ✅ continue: the passthrough's plan loop advancestrack.segmentwhenplanreturnsLand::Opens, so its plan owns all of its grouping state (rejected: rethink, extracting one grouping state machine shared bycontainer::Producer::prepareandPassthrough::plan, a larger refactor of duplication that predates this PR and that the rule doesn't need). A later review can still pick the rethink.Models
Size: L
Implementer: Claude, claude-opus-5-5, high effort
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, L)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.Head: 4ea1ee0
Base: origin/main at cb9cb51
Risk: high,
Tightening the hang wire contract and changing Rust and JS producers to merge groups and drop overlapping frames can break published consumers and deployed peers relying on prior grouping behavior; reverting restores future behavior but cannot recover discarded media.Classified: high by Codex, gpt-6.1-sol, low effort, session
01a1fc77-37de-7030-9d62-81a8eee5d05dReviewer: Codex, gpt-6.1-sol, xhigh effort
Session:
01a1fc0a-e616-7291-8ca5-449b9fee5f79, round 5Score: 10/10,
The update correctly decides group placement before trimming, preserves valid leading pictures, and introduces no new findings.Rounds: 5; first round 2 (medium 2); findings 4 (medium 4): fixed 4, refuted 0, open 0
Findings: none
Reviewer: Codex, gpt-6.1-sol, high effort, shadow
Session:
01a1fc65-727f-7b03-a523-38d199028738, round 2Score: 10/10,
The incremental change fixes the trimming regression and adds focused coverage, with no further issues found by code inspection; builds and tests were not run.Rounds: 2; first round 1 (medium 1); findings 1 (medium 1): fixed 1, refuted 0, open 0
Findings: none
The last round's findings, at dfae86b:
rs/moq-mux/src/container/fmp4/import.rs:1673:Resolve group placement before selecting the trim ceiling;rs/moq-mux/src/container/fmp4/import.rs:1678 now resolves key-based continuation before selecting the trim ceiling, preserving leading pictures in the seeked group; the added regression test covers this scenario.