Skip to content

fix(mux): a cut-short mp4 finish repairs, and only a verified index counts as finished - #9124

Merged
kixelated merged 3 commits into
mainfrom
quest/m1/precut-fixes/mp4-finish-repair
Nov 19, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m1/precut-fixes/mp4-finish-repair

Conversation

@kixelated

@kixelated kixelated commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The pre-cut audit found two holes in moq repair mp4 and the moq export mp4 finish:

  • A finish cut short partway through its 16-byte mdat header (for example, only the first 8 bytes written, 00000001 'mdat' plus zeros) left all the media and the final moov on disk, but repair read the reserved slot as a box, could not parse it, and refused the file.
  • A finished file with its trailing moov removed or cut counted as "already finished", because any mdat in the reserved slot did. ffprobe could not open that file.

The finish's durable write sequence also lived in the CLI (write_finish), so no other file-backed caller could finish a recording safely.

Approach

  • repair reads the reserved slot as bytes, always parses the fragmented moov right after it (a finish never moves it), and rebuilds the index from the fragments. It reports a file as finished only when the slot and the trailing moov are exactly what finishing writes. Otherwise it returns the finish, so a torn header, a missing, cut, extended, or altered moov, and a never-finished recording all repair to the same result. A slot whose bytes are not each the free box's or the mdat header's is refused as not a recording.
  • New mp4::Finish::write(&self, &mut std::fs::File) makes the writes durably at the format layer, and the CLI's export and repair both use it:
    0. Repair only: if a cut-short finish left the slot's first 8 bytes a header (which skips the fragmented moov), write them back to free, then sync. Otherwise cutting the trailing moov would leave a crash with no moov a player reaches.
    1. Cut the file after the media, then sync. Otherwise a moov landing over unsynced cut bytes can mix into a box kind repair refuses.
    2. Append the moov.
    3. Write the header's last 8 bytes (its 64-bit size) into the free box body, then sync.
    4. Write its first 8 bytes (size marker and mdat type), then sync.
      Until that last write, the slot reads as a box a player skips (free, a 64-bit free, or a kind it does not know), never a broken mdat header. So every crash point leaves a file that plays as-is, in a finish and in the repair of any crashed file. This takes three syncs instead of two, and a fourth when a repair reopens the slot.
  • moq export mp4 flushes its tokio file before converting it, since into_std drops the error of a write still in flight.
  • Finish's fields are private now, so nothing can make the writes in an unsafe order.

Validation

  • Crash model (unit test). a_finish_interrupted_anywhere_plays_and_repairs records the calls Finish::write makes through a private Disk trait. It then builds every file a crash could leave: what was synced, plus each call since lost or made, with writes torn at every byte of their leading box header and trailing bytes. It does this for three starting files: a clean recording, one cut mid-fragment, and one cut mid-description, where the leftover tail is longer than the moov. Every crashed file must:
    • still play, meaning the first moov a box-walking reader reaches is the fragmented one or the final one;
    • repair to the same Finish;
    • be byte-identical to the clean finish once repaired.
    • when its slot is no longer free, also survive a crash at any point of its own repair: every file that leaves must play and repair the same.
  • The model discriminates. Writing the mdat type before its size fails it ("crash 1106 left a file no player reads"). Dropping the sync after the cut fails it too: a moov-prefixed tail reads as a free box holding neither a fragment nor a moov. Skipping the slot reopen fails it ("crash 347, 2 left a file no player reads"). Dropping the sync between the moov and the header size passes, correctly, since the size lands in the free box body, so that sync was removed.
  • Verified index (unit test). only_a_verified_index_counts_as_finished covers a header torn first-to-last (the old order) and last-to-first at every length. It also covers a moov removed, cut at 1 byte, 8 bytes, and all but 1, extended with junk, or with a changed chunk offset. All repair to the same finish. Garbage in the slot and a plain fMP4 are refused.
  • Real surface. A local moq-relay, moq import ts of bbb5s_cbr.ts (HEVC and Opus) at about real time, and moq export mp4 --output produced the clean file (ffmpeg 9 ffprobe: 120 video and 231 audio frames). Six damaged copies were run through the built moq repair mp4: header torn after 8 bytes, after 4 bytes, and with only its last 8 written; moov stripped; moov cut in half; moov stripped plus a torn header. Before repair, ffprobe failed on all but the last-8 state ("moov atom not found" / "error reading header"). After repair, every copy was byte-identical to the clean file with 120/231 frames, and a second repair logged "recording already finished".
  • Killed recorder. A recorder killed with SIGKILL mid-recording repaired to 73 video and 170 audio frames. Its finished header was then torn, and it repaired byte-identical again; a second repair logged "already finished".
  • Middle states play. With ffprobe, each slot state the new write order passes through (00000010 free, 00000001 free, 00000001 mree, mdee, mdae, each with the size already written) decodes all 120/231 frames without repair.
  • just ci passed on the head (4892 tests). just rs cli-export passed (16 real-process export tests, including a_killed_mp4_recording_is_repaired and an_interrupted_mp4_recording_is_finished).

Impact

  • Public API: moq_mux::container::mp4::Finish gains write(&self, &mut std::fs::File) -> std::io::Result<()>, and its fields moov_offset, moov, mdat_offset, mdat become private. The type is not on release, so no published API changes and the title takes no !. A main caller that made the writes itself calls finish.write(&mut file) instead.
  • repair now returns Some for a regular MP4 that only looks finished, and refuses a file whose reserved slot is neither free nor its mdat header. Before, it returned None for any mdat there.
  • CLI: moq repair mp4 finishes torn or index-less files it used to refuse or skip. moq export mp4 makes one more sync when it finishes. No flag changes. Wire: none.

Decisions

  • "Finished" means the trailing moov is byte-identical to the rebuilt one. If a later build changes how Index::finish encodes the moov, repairing a file finished by an older build rewrites its moov (and logs "repaired") instead of leaving it alone. That rewrite is harmless: same media, valid index. The rejected alternative was a structural check (the moov decodes and its chunks lie inside the mdat), which would accept a corrupted index that still decodes.
  • Header size before type. Writing the 16-byte header in two synced halves costs one more fsync, but no crash point leaves a header a player cannot parse. The rejected alternative relied on a 16-byte write within one sector being atomic in practice.
  • The crash model's torn writes are prefixes, bounded to box headers and trailing bytes. A reader sees any cut in between the same way (a box whose size runs past EOF). The bound keeps the test at 0.5s instead of 2s, and nested crashes run only for files whose slot is not free, since only those repairs write anything a finish does not (all nested crashes took 28s).
  • Finish::write takes &std::fs::File, not a generic writer. It needs set_len and sync, and every file-backed caller has a std::fs::File (the tokio export converts with into_std).

Alternatives

  • Keeping Finish's fields public for non-file writers: no such consumer exists, and public fields let a caller reproduce the unsafe write order this PR fixes.
  • Re-scanning after the write to confirm the result: the repair's own exact-match check already verifies any later run, and write returns only after its final sync.

Caused by

  • moq-dev/moq#7560: repair parsed the reserved slot as a box, so it refused a torn mdat header and counted any mdat there as finished without checking the trailing moov (found by check: the November 17-18 pre-cut audit)

Follow-ups

None.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, high effort (arm A)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with high)

Review

Rewritten by quest review after each round.

  • Head: 7462f56

  • Base: origin/main at 1599097

  • Risk: high, The change rewrites and truncates users’ recordings in place, so a regression could damage media that reverting the code cannot restore.

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1f0f5-76ba-7263-a645-77cbe8c2840a, round 2

  • Score: 10/10, Both earlier findings are fixed, and code review of the incremental changes found no new issues; builds and tests were not run.

  • Raised the risk to high: The change rewrites and truncates users’ recordings in place, so a regression could damage media that reverting the code cannot restore.

  • Rounds: 2; first round 2 (medium 2); findings 2 (medium 2): fixed 2, refuted 0, open 0

  • Findings: none

  • The last round's findings, at 5775b28:

    • Fixed: medium, rs/moq-mux/src/container/mp4.rs:94: Expose the fragmented index before truncating during repair; repair sets Finish::reopen for a modified reserved header, and Finish::write_to restores and syncs its first eight bytes before truncating; the expanded test covers interrupted repairs of torn headers.
    • Fixed: medium, rs/moq-cli/src/subscribe.rs:428: Flush pending media writes before converting the file; run_mp4 now calls file.flush().await? before into_std(), propagating any pending media-write failure before finalization.

kixelated and others added 2 commits November 18, 2026 14:18
…ounts as finished

repair read the reserved slot as a box, so a torn mdat header (8 of its 16
bytes) made it refuse a file whose media and moov were intact, and any mdat
there counted as finished even with the trailing moov missing. It now reads
the slot as bytes, rebuilds the index from the fragments, and reports a file
finished only when the header and trailing moov are exactly what finishing
writes.

Finish::write durably makes the writes at the format layer, replacing the
CLI's write_finish: cut and sync, append the moov, write the header's size
before its type, then sync, so every crash point still plays and repairs to
the same finish. Finish's fields are private now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Finalize and repair interrupted MP4 recordings fix(mux): a cut-short mp4 finish repairs, and only a verified index counts as finished Nov 18, 2026
…iling moov

A finish cut short after the header's size marker leaves the reserved box a
64-bit box that skips the fragmented moov, so cutting the trailing moov first
left a repair interrupted there with no moov a player reaches. Finish now
makes the box free again, synced, before the cut. The crash model also
crashes each such repair.

The export flushes before converting its tokio file, which otherwise drops the
error of a write still in flight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 18, 2026 23:51
@coderabbitai

coderabbitai Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d71af6a9-f788-4ea6-854f-7100d05234a8
Commits

Reviewing files that changed from the base of the PR and between a7e327b and 7462f56.

Files selected for processing: 5

  • doc/bin/cli.md
  • quest/m1/precut-fixes/mp4-finish-repair.md
  • rs/moq-cli/src/repair.rs
  • rs/moq-cli/src/subscribe.rs
  • rs/moq-mux/src/container/mp4.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound. The fix moves the durable write order into the format layer (Finish::write), so the CLI export and repair now share it. repair reads the reserved slot as raw bytes and accepts the file as finished only when it matches the rebuilt finish byte for byte. A torn header and a missing or altered trailing moov therefore both go through the same repair.

Reviewed changes. I read the whole diff, traced the write order against each torn-slot state, and ran cargo test -p moq-mux --lib container::mp4 locally. All 13 tests passed, including both new tests.

  • Finish::write / write_to: when needed it reopens the slot to free and syncs. It then cuts the file and syncs, appends the moov, writes the size into the free body and syncs, then writes 00000001 mdat and syncs. Before that last write, the slot never reads as a 64-bit mdat whose size has not reached the disk.
  • repair slot check: a byte that matches neither FREE nor the rebuilt mdat header refuses the file. None now requires the trailing bytes to equal the rebuilt moov exactly and the slot to equal the mdat header. reopen is set whenever slot[..8] is no longer free's.
  • Finish fields made private: I checked that mp4.rs is absent from release, so no published API breaks.
  • CLI: run_mp4 flushes the tokio file before into_std, so an error from a write still in flight is returned rather than dropped. subscribe::write_finish is removed, and repair.rs calls Finish::write inside spawn_blocking.
  • Crash model tests: crashes builds every crash state: what was synced, plus each later call either lost or made, with writes torn inside their leading box header or trailing bytes. Every state must play, repair to the same Finish, and come out byte-identical. Repairs of files whose slot is no longer free are crashed again.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge pick (maintainer away, granted the coordinator merges it confidently recommends): ✅ /quest-merge, accepting the decisions listed in the PR body.

(Written by Claude Opus 5.5 with high)

@kixelated
kixelated added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit a40d786 Nov 19, 2026
9 checks passed
@kixelated
kixelated deleted the quest/m1/precut-fixes/mp4-finish-repair branch November 19, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant