Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ moves what remains of them once their claims merge.
- [Relay settings parse once into a validated configuration](/quest/m1/relay-config-checks.md) - settings that cannot work together are refused at load instead of accepted and ignored
- [Track drop runs are forgotten below the floor](/quest/m1/drop-runs-prune.md) - a track's dropped-group record stays proportional to its window in Rust and JS
- [A macOS camera delivers frames the renderer imports](/quest/m1/camera-nv12-macos.md) - AVFoundation capture asks for NV12, so a camera whose native format is `2vuy` reaches the renderer
- [Encoders keyframe by time, not frame count](/quest/m1/keyframe-by-time.md) - groups stay within the declared join interval when a source's frame rate drifts from the configured one

## Related

Expand Down
31 changes: 0 additions & 31 deletions quest/m1/keyframe-by-time.md

This file was deleted.

6 changes: 4 additions & 2 deletions rs/moq-video/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,14 +115,16 @@ later one goes in, and for the tail `finish()` drains. Bring your own pixels wit

Group boundaries are automatic: `Config::join` is the time between join points
(a keyframe every two seconds by default, rounded to whole frames, and a join of
zero is refused at open), so an application never has to think about them.
zero is refused at open), so an application never has to think about them. It
is measured by the frames' timestamps, so a source running slower than
`Config::framerate` still keys every join.
`Encoder::cut()` opens a group at the next frame when something outside the
encoder needs a decodable starting point there: a source group boundary, a scene
change, a source switch. The request is held until a frame
arrives, so it is safe to call before you have one. It fails with
`Error::CutUnsupported` on a backend that cannot force a boundary (a V4L2
driver without the force-keyframe control), and queues nothing then: groups
keep falling where `Config::join` puts them. `encode::Sink` answers the same
keep falling every `Config::join`'s worth of frames at `Config::framerate`. `encode::Sink` answers the same
way, awaited.

Public entry points:
Expand Down
11 changes: 7 additions & 4 deletions rs/moq-video/src/encode/backend/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,13 @@ mod vaapi;
/// whether forced by `cut` or placed by the backend's own GOP cadence.
pub(crate) trait Backend {
/// Encode one frame, opening a group at it (an IDR) when `cut` is set.
/// Backends place group boundaries on their own per [`Config::join`], so this
/// is only the caller's extra request, arriving via
/// [`Encoder::cut`](super::Encoder::cut), and only on a backend whose
/// [`can_cut`](Self::can_cut) said yes.
/// Backends place group boundaries on their own every
/// [`Config::keyframe_interval`] frames, so this is only the encoder's extra
/// request: a caller's [`Encoder::cut`](super::Encoder::cut), or a
/// [`Config::join`] that a source slower than its frame rate reached first. It
/// arrives only on a backend whose [`can_cut`](Self::can_cut) said yes. A
/// backend that drops the frame carries the cut to the next one it encodes,
/// or the group waits a whole join for its keyframe.
fn encode(&mut self, frame: &Frame, cut: bool) -> Result<Vec<Encoded>, Error>;

/// Return every access unit the codec is still holding, leaving the encoder
Expand Down
7 changes: 7 additions & 0 deletions rs/moq-video/src/encode/backend/videotoolbox.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ pub(crate) struct VideoToolbox {
framerate_numerator: i32,
framerate_denominator: i64,
frame_index: i64,
/// A cut whose frame VideoToolbox dropped, carried to the next frame so the
/// group it asked for still opens.
keyframe_pending: bool,
applied: Applied,
}

Expand Down Expand Up @@ -206,6 +209,7 @@ impl VideoToolbox {
framerate_numerator,
framerate_denominator: i64::from(config.framerate.denominator()),
frame_index: 0,
keyframe_pending: false,
applied,
})
}
Expand Down Expand Up @@ -234,6 +238,7 @@ impl Backend for VideoToolbox {
};
self.frame_index += 1;

let cut = cut || self.keyframe_pending;
let frame_properties = cut.then_some(&*self.force_keyframe);

let status = unsafe {
Expand Down Expand Up @@ -265,6 +270,8 @@ impl Backend for VideoToolbox {
"VideoToolbox encode callback failed: {status}"
)));
}
// A dropped frame comes back with no packet and no error.
self.keyframe_pending = cut && self.sink.packets.is_empty();
// `complete_frames` above forces this frame out before returning, so every
// packet collected here came from it and carries its timestamp.
Ok(std::mem::take(&mut self.sink.packets)
Expand Down
Loading
Loading