diff --git a/Cargo.lock b/Cargo.lock index bf4fa058..5867e389 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4694,6 +4694,7 @@ dependencies = [ "raw-window-handle", "resvg 0.50.0", "rune-fx", + "rustix 1.1.6", "same-file", "serde", "tempfile", diff --git a/README.md b/README.md index c167c06d..5fb33f92 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ cargo run --release -p moq-studio --features capture,openh264 -- \ - `show.toml` is the scene collection. If it does not exist yet, [guided setup](#first-launch) proposes one and creates it. Changes are saved to it a second after you stop editing, with no save step. -- `--recordings ` is where recordings are saved, by default beside +- `--recordings ` is where recordings and clips are saved, by default beside `show.toml`. - Without `--connect`, the studio composes, previews, and records but publishes nothing, and broadcast sources stay offline. @@ -141,9 +141,9 @@ macOS the system's buttons and edges stay. It holds, from the left: - The health dot, the stream's worst grade at a glance. Click it for each relay, who is watching, the encoder, and each problem, and **Details** for the [Health](#health) window. It reads **Not live** without `--connect`. -- Record, which starts or stops a [recording](#recording) with REVIEW's - picks. Its dot pulses while recording, and says for how long under the - pointer. +- Record, which starts or stops a [recording](#recording) from any page, with + the picks in Settings' Recording tab. Its dot pulses while recording, with + how long it has been recording beside it. - The live control: **LIVE** and how long the studio has been live, with `--connect`, or **NOT LIVE** without it. The studio goes live at launch, so it starts and ends nothing yet. @@ -184,7 +184,7 @@ The **strip** holds: | LIVE | Watch the program while on air, and TAKE or AUTO a cued scene. | | BUILD | Edit the scene on program, or the one open off air, shown in the middle, with its layers and what can be added on the left and the inspector on the right (see [BUILD](#build)). | | MIX | Add, rename, reorder, and remove audio scenes, pick the audio scene on program and the one to listen to and on which output, set each sound's gain, pan, mute, and delay, and send each sound and audio scene into audio scenes, with a meter on every one. The program shows small in the strip's place for the sound. | -| REVIEW | Replay the last moments of the program or a source in a replay source, slowed down if you like, or save them as a clip (see [Replay](#replay)), and start and stop recording and pick the picture and the audio scene it records (see [Recording](#recording)), beside the program. | +| REVIEW | Pick a stretch of the last 30 seconds of the program or a source from a strip of its pictures, and replay it in a replay source, slowed down if you like, or save it as a clip (see [Replay](#replay)). Beside it, this show's recordings and clips as cards with their length and size, one repaired after a crash marked so, and **Open the folder**. The program's card is the live picture. | The layout follows the program's shape and the window's size, giving the picture the most room it can. A landscape program sits above the strip; a @@ -297,6 +297,7 @@ the [hotkeys](#hotkeys), not in the scene file. | Tab | Use it to | | --- | --- | | Show | Set the canvas size and frame rate and each scene's size, see the broadcast path, whether the program is encoding, the encoder sessions open, and the encoder work of the program, the recording, and each published scene, and turn the virtual camera of the program or of each published scene on. | +| Recording | See and open the recordings folder, and pick the picture and the audio scene a recording holds, and whether each source records a file of its own too (see [Recording](#recording)). | | Hotkeys | Bind keys to the panic key, TAKE, AUTO, Studio mode, replay, save a clip, start or stop recording, and take or cue each scene, and clear them (see [Hotkeys](#hotkeys)). | | Effects | Pick how the window's [effects](#effects) move. | | Advanced | See the sources, scenes, and audio scenes as nodes, wired left to right to what draws or holds them, and wire them there (see [Scene graph](#scene-graph)). | @@ -503,7 +504,7 @@ each other audio scene. A source added from BUILD's Add, or pasted, is sent to the audio scene on program, so it is heard at once. **On program** picks the audio scene the broadcast carries, and switching it, or editing an audio scene, never drops or repeats audio. A recording holds -the audio scene on program unless REVIEW picks another, such as one +the audio scene on program unless Settings' Recording tab picks another, such as one keeping music the stream leaves out, and each published scene carries the one on program unless BUILD's inspector picks another beside its **Publish** switch; switching either while it runs never drops or repeats audio either. The audio @@ -1143,7 +1144,7 @@ placed, and so does every audio scene's mix. The studio draws it only as viewers will: over the program, on every page and in the program's virtual camera, so both show what viewers see. It goes over every other layer of its scene, and over the always-on-top layers, however the scene stacks it. Recordings hold the program as published, -without it, and recording **each source too** gives a published camera, +without it, and recording **Each source too** gives a published camera, display, window, or microphone a file of its own. Published and recorded scenes, and a scene that shows the program, leave it out too. While viewers read its track, it claims its bitrate of the connection beside the program's, and its video encodes at what the connection grants it. Its track carries one rendition, the plain capture before any filter or gain, and encodes for as long as it is @@ -1228,14 +1229,14 @@ once `moq play` plays layers. ### Recording -**Start recording** on REVIEW, or record in the title bar, saves the program at the canvas size, -or the scene picked beside it at its own size, +The record button in the title bar, on any page, saves the program at the +canvas size, or the scene picked in Settings' Recording tab at its own size, to ` .mp4` in the `--recordings` directory, live or not. The recording has an encoder of its own, opened only while it records, at full quality whatever the uplink grants the live stream, as OBS's advanced output mode records apart from streaming. It costs one more encode session and one more copy of each program frame while recording. With -**each source too**, every camera, display, window, microphone, and system +**Each source too** in Settings' Recording tab, every camera, display, window, microphone, and system audio records its own track into a file of its own beside it, ` .mp4`, holding the capture before any filter, or the sound before its gain and mute; a source being recorded @@ -1258,7 +1259,7 @@ program and local sources are read from the studio's own broadcasts, never what the relay serves, so a recording starts and stops without touching the live stream. The audio scene it holds is encoded a second time for it while recording: the one on program, so the file -holds what viewers hear, or the one REVIEW picks, which viewers never +holds what viewers hear, or the one Settings' Recording tab picks, which viewers never can. Without `--connect`, only the recording's encoder encodes the program. @@ -1275,7 +1276,7 @@ beside it, numbered if a file already has that name, a regular MP4 of everything the file holds up to its last complete fragment, leaves the original untouched, and removes the marker, so the file is repaired only once. The repair runs in the background and appears at the top of the -Health window and in the log. A file another studio is still recording into the +Health window and in the log, and REVIEW marks its card **Repaired after a crash**. A file another studio is still recording into the same directory is left alone. ### Source frames @@ -1303,8 +1304,10 @@ which its key takes, or cues in Studio mode, as its pad does; the tab says which. Press **Bind**, then the key with any modifiers; Escape, or leaving the window or the tab, cancels. Every action starts unbound, so no key a game uses is taken unasked, and a key bound to one action is refused for another -until it is cleared there. Replay, Save clip, and a recording use REVIEW's -picks. +until it is cleared there. Replay and Save clip use REVIEW's picks, but take +the last seconds as long as the stretch picked there, so a key pressed from +another page acts on what just happened, as the clip button beside the panic +key does; a recording uses Settings' Recording tab. The key pressed is the key that fires, wherever it sits, on any keyboard layout and after a layout change: the tab names it as the layout printed it when it @@ -1362,19 +1365,27 @@ and on Linux the desktop's, through its Settings portal. ## Replay Add a `replay` source and place it like any other, such as full screen in a -scene of its own. On REVIEW, pick the program or a source, a length -up to 30 seconds, and a speed of 100%, 50%, or 25%, then **Replay** plays that -many seconds up to the click in the replay source, which is transparent before -and after. Take its scene to put it on program. Sound replays only at -100%, through the replay source's strip on MIX, so a replay of a -microphone or system audio, which has only sound, plays only at 100%. A new -replay cuts -off the one playing, and a stretch the source was offline or paused replays -as the gap it was. - -**Save clip** writes the same seconds, in real time, beside the scene file as a -regular MP4: `show-clip-1.mp4` for `show.toml`, then `show-clip-2.mp4`, and so -on. Nothing is re-encoded, so a clip starts at the keyframe before those +scene of its own. On REVIEW, pick what to replay, the program or a source, by +its card, and a speed of 100%, 50%, or 25%. The strip shows the last 30 +seconds of what is picked, a picture each time its track starts a group, +which is each keyframe. Drag the amber handles to pick any stretch of it, +and click or drag elsewhere on the strip to move the scrub head: the large +picture is the one under the head. With the right handle at the strip's right +edge, the stretch is the last that many seconds up to the click; dragged +left of it, the stretch stays on the moments picked as time goes on, until it +ages out of the 30 seconds. **Replay** plays the stretch in the replay source, +which is transparent before and after; with more than one replay source, +pick which one under the speed. Take its scene to put it on program. +Sound replays only at 100%, through the replay source's strip on MIX, so a +replay of a microphone or system audio, which has only sound, plays only at +100%. A new replay cuts off the one playing, and a stretch the source was +offline or paused replays as the gap it was. When Replay or Save clip cannot +act, such as off air with no relay or for a source with no past, it says why +in place of what it would do. + +**Save clip** writes the same stretch, in real time, into the recordings +folder as a regular MP4: `show-clip-1.mp4` for `show.toml`, then +`show-clip-2.mp4`, and so on. Nothing is re-encoded, so a clip starts at the keyframe before those seconds, up to one keyframe interval earlier. It shows nothing after them, so video with B-frames can end early: the pictures shown before the end that refer to one shown after it are left out, along with everything decoded after that diff --git a/quest/m0/ux/review.md b/quest/m0/ux/review.md deleted file mode 100644 index 1e2ee4d2..00000000 --- a/quest/m0/ux/review.md +++ /dev/null @@ -1,42 +0,0 @@ -# [M] The REVIEW page - -## Goal - -The REVIEW page of [`mock/review.html`](/quest/m0/ux/mock/review.html) -replaces the Replay panel and the recording half of the Output panel: the -last 30 seconds as a strip to pick a stretch from, what to replay (the -program or a source), the speed, and Replay and Save clip; beside it, this -show's recordings and clips as cards with a picture, a length, and a size, -one that was repaired after a crash marked as such, and a button that opens -their folder. - -The bottom strip stays, and the program's card in "replay what" is the live -picture, so looking back never hides what is on air. - -The title bar's record button starts and stops a recording from any page and -shows its clock while recording. - -## Plan - -Recording and repair behave as `README.md` has them; this is their place and -look. Recommended, not decided: Replay and Save clip take any stretch of the -last 30 seconds, picked with two handles, where today they take the last N -seconds up to the click. The track already holds the 30 seconds. The -alternative pins the right handle to now. The strip's pictures are frames the program's track -already keeps for late viewers, decoded at a low rate while the page is -shown, and nothing records in the background for it. - -- The large picture is the frame under the scrub head, so the choice is - seen before it is replayed. The program's card keeps its ON AIR tag, so the - live picture never leaves the screen. -- A replay that cannot play says why where the button is: off air with no - relay, a source with no past, a speed that drops the sound. -- A recording's card comes from the files in the recordings folder that - carry this show's name. Reading their length and a first picture must not - hold up the page. -- Which picture and audio scene a recording holds are Settings, Recording. - -[Markers and clips](/quest/m2/markers.md) later adds markers to the strip -and chapters to a recording. - -Human check: scrubbing and picking a stretch by feel. diff --git a/quest/m0/ux/try-review.md b/quest/m0/ux/try-review.md new file mode 100644 index 00000000..ac83c985 --- /dev/null +++ b/quest/m0/ux/try-review.md @@ -0,0 +1,30 @@ +# [XS] Try the REVIEW page + +## Goal + +The maintainer tries the REVIEW page on their own machine and either signs it +off or files what they find as quests: what the epic's +[What a person must check](/quest/m0/ux/README.md) asks of it, which no +headless run exercises. + +## Plan + +The REVIEW page is on `main` (`cargo run -p moq-studio --features capture`). +What to try, on a show going live to a relay: + +- **Feel:** scrubbing the strip's head and dragging the two amber handles to + pick a stretch, then Replay and Save clip; whether the strip keeps up. +- **Look:** whether the page reads like its + [mock-up](/quest/m0/ux/mock/review-review.webp). +- **Readability:** the strip's ruler and AIR/AGO labels, and the cards' + length, size, and age, at a normal window size. + +An agent then turns each finding into a quest, or an edit to the quest that +owns that part of the page, and deletes this one. + +## Waiting + +- Maintainer: try the REVIEW page as the Plan lists and report what to change, + or that nothing needs to. An agent cannot judge feel or taste; a kittest run + and `just shots` already cover the clicks and the layout. Check: a comment + or quest records the result. diff --git a/quest/m2/markers.md b/quest/m2/markers.md index dfb3535b..73648af3 100644 --- a/quest/m2/markers.md +++ b/quest/m2/markers.md @@ -17,7 +17,3 @@ Planned 2026-11-09 (maintainer, /quest-plan), and held for later: ✅ m2 same keyframe limits apply. - Chapters are written when a recording becomes a regular MP4. - Marker is a hotkey action. - -## Required - -- [The REVIEW page](/quest/m0/ux/review.md) - the strip and the list markers appear on diff --git a/quest/m3/card-pictures.md b/quest/m3/card-pictures.md new file mode 100644 index 00000000..b2b11fad --- /dev/null +++ b/quest/m3/card-pictures.md @@ -0,0 +1,24 @@ +# [S] Pictures on REVIEW's recording and clip cards + +## Goal + +Each finished recording's and clip's card on the REVIEW page of +[`mock/review.html`](/quest/m0/ux/mock/review.html) shows the file's first +picture, where it shows a placeholder tile today. + +## Plan + +Decided 2026-11-15 (agent's recommendation on moq-dev/studio#177, which the +maintainer kept when picking it to merge): ✅ decode through moq-mux's regular MP4 import +(rejected: reading the first sample here with `mp4-atom`, since demuxing +belongs in moq; dropping the pictures from the mock-up). + +The picture is the file's first video sample, imported through moq-mux and +decoded small once per file in the shelf's background listing +(`rs/moq-studio/src/shelf.rs`), as the strip decodes keyframes +(`rs/moq-studio/src/past.rs`). A recording under way already shows the live +picture. The placeholder is painted in `rs/moq-studio/src/ui/review.rs`. + +## Required + +- [MP4 import pinned](/quest/m3/pin-mp4-import.md) - brings moq-mux's regular MP4 import into the `moq` submodule diff --git a/quest/m3/pin-mp4-import.md b/quest/m3/pin-mp4-import.md new file mode 100644 index 00000000..18579c6c --- /dev/null +++ b/quest/m3/pin-mp4-import.md @@ -0,0 +1,22 @@ +# [XS] moq-mux's regular MP4 import is pinned + +## Goal + +The `moq` submodule includes moq-mux's import of a regular, non-fragmented +MP4, from +[moq-mux imports a regular MP4](https://github.com/moq-dev/moq/blob/main/quest/m3/mp4-import.md). + +## Plan + +Once the wait clears, advance it with `just upgrade-moq` in its own PR, +fixing whatever the new API breaks here. + +Here in m3 because the upstream quest is in moq's m3 (maintainer, 2026-11-05); +move both up together if the studio needs it sooner. + +## Waiting + +- moq-dev/moq: the upstream quest above lands. Check: that quest's PR has + merged on moq's `main`, deleting its file, and `ContainerFormat` in + `rs/moq-mux/src/import/format.rs` offers regular MP4. A quest file that + moved, or went away without its work landing, does not count. diff --git a/rs/moq-studio/Cargo.toml b/rs/moq-studio/Cargo.toml index a6f3b012..f11020a5 100644 --- a/rs/moq-studio/Cargo.toml +++ b/rs/moq-studio/Cargo.toml @@ -68,6 +68,9 @@ moq-scene = { workspace = true } moq-stats = { workspace = true, features = ["async"] } moq-tokio = { workspace = true, default-features = false, features = ["net"] } moq-video = { workspace = true } +# A recording's length, from its movie header; the version moq-mux writes it +# with. +mp4-atom = "0.17.0" pollster = { workspace = true } # The join link's QR code. No dependencies of its own. qrcodegen = "1.8" @@ -110,6 +113,10 @@ raw-window-handle = "0.6" # tells a held key's repeats from new presses. x11rb = { version = "0.13", features = ["xkb"] } +[target.'cfg(unix)'.dependencies] +# The room left on the recordings folder's disk. moq-scene's release. +rustix = { version = "1", features = ["fs"] } + [target.'cfg(any(target_os = "macos", windows))'.dependencies] # System-wide hotkeys on macOS and Windows; X11 grabs keys itself, and Wayland # goes through the GlobalShortcuts portal. @@ -123,8 +130,9 @@ objc2-app-kit = { version = "0.3", default-features = false, features = ["std", [target.'cfg(windows)'.dependencies] # Which virtual key the foreground window's layout puts on each key, so a # hotkey registers the key pressed, and whether the system asks apps to -# reduce motion, for the effects. -windows-sys = { version = "0.61", features = ["Win32_UI_Input_KeyboardAndMouse", "Win32_UI_WindowsAndMessaging"] } +# reduce motion, for the effects, and the room left on the recordings folder's +# disk. +windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem", "Win32_UI_Input_KeyboardAndMouse", "Win32_UI_WindowsAndMessaging"] } [build-dependencies] # Rasterizes the app icon's SVG for the window. No text or embedded images, so @@ -138,8 +146,6 @@ egui_kittest = { version = "0.36", features = ["eframe", "wgpu"] } image = { version = "0.25", default-features = false, features = ["png"] } # A clock that moves wall time too, which renewed tokens' expiries read. kio = { workspace = true, features = ["manual"] } -# Reads a saved clip's timeline back; the version moq-mux writes it with. -mp4-atom = "0.17.0" # The auth server a test relay verifies guest tokens with. moq-auth = { workspace = true, features = ["serve"] } moq-relay = { workspace = true, features = ["test-support"] } diff --git a/rs/moq-studio/assets/icons/download.svg b/rs/moq-studio/assets/icons/download.svg new file mode 100644 index 00000000..6472b987 --- /dev/null +++ b/rs/moq-studio/assets/icons/download.svg @@ -0,0 +1,16 @@ + + + + + + diff --git a/rs/moq-studio/assets/icons/folder.svg b/rs/moq-studio/assets/icons/folder.svg new file mode 100644 index 00000000..7e5cb44b --- /dev/null +++ b/rs/moq-studio/assets/icons/folder.svg @@ -0,0 +1,14 @@ + + + + diff --git a/rs/moq-studio/src/controls.rs b/rs/moq-studio/src/controls.rs index 73fa847d..bee7e2a5 100644 --- a/rs/moq-studio/src/controls.rs +++ b/rs/moq-studio/src/controls.rs @@ -24,9 +24,9 @@ pub enum Action { Scene(String), /// Turn Studio mode on or off. StudioMode, - /// Play the recent past the Replay panel picks in its replay source. + /// Play the stretch of the recent past REVIEW picks in its replay source. Replay, - /// Save the recent past the Replay panel picks as a clip. + /// Save the stretch of the recent past REVIEW picks as a clip. SaveClip, /// Start recording, or stop the recording under way. Record, @@ -77,7 +77,7 @@ impl std::fmt::Display for Action { pub struct Controls { pub engine: Arc, runtime: tokio::runtime::Handle, - /// The Replay panel's picks, which a replay or clip from a hotkey takes too. + /// REVIEW's replay picks, which a replay or clip from a hotkey takes too. pub replay: Mutex, /// Whether the next recording gives each source a file of its own too. pub record_sources: AtomicBool, @@ -88,10 +88,10 @@ pub struct Controls { fired: Arc>>>, } -/// The Replay panel's choices, and how the last replay or clip went. +/// REVIEW's replay choices, and how the last replay or clip went. pub struct Replay { pub of: replay::Of, - pub length: std::time::Duration, + pub stretch: replay::Stretch, pub speed: f32, /// The replay source a replay plays in. pub into: Option, @@ -103,7 +103,7 @@ impl Default for Replay { fn default() -> Self { Self { of: replay::Of::Program, - length: replay::LENGTHS[1], + stretch: replay::Stretch::default(), speed: replay::SPEEDS[0], into: None, outcome: Arc::default(), @@ -192,15 +192,15 @@ impl Controls { } } - /// Play the Replay panel's pick in its replay source, or save it as a + /// Play REVIEW's pick in its replay source, or save it as a /// clip. Copied now, without waiting on anything, so the span ends here and /// replays apply in the order they were asked for. How it went shows in - /// the Replay panel, and in the status bar too when a hotkey `pressed` it, + /// REVIEW, and in the status bar too when a hotkey `pressed` it, /// since the panel may be hidden. fn replay(&self, play: bool, pressed: bool) -> anyhow::Result { let replay = self.replay.lock().unwrap(); let action = if play { Action::Replay } else { Action::SaveClip }; - let taken = self.take(&replay, play); + let taken = self.take(&replay, play, pressed); let message = taken .as_ref() .map(|(message, _)| message.clone()) @@ -225,16 +225,24 @@ impl Controls { } /// What a replay or clip does at once, and the save a clip still runs. + /// One `pressed` from outside REVIEW's own buttons, by a hotkey or the + /// shell's clip button, takes the last stretch as long as the one picked. fn take( &self, replay: &Replay, play: bool, + pressed: bool, ) -> anyhow::Result<(String, Option> + use<>>)> { let into = match play { true => Some(self.replay_source(replay.into.as_deref())?), false => None, }; - let take = replay::take(&self.engine, &replay.of, replay.length, into.is_some())?; + let stretch = match pressed { + true => replay.stretch.last(), + false => replay.stretch, + }; + let (ago, length) = stretch.span(kio::clock::Instant::now())?; + let take = replay::take(&self.engine, &replay.of, ago, length, into.is_some())?; let of = replay.of.clone(); if let Some(into) = into { replay::play(&self.engine, take, replay.speed, &into)?; @@ -243,7 +251,8 @@ impl Controls { let saving = format!("saving {of}"); let engine = self.engine.clone(); let save = async move { - let path = replay::save(take, engine.autosave.path()).await?; + let recorder = &engine.recorder; + let path = replay::save(take, recorder.dir(), recorder.show()).await?; Ok(format!("saved {of} to {}", path.display())) }; Ok((saving, Some(save))) diff --git a/rs/moq-studio/src/engine.rs b/rs/moq-studio/src/engine.rs index 390e6d81..3ab8d18b 100644 --- a/rs/moq-studio/src/engine.rs +++ b/rs/moq-studio/src/engine.rs @@ -5096,15 +5096,45 @@ mod tests { assert!(engine.stats.error().is_none()); // The program's recent past saves as a clip, from the groups it holds. - let take = crate::replay::take(&engine, &crate::replay::Of::Program, Duration::from_secs(1), false) + let of = crate::replay::Of::Program; + let take = crate::replay::take(&engine, &of, Duration::ZERO, Duration::from_secs(1), false) .expect("the program's past"); assert!(take.video.is_some(), "the canvas size holds video"); - let clip = kio::clock::timeout(TIMEOUT, crate::replay::save(take, engine.autosave.path())) + let (folder, show) = (engine.recorder.dir(), engine.recorder.show()); + let clip = kio::clock::timeout(TIMEOUT, crate::replay::save(take, folder, show)) .await .expect("the clip saves") .expect("a clip"); assert_eq!(clip, dir.path().join("show-clip-1.mp4")); + // REVIEW's strip decodes the keyframe each held group opens with, small, + // and it is the picture composed: the red canvas. + let view = tokio::task::spawn_blocking({ + let engine = engine.clone(); + move || crate::past::sample(&engine, &of) + }) + .await + .unwrap(); + let held = view.held.expect("sampled").expect("the program's past"); + assert!(held.video); + assert!(!view.pictures.is_empty(), "a keyframe of the past"); + for picture in &view.pictures { + assert_eq!(picture.size[0], 480, "decoded small"); + assert!(held.oldest <= picture.at && picture.at <= kio::clock::Instant::now()); + let middle = (picture.size[1] / 2 * picture.size[0] + picture.size[0] / 2) * 4; + let [r, g, b, _] = picture.rgba[middle..middle + 4] else { + unreachable!() + }; + assert!(r > 200 && g < 60 && b < 60, "red, not {r} {g} {b}"); + } + let times: Vec<_> = view.pictures.iter().map(|picture| picture.at).collect(); + assert!(times.is_sorted(), "oldest first"); + // And its sound, a packet at a time: the program's mix, silent here. + let sound = view.sound.expect("the program's sound"); + assert!(sound.len() > 10, "{} packets", sound.len()); + assert!(sound.iter().all(|(_, peak)| (0.0..=1.0).contains(peak))); + assert!(sound.iter().map(|(at, _)| *at).is_sorted(), "oldest first"); + // Quitting ends the program's video and audio cleanly once the encoders // published what they were handed, then its catalog in each format, and // closes the relay session only once the relay has them, rather than diff --git a/rs/moq-studio/src/kit/icon.rs b/rs/moq-studio/src/kit/icon.rs index 02869bf6..8074423f 100644 --- a/rs/moq-studio/src/kit/icon.rs +++ b/rs/moq-studio/src/kit/icon.rs @@ -61,12 +61,15 @@ pub enum Icon { Broadcast, Scene, Replay, + Folder, + Save, + Repaired, } impl Icon { /// Every icon, for the gallery. #[cfg(test)] - pub const ALL: [Self; 50] = [ + pub const ALL: [Self; 53] = [ Self::Undo, Self::Redo, Self::Settings, @@ -117,6 +120,9 @@ impl Icon { Self::Broadcast, Self::Scene, Self::Replay, + Self::Folder, + Self::Save, + Self::Repaired, ]; fn source(self) -> ImageSource<'static> { @@ -171,6 +177,9 @@ impl Icon { Self::Broadcast => include_image!("../../assets/icons/radio-tower.svg"), Self::Scene => include_image!("../../assets/icons/layers.svg"), Self::Replay => include_image!("../../assets/icons/rewind.svg"), + Self::Folder => include_image!("../../assets/icons/folder.svg"), + Self::Save => include_image!("../../assets/icons/download.svg"), + Self::Repaired => include_image!("../../assets/icons/shield.svg"), } } diff --git a/rs/moq-studio/src/main.rs b/rs/moq-studio/src/main.rs index d79966e9..30c6a29c 100644 --- a/rs/moq-studio/src/main.rs +++ b/rs/moq-studio/src/main.rs @@ -27,9 +27,11 @@ mod interact; mod kit; mod ladder; mod mix; +mod past; mod record; mod replay; mod setup; +mod shelf; #[cfg(test)] mod shots; mod snap; @@ -67,7 +69,7 @@ pub struct Config { #[usage(long)] pub on_demand: bool, - /// The directory recordings are saved in. Defaults to the scene file's. + /// The directory recordings and clips are saved in. Defaults to the scene file's. #[usage(long, value_hint = usage_rs::ValueHint::DirPath)] pub recordings: Option, diff --git a/rs/moq-studio/src/past.rs b/rs/moq-studio/src/past.rs new file mode 100644 index 00000000..5753a688 --- /dev/null +++ b/rs/moq-studio/src/past.rs @@ -0,0 +1,438 @@ +//! The last 30 seconds of what REVIEW replays, as pictures to scrub. +//! +//! Nothing records for it. About once a second while the page is shown, a +//! thread of its own takes the window the way a replay does, from the groups +//! the tracks keep for late viewers anyway, and decodes the keyframe each +//! group opens with, small and on the CPU, once per keyframe. A picture is +//! the moment its group starts, so the strip is as fine as the track's +//! keyframe interval. The sound is decoded too, once per group, down to the +//! loudest sample of each packet. + +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Condvar, Mutex}; +use std::task::Poll; +use std::time::Duration; + +use anyhow::Context; +use kio::clock::Instant; +use moq_mux::container::Container as _; + +use crate::engine::Engine; +use crate::replay::{self, Of}; + +/// How often the window is taken again while it is watched. +const PERIOD: Duration = Duration::from_secs(1); + +/// How long the thread keeps going once the page stops asking. +const LINGER: Duration = Duration::from_secs(2); + +/// The last [`Picture::id`] handed out, shared by every worker the page +/// starts, since the window keeps textures across them. +static LAST_ID: AtomicU64 = AtomicU64::new(0); + +/// How wide a picture is decoded: enough for the page's large picture. +const WIDTH: u32 = 480; + +/// The recent past of one thing REVIEW replays, kept fresh while watched. +pub struct Past { + shared: Arc, +} + +struct Shared { + engine: Arc, + watch: Mutex, + /// Wakes the thread early, when what it watches changes. + wake: Condvar, +} + +struct Watch { + of: Of, + /// When the page last asked, after which the thread lingers and stops. + seen: Instant, + running: bool, + view: Arc, +} + +/// What the window held when last taken. +pub struct View { + /// What it is of, which lags a change of pick by a sample. + pub of: Of, + /// The keyframe each group opens with, oldest first. + pub pictures: Vec>, + /// The loudest sample of each packet of sound, from 0 to 1, oldest first, + /// or why the sound could not be read. + pub sound: Result, String>, + /// How far back it reaches, or why there is nothing to replay. + pub held: Option>, +} + +impl Default for View { + fn default() -> Self { + Self { + of: Of::Program, + pictures: Vec::new(), + sound: Ok(Vec::new()), + held: None, + } + } +} + +/// How much of the window a sample found. +#[derive(Clone, Copy, Debug)] +pub struct Held { + /// The oldest moment held. + pub oldest: Instant, + /// Whether it holds pictures, rather than sound alone. + pub video: bool, +} + +/// A decoded keyframe. +pub struct Picture { + /// Unique for as long as the studio runs, so the window uploads it once. + pub id: u64, + /// When it was shown. + pub at: Instant, + pub size: [usize; 2], + /// Tightly packed RGBA. + pub rgba: Vec, +} + +impl Past { + pub fn new(engine: Arc) -> Self { + let watch = Watch { + of: Of::Program, + seen: Instant::now(), + running: false, + view: Arc::default(), + }; + Self { + shared: Arc::new(Shared { + engine, + watch: Mutex::new(watch), + wake: Condvar::new(), + }), + } + } + + /// Keep the past of `of` fresh, for a frame that shows it, and what the + /// last sample found. + pub fn watch(&self, of: &Of) -> Arc { + let mut watch = self.shared.watch.lock().unwrap(); + watch.seen = Instant::now(); + if watch.of != *of { + watch.of = of.clone(); + self.shared.wake.notify_all(); + } + if !watch.running { + watch.running = true; + let shared = self.shared.clone(); + let spawned = std::thread::Builder::new() + .name("review-past".into()) + .spawn(move || run(&shared)); + if let Err(err) = spawned { + watch.running = false; + watch.view = Arc::new(View { + of: of.clone(), + held: Some(Err(format!("cannot start the strip's decoder: {err}"))), + ..View::default() + }); + } + } + watch.view.clone() + } + + /// Show `view` as if sampled, for the window's views. + #[cfg(test)] + pub fn seed(&self, view: View) { + let mut watch = self.shared.watch.lock().unwrap(); + // A thread that runs would replace it. + watch.running = true; + watch.of = view.of.clone(); + watch.view = Arc::new(view); + } +} + +/// Take the window of `of` once, on a decoder of its own, as the thread does. +#[cfg(test)] +pub fn sample(engine: &Engine, of: &Of) -> View { + Decoding::default().sample(engine, of) +} + +/// Take the window about once a [`PERIOD`] while watched. +fn run(shared: &Shared) { + let mut decoding = Decoding::default(); + let mut watch = shared.watch.lock().unwrap(); + loop { + if watch.seen.elapsed() > LINGER { + watch.running = false; + return; + } + let of = watch.of.clone(); + drop(watch); + let view = decoding.sample(&shared.engine, &of); + watch = shared.watch.lock().unwrap(); + watch.view = Arc::new(view); + if watch.of == of { + watch = shared.wake.wait_timeout(watch, PERIOD).unwrap().0; + } + } +} + +/// The decoder and the pictures already decoded, of one rendition. +#[derive(Default)] +struct Decoding { + /// What they are of: the pick, and the rendition's name and config. + key: Option<(Of, String, hang::catalog::VideoConfig)>, + decoder: Option, + /// By the keyframe's timestamp, in microseconds on its broadcast's timeline. + pictures: BTreeMap>, + /// The same for the sound: what it is of, its decoder, and each group's + /// peaks by the group's sequence. + sound_key: Option<(Of, String, hang::catalog::AudioConfig)>, + sound_decoder: Option, + peaks: BTreeMap>, +} + +impl Decoding { + fn sample(&mut self, engine: &Engine, of: &Of) -> View { + let mut view = View { + of: of.clone(), + ..View::default() + }; + let now = Instant::now(); + let take = match replay::take(engine, of, Duration::ZERO, replay::WINDOW, true) { + Ok(take) => take, + Err(err) => { + self.reset(); + view.held = Some(Err(format!("{err:#}"))); + return view; + } + }; + // The moment a timestamp of the take's broadcast was shown. + let end = Duration::from(take.end); + let at = |timestamp: moq_net::Timestamp| now.saturating_sub(end.saturating_sub(Duration::from(timestamp))); + let broadcast = take.consume(); + let mut oldest = None; + if let Some((name, config)) = &take.audio { + view.sound = self + .sound(&broadcast, of, name, config, &at) + .map_err(|err| format!("{err:#}")); + oldest = view + .sound + .as_ref() + .ok() + .and_then(|sound| sound.first()) + .map(|(at, _)| *at); + } + if let Some((name, config)) = &take.video { + match self.pictures(&broadcast, of, name, config, &at) { + Ok(pictures) => view.pictures = pictures, + Err(err) => { + view.held = Some(Err(format!("{err:#}"))); + return view; + } + } + let first = view.pictures.first().map(|picture| picture.at); + oldest = oldest.into_iter().chain(first).min(); + } + view.held = Some(Ok(Held { + oldest: oldest.unwrap_or(now), + video: take.video.is_some(), + })); + view + } + + /// The keyframe of each group of the video `name`, decoding those not + /// decoded yet and letting go of those the window no longer holds. + fn pictures( + &mut self, + broadcast: &moq_net::broadcast::Consumer, + of: &Of, + name: &str, + config: &hang::catalog::VideoConfig, + at: &impl Fn(moq_net::Timestamp) -> Instant, + ) -> anyhow::Result>> { + let key = Some((of.clone(), name.to_string(), config.clone())); + if self.key != key { + self.reset(); + self.key = key; + } + let container = moq_mux::catalog::hang::Container::try_from(config)?; + let mut kept = BTreeMap::new(); + for frame in firsts(broadcast, name, container)? { + let micros = frame.timestamp.as_micros(); + let picture = match self.pictures.remove(µs) { + Some(picture) => picture, + None => { + let Some(rgba) = self.decode(config, &frame)? else { + continue; + }; + let size = rgba.size(); + Arc::new(Picture { + id: LAST_ID.fetch_add(1, Ordering::Relaxed) + 1, + at: at(frame.timestamp), + size: [size.width as usize, size.height as usize], + rgba: rgba.into_data(), + }) + } + }; + kept.insert(micros, picture); + } + self.pictures = kept; + Ok(self.pictures.values().cloned().collect()) + } + + /// Decode one keyframe on its own, small, or `None` if the decoder gave + /// back no picture for it. + fn decode( + &mut self, + config: &hang::catalog::VideoConfig, + frame: &moq_mux::container::Frame, + ) -> anyhow::Result> { + let decoder = match &mut self.decoder { + Some(decoder) => decoder, + None => { + let mut options = moq_video::decode::Config::new(); + options.output = moq_video::Output::Cpu; + let decoder = moq_video::decode::Decoder::new(config, &options) + .context("cannot decode the pictures for the strip")?; + self.decoder.insert(decoder) + } + }; + decoder.decode(&frame.payload, frame.timestamp, true)?; + // Drained at once, so the keyframe comes out without the rest of its group. + decoder.flush()?; + let mut picture = None; + while let Some(decoded) = decoder.receive()? { + picture = Some(decoded); + } + let Some(picture) = picture else { return Ok(None) }; + let size = picture.size(); + let height = (u64::from(WIDTH) * u64::from(size.height) / u64::from(size.width.max(1))) as u32; + let small = moq_video::Size::new(WIDTH, (height.max(2) + 1) & !1); + let mut resize = moq_video::resize::Config::default(); + resize.output = moq_video::Output::Cpu; + let small = picture.resize(small, &resize)?; + Ok(Some(small.surface.to_rgba(&moq_video::convert::Config::new())?)) + } + + /// The loudest sample of each packet of the audio `name`, decoding the + /// groups not decoded yet, and the newest again, which may have grown. + fn sound( + &mut self, + broadcast: &moq_net::broadcast::Consumer, + of: &Of, + name: &str, + config: &hang::catalog::AudioConfig, + at: &impl Fn(moq_net::Timestamp) -> Instant, + ) -> anyhow::Result> { + let key = Some((of.clone(), name.to_string(), config.clone())); + if self.sound_key != key { + self.sound_decoder = None; + self.peaks.clear(); + self.sound_key = key; + } + let container = moq_mux::catalog::hang::Container::try_from(config)?; + let groups = groups(broadcast, name, false)?; + let newest = groups.last().map(|(sequence, _)| *sequence); + let mut kept = BTreeMap::new(); + for (sequence, frames) in groups { + let peaks = match self.peaks.remove(&sequence) { + Some(peaks) if Some(sequence) != newest => peaks, + _ => { + let decoder = match &mut self.sound_decoder { + Some(decoder) => decoder, + None => self.sound_decoder.insert( + moq_audio::decode::Decoder::new(config, &moq_audio::decode::Config::new()) + .context("cannot decode the sound for the strip")?, + ), + }; + let mut peaks = Vec::new(); + for frame in frames { + for packet in container.decode(frame)? { + decoder.decode(&packet.payload)?; + let mut peak = 0.0_f32; + while let Some(decoded) = decoder.receive()? { + peak = decoded.samples.iter().fold(peak, |peak, sample| peak.max(sample.abs())); + } + peaks.push((packet.timestamp, peak.min(1.0))); + } + } + peaks + } + }; + kept.insert(sequence, peaks); + } + self.peaks = kept; + Ok(self + .peaks + .values() + .flatten() + .map(|(timestamp, peak)| (at(*timestamp), *peak)) + .collect()) + } + + fn reset(&mut self) { + self.key = None; + self.decoder = None; + self.pictures.clear(); + self.sound_key = None; + self.sound_decoder = None; + self.peaks.clear(); + } +} + +/// The frame each group of the track `name` opens with, oldest first, as far +/// as the take holds them now. +fn firsts( + broadcast: &moq_net::broadcast::Consumer, + name: &str, + container: moq_mux::catalog::hang::Container, +) -> anyhow::Result> { + let mut firsts = Vec::new(); + for (_, frames) in groups(broadcast, name, true)? { + let Some(frame) = frames.into_iter().next() else { + continue; + }; + // A gap's marker has an empty payload, and is no picture. + if let Some(first) = container.decode(frame)?.into_iter().next() + && !first.payload.is_empty() + { + firsts.push(first); + } + } + Ok(firsts) +} + +/// Each group of the track `name` by its sequence, oldest first, with its +/// frames, or only its first if `first`, as far as the take holds them now. +fn groups( + broadcast: &moq_net::broadcast::Consumer, + name: &str, + first: bool, +) -> anyhow::Result)>> { + let subscription = moq_net::track::Subscription::default() + .with_delay(moq_net::track::Delay::default().with_max(None)) + .with_order(moq_net::track::Order::Ascending); + // A take's tracks are written and finished before it is handed out, so + // everything is ready and nothing is waited on. + let now = kio::Waiter::noop(); + let mut source = match broadcast.track(name)?.subscribe(subscription).poll_ok(&now) { + Poll::Ready(source) => source?, + Poll::Pending => return Ok(Vec::new()), + }; + let mut groups = BTreeMap::new(); + while let Poll::Ready(group) = source.poll_recv_group(&now) { + let Some(mut group) = group? else { break }; + let mut frames = Vec::new(); + while let Poll::Ready(Ok(Some(frame))) = group.poll_read_frame(&now) { + frames.push(frame); + if first { + break; + } + } + groups.insert(group.sequence, frames); + } + Ok(groups.into_iter().collect()) +} diff --git a/rs/moq-studio/src/record.rs b/rs/moq-studio/src/record.rs index 60785f6e..5e327c5b 100644 --- a/rs/moq-studio/src/record.rs +++ b/rs/moq-studio/src/record.rs @@ -156,6 +156,11 @@ impl Recorder { &self.dir } + /// The show's name, which each file's starts with: the scene file's. + pub fn show(&self) -> &str { + &self.name + } + /// Record the broadcast sources `collection` has from now on, as each /// recording that gives sources files of their own follows them. pub fn set_sources(&self, collection: &moq_scene::Collection) { diff --git a/rs/moq-studio/src/replay.rs b/rs/moq-studio/src/replay.rs index b92f4a57..7aa296a5 100644 --- a/rs/moq-studio/src/replay.rs +++ b/rs/moq-studio/src/replay.rs @@ -1,27 +1,71 @@ -//! Instant replays and clips: the recent past of the program or a source, -//! played in a replay source or saved to disk. +//! Instant replays and clips: a stretch of the recent past of the program or a +//! source, played in a replay source or saved to disk. use std::path::{Path, PathBuf}; use std::time::Duration; use anyhow::Context; +use kio::clock::Instant; use moq_scene::Source; use moq_scene::replay::{Take, Target}; use tokio::io::{AsyncSeekExt, AsyncWriteExt}; use crate::engine::Engine; -/// The lengths the Replay panel offers. -pub const LENGTHS: [Duration; 4] = [ - Duration::from_secs(5), - Duration::from_secs(10), - Duration::from_secs(20), - moq_scene::replay::MAX_LENGTH, -]; +/// How far back a replay reaches: as far as the tracks keep their past. +pub const WINDOW: Duration = moq_scene::replay::MAX_LENGTH; -/// The speeds the Replay panel offers: real time, and slow motion. +/// The shortest stretch REVIEW's handles pick. +pub const MIN_STRETCH: Duration = Duration::from_millis(500); + +/// The speeds REVIEW offers: real time, and slow motion. pub const SPEEDS: [f32; 3] = [1.0, 0.5, 0.25]; +/// Which stretch of the recent past a replay or clip takes. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum Stretch { + /// The last this long, up to the moment it is fired. + Last(Duration), + /// From one past moment to a later one, which stay put as time goes on, + /// until they age out of the [`WINDOW`]. + Between(Instant, Instant), +} + +impl Default for Stretch { + fn default() -> Self { + Self::Last(Duration::from_secs(10)) + } +} + +impl Stretch { + /// How long before `now` it ends, and how long it is: the part of it still + /// inside the [`WINDOW`]. Refuses one that has aged out of it. + pub fn span(&self, now: Instant) -> anyhow::Result<(Duration, Duration)> { + let (from, to) = match *self { + Self::Last(length) => (length, Duration::ZERO), + Self::Between(from, to) => (now.saturating_duration_since(from), now.saturating_duration_since(to)), + }; + anyhow::ensure!( + to < WINDOW, + "the picked stretch is older than the last {} s; pick another", + WINDOW.as_secs() + ); + let length = from.min(WINDOW).saturating_sub(to); + anyhow::ensure!(!length.is_zero(), "the picked stretch is empty"); + Ok((to, length)) + } + + /// The last stretch as long as this one, as a hotkey or the shell's clip + /// button fires it: pressed away from REVIEW's strip, it means what just + /// happened, not a moment picked there earlier, which would age out. + pub fn last(self) -> Self { + match self { + Self::Between(from, to) => Self::Last(to.saturating_duration_since(from).min(WINDOW)), + last => last, + } + } +} + /// What a replay or clip is of. #[derive(Clone, Debug, Default, PartialEq, Eq)] pub enum Of { @@ -88,15 +132,16 @@ impl Engine { } } -/// Hold the `length` of `of` up to now: to play in a replay source with +/// Hold the `length` of `of` that ended `ago`: to play in a replay source with /// `play`, or as a clip to save. /// /// Takes the target whose video covers the most of the span, the first of /// those tied, such as the program's half size over a canvas size the grant /// disabled partway, or else the first one tried. -pub fn take(engine: &Engine, of: &Of, length: Duration, play: bool) -> anyhow::Result { +pub fn take(engine: &Engine, of: &Of, ago: Duration, length: Duration, play: bool) -> anyhow::Result { let mut best: Option> = None; - for target in engine.targets(of)? { + for mut target in engine.targets(of)? { + target.end = before(target.end, ago)?; let next = match play { true => Take::new(&target, length), false => Take::clip(&target, length), @@ -121,6 +166,14 @@ pub fn take(engine: &Engine, of: &Of, length: Duration, play: bool) -> anyhow::R }) } +/// The moment `ago` before `end`, on `end`'s timeline and in its scale. +fn before(end: moq_net::Timestamp, ago: Duration) -> anyhow::Result { + let at = Duration::from(end) + .checked_sub(ago) + .context("the broadcast is younger than the picked stretch")?; + Ok(moq_net::Timestamp::try_from(at)?.convert(end.scale())?) +} + /// Play `take` at `speed` in the replay source `into`. pub fn play(engine: &Engine, take: Take, speed: f32, into: &str) -> anyhow::Result<()> { let state = engine.state.lock().unwrap(); @@ -132,10 +185,10 @@ pub fn play(engine: &Engine, take: Take, speed: f32, into: &str) -> anyhow::Resu Ok(()) } -/// Save `take`, in real time, as a new MP4 file beside `scenes`, the scene -/// file. Returns where. -pub async fn save(take: Take, scenes: &Path) -> anyhow::Result { - let (path, mut file) = create(scenes).await?; +/// Save `take`, in real time, as a new MP4 file in `dir`, the recordings +/// folder, named after `show`. Returns where. +pub async fn save(take: Take, dir: &Path, show: &str) -> anyhow::Result { + let (path, mut file) = create(dir, show).await?; let written = async { let broadcast = take.consume(); let catalog = moq_mux::catalog::Consumer::<()>::new(&broadcast, moq_mux::catalog::CatalogFormat::Hang).await?; @@ -169,13 +222,11 @@ pub async fn save(take: Take, scenes: &Path) -> anyhow::Result { Ok(path) } -/// A new file beside the scene file `scenes`: for `show.toml`, -/// `show-clip-1.mp4`, or the first number not taken. -async fn create(scenes: &Path) -> anyhow::Result<(PathBuf, tokio::fs::File)> { - let dir = scenes.parent().unwrap_or(Path::new("")); - let stem = scenes.file_stem().unwrap_or_default().to_string_lossy(); +/// A new clip file in `dir`: for the show `show`, `show-clip-1.mp4`, or the +/// first number not taken. +async fn create(dir: &Path, show: &str) -> anyhow::Result<(PathBuf, tokio::fs::File)> { for n in 1.. { - let path = dir.join(format!("{stem}-clip-{n}.mp4")); + let path = dir.join(clip_name(show, n)); match tokio::fs::File::create_new(&path).await { Ok(file) => return Ok((path, file)), Err(err) if err.kind() == std::io::ErrorKind::AlreadyExists => continue, @@ -185,6 +236,49 @@ async fn create(scenes: &Path) -> anyhow::Result<(PathBuf, tokio::fs::File)> { unreachable!("some clip number is free") } +/// The name of the show `show`'s `n`th clip. +pub fn clip_name(show: &str, n: u32) -> String { + format!("{show}-clip-{n}.mp4") +} + +#[cfg(test)] +mod stretch_tests { + use super::*; + + /// The last few seconds end now, and a stretch picked between two moments + /// ends as long ago as its end, is cut where it leaves the window, and is + /// refused once it is wholly out of it. + #[test] + fn a_stretch_spans_what_the_window_holds() { + let secs = Duration::from_secs; + let now = Instant::now().saturating_add(secs(100)); + let ago = |seconds| now.saturating_sub(secs(seconds)); + assert_eq!(Stretch::Last(secs(8)).span(now).unwrap(), (Duration::ZERO, secs(8))); + let picked = Stretch::Between(ago(14), ago(6)); + assert_eq!(picked.span(now).unwrap(), (secs(6), secs(8))); + // Twenty seconds on, its start has left the 30 s window. + let later = now.saturating_add(secs(20)); + assert_eq!(picked.span(later).unwrap(), (secs(26), secs(4))); + let gone = now.saturating_add(secs(24)); + assert!(picked.span(gone).is_err(), "aged out"); + let empty = Stretch::Between(ago(3), ago(3)); + assert!(empty.span(now).is_err(), "empty"); + } + + /// A hotkey takes the last stretch as long as the one picked, even once + /// the picked one has aged out. + #[test] + fn a_hotkey_takes_the_last_stretch_as_long() { + let secs = Duration::from_secs; + let now = Instant::now().saturating_add(secs(100)); + let picked = Stretch::Between(now.saturating_sub(secs(14)), now.saturating_sub(secs(6))); + assert_eq!(picked.last(), Stretch::Last(secs(8))); + let later = now.saturating_add(secs(60)); + assert_eq!(picked.last().span(later).unwrap(), (Duration::ZERO, secs(8))); + assert_eq!(Stretch::Last(secs(5)).last(), Stretch::Last(secs(5))); + } +} + // The software encoder is the one every test machine has. #[cfg(all(test, feature = "openh264"))] mod tests { @@ -192,7 +286,7 @@ mod tests { use super::*; - /// A clip is a finished MP4 of the span, named beside the scene file + /// A clip is a finished MP4 of the span, named in the recordings folder /// without overwriting an earlier one, that shows nothing after the span's /// end even when a picture inside it refers to one shown after. #[tokio::test] @@ -239,10 +333,9 @@ mod tests { }; let dir = tempfile::tempdir().unwrap(); - let scenes = dir.path().join("show.toml"); std::fs::write(dir.path().join("show-clip-1.mp4"), b"an earlier clip").unwrap(); let take = Take::clip(&target, Duration::from_secs(1)).unwrap(); - let path = save(take, &scenes).await.unwrap(); + let path = save(take, dir.path(), "show").await.unwrap(); assert_eq!(path, dir.path().join("show-clip-2.mp4")); assert_eq!( std::fs::read(dir.path().join("show-clip-1.mp4")).unwrap(), diff --git a/rs/moq-studio/src/shelf.rs b/rs/moq-studio/src/shelf.rs new file mode 100644 index 00000000..ab95ec40 --- /dev/null +++ b/rs/moq-studio/src/shelf.rs @@ -0,0 +1,511 @@ +//! This show's recordings and clips, as REVIEW's cards list them: the files in +//! the recordings folder whose names start with the show's, read in the +//! background so a large folder or file never holds up the page. + +use std::collections::BTreeMap; +use std::io::{Read, Seek, SeekFrom}; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use kio::clock::{Instant, Wall}; +use mp4_atom::{Atom, ReadAtom, ReadFrom}; + +/// How often the folder is listed again while REVIEW shows it. +const PERIOD: Duration = Duration::from_secs(2); + +/// What a recording's start stamp looks like in its files' names: +/// `2026-11-03T12-00-00Z`. +const STAMP: usize = "2026-11-03T12-00-00Z".len(); + +/// What a repaired file's name adds. +const REPAIRED: &str = ".repaired"; + +/// The show's recordings and clips, listed again while watched. +pub struct Shelf { + dir: PathBuf, + show: String, + runtime: tokio::runtime::Handle, + shared: Arc>, +} + +#[derive(Default)] +struct Shared { + /// When the folder was last listed. + listed: Option, + /// Whether a listing runs now, so a slow disk never has two at once. + listing: bool, + cards: Arc, + lengths: Lengths, + /// Never listed again, for the window's views. + seeded: bool, +} + +/// Each file's length, by path, with its size and when it last changed, so a +/// file is read once until it changes. +type Lengths = BTreeMap, Option)>; + +/// What a listing found. +#[derive(Default)] +pub struct Listing { + /// The newest first. + pub cards: Vec, + /// Why the folder could not be listed. + pub error: Option, + /// The room left on the folder's disk, in bytes. + pub free: Option, +} + +/// A recording, all its files together, or a clip. +#[derive(Clone, Debug, PartialEq)] +pub struct Card { + pub kind: Kind, + /// The file a click on it shows: the program's, or the clip. + pub path: PathBuf, + /// When it was last written to. + pub modified: Option, + /// Every file's size together. + pub bytes: u64, + /// How long the program's files play together, once any is a regular MP4. + pub length: Option, +} + +#[derive(Clone, Debug, PartialEq)] +pub enum Kind { + Recording { + /// The UTC start stamp its files are named with. + stamp: String, + /// How many sources have files of their own beside the program's. + sources: usize, + /// Whether a file of it was repaired after a crash. + repaired: bool, + }, + /// The `n`th clip. + Clip(u32), +} + +impl Shelf { + pub fn new(dir: &Path, show: &str, runtime: tokio::runtime::Handle) -> Self { + Self { + dir: dir.to_path_buf(), + show: show.to_string(), + runtime, + shared: Arc::default(), + } + } + + /// The folder. + pub fn dir(&self) -> &Path { + &self.dir + } + + /// What the last listing found, listing again in the background once it + /// is a [`PERIOD`] old. + pub fn cards(&self) -> Arc { + let mut shared = self.shared.lock().unwrap(); + if !shared.seeded && !shared.listing && shared.listed.is_none_or(|listed| listed.elapsed() >= PERIOD) { + shared.listing = true; + let (dir, show, kept) = (self.dir.clone(), self.show.clone(), self.shared.clone()); + let lengths = shared.lengths.clone(); + self.runtime.spawn_blocking(move || { + let (listing, lengths) = list(&dir, &show, lengths); + let mut shared = kept.lock().unwrap(); + shared.cards = Arc::new(listing); + shared.lengths = lengths; + shared.listing = false; + // Counted from when it finished, so a slow disk is not asked again at once. + shared.listed = Some(Instant::now()); + }); + } + shared.cards.clone() + } + + /// Show `listing` as if listed, from now on, for the window's views. + #[cfg(test)] + pub fn seed(&self, listing: Listing) { + let mut shared = self.shared.lock().unwrap(); + shared.seeded = true; + shared.cards = Arc::new(listing); + } +} + +/// Open `path`, a folder or a recording, as the system opens it: a folder in +/// the file manager, a recording in the video player. +pub fn open(path: &Path) -> anyhow::Result<()> { + let program = match () { + _ if cfg!(target_os = "macos") => "open", + _ if cfg!(windows) => "explorer", + _ => "xdg-open", + }; + let mut child = std::process::Command::new(program) + .arg(path) + .spawn() + .map_err(|err| anyhow::anyhow!("cannot run {program} to open {}: {err}", path.display()))?; + // Reaped once it exits, so it leaves no zombie. + std::thread::spawn(move || child.wait()); + Ok(()) +} + +/// List the show `show`'s files in `dir` as cards, reading the length of any +/// file not in `lengths` or changed since. +fn list(dir: &Path, show: &str, mut lengths: Lengths) -> (Listing, Lengths) { + let mut listing = Listing { + free: free(dir), + ..Listing::default() + }; + let entries = match std::fs::read_dir(dir) { + Ok(entries) => entries, + Err(err) => { + listing.error = Some(format!("cannot list {}: {err}", dir.display())); + return (listing, lengths); + } + }; + let mut seen = BTreeMap::new(); + let mut recordings: BTreeMap = BTreeMap::new(); + for entry in entries.flatten() { + let name = entry.file_name().to_string_lossy().into_owned(); + let Some(file) = parse(show, &name) else { continue }; + let Ok(metadata) = entry.metadata() else { continue }; + let path = entry.path(); + let modified = metadata + .modified() + .ok() + .and_then(|modified| Wall::try_from(modified).ok()); + let bytes = metadata.len(); + let length = match lengths.remove(&path) { + Some((size, changed, length)) if size == bytes && changed == modified => length, + _ => length(&path), + }; + seen.insert(path.clone(), (bytes, modified, length)); + match file { + File::Clip(n) => listing.cards.push(Card { + kind: Kind::Clip(n), + path, + modified, + bytes, + length, + }), + File::Recording { + stamp, + source, + repaired, + } => { + let gathered = recordings.entry(stamp).or_default(); + gathered.bytes += bytes; + gathered.modified = gathered.modified.max(modified); + gathered.repaired |= repaired; + match source { + // A source's files count once, however many it was split + // or repaired into. + Some(source) => drop(gathered.sources.insert(source)), + None => { + if let Some(length) = length { + *gathered.length.get_or_insert_default() += length; + } + gathered.programs.push((!repaired, path.as_os_str().len(), path)); + } + } + } + } + } + listing + .cards + .extend(recordings.into_iter().filter_map(|(stamp, gathered)| { + // The first program file, its repair over it, is what a click shows. + let (_, _, path) = gathered.programs.into_iter().min()?; + Some(Card { + kind: Kind::Recording { + stamp, + sources: gathered.sources.len(), + repaired: gathered.repaired, + }, + path, + modified: gathered.modified, + bytes: gathered.bytes, + length: gathered.length, + }) + })); + listing + .cards + .sort_by(|a, b| b.modified.cmp(&a.modified).then_with(|| a.path.cmp(&b.path))); + (listing, seen) +} + +/// A recording's files, as the folder is listed. +#[derive(Default)] +struct Gathered { + bytes: u64, + modified: Option, + length: Option, + repaired: bool, + sources: std::collections::BTreeSet, + /// Each program file, after whether it is not a repair and how long its + /// name is, so the first file and its repair sort first. + programs: Vec<(bool, usize, PathBuf)>, +} + +/// What a file in the recordings folder is to the show `show`. +#[derive(Debug, PartialEq)] +pub enum File { + /// A file of the recording that started at `stamp`: the program's, or + /// the `source`'s. + Recording { + stamp: String, + source: Option, + repaired: bool, + }, + Clip(u32), +} + +/// Read a file's name as the recorder and clips name them: ` +/// [ ][ (n)][.repaired][ (n)].mp4`, or `-clip-.mp4`. +pub fn parse(show: &str, name: &str) -> Option { + let stem = name.strip_suffix(".mp4")?; + if let Some(n) = stem.strip_prefix(show).and_then(|rest| rest.strip_prefix("-clip-")) { + return n.parse().ok().filter(|n| *n > 0).map(File::Clip); + } + let rest = stem.strip_prefix(show)?.strip_prefix(' ')?; + let stamp = rest.get(..STAMP)?; + let shaped = stamp.bytes().enumerate().all(|(at, byte)| match at { + 4 | 7 | 13 | 16 => byte == b'-', + 10 => byte == b'T', + 19 => byte == b'Z', + _ => byte.is_ascii_digit(), + }); + if !shaped { + return None; + } + let mut tail = &rest[STAMP..]; + let repaired = match tail.find(REPAIRED) { + Some(at) => { + tail = &tail[..at]; + true + } + None => false, + }; + let tail = strip_number(tail); + let source = match tail { + "" => None, + source => Some(source.strip_prefix(' ')?.to_string()), + }; + Some(File::Recording { + stamp: stamp.to_string(), + source, + repaired, + }) +} + +/// `text` without a trailing ` (n)`, the number a taken name is given. +fn strip_number(text: &str) -> &str { + let Some(open) = text.rfind(" (") else { return text }; + let number = &text[open + 2..]; + match number.strip_suffix(')') { + Some(digits) if !digits.is_empty() && digits.bytes().all(|byte| byte.is_ascii_digit()) => &text[..open], + _ => text, + } +} + +/// How long the MP4 at `path` plays, from its movie header alone, skipping +/// its media. `None` for a fragmented one, whose header holds no length, and +/// for anything unreadable. +fn length(path: &Path) -> Option { + let mut file = std::io::BufReader::new(std::fs::File::open(path).ok()?); + loop { + let header = as ReadFrom>::read_from(&mut file).ok()??; + if header.kind != mp4_atom::Moov::KIND { + file.seek(SeekFrom::Current(i64::try_from(header.size?).ok()?)).ok()?; + continue; + } + let mut moov = (&mut file).take(header.size? as u64); + loop { + let child = mp4_atom::Header::read_from(&mut moov).ok()?; + if child.kind != mp4_atom::Mvhd::KIND { + std::io::copy(&mut (&mut moov).take(child.size? as u64), &mut std::io::sink()).ok()?; + continue; + } + let mvhd = mp4_atom::Mvhd::read_atom(&child, &mut moov).ok()?; + if mvhd.duration == 0 || mvhd.timescale == 0 { + return None; + } + return Some(Duration::from_secs_f64( + mvhd.duration as f64 / f64::from(mvhd.timescale), + )); + } + } +} + +/// The room left for an unprivileged user on the disk holding `dir`, in bytes. +#[cfg(unix)] +fn free(dir: &Path) -> Option { + let stat = rustix::fs::statvfs(dir).ok()?; + Some(stat.f_bavail.saturating_mul(stat.f_frsize)) +} + +#[cfg(windows)] +fn free(dir: &Path) -> Option { + use std::os::windows::ffi::OsStrExt; + let wide: Vec = dir.as_os_str().encode_wide().chain([0]).collect(); + let mut available = 0u64; + // SAFETY: `wide` is NUL-terminated and outlives the call, and the one + // out-pointer is a valid u64. + let ok = unsafe { + windows_sys::Win32::Storage::FileSystem::GetDiskFreeSpaceExW( + wide.as_ptr(), + &mut available, + std::ptr::null_mut(), + std::ptr::null_mut(), + ) + }; + (ok != 0).then_some(available) +} + +#[cfg(not(any(unix, windows)))] +fn free(_dir: &Path) -> Option { + None +} + +#[cfg(test)] +mod tests { + use super::*; + + /// Every name the recorder, a repair, and a clip give is read back as the + /// recording or clip it belongs to, and another show's, or a stranger's, + /// is not. + #[test] + fn names_read_back() { + let stamp = "2026-11-03T12-00-00Z"; + let program = |repaired| File::Recording { + stamp: stamp.into(), + source: None, + repaired, + }; + let source = |name: &str, repaired| File::Recording { + stamp: stamp.into(), + source: Some(name.into()), + repaired, + }; + let cases = [ + (format!("show {stamp}.mp4"), Some(program(false))), + (format!("show {stamp} (2).mp4"), Some(program(false))), + (format!("show {stamp}.repaired.mp4"), Some(program(true))), + (format!("show {stamp}.repaired (3).mp4"), Some(program(true))), + (format!("show {stamp} (2).repaired.mp4"), Some(program(true))), + (format!("show {stamp} mic.mp4"), Some(source("mic", false))), + ( + format!("show {stamp} desk mic (2).mp4"), + Some(source("desk mic", false)), + ), + (format!("show {stamp} mic.repaired.mp4"), Some(source("mic", true))), + ("show-clip-3.mp4".into(), Some(File::Clip(3))), + ("show-clip-0.mp4".into(), None), + ("show-clip-x.mp4".into(), None), + (format!("show 2 {stamp}.mp4"), None), + (format!("showtime {stamp}.mp4"), None), + (format!("show {stamp}.mp4.unfinished"), None), + ("show notes.mp4".into(), None), + ]; + for (name, file) in cases { + assert_eq!(parse("show", &name), file, "{name}"); + } + } + + /// The folder lists each recording once with every file's size and its + /// program files' length together, marks one repaired after a crash, and + /// lists clips on their own, newest first. + #[test] + fn the_folder_lists_cards() { + let dir = tempfile::tempdir().unwrap(); + let stamp = "2026-11-03T12-00-00Z"; + let write = |name: &str, seconds: u64, extra: usize| { + let path = dir.path().join(name); + std::fs::write(&path, mp4(seconds, extra)).unwrap(); + path + }; + write(&format!("show {stamp}.mp4"), 60, 0); + write(&format!("show {stamp} (2).mp4"), 30, 0); + write(&format!("show {stamp} mic.mp4"), 90, 100); + write(&format!("show {stamp} mic (2).mp4"), 90, 100); + let crashed = "2026-11-04T12-00-00Z"; + write(&format!("show {crashed}.repaired.mp4"), 10, 0); + std::fs::write(dir.path().join(format!("show {crashed}.mp4")), b"a fragment").unwrap(); + let clip = write("show-clip-1.mp4", 8, 0); + write(&format!("other {stamp}.mp4"), 1, 0); + // The clip is the newest. + let later = kio::clock::wall() + Duration::from_secs(60); + std::fs::File::options() + .write(true) + .open(&clip) + .unwrap() + .set_modified(later.into()) + .unwrap(); + + let (listing, lengths) = list(dir.path(), "show", BTreeMap::new()); + assert!(listing.error.is_none()); + assert_eq!(lengths.len(), 7, "every file of the show's is remembered"); + let kinds: Vec<_> = listing.cards.iter().map(|card| (&card.kind, card.length)).collect(); + assert_eq!(kinds[0], (&Kind::Clip(1), Some(Duration::from_secs(8)))); + let recording = listing + .cards + .iter() + .find(|card| matches!(&card.kind, Kind::Recording { stamp: theirs, .. } if theirs == stamp)) + .unwrap(); + assert_eq!( + recording.kind, + Kind::Recording { + stamp: stamp.into(), + sources: 1, + repaired: false + } + ); + assert_eq!(recording.length, Some(Duration::from_secs(90))); + let sizes: u64 = ["", " (2)", " mic", " mic (2)"] + .iter() + .map(|tail| { + std::fs::metadata(dir.path().join(format!("show {stamp}{tail}.mp4"))) + .unwrap() + .len() + }) + .sum(); + assert_eq!(recording.bytes, sizes); + assert_eq!(recording.path, dir.path().join(format!("show {stamp}.mp4"))); + let crash = listing + .cards + .iter() + .find(|card| matches!(&card.kind, Kind::Recording { stamp: theirs, .. } if theirs == crashed)) + .unwrap(); + assert!(matches!(crash.kind, Kind::Recording { repaired: true, .. })); + assert_eq!( + crash.length, + Some(Duration::from_secs(10)), + "the fragment adds no length" + ); + assert_eq!(crash.path, dir.path().join(format!("show {crashed}.repaired.mp4"))); + assert_eq!(listing.cards.len(), 3); + } + + /// A regular MP4's boxes, holding `seconds` in its movie header after + /// `extra` bytes of media. + fn mp4(seconds: u64, extra: usize) -> Vec { + use mp4_atom::Encode; + let mut out = Vec::new(); + mp4_atom::Ftyp { + major_brand: b"isom".into(), + minor_version: 0, + compatible_brands: vec![b"isom".into()], + } + .encode(&mut out) + .unwrap(); + mp4_atom::Mdat { data: vec![0; extra] }.encode(&mut out).unwrap(); + mp4_atom::Moov { + mvhd: mp4_atom::Mvhd { + timescale: 1000, + duration: seconds * 1000, + ..Default::default() + }, + ..Default::default() + } + .encode(&mut out) + .unwrap(); + out + } +} diff --git a/rs/moq-studio/src/shots.rs b/rs/moq-studio/src/shots.rs index 9e7dabf0..ac5d4dba 100644 --- a/rs/moq-studio/src/shots.rs +++ b/rs/moq-studio/src/shots.rs @@ -15,6 +15,7 @@ use std::time::Duration; use eframe::egui_wgpu; use egui_kittest::Harness; use egui_kittest::kittest::Queryable; +use moq_scene::Source; use moq_scene::source::Status; use crate::fx::Effects; @@ -66,6 +67,13 @@ enum View { /// The LIVE page after a mute, with the health dot's popover open and the /// pointer on undo, which names the mute. Hover, + /// REVIEW, with this show's recordings and clips, and if `picked`, the + /// last 30 seconds held and a stretch picked. With `replays`, the show + /// has two replay sources to pick between. + Review { + picked: bool, + replays: bool, + }, /// A page in Studio mode, with `Interview` cued. Cued(Page), /// A scene, by name, open for editing off air on BUILD. @@ -164,7 +172,18 @@ views! { /// MIX in a stacked window, its live picture as tall as the sound's slot /// allows. mix_compact: Still, Open::Studio { view: View::Page(Page::Mix), size: HALF, show: "show.toml" }; - review: Still, studio(View::Page(Page::Review)); + /// REVIEW off air: without a relay the program keeps no past, which the + /// strip and both actions say, beside this show's recordings and clips. + review: Still, studio(View::Review { picked: false, replays: false }); + /// REVIEW with the last 30 seconds held: a stretch picked between the + /// handles, the head inside it, the picture under the head large, and + /// Replay saying why it cannot play without a replay source. + review_picked: Still, studio(View::Review { picked: true, replays: false }); + /// REVIEW on a show with two replay sources: which one Replay plays in is + /// picked under the speed, in place of its note. + review_into: Still, studio(View::Review { picked: true, replays: true }); + /// Settings' Recording tab: the folder, and what a recording holds. + recording: Still, studio(View::Setting(Setting::Recording)); /// Settings' Advanced tab: the scene graph. advanced: Still, studio(View::Setting(Setting::Advanced)); /// The edit history, open under undo. @@ -529,8 +548,12 @@ fn shoot(open: &Open, gpu: &engine::Gpu, runtime: &tokio::runtime::Runtime, show setup::Setup::new(launch(gpu), "show.toml".into(), runtime.handle().clone()).unwrap(), )), Open::Kit { .. } => unreachable!("drawn without an engine"), - Open::Studio { show, .. } => { - std::fs::copy(shows.join(show), "show.toml").unwrap(); + Open::Studio { show, view, .. } => { + let mut text = std::fs::read_to_string(shows.join(show)).unwrap(); + if let View::Review { replays: true, .. } = view { + text.push_str("\n[sources.replay]\nkind = \"replay\"\n\n[sources.slowmo]\nkind = \"replay\"\n"); + } + std::fs::write("show.toml", text).unwrap(); start(launch(gpu), runtime) } }; @@ -563,6 +586,15 @@ fn shoot(open: &Open, gpu: &engine::Gpu, runtime: &tokio::runtime::Runtime, show studio.turn(*page); } View::OffAir(scene) => studio.edit_off_air(scene), + View::Review { picked, .. } => { + studio.turn(Page::Review); + studio.seed_shelf(shelf()); + if *picked { + let secs = Duration::from_secs; + let now = kio::clock::Instant::now(); + studio.seed_past(past(now), now, (secs(14), secs(6)), secs(9)); + } + } } let engine = started.get().expect("the studio started").clone(); runtime @@ -672,6 +704,97 @@ fn hover(harness: &mut Harness<'_, Window>) { harness.run_steps(4); } +/// The last 30 seconds as of `now`: a keyframe every 2.5 s, a sun setting +/// across a gradient, drawn rather than decoded so the view is the same on +/// every machine. +fn past(now: kio::clock::Instant) -> crate::past::View { + let (width, height) = (480usize, 270usize); + let pictures = (0..12u64) + .map(|index| { + let mut rgba = vec![0u8; width * height * 4]; + let sun = (60.0 + index as f32 * 30.0, 70.0 + index as f32 * 6.0); + for (at, pixel) in rgba.as_chunks_mut::<4>().0.iter_mut().enumerate() { + let (x, y) = ((at % width) as f32, (at / width) as f32); + let sky = y / height as f32; + let mut color = [ + (40.0 + 160.0 * sky) as u8, + (20.0 + 40.0 * sky) as u8, + (90.0 - 40.0 * sky) as u8, + ]; + if (x - sun.0).powi(2) + (y - sun.1).powi(2) < 45.0_f32.powi(2) { + color = [255, 170, 80]; + } + pixel.copy_from_slice(&[color[0], color[1], color[2], 255]); + } + Arc::new(crate::past::Picture { + id: index + 1, + at: now.saturating_sub(Duration::from_millis(29_000 - index * 2_500)), + size: [width, height], + rgba, + }) + }) + .collect::>(); + // A packet each 20 ms, swelling and fading as speech does. + let sound = (0..1450u64) + .map(|index| { + let at = now.saturating_sub(Duration::from_millis(29_000 - index * 20)); + let t = index as f32 / 50.0; + let peak = ((t * 1.3).sin() * (t * 0.17).cos() * (t * 0.049 + 1.0).sin()).abs(); + (at, peak) + }) + .collect(); + crate::past::View { + of: crate::replay::Of::Program, + held: Some(Ok(crate::past::Held { + oldest: pictures[0].at, + video: true, + })), + pictures, + sound: Ok(sound), + } +} + +/// This show's recordings and clips, as the mock-up lists them. +fn shelf() -> crate::shelf::Listing { + use crate::shelf::{Card, Kind, Listing}; + let wall = kio::clock::wall(); + let ago = |seconds: u64| Some(wall - Duration::from_secs(seconds)); + let length = |seconds: u64| Some(Duration::from_secs(seconds)); + let recording = |sources, repaired| Kind::Recording { + stamp: String::new(), + sources, + repaired, + }; + let card = |kind, modified, bytes, length| Card { + kind, + path: "show.mp4".into(), + modified, + bytes, + length, + }; + Listing { + cards: vec![ + card(Kind::Clip(3), ago(150), 11_000_000, length(8)), + card(Kind::Clip(2), ago(41 * 60), 18_000_000, length(12)), + card( + recording(0, true), + ago(2 * 86_400 + 600), + 14_200_000_000, + length(2 * 3600 + 10 * 60 + 44), + ), + card( + recording(4, false), + ago(6 * 86_400 + 600), + 21_000_000_000, + length(3 * 3600 + 2 * 60 + 10), + ), + card(Kind::Clip(1), ago(6 * 86_400 + 900), 22_000_000, length(15)), + ], + error: None, + free: Some(212_000_000_000), + } +} + /// Draw until setup has finished its checks and proposes a scene. async fn checked(harness: &mut Harness<'_, Window>) { loop { @@ -702,8 +825,11 @@ async fn settle(harness: &mut Harness<'_, Window>, engine: &engine::Engine) { let rendered = engine.stats.rendered.load(Ordering::Relaxed); let (started, framerate) = { let state = engine.state.lock().unwrap(); + // A replay source's track starts with its first replay, so one that + // idles has nothing to wait for. let started = state.sources.iter().all(|(_, feed)| { - !matches!(feed.status(), Status::Starting) && !matches!(feed.track(), Some(Status::Starting)) + !matches!(feed.status(), Status::Starting) + && (matches!(feed.source(), Source::Replay) || !matches!(feed.track(), Some(Status::Starting))) }); (started, u64::from(state.compositor.canvas().framerate)) }; diff --git a/rs/moq-studio/src/ui.rs b/rs/moq-studio/src/ui.rs index c1ab40d9..c6f1aa00 100644 --- a/rs/moq-studio/src/ui.rs +++ b/rs/moq-studio/src/ui.rs @@ -90,6 +90,8 @@ pub(crate) enum Setting { /// The canvas, each scene's size, the encoders, and the virtual cameras. #[default] Show, + /// The folder, and what a recording holds. + Recording, Hotkeys, Effects, /// The scene graph: every source, scene, and audio scene, wired. @@ -97,11 +99,18 @@ pub(crate) enum Setting { } impl Setting { - const ALL: [Self; 4] = [Self::Show, Self::Hotkeys, Self::Effects, Self::Advanced]; + const ALL: [Self; 5] = [ + Self::Show, + Self::Recording, + Self::Hotkeys, + Self::Effects, + Self::Advanced, + ]; fn name(self) -> &'static str { match self { Self::Show => "Show", + Self::Recording => "Recording", Self::Hotkeys => "Hotkeys", Self::Effects => "Effects", Self::Advanced => "Advanced", @@ -169,6 +178,7 @@ impl Studio { ) }; let controls = Arc::new(Controls::new(engine.clone(), runtime.clone())); + let review = review::Review::new(&engine, &runtime); let mut panels = Panels { engine, runtime, @@ -221,6 +231,7 @@ impl Studio { reloads: 0, levels: Levels::default(), graph: graph::View::default(), + review, }; if dropped > 0 { panels.report(Err(anyhow::anyhow!(dropped_message(dropped)))); @@ -246,6 +257,26 @@ impl Studio { self.panels.details = true; } + /// Show `listing` as REVIEW's cards, as if the folder held them. + pub(crate) fn seed_shelf(&mut self, listing: crate::shelf::Listing) { + self.panels.review.seed_shelf(listing); + } + + /// Show `view` as REVIEW's strip, as if the window held it, still at + /// `now`, with the stretch `picked` ago, from and to, and the head `head` + /// ago. + pub(crate) fn seed_past( + &mut self, + view: crate::past::View, + now: kio::clock::Instant, + picked: (Duration, Duration), + head: Duration, + ) { + self.panels.review.seed_past(view, now, head); + self.panels.controls.replay.lock().unwrap().stretch = + replay::Stretch::Between(now.saturating_sub(picked.0), now.saturating_sub(picked.1)); + } + /// Turn Studio mode on or off, as its switch would. pub(crate) fn studio_mode(&mut self, on: bool) { self.panels.controls.set_studio(on); @@ -521,6 +552,8 @@ struct Panels { /// Each sound's and audio scene's recent levels, for the meters. levels: Levels, graph: graph::View, + /// The REVIEW page's strip, cards, and what is dragged on it. + review: review::Review, } /// A layer, by the list it is in and its place there, back to front. diff --git a/rs/moq-studio/src/ui/review.rs b/rs/moq-studio/src/ui/review.rs index 45c5b5ea..13a9509b 100644 --- a/rs/moq-studio/src/ui/review.rs +++ b/rs/moq-studio/src/ui/review.rs @@ -1,255 +1,1349 @@ -//! The REVIEW page: replays, clips, and recordings. +//! The REVIEW page: the last 30 seconds as a strip to pick a stretch from, +//! what to replay and how fast, Replay and Save clip, and beside them this +//! show's recordings and clips. + +use egui::{Align2, CursorIcon, Rect, Sense, Stroke, UiBuilder, Vec2, pos2, vec2}; +use kio::clock::Instant; use super::*; +use crate::controls; +use crate::past::{self, Past}; +use crate::shelf::{self, Shelf}; -impl Panels { - /// Start and stop recording, and see what the recording wrote. - pub(super) fn record(&mut self, ui: &mut Ui) { - let engine = self.engine.clone(); +/// How wide the recordings column is, at most. +const SHELF: f32 = 384.0; +/// How wide the "replay what" column is. +const WHAT: f32 = 160.0; +/// How tall a card in it is. +const WHAT_CARD: f32 = 90.0; +/// How tall the Replay and Save clip buttons are. +const ACTION: f32 = 70.0; +/// How far from a handle a press still grabs it, in points. +const GRAB: f32 = 12.0; +/// How near the strip's right edge a dragged end pins the stretch to now. +const PIN: f32 = 8.0; +/// How tall the sound under the strip's pictures is, and how wide each bar. +const WAVE: f32 = 28.0; +const BAR: f32 = 6.0; + +/// The page's own state: the strip, the cards, and what a drag holds. +pub(super) struct Review { + past: Past, + shelf: Shelf, + /// Each picture on screen, uploaded once, by its id. + textures: BTreeMap, + /// Where the scrub head is, or `None` at the stretch's end. + head: Option, + /// What a drag on the strip moves. + grip: Option, + /// The last outcome of a replay or clip shown, so each shows once. + outcome: Option>, + /// The moment the page draws as now, held still for the window's views. + frozen: Option, +} + +/// What a drag on the strip moves. +#[derive(Clone, Copy, Debug, PartialEq)] +enum Grip { + /// The stretch's start or end. + From, + To, + /// The scrub head. + Head, +} + +impl Review { + pub(super) fn new(engine: &Arc, runtime: &tokio::runtime::Handle) -> Self { let recorder = &engine.recorder; - let status = recorder.status(); - ui.horizontal(|ui| match status.as_ref().filter(|status| !status.done) { - Some(status) => { - if ui - .add_enabled(!status.stopping, egui::Button::new("Stop recording")) - .clicked() - { - recorder.stop(); - } - if status.stopping { - ui.label("finishing"); - } else { - let elapsed = kio::clock::now().saturating_duration_since(status.started).as_secs(); - let elapsed = format!("{}:{:02}:{:02}", elapsed / 3600, elapsed / 60 % 60, elapsed % 60); - ui.colored_label(kit::ACCENT, format!("recording {elapsed}")); - } + Self { + past: Past::new(engine.clone()), + shelf: Shelf::new(recorder.dir(), recorder.show(), runtime.clone()), + textures: BTreeMap::new(), + head: None, + grip: None, + outcome: None, + frozen: None, + } + } + + /// Show `listing` as the cards, for the window's views. + #[cfg(test)] + pub(super) fn seed_shelf(&mut self, listing: shelf::Listing) { + self.shelf.seed(listing); + } + + /// Show `view` as the strip, held still at `now`, with the head `head` + /// ago, for the window's views. + #[cfg(test)] + pub(super) fn seed_past(&mut self, view: past::View, now: Instant, head: Duration) { + self.frozen = Some(now); + self.head = Some(now.saturating_sub(head)); + self.past.seed(view); + } + + fn now(&self) -> Instant { + self.frozen.unwrap_or_else(Instant::now) + } + + /// The picture shown at or before `at`, or the first after it when none was. + fn picture_at<'a>(&self, view: &'a past::View, at: Instant) -> Option<&'a past::Picture> { + let before = view.pictures.iter().rev().find(|picture| picture.at <= at); + before.or(view.pictures.first()).map(|picture| &**picture) + } + + /// Upload each picture not on the GPU yet, and let go of those gone. + fn upload(&mut self, ctx: &egui::Context, view: &past::View) { + let mut kept = BTreeMap::new(); + for picture in &view.pictures { + let texture = self.textures.remove(&picture.id).unwrap_or_else(|| { + let image = egui::ColorImage::from_rgba_unmultiplied(picture.size, &picture.rgba); + ctx.load_texture(format!("past-{}", picture.id), image, egui::TextureOptions::LINEAR) + }); + kept.insert(picture.id, texture); + } + self.textures = kept; + } + + fn texture(&self, picture: &past::Picture) -> Option { + let texture = self.textures.get(&picture.id)?; + let size = vec2(picture.size[0] as f32, picture.size[1] as f32); + Some(egui::load::SizedTexture::new(texture.id(), size)) + } +} + +/// Why REVIEW's picks cannot play or save now, if they cannot. +struct Refusals { + replay: Option, + save: Option, +} + +impl Panels { + /// The page in `stage`: the strip and its actions, and the recordings + /// column on the right. `shape` is the program's, width over height. + pub(super) fn review(&mut self, ui: &mut Ui, fx: &mut Fx, stage: Rect, shape: f32) { + let side = SHELF.min(stage.width() * 0.34); + let shelf = stage.with_min_x(stage.right() - side); + let main = stage.with_max_x(shelf.left()); + kit::wash(ui.painter(), main); + shell::bar(ui, fx, shelf); + ui.painter() + .vline(shelf.left() + 0.5, shelf.y_range(), Stroke::new(1.0, kit::LINE)); + self.replays(ui, main.shrink2(vec2(24.0, 14.0)), shape); + self.shelf(ui, shelf.shrink2(vec2(16.0, 14.0))); + } + + /// The large picture and what to replay, the strip, the speed, and the + /// two actions, top to bottom in `rect`. + fn replays(&mut self, ui: &mut Ui, rect: Rect, shape: f32) { + let replayable = self.replayable(); + let controls = self.controls.clone(); + let mut replay = controls.replay.lock().unwrap(); + if let Of::Source(name) = &replay.of + && !replayable.iter().any(|(theirs, _)| *theirs == *name) + { + replay.of = Of::Program; + } + let mut view = self.review.past.watch(&replay.of); + // Until the thread samples a new pick, nothing of the old one shows. + if view.of != replay.of { + view = Arc::new(past::View { + of: replay.of.clone(), + ..past::View::default() + }); + } + self.review.upload(ui.ctx(), &view); + ui.ctx().request_repaint_after(Duration::from_millis(250)); + + let actions = Rect::from_min_max(pos2(rect.left(), rect.bottom() - ACTION), rect.max); + let strip_height = 22.0 + (rect.width() / 6.0 * 9.0 / 16.0).clamp(48.0, 96.0) + 6.0 + WAVE + 50.0; + let strip = Rect::from_min_max( + pos2(rect.left(), actions.top() - 18.0 - strip_height), + pos2(rect.right(), actions.top() - 18.0), + ); + let top = Rect::from_min_max(rect.min, pos2(rect.right(), strip.top() - 14.0)); + let what = Rect::from_min_max(pos2(top.right() - WHAT, top.top()), top.max); + let picture = super::layout::fit(top.with_max_x(what.left() - 18.0).shrink(6.0), shape); + + self.large(ui, &view, picture, &replay); + self.what(ui, what, &replayable, &mut replay); + self.past_strip(ui, strip, &view, &mut replay); + let sound_only = view + .held + .as_ref() + .and_then(|held| held.as_ref().ok()) + .is_some_and(|held| !held.video) + || replayable + .iter() + .any(|(name, sound)| *sound && matches!(&replay.of, Of::Source(of) if of == name)); + let refusals = self.refusals(&view, &replay, sound_only); + let fired = self.actions(ui, actions, &mut replay, &refusals); + // How a replay or clip went, once, as the window's message. + let outcome = replay.outcome.lock().unwrap().clone(); + drop(replay); + if outcome != self.review.outcome { + self.review.outcome = outcome.clone(); + if let Some(outcome) = outcome { + self.message = Some((outcome, kio::clock::Instant::now())); } - None => { - let mut each = self.controls.record_sources.load(Ordering::Relaxed); - if ui.button("Start recording").clicked() { - let result = recorder.start(each); - self.check(result); - } - if ui.add(kit::Toggle::new(&mut each).label("each source too")).changed() { - self.controls.record_sources.store(each, Ordering::Relaxed); + } + if let Some(action) = fired { + let _ = controls.fire(&action); + } + } + + /// Each source with a past to replay, and whether it is sound alone. + fn replayable(&self) -> Vec<(String, bool)> { + let state = self.engine.state.lock().unwrap(); + let collection = state.collection(); + // What a source shows or plays, a clone its source's, has a past only + // if something encodes it: not an image, a color, a web page, an + // overlay, a LUT, or a scene or the program drawn into another. + collection + .sources + .keys() + .filter_map(|name| { + let captured = collection.sources.get(collection.capture(name))?; + match captured { + Source::Image { .. } + | Source::Color { .. } + | Source::Browser { .. } + | Source::Overlay { .. } + | Source::Lut { .. } + | Source::Scene { .. } + | Source::Program + | Source::Replay => None, + Source::Microphone { .. } | Source::SystemAudio => Some((name.clone(), true)), + _ => Some((name.clone(), false)), } - ui.weak(format!("into {}", recorder.dir().display())); + }) + .collect() + } + + /// The picture under the scrub head, large, with how long ago it was. + fn large(&mut self, ui: &mut Ui, view: &past::View, rect: Rect, replay: &controls::Replay) { + let painter = ui.painter(); + painter.rect_filled(rect, 10, kit::P1); + painter.rect_stroke( + rect.expand(3.0), + 12, + Stroke::new(2.0, kit::LINE), + egui::StrokeKind::Outside, + ); + let now = self.review.now(); + let head = self.head(replay, now); + let picture = self.review.picture_at(view, head); + match (picture.and_then(|picture| self.review.texture(picture)), &view.held) { + (Some(texture), _) => { + egui::Image::new(texture) + .corner_radius(10) + .uv(cover(texture.size, rect.size())) + .paint_at(ui, rect); + } + (None, held) => { + let why = match held { + None => "Reading the last 30 seconds".to_string(), + Some(Err(why)) => capitalize(why), + Some(Ok(held)) if !held.video => "Sound only: nothing to show".to_string(), + Some(Ok(_)) => "No pictures held".to_string(), + }; + let galley = ui.painter().layout(why, kit::ui(14.0), kit::DIM, rect.width() - 80.0); + ui.painter() + .galley(rect.center() - galley.size() / 2.0, galley, kit::DIM); } - }); - self.recorded_scene(ui); - self.recorded_audio(ui); - for file in status.iter().flat_map(|status| &status.files) { - let name = file.path.file_name().unwrap_or_default().to_string_lossy(); - let size = format!("{:.1} MB", file.written as f64 / 1e6); - match &file.finished { - None => ui.label(format!("{name}: {size}")), - Some(record::Finished::Saved) => ui.label(format!("{name}: saved, {size}")), - Some(record::Finished::Empty) => ui.weak(format!("{name}: nothing recorded, removed")), - Some(record::Finished::Failed(reason)) => { - ui.colored_label(ui.visuals().error_fg_color, format!("{name}: {reason}")) - } - }; } + let ago = now.saturating_duration_since(head); + let text = match ago < Duration::from_millis(500) { + true => "Now".to_string(), + false => format!("{} s ago", ago.as_secs_f32().round()), + }; + Tag::AGO.paint(ui.painter(), rect.left_top() + vec2(14.0, 14.0), &text); + } + + /// Where the scrub head is: where it was put, or the stretch's end. + fn head(&self, replay: &controls::Replay, now: Instant) -> Instant { + let (_, to) = ages(replay.stretch, now); + let oldest = now.saturating_sub(replay::WINDOW); + self.review.head.unwrap_or(now.saturating_sub(to)).max(oldest).min(now) } - /// Pick the picture recordings hold, the program or any scene, even while - /// recording, which goes on in a new file when the size changes. - pub(super) fn recorded_scene(&mut self, ui: &mut Ui) { - let engine = self.engine.clone(); - let mut state = engine.state.lock().unwrap(); - let current = state.collection().recorded().map(str::to_string); - let scenes = state.collection().scenes.iter(); - let names: Vec = scenes.map(|scene| scene.name.clone()).collect(); - let mut picked = current.clone(); - ui.horizontal(|ui| { - ui.label("Picture") - .on_hover_text("A scene of another size than the program's goes on in a new file"); - egui::ComboBox::from_id_salt("recorded scene") - .selected_text(picked.as_deref().unwrap_or("the program")) - .show_ui(ui, |ui| { - ui.selectable_value(&mut picked, None, "the program"); - for name in &names { - ui.selectable_value(&mut picked, Some(name.clone()), name); + /// What to replay: the program's live picture, then each source with a + /// past, by its live picture or, for sound alone, its name. + fn what(&mut self, ui: &mut Ui, rect: Rect, replayable: &[(String, bool)], replay: &mut controls::Replay) { + let caption = Rect::from_min_size(rect.min, vec2(rect.width(), 18.0)); + caption_text(ui.painter(), caption.left_center(), "Replay what", Align2::LEFT_CENTER); + let room = rect.with_min_y(caption.bottom() + 4.0); + ui.scope_builder(UiBuilder::new().max_rect(room), |ui| { + egui::ScrollArea::vertical() + .id_salt("replay what") + .auto_shrink(false) + .show(ui, |ui| { + ui.spacing_mut().item_spacing.y = 8.0; + let program = egui::load::SizedTexture::new( + self.program.id, + vec2(self.program_drawn.width as f32, self.program_drawn.height as f32), + ); + let card = What { + name: "Program", + picture: Some(program), + sound: false, + on: replay.of == Of::Program, + air: true, + }; + if card.show(ui).clicked() { + replay.of = Of::Program; + } + for (name, sound) in replayable { + let thumbnail = Thumbnail::Source(name.clone()); + let picture = match sound { + true => None, + false => self + .thumbnails + .get(&thumbnail) + .map(|registered| egui::load::SizedTexture::new(registered.id, vec2(16.0, 9.0))), + }; + let card = What { + name, + picture, + sound: *sound, + on: matches!(&replay.of, Of::Source(of) if of == name), + air: false, + }; + let response = card.show(ui); + if !sound && ui.is_rect_visible(response.rect) { + self.thumbnails_shown.insert(thumbnail); + } + if response.clicked() { + replay.of = Of::Source(name.clone()); + } } }); }); - if picked == current { - return; + } +} + +/// A card in the "replay what" column. +struct What<'a> { + name: &'a str, + /// Its live picture, or `None` for sound alone, or before one is drawn. + picture: Option, + /// Whether it is sound alone, which replays only at 100%. + sound: bool, + /// Whether it is picked. + on: bool, + /// Whether it is the program, tagged ON AIR. + air: bool, +} + +impl What<'_> { + /// Draw the card, as wide as `ui`. + fn show(self, ui: &mut Ui) -> egui::Response { + let Self { + name, + picture, + sound, + on, + air, + } = self; + let (rect, response) = ui.allocate_exact_size(vec2(ui.available_width(), WHAT_CARD), Sense::click()); + response.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::Button, true, on, name)); + let response = response.on_hover_cursor(CursorIcon::PointingHand); + let painter = ui.painter(); + painter.rect_filled(rect, 9, kit::P2); + match picture { + Some(picture) => { + egui::Image::new(picture) + .corner_radius(9) + .uv(cover(picture.size, rect.size())) + .paint_at(ui, rect); + let fade = Rect::from_min_max(pos2(rect.left(), rect.bottom() - 30.0), rect.max); + let mut mesh = egui::Mesh::default(); + let clear = Color32::TRANSPARENT; + let dark = Color32::from_black_alpha(210); + mesh.colored_vertex(fade.left_top(), clear); + mesh.colored_vertex(fade.right_top(), clear); + mesh.colored_vertex(fade.right_bottom(), dark); + mesh.colored_vertex(fade.left_bottom(), dark); + mesh.add_triangle(0, 1, 2); + mesh.add_triangle(0, 2, 3); + ui.painter().add(mesh); + } + None if sound => { + Icon::Mic.paint(ui, rect.translate(vec2(0.0, -8.0)), 18.0, kit::DIM); + let note = ui.painter().layout_no_wrap("100% only".into(), kit::ui(10.5), kit::DIM); + ui.painter() + .galley(rect.right_bottom() - vec2(8.0, 7.0) - note.size(), note, kit::DIM); + } + None => {} + } + let painter = ui.painter(); + let ink = if sound { kit::DIM } else { kit::TEXT }; + let label = kit::caps_galley(painter, name, kit::display_bold(13.0), ink, Some(rect.width() - 60.0)); + painter.galley( + pos2(rect.left() + 8.0, rect.bottom() - 6.0 - label.size().y), + label, + ink, + ); + if air { + Tag::AIR.paint(painter, rect.left_top() + vec2(6.0, 6.0), "On air"); } - let label = match &picked { - Some(name) => format!("Record scene `{name}`"), - None => "Record the program".into(), + let (width, color) = match (on, response.hovered()) { + (true, _) => (3.0, kit::TEXT), + (false, true) => (1.0, kit::DIM), + (false, false) => (1.0, kit::LINE), }; - let result = edit(&mut state, label, None, |draft| draft.set_recorded(picked.as_deref())); - drop(state); - self.check(result); - } - - /// Pick the audio scene recordings hold, even while recording, which goes - /// on with the new one from its next block. - pub(super) fn recorded_audio(&mut self, ui: &mut Ui) { - let engine = self.engine.clone(); - let mut state = engine.state.lock().unwrap(); - let scenes = state.collection().audio_scenes.clone(); - let program = state.collection().program_audio().unwrap_or_default(); - let on_program = format!("on program ({program})"); - let current = scenes - .iter() - .find(|scene| scene.recorded) - .map(|scene| scene.name.clone()); - let mut picked = current.clone(); - ui.horizontal(|ui| { - ui.label("Audio") - .on_hover_text("An audio scene nothing else carries stays off the relay"); - egui::ComboBox::from_id_salt("recorded audio") - .selected_text(picked.as_deref().unwrap_or(&on_program)) - .show_ui(ui, |ui| { - ui.selectable_value(&mut picked, None, &on_program); - for scene in &scenes { - ui.selectable_value(&mut picked, Some(scene.name.clone()), &scene.name); - } - }); - }); - if picked == current { - return; + painter.rect_stroke(rect, 9, Stroke::new(width, color), egui::StrokeKind::Inside); + response + } +} + +impl Panels { + /// The last 30 seconds, with the stretch picked between amber handles and + /// the scrub head, in `rect`. + fn past_strip(&mut self, ui: &mut Ui, rect: Rect, view: &past::View, replay: &mut controls::Replay) { + let now = self.review.now(); + let header = Rect::from_min_size(rect.min, vec2(rect.width(), 16.0)); + caption_text( + ui.painter(), + header.left_center(), + "The last 30 seconds", + Align2::LEFT_CENTER, + ); + let thumbs = Rect::from_min_max( + pos2(rect.left() + 2.0, header.bottom() + 12.0), + pos2(rect.right() - 2.0, rect.bottom() - 40.0 - WAVE - 6.0), + ); + let wave = Rect::from_min_max( + pos2(thumbs.left(), thumbs.bottom() + 6.0), + pos2(thumbs.right(), thumbs.bottom() + 6.0 + WAVE), + ); + let track = thumbs.union(wave); + let window = replay::WINDOW.as_secs_f32(); + let x_at = |age: Duration| thumbs.left() + (1.0 - age.as_secs_f32() / window).clamp(0.0, 1.0) * thumbs.width(); + let age_at = |x: f32| { + Duration::from_secs_f32(((1.0 - (x - thumbs.left()) / thumbs.width()) * window).clamp(0.0, window)) + }; + + // A slot a picture wide, each showing the picture shown at its middle. + let painter = ui.painter().with_clip_rect(thumbs); + painter.rect_filled(thumbs, 8, kit::P2); + let slots = (thumbs.width() / (thumbs.height() * 16.0 / 9.0)).ceil().max(1.0) as usize; + let slot = thumbs.width() / slots as f32; + let oldest = view + .held + .as_ref() + .and_then(|held| held.as_ref().ok()) + .map(|held| held.oldest); + for index in 0..slots { + let cell = Rect::from_min_size( + pos2(thumbs.left() + index as f32 * slot, thumbs.top()), + vec2(slot, thumbs.height()), + ); + let middle = now.saturating_sub(age_at(cell.center().x)); + if oldest.is_none_or(|oldest| middle < oldest) { + continue; + } + let Some(texture) = self + .review + .picture_at(view, middle) + .and_then(|picture| self.review.texture(picture)) + else { + continue; + }; + let inner = cell.shrink2(vec2(1.0, 0.0)); + painter.image(texture.id, inner, cover(texture.size, inner.size()), Color32::WHITE); + } + // What the window does not hold, such as before the source was on + // program, is darker still. + if let Some(oldest) = oldest { + let held_from = x_at(now.saturating_duration_since(oldest)); + if held_from > thumbs.left() + 1.0 { + let gone = thumbs.with_max_x(held_from); + painter.rect_filled(gone, 0, Color32::from_black_alpha(140)); + } + } + // The stretch picked, and the head. + let (from, to) = ages(replay.stretch, now); + let (from_x, to_x) = (x_at(from), x_at(to)); + let shade = Color32::from_rgba_unmultiplied(8, 8, 12, 153); + painter.rect_filled(thumbs.with_max_x(from_x), 0, shade); + painter.rect_filled(thumbs.with_min_x(to_x), 0, shade); + // The sound, a bar each few points for the loudest moment under it, + // bright inside the stretch. + if let Ok(sound) = &view.sound + && !sound.is_empty() + { + let bars = (wave.width() / BAR).floor() as usize; + for index in 0..bars { + let left = wave.left() + index as f32 * BAR; + let (newer, older) = (now.saturating_sub(age_at(left + BAR)), now.saturating_sub(age_at(left))); + let peak = sound + .iter() + .filter(|(at, _)| (older..newer).contains(at)) + .map(|(_, peak)| *peak) + .fold(None, |loudest: Option, peak| { + Some(loudest.unwrap_or(0.0).max(peak)) + }); + let Some(peak) = peak else { continue }; + let height = 3.0 + peak.sqrt() * (WAVE - 3.0); + let bar = Rect::from_center_size(pos2(left + BAR / 2.0, wave.center().y), vec2(BAR - 2.0, height)); + let inside = bar.center().x > from_x && bar.center().x < to_x; + ui.painter() + .rect_filled(bar, 1, if inside { kit::TEXT } else { kit::FAINT }); + } + } + if let Err(why) = &view.sound { + let note = format!("No sound: {why}"); + ui.painter() + .text(wave.left_center(), Align2::LEFT_CENTER, note, kit::ui(11.0), kit::DIM); + } + let pick = Rect::from_min_max(pos2(from_x, thumbs.top() - 5.0), pos2(to_x, wave.bottom() + 5.0)); + let painter = ui.painter(); + painter.rect_stroke(pick, 9, Stroke::new(3.0, kit::CUE), egui::StrokeKind::Inside); + painter.add( + egui::Shadow { + offset: [0, 0], + blur: 26, + spread: 0, + color: kit::CUE.gamma_multiply(0.2), + } + .as_shape(pick, egui::CornerRadius::same(9)), + ); + for (x, left) in [(from_x, true), (to_x, false)] { + let handle = match left { + true => Rect::from_min_max(pick.left_top(), pos2(x + 15.0, pick.bottom())), + false => Rect::from_min_max(pos2(x - 15.0, pick.top()), pick.right_bottom()), + }; + let corners = match left { + true => egui::CornerRadius { + nw: 9, + sw: 9, + ne: 0, + se: 0, + }, + false => egui::CornerRadius { + nw: 0, + sw: 0, + ne: 9, + se: 9, + }, + }; + painter.rect_filled(handle, corners, kit::CUE); + let grip = Rect::from_center_size(handle.center(), vec2(3.0, 26.0)); + painter.rect_filled(grip, 2, kit::ON_CUE); } - let label = match &picked { - Some(name) => format!("Record audio scene `{name}`"), - None => "Record the audio scene on program".into(), + let head = self.head(replay, now); + let head_x = x_at(now.saturating_duration_since(head)); + painter.vline( + head_x, + thumbs.top() - 12.0..=wave.bottom() + 5.0, + Stroke::new(2.0, Color32::WHITE), + ); + painter.rect_filled( + Rect::from_center_size(pos2(head_x, thumbs.top() - 9.0), vec2(14.0, 10.0)), + egui::CornerRadius { + nw: 3, + ne: 3, + sw: 6, + se: 6, + }, + Color32::WHITE, + ); + // Why there are no pictures, over everything. + let note = match &view.held { + None => Some("Reading the last 30 seconds".to_string()), + Some(Err(why)) => Some(capitalize(why)), + Some(Ok(held)) if !held.video => Some("Sound only".to_string()), + Some(Ok(_)) => None, }; - let mut next = scenes; - for scene in &mut next { - scene.recorded = picked.as_deref() == Some(scene.name.as_str()); + if let Some(note) = note { + let galley = painter.layout(note, kit::ui(13.0), kit::TEXT, thumbs.width() - 80.0); + let back = Rect::from_center_size(thumbs.center(), galley.size() + vec2(24.0, 12.0)); + painter.rect_filled(back, 8, kit::P1.gamma_multiply(0.92)); + painter.galley(back.center() - galley.size() / 2.0, galley, kit::TEXT); + } + + // How long it is, and when it starts and ends. + let length = from.saturating_sub(to); + let picked = painter.layout_no_wrap( + format!("{:.1} s picked", length.as_secs_f32()), + kit::mono(11.5), + kit::DIM, + ); + painter.galley( + pos2( + header.right() - picked.size().x, + header.center().y - picked.size().y / 2.0, + ), + picked, + kit::DIM, + ); + let say = |age: Duration| match age < Duration::from_millis(250) { + true => "now".to_string(), + false => format!("{} s ago", age.as_secs_f32().round()), + }; + let under = wave.bottom() + 9.0; + let from_label = painter.layout_no_wrap(say(from), kit::mono(11.0), kit::CUE); + let to_label = painter.layout_no_wrap(say(to), kit::mono(11.0), kit::CUE); + let from_at = (from_x) + .min(to_x - to_label.size().x - from_label.size().x - 8.0) + .max(thumbs.left()); + let to_at = (to_x - to_label.size().x) + .max(from_at + from_label.size().x + 8.0) + .min(thumbs.right() - to_label.size().x); + let marks = [(from_at, from_label.size().x), (to_at, to_label.size().x)]; + painter.galley(pos2(from_at, under), from_label, kit::CUE); + painter.galley(pos2(to_at, under), to_label, kit::CUE); + // The ruler, where the stretch's own labels leave room. + let ruler = under + 18.0; + for (age, text) in [(30, "30 s ago"), (20, "20"), (10, "10"), (0, "now")] { + let galley = painter.layout_no_wrap(text.into(), kit::mono(10.5), kit::FAINT); + let x = x_at(Duration::from_secs(age)); + let x = match age { + 30 => x, + 0 => x - galley.size().x, + _ => x - galley.size().x / 2.0, + }; + let clear = marks + .iter() + .all(|(at, width)| x + galley.size().x < at - 6.0 || x > at + width + 6.0); + if clear { + painter.galley(pos2(x, ruler), galley, kit::FAINT); + } } - let result = edit(&mut state, label, None, |draft| draft.set_audio_scenes(next)); + + // A press near a handle moves it, anywhere else moves the head. + let response = ui.interact( + track.expand2(vec2(8.0, 12.0)), + ui.id().with("past strip"), + Sense::click_and_drag(), + ); + let pointer = response.interact_pointer_pos().or(response.hover_pos()); + if let Some(pointer) = pointer { + let over = grip_at(pointer.x, from_x, to_x); + if over.is_some() || self.review.grip.is_some_and(|grip| grip != Grip::Head) { + ui.ctx().set_cursor_icon(CursorIcon::ResizeHorizontal); + } + if response.drag_started() || response.clicked() { + self.review.grip = Some(over.unwrap_or(Grip::Head)); + } + } + if (response.dragged() || response.clicked()) + && let (Some(grip), Some(pointer)) = (self.review.grip, response.interact_pointer_pos()) + { + let age = age_at(pointer.x); + let min = replay::MIN_STRETCH; + match grip { + Grip::Head => self.review.head = Some(now.saturating_sub(age)), + Grip::From => { + let from = age.max(to + min).min(replay::WINDOW); + replay.stretch = match replay.stretch { + replay::Stretch::Last(_) => replay::Stretch::Last(from), + replay::Stretch::Between(_, end) => replay::Stretch::Between(now.saturating_sub(from), end), + }; + } + Grip::To => { + let to = age.min(from.saturating_sub(min)); + replay.stretch = match pointer.x >= thumbs.right() - PIN { + true => replay::Stretch::Last(from), + false => replay::Stretch::Between(now.saturating_sub(from), now.saturating_sub(to)), + }; + } + } + } + if response.drag_stopped() || response.clicked() { + self.review.grip = None; + } + } + + /// Why the picks cannot replay or save now. + fn refusals(&self, view: &past::View, replay: &controls::Replay, sound_only: bool) -> Refusals { + let mut refusals = Refusals { + replay: None, + save: None, + }; + let stretch = replay + .stretch + .span(self.review.now()) + .err() + .map(|err| capitalize(&format!("{err:#}"))); + let held = match &view.held { + Some(Err(why)) => Some(capitalize(why)), + _ => None, + }; + refusals.save = held.clone().or(stretch.clone()); + let state = self.engine.state.lock().unwrap(); + let into = state + .collection() + .sources + .iter() + .any(|(_, source)| matches!(source, Source::Replay)); drop(state); - self.check(result); + refusals.replay = held.or(stretch).or_else(|| match () { + _ if !into => Some("Add a replay source to play replays in".into()), + _ if sound_only && replay.speed < 1.0 => Some("Sound plays only at 100%".into()), + _ => None, + }); + refusals } - /// Replay the recent past of the program or a source in a replay source, - /// or save it as a clip. - pub(super) fn replay(&mut self, ui: &mut Ui) { - let (replayable, replays) = { - let state = self.engine.state.lock().unwrap(); - let named = |kind: fn(&Source) -> bool| -> Vec { - let sources = &state.collection().sources; - sources - .iter() - .filter(|(_, source)| kind(source)) - .map(|(name, _)| name.clone()) - .collect() + /// The speed, Replay, and Save clip, in `rect`, returning what was fired. + fn actions(&self, ui: &mut Ui, rect: Rect, replay: &mut controls::Replay, refusals: &Refusals) -> Option { + let speed = Rect::from_min_size(rect.min, vec2(200.0_f32.min(rect.width() * 0.25), rect.height())); + caption_text( + ui.painter(), + speed.left_top() + vec2(0.0, 6.0), + "Speed", + Align2::LEFT_CENTER, + ); + let segments = Rect::from_min_size(speed.left_top() + vec2(0.0, 18.0), vec2(speed.width(), 30.0)); + let each = segments.width() / replay::SPEEDS.len() as f32; + ui.painter().rect( + segments, + 10, + kit::P2, + Stroke::new(1.0, kit::LINE), + egui::StrokeKind::Inside, + ); + for (index, speed) in replay::SPEEDS.into_iter().enumerate() { + let cell = Rect::from_min_size( + segments.min + vec2(index as f32 * each, 0.0), + vec2(each, segments.height()), + ); + let label = format!("{:.0}%", speed * 100.0); + let response = ui.interact(cell, ui.id().with(("speed", index)), Sense::click()); + let on = replay.speed == speed; + response.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::Button, true, on, &label)); + if response.clicked() { + replay.speed = speed; + } + let ink = match on { + true => { + ui.painter().rect_filled(cell.shrink(1.0), 9, kit::TEXT); + Color32::from_rgb(0x11, 0x11, 0x11) + } + false if response.hovered() => kit::TEXT, + false => kit::DIM, }; - // What a source shows or plays, a clone its source's, has a past - // only if something encodes it: not an image, a color, a web page, - // an overlay, a LUT, or a scene or the program drawn into another. - let collection = state.collection(); - let replayable = collection - .sources - .keys() - .filter(|name| { - let captured = collection.sources.get(collection.capture(name)); - captured.is_some_and(|source| { - !matches!( - source, - Source::Image { .. } - | Source::Color { .. } | Source::Browser { .. } - | Source::Overlay { .. } | Source::Lut { .. } - | Source::Scene { .. } | Source::Program - ) - }) - }) - .cloned() - .collect::>(); - (replayable, named(|source| matches!(source, Source::Replay))) - }; - let controls = self.controls.clone(); - // Held while drawing, never with the engine's state, which a replay - // locks under it. - let mut guard = controls.replay.lock().unwrap(); - let panel = &mut *guard; - if panel.into.as_ref().is_none_or(|into| !replays.contains(into)) { - panel.into = replays.first().cloned(); - } - - egui::Grid::new("replay").show(ui, |ui| { - ui.label("Replay"); - egui::ComboBox::from_id_salt("replay of") - .selected_text(panel.of.to_string()) - .show_ui(ui, |ui| { - ui.selectable_value(&mut panel.of, Of::Program, Of::Program.to_string()); - for name in replayable { - let of = Of::Source(name); - let label = of.to_string(); - ui.selectable_value(&mut panel.of, of, label); - } - }); - ui.end_row(); - ui.label("Length"); - egui::ComboBox::from_id_salt("replay length") - .selected_text(format!("{} s", panel.length.as_secs())) - .show_ui(ui, |ui| { - for length in replay::LENGTHS { - ui.selectable_value(&mut panel.length, length, format!("{} s", length.as_secs())); - } - }); - ui.end_row(); - ui.label("Speed"); - egui::ComboBox::from_id_salt("replay speed") - .selected_text(format!("{:.0}%", panel.speed * 100.0)) - .show_ui(ui, |ui| { - for speed in replay::SPEEDS { - ui.selectable_value(&mut panel.speed, speed, format!("{:.0}%", speed * 100.0)); - } - }); - ui.end_row(); - ui.label("Play in"); - match &mut panel.into { - Some(into) => { + ui.painter() + .text(cell.center(), Align2::CENTER_CENTER, label, kit::ui_medium(12.5), ink); + } + // With more than one replay source, which one plays takes the note's + // row, and the Replay button's note says what the row said. + let replays = self.replay_sources(replay); + match (&mut replay.into, replays.len() > 1) { + (Some(into), true) => { + let row = Rect::from_min_size(segments.left_bottom() + vec2(0.0, 2.0), vec2(speed.width(), 20.0)); + ui.scope_builder(UiBuilder::new().max_rect(row), |ui| { egui::ComboBox::from_id_salt("replay into") - .selected_text(into.as_str()) + .selected_text(format!("Into {into}")) + .width(speed.width()) .show_ui(ui, |ui| { - for name in replays { - let label = name.clone(); - ui.selectable_value(into, name, label); + for name in &replays { + ui.selectable_value(into, name.clone(), name.as_str()); } }); + }); + } + _ => { + ui.painter().text( + segments.left_bottom() + vec2(0.0, 12.0), + Align2::LEFT_CENTER, + "No sound below 100%", + kit::ui(11.5), + kit::DIM, + ); + } + } + + let buttons = rect.with_min_x(speed.right() + 24.0); + let gap = 14.0; + let play_width = (buttons.width() - gap) * 1.25 / 2.25; + let play_rect = buttons.with_max_x(buttons.left() + play_width); + let save_rect = buttons.with_min_x(play_rect.right() + gap); + let (_, length) = replay.stretch.span(self.review.now()).unwrap_or_default(); + let seconds = format!("{} s", length.as_secs_f32().round().max(1.0)); + let into = replay.into.as_ref().map(|into| format!("`{into}`")); + let mut pace = match replay.speed { + 1.0 => "at full speed".to_string(), + 0.5 => "at half speed".to_string(), + 0.25 => "at quarter speed".to_string(), + speed => format!("at {:.0}%", speed * 100.0), + }; + if replays.len() > 1 && replay.speed < 1.0 { + pace.push_str(" with no sound"); + } + let play_note = match &refusals.replay { + Some(why) => why.clone(), + None => format!( + "{seconds} {pace}, into {}", + into.as_deref().unwrap_or("the replay source") + ), + }; + let save_note = match &refusals.save { + Some(why) => why.clone(), + None => format!("{seconds} at full speed, into the folder"), + }; + let key = self.key(&Action::SaveClip, ui.ctx()); + let play = Big { + icon: Icon::Play, + text: "Replay", + note: &play_note, + filled: true, + enabled: refusals.replay.is_none(), + key: None, + } + .show(ui, play_rect); + let save = Big { + icon: Icon::Save, + text: "Save clip", + note: &save_note, + filled: false, + enabled: refusals.save.is_none(), + key: key.as_deref(), + } + .show(ui, save_rect); + match (play.clicked(), save.clicked()) { + (true, _) => Some(Action::Replay), + (_, true) => Some(Action::SaveClip), + _ => None, + } + } + + /// The show's replay sources, keeping what `replay` plays in one of them: + /// the first, unless it names another. + fn replay_sources(&self, replay: &mut controls::Replay) -> Vec { + let state = self.engine.state.lock().unwrap(); + let replays: Vec = state + .collection() + .sources + .iter() + .filter(|(_, source)| matches!(source, Source::Replay)) + .map(|(name, _)| name.clone()) + .collect(); + if replay.into.as_ref().is_none_or(|into| !replays.contains(into)) { + replay.into = replays.first().cloned(); + } + replays + } + + /// This show's recordings and clips as cards, with the folder below. + fn shelf(&mut self, ui: &mut Ui, rect: Rect) { + let painter = ui.painter(); + let title = kit::caps_galley(painter, "Recordings", kit::display_bold(22.0), kit::TEXT, None); + let title_height = title.size().y; + painter.galley(rect.left_top(), title, kit::TEXT); + painter.text( + pos2(rect.right(), rect.top() + title_height / 2.0), + Align2::RIGHT_CENTER, + "of this show", + kit::ui(11.5), + kit::DIM, + ); + let footer = Rect::from_min_max(pos2(rect.left(), rect.bottom() - 34.0), rect.max); + let list = Rect::from_min_max( + pos2(rect.left(), rect.top() + title_height + 12.0), + pos2(rect.right(), footer.top() - 10.0), + ); + let listing = self.review.shelf.cards(); + let last = self.engine.recorder.status(); + // Each file of the recording under way or the last that stopped + // writing early, and why. + let failed: Vec = last + .iter() + .flat_map(|status| &status.files) + .filter_map(|file| match &file.finished { + Some(record::Finished::Failed(why)) => { + let name = file.path.file_name().unwrap_or_default().to_string_lossy(); + Some(format!("{name} stopped early: {why}")) } - None => { - ui.weak("no replay source; add one in Sources"); - } + _ => None, + }) + .collect(); + let status = last.filter(|status| !status.done); + let show = self.engine.recorder.show().to_string(); + let live = status.as_ref().and_then(|status| { + let first = status.files.first()?; + match shelf::parse(&show, &first.path.file_name()?.to_string_lossy())? { + shelf::File::Recording { stamp, .. } => Some(stamp), + shelf::File::Clip(_) => None, } - ui.end_row(); }); + let mut opened = None; + ui.scope_builder(UiBuilder::new().max_rect(list), |ui| { + egui::ScrollArea::vertical() + .id_salt("recordings") + .auto_shrink(false) + .show(ui, |ui| { + ui.spacing_mut().item_spacing.y = 8.0; + for failed in &failed { + ui.label(egui::RichText::new(failed).color(kit::ERR)); + } + if let Some(status) = &status { + self.live_card(ui, status, &show); + } + for card in &listing.cards { + if matches!(&card.kind, shelf::Kind::Recording { stamp, .. } if Some(stamp) == live.as_ref()) { + continue; + } + if self.card(ui, card).clicked() { + opened = Some(card.path.clone()); + } + } + if listing.cards.is_empty() && status.is_none() { + let why = listing + .error + .clone() + .unwrap_or_else(|| "Nothing recorded for this show yet. Record from the title bar.".into()); + ui.label(egui::RichText::new(why).color(kit::DIM)); + } + }); + }); + if let Some(path) = opened { + let result = shelf::open(&path); + self.check(result); + } + let open = ui.put( + Rect::from_min_size(footer.left_center() - vec2(0.0, 15.0), vec2(150.0, 30.0)), + kit::button(Icon::Folder, "Open the folder"), + ); + let open = open.on_hover_text(self.review.shelf.dir().display().to_string()); + if open.clicked() { + let result = shelf::open(self.review.shelf.dir()); + self.check(result); + } + if let Some(free) = listing.free { + ui.painter().text( + footer.right_center(), + Align2::RIGHT_CENTER, + format!("{} free", bytes(free)), + kit::ui(11.5), + kit::DIM, + ); + } + } - let (play, save) = ui - .horizontal(|ui| { - let play = ui - .add_enabled(panel.into.is_some(), egui::Button::new("Replay")) - .on_hover_text("Play the last moments in the replay source") - .clicked(); - let save = ui - .add(kit::button(Icon::Clip, "Save clip")) - .on_hover_text( - "Save the last moments, in real time, beside the scene file. Copied without re-encoding, a \ - clip starts at the keyframe before them, and video with B-frames can end early.", - ) - .clicked(); - (play, save) + /// The recording under way, its picture the live one. + fn live_card(&self, ui: &mut Ui, status: &record::Status, show: &str) { + let written: u64 = status.files.iter().map(|file| file.written).sum(); + let sources = status + .files + .iter() + .filter_map(|file| shelf::parse(show, &file.path.file_name()?.to_string_lossy())) + .filter_map(|file| match file { + shelf::File::Recording { source, .. } => source, + shelf::File::Clip(_) => None, }) - .inner; - if let Some(outcome) = panel.outcome.lock().unwrap().as_ref() { - match outcome { - Ok(message) => ui.label(message), - Err(err) => ui.colored_label(ui.visuals().error_fg_color, err), + .collect::>() + .len(); + let elapsed = kio::clock::now().saturating_duration_since(status.started); + let picture = egui::load::SizedTexture::new( + self.program.id, + vec2(self.program_drawn.width as f32, self.program_drawn.height as f32), + ); + let (title, meta) = match status.stopping { + true => ("Finishing the recording", format!("{} written", bytes(written))), + false => ("Recording now", format!("{} so far", bytes(written))), + }; + let chip = (sources > 0).then(|| (None, format!("Program and {sources} sources"))); + let card = Card { + picture: Some(picture), + length: &clock(elapsed), + title, + meta: &meta, + chip, + live: true, + file: None, + }; + card.show(ui); + } + + /// A finished recording's or a clip's card. + fn card(&self, ui: &mut Ui, card: &shelf::Card) -> egui::Response { + let size = bytes(card.bytes); + let ago = card.modified.map(ago); + let (title, meta, chip) = match &card.kind { + shelf::Kind::Clip(n) => ( + format!("Clip {n}"), + match ago { + Some(ago) => format!("{size}, {ago}"), + None => size, + }, + Some((Some(Icon::Clip), "Clip".to_string())), + ), + shelf::Kind::Recording { sources, repaired, .. } => ( + ago.map(|ago| capitalize(&ago)).unwrap_or_else(|| "Recording".into()), + size, + match (repaired, sources) { + (true, _) => Some((Some(Icon::Repaired), "Repaired after a crash".to_string())), + (false, 0) => None, + (false, sources) => Some((None, format!("Program and {sources} sources"))), + }, + ), + }; + let length = card.length.map(clock).unwrap_or_default(); + let name = card.path.file_name().unwrap_or_default().to_string_lossy(); + let card = Card { + picture: None, + length: &length, + title: &title, + meta: &meta, + chip, + live: false, + file: Some(&name), + }; + card.show(ui) + } +} + +/// A recording's or clip's card: a picture with its length over it, the +/// title, a line under it, and a chip. +struct Card<'a> { + picture: Option, + length: &'a str, + title: &'a str, + meta: &'a str, + /// Such as CLIP, with its icon. + chip: Option<(Option, String)>, + /// Whether it is the recording under way. + live: bool, + /// The file a click opens, by name. + file: Option<&'a str>, +} + +impl Card<'_> { + /// Draw the card, as wide as `ui`. + fn show(self, ui: &mut Ui) -> egui::Response { + let Self { + picture, + length, + title, + meta, + chip, + live, + file, + } = self; + let sense = if file.is_some() { Sense::click() } else { Sense::hover() }; + let (rect, response) = ui.allocate_exact_size(vec2(ui.available_width(), 88.0), sense); + response.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, title)); + let response = match file { + Some(file) => response + .on_hover_text(format!("Open {file}")) + .on_hover_cursor(CursorIcon::PointingHand), + None => response, + }; + let painter = ui.painter(); + let border = match (live, response.hovered() && file.is_some()) { + (_, true) => kit::DIM, + (true, false) => Color32::from_rgb(0x3f, 0x3f, 0x4d), + (false, false) => kit::LINE, + }; + painter.rect(rect, 12, kit::P2, Stroke::new(1.0, border), egui::StrokeKind::Inside); + let thumb = Rect::from_min_size(rect.left_top() + vec2(7.0, 8.0), vec2(128.0, 72.0)); + match picture { + Some(picture) => { + egui::Image::new(picture) + .corner_radius(7) + .uv(cover(picture.size, thumb.size())) + .paint_at(ui, thumb); + } + // What a finished file shows waits on reading a regular MP4's first + // picture (quest/m3/card-pictures.md). + None => { + ui.painter().rect_filled(thumb, 7, kit::P3); + Icon::Review.paint(ui, thumb, 20.0, kit::FAINT); + } + } + let painter = ui.painter(); + if !length.is_empty() { + let galley = painter.layout_no_wrap(length.into(), kit::mono(10.5), Color32::WHITE); + let at = thumb.right_bottom() - vec2(5.0, 5.0) - galley.size() - vec2(5.0, 3.0); + let back = Rect::from_min_size(at, galley.size() + vec2(10.0, 6.0)); + painter.rect_filled(back, 4, Color32::from_black_alpha(204)); + painter.galley(at + vec2(5.0, 3.0), galley, Color32::WHITE); + } + let x = thumb.right() + 12.0; + let width = rect.right() - 8.0 - x; + let mut y = rect.top() + 18.0; + if live { + painter.circle_filled(pos2(x + 4.0, y + 7.0), 4.0, kit::ACCENT); + } + let title_x = if live { x + 14.0 } else { x }; + let title = painter.layout(title.into(), kit::ui_strong(13.5), kit::TEXT, width); + let title_height = title.size().y; + painter.galley(pos2(title_x, y), title, kit::TEXT); + y += title_height + 6.0; + let meta = painter.layout(meta.into(), kit::ui(11.5), kit::DIM, width); + let meta_height = meta.size().y; + painter.galley(pos2(x, y), meta, kit::DIM); + y += meta_height + 6.0; + if let Some((icon, text)) = chip { + let outlined = icon == Some(Icon::Repaired); + let galley = kit::caps_galley(painter, &text, kit::display_bold(11.0), kit::TEXT, Some(width - 20.0)); + let icon_width = if icon.is_some() { 17.0 } else { 0.0 }; + let chip = Rect::from_min_size(pos2(x, y), vec2(galley.size().x + 14.0 + icon_width, 19.0)); + match outlined { + true => painter.rect_stroke(chip, 4, Stroke::new(1.0, kit::DIM), egui::StrokeKind::Inside), + false => painter.rect_filled(chip, 4, kit::P3), }; + if let Some(icon) = icon { + icon.paint( + ui, + Rect::from_center_size(pos2(chip.left() + 7.0 + 6.0, chip.center().y), Vec2::splat(12.0)), + 12.0, + kit::TEXT, + ); + } + ui.painter().galley( + pos2(chip.left() + 7.0 + icon_width, chip.center().y - galley.size().y / 2.0), + galley, + kit::TEXT, + ); } - let action = match (play, save) { - (true, _) => Action::Replay, - (_, true) => Action::SaveClip, - _ => return, + response + } +} + +/// One of the page's two large buttons: Replay, filled, and Save clip. +struct Big<'a> { + icon: Icon, + text: &'a str, + /// What it does, or why it cannot. + note: &'a str, + filled: bool, + enabled: bool, + /// The key bound to it. + key: Option<&'a str>, +} + +impl Big<'_> { + /// Draw the button filling `rect`. + fn show(self, ui: &mut Ui, rect: Rect) -> egui::Response { + let Self { + icon, + text, + note, + filled, + enabled, + key, + } = self; + let sense = if enabled { Sense::click() } else { Sense::hover() }; + let response = ui.interact(rect, ui.id().with(text), sense); + response.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, enabled, text)); + let hovered = enabled && response.hovered(); + let (fill, ink, dim, stroke) = match (filled, enabled) { + (true, true) => ( + if hovered { Color32::WHITE } else { kit::TEXT }, + Color32::from_rgb(0x11, 0x11, 0x11), + Color32::from_rgba_unmultiplied(0x11, 0x11, 0x11, 0xbb), + Color32::TRANSPARENT, + ), + (false, true) => ( + if hovered { kit::P3 } else { kit::P2 }, + kit::TEXT, + kit::DIM, + Color32::from_rgb(0x3f, 0x3f, 0x4d), + ), + (_, false) => (kit::P1, kit::FAINT, kit::DIM, kit::LINE), }; - drop(guard); - // How it went shows above, from the next frame. - let _ = controls.fire(&action); + let painter = ui.painter(); + if filled && enabled { + painter.add( + egui::Shadow { + offset: [0, 10], + blur: 30, + spread: 0, + color: Color32::from_black_alpha(136), + } + .as_shape(rect, egui::CornerRadius::same(13)), + ); + } + painter.rect(rect, 13, fill, Stroke::new(1.0, stroke), egui::StrokeKind::Inside); + let icon_rect = Rect::from_center_size(pos2(rect.left() + 22.0 + 13.0, rect.center().y), Vec2::splat(26.0)); + icon.paint(ui, icon_rect, 26.0, ink); + let x = icon_rect.right() + 16.0; + let key_width = key.map_or(0.0, |_| 90.0); + let width = (rect.right() - 22.0 - key_width - x).max(10.0); + let painter = ui.painter(); + let word = kit::caps_galley(painter, text, kit::display_bold(28.0), ink, Some(width)); + // A refusal can take two lines. + let mut job = egui::text::LayoutJob::simple(note.into(), kit::ui(12.0), dim, width); + job.wrap.max_rows = 2; + let note = painter.layout_job(job); + let height = word.size().y + 4.0 + note.size().y; + let top = rect.center().y - height / 2.0; + let word_height = word.size().y; + painter.galley(pos2(x, top), word, ink); + painter.galley(pos2(x, top + word_height + 4.0), note, dim); + if let Some(key) = key { + let galley = painter.layout_no_wrap(key.into(), kit::mono(11.0), ink); + let back = Rect::from_min_size( + pos2(rect.right() - 22.0 - galley.size().x - 12.0, rect.center().y - 11.0), + vec2(galley.size().x + 12.0, 22.0), + ); + painter.rect_stroke(back, 4, Stroke::new(1.0, kit::FAINT), egui::StrokeKind::Inside); + painter.galley(back.center() - galley.size() / 2.0, galley, ink); + } + response + } +} + +/// How long ago the stretch starts and ends, as of `now`. +fn ages(stretch: replay::Stretch, now: Instant) -> (Duration, Duration) { + match stretch { + replay::Stretch::Last(length) => (length.min(replay::WINDOW), Duration::ZERO), + replay::Stretch::Between(from, to) => ( + now.saturating_duration_since(from).min(replay::WINDOW), + now.saturating_duration_since(to).min(replay::WINDOW), + ), + } +} + +/// Which handle a press at `x` takes, given where the stretch's start and end +/// are drawn. Where both are in reach, as when a stretch has aged out and both +/// sit on the strip's oldest edge, the one on the press's side, so the end can +/// always be dragged free. +fn grip_at(x: f32, from_x: f32, to_x: f32) -> Option { + let near = |at: f32| (x - at).abs() <= GRAB; + match (near(from_x), near(to_x)) { + (true, true) if x < (from_x + to_x) / 2.0 => Some(Grip::From), + (true, true) | (false, true) => Some(Grip::To), + (true, false) => Some(Grip::From), + (false, false) => None, + } +} + +/// The part of a `texture` that covers a rect of `room`'s shape, centered. +fn cover(texture: Vec2, room: Vec2) -> Rect { + let (texture, room) = (texture.x / texture.y.max(1.0), room.x / room.y.max(1.0)); + match texture > room { + true => { + let width = room / texture; + Rect::from_min_max(pos2((1.0 - width) / 2.0, 0.0), pos2((1.0 + width) / 2.0, 1.0)) + } + false => { + let height = texture / room; + Rect::from_min_max(pos2(0.0, (1.0 - height) / 2.0), pos2(1.0, (1.0 + height) / 2.0)) + } + } +} + +/// A small tag, such as ON AIR, its top left at `at`. +struct Tag { + fill: Color32, + ink: Color32, + size: f32, +} + +impl Tag { + /// The program's, in the accent. + const AIR: Self = Self { + fill: kit::ACCENT, + ink: Color32::WHITE, + size: 11.0, + }; + /// How long ago the large picture was, in amber. + const AGO: Self = Self { + fill: kit::CUE, + ink: kit::ON_CUE, + size: 17.0, + }; + + /// Draw `text` in it, its top left at `at`. + fn paint(&self, painter: &egui::Painter, at: egui::Pos2, text: &str) { + let galley = kit::caps_galley(painter, text, kit::display_bold(self.size), self.ink, None); + let back = Rect::from_min_size(at, galley.size() + vec2(self.size * 0.8, self.size * 0.35)); + painter.rect_filled(back, 3, self.fill); + painter.galley(back.center() - galley.size() / 2.0, galley, self.ink); + } +} + +/// A section's caption, small and spaced. +fn caption_text(painter: &egui::Painter, at: egui::Pos2, text: &str, align: Align2) { + let galley = kit::caps_galley(painter, text, kit::display(11.5), kit::DIM, None); + let rect = align.anchor_size(at, galley.size()); + painter.galley(rect.min, galley, kit::DIM); +} + +/// `text` with a capital first letter, for an error shown as a sentence. +fn capitalize(text: &str) -> String { + let mut chars = text.chars(); + match chars.next() { + Some(first) => first.to_uppercase().chain(chars).collect(), + None => String::new(), + } +} + +/// A size on disk: "11 MB", "6.1 GB". +fn bytes(bytes: u64) -> String { + match bytes { + 0..1_000_000_000 => format!("{} MB", bytes.div_ceil(1_000_000)), + _ => format!("{:.1} GB", bytes as f64 / 1e9), + } +} + +/// How long ago `then` was, as a phrase: "2 minutes ago". +fn ago(then: kio::clock::Wall) -> String { + let ago = kio::clock::wall().saturating_duration_since(then).as_secs(); + let plural = |n: u64, unit: &str| match n { + 1 => format!("1 {unit} ago"), + n => format!("{n} {unit}s ago"), + }; + match ago { + 0..60 => "just now".into(), + 60..3600 => plural(ago / 60, "minute"), + 3600..86_400 => plural(ago / 3600, "hour"), + 86_400..172_800 => "yesterday".into(), + _ => plural(ago / 86_400, "day"), + } +} + +/// A length: "0:08", "52:10", "2:10:44". +pub(super) fn clock(length: Duration) -> String { + let seconds = length.as_secs(); + match seconds { + 0..3600 => format!("{}:{:02}", seconds / 60, seconds % 60), + _ => format!("{}:{:02}:{:02}", seconds / 3600, seconds / 60 % 60, seconds % 60), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn sizes_and_lengths_read_as_the_mock_up_has_them() { + assert_eq!(bytes(11_000_000), "11 MB"); + assert_eq!(bytes(6_100_000_000), "6.1 GB"); + assert_eq!(clock(Duration::from_secs(8)), "0:08"); + assert_eq!(clock(Duration::from_secs(52 * 60 + 10)), "52:10"); + assert_eq!(clock(Duration::from_secs(2 * 3600 + 10 * 60 + 44)), "2:10:44"); + } + + /// Each handle is taken near it, and where both are in reach, the one on + /// the press's side: an aged-out stretch's end, drawn on its start at the + /// strip's oldest edge, is still taken by a press just right of it. + #[test] + fn a_press_takes_the_handle_on_its_side() { + assert_eq!(grip_at(100.0, 100.0, 300.0), Some(Grip::From)); + assert_eq!(grip_at(305.0, 100.0, 300.0), Some(Grip::To)); + assert_eq!(grip_at(200.0, 100.0, 300.0), None); + assert_eq!(grip_at(24.0, 20.0, 20.0), Some(Grip::To)); + assert_eq!(grip_at(16.0, 20.0, 20.0), Some(Grip::From)); + assert_eq!(grip_at(98.0, 100.0, 106.0), Some(Grip::From)); + assert_eq!(grip_at(104.0, 100.0, 106.0), Some(Grip::To)); + } + + /// A wide picture in a narrow room is cut at its sides, a tall one at its + /// top and bottom, and one of the room's shape is whole. + #[test] + fn a_picture_covers_its_room() { + let wide = cover(vec2(160.0, 90.0), vec2(90.0, 90.0)); + assert!((wide.width() - 90.0 / 160.0).abs() < 1e-6 && wide.height() == 1.0); + let tall = cover(vec2(90.0, 160.0), vec2(90.0, 90.0)); + assert!((tall.height() - 90.0 / 160.0).abs() < 1e-6 && tall.width() == 1.0); + assert_eq!( + cover(vec2(16.0, 9.0), vec2(160.0, 90.0)), + Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)) + ); } } diff --git a/rs/moq-studio/src/ui/settings.rs b/rs/moq-studio/src/ui/settings.rs index 3d5bbffa..19fef6fe 100644 --- a/rs/moq-studio/src/ui/settings.rs +++ b/rs/moq-studio/src/ui/settings.rs @@ -1,5 +1,5 @@ -//! Settings: what a broadcaster sets once, such as the canvas and the -//! hotkeys; and the edit history, under the title bar's undo. +//! Settings: what a broadcaster sets once, such as the canvas, recording, and +//! the hotkeys; and the edit history, under the title bar's undo. use super::*; @@ -410,4 +410,112 @@ impl Panels { drop(state); self.check(result); } + + /// What the record button saves, from any page: the folder, the picture, + /// the sound, and whether each source gets a file of its own too. + pub(super) fn recording(&mut self, ui: &mut Ui) { + let dir = self.engine.recorder.dir().to_path_buf(); + ui.weak("What the record button saves, from any page."); + ui.add_space(8.0); + ui.horizontal(|ui| { + ui.label("Folder"); + ui.monospace(dir.display().to_string()); + if ui.add(kit::button(Icon::Folder, "Open")).clicked() { + let result = crate::shelf::open(&dir); + self.check(result); + } + }); + ui.weak("Set with --recordings. Clips are saved here too."); + ui.add_space(8.0); + self.recorded_scene(ui); + ui.weak("The program is what viewers see. A scene records at its own size."); + ui.add_space(8.0); + self.recorded_audio(ui); + ui.weak("The audio scene the recording holds. It can differ from the stream's, to keep music out."); + ui.add_space(8.0); + let mut each = self.controls.record_sources.load(Ordering::Relaxed); + if ui.add(kit::Toggle::new(&mut each).label("Each source too")).changed() { + self.controls.record_sources.store(each, Ordering::Relaxed); + } + ui.weak( + "Every camera, screen, and microphone also saves a file of its own, before its filters. Each costs an \ + encoder and its share of the disk.", + ); + ui.add_space(12.0); + ui.weak("A recording cut short by a crash is repaired the next time the studio opens."); + } + + /// Pick the picture recordings hold, the program or any scene, even while + /// recording, which goes on in a new file when the size changes. + fn recorded_scene(&mut self, ui: &mut Ui) { + let engine = self.engine.clone(); + let mut state = engine.state.lock().unwrap(); + let current = state.collection().recorded().map(str::to_string); + let scenes = state.collection().scenes.iter(); + let names: Vec = scenes.map(|scene| scene.name.clone()).collect(); + let mut picked = current.clone(); + ui.horizontal(|ui| { + ui.label("Picture") + .on_hover_text("A scene of another size than the program's goes on in a new file"); + egui::ComboBox::from_id_salt("recorded scene") + .selected_text(picked.as_deref().unwrap_or("The program")) + .show_ui(ui, |ui| { + ui.selectable_value(&mut picked, None, "The program"); + for name in &names { + ui.selectable_value(&mut picked, Some(name.clone()), name); + } + }); + }); + if picked == current { + return; + } + let label = match &picked { + Some(name) => format!("Record scene `{name}`"), + None => "Record the program".into(), + }; + let result = edit(&mut state, label, None, |draft| draft.set_recorded(picked.as_deref())); + drop(state); + self.check(result); + } + + /// Pick the audio scene recordings hold, even while recording, which goes + /// on with the new one from its next block. + fn recorded_audio(&mut self, ui: &mut Ui) { + let engine = self.engine.clone(); + let mut state = engine.state.lock().unwrap(); + let scenes = state.collection().audio_scenes.clone(); + let program = state.collection().program_audio().unwrap_or_default(); + let on_program = format!("On program ({program})"); + let current = scenes + .iter() + .find(|scene| scene.recorded) + .map(|scene| scene.name.clone()); + let mut picked = current.clone(); + ui.horizontal(|ui| { + ui.label("Sound") + .on_hover_text("An audio scene nothing else carries stays off the relay"); + egui::ComboBox::from_id_salt("recorded audio") + .selected_text(picked.as_deref().unwrap_or(&on_program)) + .show_ui(ui, |ui| { + ui.selectable_value(&mut picked, None, &on_program); + for scene in &scenes { + ui.selectable_value(&mut picked, Some(scene.name.clone()), &scene.name); + } + }); + }); + if picked == current { + return; + } + let label = match &picked { + Some(name) => format!("Record audio scene `{name}`"), + None => "Record the audio scene on program".into(), + }; + let mut next = scenes; + for scene in &mut next { + scene.recorded = picked.as_deref() == Some(scene.name.as_str()); + } + let result = edit(&mut state, label, None, |draft| draft.set_audio_scenes(next)); + drop(state); + self.check(result); + } } diff --git a/rs/moq-studio/src/ui/shell.rs b/rs/moq-studio/src/ui/shell.rs index 14309108..cc82d16a 100644 --- a/rs/moq-studio/src/ui/shell.rs +++ b/rs/moq-studio/src/ui/shell.rs @@ -267,22 +267,8 @@ impl Panels { ui.separator(); self.audio(ui); }), - Page::Review => { - // The live picture beside replays and recordings. - let side = (stage.width() * 0.32).clamp(220.0, 420.0); - let card = stage.with_min_x(stage.right() - side); - region(ui, stage.with_max_x(card.left()), |ui| { - self.replay(ui); - ui.separator(); - self.record(ui); - }); - bar(ui, fx, card); - ui.painter() - .vline(card.left() + 0.5, card.y_range(), Stroke::new(1.0, kit::LINE)); - let room = card.shrink(MARGIN + 10.0); - let room = room.with_max_y(room.top() + room.width() / snapshot.shape.max(0.5)); - self.on_air(ui, Some(fx), fit(room, snapshot.shape)); - } + // The program's card in "replay what" is the live picture. + Page::Review => self.review(ui, fx, stage, snapshot.shape), } } @@ -581,18 +567,23 @@ impl Panels { } } - /// Start or stop recording, pulsing while it records. + /// Start or stop recording, from any page, pulsing and showing its clock + /// while it records. fn record_button(&mut self, ui: &mut Ui, fx: &mut Fx) { let status = self.engine.recorder.status().filter(|status| !status.done); let key = self.key(&Action::Record, ui.ctx()); - let (rect, response) = ui.allocate_exact_size(Vec2::splat(30.0), Sense::click()); - let name = match &status { - None => "Start recording".to_string(), - Some(status) if status.stopping => "Finishing the recording".to_string(), - Some(status) => { - let elapsed = kio::clock::now().saturating_duration_since(status.started); - format!("Stop recording, {} in", clock(elapsed)) - } + let elapsed = status + .as_ref() + .map(|status| super::review::clock(kio::clock::now().saturating_duration_since(status.started))); + let time = elapsed + .clone() + .map(|elapsed| ui.painter().layout_no_wrap(elapsed, kit::mono(12.0), kit::TEXT)); + let width = time.as_ref().map_or(30.0, |time| 30.0 + time.size().x + 8.0); + let (rect, response) = ui.allocate_exact_size(vec2(width, 30.0), Sense::click()); + let name = match (&status, &elapsed) { + (None, _) => "Start recording".to_string(), + (Some(status), _) if status.stopping => "Finishing the recording".to_string(), + (Some(_), elapsed) => format!("Stop recording, {} in", elapsed.as_deref().unwrap_or_default()), }; let response = kit::tip(response, &name, key.as_deref()); let painter = ui.painter(); @@ -610,11 +601,15 @@ impl Panels { egui::StrokeKind::Inside, ); let pulse = (recording && fx.on()).then(|| Mark::new(ui.painter())); - let dot = Rect::from_center_size(rect.center(), Vec2::splat(9.0)); + let dot = Rect::from_center_size(pos2(rect.left() + 15.0, rect.center().y), Vec2::splat(9.0)); ui.painter().circle_filled(dot.center(), 4.5, kit::ACCENT); if let Some(pulse) = pulse { fx.recording(&pulse.around(dot)); } + if let Some(time) = time { + let at = pos2(dot.right() + 7.0, rect.center().y - time.size().y / 2.0); + ui.painter().galley(at, time, kit::TEXT); + } if response.clicked() { self.controls.press(&Action::Record); } @@ -1011,7 +1006,7 @@ impl Panels { .look(kit::Look::Boxed) .key(clip_key.as_deref()) .note(Some( - "Save the last moments, as Review sets them, beside the scene file", + "Save the last seconds, as many as REVIEW picks, as a clip into the recordings folder", )); let (panic_at, clip_at) = match tall { true => ( @@ -1273,6 +1268,9 @@ impl Panels { Setting::Show => { egui::ScrollArea::vertical().show(ui, |ui| self.output(ui)); } + Setting::Recording => { + egui::ScrollArea::vertical().show(ui, |ui| self.recording(ui)); + } Setting::Hotkeys => { egui::ScrollArea::vertical().show(ui, |ui| self.hotkeys(ui, fx)); }