Skip to content

FMV: bind HD replacement to guest movie timeline - #761

Open
tetrisgm wants to merge 13 commits into
RetroPortingToolKit:masterfrom
tetrisgm:fix/fmv-guest-timeline-upstream
Open

tetrisgm wants to merge 13 commits into
RetroPortingToolKit:masterfrom
tetrisgm:fix/fmv-guest-timeline-upstream

Conversation

@tetrisgm

@tetrisgm tetrisgm commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

HD FMV replacement in #718 could select the wrong replacement after a seek or state load, and advanced subframes from host time even while the guest was paused. This continuation keeps the original contributor's HD movie feature while binding lookup and subframe timing to guest events.

The renderer now receives an immutable guest timeline snapshot. FMVHD003 packs identify a movie by its SetLoc range and each source frame by the delivered CD sector interval, with the native frame signature as a second check. Duplicate-looking frames are distinct. Guest cycles drive subframe phase; MDEC decode events, seeks, state loads, rollback, and movie exit reset it. An unmatched or failed lookup displays the native PS1 frame. Signature-only FMVHD002 packs require explicit PSX_FMV_HD_LEGACY=1 because they cannot resolve seek and repeated-frame ambiguity.

This supersedes the unresolved timeline design in #718. The original HD replacement implementation is by @yacineolivier-maker; this branch includes that work and the reviewer-requested safety changes. Please review this PR alongside #718. It does not modify or close the contributor's branch.

Verification:

  • Windows and macOS: full psx-runtime builds and focused gpu_fmv_hd_test passed. The synthetic test covers malformed packs, worker/decode lifecycle, pause, guest-cycle subframe advancement, duplicate frames, seek generation, movie exit, and load/rollback epochs.
  • R4 local-disc replay with an 80-frame local synthetic FMVHD003 pack: expected HD replacements at SetLoc 192822 and delivered LBAs 194127–194324, zero dispatch misses. First replacement at guest frame 1475/cycle 832608000, delivered LBA 194129, MDEC ordinal 260.
  • Host pause for 0.5 s held guest cycle, HD count, and audio counters. Save/load advanced execution epoch 1→2 and playback resumed.
  • Across one unpaused movie interval, guest time advanced 366.67 ms, SPU output advanced 16,170 samples/366.67 ms, CD input 16,464 samples/373.33 ms, and host output 15,876 samples/360.00 ms (44.1 kHz). This measures cadence within 6.67 ms, not sound-to-image content alignment.
  • TCP Start skipped the movie. At guest frame 1494, HD source tracking reset to -1; HD frame count/lookups stopped at 17/530 through frame 1514. CD input stopped while queued host audio drained. Zero dispatch misses.

R4's intro has no movie seek control. Local Rewind was off in this test configuration and netplay rollback inactive; the focused test exercises seek generation and rollback epoch changes. The replacement images are conspicuous synthetic frames, so content-level A/V lip-sync is not measured. No retail movie assets or synthetic pack are included.

yacineolivier-maker and others added 13 commits October 24, 2026 08:34
An optional pack replaces a title's MDEC movies with high-definition frames
while the game keeps its own audio, timing and skip control:
- fmv_hd/<movie>/index.bin: header, then a luminance signature per source
  frame; fmv_hd/<movie>/frames/NNNNNN.jpg: `sub` HD frames per source frame.
- Every presented 24-bit frame is matched by signature (mean absolute
  difference), first around the last recognised frame, then over the whole
  pack when tracking is lost; a near global match is also tried a few lines
  up or down (each movie has its own vertical placement on screen) and the
  offset kept. Worker threads decode the upcoming frames ahead into a small
  RGBA ring.
- Format FMVHD002 records the native display the movie is shown in and its
  position, so the HD frame lands exactly where the original was.
- The last HD frame is held briefly when the next one is not ready or not
  recognised (no flicker back to the 4:3 original mid-movie), and is
  uploaded to the GPU only when it changes.
- Debug: TCP {"cmd":"fmv_hd"} reports the pack, the matches and lost frames.
No pack: no effect. Pack frames are the user's own upscale of their own disc;
nothing is distributed with the runtime.

Used for R4's intro and ending movies (SeedVR2 upscales, 25 fps sources).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rkyma8rbyPiccs9n2bkyG6
Reject invalid index products and unsafe paths, cap encoded/decoded residency, recover allocation and partial worker-start failures, release GL movie textures with their context, and serialize diagnostics through the presenter. Real SDL decode/restart/failure test passes and SDL2 C seam compiles warning-free. Guest/present timestamp ownership remains a separate design gate. Tracking: beads-eio.3.388.
Replace host-time phase and hold with captured guest cycles, reset on state timeline changes and non-movie scanout, and fall back to native frames on misses.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
FMVHD003 carries movie and per-frame LBA ranges; legacy signature-only packs require explicit opt-in. Capture CD stream generation and MDEC ordinal with each present so seeks and repeated images cannot reuse an old interpolation phase.

Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@tetrisgm
tetrisgm marked this pull request as ready for review October 24, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants