Skip to content

track.start shifts the muxed audio but not the analysis audio-reactive components read #190

Description

@LeadcodeDev

Symptom

An audio track with start: 73 plays the file from the beginning at t=73 in the video. A waveform or audio_spectrum at t=73 draws the file's content at 73 seconds, not at 0. Picture and sound disagree by exactly start.

Cause

Two independent time bases:

  • crates/rustmotion/src/encode/audio.rs places a track at track.start * TARGET_SAMPLE_RATE on the scenario timeline and copies the file from its own sample 0.
  • crates/rustmotion/src/encode/audio_analysis.rs analyses the raw file, indexed from its own frame 0, and the painters call amplitude_at(ctx.time) with scenario time. track.start appears nowhere in that file.

So the mapping is 1:1 only when start == 0. end has the same issue in reverse: the analysis keeps running past the point where the track has been cut.

Practical consequence

Gating a soundtrack to one scene is a natural thing to want, and the obvious way to write it — start/end — silently desynchronises the visualisation. The workaround is volume_keyframes, which leaves the time mapping intact but is not what the fields suggest.

Proposed fix

  1. Offset the lookup by the track's placement: the painters resolve a track (by style.audio-reactive.track, or the first cached entry), so the same resolution can carry start, and read amplitude_at(ctx.time - track.start).
  2. Return 0 outside [start, end) so a visualisation goes flat exactly when the track is silent, instead of drawing an envelope for audio nobody hears.
  3. volume_keyframes and the fades are a separate question — the analysis is of the source, not of the mix. Worth deciding explicitly and documenting either way, since feat(studio): play the scenario's audio while previewing #182 made the studio play the mixed output, so a viewer now hears the envelope while the waveform draws the raw one.

A test with a track at start: 1.0 whose first second is silence, asserting the visualisation is flat over [0,1) and active after, pins it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions