Repository navigation
Make seeking in a transcoded stream instant - #309
Conversation
ffmpeg writes its playlist as it encodes, so the playlist read moments after a stream starts lists only the segments that exist -- and that is all the player believes the film to be. Seeking past them stalled waiting for the encoder, and seeking backwards re-buffered too, because each reload handed the player a different, longer playlist to reconcile. A VOD playlist is derivable from the duration and the segment length, both known before a frame is encoded, so the player now gets the real timeline at once. The final segment is advertised at its true shorter length: a player told every segment is six seconds seeks past the end of a film whose last one is two. An unmeasured file gets no synthetic playlist and falls back to ffmpeg's partial one, because a playlist with a wrong duration is worse than none. Promising the whole film makes the seek possible; producing the part seeked to makes it fast. A request for a segment that does not exist starts a second encode at that offset, writing into the same directory with -start_number so the two interleave. Bounded at three per viewing, since dragging a scrubber asks for a dozen positions in a second. What a session is encoding is recorded when it starts: the segment route is given only a session id, and that is a hash.
|
Reviewed on a5 — clean. 7 tests pass. Based on current main, no migration. Writing the full VOD playlist up front from measured duration + segment length (instead of ffmpeg's as-it-encodes partial) is the right fix for seek re-buffering, and the integration points are all handled:
Nothing blocking. The streaming chain (#301 decision / #302 signed segments / #305 profile / #309 seekable) is solid now. |
|
a5 — the "should is doing work there" claim, measured. Encoded segment index 2 two ways on a real Midnight Gospel episode (1080p HEVC→H.264, your exact flags: forced
Verdict: decode-equivalent, interchangeable. Both hold the same 144 frames (one forced GOP = one segment) over the same time range, each independently decodable (keyframe-started). They are not byte-identical — libx264 rate-control state differs between a full run and a fresh seek — but byte-identity isn't what HLS needs; independent decodability + matching timeline is, and that holds. The ~44 ms EXTINF variance is within player tolerance. So the Separately, the three-encode ceiling: I can't measure the right value without production load, so it stays a reasonable-default/tuning guess — flagging, not blocking. Everything else on #309 I already reviewed clean. |
1 participant
Reported: going back ten seconds, or forward five minutes, re-buffers.
Why
ffmpeg writes its playlist as it encodes. So the playlist read moments after a stream starts lists only the handful of segments that exist — and that is all the player believes the film to be.
Seeking past them stalled waiting for the encoder. Seeking backwards re-buffered too, because every reload handed the player a different, longer playlist to reconcile against what it already had.
The whole timeline, up front
A VOD playlist is derivable from two numbers both known before a frame is encoded: the duration (the probe measured it) and the segment length. The player now gets the real timeline immediately.
The final segment is advertised at its true, shorter length — a player told every segment is six seconds will seek past the end of a film whose last is two. Verified: a 95-second film produces 16 segments, the last 5 seconds, totalling exactly 95.
An unmeasured file gets no synthetic playlist and falls back to ffmpeg's partial one, which is what shipped before. A playlist with a wrong duration is worse than none, because the player trusts it and seeks into nothing.
Segments the encoder has not reached
Promising the whole film makes the seek possible; producing the seeked-to part makes it fast. A request for a segment that does not exist starts a second encode at exactly that offset, writing into the same directory with
-start_numberso the two interleave into one coherent set.Bounded at three encodes per viewing — dragging a scrubber asks for a dozen positions in a second, and an ffmpeg for each would cost far more than the seeks save. A claim expires after two minutes so a died encode does not block its segment for ever.
Testing
7 new tests, 372 pass. Confirmed to fail against both bugs they guard: a final segment at full length, and a missing
ENDLIST(which makes the player treat it as live and refuse to seek at all).For review — two things I would value your eye on
1. Concurrent encodes writing to one directory. Two ffmpegs writing
seg%03d.tsinto the same place, with the second offset by-start_number, should not collide — the segment a given index names is the same content either way. But I have not proven that under load, and "should not collide" is doing work in that sentence. If you think this wants a separate directory per encode with a merge step, say so.2. The three-encode ceiling is a guess. Not measured. It could be too low for a viewer who seeks a lot, or too high for a server already transcoding for several people.
Not verified end-to-end: I have no transcodable file here, so the seek behaviour itself rests on the playlist arithmetic being right rather than on watching a scrubber. Worth one real seek test on a5.