Skip to content

Fix slow frame grabbing when max_fps is set on FileStreamFrameGrabber#95

Merged
timmarkhuff merged 3 commits into
mainfrom
tim/fix-max-fps
Mar 17, 2026
Merged

Fix slow frame grabbing when max_fps is set on FileStreamFrameGrabber#95
timmarkhuff merged 3 commits into
mainfrom
tim/fix-max-fps

Conversation

@timmarkhuff

@timmarkhuff timmarkhuff commented Mar 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

_drop_frames() in FileStreamFrameGrabber used cv2.CAP_PROP_POS_FRAMES to seek past unwanted frames. On H.264/H.265 video, every seek forces the decoder to locate the nearest keyframe and decode forward to the target frame, producing a sawtooth timing pattern where grab times ramp up between keyframes. This made grabbing fewer frames (e.g. 5fps from a 30fps source) paradoxically much slower than reading every frame sequentially.

The fix replaces the seek with sequential capture.read() calls that discard unwanted frames. Sequential reads use the decoder's internal buffer and cost ~1ms each, while seeks were averaging ~40-120ms depending on distance from the nearest keyframe.

The bug

_drop_frames() called:

current_pos = self.capture.get(cv2.CAP_PROP_POS_FRAMES)
self.capture.set(cv2.CAP_PROP_POS_FRAMES, current_pos + frames_to_drop)

The fix

Replaced with:

for _ in range(frames_to_drop):
    self.capture.read()

How this bug was introduced

Git archeology shows this bug did not exist in the original implementation. The first commit on the feature branch (27a5f34, Dec 13, 2024) correctly used sequential capture.read() calls to discard frames. Four days later, a cleanup commit (f109a87, Dec 17, 2024) on the same branch replaced the sequential reads with CAP_PROP_POS_FRAMES seeking -- likely as a perceived optimization. The branch was then squash-merged to main as a4f1fd8 ("Add FileStreamFrameGrabber (#62)"), baking the regression into the mainline. The _drop_frames method has not been modified since.

Performance results (25fps H.264 source, max_fps=5, 200 frames)

Metric Before (seek) After (sequential read)
Mean grab time 39.5 ms 2.9 ms
Max grab time 98.2 ms 11.8 ms
Std deviation 18.4 ms 1.1 ms
Sawtooth pattern Yes No

@timmarkhuff timmarkhuff changed the title Fixing max_fps Fix slow frame grabbing when max_fps is set on FileStreamFrameGrabber Mar 17, 2026
@timmarkhuff timmarkhuff requested a review from sunildkumar March 17, 2026 20:58
Comment thread src/framegrab/grabber.py Outdated
Comment on lines +1559 to +1563
"""Drop frames to achieve target frame rate by reading and discarding.

Sequential reads are used instead of CAP_PROP_POS_FRAMES seeking
because seeking forces the decoder to find the nearest keyframe and
decode forward, which is extremely expensive on H.264/H.265 video.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: some of this docstring reads as being very focused on this specific bug fix. (because claude thinks it is the center of the universe...) I would move this implementation detail to a comment instead.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cleaned up the docstring. It was actually written by ChatGPT

@timmarkhuff timmarkhuff merged commit 17f5508 into main Mar 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants