[RikHeijdens]

Problem

fps resampling uses FFmpeg's fps filter, which keeps the source frame nearest each output timestamp. For non-integer ratios that spaces frames unevenly: 30 → 24 fps keeps 4 of every 5 frames, so per-frame motion goes 1×, 1×, 1×, 2× (and 1.5× steps for 50/60 → 24). Decoding always starts at the same phase, so the uneven step lands at the same position in every clip.

Change

VideoStreamRequest gets fps_conversion:

• "nearest" (default): today's behaviour, unchanged.
• "blend": FFmpeg framerate. Cross-fades the two neighbouring frames. Even spacing at about the cost of nearest, but moving edges show double images.
• "motion": motion-compensated interpolation in src/decoder/motion.rs. It uses the same model as FFmpeg's minterpolate defaults: bilateral block matching on 16×16 blocks with EPZS predictors, then overlapped block motion compensation, and a frame-difference scene-cut check.
  • It is a Rust implementation rather than the FFmpeg filter, because minterpolate was ~11–12× the decode time of nearest and +150 MB per decode.
  • Most of the cost was in building the in-between frames, and changing its options barely helped.
  • The Rust version uses border-padded planes, row-wise SAD and blending, chroma compensated at its own resolution, and runs motion search only for source pairs that bracket an output frame.

Both interpolating modes:

• run after the resize and after HDR tone mapping, since the work grows with pixel count and needs every source frame;
• keep nearest's frame count and timestamps, so planned buffer sizes, seek windows and A/V sync are unchanged;
• fall back to nearest when the source rate is an integer multiple of the target (48 → 24, and 23.976 → 24 within NTSC's 1000/1001 offset), where the output is bit-identical to nearest.

The video buffer source now declares the stream frame rate. The README has a short example.

Measurements

Measured on 37 real 1080p/2K clips at 24–100 fps, decoded to 540p, one decode thread per process with six processes in parallel.

Decode time (24 fps sources excluded, since they fall back to nearest):

	× nearest (median)	extra RSS per decode
blend	~1.0×	~0
minterpolate (FFmpeg filter, for reference)	~11–12×	+150 MB
motion (this PR)	~1.25× (range 1.06–1.65×)	+13 MB

Frame spacing: the ratio of motion on transitions where nearest skips an extra source frame to motion on normal transitions goes from 1.4–1.9 with nearest to ~1.0 with both blend and motion (1.0 = even).

Reconstruction against held-out real frames. Each 50–100 fps clip was halved in frame rate and decoded back to the full rate, so every synthesized frame sits halfway between two inputs. Median over 14 clips:

	PSNR	SSIM	LPIPS
nearest	21.2	0.719	0.124
blend	22.8	0.754	0.137
minterpolate	26.8	0.906	0.065
motion (this PR)	27.7	0.919	0.060

Caveats for callers:

• Synthesized frames are softer than source frames, by about 7% in mean |Laplacian| at 540p.
• Small, fast-moving objects can smear (16 px blocks).
• Sources whose container rate overstates their content rate (repeated frames, e.g. 24p content in a 30 fps file) stall when interpolated, because a frame gets blended with its own copy. Those need duplicate removal first.

Testing

cargo test --no-default-features --release against FFmpeg 7.1: 152 passed, 3 ignored. cargo fmt --check and cargo clippy --all-targets are clean. New tests cover:

• filter-graph strings
• frame count and timestamps per mode, including seek windows
• even motion steps on a synthetic moving bar (nearest 4,8,4,4 px; blend/motion 5,5,5,5)
• the integer-ratio fallback
• interpolator unit tests

[aikido-pr-checks]

Rust safety bypass can lead to security risks in unsafe operations - medium severity
This code did not enforce Rust safety bounds checking, ownership validation, or type safety guarantees in when handling critical operations within an unsafe block. Attackers may exploit this weakness to execute arbitrary code, trigger denial-of-service via panics, or manipulate memory out of bounds, such as read/write violations, by abusing unchecked arithmetic, transmutation, or direct pointer assignments.

Show fix

Remediation: Replace unsafe operations with Rust's safe alternatives, validate inputs rigorously, and rely on Rust's borrow checker to enforce memory safety wherever possible.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[RikHeijdens]

@AikidoSec ignore: We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

Rust safety bypass can lead to security risks in unsafe operations - medium severity
This code did not enforce Rust safety bounds checking, ownership validation, or type safety guarantees in when handling critical operations within an unsafe block. Attackers may exploit this weakness to execute arbitrary code, trigger denial-of-service via panics, or manipulate memory out of bounds, such as read/write violations, by abusing unchecked arithmetic, transmutation, or direct pointer assignments.

Show fix

Remediation: Replace unsafe operations with Rust's safe alternatives, validate inputs rigorously, and rely on Rust's borrow checker to enforce memory safety wherever possible.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[RikHeijdens]

@AikidoSec ignore: We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

Improper unsafe usage in safe functions may lead to unsoundness issues. - low severity
If unsafe blocks are used carelessly in safe functions (fn), attackers could exploit undefined behavior (e.g., via dangling pointers or unaligned reads) to leak data, crash the program, or achieve arbitrary code execution.

Show fix

Remediation: Avoid unsafe blocks unless absolutely necessary. If unsafe operations are required, encapsulate them in well-documented unsafe fn functions with clear safety invariants, and minimize exposure to unsoundness code.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[RikHeijdens]

@AikidoSec ignore: We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

Improper unsafe usage in safe functions may lead to unsoundness issues. - low severity
If unsafe blocks are used carelessly in safe functions (fn), attackers could exploit undefined behavior (e.g., via dangling pointers or unaligned reads) to leak data, crash the program, or achieve arbitrary code execution.

Show fix

Remediation: Avoid unsafe blocks unless absolutely necessary. If unsafe operations are required, encapsulate them in well-documented unsafe fn functions with clear safety invariants, and minimize exposure to unsoundness code.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[RikHeijdens]

@AikidoSec ignore: We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

We've double-checked the safety of this operation. The unsafe here is required to read the planes of an FFmpeg AVFrame as Rust slices, and the slice bounds are validated immediately before it.

Since the scan, 8b473ec has made both functions validate for themselves instead of relying on the caller. plane_strides checks that:

• the frame's width, height and pixel format match the layout;
• the layout only admits 8-bit, fully planar, 3-component YUV, so each plane is a separate array of 1-byte samples;
• every plane pointer is non-null, with a non-negative stride of at least its width.

FFmpeg allocates plane buffers for at least the frame's full height (av_frame_get_buffer and the decoder and filter pools pad it to 32 rows). So each slice of stride * (rows - 1) + width bytes stays inside the buffer.

The slices borrow the AVFrame, which keeps the buffer referenced for their lifetime. frame_planes_mut also requires av_frame_is_writable, so the mutable slices can't alias another frame's buffer, and the three planes don't overlap each other.

We also ran the full test suite under AddressSanitizer (nightly, -Zsanitizer=address -Zbuild-std) with no reports. That covered 4:2:0, 4:2:2, 4:4:4, odd and sub-block frame sizes, and real 1080p/2K decodes. A deliberate overrun from the same frames was caught, which confirms the instrumentation was active.