fix(encoder): release MediaCodec output buffers immediately instead of on Frame.close() - #304
Closed
filippoadessi wants to merge 1 commit into
Closed
Conversation
…f on Frame.close() createFrame() held the MediaCodec output ByteBuffer by reference and only called releaseOutputBuffer() in Frame.onClosed(), which only runs after the frame has traveled through the whole downstream pipeline including the network send. MediaCodec's small output buffer pool (~4 buffers) exhausts if the network send lags (real congestion, not hypothetical), stalling the encoder for multi-second gaps. Copy the bytes out and release the codec buffer immediately in createFrame(), decoupling encoder throughput from downstream/network speed (same approach as RootEncoder). Confirmed on-device: encoder output continues uninterrupted under uplink bandwidth constrained below the configured bitrate, no multi-second gaps in the captured output (previously reproducible via byte-level capture).
Owner
|
As said in the link issue it is not correct fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #302
Title: Encoder stalls under downstream backpressure (network congestion / slow endpoint) because MediaCodec output buffers aren't released until the whole pipeline finishes with the Frame
Environment
MediaCodecEncoderconstrained uplink bandwidth, but the mechanism is endpoint-agnostic)
lifetime, not bitstream content)
Symptom
Under real network congestion (limited uplink bandwidth, or a downstream
consumer that reads slower than the encoder produces), the encoder
periodically stops producing output entirely for multiple seconds at a time,
then resumes — visible downstream as multi-second freezes/gaps in the
published stream, confirmed via byte-level capture of the muxed output
(literal gaps of missing bytes, not just paced/throttled ones). This is not a
graceful bitrate/quality adaptation — it's a hard stall of encoder output.
Root cause
MediaCodecEncoder.FrameFactory.createFrame()wraps theByteBufferhandedout by
MediaCodec.getOutputBuffer(index)by reference (no copy) into theFrameobject, and only callscodec.releaseOutputBuffer(index, false)inside the
Frame'sonClosedcallback:Frame.close()(and thereforeonClosed) only runs after the frame has beenfully consumed by everything downstream: the processing pipeline, the muxer,
and ultimately the network send call of the endpoint.
MediaCodechas asmall, fixed pool of output buffers (commonly ~4 on Android hardware
encoders). If the network send blocks or falls behind — exactly the
congestion scenario a live encoder needs to survive — frames stop being
close()d, the output buffer pool is exhausted, andMediaCodecitself stopsbeing able to produce new encoded output until buffers free up. The encoder
is effectively backpressured all the way from the socket write into the
hardware codec, with nothing in between to absorb the mismatch.
RootEncoder (pedroSG94/RootEncoder) doesn't hit this: it copies the encoded
bytes out of the
MediaCodecbuffer immediately and releases the buffer rightaway, decoupling encoder throughput from however fast the network consumer
is.
Fix
Copy the encoded bytes out of the
MediaCodecoutput buffer and callcodec.releaseOutputBuffer(index, false)immediately increateFrame(),instead of holding the codec buffer by reference until
Frame.close():This decouples encoder throughput from downstream/network speed: the codec
buffer pool can never be exhausted by a slow consumer, and the encoder keeps
producing frames into the pool-managed
Framebuffers at its own pace.The one behavioral trade-off is one extra copy of each encoded frame
(
ByteBuffer.allocate+put) compared to the previous zero-copy path —negligible relative to the cost of encoding itself, and the same approach
RootEncoder has shipped for years.
Verified on-device: with uplink bandwidth constrained below the configured
bitrate, encoder output continues uninterrupted (no multi-second gaps in the
muxed output; previously reproducible via byte-level capture). Core unit
tests pass.