Add Vulkan NV12 texture handle support for QVideoFrame/QML rendering#725
Conversation
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
|
@copilot It does not work: $ QSG_RHI_BACKEND=vulkan ./qml_video ../../tests/auto/integration/testdata/DHC0413_CreaseOrNot.mp4
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447 |
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
|
|
||
| if (planeFormats && planeCount > 1) { | ||
| for (int plane = 0; plane < AV_NUM_DATA_POINTERS; ++plane) | ||
| hwctx->format[plane] = plane < planeCount ? planeFormats[plane] : VK_FORMAT_UNDEFINED; |
There was a problem hiding this comment.
@copilot this now returns vk_frame->img[0] and vk_frame->img[1] images, but rendering is 100% green
There was a problem hiding this comment.
Addressed in f28e197. The Vulkan handle path is now gated on the FFmpeg frame and Qt QRhi using the same Vulkan device; otherwise it stops exposing raw VkImage handles so Qt falls back to the mapped-memory upload path instead of sampling foreign-device images.
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
| return nullptr; | ||
|
|
||
| const int layoutIndex = vkFrame->layout[plane] ? plane : 0; | ||
| if (!texture->createFrom({handle, int(vkFrame->layout[layoutIndex])})) |
There was a problem hiding this comment.
@copilot this produces green artifacts, but recognizable image
There was a problem hiding this comment.
Addressed in 686b2ae. When the Vulkan-specific mapTextures() path fails, it now suppresses the generic textureHandle() fallback so Qt drops to the mapped-memory upload path instead of sampling the broken raw Vulkan handles that produced the green artifacts.
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Vulkan-decoded NV12 frames were exposed to Qt as if both planes shared the same importable texture handle, which breaks QML rendering and produces the green/noisy output shown in the issue. This change aligns the Vulkan frame export path with Qt's per-plane texture import expectations.
Vulkan frame allocation
AV_VK_FRAME_FLAG_DISABLE_MULTIPLANEso NV12 is backed by importable per-planeVkImages instead of a single multi-planar image.QVideoFrame handle plumbing
QAVVideoFrame::VulkanTextureHandle.QAVVideoFrameinstances to Qt'sRhiTextureHandlepath inQVideoFrameconversion.Per-plane Vulkan handle export
VkImagehandle per plane for Vulkan/NV12 frames instead of duplicatingimg[0].API contract
QAVVideoFrame::handle()may return aQList<quint64>for multi-plane hardware frames.Coverage