Skip to content

Add Vulkan NV12 texture handle support for QVideoFrame/QML rendering - #725

Draft
valbok with Copilot wants to merge 10 commits into
masterfrom
copilot/add-vulkan-handle-to-qavvideo-frame
Draft

Add Vulkan NV12 texture handle support for QVideoFrame/QML rendering#725
valbok with Copilot wants to merge 10 commits into
masterfrom
copilot/add-vulkan-handle-to-qavvideo-frame

Conversation

Copilot AI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Vulkan-decoded NV12 frames were exposed to Qt as if both planes shared the same importable texture handle, which breaks QML rendering and produces the green/noisy output shown in the issue. This change aligns the Vulkan frame export path with Qt's per-plane texture import expectations.

  • Vulkan frame allocation

    • Configure FFmpeg Vulkan frames with AV_VK_FRAME_FLAG_DISABLE_MULTIPLANE so NV12 is backed by importable per-plane VkImages instead of a single multi-planar image.
  • QVideoFrame handle plumbing

    • Add QAVVideoFrame::VulkanTextureHandle.
    • Map Vulkan-backed QAVVideoFrame instances to Qt's RhiTextureHandle path in QVideoFrame conversion.
  • Per-plane Vulkan handle export

    • Introduce Vulkan-specific video buffer handling for Qt 6.
    • Export one VkImage handle per plane for Vulkan/NV12 frames instead of duplicating img[0].
    • Centralize Vulkan handle extraction and guard against invalid frame context / plane count mismatches.
  • API contract

    • Document that QAVVideoFrame::handle() may return a QList<quint64> for multi-plane hardware frames.
  • Coverage

    • Extend the existing integration test path to assert Vulkan handle extraction when Vulkan decoding is active.
QList<quint64> textures;
for (int plane = 0; plane < planeCount; ++plane)
    textures.push_back(quint64(vk_frame->img[plane]));
return QVariant::fromValue(textures);

Copilot AI linked an issue Sep 11, 2026 that may be closed by this pull request
Copilot AI and others added 5 commits September 11, 2026 06:49
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Copilot AI changed the title [WIP] Add Vulkan handle support to QAVVideoFrame Add Vulkan NV12 texture handle support for QVideoFrame/QML rendering Sep 11, 2026
Copilot AI requested a review from valbok September 11, 2026 06:55
@valbok

valbok commented Sep 11, 2026

Copy link
Copy Markdown
Owner

@copilot It does not work:
Run like this:

$ QSG_RHI_BACKEND=vulkan ./qml_video ../../tests/auto/integration/testdata/DHC0413_CreaseOrNot.mp4 
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447
Missing Vulkan image for plane 1 of 2
No Vulkan image handles in the frame 1.13447

Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Comment thread src/QtAVPlayer/qavhwdevice_vulkan.cpp Outdated
Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>

Copilot AI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@copilot It does not work:
Run like this:

...

Addressed in aa13c2c. The Vulkan frames init now overrides FFmpeg's prefilled multi-planar NV12 format with the per-plane fallback formats before av_hwframe_ctx_init(), instead of falling back to img[0] when plane 1 is missing.

Copilot AI requested a review from valbok September 11, 2026 07:19

if (planeFormats && planeCount > 1) {
for (int plane = 0; plane < AV_NUM_DATA_POINTERS; ++plane)
hwctx->format[plane] = plane < planeCount ? planeFormats[plane] : VK_FORMAT_UNDEFINED;

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this now returns vk_frame->img[0] and vk_frame->img[1] images, but rendering is 100% green

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f28e197. The Vulkan handle path is now gated on the FFmpeg frame and Qt QRhi using the same Vulkan device; otherwise it stops exposing raw VkImage handles so Qt falls back to the mapped-memory upload path instead of sampling foreign-device images.

Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Copilot AI requested a review from valbok September 11, 2026 07:44
return nullptr;

const int layoutIndex = vkFrame->layout[plane] ? plane : 0;
if (!texture->createFrom({handle, int(vkFrame->layout[layoutIndex])}))

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this produces green artifacts, but recognizable image

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 686b2ae. When the Vulkan-specific mapTextures() path fails, it now suppresses the generic textureHandle() fallback so Qt drops to the mapped-memory upload path instead of sampling the broken raw Vulkan handles that produced the green artifacts.

Co-authored-by: valbok <1194376+valbok@users.noreply.github.com>
Copilot AI requested a review from valbok September 11, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add vulkan handle to QAVVideoFrame

2 participants