Skip to content

feat(moq-ffi): the decode pixel format reaches every uniffi binding - #3892

Merged
kixelated merged 5 commits into
mainfrom
quest/main/ffi-decode-format
Sep 22, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/main/ffi-decode-format

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

moq_video_decoder_output in libmoq lets a C consumer choose the decoded CPU
pixel format (#3674). MoqVideoDecoderOutput in rs/moq-ffi had only a
best-effort resize and documented that there was no format to choose, so
Python, Swift, Kotlin, and Go consumers always got I420 and had to convert
themselves.

Completes quest/next/ffi-decode-format.md, which moves under quest/main/
(additive record field, no wire change) and is then deleted by this PR.

Approach

  • MoqVideoDecoderOutput gains format, and the delivery loop converts to it
    the way libmoq's already does (into_i420 / to_rgba).
  • MoqVideoDecodedFrame gains format, so a frame names the layout it was
    decoded to rather than the caller having to remember what it asked for.
  • resize stays best effort, but its doc now names the backends: only NVDEC
    has a built-in scaler, every other backend decodes at the stream's size.
  • One RGBA decode test per suite: moq-ffi, Python, Go, Kotlin, and Swift.
  • Each binding's doc/lib/*/index.md states the decode output contract the C
    page's "Raw decode output" bullet does.

format is Option<MoqVideoPixelFormat>, where None means I420. uniffi 0.32
has no enum-valued default in its proc-macro metadata (LIT_* covers str, int,
float, bool, none, some, and empty seq only), so an I420-valued field would have
to be required, which breaks every existing caller. None also lands right in
Go, which gets no uniffi defaults at all and would otherwise read a zero
discriminant. Both records append the field, so no positional constructor in any
binding shifts.

Impact

Public API, all additive:

  • moq_ffi::video::MoqVideoDecoderOutput gains
    format: Option<MoqVideoPixelFormat>, appended, #[uniffi(default = None)],
    where None delivers I420. It also derives Default.
  • moq_ffi::video::MoqVideoDecodedFrame gains format: MoqVideoPixelFormat,
    appended. The library constructs this record; callers only read it.
  • MoqVideoPixelFormat is unchanged, now reachable from the decode side too.
  • Every wrapper re-exports both records by alias, so Python
    VideoDecoderOutput(format=...), Swift VideoDecoderOutput(format:), Kotlin
    VideoDecoderOutput(format = ...), and Go VideoDecoderOutput{Format: &f}
    all follow without wrapper code.

Wire: none. libmoq's C ABI: unchanged.

Verification

  • just check, just rs check, just go check, just kt check,
    just rs test -p moq-ffi (144 passed), just py test (66 passed).
  • just test smoke --all: the full 4x8 matrix passes (rust, python, go, js
    publishers against rust, python, go, js, js-native-node, js-native-bun, c,
    gst subscribers).
  • Not run here: the Swift test. The Linux dev shell ships no Swift toolchain, so
    swift/Tests/MoqTests/SmokeTests.swift is uncompiled; macOS CI is the first
    thing to build it.
  • just dart check fails on this branch and on main alike: dart pub get --enforce-lockfile refuses because test 1.32.0 has been published since
    dart/moq/pubspec.lock was written. Nothing here touches Dart.

Alternatives

  • libmoq reusing the shared record, as the quest's plan asked. Not possible:
    moq-ffi is crate-type = ["staticlib", "cdylib"] with no rlib, so libmoq
    cannot depend on it, and there is no third crate holding FFI-facing types. The
    two crates were split apart in 25573e1be by copying, and every video feature
    since has landed twice by hand. libmoq needs no change here anyway, since its
    C ABI already carries format. See Follow-ups.
  • A required format: MoqVideoPixelFormat, matching MoqVideoEncoderInput.
    Rejected: uniffi cannot default it, so it breaks every released binding's
    constructor, which would make this a dev change rather than an additive one.
  • Enforcing resize on the CPU, as libmoq does. Out of scope: the quest
    keeps resize a hint, and documenting which backend honors it is the honest
    fix.
  • A Dart wrapper field. Dart's bindings are generated with
    --no-default-features, so they carry no codecs and have no decodeVideo to
    extend.

Follow-ups

Worth a quest each, not created here:

  • libmoq and moq-ffi share their video boundary. The two crates duplicate
    finalize, block_on, video_ceiling, follow_reservation, the
    encode::Config assembly, the pixel-format/codec/encoder-kind conversions,
    and the encode-side format-to-Surface match, character for character. A
    shared crate (or an rlib on moq-ffi) would collapse it, but that is an API and
    packaging decision, not a drive-by.
  • Python cannot name a VideoEncoderKind variant without a cast. uniffi
    nests each variant inside the enum without declaring it a subclass, so pyright
    rejects VideoEncoderKind.SOFTWARE(). Go has SoftwareEncoder() and Kotlin
    has softwareEncoder for exactly this; Python has nothing. Belongs to
    quest/next/binding-parity.md.
  • The C decode path enforces resize and the uniffi one does not. libmoq
    CPU-resizes a frame the backend did not scale; moq-ffi hands back the native
    size. Both are documented, but one contract would be better.

🤖 Generated with Claude Code

(written by Opus 5)

@kixelated
kixelated marked this pull request as ready for review September 22, 2026 14:10
@moq-bot

moq-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

No issues found.
Verdict: approve
(Written by Muse Spark)

New%20session%20-%202026-09-22T14%3A13%3A57.501Z
opencode session  |  github run

@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 4965948a-5781-4e79-80c6-c84fcb9e1480

📥 Commits

Reviewing files that changed from the base of the PR and between 5c8d3ff and f63ac23.

📒 Files selected for processing (8)
  • go/wrapper/moq_test.go
  • go/wrapper/types.go
  • kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt
  • py/moq-rs/moq/subscribe.py
  • quest/dev/release.md
  • quest/next/README.md
  • rs/moq-ffi/src/test.rs
  • swift/Tests/MoqTests/SmokeTests.swift
💤 Files with no reviewable changes (1)
  • quest/next/README.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • py/moq-rs/moq/subscribe.py
  • go/wrapper/types.go

Included review availability: Your plan provides up to 4 included reviews per hour; 1 remains after this review.


Walkthrough

The FFI decoder now accepts an optional I420 or RGBA output format and reports that format on each decoded frame. Unset output defaults to I420. Rust, Go, Kotlin, Python, and Swift tests verify buffer sizes and opaque RGBA alpha values. Binding comments and library documentation describe format selection and best-effort resizing. Release planning files now track the additive decoder output field.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to f63ac

The decoder adds optional RGBA output while preserving default I420 behavior and reports each frame’s layout. Cross-binding tests cover the new formats, so the change is merge-ready.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 61.90% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 13 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly explains the decode pixel-format change, compatibility approach, testing, and scope. It directly matches the changeset.
Title check ✅ Passed The title clearly identifies the main change: decode pixel-format support reaching every uniffi binding.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 61.90% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 13 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Review round, both verdicts in: moq-bot says "No issues found. Verdict: approve", and CodeRabbit generated no actionable comments. Two things to answer anyway.

Turning down the Docstring Coverage pre-merge check (52.38% vs an 80% threshold). The 21 functions it counted are almost entirely the new per-binding tests, and every one of them that the repo's conventions ask to document already is: video_decode_format, TestDecodeVideoFormat, test_decode_video_format, and testDecodeVideoFormat each carry a doc comment stating the contract they pin. What the check is scoring as missing is test bodies and the wrapper accessors, and CLAUDE.md asks for a plain line on every exported symbol, which tests are not. Adding docstrings to satisfy a percentage would be noise, so I am leaving it.

Turning down the "resolve merge conflicts" finishing touch. There is no conflict: GitHub reports the PR MERGEABLE against main. That suggestion appears to be spurious.

One fix pushed (10c5632), from macOS CI rather than from the review: the new Swift test did not compile. XCTUnwrap takes an autoclosure, which cannot carry an await, so try XCTUnwrap(try await catalogs.next()) and its two siblings were errors. Each awaited value is now bound first. The Linux dev shell ships no Swift toolchain, so this was the first build of that test, exactly as the PR description flagged. The same commit also corrects a comment in the Rust test that described mid-gray round-tripping while the assertion under it checks alpha opacity.

No behavior, API, or wire change in that commit: it is the test and a comment.

(written by Opus 5)

@kixelated
kixelated force-pushed the quest/main/ffi-decode-format branch from 10c5632 to ff7cd28 Compare September 22, 2026 14:20
@kixelated

Copy link
Copy Markdown
Collaborator Author

Correction to my previous comment: CodeRabbit's merge-conflict note was right, just early. GitHub reported MERGEABLE when I wrote that, and the conflict appeared minutes later when #3889 landed on main and removed the JSON mutate quest from the same two lists this PR edits.

Rebased onto main (ff7cd28). Three conflicts, all in the quest bookkeeping, no code:

  • quest/next/README.md: main dropped the JSON mutate line, this PR drops the Decode format line, so both go.
  • quest/dev/release.md Required list: same, in the intermediate commit and again in the final one.

The net diff against main is unchanged from what was reviewed, and #3889's removals are preserved rather than reverted. Everything outside quest/ is untouched by the rebase.

(written by Opus 5)

@kixelated
kixelated enabled auto-merge (squash) September 22, 2026 14:35
@kixelated

Copy link
Copy Markdown
Collaborator Author

Auto-merge enabled (squash). Summary of what happened after review.

Review verdict. moq-bot: "No issues found. Verdict: approve". CodeRabbit: no actionable comments, merge risk minimal. One round, no re-review requested.

Fixed.

  • The new Swift test did not compile. XCTUnwrap takes an autoclosure, which cannot carry an await, so the three try XCTUnwrap(try await ...) lines were errors. Each awaited value is now bound first. This was the first build of that test anywhere, since the Linux dev shell has no Swift toolchain. macOS CI now runs it: testDecodeVideoFormat passed in 0.775s, 17 of 17 Swift tests green.
  • A comment in the Rust test claimed mid-gray round-tripping while the assertion under it checks alpha opacity. Reworded to match.
  • Rebased onto main after feat(json): edit a snapshot value through a closure #3889 landed and took the JSON mutate quest out of the same two lists this PR edits. Three conflicts, all quest bookkeeping, no code.

Turned down.

  • CodeRabbit's Docstring Coverage check (52.38% against an 80% threshold). The functions it counted are the new per-binding tests, and each one already carries a doc comment stating the contract it pins. What it scores as missing is test bodies, and CLAUDE.md asks for a line on every exported symbol. Padding to hit a percentage would be noise.

Verified locally on the rebased branch, beyond CI: just rs test -p moq-ffi 144/144 including video_decode_format, just py test 66/66, just kt check, just go check.

Unchanged from the reviewed diff: the API shape, the wire (none), and the decision not to touch libmoq. Public API is additive only: MoqVideoDecoderOutput.format: Option<MoqVideoPixelFormat> defaulting to None (I420), and MoqVideoDecodedFrame.format, both appended so no positional constructor shifts.

(written by Opus 5)

@moq-bot

moq-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor
  1. VideoHint doc comment now attaches to VideoDecodedFrame, leaving VideoHint undocumented (go/wrapper/types.go:54, AGENTS.md#public-api). Move VideoHint = ffi.MoqVideoHint directly below its comment so each alias keeps its own doc line.

Verdict: request changes
(Written by Muse Spark)

New%20session%20-%202026-09-22T14%3A33%3A11.776Z
opencode session  |  github run

@kixelated
kixelated disabled auto-merge September 22, 2026 14:36
@kixelated

Copy link
Copy Markdown
Collaborator Author

Taking the second review's finding: agreed and fixed in 57598f1.

// VideoHint supplies catalog fields a video stream can't reveal itself... sat directly above VideoDecodedFrame, so VideoHint = ffi.MoqVideoHint aliased two lines further down with no doc line at all. The alias now sits under its own comment.

For the record, the misplacement predates this PR: it reads the same way on main, and this PR only reworded the VideoDecodedFrame line underneath it. Fixing it here rather than splitting it out, since CLAUDE.md asks to fix stale docs and comments inline and it is two lines inside a block this PR already edits.

One thing I am not fixing, to keep the PR focused: gofmt -l also flags go/wrapper/moq_test.go:376 and several spots in go/wrapper/errors_test.go. Both predate this PR, neither is on a line it touches, and just go check passes, so the repo's Go gate evidently does not run plain gofmt. That belongs in its own change.

Auto-merge was disabled while I pushed this and will go back on once the required checks are green.

(written by Opus 5)

@moq-bot

moq-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

No issues found.
Verdict: approve
(Written by Muse Spark)

New%20session%20-%202026-09-22T14%3A37%3A57.068Z
opencode session  |  github run

@kixelated
kixelated enabled auto-merge (squash) September 22, 2026 14:44
@kixelated

Copy link
Copy Markdown
Collaborator Author

Assessment

  • Positive improvement? Yes. Brings the C ABI’s decode format knob to every uniffi binding: MoqVideoDecoderOutput.format (Option, None = I420) and MoqVideoDecodedFrame.format, with docs, wrappers, and one RGBA round-trip test per binding suite.
  • Worth the complexity? Yes. Additive append-only fields; Option avoids uniffi’s missing enum default and keeps Go (no defaults) correct. Wire and libmoq C ABI unchanged. Honest “resize is NVDEC-only” docs beat pretending every backend scales.
  • Different approach better? A required I420-valued field would break every released constructor—correctly rejected. Sharing records with libmoq needs an rlib/shared crate and is the right deferred follow-up, not a blocker here.

MERGE — clean additive FFI parity with solid cross-binding tests; land when checks that touch rs/moq-ffi / bindings are green.
This is an automated review, not the maintainer's decision
(Written by Grok)

kixelated and others added 5 commits September 22, 2026 08:12
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Additive field on a uniffi record and its wrappers, no wire change, so it
lands on main per quest/README.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`MoqVideoDecoderOutput` gains `format`, so a Python, Swift, Kotlin, or Go
consumer picks I420 or RGBA decode output the way a C consumer does through
`moq_video_decoder_output`. `MoqVideoDecodedFrame` carries the layout it was
decoded to, and `resize` stays best effort but now names the one backend that
honors it.

The field is `Option<MoqVideoPixelFormat>` defaulting to `None`, which means
I420: uniffi 0.32 has no enum-valued default in its proc-macro metadata, so an
I420-valued field would have to be required, breaking every existing caller.
`None` also lands right in Go, which gets no defaults at all. Both records
append the field, so no existing positional constructor shifts.

libmoq is unchanged: its C ABI already carries `format`, and it cannot reuse
the shared record because moq-ffi is a staticlib/cdylib with no rlib. Dart
ships no codecs, so it has no decode API to extend.

One RGBA decode test per binding suite: moq-ffi, Python, Go, Kotlin, and
Swift (uncompiled here, since the dev shell has no Swift on Linux).

Completes the quest.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
XCTUnwrap takes an autoclosure, which cannot carry an await, so macOS CI
refused to compile the new test. Bind each awaited value first.

Also correct a test comment that described mid-gray round-tripping while the
assertion below it checks alpha opacity.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The VideoHint comment sat directly above VideoDecodedFrame, so VideoHint
aliased undocumented two lines down. Move the alias under its comment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the quest/main/ffi-decode-format branch from 57598f1 to f63ac23 Compare September 22, 2026 15:15
@kixelated

Copy link
Copy Markdown
Collaborator Author

Rebased again (f63ac23): #3885, #3891 and #3894 landed on main while the first rebase was in CI.

Five conflicts this time, one of them in code:

quest -- check passes (359 documents ok) and just rs test -p moq-ffi is 144/144 on the rebased tree. The net diff against main is unchanged from what was approved.

Worth noting for anyone reading the earlier comment: #3894 landed the Dart lockfile refresh, so the Check job that just dart check was breaking is green again on its own.

Auto-merge is still armed.

(written by Opus 5)

@moq-bot

moq-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

No issues found.
Verdict: approve
(Written by Muse Spark)

New%20session%20-%202026-09-22T15%3A15%3A35.198Z
opencode session  |  github run

@kixelated
kixelated merged commit 84b34f5 into main Sep 22, 2026
24 checks passed
@kixelated
kixelated deleted the quest/main/ffi-decode-format branch September 22, 2026 15:40
@moq-bot moq-bot Bot mentioned this pull request Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant