Skip to content

feat(ffi): generate C++ bindings over the C ABI - #163

Merged
kixelated merged 5 commits into
mainfrom
quest/m1/ffi-export/cpp
Oct 28, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m1/ffi-export/cpp

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The exporter only emits C. moq ships C++ (its OBS plugin builds C++17 and C17), and the epic needs a C++ generator over the one C ABI, with native futures, that builds at C++17, 20, and 23 with exceptions and RTTI on or off.

Approach

  • src/cpp.rs writes include/<prefix>.hpp beside the C header on every export. It's header-only and has no config. The runtime it shares, runtime/rt.hpp, holds view, buf, result, waker, future, and task. It throws nothing and reads no type information.
  • Names: each C name without the prefix, in a namespace of the prefix. Keywords take a trailing _, as does a member named as its class or one of its fields. Two members that still clash are an export error.
  • Objects are classes that own their handle. They're move-only, and copyable only if Clone, where a copy clones. &self methods are const, and self methods are &&.
  • Records are structs. Enums are an enum class, or for an enum with data a std::variant of a struct per variant, with methods and traits as free functions. Traits are abstract classes passed as a unique_ptr. Bytes go in as a view and come back as a buf that owns Rust's buffer with no copy.
  • Futures:
    • poll(waker) polls once.
    • get() blocks the thread.
    • then(exec, cb) polls on a caller-given executor and returns a task that cancels when destroyed.
    • From C++20, co_await f.via(exec) resumes the coroutine on the executor.
    • A wake that lands during a poll is latched and polls again.
    • A cancel loses nothing, since the C ABI hands out a value only in the poll that finishes.
    • There is one poller at a time, because awaiting moves the future.
  • Closures lent for the call (from feat(ffi): export closures lent for the call #151) are a template parameter taking any callable. A noexcept trampoline lends each object as a reference to its class for that call, then gives the handle back unfreed. If the call moved the object out, it aborts.
  • The header checks <prefix>_contract() against <PREFIX>_CONTRACT at load, before any other call, in an inline variable's initializer (feat(ffi): a contract symbol each header and library check #146). A mismatch aborts.
  • Fixing a bug moq exposed: moq's C header didn't compile as C++ (a field named namespace). C's keyword escape now covers C++ keywords too.
  • Tests:
    • tests/api.cpp runs the transcript script through the C++ header at C++17, 20, and 23, with no exceptions or RTTI at 17 and 23. Its output must match native Rust line for line.
    • The same test then checks the future semantics: the in-poll wake (a new fixture, api::yielded, wakes itself before returning pending), a cancel between wake and poll losing nothing, cancel from the callback, and a coroutine resumed or destroyed. It also checks the abort paths.
    • The same test calls each lent closure, and aborts when a closure moves its lent object into into_label or keeps it past the call. A program whose header names another contract aborts at load.
    • moq/moq.cpp ports moq/moq.c, including owned paths and catalog::Producer::mutate, against the real moq library at C++17 without exceptions or RTTI, in just ffi moq.
    • moq/bench/export.cpp joins just ffi bench, and both benches gained a read path.

Numbers

just ffi bench's export lines, 3 runs, local and loaded host (ns per 100 B frame):

stamped write reused stamp handle call write + read (future per read)
C 222 to 238 211 to 228 6.8 to 6.9 364 to 370
C++ 225 to 236 217 to 220 6.6 to 6.9 374 to 389

Generated size for moq's export: moq.hpp is 1,312,535 B raw and 228,494 B gzipped, on top of moq.h (708,988 B raw, 173,586 B gzipped). The fork's C++ for moq-ffi is 809,789 B raw and 98,503 B gzipped. That's not like for like: the export covers the full pub API of 7 crates, while moq-ffi is a hand-picked surface.

Impact

  • New: include/<prefix>.hpp on every export.
  • New: the C++ header aborts at load when its library is from another build.
  • Breaking (C): a field, parameter, or variant named a C++ keyword takes a leading _ in C (moq's namespace becomes _namespace). See ffi/MIGRATE.md.
  • Breaking (naming rule): the C++ runtime's names (view, result, task, future, poll_t, detail) and every variant's payload name <enum>_<variant> are now claimed, so an item named one of them clashes. See ffi/MIGRATE.md.

Choices made without asking

  • No exceptions anywhere. result<T, E> is its own type, and value() on an error aborts. It isn't std::expected at C++23, because the type would differ by standard, and that's an ODR violation when translation units mix standards (OBS's C++17 plugin inside a C++20 app). For the same reason, via is declared at every standard and only defined from C++20.

  • The C++ header is always written beside the C header, with no flag. A C++-only member clash therefore fails a C-only export too.

  • Payload structs are named <enum>_<variant> even for variants without fields, and that name is claimed in C.

  • The waker/future latch: each poll registers the same waker node, so Rust reuses it. A wake that lands mid-poll sets a flag and re-posts the poll to the executor instead of looping, so a self-waking future can't starve the executor.

  • TSan on a cancel/wake stress test reports races only where the happens-before edge goes through Rust's Arc count, which isn't instrumented. ASan/UBSan/LSan stay clean over 60k iterations, and the counts match, so I'm treating them as false positives.

  • A lent closure is a template parameter, not a type-erased function_ref. It needs no runtime type and no second indirect call, and its trampoline is noexcept, so a throwing callable terminates instead of unwinding through Rust.

Decisions

Settled by the maintainer, keeping each as built:

  1. How async resumes: ✅ a caller-given executor, with no default thread. Rejected: a default background dispatcher, as the fork has; requiring the coroutine's promise type to provide the executor.
  2. What new maps to: ✅ new_, by the one keyword rule. Rejected: a C++ constructor (a fallible new can't be one without exceptions); init, as moq's fork-based C++ has it.
  3. How objects are owned: ✅ move-only owning classes whose copy is a Rust Clone. Rejected: std::shared_ptr<T> everywhere, as the fork has.

Alternatives

  • Generating a .cpp beside the header, as the fork does: a header is simpler to ship and to consume.
  • std::chrono for Duration: Rust's range doesn't fit std::chrono::nanoseconds, so duration stays the exact C struct.

Checks

  • just ffi test: pass (14 tests), including the C++ matrix with GCC 16, after merging main.
  • just ffi moq: C and C++ both pass, after merging main.
  • Clang 21, before the merge (local, nix shell nixpkgs#clang): all 6 standard × exceptions/RTTI combinations pass the transcript and the futures checks.
  • ASan/UBSan/LSan on api.cpp and on a cancel/wake stress test: clean, before the merge.
  • just check: pass
  • Not run: MSVC and Apple Clang. Those are the rewritten quest.

Follow-ups

  • quest/m1/ffi-export/cpp.md is rewritten to what remains: MSVC, Apple Clang, and Clang in CI.
  • Suggested quest: run moq's bench/ffi C++ runner through this header once moq's C++ package moves off the fork.
  • Suggested quest: shrink the generated header, for example by not repeating each item's docs in both the C and C++ headers.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 27, 2026 23:59
Quest-Claim: d41b916a419f47a5
Writes include/<prefix>.hpp beside the C header: header-only C++17/20/23, with
exceptions and RTTI on or off, and futures that poll, block, call back on an
executor, or co_await. C's keyword escape now covers C++ keywords, so the C
header compiles as C++.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title C++ bindings feat(ffi): generate C++ bindings over the C ABI Oct 28, 2026
A closure lent for the call is a template parameter in C++, called through a
noexcept trampoline that lends each object as its class and gives the handle
back unfreed, aborting if the call moved it out. The header checks the
library's contract at load. moq.cpp follows moq.c onto owned paths and
catalog::Producer::mutate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 16:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T17:04:15.935443Z 9761307 New commits
🔒 Security Review ✅ Completed 2026-10-28T17:08:25.820772Z 9761307 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f80f92c6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ffi/src/cpp.rs Resolved
Comment thread ffi/runtime/rt.hpp Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 9f80f92c6d7a96c79886709c0b0e55c91f728c6a.

Two actionable buffer-conversion issues remain, already reported independently; referencing them rather than duplicating inline comments:

  • ffi/src/cpp.rs:167: an engaged optional empty C++ buf lowers its null storage pointer as None. Normalize through view before lowering, preserving Some(empty). Existing finding.
  • ffi/runtime/rt.hpp:119-122: view(nullptr, nonzero) substitutes a one-byte sentinel without changing the length, bypassing Rust's null-buffer validation and allowing out-of-bounds reads. Reject that malformed input; reserve the sentinel for empty buffers. Existing finding.

Overall direction is sound: header-only wrappers reuse the single C ABI, and explicit executor ownership avoids adding a background runtime. The wake latch and cancellation synchronization justify their complexity. No additional actionable bugs found in the inspected generator, runtime, ABI changes and tests.

Verification: static review only; I did not run builds, differential tests, sanitizers or platform compiler checks.

(Written by OpenAI)

…h a length

An engaged optional buffer lowers through a view, so an empty buffer C++ made
crosses as Some, not None. A view of null bytes with a nonzero length aborts
instead of standing in a one-byte sentinel for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 9761307a3305e665b7dbb9fcdb3b050bd1361338, compared with 9f80f92c6d7a96c79886709c0b0e55c91f728c6a.

Both findings referenced in my previous review are fixed: optional buffers now lower through view, preserving Some(empty), and null data with nonzero length aborts before sentinel substitution. The native/C/C++ transcript additions distinguish empty text from absent text, and the C++ abort case covers malformed views.

No new actionable bugs found in this update. These are focused fixes at the shared conversion and validation boundaries, with no additional machinery needed; the overall direction remains sound.

Verification: inspected the complete one-commit delta and relevant surrounding code. Static review only; I did not run builds or tests.

(Written by OpenAI)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Landing at 9761307, reviewed clean by Codex.

  • Decisions settled by the maintainer and recorded in the body: a caller-supplied executor, new_, and move-only owning classes.
  • Merging main added C++ for lent closures (a template parameter), the contract check at load, and moq's owned paths and mutate.
  • Fixed Codex's two findings: empty optional buffers crossed as None, and null views with a length slipped past validation.
  • just ffi test, just ffi moq, and just check pass locally, and CI passes.

(Written by Claude Opus 5.5)

@kixelated
kixelated merged commit c4b7eb0 into main Oct 28, 2026
1 check passed
@kixelated
kixelated deleted the quest/m1/ffi-export/cpp branch October 28, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant