Skip to content

feat(ffi)!: Copy types changed in place cross by value - #808

Merged
kixelated merged 8 commits into
mainfrom
quest/m1/ffi-export/copy-values
Nov 14, 2026
Merged

kixelated merged 8 commits into
mainfrom
quest/m1/ffi-export/copy-values

Conversation

@kixelated

@kixelated kixelated commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes Copy types that change cross by value. A Copy type with a pub field or a pub &mut self method now crosses as an opaque value, as other Copy types have since #742. It is no longer an Arc handle. A change reaches only the copy it was made on, as in Rust.

Changes

  • collect.rs: changed is gone, so such a type is a value. A value's pub fields export as a getter and setter, as an object's do. Its &mut self methods export too, except two that are refused at their span (written_back): one returning a future and one returning a Result.
  • C ABI: a value's &mut self, its field setters included, takes <T> *self (the new Ty::InOut). The glue writes back what the call left through it.
  • C++: the method is non-const and assigns *this. Swift: the method is mutating and assigns self. Go: the method has a pointer receiver and assigns *self.
  • Kotlin, Python, Dart: the method returns the value as the call left it: x = x.add(o). If Rust also returns something, the method returns a Pair, a tuple, or a record, with each element of a returned tuple after the value. Abi::handed_back gives each generator that function's signature and a Handed, whose call writes the glue's call inline: it copies the value, calls the in-place C function on the copy, and writes the copy (and Rust's return) to the outs. There is no separate Rust shim. All three generators' fast paths (flat records, tuples, written-back Mut arguments) work unchanged.
  • README: the rules table, the C ABI section, and each language's section are updated. The quest file is deleted, along with its links from ship.md and kotlin-values.md.

Public API

Breaking (!). Every generated binding holds these types differently:

  • moq's stats::Traffic, stats::Presence, stats::Failures, and stats::Refusals are now values. Their fields are <field>_get/<field>_set accessors, and add writes back in place in C, C++, Swift, and Go.
  • In Kotlin, Python, and Dart, add returns the new value: t = t.add(o).
  • <T>_free and <T>_clone of these types are gone.

Migration: in C, hold moq_stats_Traffic by value and pass &t. Drop frees and clones. In Kotlin, Python, and Dart, reassign the method's result. No wire impact.

Decisions

Settled by the maintainer on 2026-11-14 (comment):

  • ✅ Refuse a value's &mut self method returning a Result or a future at its span in every language (rejected: refuse only in Kotlin, Python, and Dart). In those three the error would be raised as an exception, losing a change Rust made before failing, and a future outlives the call that has to hand the value back. Refusing in one language only would split one Rust type's API. None of moq's methods is affected; the reject fixture covers both refusals.
  • ✅ Kotlin, Python, and Dart flatten a returned tuple after the value (rejected: nest it). For example absorb -> (u64, u64) returns Triple<Score, ULong, ULong>. Nesting would need a C tuple struct the API never declares.
  • ✅ The Kotlin, Python, and Dart hand-back is written natively in each generator's call path (rejected: a Rust back_<C name>::call shim in the glue). Each generator's glue call (the JNI native, the Python glue function, the Dart serve thunk) now copies the value, calls the in-place C function on the copy, and writes the copy back inline. Handed::call in abi.rs builds that expression once for all three, so the argument mapping lives in one place.

Validation

  • The api fixture's differential test compares every language's transcript with native Rust's. It passes in all seven languages: C, C++17/20, Swift, Go, Kotlin (JVM), Python (plus api_py_types), and Dart.
  • The new transcript lines exercise these cases:
    • Score::take (&mut self returning a value), mutated on a copy, with the original left as it was.
    • Score::absorb (&mut self plus a written-back &mut Score argument, returning a tuple).
    • Hits: #[non_exhaustive] with all fields pub, as moq's stats are. It is changed by add and by a setter, and a copy kept before the change is unchanged.
    • Extent: a pub field beside a private one, with slide returning the old start, and a setter.
    • Hits::eq(&mut self, …): its C name clashes with PartialEq's api_Hits_eq, so it is renamed api_Hits_eq_. The hand-back must keep the renamed name; the local review found this case.
  • just ffi test (all fixture tests, including reject with its two new refusals) passes.
  • just ffi moq exports moq at its pin, builds the export, and runs its C, C++, Swift, Kotlin, Python, Go, and Dart programs and linters. All pass. In the header, moq_stats_{Traffic,Presence,Failures,Refusals}_add(moq_stats_X *self, moq_stats_X other) take the value in place.
  • just ci passes on the head.
  • After the shim was replaced by the inline call: the api transcripts pass again in all seven languages, and just ffi moq and just ci pass. The generated Kotlin, Python, and Dart glue has no back_ module. Its inline calls keep the renamed api_Hits_eq_ and pass absorb's tuple outs after the value's.
  • Local review: quest review --harness codex (gpt-6.1-sol, high). On the earlier shim head, round 1 found one medium issue (a renamed C name lost in the hand-back), fixed in d318c87. On this head, a new session's round 1 flagged that a value's &mut self returning a borrow of itself would not compile in the glue. That was refuted: collect.rs already refuses it at its span. A reject fixture case, Counter::bump_ref, now covers it. Round 2 found nothing (10/10). Main was then merged in (only reject/expected.txt conflicted; it was regenerated), and round 3 on the merge found nothing (10/10). just ci and just ffi moq pass on the merge.
  • Not run: just ffi sanitize (it needs the sanitize shell) and the nightly shapes run. The bench was not re-measured, since this PR makes no performance claim.

Follow-ups

  • clock::WallMap stays a Copy handle. Its Option<(Instant, Wall)> leaves bytes uninitialized for None, so it fails the "value-independent padding" rule that this quest's Goal assumes. The quest listed it among the five, but the padding rule, not &mut self, is what keeps it a handle. Making it a value would take a padding rule that handles enum variants, which is close to padding-layout's open decision. No new quest is added here.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, M)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Summary by CodeRabbit

  • New Features
    • Eligible copyable values with public fields or mutable methods are exposed as independent values across supported language bindings.
    • Mutable method calls return the updated value in bindings that use immutable-by-reference values; C and C++ calls update the value passed to the call.
    • Public fields on eligible values are available through generated getters and setters.
  • Documentation
    • Updated FFI guidance to explain copy behavior, field access, and how mutable calls return or write back updated values.

Review

Rewritten by quest review after each round.

  • Head: 82443af

  • Base: origin/main at 0f2331b

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1db7c-8957-7873-9e46-5226f94a8527, round 3

  • Score: 10/10, The merge preserves the reviewed behavior and correctly updates rejection diagnostics, with no new findings from code inspection; builds and tests were not run as instructed.

  • Rounds: 3; first round 1 (medium 1); findings 1 (medium 1): fixed 0, refuted 1, open 0

  • Findings: none

@coderabbitai

coderabbitai Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Eligible opaque Copy + Send values can expose public fields and mutable methods. The FFI passes mutable copies through updated-value paths. Language bindings and fixtures now reflect these value semantics.

Changes: Opaque Copy Value Handling

Layer / File(s) Summary
Value classification and eligibility
ffi/src/collect.rs, ffi/src/ir.rs, ffi/fixtures/reject/*, ffi/README.md, quest/m1/ffi-export/*, quest/m1/ffi-speed/kotlin-values.md
The collector allows eligible Copy values with public fields or mutable methods and generates field accessors. It excludes mutable methods that return a future or Result. The README and rejection fixtures describe these rules; related planning text and links were removed.
ABI and Rust write-back
ffi/src/abi.rs, ffi/src/c.rs, ffi/src/cpp.rs, ffi/src/glue.rs, ffi/README.md
Mutable value receivers use an in-out pointer. Rust glue and C++ wrappers write updated receiver values back after calls.
Language binding integration
ffi/src/dart.rs, ffi/src/go.rs, ffi/src/kotlin.rs, ffi/src/python.rs, ffi/src/swift.rs, ffi/README.md
The language generators route eligible mutable value methods through updated-value handling. Generated bindings use language-specific receiver or return behavior and document the updated copy.
Value API examples and tests
ffi/fixtures/api/src/lib.rs, ffi/fixtures/api/examples/transcript.rs, ffi/tests/api.*
The fixtures add Score, Hits, and Extent operations and test copied values, mutable calls, field updates, and method results across the supported language examples.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Binding as Generated binding
  participant ABI as C ABI
  participant Glue as Rust glue
  participant Method as Rust mutable method
  Binding->>ABI: Pass receiver as an in-out pointer
  ABI->>Glue: Call wrapper with receiver pointer
  Glue->>Method: Call method with mutable receiver
  Method-->>Glue: Return method result
  Glue-->>ABI: Write updated receiver and result
  ABI-->>Binding: Return call outputs
Loading
Merge Risk: Minimal · up to 82443

This change makes certain Copy types cross the FFI by value, which removes _free and _clone for the affected types. The change is documented and tested across the language bindings. No concrete merge-blocking issue was found.

Security architecture risk: Minimal · up to 82443

This is a breaking FFI change, but the inspected paths use per-copy ownership, reject unsupported mutation lifetimes and retain compatibility checks. No introduced security or failure-containment defect was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The demonstrated exposure is the in-process foreign-language-to-native boundary for APIs exporting eligible mutable Copy types. Native callers supply receiver storage that Rust reads and writes; the supplied public-entrypoint signals for Counter and the Python script are rejection and binding-test fixtures, respectively.

Trust Boundaries and Controls

  • observed — The collector rejects mutable value methods whose future or Result would conflict with handing mutation back, and rejects returns borrowing a value being written back. The retained-method path therefore avoids those unresolved receiver lifetime and error-state transitions.
  • observed — Kotlin consumes the transformed read-only receiver signature, not the raw in-out receiver. Returned outputs use signature-derived native locals and serialization; opaque word-array inputs retain explicit length checks and output serialization reserves storage before copying.

Resilience and Maintainability Implications

  • observed — Receiver publication occurs after normal method return. Panics retain the existing documented abort behavior, rather than exposing a recoverable partially updated receiver. Mutable value methods returning futures are refused, so this receiver transition has no asynchronous cancellation or recovery phase.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: Copy types that change in place now cross the FFI by value. The raw summary confirms that Copy types with public fields or &mut self methods became opaque values, the …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: Copy types that change in place now cross the FFI by value. The raw summary confirms that Copy types with public fields or &mut self methods became opaque values, the C ABI gained an InOut receiver, and the handle-based _free and _clone calls were removed for affected types. The title is specific and matches the changeset. The conventional-commit prefix and the breaking-change marker are consistent with the removed APIs.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d08e65cd-37f1-4f36-b36e-0ec2805d0054
Commits

Reviewing files that changed from the base of the PR and between 7e82a7a and 82443af.

Files selected for processing: 5

  • ffi/fixtures/reject/expected.txt
  • ffi/fixtures/reject/src/lib.rs
  • ffi/src/glue.rs
  • ffi/src/kotlin.rs
  • ffi/tests/api.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

kixelated and others added 3 commits November 14, 2026 03:05
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Copy types that change cross by value feat(ffi)!: Copy types changed in place cross by value Nov 14, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 14, 2026 11:41
pullfrog[bot]
pullfrog Bot previously approved these changes Nov 14, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

I reviewed the whole PR: a Copy type changed in place now crosses as an opaque value, and its &mut self methods and field setters are written back in each language.

  • Values with pub fields or &mut self methods: changed() is gone, so these types are now Decl::Value. A value struct's fields get accessors. written_back refuses, at its span, a &mut self method that returns a future or a Result.
  • Ty::InOut in the C ABI: a value's &mut self is passed as <T> *self. The glue lifts a mut copy, calls with &mut a_self, and writes it back through this before finish.
  • In-place languages: C++ assigns *this. Swift is mutating with a defer { self = … }. Go uses a pointer receiver and assigns *self.
  • Hand-back languages: in Kotlin, Python, and Dart, Abi::handed_back builds a &self clone that returns (Self, …ret). It keeps the real method's renamed C name. A back_<C name>::call shim copies *p0, calls the real C function, and writes the copy to the first out.
  • Docs and quest: I checked the README rules table and each language's section against the generators. The quest file and its links are removed.

I checked the shim's parameter mapping by exporting a scratch crate whose value had &mut self methods returning Option<u64>, String, and a 4-tuple, plus ones taking closures and &mut Self with &str. Real params are [self, params, written outs, tuple outs]; the back shim's are [self, params, written outs, out_v, …]. Every index lined up. The glue built with --features kotlin,python,dart. A &mut self -> &u64 on a value is already refused at its span (returned borrow of a value written back), so the new in-place write-back can't hit a glue borrow conflict.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-14, via /quest-run):

  • ✅ /quest-merge pick (rejected: skip, /quest-delete).
  • ✅ A value's &mut self method returning a Result or a future is refused at its span in every language (rejected: refuse only in Kotlin, Python, and Dart).
  • ✅ Kotlin, Python, and Dart flatten a returned tuple after the value (rejected: nest it).
  • ✅ The Kotlin, Python, and Dart hand-back is written natively in each generator's call path, not as a Rust back_<C name>::call shim in the glue (rejected: the Rust shim).

(Written by Claude Opus 5.5 with high effort)

kixelated and others added 2 commits November 14, 2026 11:05
… Rust shim

Kotlin's, Python's, and Dart's glue now call a value's in-place C function
on their own copy and write that copy back inline. Abi::handed_back no
longer builds the back_<C name>::call module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Applied the maintainer's decision on head 7e82a7a: the Rust back_<C name>::call shim is gone. Kotlin's JNI native, Python's glue function, and Dart's serve thunk each copy the value, call the in-place C function on the copy, and write the copy back inline. Handed::call in ffi/src/abi.rs builds that expression once for all three. The other two decisions were already implemented and are now recorded with ✅ under Decisions. The api transcripts pass in all seven languages, and just ffi moq and just ci pass.

(Written by Claude Opus 5.5)

# Conflicts:
#	ffi/fixtures/reject/expected.txt
pullfrog[bot]
pullfrog Bot previously approved these changes Nov 14, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Since d318c87, the Rust back_<C name> shim is replaced by an expression each generator writes into its own glue call. A reject case also now covers a value's &mut self that returns a borrow of itself.

  • abi::Handed replaces the shim: handed_back returns a Handed { name, lead, unit, tuple } instead of shim source. Handed::call emits an inline unsafe block that copies *args[0], calls the real C function with &mut handed_, and writes the copy (and Rust's return) through the outs. I compared it with the removed shim branch by branch. The argument slicing (args[1..=lead], then the tuple outs from lead + 2) and the three return shapes are unchanged. It still uses the real, possibly renamed, C name.
  • Kotlin, Python, and Dart call sites: the Kotlin rust_call closure, Python's rt::call(|| …), and Dart's served thunk (keyed by C name in the new BTreeMap) use Handed::call. Each argument expression still appears exactly once, so it is evaluated once, as it was when passed to the shim. The unsafe block nested inside Dart's outer unsafe is covered by the glue's crate-wide #![allow(unused_unsafe)].
  • Counter::bump_ref reject case: the fixture line (1218) and the expected diagnostic, including its place in the sorted output, match collect.rs's "returned borrow of a value written back" refusal.

I checked this by reading the code. No nightly toolchain was cached in this shell, so I did not rebuild the glue. The PR body reports the api transcripts passing in all seven languages on this head.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Since 7e82a7a, the only new commit merges origin/main (0f2331b) into the branch.

  • Regenerated reject/expected.txt: lines that main removed moved the fixture source up by 9 lines. I checked every entry the range-diff touched against the merged ffi/fixtures/reject/src/lib.rs. Counter::try_bump is at 1199, bump_later at 1204, and bump_ref at 1209. hidden_* and boxed_bytes are at 1168 to 1183, poll_ready at 127, Link::session at 144, and measure at 161. All of them match, and their order follows the file's existing byte-wise sort.
  • kotlin.rs context: the range-diff's only other change is main renaming _Native.Pollable to _Native.__Pollable. It sits next to this PR's Kotlin code but changes none of the PR's own lines.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit 039280e Nov 14, 2026
9 checks passed
@kixelated
kixelated deleted the quest/m1/ffi-export/copy-values branch November 14, 2026 20:04
kixelated added a commit that referenced this pull request Nov 14, 2026
Main's #808 makes Score an opaque value, so the refusal fixtures hold a new
checked Copy handle, Mark. Its handed-back `&mut self` calls (#808) now pass a
status, which `Handed` places last; `Score::note(&Mark)` covers it. #807's
underscore names extend to `__Close` and `__Refused`, and Swift's moq program
drops the `try`s #808's values made unneeded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant