Skip to content

perf(ffi): Go's calls pass C no pointer to a Go local - #837

Merged
kixelated merged 6 commits into
mainfrom
quest/m1/ffi-speed/go-pointers
Nov 16, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m1/ffi-speed/go-pointers

Conversation

@kixelated

@kixelated kixelated commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m1/ffi-speed/go-pointers.md: a generated Go call no longer passes C a pointer to a Go local, so cgo moves none of them to the heap.

Summary

kixel-ai/rust2#829 gave each checked call a C shim in the package's cgo preamble that kept the status in a C local. This PR extends that shim to every C call whose parameters include an In, Out, or InOut pointer, checked or not, and replaces checked_c/call_checked with one call. Each such call now goes through <prefix>GoCall_<name>, which:

  • takes each value C reads (In: a value receiver c_self_, Equal's c_other_) by value, and passes C a pointer to its own copy;
  • keeps each value C writes (Out: c_out_, c_out<k>_, a &mut param's written value, c_err_, the status, a future's handle f_) in a zeroed C local and returns it;
  • returns what an InOut (&mut self of a value) holds after the call, which Go writes back to *self.

A call with one result returns it by itself, and one with more returns them in a <prefix>GoReturned_<name> struct. Go binds each result to the local the old code passed by pointer, so the rest of each call site stays as it was. A C local never moves, and cgo's wrapper writes the returned value into the goroutine's stack even after a callback moved that stack. So the shim works no matter when C writes, including after a callback, like #829's status.

The C locals start zeroed so Go never reads a stale word as a pointer where C wrote nothing, as when a poll pends or a check fails. The GC scans those fields.

Every Go call site goes through it: functions and methods, eager and lazy futures (make, start, poll), the String/GoString/Clone/Equal traits, and a checked type's drop_. Calls that pass no such pointer emit the same code as before.

The C ABI, the Rust glue, and every generated Go API are unchanged.

Validation

  • Escape analysis (go build -a -gcflags=-m ./...), before → after:
    • The api fixture's generated packages: c_self_ 201 → 0, c_out_ 71 → 0, c_other_ 65 → 0, c_err_ 35 → 0, f_ 20 → 0, c_out0_ 15 → 0, c_out1_ 14 → 0.
    • moq's packages, exported by just ffi moq after merging main: no c_*_ or f_ moves left.
    • What still moves is outside this quest: v (the *T an Option result lifts to), s/b in runtime helpers, and k*_ (the cgo.Handle a lent closure or reader passes as its data; see Decisions).
  • Tests: just ffi test (api fixtures in every language, including api_go's script under go vet), just ffi shapes (the nightly round trip of every sampled type shape in every language: shapes::nightly::go ok), just ffi moq (moq's export through C, C++, Kotlin, Python, Swift, Go, and Dart), and just ci. All pass, re-run after each merge of main (Shared borrows that don't contend #834, feat(ffi)!: lossless binding types #832).
  • Benchmark: Go is not on CodSpeed (bench/ffi/README.md says why), so the numbers below were taken by hand. Base is this branch's merge base (06efe2d) exported by its own exporter. Both builds link the same libmoq.a and ran alternately on a host shared with other agents (load average ~10-13 on 32 threads).
    • A focused probe (the bench's calls alone, 20 interleaved rounds of 200k calls, medians, heap allocations per call from runtime.MemStats):

      case base ns branch ns Δ base allocs/call branch allocs/call
      record return 57.9 57.5 -0.8% 0 0
      object return 251.5 263.9 +4.9% (noise, IQRs 240-284 vs 249-334) 3 3
      write 427.9 407.8 -4.7% 1.05 0.03
      stamped write 489.6 459.6 -6.1% 3.05 0.03
      TimestampFromMicros 59.5 54.5 -8.4% 2 0
    • The bench's own Go program (ffi/moq/bench/export.go), 12 interleaved rounds pinned to the same 8 CPUs, wall-time medians: record 65.2 → 62.2, object 243.8 → 229.5, stamped 331.7 → 317.4. The write row is bimodal on this host, with IQRs of 234-420 on both builds, so it cannot resolve a 20 ns change. An earlier unpinned 10-round pass was just as noisy: stamped read +25% there, while call and getter, whose code did not change, moved 3-4%. The allocation counts are the measurement that doesn't depend on load.

    • Object and record return already passed C no Go-local pointer after perf(ffi): Go's checked calls keep their status in C #829, which moved their status into C. So they do not move here; their 3 allocations are the object and its cleanup, which object-ownership owns.

Decisions

  • One generic shim for every call that passes a pointer, not just checked ones. Rejected: extending only the checked shim as the Plan worded it, which would leave the unchecked calls' c_self_/c_out_/c_other_ (most of the 201/71/65) on the heap, though the Goal counts them.
  • The shim's C locals are zeroed (= {0}). Rejected: leaving them uninitialized, as the Go side would then copy stack garbage into fields the GC scans as pointers whenever C wrote nothing (a pending poll, a failed check).
  • Results bind to the old local names (c_out_ := c_ret_.p2), and fields are named by position (p<i>, plus value and status). Rejected: naming fields after the C parameter, which could collide with value/status or a Go keyword under cgo's renaming.
  • A lazily made future that fails its check is freed only once made (if c_status_ == 0 { free }). Before, the code freed whatever the shim returned. Rejected: binding the status and leaving it unused, which Go refuses to compile.
  • The lent closure's and reader's cgo.Handle (k*_) still moves to the heap. It is outside the Goal's list. Moving it into C means special-casing Ty::Data, and the cgo.NewHandle beside it costs far more than one word on the heap.
  • Settled before this PR, in perf(ffi): Go's checked calls keep their status in C #829: keep this quest (rejected: drop it). The Plan's rejections stand: a uintptr hide (checkptr refuses it under -race) and #cgo noescape (it applies only beside nocallback, and these calls can call back).

Public API

None. The generated Go API, the C ABI, and the Rust glue are unchanged. Only the cgo preamble's private shims and the generated function bodies change.

Caused by

None: this is a speedup, not a fix. The escapes are as old as the Go exporter.

Follow-ups

None.

Models

Size: M
Implementer: Claude Opus 5.5, claude-opus-5-5, medium effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with medium effort)

Review

Rewritten by quest review after each round.

  • Head: cc28423

  • Base: origin/main at 72880f4

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, high effort

  • Session: 01a1e0cd-5a60-77f3-b393-02ca219105e3, round 2

  • Score: 8/10, The incremental change correctly relocates a documentation comment and introduces no actionable defects; builds and tests remain unrun, as instructed.

  • Rounds: 2; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

Summary by CodeRabbit

  • Improvements
    • Improved Go-to-C calls involving input, output, and in/out parameters, including returning results and writing modified values back.
    • Updated generated calls for regular functions, futures, trait methods, and object cleanup to handle these values consistently.

Summary by CodeRabbit

  • Improvements
    • Improved Go-to-C calls with input, output, and in/out parameters, including returning results and applying modified values.
    • Standardized generated calls for regular functions, futures, trait methods, and object cleanup, including operations that can report errors.
    • Calls without input, output, or in/out parameters continue to use the direct C-call path.

@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Go code generation now routes calls with input, output, or in/out parameters through generated C shims. Trait operations, object drop, regular functions, and futures use the shared call path. The pointer-passing quest document was removed.

Changes: Go FFI call generation

Layer / File(s) Summary
Shared call-shim generation
ffi/src/go.rs, quest/m1/ffi-speed/go-pointers.md
Gen::call creates shims for calls with input, output, or in/out parameters. The shims copy inputs into C locals and return results and outputs. Calls without these parameters remain direct C calls. The pointer-passing quest document was deleted.
Trait and object call integration
ffi/src/go.rs
Trait operations, value receivers, equality calls, and checked object drop now use the shared call path.
Function and future call integration
ffi/src/go.rs
Regular function calls and future polling and creation use the shared call path. Call statements are emitted only when present, and the separate output-variable declaration helper was removed.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Merge Risk: Minimal · up to cc284

This change routes generated Go FFI calls that pass pointers through C shims so the pointers refer to C-owned locals. No concrete merge-blocking risk was found, and the reported validation covers the changed paths.

Security architecture risk: Minimal · up to c46f3

Calls now use C-owned temporary storage without expanding exposed operations. The inspected failure and cancellation paths preserve checks and cleanup; no material security or ownership regression was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The affected boundary is in-process Go/native memory transfer across regenerated bindings, including mutable receivers and asynchronous operations. The inspected change does not establish a new network entrypoint, tenant boundary, or privilege grant; its security-relevant scope is storage lifetime and ownership within consuming processes.

Trust Boundaries and Controls

  • observed — The shared shim forwards to the selected native function rather than introducing another authority path. Trait calls retain status-based failure handling, and checked object drop still forwards the same handle to the drop symbol and converts its returned status.

Resilience and Maintainability Implications

  • inferred — The inspected future transitions preserve ownership through failure, completion, pending polls, and cancellation. Failed creation returns a null handle, so the new success-only free in the already-cancelled lazy branch is equivalent to the prior null-tolerant cleanup. Start frees completed futures and transfers only pending handles to Go. Go defers freeing retained handles, while await polls serially and returns on completion or cancellation. The existing runtime rejects concurrent polling and polling after completion.
Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "perf(ffi): Go's calls pass C no pointer to a Go local" matches the main change. The PR routes Go FFI calls that take In, Out, or InOut pointers through a C shim, so C receives pointers to C…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title "perf(ffi): Go's calls pass C no pointer to a Go local" matches the main change. The PR routes Go FFI calls that take In, Out, or InOut pointers through a C shim, so C receives pointers to C-owned locals instead of Go locals. The title names the performance goal and the core behavior. It uses a Conventional Commits style prefix, and the wording is compact, though slightly awkward.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5e4b8b61-43b4-465f-8555-57c66a71bcb6
Commits

Reviewing files that changed from the base of the PR and between c46f3a0 and cc28423.

Files selected for processing: 1

  • ffi/src/go.rs
Files skipped from review as they are similar to previous changes: 1

  • ffi/src/go.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

kixelated and others added 4 commits November 15, 2026 10:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Go's calls pass C no pointer to a Go local perf(ffi): Go's calls pass C no pointer to a Go local Nov 15, 2026
@kixelated
kixelated marked this pull request as ready for review November 15, 2026 20:04

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No correctness issues found. One minor suggestion inline.

Reviewed changes

I reviewed the full diff: the Go exporter's new generic cgo shim, and every call site moved onto it.

  • One shim for every pointer-passing call: checked_c/call_checked become Gen::call/call_sig. Any signature with an In, Out, or InOut parameter goes through <prefix>GoCall_<name>. That shim takes values by copy and returns what C writes from a zeroed C local, alone or in <prefix>GoReturned_<name>.
  • Go call sites rebind results to the old local names: plain calls, eager and lazy futures (make/start/poll), the String/GoString/Clone/Equal traits, and a checked drop_ all bind the shim's results to the names they used before (c_status_, c_<n>_, f_, c_self_ write-back), so the code after each call is unchanged.
  • The lazy make frees only on success: this is equivalent, since a failed check's wrap returns a null future.
  • The quest is finished: quest/m1/ffi-speed/go-pointers.md is deleted, and nothing links to it.

I checked the one shape that could emit an unused c_ret_.value statement: a non-void return with outs and no error or status. The ABI produces it only with written &mut outs, and c_out_ := {call} already handles that case. No &c_*/&f_ Go-local pointer remains in go.rs.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread ffi/src/go.rs Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

I reviewed what changed since the prior Pullfrog review (c46f3a0).

  • Doc comment restored: out_name's /// The C name of the out value… line is back on out_name, and then keeps only its own doc. This addresses the prior finding.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 4aa85e9 Nov 16, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/ffi-speed/go-pointers branch November 16, 2026 08:56
kixelated added a commit that referenced this pull request Nov 16, 2026
Its other children landed in #837, #838, and #849, so deleting
shared-borrow.md would leave the epic with no children. Per the
maintainer's decision, delete its README, rewrite the links to it, and
replace bench/ffi's link to the deleted go-pointers.md with #837.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant