Skip to content

feat(registry): spike fixture package builds and registry constraints - #761

Merged
kixelated merged 20 commits into
mainfrom
quest/m2/registry
Nov 17, 2026
Merged

kixelated merged 20 commits into
mainfrom
quest/m2/registry

Conversation

@kixelated

@kixelated kixelated commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Completes the registry-packaging spike: can one workflow replace hand-maintained binding releases across registries?

Approach

Add a no-input reusable workflow for the fixed api fixture, native CPython wheels, a host macOS XCFramework plus Swift source package, and clean consumer jobs comparing each installed package with native Rust. The macOS builder selects the installed Xcode SDK and exports the fixture suite's current WASI target. The Swift fixture avoids a helper name shadow in Swift 6. Also close a base-branch Node launcher signal race exposed by local CI, with a deterministic regression test. The maintainer accepted the no-go and chose release templates with shared builders. Both artifact consumers passed in GitHub Actions; the completed spike quest is deleted.

The report surveys Maven, npm, NuGet, pub.dev and Go. Accepted direction: release templates with shared builders. Cargo metadata cannot establish registry namespace ownership.

Impact

  • Public API: adds an experimental no-input registry.yml workflow call for this fixture only. No caller-crate packaging or publication is implemented.
  • Wire and generated bindings: unchanged.
  • Adds a Node launcher startup-signal regression to the existing release suite (0.023 s in isolation). Package consumers run nightly and through the existing manual Check workflow; nightly reporting includes their result. Workflow changes select the existing full local suite.
  • Node launcher fallback: supervisor signals are caught before spawning, preventing premature launcher death while its child is starting.

Alternatives

Use maturin for Python alone; reuse BoltFFI's packaging but accept its different generation model; infer names from Cargo. None solves existing registry identities without platform setup.

Decisions

  • Maintainer merge pick, 2026-11-16: ✅ /quest-merge; keep the release-templates follow-up unchanged (rejected: drop it). The coordinator will queue the resulting head.

  • Maintainer decision, 2026-11-16: ✅ accept the report's no-go; the full version is release templates with shared builders (rejected: keep investigating a configless publisher; stop the spike with no full version). Cargo metadata cannot establish registry identities and hosting policy.

Choices

  • Apply the accepted no-go: templates with shared builders are the full version; this PR completes the fixture spike.
  • Use Linux arm64 and macOS arm64, both in crabx's planned matrix, with a host-only XCFramework. No claim of iOS, universal Apple binaries, manylinux portability, or moq migration.
  • Keep Swift source over a binary Rust C ABI, with the exporter's pinned BasicContainers dependency.
  • Pin external calls to an immutable fixture/exporter/tools commit; rust2 calls exercise the triggering commit. Callers need only the workflow call; the spike does not inspect their crate.
  • Use the existing fixture exclusion manifest; this does not establish annotation-free export of arbitrary crates.

Validation

  • Pullfrog pin fix at 8d942c3477176729d4861297cc041c8e3b4ebb19: applied the exact v8.0.2 download-artifact pin already used in nightly.yml and crabx.yml; checked the pinned action's name and path inputs. quest gate nix develop --command just ci and git diff --check passed. Claude round 3 reviewed this head at 9/10 with no findings. The finding thread is answered and resolved. No additional artifact-consumer run was needed for this pin alignment.

  • Repository-pinned nix develop --command quest check: 43 documents pass.

  • actionlint, shellcheck and shfmt for the new scripts pass.

  • Final quest gate nix develop --command just ci: exit 0 after integration fixes, including api_swift and all 92 TS differential fixtures. git diff --check also passed.

  • quest gate nix develop --command bench/registry/build.sh: Linux x86_64 wheel built.

  • Installed the wheel via uv into a fresh CPython 3.14.10 venv with no index/dependencies; ran from a clean directory with -I: byte-for-byte native transcript plus futures, ownership, values and calls passed.

  • Independently verified wheel RECORD coverage, sizes and SHA-256 hashes. After review, stripped Linux RUNPATH and bounded native dependencies and GLIBC versions; installed consumer passed again.

  • Initial export failed because imported Cargo build directories lacked CACHEDIR.TAG; repaired only local generated cache markers. No harness or cache tool was disabled.

  • macOS validation on kixtop, 2026-11-16: macOS 26.6.2 arm64, Xcode 26.6 (17F113), Apple Swift 6.3.3. Both external-call pin 901f7baa and the merged source at 8b7af48d passed. Main was merged only to resolve the Swift fixture conflict, retaining its lossless RustDuration implementation.

  • Ran nix develop --command quest gate bench/registry/build.sh via gated SSH: exit 0; xcodebuild -create-xcframework produced Api.zip with the C static library and Swift source package.

  • Copied only Api.zip, native.txt, consumer.swift, and consume-swift.sh (as verify-swift.sh) into a fresh artifact directory, then ran unzip -q Api.zip. Ran the verifier via nix develop --command quest gate env -i HOME=/Users/kixelated PATH=/usr/bin:/bin:/usr/sbin:/sbin /bin/bash <artifact-directory>/verify-swift.sh <artifact-directory>: exit 0. Apple's SwiftPM built the release consumer, its transcript matched native Rust byte for byte, then it printed futures ok and owned ok.

  • Independent checks: plutil -p Api/ApiFFI.xcframework/Info.plist and lipo -info Api/ApiFFI.xcframework/macos-arm64/libapi.a confirmed one macOS arm64 slice. shasum -a 256 native.txt consumer.*/actual.txt returned 729ffddcec8928c564d29fd1ee3c4fc34ccaecf18900d34be63c4e6fd0a50714 for both merged-source transcripts.

  • Reproduced and fixed the Mac failures: Nix's incomplete SDK/developer-directory selection, a Swift 6 helper name shadow, and the spike's stale wasm target. The older external pin also needed an explicit @Sendable helper type; the merged base no longer captures that helper. Apple Swift still warns about the fixture's separate shared-call stress test's pointer capture; this consumer does not execute that stress test.

  • The first attempt to launch the merged consumer referenced a garbage-collected Quest store path. Using the current pinned CLI to launch the clean Apple environment above succeeded.

  • CI exposed a timeout in crabx's Node signal test. A deterministic regression sent real SIGTERM during spawn and reproduced premature launcher death (exit -15). Installing signal handlers before spawning the child fixes the startup window; the regression passed after the fix and checks signal forwarding and the child's exit code. The final full CI run passed all 28 release tests and all remaining suites.

  • Ran gh workflow run check.yml --repo kixel-ai/rust2 --ref quest/m2/registry. Direct dispatch of the new registry.yml returned 404 because it is not on the default branch; the existing registered Check workflow now invokes it locally. First run 43968322384 built both packages and passed the Linux consumer, but the clean Mac wheel import exposed a Nix-store libiconv dependency. Fixed the builder to use Apple's linker; wheel packaging now normalizes the dylib ID and rejects non-system dependencies. Kixtop was unreachable for this repeat, so validation used hosted macOS.

  • Run 43971814703 invoked the reusable workflow at 13710f7b7f1dcc649fa5cdabff348b497bc835ff. Both builders passed and uploaded registry-ubuntu-24.04-arm and registry-macos-26. Separate consumers had no checkout, Nix, Rust or exporter: they created fresh venvs, installed via pip install --no-index --no-deps artifacts/*.whl, and ran the verifier with Python -I from a clean directory. Linux arm64 and macOS arm64 each printed installed wheel: transcript matches Rust; futures, ownership, values and calls pass.

  • The same macOS 26.7.2 consumer unpacked Api.zip and ran bash verify-swift.sh .: release SwiftPM build passed, transcript matched Rust byte for byte, then futures ok and owned ok. Both consumer workspaces were cleaned. The builder's independent otool -L check showed only the relative dylib ID, system libiconv and libSystem. The external fixture pin now uses this validated revision.

  • Downloaded both run artifacts with gh run download: independently checked complete wheel RECORD coverage, sizes and SHA-256 hashes, and the XCFramework plist's single macOS arm64 slice. Both packaged native transcripts have SHA-256 729ffddcec8928c564d29fd1ee3c4fc34ccaecf18900d34be63c4e6fd0a50714.

  • All five jobs in the dispatched Check run finished successfully, including its existing full just gate job.

  • Review hardening: final quest gate nix develop --command just ci passed with the job timeouts, Linux libm allowlist, Swift-only revision parsing and mismatch diagnostics. A forced transcript mismatch printed a byte-level unified diff and exited 1; importing the wheel packager while refusing reads of ffi/justfile passed. Two consecutive pinned-toolchain cargo rustc -- --print=native-static-libs runs replayed the same note, including the cached 0.00 s run. Real Node signal probes proved that its event-loop turn is required; the signal regression passed in the full suite.

  • No moq migration, release publication, iOS or universal Apple packaging is claimed.

Caused by

  • kixel-ai/rust2#832 - introduced the Swift fixture variable shadowing its debug() helper in Swift 6 (found by check).
  • kixel-ai/rust2#826 - started the Node fallback child before registering supervisor-signal handlers, leaving a startup race (found by check).

Follow-ups

  • Release templates with shared binding package builders, size L: migrate moq's Python and Swift packaging while preserving its current identities, targets, hosting and deployment policy. Other registry constraints stay documented; no open design decision is left in this follow-up. The maintainer kept this follow-up unchanged at the 2026-11-16 merge pick.

Models

Size: M
Implementer: Codex, gpt-6.1-sol, high effort
Implementer: Codex, gpt-6, unknown effort (iterate, XS)

Review

Rewritten by quest review after each round.

  • Head: 8d942c3

  • Base: origin/main at 2c10cde

  • Risk: high, it changes .github/workflows/check.yml, which is always high, as are 2 more of its paths: workflows run with the repository's secrets and decide what merges and ships

  • Classified: high by the always-high paths

  • Reviewer: Claude Code, claude-opus-5-5, medium effort

  • Session: 6d852b42-26ff-4ca5-88a2-1a1ddf458cb1, round 3

  • Score: 9/10, The new commit only changes the download-artifact pin to the v8.0.2 SHA the other workflows use and adds a plan note, so no findings remain, but the registry CI wiring is still high risk.

  • Rounds: 3; first round 9 (medium 3, low 6); findings 9 (medium 3, low 6): fixed 4, refuted 5, open 0

  • Findings: none

Summary by CodeRabbit

  • Chores
    • Added automated checks that build and test Python and Swift packages on supported platforms. These checks use temporary artifacts and do not publish packages.
    • Improved signal handling in the command-line launcher so termination signals are forwarded reliably during startup.
  • Documentation
    • Updated release guidance to describe shared package-building templates and clarify their scope and limitations.

(Written by GPT-6)

Summary by CodeRabbit

  • Bug Fixes
    • Improved signal handling in the crabx launcher so termination signals are forwarded reliably, including when they arrive during process startup.
  • Documentation
    • Added guidance for Python and Swift release templates, including packaging and validation expectations.
    • Updated registry packaging documentation to describe the shared-builder approach and its limitations.
  • Tests
    • Added automated checks for registry-built Python and Swift packages and launcher signal handling.

Quest-Claim: ecf7c9e0a12360fc
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

This PR adds Python and Swift fixture packaging and consumer checks, wires them into check and nightly workflows, and updates the related quest and spike documentation. It also changes crabx signal-handler order during child spawn and renames a local variable in a Swift test.

Changes: Registry packaging spike

Layer / File(s) Summary
Fixture packaging and consumers
bench/registry/build.sh, bench/registry/package.py, bench/registry/consume.py, bench/registry/consume-swift.sh
Adds fixture build scripts, wheel and Swift package creation, and Python and Swift consumer checks. Packaging code validates native dependencies and writes package metadata.
Reusable workflow and CI wiring
.github/workflows/registry.yml, .github/workflows/check.yml, .github/workflows/nightly.yml
Adds a callable and manually dispatched workflow with package and consumer jobs on Linux and macOS. Check and nightly workflows invoke it. The nightly report waits for it.
Quest and spike documentation
bench/registry/README.md, quest/m2/README.md, quest/m2/registry.md, quest/m2/release-templates.md
Documents the fixture validation, workflow scope, and limits. Updates the quest direction to release templates, removes the registry quest, and adds a release-templates quest.

crabx spawn signal handling

Layer / File(s) Summary
Spawn order and regression test
crabx/npm/crabx.js, crabx/test_release.py
spawnAndWait now registers signal handlers before spawning the child. A regression test checks that SIGTERM during spawn is forwarded and that the launcher returns the child exit code.

Swift test cleanup

Layer / File(s) Summary
Local test variable rename
ffi/tests/api.swift
Renames a local test variable from debug to debuggable. The assertion remains unchanged.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller as check.yml or nightly.yml
  participant Registry as registry.yml
  participant Artifacts as uploaded artifacts
  participant Consumer as registry consumer job
  participant Report as nightly report
  Caller->>Registry: invoke reusable workflow
  Registry->>Artifacts: upload package artifacts
  Artifacts->>Consumer: provide artifacts for verification
  Consumer->>Consumer: verify Python and macOS Swift consumers
  Consumer-->>Report: complete before nightly report
Loading
Merge Risk: Minimal · up to 8d942

This change adds CI-only fixture packaging checks and fixes a signal-handling startup race in the crabx launcher. No concrete merge-blocking risk was found.

Security architecture risk: Minimal · up to 8d942

The inspected changes do not establish an introduced or worsened material security or containment risk. Package validation uses read-only repository permissions, passes no caller secrets, and does not publish packages. The launcher change registers signal handlers earlier while preserving its existing child-exit behavior.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The directly affected security scope is the selected CI runners and their accessible workspace and ambient resources, plus processes using the crabx fallback launcher. Downloaded native libraries and verifier scripts execute with consumer-runner authority; a fresh directory and Python isolated mode do not establish host isolation.

Trust Boundaries and Controls

  • observed — The inspected entrypoints are manual dispatch and scheduled execution, not pull-request triggers. Executable artifacts come from the package job in the same workflow run. The issue-writing report remains a separate scheduled-only job and does not execute downloaded package artifacts.

Resilience and Maintainability Implications

  • inferred — Registering launcher handlers before spawn closes the prior default-signal-disposition window. Signal callbacks execute after the synchronous startup block, preserving the child binding before forwarding. Spawn-error termination and child exit-code or signal propagation remain intact; no new orphaning or recovery failure was established.
Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "feat(registry): spike fixture package builds and registry constraints" matches the main change. The PR adds a reusable registry-packaging workflow and bench/registry scripts that build and …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title "feat(registry): spike fixture package builds and registry constraints" matches the main change. The PR adds a reusable registry-packaging workflow and bench/registry scripts that build and test the fixture's Python wheel and macOS XCFramework, and it documents registry constraints in the README and release-templates docs. The title omits the Node launcher signal-handling fix and the release-template docs, but those are secondary to the main change.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0fe58c89-964a-44ff-8bef-8ec1c1b097d4
Commits

Reviewing files that changed from the base of the PR and between 31f4aa2 and 8d942c3.

Files selected for processing: 2

  • .github/workflows/registry.yml
  • quest/m2/release-templates.md
Files skipped from review as they are similar to previous changes: 1

  • quest/m2/release-templates.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

@kixelated kixelated changed the title Spike: publish bindings to every registry from one workflow feat(registry): spike fixture package builds and registry constraints Nov 13, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-11-16, via /quest-run):

  • ✅ Accept the report's no-go: the full version is release templates with shared builders, since registry identities (Maven groupId and signing, npm scope, NuGet owner, pub.dev publisher) cannot come from Cargo.toml. (rejected: keep investigating a configless publisher; stop the spike with no full version)

(Written by Claude Opus 5.5)

Copy link
Copy Markdown
Collaborator Author

Applied the maintainer's decision in 9677e99: accept the report's no-go, with release templates and shared builders as the full version. The quest records the decision and rejected alternatives; the PR body records it under Decisions. Added the release templates follow-up for moq's Python and Swift releases.

The remaining reusable-workflow validation is now running through the branch's manual Check invocation. No merge has been selected.

(Written by GPT-6)

@kixelated
kixelated marked this pull request as ready for review November 16, 2026 23:11

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Head: 31f4aa2c31cc07f7de4639687717b3f7d3b78d12

No blocker for the fixture spike. The CPython tag matches the generator’s stable-ABI setting, and the clean consumers compare installed artifacts against native Rust rather than importing from the build tree. The Swift artifact establishes the host slice, not a portable release matrix. Shared builders plus per-project release templates are the right boundary here: the fixture proves packaging, not registry ownership or publishing policy.

The Node launcher now registers signal handlers before spawn, and the regression exercises a real OS signal during the startup window. I verified the successful consumer jobs at 13710f7b, not this head. Since this head changes the workflow timeouts and packaging checks, rerun Check here before treating that evidence as current. Static review; I did not build or publish anything.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one pinning nit inline.

Reviewed changes

The full registry-packaging spike at 31f4aa2: the reusable workflow and its callers, the package builders and consumers, the crabx launcher signal fix, the Swift fixture rename and the quest handoff.

  • Reusable registry.yml: builds the api fixture's wheel and XCFramework on Linux arm64 and macOS arm64, then checks the installed artifacts in fresh consumer jobs that have no checkout. Runs in-repo use the triggering commit; external callers use a fixed revision.
  • Callers: the manual Check and Nightly workflows invoke it. Nightly's report job now counts it, so a spike regression opens the nightly-failure issue.
  • package.py / build.sh: the wheel gets a complete hashed RECORD and native-dependency allowlists (RUNPATH stripped and glibc ≤ 2.39 on Linux; @rpath ID and system-only otool -L on macOS). The Swift package's link settings come from rustc's native-static-libs. On macOS the build uses Xcode's SDK and Apple's linker.
  • Consumers: consume.py compares the Python transcript byte for byte, prints a unified diff on mismatch, then runs the extra cases. consume-swift.sh builds a fresh SwiftPM executable against the unpacked package.
  • crabx launcher: supervisor-signal handlers are now installed before spawn. Before this, a SIGTERM that arrived during startup hit the default disposition and killed the launcher. The new regression test sends a real signal from inside a mocked spawn.
  • Swift fixture: renames a local that shadowed the debug() helper under Swift 6.
  • Quests: deletes quest/m2/registry.md and adds the release-templates follow-up. No stale links to the deleted file remain.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread .github/workflows/registry.yml Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Head: 8d942c3477176729d4861297cc041c8e3b4ebb19

No new blocker in the downloader update. The new pinned SHA already matches crabx.yml and nightly.yml, and the fixture still downloads an exact artifact name from its own run. The earlier successful build/install transcript predates this pin change, and ordinary PR checks don't run the dispatch-only registry fixture. Capture a fresh fixture run before treating the updated workflow as validated for reuse. Static review; I did not dispatch anything.

@kixelated
kixelated enabled auto-merge November 16, 2026 23:49

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

The delta since the prior Pullfrog review at 31f4aa2.

  • Aligned the download-artifact pin: registry.yml now pins actions/download-artifact at v8.0.2 (9000827c…), matching nightly.yml and crabx.yml, which resolves the earlier inline finding.
  • Recorded the merge pick: quest/m2/release-templates.md notes the maintainer's decision to keep the follow-up unchanged.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit a705aa5 Nov 17, 2026
9 checks passed
@kixelated
kixelated deleted the quest/m2/registry branch November 17, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant