Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ccb4cd6
chore(quest): claim quest/m0/ci/hakari.md
kixelated Oct 28, 2026
765c265
ci: unify dependency features with a cargo-hakari workspace-hack
kixelated Oct 28, 2026
5bb9a84
ci: isolate third-party features per crate nightly
kixelated Oct 28, 2026
a1b9b43
chore(quest): narrow hakari to the FFI builds and hosted numbers
kixelated Oct 28, 2026
cf2ca1e
ci: leave the workspace-hack's formatting to cargo hakari
kixelated Oct 28, 2026
649b1d2
Merge remote-tracking branch 'origin/main' into quest/m0/ci/hakari
kixelated Oct 28, 2026
6210536
chore(quest): claim quest/m0/ci/hakari.md
kixelated Nov 3, 2026
34fe18a
chore: merge current main and the new hakari claim
kixelated Nov 3, 2026
946bd85
ci: share host FFI dependency build identities
kixelated Nov 3, 2026
337471d
style(dart): format the hook with its resolved language version
kixelated Nov 3, 2026
69ee071
Merge remote-tracking branch 'origin/main' into quest/m0/ci/hakari
kixelated Nov 3, 2026
bf59c42
test(ffi): distinguish each packaged build from stale artifacts
kixelated Nov 3, 2026
973e007
Merge remote-tracking branch 'origin/quest/m0/ci/hakari' into t3/dc71…
kixelated Nov 4, 2026
1abd720
ci: continue independent platform checks after suite failures
kixelated Nov 4, 2026
a695fc8
docs: refresh scheduled hakari evidence and remaining wait
kixelated Nov 4, 2026
d2c44da
docs: make the missing CPU measurement decision explicit
kixelated Nov 4, 2026
eb32a9c
fix(ci): preserve the native Dart release build environment
kixelated Nov 4, 2026
d0d7ba9
fix(dart): resolve the shared release artifact directory through Cargo
kixelated Nov 4, 2026
f3e90d3
style: wrap Dart fixture command strings
kixelated Nov 4, 2026
64d4966
style: apply fixture string formatting
kixelated Nov 4, 2026
7c935b3
docs(ci): record accepted cache evidence and scheduled wait
kixelated Nov 4, 2026
324945e
test(ffi): disable flaky socket datagram receive
kixelated Nov 4, 2026
1773cde
test: track disabled datagram coverage in flake epic
kixelated Nov 4, 2026
4829955
Merge origin/main into FFI cache alignment
kixelated Nov 5, 2026
dff984d
fix(ffi): select Cargo-reported native artifacts
kixelated Nov 5, 2026
73664b4
test(ffi): restore default connection datagram coverage
kixelated Nov 5, 2026
089ccc0
Merge remote-tracking branch 'origin/main' into fix/hakari-datagrams
kixelated Nov 5, 2026
0dc39bb
fix(ffi): resolve Android artifacts after cargo-ndk consumes JSON
kixelated Nov 5, 2026
eaa9f82
style: wrap the Android packaging fixture assertion
kixelated Nov 5, 2026
47839e2
test(ffi): isolate the packaging fixture from a developer's linker env
kixelated Nov 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@ moq = "run --bin moq --"
# name carries no feature hash, so a build with other features replaces
# `libmoq_ffi.so` under everyone reading it; such a build takes a target
# directory of its own (py, `sh/size/collect.sh`, `build.sh --no-default-features`)
# or a `--target` triple no default build uses (dart's `testing` build).
# or dart's `target/dart` directory for its `testing` build.
# Aliases resolve from the working directory, so run it inside the checkout.
build-ffi = "build --locked --package moq-ffi --features openh264"
6 changes: 5 additions & 1 deletion .github/ffi.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,14 @@ stripped for `moq-ffi`. The toolchain action installed stable Rust 1.99.0,
but the repository's `rust-toolchain.toml` selects Rust 1.98.1 (`48a229cea`)
for the actual Cargo builds on both paths. C++ invokes
`cargo build --locked --package moq-ffi --release` through
CMake; the shared candidate adds `--target <triple>` through
CMake; in this sampled revision, the shared candidate adds `--target <triple>` through
`rs/moq-ffi/build.sh`. C++ is always a native host build. Only the shared ARM
Linux candidate crosses from an x86 Linux host.

Current native packaging omits `--target`, matching the C++ invocation. The
table and debug-path differences below describe the older sampled invocation;
only cross-builds still pass an explicit target.

| Target | C++ host | Shared candidate host | Native archive comparison |
| --- | --- | --- | --- |
| x86\_64-unknown-linux-gnu | Ubuntu 24.04, image 20261004.327.1 | Same image and architecture | 772 common objects byte-identical; one differently named Rust LTO object |
Expand Down
30 changes: 19 additions & 11 deletions .github/workflows/platform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,24 +102,28 @@ jobs:
if: needs.scope.outputs.c == 'true' || needs.scope.outputs.cpp == 'true' || needs.scope.outputs.obs == 'true'
with:
generator: cpp
# Later suites are independent once tool and generator setup succeeds.
- name: Setup complete
id: setup
run: echo "Platform tools ready"
- name: Rust compile and tests
if: needs.scope.outputs.rs == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.rs == 'true') }}
shell: bash
env:
CI_BASE: ${{ github.event_name != 'pull_request' && '--all' || '' }}
run: |
just rs windows "$CI_BASE"
just rs windows-test
- name: C
if: needs.scope.outputs.c == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.c == 'true') }}
shell: bash
run: just c check
- name: C++
if: needs.scope.outputs.cpp == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.cpp == 'true') }}
shell: bash
run: just cpp check
- name: OBS
if: needs.scope.outputs.obs == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.obs == 'true') }}
shell: bash
run: just obs package --target x86_64-pc-windows-msvc --output "$RUNNER_TEMP/obs"

Expand Down Expand Up @@ -151,36 +155,40 @@ jobs:
if: needs.scope.outputs.obs == 'true'
with:
generator: cpp
# Later suites are independent once tool and generator setup succeeds.
- name: Setup complete
id: setup
run: echo "Platform tools ready"
- name: Rust compile and tests
if: needs.scope.outputs.rs == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.rs == 'true') }}
env:
CI_BASE: ${{ github.event_name != 'pull_request' && '--all' || '' }}
run: |
just rs macos "$CI_BASE"
just rs macos-test
- name: Swift simulator check
if: needs.scope.outputs.swift == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.swift == 'true') }}
run: just swift simulator
- name: Swift
if: needs.scope.outputs.swift == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.swift == 'true') }}
run: just swift check
# At a token budget, so the runner keeps working at this commit; rust2's
# `just bench-ffi` measures it at its pinned moq revision.
- name: Swift binding benchmark runner
if: inputs.benchmarks
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (inputs.benchmarks) }}
env:
MOQ_BENCH_SECONDS: "0.05"
run: |
just swift bench
test -s target/bench/ffi/swift.json
- name: OBS
if: needs.scope.outputs.obs == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.obs == 'true') }}
run: just obs package --target aarch64-apple-darwin --output "$RUNNER_TEMP/obs"
- name: Installer fixtures
if: needs.scope.outputs.moq_sh == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.moq_sh == 'true') }}
run: just infra moq-sh test
- name: Installer smoke
if: needs.scope.outputs.moq_sh == 'true'
if: ${{ !cancelled() && steps.setup.outcome == 'success' && (needs.scope.outputs.moq_sh == 'true') }}
env:
GH_TOKEN: ${{ github.token }}
run: just infra moq-sh smoke
63 changes: 55 additions & 8 deletions dart/moq_ffi/hook/build.dart
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import 'dart:convert';
import 'dart:io';

import 'package:code_assets/code_assets.dart';
Expand Down Expand Up @@ -88,17 +89,38 @@ Future<_Resolved> _resolveLibrary(
/// another recipe with different features. Cargo's own timestamp check makes
/// the no-op case cheap.
Future<_Resolved> _build(Uri workspace, String target, String library) async {
// The shared default build, an alias .cargo/config.toml defines for Cargo
// runs in the workspace; it ships the software H.264 fallback. A checkout build is a test build, carrying the hooks
// leak_test.dart reads; `--target` keeps it out of the default build's
// directory, since a cdylib's file name carries no feature hash.
final cargoArgs = ['build-ffi', '--features', 'testing'];
// Keep the leak-test allocator out of the shared cdylib. A separate output
// directory preserves Cargo's native compilation mode and dependency cache
// identities; passing the host as --target would split them.
final targetDir = workspace.resolve('target/dart/');
final cargoArgs = [
'build-ffi',
'--message-format=json-render-diagnostics',
'--features',
'testing',
'--target-dir',
targetDir.toFilePath(),
];
final rustc = await Process.run('rustc', [
'-vV',
], workingDirectory: workspace.toFilePath());
if (rustc.exitCode != 0) {
throw StateError('rustc -vV failed: ${rustc.stderr}');
}
final host = LineSplitter.split(
rustc.stdout as String,
).firstWhere((line) => line.startsWith('host: ')).substring('host: '.length);
final native = target == host;
final android = target.contains('-android');

// Android needs the NDK's linker and sysroot, which only cargo-ndk sets up.
// This mirrors rs/moq-ffi/build.sh.
final args = target.contains('-android')
final args = android
? ['ndk', '--target', target, '--platform', '24', '--', ...cargoArgs]
: [...cargoArgs, '--target', target];
: [
...cargoArgs,
if (!native) ...['--target', target],
];

// The iOS floor release-ffi.yml builds with, which also avoids its
// ___chkstk_darwin linker errors. The hook's filtered environment drops it.
Expand All @@ -116,7 +138,32 @@ Future<_Resolved> _build(Uri workspace, String target, String library) async {
);
}

final built = workspace.resolve('target/$target/debug/$library');
final scoped = targetDir.resolve('$target/debug/$library');
final Uri built;
if (android) {
// cargo-ndk consumes Cargo's artifact messages and always sets --target.
built = scoped;
} else {
final artifacts = LineSplitter.split(result.stdout as String)
.map((line) => jsonDecode(line) as Map<String, dynamic>)
.where(
(message) =>
message['reason'] == 'compiler-artifact' &&
(message['target'] as Map<String, dynamic>)['name'] == 'moq_ffi',
)
.expand((message) => (message['filenames'] as List).cast<String>())
.map((path) => File(path).uri)
.where((uri) => uri.pathSegments.last == library)
.toSet();
if (artifacts.length != 1) {
throw StateError('expected one moq-ffi cdylib from Cargo: $artifacts');
}
built = artifacts.single;
final unscoped = targetDir.resolve('debug/$library');
if (built != scoped && !(native && built == unscoped)) {
throw StateError('Cargo artifact does not match target $target: $built');
}
}
if (!await File.fromUri(built).exists()) {
throw StateError('cargo did not produce ${built.toFilePath()}');
}
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ The historical samples locate candidates; they are not controlled benchmarks.
- [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out
- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - confirm the hosted Nightly jobs reuse the previous Nightly, then measure restore time and expiry
- [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm releases use `nix.moq.dev`, check its bucket sizes, and retire Cachix
- [Unify dependency features](/quest/m0/ci/hakari.md) - align the FFI builds' features and record the hosted platform jobs' hits once the workspace-hack lands
- [Record hosted cargo-hakari evidence](/quest/m0/ci/hakari.md) - compare scheduled macOS and Windows cache hits and estimated compiler work saved before and after the workspace-hack
- [Drop main from the publish Environment](/quest/m0/ci/publish-release-only.md) - the maintainer narrows `publish` to `release` and release tags once no job naming it runs as `main`
- [Binary release publishers run on their tag](/quest/m0/ci/release-tag-publishers.md) - brew, Go and Swift releases stop running through `workflow_run` on main
- [The bench comparison builds moq-gst's catalog bench on its base](/quest/m0/ci/bench-base-build.md) - A paired bench comparison that selects `rust/gst/moq-gst/catalog` against `origin/main` builds the base side and reports a comparison, instead of marking it unavailable with "base build failed"
Expand Down
79 changes: 54 additions & 25 deletions quest/m0/ci/hakari.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,60 @@
# [S] Unify dependency features with cargo-hakari
# [S] Record hosted cargo-hakari cache evidence

## Goal

The FFI builds compile each dependency with the same features as each other,
so mbx's cache hits across them, and Nightly's hosted platform jobs record
their hit rate and CPU time before and after the workspace-hack.
Nightly's hosted macOS and Windows jobs record their mbx cache-hit evidence and
estimated compiler work saved before and after the workspace-hack.

## Plan

Decided with the maintainer on 2026-10-25: use cargo-hakari, the standard tool
from guppy's authors, rather than only trimming CI's feature permutations. mbx
keys every compilation on its inputs, so a dependency built with a different
feature set in another job is a miss, however warm the cache.

[moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985) added
`rs/workspace-hack` as a path-only dev-dependency of every member outside
`.config/hakari.toml`'s excludes. Test, lint, and bench builds now unify;
library-only builds do not, since dev-dependencies stay out of them. That
leaves the FFI builds (`cargo build --release -p moq-ffi` in the bindings and
release workflows), which also split on their own flags: the Dart build passes
the host triple as `--target` and adds the `testing` feature. Align those
invocations rather than add the hack to them, unless the PR's decision on the
layout says otherwise.

Nightly's macOS and Windows jobs read and write R2, so their before and after
can only come from scheduled Nightlies on `main`
(`gh run list --workflow nightly.yml --branch main --event schedule`): compare
the mbx summaries of the last run before the merge with the first two after.

Public API: none; the workspace-hack crate is not published.
Decided 2026-11-04 (maintainer, /quest-unblock): ✅ accept the mbx cache-hit
evidence and estimated compiler savings as sufficient. Not chosen: add per-job
CPU instrumentation and measure CPU totals. The estimates are not measured CPU
savings.

Decided with the maintainer on 2026-10-25: use cargo-hakari rather than only
trimming CI's feature permutations.
[moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985) added the path-only
dev-dependency layout, unifying test, lint, and bench builds. Keep that layout;
library-only builds do not need the hack.

The remaining FFI invocation alignment is implemented in
[moq-dev/moq#7666](https://github.com/moq-dev/moq/pull/7666): default builds use
`cargo build-ffi` with `openh264`, native builds omit `--target`, and only cross
builds name a target. Python and Dart host tests retain `testing` for the manual
clock and leak allocator, with separate target directories so they cannot
replace the default cdylib. Their third-party feature identities match the
default host build; the intentional `kio/manual` split affects workspace crates.

Compare the mbx summaries of the last scheduled Nightly before the
[2026-11-02 merge](https://github.com/moq-dev/moq/commit/8e53bf32185fec578e460ff6ccbe41a80a039a6a)
with the first two after. Query only scheduled runs on `main`:
`gh run list --workflow nightly.yml --branch main --event schedule`.
The baseline is [2026-11-01](https://github.com/moq-dev/moq/actions/runs/41313466339).
The [2026-11-02 run](https://github.com/moq-dev/moq/actions/runs/41483654426)
has partial macOS summaries before a websocket-test timeout and no Windows
compilation (NASM installation returned HTTP 503).
[2026-11-03](https://github.com/moq-dev/moq/actions/runs/41656613559) completed
both platform jobs. [2026-11-04](https://github.com/moq-dev/moq/actions/runs/41852517544)
completed Windows, but cancelled macOS before any step ran. Windows therefore
has two complete after-runs; macOS has only one. Logs expose object-cache counts
and estimated compiler work saved, but no measured user/system CPU totals.
Do not call elapsed time CPU time, or attribute these earlier runs to the FFI
alignment that has not landed yet.

The 2026-11-04 maintainer decision accepts hosted CI as the strict Nix gate;
a local namespace-probe skip is expected. The pkg-config and Gradle repairs
have landed, so neither remains an external prerequisite.

Public API and wire: none.

## Waiting

- moq-dev/moq: inspect the next scheduled Nightly for a second complete macOS
after-run. As of 2026-11-04, the latest scheduled run is
[Nov 4](https://github.com/moq-dev/moq/actions/runs/41852517544); its
[macOS job](https://github.com/moq-dev/moq/actions/runs/41852517544/job/126656206251)
was cancelled with no steps. Run
`gh run list --repo moq-dev/moq --workflow nightly.yml --event schedule`,
inspect the next macOS job and record its mbx summaries alongside Nov 3.
This measurement wait outlives the implementation PR; never dispatch Nightly.
Loading
Loading