Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ check:

# Tests of repository tooling; nightly, not on every pull request.
test:
python3 sh/size/report.test.py
python3 sh/size/release.test.py
python3 sh/size/nightly.test.py
python3 bench/compare.test.py
python3 bench/summary.test.py
python3 bench/relay.test.py
Expand Down Expand Up @@ -45,6 +48,19 @@ install-formula *args:
trigger-repo-publish *args:
sh/gh/trigger-repo-publish.sh "$@"

# Measure a workflow run's release uploads, the minimal FFI build, and bundled first-load JS entries.
size-collect output run="":
sh/size/collect.sh '{{ output }}' '{{ run }}'

# Compare a size report with its retained nightly history.
[positional-arguments]
size-report *args:
python3 sh/size/report.py "$@"

# Collect and compare nightly shipping sizes through the existing alert path.
size-nightly output run:
sh/size/nightly.sh '{{ output }}' '{{ run }}'

# Install the rendered GStreamer formula against the current macOS tarball.
[positional-arguments]
gst-formula *args:
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,45 @@ jobs:
permissions:
contents: read

sizes:
name: Shipping sizes
# Measures what these release jobs upload rather than rebuilding it.
needs: [release-py-ffi, release-kt-ffi, release-swift-ffi, release-go-ffi, release-moq-c, release-moq-relay, release-moq-cli]
runs-on: ubuntu-24.04-arm
timeout-minutes: 120
permissions:
contents: read
actions: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: DeterminateSystems/nix-installer-action@1d87d45818068401a10cf16bdc5f00b24994a83f # main
with:
determinate: false
extra-conf: |
extra-substituters = https://kixelated.cachix.org
extra-trusted-public-keys = kixelated.cachix.org-1:CmFcV0lyM6KuVM2m9mih0q4SrAa0XyCsiM7GHrz3KKk=
- uses: ./.github/actions/rust-cache
- name: Collect and compare sizes
run: nix develop --command just gh size-nightly size-report "$GITHUB_RUN_ID"
env:
GH_TOKEN: ${{ github.token }}
- name: Preserve sizes and attribution
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: nightly-size-history
path: |
size-report/history.json
size-report/sizes.json
size-report/summary.md
size-report/ffi-bloat.txt
size-report/js/bytes.json
size-report/js/*.metafile.json
retention-days: 45
if-no-files-found: error

fuzz:
name: Fuzz (${{ matrix.target }})
runs-on: ubuntu-24.04-arm
Expand Down
1 change: 1 addition & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@
cargo-edit
cargo-semver-checks
cargo-deny
cargo-bloat
# Per-crate feature matrices (`just rs features`), where workspace
# unification would otherwise hide a broken single-crate build.
cargo-hack
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,6 @@ session outcomes, and embedder refusals inside session outcomes.
- [moqsrc A/V alignment](/quest/m1/gst-src-av-sync.md) - every `moqsrc` pad of a run shares one timestamp reference and segment base
- [TS passthrough export](/quest/m1/ts-passthrough-export.md) - `export ts --passthrough` writes the `m2ts` track back byte-identical (less late drops) on a fixed delay
- [FLV and MKV export delay](/quest/m1/export-delay.md) - FLV and MKV interleave through the shared jitter buffer on a fixed delay, breaking the CLI once
- [Size report](/quest/m1/size-report.md) - a nightly job reports every shipped artifact's size, native and JS, and alerts when one grows
- [Bindings size profile](/quest/m1/ffi-size-profile.md) - a benchmark decides whether the moq-ffi builds ship at opt-level "s", which halves the dylib
- [Go mirror delivery](/quest/m1/go-mirror-delivery.md) - the Go binding's staticlibs stop growing git history by ~210 MiB per release
- [Dart on iOS](/quest/m1/dart-ios.md) - prove the shipped iOS native asset actually loads on a device, which no CI can
Expand Down
64 changes: 0 additions & 64 deletions quest/m1/size-report.md

This file was deleted.

23 changes: 23 additions & 0 deletions sh/size/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Nightly artifact sizes

`just gh size-collect OUTPUT [RUN]` measures what a workflow run's release jobs
uploaded, plus the builds no release job makes: the minimal-feature FFI library
and real browser consumer bundles (raw, gzip and Brotli). `just gh size-report
OUTPUT [--history HISTORY]` compares the report with its previous report (5%
growth) and the newest report at least 30 days old (10% growth).

Nightly passes its own run, so the sizes are the shipping artifacts for every
release target, Linux `x86_64` and `aarch64` included, without rebuilding them. Any
retained run works locally too. Each release job's `Build` step durations are
recorded and summarized but never compared, since hosted build time is noisy.
The Go mirror's 100 MiB file limit is enforced by the Go FFI package step;
the report tracks those staticlibs.

Keys name the artifact (which carries the target) and the filename without its
release version, so a version bump keeps the history comparable. Duplicate keys
and missing release artifacts are refused. An artifact-set change is refused
too: review the new set, then start a fresh history by reporting without one.

A regression fails Nightly and reaches its existing alert workflow. The job
summary carries the byte counts, build times and FFI crate attribution, the
latter from the symbol-preserving `profiling` profile.
71 changes: 71 additions & 0 deletions sh/size/bundles.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
// Measure consumer bundles from published dist entry points, including only static chunks.

import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join, resolve } from "node:path";
import { brotliCompressSync, gzipSync } from "node:zlib";
import { build } from "esbuild";

const root = process.cwd();
if (!process.argv[2]) throw new Error("usage: bundles.ts OUTPUT");
const out = resolve(process.argv[2]);
mkdirSync(out, { recursive: true });
const sizes: Record<string, number> = {};
for (const [name, code] of Object.entries({
net: 'export * from "@moq/net";',
watch: 'import "@moq/watch/element";',
"watch-ui": 'import "@moq/watch/element"; import "@moq/watch/ui";',
publish: 'import "@moq/publish/element";',
})) {
const result = await build({
stdin: { contents: code, resolveDir: root, sourcefile: `${name}.js` },
bundle: true,
minify: true,
splitting: true,
format: "esm",
platform: "browser",
metafile: true,
write: false,
outdir: join(out, name),
plugins: [
{
name: "built-packages",
setup(b) {
b.onResolve({ filter: /^@moq\// }, ({ path }) => {
const [, pkg, ...subpath] = path.split("/");
const dir = join(root, "js", pkg, "dist");
// Only packages outside this workspace fall through to node_modules.
if (!existsSync(join(root, "js", pkg, "package.json"))) return;
const manifest: { exports: Record<string, string | { default: string }> } = JSON.parse(
readFileSync(join(dir, "package.json"), "utf8"),
);
const entry = manifest.exports[subpath.length ? `./${subpath.join("/")}` : "."];
if (!entry) throw new Error(`missing built export: ${path}`);
return { path: resolve(dir, typeof entry === "string" ? entry : entry.default) };
});
},
},
],
});
if (!result.metafile || !result.outputFiles) throw new Error(`bundle outputs missing: ${name}`);
writeFileSync(join(out, `${name}.metafile.json`), JSON.stringify(result.metafile, null, 2));
const outputs = result.metafile.outputs;
const first = new Set<string>();
const visit = (path: string) => {
if (first.has(path)) return;
first.add(path);
for (const dep of outputs[path].imports) if (!dep.external && dep.kind !== "dynamic-import") visit(dep.path);
};
for (const [path, output] of Object.entries(outputs)) if (output.entryPoint === `${name}.js`) visit(path);
if (!first.size) throw new Error(`entry output missing: ${name}`);
for (const file of result.outputFiles) {
if (![...first].some((path) => resolve(root, path) === file.path)) continue;
for (const [format, bytes] of Object.entries({
raw: file.contents.length,
gzip: gzipSync(file.contents).length,
brotli: brotliCompressSync(file.contents).length,
}))
sizes[`js/${name}/${format}`] = (sizes[`js/${name}/${format}`] ?? 0) + bytes;
}
if (!sizes[`js/${name}/raw`]) throw new Error(`no first-load bytes: ${name}`);
}
writeFileSync(join(out, "bytes.json"), JSON.stringify(sizes, null, 2));
38 changes: 38 additions & 0 deletions sh/size/collect.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
# Measure a workflow run's release uploads and the builds no release job makes, retaining diagnostics.
set -euo pipefail
out=$(realpath -m "${1:?usage: collect.sh OUTPUT [RUN]}")
run=${2:-}
mkdir -p "$out/native"
target=$(cargo metadata --locked --format-version 1 --no-deps | jq -r .target_directory)
case $(uname -s) in Linux) dylib=so ;; Darwin) dylib=dylib ;; *)
echo 'native size capture requires Linux or macOS' >&2
exit 1
;;
esac
# Release jobs ship default features only; this shows what the feature gates save.
cargo build --locked --release -p moq-ffi --no-default-features
cp "$target/release/libmoq_ffi.$dylib" "$out/native/libmoq_ffi-minimal.$dylib"
cp "$target/release/libmoq_ffi.a" "$out/native/libmoq_ffi-minimal.a"
bun install --frozen-lockfile
just js build
bun sh/size/bundles.ts "$out/js"
# Symbols are necessary only for attribution; they never replace the measured shipping sizes.
cargo bloat --locked --profile profiling -p moq-ffi --crates --lib >"$out/ffi-bloat.txt"
python3 - "$out" "$run" <<'PY'
import datetime, json, pathlib, subprocess, sys
sys.path.insert(0, "sh/size")
import release
out, run = pathlib.Path(sys.argv[1]), sys.argv[2]
host = next(line.split(': ', 1)[1] for line in subprocess.check_output(['rustc', '-vV'], text=True).splitlines() if line.startswith('host: '))
sizes = {f'native/{host}/{p.name}': p.stat().st_size for p in (out / 'native').iterdir()}
Comment on lines +27 to +28

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Measure the minimal FFI build on x86_64 too

The scheduled sizes job runs only on ubuntu-24.04-arm, and this records only the host's minimal-feature artifacts. Consequently, an x86_64-specific size regression in moq-ffi --no-default-features is invisible; the downloaded x86_64 release artifacts use default features and do not cover it. Build or cross-build the minimal cdylib/staticlib for both Linux architectures before treating the report as x86_64 and aarch64 coverage.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining. Nothing ships the minimal-feature build. It's there to show what the feature gates save, which hardly depends on the architecture. The x86_64 shipping builds (Go staticlib and cdylib, wheel, relay, CLI, moq-c) are all measured from the release uploads. Adding a cross-compiled x86_64 fat-LTO build to an aarch64 runner would cost about 10 minutes per night to watch an artifact nobody downloads.

(Written by Claude Opus 5.5)

sizes.update(json.loads((out / 'js/bytes.json').read_text()))
report = {'version': 1, 'date': datetime.date.today().isoformat(),
'revision': subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip(),
'bytes': sizes}
if run:
release.download(run, out / 'release')
sizes.update(release.sizes(out / 'release'))
report['seconds'] = release.seconds(run)
(out / 'sizes.json').write_text(json.dumps(report, indent=2) + '\n')
PY
33 changes: 33 additions & 0 deletions sh/size/nightly.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
# Load the latest completed history, then retain the new report even if growth fails.
set -euo pipefail
out=${1:?usage: nightly.sh OUTPUT RUN}
run=${2:?usage: nightly.sh OUTPUT RUN}
mkdir -p "$out"
history=()
# A packaging or collector failure may leave no history artifact in the newest run.
# Search retained completed runs rather than silently discarding the older baseline.
runs=$(gh run list --workflow nightly.yml --branch main --limit 45 --status completed --json databaseId --jq '.[].databaseId')
while IFS= read -r prior; do
[[ -n "$prior" ]] || continue
artifact=$(gh api "repos/{owner}/{repo}/actions/runs/$prior/artifacts" --jq '.artifacts[] | select(.name == "nightly-size-history" and .expired == false) | .id')
if [[ -n "$artifact" ]]; then
previous="$out/previous/$prior"
gh run download "$prior" --name nightly-size-history --dir "$previous"
# Failed collectors can upload diagnostics without producing a history.
if [[ -f "$previous/history.json" ]]; then
history=(--history "$previous/history.json")
break
fi
fi
done <<<"$runs"
sh/size/collect.sh "$out" "$run"
status=0
python3 sh/size/report.py "$out" "${history[@]}" || status=$?
{
cat "$out/summary.md"
printf '\n<details><summary>FFI crate attribution</summary>\n\n```text\n'
cat "$out/ffi-bloat.txt"
printf '\n```\n</details>\n'
} >>"${GITHUB_STEP_SUMMARY:-/dev/stdout}"
exit "$status"
56 changes: 56 additions & 0 deletions sh/size/nightly.test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Exercise history recovery without network access or native builds."""
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest

SCRIPT = Path(__file__).with_name("nightly.sh").resolve()
REPORT = Path(__file__).with_name("report.py").resolve()


class History(unittest.TestCase):
def test_skips_retained_artifact_without_history(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "bin").mkdir()
(root / "sh/size").mkdir(parents=True)
(root / "sh/size/report.py").symlink_to(REPORT)
gh = root / "bin/gh"
gh.write_text('''#!/usr/bin/env python3
import json, pathlib, sys
args = sys.argv[1:]
if args[:2] == ['run', 'list']:
print('2\\n1')
elif args[0] == 'api':
print('artifact')
elif args[:2] == ['run', 'download']:
out = pathlib.Path(args[args.index('--dir') + 1])
out.mkdir(parents=True)
if args[2] == '1':
(out / 'history.json').write_text(json.dumps([{'version': 1, 'date': '2026-10-09', 'bytes': {'a': 100}}]))
else:
(out / 'sizes.json').write_text('{}')
else:
raise SystemExit(args)
''')
gh.chmod(0o755)
collect = root / "sh/size/collect.sh"
collect.write_text('''#!/usr/bin/env bash
set -eu
printf '%s' '{"version":1,"date":"2026-10-10","bytes":{"a":100}}' >"$1/sizes.json"
printf 'attribution' >"$1/ffi-bloat.txt"
''')
collect.chmod(0o755)
env = dict(os.environ, PATH=f"{root / 'bin'}:{os.environ['PATH']}",
GITHUB_STEP_SUMMARY=str(root / "summary"))
result = subprocess.run(["bash", str(SCRIPT), "out", "3"], cwd=root,
env=env, capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
history = json.loads((root / "out/history.json").read_text())
self.assertEqual([row['date'] for row in history], ['2026-10-09', '2026-10-10'])


unittest.main()
Loading
Loading