Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ test:
python3 sh/rs/package-rename.test.py
python3 sh/rs/select.test.py
python3 sh/js/workspace.test.py
python3 sh/rs/test-lints.test.py
sh/go/publish-wrapper.test.sh
bash .claude/hooks/direnv.test.sh
python3 sh/rs/uring.test.py
Expand Down
7 changes: 2 additions & 5 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,8 @@ concurrency:
cancel-in-progress: true

jobs:
# `check` and `test` are separate jobs because they share almost no build
# artifacts: clippy emits rmeta without codegen, while nextest codegens and
# links every test binary. Running them in sequence meant paying both in
# series for no reuse, so they run concurrently and the job takes the slower
# of the two rather than their sum.
# Test enforces default-feature clippy while compiling its binaries. Check
# runs the other lints, docs, and distinct feature/target builds in parallel.
check:
name: Check
# Skip the actual work on close/merge: this run exists only to cancel the
Expand Down
1 change: 0 additions & 1 deletion quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ Measure each change against the numbers above and record the result in its PR.
- [Require platform gates](/quest/m0/ci/platform-queue.md) - activate the consolidated platform checks in the main and release rulesets after the workflow lands
- [mbx on R2](/quest/m0/ci/mbx-r2.md) - Rust builds on every OS read an R2 cache that only `main` writes, escaping the 10 GB Actions cache
- [Nix cache](/quest/m0/ci/nix-cache.md) - `nix.moq.dev` on R2 replaces Cachix for CI and releases
- [Clippy in Test](/quest/m0/ci/clippy-test.md) - Test's clippy-driver compile denies warnings, so Check stops repeating the default Clippy pass
- [Nightly finishes](/quest/m0/ci/nightly-split.md) - every Nightly step runs within its timeout
- [FFI artifact reuse](/quest/m0/ci/ffi-reuse.md) - identical FFI release builds are built once and shared across the language packages
- [Feature permutations](/quest/m0/ci/features.md) - PR feature checks and test builds compile only what they run, measured first
16 changes: 0 additions & 16 deletions quest/m0/ci/clippy-test.md

This file was deleted.

28 changes: 15 additions & 13 deletions rs/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,22 +30,20 @@ default:
[positional-arguments]
check *args:
cargo clippy --locked --all-targets "$@" -- -D warnings
just rs _lint "$@"
just rs lint "$@"

# `check` with clippy replaced by the `test` build, whose clippy-driver reports
# the same findings. `build.warnings` turns them into errors without entering
# cargo's fingerprint, so a later plain `test` reuses the artifacts. A
# selection can hold nothing testable.
# `check` with clippy enforced by the `test` build. A selection can hold
# nothing testable.

# Compile, lint, and test in one build, plus everything else `check` verifies.
[positional-arguments]
check-test *args:
CARGO_BUILD_WARNINGS=deny RUSTC_WORKSPACE_WRAPPER={{ require("clippy-driver") }} bash sh/rs/nextest.sh cargo nextest run --locked --all-targets --no-tests=pass "$@"
just rs _lint "$@"
just rs test --no-tests=pass "$@"
just rs lint "$@"

# Everything `check` verifies besides clippy.
[positional-arguments]
_lint *args:
lint *args:
python3 sh/rs/stats-docs.py
cargo fmt --all --check
# quinn-udp keeps quinn's formatting; cargo fmt skips it (see moq-sock's lib.rs).
Expand All @@ -67,6 +65,10 @@ fix *args:
check-changed $LIST:
sh/rs/select.sh check "$LIST"

# Like `lint`, with the feature and target checks for the selected crates.
lint-changed $LIST:
sh/rs/select.sh lint "$LIST"

# Like `check-test`, but only the crates in a changed-file list plus their dependents.
check-test-changed $LIST:
sh/rs/select.sh check-test "$LIST"
Expand Down Expand Up @@ -250,14 +252,14 @@ media-features:

# nextest kills a wedged test (`.config/nextest.toml` sets the timeout), but
# skips doctests; `just rs doctest` covers those. clippy-driver as the workspace
# wrapper reports lint findings from the same compile, as warnings: `check-test`
# is the gate. It is part of cargo's fingerprint, and RUSTFLAGS would split the
# artifact cache, which is why neither is set any other way.
# wrapper enforces lint findings from the same compile, including benches and
# examples. `build.warnings` denies warnings without changing cargo's artifact
# fingerprint, unlike RUSTFLAGS.

# Run the test suite, reporting clippy findings from the same compile.
# Run the test suite and enforce clippy findings from the same compile.
[positional-arguments]
test *args:
RUSTC_WORKSPACE_WRAPPER="$(command -v clippy-driver || true)" bash sh/rs/nextest.sh cargo nextest run --locked --all-targets "$@"
CARGO_BUILD_WARNINGS=deny RUSTC_WORKSPACE_WRAPPER={{ require("clippy-driver") }} bash sh/rs/nextest.sh cargo nextest run --locked --all-targets "$@"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore a warning gate that works with nextest JSON

cargo nextest builds tests using cargo test --no-run --message-format json-render-diagnostics, but Cargo 1.98 does not honor build.warnings="deny" when processing JSON diagnostics, so Clippy warnings leave Cargo with exit code 0 and nextest proceeds. Because ci-check now omits the separate default-feature cargo clippy ... -D warnings pass, such warnings can merge despite the intended gate. Keep an explicit denial mechanism that Clippy itself enforces until Cargo's JSON warning-accounting bug is fixed. See nextest's documented Cargo invocation and Cargo's open build.warnings JSON bug.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining: the gate does fire through nextest's JSON build on Cargo 1.98. The X11 capture job on the previous head failed exactly this way: cargo test --no-run --message-format json-render-diagnostics --package moq-video ... reported "moq-video (lib test) generated 2 warnings" then "warnings are denied by build.warnings configuration" and exited 101 (log). sh/rs/test-lints.test.py also pins it: it runs just rs test (nextest) on a crate with a clippy::len_zero finding and asserts a nonzero exit, then a clean build passes. It passes locally and runs in CI via .github/justfile, so a Cargo regression here would fail loudly.

(Written by Claude Opus 5.5)


# Exercise the ignored hardware tests for every GPU vendor on this Linux host.
gpu:
Expand Down
4 changes: 2 additions & 2 deletions rs/moq-video/src/capture/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -374,12 +374,12 @@ impl Stream {

/// A test-driven capture device: fixtures push frames at explicit acquisition
/// instants or device timestamps, standing in for a camera behind a [`Stream`].
#[cfg(test)]
#[cfg(all(test, feature = "openh264"))]
pub(crate) struct Synthetic {
chan: Arc<FrameChannel>,
}

#[cfg(test)]
#[cfg(all(test, feature = "openh264"))]
impl Synthetic {
/// Open a synthetic device with the given geometry, returning it and the stream reading it.
pub(crate) fn open(size: crate::Size, framerate: Rate) -> (Self, Stream) {
Expand Down
23 changes: 23 additions & 0 deletions sh/dispatch-query.test.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,29 @@ def test_platform_is_only_a_query(self):
self.assertEqual(result.returncode, 2)
self.assertEqual(result.stdout, "")

def test_ci_check_leaves_default_rust_lints_to_test(self):
for tool in ("bun", "python3", "cargo", "wasm-bindgen", "jq", "shellcheck",
"shfmt", "taplo", "nixfmt", "actionlint", "quest", "nix"):
path = self.bin / tool
path.write_text("#!/bin/sh\nexit 0\n")
path.chmod(0o755)
just = self.bin / "just"
just.write_text('#!/usr/bin/env bash\nprintf "%s\\n" "$*" >>"$CALLS"\n')
just.chmod(0o755)
self.env.update(MOQ_DEV_SHELL="1", CALLS=str(self.root / "calls"))
self.change("rs/moq-clock/src/lib.rs")
self.env["PATH"] += f":{os.environ['PATH']}"
for script in ("markdown.sh", "shell.sh"):
path = self.repo / "sh" / script
path.write_text("#!/bin/sh\nexit 0\n")
path.chmod(0o755)
result = subprocess.run(["bash", "sh/dispatch.sh", "ci-check", "origin/main"],
cwd=self.repo, env=self.env, text=True, capture_output=True)
self.assertEqual(result.returncode, 0, result.stderr)
calls = (self.root / "calls").read_text().splitlines()
self.assertTrue(any(call.startswith("rs lint-changed ") for call in calls), calls)
self.assertFalse(any(call.startswith("rs check-changed ") for call in calls), calls)


def benchmark():
# Sweep path count and both job selections. No timing threshold: hosted setup
Expand Down
6 changes: 3 additions & 3 deletions sh/dispatch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -228,9 +228,9 @@ for module in "${selected[@]}"; do
check:js) just js check-changed "$js_list" ;;
check:workers) just js workers ;;
check:moq_sh) just infra moq-sh test ;;
# `check` lints Rust through its test build, so the crates compile once;
# `ci-check` uses clippy, which is quicker when the tests run elsewhere.
check:rs) if [[ "$action" == check ]]; then just rs check-test-changed "$rs_list"; else just rs check-changed "$rs_list"; fi ;;
# Test enforces default-feature clippy through its compile. Check keeps
# the other lints and distinct feature/target builds.
check:rs) if [[ "$action" == check ]]; then just rs check-test-changed "$rs_list"; else just rs lint-changed "$rs_list"; fi ;;
check:bench) cargo check --locked --package moq-relay --package moq-bench --features moq-relay/io-uring ;;
check:quest) quest check ;;
check:drill) just test drill-sensitivity --apply-only ;;
Expand Down
14 changes: 7 additions & 7 deletions sh/rs/select.sh
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#!/usr/bin/env bash
# Run `just rs check|check-test|fix|test` on the crates a diff touches, plus
# Run `just rs check|check-test|lint|fix|test` on the crates a diff touches, plus
# their dependents. `platform` is the Windows and macOS compile.
#
# Usage: sh/rs/select.sh check|check-test|fix|test|platform|uring LISTFILE|--all
# Usage: sh/rs/select.sh check|check-test|lint|fix|test|platform|uring LISTFILE|--all
#
# LISTFILE holds the changed paths, one per line. `--all` is the whole
# workspace, as is a change to anything every crate is built or tested by.
set -euo pipefail

usage="usage: sh/rs/select.sh check|check-test|fix|test|platform|uring LISTFILE|--all"
usage="usage: sh/rs/select.sh check|check-test|lint|fix|test|platform|uring LISTFILE|--all"
action=${1:?$usage}
list=${2:?$usage}

Expand Down Expand Up @@ -101,7 +101,7 @@ if [[ -f "$list" ]] && grep -qx 'sh/rs/net-features.sh' "$list"; then net_script

case "$packages" in
"")
if [[ "$action" == check* && -n "$media_script$net_script" ]]; then
if [[ ("$action" == check* || "$action" == lint) && -n "$media_script$net_script" ]]; then
if [[ -n "$media_script" ]]; then just rs media-features; fi
if [[ -n "$net_script" ]]; then just rs net-features; fi
else
Expand All @@ -126,7 +126,7 @@ case "$action" in
uring)
if wants '(moq-uring|moq-relay)'; then just rs uring; fi
;;
check | check-test)
check | check-test | lint)
just rs "$action" "${flags[@]}"
# Workspace feature unification hides a broken moq-tokio feature set
# whenever any dependent enables a transport.
Expand All @@ -139,8 +139,8 @@ case "$action" in
if wants '(moq-wasm|moq-mux|moq-ffi|moq-clock)'; then just rs wasm; fi
# Device code behind the off-by-default `capture` feature. check-test
# lints it through the test build, as for the default features.
if wants '(moq-video|moq-audio)'; then
if [[ "$action" == check-test ]]; then CARGO_BUILD_WARNINGS=deny just rs capture-test; else just rs capture; fi
if [[ "$action" != lint ]] && wants '(moq-video|moq-audio)'; then
if [[ "$action" == check-test ]]; then just rs capture-test; else just rs capture; fi
fi
# moq-cli's media tasks are behind `play`; only check-test runs tests.
if [[ "$action" == check-test ]] && wants moq-cli; then just rs play; fi
Expand Down
10 changes: 8 additions & 2 deletions sh/rs/select.test.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@ def test_nextest_launcher_selects_workspace(self):
just.chmod(0o755)
changed = scratch / "changed"
changed.write_text("sh/rs/nextest.sh\n")
env = dict(os.environ, PATH=f"{scratch}:{os.environ['PATH']}", SELECT_CALLS=str(calls))
for action in ("check", "check-test", "test"):
env = dict(os.environ, PATH=f"{scratch}:{os.environ['PATH']}", SELECT_CALLS=str(calls),
BASH_ENV=os.devnull)
for action in ("check", "check-test", "lint", "test"):
with self.subTest(action=action):
calls.write_text("")
subprocess.run(
Expand All @@ -32,6 +33,11 @@ def test_nextest_launcher_selects_workspace(self):
f"rs {action} --workspace --exclude moq-net-fuzz",
calls.read_text().splitlines(),
)
if action == "lint":
selected = calls.read_text().splitlines()
for recipe in ("tokio-features", "wasm", "uring-check", "media-features"):
self.assertIn(f"rs {recipe}", selected)
self.assertNotIn("rs capture", selected)


if __name__ == "__main__":
Expand Down
48 changes: 48 additions & 0 deletions sh/rs/test-lints.test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""A test build must reject Clippy findings without a separate lint compile."""

import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest


ROOT = Path(__file__).resolve().parents[2]


class TestLints(unittest.TestCase):
def test_clippy_warning_fails_test_build(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "rs").mkdir()
(root / "sh" / "rs").mkdir(parents=True)
(root / "src").mkdir()
(root / "justfile").write_text("mod rs\n")
shutil.copy(ROOT / "rs" / "justfile", root / "rs" / "justfile")
shutil.copy(ROOT / "sh" / "rs" / "nextest.sh", root / "sh" / "rs" / "nextest.sh")
(root / "Cargo.toml").write_text(
'[package]\nname = "test-lint-gate"\nversion = "0.0.0"\nedition = "2024"\n'
)
(root / "src" / "lib.rs").write_text(
'pub fn empty(values: &[u8]) -> bool { values.len() == 0 }\n'
'#[test]\nfn runs() { assert!(empty(&[])); }\n'
)
toolchain = Path(shutil.which("clippy-driver")).parent
env = dict(os.environ, CARGO_TARGET_DIR=str(root / "target"),
PATH=f"{toolchain}:{os.environ['PATH']}")
subprocess.run(["cargo", "generate-lockfile", "--offline"], cwd=root,
env=env, check=True, capture_output=True)
result = subprocess.run(["just", "rs", "test"], cwd=root, env=env,
text=True, capture_output=True)
self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertIn("clippy::len_zero", result.stderr)
source = root / "src" / "lib.rs"
source.write_text(source.read_text().replace("values.len() == 0", "values.is_empty()"))
result = subprocess.run(["just", "rs", "test"], cwd=root, env=env,
text=True, capture_output=True)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)


if __name__ == "__main__":
unittest.main()
Loading