diff --git a/bench/README.md b/bench/README.md index 8257c70f452..12461ba1081 100644 --- a/bench/README.md +++ b/bench/README.md @@ -444,7 +444,7 @@ run by default: - `group`, on request: every subscriber drains one finished 512-frame group in object batches through the relay's private reader, as moq-net's `group_relay_objects/plain/512/*` bench does, so that bench's cost is counted - without its wall-clock noise. Revisions before the hidden `ObjectBatch` helper + without its wall-clock noise. Revisions before the `fuzz::ObjectBatch` helper report unsupported; historical frame-batch counts remain in the bisect report. Each `bench/fanout/` crate builds against each base's and the current diff --git a/quest/m2/api-shape/README.md b/quest/m2/api-shape/README.md index 1c894a9ff2c..1b8dd7965db 100644 --- a/quest/m2/api-shape/README.md +++ b/quest/m2/api-shape/README.md @@ -59,5 +59,4 @@ Rejected in planning: ## Required -- [Ban doc(hidden) in Rust](/quest/m2/api-shape/doc-hidden.md) - `just check` bans it; each use becomes private or documented, test-support modules documented behind their feature - [A peer's closed session reads as Closed in the bindings](/quest/m2/api-shape/session-closed.md) - a remote SESSION_CLOSED maps to `MoqError::Closed`, as a local close does diff --git a/quest/m2/api-shape/doc-hidden.md b/quest/m2/api-shape/doc-hidden.md deleted file mode 100644 index 706126273e0..00000000000 --- a/quest/m2/api-shape/doc-hidden.md +++ /dev/null @@ -1,53 +0,0 @@ -# [M] Ban doc(hidden) in Rust - -## Goal - -`just check` fails on any `#[doc(hidden)]` in the repository's Rust, vendored -code included. Each current use becomes private (`pub(crate)` or narrower), or -documented public API. A public item is either documented or not public. - -JS `@internal` is out of scope: it is how a JS package shares code across its -own modules without publishing it, which TypeScript has no visibility for. - -## Plan - -Planned 2026-11-09 (maintainer, /quest-complete follow-up of -[moq-dev/moq#8369](https://github.com/moq-dev/moq/pull/8369), which un-hides -moq-net's route stream: "audit any other doc(hidden). In fact just ban it"). - -Decisions: - -- ✅ The ban covers `rs/moq-webrtc/src/vendor` too (rejected: exclude vendored - code). Those modules are private and already trimmed by hand, so their - `doc(hidden)` hides nothing and can be deleted. `vendor/mod.rs` records - upstream revisions, not local edits, so the removal needs no note there. -- ✅ Feature-gated test support stays `pub` behind its feature and is - documented, like tokio's `test-util` (rejected: make it private by moving - bench and fuzz bodies into the crate). This covers `pub mod fuzz` (moq-net, - moq-quic, moq-mux, moq-rtmp), `pub mod sim` (moq-quic, moq-tokio, and - `quic::driver::sim`), and the bench exports (`moq_json::bench`, - `moq_stats::produce::bench`, `moq_quic::connection::BenchSentPackets`). Each one's doc - line says it is test support and names the feature that enables it. -- ✅ Rust only (rejected: audit or ban JS `@internal`). -- ✅ m2, in this epic (rejected: m3 with the audits). - -Guidance for the rest: `moq_tokio::settings` (used by moq-relay and -moq-bench), `moq_quic::mux` and its `connection::mux` re-export, and -`moq_webrtc::p2p` follow this epic's rules: private when only the crate uses -it, deleted when nothing does, otherwise documented. The two hides on moq-net's -route stream go with #8369; if it has not landed when this is worked, remove -them here. - -Add the check where `just check` already scans tracked text (`sh/text.sh` -greps every tracked file for em dashes), so it runs in the merge gate. Update -[wire fuzz coverage](/quest/m4/wire-fuzz-coverage.md), which asks for a -"hidden `fuzz` module", and [the relay over QUIC on the sim](/quest/m4/quic-sim.md), -which describes moq-tokio's "hidden `sim` feature", to match. - -Public API: newly documented items in moq-net, moq-quic, moq-mux, moq-rtmp, -moq-json, moq-stats, moq-tokio, and moq-webrtc; anything made private or -deleted is breaking. Wire: none. - -## Related - -- [Lint audit](/quest/m3/audit-lints.md) - the wider lint configuration diff --git a/quest/m4/quic-sim.md b/quest/m4/quic-sim.md index 1efe4d0ae12..ca3f468c9a7 100644 --- a/quest/m4/quic-sim.md +++ b/quest/m4/quic-sim.md @@ -11,7 +11,7 @@ deterministically with no sockets. Already landed: MoQ sessions over moq-quic on the sim. The QUIC driver spawns on the simulator inside `kio::clock::sim::run`, under moq-tokio's -hidden `sim` feature. `quic::driver::sim::Network` binds a socket per address +`sim` feature. `quic::driver::sim::Network` binds a socket per address and carries datagrams across moq-quic's seeded `Link` (now `moq_quic::sim`), so one seed replays a run. `rs/moq-tokio/tests/integration/sim.rs` builds both endpoints on the raw @@ -22,8 +22,9 @@ What remains: - moq-tokio's `Client` and `Server` bind their QUIC endpoint from an address (`crate::bind::udp`), so a sim test cannot reach them yet. Pick how a - caller hands either one a `driver::endpoint::Socket`, such as a hidden - `sim`-only hook. That API choice belongs to the maintainer. + caller hands either one a `driver::endpoint::Socket`, such as a + documented `sim`-only hook (`just check` bans `#[doc(hidden)]`). That API + choice belongs to the maintainer. - Route the session-level spawns that `server.rs`, `connection.rs`, and `origin.rs` still send through `crate::runtime::spawn` through `runtime::Spawner`, as `transport.rs` and the WebTransport session already diff --git a/quest/m4/wire-fuzz-coverage.md b/quest/m4/wire-fuzz-coverage.md index 0a2c0b4dd36..2441cf70caf 100644 --- a/quest/m4/wire-fuzz-coverage.md +++ b/quest/m4/wire-fuzz-coverage.md @@ -16,7 +16,7 @@ Decided 2026-10-22 from the security audit, with the maintainer: The point is the invariant, not the coverage number: a target that only checks for a panic would miss the bugs this audit is about. -Each crate follows moq-net's shape (`rs/moq-net/fuzz/README.md`): the target body in a hidden `fuzz` module so the crate's own tests replay `fuzz/regressions/` on stable, a one-line shim per target, an `--exclude` in the `no_fuzz` list of `rs/justfile`, and a row in `nightly.yml`'s fuzz matrix. Target names must stay unique across crates. +Each crate follows moq-net's shape (`rs/moq-net/fuzz/README.md`): the target body in a `fuzz` module, documented as test support behind the `fuzz` feature, so the crate's own tests replay `fuzz/regressions/` on stable, a one-line shim per target, an `--exclude` in the `no_fuzz` list of `rs/justfile`, and a row in `nightly.yml`'s fuzz matrix. Target names must stay unique across crates. Ordering: diff --git a/rs/moq-json/Cargo.toml b/rs/moq-json/Cargo.toml index 433a2143a85..67a768dacf5 100644 --- a/rs/moq-json/Cargo.toml +++ b/rs/moq-json/Cargo.toml @@ -19,8 +19,8 @@ features = ["async"] default = [] # Generated async wrappers for the poll methods. Enable where you `.await` them. async = ["kio/async"] -# Exposes the merge-patch diff to the codec bench, through the hidden `bench` module. Off by -# default and hidden from the docs. Not a supported API. +# Exposes the merge-patch diff to the codec bench, through the `bench` module. Off by +# default. bench = [] [dependencies] diff --git a/rs/moq-json/src/lib.rs b/rs/moq-json/src/lib.rs index c440c2543ee..ba1641e56cd 100644 --- a/rs/moq-json/src/lib.rs +++ b/rs/moq-json/src/lib.rs @@ -31,10 +31,8 @@ pub mod stream; mod strict; pub mod window; -/// The merge-patch diff, for the codec bench. Not a public API: compiled only with the `bench` -/// feature, which nothing enables by default. +/// Test support for the codec bench, enabled by the `bench` feature: the merge-patch diff. #[cfg(feature = "bench")] -#[doc(hidden)] pub mod bench { pub use crate::diff::{Diff, diff}; } diff --git a/rs/moq-mux/Cargo.toml b/rs/moq-mux/Cargo.toml index 8c239c3c607..1810f50783e 100644 --- a/rs/moq-mux/Cargo.toml +++ b/rs/moq-mux/Cargo.toml @@ -21,7 +21,7 @@ default = [] # poll of their own. Enable where you `.await` them. async = ["kio/async", "moq-net/async"] # Exposes the media parsers' fuzz target bodies and seeds to the `fuzz/` harness through the -# hidden `fuzz` module. Off by default and hidden from the docs. +# `fuzz` module. Off by default. fuzz = [] [dependencies] diff --git a/rs/moq-mux/fuzz/Cargo.toml b/rs/moq-mux/fuzz/Cargo.toml index 3bc135f909b..d13a729fff0 100644 --- a/rs/moq-mux/fuzz/Cargo.toml +++ b/rs/moq-mux/fuzz/Cargo.toml @@ -6,7 +6,7 @@ # excluded from every gate that means "all of it" instead: libFuzzer needs a nightly # toolchain and sanitizer flags, so nothing in `check` or `test` may compile it. # -# The target bodies live in `moq-mux`'s hidden `fuzz` module, not here: sharing them +# The target bodies live in `moq-mux`'s `fuzz` module, not here: sharing them # is what lets the crate's own tests replay the corpus on stable, allocation bound # included. [package] diff --git a/rs/moq-mux/fuzz/README.md b/rs/moq-mux/fuzz/README.md index 55853448c6d..3a2d0079bd7 100644 --- a/rs/moq-mux/fuzz/README.md +++ b/rs/moq-mux/fuzz/README.md @@ -19,7 +19,7 @@ minimized corpus, and uploads failure inputs as workflow artifacts. ## How it fits together -The target bodies are not in `fuzz_targets/`; they are in `moq-mux`'s hidden `fuzz` +The target bodies are not in `fuzz_targets/`; they are in `moq-mux`'s `fuzz` module (`src/fuzz.rs`), and the files here are shims that also register its counting allocator. `just check` replays the same bodies on the pinned stable toolchain, bound included, so a failure found here becomes a regression test that CI runs without anyone diff --git a/rs/moq-mux/src/fuzz.rs b/rs/moq-mux/src/fuzz.rs index cbec409c6c4..fd5cf3caa5f 100644 --- a/rs/moq-mux/src/fuzz.rs +++ b/rs/moq-mux/src/fuzz.rs @@ -1,4 +1,5 @@ -//! Fuzz target bodies for the media parsers an untrusted publisher reaches, plus their seeds. +//! Test support, enabled by the `fuzz` feature: fuzz target bodies for the media parsers an +//! untrusted publisher reaches, plus their seeds. //! //! Each target takes raw fuzzer bytes, drives one parser through its public entry point the way //! ingest or a subscriber does, and returns whether the input was accepted. [`run`] wraps a target @@ -7,8 +8,7 @@ //! them on stable, which is what turns a crash the fuzzer found into a regression `just check` //! runs. //! -//! Compiled only under `cfg(test)` or the `fuzz` feature, so none of this is part of the published -//! API. See `fuzz/README.md` for the workflow. +//! It tracks the crate's internals, so it can change in any release. See `fuzz/README.md` for the workflow. use std::task::Poll; diff --git a/rs/moq-mux/src/lib.rs b/rs/moq-mux/src/lib.rs index 496f9b6fc17..02928ab45c8 100644 --- a/rs/moq-mux/src/lib.rs +++ b/rs/moq-mux/src/lib.rs @@ -49,10 +49,7 @@ pub mod codec; pub mod container; mod error; pub mod follow; -// Not part of the public API: compiled only for the crate's own tests and for the `fuzz/` -// harness. #[cfg(any(test, feature = "fuzz"))] -#[doc(hidden)] pub mod fuzz; pub mod import; mod ingest; diff --git a/rs/moq-net-tests/Cargo.toml b/rs/moq-net-tests/Cargo.toml index a1e43a5349a..d22da1934d0 100644 --- a/rs/moq-net-tests/Cargo.toml +++ b/rs/moq-net-tests/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "moq-net-tests" -description = "moq-net tests that need its hidden fuzz module or JSON consumers." +description = "moq-net tests that need its `fuzz` module or JSON consumers." repository = "https://github.com/moq-dev/moq" license = "MIT OR Apache-2.0" version = "0.0.0" diff --git a/rs/moq-net-tests/tests/integration/hop_limit.rs b/rs/moq-net-tests/tests/integration/hop_limit.rs index 1b88be9f7f9..8c88c3db127 100644 --- a/rs/moq-net-tests/tests/integration/hop_limit.rs +++ b/rs/moq-net-tests/tests/integration/hop_limit.rs @@ -2,7 +2,7 @@ //! forwards one it has no room to stamp. //! //! Only a peer's route arrives with hops, so the routes under test land through the -//! hidden `fuzz` module as a session lands them. +//! `fuzz` module as a session lands them. use crate::support::{ Kind, drain, diff --git a/rs/moq-net-tests/tests/integration/ietf_conformance.rs b/rs/moq-net-tests/tests/integration/ietf_conformance.rs index 082601de4d5..57b56cf64a1 100644 --- a/rs/moq-net-tests/tests/integration/ietf_conformance.rs +++ b/rs/moq-net-tests/tests/integration/ietf_conformance.rs @@ -1,7 +1,7 @@ //! Moxygen announces one prefix, then serves its sixteen-field test namespaces. //! //! It answers each namespace without announcing its exact path, as only a session's peer -//! claim does, so the claim lands through the hidden `fuzz` module. +//! claim does, so the claim lands through the `fuzz` module. use crate::support::harness::{MockConnectOptions, connect_mock, spawn}; use moq_net::{Version, broadcast, origin}; diff --git a/rs/moq-net-tests/tests/integration/main.rs b/rs/moq-net-tests/tests/integration/main.rs index f0c63cca5bc..cbb6900f947 100644 --- a/rs/moq-net-tests/tests/integration/main.rs +++ b/rs/moq-net-tests/tests/integration/main.rs @@ -1,4 +1,4 @@ -//! Tests that need JSON consumers or moq-net's hidden fuzz hooks. +//! Tests that need JSON consumers or moq-net's `fuzz` module. mod hop_limit; mod ietf_conformance; diff --git a/rs/moq-net-tests/tests/integration/restart.rs b/rs/moq-net-tests/tests/integration/restart.rs index 8cf357e6321..0033f6a3413 100644 --- a/rs/moq-net-tests/tests/integration/restart.rs +++ b/rs/moq-net-tests/tests/integration/restart.rs @@ -185,7 +185,7 @@ relay_chain_tests! { /// A worker claiming `pool` under its own instance, `worker-`, answering each request /// with an output whose `video` track carries the worker's name. The claim is a peer's, landed through -/// the hidden `fuzz` module as a session lands it, so its answers carry no exact route. +/// the `fuzz` module as a session lands it, so its answers carry no exact route. struct Worker { origin: origin::Producer, _claim: Rc, @@ -305,7 +305,7 @@ async fn prefix_pool_lite07() { /// A claim at `live` whose handler answers each request with a fresh output, kept so /// the test can write into it: `outputs[n]` answered the `n`th request. Only a peer's -/// claim moves to another instance or re-prices in place, so it lands through the hidden +/// claim moves to another instance or re-prices in place, so it lands through the /// `fuzz` module as a session lands it. struct Claim { dynamic: Rc, diff --git a/rs/moq-net-tests/tests/integration/wide_hop.rs b/rs/moq-net-tests/tests/integration/wide_hop.rs index 50a46238047..923186b777b 100644 --- a/rs/moq-net-tests/tests/integration/wide_hop.rs +++ b/rs/moq-net-tests/tests/integration/wide_hop.rs @@ -1,7 +1,7 @@ //! Hop IDs span 64 bits. A link whose varint stops at 2^62 - 1 carries a larger //! one as 0, the reserved unknown hop. //! -//! The hop chain is crate-private, so it is read through the hidden `fuzz` module. +//! The hop chain is crate-private, so it is read through the `fuzz` module. use crate::support; diff --git a/rs/moq-net/Cargo.toml b/rs/moq-net/Cargo.toml index ab3685f2cf2..acd353c3f80 100644 --- a/rs/moq-net/Cargo.toml +++ b/rs/moq-net/Cargo.toml @@ -25,7 +25,7 @@ async = ["dep:futures", "kio/async", "kio/runtime"] # Exposes the wire codecs to the `fuzz/` harness and the announce, codec, path, varint, frame, and canonical storage benches, # session views of an origin to the viewers bench, the lite publisher's run loop to the # publisher bench, and a lite-07 publisher's frame bounds to the ranges bench, through the -# hidden `fuzz` module. Off by default and hidden from the docs. +# `fuzz` module. Off by default. fuzz = ["async"] [dependencies] @@ -77,43 +77,43 @@ name = "fuzz-seeds" path = "examples/fuzz-seeds.rs" required-features = ["fuzz"] -# Lands peer claims through the hidden `fuzz` module, as a session does. +# Lands peer claims through the `fuzz` module, as a session does. [[bench]] name = "origin" harness = false required-features = ["fuzz"] -# Reaches the crate-private session view of an origin through the hidden `fuzz` module. +# Reaches the crate-private session view of an origin through the `fuzz` module. [[bench]] name = "viewers" harness = false required-features = ["fuzz"] -# Reaches the private announce codec through the hidden `fuzz` module. +# Reaches the private announce codec through the `fuzz` module. [[bench]] name = "announce" harness = false required-features = ["fuzz"] -# Reaches the crate-private session snapshot through the hidden `fuzz` module. +# Reaches the crate-private session snapshot through the `fuzz` module. [[bench]] name = "synced" harness = false required-features = ["fuzz"] -# Reaches the crate-private session interests through the hidden `fuzz` module. +# Reaches the crate-private session interests through the `fuzz` module. [[bench]] name = "interests" harness = false required-features = ["fuzz"] -# Reaches the crate-private route layer through the hidden `fuzz` module. +# Reaches the crate-private route layer through the `fuzz` module. [[bench]] name = "routes" harness = false required-features = ["fuzz"] -# Reaches the private message codecs through the hidden `fuzz` module. +# Reaches the private message codecs through the `fuzz` module. [[bench]] name = "codec" harness = false @@ -125,25 +125,25 @@ name = "parameters" harness = false required-features = ["fuzz"] -# Reaches the private lite codec through the hidden `fuzz` module. +# Reaches the private lite codec through the `fuzz` module. [[bench]] name = "varint" harness = false required-features = ["fuzz"] -# Reaches the private frame ingest through the hidden `fuzz` module. +# Reaches the private frame ingest through the `fuzz` module. [[bench]] name = "frame" harness = false required-features = ["fuzz"] -# Reaches a lite-07 publisher's frame bounds through the hidden `fuzz` module. +# Reaches a lite-07 publisher's frame bounds through the `fuzz` module. [[bench]] name = "ranges" harness = false required-features = ["fuzz"] -# Receives frames through the private session reader via the hidden `fuzz` module. +# Receives frames through the private session reader via the `fuzz` module. [[bench]] name = "backing" harness = false @@ -172,7 +172,7 @@ name = "tail" harness = false required-features = ["fuzz"] -# Drives the lite publisher's run loop through the hidden `fuzz` module. +# Drives the lite publisher's run loop through the `fuzz` module. [[bench]] name = "publisher" harness = false @@ -208,13 +208,13 @@ required-features = ["async"] name = "refresh" harness = false -# Follows the session's probe target through the hidden `fuzz` module. +# Follows the session's probe target through the `fuzz` module. [[bench]] name = "bandwidth" harness = false required-features = ["fuzz"] -# Uses the hidden fuzz module to observe source selection without public API. +# Uses the `fuzz` module to observe source selection without public API. [[bench]] name = "retirement" harness = false diff --git a/rs/moq-net/benches/interests.rs b/rs/moq-net/benches/interests.rs index dc4318801e3..89cef731b4c 100644 --- a/rs/moq-net/benches/interests.rs +++ b/rs/moq-net/benches/interests.rs @@ -10,7 +10,7 @@ //! - uncovered: a watch on a fresh prefix comes and goes, which every session hears. //! - covered: a watch beneath one already held comes and goes, which no session hears. //! -//! The session side is crate-private, so this reaches it through the hidden `fuzz` module. +//! The session side is crate-private, so this reaches it through the `fuzz` module. //! //! Run with `cargo bench -p moq-net --features fuzz --bench interests`. diff --git a/rs/moq-net/benches/origin.rs b/rs/moq-net/benches/origin.rs index 4f25e31b27f..77c1b5ccecd 100644 --- a/rs/moq-net/benches/origin.rs +++ b/rs/moq-net/benches/origin.rs @@ -32,7 +32,7 @@ //! A claim's answer, and its retraction moving the claim to a fresh instance, is swept //! over the paths the claim already answered and the cursors watching it. //! -//! Peer claims land through the hidden `fuzz` module, as a session lands them. +//! Peer claims land through the `fuzz` module, as a session lands them. //! //! Run with `cargo bench -p moq-net --features fuzz --bench origin`. diff --git a/rs/moq-net/benches/routes.rs b/rs/moq-net/benches/routes.rs index 6f862af4cc0..034a7a30051 100644 --- a/rs/moq-net/benches/routes.rs +++ b/rs/moq-net/benches/routes.rs @@ -12,7 +12,7 @@ //! batches behind the first, and each hears its next batch. Swept over the claims //! the origin holds and how far the second lags: neither axis is expected to grow. //! -//! The route layer is crate-private, so this reaches it through the hidden `fuzz` module. +//! The route layer is crate-private, so this reaches it through the `fuzz` module. //! //! Run with `cargo bench -p moq-net --features fuzz --bench routes`. diff --git a/rs/moq-net/benches/synced.rs b/rs/moq-net/benches/synced.rs index a04daad9327..86a5af13253 100644 --- a/rs/moq-net/benches/synced.rs +++ b/rs/moq-net/benches/synced.rs @@ -9,7 +9,7 @@ //! - land: a snapshot starts and lands under one prefix, releasing its one consumer. //! - register: a consumer of one prefix registers, counting the snapshots it waits on. //! -//! The snapshot is crate-private, so this reaches it through the hidden `fuzz` module. +//! The snapshot is crate-private, so this reaches it through the `fuzz` module. //! //! Run with `cargo bench -p moq-net --features fuzz --bench synced`. diff --git a/rs/moq-net/benches/viewers.rs b/rs/moq-net/benches/viewers.rs index 4999fe2eba4..0be9c30317c 100644 --- a/rs/moq-net/benches/viewers.rs +++ b/rs/moq-net/benches/viewers.rs @@ -4,7 +4,7 @@ //! may depend on how many came before. //! //! Each viewer excludes its own hop the way a session serves its peer. That view -//! is crate-private, so this reaches it through the hidden `fuzz` module. Its own +//! is crate-private, so this reaches it through the `fuzz` module. Its own //! target keeps the counting allocator out of the `origin` bench. //! //! Run with `cargo bench -p moq-net --features fuzz --bench viewers`. diff --git a/rs/moq-net/fuzz/Cargo.toml b/rs/moq-net/fuzz/Cargo.toml index e9812d7ac2b..3b24c6596aa 100644 --- a/rs/moq-net/fuzz/Cargo.toml +++ b/rs/moq-net/fuzz/Cargo.toml @@ -6,7 +6,7 @@ # excluded from every gate that means "all of it" instead: libFuzzer needs a nightly # toolchain and sanitizer flags, so nothing in `check` or `test` may compile it. # -# The target bodies live in `moq-net`'s hidden `fuzz` module, not here: `lite` and +# The target bodies live in `moq-net`'s `fuzz` module, not here: `lite` and # `ietf` are private modules, so this crate cannot reach a single decoder, and sharing # the bodies is what lets the crate's own tests replay the corpus on stable. [package] diff --git a/rs/moq-net/fuzz/README.md b/rs/moq-net/fuzz/README.md index f4b4b4809ab..76c836071e4 100644 --- a/rs/moq-net/fuzz/README.md +++ b/rs/moq-net/fuzz/README.md @@ -18,7 +18,7 @@ minimized corpus, and uploads failure inputs as workflow artifacts. ## How it fits together -The target bodies are not in `fuzz_targets/`; they are in `moq-net`'s hidden `fuzz` +The target bodies are not in `fuzz_targets/`; they are in `moq-net`'s `fuzz` module (`src/fuzz.rs`), and the files here are one-line shims. Two reasons: - `lite` and `ietf` are private modules, so an outside crate cannot reach a single diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 6c61cb1013f..4cbcf800809 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -1,4 +1,5 @@ -//! Fuzz target bodies for the wire codecs, plus the seeds they start from. +//! Test support, enabled by the `fuzz` feature: fuzz target bodies for the wire codecs and their +//! seeds, plus the hooks the benches and `moq-net-tests` reach private state through. //! //! Each `pub fn` here is one target: it takes raw fuzzer bytes, drives a decoder, and //! checks that whatever came out survives a re-encode. The bodies live in the crate @@ -7,8 +8,7 @@ //! here also lets the tests below replay them on stable, which is what turns a crash //! the fuzzer found into a regression `just check` runs. //! -//! Compiled only under `cfg(test)` or the `fuzz` feature, so none of this is part of -//! the published API. See `fuzz/README.md` for the workflow. +//! It tracks the crate's internals, so it can change in any release. See `fuzz/README.md` for the workflow. mod ingest; mod request; @@ -215,9 +215,12 @@ impl ObjectBatch { pub struct CanonicalStorage(crate::model::group::StorageBench); impl CanonicalStorage { + /// A group of `frames` 64-byte frames, each held by `views` extra owners. pub fn new(frames: usize, views: usize) -> Self { Self(crate::model::group::StorageBench::new(frames, views)) } + + /// The group's stored bytes. pub fn size(&self) -> u64 { self.0.size() } @@ -789,7 +792,7 @@ impl Interests { /// Add `n` publisher-side payload bytes on `(path, tier)`, creating the entry /// when the path is tracked. The stats producer bench marks a share of held /// paths changed this way, without a media frame. A later [`crate::stats::Registry::report`] -/// drops the entry unless some guard still holds it. Not a public API. +/// drops the entry unless some guard still holds it. pub fn bump_publisher_bytes( registry: &crate::stats::Registry, path: impl Into, diff --git a/rs/moq-net/src/lib.rs b/rs/moq-net/src/lib.rs index 8ca3ed74a7e..b9f9565a26a 100644 --- a/rs/moq-net/src/lib.rs +++ b/rs/moq-net/src/lib.rs @@ -97,18 +97,13 @@ mod discovery; #[cfg(feature = "async")] mod driver; mod error; +#[cfg(all(feature = "async", any(test, feature = "fuzz")))] +pub mod fuzz; #[cfg(feature = "async")] pub mod goaway; #[cfg(not(feature = "async"))] #[allow(dead_code, unused_imports, reason = "async-only")] mod goaway; -#[cfg(test)] -mod test_tracing; -// Not part of the public API: compiled only for the crate's own tests and for the -// `fuzz/` harness. -#[cfg(all(feature = "async", any(test, feature = "fuzz")))] -#[doc(hidden)] -pub mod fuzz; mod ietf; mod instance; mod ledger; @@ -129,6 +124,8 @@ mod setup; mod tail; #[cfg(all(test, feature = "async"))] mod test_interop; +#[cfg(test)] +mod test_tracing; mod util; mod version; diff --git a/rs/moq-net/src/stats.rs b/rs/moq-net/src/stats.rs index 95ea379d967..6d7c6fd962c 100644 --- a/rs/moq-net/src/stats.rs +++ b/rs/moq-net/src/stats.rs @@ -1233,7 +1233,7 @@ impl Registry { } } - /// Add `n` publisher-side payload bytes on `(path, tier)`. The hidden `fuzz` + /// Add `n` publisher-side payload bytes on `(path, tier)`. The `fuzz` /// module exposes this to the stats producer bench. A report drops the entry /// unless a guard still holds it. #[cfg(any(test, feature = "fuzz"))] diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index 32080d85de9..e87fc5e9185 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -1,6 +1,6 @@ //! Integration tests without feature-enabling dependencies, linked into one binary. //! -//! Tests needing JSON consumers or the hidden `fuzz` module live in `moq-net-tests`. +//! Tests needing JSON consumers or the `fuzz` module live in `moq-net-tests`. //! Each `tests/*.rs` file is its own crate that links this package and its //! dependencies again, so add new integration tests here as a module. //! `send.rs` stays apart because it builds on another toolchain. diff --git a/rs/moq-quic/Cargo.toml b/rs/moq-quic/Cargo.toml index e47aea99103..dc598b73cd6 100644 --- a/rs/moq-quic/Cargo.toml +++ b/rs/moq-quic/Cargo.toml @@ -19,13 +19,12 @@ qlog = ["dep:qlog"] # transport. Driven by the `qmux` crate. mux = [] # The seeded, impaired network path (`sim::Link`) the tests run endpoints across, for a -# driver that runs QUIC on kio's simulator. Hidden from the docs; meant for -# dev-dependencies. +# driver that runs QUIC on kio's simulator. Meant for dev-dependencies. sim = [] # Exposes the `fuzz/` harness's target bodies, and the plaintext crypto the connection -# target and the benches run on, through the hidden `fuzz` module. rind keys the endpoint's -# reset and token secrets. Off by default and hidden from the docs. +# target and the benches run on, through the `fuzz` module. rind keys the endpoint's +# reset and token secrets. Off by default. fuzz = ["dep:arbitrary"] [dependencies] @@ -75,7 +74,7 @@ name = "fuzz-seeds" path = "examples/fuzz-seeds.rs" required-features = ["fuzz"] -# The benches run on the hidden `fuzz` module's plaintext crypto. +# The benches run on the `fuzz` module's plaintext crypto. [[bench]] name = "streams" harness = false diff --git a/rs/moq-quic/MIGRATE.md b/rs/moq-quic/MIGRATE.md index 6efd72b6225..a556c7c354a 100644 --- a/rs/moq-quic/MIGRATE.md +++ b/rs/moq-quic/MIGRATE.md @@ -102,9 +102,9 @@ Unused `Connection::local_ip`, `min_rtt`, `has_pending_retransmits`, `ping`, `fo To rotate TLS certificates on a running endpoint, give the `ServerConfig`'s rustls config a `ResolvesServerCert` that returns the current certificate, as moq-tokio's `tls::Serve` does. Use `Connection::stats` for RTT metrics and receive metadata for local addresses. -### `mux` is an internal integration surface +### `connection::mux` is the QMux connection -The `mux` module is hidden from public API documentation; use the `qmux` crate for reliable-transport sessions. It remains public only so qmux can share the stream state machine. +The `mux` feature adds `connection::mux`, the sans-IO QMux connection that shares the stream state machine; use the `qmux` crate for reliable-transport sessions. Each entry is a change from `quinn-proto`. @@ -148,7 +148,7 @@ priority. Higher still goes first. ### `fuzzing` module and `arbitrary` feature removed The `cfg(fuzzing)` `fuzzing` module and the `arbitrary` feature are gone. The fuzz -targets run through the hidden `fuzz` module behind the `fuzz` feature; see +targets run through the `fuzz` module behind the `fuzz` feature; see [`fuzz/`](fuzz/README.md). ### `congestion::NewReno` removed diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index 22885a320ad..41c21c6a560 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -85,7 +85,7 @@ Changes on top of the upstream commit, besides the renames: - **Sharded connection IDs** (`RandomConnectionIdGenerator::set_shard`): every CID names shard `index` of a `count`-endpoint group in its first byte, modulo `count`, and sets a marker in its fourth byte, so a `SO_REUSEPORT` steering filter delivers a packet to the endpoint that issued its CID. `validate` refuses a CID without this shard's prefix and marker, so a misrouted packet is dropped instead of answered with a stateless reset. Not offered upstream; it serves moq-sock's per-core sharding. - **`RESET_STREAM_AT`** ([draft-ietf-quic-reliable-stream-reset](https://datatracker.ietf.org/doc/draft-ietf-quic-reliable-stream-reset/)): the `reset_stream_at` transport parameter is always advertised, `send::Stream::reset_at` keeps the data written before an offset reliable, holding a plain `RESET_STREAM` until that data is acknowledged when the peer lacks the extension, and a received reset surfaces only once the reliable prefix has been read. Stopping a stream after it was reset no longer credits its unread data twice. - **ACK progress** (`send::Stream::offset`, `send::Stream::acked`, `poll_acked`): `acked` reports whether the peer acknowledged every byte before an offset, by comparing against the stream's contiguous acknowledged prefix, which a reset that frees the stream keeps with its end. A `false` stores the lowest polled offset on the stream, and the ACK that advances the prefix past it, or a reset or `STOP_SENDING` that makes it fail, wakes the stream's `poll_acked` waiters, so a port touching `StreamsState::received_ack_of`, `received_stop_sending`, or the reset path has to keep that wake. Not offered upstream: quinn has no API for when sent data was acknowledged. -- **Fuzzing** runs through the hidden `fuzz` module behind a `fuzz` feature, with one-line shims in [`fuzz/`](fuzz/README.md): upstream's `packet`, `params`, `streamid`, and `streams` targets, plus `frame` and a connection op sequence. Upstream's `cfg(fuzzing)` `fuzzing` module and `arbitrary` feature are gone, so its fuzz patches port by hand. +- **Fuzzing** runs through the `fuzz` module behind a `fuzz` feature, with one-line shims in [`fuzz/`](fuzz/README.md): upstream's `packet`, `params`, `streamid`, and `streams` targets, plus `frame` and a connection op sequence. Upstream's `cfg(fuzzing)` `fuzzing` module and `arbitrary` feature are gone, so its fuzz patches port by hand. - **Draining endpoint event**: `EndpointEvent::is_draining` fires once per connection when it stops being active, before the drained event, so the driver's `Endpoint::wait_all_draining` can end a shutdown without sitting out every closing period. - **Late stream limits**: peer transport parameters that raise a stream limit after an open was refused wake `Streams::poll_open`, as `MAX_STREAMS` does. A server accepted before the rest of a multi-packet ClientHello (a post-quantum key share is enough) otherwise leaves a 0.5-RTT `open_uni` parked until the idle timeout. Not offered upstream yet. - **Receive leases** (`streams/lease.rs`, `recv::Stream::lease`), building on [#5675](https://github.com/moq-dev/moq/pull/5675)'s held credit: a lease reads a stream's ordered bytes out of the assembler but keeps their connection credit, counted in `StreamsState::held`, until the reader `advance`s or `take`s past them and `release`s, while stream credit advances as usual. A lease copies the bytes it reads out of their datagram buffers (`Leased::extend`), so a byte of credit never pins a receive batch, whether its reader waits, parks, or takes it. The lease lives in the stream's `Recv`, which a leased stream keeps, with its stream ID, until the application stops it, even past the FIN. When the peer has spent the connection window and no reader can make progress, the stream ranked lowest by `set_priority` (the newest on a tie) is evicted: its bytes and credit are freed, the stream is stopped without `STOP_SENDING`, and its next lease fails with `recv::Error::Evicted`; the application's `stop` then tells the peer and frees the stream once its final size is known. Leased streams that may hold credit are kept in `StreamsState::ranks`, in eviction order, so an eviction costs a logarithm rather than a scan of every stream. `MAX_DATA` goes out for any increase once the peer is within an eighth of the window (`max_data_due`), since what leases release may never add up to the usual threshold. Streams with data their reader has not looked at hold eviction off; they are kept in `StreamsState::unlooked`, so the stuck check stays constant time however many readers wait. `ConnectionStats::credit` reports the leased and available bytes, the streams evicted, and the peak credit spent. A `recv::Lease` owns the chunks it took, so a reader holds no connection borrow while it parses; `restore` or `release` hands it back, and an eviction pass waits while one is out. A port touching `Chunks` or `add_read_credits` has to keep `held` balanced. diff --git a/rs/moq-quic/benches/common/mod.rs b/rs/moq-quic/benches/common/mod.rs index 82954b66b8a..24555bd10df 100644 --- a/rs/moq-quic/benches/common/mod.rs +++ b/rs/moq-quic/benches/common/mod.rs @@ -1,7 +1,7 @@ //! A sans-IO client/server harness for the benchmarks, built from the public API so the suite //! compiles on both sides of a rewrite. //! -//! Both sides run the hidden `fuzz` module's plaintext crypto, which skips TLS and leaves packets +//! Both sides run the `fuzz` module's plaintext crypto, which skips TLS and leaves packets //! unprotected, and the congestion window is unlimited, so a benchmark measures protocol CPU //! rather than AES or a controller's model. Time is virtual: it only advances to //! the next timer once no datagram is in flight. Cases are timed in thread CPU time, so a busy diff --git a/rs/moq-quic/fuzz/Cargo.toml b/rs/moq-quic/fuzz/Cargo.toml index ae75a6f32d8..7735e9ace65 100644 --- a/rs/moq-quic/fuzz/Cargo.toml +++ b/rs/moq-quic/fuzz/Cargo.toml @@ -6,7 +6,7 @@ # excluded from every gate that means "all of it" instead: libFuzzer needs a nightly # toolchain and sanitizer flags, so nothing in `check` or `test` may compile it. # -# The target bodies live in `moq-quic`'s hidden `fuzz` module, not here: the frame +# The target bodies live in `moq-quic`'s `fuzz` module, not here: the frame # decoder and stream state are private, and sharing the bodies is what lets the # crate's own tests replay the corpus on stable. [package] diff --git a/rs/moq-quic/fuzz/README.md b/rs/moq-quic/fuzz/README.md index b0f8bc3f33b..8d913943bca 100644 --- a/rs/moq-quic/fuzz/README.md +++ b/rs/moq-quic/fuzz/README.md @@ -19,7 +19,7 @@ minimized corpus, and uploads failure inputs as workflow artifacts. ## How it fits together -The target bodies are not in `fuzz_targets/`; they are in `moq-quic`'s hidden `fuzz` +The target bodies are not in `fuzz_targets/`; they are in `moq-quic`'s `fuzz` module (`src/fuzz.rs`), and the files here are one-line shims. Two reasons: - The frame decoder and the stream state are private, so an outside crate cannot reach diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index fba1f2093fb..d86202b5dc7 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -58,7 +58,6 @@ pub use datagrams::Datagrams; mod mtud; #[cfg(feature = "mux")] -#[doc(hidden)] pub mod mux; mod packet_builder; @@ -81,7 +80,6 @@ mod receive_timestamps; mod sent_packets; #[cfg(feature = "fuzz")] -#[doc(hidden)] pub use sent_packets::Bench as BenchSentPackets; mod slots; mod spaces; diff --git a/rs/moq-quic/src/connection/sent_packets.rs b/rs/moq-quic/src/connection/sent_packets.rs index ccc283a8558..859f15d6cb9 100644 --- a/rs/moq-quic/src/connection/sent_packets.rs +++ b/rs/moq-quic/src/connection/sent_packets.rs @@ -84,7 +84,8 @@ impl SentPackets { } } -/// `SentPackets` over packets of only a size and an ACK-eliciting flag, for the benches. +/// Test support for the sent-packet bench, enabled by the `fuzz` feature: `SentPackets` over +/// packets of only a size and an ACK-eliciting flag. #[cfg(feature = "fuzz")] pub struct Bench { packets: SentPackets, diff --git a/rs/moq-quic/src/fuzz.rs b/rs/moq-quic/src/fuzz.rs index fee231bb530..affd600df45 100644 --- a/rs/moq-quic/src/fuzz.rs +++ b/rs/moq-quic/src/fuzz.rs @@ -1,4 +1,5 @@ -//! Fuzz target bodies, plus the seeds they start from. +//! Test support, enabled by the `fuzz` feature: fuzz target bodies and their seeds, plus the +//! plaintext crypto and sent-packet storage the benches run on. //! //! Each `pub fn` here is one target: it takes raw fuzzer bytes and drives a parser or //! a state machine with them. The bodies live in the crate rather than in @@ -7,8 +8,8 @@ //! fuzzer found into a regression `just check` runs. //! //! `packet`, `params`, `streamid`, and `streams` are ported from upstream quinn's -//! `fuzz/fuzz_targets/`. Compiled only under `cfg(test)` or the `fuzz` feature, so none -//! of this is part of the published API. See `fuzz/README.md` for the workflow. +//! `fuzz/fuzz_targets/`. It tracks the crate's internals, so it can change in any release. See `fuzz/README.md` +//! for the workflow. //! //! [`Plaintext`] is the crypto the `connection` target and the benchmarks run on, and //! [`SentPackets`] is the sent-packet storage its benchmark drives. diff --git a/rs/moq-quic/src/lib.rs b/rs/moq-quic/src/lib.rs index 5d29677ce32..c1ff1f1505b 100644 --- a/rs/moq-quic/src/lib.rs +++ b/rs/moq-quic/src/lib.rs @@ -23,15 +23,10 @@ mod cid_queue; mod coding; mod constant_time; mod entropy; -// Not part of the public API: compiled only for the crate's own tests, the benches, and -// the `fuzz/` harness. The connection target uses rind for the endpoint's keys. #[cfg(any(test, feature = "fuzz"))] -#[doc(hidden)] pub mod fuzz; mod range_set; -// Not part of the public API: the network path the crate's tests and simulated drivers share. #[cfg(any(test, feature = "sim"))] -#[doc(hidden)] pub mod sim; #[cfg(test)] mod tests; @@ -47,9 +42,6 @@ pub mod connection; pub use crate::connection::streams::{recv, send}; pub mod stream; -#[cfg(feature = "mux")] -#[doc(hidden)] -pub use connection::mux; pub(crate) use stream::{Dir, Id as StreamId}; mod config; diff --git a/rs/moq-quic/src/sim.rs b/rs/moq-quic/src/sim.rs index 1d0a37d9680..333e83cd392 100644 --- a/rs/moq-quic/src/sim.rs +++ b/rs/moq-quic/src/sim.rs @@ -1,9 +1,8 @@ -//! A seeded, impaired network path: a rate-limited bottleneck queue, then random loss, -//! duplication, jitter, and reordering +//! Test support, enabled by the `sim` feature: a seeded, impaired network path, a +//! rate-limited bottleneck queue, then random loss, duplication, jitter, and reordering //! -//! Not part of the public API. The crate's own tests drive two endpoints across a [`Link`] -//! each way, and the `sim` feature exposes it for drivers that run QUIC on kio's -//! simulator with no sockets. +//! The crate's own tests drive two endpoints across a [`Link`] each way, and drivers that +//! run QUIC on kio's simulator with no sockets use it too. It tracks the crate's internals, so it can change in any release. use rand::{RngExt, rngs::StdRng}; @@ -24,6 +23,7 @@ pub struct Bottleneck { timed: bool, /// When the link finishes serving everything queued so far busy_until: Option, + /// What the queue has dropped, marked, and delayed so far pub stats: QueueStats, /// Each queued packet's arrival and queueing delay, once set to `Some` pub trace: Option>, diff --git a/rs/moq-relay/Cargo.toml b/rs/moq-relay/Cargo.toml index 6c2f71110c9..a6712c1682f 100644 --- a/rs/moq-relay/Cargo.toml +++ b/rs/moq-relay/Cargo.toml @@ -117,7 +117,7 @@ h2 = { workspace = true } kio = { path = "../kio", features = ["manual", "sim"] } moq-auth = { path = "../moq-auth", features = ["client", "serve"] } moq-congestion = { path = "../moq-congestion" } -# The shared session bench fills FETCH runs through the hidden `fuzz` module. +# The shared session bench fills FETCH runs through the `fuzz` module. moq-net = { path = "../moq-net", features = ["fuzz"] } moq-shaper = { path = "../moq-shaper", features = ["test"] } pem = { workspace = true } diff --git a/rs/moq-rtmp/Cargo.toml b/rs/moq-rtmp/Cargo.toml index e8241e44006..52977bc2743 100644 --- a/rs/moq-rtmp/Cargo.toml +++ b/rs/moq-rtmp/Cargo.toml @@ -21,8 +21,8 @@ default = ["tls"] # `pub use rustls`. Plaintext RTMP needs none of this, so an embedder can drop # TLS with `default-features = false`. tls = ["dep:rustls", "dep:tokio-rustls"] -# Exposes the AMF0 fuzz target body and seeds to the `fuzz/` harness through the hidden `fuzz` -# module. Off by default and hidden from the docs. +# Exposes the AMF0 fuzz target body and seeds to the `fuzz/` harness through the `fuzz` +# module. Off by default. fuzz = [] # Pure library: the RTMP <-> MoQ gateway. The CLI lives in `moq-cli` (the `rtmp` diff --git a/rs/moq-rtmp/fuzz/Cargo.toml b/rs/moq-rtmp/fuzz/Cargo.toml index 857e85c0d7f..eabc7487ad1 100644 --- a/rs/moq-rtmp/fuzz/Cargo.toml +++ b/rs/moq-rtmp/fuzz/Cargo.toml @@ -6,7 +6,7 @@ # excluded from every gate that means "all of it" instead: libFuzzer needs a nightly # toolchain and sanitizer flags, so nothing in `check` or `test` may compile it. # -# The target body lives in `moq-rtmp`'s hidden `fuzz` module, not here: the vendored +# The target body lives in `moq-rtmp`'s `fuzz` module, not here: the vendored # `rml` module is private, and sharing the body is what lets the crate's own tests # replay the corpus on stable. [package] diff --git a/rs/moq-rtmp/fuzz/README.md b/rs/moq-rtmp/fuzz/README.md index 627f13e5a0b..c12295d3157 100644 --- a/rs/moq-rtmp/fuzz/README.md +++ b/rs/moq-rtmp/fuzz/README.md @@ -14,7 +14,7 @@ run's minimized corpus, and uploads failure inputs as workflow artifacts. ## How it fits together -The target body is not in `fuzz_targets/`; it is in `moq-rtmp`'s hidden `fuzz` module +The target body is not in `fuzz_targets/`; it is in `moq-rtmp`'s `fuzz` module (`src/fuzz.rs`), because the vendored `rml` module is private, and so `just check` replays it on the pinned stable toolchain. The shim here also registers the counting allocator the body measures with. diff --git a/rs/moq-rtmp/src/fuzz.rs b/rs/moq-rtmp/src/fuzz.rs index cfb46f199a5..a8c2a7f9e3b 100644 --- a/rs/moq-rtmp/src/fuzz.rs +++ b/rs/moq-rtmp/src/fuzz.rs @@ -1,4 +1,5 @@ -//! The fuzz target body for the AMF0 receive views, plus the seeds it starts from. +//! Test support, enabled by the `fuzz` feature: the fuzz target body for the AMF0 receive views, +//! plus its seeds. //! //! The target reads an RTMP AMF0 message through the borrowed views a session uses, walking every //! value a peer supplied, and checks that it allocates nothing: the views keep the encoded bytes @@ -6,8 +7,7 @@ //! lives here rather than in `fuzz/fuzz_targets/` because the `rml` module is private, and so the //! tests below replay it on stable. //! -//! Compiled only under `cfg(test)` or the `fuzz` feature, so none of this is part of the published -//! API. See `fuzz/README.md` for the workflow. +//! It tracks the crate's internals, so it can change in any release. See `fuzz/README.md` for the workflow. use crate::rml::amf0::{Amf0Value, Value, allocation::measure, serialize}; use crate::rml::messages::MessagePayload; diff --git a/rs/moq-rtmp/src/lib.rs b/rs/moq-rtmp/src/lib.rs index 2d43e987003..a09b301e0c3 100644 --- a/rs/moq-rtmp/src/lib.rs +++ b/rs/moq-rtmp/src/lib.rs @@ -63,10 +63,7 @@ use std::time::Duration; mod dial; mod error; mod flv; -// Not part of the public API: compiled only for the crate's own tests and for the `fuzz/` -// harness. #[cfg(any(test, feature = "fuzz"))] -#[doc(hidden)] pub mod fuzz; mod metadata; // Vendored forks of rml_rtmp and rml_amf0; see the module docs. diff --git a/rs/moq-stats/Cargo.toml b/rs/moq-stats/Cargo.toml index 9c4c50bacf9..da52b77a512 100644 --- a/rs/moq-stats/Cargo.toml +++ b/rs/moq-stats/Cargo.toml @@ -19,9 +19,8 @@ features = ["async"] default = [] # Generated async wrappers for the poll methods, and `Publisher::run`. Enable where you `.await` them. async = ["kio/async"] -# Exposes one producer tick to the producer_tick bench, through the hidden -# `produce::bench` module, and moq-net's hidden counter bump. Off by default -# and hidden from the docs. Not a supported API. +# Exposes one producer tick to the producer_tick bench, through the +# `produce::bench` module, and moq-net's counter bump. Off by default. bench = ["moq-net/fuzz"] [dependencies] @@ -54,7 +53,7 @@ harness = false name = "aggregate" harness = false -# Reaches the producer drain through the hidden `bench` module. +# Reaches the producer drain through the `bench` module. [[bench]] name = "producer_tick" harness = false diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index cc046adb132..ec5ee7a5c20 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -402,10 +402,7 @@ impl Drain { } } -/// Drives one producer tick for the benchmark. Not a public API: compiled only -/// with the `bench` feature, which nothing enables by default. #[cfg(feature = "bench")] -#[doc(hidden)] #[path = "produce_bench.rs"] pub mod bench; diff --git a/rs/moq-stats/src/produce_bench.rs b/rs/moq-stats/src/produce_bench.rs index 628880e1bd0..091163f7130 100644 --- a/rs/moq-stats/src/produce_bench.rs +++ b/rs/moq-stats/src/produce_bench.rs @@ -1,5 +1,5 @@ -//! One producer tick, without the publish interval. The benchmark drives this -//! over a registry it filled itself. Not a public API. +//! Test support for the producer bench, enabled by the `bench` feature: one producer tick, +//! without the publish interval, over a registry the bench filled itself. use std::time::Duration; diff --git a/rs/moq-tokio/src/lib.rs b/rs/moq-tokio/src/lib.rs index b6ba044fbb2..6a1767ad08d 100644 --- a/rs/moq-tokio/src/lib.rs +++ b/rs/moq-tokio/src/lib.rs @@ -58,10 +58,8 @@ mod runtime; pub mod secret; #[cfg(feature = "_transport")] pub mod server; -#[doc(hidden)] pub mod settings; #[cfg(all(feature = "sim", feature = "net"))] -#[doc(hidden)] pub mod sim; #[cfg(feature = "tcp")] pub mod tcp; diff --git a/rs/moq-tokio/src/quic/driver/mod.rs b/rs/moq-tokio/src/quic/driver/mod.rs index 9040c29adc0..1ae87088d29 100644 --- a/rs/moq-tokio/src/quic/driver/mod.rs +++ b/rs/moq-tokio/src/quic/driver/mod.rs @@ -58,7 +58,6 @@ pub mod recv; mod runtime; pub mod send; #[cfg(feature = "sim")] -#[doc(hidden)] pub mod sim; mod work_limiter; diff --git a/rs/moq-tokio/src/quic/driver/sim.rs b/rs/moq-tokio/src/quic/driver/sim.rs index 63c590e756f..417e6569f2d 100644 --- a/rs/moq-tokio/src/quic/driver/sim.rs +++ b/rs/moq-tokio/src/quic/driver/sim.rs @@ -1,4 +1,5 @@ -//! QUIC with no sockets: endpoints on kio's simulator, joined by seeded, impaired paths +//! Test support, enabled by the `sim` feature: QUIC with no sockets, endpoints on kio's +//! simulator joined by seeded, impaired paths //! //! A [`Network`] binds a [`Socket`] per address for [`Endpoint::new`](super::endpoint::Endpoint::new). //! Each datagram crosses the [`Path`] from its sender to its destination: the [`Link`]'s @@ -6,8 +7,9 @@ //! reordering, all drawn from the network's seed. With each endpoint's `rng_seed` set too, a //! seed replays a run exactly. A destination nobody bound drops the datagram, as UDP does. //! -//! Not part of the public API. It runs only inside `kio::clock::sim::run`, where every -//! arrival is a simulated timer. +//! It runs only inside `kio::clock::sim::run`, where every arrival is a simulated timer. +//! +//! It tracks the crate's internals, so it can change in any release. use std::{ collections::{HashMap, VecDeque}, diff --git a/rs/moq-tokio/src/settings.rs b/rs/moq-tokio/src/settings.rs index c5d72a9e304..2d62e4eb456 100644 --- a/rs/moq-tokio/src/settings.rs +++ b/rs/moq-tokio/src/settings.rs @@ -12,7 +12,6 @@ /// Accept-side settings (`[listen]`, `--listen-*`). #[derive(usage_rs::Config)] #[usage(prefix = "listen")] -#[doc(hidden)] pub struct Listen { #[usage(env = "MOQ_LISTEN", cli("--listen"))] bind: Option, @@ -130,7 +129,6 @@ struct ListenUnixAllow { /// Dial-side settings (`[connect]`, `--connect-*`). #[derive(usage_rs::Config)] #[usage(prefix = "connect")] -#[doc(hidden)] pub struct Connect { #[usage(env = "MOQ_CONNECT", cli("--connect"))] url: Option, @@ -240,7 +238,6 @@ struct ConnectWebsocket { /// Shared QUIC tuning (`[quic]`, `--quic-*`). #[derive(usage_rs::Config)] #[usage(prefix = "quic")] -#[doc(hidden)] pub struct Quic { #[usage(env = "MOQ_QUIC_MAX_STREAMS", cli("--quic-max-streams"))] max_streams: Option, @@ -279,7 +276,6 @@ pub struct Quic { /// Log settings (`[log]`, `--log-level`). #[derive(usage_rs::Config)] #[usage(prefix = "log")] -#[doc(hidden)] pub struct Log { #[usage(env = "MOQ_LOG_LEVEL", cli("--log-level"))] level: Option, diff --git a/rs/moq-tokio/src/sim.rs b/rs/moq-tokio/src/sim.rs index 8610391ec7d..51d1cac163f 100644 --- a/rs/moq-tokio/src/sim.rs +++ b/rs/moq-tokio/src/sim.rs @@ -1,9 +1,12 @@ -//! A network inside the process, for tests on `kio::clock::sim`. +//! Test support, enabled by the `sim` and `net` features: a network inside the process, for +//! tests on `kio::clock::sim`. //! //! A [`Client`](crate::Client) given a [`Network`] opens no sockets: each dial becomes a //! [`Dial`] on its [`Dials`], and whoever routes those answers it with an in-memory link. //! Everything else a dial does, the handshake, redials, and GOAWAY handling, runs as it //! would over a socket. +//! +//! It tracks the crate's internals, so it can change in any release. use futures::{ StreamExt, diff --git a/rs/moq-webrtc/src/lib.rs b/rs/moq-webrtc/src/lib.rs index 8bb4df6704d..28bc077c9d8 100644 --- a/rs/moq-webrtc/src/lib.rs +++ b/rs/moq-webrtc/src/lib.rs @@ -46,7 +46,6 @@ pub mod client; #[cfg(feature = "p2p")] -#[doc(hidden)] pub mod p2p; mod rate; pub mod server; diff --git a/rs/moq-webrtc/src/vendor/is/candidate.rs b/rs/moq-webrtc/src/vendor/is/candidate.rs index fbf6ec45001..ecc9ae62551 100644 --- a/rs/moq-webrtc/src/vendor/is/candidate.rs +++ b/rs/moq-webrtc/src/vendor/is/candidate.rs @@ -680,30 +680,24 @@ impl<'de> Deserialize<'de> for Candidate { } } -#[doc(hidden)] /// Marker for a builder requiring a protocol selection. pub struct NoProtocol(()); -#[doc(hidden)] /// State indicating a transport protocol has been assigned to the builder. pub struct HasProtocol

{ _marker: PhantomData

, protocol: Protocol, } -#[doc(hidden)] /// Marker for a builder using the UDP protocol. pub struct Udp(()); -#[doc(hidden)] /// Marker for a builder using a TCP-based protocol. pub struct Tcp(()); -#[doc(hidden)] /// Marker indicating the candidate's network route (addresses and kind) is not yet defined. pub struct NoRoute(()); -#[doc(hidden)] /// State containing the network route and addresses required to build a candidate. pub struct HasRoute { kind: CandidateKind, diff --git a/rs/moq-webrtc/src/vendor/is/stun.rs b/rs/moq-webrtc/src/vendor/is/stun.rs index 5b3a65288f5..7ffa8faf76a 100644 --- a/rs/moq-webrtc/src/vendor/is/stun.rs +++ b/rs/moq-webrtc/src/vendor/is/stun.rs @@ -1151,12 +1151,10 @@ mod builder { use std::net::SocketAddr; /// Type state representing a builder where the STUN method has not yet been set. - #[doc(hidden)] #[derive(Default, Debug, Clone)] pub struct NoMethod; /// Type state representing a builder where the STUN method has been set, but the class has not. - #[doc(hidden)] #[derive(Debug, Clone)] pub struct HasMethod { method: Method, @@ -1164,7 +1162,6 @@ mod builder { /// Type state representing a builder where both the STUN method and class have been set. /// Attributes can now be added. - #[doc(hidden)] #[derive(Debug, Clone)] pub struct HasClass { method: Method, diff --git a/rs/moq-webrtc/src/vendor/sctp_proto/config.rs b/rs/moq-webrtc/src/vendor/sctp_proto/config.rs index b85156e9d77..0549f5f556b 100644 --- a/rs/moq-webrtc/src/vendor/sctp_proto/config.rs +++ b/rs/moq-webrtc/src/vendor/sctp_proto/config.rs @@ -289,7 +289,6 @@ impl EndpointConfig { /// /// While `get_` accessors are typically unidiomatic in Rust, we favor concision for setters, /// which will be used far more heavily. - #[doc(hidden)] pub fn get_max_payload_size(&self) -> u32 { self.max_payload_size } diff --git a/rs/moq-webrtc/src/vendor/sctp_proto/shared.rs b/rs/moq-webrtc/src/vendor/sctp_proto/shared.rs index 1535a8e7278..5d66eef2a58 100644 --- a/rs/moq-webrtc/src/vendor/sctp_proto/shared.rs +++ b/rs/moq-webrtc/src/vendor/sctp_proto/shared.rs @@ -53,11 +53,8 @@ pub type AssociationId = u32; #[repr(u8)] #[derive(Debug, Copy, Clone, Eq, PartialEq)] pub enum EcnCodepoint { - #[doc(hidden)] Ect0 = 0b10, - #[doc(hidden)] Ect1 = 0b01, - #[doc(hidden)] Ce = 0b11, } diff --git a/rs/moq-webrtc/src/vendor/str0m/format/codec.rs b/rs/moq-webrtc/src/vendor/str0m/format/codec.rs index 4274f719f59..3e4f808568c 100644 --- a/rs/moq-webrtc/src/vendor/str0m/format/codec.rs +++ b/rs/moq-webrtc/src/vendor/str0m/format/codec.rs @@ -32,21 +32,16 @@ pub enum Codec { AmrWb, H264, // TODO show this when we support h265. - #[doc(hidden)] H265, Vp8, Vp9, // TODO show this when we support Av1. - #[doc(hidden)] Av1, /// Technically not a codec, but used in places where codecs go /// in `a=rtpmap` lines. - #[doc(hidden)] Rtx, /// For RTP mode. No codec. - #[doc(hidden)] Null, - #[doc(hidden)] Unknown, } diff --git a/rs/moq-webrtc/src/vendor/str0m/lib.rs b/rs/moq-webrtc/src/vendor/str0m/lib.rs index 7ed73e7b453..b750ca9de11 100644 --- a/rs/moq-webrtc/src/vendor/str0m/lib.rs +++ b/rs/moq-webrtc/src/vendor/str0m/lib.rs @@ -704,7 +704,6 @@ pub mod config { // The ICE API is not necessary to interact with directly for "regular" // use of str0m. This is exported for other libraries that want to // reuse str0m's ICE implementation. This is now in the `is` crate. -#[doc(hidden)] pub mod ice { pub use crate::vendor::is::IceCreds; pub use crate::vendor::is::stun::{StunMessage, StunMessageBuilder, StunPacket, TransId}; @@ -2116,7 +2115,6 @@ impl fmt::Debug for Rtc { } #[cfg(test)] -#[doc(hidden)] pub fn init_crypto_default() { crate::vendor::str0m::crypto::from_feature_flags().install_process_default(); } diff --git a/rs/moq-webrtc/src/vendor/str0m/rtp/ext.rs b/rs/moq-webrtc/src/vendor/str0m/rtp/ext.rs index df5520a9002..95cbee6d782 100644 --- a/rs/moq-webrtc/src/vendor/str0m/rtp/ext.rs +++ b/rs/moq-webrtc/src/vendor/str0m/rtp/ext.rs @@ -65,7 +65,6 @@ pub enum Extension { ColorSpace, /// Not recognized URI, but it could still be user parseable. - #[doc(hidden)] UnknownUri(String, Arc), } @@ -866,30 +865,18 @@ pub struct ExtensionValues { // The values below are considered internal until we have a reason to expose them. // Generally we want to avoid expose experimental features unless there are strong // reasons to do so. - #[doc(hidden)] pub video_content_type: Option, // 0 = unspecified, 1 = screenshare - #[doc(hidden)] pub tx_time_offs: Option, - #[doc(hidden)] pub abs_send_time: Option, - #[doc(hidden)] pub abs_capture_time: Option, - #[doc(hidden)] pub transport_cc: Option, // (buf[0] << 8) | buf[1]; - #[doc(hidden)] // https://webrtc.googlesource.com/src/+/refs/heads/master/docs/native-code/rtp-hdrext/playout-delay pub play_delay_min: Option, - #[doc(hidden)] pub play_delay_max: Option, - #[doc(hidden)] pub video_timing: Option, - #[doc(hidden)] pub rid: Option, - #[doc(hidden)] pub rid_repair: Option, - #[doc(hidden)] pub mid: Option, - #[doc(hidden)] pub frame_mark: Option, /// User values for [`ExtensionSerializer`] to parse into and write from. diff --git a/rs/moq-webrtc/src/vendor/str0m/sdp/data.rs b/rs/moq-webrtc/src/vendor/str0m/sdp/data.rs index 0013e11b1ef..c19eb1c40d5 100644 --- a/rs/moq-webrtc/src/vendor/str0m/sdp/data.rs +++ b/rs/moq-webrtc/src/vendor/str0m/sdp/data.rs @@ -778,7 +778,6 @@ pub enum MediaType { Application, // If a parsed SDP has a value we don't recognize, we stick it // in here. We could consider making that a parse exception instead. - #[doc(hidden)] Unknown(String), } diff --git a/rs/qmux/src/config.rs b/rs/qmux/src/config.rs index 43c3ebf5e4f..568721dc07a 100644 --- a/rs/qmux/src/config.rs +++ b/rs/qmux/src/config.rs @@ -1,6 +1,6 @@ use std::time::Duration; -use moq_quic::{Limits, VarInt, mux}; +use moq_quic::{Limits, VarInt, connection::mux}; use crate::Version; diff --git a/rs/qmux/src/lib.rs b/rs/qmux/src/lib.rs index 935455690a9..bf282f169fc 100644 --- a/rs/qmux/src/lib.rs +++ b/rs/qmux/src/lib.rs @@ -65,7 +65,7 @@ pub use driver::Driver; pub use error::Error; #[cfg(feature = "ws")] pub use error::WebSocketError; -pub use moq_quic::mux::Version; +pub use moq_quic::connection::mux::Version; pub use moq_quic::{Limits, LimitsError}; pub use session::{RecvStream, SendGroup, SendStream, Session}; #[cfg(any(target_os = "linux", target_os = "macos"))] diff --git a/rs/qmux/src/session.rs b/rs/qmux/src/session.rs index 3419b3b824c..9c706c89de3 100644 --- a/rs/qmux/src/session.rs +++ b/rs/qmux/src/session.rs @@ -13,7 +13,11 @@ use std::time::Duration; use bytes::{Buf, Bytes}; use moq_net::transport::{self as net, Address, poll}; use moq_quic::stream::{Dir, Id as StreamId}; -use moq_quic::{Side, VarInt, connection::Error as ConnectionError, mux, recv, send}; +use moq_quic::{ + Side, VarInt, + connection::{Error as ConnectionError, mux}, + recv, send, +}; use crate::transport::Transport; use crate::{Config, Driver, Error, SharedSocketStats}; diff --git a/sh/text.sh b/sh/text.sh index 65813a3ce71..a16ae6d0fa2 100755 --- a/sh/text.sh +++ b/sh/text.sh @@ -4,22 +4,37 @@ set -euo pipefail cd "$(git rev-parse --show-toplevel)" +failed=0 + +# Fail with the message when git grep matches. Matches stay on stdout. Stderr +# is captured because Git reports a file it cannot read there, yet still exits +# 1 when nothing else matches. +forbid() { + local message=$1 errors status=0 + shift + { errors="$(git --no-pager grep --no-textconv -n -I "$@" 2>&1 >&3 3>&-)" || status=$?; } 3>&1 + if [[ -n "$errors" ]]; then + printf '%s\n' "$errors" >&2 + echo "error: git grep could not scan every tracked file." >&2 + exit $((status > 1 ? status : 2)) + fi + case "$status" in + 0) + echo "error: $message" >&2 + failed=1 + ;; + 1) ;; + *) exit "$status" ;; + esac +} + # Git decides what is binary. Lockfiles are generated dependency metadata. # Spell the UTF-8 bytes as escapes so the match works in every locale. -# Matches stay on stdout. Stderr is captured because Git reports a file it -# cannot read there, yet still exits 1 when nothing else matches. -status=0 -{ errors="$(git --no-pager grep --no-textconv -n -I -F $'\xe2\x80\x94' -- . ':!*.lock' 2>&1 >&3 3>&-)" || status=$?; } 3>&1 -if [[ -n "$errors" ]]; then - printf '%s\n' "$errors" >&2 - echo "error: git grep could not scan every tracked file." >&2 - exit $((status > 1 ? status : 2)) -fi -case "$status" in - 0) - echo "error: em dashes are forbidden; replace them with plain punctuation (UI placeholders use '-')." >&2 - exit 1 - ;; - 1) exit 0 ;; - *) exit "$status" ;; -esac +forbid "em dashes are forbidden; replace them with plain punctuation (UI placeholders use '-')." \ + -F $'\xe2\x80\x94' -- . ':!*.lock' + +# A public Rust item is documented or not public. +forbid "#[doc(hidden)] is forbidden; make the item private, delete it, or document it." \ + -E 'doc[[:space:]]*\([^)]*hidden' -- '*.rs' + +exit "$failed" diff --git a/sh/text.test.py b/sh/text.test.py index 4df5870d47a..f292bd3dfb1 100644 --- a/sh/text.test.py +++ b/sh/text.test.py @@ -83,6 +83,27 @@ def test_hidden_generated_and_extensionless_text_are_checked(self): self.assertIn(f"{name}:1:", result.stdout) self.git("rm", "-f", name) + def test_doc_hidden_fails_only_in_rust(self): + self.write("notes.md", "#[doc(hidden)]\n") + self.git("add", ".") + self.assertEqual(self.check().returncode, 0) + for text in ("#[doc(hidden)]", "#[cfg_attr(test, doc( hidden ))]", '#[doc(alias = "Old", hidden)]'): + with self.subTest(text=text): + self.write("src/vendor/lib.rs", f"pub mod a;\n{text}\n") + self.git("add", ".") + result = self.check() + self.assertEqual(result.returncode, 1, result.stderr) + self.assertIn("src/vendor/lib.rs:2:", result.stdout) + self.assertIn("doc(hidden)] is forbidden", result.stderr) + + def test_every_violation_is_reported(self): + self.write("a.rs", f"// {EM_DASH}\n#[doc(hidden)]\n") + self.git("add", ".") + result = self.check() + self.assertEqual(result.returncode, 1, result.stderr) + self.assertIn("em dashes are forbidden", result.stderr) + self.assertIn("doc(hidden)] is forbidden", result.stderr) + def test_git_errors_fail(self): self.git("config", "grep.patternType", "invalid") result = self.check()