diff --git a/quest/m1/README.md b/quest/m1/README.md index 589159cfc9c..5cde7ab6459 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -197,7 +197,6 @@ folded into Dropped sources. - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [A/V sync across a break](/quest/m1/watch-break-av-sync.md) - `@moq/watch` never plays pre-break audio out of sync with pre-break video - [moqsrc reconnect](/quest/m1/moqsrc-reconnect.md) - `moqsrc` redials after losing its relay and resumes on the same pads -- [moq fetch closes cleanly](/quest/m1/moq-fetch-close.md) - `moq fetch` closes its session before exiting instead of leaving the relay to time it out - [Plan: HLS export failure signal](/quest/m1/hls-failure-signal.md) - decide what players see when the exporter's timeline fails, live or history, so a truncated recording never looks complete - [Publisher linger](/quest/m1/publisher-linger.md) - every finishing publisher keeps its broadcast up for the media max-age so subscribers can drain it diff --git a/quest/m1/moq-fetch-close.md b/quest/m1/moq-fetch-close.md deleted file mode 100644 index 2c8216e4588..00000000000 --- a/quest/m1/moq-fetch-close.md +++ /dev/null @@ -1,26 +0,0 @@ -# [XS] moq fetch closes its connection cleanly - -## Goal - -`moq fetch` (`rs/moq-cli/src/fetch.rs`) closes its session before exiting, -so the relay sees a clean close instead of timing the connection out. - -## Plan - -Found while merging #5174 (2026-10-10), which taught every interop client to -close cleanly and made the harness fail a round on an idled-out connection. -`moq fetch` still drops its connection without `Connection::close` or -`Client::close`; the js-native FETCH_DATA wire-compat cell reproduced the -relay timing it out. Only the wire-compat lanes reach it, and they skip the -idle-out check, so it doesn't fail CI today. - -Close the session the way #5174 did for the other clients, on every exit -after connecting, including an error or the `timeout_at` deadline. For the -regression, run a FETCH round with the current relay and the current -`moq fetch` under the idle-out check. No such round exists today (FETCH only -runs in the wire-compat lanes), so add one and wire it into CI. Don't enable -the check for the wire-compat cells: they mix in released relays and clients, -which it excludes because they may not close cleanly or log which connection -closed. - -Public API: none. Wire: none. diff --git a/rs/moq-cli/src/fetch.rs b/rs/moq-cli/src/fetch.rs index 3b0c15b7632..9b57a7dfa76 100644 --- a/rs/moq-cli/src/fetch.rs +++ b/rs/moq-cli/src/fetch.rs @@ -74,19 +74,26 @@ async fn fetch( .with_subscriber(origin.clone()) .with_reconnect(false); - fetch_from( + let connection = client.connect(url); + let result = fetch_from( Request { args, path: &broadcast, deadline, }, async { - let connection = client.connect(url).established().await.context("failed to connect")?; + let connection = connection.clone().established().await.context("failed to connect")?; anyhow::Ok((origin, connection)) }, out, ) - .await + .await; + // Keep an independent handle outside the timed operation: cancellation of setup or + // a read must still send the session close before the runtime can stop. + let closed = connection.close().await; + client.close().await; + result?; + closed.context("failed to close fetch connection") } /// The operation's selection and one absolute deadline, including setup. @@ -104,7 +111,7 @@ async fn fetch_from( ) -> anyhow::Result<()> { let Request { args, path, deadline } = request; let result = timeout_at(deadline, async { - // Held until the last frame is read; dropping it closes the session. + // Held until the last frame is read; the caller closes the connection afterward. let (origin, _connection) = setup.await?; // Wait for a covering route rather than asking on the spot: the announcement diff --git a/test/interop/README.md b/test/interop/README.md index 310357f7a18..1dcbf80ce51 100644 --- a/test/interop/README.md +++ b/test/interop/README.md @@ -23,7 +23,7 @@ The normal run also checks a finite raw track through the relay. Its publisher announces end 4 before writing four 256 KiB groups; the reader verifies every byte, exactly groups 0 through 3, and a clean end at 4. Rust-to-Rust always runs. Selecting native JS adds both Rust-to-JS and JS-to-Rust under that runtime, so `--all` covers -Node and Bun. `--tail` runs just these five lanes. The wire-compat lanes, which swap in released relays and clients, skip them. +Node and Bun. The current CLI also FETCHes group 0, verifies every byte, and closes under the relay idle-out check. `--tail` runs this FETCH round and the five finite-tail lanes. The wire-compat lanes, which swap in released relays and clients, skip them. The finite clients keep their session alive until the harness acknowledges the reader's complete clean end over stdin, so a lane tests delivery rather than @@ -141,7 +141,7 @@ just test interop --all # Pick your own axes: just test interop --publishers rust,python --subscribers rust,c,js-native-bun -# Finite track tails only: Rust-to-Rust and Rust/Node/Bun in both directions. +# CLI FETCH close and finite track tails only: Rust-to-Rust and Rust/Node/Bun in both directions. just test interop --tail # Subscription termination: Rust/JS response bytes over in-memory transports. diff --git a/test/interop/clients/rust/tail.rs b/test/interop/clients/rust/tail.rs index 4f7f9985837..cebf55d2604 100644 --- a/test/interop/clients/rust/tail.rs +++ b/test/interop/clients/rust/tail.rs @@ -26,7 +26,12 @@ async fn main() -> anyhow::Result<()> { moq_net::track::Info::default().with_timescale(moq_net::Timescale::MILLI), )?; broadcast.announce(Default::default())?; - let _session = client.with_publisher(&origin).connect(url).established().await?; + let session = client + .clone() + .with_publisher(&origin) + .connect(url) + .established() + .await?; track.demand().used().await?; // Declare the end before the final group's payload is even available. track.finish_at(moq_net::track::Position::group(GROUPS))?; @@ -43,11 +48,17 @@ async fn main() -> anyhow::Result<()> { .await?; ensure!(ack == "clean end\n", "reader did not acknowledge a clean end"); broadcast.close(); + session.close().await?; eprintln!("tail acknowledged"); } "subscribe" => { let consumer = origin.consume(); - let _session = client.with_subscriber(origin).connect(url).established().await?; + let session = client + .clone() + .with_subscriber(origin) + .connect(url) + .established() + .await?; let broadcast = consumer.routed_broadcast(&args[3]).await?; let options = moq_net::track::Subscription::default() .with_max_delay(Duration::from_secs(5)) @@ -72,9 +83,12 @@ async fn main() -> anyhow::Result<()> { track.finished().await? == moq_net::track::Position::group(GROUPS), "wrong declared end" ); + session.close().await?; eprintln!("tail clean end=4 groups=0,1,2,3 bytes=1048576"); } _ => anyhow::bail!("unknown role: {}", args[1]), } + // Dropping only queues the QUIC close; send it before the runtime stops. + client.close().await; Ok(()) } diff --git a/test/interop/interop.sh b/test/interop/interop.sh index 43de7f0b775..4a5da418f4f 100755 --- a/test/interop/interop.sh +++ b/test/interop/interop.sh @@ -1003,6 +1003,40 @@ run_tail_pair() { rm "$fifo" } +# Exercise this checkout's CLI FETCH under the same relay close check as live readers. +run_fetch_round() { + local name="fetch-rust" fifo="$HARNESS_RUN/fetch-ack" broadcast="fetch-$$-$RANDOM" pid from + from=$(relay_next_line) + mkfifo "$fifo" + exec 9<>"$fifo" + harness_spawn "$name-pub" "$HARNESS_RUN/$name-pub.log" run_tail_publisher "$fifo" rust publish "$broadcast" "$(token_url --publish "$broadcast")" + pid="$HARNESS_PID" + if timeout -k 3 "$TIMEOUT" "$MOQ" --connect "$(token_url --subscribe "**/$broadcast")" \ + --broadcast "$broadcast" fetch tail --group 0 >"$HARNESS_RUN/$name.raw" && + python3 - "$HARNESS_RUN/$name.raw" <<'PYCODE' +import pathlib, sys +assert pathlib.Path(sys.argv[1]).read_bytes() == bytes(256 * 1024), "incorrect FETCH payload" +PYCODE + then + printf 'clean end\n' >&9 + harness_wait "$pid" || true + if grep -qx 'tail acknowledged' "$HARNESS_RUN/$name-pub.log"; then + echo " PASS CLI FETCH (group 0; 262144 bytes)" + else + echo " FAIL CLI FETCH (publisher did not acknowledge)" + overall=1 + cat "$HARNESS_RUN/$name-pub.log" + fi + else + echo " FAIL CLI FETCH" + overall=1 + harness_reap "$pid" + fi + exec 9>&- + rm "$fifo" + check_idle_outs "$name" "$from" +} + # One media.ts invocation. It reports its own verdict (a negative control passes by failing on the # assertion it names), so the exit code is the whole answer. run_media() { @@ -1131,6 +1165,8 @@ elif [[ "$TAIL_ONLY" -eq 0 ]]; then fi if [[ "$NEGATIVE" -eq 0 && "$MEDIA" -eq 0 && "$IN_TREE" -eq 1 ]]; then + echo "=== CLI FETCH close ===" + run_fetch_round echo "=== finite track tails ===" run_tail_pair rust rust for runtime in js-native-node js-native-bun; do diff --git a/test/interop/relay-log.sh b/test/interop/relay-log.sh index bc96ec5ea48..09b4eed71db 100644 --- a/test/interop/relay-log.sh +++ b/test/interop/relay-log.sh @@ -6,24 +6,23 @@ relay_log_since() { tail -n +"$1" "$HARNESS_RUN/relay.log" | sed 's/\x1b\[[0-9;]*m//g' } -# Print each relay connection that appears from log line FROM on, with how it closed: the +# Print each relay connection opened from log line FROM on, with how it closed: the # relay's close error, "ended" for a close without one, or "open" while it hasn't closed yet. # The patterns follow the relay's `conn{id=...}` span and its `connection closed` log -# (`rs/moq-relay/src/relay.rs`). Only connections opened in the window count, so a straggler -# from an earlier round closing here isn't blamed on this one. +# (`rs/moq-relay/src/relay.rs`). The relay numbers connections in accept order, so an id at +# or below the highest one logged before FROM was opened earlier: a straggler from an earlier +# round that logs or idles out here isn't blamed on this one. relay_connections() { - relay_log_since "$1" | awk ' - match($0, /conn\{id=[0-9]+ /) { - id = substr($0, RSTART + 8, RLENGTH - 9) - if (!(id in state)) { state[id] = "open"; order[++n] = id } - } - match($0, /connection closed id=[0-9]+/) { - id = substr($0, RSTART + 21, RLENGTH - 21) - if (id in state) { - rest = substr($0, RSTART + RLENGTH) - state[id] = sub(/^ err=/, "", rest) ? rest : "ended" - } - } + relay_log_since 1 | awk -v from="$1" ' + BEGIN { before = -1 } + { id = -1; closed = match($0, /connection closed id=[0-9]+/) } + closed { id = substr($0, RSTART + 21, RLENGTH - 21) + 0; rest = substr($0, RSTART + RLENGTH) } + !closed && match($0, /conn\{id=[0-9]+ /) { id = substr($0, RSTART + 8, RLENGTH - 9) + 0 } + id < 0 { next } + NR < from { if (id > before) before = id; next } + id <= before { next } + !(id in state) { state[id] = "open"; order[++n] = id } + closed { state[id] = sub(/^ err=/, "", rest) ? rest : "ended" } END { for (i = 1; i <= n; i++) print order[i], state[order[i]] } ' } @@ -49,9 +48,9 @@ check_idle_outs() { fi while read -r id reason; do [[ "$reason" == "open" || "$reason" == *"timed out"* ]] || continue - # The relay subscribes upstream only on the publisher's connection. + # The relay subscribes or fetches upstream only on the publisher's connection. role=subscriber - relay_log_since "$from" | grep "conn{id=$id .*::subscriber: subscribe started" >/dev/null && role=publisher + relay_log_since "$from" | grep -E "conn\{id=$id .*::subscriber: (subscribe|fetch) started" >/dev/null && role=publisher echo " FAIL $pub round: relay connection $id ($role) idled out instead of closing ($reason)" relay_log_since "$from" | grep -E "conn\{id=$id |connection closed id=$id " | sed 's/^/ /' failed=1 diff --git a/test/interop/relay-log.test.sh b/test/interop/relay-log.test.sh index 39254fe9052..1ddf13846c8 100755 --- a/test/interop/relay-log.test.sh +++ b/test/interop/relay-log.test.sh @@ -24,4 +24,8 @@ overall=0 check_idle_outs timeout 6 >"$HARNESS_RUN/result" [[ "$overall" == 1 ]] grep 'idled out instead of closing' "$HARNESS_RUN/result" + +# An earlier round's connection that logs or idles out in this window isn't counted. +printf '%s\n' 'WARN conn{id=1 remote=localhost}: refused' 'INFO conn{id=4 remote=localhost}: accepted' 'WARN connection closed id=3 err=timed out' 'INFO connection closed id=4' >>"$HARNESS_RUN/relay.log" +[[ "$(relay_connections 8)" == '4 ended' ]] echo 'relay log regression passed' diff --git a/test/justfile b/test/justfile index 7f54d2b8ca2..fdacef8cd45 100644 --- a/test/justfile +++ b/test/justfile @@ -34,7 +34,7 @@ harness: # browser + C++ publish; everyone subscribes (incl. native node/bun JS, the # moq-c C client, and the GStreamer moqsrc plugin). Other flags pass through, e.g. # `just test interop --publishers rust,python --subscribers rust,c`. -# Finite track tails run too; `--tail` selects just Rust/Node/Bun tail checks. +# Finite track tails run too; `--tail` selects just the CLI FETCH close and Rust/Node/Bun tail checks. # Both runs first check js/net's varints against moq-net's encoder at every size boundary, # and on lite-06/07 messages (the filter also matches `lite_varint_interop`), then # moq-transport OBJECT_DATAGRAMs both ways on every draft, and Lite AUTH_ERROR