Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions quest/m0/test-flakes-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,10 @@ Added 2026-11-08 (maintainer, /quest-complete): `js/net`'s widened max-age
re-drop test, which failed on [moq-dev/moq#8230](https://github.com/moq-dev/moq/pull/8230)
with the `Cancel` reset [moq-dev/moq#8195](https://github.com/moq-dev/moq/pull/8195) intends for a reply still pending at the stream's end; the test races its finish against that reply.

Added 2026-11-08 (maintainer's standing rule that flakes join this epic):
`bench/relay.test.py`'s stream-transport test, which binds UDP on the number the
OS picked for TCP and failed a local run while other checks held that UDP port.

Added 2026-11-08 (maintainer's standing rule that flakes join this epic): moq-ffi's
raw media warmup test, which failed in the merge queue for
[moq-dev/moq#8272](https://github.com/moq-dev/moq/pull/8272) when its reader skipped one of three
Expand All @@ -224,4 +228,5 @@ Public API: none. Wire: none.
- [js/net's lite-07 datagram queue test holds up under load](/quest/m0/test-flakes-2/datagram-queues.md) - the createWritable queue test waits on the writes it asserts, and its `test.skip` is removed
- [js/net's lite-06 session-close test keeps the end under load](/quest/m0/test-flakes-2/session-close-end.md) - a session close right after a datagram track's end returns the end, not a `remote error: 0`, and its lite-06 skip is removed
- [js/net's widened max-age re-drop test ends cleanly, not with a reset](/quest/m0/test-flakes-2/widened-max-age-reset.md) - the test finishes the track only after the re-drop reply arrives, so the end is a FIN, and drops its wall-clock assert
- [bench/relay.test.py's stream-transport test gets a port free for TCP and UDP](/quest/m0/test-flakes-2/relay-port-pair.md) - the test binds TCP and UDP to one number free in both, so parallel checks can't collide with it
- [moq-ffi's warmup test keeps every group under load](/quest/m0/test-flakes-2/ffi-warmup-wall-clock.md) - `raw_media_catalog_warmup_reaches_the_subscription` keeps the group between its warmup start and the live edge, and its `#[ignore]` is removed
50 changes: 50 additions & 0 deletions quest/m0/test-flakes-2/relay-port-pair.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# [XS] bench/relay.test.py's stream-transport test gets a port free for TCP and UDP

## Goal

`bench/relay.test.py`'s `PortOwnership.test_a_stream_transport_needs_its_own_listener`
passes while other checks run in parallel on the same host, never failing its
setup with `Address already in use`.

## Plan

Planned 2026-11-08 (maintainer's standing rule that flakes join this epic). It
failed once locally while other sessions ran checks in parallel. None of the 40
most recent failed merge-queue runs (2026-11-05 to 2026-11-08) failed it, so it
is not quarantined.

The test does not bind a fixed port: it binds its TCP listener to port 0
(line 278), then binds UDP to the same number (line 281,
`udp.bind(("127.0.0.1", port))`). TCP and UDP have separate port spaces, so
the OS's free TCP port says nothing about UDP. A parallel check's QUIC client
holding that UDP port fails line 281.

Decided (rejected options in parentheses):

- ✅ Find a number free in both protocols: bind UDP to port 0, then TCP to the
same number, and draw again while TCP refuses it. Keep the refused sockets
open until a pair binds, so each draw gets a new number and the search ends.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
This searches for a free resource rather than retrying a timing race: no OS
call reserves one number in both protocols. (rejected: a private network
namespace via `unshare`, which needs unprivileged user namespaces that
`sh/dispatch.sh` already probes for and some hosts deny; a fake `lsof`, which
stops testing the real `lsof` filters `relay_owns_port` uses; dropping the
same-number requirement, which is what the relay binds and the check proves.)
- The sibling tests need no change: `test_tcp_without_udp_is_not_owned_as_a_relay_port`
binds only TCP to port 0, and the relay startup tests go through
`start_relay`, which already draws again when a port is taken.

Reproduce first, by hand rather than as a committed test: from a helper process,
hold UDP sockets across all but a few ports of the ephemeral range, and watch
line 281 fail. Then show the fixed test passing under the same pressure. Next,
have the helper hold TCP listeners on all but a few ports instead, so TCP
refuses nearly every draw, and show the search redraws until a pair binds.
Each held socket costs a descriptor: the helper holds one per port, and the
test keeps one UDP socket per refused draw. Raise both processes' soft
`RLIMIT_NOFILE` to cover the range plus a few spare (the Linux default hard
limit, 524288, allows it). The helper is a separate process because
`relay_owns_port` checks the test's own PID, so a pressure listener the test
owned at the drawn port plus two would make the negative `tls` assertion fail.
Real collisions are rare, so the search itself needs no cap.

Public API: none. Wire: none.
1 change: 1 addition & 0 deletions quest/m3/webrtc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Decided 2026-10-30 with the maintainer (planning interview), after the str0m wor
- [WHIP/WHEP builds carry no SCTP](/quest/m3/webrtc/sctp-feature.md) - SCTP compiles only with the `p2p` feature
- [The vendored stack keeps only what browsers negotiate](/quest/m3/webrtc/browser-only.md) - PSK suites, `ccm`, and anything else no browser uses are gone
- [The vendored stack exposes only what moq-webrtc calls](/quest/m3/webrtc/dead-api.md) - dead upstream API is deleted once clock settles the time seams
- [A failed client socket ends a WebRTC session with an error](/quest/m3/webrtc/socket-error.md) - a client or `p2p` session whose socket fails ends with its I/O error, not `Ok` or `Closed`

## Related

Expand Down
89 changes: 89 additions & 0 deletions quest/m3/webrtc/socket-error.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# [M] A failed client socket ends a WebRTC session with an error

## Goal

When the UDP socket under a moq-webrtc client session fails to receive,
`Client::publish` and `Client::subscribe` end with `Error::Io` carrying the
socket's `io::Error`, instead of `Ok` as if the peer had disconnected. A
`p2p` connection that fails this way hands its opener `p2p::Error::Io` before
its channel opens, including an opener that asks after the failure, and fails
the MoQ session over an opened channel with `qmux::Error::Io`, instead of
`Closed`. A peer disconnect still ends `Ok`. The first cause wins: a peer
close that lands before the socket fails stays a close.

## Plan

Planned 2026-11-08 from [moq-dev/moq#8268](https://github.com/moq-dev/moq/pull/8268)'s
follow-ups, where CodeRabbit raised it and the author deferred it as
pre-existing.

Where it goes wrong today:

1. `ice::reader` (`rs/moq-webrtc/src/ice/wire.rs`) logs a `recv_from` error
and breaks, dropping its queue's sender. Client sessions (`session::bind_udp`)
and the `p2p` transport (`p2p::bind`) use it.
2. `Agent::next` (`rs/moq-webrtc/src/ice/agent.rs`) sees the queue end and
fails with `ice::Error::Closed`, the same variant as an ICE disconnect or
a dead `Rtc`.
3. `Session::run_loop` (`rs/moq-webrtc/src/session.rs`) maps `Closed` to
`Ok(())`. `p2p`'s `Error::ice` maps it to its own `Closed`.

The cause no caller can see is step 1: the reader drops the error. Fix it there.
The reader hands its `io::Error` to the agent, for example as the queue's last
item, and the agent fails with a new `ice::Error` variant named by role (such as
`Socket`). `Agent::next` returns a clone of the stored failure on every later
call, so `ice::Error` stays `Clone` and the variant holds an `Arc<io::Error>`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
The session maps it to the existing `moq_webrtc::Error::Io`, which is already
documented as covering a receive error on the media socket. `p2p`'s
`Error::ice` maps it to `p2p::Error::Io`, whose doc widens from bind and STUN
failures to cover it. Rebuild an owned error from the `Arc` that keeps what
callers read: `io::Error::from_raw_os_error` when it has an OS code, so
`raw_os_error()` still answers, else `io::Error::new(err.kind(), err)`.

Every handle on a `p2p` connection reads its end as `Closed` today, whatever the
cause: an opener that asks after the driver exited (the dropped command channel),
and an opened channel's `Reader` (the dropped `Inbox`) and `Writer` (the closed
outbox). Pushing the error into the `Inbox` alone would race: qmux's driver can
find the outbox closed before it reads the `Inbox`. Instead the driver records
its terminal error in one slot the handles share before it drops anything, and
each handle that finds its channel gone reports that error, as `p2p::Error` or
`qmux::Error::Io`, instead of `Closed`. That fixes every cause, not only the
socket.

`Agent::next` judges the establishment deadline before it reads the inbound
queue, so a socket error queued just before that deadline would surface as
`Timeout`: drain the queue, datagrams and any socket error behind them, before
judging the deadline, since the socket error is the cause.
Don't match on error kinds to decide which errors are fatal (AGENTS.md).

Boundary: the server's shared-socket `server::mux::demux` logs and continues on
every recv error, since one transient error, such as an ICMP unreachable,
must not end every session on the socket. It is out of scope. Do check whether
the client reader has the same transient case. On Windows an unconnected UDP
socket reports an ICMP port unreachable as `WSAECONNRESET` unless
`SIO_UDP_CONNRESET` is off. If tokio leaves it on, a client's ICE check to a
dead candidate ends the session today. Turn it off when the socket is bound,
as a known platform capability, rather than ignoring that error at runtime.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
Linux reports ICMP errors on an unconnected UDP socket only with `IP_RECVERR`,
which nothing here sets, so there a receive error is the socket failing.

Reproducing test, failing on main first. On Linux, a connected UDP socket
reports a loopback ICMP port unreachable as `ECONNREFUSED` on its next
receive, with no rate limit on loopback. Connect a socket to a closed loopback
port, send once, and feed the socket to `ice::reader` and an agent, as
`agent.rs`'s `unpolled` fixture does. Then assert that the agent and the
session end with the socket error, not `Closed` or `Ok`. Also assert that a
`p2p` opener gets `p2p::Error::Io` with the underlying error whether it asked
before or after the failure. With a channel opened between two peers and a
write pending, and the outbox's closure reaching qmux before the `Inbox`'s,
failing one side's socket ends its MoQ session with
`qmux::Error::Io` and the OS code. A socket error queued behind a datagram,
read after the establishment deadline, still wins over `Timeout`. Keep the
existing peer-disconnect path asserting `Ok`.

Public API: none. The errors map to existing `Io` variants, and the new
`ice::Error` variant is crate-private. Wire: none.

## Related

- [A typed DTLS handshake timeout error](/quest/m3/webrtc/dtls-timeout.md) - also adds an `ice::Error` variant mapped through the session loop and `p2p`
Loading