Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .claude/skills/verify-reconnect/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: Model-check the client connect policy (`rs/moq-net/src/reconnect.rs

## Rules

Each rule is an INVARIANT; the harm says why it matters. All but `NoOrphan` and `RedirectFollowed` are checked only while `run` waits with the queue drained, which is when another task can read the handle.
Each rule is an INVARIANT; the harm says why it matters. All but `NoOrphan`, `RedirectFollowed`, and `FirstWalkFailsFast` are checked only while `run` waits with the queue drained, which is when another task can read the handle.

| Rule | Holds when | Harm if broken |
| --- | --- | --- |
Expand All @@ -18,6 +18,7 @@ Each rule is an INVARIANT; the harm says why it matters. All but `NoOrphan` and
| `OfflineMeansNothingServes` | Disconnected is reported only when no predecessor still serves (`Action::Offline`: "nothing serves anymore"). | An app waiting on `established` stalls, and the UI says disconnected, while a session still carries its media. |
| `NoOrphan` | Until the policy is done, every open session is the live one or the predecessor. | A transport and the peer's state stay open with nothing able to reach them. |
| `PresenceBalanced` | `sessions_started - sessions_ended` is 1 while connected and 0 otherwise. | Presence counts drift on every migration. |
| `FirstWalkFailsFast` | Nothing backs off before a session has been established: a failed first walk ends the policy (a refusal of since-replaced tokens is the exception, and tokens are not modeled). | A setup mistake (an untrusted certificate, no version in common) retries forever instead of surfacing. |
| `RedirectFollowed` | A dial never goes elsewhere than a redirect, named to a session the policy still holds, that the policy never acted on. | The replacement lands back on the node that is leaving. |

## Run
Expand Down Expand Up @@ -54,6 +55,7 @@ Sync commit: `103052ed074a87f00269ecb0d639905482ecfb6f`. Run `git log 103052ed07
| Code | Model |
| --- | --- |
| `reconnect.rs` `Reconnect::poll` (queue, predecessor, phases) | `Pop`, `Retire`, `IdleClose`, `Dial`, `Abandon`, `BackoffEnds`, `LiveEnds`, `ProbeAsk`, `ProbeAbandon` |
| `reconnect.rs` `established`, and `walk_failed` ending the first walk | `Established`, `Failed`, `Abandon` |
| `reconnect.rs` `connected`, `fallback`, `failed`, `probed` | `Connected`, `Failed`, `ProbeFails`, `ProbeGetsThrough` |
| `reconnect.rs` `end`, `closed`, `done`, `Predecessor` | `EndGoaway`, `EndClosed`, `Done`, `Retire`, `IdleClose` |
| `reconnect.rs` `Restart::Wait` | `LiveEnds` with `Restart = "wait"` (`RestartWait.cfg`) |
Expand Down
18 changes: 6 additions & 12 deletions c/moq/test/probe.c
Original file line number Diff line number Diff line change
Expand Up @@ -549,9 +549,8 @@ int main(void)
moq_route_free(route);
moq_origin_producer_free(origin);

/* A first dial that keeps failing still returns the session, whose state names each failed
* attempt while it retries, then ends closed once cancelled; the server is gone, so every
* dial fails. */
/* A first dial that fails still returns the session, whose state ends closed with the
* dial's error rather than retrying; the server is gone, so the dial fails. */
moq_client_config retry_config = moq_client_config_default();
retry_config.timeout.has_value = true;
retry_config.timeout.value = 100000000; /* 100ms */
Expand All @@ -561,20 +560,15 @@ int main(void)
ok(moq_client_new(&retry_config, &retrying), "client new");
result dialing = {0};
task = moq_client_connect(retrying, url, on_session, &dialing);
moq_session *redialing = await(&dialing, "retrying connect");
moq_session *redialing = await(&dialing, "failing connect");
moq_task_free(task);
moq_connection_state *state = moq_session_state(redialing);
CHECK(state->phase.tag == MOQ_CONNECTION_PHASE_CONNECTING);
while (!state->phase.connecting.failure) {
state = next_state(redialing, state);
CHECK(state->phase.tag == MOQ_CONNECTION_PHASE_CONNECTING);
}
CHECK(state->phase.connecting.failure->tag == MOQ_ERROR_CONNECT);
CHECK(state->connects == 0);
moq_session_cancel(redialing, 0);
while (state->phase.tag != MOQ_CONNECTION_PHASE_CLOSED) {
CHECK(state->phase.tag == MOQ_CONNECTION_PHASE_CONNECTING);
state = next_state(redialing, state);
}
CHECK(state->phase.closed.failure);
CHECK(state->phase.closed.failure->tag == MOQ_ERROR_CONNECT);
CHECK(state->connects == 0);
moq_connection_state_free(state);
moq_session_free(redialing);
Expand Down
3 changes: 2 additions & 1 deletion cpp/obs/src/moq-dock.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1212,7 +1212,8 @@ void MoQDock::UpdateStatus()
const bool everConnected = moq && (moq->IsLiveSession() || obs_output_get_connect_time_ms(output) > 0);
if (!everConnected) {
if (!failText.isEmpty()) {
// The first dial is still retrying; one that gave up stops the output instead.
// A first dial that reaches no session stops the output, so this shows only
// until that stop lands.
status->setText(QString("● Connecting… · %1").arg(failText));
status->setStyleSheet("color: #d08b1d;");
statsBox->setText(failText);
Expand Down
3 changes: 2 additions & 1 deletion cpp/obs/src/moq-settings.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,8 @@ const std::vector<Field> &Fields()
600000, 1000, "Ceiling on the growing reconnect delay."));
f.push_back(Number(BACKOFF_TIMEOUT, "Give up after (ms)", Millis(d.backoff.timeout, BACKOFF_TIMEOUT), 0,
3600000, 1000,
"Total time to keep retrying before the stream fails. 0 retries forever."));
"Total time to keep reconnecting after a drop before the stream fails. 0 retries "
"forever. A first connect that reaches no server fails the stream at once."));

// QUIC transport.
f.push_back(Choose(
Expand Down
29 changes: 15 additions & 14 deletions cpp/obs/test/moq-output-test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1008,8 +1008,9 @@ int main()
}
printf("names the failure while reconnecting: ok\n");

// While the first dial keeps failing, the dock names why instead of a bare
// "Connecting…". The relay is gone before the start, so nothing answers.
// A first dial that reaches no session fails the start for good rather than
// retrying: nothing has ever connected, so the cause is likely setup. The relay
// is gone before the start, so nothing answers.
{
std::string url;
{
Expand All @@ -1019,21 +1020,21 @@ int main()
reset(url);
MoQOutput o(nullptr, OUTPUT);
CHECK(o.Start());
std::string reason;
CHECK(WaitFor([&] {
int code = 0;
o.CopyLastFailure(&code, &reason);
return !reason.empty();
}));
// Still retrying rather than failed, so this is the first-dial path.
CHECK(!o.IsLiveSession());
CHECK(signalCount() == 0);
o.Stop();
CHECK(WaitFor([] { return signalCount() == 1; }));
// Never reached the server, so OBS must not retry the endpoint.
CHECK(signalAt(0).code == OBS_OUTPUT_CONNECT_FAILED);
CHECK(signalAt(0).begin_capture_at == 1);
CHECK(!signalAt(0).last_error.empty());
int code = 0;
std::string reason;
o.CopyLastFailure(&code, &reason);
CHECK(reason.empty());
CHECK(reason == signalAt(0).last_error);
CHECK(!o.IsLiveSession());
// The failure retired the attempt, so nothing it queued can report again.
MoQOutputStatsTest::Drain(o);
CHECK(signalCount() == 1);
}
printf("names the failure while the first dial retries: ok\n");
printf("a failing first dial fails the start: ok\n");

// Stopping while the connect is still pending cancels it: nothing but the stop
// reaches OBS, then or later.
Expand Down
2 changes: 2 additions & 0 deletions dart/moq/lib/src/aliases.dart
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,8 @@ typedef ConnectionPhaseClosed = ClosedMoqConnectionPhase;
typedef ConnectionState = MoqConnectionState;

/// Retry pacing for the automatic reconnect: initial delay, multiplier, ceiling, and give-up window.
/// It applies only once a session has been established: a first dial that
/// reaches no session ends the connection with its error rather than retrying.
typedef Backoff = MoqBackoff;

/// Whether a protocol code is from the session or stream registry.
Expand Down
5 changes: 3 additions & 2 deletions dart/moq/lib/src/client.dart
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,9 @@ final class ConnectOptions {
/// The WebSocket fallback raced against QUIC for `http(s)` URLs.
final WebSocketConfig? websocket;

/// Set true for a one-shot dial. By default the session redials with
/// backoff whenever the transport drops.
/// Set true for a one-shot dial. By default an established session redials
/// with backoff whenever the transport drops; a first dial that reaches no
/// session ends it with that error instead.
final bool once;

/// Retry pacing for the automatic reconnect.
Expand Down
4 changes: 1 addition & 3 deletions dart/moq_ffi/lib/src/moq.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ import "dart:ffi";
import "dart:io" show Platform, File, Directory;
import "dart:isolate";
import "dart:typed_data";

import "package:ffi/ffi.dart";

import "uniffi_runtime.dart";
export "uniffi_runtime.dart";

Expand Down Expand Up @@ -25662,7 +25660,7 @@ void _checkApiChecksums() {
if (uniffi_moq_ffi_checksum_method_moqclient_cancel() != 29949) {
throw UniffiInternalError.panicked("UniFFI API checksum mismatch");
}
if (uniffi_moq_ffi_checksum_method_moqclient_connect() != 11393) {
if (uniffi_moq_ffi_checksum_method_moqclient_connect() != 30086) {
throw UniffiInternalError.panicked("UniFFI API checksum mismatch");
}
if (uniffi_moq_ffi_checksum_method_moqsession_allocator() != 12118) {
Expand Down
2 changes: 0 additions & 2 deletions dart/moq_ffi/lib/src/uniffi_runtime.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import "dart:ffi";
import "dart:io" show Platform, File, Directory;
import "dart:isolate";
import "dart:typed_data";

import "package:ffi/ffi.dart";

class UniffiInternalError implements Exception {
Expand Down Expand Up @@ -253,7 +252,6 @@ mixin FfiConverterPrimitive<T> on FfiConverter<T, T> {
@override
T lower(T value) => value;
}

Uint8List createUint8ListFromInt(int value) {
int length = value.bitLength ~/ 8 + 1;
if (length != 4 && length != 8) {
Expand Down
8 changes: 5 additions & 3 deletions doc/bin/gstreamer.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,9 @@ codec config, a pad requested again, or a stream resumed after EOS publishes as
`url`, `broadcast`, `linger`, `connect-version`, `tls-disable-verify`, `quic-idle-timeout`, `quic-keylog`,
`quic-keep-alive`, and read-only `status`, `connected`, `moq-version`, and
`estimated-send-rate`, `estimated-recv-rate`, `connection-stats`, and
`sessions`. The sink reconnects for as long as the pipeline runs and only
reports `failed` on an answer redialing can't change, such as a rejected token.
`sessions`. Once connected, the sink reconnects for as long as the pipeline runs and only
reports `failed` on an answer redialing can't change, such as a rejected token. A first
connect that reaches no relay reports `failed` and posts an error instead of retrying.
Both elements accept the same connection properties. `connect-version` is a
comma-separated list, matching the CLI's repeatable `--connect-version`;
unknown versions fail startup. QUIC durations are milliseconds. Zero disables the local idle timeout; a zero
Expand Down Expand Up @@ -168,7 +169,8 @@ A rendition that leaves the catalog
gets EOS once its track ends. A broadcast that ends or loses its publisher holds
its pads without EOS until it is announced again, so the pipeline does not end
on its own. Losing the relay connection also holds the pads: `moqsrc` redials
without a retry deadline and resumes when the broadcast returns. A refused
without a retry deadline and resumes when the broadcast returns. A first connect
that reaches no relay posts an error rather than retrying. A refused
connection, refused path or catalog, or malformed protocol input or catalog posts
an error. Properties: `url`,
`broadcast`, `max-delay`, `connect-version` (comma-separated protocol versions to offer, most
Expand Down
2 changes: 1 addition & 1 deletion doc/lib/cpp/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@ Set `ClientConfig::tokens` instead of putting the credential in the URL. The fir

## Stats

`session->stats()` reports the transport counters, each a `std::optional` that is empty when the transport does not report it, plus the negotiated protocol in `version`. Connect returns a connecting session. Read `session->state()` and wait with `state_changed(state.revision)` to observe its first-dial failures and reconnects; independent watchers keep their own revisions.
`session->stats()` reports the transport counters, each a `std::optional` that is empty when the transport does not report it, plus the negotiated protocol in `version`. Connect returns a connecting session. Read `session->state()` and wait with `state_changed(state.revision)` to observe its reconnects (a first dial that reaches no session ends the session with that error rather than retrying); independent watchers keep their own revisions.

## Shutdown

Expand Down
2 changes: 1 addition & 1 deletion doc/lib/dart/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ shows which module holds each.
- **Decoded frames hold decoder buffers.** `dispose()` each frame from a `video.Consumer` promptly rather than waiting for the garbage collector, or the decoder stalls. `resize` is best effort, so read each frame's `width()` and `height()`.
- **Closing.** `await moq.close()` drains for up to one second and reports its result. `session.shutdown()` requests the drain synchronously; `await session.closed()` waits for its result. `session.cancel(code: 0)` closes at once. Finish or abort live tracks first. Awaiting cancellation of an announcement, grant, or request subscription releases its pending native read. Announcements and grants also release their consumer; server listeners leave their shared server usable. moq-lite-05 and newer wait for the peer to receive finished tails; older Lite versions and IETF wait only for transport ACKs.
- **Ending cleanly.** When the input ends, finish every track, then call `finish()` on its `CatalogProducer` to finish the catalog, which ends the content. The broadcast stays announced and keeps serving the finished tracks, so a subscriber still catching up can drain them, until `broadcast.close()` or disposing its `MoqRoute` takes it offline. How long to keep it up is yours to decide. A track still open at the close, or released without a finish, ends with `Unroutable` (`Internal` over moq-transport, which has no such code). Readers follow a publisher's runs by announcement, not by that error: see [identity and recovery](/concept/identity).
- **Stats.** `session.stats()` reports `rtt`, `estimatedSendRateBps`, and the byte and packet counters (`bytesSent`, `bytesReceived`, `bytesLost`, `packetsSent`, `packetsReceived`, `packetsLost`), and `datagramsUnrouted`, the datagrams received for a subscription with no datagram track, reported only on moq-lite 05 and later, which carry datagrams. A field is `null` when the transport does not report it, which is not the same as zero. `version` names the negotiated protocol. Connect returns a connecting session; read its coherent `state()` snapshot and wait with that snapshot's revision to observe first-dial failures and reconnects. Each watcher keeps its own revision. Hold the owned `BandwidthConsumer` returned by `session.recvBandwidth()` to receive bandwidth estimates. `rate()` reads the current estimate in bits per second and `rateChanged()` waits for a new one. Release the consumer or call `cancel()` when finished; releasing the last consumer stops PROBE; the session and stats snapshots do not request it. Client consumers survive reconnects and report no estimate while disconnected.
- **Stats.** `session.stats()` reports `rtt`, `estimatedSendRateBps`, and the byte and packet counters (`bytesSent`, `bytesReceived`, `bytesLost`, `packetsSent`, `packetsReceived`, `packetsLost`), and `datagramsUnrouted`, the datagrams received for a subscription with no datagram track, reported only on moq-lite 05 and later, which carry datagrams. A field is `null` when the transport does not report it, which is not the same as zero. `version` names the negotiated protocol. Connect returns a connecting session; read its coherent `state()` snapshot and wait with that snapshot's revision to observe reconnects. A first dial that reaches no session ends the session with that error rather than retrying. Each watcher keeps its own revision. Hold the owned `BandwidthConsumer` returned by `session.recvBandwidth()` to receive bandwidth estimates. `rate()` reads the current estimate in bits per second and `rateChanged()` waits for a new one. Release the consumer or call `cancel()` when finished; releasing the last consumer stops PROBE; the session and stats snapshots do not request it. Client consumers survive reconnects and report no estimate while disconnected.
- **Viewers.** `client.viewers()` divides the connection's receive estimate among the broadcasts you watch, max-min fair by each one's ceiling. `watch(maxBps: ...)` returns a `Viewer` claiming up to what the largest rendition the player would switch to needs; `rate()` reads its slice in bits per second, `null` while there is no estimate, so hold the current selection. `update(maxBps: ...)` changes the ceiling, such as when the catalog enables or disables a rendition. Holding a viewer requests PROBE, and the session sends the viewers' summed ceilings as the PROBE target; `dispose()` it once the broadcast stops being watched. `setTarget(targetBps: ...)` sends another target, `0` sends none, and `null` follows the viewers again. Ceilings or a target past 2^53 - 1 bps, the most a PROBE target carries, throw `OutOfRangeMoqException`. Client viewers survive reconnects and read `null` while disconnected.
- **Authorization.** On `moq-lite-07-wip` (opt in with `ConnectOptions(versions: ...)`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), listen to `moq.grants().updates()` for the `Grant` the relay sent back and each change, `null` until it answers and while a client session reconnects.
- **Tokens.** Pass `ConnectOptions(tokens: [...])` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `moq.tokens().set_(tokens: [...])` replaces them before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. `await tokens().answered()` returns once the relay accepts them and throws its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release.
Expand Down
Loading
Loading