Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/bin/inspect.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ The node name (`local/host` here) is `stats.node`. Each fetch returns the
newest snapshot of cumulative counters. `totals.json` sums each tier's traffic
and sessions, `publisher.json` is egress per broadcast path,
`subscriber.json` is ingress, and `sessions.json` counts sessions per auth
root. Fetch twice and divide by the interval for a
root. A subscriber may also ask for `<path>/traffic.json`, one broadcast
prefix rolled up with each child, produced only while held. Fetch twice and divide by the interval for a
rate. The `.json.z` twins carry compressed patches, which these tools print as
raw bytes. [Stats](/concept/stats) describes every field.
41 changes: 37 additions & 4 deletions doc/concept/stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ comes in two encodings:
| `[<tier>/]subscriber.json` | broadcast path | [Traffic](#traffic) this node received (ingress) |
| `[<tier>/]sessions.json` | auth root | [Sessions](#sessions) connected now |
| `[<root>/]presence.json` | tier | [Sessions](#sessions) under one auth root |
| `[<path>/]traffic.json` | tier | [Prefix](#prefix) traffic under one broadcast prefix |

The default tier's per-tier tracks are unprefixed. A named tier prefixes each
name with its label and a slash: tier `region/sjc` publishes
Expand All @@ -107,7 +108,9 @@ A presence track exists only while subscribed. Ask for `<root>/presence.json`
for any auth root in the group (`presence.json` for the empty root). The relay
answers with the root's current sessions and counts from there, holding a root
with no sessions open at `{}`, and stops producing it once the last subscriber
leaves. A root outside the group, or any other name, is refused.
leaves. A prefix track works the same way for any broadcast prefix in the
group: ask for `<path>/traffic.json` (`traffic.json` for the empty prefix). A
root or prefix outside the group, or any other name, is refused.

## Frames

Expand Down Expand Up @@ -159,6 +162,36 @@ A presence track's entries are cumulative from when it was first held: a
session that ends stays counted, and so does every one after it, while
anyone holds the track. A track held again starts over.

### Prefix

```json
{
"": {
"publisher": { ... }, "subscriber": { ... },
"self": { "publisher": { ... }, "subscriber": { ... } },
"children": { "live": { "publisher": { ... }, "subscriber": { ... } } }
}
}
```

Each tier the prefix recorded on, with three views of its [Traffic](#traffic):
the top-level `publisher` and `subscriber` roll up every broadcast at or under
the prefix, so a [totals](#totals) reader parses them unchanged; `self` is the
broadcast at exactly the prefix, apart from any nested under it; and
`children` rolls up each direct child, keyed by its next path segment.

Like presence, the rollup and `self` are cumulative from when the track was
first held: each frame sums the live broadcasts' counters plus the final
counters of those that ended since. A broadcast that was live before the hold
carries its history from before it, so absolute values are not authoritative:
take differences within one subscription, and read the [totals](#totals) for
real sums. Started minus ended stays exact. A child appears while a broadcast
under it has counters, and leaves after the frame carrying its last one; its
tail stays in the rollup, and a child that returns starts over. Children are
never capped, so roughly ten thousand across four tiers outgrow the 32 MB frame
limit: that encoding's track then ends with `FRAME_TOO_LARGE` instead of going
stale.

## Entries

### Traffic
Expand Down Expand Up @@ -252,9 +285,9 @@ count is started minus ended. A frame never shows ended above started.
A new epoch starts every counter from zero, so a reader summing a node over
time adds each epoch's counters rather than diffing across them. Within one
epoch the totals never go down. A per-tier counter going **down** means the
entry was dropped and re-created, and a presence counter going down means the
track was held again. Treat it as the start of a fresh segment rather than a
negative rate. On a route without an epoch (an older relay, or a session older
entry was dropped and re-created, and a presence or prefix counter going down
means the track was held again. Treat it as the start of a fresh segment
rather than a negative rate. On a route without an epoch (an older relay, or a session older
than moq-lite 07), a counter going down also means the node restarted.

A reader ignores unknown fields, so a newer relay can add counters, and
Expand Down
2 changes: 0 additions & 2 deletions quest/m0/broadcast-epoch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,5 +128,3 @@ End+Start. The child quests own their focused regressions.
## Related

- [JS consume identity](https://github.com/moq-dev/moq/pull/5254) - requests carry the serving prefix's epoch and new winners cannot reuse an old cached broadcast

- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) - deferred with QoS to m2; neither gates epoch correctness or the next cut
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ ranked just below deterministic TS legs, which it requires.
- [Kotlin calls](/quest/m2/kt-ffi-calls.md) - Kotlin calls into moq-ffi cost about what Python's and Go's do, sync and async, once the fork's Kotlin generator drops JNA's per-call overhead
- [Poll-native async](/quest/m2/ffi-poll-migrate.md) - moq-ffi's pure-kio async exports poll their state directly instead of going through `ffi::Task`
- [Foreign poll for hot reads](/quest/m2/ffi-poll-protocol.md) - the fork's generators give each wrapper a sync poll plus a registered waker for track and group reads, so a ready read skips the event-loop round trip and a cancel never loses a frame
- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) - additive tracks required by the breaking lag API; existing maps remain supported
- [QoS](/quest/m2/qos/README.md) - broadcast health: relay starvation and timeliness histograms
- [Plan: watch worker](/quest/m2/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides
- [Watch worker](/quest/m2/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio
Expand Down
6 changes: 2 additions & 4 deletions quest/m2/qos-lag-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,8 @@ and per node like the existing counters.

## Plan

- Read both from the stats totals and [prefix
tracks](/quest/m2/stats-prefix-tracks.md), which land first,
rather than the `publisher.json` rows they sit on today (decided in the
- Read both from the stats totals and prefix tracks
(`<path>/traffic.json`) rather than the `publisher.json` rows they sit on today (decided in the
2026-10-06 audit; the rows stay for existing readers, 2026-10-12); [viewer lag histogram](/quest/m2/qos/lag-histogram.md)
lands them on the egress side of its per-group totals and prefix tracks
(decided 2026-10-08). `lag` is a cumulative byte count per bucket keyed
Expand All @@ -35,4 +34,3 @@ Public API: none. Wire: none.
## Required

- [Viewer lag histogram](/quest/m2/qos/lag-histogram.md) - the `lag` histogram and `dropped` counters the dashboard reads
- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) - the prefix tracks the dashboard reads lag from, beside the totals
6 changes: 2 additions & 4 deletions quest/m2/qos-publisher-timeliness.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,8 @@ milliseconds, and the re-anchor happens on its own as old minima expire.
Document the window length beside the bucket edges.

Aggregate as a byte-weighted cumulative histogram of drift on the ingress
(`Role::Subscriber`) side of the stats totals and [prefix
tracks](/quest/m2/stats-prefix-tracks.md), which land first,
rather than the per-path rows (decided in the 2026-10-06 audit), with the same
(`Role::Subscriber`) side of the stats totals and prefix tracks
(`<path>/traffic.json`) rather than the per-path rows (decided in the 2026-10-06 audit), with the same
bucket edges and the same monotonic contract as the egress `lag` histogram. Beside it keep two cumulative
counters: `timestamp_regressions`, frames whose timestamp is below the
previous group's newest timestamp, and `stalls`, gaps between complete frames
Expand All @@ -56,4 +55,3 @@ excluded.
## Required

- [Viewer lag histogram](/quest/m2/qos/lag-histogram.md) - the egress histogram whose type and edges this reuses
- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) - the prefix tracks the drift histogram lands on, beside the totals
8 changes: 2 additions & 6 deletions quest/m2/qos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,8 @@ The per-broadcast verdict combining client reports, the relay's starvation,
and publisher timeliness lives downstream. This line only reports the
counters.

Decided (2026-10-06 audit): [stats prefix
tracks](/quest/m2/stats-prefix-tracks.md) lands first, since it is
a prerequisite of the lag API; both are deferred to m2 by the maintainer
Decided (2026-10-06 audit): stats prefix tracks (`<path>/traffic.json`)
landed first, since they are a prerequisite of the lag API; both are deferred to m2 by the maintainer
on 2026-10-15; the lag histogram lands on the stats totals and
prefix tracks rather than the per-path `publisher.json` and
`subscriber.json` map rows, which stay unchanged for existing readers
Expand All @@ -61,9 +60,6 @@ on `main`.

## Related

- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) -
lands first beside the map rows; this line's histograms land on the
totals and prefix tracks
- [Media stats](/quest/m2/stats/README.md) - publishers and viewers report
their own media, transport, and playback health, the media half of a verdict
- [Loss delay](/quest/m3/cut-through/loss-delay.md) - an ingress counter of
Expand Down
5 changes: 1 addition & 4 deletions quest/m2/qos/lag-histogram.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The relay's `moq-stats` egress side reports, per broadcast, how far behind
its viewers are according to what the network has acknowledged: a
byte-weighted cumulative `lag` histogram and `dropped` counters (duration,
bytes, and groups dropped before they were acknowledged), on the stats totals
and [prefix tracks](/quest/m2/stats-prefix-tracks.md). A spliced subscription
and prefix tracks (`<path>/traffic.json`). A spliced subscription
stays exact across a route switch, and a closing subscription records its
last partial interval.

Expand Down Expand Up @@ -55,6 +55,3 @@ monotonic so readers diff two samples and sum nodes bucket by bucket. Update
Public API: breaking in `moq-stats`, as the line already decided. Wire: the
stats JSON gains the egress fields.

## Required

- [Stats prefix tracks](/quest/m2/stats-prefix-tracks.md) - the prefix tracks the histogram lands on, beside the totals
100 changes: 0 additions & 100 deletions quest/m2/stats-prefix-tracks.md

This file was deleted.

76 changes: 72 additions & 4 deletions rs/moq-stats/benches/producer_tick.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@
//! `stats/presence` sweeps the same tick over connected auth roots and the
//! presence tracks held for them.
//!
//! `stats/prefix` sweeps it over held paths, path depth, and held prefix
//! tracks: each row walks its ancestors once a prefix is held, so the cost
//! grows with rows times depth, plus each held track's encode.
//!
//! Run `cargo bench -p moq-stats --features bench --bench producer_tick`.

use std::alloc::{GlobalAlloc, Layout, System};
Expand Down Expand Up @@ -83,15 +87,18 @@ struct Relay {
_publishers: Vec<broadcast::Producer>,
_cursors: Vec<announce::Consumer>,
_sessions: Vec<stats::Session>,
_held: Vec<moq_net::track::Subscriber>,
}

impl Relay {
fn build(paths: usize, tiers: usize, changed_percent: usize) -> Self {
/// `depth` is each path's segment count, at least 2.
fn build(paths: usize, tiers: usize, changed_percent: usize, depth: usize) -> Self {
let registry = stats::Registry::new(
stats::Config::new().with_exclude(moq_net::Pattern::subtree(".stats").expect("literal prefix")),
);
let (origin, origin_driver) = origin::Producer::new(origin::Config::default());
let path_names: Vec<String> = (0..paths).map(|index| format!("room/{index:06}")).collect();
let nested = "/s".repeat(depth - 2);
let path_names: Vec<String> = (0..paths).map(|index| format!("room/{index:06}{nested}")).collect();
let publishers: Vec<_> = path_names
.iter()
.map(|path| {
Expand Down Expand Up @@ -130,6 +137,15 @@ impl Relay {
_publishers: publishers,
_cursors: cursors,
_sessions: sessions,
_held: Vec::new(),
}
}

/// Hold the first `held` paths' `room/<index>` prefix tracks.
fn hold(&mut self, held: usize) {
for index in 0..held {
let sub = self.driver.hold_prefix(&format!("room/{index:06}"));
self._held.push(sub);
}
}

Expand Down Expand Up @@ -213,7 +229,7 @@ fn bench(c: &mut Criterion) {
let mut relay = None;
group.bench_function(id, |b| {
let relay = relay.get_or_insert_with(|| {
let mut relay = Relay::build(paths, tiers, changed);
let mut relay = Relay::build(paths, tiers, changed, 2);
let row = sample(&mut relay);
let micros = row.elapsed.as_secs_f64() * 1e6;
println!(
Expand Down Expand Up @@ -327,5 +343,57 @@ fn presence(c: &mut Criterion) {
group.finish();
}

criterion_group!(benches, bench, presence);
const PREFIX_PATHS: [usize; 2] = [1_000, 10_000];

/// Segments per path.
const PREFIX_DEPTH: [usize; 2] = [2, 8];

/// Prefix tracks a reader holds.
const PREFIX_HELD: [usize; 3] = [0, 10, 100];

/// What holding prefix tracks costs a tick, with a tenth of the paths moving.
fn prefix(c: &mut Criterion) {
println!("paths depth held allocs/tick us/tick");

let mut group = c.benchmark_group("stats/prefix");
group.sample_size(10);
group.warm_up_time(Duration::from_millis(200));
group.measurement_time(Duration::from_secs(1));
for &paths in &PREFIX_PATHS {
for &depth in &PREFIX_DEPTH {
for &held in &PREFIX_HELD {
let id = BenchmarkId::from_parameter(format!("{paths}p_{depth}d_{held}h"));
group.throughput(Throughput::Elements(paths as u64));
let mut relay = None;
group.bench_function(id, |b| {
let relay = relay.get_or_insert_with(|| {
let mut relay = Relay::build(paths, 1, 10, depth);
black_box(relay.driver.tick());
relay.hold(held);
let row = sample(&mut relay);
let micros = row.elapsed.as_secs_f64() * 1e6;
println!(
"{paths:>5} {depth:>5} {held:>4} {allocs:>11} {micros:>8.1}",
allocs = row.allocs
);
relay
});
b.iter_custom(|iters| {
let mut total = Duration::ZERO;
for _ in 0..iters {
relay.touch();
let start = Instant::now();
black_box(relay.driver.tick());
total += start.elapsed();
}
total
});
});
}
}
}
group.finish();
}

criterion_group!(benches, bench, presence, prefix);
criterion_main!(benches);
Loading