Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2796276
chore(quest): claim quest/m2/archive-time-ranges.md
kixelated Oct 30, 2026
a135897
wip: archive time ranges
kixelated Oct 30, 2026
c4941dc
wip: fmt
kixelated Oct 30, 2026
228a8c2
wip: lookups in filter
kixelated Oct 30, 2026
cc65eb4
feat(net)!: a time start the cache can't resolve is looked up through…
kixelated Oct 30, 2026
210cb27
Merge remote-tracking branch 'origin/main' into quest/m2/archive-time…
kixelated Oct 30, 2026
c652836
fix(net): look up a time start in a cache gap, and wait on lookups be…
kixelated Oct 30, 2026
a888e62
fix(net): look up a time start in any cache gap, evicted arrivals inc…
kixelated Oct 30, 2026
253787d
test(net): push the gone run as the other window tests do
kixelated Oct 30, 2026
e54ba43
Merge remote-tracking branch 'origin/main' into quest/m2/archive-time…
kixelated Oct 30, 2026
234162e
wip(net,archive): bound time lookups by group starts, not evicted arr…
kixelated Oct 31, 2026
746417a
fix(net): count a filled group's start for time ranges on a history-s…
kixelated Oct 31, 2026
94cdc3a
fix(net,archive): settle a time end after an unanswered lookup, skip …
kixelated Oct 31, 2026
6515d2a
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Nov 1, 2026
dd73c21
fix(archive): drop lookup cancellation after main removed Fill::unused
kixelated Nov 1, 2026
aed225f
Merge base into PR 7310 preserving time lookups and cursor updates
Nov 1, 2026
4ac5799
Merge origin/main into quest/m2/archive-time-ranges
kixelated Nov 1, 2026
b3af034
Merge base into PR 7310 preserving time lookups and stale accounting
Nov 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,6 @@ lite-07; bitrate enforcement is not part of this release.
- [The relay over QUIC on the sim](/quest/m2/quic-sim.md) - the relay and moq-tokio's `Client` and `Server` run over `moq-quic` on the seeded sim
- [moq-shaper's own socket never passes for path loss](/quest/m2/shaper-socket.md) - relay-sized buffers and its own socket drops reported beside path drops

- [Archive time ranges](/quest/m2/archive-time-ranges.md) - a `Time(pts)` range on an archive consumer broadcast resolves from the archive's keys, however old
- [DVR rewind](/quest/m2/archive-dvr.md) - follow-on capability after the release contract is stable
- [Archive lookups cancel when abandoned](/quest/m2/archive-lookup-withdrawal.md) - a public withdrawal signal on `Fill` stops a time lookup's store read once its readers leave

Expand Down
1 change: 0 additions & 1 deletion quest/m2/archive-dvr.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,3 @@ over missing groups, and return to live.
## Required

- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - the `Time(pts)` range a seek sends
- [Archive time ranges](/quest/m2/archive-time-ranges.md) - the archive consumer resolves time ranges from its keys
35 changes: 0 additions & 35 deletions quest/m2/archive-time-ranges.md

This file was deleted.

1 change: 0 additions & 1 deletion quest/m3/studio/replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,5 @@ any other.

## Related

- [Archive time ranges](/quest/m2/archive-time-ranges.md) - ranges reaching past the cache
- [Archive DVR](/quest/m2/archive-dvr.md) - viewer scrubbing beyond the relay cache
- [Recording](/quest/m3/studio/recording.md) - clips use the same file writer
86 changes: 74 additions & 12 deletions rs/moq-archive/src/consumer/index.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
//! One track's object keys, listed once and extended after its newest key.

use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};

use futures::TryStreamExt;
use moq_net::Timescale;
use moq_net::{Timescale, Timestamp};
use object_store::path::Path;
use object_store::{ObjectStore, ObjectStoreExt};

Expand All @@ -13,11 +13,15 @@ use crate::{Error, Info, Result};
/// Where an object starts: its first sequence and that sequence's first frame index.
pub(super) type Start = (u64, u64);

/// One listed object: where it starts, its first frame's timestamp on a timed track, and
/// its key.
pub(super) type Listing = (Start, Option<u64>, Path);

/// What one listing of a track's directory found.
#[derive(Default)]
pub(super) struct Listed {
/// The objects new since the cursor, oldest first.
pub objects: Vec<(Start, Path)>,
pub objects: Vec<Listing>,
/// The track's `end`, when asked for.
pub end: Option<End>,
}
Expand All @@ -26,7 +30,13 @@ pub(super) struct Listed {
pub(super) struct Index {
pub dir: Path,
pub timescale: Option<Timescale>,
objects: BTreeMap<Start, Path>,
/// Each object's first frame timestamp, on a timed track, and key.
objects: BTreeMap<Start, (Option<u64>, Path)>,
/// Each timed object starting a group, by that group's start timestamp with where it
/// starts, so a time finds the newest such object at or before it in one lookup. Group
/// starts never rewind, so they sort as the objects do. An object continuing a group is
/// left out: its first frame says nothing about where the groups after it start.
times: BTreeSet<(u64, Start)>,
/// The newest key listed, which the next listing starts after.
cursor: Path,
}
Expand All @@ -38,6 +48,7 @@ impl Index {
dir,
timescale: super::track::timescale(info)?,
objects: BTreeMap::new(),
times: BTreeSet::new(),
cursor,
})
}
Expand All @@ -52,9 +63,14 @@ impl Index {
}

/// Take in the objects a listing after [`Self::cursor`] found.
pub fn extend(&mut self, objects: &[(Start, Path)]) {
for (start, path) in objects {
self.objects.insert(*start, path.clone());
pub fn extend(&mut self, objects: &[Listing]) {
for (start, pts, path) in objects {
self.objects.insert(*start, (*pts, path.clone()));
if let Some(pts) = pts
&& start.1 == 0
{
self.times.insert((*pts, *start));
}
self.cursor = path.clone();
}
}
Expand All @@ -63,19 +79,19 @@ impl Index {
pub fn newest(&self) -> Option<(Start, Path)> {
self.objects
.last_key_value()
.map(|(start, path)| (*start, path.clone()))
.map(|(start, (_, path))| (*start, path.clone()))
}

/// The objects holding group `sequence` from frame `frame` on, oldest first: the last one
/// starting at or before it, then each continuing the group. Also the start of the first
/// object after them, which bounds a gap.
pub fn holding(&self, sequence: u64, frame: u64) -> (Vec<(Start, Path)>, Option<Start>) {
let mut holding = Vec::new();
if let Some((start, path)) = self.objects.range(..=(sequence, frame)).next_back() {
if let Some((start, (_, path))) = self.objects.range(..=(sequence, frame)).next_back() {
holding.push((*start, path.clone()));
}
let mut after = None;
for (start, path) in self.objects.range((sequence, frame.saturating_add(1))..) {
for (start, (_, path)) in self.objects.range((sequence, frame.saturating_add(1))..) {
if start.0 != sequence {
after = Some(*start);
break;
Expand All @@ -85,9 +101,53 @@ impl Index {
(holding, after)
}

/// The objects that may hold the newest group starting at or before `pts`, newest first:
/// from the newest object starting a group at or before it, or the oldest object when none
/// does, up to the next object starting a group after it, which holds only later groups.
/// One continuing a group the next object still continues starts none, so it is skipped.
/// Empty on an untimed track.
pub fn around(&self, pts: Timestamp) -> Vec<(Start, Path)> {
let Some(timescale) = self.timescale else {
return Vec::new();
};
// A stored timestamp is whole, so it is at or before `pts` exactly when it is at or
// before `pts` rounded down to the track's timescale.
let limit = u64::try_from(pts.as_scale(timescale)).unwrap_or(u64::MAX);
let from = self
.times
.range(..=(limit, (u64::MAX, u64::MAX)))
.next_back()
.map_or((0, 0), |(_, start)| *start);

let mut around = Vec::new();
let mut objects = self.objects.range(from..).peekable();
while let Some((start, (first, path))) = objects.next() {
if start.1 == 0 && first.is_some_and(|first| first > limit) {
break;
}
let continued = start.1 > 0 && objects.peek().is_some_and(|(next, _)| next.0 == start.0);
if !continued {
around.push((*start, path.clone()));
}
}
around.reverse();
around
}

/// The object just after the one starting at `start`, or the oldest for `None`.
pub fn after(&self, start: Option<Start>) -> Option<(Start, Path)> {
let (start, (_, path)) = match start {
Some(start) => self.objects.range((start.0, start.1.saturating_add(1))..).next()?,
None => self.objects.first_key_value()?,
};
Some((*start, path.clone()))
}

/// Forget an object the store no longer has, such as one a DVR pruned.
pub fn forget(&mut self, start: Start) {
self.objects.remove(&start);
if let Some((Some(pts), _)) = self.objects.remove(&start) {
self.times.remove(&(pts, start));
}
}
}

Expand Down Expand Up @@ -117,7 +177,9 @@ pub(super) async fn list<S: ObjectStore>(
match Name::parse(filename, timed)? {
Name::Info => {}
Name::End => found = Some(meta.location),
Name::Object(start) => listed.objects.push(((start.group, start.frame), meta.location)),
Name::Object(start) => listed
.objects
.push(((start.group, start.frame), start.pts, meta.location)),
}
}
listed.objects.sort();
Expand Down
13 changes: 12 additions & 1 deletion rs/moq-archive/src/consumer/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
//! demand from the store through [`track::Requests`](moq_net::track::Requests) fills, each sequence in its recorded form: a
//! group as a group, a datagram as a datagram.
//!
//! A reader's time start older than the newest group is looked up from the object keys, each of
//! which carries its first frame's timestamp, so the range starts at the stored group containing
//! it however old, and a time end stops just past the group containing it.
//!
//! A growing archive is followed: every [`Config::refresh`], one delimiter LIST of the prefix
//! finds added tracks, and one LIST per track after its newest key finds new objects. Once the
//! prefix holds `.end`, each track ends as recorded and refreshing stops. Without it the archive
Expand Down Expand Up @@ -330,7 +334,14 @@ async fn refresh<S: ObjectStore>(shared: Arc<Shared<S>>, mut tracks: Vec<Snapsho

let (objects, from) = match track.started {
// Everything new, in order.
true => (listed.objects, 0),
true => (
listed
.objects
.into_iter()
.map(|(start, _, path)| (start, path))
.collect(),
0,
),
// Only the newest sequence: fills serve the rest.
false => {
let Some((start, path)) = track.index.lock().unwrap().newest() else {
Expand Down
Loading