Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions doc/concept/archives.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ Tracks have separate directories, with each track name percent-encoded as one
segment. Within one broadcast prefix:

- Each track's `.info` stores its timescale, when timed, and priority.
- Data objects are keyed by their first group's start, frame index, and
presentation timestamp; untimed tracks omit the timestamp. The sorted key listing is the
- Data objects are keyed by their first group's start and frame index. On a
timed track a group's start is its time, so the sorted key listing is the
index, with no manifest or timeline object.
- Each track's `end` records its final position or stream error. The broadcast's
`.end` marks the recording complete, after its track markers are written.
Expand All @@ -82,7 +82,10 @@ window and grace; the newest object of each track stays. Pruning runs only while
the recorder runs: once a recording completes or its recorder stops, objects not
yet due stay until something else deletes them, such as a store lifecycle rule.

The [moq-archive rustdoc](https://docs.rs/moq-archive) specifies the format for
The store format is versioned, and a reader accepts only its own version: a
prefix written in another one is refused with the version it found. Version 1
shipped first; the current format is version 2, which cannot read version 1
recordings. The [moq-archive rustdoc](https://docs.rs/moq-archive) specifies the format for
the published version. This page follows the
[layout documented in current source](https://github.com/moq-dev/moq/blob/main/rs/moq-archive/src/lib.rs#L8),
which can differ from an installed package; use the reference matching your version.
Expand Down
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ removal, a rename, or a wire, catalog, or format change that is free until
- **Catalog, drafts, and formats.** [hang-04](/quest/m2/hang04/README.md)
(the `sync` section, a version per entry, no video `warmup`, `m2ts` under
`mpegts`, one strict rule);
[the archive as version 2](/quest/m2/archive-v2.md);
[declared ends](/quest/m2/time-addressing/consumers.md), back from m3;
[finish](/quest/m2/finish-sweep.md), [TS outcomes](/quest/m2/ts-typed-outcomes.md),
and [moq-json writes](/quest/m2/json-write-at.md).
Expand Down
84 changes: 0 additions & 84 deletions quest/m2/archive-v2.md

This file was deleted.

1 change: 0 additions & 1 deletion quest/m2/time-addressing/consumers.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,5 +139,4 @@ them.

## Related

- [The archive format ships as version 2](/quest/m2/archive-v2.md) - freezes the keys this made starts
- [A group's frames present before the next group starts](/quest/m3/groups-never-overlap.md) - tightens the group rule a declared end relies on
10 changes: 2 additions & 8 deletions rs/moq-archive/src/consumer/index.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,13 +134,7 @@ impl Index {
/// read its `end` when `end` is set. `.info` sorts before every object key and `end` after,
/// so listing after `.info` lists everything else. Object stores promise no listing order, so
/// it is sorted here.
pub(super) async fn list<S: ObjectStore>(
store: &S,
dir: &Path,
cursor: &Path,
timed: bool,
end: bool,
) -> Result<Listed> {
pub(super) async fn list<S: ObjectStore>(store: &S, dir: &Path, cursor: &Path, end: bool) -> Result<Listed> {
let metas: Vec<_> = store.list_with_offset(Some(dir), cursor).try_collect().await?;
let mut listed = Listed::default();
let mut found = None;
Expand All @@ -153,7 +147,7 @@ pub(super) async fn list<S: ObjectStore>(
.location
.filename()
.ok_or_else(|| Error::Path(meta.location.to_string()))?;
match Name::parse(filename, timed)? {
match Name::parse(filename)? {
Name::Info => {}
Name::End => found = Some(meta.location),
Name::Object(start) => listed.objects.push(((start.group, start.frame), meta.location)),
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-archive/src/consumer/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ async fn refresh<S: ObjectStore>(shared: Arc<Shared<S>>, mut tracks: Vec<Snapsho
let index = track.index.lock().unwrap();
(index.dir.clone(), index.cursor().clone(), index.timed())
};
let listed = index::list(store.inner(), &dir, &cursor, timed, ended).await?;
let listed = index::list(store.inner(), &dir, &cursor, ended).await?;
track.index.lock().unwrap().extend(&listed.objects);
if let Some(end) = listed.end {
ends.push((track.name.clone(), end));
Expand Down
20 changes: 20 additions & 0 deletions rs/moq-archive/src/consumer/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -655,6 +655,26 @@ async fn an_empty_or_unrecognized_prefix_is_refused() {
archive.raw("video/segments/0000000000000000000", b"x").await;
assert!(matches!(open(archive.store.clone()).await, Some(Error::Path(_))));

// Release's version 1 layout and an older main build's version 4 are refused by version,
// before any of their keys is read.
for (info, key, version) in [
(
&br#"{"version":1,"priority":0,"timescale":1000}"#[..],
"video/segments/0000000000000000000",
1,
),
(
br#"{"version":4,"priority":0,"timescale":1000}"#,
"video/0000000000000000000-0000000000000000000-0000000000000000000",
4,
),
] {
let archive = Archive::new();
archive.raw("video/.info", info).await;
archive.raw(key, b"x").await;
assert_eq!(open(archive.store.clone()).await, Some(Error::Version(version)));
}

// A directory that is not a canonical track name.
let archive = Archive::new();
archive
Expand Down
9 changes: 2 additions & 7 deletions rs/moq-archive/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ pub enum Error {
#[error("not found: {0}")]
NotFound(String),

/// The recording format version is unknown, or older than the current one when writing.
#[error("unsupported version {0}")]
/// The archive format version found is not [`VERSION`](crate::VERSION), the only one supported.
#[error("archive format version {0} is not supported, only version {supported}", supported = crate::VERSION)]
Version(u64),

/// A sequence, frame index, or timestamp is outside 0 through 2^53 - 1.
Expand Down Expand Up @@ -44,11 +44,6 @@ pub enum Error {
#[error("empty object")]
Empty,

/// A version 2 or 3 object entry is not a group: the datagram form, which an archive
/// does not replay, or an unknown one.
#[error("unsupported entry form {0}")]
Form(u64),

/// Group sequences are not strictly ascending.
#[error("group sequences are not strictly ascending")]
Sequence,
Expand Down
Loading