Repository navigation
fix(net): an empty replacement track ends past its predecessor's groups #6376
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
acd25a2
28a93c9
298adfe
ef213e8
e923230
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -763,6 +763,10 @@ export class Producer { | |
| // `max_sequence`, and where the next append lands. The namespace above can run ahead of | ||
| // it: earlier producers of the same broadcast track advanced it too. | ||
| #received = 0; | ||
| // Where the namespace stood when a replacement claimed the name. From then on it is this | ||
| // producer's inherited edge, so the replacement's groups never count as its progress, and | ||
| // it writes nothing more, so it can't land a group past the replacement's end. | ||
| #handoff?: number; | ||
|
|
||
| // Recently written source groups, retained for replay to late subscribers and | ||
| // pruned once idle for longer than the cache window. Each entry tracks the mirror | ||
|
|
@@ -839,6 +843,11 @@ export class Producer { | |
| bindProducerSequence = (producer, sequence) => { | ||
| const own = producer.#sequence; | ||
| if (own === sequence) return; | ||
| if (producer.#state.closed.peek() === undefined) { | ||
| const owner = sequence.owner; | ||
| if (owner && owner.#state.closed.peek() === undefined) owner.#handoff = sequence.next; | ||
| sequence.owner = producer; | ||
| } | ||
| sequence.next = Math.max(sequence.next, producer.#received); | ||
| // What earlier producers of the name wrote is gone from this track's cache, so their | ||
| // highest checked start is its eviction watermark. | ||
|
|
@@ -1448,6 +1457,7 @@ export class Producer { | |
| // Refuse a write once the track is closed, or at or past its declared end. | ||
| #writable(sequence: number): void { | ||
| if (this.#state.closed.peek() !== undefined) throw new Error("track is closed"); | ||
| if (this.#handoff !== undefined) throw new Error(`track ${this.name} was replaced`); | ||
| const end = this.#state.end.peek(); | ||
| if (end !== undefined && sequence >= finalSequence(end)) { | ||
| throw new Error(`sequence ${sequence} is at or past the track's end ${end.group}/${end.frame}`); | ||
|
|
@@ -1599,21 +1609,33 @@ export class Producer { | |
| * at or past it is refused. Unlike {@link close} it is not terminal: call `close()` once | ||
| * the remaining groups are written. Readers end once the live edge reaches the end's | ||
| * group, independently of closure. Throws if the track is closed, already has an end, or | ||
| * `end` would orphan a group or frame already produced. | ||
| * `end` falls before or inside a group already produced, including one an earlier producer | ||
| * of the name wrote, which a replacement continues. | ||
| */ | ||
| finishAt(end: Location): void { | ||
| // A replacement continues its name's timeline, so it can't end before or inside a group | ||
| // an earlier producer wrote; only its own last group can be cut mid-way. | ||
| const ownLast = end.frame !== 0 && this.#received === end.group + 1; | ||
| if (end.group < this.#inherited() && !ownLast) { | ||
| throw new ProtocolViolation(`track end ${end.group}/${end.frame} is inside the name's earlier groups`); | ||
| } | ||
| this.#finishAt(end); | ||
| } | ||
|
|
||
| #finishAt(end: Location): void { | ||
| if (this.#state.closed.peek() !== undefined) throw new Error("track is closed"); | ||
| const declared = this.#state.end.peek(); | ||
| if (declared !== undefined) throw new Error(`track already ends at ${declared.group}/${declared.frame}`); | ||
| this.#declareEnd(end); | ||
| } | ||
|
|
||
| // The wire's end: repeating it is a no-op, and a mid-group end may refine the group | ||
| // boundary it falls just before. Mirrors the Rust `finish_at` on a `received()` copy. | ||
| // boundary it falls just before. Mirrors the Rust `finish_at` on a `received()` copy, | ||
| // which answers to its publisher's end rather than the name's local writes. | ||
| #endPending(end: Location): void { | ||
| const declared = this.#state.end.peek(); | ||
| if (declared === undefined) { | ||
| this.finishAt(end); | ||
| this.#finishAt(end); | ||
| } else if (declared.group !== end.group || declared.frame !== end.frame) { | ||
| if (declared.frame !== 0 || end.frame === 0 || finalSequence(end) !== declared.group) { | ||
| throw new ProtocolViolation(`conflicting track end: ${end.group}/${end.frame}`); | ||
|
|
@@ -1672,6 +1694,18 @@ export class Producer { | |
| return !this.#arrivals.covers(Math.min(floor, final), final); | ||
| } | ||
|
|
||
| // One past the highest sequence produced: where the live edge stands against the end. A | ||
| // replacement starts at its name's, since what earlier producers wrote never arrives here. | ||
| // Mirrors the Rust `edge`. | ||
| #liveEdge(): number { | ||
| return Math.max(this.#received, this.#inherited()); | ||
| } | ||
|
|
||
| // One past what the name's producers wrote, up to the replacement that superseded this one. | ||
| #inherited(): number { | ||
| return this.#handoff ?? this.#sequence.next; | ||
| } | ||
|
|
||
| #declareEnd(end: Location): void { | ||
| const { group, frame } = end; | ||
| if (!Number.isSafeInteger(group) || group < 0 || !Number.isSafeInteger(frame) || frame < 0) { | ||
|
|
@@ -1703,7 +1737,7 @@ export class Producer { | |
| #endLargest(): void { | ||
| if (this.#state.largestEnded.peek() || this.#largestOpen) return; | ||
| const end = this.#state.end.peek(); | ||
| if (end === undefined || this.#received < finalSequence(end)) return; | ||
| if (end === undefined || this.#liveEdge() < finalSequence(end)) return; | ||
| for (const state of [this.#state, ...this.#sinks]) state.largestEnded.set(true); | ||
| } | ||
|
|
||
|
|
@@ -1722,14 +1756,17 @@ export class Producer { | |
| // A gap past the grace was lost before the abort rather than cut off by it. | ||
| if (abort && this.#tailPending) this.#arrivals.expire(now(), this.#grace()); | ||
| if (abort && this.#settled()) abort = undefined; | ||
| // A replacement continues its name's timeline: ending below an earlier producer's | ||
| // groups would contradict the ones subscribers already received. | ||
| if (abort === undefined && this.#state.end.peek() === undefined) { | ||
| this.#declareEnd({ group: this.#received, frame: 0 }); | ||
| this.#declareEnd({ group: this.#liveEdge(), frame: 0 }); | ||
|
Comment on lines
1761
to
+1762
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When AGENTS.md reference: AGENTS.md:L65-L65 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in e923230 for JS, the only side where this can happen: once a replacement claims the name, the removed producer's writes fail ( Rust has no such overlap: (Written by Claude Opus 5.5) |
||
| } | ||
| // Nobody will finish these, so a subscriber that has not taken one yet never sees it. | ||
| // Not evicted: a reader already holding one keeps its frames and sees the abort. | ||
| const open = abort ? this.#cache.filter((entry) => entry.group.closed.peek() === undefined) : []; | ||
| closeTrackState(this.#state, abort); | ||
| this.#sequence.holders--; | ||
| if (this.#sequence.owner === this) this.#sequence.owner = undefined; | ||
| for (const { group } of this.#cache) group.close(abort); | ||
| for (const entry of open) { | ||
| this.#unlink(entry); | ||
|
|
@@ -1752,7 +1789,7 @@ export class Producer { | |
| const end = this.#state.end.peek(); | ||
| if (end === undefined) return false; | ||
| const final = finalSequence(end); | ||
| if (this.#received < final || this.#owes(final)) return false; | ||
| if (this.#liveEdge() < final || this.#owes(final)) return false; | ||
| return this.#cache.every(({ group }) => group.sequence >= final || group.closed.peek() === null); | ||
| } | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -2198,8 +2198,11 @@ impl TrackState { | |||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| /// One past the highest sequence produced: where the live edge stands against the end. | ||||||||||||||||||||||
| /// A replacement starts at its name's, since what earlier producers wrote never arrives here. | ||||||||||||||||||||||
| fn edge(&self) -> u64 { | ||||||||||||||||||||||
| self.max_sequence.map_or(0, |max| max.saturating_add(1)) | ||||||||||||||||||||||
| self.max_sequence | ||||||||||||||||||||||
| .map_or(0, |max| max.saturating_add(1)) | ||||||||||||||||||||||
| .max(self.sequence.next()) | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| /// Whether a group below `fin` the wire subscription still owes has yet to arrive. | ||||||||||||||||||||||
|
|
@@ -2924,7 +2927,8 @@ impl Producer { | |||||||||||||||||||||
|
|
||||||||||||||||||||||
| /// Mark the track as finished after the last appended group. | ||||||||||||||||||||||
| /// | ||||||||||||||||||||||
| /// Ends the track at the head of the group after the current max_sequence. | ||||||||||||||||||||||
| /// Ends the track at the head of the group after the largest one written to its name, | ||||||||||||||||||||||
| /// including by an earlier producer this one replaces. | ||||||||||||||||||||||
| /// No new groups at or above it can be appended. | ||||||||||||||||||||||
| /// NOTE: Old groups with lower sequence numbers can still arrive. | ||||||||||||||||||||||
| pub fn finish(&self) -> Result<()> { | ||||||||||||||||||||||
|
|
@@ -2933,6 +2937,9 @@ impl Producer { | |||||||||||||||||||||
| Some(max) => max.checked_add(1).ok_or(coding::BoundsExceeded)?, | ||||||||||||||||||||||
| None => 0, | ||||||||||||||||||||||
| }; | ||||||||||||||||||||||
| // A replacement continues its name's timeline: ending below an earlier producer's | ||||||||||||||||||||||
| // groups would contradict the ones subscribers already received. | ||||||||||||||||||||||
| let final_sequence = final_sequence.max(state.sequence.next()); | ||||||||||||||||||||||
| state.set_end(Position::group(final_sequence)) | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
|
|
@@ -2953,13 +2960,21 @@ impl Producer { | |||||||||||||||||||||
| /// | ||||||||||||||||||||||
| /// Repeating the declared end is a no-op, and a mid-group end may refine the group | ||||||||||||||||||||||
| /// boundary it falls just before (a peer may name only the boundary first). Any other | ||||||||||||||||||||||
| /// second end is [`Error::ProtocolViolation`]. | ||||||||||||||||||||||
| /// second end is [`Error::ProtocolViolation`], and so is one before or inside a group an | ||||||||||||||||||||||
| /// earlier producer of the name wrote, which a replacement continues. | ||||||||||||||||||||||
| /// | ||||||||||||||||||||||
| /// On a [received](Self::received) copy, groups below the end may still be in flight: | ||||||||||||||||||||||
| /// readers wait for a missing one until a grace passes, the subscription's max delay | ||||||||||||||||||||||
| /// or one second without one, before ending at the boundary. | ||||||||||||||||||||||
| pub fn finish_at(&mut self, end: Position) -> Result<()> { | ||||||||||||||||||||||
| let mut state = self.modify()?; | ||||||||||||||||||||||
| // A replacement continues its name's timeline, so it can't end before or inside a group | ||||||||||||||||||||||
| // an earlier producer wrote; only its own last group can be cut mid-way. A received | ||||||||||||||||||||||
| // copy answers to its publisher's end, which the session judges. | ||||||||||||||||||||||
| let own_last = end.frame != 0 && state.max_sequence == Some(end.group); | ||||||||||||||||||||||
| if !state.received && end.group < state.sequence.next() && !own_last { | ||||||||||||||||||||||
| return Err(Error::ProtocolViolation); | ||||||||||||||||||||||
| } | ||||||||||||||||||||||
|
Comment on lines
+2974
to
+2977
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: set -u
rg -n -F --glob '*.rs' -e 'fn create_group' -e 'fn edge' -e 'max_sequence' -e 'fn finish_at' -e 'fn finish(' -e 'own_last' -e 'struct TrackState' -e 'sequence:' -- rs/moq-net/src/model/ || test "$?" -eq 1
awk 'NR>=2920 && NR<=3000 {printf "%5d %s\n", NR, $0}' rs/moq-net/src/model/track.rs
rg -n -F --glob '*.ts' -e 'finishAt' -e '#received' -- js/net/src/ || test "$?" -eq 1Repository: moq-dev/moq Length of output: 40396 🏁 Script executed: set -u
rg -n -F -A 30 -e 'fn set_end' -e 'fn final_sequence' -- rs/moq-net/src/model/track.rs rs/moq-net/src/model/track/ || test "$?" -eq 1Repository: moq-dev/moq Length of output: 6974 Restrict the
Proposed fix- let own_last = end.frame != 0 && state.max_sequence == Some(end.group);
+ let own_last = end.frame != 0
+ && state.max_sequence == Some(end.group)
+ && end.group.saturating_add(1) >= state.sequence.next();Add a test. A predecessor writes group 8. The replacement calls The JS 📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||
| match state.end { | ||||||||||||||||||||||
| Some(declared) if declared == end => return Ok(()), | ||||||||||||||||||||||
| Some(declared) if declared.frame == 0 && end.frame != 0 && end.final_sequence() == Some(declared.group) => { | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 41892
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 42239
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 32910
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 20256
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 28467
Advance subscriber progress to the shared live edge.
When a replacement inherits a namespace ahead of its own writes,
finishAt({ group: 1, frame: 0 })wakesrecvGroup()but does not complete it. The sink'sreceivedremains0, while#complete()requires it to reach1. The read can remain pending untilclose(). Apply the live edge when binding the namespace, declaring the end, and creating a late sink.Suggested fix
Add coverage that calls
finishAt()and awaitsrecvGroup()before callingclose().🤖 Prompt for AI Agents