Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ above them.
- [qmux credit](/quest/m0/qmux-credit.md) - dropped/stopped streams return credit and close is delivered

- [Restart followers](/quest/m1/restart-followers.md) - first-party consumers follow an epochless `Restart`, from a GOAWAY drain or a publisher replacement, by resolving the path again
- [Cross-node flapping](/quest/m1/cross-node-flapping.md) - incident-sensitive baseline and residual stale-revival counterexamples before the route rewrite
- [Cluster routing](/quest/m1/cluster-routing/README.md) - prove shared routing contracts, implement per-origin routes, and connect outbound local clusters
- [P2P](/quest/m1/p2p/README.md) - application-selected peer sessions use shared priorities, routing, and same-epoch handover
- [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation/P2P setup
Expand Down
1 change: 0 additions & 1 deletion quest/m1/cluster-routing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ setup after the capabilities land.
- [Route health](/quest/m2/route-health/README.md) - epoch-only active/passive delivery monitoring and automatic recovery
- [P2P](/quest/m1/p2p/README.md) - application-selected peer sessions feed the shared routing model
- [Routing guide](/quest/m1/routing-guide.md) - concept page and practical federation/P2P setup
- [Cross-node flapping](/quest/m1/cross-node-flapping.md) - incident-sensitive regressions for residual withdrawal hunting
- [Route wakes](/quest/m2/route-wakes.md) - local work follows affected fronts rather than the whole covered table
- [Link quality](/quest/m2/link-quality.md) - optional measured link cost, distinct from delivery-health failover
- [Same-epoch importers](/quest/m1/hop-aligned-import.md) - independent origins serving the same explicit epoch
1 change: 0 additions & 1 deletion quest/m1/cluster-routing/handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,4 @@ bounded seqno-request attempt ends and no further unsolicited update arrives.

## Related

- [Cross-node flapping](/quest/m1/cross-node-flapping.md) - residual stale-revival fixtures
- [Lazy discovery](/quest/m1/lazy-discovery/README.md) - interest changes must not erase freshness protection
18 changes: 16 additions & 2 deletions quest/m1/cluster-routing/routes.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,21 @@ Open, for the implementer to settle and record:
with deterministic acceptance tests before the wire is written.
- How much of today's route trie, fronts, and `route_order` survives intact.

The #4349 counterexamples are the `residual_*` tests in
`rs/moq-net/tests/mesh_withdraw.rs`. They pin today's hunting on lite-06:
a stale path outlasting the 300 ms hold, a revival the quarantine misses,
and 179 hunting relays (3 reviving) over 200 seeded meshes. Turn each into a
single retraction everywhere, and keep `cross_node_close_retracts_once`
passing. On `main` an epochless switch to a stale path is a Restart, which a
lite-06 client sees as a retraction and an announcement; the release sent an
in-place update.

Publishing toggles too, decided 2026-10-13: add a time-mocked fixture where,
on some rings, a longer path arrives first and the better route then replaces
it as a Restart, so a lite-06 client sees `Start, End, Start` while the
broadcast stays live. Pin today's sequence beside the `residual_*` tests;
the epochless Restart rule above decides what it becomes.

Tests, time mocked: #4644's live-graph withdrawal and failover tests
(`quest/m0/path-hunting` branch) as acceptance, run on the wip version; a
link flap with many broadcasts behind one origin costs one ROUTE change per
Expand Down Expand Up @@ -219,11 +234,10 @@ release.

- [Routing contracts](/quest/m1/cluster-routing/contracts.md) - authenticated incarnation, local priorities, forwarding progress, and export views
- [Next-hop handoff](/quest/m1/cluster-routing/handoff.md) - complete and fresh claims with bounded state
- [Cross-node flapping](/quest/m1/cross-node-flapping.md) - incident-sensitive baseline and residual counterexamples

## Closes

- [#4349](https://github.com/moq-dev/moq/issues/4349) - close once the residual cross-node flapping counterexamples pass
- [#4349](https://github.com/moq-dev/moq/issues/4349) - close once the `residual_*` tests in `mesh_withdraw.rs` retract once everywhere

## Related

Expand Down
53 changes: 0 additions & 53 deletions quest/m1/cross-node-flapping.md

This file was deleted.

61 changes: 52 additions & 9 deletions rs/moq-net/sim/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
//! Time never moves on its own. Once every task has stalled, [`run`] jumps the clock
//! to the earliest armed timer and wakes it, like tokio's paused clock, so a test
//! never waits on the wall clock. A run whose tasks are all parked with nothing
//! armed can never finish, so it panics rather than hang.
//! armed can never finish, so it panics rather than hang, unless a task awaits
//! [`idle`], which completes at exactly that point.
//!
//! Protocol drivers take time from their caller: [`drive`] polls one with the
//! simulated instant, and [`attach`] keeps an externally owned clock in step.

use std::{
cell::RefCell,
cell::{Cell, RefCell},
collections::BTreeMap,
future::{Future, IntoFuture},
pin::{Pin, pin},
Expand Down Expand Up @@ -41,6 +42,8 @@ struct Runtime {
spawned: RefCell<Vec<Task>>,
time: RefCell<Time>,
sources: RefCell<Vec<Source>>,
/// Awaiting [`idle`]: each flag is raised, and its waker woken, once nothing else can run.
idle: RefCell<Vec<(Rc<Cell<bool>>, Waker)>>,
}

struct Time {
Expand Down Expand Up @@ -135,7 +138,7 @@ impl Drop for Enter {
/// # Panics
///
/// When called inside another `run`, and when every task is parked with no timer
/// armed, which is a deadlock.
/// armed and none awaits [`idle`], which is a deadlock.
pub fn run<F: IntoFuture>(future: F) -> F::Output {
let runtime = Rc::new(Runtime {
spawned: RefCell::new(Vec::new()),
Expand All @@ -145,6 +148,7 @@ pub fn run<F: IntoFuture>(future: F) -> F::Output {
timers: BTreeMap::new(),
}),
sources: RefCell::new(Vec::new()),
idle: RefCell::new(Vec::new()),
});
CURRENT.with(|current| {
let mut current = current.borrow_mut();
Expand Down Expand Up @@ -177,17 +181,23 @@ pub fn run<F: IntoFuture>(future: F) -> F::Output {
if root.0.load(Ordering::SeqCst) || pool.0.load(Ordering::SeqCst) || !runtime.spawned.borrow().is_empty() {
continue;
}
assert!(
runtime.advance(),
"deadlock: every task is parked and no timer is armed"
);
if runtime.advance() {
continue;
}
let idle = runtime.idle.take();
assert!(!idle.is_empty(), "deadlock: every task is parked and no timer is armed");
for (flag, waker) in idle {
flag.set(true);
waker.wake();
}
};

drop(tasks);
drop(runtime.spawned.take());
drop(enter);
// Sources and timers may hold wakers into the dropped tasks.
// Sources, timers and idle waiters may hold wakers into the dropped tasks.
runtime.sources.borrow_mut().clear();
runtime.idle.borrow_mut().clear();
runtime.time.borrow_mut().timers.clear();
output
}
Expand Down Expand Up @@ -216,6 +226,20 @@ pub async fn advance(duration: Duration) {
yield_now().await;
}

/// Complete once every other task is parked and no timer or source is armed, so
/// nothing can happen until something outside the run acts: the system has settled.
pub fn idle() -> impl Future<Output = ()> {
let runtime = current();
let settled = Rc::new(Cell::new(false));
std::future::poll_fn(move |cx| {
if settled.get() {
return Poll::Ready(());
}
runtime.idle.borrow_mut().push((settled.clone(), cx.waker().clone()));
Poll::Pending
})
}

/// Yield once, letting every other ready task run first.
pub fn yield_now() -> impl Future<Output = ()> {
let mut yielded = false;
Expand Down Expand Up @@ -509,7 +533,8 @@ mod tests {
use std::cell::Cell;

use super::{
Duration, Elapsed, Poll, Rc, RefCell, Waker, advance, attach, drive, now, run, sleep, spawn, timeout, yield_now,
Duration, Elapsed, Poll, Rc, RefCell, Waker, advance, attach, drive, idle, now, run, sleep, spawn, timeout,
yield_now,
};

#[test]
Expand Down Expand Up @@ -617,6 +642,24 @@ mod tests {
run(std::future::pending::<()>());
}

#[test]
fn idle_waits_for_every_timer() {
run(async {
let start = now();
let fired = Rc::new(Cell::new(0));
for secs in [1, 5, 3] {
let fired = fired.clone();
drop(spawn(async move {
sleep(Duration::from_secs(secs)).await;
fired.set(fired.get() + 1);
}));
}
idle().await;
assert_eq!(fired.get(), 3);
assert_eq!(now(), start + Duration::from_secs(5));
});
}

#[test]
fn attached_sources_advance_with_time() {
run(async {
Expand Down
Loading
Loading