From 64d216ff7ca904c8c7009a1507eade1dd241440c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 18:04:19 -0800 Subject: [PATCH 1/4] chore(quest): claim quest/m2/unbounded-groups/run-fill-pacing.md Quest-Claim: 631390826512d3b7 From d1f4ae2d6eb8a82aa172f01444775a06101b928d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 18:45:14 -0800 Subject: [PATCH 2/4] feat(net): pace a moq-transport FETCH run once a reader lost a group to eviction A relay filling a run of groups with one FETCH wrote the whole run at the stream's speed, so on a cache smaller than the run a slow reader lost most of it. Once a reader of the track lost a group it judged cached, the run now holds its fetch stream unread at each group boundary until a reader picked up the group before, through the session's pause budget, so flow control holds the publisher back. The quest is rewritten to the lite-07 remainder, which needs a maintainer decision on its mechanism. Co-Authored-By: Claude Opus 5.5 --- quest/m2/unbounded-groups/README.md | 2 +- quest/m2/unbounded-groups/run-fill-pacing.md | 68 +++++++-- rs/moq-net/src/coding/reader.rs | 7 +- rs/moq-net/src/ietf/subscriber.rs | 151 +++++++++++++++++-- rs/moq-net/src/lite/test_transport.rs | 14 ++ rs/moq-net/src/model/track/fill.rs | 11 ++ 6 files changed, 221 insertions(+), 32 deletions(-) diff --git a/quest/m2/unbounded-groups/README.md b/quest/m2/unbounded-groups/README.md index 5e8ba9843fe..25122dceb96 100644 --- a/quest/m2/unbounded-groups/README.md +++ b/quest/m2/unbounded-groups/README.md @@ -183,4 +183,4 @@ the model layer (maintainer, 2026-10-30). - [Byte holes](/quest/m2/unbounded-groups/byte-holes.md) - segments inside a frame shed like front frames and flights start at `(frame, byte)` on the current lite wip; the frame cap goes - [JS byte holes](/quest/m2/unbounded-groups/js-byte-holes.md) - `@moq/net` mirrors the byte-offset wire and continues a frame cut off mid-way - [Compressed frames have no size cap and inflate as a bounded stream](/quest/m2/unbounded-groups/compressed-stream.md) - A compressed frame of any size publishes and is received in Rust and `@moq/net`, like a plain one -- [Run-based relay fills are paced](/quest/m2/unbounded-groups/run-fill-pacing.md) - FETCH runs and lite-07 range fills backpressure upstream once a reader loses a batch to eviction +- [Lite-07 history runs are paced](/quest/m2/unbounded-groups/run-fill-pacing.md) - range fills, recoveries, and a reader's own history range backpressure upstream once a reader loses a group to eviction diff --git a/quest/m2/unbounded-groups/run-fill-pacing.md b/quest/m2/unbounded-groups/run-fill-pacing.md index a248499dcf1..a508a069f4e 100644 --- a/quest/m2/unbounded-groups/run-fill-pacing.md +++ b/quest/m2/unbounded-groups/run-fill-pacing.md @@ -1,23 +1,67 @@ -# [M] Run-based relay fills are paced +# [M] Lite-07 history runs are paced ## Goal -A relay pacing a run-based fill (a moq-transport FETCH run, or a lite-07 -range fill) backpressures the upstream stream once it sees a downstream reader -lose a batch to eviction, so a slow reader of a long run stays within the -tail instead of falling off it. +A lite-07 copy bringing a long run of history to a slow reader (a range fill, a +recovery, or the reader's own history range on the copy's subscription) +backpressures the upstream once it sees a downstream reader lose a group to +eviction, so a slow reader of a long run stays within the tail instead of +falling off it. ## Plan Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`), from [moq-dev/moq#8017](https://github.com/moq-dev/moq/pull/8017), which paces a -batch fill's handler by its fastest reader. Run-based fills still read -upstream at full speed. +batch fill's handler by its fastest reader. Pace on evidence, not up front: an +unpaced run is fine while every reader keeps up. -Pace on evidence, not up front: an unpaced run is fine while every reader -keeps up, so start backpressure only after the relay sees a downstream -reader lose a batch to eviction, then hold the upstream stream as #8017 -holds a batch handler. Test a slow reader of a long FETCH run and of a -lite-07 range fill through a relay. +Landed in [moq-dev/moq#8117](https://github.com/moq-dev/moq/pull/8117): the +evidence is the track's eviction count (`TrackState::evictions`, bumped when a +reader loses a group it judged cached), and a moq-transport FETCH run holds its +fetch stream unread at each group boundary until a reader picked up the group +before (`Fill::poll_run_paced`), through the session's pause budget. + +What remains is lite-07, which that PR's prototypes found harder than the plan +assumed: + +- History reaches a lite-07 copy three ways, all unpaced: the copy's upstream + subscription carrying a reader's own history range (the common case, and not a + fill), `RangeFills` asking a held fill's run through that subscription, and a + `Recovery` subscription asking again for what the subscription already + accounted for. Each arrives as one group stream per group, concurrently. +- Holding each arriving group stream unread until the fill's readers picked up + the one before admits streams in arrival order, not the fill's, so groups a + reader takes last evict each other first. A slow reader of 40 groups on a + 4-group cache still got 10. +- Fetching a paced fill a group at a time instead (lite's `FillServeRun`, paced + since #8017) got that reader 26 of 40, but costs a round trip per group, which + #8017 and [moq-dev/moq#7339](https://github.com/moq-dev/moq/pull/7339) + rejected, and paces neither the subscription's own history range nor several + fills at once. +- #8017's per-fill pacing does not hold when one reader has several fills out, + split by groups already cached or by refills: each fill moves on as the reader + picks up its answer, so together they still outrun it. This affects batch + handlers and FETCH runs too. + +The mock transport moves a whole run in an instant, before any reader polls, so +a test needs the evidence first: one slow reader loses part of a run, then a +second slow reader of another long run gets all of it. Public API: none expected. Wire: none. + +## Waiting + +- Maintainer: pick lite-07's pacing mechanism. (a) Hold arriving group streams + unread and admit them in the fill's order, within the session's pause budget, + extended to the history range a reader's subscription carries: no round trips, + but out-of-order streams spend the pause budget and read on unpaced once it + runs out. (b) Narrow the subscription's ranges to the next owed group once + paced: in-order, but a SUBSCRIBE_UPDATE and a round trip per group. (c) Fetch + paced runs a group at a time: reuses `FillServeRun`, but a round trip per group + and the subscription's own range stays unpaced. (d) Drop lite-07 pacing and + rely on refill-once. Recommended: (a), since it is the plan's "hold the + upstream stream" without round trips; pair it with pacing per reader across + its fills (a read picks up one answer per group it takes, all of them while it + waits), so several fills of one reader cannot outrun it. An agent cannot take + this: (b) and (c) reopen #7339's rejection, and the per-reader change rewrites + #8017's settled pick-up rule. diff --git a/rs/moq-net/src/coding/reader.rs b/rs/moq-net/src/coding/reader.rs index c4e343f17df..6c3485637ff 100644 --- a/rs/moq-net/src/coding/reader.rs +++ b/rs/moq-net/src/coding/reader.rs @@ -474,12 +474,13 @@ impl Reader { self.poll_pace(waiter, group.poll_admit(waiter, header, self.stream.window())) } - /// Wait out `paced`, the group's pace gate, if the session has room for the credit this - /// stream holds while paused (see [`Pause`]); otherwise read on, and the group sheds. + /// Wait out `paced`, a pace gate on what this stream writes next, if the session has room + /// for the credit this stream holds while paused (see [`Pause`]); otherwise read on, + /// unpaced. /// /// A paused stream is not read, so it is watched for a reset instead, which fails the /// wait rather than leaving it to the group's reader to notice. - fn poll_pace(&mut self, waiter: &kio::Waiter, paced: Poll<()>) -> Poll> + pub(crate) fn poll_pace(&mut self, waiter: &kio::Waiter, paced: Poll<()>) -> Poll> where S: Pause, { diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index f384780a246..05902b539e0 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -7528,6 +7528,12 @@ impl RecvGroupFetch { /// properties, never past FETCH_OK's End Location. A group's objects end where the next /// group's begin, since a FETCH carries every object of its range in order. Before /// draft-22 the stream must also reach an object bound inside the last group. +/// +/// Once a reader of the track lost a group to eviction, each group after the first waits, +/// its stream unread, until a reader picked up the one before, so a run wider than the +/// cache never evicts its own groups ahead of its readers (see [`track::Fill::poll_run_paced`]). +/// The pause holds receive credit, so it is taken only while the session's paused streams +/// leave room for it, as within a group. struct GroupFetchObjects { version: Version, /// The next Object ID expected in the current group. @@ -7536,6 +7542,9 @@ struct GroupFetchObjects { first: bool, /// The object whose payload is being read: its group, its Object ID, and where it goes. payload: Option<(u64, u64, FetchPayloadRead)>, + /// The first object of the next group and its sequence, read while the run waits for its + /// readers to pick up the group before. + paced: Option<(FetchedObject, u64)>, } /// Where a fetch object's payload goes. @@ -7557,6 +7566,7 @@ impl GroupFetchObjects { prior_group: None, first: true, payload: None, + paced: None, } } @@ -7585,22 +7595,38 @@ impl GroupFetchObjects { } } - let Some(object) = ready!(poll_fetch_object(stream, self.first, waiter))? else { - break; - }; - self.first = false; - if !object.subgroup_ok && !object.datagram { - tracing::warn!("subgroup ID is not supported, dropping group fetch"); - return Poll::Ready(Err(Error::Unsupported)); - } + let (object, sequence) = match self.paced.take() { + Some(paced) => paced, + None => { + let Some(object) = ready!(poll_fetch_object(stream, self.first, waiter))? else { + break; + }; + self.first = false; + if !object.subgroup_ok && !object.datagram { + tracing::warn!("subgroup ID is not supported, dropping group fetch"); + return Poll::Ready(Err(Error::Unsupported)); + } - let group = resolve_fetch_group(self.prior_group, object.group)?; - if let Some(group) = group { - self.prior_group = Some(group); - } - // The first object always names its group, so a later one leaving it out stays in it. - let sequence = group.or(writer.sequence()).ok_or(Error::ProtocolViolation)?; + let group = resolve_fetch_group(self.prior_group, object.group)?; + if let Some(group) = group { + self.prior_group = Some(group); + } + // The first object always names its group, so a later one leaving it out stays in it. + let sequence = group.or(writer.sequence()).ok_or(Error::ProtocolViolation)?; + (object, sequence) + } + }; if writer.sequence() != Some(sequence) { + // The group before is whole, so its readers see it end while the stream waits, + // unread, for them to pick it up: flow control then holds the publisher back. + writer.finish()?; + match stream.poll_pace(waiter, writer.fetch.fill.poll_run_paced(waiter)) { + Poll::Ready(res) => res?, + Poll::Pending => { + self.paced = Some((object, sequence)); + return Poll::Pending; + } + } self.next = writer.open(sequence, object.datagram)?; } else if object.datagram { // A datagram group has one object, so it never shares one with a stream. @@ -17115,6 +17141,97 @@ mod stitch_tests { assert!(matches!(receive.await, Err(Error::Evicted))); } + /// What a slow reader of a FETCH run ten times wider than the cache gets, taking a group + /// each time the run's stream is polled, once a reader of the track lost a group to + /// eviction or not. + async fn read_a_fetch_run_slowly(outran: bool) -> Vec { + const GROUPS: u64 = 40; + const SIZE: usize = 10_000; + let payload = vec![0u8; SIZE]; + let mut objects = Vec::new(); + // The first Group ID is 0, and each later one, a delta, 0 again: the next group. + for _ in 0..GROUPS { + objects.extend(group_fetch_objects(0, 0, &[&payload[..]])); + } + // The session grants a stream credit of one group, so a paused stream holds one. + let window = crate::transport::ReceiveWindow::new(GROUPS * SIZE as u64, SIZE as u64); + let session = ScriptedSession::per_stream_eof(vec![objects]).with_receive_window(window); + let mut run = GroupFetchRun::with_session(VERSION, session).await; + let pool = crate::cache::Pool::new(crate::cache::Config::default().with_capacity(4 * SIZE as u64)); + run.track = track::Producer::new( + std::sync::Arc::new(crate::broadcast::Info { + pool, + ..Default::default() + }), + "video", + track::Info::default().with_timescale(None).with_max_age(None), + ); + let dynamic = run.track.requests(); + run.track.start_at(GROUPS).unwrap(); + let mut live = run.track.create_group(group::Info { sequence: GROUPS }).unwrap(); + live.write_frame(None, &payload[..]).unwrap(); + let waiter = kio::Waiter::noop(); + if outran { + // A reader judges the live edge cached and loses it before taking it. + let mut lost = run.track.subscribe( + track::Subscription::default() + .with_ranges([track::Range::groups(0..GROUPS + 1)]) + .with_delay(track::Delay::default().with_max(None)), + ) + .ordered(); + assert!(lost.poll_next_group(&waiter).is_pending()); + run.track.evict(GROUPS); + } + + let mut reader = run.track.subscribe( + track::Subscription::default() + .with_ranges([track::Range::groups(0..GROUPS)]) + .with_order(track::Order::Ascending) + .with_delay(track::Delay::default().with_max(None)), + ) + .ordered(); + assert!(reader.poll_next_group(&waiter).is_pending(), "group 0 is a miss"); + let fill = dynamic.requested_fill().await.expect("the reader asked for the run"); + assert_eq!((fill.start().group, fill.end()), (0, GROUPS)); + let slot = kio::Producer::new(GroupFetch::Ready(RangeFetch { + fill: std::sync::Arc::new(fill), + timescale: None, + first: 0, + last: GROUPS - 1, + start: 0, + end: None, + })); + let mut receiving = run.subscriber.clone(); + let mut receive = std::pin::pin!(receiving.recv_group_fetch(&mut run.stream, slot)); + let (mut read, mut received) = (Vec::new(), false); + for _ in 0..4 * GROUPS { + if !received && let Poll::Ready(res) = futures::poll!(&mut receive) { + res.expect("the run is received"); + received = true; + } + match reader.poll_next_group(&waiter) { + Poll::Ready(Ok(Some(group))) => read.push(group.sequence), + // What it lost is given up, which ends its range. + Poll::Ready(Ok(None)) => break, + Poll::Ready(Err(err)) => panic!("the reader failed: {err}"), + Poll::Pending if received => break, + Poll::Pending => {} + } + } + read + } + + /// Once a reader of the track lost a group to eviction, a FETCH run writes its next group + /// only once a reader picked up the one before, leaving the stream unread meanwhile, so a + /// slow reader of a run far wider than the cache gets all of it. Until then the run moves + /// at the stream's speed and evicts what the reader has yet to take. + #[kio::test] + async fn a_fetch_run_waits_for_its_reader_once_one_lost_a_group() { + assert_eq!(read_a_fetch_run_slowly(true).await, (0..40).collect::>()); + let unpaced = read_a_fetch_run_slowly(false).await; + assert!(unpaced.len() < 40, "the unpaced run outran its reader: {unpaced:?}"); + } + #[kio::test] async fn internal_group_fetch_setup_cancellation_is_cancel() { let mut run = GroupFetchRun::new(VERSION, Vec::new()).await; @@ -17160,7 +17277,9 @@ mod stitch_tests { tasks.0.clone(), Default::default(), ); - let (_, recv) = session.open_bi(Default::default()).await.unwrap(); + // Through the subscriber's transport, so it pauses as a session's stream does where + // the session grants a receive window. + let stream = Stream::open(&subscriber.session, version).await.unwrap(); let track = track::Producer::new( std::sync::Arc::new(crate::broadcast::Info::default()), "video", @@ -17169,7 +17288,7 @@ mod stitch_tests { Self { subscriber, - stream: Reader::new(crate::transport::boxed::RecvStream::new(recv), version), + stream: stream.reader, track, _tasks: tasks, } diff --git a/rs/moq-net/src/lite/test_transport.rs b/rs/moq-net/src/lite/test_transport.rs index 3480c8db5ed..8f9adc42d0a 100644 --- a/rs/moq-net/src/lite/test_transport.rs +++ b/rs/moq-net/src/lite/test_transport.rs @@ -974,6 +974,8 @@ pub struct ScriptedSession { groups: bool, /// Set by [`Self::with_send_window`]. send_window: Option, + /// Set by [`Self::with_receive_window`]. + receive_window: Option, } impl ScriptedSession { @@ -1014,9 +1016,17 @@ impl ScriptedSession { unacked_unis: Default::default(), groups: false, send_window: None, + receive_window: None, } } + /// Report `window` as the receive windows in force, so a stream the session stops reading + /// holds credit. + pub fn with_receive_window(mut self, window: crate::transport::ReceiveWindow) -> Self { + self.receive_window = Some(window); + self + } + /// Let each opened bidi stream send only its first `bytes`, then park its writes for good, /// like a peer that never grants more send credit. pub fn with_send_window(mut self, bytes: usize) -> Self { @@ -1147,6 +1157,10 @@ impl poll::Session for ScriptedSession { type RecvStream = ScriptedRecv; type Error = SinkError; + fn receive_window(&self) -> Option { + self.receive_window + } + fn poll_accept_uni(&self, waiter: &kio::Waiter) -> Poll> { let Some(script) = self.incoming_unis.lock().unwrap().pop_front() else { return self.log.poll_closed(waiter).map(Err); diff --git a/rs/moq-net/src/model/track/fill.rs b/rs/moq-net/src/model/track/fill.rs index 39b8b64a720..802bdaf7c0f 100644 --- a/rs/moq-net/src/model/track/fill.rs +++ b/rs/moq-net/src/model/track/fill.rs @@ -1060,6 +1060,17 @@ impl Fill { .map(drop) } + /// Poll until a run's next group may be written: at once until a reader of the track lost + /// a group to eviction, then as [`Self::poll_paced`]. An unpaced run moves at its source's + /// speed, which is fine while every reader keeps up. + #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] + pub(crate) fn poll_run_paced(&self, waiter: &kio::Waiter) -> Poll<()> { + if self.inner.state.read().evictions == 0 { + return Poll::Ready(()); + } + self.poll_paced(waiter) + } + /// Refuse every group still owed with `err`, which says nothing about whether the source /// has them, so nothing is recorded dropped. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] From 872e887f3ab5ce4a66a64646258c3cf5fe0bc743 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 18:47:09 -0800 Subject: [PATCH 3/4] style(net): rustfmt the FETCH run pacing test Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/ietf/subscriber.rs | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 05902b539e0..39eaaac151e 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -17173,23 +17173,27 @@ mod stitch_tests { let waiter = kio::Waiter::noop(); if outran { // A reader judges the live edge cached and loses it before taking it. - let mut lost = run.track.subscribe( - track::Subscription::default() - .with_ranges([track::Range::groups(0..GROUPS + 1)]) - .with_delay(track::Delay::default().with_max(None)), - ) - .ordered(); + let mut lost = run + .track + .subscribe( + track::Subscription::default() + .with_ranges([track::Range::groups(0..GROUPS + 1)]) + .with_delay(track::Delay::default().with_max(None)), + ) + .ordered(); assert!(lost.poll_next_group(&waiter).is_pending()); run.track.evict(GROUPS); } - let mut reader = run.track.subscribe( - track::Subscription::default() - .with_ranges([track::Range::groups(0..GROUPS)]) - .with_order(track::Order::Ascending) - .with_delay(track::Delay::default().with_max(None)), - ) - .ordered(); + let mut reader = run + .track + .subscribe( + track::Subscription::default() + .with_ranges([track::Range::groups(0..GROUPS)]) + .with_order(track::Order::Ascending) + .with_delay(track::Delay::default().with_max(None)), + ) + .ordered(); assert!(reader.poll_next_group(&waiter).is_pending(), "group 0 is a miss"); let fill = dynamic.requested_fill().await.expect("the reader asked for the run"); assert_eq!((fill.start().group, fill.end()), (0, GROUPS)); From 2e15d67dc75560b76deeb17abfb7c208f38d70b1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 20:55:51 -0800 Subject: [PATCH 4/4] test(net): a descending reader keeps up with a paced FETCH run too Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/ietf/subscriber.rs | 47 ++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 16 deletions(-) diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 39eaaac151e..136e906266e 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -17143,8 +17143,9 @@ mod stitch_tests { /// What a slow reader of a FETCH run ten times wider than the cache gets, taking a group /// each time the run's stream is polled, once a reader of the track lost a group to - /// eviction or not. - async fn read_a_fetch_run_slowly(outran: bool) -> Vec { + /// eviction or not. An ascending reader reads in sequence, a descending one as groups + /// arrive. + async fn read_a_fetch_run_slowly(outran: bool, order: track::Order) -> Vec { const GROUPS: u64 = 40; const SIZE: usize = 10_000; let payload = vec![0u8; SIZE]; @@ -17185,16 +17186,24 @@ mod stitch_tests { run.track.evict(GROUPS); } - let mut reader = run - .track - .subscribe( - track::Subscription::default() - .with_ranges([track::Range::groups(0..GROUPS)]) - .with_order(track::Order::Ascending) - .with_delay(track::Delay::default().with_max(None)), - ) - .ordered(); - assert!(reader.poll_next_group(&waiter).is_pending(), "group 0 is a miss"); + let subscriber = run.track.subscribe( + track::Subscription::default() + .with_ranges([track::Range::groups(0..GROUPS)]) + .with_order(order) + .with_delay(track::Delay::default().with_max(None)), + ); + type Read = Box Poll>>>; + let mut reader: Read = match order { + track::Order::Ascending => { + let mut ordered = subscriber.ordered(); + Box::new(move |waiter| ordered.poll_next_group(waiter)) + } + track::Order::Descending => { + let mut subscriber = subscriber; + Box::new(move |waiter| subscriber.poll_recv_group(waiter)) + } + }; + assert!(reader(&waiter).is_pending(), "group 0 is a miss"); let fill = dynamic.requested_fill().await.expect("the reader asked for the run"); assert_eq!((fill.start().group, fill.end()), (0, GROUPS)); let slot = kio::Producer::new(GroupFetch::Ready(RangeFetch { @@ -17213,7 +17222,7 @@ mod stitch_tests { res.expect("the run is received"); received = true; } - match reader.poll_next_group(&waiter) { + match reader(&waiter) { Poll::Ready(Ok(Some(group))) => read.push(group.sequence), // What it lost is given up, which ends its range. Poll::Ready(Ok(None)) => break, @@ -17231,9 +17240,15 @@ mod stitch_tests { /// at the stream's speed and evicts what the reader has yet to take. #[kio::test] async fn a_fetch_run_waits_for_its_reader_once_one_lost_a_group() { - assert_eq!(read_a_fetch_run_slowly(true).await, (0..40).collect::>()); - let unpaced = read_a_fetch_run_slowly(false).await; - assert!(unpaced.len() < 40, "the unpaced run outran its reader: {unpaced:?}"); + for order in [track::Order::Ascending, track::Order::Descending] { + let paced = read_a_fetch_run_slowly(true, order).await; + assert_eq!(paced, (0..40).collect::>(), "{order:?}"); + let unpaced = read_a_fetch_run_slowly(false, order).await; + assert!( + unpaced.len() < 40, + "the unpaced run outran its {order:?} reader: {unpaced:?}" + ); + } } #[kio::test]