Skip to content

The media-aware TS lane and MSFTS ES-level carriage are not the same mode: six decisions #3731

Open

Description

@t0ms

Summary

draft-gregoire-moq-msfts has been restructured and now defines three named carriage modes for
MPEG-2 Transport Streams over MoQ. The third of them — "modified ES-level carriage", §5.5.3 — reads
as an attempt to describe a media-aware lane of the kind this repository implements. Read against
origin/main, it describes a different design, and the differences are structural rather than
cosmetic.

That matters now rather than later, because the draft is where this gets settled for anyone's
implementation, and because a subscriber cannot be written to satisfy both descriptions at once. This
issue sets out the divergence as I read it from the code, then asks for decisions. It does not
propose a patch, and it does not assume the draft is right where the two differ — on three of the six
points below I think the divergence favours what is implemented here, and would argue for the draft
moving rather than the code.

Draft text: https://mondain.github.io/msfts/draft-gregoire-moq-msfts.html

What the draft says mode 3 is

§5.5.3 When m2tsEsPid is present, the track carries a single elementary stream or signaling
table. The track payload contains only TS packets for the PID identified by m2tsEsPid; PAT, PMT,
and null packets are not included.

and

§5.5.3 A publisher producing multiple ES-level tracks for the same program MUST align Group
boundaries across all those tracks so that matching Group numbers correspond to the same
presentation position.

and, from §8 (excerpted — the paragraph opens with a requirement to align on the same starting Group
number across all subscribed ES-level tracks, which is the subject of point 4 below):

§8 The subscriber constructs the combined TS output by building a PAT listing the carried
program and a PMT listing the PIDs of all subscribed ES-level tracks, then interleaving packets
from all tracks. PCR is sourced from the track where m2tsPcrPid equals m2tsEsPid.

So: the payload unit is a 188-octet TS packet, filtered to one PID; continuity counters,
adaptation fields and PES framing travel inside those packets; the timing reference travels with the
media, in the PCR-bearing track; and group numbers are a cross-track synchronisation primitive.

What origin/main does

Read from rs/moq-mux/src/container/ts/{import,export,catalog}.rs:

MSFTS §5.5.3 moq-dev on origin/main
Track payload 188-octet TS packets, one PID Decoded access units — N.avc3 length-prefixed NALUs, N.aac ADTS frames; reassembled PES payloads (N.ts) for undecoded ES; complete sections for SCTE-35
Continuity counters preserved inside the packets observed for resync, then discarded; regenerated at export
PAT / PMT out of band in initDataList; subscriber constructs a PMT regenerated at export from catalog identity, re-emitted at keyframes and every 500 ms
PCR carried in the track where m2tsPcrPid == m2tsEsPid not carried. Export synthesises a uniform 25 ms grid; source PCR PID survives as an identifier
Group boundaries MUST align across ES tracks at one presentation position video cuts at keyframes; audio cuts every frame; verbatim PES per PES; sections per section
SI tables separate tracks, role: nit/sdt/eit/tdt NIT and SDT as opaque section sets in the catalog (mpegts.si, with an interval); EIT and TDT/TOT not carried
Mux rate m2tsMuxRate, though it MUST be absent in this mode no mux-rate concept in the TS code
Catalog MSF, packaging: "m2ts" Hang catalog with a typed mpegts extension; moq-msf's Packaging enum has no m2ts
MPTS mode 1 with m2tsMpts: true first non-zero PAT programme only; export rebuilds single-programme PSI

So this is not mode 3. It is a fourth mode the draft does not have: access-unit carriage with
transport-stream re-synthesis at egress.
m2tsPacketSize and m2tsPacketsPerObject have no
meaning for these tracks, because there are no source packets in the payload.

The view I hold, so it is on the record before the questions

I think the implementation's choice is the right one for a media-aware lane, and that the draft's
mode 3 sits between two coherent designs without committing to either.

Filtered-TS carriage preserves counters, adaptation fields and PES framing exactly, and it is a PID
filter rather than a demuxer, so it carries components the publisher does not understand. That is
real, and it is the honest form of "select the components you want".

But it buys selective component subscription, not media awareness, while taking on the full cost
of per-ES carriage: cross-track group alignment, PAT and PMT reconstruction, PCR sourced from a
different track, and re-interleaving N independently-flowing tracks into one multiplex that still has
to satisfy the T-STD. Access-unit carriage takes that same complexity and gets something for it —
group boundaries land on an IDR by construction, so a join is decodable without scanning; objects are
frame-sized, so priority and shedding are per frame; codec configuration reaches the catalog, so a
subscriber initialises without parsing TS.

The price of the implementation's choice is exactly what it regenerates, and it is not free. In our
own measurements the ungroomed egress of this lane carries no stuffing, thins PAT repetition from
8.04/s to 2.51/s, and puts 8.19 % of PCR intervals above 40 ms with a worst of 319.94 ms, against a
source with none above 40 ms in 600 s — cross-host, wire domain, eab960192. With a rate-controlled
egress in front of it the same lane measures 0 of 20,193 intervals above 40 ms over 300 s, and holds
that across a 24.01 h run. Nothing here has been graded on a hardware IRD, so those are software
conformance figures and not a lock claim.

None of that is an argument against the design. It is an argument that the format has to say the
egress must restore timing, which I have asked the draft for separately as
mondain/msfts#32.

The six decisions

Only you can make these, and a subscriber author needs every one of them answered.

  1. Is MSFTS compatibility an objective at all? Today the TS lane publishes a Hang catalog with an
    mpegts extension, and MSFTS extends the MSF catalog with packaging: "m2ts". These are two
    documents, not two dialects. If compatibility is wanted, the realistic first step is a published
    mapping — the mpegts section is a superset of MSFTS's per-track fields in information content —
    rather than a rewrite. If it is not wanted, that is a legitimate answer and the draft should stop
    implying otherwise.
  2. Does the payload unit converge, and in which direction? Access units or filtered TS packets.
    I think the draft should move, but this is the decision everything else hangs from.
  3. Does the timing reference travel with the media, or is it reconstructed at egress? The draft
    assumes the former; the exporter does the latter, on a synthetic 25 ms grid. A subscriber cannot
    do both, and a subscriber that expects source PCR and receives a regenerated grid will mis-measure
    drift against the source. Whichever it is, it needs to be stated in the format — a regular 25 ms
    grid is comfortably inside the 40 ms repetition limit and arguably better behaved than most
    sources, but that is only true if the receiver is told it is looking at a new clock.
  4. Do you accept §5.5.3's group-alignment MUST? I do not think you can, and I do not think you
    should. Audio cuts a group per frame here, which is a deliberate latency choice — one QUIC stream
    per frame, so a lost audio frame does not head-of-line-block the next — and aligning audio groups
    to video GOPs would cost up to a GOP of audio latency. The requirement is also unsatisfiable
    literally: a 1024-sample AAC frame at 48 kHz is 21.333 ms against a 40 ms video frame at 25 fps, so
    "the same presentation position" has no solution without splitting or padding, and the MUST sweeps
    in sparse signalling tracks such as SCTE-35 whose sections have no presentation time of their own.
    If you agree, saying so is what gets it restated as a presentation-time correspondence
    requirement instead.
  5. SI in the catalog, or SI as tracks? The catalog route has an advantage the draft has not
    considered: a joining subscriber gets SDT and NIT immediately, rather than waiting a repetition
    cycle, and mpegts.si's interval is the same idea as m2tsPsiInterval. I would argue for the
    draft permitting both and saying when to choose each. The counter-question is coverage: EIT and
    TDT/TOT are not carried on origin/main, and broadcast time is something an IRD wants — is the
    #2909 snapshot-track work intended for main, and is TDT/TOT in scope with it?
  6. Will the observed mux rate be recorded in the catalog? This is the smallest item and the one
    with the largest effect. Null stripping is a genuine gain — our reference multiplex is 4.57 %
    stuffing and this lane runs about 5.3 % below SRT on the same content, measured on a WAN path over
    60 s — and MSFTS §5.5.2 already anticipates exactly this case and provides m2tsMuxRate for it.
    Recording the rate observed at import in the mpegts section would make this lane's egress
    re-pacable by any generic groomer, with no catalog convergence needed at all.

I have put the same argument to the draft's authors as
mondain/msfts#33, so this is not being raised in only one venue.

What I am not asking for

Not a patch, not a commitment, and not a change before you have a position. If the answer to (1) is
"no", then (2) through (5) stop being convergence questions and become documentation questions, and
that is a perfectly good outcome — it just needs to be said out loud, because the draft currently
reads as though this implementation is an instance of its mode 3.


Activity

  1. kixelated commented on Sep 17, 2026

    @kixelated
    Collaborator

    I think the implementation's choice is the right one for a media-aware lane, and that the draft's
    mode 3 sits between two coherent designs without committing to either.

    I agree with mr AI. I don't agree with the msfts approach because it's the worst of both worlds IMO. You have all of the overhead of TS and it's still a separate format that players must explicitly support.

    Is MSFTS compatibility an objective at all?

    We should try to converge. My goal is to convert TS to/from MoQ, not to transport a legacy protocol verbatim. It's the same reason why I'm not making RTP over MoQ or RTMP over MoQ.

    Does the timing reference travel with the media, or is it reconstructed at egress?

    I think it's kind of dumb to transmit PCR over the network, since it's intended for hardware decoders that don't have their own clocks? But IDK it doesn't really matter.

    Do you accept §5.5.3's group-alignment MUST?

    I don't like forcing group alignment. Use timestamps for synchronization, not group IDs IMO. But it can be optional.

    SI in the catalog, or SI as tracks?

    IDK.

    Will the observed mux rate be recorded in the catalog?

    I'd be okay putting the mux rate in the catalog and we can generate padding on export. I feel like we can just report it as maxBitrate or something on import, and add an option on export to pad to that value. Or frankly just always pad I guess.

  2. t0ms commented on Sep 18, 2026

    @t0ms
    ContributorAuthor

    The demux/remux mode feels more elegant for sure, my concern is the differences between the msfts demux mode and your demux mode which aren't currently interoperable. It's probably worth pausing on this until the latest revision of msfts is posted, which I understand from Gwendal will be in the next couple of weeks. Then, we can re-assess how far the two demux/remux approaches are.

    Regarding the opaque (aka transparent) mode, we conceded we would not pursue further unless we can prove a definitive industry need. I have found 1 reason why this mode might have a purpose: MOQ secures transport from publishers to subscribers but not upstream and downsteam from those. An org with contractual/licensing requirements to protect content end-to-end might want an additional content protection layer such as BISS-CA, which would ensure content security compliance upstream and downstream. A couple of vendors I spoke to at IBC have suggested layering BISS-CA, but BISS-CA would not work with the demux/remux mode (afaik). Having said that, I'm not convinced yet BISS-CA is a must-have requirement, as broadcasters' internet native distribution workflows leveraging Zixi/SRT do not layer BISS-CA on top.

  3. added 2 commits that reference this issue on Sep 18, 2026
  4. kixelated commented on Sep 21, 2026

    @kixelated
    Collaborator

    The demux/remux mode feels more elegant for sure, my concern is the differences between the msfts demux mode and your demux mode which aren't currently interoperable. It's probably worth pausing on this until the latest revision of msfts is posted, which I understand from Gwendal will be in the next couple of weeks. Then, we can re-assess how far the two demux/remux approaches are.

    Regarding the opaque (aka transparent) mode, we conceded we would not pursue further unless we can prove a definitive industry need. I have found 1 reason why this mode might have a purpose: MOQ secures transport from publishers to subscribers but not upstream and downsteam from those. An org with contractual/licensing requirements to protect content end-to-end might want an additional content protection layer such as BISS-CA, which would ensure content security compliance upstream and downstream. A couple of vendors I spoke to at IBC have suggested layering BISS-CA, but BISS-CA would not work with the demux/remux mode (afaik). Having said that, I'm not convinced yet BISS-CA is a must-have requirement, as broadcasters' internet native distribution workflows leveraging Zixi/SRT do not layer BISS-CA on top.

    Yeah, there's no right answer here. It's transmuxing vs proxying. It's a similar analogy with RoQ (RTP) versus MoQ.

    Preserving a legacy format verbatim is easy and provides maximum compatibility, but in this case it costs you 5-20% bitrate. My approach is designed to work over the internet while still preserving elementary streams. It's cheaper for you and works with stock MSF/hang players.

    But Gwendal knows waaaay more about TS than me and I may be in Dunning-Kruger territory. There might just be some unsolvable problem, like PCR/padding, but I feel like we're close.

    Zooming out, I reaaaaallly don't want to support an inefficient proxy mode for every legacy format (ex. RTP, FLV, MKV, MP4, etc). I think transmuxing is the long-term solution and if we can make it good/reliable enough, the short-term solution too.

  5. t0ms commented on Sep 22, 2026

    @t0ms
    ContributorAuthor

    The MSFTS revision has landed (mondain/msfts#35), so
    here is the re-assessment I said I'd bring back. Short version: four of the six are resolved,
    three of them in the direction you argued for, and the one structural disagreement is now the only
    one left open.

    Resolved

    (1) Is compatibility an objective? Both documents now scope themselves explicitly, and
    compatibly. MSFTS's Scope section is "carry an MPEG-2 Transport Stream over MOQT without changing
    the transport stream itself
    ", with a relay that propagates "without parsing". Your draft opens by
    naming the same split — MSFTS is the intact-multiplex shape, draft-lcurley-moq-mpegts is the
    demultiplexed one. A subscriber author can now read either and know which they are implementing,
    which is the thing that was genuinely broken when I filed this.

    (4) Group alignment. The MUST is now a SHOULD, and the draft adds "elementary streams have
    different frame durations, so exact alignment is not always possible". That is your position, and
    it removes the requirement that was unsatisfiable for per-frame audio groups.

    (6) Mux rate. Both sides landed it in the same week, and they agree on the semantics: it is a
    stuffing target and not a timing source, counted over 188-octet packets, declared when the
    publisher has removed null packets. MSFTS also dropped the prohibition that made
    m2tsMuxRate absent in the one mode that needed it, and now recommends it on ES-level tracks.
    mpegts.muxRate from #3831 is the same quantity under a different name.

    (3) Egress timing — resolved as a venue question, and it names the residue rather than closing
    it.
    The revision adds a normative Egress Timing section that says what you and I were circling:
    delivery-schedule conformance is a property of how the subscriber hands packets to its receiver,
    not of the carriage. It puts the obligation there, with {{TR101290}} for a DVB output and
    mpeg2tsMuxRate as the stuffing target. So a regenerated grid is fine, provided the egress delivers
    on a schedule the values describe — which is the answer I think we both wanted.

    Worth reading its last two sentences closely, though, because they are the honest part and they bear
    on your side rather than on MSFTS's: "A stuffing target does not reproduce the source schedule.
    Reproducing it requires timing information that this document does not define."
    That is exactly
    right, and it is the gap mpegts.muxRate has too — padding to a rate makes the total correct
    without making the byte positions correct, and a receiver clocking off packet arrival grades the
    positions. I've measured the size of that gap on export ts and it is large; I'm keeping the
    numbers out of this thread because they belong in
    #2829 and a successor to
    #3334 rather than in a design-convergence
    discussion. Flagging it only because the revision has now written down, normatively, the one thing
    neither document currently specifies.

    Still open, and it is the one you'd expect

    (2) The payload unit. Unchanged on both sides: MSFTS ES-level carriage is filtered 188-octet
    packets, your lane is access units. This is the real disagreement, it is tracked at
    msfts#33, and Gwendal is evaluating it — so I'm
    reporting it rather than pushing it.

    (5) SI has moved in a way worth flagging rather than deciding. MSFTS dropped the
    m2tsPsiInterval catalog field and made repetition a subscriber obligation instead. Your per-
    table_id interval is now a divergence rather than an overlap, and I think yours is the better
    granularity, because one PID carries tables with different repetition requirements.

  6. added
    questBeing tracked/planned in a quest. See `quest/`
    on Sep 23, 2026
  7. kixelated commented on Sep 28, 2026

    @kixelated
    Collaborator

    Summary of the planning decision (#3987):

    • Keep it as a deferred quest instead of closing. Four of the six divergences are settled (scope, group alignment as a SHOULD, mux rate, and egress timing).
    • Waiting on MSFTS. The payload unit (access units, or filtered 188-byte packets) is MSFTS's call at What is ES-level carriage for? As drafted it takes the costs of two designs and the benefits of neither mondain/msfts#33, which is still open. Once it settles, we'll decide what converges: a published mapping from the mpegts catalog section to the m2ts fields, or a change on either side. Then we'll update draft-lcurley-moq-mpegts and the concept docs.
    • Non-goal: carrying TS verbatim.
    • Status: tracked in quest/m4/msfts-convergence.md. This issue was closed when the plan merged, not when the work was done.

    (Written by Claude Opus 5.5)

  8. t0ms commented on Oct 16, 2026

    @t0ms
    ContributorAuthor

    A data point for msfts-convergence, from running OpenMOQ's moq5 (0729c49: an MSFTS sender, receiver and relay) against 3c0daa21b on drafts 16, 18 and 22, raw QUIC, loopback.

    The relays carry each other's tracks; the catalogs are where the two stop. moq5's sender through moq-relay to moq5's receiver delivers every elementary-stream PID byte-exact on all three drafts, and a demultiplexed moq import ts through moq5-relay to moq export ts is content-exact. Every arm that needs one side to find the other's TS track through its catalog fails at discovery, in both directions:

    • Demultiplexed lane to an MSFTS subscriber. The MSF catalog lists 0.avc3 and 6.mp2 with legacy packaging. moq5 does not parse legacy, so it subscribes to neither. This is the quest's Goal seen from the subscriber's side: an MSFTS implementation finds nothing in the MSF catalog it can map today.
    • Passthrough to an MSFTS subscriber. The MSF catalog is empty and the track is named only in the hang catalog's m2ts member. That is the TS draft as written ("An MSF catalog does not list the m2ts track itself"); the converter it describes, which strips the timestamps and republishes with packaging: mpeg2ts, does not exist yet, so moq5 has nothing to subscribe to.
    • MSFTS to moq export ts. The export reads moq5's catalog only under a .msf broadcast name (otherwise catalog.json is refused not found), then logs skipping MSF video track with unsupported packaging track=ts packaging=mpeg2ts and waits on the catalog until the broadcast ends.

    So the passthrough mapping the TS draft defines runs one way on paper and is implemented in neither direction. Is a converter in either direction planned, and if so, would it sit under this quest or a sibling of ts-passthrough-export? Happy to rerun the matrix against whatever lands.

    (Written by Claude Opus 5.5)

  9. t0ms commented on Nov 10, 2026

    @t0ms
    ContributorAuthor

    A named consumer, should msfts-convergence be re-planned (it was pruned in #8423 for want of one): OpenMOQ's moq2ts, which carries MPEG-TS under MSF and MSFTS, publishing through moqxr and receiving on moq5, and is under active development. Its catalog declares each program as a track with "packaging": "mpeg2ts" and "mpeg2tsMode": "unmodified-program", beside a mediatimeline track.

    Where the two stand on main (cd560bde5e), by reading the code:

    • moq-mux's MSF consumer (rs/moq-mux/src/catalog/msf/consumer.rs) maps only LOC, legacy and CMAF tracks; any other audio or video packaging is skipped with a warning. moq_msf::Packaging has no mpeg2ts value, so moq2ts's track parses as Unknown and is skipped.
    • moq import ts --passthrough describes the whole multiplex only in the Hang catalog's m2ts section, which an MSF reader does not look at.

    So neither finds the other's transport stream through MSF, which is where the October data point above stopped for moq5 too. A re-plan could take as its acceptance the two directions: moq export ts receiving a moq2ts unmodified-program track byte-exact, and moq2ts's receiver finding and receiving moq import ts --passthrough's track through the MSF catalog. We can run either direction against a branch.

    (Written by Claude Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questBeing tracked/planned in a quest. See `quest/`

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions