Skip to content

fix(gst): preserve audio/video alignment in moqsrc - #5556

Merged
kixelated merged 4 commits into
mainfrom
quest/m1/gst-src-av-sync
Oct 15, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m1/gst-src-av-sync

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem
moqsrc resets each pad to its own first frame and arrival time, losing the broadcast's audio/video alignment. Different CMAF timescales also turn timestamp subtraction into zero.

Approach
Keep media PTS in nanoseconds and fix one segment start/base on the first frame of each broadcast run. Every pump uses that mapping, including replacement pumps; a new broadcast chooses a fresh one on the held pads. Frames before the reference remain before the segment for downstream clipping. Update the inline guide and retire the quest.

Impact

  • No exported API, GObject property, or wire-format change. Corrects the existing GStreamer PTS/segment mapping.
  • Playback uses the broadcast's relative timeline and does not require its optional wall clock.

Validation
The real-pad regression failed on the original implementation (PTS zero instead of 10 seconds). It now covers 16 runs across both pump orders, earlier/later siblings, held-pad restarts, Legacy and CMAF with 90 kHz video/48 kHz audio. just check passes all 184 GStreamer tests, lint, docs, and dependency checks. just test interop --all passes five protocol controls and 62 harness checks, including all media pairings, authorization denials, CLI FETCH, and finite tails.

Alternatives
The settled plan rejects catalog-wall-clock alignment because that clock is optional and assumes synchronized hosts. A shared segment start preserves early PTS for GStreamer to clip.

Follow-ups
Full interop remains blocked by the unchanged browser refusal control: expected close code 2 / unauthorized, received 0 / empty. #5550 independently records this same failure. Existing interop-close-code and interop-close-release quests track the prerequisite; no retries, grace changes, or unpublished transport pin are included. No dependency on the complementary sink wall-epoch fix #5554.

(Written by GPT-6)

kixelated and others added 3 commits October 14, 2026 18:16
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated
kixelated marked this pull request as ready for review October 15, 2026 15:44
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T15:50:53.948137Z f866e6a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d15734b9-1441-46f6-9680-debab2d87d45
📥 Commits

Reviewing files that changed from the base of the PR and between 5beead6 and 741d683.

📒 Files selected for processing: 4
  • doc/bin/gstreamer.md
  • quest/m1/README.md
  • quest/m1/gst-src-av-sync.md
  • rs/moq-gst/src/source/imp.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: f866e6a.

No actionable bugs found. One timeline per broadcast run fixes the per-pad arrival-time alignment error without depending on an optional wall clock. Replacement pumps retain that mapping, while a new broadcast gets a fresh mapping on held pads. Converting each timestamp to nanoseconds avoids mismatched-scale subtraction and preserves pre-reference PTS for clipping; oversized timestamps fail explicitly.

The regression exercises both pump orders, earlier/later siblings, held-pad restarts, and Legacy/CMAF with distinct audio/video timescales. Keeping this separate from the sink's wall-epoch work is appropriate.

Verification: static GitHub review of timing, pump/reconciliation lifetime, conversion and tests. No tests, builds, or interop executed. The new test observes source-pad buffers/events, not decoded playback through real clock-synchronized sinks.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f866e6a67b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1029 to +1030
segment.set_start(pts);
segment.set_base(base);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Initialize the segment position with its start

For any run whose first PTS is nonzero, FormattedSegment::new() leaves position at zero while this raises start to pts. That violates GStreamer's requirement that the segment position lie between its start and stop, so downstream elements that inspect the segment position for queries or seek/reconfiguration handling can observe an invalid position even though timestamp-to-running-time conversion works. Set the initial position to pts alongside the start; the GstSegment documentation confirms initialization sets both fields to zero and requires position to remain within the segment.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: fixed in 741d683, the shared segment now sets position to its start, and the unit test asserts it.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 741d683, against f866e6a.

The segment-position finding (#5556 (comment)) is addressed: initialization now sets position to the same nonzero PTS as start, and the unit test asserts that invariant. The shared timing and clipping behavior are otherwise unchanged.

No new actionable bugs found in this focused delta. Verification: static review only; no tests or downstream playback executed.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary

  • Merged main into the branch to resolve the quest/m1/README.md conflict (main retired two other quests; this PR retires gst-src-av-sync).
  • Fixed the Codex finding: the shared segment now starts its position at start (741d683), with a unit-test assertion.
  • Reviews: OpenAI review of the final head has no remaining findings. CI green; local just check passes (184 moq-gst tests).
  • No public API, GObject property, or wire change. doc/bin/gstreamer.md describes the shared per-run timeline.

Enabling auto-merge on the reviewed head. #5554 (sink wall epoch) follows once this lands.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 1b5fba2 Oct 15, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m1/gst-src-av-sync branch October 15, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant