Skip to content

test(browser): run media QA with alternate player engines - #5640

Merged
kixelated merged 5 commits into
mainfrom
quest/m2/browser-media-qa-engines
Oct 16, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m2/browser-media-qa-engines

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Media QA exercised only Chromium and treated publisher capture/encode APIs as player requirements. A player on another engine needs its own capability check and explicit engine reporting.

Approach

just test media --media-engines chromium,firefox,webkit runs each requested player against a Chromium fixture. Firefox and WebKit players use the WebSocket fallback. The capability report names the engine version and lists missing publisher and player APIs separately, and fails by name instead of skipping. The lifecycle cases, shared tolerances, and five negative controls run unchanged per engine. Nightly runs Firefox and WebKit players (--with-deps); the Chromium player already runs in interop.yml on PRs and nightly.

The player logs each sample to the console, and the harness reads player state only from those logs, for every engine. Any Playwright evaluation grants user activation and would invalidate the pre-gesture autoplay checks; Chromium avoided that with a CDP read, but Firefox and WebKit have no equivalent. One read path replaces the CDP player read and the per-engine branches. A navigation clears the previous document's sample so the matrix driver's reload cannot read stale state. The Chromium fixture is still read over CDP with userGesture: false.

Validation

  • just test media --media-engines chromium,firefox locally on the final code: all 12 rows PASS (media output + lifecycle and the five negative controls, per engine), including the gesture gate and capture permissions on the Firefox player. Firefox 155.0 lacks only MediaStreamTrackProcessor, so it can play this fixture but not publish it. An earlier Firefox run broke its leaked-session control at cold-start tone presence (71/79 against the 90% floor), the existing flake tracked by quest/m1/publish-audio-channel-count.md; it passed on the rerun and no tolerance changed.
  • The earlier "audio stays suspended after the click" result did not reproduce. A bare AudioContext driven the way Gesture.unlock does it (resume() on pointerdown/pointerup) resumes in headless Firefox 155 with and without the autoplay-block prefs, and the full player run passes the gesture gate. Its cause is unconfirmed; this host has a working audio server.
  • WebKit 2359 was not run locally: the host is missing its shared libraries and installing them needs root. Nightly installs them with --with-deps; its first run is the first WebKit evidence. Playwright WebKit is not Safari.
  • Unit tests: role requirements, no page evaluation for player reads, and a navigation discarding the stale sample (fails without the framenavigated clear). just check passes.

Decisions

Settled by the iterate agent with the recommended option; challenge any of them.

  1. Player state read path
    • ✅ Console samples for every engine (one path, deletes the CDP player read and engine branches)
    • Console for Firefox/WebKit, CDP for Chromium (two paths that can disagree)
  2. This quest
    • ✅ Complete: the goal is "a Firefox or WebKit player", and Firefox passes. Delete quest/m2/browser-media-qa-engines.md.
    • Keep open until the first nightly WebKit result
  3. Nightly matrix
    • ✅ Firefox and WebKit only; the Chromium player already runs in interop.yml nightly and on PRs
    • Keep a duplicate Chromium row

Impact

  • Public API: none. Test-only CLI option --media-engines on just test media.
  • Wire: unchanged. No production media behavior or tolerance changes.

Follow-ups

🤖 Generated with Claude Code

(Written by GPT-6, iterated by Claude Opus 5.5)

kixelated and others added 5 commits October 15, 2026 18:37
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Every engine now reads the player through the samples it logs, so the
Chromium CDP read and the per-engine branches go away, and a navigation
clears the previous document's sample. The Firefox player passes the
full media run locally, so the quest is complete. Nightly drops its
Chromium row, which interop.yml already runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…f8f2b9b8148

# Conflicts:
#	quest/m2/README.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 16, 2026 22:13
@coderabbitai

coderabbitai Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b980f3a5-f495-4bf3-a135-5faa99a57513
📥 Commits

Reviewing files that changed from the base of the PR and between 977223d and c116f19.

📒 Files selected for processing: 14
  • .github/workflows/nightly.yml
  • quest/m2/README.md
  • quest/m2/browser-media-qa-engines.md
  • test/interop/README.md
  • test/interop/clients/js/driver.ts
  • test/interop/clients/js/harness.test.ts
  • test/interop/clients/js/harness.ts
  • test/interop/clients/js/media.ts
  • test/interop/clients/js/src/capabilities.test.ts
  • test/interop/clients/js/src/capabilities.ts
  • test/interop/clients/js/src/contract.ts
  • test/interop/clients/js/src/probe.ts
  • test/interop/interop.sh
  • test/justfile
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T22:16:10.580166Z c116f19 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging at c116f19 with all three decisions accepted:

  1. The harness reads player state only from console samples, on every engine. The Chromium CDP player read and the per-engine branches are gone, and a navigation clears the stale sample.
  2. The quest is complete and deleted: the Firefox player passes the full media run (all 12 rows with Chromium, locally).
  3. Nightly runs Firefox and WebKit players only; interop.yml already covers Chromium.

CI is green on this head and the Codex review found nothing. WebKit is unverified until its first nightly run.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 2aa7a95 Oct 16, 2026
16 checks passed
@kixelated
kixelated deleted the quest/m2/browser-media-qa-engines branch October 16, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant