Skip to content

js/publish: video restore can replace the retained aspect ratio with its loading size #7826

Closed

Description

@shermerL

Problem

With a responsive <video> inside <moq-publish-ui>, quickly showing and hiding a camera preview can replace the retained playback aspect ratio with the video's temporary pre-metadata size. In a 944px-wide fixture, a previously stable 531px player ends up 150px tall while hidden. Showing it again and letting the media size settle restores 531px.

Keeping space for the controls while video is hidden is expected. This report concerns the retained ratio changing to the loading placeholder's ratio, not the existence of empty space. Native video's temporary default size is normal browser behavior, not a browser defect.

Reproduction and evidence

Reproduced with unmodified publish production code from cd5a8476cb21681ad08f5ac96eb433253f57547c. Both js/publish/src/element.ts and js/publish/src/ui/element.ts are byte-identical in 4506e3fdc59bfa7827eb46b9e2d2b4392e959806; their GitHub main blobs still matched when checked on 2026-11-04.

Environment: macOS arm64, Bun 1.4.2, Playwright 1.64.1, Chromium 156.0.8078.4, fake camera/microphone, no relay. Video CSS is width:100%; height:auto; captured video is 1280×720. Verified in headless Chromium and once in a visible Chromium window.

User-facing sequence:

  1. Use the existing test/interop/clients/js/publish-ui fixture, adding <video muted autoplay style="width:100%;height:auto"> to <moq-publish>; use capture="hold".
  2. Select Camera through Settings and let the preview stabilize.
  3. Hide video, then quickly activate Show video and Hide video again.
  4. The off-state player can remain much shorter than the last stable preview. In an automated run activating the actual buttons with Playwright press("Enter"), this happened in 1 of 10 attempts on the unmodified source. That count is not an estimate of real-world frequency.

To target the timing window, hide on the next loadedmetadata event after restoring the preview. This reproduced 10/10 times on the unmodified source. The same targeted sequence also reproduced with capture="release", and with preview="none" / "source" instead of invisible. These are event-driven diagnostics, not manual-camera tests.

The event trace is:

Event <moq-publish> height UI player height Last observed host height videoWidth
Restoring, ResizeObserver callback 150 150 150 0
loadedmetadata, immediately before hide 531 531 150 1280
ResizeObserver after hide 0 150 0 0
After five additional animation frames 0 150 0 0

A native <video> + ordinary parent control, with the same stream rebinding and display:none sequence, also briefly measures 150px but ends with both element and parent at 0px. In the MoQ UI, the hidden video/host is already 0px; .player retains 150px through its inline aspect-ratio: 6.29333 / 1.

Runnable reproduction using the existing browser fixture

In a checkout at the revision above, install the workspace dependencies and Playwright Chromium using the repository's normal setup. Save this as .tmp/video-height-repro.ts, then run bun .tmp/video-height-repro.ts from the repository root. It exits nonzero with Expected retained height 531, got 150. The absolute 531px expectation is specific to this fixture and fake camera, not a proposed production constant.

import { createServer } from 'vite';
import { chromium } from 'playwright';
const root = process.argv[2] ?? process.cwd();
const { worklet } = await import(`${root}/js/common/vite-plugin-worklet.ts`);
const source = `${root}/js/publish/src`;
const server = await createServer({
  configFile: false,
  root: `${root}/test/interop/clients/js/publish-ui`,
  resolve: { alias: [
    { find: '@moq/publish/element', replacement: `${source}/element.ts` },
    { find: '@moq/publish/ui', replacement: `${source}/ui/element.ts` },
    { find: /^@moq\/publish$/, replacement: `${source}/index.ts` },
  ] },
  plugins: [worklet()],
  server: { host: '127.0.0.1', port: 0 },
});
await server.listen();
const browser = await chromium.launch({
  channel: 'chromium',
  args: ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'],
});
try {
  const page = await browser.newPage({ viewport: { width: 960, height: 720 } });
  page.setDefaultTimeout(15_000);
  await page.goto(server.resolvedUrls!.local[0]);
  await page.getByRole('button', { name: 'Settings', exact: true }).waitFor();
  await page.evaluate(() => {
    const p: any = document.querySelector('moq-publish');
    p.capture = 'hold';
    p.backend = '2d';
    const v = document.createElement('video');
    v.autoplay = true;
    v.muted = true;
    v.style.cssText = 'width:100%;height:auto';
    p.append(v);
    const w: any = window;
    w.lastHeight = -1;
    w.trace = [];
    const player = p.parentElement.shadowRoot.querySelector('.player');
    w.snapshot = (event: string) => {
      const row = { event, host: p.getBoundingClientRect().height,
        player: player.getBoundingClientRect().height,
        observed: w.lastHeight, videoWidth: v.videoWidth };
      w.trace.push(row);
      return row;
    };
    new ResizeObserver(([entry]) => {
      w.lastHeight = entry.contentRect.height;
      w.snapshot('RO');
    }).observe(p);
  });
  await page.getByRole('button', { name: 'Settings', exact: true }).click();
  await page.getByRole('tabpanel', { name: 'Source' })
    .getByRole('button', { name: 'Camera', exact: true }).click();
  await page.waitForFunction(() => (window as any).lastHeight === 531);
  await page.evaluate(() => { (document.querySelector('moq-publish') as any).invisible = true; });
  await page.waitForFunction(() => (window as any).lastHeight === 0);
  await page.evaluate(() => {
    const p: any = document.querySelector('moq-publish');
    const v = p.querySelector('video');
    const w: any = window;
    w.trace = [];
    v.addEventListener('loadedmetadata', () => {
      w.snapshot('metadata-before-hide');
      p.invisible = true;
    }, { once: true });
    p.invisible = false;
  });
  await page.waitForFunction(() => {
    const w: any = window;
    return w.trace.some((r: any) => r.event === 'metadata-before-hide') && w.lastHeight === 0;
  });
  const result = await page.evaluate(async () => {
    for (let i = 0; i < 5; i++) await new Promise(requestAnimationFrame);
    const w: any = window;
    return { final: w.snapshot('final'), trace: w.trace };
  });
  console.log(JSON.stringify({ chromium: browser.version(), ...result }, null, 2));
  if (result.final.player !== 531) throw new Error(`Expected retained height 531, got ${result.final.player}`);
} finally {
  await browser.close();
  await server.close();
}

Cause and scope

The video preview effect clears srcObject on hide, then attaches a new MediaStream and immediately shows the element on restore. Before metadata is ready it contributes the temporary size. The UI size observer accepts that nonzero size while capture is live and overwrites lastAspect. If hidden before the media-size observation is delivered, the next observation is zero and the placeholder ratio is retained.

This is a recoverable local layout problem. No relay or remote playback failure is established. Real cameras and other browsers were not tested. The current publish demo uses canvas; the documented video embedding is the relevant entry point.

Related: rendered-preview sizing PR #7247 and the existing publish-empty-previews quest. This supplies evidence for the video restore transition; it is not a request to duplicate that quest. No production fix is included. Should the UI retain the last media-ready ratio across this transition rather than the temporary placeholder ratio?

(Written by GPT-6)

Activity

  1. coderabbitai commented on Nov 4, 2026

    @coderabbitai
    Contributor
    🔗 Related PRs

    #7247 - fix(publish): size controls from rendered preview content [closed]
    #7157 - feat(publish)!: muted and invisible keep capture open [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  2. added
    questBeing tracked/planned in a quest. See `quest/`
    on Nov 4, 2026
  3. kixelated commented on Nov 4, 2026

    @kixelated
    Collaborator

    Thanks for the detailed report and the targeted reproduction. I checked the diagnosis against the code. js/publish/src/element.ts shows the <video> in the same tick it sets srcObject, so the video lays out at its 300x150 pre-metadata size. js/publish/src/ui/element.ts then keeps that ratio while capture is live.

    Decision: fix it at the source. <moq-publish> will keep a <video> preview display: none until it reports videoWidth > 0, then show it. That extends the rule from publish-empty-previews that a preview showing nothing is hidden. The UI keeps sizing purely from what's rendered. We decided against making the UI ignore an unsized video, because that couples it to the preview element type. We also decided against pre-sizing from track.getSettings(), because those values can be missing or pre-rotation. The test reuses your loadedmetadata-targeted sequence in publish-ui.browser.ts.

    moq-dev/moq#7834 tracks this as quest/m2/publish-video-loading.md in m2. That quest requires publish-empty-previews (moq-dev/moq#7832) to land first, because both edit the same preview effect. The quest's PR will close this issue.

    Anyone who wants to donate tokens can pick it up with /quest-start quest/m2/publish-video-loading once it's unblocked.

    (Written by Claude Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questBeing tracked/planned in a quest. See `quest/`

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions