Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions doc/lib/js/publish.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ WebCodecs, writes the catalog, and publishes a hang broadcast.
| `url`, `name` | Relay URL (with `?jwt=` if needed) and broadcast name. |
| `versions` | Protocol versions to offer, most preferred first, comma-separated like `moq-transport-18,moq-lite-06`. Every stable version by default. |
| `source` | `camera`, `screen`, or `file`. |
| `muted`, `invisible` | Disable audio or video capture. |
| `muted`, `invisible` | Stop publishing audio or video. The microphone and camera stay open, so unmuting or unhiding is instant and the browser's capture indicators stay lit; clear `source` or remove the element to release them. `source` covers both devices, so releasing one while keeping the other takes the library: toggle `Source.Microphone` or `Source.Camera` `enabled`. |
| `preview` | What the nested element shows: the raw `source` (default), a decoded copy of the `encoded` stream to see what viewers get, or `none`. |
| `backend` | How a `<canvas>` preview is drawn; see [@moq/video](/lib/js/video). `auto` (default), `webgpu`, or `2d`. |
| `announce` | When to advertise: once the `source` is live (default), or `always`. Nobody can see or subscribe to the broadcast until it is announced. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. |
Expand Down Expand Up @@ -170,7 +170,7 @@ browsers suspend Web Audio until then. Every input and output is a signal from
`out.error` and disables only its rendition; siblings and the broadcast keep
running. It does not retry on demand or bandwidth changes, and enable cycles
that keep the same capture do not retry. A new capture source (such as the
track a camera or microphone reopens on unmute), changed encoder settings, or
track a camera or microphone reopens after a device swap), changed encoder settings, or
a replacement encoder recovers it. Recovery keeps the catalog ID and uses a
fresh `track` name.

Expand Down
7 changes: 7 additions & 0 deletions js/publish/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first.

## Unreleased

### `muted` and `invisible` keep capture open

`<moq-publish muted>` and `invisible` now stop only the encoder, so the microphone and camera stay open and the browser's capture indicators stay lit.
Unmuting no longer waits on getUserMedia, a new AudioContext, or a camera reopen.
A camera source opens both devices even when it starts muted or hidden.
To release the devices, clear `source` or remove the element; to release one and keep the other, build the pipeline with the library and toggle `Source.Microphone` or `Source.Camera` `enabled`.

### Element reflection and invalid configuration

Property and built-in UI changes now reflect into attributes. Invalid source/preview/backend/announce modes, URLs, and version lists throw instead of selecting defaults; remove an attribute to use its default.
Expand Down
4 changes: 2 additions & 2 deletions js/publish/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ The simplest way to publish a stream:
| `name` | string | required | Broadcast name |
| `versions` | string | stable versions | Comma-separated protocol versions to offer, most preferred first, like `moq-transport-18,moq-lite-06` |
| `source` | string | — | `"camera"`, `"screen"`, `"file"` |
| `muted` | boolean | false | Mute audio capture |
| `invisible` | boolean | false | Disable video capture |
| `muted` | boolean | false | Stop publishing audio; the microphone stays open |
| `invisible` | boolean | false | Stop publishing video; the camera stays open |
| `preview` | string | `"source"` | What the preview renders: `"source"`, `"encoded"`, `"none"` |
| `backend` | string | `"auto"` | Graphics API for a `<canvas>` preview: `"auto"`, `"webgpu"`, `"2d"` |
| `announce` | string | `"source"` | When to advertise: `"always"` or `"source"` (once media is captured). A camera source waits for every enabled track, and `"source"` waits until each captured track's config resolves or fails, so the first catalog lists every rendition. After its first announcement, a camera stays announced until deselected or disconnected, including mute, hide, device swaps, and capture retries. The broadcast is created while connected either way. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. |
Expand Down
9 changes: 4 additions & 5 deletions js/publish/src/announce.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,11 @@ export function run(effect: Effect, announcing: Signal<boolean>, props: Props):
effect.run((effect) => {
const mode = effect.get(props.mode);
const source = effect.get(props.source);
const video = effect.get(props.video.source) !== undefined;
const audio = effect.get(props.audio.source) !== undefined;
const videoEnabled = effect.get(props.video.enabled);
const audioEnabled = effect.get(props.audio.enabled);
// A muted or hidden track keeps capturing but publishes nothing, so it counts as absent.
const video = videoEnabled && effect.get(props.video.source) !== undefined;
const audio = audioEnabled && effect.get(props.audio.source) !== undefined;

// A camera captures video and audio as one publication, so it waits for every enabled track and
// denying either permission can't advertise a partial broadcast.
Expand All @@ -44,9 +45,7 @@ export function run(effect: Effect, announcing: Signal<boolean>, props: Props):
const live = (video || audio) && ready;
const active = source !== undefined && (camera || live);

const settled =
(!videoEnabled || !video || !!effect.get(props.video.settled)) &&
(!audioEnabled || !audio || !!effect.get(props.audio.settled));
const settled = (!video || !!effect.get(props.video.settled)) && (!audio || !!effect.get(props.audio.settled));

const announced =
mode === "always" || (mode === "source" && active && (latched === source || (live && settled)));
Expand Down
4 changes: 2 additions & 2 deletions js/publish/src/audio/capture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ export class Capture {
}

// Decoded samples need no graph and declare their own format, so this only converts them to the
// planar shape the framer wants. Note it never reads `enabled`: muting must not tear the stream
// planar shape the framer wants. Note it never reads `enabled`: disabling must not tear the stream
// down, because a one-shot stream can't be read again.
#runSamples(source: SampleSource, effect: Effect): void {
const { sampleRate, channelCount } = source;
Expand All @@ -173,7 +173,7 @@ export class Capture {
}

#runTrack(source: SourceConfig, effect: Effect): void {
// Releasing the capture device while muted is the whole point of gating on `enabled`.
// Releasing the capture device while disabled is the whole point of gating on `enabled`.
if (!effect.get(this.in.enabled)) return;

const settings = source.track.getSettings();
Expand Down
52 changes: 27 additions & 25 deletions js/publish/src/element.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,13 @@ export default class MoqPublish extends HTMLElement {
// The preview element, either a <video> (raw source via srcObject) or a <canvas> (rendered frames).
#preview = new Signal<HTMLVideoElement | HTMLCanvasElement | undefined>(undefined);

// The connected tracks allowed by `muted` and `invisible`.
#videoEnabled: Signal<boolean>;
#audioEnabled: Signal<boolean>;
#eitherEnabled: Signal<boolean>;
// Whether to encode each track: connected and allowed by `muted` and `invisible`. Capture follows
// `#enabled` instead, so muting or hiding keeps the devices open and resuming is instant.
#videoEnabled = new Signal(false);
#audioEnabled = new Signal(false);

// Set when the element is connected to the DOM.
// Set when the element is connected to the DOM. Gates capture: devices stay open while connected
// with a source selected, whatever `muted` and `invisible` say.
#enabled = new Signal(false);
#connected = false;
#invalid = new Signal<ReadonlySet<string>>(new Set());
Expand Down Expand Up @@ -190,19 +191,10 @@ export default class MoqPublish extends HTMLElement {
});
this.signals.cleanup(() => this.connection.close());

// The connected tracks allowed by `muted` and `invisible`.
// TODO make this.signals.computed to simplify the code.
this.#videoEnabled = new Signal(false);
this.#audioEnabled = new Signal(false);
this.#eitherEnabled = new Signal(false);

this.signals.run((effect) => {
const enabled = effect.get(this.#enabled);
const muted = effect.get(this.controls.muted);
const invisible = effect.get(this.controls.invisible);
this.#videoEnabled.set(enabled && !invisible);
this.#audioEnabled.set(enabled && !muted);
this.#eitherEnabled.set(enabled && (!muted || !invisible));
this.#videoEnabled.set(enabled && !effect.get(this.controls.invisible));
this.#audioEnabled.set(enabled && !effect.get(this.controls.muted));
});

this.#capture = new Video.Capture({ source: this.#videoSource, orientation: this.#orientation });
Expand All @@ -211,7 +203,7 @@ export default class MoqPublish extends HTMLElement {
// Reached as `audio.capture` rather than a field of its own, so audio and video read alike.
const audioCapture = new Audio.Capture({
source: this.#audioSource,
enabled: this.#audioEnabled,
enabled: this.#enabled,
});
this.signals.cleanup(() => audioCapture.close());

Expand Down Expand Up @@ -295,7 +287,8 @@ export default class MoqPublish extends HTMLElement {
return;
}

const source = effect.get(this.#videoSource);
// A hidden camera stays open, so blank the preview to match what viewers get.
const source = effect.get(this.#videoEnabled) ? effect.get(this.#videoSource) : undefined;
if (!source) {
preview.style.display = "none";
return;
Expand Down Expand Up @@ -423,12 +416,13 @@ export default class MoqPublish extends HTMLElement {
effect.set(this.#orientation, 0, 0);

// Sources stay alive while disconnected so device and file selections survive a reconnect;
// the connection-gated enabled signals release their capture and device discovery.
// the connection gate releases their capture and device discovery. `muted` and `invisible`
// only stop the encoders, so a source holds its devices until it is deselected or disconnected.
if (!source) return;

if (source === "camera") {
const video = new Source.Camera({ enabled: this.#videoEnabled, device: { enabled: this.#enabled } });
const audio = new Source.Microphone({ enabled: this.#audioEnabled, device: { enabled: this.#enabled } });
const video = new Source.Camera({ enabled: this.#enabled, device: { enabled: this.#enabled } });
const audio = new Source.Microphone({ enabled: this.#enabled, device: { enabled: this.#enabled } });

effect.set(this.#sources.video, video);
effect.set(this.#sources.audio, audio);
Expand All @@ -448,7 +442,7 @@ export default class MoqPublish extends HTMLElement {

if (source === "screen") {
const screen = new Source.Screen({
enabled: this.#eitherEnabled,
enabled: this.#enabled,
});

effect.run((nested) => {
Expand All @@ -471,7 +465,7 @@ export default class MoqPublish extends HTMLElement {
const fileSource = new Source.File({
// If a File is provided, use it directly.
file: source instanceof File ? source : undefined,
enabled: this.#eitherEnabled,
enabled: this.#enabled,
});

// Otherwise prompt the user to pick one. The selection click is still the
Expand Down Expand Up @@ -539,7 +533,11 @@ export default class MoqPublish extends HTMLElement {
this.#accept("source");
}

/** Whether audio is muted, mirrored by the `muted` attribute. */
/**
* Whether audio is muted, mirrored by the `muted` attribute. Muting stops encoding but keeps the
* microphone open, so unmuting is instant and the browser's microphone indicator stays lit.
* Clear `source` (or remove the element) to release the device.
*/
get muted(): boolean {
return this.controls.muted.peek();
}
Expand All @@ -549,7 +547,11 @@ export default class MoqPublish extends HTMLElement {
this.#accept("muted");
}

/** Whether video is disabled, mirrored by the `invisible` attribute. */
/**
* Whether video is hidden, mirrored by the `invisible` attribute. Hiding stops encoding and the
* preview but keeps the camera open, so showing it again is instant and the browser's camera
* indicator stays lit. Clear `source` (or remove the element) to release the device.
*/
get invisible(): boolean {
return this.controls.invisible.peek();
}
Expand Down
3 changes: 2 additions & 1 deletion js/publish/src/ui/element.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,8 @@ export default class MoqPublishUi extends HTMLElement {
// last live preview's aspect ratio (defaults to 16:9) instead of snapping.
let lastAspect = 16 / 9;
effect.run((e) => {
const capture = e.get(publish.video.in.capture);
// A hidden camera stays open but shows no preview, so it sizes the box like no video at all.
const capture = e.get(publish.controls.invisible) ? undefined : e.get(publish.video.in.capture);
const src = capture ? e.get(capture.in.source) : undefined;
if (src) {
player.classList.remove("player--empty");
Expand Down
1 change: 0 additions & 1 deletion quest/m1/push-to-talk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ encoder, and record why.

## Required

- [`muted` and `invisible` keep capture open](/quest/m1/push-to-talk/mute.md) - the element disables only the encoder, so unmute skips getUserMedia and AudioContext setup
- [Watch caches audio codec support](/quest/m1/push-to-talk/audio-support-cache.md) - an unmute resumes audio without waiting on a support check

## Closes
Expand Down
47 changes: 0 additions & 47 deletions quest/m1/push-to-talk/mute.md

This file was deleted.

3 changes: 2 additions & 1 deletion test/audio-quality/clients/js/src/mic.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
/**
* The publisher page for the microphone trace: the demo's `<moq-publish>` capturing audio only.
* The publisher page for the microphone trace: the demo's `<moq-publish>` publishing audio only
* (`invisible` still opens the camera, but nothing encodes it).
*
* Whatever device Chromium offers is captured through the element's real path (getUserMedia, the
* capture worklet, WebCodecs Opus), so the trace carries a browser publisher's own cadence. Under
Expand Down
Loading
Loading