Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/lib/js/publish.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ WebCodecs, writes the catalog, and publishes a hang broadcast.
| `url`, `name` | Relay URL (with `?jwt=` if needed) and broadcast name. |
| `versions` | Protocol versions to offer, most preferred first, comma-separated like `moq-transport-18,moq-lite-06`. Every stable version by default. |
| `source` | `camera`, `screen`, or `file`. |
| `muted`, `invisible` | Stop publishing audio or video. The microphone and camera stay open, so unmuting or unhiding is instant and the browser's capture indicators stay lit; clear `source` or remove the element to release them. `source` covers both devices, so releasing one while keeping the other takes the library: toggle `Source.Microphone` or `Source.Camera` `enabled`. |
| `muted`, `invisible` | Stop publishing audio or video. By default the microphone and camera stay open, so unmuting or unhiding is instant and the browser's capture indicators stay lit. |
| `capture` | What a muted or invisible track does with its device: `hold` (default) keeps it open, `release` closes it so its capture indicator goes dark, and resuming reopens it. A screen share or file stops once both tracks are off. Clearing `source` or removing the element releases everything either way. |
| `preview` | What the nested element shows: the raw `source` (default), a decoded copy of the `encoded` stream to see what viewers get, or `none`. |
| `backend` | How a `<canvas>` preview is drawn; see [@moq/video](/lib/js/video). `auto` (default), `webgpu`, or `2d`. |
| `announce` | When to advertise: once the `source` is live (default), or `always`. Nobody can see or subscribe to the broadcast until it is announced. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. |
Expand Down
2 changes: 1 addition & 1 deletion js/publish/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ How to update your code for each breaking change, newest first.
`<moq-publish muted>` and `invisible` now stop only the encoder, so the microphone and camera stay open and the browser's capture indicators stay lit.
Unmuting no longer waits on getUserMedia, a new AudioContext, or a camera reopen.
A camera source opens both devices even when it starts muted or hidden.
To release the devices, clear `source` or remove the element; to release one and keep the other, build the pipeline with the library and toggle `Source.Microphone` or `Source.Camera` `enabled`.
To release each device while it is muted or hidden, as before, set `capture="release"`.

### Element reflection and invalid configuration

Expand Down
5 changes: 3 additions & 2 deletions js/publish/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,9 @@ The simplest way to publish a stream:
| `name` | string | required | Broadcast name |
| `versions` | string | stable versions | Comma-separated protocol versions to offer, most preferred first, like `moq-transport-18,moq-lite-06` |
| `source` | string | - | `"camera"`, `"screen"`, `"file"` |
| `muted` | boolean | false | Stop publishing audio; the microphone stays open |
| `invisible` | boolean | false | Stop publishing video; the camera stays open |
| `muted` | boolean | false | Stop publishing audio |
| `invisible` | boolean | false | Stop publishing video |
| `capture` | string | `"hold"` | What a muted or invisible track does with its device: `"hold"` keeps it open so resuming is instant, `"release"` closes it so the browser's capture indicator goes dark. A screen share or file stops once both are off. |
| `preview` | string | `"source"` | What the preview renders: `"source"`, `"encoded"`, `"none"` |
| `backend` | string | `"auto"` | Graphics API for a `<canvas>` preview: `"auto"`, `"webgpu"`, `"2d"` |
| `announce` | string | `"source"` | When to advertise: `"always"` or `"source"` (once media is captured). A camera source waits for every enabled track, and `"source"` waits until each captured track's config resolves or fails, so the first catalog lists every rendition. After its first announcement, a camera stays announced until deselected or disconnected, including mute, hide, device swaps, and capture retries. The broadcast is created while connected either way. To keep it out of discovery, use a hidden name with a `.`-prefixed segment; it is still subscribable by name. |
Expand Down
9 changes: 7 additions & 2 deletions js/publish/api/element.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,27 @@ import { Video } from "@moq/publish";

export type AnnounceMode = "always" | "source";

export type CaptureMode = "hold" | "release";

export type SourceType = "camera" | "screen" | "file";

export default class extends HTMLElement {
constructor();
get announce(): AnnounceMode | undefined;
set announce(value: AnnounceMode);
attributeChangedCallback(name: "source" | "url" | "stats" | "announce" | "name" | "backend" | "versions" | "muted" | "invisible" | "preview" | "echo-path", oldValue: string | null, newValue: string | null): void;
attributeChangedCallback(name: "source" | "url" | "stats" | "announce" | "name" | "capture" | "backend" | "versions" | "muted" | "invisible" | "preview" | "echo-path", oldValue: string | null, newValue: string | null): void;
audio: Audio.Encoder;
get backend(): Preview.Backend;
set backend(value: Preview.Backend);
broadcast: Broadcast;
get capture(): CaptureMode;
set capture(value: CaptureMode);
connectedCallback(): void;
connection: Net.Connection;
readonly controls: {
announce: Signals.Signal<AnnounceMode | undefined>;
backend: Signals.Signal<Preview.Backend>;
capture: Signals.Signal<CaptureMode>;
echoPath: Signals.Signal<Net.Path.Relative | undefined>;
invisible: Signals.Signal<boolean>;
muted: Signals.Signal<boolean>;
Expand All @@ -43,7 +48,7 @@ export default class extends HTMLElement {
set muted(value: boolean);
get name(): Net.Path.Valid;
set name(value: string | Net.Path.Valid);
static observedAttributes: readonly ["url", "versions", "name", "muted", "invisible", "source", "preview", "backend", "announce", "stats", "echo-path"];
static observedAttributes: readonly ["url", "versions", "name", "muted", "invisible", "source", "preview", "backend", "announce", "stats", "echo-path", "capture"];
get preview(): Preview.Mode;
set preview(value: Preview.Mode);
readonly signals: Signals.Effect;
Expand Down
74 changes: 58 additions & 16 deletions js/publish/src/element.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ const OBSERVED = [
"announce",
"stats",
"echo-path",
"capture",
] as const;

/** The built-in capture sources selectable via the `source` attribute. */
Expand All @@ -67,6 +68,15 @@ export type SourceType = "camera" | "screen" | "file";
*/
export type AnnounceMode = "always" | "source";

/**
* What a muted or invisible track does with its capture.
*
* `hold` keeps the device open, so resuming is instant but the browser's capture indicator stays lit.
* `release` closes it, so the indicator goes dark and resuming reopens it. A screen share or file
* stops only once both tracks are off. Defaults to `hold`.
*/
export type CaptureMode = "hold" | "release";

/**
* Parse a boolean attribute: absent uses `defaultValue`, bare presence is true, and an explicit
* `"false"`/`"0"` is false. Matches `<moq-watch>`.
Expand All @@ -89,6 +99,12 @@ function parseAnnounce(value: string | null): AnnounceMode {
throw invalidAttribute("announce", value, '"source" or "always"');
}

function parseCapture(value: string | null): CaptureMode {
if (value === null) return "hold";
if (value === "hold" || value === "release") return value;
throw invalidAttribute("capture", value, '"hold" or "release"');
}

function parsePreview(value: string | null): Preview.Mode {
if (value === null) return "source";
if (value === "encoded" || value === "source" || value === "none") return value;
Expand Down Expand Up @@ -132,6 +148,8 @@ export default class MoqPublish extends HTMLElement {
stats: new Signal(false),
// Where viewers may publish feedback, relative to the broadcast, or undefined to solicit none.
echoPath: new Signal<Moq.Path.Relative | undefined>(undefined),
// Whether a muted or invisible track keeps its capture open.
capture: new Signal<CaptureMode>("hold"),
};

/**
Expand Down Expand Up @@ -178,13 +196,17 @@ export default class MoqPublish extends HTMLElement {
// The preview element, either a <video> (raw source via srcObject) or a <canvas> (rendered frames).
#preview = new Signal<HTMLVideoElement | HTMLCanvasElement | undefined>(undefined);

// Whether to encode each track: connected and allowed by `muted` and `invisible`. Capture follows
// `#enabled` instead, so muting or hiding keeps the devices open and resuming is instant.
// Whether to encode each track: connected and allowed by `muted` and `invisible`.
#videoEnabled = new Signal(false);
#audioEnabled = new Signal(false);

// Set when the element is connected to the DOM. Gates capture: devices stay open while connected
// with a source selected, whatever `muted` and `invisible` say.
// Whether to capture each track: connected, and with `capture="release"` also allowed by `muted`
// and `invisible`. A screen share or file carries both tracks, so it captures while either does.
#videoCapture = new Signal(false);
#audioCapture = new Signal(false);
#eitherCapture = new Signal(false);

// Set when the element is connected to the DOM.
#enabled = new Signal(false);
#connected = false;
#invalid = new Signal<ReadonlySet<string>>(new Set());
Expand Down Expand Up @@ -218,13 +240,23 @@ export default class MoqPublish extends HTMLElement {
this.#audioEnabled.set(enabled && !effect.get(this.controls.muted));
});

this.signals.run((effect) => {
const enabled = effect.get(this.#enabled);
const hold = effect.get(this.controls.capture) === "hold";
const video = enabled && (hold || !effect.get(this.controls.invisible));
const audio = enabled && (hold || !effect.get(this.controls.muted));
this.#videoCapture.set(video);
this.#audioCapture.set(audio);
this.#eitherCapture.set(video || audio);
});

this.#capture = new Video.Capture({ source: this.#videoSource, orientation: this.#orientation });
this.signals.cleanup(() => this.#capture.close());

// Reached as `audio.capture` rather than a field of its own, so audio and video read alike.
const audioCapture = new Audio.Capture({
source: this.#audioSource,
enabled: this.#enabled,
enabled: this.#audioCapture,
});
this.signals.cleanup(() => audioCapture.close());

Expand Down Expand Up @@ -311,7 +343,7 @@ export default class MoqPublish extends HTMLElement {
return;
}

// A hidden camera stays open, so blank the preview to match what viewers get.
// A hidden camera may stay open, so blank the preview to match what viewers get.
const source = effect.get(this.#videoEnabled) ? effect.get(this.#videoSource) : undefined;
if (!source) {
preview.style.display = "none";
Expand Down Expand Up @@ -361,6 +393,7 @@ export default class MoqPublish extends HTMLElement {
this.signals.run((effect) => this.#reflect(effect, "announce", effect.get(this.controls.announce)));
this.signals.run((effect) => this.#reflect(effect, "stats", effect.get(this.controls.stats) ? "" : undefined));
this.signals.run((effect) => this.#reflect(effect, "echo-path", effect.get(this.controls.echoPath)));
this.signals.run((effect) => this.#reflect(effect, "capture", effect.get(this.controls.capture)));
this.signals.run(this.#runSource.bind(this));
}

Expand Down Expand Up @@ -392,6 +425,7 @@ export default class MoqPublish extends HTMLElement {
else if (name === "backend") this.backend = parseBackend(newValue);
else if (name === "stats") this.stats = parseBoolean(newValue, false);
else if (name === "echo-path") this.echoPath = parseEchoPath(newValue);
else if (name === "capture") this.capture = parseCapture(newValue);
else {
const exhaustive: never = name;
throw new Error(`Invalid attribute: ${exhaustive}`);
Expand Down Expand Up @@ -445,12 +479,12 @@ export default class MoqPublish extends HTMLElement {

// Sources stay alive while disconnected so device and file selections survive a reconnect;
// the connection gate releases their capture and device discovery. `muted` and `invisible`
// only stop the encoders, so a source holds its devices until it is deselected or disconnected.
// release a device only with `capture="release"`.
if (!source) return;

if (source === "camera") {
const video = new Source.Camera({ enabled: this.#enabled, device: { enabled: this.#enabled } });
const audio = new Source.Microphone({ enabled: this.#enabled, device: { enabled: this.#enabled } });
const video = new Source.Camera({ enabled: this.#videoCapture, device: { enabled: this.#enabled } });
const audio = new Source.Microphone({ enabled: this.#audioCapture, device: { enabled: this.#enabled } });

effect.set(this.#sources.video, video);
effect.set(this.#sources.audio, audio);
Expand All @@ -470,7 +504,7 @@ export default class MoqPublish extends HTMLElement {

if (source === "screen") {
const screen = new Source.Screen({
enabled: this.#enabled,
enabled: this.#eitherCapture,
});

effect.run((nested) => {
Expand All @@ -493,7 +527,7 @@ export default class MoqPublish extends HTMLElement {
const fileSource = new Source.File({
// If a File is provided, use it directly.
file: source instanceof File ? source : undefined,
enabled: this.#enabled,
enabled: this.#eitherCapture,
});

// Otherwise prompt the user to pick one. The selection click is still the
Expand Down Expand Up @@ -562,9 +596,8 @@ export default class MoqPublish extends HTMLElement {
}

/**
* Whether audio is muted, mirrored by the `muted` attribute. Muting stops encoding but keeps the
* microphone open, so unmuting is instant and the browser's microphone indicator stays lit.
* Clear `source` (or remove the element) to release the device.
* Whether audio is muted, mirrored by the `muted` attribute. Muting stops encoding; `capture`
* decides whether the microphone stays open.
*/
get muted(): boolean {
return this.controls.muted.peek();
Expand All @@ -577,8 +610,7 @@ export default class MoqPublish extends HTMLElement {

/**
* Whether video is hidden, mirrored by the `invisible` attribute. Hiding stops encoding and the
* preview but keeps the camera open, so showing it again is instant and the browser's camera
* indicator stays lit. Clear `source` (or remove the element) to release the device.
* preview; `capture` decides whether the camera stays open.
*/
get invisible(): boolean {
return this.controls.invisible.peek();
Expand All @@ -589,6 +621,16 @@ export default class MoqPublish extends HTMLElement {
this.#accept("invisible");
}

/** What a muted or invisible track does with its capture, mirrored by the `capture` attribute. */
get capture(): CaptureMode {
return this.controls.capture.peek();
}

set capture(value: CaptureMode) {
this.controls.capture.set(parseCapture(value));
this.#accept("capture");
}

/** What a <canvas> preview renders, mirrored by the `preview` attribute. */
get preview(): Preview.Mode {
return this.controls.preview.peek();
Expand Down
73 changes: 73 additions & 0 deletions test/interop/clients/js/publish-ui.browser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,79 @@ test("a camera holds both devices until its source clears or the element detache
}
}, 30_000);

test('capture="release" closes each device while it is muted or hidden', async () => {
const server = await createServer({
configFile: false,
root: `${import.meta.dir}/publish-ui`,
plugins: [worklet()],
server: { host: "127.0.0.1", port: 0 },
});
await server.listen();
const browser = await launch(["--use-fake-device-for-media-stream", "--use-fake-ui-for-media-stream"]);
try {
const page = await browser.newPage();
await page.goto(server.resolvedUrls?.local[0] ?? "");
await page.getByRole("button", { name: /mute microphone/i }).waitFor();
const publication = await page.evaluateHandle(() => {
const publish = document.querySelector<MoqPublish>("moq-publish");
if (!publish) throw new Error("missing publish element");
publish.setAttribute("capture", "release");
return { publish };
});
expect(await publication.evaluate(({ publish }) => publish.capture)).toBe("release");
// Wait until the microphone and camera are each open or closed as given.
const opened = (audio: boolean, video: boolean) =>
page.waitForFunction(
([{ publish }, audio, video]) =>
!!publish.sources.audio.peek()?.out.source.peek() === audio &&
!!publish.sources.video.peek()?.out.source.peek() === video,
[publication, audio, video] as const,
);

// Muted and hidden from the start opens neither device.
await publication.evaluate(({ publish }) => {
publish.muted = true;
publish.invisible = true;
publish.source = "camera";
});
expect(
await publication.evaluate(({ publish }) =>
[publish.sources.audio.peek(), publish.sources.video.peek()].map((source) => source?.in.enabled.peek()),
),
).toEqual([false, false]);
Comment on lines +396 to +400

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
expect(
await publication.evaluate(({ publish }) =>
[publish.sources.audio.peek(), publish.sources.video.peek()].map((source) => source?.in.enabled.peek()),
),
).toEqual([false, false]);
expect(
await publication.evaluate(({ publish }) => {
const audio = publish.sources.audio.peek();
const video = publish.sources.video.peek();
return {
inputs: [audio?.in.enabled.peek(), video?.in.enabled.peek()],
outputs: [audio?.out.source.peek(), video?.out.source.peek()],
};
}),
).toEqual({ inputs: [false, false], outputs: [undefined, undefined] });


await publication.evaluate(({ publish }) => {
publish.muted = false;
publish.invisible = false;
});
await opened(true, true);
const first = await captured(publication);

// Muting ends only the microphone; unmuting opens a new one.
await publication.evaluate(({ publish }) => {
publish.muted = true;
});
await page.waitForFunction((tracks) => tracks[0]?.readyState === "ended", first);
expect(await first.evaluate((tracks) => tracks[1]?.readyState)).toBe("live");
await publication.evaluate(({ publish }) => {
publish.muted = false;
});
await opened(true, true);
const second = await captured(publication);
expect(await second.evaluate((tracks) => tracks.map((track) => track?.readyState))).toEqual(["live", "live"]);

// Hiding ends only the camera.
await publication.evaluate(({ publish }) => {
publish.invisible = true;
});
await page.waitForFunction((tracks) => tracks[1]?.readyState === "ended", second);
expect(await second.evaluate((tracks) => tracks[0]?.readyState)).toBe("live");
} finally {
await browser.close();
await server.close();
}
}, 30_000);

test("watch property and UI state reflects without resetting a selected publish File", async () => {
const server = await createServer({
configFile: false,
Expand Down