diff --git a/README.md b/README.md index 6f6f8faf..158438f8 100644 --- a/README.md +++ b/README.md @@ -480,6 +480,7 @@ setting costs. Escape or a click outside closes it. | Recording | Change or open the recordings folder, and pick the picture and the audio scene a recording holds, and whether each source records a file of its own too (see [Recording](#recording)). | | Hotkeys | Bind keys, by what they are for: going live, ending the stream, the panic key, and recording; Studio mode, TAKE, and AUTO; each scene; each sound's mute; and replay, clips, and markers (see [Hotkeys](#hotkeys)). | | Guests | Set the room guests join, its key, the join page, how long a link lasts, and whether guests appear on their own (see [Guests](#guests)). | +| Remote | Pair a phone as a remote by its QR code, see which are connected, and remove one (see [Phone remote](#phone-remote)). | | Advanced | Publish scenes as broadcasts of their own and sources independently, turn the virtual camera of the program or of each published scene on, pick how the window's [effects](#effects) move, and open the [scene graph](#scene-graph). | On air, a new canvas size or frame rate, or a published scene's new size, @@ -1118,7 +1119,9 @@ why under Guests, with a button to the pane. without reconnecting. A guest's token may read only their own renewals and return feed, at `.renew//` and `.return//` in the room, and the program at `.program`, but anything else the relay lets - subscribe there can read every guest's, so grant that only to the studio. A guest who + subscribe there can read every guest's, so grant that only to the studio. + The same session publishes each [phone remote](#phone-remote)'s desk under + `.remote` and reads their presses under `.press`. A guest who leaves needs a link, or a renewed token, that has not expired to come back. BUILD's Add says, under Guests, when renewal is failing. - **Upgrading from a studio without renewal:** generate a new room key, drop @@ -1181,6 +1184,45 @@ encode of their mix-minus. A guest's screen share stays in their feed, since only their camera's microphone is taken out. +## Phone remote + +A phone paired as a remote drives the show from wherever it reaches the +relay, with nothing installed: its page shows the scene pads with their +tally (ON AIR, FIRST, or NEXT, as the studio's own pads), TAKE and AUTO in +Studio mode, each sound's mute, and the live clock, and the whole page is +framed in the accent while the program is on air. A pad the studio cannot +take now is greyed out. Each press shows in the studio's status +bar, as a key's does, and on the phone. + +Settings, Remote, pairs one: name it, **Pair a phone**, and scan the QR code +with the phone's camera, or **Copy link** to send it. A remote joins the +[guests' room](#guests), so it needs the room, its key, and the join page set +there first; its page is `remote.html` beside the join page, and the link +lasts as long as a guest's (`--guest-expiry`). The pane lists each phone +paired, whether it is connected, and its link, with **Show link** while that +link still works, **New link**, and **Remove**. Treat the link like a +password: anyone holding it can drive the show until you remove it. + +- A remote presses only what is safe to press by mistake from a pocket: a + scene's pad, TAKE, AUTO, and a sound's mute. Going live, ending the + stream, the panic key, recording, and replay stay on the studio. +- A remote's token, signed by the room key like a guest's, may publish only + its presses, at `.press//` in the room, and read only the + desk the studio publishes it at `.remote//`, which carries + its renewed tokens. Remote names are their own, apart from guests'. +- The studio reads a remote's newest press only, at most one a program frame, + and refuses one that is not a single action of at most 4 KiB, so a page + pressing faster costs the studio nothing more. The page sends its next + press once the desk says how the last went, and the desk catches up with + what changed at the studio four times a second. A press the phone made + before the studio restarted or rejoined the room is not done again, since + it was most likely done before: the page asks you to check it first. +- **Remove** stops reading its presses at once, and stops renewing its token, + so it is disconnected within `--guest-expiry`; **New link** pairs it again + under a new generation, which a link from before cannot reach. Pairings and + removals outlast a restart, kept beside the room key as + `room.remotes.toml` for `room.jwk`. + ## Scene file The collection is TOML, so it diffs and reviews like code. It names devices, diff --git a/bun.lock b/bun.lock index 75ea1ba1..09fc2307 100644 --- a/bun.lock +++ b/bun.lock @@ -12,6 +12,8 @@ "name": "@moq/studio-join", "version": "0.0.0", "dependencies": { + "@moq/clock": "workspace:*", + "@moq/json": "workspace:*", "@moq/net": "workspace:*", "@moq/publish": "workspace:*", "@moq/room": "workspace:*", diff --git a/js/join/package.json b/js/join/package.json index 743a1f7d..e61af568 100644 --- a/js/join/package.json +++ b/js/join/package.json @@ -3,7 +3,7 @@ "private": true, "type": "module", "version": "0.0.0", - "description": "The page a guest opens to join a kixel.live Studio room", + "description": "The pages a guest opens to join a kixel.live Studio room, and a phone opens to drive it", "license": "(MIT OR Apache-2.0)", "repository": "github:moq-dev/studio", "scripts": { @@ -13,6 +13,8 @@ "test": "bun test" }, "dependencies": { + "@moq/clock": "workspace:*", + "@moq/json": "workspace:*", "@moq/net": "workspace:*", "@moq/publish": "workspace:*", "@moq/room": "workspace:*", diff --git a/js/join/remote.html b/js/join/remote.html new file mode 100644 index 00000000..215920ac --- /dev/null +++ b/js/join/remote.html @@ -0,0 +1,47 @@ + + + + + + + + + + Studio remote + + + + +
+ + + Connecting + +
+ +
+ +

Waiting for the studio. Keep the studio open with this phone paired.

+ + +
+ + + + + diff --git a/js/join/src/desk.test.ts b/js/join/src/desk.test.ts new file mode 100644 index 00000000..b7494e6a --- /dev/null +++ b/js/join/src/desk.test.ts @@ -0,0 +1,42 @@ +import { expect, test } from "bun:test"; +import { clock, type Desk, settled, tally } from "./desk.ts"; + +const desk = (change: Partial): Desk => ({ + air: "off", + clock: null, + studio: false, + program: "Gameplay", + cued: null, + scenes: [], + take_refused: null, + sounds: [], + done: null, + ...change, +}); + +test("the clock reads hours, minutes, and seconds, past a day too", () => { + expect(clock(0)).toBe("00:00:00"); + expect(clock(5047)).toBe("01:24:07"); + expect(clock(100 * 3600 + 59)).toBe("100:00:59"); + expect(clock(-3)).toBe("00:00:00"); +}); + +test("the scene on program is on air only while live, and first before", () => { + expect(tally(desk({ air: "live" }), "Gameplay")).toEqual({ tag: "On air", on: "air" }); + expect(tally(desk({ air: "ending" }), "Gameplay")?.on).toBe("air"); + expect(tally(desk({ air: "off" }), "Gameplay")).toEqual({ tag: "First", on: "cue" }); + expect(tally(desk({ air: "starting" }), "Gameplay")?.tag).toBe("First"); + expect(tally(desk({ cued: "Chatting" }), "Chatting")).toEqual({ tag: "Next", on: "cue" }); + expect(tally(desk({ cued: "Chatting" }), "BRB")).toBeUndefined(); +}); + +test("a press is settled only by the desk naming this page and it or a later one", () => { + const done = (id: string, seq: number) => desk({ done: { id, seq, ok: true, said: "" } }); + expect(settled(undefined, "a", 0)).toBe(false); + expect(settled(desk({}), "a", 0)).toBe(false); + expect(settled(done("a", 0), "a", 0)).toBe(true); + expect(settled(done("a", 3), "a", 2)).toBe(true); + expect(settled(done("a", 2), "a", 3)).toBe(false); + // Another page, such as this one before a reload. + expect(settled(done("b", 9), "a", 3)).toBe(false); +}); diff --git a/js/join/src/desk.ts b/js/join/src/desk.ts new file mode 100644 index 00000000..6af1cae8 --- /dev/null +++ b/js/join/src/desk.ts @@ -0,0 +1,98 @@ +/** + * The desk: the show as the remote shows it, which the studio publishes at a + * path only the remote's own token may read, and the presses the remote + * publishes back. The shapes are `Desk` and `Press` in + * `rs/moq-studio/src/guests/remote.rs`. + * + * @module + */ + +import * as Net from "@moq/net"; + +/** Where the studio publishes `name`'s desk in `generation`, with its renewed tokens. */ +export function path(name: string, generation: number): Net.Path.Valid { + return Net.Path.from(`.remote/${name}/${generation}`); +} + +/** Where the remote called `name` publishes its presses in `generation`. */ +export function pressPath(name: string, generation: number): Net.Path.Valid { + return Net.Path.from(`.press/${name}/${generation}`); +} + +/** The desk's track, a JSON snapshot. */ +export const TRACK = "desk"; + +/** The presses' track, a JSON snapshot. */ +export const PRESS_TRACK = "press"; + +/** Whether the program is on air. */ +export type Air = "off" | "starting" | "live" | "ending"; + +/** The show as the studio last published it. */ +export interface Desk { + air: Air; + /** Whole seconds on air, while live. */ + clock: number | null; + /** Whether a pad cues its scene rather than taking it. */ + studio: boolean; + program: string | null; + cued: string | null; + scenes: { name: string; refused: string | null }[]; + /** Why TAKE and AUTO cannot run now, if they cannot. */ + take_refused: string | null; + sounds: { name: string; muted: boolean }[]; + /** How this page's last press went, once the studio has read one. */ + done: Done | null; +} + +/** How a press went. */ +export interface Done { + id: string; + seq: number; + ok: boolean; + said: string; +} + +/** What a remote may press: `Action` in `rs/moq-studio/src/controls.rs`, as serde writes it. */ +export type Action = "Cut" | "Transition" | { Scene: string } | { Mute: string }; + +/** A press: this page, its number, and the action, or none to say the page is there. */ +export interface Press { + id: string; + seq: number; + action: Action | null; +} + +/** + * Whether `desk` says how the press numbered `seq` from the page `id` went: + * until it does, the page sends nothing more, so the studio, which reads only + * the newest press, never drops one. + */ +export function settled(desk: Desk | undefined, id: string, seq: number): boolean { + return desk?.done?.id === id && desk.done.seq >= seq; +} + +/** `seconds` as the studio's live clock shows it: hours, minutes, and seconds. */ +export function clock(seconds: number): string { + const whole = Math.max(0, Math.floor(seconds)); + const two = (value: number) => value.toString().padStart(2, "0"); + return `${two(Math.floor(whole / 3600))}:${two(Math.floor(whole / 60) % 60)}:${two(whole % 60)}`; +} + +/** A pad's tally: on air in the accent, or first or next in amber. */ +export type Tally = { tag: string; on: "air" | "cue" } | undefined; + +/** + * The tally of the pad for `scene`, as the studio's own pads show it: the + * scene on program is ON AIR while live, and FIRST otherwise, the one the + * stream opens on; the scene cued is NEXT. + */ +export function tally(desk: Desk, scene: string): Tally { + if (desk.program === scene) { + return desk.air === "live" || desk.air === "ending" + ? { tag: "On air", on: "air" } + : { tag: "First", on: "cue" }; + } + if (desk.cued === scene) return { tag: "Next", on: "cue" }; + return undefined; +} diff --git a/js/join/src/index.ts b/js/join/src/index.ts index 09b416e2..3c876a74 100644 --- a/js/join/src/index.ts +++ b/js/join/src/index.ts @@ -13,47 +13,13 @@ import { Local } from "@moq/room"; import { Effect, Signal } from "@moq/signals"; import * as Watch from "@moq/watch"; import * as Feed from "./feed.ts"; -import { format, type Link, parse } from "./link.ts"; +import { format, type Link } from "./link.ts"; +import { element, fail, read, stored } from "./page.ts"; import * as Renew from "./renew.ts"; /** Where the fragment is kept once it leaves the address bar, so a reload rejoins. */ const STORED = "moq-studio-join"; -function element(id: string): T { - const found = document.getElementById(id); - if (!found) throw new Error(`missing #${id}`); - return found as T; -} - -function fail(err: unknown): void { - const error = element("error"); - error.textContent = err instanceof Error ? err.message : String(err); - error.hidden = false; -} - -/** - * `use` on the page's session storage, or `undefined` where the browser denies - * it, which only costs rejoining after a reload. - */ -function stored(use: (storage: Storage) => T): T | undefined { - try { - return use(sessionStorage); - } catch { - return undefined; - } -} - -/** The link this page was opened with, moved out of the address bar. */ -function read(): Link { - const fragment = location.hash.slice(1) || stored((storage) => storage.getItem(STORED)); - if (!fragment) throw new Error("open the join link the host sent you"); - const link = parse(fragment); - // Out of the address bar, history, and anything that shows them, such as a screen share. - stored((storage) => storage.setItem(STORED, fragment)); - history.replaceState(null, "", location.pathname + location.search); - return link; -} - /** Publish into the room until the guest leaves. Returns how to leave. */ function join(link: Link): () => void { const signals = new Effect(); @@ -141,7 +107,7 @@ function join(link: Link): () => void { function main(): void { let link: Link; try { - link = read(); + link = read(STORED); } catch (err) { fail(err); return; diff --git a/js/join/src/page.ts b/js/join/src/page.ts new file mode 100644 index 00000000..523bd180 --- /dev/null +++ b/js/join/src/page.ts @@ -0,0 +1,48 @@ +/** + * What the join page and the remote share: finding their elements, saying what + * failed, and reading the link they were opened with out of the address bar. + * + * @module + */ + +import { type Link, parse } from "./link.ts"; + +/** The element with `id`, which the page's HTML always has. */ +export function element(id: string): T { + const found = document.getElementById(id); + if (!found) throw new Error(`missing #${id}`); + return found as T; +} + +/** Show `err` in the page's `#error`. */ +export function fail(err: unknown): void { + const error = element("error"); + error.textContent = err instanceof Error ? err.message : String(err); + error.hidden = false; +} + +/** + * `use` on the page's session storage, or `undefined` where the browser denies + * it, which only costs rejoining after a reload. + */ +export function stored(use: (storage: Storage) => T): T | undefined { + try { + return use(sessionStorage); + } catch { + return undefined; + } +} + +/** + * The link this page was opened with, moved out of the address bar and kept + * under `key` in session storage, so a reload rejoins. + */ +export function read(key: string): Link { + const fragment = location.hash.slice(1) || stored((storage) => storage.getItem(key)); + if (!fragment) throw new Error("open the link the host sent you"); + const link = parse(fragment); + // Out of the address bar, history, and anything that shows them, such as a screen share. + stored((storage) => storage.setItem(key, fragment)); + history.replaceState(null, "", location.pathname + location.search); + return link; +} diff --git a/js/join/src/remote.css b/js/join/src/remote.css new file mode 100644 index 00000000..174459c3 --- /dev/null +++ b/js/join/src/remote.css @@ -0,0 +1,288 @@ +/* The studio's LIVE look at a phone's width: its tokens are the app's dark chrome (quest/m0/ux/mock/kit.css). */ +:root { + color-scheme: dark; + --bg: #09090c; + --p1: #111116; + --p2: #18181f; + --p3: #22222b; + --line: #2a2a35; + --text: #f3f3f6; + --dim: #8d8d9c; + --accent: #4f8cff; + --cue: #ffb020; + --on-cue: #1a1200; + --ok: #35d07f; + --err: #ff2d46; + --display: "Barlow Condensed", "Arial Narrow", sans-serif; + --mono: "JetBrains Mono", ui-monospace, monospace; + font: + 15px / 1.35 Inter, + system-ui, + sans-serif; + background: var(--bg); + color: var(--text); +} + +* { + box-sizing: border-box; +} + +body { + margin: 0; + min-height: 100dvh; + /* The tally: the whole page is framed in the accent while on air. */ + box-shadow: inset 0 0 0 3px transparent; + transition: box-shadow 0.2s; +} + +body[data-air="live"], +body[data-air="ending"] { + box-shadow: + inset 0 0 0 3px var(--accent), + inset 0 0 40px #4f8cff33; +} + +.top { + position: sticky; + top: 0; + z-index: 1; + display: flex; + align-items: center; + gap: 10px; + padding: max(10px, env(safe-area-inset-top)) 14px 10px; + background: var(--p1); + border-bottom: 1px solid var(--line); +} + +.logo { + flex: none; + width: 22px; + height: 22px; + border-radius: 7px; + background: conic-gradient(from 210deg, #ff2d46, #ff8a3d, #ff2d46); +} + +.name { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + font-weight: 600; +} + +.pill { + display: flex; + align-items: center; + gap: 6px; + height: 26px; + padding: 0 9px; + border-radius: 13px; + background: var(--p2); + border: 1px solid var(--line); + font-size: 12px; + white-space: nowrap; +} + +.dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--cue); +} + +.dot.connected { + background: var(--ok); +} + +.dot.disconnected { + background: var(--err); +} + +/* The live control: OFF AIR in amber, LIVE with the clock in the accent. */ +.air { + height: 28px; + padding: 0 10px; + border-radius: 6px; + font: 700 16px / 28px var(--display); + letter-spacing: 0.1em; + text-transform: uppercase; + white-space: nowrap; + font-variant-numeric: tabular-nums; +} + +.air:empty { + display: none; +} + +.air[data-air="off"], +.air[data-air="starting"] { + background: var(--cue); + color: var(--on-cue); +} + +.air[data-air="live"], +.air[data-air="ending"] { + background: var(--accent); + color: #fff; + box-shadow: 0 0 22px #4f8cff66; +} + +main { + max-width: 40rem; + margin: 0 auto; + padding: 14px 14px max(14px, env(safe-area-inset-bottom)); +} + +section + section { + margin-top: 22px; +} + +.cap { + margin: 0 0 8px; + font: 600 12px / 1 var(--display); + letter-spacing: 0.14em; + text-transform: uppercase; + color: var(--dim); +} + +button { + font: inherit; + color: var(--text); + background: var(--p2); + border: 2px solid transparent; + border-radius: 10px; + cursor: pointer; + touch-action: manipulation; + -webkit-tap-highlight-color: transparent; +} + +/* A tap always lands on the button itself, whatever it holds. */ +button > * { + pointer-events: none; +} + +button:active:not(:disabled) { + background: var(--p3); +} + +button:disabled { + cursor: default; + opacity: 0.45; +} + +/* A scene's pad, its tally in its border and tag, as on the studio's dock. */ +.pads { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(9rem, 1fr)); + gap: 10px; +} + +.pad { + position: relative; + min-height: 76px; + padding: 30px 10px 10px; + text-align: left; + font: 600 17px / 1.1 var(--display); + letter-spacing: 0.06em; + text-transform: uppercase; + overflow-wrap: anywhere; +} + +.pad[data-tally="air"] { + border-color: var(--accent); + box-shadow: 0 0 24px #4f8cff40; +} + +.pad[data-tally="cue"] { + border-color: var(--cue); + box-shadow: 0 0 24px #ffb02033; +} + +.tag { + position: absolute; + left: 8px; + top: 8px; + padding: 0 5px; + border-radius: 3px; + font: 700 11px / 16px var(--display); + letter-spacing: 0.12em; + background: var(--accent); + color: #fff; +} + +.pad[data-tally="cue"] .tag { + background: var(--cue); + color: var(--on-cue); +} + +/* Shown only in Studio mode, so its display yields to `hidden`. */ +.take:not([hidden]) { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 10px; + margin-top: 10px; +} + +.take button { + height: 52px; + font: 700 18px / 1 var(--display); + letter-spacing: 0.12em; + text-transform: uppercase; + background: var(--cue); + color: var(--on-cue); +} + +.sounds { + display: flex; + flex-direction: column; + gap: 8px; +} + +.sound { + display: flex; + align-items: center; + gap: 10px; + min-height: 48px; + padding: 6px 6px 6px 12px; + border-radius: 10px; + background: var(--p1); + border: 1px solid var(--line); +} + +.sound span { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.sound button { + flex: none; + min-width: 88px; + height: 38px; + font-weight: 600; + border-color: var(--line); +} + +.sound button[aria-pressed="true"] { + background: var(--err); + border-color: var(--err); + color: #fff; +} + +.hint { + color: var(--dim); + font-size: 13px; +} + +.said { + margin-top: 18px; + color: var(--dim); + font-size: 13px; +} + +.error { + color: var(--err); +} diff --git a/js/join/src/remote.ts b/js/join/src/remote.ts new file mode 100644 index 00000000..98f45ee7 --- /dev/null +++ b/js/join/src/remote.ts @@ -0,0 +1,269 @@ +/** + * The remote: a phone opens the studio's QR code and drives the show from + * here, wherever it reaches the relay: the scene pads, TAKE and AUTO, the + * mutes, the live clock, and a tally. Its token publishes only its presses and + * reads only the desk the studio publishes it, with its renewed tokens. + * + * @module + */ + +import { Duration } from "@moq/clock"; +import { Snapshot } from "@moq/json"; +import * as Net from "@moq/net"; +import { Effect, Signal } from "@moq/signals"; +import * as Desk from "./desk.ts"; +import { format, type Link } from "./link.ts"; +import { element, fail, read, stored } from "./page.ts"; +import * as Renew from "./renew.ts"; + +/** Where the fragment is kept once it leaves the address bar, so a reload reconnects. */ +const STORED = "moq-studio-remote"; + +/** Drive the studio until the page closes. */ +function drive(link: Link): void { + const signals = new Effect(); + // The token rides SETUP rather than the URL, and each renewal replaces it in band. + const tokens = new Signal([link.jwt]); + const connection = new Net.Connection({ url: link.url, versions: [Renew.VERSION], tokens, enabled: true }); + const desks = Desk.path(link.name, link.generation); + + Renew.follow(connection.origin, desks, (jwt) => { + tokens.set([jwt]); + // So a reload uses the newest token once the link's has expired. + link.jwt = jwt; + stored((storage) => storage.setItem(STORED, format(link))); + }); + + // This page, so a desk saying how another's press went, such as this one's before a reload, settles nothing. + const id = crypto.randomUUID(); + const desk = new Signal(undefined); + // The number of the newest press sent, which the next waits for the desk to settle. + const sent = new Signal(undefined); + const presses = new Signal | undefined>(undefined); + let next = 0; + const send = (producer: Snapshot.Producer, action: Desk.Action | null) => { + const seq = next++; + producer.update({ value: { id, seq, action }, at: Net.Time.Timestamp.unscaled(seq) }); + sent.set(seq); + }; + const press = (action: Desk.Action) => { + const producer = presses.peek(); + const last = sent.peek(); + if (!producer || last === undefined || !Desk.settled(desk.peek(), id, last)) return; + send(producer, action); + }; + + signals.run((effect) => { + const origin = effect.get(connection.origin); + if (!origin) return; + const route = origin.route(Desk.pressPath(link.name, link.generation)); + effect.cleanup(() => route.close()); + const producer = new Snapshot.Producer({ + track: route.broadcast.createTrack(Desk.PRESS_TRACK), + // Each press whole, since the studio reads only the newest. + deltas: false, + }); + // The studio never does the first press it reads, which it may have done before it was + // listening, so each route opens with the page saying it is there. + send(producer, null); + presses.set(producer); + effect.cleanup(() => presses.set(undefined)); + }); + + signals.run((effect) => { + const origin = effect.get(connection.origin); + if (!origin) return; + const following = origin.follow(desks); + effect.cleanup(() => following.close()); + effect.run((effect) => { + const broadcast = effect.get(following.active); + if (!broadcast) { + desk.set(undefined); + return; + } + const consumer = new Snapshot.Consumer({ track: broadcast.track(Desk.TRACK) }); + effect.cleanup(() => consumer.close()); + effect.spawn(async () => { + for (;;) { + const latest = await effect.race(consumer.latest()); + if (!latest) return; + desk.set(latest.value); + } + }); + }); + }); + + signals.run((effect) => { + const status = effect.get(connection.status); + element("status").textContent = status.charAt(0).toUpperCase() + status.slice(1); + element("status-dot").className = `dot ${status}`; + const error = effect.get(connection.error); + if (error) fail(error); + }); + + // The clock counts on from the whole seconds of the newest desk. + const live = new Signal<{ seconds: number; at: number } | undefined>(undefined); + signals.run((effect) => { + const seconds = effect.get(desk)?.clock; + live.set(seconds == null ? undefined : { seconds, at: performance.now() }); + }); + signals.run((effect) => { + const current = effect.get(desk); + const on = effect.get(live); + const air = element("air"); + const tick = () => { + const seconds = on ? on.seconds + (performance.now() - on.at) / 1000 : 0; + air.textContent = current ? said(current.air, Desk.clock(seconds)) : ""; + }; + tick(); + if (on) effect.interval(tick, Duration.millis(250)); + }); + + const show = view(press); + signals.run((effect) => { + const current = effect.get(desk); + const last = effect.get(sent); + const ready = last !== undefined && Desk.settled(current, id, last); + show(current, ready, current?.done?.id === id ? current.done : undefined); + }); +} + +/** What the live control says while the program is `air`, with its `clock` while live. */ +function said(air: Desk.Air, clock: string): string { + switch (air) { + case "off": + return "Off air"; + case "starting": + return "Going live"; + case "live": + return `Live ${clock}`; + case "ending": + return "Ending"; + } +} + +/** + * Show each desk, its controls pressable once `ready`, and how this page's + * last press went. A control stays the same element from one desk to the + * next, changed in place, since a desk arrives each second while live: a tap + * that spans one still lands, and focus stays where it was. + */ +function view( + press: (action: Desk.Action) => void, +): (desk: Desk.Desk | undefined, ready: boolean, done: Desk.Done | undefined) => void { + // By the action each presses, which never changes for an element. + const controls = new Map(); + const control = (action: Desk.Action, className: string): HTMLButtonElement => { + const key = JSON.stringify(action); + let button = controls.get(key); + if (!button) { + button = document.createElement("button"); + button.type = "button"; + button.className = className; + button.addEventListener("click", () => press(action)); + controls.set(key, button); + } + return button; + }; + const take = control("Cut", ""); + const auto = control("Transition", ""); + take.textContent = "Take"; + auto.textContent = "Auto"; + element("take").replaceChildren(take, auto); + const rows = new Map(); + + return (desk, ready, done) => { + element("waiting").hidden = desk !== undefined; + element("desk").hidden = desk === undefined; + document.body.dataset.air = desk?.air ?? "off"; + element("air").dataset.air = desk?.air ?? "off"; + if (!desk) return; + + const enable = (button: HTMLButtonElement, refused: string | null) => { + button.disabled = !ready || refused !== null; + if (refused) button.title = refused; + else button.removeAttribute("title"); + }; + + const pads = desk.scenes.map((scene) => { + const pad = control({ Scene: scene.name }, "pad"); + if (!pad.firstChild) { + const tag = document.createElement("span"); + tag.className = "tag"; + const name = document.createElement("span"); + name.textContent = scene.name; + pad.append(tag, name); + } + const tally = Desk.tally(desk, scene.name); + const tag = pad.firstChild as HTMLSpanElement; + tag.hidden = !tally; + text(tag, tally?.tag ?? ""); + if (tally) pad.dataset.tally = tally.on; + else delete pad.dataset.tally; + enable(pad, scene.refused); + return pad; + }); + children(element("scenes"), pads); + text(element("mode"), desk.studio ? "Studio mode: a pad cues its scene" : "A pad takes its scene"); + + // TAKE and AUTO put the cued scene on air, in Studio mode. + element("take").hidden = !desk.studio; + enable(take, desk.take_refused); + enable(auto, desk.take_refused); + + const sounds = desk.sounds.map((sound) => { + let row = rows.get(sound.name); + const mute = control({ Mute: sound.name }, ""); + if (!row) { + row = document.createElement("div"); + row.className = "sound"; + const name = document.createElement("span"); + name.textContent = sound.name; + row.append(name, mute); + rows.set(sound.name, row); + } + text(mute, sound.muted ? "Muted" : "Mute"); + mute.setAttribute("aria-pressed", String(sound.muted)); + mute.setAttribute("aria-label", `${sound.muted ? "Unmute" : "Mute"} ${sound.name}`); + enable(mute, null); + return row; + }); + children(element("sounds"), sounds); + element("no-sounds").hidden = sounds.length > 0; + + // Forget the controls of scenes and sounds the show no longer has. + const kept = new Set([take, auto, ...pads, ...sounds.map((row) => row.lastChild)]); + for (const [key, button] of controls) if (!kept.has(button)) controls.delete(key); + for (const [name, row] of rows) if (!sounds.includes(row)) rows.delete(name); + + const message = element("said"); + message.hidden = !done; + text(message, done?.said ?? ""); + message.classList.toggle("error", done?.ok === false); + }; +} + +/** Set `node`'s text, unless it already says it, which would replace its text node. */ +function text(node: HTMLElement, said: string): void { + if (node.textContent !== said) node.textContent = said; +} + +/** Make `nodes` the children of `parent`, in order, moving none that already are. */ +function children(parent: HTMLElement, nodes: readonly Node[]): void { + const same = parent.childNodes.length === nodes.length && nodes.every((node, i) => parent.childNodes[i] === node); + if (!same) parent.replaceChildren(...nodes); +} + +function main(): void { + let link: Link; + try { + link = read(STORED); + } catch (err) { + fail(err); + return; + } + element("name").textContent = link.name; + drive(link); +} + +main(); diff --git a/js/join/vite.config.ts b/js/join/vite.config.ts index 8a27d5d5..58d8b980 100644 --- a/js/join/vite.config.ts +++ b/js/join/vite.config.ts @@ -1,3 +1,4 @@ +import { fileURLToPath } from "node:url"; import { defineConfig } from "vite"; import { worklet } from "../../moq/js/common/vite-plugin-worklet.ts"; @@ -8,6 +9,13 @@ export default defineConfig({ plugins: [worklet()], build: { target: "esnext", + // The join page, and the remote beside it, which the studio links as `remote.html`. + rollupOptions: { + input: { + index: fileURLToPath(new URL("index.html", import.meta.url)), + remote: fileURLToPath(new URL("remote.html", import.meta.url)), + }, + }, // @moq/publish's capture and encode pipeline is most of the page, in one chunk. chunkSizeWarningLimit: 1024, }, diff --git a/quest/m2/phone-remote.md b/quest/m2/phone-remote.md deleted file mode 100644 index a8cd578f..00000000 --- a/quest/m2/phone-remote.md +++ /dev/null @@ -1,23 +0,0 @@ -# [L] A phone as a remote - -## Goal - -A QR code in the studio opens a page on a phone that shows the scene pads, -the mutes, the live clock, and a tally, and drives them. No app is -installed, and it works wherever the phone reaches the relay, as a guest's -join page does. - -## Plan - -Planned 2026-11-09 (maintainer, /quest-plan), and held for later: ✅ m2 -(rejected: m1). - -- It is a page beside the join page (`js/join`), over the room the studio - already owns: the studio publishes its state there, and the remote - publishes commands the studio subscribes to. -- A remote's token is signed like a guest's and lets it do only that. Its - commands are untrusted input from the network: a fixed, small vocabulary, - rejected whole when malformed, and bounded in rate by design. -- Removing a remote stops renewing its token, as for a guest. -- The page follows the look of the studio's LIVE page - ([mock-up](/quest/m0/ux/mock/shell-live.webp)) at a phone's width. diff --git a/quest/m2/try-phone-remote.md b/quest/m2/try-phone-remote.md new file mode 100644 index 00000000..56e2b46a --- /dev/null +++ b/quest/m2/try-phone-remote.md @@ -0,0 +1,38 @@ +# [XS] Try the phone remote + +## Goal + +The maintainer drives a show from a real phone and either signs the remote +off or files what they find as quests: how it feels to use, which no +headless or desktop-browser run exercises. + +## Plan + +The remote is on `main` once +[moq-dev/studio#212](https://github.com/moq-dev/studio/pull/212) merges, and +the join page deploy serves it at `https://join.studio.moq.dev/remote.html`. +With a room set up in Settings, Guests (see the README's Guests and Phone +remote sections), pair a phone in Settings, Remote, and scan its QR code. +What to try, beside the [LIVE mock-up](/quest/m0/ux/mock/shell-live.webp): + +- **Reach:** the phone on cellular rather than the studio's network, then + back on Wi-Fi; whether the page reconnects on its own and says so. +- **Feel:** the pads, TAKE, AUTO, and the mutes under a thumb, with the + phone locked and woken in between; whether a press ever seems lost or done + twice; whether the time between a press and the program changing is short + enough to cut on. +- **Readability:** the tally (ON AIR, NEXT, FIRST) and the frame in the + accent while live, at arm's length. +- **Removal:** Remove in Settings, Remote, and how the page shows it. + +An agent then turns each finding into a quest, or an edit to the quest that +owns the remote, and deletes this one. + +## Waiting + +- Maintainer: try the remote from a phone as the Plan lists and report what + to change, or that nothing needs to. An agent has no phone to hold and + cannot judge feel; a headless run already covers the protocol + (`guests::remote::tests`), and the agent Chrome at phone size covered the + page's clicks. + Check: a comment or quest records the result. diff --git a/rs/moq-studio/assets/icons/smartphone.svg b/rs/moq-studio/assets/icons/smartphone.svg new file mode 100644 index 00000000..162a8f63 --- /dev/null +++ b/rs/moq-studio/assets/icons/smartphone.svg @@ -0,0 +1,15 @@ + + + + + diff --git a/rs/moq-studio/src/controls.rs b/rs/moq-studio/src/controls.rs index 4a7c5d54..a29087e6 100644 --- a/rs/moq-studio/src/controls.rs +++ b/rs/moq-studio/src/controls.rs @@ -250,6 +250,23 @@ impl Action { fixed => fixed.fixed().words, } } + + /// Whether a phone paired as a remote may fire it: what its page shows, + /// a scene's pad, a mute, TAKE, and AUTO. Every action says, so a new one + /// is kept off the network until it is placed here. + pub fn remote(&self) -> bool { + match self { + Self::Scene(_) | Self::Mute(_) | Self::Cut | Self::Transition => true, + Self::StudioMode + | Self::Replay + | Self::SaveClip + | Self::Marker + | Self::Record + | Self::Panic + | Self::GoLive + | Self::EndStream => false, + } + } } impl std::fmt::Display for Action { @@ -623,7 +640,7 @@ impl Controls { /// so the next go-live, end of stream, or save tries them again; and join /// its guests' room on them, unless joined already. Call it holding /// [`Self::following`], inside the runtime. - fn follow(&self, stream: &Stream) { + fn follow(self: &Arc, stream: &Stream) { if !crate::show::follow(&self.engine, stream) { self.rewatch.store(true, Ordering::Release); } @@ -634,7 +651,7 @@ impl Controls { /// guests' room if the last try failed, so a room whose key file is back /// joins at the next go-live, end of stream, or save, as well as on Try /// again. Call it holding [`Self::following`], inside the runtime. - fn catch_up(&self, stream: &Stream, waiting: bool) { + fn catch_up(self: &Arc, stream: &Stream, waiting: bool) { match waiting { true => self.follow(stream), false => self.join_room(stream), @@ -646,7 +663,7 @@ impl Controls { /// Call it holding [`Self::following`], which keeps two joins apart, as /// [`Self::joined`] is let go while one reads the key file, so the UI /// reading the failure never waits on the disk; inside the runtime. - fn join_room(&self, stream: &Stream) { + fn join_room(self: &Arc, stream: &Stream) { let room = (stream.relay().cloned(), stream.guests()); { let mut joined = self.joined.lock().unwrap(); @@ -667,6 +684,9 @@ impl Controls { } let joining = crate::guests::Joining::connect(&room.1, room.0.as_ref(), &stream.client(), stream.quic().clone()); + // Weak, so the remotes never keep the controls, and the engine, alive. + let board: std::sync::Weak = Arc::downgrade(self); + let joining = joining.map(|joining| joining.map(|joining| joining.driving(board))); // A room that cannot be leaves the last one too, since its settings // are gone; it is tried again once they change. let result = match joining { diff --git a/rs/moq-studio/src/engine.rs b/rs/moq-studio/src/engine.rs index c6eee99e..bae3cf0c 100644 --- a/rs/moq-studio/src/engine.rs +++ b/rs/moq-studio/src/engine.rs @@ -1435,7 +1435,7 @@ impl Engine { // the two never publish at once. *joined = None; *self.feeds.lock().unwrap() = None; - let Some(crate::guests::Joining { room, renewal }) = joining else { + let Some(crate::guests::Joining { room, renewal, remotes }) = joining else { return Ok(()); }; anyhow::ensure!(!*self.stopped.borrow(), "the show is closing"); @@ -1456,6 +1456,9 @@ impl Engine { if let Some(renewal) = renewal { tasks.push(self.until_stopped(renewal.run())); } + if let Some(remotes) = remotes { + tasks.push(self.until_stopped(remotes.run())); + } // Who has a return feed follows who is renewed, synced with the audio, // including any change made since the feeds started. let mut changes = room.changes(); @@ -4543,6 +4546,34 @@ mod tests { assert_eq!(format!("{err:#}"), why); } + // A phone paired as a remote shows the pads and mutes the palette + // lists, with the tally, and presses as a key does. + { + use crate::controls::Action; + use crate::guests::remote::Board; + let desk = controls.desk(); + let pads: Vec<&str> = desk.scenes.iter().map(|pad| pad.name.as_str()).collect(); + assert!(pads.contains(&"Guest") && pads.contains(&main.as_str()), "{pads:?}"); + let portrait = desk.scenes.iter().find(|pad| pad.name == "Portrait").unwrap(); + assert!(portrait.refused.is_some(), "a pad says why it cannot go on"); + assert_eq!(desk.program.as_deref(), Some("Guest")); + assert_eq!(desk.take_refused.as_deref(), Some("nothing is cued")); + let mutes: Vec = Action::listed(&engine.state.lock().unwrap()) + .into_iter() + .filter_map(|action| match action { + Action::Mute(name) => Some(name), + _ => None, + }) + .collect(); + let sounds: Vec = desk.sounds.iter().map(|sound| sound.name.clone()).collect(); + assert_eq!(sounds, mutes, "the palette's mutes, in its order"); + assert!(!desk.studio); + let take = |scene: &str| controls.clone().press(&Action::Scene(scene.into())); + assert_eq!(take(&main), Ok(format!("took `{main}`"))); + assert_eq!(controls.desk().program.as_deref(), Some(main.as_str())); + take("Guest").unwrap(); + } + // A sound's key or palette row flips its mute, an edit undo takes back. let mute = crate::controls::Action::Mute("red".into()); assert_eq!(controls.fire(&mute).unwrap(), "muted `red`"); diff --git a/rs/moq-studio/src/guests.rs b/rs/moq-studio/src/guests.rs index f0a8296f..478f70e3 100644 --- a/rs/moq-studio/src/guests.rs +++ b/rs/moq-studio/src/guests.rs @@ -22,6 +22,10 @@ //! A renewed guest also gets a return feed, the program with a mix-minus, at //! `.return//`, which only their token may read either; //! see [`crate::feed`]. +//! +//! A phone paired as a [remote] drives the show from the same room. + +pub mod remote; use std::collections::{BTreeMap, BTreeSet}; use std::path::PathBuf; @@ -147,24 +151,42 @@ impl Config { .clone() .unwrap_or_else(|| Url::parse(JOIN_PAGE).expect("the default join page is a URL")); let connect = connect.context("guests join through the relay; set one in Settings, Stream")?; - let removals = removals_path(key); + let removals = beside(key, "guests"); + let remotes = beside(key, "remotes"); let key = moq_auth::Key::from_file(key).with_context(|| format!("reading the room key {}", key.display()))?; let expiry = self.guest_expiry.map_or(DEFAULT_EXPIRY, Duration::from); let room = Room::new(connect, room, key, page, expiry)?; - Ok(Some(room.remember(removals)?)) + let mut room = room.remember(removals)?; + room.remotes = remote::Remotes::load(remotes)?; + Ok(Some(room)) } } -/// Where the guests removed from scenes, how often each was let back in, and -/// who appeared on their own are kept: `room.guests.toml` for `room.jwk`. Beside the key, since the -/// tokens they bound are the key's, whichever scene file the studio opens. -fn removals_path(key: &std::path::Path) -> PathBuf { +/// Where what the room key's tokens bound is kept beside it, whichever scene +/// file the studio opens: `room..toml` for `room.jwk`. The guests removed +/// from scenes, how often each was let back in, and who appeared on their own +/// are in `guests`, and the phones paired as remotes in `remotes`. +fn beside(key: &std::path::Path, what: &str) -> PathBuf { let mut name = key.file_stem().unwrap_or_default().to_os_string(); - name.push(".guests.toml"); + name.push(format!(".{what}.toml")); key.with_file_name(name) } -/// The file at [`removals_path`]. +/// Replace the file at `path` with `text` whole, so a crash never leaves half +/// a file. +fn replace(path: &std::path::Path, text: &str) -> anyhow::Result<()> { + let context = || format!("writing {}", path.display()); + let dir = match path.parent() { + Some(dir) if !dir.as_os_str().is_empty() => dir, + _ => std::path::Path::new("."), + }; + let mut file = tempfile::NamedTempFile::new_in(dir).with_context(context)?; + std::io::Write::write_all(&mut file, text.as_bytes()).with_context(context)?; + file.persist(path).map_err(|err| err.error).with_context(context)?; + Ok(()) +} + +/// The guests' file [`beside`] the key. #[derive(Default, serde::Serialize, serde::Deserialize)] #[serde(deny_unknown_fields)] struct Removals { @@ -201,6 +223,10 @@ pub struct Room { session: OnceLock, /// What that session publishes, once it dials. origin: OnceLock, + /// What that session reads, once it dials: the remotes' presses. + heard: OnceLock, + /// The phones paired as remotes. + remotes: remote::Remotes, } /// One guest the studio knows of. @@ -301,11 +327,14 @@ impl Room { changed: Default::default(), session: Default::default(), origin: Default::default(), + heard: Default::default(), + remotes: Default::default(), }; // Sign once now, so a key that cannot sign for guests or for the studio's // own session fails at startup. room.sign("guest", 0)?; room.sign_session()?; + room.sign_remote("remote", 0)?; Ok(room) } @@ -364,16 +393,7 @@ impl Room { .collect(), appeared: appeared.map(|(identity, _)| identity.clone()).collect(), }; - let context = || format!("writing {}", path.display()); - let dir = match path.parent() { - Some(dir) if !dir.as_os_str().is_empty() => dir, - _ => std::path::Path::new("."), - }; - // Replaced whole, so a crash never leaves half a file. - let mut file = tempfile::NamedTempFile::new_in(dir).with_context(context)?; - std::io::Write::write_all(&mut file, toml::to_string(&removals)?.as_bytes()).with_context(context)?; - file.persist(path).map_err(|err| err.error).with_context(context)?; - Ok(()) + replace(path, &toml::to_string(&removals)?) } /// The room, relative to the studio's connection. @@ -412,14 +432,7 @@ impl Room { participant.generation += 1; })?; } - let mut link = self.page.clone(); - let fragment = url::form_urlencoded::Serializer::new(String::new()) - .append_pair("url", self.url.as_str()) - .append_pair("name", name) - .append_pair("jwt", &token) - .append_pair("generation", &(generation + u64::from(removed)).to_string()) - .finish(); - link.set_fragment(Some(&fragment)); + let link = self.link(self.page.clone(), name, &token, generation + u64::from(removed)); let invite = Invite { link, expires }; // A new link may appear on its own again. if appeared { @@ -431,6 +444,19 @@ impl Room { Ok(invite) } + /// `page` with the room's relay, `name`, its `token`, and its `generation` + /// in the fragment, which the browser never sends to a server. + fn link(&self, mut page: Url, name: &str, token: &str, generation: u64) -> Url { + let fragment = url::form_urlencoded::Serializer::new(String::new()) + .append_pair("url", self.url.as_str()) + .append_pair("name", name) + .append_pair("jwt", token) + .append_pair("generation", &generation.to_string()) + .finish(); + page.set_fragment(Some(&fragment)); + page + } + /// Stop renewing the token of the guest called `identity`, across restarts, /// until the host makes them a new link. pub fn stop_renewing(&self, identity: &str) -> anyhow::Result<()> { @@ -498,13 +524,18 @@ impl Room { } /// A token for the studio's own session in the room, which publishes the - /// renewals, the return feeds, and the program they show, and nothing else. + /// renewals, the return feeds, the program they show, and the remotes' + /// desks, and reads only the remotes' presses. fn sign_session(&self) -> anyhow::Result<(String, kio::clock::Wall)> { - let claims = moq_auth::Claims::default().with_root(self.root.as_str()).with_publish([ - moq_auth::Pattern::subtree(RENEW)?, - moq_auth::Pattern::subtree(RETURN)?, - moq_auth::Pattern::subtree(PROGRAM)?, - ]); + let claims = moq_auth::Claims::default() + .with_root(self.root.as_str()) + .with_publish([ + moq_auth::Pattern::subtree(RENEW)?, + moq_auth::Pattern::subtree(RETURN)?, + moq_auth::Pattern::subtree(PROGRAM)?, + moq_auth::Pattern::subtree(remote::DESK)?, + ]) + .with_subscribe([moq_auth::Pattern::subtree(remote::PRESS)?]); self.sign_claims(claims) .context("the room key cannot sign the studio's own session in the room") } @@ -962,10 +993,12 @@ pub async fn run(engine: Arc, room: Arc, origin: moq_net::origin:: } /// A room ready for the engine to [join](Engine::join), with the studio's -/// session in it, which renews its guests' tokens. +/// session in it, which renews its guests' tokens, and what its remotes +/// drive, if any. pub struct Joining { pub(crate) room: Arc, pub(crate) renewal: Option, + pub(crate) remotes: Option, } impl Joining { @@ -985,15 +1018,27 @@ impl Joining { Ok(Some(Self { room, renewal: Some(renewal), + remotes: None, })) } + /// Let the room's remotes drive `board` for as long as the engine + /// follows the room. + pub fn driving(self, board: std::sync::Weak) -> Self { + let remotes = remote::Service::new(self.room.clone(), board); + Self { + remotes: Some(remotes), + ..self + } + } + /// `room` with no session in it, so no guest is renewed or fed. #[cfg(test)] pub fn offline(room: Room) -> Self { Self { room: Arc::new(room), renewal: None, + remotes: None, } } } @@ -1033,9 +1078,14 @@ impl Renewal { .init(quic) .context("cannot set up the studio's session in the room")?; let origin = moq_tokio::origin::spawn(); - let connection = client.with_publisher(&origin).connect(room.url.clone()); + let heard = moq_tokio::origin::spawn(); + let connection = client + .with_publisher(&origin) + .with_subscriber(heard.clone()) + .connect(room.url.clone()); let _ = room.session.set(connection.monitor()); let _ = room.origin.set(origin.clone()); + let _ = room.heard.set(heard); let renewals = Renewals::new(room, origin); Ok(Self { renewals, connection }) } diff --git a/rs/moq-studio/src/guests/remote.rs b/rs/moq-studio/src/guests/remote.rs new file mode 100644 index 00000000..f3d80705 --- /dev/null +++ b/rs/moq-studio/src/guests/remote.rs @@ -0,0 +1,1164 @@ +//! Remotes: a phone that scans a QR code opens a page showing the scene pads, +//! the mutes, the live clock, and a tally, and drives them from wherever it +//! reaches the relay, with nothing installed. +//! +//! A remote is paired by name in the guests' room. Its token, signed by the +//! room key like a guest's, publishes only its presses, at +//! `.press//`, and reads only the desk the studio publishes +//! it at `.remote//`: the show as its page shows it, how its +//! last press went, and its renewed tokens. The studio's session in the room +//! reads the presses of each remote paired and not removed, and publishes its +//! desk while its page is connected. +//! +//! A press is untrusted input from the network: one JSON value of at most +//! [`MAX_PRESS`] bytes naming one action [`Action::remote`] admits, refused +//! whole otherwise. The studio reads a remote's newest press only, at most one +//! a program frame, so a page pressing faster loses its own presses and costs +//! the studio no more; the page sends its next press once the desk says how +//! the last went. Removing a remote stops reading its presses at once, and +//! stops renewing its token, across restarts, until a new link moves it to the +//! next generation, which its old token cannot reach. + +use std::collections::BTreeMap; +use std::path::PathBuf; +use std::sync::{Arc, Mutex, Weak}; +use std::task::{Poll, ready}; +use std::time::Duration; + +use anyhow::Context; +use moq_net::Path; + +use super::{Invite, Room, TOKEN_TRACK}; +use crate::controls::{Action, Controls}; +use crate::engine::OnAir; + +/// The prefix of each remote's desk, which only its token may read. +pub(super) const DESK: &str = ".remote"; + +/// The prefix of each remote's presses, which only its token may publish. +pub(super) const PRESS: &str = ".press"; + +/// The track of a remote's desk, a JSON snapshot. `js/join` reads the same name. +const DESK_TRACK: &str = "desk"; + +/// The track of a remote's presses, a JSON snapshot. `js/join` writes the same name. +const PRESS_TRACK: &str = "press"; + +/// The most of one press the studio reads. A press names one scene or sound, +/// so this only stops a page sending something that is not one. +const MAX_PRESS: usize = 4096; + +/// How often a connected remote's desk catches up with what the studio +/// changed itself, such as a scene taken at the keyboard. +const TALLY: Duration = Duration::from_millis(250); + +/// What a remote's page is told of a press it made before the studio +/// restarted or rejoined the room, which the studio cannot tell was done. +const RESTARTED: &str = "the studio restarted; check whether this was done before pressing again"; + +/// The remote's page, beside the join page: `js/join/remote.html`. +const PAGE: &str = "remote.html"; + +/// One phone paired as a remote. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Remote { + /// The newest link the host made for it this session, if any. + pub invite: Option, + /// How often a new link paired it again after a removal. Its token and + /// desk are under it, so a token from before cannot read or press. + pub generation: u64, + /// Whether the host removed it, which stops reading its presses and + /// renewing its token, across restarts, until a new link. + pub removed: bool, + /// Whether its page is connected now. + pub connected: bool, +} + +/// The remotes the room key paired, kept beside it. +#[derive(Default)] +pub(super) struct Remotes { + paired: Mutex>, + /// Where they are kept, or `None` to keep them in memory alone. + file: Option, + /// Bumped when which remotes are read changes. + changed: tokio::sync::watch::Sender<()>, +} + +/// One remote as its file keeps it. +#[derive(serde::Serialize, serde::Deserialize)] +#[serde(deny_unknown_fields)] +struct Kept { + generation: u64, + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + removed: bool, +} + +impl Remotes { + /// The remotes kept in the file at `path`, which keeps them from now on. + pub(super) fn load(path: PathBuf) -> anyhow::Result { + let kept: BTreeMap = match std::fs::read_to_string(&path) { + Ok(text) => toml::from_str(&text).with_context(|| format!("reading {}", path.display()))?, + Err(err) if err.kind() == std::io::ErrorKind::NotFound => BTreeMap::new(), + Err(err) => return Err(err).with_context(|| format!("reading {}", path.display())), + }; + let paired = kept.into_iter().map(|(name, kept)| { + let remote = Remote { + generation: kept.generation, + removed: kept.removed, + ..Remote::default() + }; + (name, remote) + }); + Ok(Self { + paired: Mutex::new(paired.collect()), + file: Some(path), + changed: Default::default(), + }) + } + + /// Write `paired` to the file, if they are kept in one. + fn save(&self, paired: &BTreeMap) -> anyhow::Result<()> { + let Some(path) = &self.file else { return Ok(()) }; + let kept: BTreeMap<&String, Kept> = paired + .iter() + .map(|(name, remote)| { + let kept = Kept { + generation: remote.generation, + removed: remote.removed, + }; + (name, kept) + }) + .collect(); + super::replace(path, &toml::to_string(&kept)?) + } + + /// Apply `change` to the remote called `name` once the file holds it, so + /// a failed write leaves both as they were. + fn change(&self, name: &str, change: impl FnOnce(&mut Remote)) -> anyhow::Result { + let mut paired = self.paired.lock().unwrap(); + let before = paired.get(name).cloned(); + let remote = paired.entry(name.to_string()).or_default(); + change(remote); + let changed = remote.clone(); + if let Err(err) = self.save(&paired) { + match before { + Some(before) => paired.insert(name.to_string(), before), + None => paired.remove(name), + }; + return Err(err); + } + drop(paired); + self.changed.send_replace(()); + Ok(changed) + } + + /// The remotes whose presses are read, and in which generation. + fn read(&self) -> Vec<(String, u64)> { + let paired = self.paired.lock().unwrap(); + let read = paired.iter().filter(|(_, remote)| !remote.removed); + read.map(|(name, remote)| (name.clone(), remote.generation)).collect() + } + + /// Note whether the page of the remote called `name` is connected. + fn connect(&self, name: &str, connected: bool) { + if let Some(remote) = self.paired.lock().unwrap().get_mut(name) { + remote.connected = connected; + } + } +} + +impl Room { + /// Make a link pairing a phone called `name` as a remote, replacing any + /// earlier one, which keeps working until it expires: a signed token + /// cannot be taken back. A removed remote is paired again under the next + /// generation. + pub fn pair(&self, name: &str) -> anyhow::Result { + let name = name.trim(); + anyhow::ensure!(!name.is_empty(), "a remote needs a name"); + anyhow::ensure!( + !name.contains(['/', '\\']), + "a remote's name is one path segment, without `/` or `\\`" + ); + anyhow::ensure!(!name.starts_with('.'), "a remote's name cannot start with `.`"); + let page = self.page.join(PAGE).context("the join page has no page beside it")?; + let before = self.remotes.paired.lock().unwrap().get(name).cloned(); + let generation = before + .as_ref() + .map_or(0, |remote| remote.generation + u64::from(remote.removed)); + // Refused before anything changes, as a name that cannot be a remote. + let (token, expires) = self.sign_remote(name, generation)?; + let invite = Invite { + link: self.link(page, name, &token, generation), + expires, + }; + self.remotes.change(name, |remote| { + remote.removed = false; + remote.generation = generation; + remote.invite = Some(invite.clone()); + })?; + Ok(invite) + } + + /// Stop reading the presses of the remote called `name`, and renewing its + /// token, across restarts, until the host makes it a new link. + pub fn unpair(&self, name: &str) -> anyhow::Result<()> { + self.remotes.change(name, |remote| remote.removed = true)?; + Ok(()) + } + + /// Every phone paired as a remote, by name. + pub fn remotes(&self) -> BTreeMap { + self.remotes.paired.lock().unwrap().clone() + } + + /// A token for the remote called `name` in `generation`, and when it + /// expires: it publishes only its presses and reads only its desk. + pub(super) fn sign_remote(&self, name: &str, generation: u64) -> anyhow::Result<(String, kio::clock::Wall)> { + let invalid = || format!("cannot pair `{name}`"); + let claims = moq_auth::Claims::default() + .with_root(self.root.as_str()) + .with_publish([moq_auth::Pattern::literal( + press_path(name, generation).with_context(invalid)?.as_str(), + )?]) + .with_subscribe([moq_auth::Pattern::literal( + desk_path(name, generation).with_context(invalid)?.as_str(), + )?]); + self.sign_claims(claims) + .context("the room key cannot sign a remote's token") + } +} + +/// Where the remote called `name` in `generation` publishes its presses, in the room. +fn press_path(name: &str, generation: u64) -> anyhow::Result { + Ok(Path::new(&format!("{PRESS}/{name}/{generation}"))?) +} + +/// Where the studio publishes the desk of the remote called `name` in +/// `generation`, in the room. +fn desk_path(name: &str, generation: u64) -> anyhow::Result { + Ok(Path::new(&format!("{DESK}/{name}/{generation}"))?) +} + +/// What a remote drives: the studio's controls, or a test's stand-in. +pub trait Board: Send + Sync { + /// The show as a remote shows it now, without how its last press went. + fn desk(&self) -> Desk; + + /// How long the program shows each picture: no remote presses faster. + fn frame(&self) -> Duration; + + /// Do `action`, saying what it did or why it could not. + fn press(self: Arc, action: &Action) -> Result; +} + +/// The show as a remote's page shows it. +#[derive(Clone, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +pub struct Desk { + /// Whether the program is on air. + pub air: Air, + /// Whole seconds on air, while live, which the page counts on from. + pub clock: Option, + /// Whether Studio mode is on, so a pad cues its scene rather than taking it. + pub studio: bool, + /// The scene on program. + pub program: Option, + /// The scene cued, which TAKE and AUTO put on program. + pub cued: Option, + /// Every scene's pad, in the show's order. + pub scenes: Vec, + /// Why TAKE and AUTO cannot run now, if they cannot. + pub take_refused: Option, + /// Every sound's mute. + pub sounds: Vec, + /// How the page's last press went, once the studio has read one. + pub done: Option, +} + +/// Whether the program is on air, as [`OnAir`] says. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Air { + #[default] + Off, + Starting, + Live, + Ending, +} + +/// A scene's pad. +#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +pub struct Pad { + pub name: String, + /// Why it cannot go on program, if it cannot. + pub refused: Option, +} + +/// A sound's mute. +#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +pub struct Mute { + pub name: String, + pub muted: bool, +} + +/// How a press went. +#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +pub struct Done { + /// The page that pressed, as its press named it. + pub id: String, + /// The press, by its number. + pub seq: u64, + /// Whether it did what it asked. + pub ok: bool, + /// What it did, or why it did not. + pub said: String, +} + +/// A press, as a remote's page publishes it: its page, its number, and the +/// action, or none for the page saying it is there. +#[derive(Debug, PartialEq, serde::Serialize, serde::Deserialize)] +#[serde(deny_unknown_fields)] +struct Press { + id: String, + seq: u64, + action: Option, +} + +impl Board for Controls { + fn desk(&self) -> Desk { + let studio = self.studio(); + let on_air = self.engine.on_air(); + let state = self.engine.state.lock().unwrap(); + let collection = state.collection(); + let mut scenes = Vec::new(); + let mut sounds = Vec::new(); + for action in Action::listed(&state) { + match &action { + Action::Scene(name) => scenes.push(Pad { + name: name.clone(), + refused: self.refused(&state, &action), + }), + Action::Mute(name) => sounds.push(Mute { + name: name.clone(), + muted: collection.audio.get(name).is_some_and(|audio| audio.muted), + }), + _ => {} + } + } + let (air, clock) = match &on_air { + OnAir::Off => (Air::Off, None), + OnAir::Starting => (Air::Starting, None), + OnAir::Live(live) => { + let on = kio::clock::now().saturating_duration_since(live.since); + (Air::Live, Some(on.as_secs())) + } + OnAir::Ending => (Air::Ending, None), + }; + Desk { + air, + clock, + studio, + program: state.compositor.program_scene().map(str::to_string), + cued: state.cued().map(str::to_string), + scenes, + take_refused: self.refused(&state, &Action::Cut), + sounds, + done: None, + } + } + + fn frame(&self) -> Duration { + let framerate = self.engine.state.lock().unwrap().compositor.canvas().framerate; + Duration::from_secs(1) / framerate.max(1) + } + + fn press(self: Arc, action: &Action) -> Result { + let result = self.fire(action).map_err(|err| format!("{err:#}")); + // The status bar says what a remote did, as it does a hotkey. + self.report(action, result.clone()); + result + } +} + +/// Reads the presses of the room's remotes and publishes their desks, for as +/// long as the engine follows the room. +pub struct Service { + room: Arc, + board: Weak, +} + +impl Service { + /// Let `room`'s remotes drive `board`. + pub fn new(room: Arc, board: Weak) -> Self { + Self { room, board } + } + + /// Serve each remote paired and not removed, through the studio's session + /// in the room, until dropped. A room with no session serves none. + pub async fn run(self) { + let (Some(publish), Some(heard)) = (self.room.origin.get(), self.room.heard.get()) else { + return; + }; + let mut changes = self.room.remotes.changed.subscribe(); + let mut serving = BTreeMap::::new(); + loop { + let read = self.room.remotes.read(); + serving.retain(|name, serving| read.contains(&(name.clone(), serving.generation))); + for (name, generation) in read { + if serving.contains_key(&name) { + continue; + } + let paired = Paired { + room: self.room.clone(), + board: self.board.clone(), + name: name.clone(), + generation, + publish: publish.clone(), + }; + let task = tokio::spawn(paired.serve(heard.consume())); + let served = Serving { + room: self.room.clone(), + name: name.clone(), + generation, + task, + }; + serving.insert(name, served); + } + // The room outlives this loop, so its changes never end. + if changes.changed().await.is_err() { + return; + } + } + } +} + +/// A remote served, until dropped. +struct Serving { + room: Arc, + name: String, + generation: u64, + task: tokio::task::JoinHandle<()>, +} + +impl Drop for Serving { + fn drop(&mut self) { + self.task.abort(); + self.room.remotes.connect(&self.name, false); + } +} + +/// One remote the studio serves. +struct Paired { + room: Arc, + board: Weak, + name: String, + generation: u64, + /// What the studio's session in the room publishes. + publish: moq_net::origin::Producer, +} + +/// A remote's desk, published until dropped. +struct Published { + route: moq_net::origin::Route, + tokens: moq_net::track::Producer, + /// The key of the next token's group, since the token track has no timescale. + next: u64, + desks: moq_json::snapshot::Producer, + /// The key of the next desk's group, likewise. + written: u64, +} + +impl Drop for Published { + fn drop(&mut self) { + let _ = self.tokens.clone().finish(); + let _ = self.desks.clone().finish(); + self.route.broadcast().close(); + } +} + +impl Paired { + /// Read the presses of each page that connects, one after another, and + /// publish it a desk while it is connected, until dropped. + async fn serve(self, heard: moq_net::origin::Consumer) { + let path = match press_path(&self.name, self.generation) { + Ok(path) => path, + Err(err) => return tracing::error!(remote = self.name, error = format!("{err:#}"), "cannot read a remote"), + }; + let mut following = match heard.follow(&path, None) { + Ok(following) => following, + Err(err) => return tracing::error!(remote = self.name, %err, "cannot read a remote"), + }; + // How the last press went, across reconnects, so one read again is not done again. + let mut done = None; + let mut next = following.available().await; + loop { + let broadcast = match next { + Ok(Some(broadcast)) => broadcast, + // Not connected until a page serves the path. + Ok(None) => { + next = following.available().await; + continue; + } + // A closed origin answers every later call at once. + Err(moq_net::Error::Closed) => return, + Err(err) => { + tracing::debug!(remote = self.name, %err, "cannot follow a remote"); + next = following.available().await; + continue; + } + }; + self.room.remotes.connect(&self.name, true); + tokio::select! { + result = self.drive(broadcast, &mut done) => { + if let Err(err) = result { + tracing::debug!(remote = self.name, error = format!("{err:#}"), "stopped reading a remote"); + } + next = following.available().await; + } + changed = following.available() => next = changed, + } + self.room.remotes.connect(&self.name, false); + } + } + + /// Publish the page at `broadcast` its desk and renew its token, and do + /// each press it makes, keeping how the last went in `done`, until it + /// leaves or the studio closes. + async fn drive(&self, broadcast: moq_net::broadcast::Consumer, done: &mut Option) -> anyhow::Result<()> { + let route = self.publish.route( + desk_path(&self.name, self.generation)?, + moq_net::origin::RouteOptions::hidden(), + )?; + let tokens = route.broadcast().create_track(TOKEN_TRACK, None)?; + let desk = route.broadcast().create_track(DESK_TRACK, None)?; + // Each desk whole, so a page joining late reads one group. + let desks = moq_json::snapshot::Producer::new(desk, moq_json::snapshot::Config::default().with_deltas(false)); + let mut published = Published { + route, + tokens, + next: 0, + desks, + written: 0, + }; + self.renew(&mut published)?; + let board = self.board.upgrade().context("the studio closed")?; + let desk = Desk { + done: done.clone(), + ..board.desk() + }; + self.show(&mut published, desk)?; + drop(board); + published.route.announce()?; + + let mut presses = Presses::new(broadcast.track(PRESS_TRACK)?); + let period = self.room.expiry / 2; + let mut renew = kio::clock::interval_at(kio::clock::now().saturating_add(period), period); + // The desk is built again only on a press or a tally tick, since each + // build waits on the state the render loop holds. + let mut tally = kio::clock::interval_at(kio::clock::now().saturating_add(TALLY), TALLY); + tally.set_missed_tick_behavior(kio::clock::MissedTickBehavior::Skip); + let mut listening = false; + // When the next press may be read: a program frame after the last. + let mut open = kio::clock::now(); + loop { + let mut pressed = false; + tokio::select! { + _ = tally.tick() => {} + _ = renew.tick() => { + self.renew(&mut published)?; + continue; + } + press = async { + kio::clock::sleep_until(open).await; + presses.latest().await + } => { + pressed = true; + match press { + Ok(Some(press)) => *done = Some(self.handle(press, &mut listening, done.as_ref())), + Ok(None) => return Ok(()), + Err(moq_json::Error::Net(err)) => return Err(err.into()), + // Refused whole, as one the page cannot be told about. + Err(err) => tracing::debug!(remote = self.name, %err, "refused a press"), + } + } + } + let board = self.board.upgrade().context("the studio closed")?; + if pressed { + // The frame now, as the show's frame rate may have changed. + open = kio::clock::now().saturating_add(board.frame()); + } + let desk = Desk { + done: done.clone(), + ..board.desk() + }; + self.show(&mut published, desk)?; + } + } + + /// Publish `desk`, unless it is the one published last. + fn show(&self, published: &mut Published, desk: Desk) -> anyhow::Result<()> { + let at = moq_net::Timestamp::unscaled(published.written); + if published.desks.update(moq_json::Timed::new(&desk, at))?.is_some() { + published.written += 1; + } + Ok(()) + } + + /// Publish the remote a fresh token. + fn renew(&self, published: &mut Published) -> anyhow::Result<()> { + let (token, _) = self.room.sign_remote(&self.name, self.generation)?; + let at = moq_net::Timestamp::unscaled(published.next); + published.tokens.write_frame(at, token)?; + published.next += 1; + Ok(()) + } + + /// Do `press`, saying how it went, after the `last` one. The first press + /// read from a connection is one the studio may have done before it was + /// listening, so it is never done again: it is the `last` if the page + /// reconnected. The page names itself first on each connection, so any + /// other first press is one the page made before the studio restarted or + /// rejoined the room, which was almost surely done then; pressing it + /// again could undo it, as with a mute, so the page is told to check. + fn handle(&self, press: Press, listening: &mut bool, last: Option<&Done>) -> Done { + let first = !std::mem::replace(listening, true); + if first && let Some(last) = last.filter(|last| last.id == press.id && press.seq <= last.seq) { + return last.clone(); + } + let result = match press.action { + None => Ok("connected".to_string()), + Some(_) if first => Err(RESTARTED.to_string()), + Some(action) if !action.remote() => Err(format!("{action} is not for a remote")), + Some(action) => match self.board.upgrade() { + Some(board) => board.press(&action), + None => Err("the studio closed".to_string()), + }, + }; + let (ok, said) = match result { + Ok(said) => (true, said), + Err(said) => (false, said), + }; + Done { + id: press.id, + seq: press.seq, + ok, + said, + } + } +} + +/// A remote's presses, each costing the studio one parse of at most +/// [`MAX_PRESS`] bytes: only the first frame of the newest group, decoded on +/// its own. The page writes each press as a group of its own; whatever else a +/// group holds, such as the merge patches a snapshot track may carry, is +/// never read, so a page cannot make one read do more work. +struct Presses { + track: Pressing, + decoder: moq_json::snapshot::Decoder, +} + +enum Pressing { + Subscribing(moq_net::track::Subscribing), + Reading { + // Boxed: the reader dwarfs the pending subscription. + groups: Box, + /// The newest group, until its first frame is read. + group: Option, + /// Whether the track has ended. + ended: bool, + }, +} + +impl Presses { + fn new(track: moq_net::track::Consumer) -> Self { + let mut config = moq_json::snapshot::consumer::Config::default(); + config.max_size = Some(MAX_PRESS); + Self { + track: Pressing::Subscribing(track.subscribe(None).into_inner()), + decoder: moq_json::snapshot::Decoder::new(config), + } + } + + /// The newest press, skipping those a newer one superseded, or `None` once + /// the page stops publishing. An error other than [`moq_json::Error::Net`] + /// refuses that press alone. + async fn latest(&mut self) -> Result, moq_json::Error> { + kio::wait(|waiter| self.poll_latest(waiter)).await + } + + fn poll_latest(&mut self, waiter: &kio::Waiter) -> Poll, moq_json::Error>> { + if let Pressing::Subscribing(pending) = &self.track { + let groups = ready!(pending.poll_ok(waiter))?.ordered(); + self.track = Pressing::Reading { + groups: Box::new(groups), + group: None, + ended: false, + }; + } + let Pressing::Reading { groups, group, ended } = &mut self.track else { + unreachable!("subscribed above"); + }; + loop { + // The newest group the reader has; a newer one supersedes the rest. + while !*ended { + match groups.poll_next_group(waiter)? { + Poll::Ready(Some(item)) => { + if let Some(newer) = item.group() { + *group = Some(newer); + } + } + Poll::Ready(None) => *ended = true, + Poll::Pending => break, + } + } + let Some(current) = group else { + return match ended { + true => Poll::Ready(Ok(None)), + false => Poll::Pending, + }; + }; + let read = ready!(current.poll_read_frame(waiter)); + // One frame a group, whatever follows it. + *group = None; + match read { + Ok(Some(frame)) => { + self.decoder.snapshot(&frame.payload)?; + return Poll::Ready(self.decoder.decode()); + } + Ok(None) => continue, + Err(err) => { + // A newer group follows, as a lost one is superseded. + tracing::debug!(%err, "lost a press"); + continue; + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::super::tests::{EXPIRY, Relay, fragment, key}; + use super::*; + use url::Url; + + /// A board that keeps what it was asked to do, and how often its desk was built. + struct Fake { + desk: Mutex, + frame: Mutex, + pressed: Mutex>, + built: std::sync::atomic::AtomicUsize, + } + + impl Fake { + fn new(frame: Duration) -> Arc { + let desk = Desk { + scenes: vec![Pad { + name: "Gameplay".into(), + refused: None, + }], + sounds: vec![Mute { + name: "Desk mic".into(), + muted: false, + }], + ..Desk::default() + }; + Arc::new(Self { + desk: Mutex::new(desk), + frame: Mutex::new(frame), + pressed: Mutex::default(), + built: Default::default(), + }) + } + + fn pressed(&self) -> Vec { + self.pressed.lock().unwrap().clone() + } + + fn built(&self) -> usize { + self.built.load(std::sync::atomic::Ordering::Relaxed) + } + } + + impl Board for Fake { + fn desk(&self) -> Desk { + self.built.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + self.desk.lock().unwrap().clone() + } + + fn frame(&self) -> Duration { + *self.frame.lock().unwrap() + } + + fn press(self: Arc, action: &Action) -> Result { + self.pressed.lock().unwrap().push(action.clone()); + Ok(format!("did {action}")) + } + } + + fn room(key: moq_auth::Key) -> Room { + let connect = Url::parse("https://relay.example/show").unwrap(); + let page = Url::parse("https://studio.example/join/").unwrap(); + Room::new(&connect, "guests", key, page, EXPIRY).unwrap() + } + + fn press(seq: u64, action: Option) -> Press { + Press { + id: "page".into(), + seq, + action, + } + } + + /// A page publishing presses for the remote called `name` in + /// `generation` into `origin`, as `js/join/src/remote.ts` does. + struct Page { + _route: moq_net::origin::Route, + track: moq_net::track::Producer, + presses: moq_json::snapshot::Producer, + /// The next group's start, as each write is a group of its own. + next: u64, + } + + impl Page { + fn open(origin: &moq_net::origin::Producer, name: &str, generation: u64) -> Self { + let route = origin + .route(press_path(name, generation).unwrap(), Default::default()) + .unwrap(); + let track = route.broadcast().create_track(PRESS_TRACK, None).unwrap(); + let config = moq_json::snapshot::Config::default().with_deltas(false); + Self { + _route: route, + track: track.clone(), + presses: moq_json::snapshot::Producer::new(track, config), + next: 0, + } + } + + fn press(&mut self, press: Press) { + let at = moq_net::Timestamp::unscaled(self.next); + self.next += 1; + self.presses.update(moq_json::Timed::new(&press, at)).unwrap(); + } + + /// Write each of `frames` in one group, whatever they are. + fn raw(&mut self, frames: &[&[u8]]) { + let at = moq_net::Timestamp::unscaled(self.next); + self.next += 1; + let mut group = self.track.append_group(at).unwrap(); + for frame in frames { + group.write_frame(at, frame.to_vec()).unwrap(); + } + group.finish().unwrap(); + } + } + + /// The desks the studio publishes the remote called `name` in `generation`. + async fn read_desks( + origin: moq_net::origin::Consumer, + name: &str, + generation: u64, + ) -> moq_json::snapshot::Consumer { + let broadcast = origin + .follow(desk_path(name, generation).unwrap(), None) + .unwrap() + .available() + .await + .unwrap() + .expect("an instance"); + moq_json::snapshot::Consumer::new(broadcast.track(DESK_TRACK).unwrap(), Default::default()) + } + + /// The first desk that says how the press numbered `seq` went. + async fn desk(desks: &mut moq_json::snapshot::Consumer, seq: u64) -> Desk { + loop { + let desk = desks.latest().await.unwrap().expect("a desk").value; + if desk.done.as_ref().is_some_and(|done| done.seq >= seq) { + return desk; + } + } + } + + /// How the press numbered `seq` went, once the desk says. + async fn done(desks: &mut moq_json::snapshot::Consumer, seq: u64) -> Done { + desk(desks, seq).await.done.expect("how it went") + } + + /// A link pairs one remote, whose token publishes only its presses and + /// reads only its desk, in its generation, which a removal moves on. + #[test] + fn a_link_pairs_one_remote_in_its_generation() { + let key = key("show/guests", &["**"], &["**"]); + let room = room(key.clone()); + let invite = room.pair(" phone ").unwrap(); + assert_eq!(invite.link.path(), "/join/remote.html"); + let link = fragment(&invite.link); + assert_eq!(link["url"], "https://relay.example/show/guests"); + assert_eq!(link["name"], "phone"); + assert_eq!(link["generation"], "0"); + let claims = key.verify(&link["jwt"]).unwrap(); + assert_eq!(claims.root, "show/guests"); + let patterns = |texts: &[&str]| texts.iter().map(|text| text.parse().unwrap()).collect(); + assert_eq!(claims.publish.path, patterns(&[".press/phone/0"])); + assert_eq!(claims.subscribe.path, patterns(&[".remote/phone/0"])); + + // A new link while paired keeps the generation, and a removal moves it on. + assert_eq!(fragment(&room.pair("phone").unwrap().link)["generation"], "0"); + room.unpair("phone").unwrap(); + assert!(room.remotes.read().is_empty(), "a removed remote is still read"); + assert_eq!(fragment(&room.pair("phone").unwrap().link)["generation"], "1"); + assert_eq!(room.remotes.read(), [("phone".to_string(), 1)]); + + for name in ["", " ", "a/b", ".hidden"] { + assert!(room.pair(name).is_err(), "{name:?}"); + } + assert_eq!(room.remotes().len(), 1, "a refused name is paired"); + } + + /// Who is paired, removed, and in which generation outlasts a restart, + /// whichever scene file the studio opens, so a removed remote stays + /// unread. + #[test] + fn pairing_outlasts_a_restart() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("room.jwk"); + key("show/guests", &["**"], &["**"]).to_file(&path).unwrap(); + let connect = Url::parse("https://relay.example/show").unwrap(); + let config = super::super::Config { + room: Some("guests".into()), + room_key: Some(path), + join_page: None, + guest_expiry: Some(EXPIRY.into()), + }; + let start = || config.load(Some(&connect)).unwrap().unwrap(); + + let room = start(); + room.pair("phone").unwrap(); + room.pair("tablet").unwrap(); + room.unpair("tablet").unwrap(); + let file = dir.path().join("room.remotes.toml"); + assert!(file.exists()); + + let room = start(); + assert_eq!(room.remotes.read(), [("phone".to_string(), 0)]); + assert!(room.remotes()["tablet"].removed); + room.pair("tablet").unwrap(); + assert_eq!(start().remotes()["tablet"].generation, 1); + + // A removal the file cannot hold is not made. + std::fs::remove_file(&file).unwrap(); + std::fs::create_dir(&file).unwrap(); + assert!(room.unpair("phone").is_err()); + assert!(!room.remotes()["phone"].removed); + } + + /// A page's presses drive the board, its newest only, at most one a + /// frame, and only those a remote may fire; the desk says how each went. + /// A page's first press is never done, since the studio may have done it + /// before it restarted. A removal stops reading it at once. + #[tokio::test] + async fn a_remote_drives_the_board_one_press_a_frame() { + const FRAME: Duration = Duration::from_millis(20); + let clock = kio::clock::Manual::new().install(&tokio::runtime::Handle::current()); + let room = Arc::new(room(key("show/guests", &["**"], &["**"]))); + let publish = moq_tokio::origin::spawn(); + let heard = moq_tokio::origin::spawn(); + let _ = room.origin.set(publish.clone()); + let _ = room.heard.set(heard.clone()); + let fake = Fake::new(FRAME); + let board: Arc = fake.clone(); + let _service = tokio::spawn(Service::new(room.clone(), Arc::downgrade(&board)).run()); + room.pair("phone").unwrap(); + + let mut page = Page::open(&heard, "phone", 0); + page.press(press(0, None)); + let mut desks = read_desks(publish.consume(), "phone", 0).await; + let connected = desk(&mut desks, 0).await; + assert_eq!(connected.done.unwrap().said, "connected"); + assert_eq!(connected.scenes, fake.desk.lock().unwrap().scenes, "the board's desk"); + assert!(room.remotes()["phone"].connected); + + let scene = Action::Scene("Gameplay".into()); + page.press(press(1, Some(scene.clone()))); + clock.advance(FRAME); + let first = done(&mut desks, 1).await; + assert!(first.ok, "{first:?}"); + assert_eq!(fake.pressed(), std::slice::from_ref(&scene)); + + // A burst within one frame: only the newest is done, a frame later. + for seq in 2..=11 { + page.press(press(seq, Some(Action::Mute(format!("sound {seq}"))))); + } + clock.advance(FRAME); + assert_eq!(done(&mut desks, 2).await.seq, 11); + assert_eq!(fake.pressed(), [scene.clone(), Action::Mute("sound 11".into())]); + + // Outside the vocabulary. + page.press(press(12, Some(Action::GoLive))); + clock.advance(FRAME); + let refused = done(&mut desks, 12).await; + assert!(!refused.ok); + assert_eq!(refused.said, "Go live is not for a remote"); + assert_eq!(fake.pressed().len(), 2); + + // Not a press, or more than one could be: refused whole, saying + // nothing, and the next press is still read. + // Each is read on a frame of its own, as `latest` skips any it passed. + let large = vec![b' '; MAX_PRESS + 1]; + for frame in [ + br#"{"id":"page","seq":13,"action":{"Launch":"rockets"}}"#.as_slice(), + &large, + ] { + page.raw(&[frame]); + clock.advance(FRAME); + for _ in 0..20 { + tokio::task::yield_now().await; + } + } + // A press is its group's first frame alone: patches after it, as many + // as a page sends, are never applied. + let patch = br#"{"action":{"Scene":"Gameplay"}}"#.as_slice(); + let mut frames = vec![br#"{"id":"page","seq":13,"action":"GoLive"}"#.as_slice()]; + frames.extend(std::iter::repeat_n(patch, 1000)); + page.raw(&frames); + clock.advance(FRAME); + let refused = done(&mut desks, 13).await; + assert_eq!(refused.said, "Go live is not for a remote"); + assert_eq!(fake.pressed().len(), 2); + + // The page back on a new connection: the press read first is the + // last it made, which is not done again. + drop(page); + while desks.latest().await.unwrap().is_some() {} + let mut page = Page::open(&heard, "phone", 0); + page.press(press(13, Some(Action::GoLive))); + let mut desks = read_desks(publish.consume(), "phone", 0).await; + clock.advance(FRAME); + assert_eq!(done(&mut desks, 13).await, refused, "the last press, again"); + + // A page whose press the studio may have done before it restarted, + // which is told to check it, not done again. + drop(page); + while desks.latest().await.unwrap().is_some() {} + let mut page = Page::open(&heard, "phone", 0); + page.press(Press { + id: "reloaded".into(), + ..press(1, Some(scene.clone())) + }); + let mut desks = read_desks(publish.consume(), "phone", 0).await; + clock.advance(FRAME); + let again = loop { + let desk = desks.latest().await.unwrap().unwrap().value; + if let Some(done) = desk.done.filter(|done| done.id == "reloaded") { + break done; + } + }; + assert!(!again.ok); + assert_eq!(again.said, RESTARTED); + assert_eq!(fake.pressed().len(), 2, "done again"); + + // Removed: the desk ends, and nothing more is read. + room.unpair("phone").unwrap(); + while desks.latest().await.is_ok_and(|desk| desk.is_some()) {} + page.press(press(2, Some(scene))); + clock.advance(FRAME); + tokio::task::yield_now().await; + assert_eq!(fake.pressed().len(), 2, "a removed remote drives"); + assert!(!room.remotes()["phone"].connected); + } + + /// A connected page's desk is built on a press and at the tally rate, + /// not every program frame, and still shows what the studio changed + /// itself. Presses wait out the frame the program shows now, whatever it + /// was when the page connected. + #[tokio::test] + async fn a_remote_keeps_pace_with_the_show() { + const FRAME: Duration = Duration::from_millis(20); + let clock = kio::clock::Manual::new().install(&tokio::runtime::Handle::current()); + let room = Arc::new(room(key("show/guests", &["**"], &["**"]))); + let publish = moq_tokio::origin::spawn(); + let heard = moq_tokio::origin::spawn(); + let _ = room.origin.set(publish.clone()); + let _ = room.heard.set(heard.clone()); + let fake = Fake::new(FRAME); + let board: Arc = fake.clone(); + let _service = tokio::spawn(Service::new(room.clone(), Arc::downgrade(&board)).run()); + room.pair("phone").unwrap(); + + let mut page = Page::open(&heard, "phone", 0); + page.press(press(0, None)); + let mut desks = read_desks(publish.consume(), "phone", 0).await; + desk(&mut desks, 0).await; + + // A second of program frames with nothing pressed builds a desk a tally tick. + let built = fake.built(); + for _ in 0..Duration::from_secs(1).div_duration_f64(FRAME) as u32 { + clock.advance(FRAME); + for _ in 0..20 { + tokio::task::yield_now().await; + } + } + let ticks = Duration::from_secs(1).div_duration_f64(TALLY) as usize; + assert_eq!(fake.built() - built, ticks, "desks built in a second"); + + // What the studio changed itself reaches the page within a tick. + fake.desk.lock().unwrap().program = Some("Gameplay".into()); + clock.advance(TALLY); + let program = loop { + let desk = desks.latest().await.unwrap().unwrap().value; + if desk.program.is_some() { + break desk.program; + } + }; + assert_eq!(program.as_deref(), Some("Gameplay")); + + // The show slows to a frame a second: the next press waits for it. + *fake.frame.lock().unwrap() = Duration::from_secs(1); + let scene = Action::Scene("Gameplay".into()); + page.press(press(1, Some(scene.clone()))); + clock.advance(FRAME); + assert!(done(&mut desks, 1).await.ok); + page.press(press(2, Some(scene.clone()))); + for _ in 0..10 { + clock.advance(FRAME); + for _ in 0..20 { + tokio::task::yield_now().await; + } + } + assert_eq!(fake.pressed().len(), 1, "pressed within the program's frame"); + clock.advance(Duration::from_secs(1)); + assert!(done(&mut desks, 2).await.ok); + assert_eq!(fake.pressed(), [scene.clone(), scene]); + } + + /// A page that opens the link reaches the relay with its token, presses + /// through it to the studio's session in the room, and reads its desk. + #[tokio::test(flavor = "multi_thread")] + async fn a_remote_drives_through_a_relay_that_verifies_tokens() { + const TIMEOUT: Duration = Duration::from_secs(10); + let relay = Relay::start().await; + let room = relay.room(); + let fake = Fake::new(Duration::from_millis(10)); + let board: Arc = fake.clone(); + let _service = tokio::spawn(Service::new(room.clone(), Arc::downgrade(&board)).run()); + + let published = moq_tokio::origin::spawn(); + let read = moq_tokio::origin::spawn(); + let _page = relay.join(&room.pair("phone").unwrap().link, &published, &read); + let mut page = Page::open(&published, "phone", 0); + page.press(press(0, None)); + let mut desks = kio::clock::timeout(TIMEOUT, read_desks(read.consume(), "phone", 0)) + .await + .expect("the studio publishes the desk"); + kio::clock::timeout(TIMEOUT, done(&mut desks, 0)) + .await + .expect("the studio hears the page"); + + let scene = Action::Scene("Gameplay".into()); + page.press(press(1, Some(scene.clone()))); + let done = kio::clock::timeout(TIMEOUT, done(&mut desks, 1)) + .await + .expect("the press reaches the studio"); + assert!(done.ok, "{done:?}"); + assert_eq!(fake.pressed(), [scene]); + } +} diff --git a/rs/moq-studio/src/kit/icon.rs b/rs/moq-studio/src/kit/icon.rs index 6bc4792f..94feccfb 100644 --- a/rs/moq-studio/src/kit/icon.rs +++ b/rs/moq-studio/src/kit/icon.rs @@ -81,12 +81,14 @@ pub enum Icon { Search, /// A moment marked in the show. Marker, + /// A phone paired as a remote. + Phone, } impl Icon { /// Every icon, for the gallery. #[cfg(test)] - pub const ALL: [Self; 67] = [ + pub const ALL: [Self; 68] = [ Self::Undo, Self::Redo, Self::Settings, @@ -154,6 +156,7 @@ impl Icon { Self::Info, Self::Search, Self::Marker, + Self::Phone, ]; fn source(self) -> ImageSource<'static> { @@ -225,6 +228,7 @@ impl Icon { Self::Info => include_image!("../../assets/icons/info.svg"), Self::Search => include_image!("../../assets/icons/search.svg"), Self::Marker => include_image!("../../assets/icons/flag.svg"), + Self::Phone => include_image!("../../assets/icons/smartphone.svg"), } } diff --git a/rs/moq-studio/src/shots.rs b/rs/moq-studio/src/shots.rs index d95e7662..a8ae8e27 100644 --- a/rs/moq-studio/src/shots.rs +++ b/rs/moq-studio/src/shots.rs @@ -84,6 +84,9 @@ enum View { /// Settings' Guests pane, its room unable to join at launch, as its key /// file is missing. GuestsFailed, + /// Settings' Remote pane in a room, one phone paired and removed, and + /// another just paired by its name and Pair a phone. + Paired, /// The scene graph's window. Graph, /// The stream's health in detail. @@ -439,6 +442,12 @@ views! { /// Settings, Guests, when the room saved cannot be joined, such as when /// its key file is gone: why, and Try again. settings_guests_failed: Live, relayed(View::GuestsFailed); + /// Settings, Remote with no room yet, which a remote joins. + settings_remote: Still, studio(View::Setting(Setting::Remote)); + /// Settings, Remote with a phone just paired: the code its camera scans, + /// when its link expires, and every phone paired, one removed. Live, as + /// each link carries a token of its own. + settings_remote_paired: Live, studio(View::Paired); /// Settings, Advanced: published scenes and sources, the virtual camera, /// the effects, and the scene graph. settings_advanced: Still, studio(View::Setting(Setting::Advanced)); @@ -1018,7 +1027,8 @@ fn the_palette_finds_and_runs_anything() { /// the format turn the program, keeping its frame rate. A show whose room /// key is missing still opens, and once the key is back, Try again joins its /// room. The Guests pane refuses a room guests could not join, saying why, -/// and moves to the room it saves at once, off air. The recordings folder +/// and moves to the room it saves at once, off air, dropping a phone's link +/// into the room it left. The recordings folder /// moves only once no recording is under way. The scene /// graph's Open closes Settings for the graph. It draws no view. #[test] @@ -1120,6 +1130,12 @@ fn settings_change_the_show_and_move_the_guests() { engine.state.lock().unwrap().collection().audio["alice"].muted, "{muted}" ); + // Remote, with no room to pair in, sends the host to the room's pane. + harness.get_by_label("Remote").click(); + harness.run_steps(2); + harness.get_by_label("Set up the room").click(); + harness.run_steps(2); + assert!(harness.query_by_label("Try again").is_some(), "Guests is open"); // The key is back in place, under the same name; the relay's path is the // room's parent. for room in ["guests", "other"] { @@ -1129,6 +1145,15 @@ fn settings_change_the_show_and_move_the_guests() { } harness.get_by_label("Try again").click(); joined(&mut harness, "guests"); + // A phone paired in this room, whose link stays on screen... + harness.get_by_label("Remote").click(); + harness.run_steps(2); + fill(&mut harness, 0, "phone"); + harness.get_by_label("Pair a phone").click(); + harness.run_steps(2); + assert!(harness.query_by_label("Copy link").is_some(), "the phone's link shows"); + harness.get_by_label("Guests").click(); + harness.run_steps(2); fill(&mut harness, 0, "other"); fill(&mut harness, 1, ""); harness.get_by_label("Save").click(); @@ -1149,6 +1174,13 @@ fn settings_change_the_show_and_move_the_guests() { joined(&mut harness, "other"); let saved = std::fs::read_to_string("show.stream.toml").unwrap(); assert!(saved.contains("room = \"other\""), "{saved}"); + // ...until the room is another, which its link cannot join. + harness.get_by_label("Remote").click(); + harness.run_steps(2); + assert!( + harness.query_by_label("Copy link").is_none(), + "a link into the room left shows" + ); // The recordings folder stays put while a recording is under way, then // moves with the file naming it. @@ -1852,6 +1884,7 @@ fn shoot(open: &Open, gpu: &engine::Gpu, runtime: &tokio::runtime::Runtime, show View::Setting(setting) | View::OnAirSetting(setting) => studio.open(*setting), View::Restart | View::CustomCanvas => studio.open(Setting::Show), View::GuestsFailed => studio.open(Setting::Guests), + View::Paired => studio.open(Setting::Remote), View::Graph => studio.open_graph(), View::Health => studio.details(), View::Going => studio.going(), @@ -1936,6 +1969,15 @@ fn shoot(open: &Open, gpu: &engine::Gpu, runtime: &tokio::runtime::Runtime, show let stalled = Duration::from_secs(4); engine.state.lock().unwrap().stalled.insert("host".into(), stalled); } + if let View::Paired = view { + let connect = url::Url::parse("https://relay.example/show").unwrap(); + let page = url::Url::parse("https://join.studio.moq.dev/").unwrap(); + let key = crate::guests::tests::key("show/room", &["**"], &["**"]); + let room = crate::guests::Room::new(&connect, "room", key, page, crate::guests::tests::EXPIRY).unwrap(); + room.pair("tablet").unwrap(); + room.unpair("tablet").unwrap(); + engine.join(Some(crate::guests::Joining::offline(room))).unwrap(); + } if let View::Joined = view { let mut state = engine.state.lock().unwrap(); let connect = url::Url::parse("https://relay.example/show").unwrap(); @@ -1982,6 +2024,14 @@ fn shoot(open: &Open, gpu: &engine::Gpu, runtime: &tokio::runtime::Runtime, show harness.get_all_by_label("Custom").next().unwrap().click(); harness.run_steps(4); } + View::Paired => { + harness.get_by_role(Role::TextInput).click(); + harness.run_steps(2); + harness.get_by_role(Role::TextInput).type_text("Phone"); + harness.run_steps(2); + harness.get_by_label("Pair a phone").click(); + harness.run_steps(4); + } View::GuestsFailed => runtime .block_on(kio::clock::timeout(SETTLE, async { while harness.query_by_label_contains("Guests cannot join").is_none() { diff --git a/rs/moq-studio/src/ui.rs b/rs/moq-studio/src/ui.rs index fe13ba52..a56cc090 100644 --- a/rs/moq-studio/src/ui.rs +++ b/rs/moq-studio/src/ui.rs @@ -44,6 +44,7 @@ mod live; mod mixer; mod multiview; mod palette; +mod remotes; mod review; mod settings; mod shell; @@ -109,18 +110,21 @@ pub(crate) enum Setting { Hotkeys, /// The room guests join, its key, the join page, and how long a link lasts. Guests, + /// The phones paired as remotes. + Remote, /// Published scenes and sources, the virtual camera, the effects, and the /// scene graph. Advanced, } impl Setting { - pub(crate) const ALL: [Self; 6] = [ + pub(crate) const ALL: [Self; 7] = [ Self::Show, Self::Stream, Self::Recording, Self::Hotkeys, Self::Guests, + Self::Remote, Self::Advanced, ]; @@ -131,6 +135,7 @@ impl Setting { Self::Recording => "Recording", Self::Hotkeys => "Hotkeys", Self::Guests => "Guests", + Self::Remote => "Remote", Self::Advanced => "Advanced", } } @@ -142,6 +147,7 @@ impl Setting { Self::Recording => Icon::Record, Self::Hotkeys => Icon::Hotkeys, Self::Guests => Icon::Guests, + Self::Remote => Icon::Phone, Self::Advanced => Icon::Advanced, } } @@ -175,6 +181,7 @@ impl Setting { ("Link lifetime", &["expiry", "expires"]), ("Appear on their own", &["tile"]), ], + Self::Remote => &[], Self::Advanced => &[ ("Publish a scene as its own broadcast", &[]), ("Publish a source independently", &[]), @@ -193,6 +200,7 @@ impl Setting { Self::Recording => &["folder", "record"], Self::Hotkeys => &["keys", "shortcuts", "bind"], Self::Guests => &["invite", "join", "room"], + Self::Remote => &["phone", "pair", "qr", "control"], Self::Advanced => &["publish"], } } @@ -205,6 +213,9 @@ impl Setting { Self::Recording => "What the record button saves, from any page.", Self::Hotkeys => "Kept on this computer, not in the show.", Self::Guests => "People who join from a web page, appearing as sources. Kept beside the show.", + Self::Remote => { + "Phones that drive the show from a page their code opens: the scenes, TAKE, and the mutes. Kept beside the room key." + } Self::Advanced => "Things most shows never touch.", } } @@ -335,6 +346,8 @@ impl Studio { bound: BTreeMap::new(), guest: String::new(), shown: None, + phone: String::new(), + paired: None, message: None, faults: Vec::new(), reloads: 0, @@ -812,6 +825,10 @@ struct Panels { guest: String, /// The join link on screen. shown: Option, + /// The name typed for the next remote. + phone: String, + /// The remote's link on screen. + paired: Option, /// The outcome of the last action that happened nowhere in particular, /// and when, shown for a while. message: Option<(Result, kio::clock::Instant)>, @@ -2121,6 +2138,16 @@ struct Shown { invite: crate::guests::Invite, /// Or why the link cannot be one, so it can still be copied. qr: Result, + /// The room it joins. Weak, so a room the studio left is never kept, and + /// another never takes its place in [`Self::joins`]. + room: std::sync::Weak, +} + +impl Shown { + /// Whether it joins `room`, rather than one the studio has left. + fn joins(&self, room: &Arc) -> bool { + std::ptr::eq(self.room.as_ptr(), Arc::as_ptr(room)) + } } /// `text` as a QR code, one texel per module, inside the quiet zone scanners need. diff --git a/rs/moq-studio/src/ui/layers.rs b/rs/moq-studio/src/ui/layers.rs index d30ee10f..37de8254 100644 --- a/rs/moq-studio/src/ui/layers.rs +++ b/rs/moq-studio/src/ui/layers.rs @@ -753,6 +753,8 @@ impl Panels { } return; }; + // A link into a room the studio has left joins nothing. + self.shown.take_if(|shown| !shown.joins(&room)); ui.horizontal(|ui| { ui.label("Name"); @@ -762,7 +764,7 @@ impl Panels { let name = self.guest.trim().to_string(); match room.invite(&name) { Ok(invite) => { - self.show(ui.ctx(), name, invite); + self.show(ui.ctx(), &room, name, invite); self.guest.clear(); } Err(err) => self.fail(Spot::Add, Err(err)), @@ -859,11 +861,11 @@ impl Panels { }); if let Some((name, link)) = show { - self.show(ui.ctx(), name, link); + self.show(ui.ctx(), &room, name, link); } if let Some(name) = invite { match room.invite(&name) { - Ok(link) => self.show(ui.ctx(), name, link), + Ok(link) => self.show(ui.ctx(), &room, name, link), Err(err) => self.fail(Spot::Add, Err(err)), } } @@ -889,8 +891,15 @@ impl Panels { } /// Put `invite`'s link for `name` on screen as a QR code. - pub(super) fn show(&mut self, ctx: &egui::Context, name: String, invite: crate::guests::Invite) { + pub(super) fn show( + &mut self, + ctx: &egui::Context, + room: &Arc, + name: String, + invite: crate::guests::Invite, + ) { let qr = qr(ctx, invite.link.as_str()); - self.shown = Some(Shown { name, invite, qr }); + let room = Arc::downgrade(room); + self.shown = Some(Shown { name, invite, qr, room }); } } diff --git a/rs/moq-studio/src/ui/remotes.rs b/rs/moq-studio/src/ui/remotes.rs new file mode 100644 index 00000000..5443c35f --- /dev/null +++ b/rs/moq-studio/src/ui/remotes.rs @@ -0,0 +1,169 @@ +//! Settings, Remote: pair a phone by its QR code, see which are connected, +//! and remove one. + +use std::sync::Arc; + +use egui::{RichText, Ui}; + +use super::{Panels, Setting, Shown, expiry, kit, qr}; +use crate::kit::Icon; + +impl Panels { + /// Pair phones as remotes of the guests' room, and list those paired. + pub(super) fn remote_settings(&mut self, ui: &mut Ui) { + let Some(room) = self.engine.room() else { + let said = match self.controls.room_failure() { + Some(failure) => format!("The room cannot be joined: {failure}"), + None => "A remote joins the guests' room on your relay, which needs a room and its key.".into(), + }; + ui.label(RichText::new(said).font(kit::ui(12.5)).color(kit::TEXT)); + ui.add_space(8.0); + if ui.add(kit::button(Icon::Guests, "Set up the room")).clicked() { + self.setting = Some(Setting::Guests); + } + return; + }; + // A link into a room the studio has left joins nothing. + self.paired.take_if(|shown| !shown.joins(&room)); + + let mut pair = None; + ui.horizontal(|ui| { + ui.label("Name"); + let field = ui.add( + egui::TextEdit::singleline(&mut self.phone) + .id_salt("remote name") + .hint_text("phone"), + ); + let entered = field.lost_focus() && ui.input(|input| input.key_pressed(egui::Key::Enter)); + if ui.add(kit::button(Icon::Phone, "Pair a phone")).clicked() || entered { + pair = Some(self.phone.trim().to_string()); + } + }); + + let now = kio::clock::wall(); + if let Some(shown) = &self.paired { + ui.add_space(10.0); + ui.label(format!( + "Scan with {}'s camera. The link {}.", + shown.name, + expiry(shown.invite.expires, now) + )); + ui.add_space(8.0); + // The link is the phone's credential, so it shows only as a code to scan. + match &shown.qr { + Ok(code) => { + ui.image((code.id(), egui::vec2(240.0, 240.0))); + } + Err(err) => { + ui.colored_label(kit::ERR, format!("{err}; copy the link instead.")); + } + } + ui.add_space(8.0); + let done = ui.horizontal(|ui| { + if ui.add(kit::button(Icon::Copy, "Copy link")).clicked() { + ui.ctx().copy_text(shown.invite.link.to_string()); + } + ui.add(kit::button(Icon::Close, "Done")).clicked() + }); + if done.inner { + self.paired = None; + } + } + + ui.add_space(10.0); + ui.separator(); + if let Some(failure) = room.renewal_failure() { + ui.colored_label( + kit::ERR, + format!("The studio is not in the room, so no remote connects: {failure}"), + ); + } + let remotes = room.remotes(); + if remotes.is_empty() { + ui.weak("No phone paired yet."); + } + let mut show = None; + let mut remove = None; + egui::Grid::new("remotes").striped(true).show(ui, |ui| { + if remotes.is_empty() { + return; + } + for heading in ["Phone", "Status", "Link", ""] { + ui.strong(heading); + } + ui.end_row(); + for (name, remote) in &remotes { + ui.label(name); + match (remote.removed, remote.connected) { + (true, _) => ui.weak("removed"), + (false, true) => ui.colored_label(kit::OK, "connected"), + (false, false) => ui.weak("offline"), + }; + // A connected remote no longer depends on its link's expiry. + match &remote.invite { + _ if remote.connected => ui.label("renewing"), + _ if remote.removed => ui.weak("not renewed"), + Some(link) => ui.label(expiry(link.expires, now)), + None => ui.weak("paired before this session"), + }; + ui.horizontal(|ui| { + // Only a link that still works is worth scanning again. + if let Some(link) = &remote.invite + && !remote.removed + && link.expires > now + && ui.small_button("Show link").clicked() + { + show = Some((name.clone(), link.clone())); + } + if ui.small_button("New link").clicked() { + pair = Some(name.clone()); + } + if !remote.removed + && ui + .small_button("Remove") + .on_hover_text( + "Stop reading its presses at once. Its token stops renewing, so it is disconnected within the link lifetime in Settings, Guests. Only New link pairs it again.", + ) + .clicked() + { + remove = Some(name.clone()); + } + }); + ui.end_row(); + } + }); + + if let Some((name, invite)) = show { + self.show_remote(ui.ctx(), &room, name, invite); + } + if let Some(name) = pair { + match room.pair(&name) { + Ok(invite) => { + self.show_remote(ui.ctx(), &room, name, invite); + self.phone.clear(); + } + Err(err) => self.report(Err(err)), + } + } + if let Some(name) = remove { + let result = room.unpair(&name).map(|()| format!("removed the remote {name}")); + if result.is_ok() && self.paired.as_ref().is_some_and(|shown| shown.name == name) { + self.paired = None; + } + self.report(result); + } + } + + /// Put `invite`'s link for the remote `name` on screen as a QR code. + fn show_remote( + &mut self, + ctx: &egui::Context, + room: &Arc, + name: String, + invite: crate::guests::Invite, + ) { + let qr = qr(ctx, invite.link.as_str()); + let room = Arc::downgrade(room); + self.paired = Some(Shown { name, invite, qr, room }); + } +} diff --git a/rs/moq-studio/src/ui/settings.rs b/rs/moq-studio/src/ui/settings.rs index ee73115c..b742cb81 100644 --- a/rs/moq-studio/src/ui/settings.rs +++ b/rs/moq-studio/src/ui/settings.rs @@ -162,6 +162,7 @@ impl Panels { let nav = Rect::from_min_size(rect.min, vec2(if icons { NAV_ICONS } else { NAV }, rect.height())); let pane = Rect::from_min_max(pos2(nav.right(), rect.top()), rect.max); self.settings_nav(ui, nav, icons, &mut setting); + self.setting = Some(setting); let builder = egui::UiBuilder::new().max_rect(pane.shrink2(vec2(MARGIN, 0.0))); ui.scope_builder(builder, |ui| { ui.add_space(24.0); @@ -183,11 +184,16 @@ impl Panels { Setting::Recording => self.recording(ui), Setting::Hotkeys => self.hotkeys(ui, fx), Setting::Guests => self.guest_settings(ui), + Setting::Remote => self.remote_settings(ui), Setting::Advanced => self.advanced(ui, fx), }); // A pane closes the window by clearing it, as the scene - // graph's Open does. - closed |= self.setting.is_none(); + // graph's Open does, or opens another by naming it, as + // Remote's Set up the room does. + match self.setting { + None => closed = true, + Some(asked) => setting = asked, + } }); }); // Escape while a key is being bound cancels the binding instead.