From d12ed28093036cb80015e36f7989f7fc48bec2b2 Mon Sep 17 00:00:00 2001 From: Cindy Zhang Date: Mon, 26 Oct 2026 09:07:09 -0700 Subject: [PATCH 1/3] feat(interaction): cast filmstrips, rect sampling, a Storybook highlight default and the WebKit shader cache - castStoryboard: a filmstrip of real screencast frames (e.g. a busy window), labelled from a zero. - sampleRects: one element rect per animation frame, the cheap path when the main thread is free. - openStory turns Storybook's highlight module off (it reads every element's computed style on each insertion, inside the inserting task) and checks it is off; storybookHighlight: true keeps it. - The WebKit container keeps Mesa's shader cache on the host, so only the first run on a machine pays 450-700 ms per new shader. Self-tests for each, README. Split out of the frame evidence pull request for scope; the code is unchanged. Co-authored-by: Christopher Chedeau --- internal/interaction-harness/README.md | 25 ++- .../scripts/webkit-container.mjs | 26 ++++ internal/interaction-harness/src/frames.ts | 70 ++++++++- .../src/harness.interaction.browser.spec.ts | 142 +++++++++++++++++- internal/interaction-harness/src/story.ts | 47 +++++- .../interaction-harness/src/storyboard.ts | 64 +++++++- .../src/storybook.interaction.browser.spec.ts | 104 +++++++++++++ 7 files changed, 466 insertions(+), 12 deletions(-) create mode 100644 internal/interaction-harness/src/storybook.interaction.browser.spec.ts diff --git a/internal/interaction-harness/README.md b/internal/interaction-harness/README.md index 7daf770400e6e..8247bb710163c 100644 --- a/internal/interaction-harness/README.md +++ b/internal/interaction-harness/README.md @@ -33,6 +33,12 @@ under a rootful docker daemon the container runs as your user so its node internal/interaction-harness/scripts/webkit-container.mjs [playwright test args…] ``` +WebKit composites through Mesa's software GL there, which compiles each shader +the first time a page draws it (450–700 ms for the first frame of a blurred or +newly animated layer). The script keeps Mesa's shader cache on the host +(`~/.cache/astryx-webkit-shaders/`, or `ASTRYX_WEBKIT_SHADER_CACHE`), so +only the first run on a machine pays for them. + Evidence (storyboards, curve traces, busy-main-thread frames) is written to `test-results/interaction-evidence/` (override with `ASTRYX_INTERACTION_EVIDENCE`) and attached to the test report. It is never @@ -44,12 +50,12 @@ committed: a pull request links or attaches it. | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `scripts.ts` | Touch scripts as data: `tap`, `drag` (hold, rest), `flick` (exact final-move speed, velocity profiles), `holdThenDrag`, `pinch`, `rotate`, `path`, `merge` (a second finger mid-gesture), `releaseVelocity`, `toTouchEvents`. | | `touch.ts` | `touchDriver(page).play(script)`: the script as real engine input. `installInputLog` / `readInputLog`: what the page received. | -| `frames.ts` | `startScreencast` (timestamped composited frames), `markElement` (a marker chip or outline), `trackMarker` (marker box per frame), `frameStats`. | +| `frames.ts` | `startScreencast` (timestamped composited frames), `markElement` (a marker chip or outline), `trackMarker` (marker box per frame), `sampleRects` (per-rAF rects), `frameStats`. | | `busy.ts` | `blockMainThread(page, ms)` and `busyWindowProof(tracked, window)`: frames presented, frames that moved, longest stall, a verdict; `presented` adds the compositor's own frames. | | `trace.ts` | `traceRendering(page, body)`: main-thread Layout / Paint / UpdateLayoutTree / PrePaint counts, `countBetween`, long tasks; `installLongTaskLog` / `readLongTasks`; `startCompositorTrace`: when the compositor presents. | | `curves.ts` | `holdAnimations`, `seekAnimations`, `sampleCurve` (presented transform / opacity at any t), `waitForAnimation`, `watchAnimations`, `waitForRest`, `releaseAnimations`. | | `math.ts` | Oracles independent of the code under test: `cubicBezier`, `timingFunction`, `springStep` (SwiftUI response / dampingFraction, initial velocity), `projectDecay`, `rubberBand`, `parseTransform`, `compareCurve`, `timeline`. | -| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event): PNG contact sheet + JSON trace. | +| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event), `castStoryboard` (real screencast frames): PNG contact sheet + JSON trace. | | `output.ts` | `evidenceDir`, `writeEvidence`: where evidence goes and how it is attached to the report. | | `story.ts` | `phone`, `withStorybook`, `openStory`, `frames`, and the timer gate (`installTimerGate`, `freezeTimers`, `thawTimers`). | @@ -65,6 +71,14 @@ dispatch latency, and the events go through the compositor's gesture detection the limits below. Read positions a frame after a move: Chromium delivers `pointermove` aligned to animation frames. +**Stories, not Storybook.** `openStory` turns Storybook's highlight module off +on every page it opens, and checks that its root element is absent. Left on, +the module observes `#storybook-root` and, on every insertion, reads the +computed style of every element on the page inside the inserting task (117 +reads and two forced style recalculations for one exit copy in a Button story), +so a proof would measure Storybook as well as the component. +`openStory(page, origin, id, args, {storybookHighlight: true})` keeps it on. + **Curve sampling.** Pause every running animation (CSS transitions and animations are Web Animations too), seek to a time, read `getComputedStyle`: the presented value at that exact time, independent of frame timing. Finished @@ -135,7 +149,8 @@ allowed; a frame it did not deliver is a gap, for the stall check). Pass `maxMedianFrameMs` (20 at 60 Hz) only in a proof that claims display rate. `verdict.failures` names each miss with its numbers, and `summary` carries the raw numbers for the report. Frames are judged in presentation order: a loaded -machine can deliver them out of order. +machine can deliver them out of order. `castStoryboard` turns the frames of the +block into a filmstrip. A just-started animation can reach the screen after the block began (50–100 ms later in WebKit on a loaded machine). When the cast saw the element at rest @@ -184,14 +199,14 @@ backstop, a cleanup render) would do so behind a held storyboard's back: `installTimerGate` before the animation starts and `freezeTimers` while photographing. The page can still cancel a held callback, a held frame callback gets the time it actually runs, and the harness's own waits (its frame waits, -`blockMainThread` with `during`) are not held. +`blockMainThread` with `during`, `sampleRects`) are not held. An animation that repeats forever does not set a storyboard's default length; with nothing else to film, pass `times`. A touch filmstrip keeps the scripted event `timeStamp`s (Chromium), but a handler that reads `performance.now()` or `Date.now()` sees one event per photograph, about 100 ms apart, so a clock-timed release flings slower when filmed than when played: a storyboard cannot show such a gesture; record it in real time instead, with -`startScreencast`. Chromium can return an empty copy of +`startScreencast` and `castStoryboard`. Chromium can return an empty copy of the page's surface right after it changes ("Unable to capture screenshot"); a storyboard takes that photograph again two frames later, up to three attempts in all, and lists each retake under `recaptures` in the trace. diff --git a/internal/interaction-harness/scripts/webkit-container.mjs b/internal/interaction-harness/scripts/webkit-container.mjs index cb20fd44aa725..11ef11bdfda80 100644 --- a/internal/interaction-harness/scripts/webkit-container.mjs +++ b/internal/interaction-harness/scripts/webkit-container.mjs @@ -16,10 +16,19 @@ * docker daemon the container runs as the invoking user (rootless podman and * rootless docker already map it), so the `test-results/` it writes into the * mounted checkout stay the host user's and the next host run can clear them. + * + * WebKit composites through Mesa's software GL in the image, which compiles + * each shader variant the first time a page draws it: 450–700 ms for the + * first frame of a blurred or newly animated layer, long enough for a short + * animation to end before it is ever presented. Mesa caches what it compiles, + * and the cache is kept on the host (`~/.cache/astryx-webkit-shaders/`, + * or `ASTRYX_WEBKIT_SHADER_CACHE`), so a run starts with every shader an + * earlier run compiled instead of paying for them in its first tests. */ import {spawnSync} from 'node:child_process'; import fs from 'node:fs'; import {createRequire} from 'node:module'; +import os from 'node:os'; import path from 'node:path'; import {fileURLToPath} from 'node:url'; @@ -42,6 +51,19 @@ if (engine == null) { process.exit(1); } const args = process.argv.slice(2); +// Where the container sees the host's shader cache (see the header). +const containerShaderCache = '/astryx-shader-cache'; +const shaderCache = + process.env.ASTRYX_WEBKIT_SHADER_CACHE ?? + path.join( + os.homedir(), + '.cache', + 'astryx-webkit-shaders', + image.replace(/[^\w.-]+/g, '_'), + ); +fs.mkdirSync(shaderCache, {recursive: true}); +// The harness self-test reads this marker inside the container. +fs.writeFileSync(path.join(shaderCache, 'ASTRYX-SHADER-CACHE'), `${image}\n`); const env = [ 'ASTRYX_STORYBOOK_DIR', 'ASTRYX_INTERACTION_EVIDENCE', @@ -79,6 +101,10 @@ const command = [ '--ipc=host', '-v', `${root}:${root}`, + '-v', + `${shaderCache}:${containerShaderCache}`, + '-e', + `MESA_SHADER_CACHE_DIR=${containerShaderCache}`, '-w', root, ...env, diff --git a/internal/interaction-harness/src/frames.ts b/internal/interaction-harness/src/frames.ts index c61ea7e171c6f..83841d259c17f 100644 --- a/internal/interaction-harness/src/frames.ts +++ b/internal/interaction-harness/src/frames.ts @@ -7,11 +7,12 @@ * `Screencast` domain) * @output `startScreencast` (timestamped composited frames), `markElement` * (a marker outline the frames can find), `trackMarker` (per-frame marker - * box), `frameStats`. + * box), `sampleRects` (per-rAF rect sampling), `frameStats`. * @position Frame evidence for motion proofs. The screencast is captured off * the page's main thread, from what the compositor presents, so it keeps * delivering frames while a script blocks the main thread: that is what makes - * the busy-main-thread proof possible. + * the busy-main-thread proof possible. Rect sampling is the cheap path when + * the main thread is free. * * Marker tracking needs no image library: frames are decoded and scanned in a * scratch page of the same browser (createImageBitmap + canvas), so the @@ -399,6 +400,71 @@ export async function trackMarker( } } +export interface RectSample { + /** Epoch ms of the rAF callback. */ + readonly t: number; + readonly box: Box | null; + readonly transform: string; + readonly opacity: number; +} + +/** + * Samples the element's rect, transform and opacity on every animation frame + * for `durationMs`. Main-thread sampling: it stops while the main thread is + * blocked, which is the point of the screencast path above. + */ +export async function sampleRects( + locator: Locator, + durationMs: number, +): Promise { + return locator.evaluate( + (element, duration) => + new Promise< + { + t: number; + box: {x: number; y: number; width: number; height: number} | null; + transform: string; + opacity: number; + }[] + >(resolve => { + // The harness's own waits run on the real timers: a frozen timer gate + // (story.ts) holds only the page's. + const gate = ( + window as unknown as { + __harnessTimerGate?: { + frame?: typeof requestAnimationFrame; + }; + } + ).__harnessTimerGate; + const frame = gate?.frame ?? window.requestAnimationFrame.bind(window); + const samples: { + t: number; + box: {x: number; y: number; width: number; height: number} | null; + transform: string; + opacity: number; + }[] = []; + const end = performance.now() + duration; + const tick = (now: number) => { + const rect = element.getBoundingClientRect(); + const style = getComputedStyle(element); + samples.push({ + t: performance.timeOrigin + now, + box: {x: rect.x, y: rect.y, width: rect.width, height: rect.height}, + transform: style.transform, + opacity: Number(style.opacity), + }); + if (now < end) { + frame(tick); + } else { + resolve(samples); + } + }; + frame(tick); + }), + durationMs, + ); +} + export interface FrameStats { readonly frames: number; /** Frames whose tracked value differs from the frame before. */ diff --git a/internal/interaction-harness/src/harness.interaction.browser.spec.ts b/internal/interaction-harness/src/harness.interaction.browser.spec.ts index e1d1a422b177f..a25c9f5ee36af 100644 --- a/internal/interaction-harness/src/harness.interaction.browser.spec.ts +++ b/internal/interaction-harness/src/harness.interaction.browser.spec.ts @@ -25,14 +25,19 @@ * two frames later; a touch filmstrip keeps scripted event times while a * handler clock sees its pacing; the timer gate holds a component timer * until the filmstrip is done, drops callbacks the page cancels, stamps held - * frames with the time they run, and leaves the harness its own frames. + * frames with the time they run, and leaves the harness its own frames; a + * cast filmstrip keeps exactly the presented frames of its window, labelled + * from its zero, as PNG. * @position The harness's own test. A component spec trusts these * instruments; this file is why it may. */ +import fs from 'node:fs'; +import path from 'node:path'; import {expect, test, type Page} from '@playwright/test'; import { animationStoryboard, + castStoryboard, blockMainThread, busyWindowProof, compareCurve, @@ -56,6 +61,7 @@ import { readLongTasks, releaseAnimations, sampleCurve, + sampleRects, seekAnimations, springStep, startCompositorTrace, @@ -508,6 +514,83 @@ test.describe('frame evidence', () => { expect(proof.monotonic).toBe(true); }); + test('in the WebKit container, compiled shaders outlive the run', async ({ + browser, + }, testInfo) => { + // The official Playwright image keeps its browsers here. + test.skip( + testInfo.project.name !== 'webkit' || + process.env.PLAYWRIGHT_BROWSERS_PATH !== '/ms-playwright', + 'runs inside webkit-container.mjs', + ); + // A cache inside the container dies with it: every run then compiled each + // shader again, 450–700 ms in the first frame of a blurred layer. + const dir = process.env.MESA_SHADER_CACHE_DIR; + expect(dir, 'MESA_SHADER_CACHE_DIR names the host-kept cache').toBeTruthy(); + // Written on the host by the script before the container starts. + expect( + fs.readFileSync(path.join(dir ?? '', 'ASTRYX-SHADER-CACHE'), 'utf8'), + ).toContain('playwright'); + // Mesa writes what it compiles there once a page draws a blurred layer. + // Earlier runs have filled the cache already, so this run's browser gets + // an empty directory beside them, in the same host-kept mount, and the + // test watches its own writes arrive. + const fresh = fs.mkdtempSync(path.join(dir ?? '', 'self-test-')); + try { + const own = await browser.browserType().launch({ + env: {...process.env, MESA_SHADER_CACHE_DIR: fresh}, + }); + try { + const page = await own.newPage(); + await page.setContent( + '
', + ); + await page.evaluate( + () => + document + .getElementById('glass') + ?.animate([{scale: '1'}, {scale: '0.9'}], {duration: 200}) + .finished, + ); + } finally { + await own.close(); + } + expect(fs.readdirSync(fresh, {recursive: true})).not.toEqual([]); + } finally { + fs.rmSync(fresh, {recursive: true, force: true}); + } + }); + + test('rect sampling reads one rect per animation frame while the main thread is free', async ({ + page, + browserName, + }) => { + await fixture(page); + const box = page.locator('#box'); + await box.evaluate(el => { + el.animate( + [{transform: 'translateX(0px)'}, {transform: 'translateX(200px)'}], + {duration: 600, easing: 'linear'}, + ); + }); + const samples = await sampleRects(box, 400); + const xs = samples.map(s => s.box?.x ?? NaN); + const gaps = samples + .slice(1) + .map((s, i) => s.t - samples[i].t) + .sort((a, b) => a - b); + const cadence = gaps[Math.floor(gaps.length / 2)]; + expect(cadence).toBeLessThan(34); + expect(samples.length).toBeGreaterThanOrEqual(10); + if (browserName === 'chromium') { + // One sample per presented frame at 60 Hz. (Headless WebKit's animation + // frames are irregular, so only the cadence is checked there.) + expect(samples.length).toBeGreaterThanOrEqual(20); + } + expect(xs.every((x, i) => i === 0 || x >= xs[i - 1])).toBe(true); + expect(xs.at(-1) ?? 0).toBeGreaterThan(xs[0] + 80); + }); + test('the trace counts no Layout for a transform animation and one per frame for `top`', async ({ page, browserName, @@ -1871,7 +1954,7 @@ test.describe('storyboards', {tag: '@storyboard'}, () => { } }); - test("the harness's own waits run on the real timers: a frozen timer gate does not hold blockMainThread's block", async ({ + test("the harness's own waits run on the real timers: a frozen timer gate holds neither blockMainThread's block nor sampleRects", async ({ page, }) => { await fixture(page); @@ -1895,6 +1978,7 @@ test.describe('storyboards', {tag: '@storyboard'}, () => { expect( await within(blockMainThread(page, 100, {during: box, timeout: 2_000})), ).toBe('settled'); + expect(await within(sampleRects(box, 200))).toBe('settled'); await thawTimers(page); }); @@ -2083,4 +2167,58 @@ test.describe('storyboards', {tag: '@storyboard'}, () => { expect(state).toMatchObject({timer: false, frame: false}); expect(state.heldFor).toBeGreaterThanOrEqual(140); }); + + test('a cast filmstrip keeps exactly the presented frames of its window, labelled from its zero', async ({ + page, + }, testInfo) => { + await fixture(page); + const box = page.locator('#box'); + // JPEG frames (WebKit's are always JPEG) take the conversion path. + const cast = await startScreencast(page, {format: 'jpeg'}); + await box.evaluate(el => + el.animate( + [{transform: 'translateX(0px)'}, {transform: 'translateX(200px)'}], + {duration: 600}, + ), + ); + await page.waitForTimeout(500); + const presented = await cast.stop(); + expect(presented.length).toBeGreaterThan(8); + // A window of the middle frames, zeroed on one of them. + const zero = presented[2].t; + const from = presented[3].t; + const to = presented[presented.length - 3].t; + const inside = presented.filter(f => f.t >= from && f.t <= to); + const seen: number[] = []; + const film = await castStoryboard(page, presented, { + name: 'harness-self-test-cast', + zero, + from, + to, + data: t => { + seen.push(t); + return {t}; + }, + }); + expect(film.frames.map(f => f.t)).toEqual(inside.map(f => f.t - zero)); + expect(film.frames.map(f => f.label)).toEqual( + inside.map(f => `t=${(f.t - zero).toFixed(1)} ms`), + ); + expect(seen).toEqual(inside.map(f => f.t)); + // Every cell is a PNG, whatever the screencast delivered. + for (const frame of film.frames) { + expect([...frame.png.subarray(0, 4)]).toEqual([0x89, 0x50, 0x4e, 0x47]); + } + const files = await film.write(testInfo); + expect(files['sheet.png']).toMatch(/sheet\.png$/); + // A window with no presented frame in it films nothing. + const empty = await castStoryboard(page, presented, { + name: 'harness-self-test-cast-empty', + zero, + from: to + 1, + to: to + 2, + }); + expect(empty.frames).toEqual([]); + await expect(empty.write(testInfo)).rejects.toThrow(/no frames/); + }); }); diff --git a/internal/interaction-harness/src/story.ts b/internal/interaction-harness/src/story.ts index 5a1f2237c6725..627969a6637f3 100644 --- a/internal/interaction-harness/src/story.ts +++ b/internal/interaction-harness/src/story.ts @@ -6,7 +6,8 @@ * static server over a built Storybook * @output `phone` (the touch-phone browser context every interaction proof * runs in), `withStorybook` (serve the built Storybook for a spec file), and - * `openStory` (load one story and wait until it has rendered). + * `openStory` (load one story, with Storybook's highlight module off, and + * wait until it has rendered). * @position Stories are the reproduction path, as for every other browser * check in this repository: a scenario used for verification is checked in, * not built in a one-off page. @@ -59,13 +60,47 @@ export function withStorybook(test: TestType): () => string { }; } -/** Loads a story in the page and waits for its first render, its fonts and two frames. */ +/** Pages whose Storybook highlight module `openStory` turned off. */ +const highlightOff = new WeakSet(); + +export interface OpenStoryOptions { + /** + * Keeps Storybook's highlight module on. It is off by default: when on, it + * observes `#storybook-root` and, on every insertion, reads the computed + * style of every element on the page inside the inserting task, so a proof + * would measure Storybook as well as the component. + */ + readonly storybookHighlight?: boolean; +} + +/** + * Loads a story in the page and waits for its first render, its fonts and two + * frames. Storybook's highlight module is off (see `OpenStoryOptions`), and + * that is checked: its root element must be absent once the story renders. + */ export async function openStory( page: Page, origin: string, storyId: string, args?: Readonly>, + {storybookHighlight = false}: OpenStoryOptions = {}, ): Promise { + if (storybookHighlight) { + if (highlightOff.has(page)) { + throw new Error( + "openStory: this page already turned Storybook's highlight module off; open the story in a new page to keep it on", + ); + } + } else if (!highlightOff.has(page)) { + highlightOff.add(page); + // Storybook's highlight module returns at once when this is set before + // its preview runs. + await page.addInitScript(() => { + ( + globalThis as unknown as {__STORYBOOK_HIGHLIGHT_INITIALIZED?: boolean} + ).__STORYBOOK_HIGHLIGHT_INITIALIZED = true; + }); + } const params = new URLSearchParams({id: storyId, viewMode: 'story'}); if (args != null) { params.set( @@ -86,6 +121,14 @@ export async function openStory( requestAnimationFrame(() => requestAnimationFrame(() => resolve())), ); }); + if ( + !storybookHighlight && + (await page.locator('#storybook-highlights-root').count()) > 0 + ) { + throw new Error( + "openStory: Storybook's highlight module is on (#storybook-highlights-root exists), so every insertion would also read the style of every element on the page; this Storybook may no longer read __STORYBOOK_HIGHLIGHT_INITIALIZED", + ); + } } /** Resolves after `count` animation frames have been presented in the page. */ diff --git a/internal/interaction-harness/src/storyboard.ts b/internal/interaction-harness/src/storyboard.ts index 2dc22fdc8cf04..4a9cbec360bb4 100644 --- a/internal/interaction-harness/src/storyboard.ts +++ b/internal/interaction-harness/src/storyboard.ts @@ -336,7 +336,8 @@ export interface TouchStoryboardOptions extends StoryboardOptions { * photograph per event, about 100 ms between moves instead of the script's * 16.7. Its release velocity is that much lower, so a flick that dismisses * when played can settle back when filmed: a storyboard cannot show such a - * gesture; record it in real time instead. + * gesture; record it in real time instead (`startScreencast`, + * `castStoryboard`). */ export async function touchStoryboard( page: Page, @@ -364,3 +365,64 @@ export async function touchStoryboard( }); return storyboard; } + +/** + * A filmstrip of screencast frames (real presented frames, e.g. the frames of + * a busy-main-thread window), labelled with their time relative to `zero`. + * Evidence for what the compositor presented, not a deterministic storyboard. + */ +export async function castStoryboard( + page: Page, + frames: readonly { + readonly t: number; + readonly data: string; + readonly format: 'png' | 'jpeg'; + }[], + options: StoryboardOptions & { + readonly zero: number; + readonly from?: number; + readonly to?: number; + readonly data?: (t: number) => Readonly>; + }, +): Promise { + const storyboard = new Storyboard(page, options); + for (const frame of frames) { + if (options.from != null && frame.t < options.from) { + continue; + } + if (options.to != null && frame.t > options.to) { + continue; + } + const png = + frame.format === 'png' + ? Buffer.from(frame.data, 'base64') + : await toPng(page, frame.data); + const t = frame.t - options.zero; + storyboard.frames.push({ + label: `t=${t.toFixed(1)} ms`, + t, + data: options.data?.(frame.t) ?? {}, + png, + }); + } + return storyboard; +} + +async function toPng(page: Page, jpegBase64: string): Promise { + const scratch = await page.context().newPage(); + try { + const url = await scratch.evaluate(async data => { + const bitmap = await createImageBitmap( + await (await fetch(`data:image/jpeg;base64,${data}`)).blob(), + ); + const canvas = document.createElement('canvas'); + canvas.width = bitmap.width; + canvas.height = bitmap.height; + canvas.getContext('2d')?.drawImage(bitmap, 0, 0); + return canvas.toDataURL('image/png'); + }, jpegBase64); + return Buffer.from(url.slice(url.indexOf(',') + 1), 'base64'); + } finally { + await scratch.close(); + } +} diff --git a/internal/interaction-harness/src/storybook.interaction.browser.spec.ts b/internal/interaction-harness/src/storybook.interaction.browser.spec.ts new file mode 100644 index 0000000000000..a711feb62cb84 --- /dev/null +++ b/internal/interaction-harness/src/storybook.interaction.browser.spec.ts @@ -0,0 +1,104 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file storybook.interaction.browser.spec.ts + * @input The built Storybook (`withStorybook`) and one of its stories + * @output `openStory` turns Storybook's highlight module off, so an insertion + * in a story carries no `getComputedStyle` scan of the page; with the module + * on (the control), every insertion carries one. Both engines; Chromium's + * trace also counts the style recalculations the scan forces + * @position Self-test for story.ts; it needs the built Storybook, as every + * component proof does + */ + +import {expect, test, type Page} from '@playwright/test'; +import {openStory, withStorybook} from './story'; +import {countBetween, traceRendering} from './trace'; + +const storybook = withStorybook(test); + +/** Counts the page's `getComputedStyle` calls, from before Storybook loads. */ +function countStyleReads(): void { + const w = window as unknown as {__styleReads: number}; + w.__styleReads = 0; + const read = window.getComputedStyle.bind(window); + window.getComputedStyle = ((element: Element, pseudo?: string | null) => { + w.__styleReads += 1; + return read(element, pseudo); + }) as typeof window.getComputedStyle; +} + +/** + * Inserts an exit copy of the story's first element into `#storybook-root`, + * as an exit animation's copy is, lets the insertion's mutation microtasks run, + * and reports what ran between its marks. + */ +async function insertCopy(page: Page) { + return page.evaluate(async () => { + const w = window as unknown as {__styleReads: number}; + const root = document.getElementById('storybook-root') as HTMLElement; + const reads = w.__styleReads; + const from = + performance.timeOrigin + performance.mark('copy:start').startTime; + const copy = (root.firstElementChild as HTMLElement).cloneNode( + true, + ) as HTMLElement; + copy.setAttribute('aria-hidden', 'true'); + root.appendChild(copy); + // A MutationObserver's callback is a microtask the insertion queued: + // these run after it. + await Promise.resolve(); + await Promise.resolve(); + const to = performance.timeOrigin + performance.mark('copy:end').startTime; + const scanned = w.__styleReads - reads; + const elements = document.body.querySelectorAll('*').length; + copy.remove(); + return {from, to, scanned, elements}; + }); +} + +test("openStory turns Storybook's highlight module off: an insertion in a story carries no getComputedStyle scan of the page", async ({ + page, + browserName, +}) => { + // The insertion's reads in every engine, and in Chromium (whose trace the + // harness reads) the style recalculations between its marks. + const measure = async (p: Page) => { + if (browserName !== 'chromium') { + return {copy: await insertCopy(p), styleRecalc: null}; + } + let copy = {from: 0, to: 0, scanned: 0, elements: 0}; + const trace = await traceRendering(p, async () => { + copy = await insertCopy(p); + }); + return { + copy, + styleRecalc: + countBetween(trace, copy.from, copy.to).UpdateLayoutTree ?? 0, + }; + }; + // The harness default. + await page.addInitScript(countStyleReads); + await openStory(page, storybook(), 'core-button--primary'); + expect(await page.locator('#storybook-highlights-root').count()).toBe(0); + const off = await measure(page); + expect(off.copy.scanned).toBe(0); + if (off.styleRecalc != null) { + expect(off.styleRecalc).toBe(0); + } + + // Control: Storybook's module kept on reads the style of every element on + // the page inside the inserting task. + const control = await page.context().newPage(); + await control.addInitScript(countStyleReads); + await openStory(control, storybook(), 'core-button--primary', undefined, { + storybookHighlight: true, + }); + expect(await control.locator('#storybook-highlights-root').count()).toBe(1); + const on = await measure(control); + expect(on.copy.scanned).toBeGreaterThanOrEqual(on.copy.elements); + if (on.styleRecalc != null) { + expect(on.styleRecalc).toBeGreaterThan(0); + } + await control.close(); +}); From 6ccd5b46cdda1309c4d5598e6acbc01fc22b4192 Mon Sep 17 00:00:00 2001 From: Cindy Zhang Date: Mon, 26 Oct 2026 13:10:35 -0700 Subject: [PATCH 2/3] fix(interaction): castStoryboard takes no clip it would ignore castStoryboard accepted StoryboardOptions' clip through its options type but never applied it to the frames. Its options now leave clip out, so a caller can't pass one that does nothing. --- internal/interaction-harness/src/storyboard.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/internal/interaction-harness/src/storyboard.ts b/internal/interaction-harness/src/storyboard.ts index 4a9cbec360bb4..a49207c4f81c4 100644 --- a/internal/interaction-harness/src/storyboard.ts +++ b/internal/interaction-harness/src/storyboard.ts @@ -378,7 +378,8 @@ export async function castStoryboard( readonly data: string; readonly format: 'png' | 'jpeg'; }[], - options: StoryboardOptions & { + // A cast is already the screen's frames: no clip is applied to them. + options: Omit & { readonly zero: number; readonly from?: number; readonly to?: number; From f79b8a9aad357837a0f8d6f74edac48c8be004a0 Mon Sep 17 00:00:00 2001 From: Cindy Zhang Date: Mon, 26 Oct 2026 14:01:02 -0700 Subject: [PATCH 3/3] fix(interaction): resolve ASTRYX_WEBKIT_SHADER_CACHE to a host path A bare relative ASTRYX_WEBKIT_SHADER_CACHE was mounted as a named volume instead of a host directory, and an empty one made mkdirSync throw. The script now resolves a set value with path.resolve and falls back to the default cache directory on an empty one. --- .../scripts/webkit-container.mjs | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/internal/interaction-harness/scripts/webkit-container.mjs b/internal/interaction-harness/scripts/webkit-container.mjs index 11ef11bdfda80..a077eae415ed5 100644 --- a/internal/interaction-harness/scripts/webkit-container.mjs +++ b/internal/interaction-harness/scripts/webkit-container.mjs @@ -53,14 +53,16 @@ if (engine == null) { const args = process.argv.slice(2); // Where the container sees the host's shader cache (see the header). const containerShaderCache = '/astryx-shader-cache'; -const shaderCache = - process.env.ASTRYX_WEBKIT_SHADER_CACHE ?? - path.join( - os.homedir(), - '.cache', - 'astryx-webkit-shaders', - image.replace(/[^\w.-]+/g, '_'), - ); +// Resolved to an absolute path: a bare relative value would be mounted as a +// named volume rather than a host directory. An empty value means the default. +const shaderCache = process.env.ASTRYX_WEBKIT_SHADER_CACHE + ? path.resolve(process.env.ASTRYX_WEBKIT_SHADER_CACHE) + : path.join( + os.homedir(), + '.cache', + 'astryx-webkit-shaders', + image.replace(/[^\w.-]+/g, '_'), + ); fs.mkdirSync(shaderCache, {recursive: true}); // The harness self-test reads this marker inside the container. fs.writeFileSync(path.join(shaderCache, 'ASTRYX-SHADER-CACHE'), `${image}\n`);