Repository navigation
Conversation
Adds internal/interaction-harness (@astryxdesign/interaction-harness) and playwright.interaction.config.ts: phone-sized touch projects in Chromium and WebKit for *.interaction.browser.spec.ts specs. Touch scripts are data (tap, drag with hold and rest, flick with an exact release velocity and velocity profiles, hold-then-drag, pinch, rotate, any path, a second finger mid-gesture), played as real engine input: CDP Input.dispatchTouchEvent with scripted timestamps in Chromium, WebKit's own input command in WebKit. The harness's self-test proves the timestamps, a native fling and multi-touch delivery; a pr-interaction CI job runs the proofs against each PR's Storybook build, and a container script runs the WebKit project where host libraries are missing.
… Chromium The Chromium driver sent a lift of one finger of several as a touchMove listing the fingers still down, assuming CDP releases a point a move omits. It does not: the lifted finger stayed down. A touchEnd that lists the lifting fingers releases exactly those while the others keep moving, so the driver sends that, and a self-test lifts one of two fingers and checks the release, the other finger's moves and the touchend's remaining touches. Playwright's WebKit sends that touchend with no remaining touches, which iOS never does. The harness does not work around it: drivers expose partialLifts, and PARTIAL_LIFT_SKIP is the reason a proof cites when it skips; the README lists it with the other engine limits.
… lift profiled flicks at their velocity - pr-interaction now always reports and the `test` join requires its success (AST-030 FR7, FR11), shaped like test-native-swift: a Linux interaction-scope job decides, with the base's scope helper and failing closed, whether the harness, a component, Storybook, internal/a11y-spec, the workflow, package.json or pnpm-lock.yaml changed. Only then do the proofs take a 4-core runner; otherwise, and in merge groups (as with pr-a11y), the job passes without work. - A profiled flick's last move runs at velocity times profile(1), so the finger lifts at the documented velocity, and releaseVelocity reads the last move instead of a 100 ms window mean. - Docs: drop the /math subpath this package does not export yet; scripted timestamps arrive to 0.1 ms; the job comment describes the self-test it runs.
…ed its previous position A sample that repeats a position is not a move: frames that merge() inserts for another finger, where this finger did not move, made the last move look shorter, so a 10 Hz drag merged with a tap read 375 px/s instead of 300 and a 60 Hz drag whose other finger lifted 5 ms before its last move read about 3.6x too fast. The previous sample is now the first one at the previous position, as the page sees it, with a merged case in scripts.test.ts.
Pause every Web Animation, seek it to a time and read the presented transform and opacity, then compare the curve with oracles that share no code with the motion under test: cubic-bezier, the SwiftUI spring (response, dampingFraction, initial velocity), UIScrollView deceleration and UIKit's rubber band. Storyboards photograph seeked animations at t = 0, 16.7, 33.3 ... ms or each step of a scripted touch, and write a PNG contact sheet plus the JSON trace a spec asserts, under the gitignored test-results/interaction-evidence. A timer gate holds a component's own timers while a filmstrip is taken. pnpm storyboards runs the tagged tests.
…sented values An animation that keeps its end value in a run of identical final keyframes (held so a later interruption can read it) reported its whole duration as the curve: storyboards filmed the hold, waitForAnimation's interval reached past the landing, and waitForRest, which also required every animation to leave the running state and read only transform, timed out. The curve now ends where that run starts (HeldAnimation.curveEndMs, waitForAnimation's to, the default storyboard times), and rest is no animation pending or before its curve's end plus the presented transform, translate, rotate, scale and opacity (and ::backdrop opacity) still for 3 frames, whatever the playState. sampleCurve composes translate, rotate and scale with transform, so curves of the individual properties are sampled.
…film no endless curve, let the page cancel gated callbacks - holdAnimations holds only running document-timeline animations: a finished fill animation keeps its end value, an animation the page paused stays paused (release resumes only what was held), and a scroll-driven one is not seeked. Each held animation keeps its offset from the earliest start, so staggered starts stay staggered. - A sample or storyboard frame at or past an end reports the time it was read at (presentedAt, half a millisecond inside the end), and curveOf reads oracles there. - An animation that repeats forever no longer sets a storyboard's default length (it ran the worker out of memory); with nothing else to film, animationStoryboard asks for times. - The timer gate queues held callbacks by id, so the page's clearTimeout and cancelAnimationFrame still cancel them; a held frame callback gets the time it runs; frames(), waitForRest and touchStoryboard use the real requestAnimationFrame, so they resolve while frozen. - The storyboard self-tests carry the @Storyboard tag that `pnpm storyboards` selects.
…s from the element's own animations, resume endless ones
- animationStoryboard's default times run from the earliest start to the
latest curve end among the animations that end, each seek at
origin + t: beside a spinner that started earlier the animation under
test no longer shows its start in every frame, and a second animation
that starts later is filmed to its end.
- A sample's presentedAt (per channel: transform, opacity) comes from the
sampled element's own animations, by keyframe property, not from any
animation on the page.
- releaseAnimations({finish: true}) resumes an animation that repeats
forever instead of finishing it (which throws), and always clears the
held set.
- The interaction proofs' scope includes packages/themes/*/src/.
…before it looked waitForAnimation polled getAnimations() after the action, so an entrance shorter than the first look (a cold page answers its first evaluate late) had already finished: "no animation started". The page now records every animation from the first call into curves.ts on it, and on every later document: a frame loop on its own requestAnimationFrame, the CSS animation and transition events, and Element.animate as it is called, pending animations included (WebKit can start and finish a short script animation between two frames of a fresh page). waitForAnimation also reports an animation that started after the last waitForRest or waitForAnimation and has finished, with finishedAt. watchAnimations(locator) arms the record before an action that may be the page's first contact with the harness.
…uest This pull request keeps the curve-sampling instruments: holding and seeking animations, sampling presented values, the oracles, waiting for an animation and for rest. The storyboards (storyboard.ts, the evidence output, the timer gate and the `pnpm storyboards` script) and their tests move to a pull request stacked on this one.
`holdAnimations` replaced the held set on each call, so a second hold left the first set paused after `releaseAnimations` (it resumed only the newest set), and `waitForRest` then reported the frozen frame as rest. A later hold now adds to the set: everything held stays held until the release, and offsets count from the earliest start of all of it. The self-test holds two elements in turn, releases, and checks the first runs to its end; on the old code it stays frozen 50 px into a 300 px move. The self-test's header and the `curves.ts` SYNC line no longer describe the storyboard and timer-gate layer, which moved to its own pull request.
animationStoryboard films held animations by seeking them (t = 0, 16.7, 33.3 … ms, from the earliest start to the latest curve end among the animations that end, each at its own offset); touchStoryboard takes a frame per scripted touch event. Each writes a PNG contact sheet and a JSON trace of its measurements to the evidence directory and attaches them to the report; the spec asserts on the trace. The timer gate (installTimerGate, freezeTimers, thawTimers) holds the page's timers and frame callbacks while a storyboard photographs, so a component that finishes an animation from a timer cannot do it behind the held animation's back; the page can still cancel a held callback, and the harness's own frame waits (and the animation recorder) keep the page's real requestAnimationFrame. `pnpm storyboards` runs only the tests tagged @Storyboard.
`touchStoryboard` said velocity-dependent behavior is unchanged in Chromium. That holds for velocity read from event timestamps, which keep their scripted times, but a handler that reads `performance.now()` or `Date.now()` (Item's and Toast's swipes) sees one event per photograph, about 100 ms apart, so a flick that dismisses when played can settle back when filmed. The doc and the README now state that limit and point such gestures at a screencast, and a self-test shows both clocks: scripted 16.7 ms gaps in `timeStamp`, more than two frames in the handler's clock.
Timestamped composited frames (Chromium's screencast and WebKit's own), a marker the frames can find, and the busy-main-thread proof: block the page's main thread mid-flight and show the element still moving every frame, with a requestAnimationFrame animation that freezes as the negative control. Chromium trace counts of Layout, Paint and style work inside an animation window, and long tasks. The exemplar spec proves the BottomSheet drag to a detent with all of it: the drag follows the finger, a flick dismisses, the settle matches its timing function, keeps advancing through a blocked main thread in Chromium and WebKit, does no Layout or Paint, and gets a drag and settle storyboard.
…ed machine The proof asserted a stall under 40 ms and near-exact frame counts, so a dropped screencast frame or two on a loaded runner failed it, and frames delivered out of order read as the element stepping backwards. busyWindowProof now returns a verdict that tolerates what load does to a screencast and nothing a frozen animation does: a median frame interval of at most 20 ms, no stall longer than three screencast intervals, and at least 90% of the window's whole frame intervals presenting a new position (one miss always allowed), with the raw numbers in a summary. Screencast frames are sorted by presentation time, the cadence comes from the frames around the window, and blockMainThread can block from inside the page in the frame an animation has run, so a short settle is still judged at length on a busy machine. The exemplar judges its settle until it is within 2 px of rest, since the screencast rounds to whole pixels. A unit test covers the limits; the rAF negative control still fails every criterion.
…ading them from the animation The settle-curve oracle took its easing and duration from the animation under test, so it passed a 900 ms ease-in or a 150 ms linear settle as readily as the real one. The spec now asserts the theme's values (the default --ease-standard, cubic-bezier(0.24, 1, 0.4, 1), over the neutral theme's --duration-medium, 300 ms) and builds the oracle from them; the header no longer claims 410 ms.
…ebKit frames on its clock - `blockMainThread`'s `during` path ran its loop inside the rAF callback of the frame that saw the animation. WebKit hands a newly started animation to its compositor in that frame's rendering update, which runs after rAF callbacks, so the loop held the hand-off and even a compositor animation froze (2 of 29 frames moved). The loop now runs from a task queued in that frame. - WebKit's screencast frames were rebased onto the wall clock by taking the first frame as the start, which shifted every frame earlier by however long that frame took to come (60–100 ms). They are now rebased by the measured offset between the wall clock and CLOCK_MONOTONIC, which WebKit stamps them on. - `busyWindowProof` fails a judged part shorter than four frames (the animation under test was not moving during the block) instead of passing it, takes the end value from `settledAt` when given (WebKit can present a stale position once an accelerated animation ends while the main thread is blocked), and reports where the longest stall starts. - The BottomSheet busy case arms the block before the release and reads where the settle ends from the page.
Retargeting a CSS transition cancels it. If that happened while it was held, `seekAnimations` and `sampleCurve` set its time again, which revived it over the page's own new transition, and `releaseAnimations` replayed it from the start. Seeking, sampling and releasing now drop an `idle` animation from the held set first, and times count from a fixed origin (the earliest start held until the release), so dropping one never moves the others. The self-test retargets a held transition, seeks, releases and waits for rest: the cancelled transition stays idle and the box ends where the page sent it. On the old code the cancelled transition is still held after the seek.
# Conflicts: # internal/interaction-harness/src/harness.interaction.browser.spec.ts
# Conflicts: # internal/interaction-harness/src/harness.interaction.browser.spec.ts
… instrument `touchStoryboard`'s doc and the README pointed a clock-timed gesture at `startScreencast` and `castStoryboard`, which this branch does not have. They now say a storyboard cannot show such a gesture and to record it in real time.
…cast instruments With frame evidence in the harness, a clock-timed gesture a storyboard cannot show is recorded with `startScreencast` and `castStoryboard`: the doc and the README name them.
`pause()` only schedules the pause: the engine takes the hold time at its next frame, so a held animation stayed held a frame past `heldAtMs`, and a read right after the hold varied by a frame between runs. Setting the current time right after `pause()` completes a pending pause at once, where `heldAtMs` read it. The self-test holds a 1 s move and checks it is still at `heldAtMs` two frames later; on the previous code it reads one frame later in Chromium (16.6–16.7 ms) and 15–70 ms later in WebKit.
# Conflicts: # internal/interaction-harness/src/harness.interaction.browser.spec.ts
# Conflicts: # internal/interaction-harness/src/harness.interaction.browser.spec.ts
`blockMainThread({during})` counted an animation as running as soon as its
current time reached `minProgressMs`. An animation created in the frame the
page looks is still pending, not on the compositor yet, and at
`minProgressMs: 0` the block started on it and froze it: in WebKit
deterministically, and in Chromium on a Lightbox page while the block still
ran inside the frame's callback. A pending animation no longer counts, and
`minProgressMs` is raised to one frame (16 ms), since in WebKit an
animation's start time resolves before its layer is flushed.
The self-test creates the animation in the frame the helper looks and
blocks at `minProgressMs: 0`. On the previous code it fails 6/6 in WebKit
(0 frames presented through the block); now it passes 10/10 in WebKit and
6/6 in Chromium.
A bare relative ASTRYX_WEBKIT_SHADER_CACHE was mounted as a named volume instead of a host directory, and an empty one made mkdirSync throw. The script now resolves a set value with path.resolve and falls back to the default cache directory on an empty one.
… path) into the recorder stand-down # Conflicts: # internal/interaction-harness/src/harness.interaction.browser.spec.ts
…ce; await it before the counted window The README, waitForRest's JSDoc and traceRendering's JSDoc said a traced waitForRest reads no style. Its one read of the element's animations, as it is called, restyles the element when an animation is already running there (2 Layouts and 2 style recalculations in the anchored fixture, 200 ms into its animation). They now say so and tell a proof to await it right after the action that starts the motion, before the window it counts.
… against its span In WebKit the rAF negative control could read "1/1 moved": one frame from before the block arrived 45 ms into it, and the moved criterion divided by the delivered frames. A part that delivered fewer than minJudgedFrames frames is now judged against its span, as one that delivered none was (busy.test.ts case; it fails on the previous rule). The README and the self-test header now say only the busy-main-thread proof carries a negative control beside its positive check.
…into n8548 # Conflicts: # internal/interaction-harness/README.md
![astracat-bot[bot]](https://avatars.githubusercontent.com/in/4796231?s=60&v=4){kind=small}
There was a problem hiding this comment.
Changes requested. The recorder and the waits now stand down inside traceRendering as described, and the remaining costs of a traced watch and of a traced waitForRest awaited mid-animation are documented where proof authors look.
- The diff still carries the open #8045, #8548 and #8549 (4,146 authored lines, 943 of them this PR's own); please land those first and update this branch so it carries only its own commits.
- The carried thin-window rule in
busyWindowProof(busy.tsL475–L477) counts an undelivered frame as a miss when fewer than four frames were delivered, while the README's busy-main-thread paragraph and the JSDoc (busy.tsL202–L205, L228–L231) still say it never is; one clause in each, in #8045, would fix it.
[Automated review]
Comment on lines
+228
to
+231
| * - at least `minMovedFraction` of the frames judged moved the tracked axis | ||
| * (a short window may still miss one): the frames the screencast delivered | ||
| * in the judged part, so an irregular cast is judged on what it delivered | ||
| * and a frame it did not deliver is a gap for the stall check, not a miss; |
There was a problem hiding this comment.
Since the thin-window rule (L475–L477), a judged part with fewer than minJudgedFrames delivered frames is judged against the span's expectedFrames, so an undelivered frame counts as a miss there. This bullet, movedFraction's JSDoc (L202–L205: "when nothing moved or was delivered") and the README's busy-main-thread paragraph ("a frame it did not deliver is a gap, for the stall check") still say it never is. A 70 ms judged part with every other frame delivered, both moved and a longest stall of 2 intervals, now fails "2 of 4 expected frames moved (50% < 90%)". One clause in each, as the inline comment at L457–L460 already has, would bring them in line (the header's SYNC note asks for that).
The README's busy-main-thread paragraph and busyWindowProof's JSDoc said an undelivered frame is never a miss; when fewer than minJudgedFrames frames were delivered in the judged part, its span counts every frame. One clause in each says so.
…ile it ran A page opened after startCompositorTrace and closed before stop() was still credited: its frames are in the browser-wide record, and neither check saw it. The record now watches for new pages from start to stop, in the page's own context and through the browser's target discovery in every context, and stop() throws if one opened. Negative controls: a page opened and closed mid-record, and one in another context, both make stop() refuse; alone, it records.
There was a problem hiding this comment.
Changes requested. The recorder and the waits now stand down inside traceRendering as described, and the busy verdict's docs now state its thin-window rule. The diff still carries the open #8045, #8548 and #8549 (4,151 authored lines, 943 of them this PR's own); please land those first and update this branch so it carries only its own commits.
[Automated review]
There was a problem hiding this comment.
Changes requested. The recorder and the waits now stand down inside traceRendering as described: the anchored self-tests count 0 harness-caused Layouts in the steady middle, against 12 to 24 with the stand-down disabled.
- The diff still carries the open #8045, #8548 and #8549 (4,227 authored lines, 943 of them this pull request's own); please land those first and update this branch so it carries only its own commits.
- Inside a trace,
waitForAnimationandwatch.started()on an element the page removed reject with “no animation started on the element” instead of “the element is not in the document” (curves.tsL958–L960), unlike the untraced path and the self-test atharness.interaction.browser.spec.tsL2107; please report a missing element as missing there too, with a traced twin of that self-test.
[Automated review]
| `waitForAnimation: ${String(locator)} matches ${handles.length} elements`, | ||
| ); | ||
| } | ||
| return handles.length === 0 |
There was a problem hiding this comment.
Inside a trace, a locator that matches nothing returns [] here, so waitForAnimation and watch.started() poll to their deadline and reject with “no animation started on the element … take watchAnimations(locator) before it” (L1252–L1258), where the untraced path rejects with “the element is not in the document … (watch the parent)” (withinTimeout, L987–L991), the outcome the self-test at harness.interaction.browser.spec.ts L2107 asserts. With that self-test's steps inside traceRendering, both waits rejected with the first message. Could the traced deadline check locator.count() and throw the same “not in the document” error, with a traced twin of that self-test?
Inside a trace, waitForAnimation and watch.started() read the element through handles, which find no element rather than wait for one, so a wait on an element the page removed rejected with "no animation started on the element" instead of "the element is not in the document", unlike the untraced path. At its deadline, a traced wait with no element now rejects as missing. Traced twin of the missing-element self-test; it fails on the previous code. Co-authored-by: Christopher Chedeau <vjeuxx@gmail.com>
There was a problem hiding this comment.
Changes requested, for scope only: the recorder and the waits now stand down inside traceRendering as described, and the anchored self-tests count 0 harness-caused Layouts in the steady middle against 12 to 24 with the stand-down disabled. The diff still carries the open #8045, #8548 and #8549 (4,285 changed lines, 1,001 of them this pull request's own), so merging it would land all three; please land those first and update this branch so it carries only its own commits.
[Automated review]
4 tasks done
4 tasks done
4 tasks done
Contributor
|
Posted by Cindy's agent on her behalf: closed and reopened only to restart this PR's CI on the current workflow after tonight's queue-wide cancel; nothing in the PR changed. |
# Conflicts: # internal/interaction-harness/README.md # internal/interaction-harness/src/busy.test.ts # internal/interaction-harness/src/busy.ts # internal/interaction-harness/src/frames.ts # internal/interaction-harness/src/harness.interaction.browser.spec.ts
There was a problem hiding this comment.
Changes requested, for scope only: the recorder and the waits stand down inside traceRendering as described, and the anchored self-tests pass with no harness-caused Layout in the steady middle. With #8045 landed, the diff still carries the open #8548 and #8549 (2,442 changed lines, 1,001 of them this pull request's own), so merging it would land both; please land those first and merge main into this branch so it carries only its own commits.
[Automated review]
This was referenced Oct 29, 2026
3 tasks done
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Maintainer impact
A per-frame "no Layout or Paint" proof counted layouts that the page under test never caused. The harness's animation recorder, and its waits, made every frame a main-thread frame, so Blink restyled each running animation's target and laid out an anchored one.
Intended invariant
No product change. A rendering trace counts only the page's own main-thread work.
Change
traceRenderingsets__harnessRenderingTracebefore its start mark, then clears it after its end mark and resumes the recorder, whose catch-up read lands outside the counted window.animate()wrapper still records script animations.waitForReststands down the same way: it waits on the gate's real timers for the element's scheduled curve ends and reads no style.waitForAnimationand a watch'sstarted()stand down too.record()from outside the recorder andarm()'s read do nothing; the wait takes the element's animations from the recorder's log, judges a started one by its own times, reads timing throughgetTiming(), and reaches the element through a handle. Each alone laid out an anchored element every frame:locator.evaluate(each call makes a main-thread frame),document.getAnimations(),element.getAnimations(),getComputedTiming().traceRenderingandwaitForAnimation.Depends on #8549 (on #8548 and #8045; #8348 and #8505 have landed); review the commits after #8549's head.
Evidence
waitForRestin the body, and 24–25 withwaitForAnimationandwatch.started()polling tominProgressMs: 450.Scope
Testing