feat(interaction): cast filmstrips, rect sampling, a Storybook highlight default and the WebKit shader cache

internal/interaction-harness/README.md

@@ -38,6 +38,12 @@ under a rootful docker daemon the container runs as your user so its
 node internal/interaction-harness/scripts/webkit-container.mjs [playwright test args…]
 ```
 
+WebKit composites through Mesa's software GL there, which compiles each shader
+the first time a page draws it (450–700 ms for the first frame of a blurred or
+newly animated layer). The script keeps Mesa's shader cache on the host
+(`~/.cache/astryx-webkit-shaders/<image>`, or `ASTRYX_WEBKIT_SHADER_CACHE`), so
+only the first run on a machine pays for them.
+
 Evidence (storyboards, curve traces) is written to
 `test-results/interaction-evidence/` (override with
 `ASTRYX_INTERACTION_EVIDENCE`) and attached to the test report. It is never
@@ -49,13 +55,13 @@ committed: a pull request links or attaches it.
 | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `scripts.ts`    | Touch scripts as data: `tap`, `drag` (hold, rest), `flick` (exact final-move speed, velocity profiles), `holdThenDrag`, `pinch`, `rotate`, `path`, `merge` (a second finger mid-gesture), `releaseVelocity`, `toTouchEvents`.                                                |
 | `touch.ts`      | `touchDriver(page).play(script)`: the script as real engine input. `installInputLog` / `readInputLog`: what the page received.                                                                                                                                               |
-| `frames.ts`     | `startScreencast` (timestamped composited frames), `markElement` (a marker chip or outline), `trackMarker` (marker box per frame), `frameStats`.                                                                                                                             |
+| `frames.ts`     | `startScreencast` (timestamped composited frames), `markElement` (a marker chip or outline), `trackMarker` (marker box per frame), `sampleRects` (per-rAF rects), `frameStats`.                                                                                              |
 | `busy.ts`       | `blockMainThread(page, ms)` and `busyWindowProof(tracked, window)`: frames presented, frames that moved, longest stall, a verdict; `presented` adds the compositor's own frames.                                                                                             |
 | `webInput.ts`   | `webInputDriver(page)`: `wheel({x, y, dx, dy, ctrl})` and `pen(points)` as real engine input, for proofs across pointers (`spec:AST-100` FR4); `wheel` resolves once the page has the step; read back through the input log.                                                 |
 | `trace.ts`      | `traceRendering(page, body)`: main-thread counts of UpdateLayoutTree / Layout / PrePaint / Paint / Layerize / UpdateLayer / ScrollLayer / HitTest, `countBetween`, long tasks; `installLongTaskLog` / `readLongTasks`; `startCompositorTrace`: when the compositor presents. |
 | `curves.ts`     | `holdAnimations`, `armHold`, `seekAnimations`, `sampleCurve` (presented transform / opacity at any t), `waitForAnimation`, `watchAnimations`, `waitForRest`, `releaseAnimations`, `releaseAnimationLog` (frees what the recorder kept, before a heap or listener count).     |
 | `math.ts`       | Oracles independent of the code under test: `cubicBezier`, `timingFunction`, `springStep` (SwiftUI response / dampingFraction, initial velocity), `projectDecay`, `rubberBand`, `parseTransform`, `compareCurve`, `timeline`.                                                |
-| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event): PNG contact sheet + JSON trace.                                                                                                                                      |
+| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event), `castStoryboard` (real screencast frames): PNG contact sheet + JSON trace.                                                                                           |
 | `output.ts`     | `evidenceDir`, `writeEvidence`: where evidence goes and how it is attached to the report.                                                                                                                                                                                    |
 | `story.ts`      | `phone`, `withStorybook`, `openStory`, `frames`, and the timer gate (`installTimerGate`, `freezeTimers`, `thawTimers`).                                                                                                                                                      |
 
@@ -71,6 +77,14 @@ dispatch latency, and the events go through the compositor's gesture detection
 the limits below. Read positions a frame after a move: Chromium delivers
 `pointermove` aligned to animation frames.
 
+**Stories, not Storybook.** `openStory` turns Storybook's highlight module off
+on every page it opens, and checks that its root element is absent. Left on,
+the module observes `#storybook-root` and, on every insertion, reads the
+computed style of every element on the page inside the inserting task (117
+reads and two forced style recalculations for one exit copy in a Button story),
+so a proof would measure Storybook as well as the component.
+`openStory(page, origin, id, args, {storybookHighlight: true})` keeps it on.
+
 **Curve sampling.** Pause every running animation (CSS transitions and
 animations are Web Animations too), seek to a time, read `getComputedStyle`:
 the presented value at that exact time, independent of frame timing. Finished
@@ -152,7 +166,8 @@ every frame, so one late frame is not "1 of 1 moved"). Pass
 `maxMedianFrameMs` (20 at 60 Hz) only in a proof that claims display rate.
 `verdict.failures` names each miss with its numbers, and `summary` carries the
 raw numbers for the report. Frames are judged in presentation order: a loaded
-machine can deliver them out of order.
+machine can deliver them out of order. `castStoryboard` turns the frames of the
+block into a filmstrip.
 
 A just-started animation can reach the screen after the block began (50–100 ms
 later in WebKit on a loaded machine). When the cast saw the element at rest
@@ -255,14 +270,14 @@ backstop, a cleanup render) would do so behind a held storyboard's back:
 `installTimerGate` before the animation starts and `freezeTimers` while
 photographing. The page can still cancel a held callback, a held frame callback
 gets the time it actually runs, and the harness's own waits (its frame waits,
-`blockMainThread` with `during`) are not held.
+`blockMainThread` with `during`, `sampleRects`) are not held.
 An animation that repeats forever does not set a storyboard's default length;
 with nothing else to film, pass `times`. A touch filmstrip keeps the scripted
 event `timeStamp`s (Chromium), but a handler that reads `performance.now()` or
 `Date.now()` sees one event per photograph, about 100 ms apart, so a
 clock-timed release flings slower when filmed than when played: a storyboard
 cannot show such a gesture; record it in real time instead, with
-`startScreencast`. Chromium can return an empty copy of
+`startScreencast` and `castStoryboard`. Chromium can return an empty copy of
 the page's surface right after it changes ("Unable to capture screenshot"); a
 storyboard takes that photograph again two frames later, up to three attempts
 in all, and lists each retake under `recaptures` in the trace.

internal/interaction-harness/scripts/webkit-container.mjs

@@ -16,10 +16,19 @@
  * docker daemon the container runs as the invoking user (rootless podman and
  * rootless docker already map it), so the `test-results/` it writes into the
  * mounted checkout stay the host user's and the next host run can clear them.
+ *
+ * WebKit composites through Mesa's software GL in the image, which compiles
+ * each shader variant the first time a page draws it: 450–700 ms for the
+ * first frame of a blurred or newly animated layer, long enough for a short
+ * animation to end before it is ever presented. Mesa caches what it compiles,
+ * and the cache is kept on the host (`~/.cache/astryx-webkit-shaders/<image>`,
+ * or `ASTRYX_WEBKIT_SHADER_CACHE`), so a run starts with every shader an
+ * earlier run compiled instead of paying for them in its first tests.
  */
 import {spawnSync} from 'node:child_process';
 import fs from 'node:fs';
 import {createRequire} from 'node:module';
+import os from 'node:os';
 import path from 'node:path';
 import {fileURLToPath} from 'node:url';
 
@@ -42,6 +51,21 @@ if (engine == null) {
   process.exit(1);
 }
 const args = process.argv.slice(2);
+// Where the container sees the host's shader cache (see the header).
+const containerShaderCache = '/astryx-shader-cache';
+// Resolved to an absolute path: a bare relative value would be mounted as a
+// named volume rather than a host directory. An empty value means the default.
+const shaderCache = process.env.ASTRYX_WEBKIT_SHADER_CACHE
+  ? path.resolve(process.env.ASTRYX_WEBKIT_SHADER_CACHE)
+  : path.join(
+      os.homedir(),
+      '.cache',
+      'astryx-webkit-shaders',
+      image.replace(/[^\w.-]+/g, '_'),
+    );
+fs.mkdirSync(shaderCache, {recursive: true});
+// The harness self-test reads this marker inside the container.
+fs.writeFileSync(path.join(shaderCache, 'ASTRYX-SHADER-CACHE'), `${image}\n`);
 const env = [
   'ASTRYX_STORYBOOK_DIR',
   'ASTRYX_INTERACTION_EVIDENCE',
@@ -84,6 +108,10 @@ const command = [
   '--pids-limit=-1',
   '-v',
   `${root}:${root}`,
+  '-v',
+  `${shaderCache}:${containerShaderCache}`,
+  '-e',
+  `MESA_SHADER_CACHE_DIR=${containerShaderCache}`,
   '-w',
   root,
   ...env,

internal/interaction-harness/src/frames.ts

@@ -7,11 +7,12 @@
  *   `Screencast` domain)
  * @output `startScreencast` (timestamped composited frames), `markElement`
  *   (a marker outline the frames can find), `trackMarker` (per-frame marker
- *   box), `frameStats`.
+ *   box), `sampleRects` (per-rAF rect sampling), `frameStats`.
  * @position Frame evidence for motion proofs. The screencast is captured off
  *   the page's main thread, from what the compositor presents, so it keeps
  *   delivering frames while a script blocks the main thread: that is what makes
- *   the busy-main-thread proof possible.
+ *   the busy-main-thread proof possible. Rect sampling is the cheap path when
+ *   the main thread is free.
  *
  * Marker tracking needs no image library: frames are decoded and scanned in a
  * scratch page of the same browser (createImageBitmap + canvas), so the
@@ -399,6 +400,71 @@ export async function trackMarker(
   }
 }
 
+export interface RectSample {
+  /** Epoch ms of the rAF callback. */
+  readonly t: number;
+  readonly box: Box | null;
+  readonly transform: string;
+  readonly opacity: number;
+}
+
+/**
+ * Samples the element's rect, transform and opacity on every animation frame
+ * for `durationMs`. Main-thread sampling: it stops while the main thread is
+ * blocked, which is the point of the screencast path above.
+ */
+export async function sampleRects(
+  locator: Locator,
+  durationMs: number,
+): Promise<RectSample[]> {
+  return locator.evaluate(
+    (element, duration) =>
+      new Promise<
+        {
+          t: number;
+          box: {x: number; y: number; width: number; height: number} | null;
+          transform: string;
+          opacity: number;
+        }[]
+      >(resolve => {
+        // The harness's own waits run on the real timers: a frozen timer gate
+        // (story.ts) holds only the page's.
+        const gate = (
+          window as unknown as {
+            __harnessTimerGate?: {
+              frame?: typeof requestAnimationFrame;
+            };
+          }
+        ).__harnessTimerGate;
+        const frame = gate?.frame ?? window.requestAnimationFrame.bind(window);
+        const samples: {
+          t: number;
+          box: {x: number; y: number; width: number; height: number} | null;
+          transform: string;
+          opacity: number;
+        }[] = [];
+        const end = performance.now() + duration;
+        const tick = (now: number) => {
+          const rect = element.getBoundingClientRect();
+          const style = getComputedStyle(element);
+          samples.push({
+            t: performance.timeOrigin + now,
+            box: {x: rect.x, y: rect.y, width: rect.width, height: rect.height},
+            transform: style.transform,
+            opacity: Number(style.opacity),
+          });
+          if (now < end) {
+            frame(tick);
+          } else {
+            resolve(samples);
+          }
+        };
+        frame(tick);
+      }),
+    durationMs,
+  );
+}
+
 export interface FrameStats {
   readonly frames: number;
   /** Frames whose tracked value differs from the frame before. */