diff --git a/.changeset/build-stylex-cross-file-consts.md b/.changeset/build-stylex-cross-file-consts.md new file mode 100644 index 0000000000000..736b20da4327d --- /dev/null +++ b/.changeset/build-stylex-cross-file-consts.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/build': patch +--- + +[fix] `astryxStylex` resolves `defineConsts` values read from another file. With the `{stylexOptions}` form (the one Storybook uses), a file that read only constants from a `.stylex.ts` module lost that import once StyleX compiled it, so the constants never reached the CSS: a constant used as a media-query key failed the build ("Invalid empty selector") and one used as a value shipped as a custom property nothing declares. In either form, a constant defined on the other side of the layer split, or in an Astryx package (compiled with the library class-name prefix) and read from an app file (compiled with its own), stayed unresolved. + +@vjeux diff --git a/.changeset/contextmenu-glass-morph.md b/.changeset/contextmenu-glass-morph.md new file mode 100644 index 0000000000000..cb0e131fcc9a6 --- /dev/null +++ b/.changeset/contextmenu-glass-morph.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[feat] Under a finger, on every touch device, when a held finger opens a ContextMenu, the menu now grows out of the finger as iOS 26's does: it starts small at the press point and grows into its place beside the lifted trigger, overshooting slightly, as it fades in. Only its position, scale and opacity move, on the compositor. Under Reduce Motion the menu cross-fades in where it is instead, in 200 ms: nothing moves or grows. + +@vjeux diff --git a/.changeset/contextmenu-lift-presentation.md b/.changeset/contextmenu-lift-presentation.md new file mode 100644 index 0000000000000..25de19d5cf964 --- /dev/null +++ b/.changeset/contextmenu-lift-presentation.md @@ -0,0 +1,10 @@ +--- +'@astryxdesign/core': patch +'@astryxdesign/cli': patch +--- + +[feat] Under a finger, on every touch device (iOS and Android alike), a ContextMenu opened by a held finger now presents as iOS 26's does. A block trigger itself lifts out of the page, with no copy and with its spot in the page held: it grows on from the press to 26 px larger on its longer side (15% at most), with the theme's high shadow fading in under it, over the theme's overlay, and it stays inside the layer runtime's viewport gutter. A tap on the tint closes the menu; while the lift is still growing, a tap or a drag closes nothing and reaches nothing under it, and the menu finishes opening (keys still work). The menu sits 16 px from the lifted trigger, below it when it fits and above it otherwise, centered on it or flush with the edge it is held against (right-to-left pages included), and fades in, with no row lit until the finger moves onto one. Closing returns the trigger to its place. Under Reduce Motion the animator cross-fades everything in place instead, in 200 ms, as the reduced-motion design record asks: nothing travels or grows. Inline triggers, pointer opens and the bottom sheet are unchanged. A fine pointer keeps today's menu at the cursor. + +The tint is the theme's modal overlay (`--color-overlay`). + +@vjeux diff --git a/.changeset/contextmenu-presentation-popover.md b/.changeset/contextmenu-presentation-popover.md new file mode 100644 index 0000000000000..b5ee8f739742e --- /dev/null +++ b/.changeset/contextmenu-presentation-popover.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[feat] Under a finger, a ContextMenu whose `presentation` is omitted now opens from a held finger on iOS's timeline (the touch model). Pass `presentation="popover"` to keep today's menu on every device: the menu at the pointer, opened by a finger held for 500 ms (the long press). Where the primary pointer is fine (a mouse or trackpad), a finger keeps that same long press. + +@vjeux diff --git a/.changeset/contextmenu-press-constants.md b/.changeset/contextmenu-press-constants.md new file mode 100644 index 0000000000000..a598482bd7bc7 --- /dev/null +++ b/.changeset/contextmenu-press-constants.md @@ -0,0 +1,8 @@ +--- +'@astryxdesign/core': patch +'@astryxdesign/cli': patch +--- + +[chore] The UIKit constants table gains ContextMenu's touch timeline as measured on iOS 26.2: the press feedback starts at 150 ms, the press commits at 400 ms, and the menu displays at 735 ms. It also records the drift tolerance, and the press feedback's growth, spring and display latency. A closed-form press-feedback curve, checked against UIKit's per-frame presentation, comes with them. ContextMenu's touch timeline uses them. + +@vjeux diff --git a/.changeset/contextmenu-touch-timeline.md b/.changeset/contextmenu-touch-timeline.md new file mode 100644 index 0000000000000..2a279da741343 --- /dev/null +++ b/.changeset/contextmenu-touch-timeline.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[feat] ContextMenu opens from a held finger on iOS 26's measured timeline, on every touch device (a coarse primary pointer, iOS and Android alike). The trigger starts to grow at 150 ms (the onset of core's one press machine), following UIKit's measured press curve on the compositor, and the hold runs on useLongPress's detector with UIKit's measured moments. At 400 ms the press takes the finger: its lift no longer sends a click, and the page cannot scroll under it. The menu opens at 735 ms if the finger is still down, or as soon as it lifts after 400 ms. A finger that lifts before 400 ms, or drifts more than 11.5 px on either axis before the menu opens, gets no menu, and the trigger springs back. The page stays still while the menu is up, and the trigger returns to its size when the menu closes. A touch open lights no row: the menu takes the focus, and a row lights only under the finger, as in UIKit (keyboard and pointer opens still focus the first item). A second finger cancels the press, and Android's own long-press menu no longer opens a second menu. A fine pointer (a mouse) keeps the right-click menu, and the touch recognizer stays off there. + +@vjeux diff --git a/.changeset/gesture-arena-moving-cancel.md b/.changeset/gesture-arena-moving-cancel.md new file mode 100644 index 0000000000000..5b7e52cad131a --- /dev/null +++ b/.changeset/gesture-arena-moving-cancel.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: a gesture that cancels itself on a finger's move (a long press dragged past its tolerance) keeps that finger claimed until it lifts, so the page never takes the drag over midway; a resting finger is still given back when its gesture is cancelled. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-browser-negotiation.md b/.changeset/gesture-browser-negotiation.md new file mode 100644 index 0000000000000..1930e34496d7e --- /dev/null +++ b/.changeset/gesture-browser-negotiation.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: touch-action derivation, the native scroll probe and haptics for the gesture system. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-hook-fixes.md b/.changeset/gesture-hook-fixes.md new file mode 100644 index 0000000000000..87f08a0948e31 --- /dev/null +++ b/.changeset/gesture-hook-fixes.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the gesture hooks call nothing back after unmount; `usePressTracker` brakes with the host by default and keeps one tracker, so an option change or a new ref never drops a press in progress; the recognizer's arena plumbing is `protected`. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-host-fixes.md b/.changeset/gesture-host-fixes.md new file mode 100644 index 0000000000000..2fc0915c974a6 --- /dev/null +++ b/.changeset/gesture-host-fixes.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: gesture host fixes from review. A page scroll under a still finger keeps the press and its click; watchers and touches nothing can claim get passive listeners only; a claimed pen holds only its own stylus touches; an element's own `touch-action` is never widened; slotted content scrolls with its shadow-tree scroller. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-host-owner.md b/.changeset/gesture-host-owner.md new file mode 100644 index 0000000000000..151a939aff001 --- /dev/null +++ b/.changeset/gesture-host-owner.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the gesture host becomes the one document-level touch owner: a scroll lock's page hold, finger scrolls, the finger count, touch gates, and a kept press for continuous-tracking controls, whose element gets `touch-action: manipulation` where the pointer is coarse. A second finger fails a one-finger pan that has not claimed, so the page can pinch. A scroll-edge pan holds a lone finger from the first move that pulls away from an edge every scroller under it is at, so those moves are cancelled before it begins; with a second finger anywhere on the screen the page keeps its pinch. Where the primary pointer is fine, the host attaches nothing and registers no listener, following the pointer when it changes; a recognizer that reads each event's pointer itself opts out with `perEventPointer`, and a press tracker with it paints a finger's press feedback only, with no control event and no mouse or pen press. A press tracker whose hold acts (a stepper's hold-repeat) can bring a selection lock for its presses, as a recognizer can. Internal, for the touch components that adopt them; not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-host.md b/.changeset/gesture-host.md new file mode 100644 index 0000000000000..eb9f2fec42db2 --- /dev/null +++ b/.changeset/gesture-host.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the gesture system's DOM host (claims, click suppression, the browser's takeover, the pointer behind a contact). Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-pan.md b/.changeset/gesture-pan.md index 12eb02fc42391..908da1b367575 100644 --- a/.changeset/gesture-pan.md +++ b/.changeset/gesture-pan.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: the pan gesture recognizer, with UIKit's measured hysteresis and velocity filter. Not exported from the package entry point; no component uses it yet. +[chore] Internal: the pan gesture recognizer, with UIKit's measured hysteresis and velocity filter. Not exported from the package entry point. @vjeux diff --git a/.changeset/gesture-press-tracking.md b/.changeset/gesture-press-tracking.md new file mode 100644 index 0000000000000..1c66836c74243 --- /dev/null +++ b/.changeset/gesture-press-tracking.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: UIControl press tracking with control and cell profiles and the gesture hooks, with every press painter writing `data-astryx-press` through one internal writer in `utils/pressFeedback.ts`. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-readme.md b/.changeset/gesture-readme.md new file mode 100644 index 0000000000000..9a6293b811b68 --- /dev/null +++ b/.changeset/gesture-readme.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the gesture system's README for component authors, with its browser proof in Chromium and WebKit. + +@vjeux diff --git a/.changeset/gesture-recognizer-core.md b/.changeset/gesture-recognizer-core.md index 0b0bfc389235f..28caef0d5579c 100644 --- a/.changeset/gesture-recognizer-core.md +++ b/.changeset/gesture-recognizer-core.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: the gesture recognizer core for touch presentations: UIKit's recognizer state machine and the arena that settles which recognizer owns a touch. Not exported from the package entry point; no component uses it yet. +[chore] Internal: the gesture recognizer core for touch presentations: UIKit's recognizer state machine and the arena that settles which recognizer owns a touch. Not exported from the package entry point. @vjeux diff --git a/.changeset/gesture-swipe-pinch-rotation-edge-hover.md b/.changeset/gesture-swipe-pinch-rotation-edge-hover.md new file mode 100644 index 0000000000000..ffeadde679ef5 --- /dev/null +++ b/.changeset/gesture-swipe-pinch-rotation-edge-hover.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: swipe, pinch, rotation, screen-edge pan and hover gesture recognizers. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/gesture-tap-long-press.md b/.changeset/gesture-tap-long-press.md index 9ef786da65842..f1efef8d29014 100644 --- a/.changeset/gesture-tap-long-press.md +++ b/.changeset/gesture-tap-long-press.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: tap and long-press gesture recognizers with UIKit's measured thresholds. Not exported from the package entry point; no component uses them yet. +[chore] Internal: tap and long-press gesture recognizers with UIKit's measured thresholds. Not exported from the package entry point. @vjeux diff --git a/.changeset/gesture-watcher.md b/.changeset/gesture-watcher.md new file mode 100644 index 0000000000000..bec7cd286e49d --- /dev/null +++ b/.changeset/gesture-watcher.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: a recognizer created with `claimsTouches: false` is a watcher: passive listeners only, simultaneous with every recognizer, and it follows a touch the browser takes for a scroll to the lift. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/layer-exit-copy.md b/.changeset/layer-exit-copy.md new file mode 100644 index 0000000000000..441788643f6d2 --- /dev/null +++ b/.changeset/layer-exit-copy.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the layer runtime gains one exit copy for touch presentations that animate a popover out: a static, inert copy of the closing popover, shown only in the top layer where the popover was drawn, for the close to play on. + +@vjeux diff --git a/.changeset/layer-input-hold.md b/.changeset/layer-input-hold.md new file mode 100644 index 0000000000000..89eb007172d37 --- /dev/null +++ b/.changeset/layer-input-hold.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: one input hold for every layer (`Layer/inputHold`). While a layer presents, a touch, a press or a click anywhere reaches nothing (keys and assistive technology still act, and a pinch still zooms the page); while it dismisses, the same, and the layer's content is inert. The gesture host holds the layer's touches through its touch gates, and `isInputShield` lets a hit test see past a hold's shield. Not exported from the package entry point; ContextMenu's touch lift uses it. + +@vjeux diff --git a/.changeset/lift-module.md b/.changeset/lift-module.md new file mode 100644 index 0000000000000..de42637a5b97a --- /dev/null +++ b/.changeset/lift-module.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: one opt-in lift-from-trigger presentation for touch surfaces. The trigger itself lifts into the top layer above a tinted backdrop; a surface grows out of the press beside it on the shared rect morph and shrinks back into it on the layer runtime's exit copy; the trigger returns to its place. Translate, scale and opacity only; nothing mounted at rest, four elements while lifted. Not exported from the package entry point; ContextMenu's touch lift uses it. + +@vjeux diff --git a/.changeset/motion-progress.md b/.changeset/motion-progress.md new file mode 100644 index 0000000000000..6f5882f7f6e02 --- /dev/null +++ b/.changeset/motion-progress.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: one progress module for the motion foundation. `progressCurve` gives a spring's progress from 0 to 1 as a closed-form curve, `mapProgress` maps any curve from another (a companion's dim following a sheet, a clamped range), `mapRange` is the clamped linear map they share, and `presentedProgressVelocity` reads the velocity an interrupted motion continues from. One velocity unit throughout: progress per second. Nothing public changes; ContextMenu's touch lift uses it. + +@vjeux diff --git a/.changeset/motion-rect-morph.md b/.changeset/motion-rect-morph.md new file mode 100644 index 0000000000000..8598e899b7e10 --- /dev/null +++ b/.changeset/motion-rect-morph.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the rect morph, one way for a touch presentation to grow out of a source rect and shrink back into it. `lerpPath` (a straight line between two rects) and `edgeGrowPath` (UIKit's popover, growing out of a point on its edge) name the rect an element shows at each progress; `morphCurves` turns a path into compositor curves (`translate`, `scale` and a companion `opacity`); `createRectMorph` presents an element along a path and dismisses it back on a copy or on itself, each continuing the other from the rect on screen with its velocity, with Reduced Motion left to the animator. It runs on new progress helpers (`progressCurve`, `mapProgress`, `mapRange`, `presentedProgressVelocity`). Not exported from the package entry point; ContextMenu's touch lift uses it. + +@vjeux diff --git a/.changeset/presentation-resolver.md b/.changeset/presentation-resolver.md new file mode 100644 index 0000000000000..decee65fa0585 --- /dev/null +++ b/.changeset/presentation-resolver.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] An internal `usePresentation` / `readPresentation` resolves a component's `presentation` prop into the surface to render and whether its touch input modes apply, keeping an omitted value distinct from an explicit `'popover'` so a caller's choice always wins over a touch default. ContextMenu reads it; its touch behavior has its own entry. + +@vjeux diff --git a/.changeset/reduced-motion-record.md b/.changeset/reduced-motion-record.md new file mode 100644 index 0000000000000..2ec44f4c1f9c8 --- /dev/null +++ b/.changeset/reduced-motion-record.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: Reduce Motion for the touch presentations follows design:reduced-motion beyond the animator too. `reducedMotionFade()` gives code beside the animator the animator's own fade (an opacity-only 200 ms cross-fade on a coarse pointer or after a touch release, at once on any other pointer), `animator.reducedMotion()` reports which form a change takes for feedback that plays no fade of its own, and `reducedMotion.stylex.ts` holds the fade's constants for StyleX transitions. The fade's length is not a theme motion role. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/touch-selection-lock-gestures.md b/.changeset/touch-selection-lock-gestures.md new file mode 100644 index 0000000000000..d1f3ab7c2b7fc --- /dev/null +++ b/.changeset/touch-selection-lock-gestures.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the gesture host applies a recognizer's own `selectionLock` to that recognizer's contact and imports no lock itself; a long press brings one only when it has a hold (a `minimumPressDuration` above 0), and `selectionLock: null` opts out. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/touch-selection-lock.md b/.changeset/touch-selection-lock.md new file mode 100644 index 0000000000000..e5285afc1f8ad --- /dev/null +++ b/.changeset/touch-selection-lock.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[fix] Where the primary pointer is fine (a mouse, a trackpad, a touchscreen laptop, a desktop pen tablet), the press feedback controller no longer registers a touch listener: it reads the primary pointer at the first finger or pen press and keeps its `touchcancel` listener only while that pointer is coarse. + +@vjeux diff --git a/.changeset/uikit-constants-table.md b/.changeset/uikit-constants-table.md index 01fd41a182b0b..2c063cdb50bc7 100644 --- a/.changeset/uikit-constants-table.md +++ b/.changeset/uikit-constants-table.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: the UIKit constants table for touch presentations (springs, scroll physics, timing curves and navigation values), each value with its source and a checked list of the simulator measurements still owed. Not exported from the package entry point; no component reads it yet. +[chore] Internal: the UIKit constants table for touch presentations (springs, scroll physics, timing curves and navigation values), each value with its source and a checked list of the simulator measurements still owed. Not exported from the package entry point. @vjeux diff --git a/.changeset/uikit-motion-animator-pseudo-element.md b/.changeset/uikit-motion-animator-pseudo-element.md new file mode 100644 index 0000000000000..7ccb10bab35da --- /dev/null +++ b/.changeset/uikit-motion-animator-pseudo-element.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: the touch presentations' animator drives a pseudo-element such as a dialog's ::backdrop: a static filling animation holds its value after a curve ends and after `set`, a custom property on its host promotes it, and a Reduce Motion dip borrows and hands back the pseudo-element's own opacity. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/uikit-motion-animator.md b/.changeset/uikit-motion-animator.md index d489e3022e6db..6b23ce1985186 100644 --- a/.changeset/uikit-motion-animator.md +++ b/.changeset/uikit-motion-animator.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: a compositor animator for touch presentations. It plays UIKit's springs, and scroll coasts (built by a separate coast module, so a surface that never coasts does not carry them; `decay` plays momentum, and sends a spring back from past an edge through the Reduce Motion policy: a cross-fade on a coarse pointer or after a touch release, at once elsewhere) as Web Animations on translate, rotate, scale and opacity, so they keep running while JavaScript is busy, and reads values and velocities back from the curves for interruption, retargeting and Reduce Motion cross-fades. Not exported from the package entry point; no component uses it yet. +[chore] Internal: a compositor animator for touch presentations. It plays UIKit's springs, and scroll coasts (built by a separate coast module, so a surface that never coasts does not carry them; `decay` plays momentum, and sends a spring back from past an edge through the Reduce Motion policy: a cross-fade on a coarse pointer or after a touch release, at once elsewhere) as Web Animations on translate, rotate, scale and opacity, so they keep running while JavaScript is busy, and reads values and velocities back from the curves for interruption, retargeting and Reduce Motion cross-fades. Not exported from the package entry point. @vjeux diff --git a/.changeset/uikit-motion-math.md b/.changeset/uikit-motion-math.md index 7af3d73a92a98..cbb9399568857 100644 --- a/.changeset/uikit-motion-math.md +++ b/.changeset/uikit-motion-math.md @@ -2,6 +2,6 @@ '@astryxdesign/core': patch --- -[chore] Internal: the spring, decay, rubber-band and projection math of UIKit for touch presentations, as pure functions checked against an independent integration and against values read from UIKit. Not exported from the package entry point; no component uses it yet. +[chore] Internal: the spring, decay, rubber-band and projection math of UIKit for touch presentations, as pure functions checked against an independent integration and against values read from UIKit. Not exported from the package entry point. @vjeux diff --git a/.changeset/uikit-motion-use-animator.md b/.changeset/uikit-motion-use-animator.md new file mode 100644 index 0000000000000..b234a4e987752 --- /dev/null +++ b/.changeset/uikit-motion-use-animator.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[chore] Internal: useAnimator, a React wrapper around the touch presentations' animator that never re-renders per frame, and browser proofs on the interaction harness that the animator's motion keeps running through a blocked main thread in Chromium and WebKit. Not exported from the package entry point. + +@vjeux diff --git a/.changeset/uselongpress-stages.md b/.changeset/uselongpress-stages.md new file mode 100644 index 0000000000000..a6c0bb3c92218 --- /dev/null +++ b/.changeset/uselongpress-stages.md @@ -0,0 +1,7 @@ +--- +'@astryxdesign/core': patch +--- + +[feat] useLongPress takes `stages`: later moments of the same hold, each fired once the finger has stayed that long from the start and still within `moveCancelPx`. ContextMenu claims the finger at its long press and shows its menu at a later stage. `start` also takes the press's start time, so the deadlines can count from a touch's own timestamp. + +@vjeux diff --git a/apps/storybook/stories/ContextMenu.stories.tsx b/apps/storybook/stories/ContextMenu.stories.tsx index 5c144b8bf45a9..9deadaa00db79 100644 --- a/apps/storybook/stories/ContextMenu.stories.tsx +++ b/apps/storybook/stories/ContextMenu.stories.tsx @@ -589,3 +589,72 @@ export const InScrollContainer: Story = { ), }; + +const touchStyles = stylex.create({ + card: { + width: '200px', + height: '120px', + display: 'grid', + placeItems: 'center', + borderRadius: '12px', + backgroundColor: '#e5e7eb', + color: '#374151', + userSelect: 'none', + }, +}); + +export const TouchLongPress: Story = { + name: 'Touch / long press', + parameters: { + docs: { + description: { + story: + 'On a touch screen (iOS or Android), hold a finger on the card. As on iOS, it starts to grow after 150 ms, a haptic plays at 400 ms where the device can play one, and the menu opens at 735 ms, or as soon as the finger lifts after 400 ms. A finger that lifts before 400 ms, or drifts more than 11.5 px on either axis before the menu opens, gets no menu, and the card springs back. While the menu is up the page stays still, and the card returns to its size when the menu closes.', + }, + }, + }, + render: () => ( + {}}, + {label: 'Share', icon: ShareIcon, onClick: () => {}}, + {label: 'Save to Files', icon: ArrowDownTrayIcon, onClick: () => {}}, + {type: 'divider'}, + { + label: 'Delete', + icon: TrashIcon, + variant: 'destructive', + onClick: () => {}, + }, + ]}> +
Hold for actions
+ + ), +}; + +export const TouchLongPressPopover: Story = { + name: 'Touch / long press, presentation="popover"', + parameters: { + docs: { + description: { + story: + 'With `presentation="popover"`, ContextMenu keeps today’s menu on every device: a finger held for 500 ms opens it at the finger, with no growth and no iOS timeline. A finger on a device whose primary pointer is fine (a mouse or a trackpad) gets the same long press.', + }, + }, + }, + render: () => ( + {}}, + {label: 'Share', icon: ShareIcon, onClick: () => {}}, + {label: 'Save to Files', icon: ArrowDownTrayIcon, onClick: () => {}}, + ]}> +
Hold for actions
+ + ), +}; diff --git a/docs/architecture/layer-runtime.md b/docs/architecture/layer-runtime.md index 0fab0e43de33e..5404257cc43cd 100644 --- a/docs/architecture/layer-runtime.md +++ b/docs/architecture/layer-runtime.md @@ -33,6 +33,8 @@ verified_by: packages/core/src/Layer/anchorName.test.ts, packages/core/src/Layer/useLayerDismissal.test.tsx, packages/core/src/Layer/layerDismissalInvariants.test.tsx, + packages/core/src/Layer/layerExitCopy.test.ts, + packages/core/src/Layer/__tests__/layerExitCopy.interaction.browser.spec.ts, packages/core/src/Popover/Popover.test.tsx, packages/core/src/DropdownMenu/DropdownMenu.test.tsx, packages/core/src/ContextMenu/ContextMenu.test.tsx, @@ -62,6 +64,7 @@ deciding_specs: [spec:AST-038, spec:AST-059] "INV7", "INV11", "INV12", + "INV13", "INV14", "INV15" ], @@ -297,6 +300,25 @@ during a press so native light dismissal does not mistake them for outside interaction. Their click action is cancelled as an invoker toggle so the control can perform its own action without closing the layer. +### Exit copy + +The browser hides a popover the moment it closes, whether by a pick, Escape, or +its own light dismiss. A touch presentation whose close stays on screen plays it +on an exit copy: a static copy of the layer that `placeLayerExitCopy` takes as +the layer closes. The copy is laid out on the layer's own box and carries the +transform and opacity the layer is drawn with, so a copy taken while the layer +is still entering holds where it was drawn. It carries no ids, form names, or +form owner; it is `inert`, hidden from assistive technology, and out of pointer +reach; it keeps the layer's scroll positions. It is shown only in the top layer, +as a manual popover, so a clipping, transformed, or sticky ancestor neither cuts +nor moves it; a fixed box or a larger `z-index` is no substitute +(`spec:AST-027` FR8). Where the browser refuses that show during the closing +popover's own toggle, the copy waits, hidden, and enters the top layer once the +toggle is over: after the script that closed the layer, or in the next frame +before it paints. Where there is no top layer (no popover API), or the browser +still refuses, there is no copy and the close is immediate. The presentation +that placed the copy animates it and removes it when its close settles. + ### Current dismissal plumbing `family:overlay-dismissal` owns the shipped Escape/platform-close membership @@ -480,6 +502,12 @@ layers use Layer rendering without joining Escape/platform dismissal. correct, and the runtime polls nothing: an anchor that a scrolled container hides is followed again when an intersection observer reports it back in view. +- **INV13 — One exit copy per closing layer.** A layer's close plays on at most + one exit copy, made by `placeLayerExitCopy`: drawn where the layer was drawn, + inert, hidden from assistive technology, without ids, form names, or form + owner, shown only in the top layer, and removed by its owner when the close + settles. Placing a second copy for the same layer removes the first; another + layer's copy stays. - **INV14 — The runtime owns every layer's shared mechanism.** Internally, the layer runtime owns the shared mechanism layers use: top-layer mounting, close-request delivery, the presentation hold (keeping a layer's chosen presentation from the open until @@ -533,11 +561,13 @@ Current gaps are observable facts, not current target behavior: migration, verification, and completion criteria for these gaps. This record must be updated only as that work ships. -INV14 and INV15 are not met yet, and `spec:AST-003` does not cover them: no -shared presentation hold, exit copy, runtime backdrop tint, or lift module -exists, and Dialog, Drawer, BottomSheet, Lightbox, MobileNav, ToastViewport, -and Selector each build their own top-layer mounting, exit, or backdrop, and -Popover builds its own presentation hold for its sheet presentation. +INV14 and INV15 are not met yet, and `spec:AST-003` does not cover them. The +runtime owns the shared exit copy (INV13, `placeLayerExitCopy`), and the one +lift-from-trigger module exists (INV15, `packages/core/src/liftFromTrigger/`); +no shared presentation hold or runtime backdrop tint exists, Dialog, Drawer, +BottomSheet, Lightbox, MobileNav, ToastViewport, and Selector each build their +own top-layer mounting, exit, or backdrop, and Popover builds its own +presentation hold for its sheet presentation. BottomSheet's own includes its sheet stack's mounting — the re-show that keeps a document's sheets in open order, a step dialog's modal present, and the non-modal sheet's popover host — alongside its scrim; the ordering rules @@ -568,6 +598,12 @@ above are the stack's, the mounting under them is this gap's. entry/return, and controlled close behavior. - A browser fallback change updates public `useLayer` and Popover docs and tests for both the native path and actual fallback. +- An exit-copy change verifies, where each can be observed: the copy's reach, + accessibility and form exclusion in jsdom; its top-layer show (from script + and by light dismiss), its place mid-entrance, and that a layer keeps one + copy, in jsdom and in Chromium and WebKit; and the refused show's wait in + jsdom and on Android Chromium, the engine that refuses a show during another + popover's toggle. ## Owning code @@ -589,6 +625,8 @@ above are the stack's, the mounting under them is this gap's. transitions of a layer and its content across the re-placement's display reset. - `Layer/gestureCounter.ts` owns physical pointer/key gesture identity. +- `Layer/layerExitCopy.ts` owns the exit copy a touch presentation's close plays + on. - `Layer/layerStack.ts`, `Layer/useLayerDismissal.ts`, and `Layer/LayerDepthContext.tsx` own current registration, presence, ordering, and Escape/platform routing. @@ -638,6 +676,7 @@ this current architecture record. | INV3–INV5 | `useLayer.test.tsx` and `anchorName.test.ts` | A mode leaks geometry, RTL resolves from the wrong context, fallback clips, or a sibling anchor is lost | | INV11 | `layerViewportInset.test.ts`, `useLayer.test.tsx`, and the `Core/Layer` viewport-inset stories under the story play guard at a desktop and a phone viewport | A second gutter definition, a `100%`-of-span cap, a layer narrower than its explicit size, a layer box wider than the viewport, a layer inside the gutter, or a layer that slides toward an off-screen anchor | | INV12 | The `Core/Layer` viewport-inset stories' first-frame comparison, and `useLayer.test.tsx`'s synchronous anchor read | A layer whose first painted rectangle differs from its settled one | +| INV13 | `layerExitCopy.test.ts` and `layerExitCopy.interaction.browser.spec.ts` | A copy with an id, a form name, a form owner, or pointer reach; a copy off the layer's drawn box; a copy outside the top layer; a second copy for one layer | | INV6, INV7 | `useLayer.test.tsx`, `Popover.test.tsx`, `DropdownMenu.test.tsx`, and `useMenuHover.test.tsx` | Duplicate close callback or the same press/re-hover reopens a surface | | INV8 | `useLayerDismissal.test.tsx`, `layerDismissalInvariants.test.tsx`, and `useFocusTrap.test.tsx` | Current top registered layer is skipped, two layers close, or a blocker leaks through | | INV9 | Representative Dialog, ContextMenu, Tooltip/HoverCard, and BottomSheet source/tests | A current local channel silently changes ownership or policy | diff --git a/docs/specs/AST-057-row-actions/spec.md b/docs/specs/AST-057-row-actions/spec.md index 2b3f6c57495e1..4c1ed5a60d229 100644 --- a/docs/specs/AST-057-row-actions/spec.md +++ b/docs/specs/AST-057-row-actions/spec.md @@ -131,10 +131,11 @@ and its panels stay put, so no element is added to any row. - **A render prop for a custom panel.** Deferred, not foreclosed: a `render*`-shaped addition can land when a caller needs something the vocabulary cannot say (DEC-1). -- **Gesture tuning.** Axis-lock distance, commit ratio, fling velocity, slide - and spring durations, resistance past the panel: behavior constants in - source, not public API and not design tokens (the same ruling the touch - press model received for its clocks). +- **Gesture tuning.** Axis-lock distance, commit ratio, fling velocity, + resistance past the panel: behavior constants in source, not public API and + not design tokens (the same ruling the touch press model received for its + clocks). The row's slide and spring motion plays on theme motion roles + (`spec:AST-094` FR4). - **Vertical swipes**, and a swipe whose only meaning is dismissal (Toast already has that in `useToastGesture`). - **Public out-of-flow slots on `Item`.** Deliberately not admitted: the @@ -219,9 +220,10 @@ and its panels stay put, so no element is added to any row. | open state | closed, resting open | The row's own. A pointer landing anywhere outside a resting row closes it, which is also how a drag on a neighbour closes it; no group state exists. | closed | `spec:AST-057` | proposed | | the modality split | internal | A mouse drag never starts the swipe; a finger's or a pen's pointer event or a horizontal wheel gesture does, read from the input event, not a prop or a media query (`architecture:interaction-modality` INV6). | — | `spec:AST-057` | proposed | -Not public: axis-lock distance, commit ratio, fling velocity, durations, -resistance, panel widths, the custom property's name. They are behavior -constants. +Not public: axis-lock distance, commit ratio, fling velocity, resistance, +panel widths, the custom property's name. They are behavior constants. The +row's slide and spring motion plays on theme motion roles (`spec:AST-094` +FR4). ## Requirements diff --git a/docs/specs/AST-094-touch-presentations/spec.md b/docs/specs/AST-094-touch-presentations/spec.md index 2bdaf010cac3b..d8f90053edefb 100644 --- a/docs/specs/AST-094-touch-presentations/spec.md +++ b/docs/specs/AST-094-touch-presentations/spec.md @@ -122,25 +122,28 @@ a measured, sourced difference exists. wherever the native motion settles with an overshoot or stretches with a rubber band. - | Role | Default | Source | - | -------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------- | - | `springs.fingerTracking` | response 0.15 s, damping ratio 0.86 | Apple: SwiftUI `interactiveSpring()` | - | `springs.emphasis` | duration 0.5 s, bounce 0.3 | Apple: SwiftUI `.bouncy` | - | `springs.settle` | duration 0.5 s, bounce 0 | Apple: SwiftUI `.smooth` | - | `springs.quickChange` | duration 0.5 s, bounce 0.15 | Apple: SwiftUI `.snappy` | - | `springs.viewChange` | mass 3, stiffness 1000, damping 500 | measured: UIKit `UISpringTimingParameters()`, iOS 26.2 simulator, 2026-10-23 | - | `springs.timedChange` | duration 0.5 s, bounce 0 | Apple: UIKit `UIView.animate(springDuration:bounce:)` | - | `springs.stateChange` | response 0.55 s, damping ratio 1 | Apple: SwiftUI `Animation.default` | - | `springs.directChange` | response 0.5 s, damping ratio 0.825 | Apple: SwiftUI `.spring(response:dampingFraction:)` | - | `springs.menuOpen` | damping ratio 0.8, response 0.32 s | measured: UIMenu opening, iOS 26.2 simulator, 2026-10-23 | - | `springs.menuClose` | damping ratio 0.85, response 0.35 s | measured: UIMenu closing, iOS 26.2 simulator, 2026-10-23 | - | `springs.subMenuOpen` | damping ratio 0.85, response 0.4 s | measured: UIKit's own spring for UIMenu's sub-menu opening, iOS 26.2 simulator, 2026-10-24 | - | `springs.subMenuBack` | damping ratio 1, response 0.4 s | measured: UIKit's own spring for UIMenu's sub-menu header back, iOS 26.2 simulator, 2026-10-24 | - | `timing.stateChangeMs` | 250 ms | measured: Core Animation's implicit transaction, iOS 26.2 simulator, 2026-10-23 | - | `timing.curves.viewChange` | cubic-bezier(0.25, 0.1, 0.25, 1) | Apple: `CAMediaTimingFunctionName.default` | - | `timing.curves.exit` | cubic-bezier(0.42, 0, 1, 1) | measured: UIKit's ease-in, iOS 26.2 simulator, 2026-10-23 | - | `timing.curves.entrance` | cubic-bezier(0, 0, 0.58, 1) | measured: UIKit's ease-out, iOS 26.2 simulator, 2026-10-23 | - | `timing.curves.move` | cubic-bezier(0.42, 0, 0.58, 1) | measured: UIKit's ease-in-out, iOS 26.2 simulator, 2026-10-23 | + | Role | Default | Source | + | ---------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------- | + | `springs.fingerTracking` | response 0.15 s, damping ratio 0.86 | Apple: SwiftUI `interactiveSpring()` | + | `springs.emphasis` | duration 0.5 s, bounce 0.3 | Apple: SwiftUI `.bouncy` | + | `springs.settle` | duration 0.5 s, bounce 0 | Apple: SwiftUI `.smooth` | + | `springs.quickChange` | duration 0.5 s, bounce 0.15 | Apple: SwiftUI `.snappy` | + | `springs.viewChange` | mass 3, stiffness 1000, damping 500 | measured: UIKit `UISpringTimingParameters()`, iOS 26.2 simulator, 2026-10-23 | + | `springs.timedChange` | duration 0.5 s, bounce 0 | Apple: UIKit `UIView.animate(springDuration:bounce:)` | + | `springs.stateChange` | response 0.55 s, damping ratio 1 | Apple: SwiftUI `Animation.default` | + | `springs.directChange` | response 0.5 s, damping ratio 0.825 | Apple: SwiftUI `.spring(response:dampingFraction:)` | + | `springs.menuOpen` | damping ratio 0.8, response 0.32 s | measured: UIMenu opening, iOS 26.2 simulator, 2026-10-23 | + | `springs.menuClose` | damping ratio 0.85, response 0.35 s | measured: UIMenu closing, iOS 26.2 simulator, 2026-10-23 | + | `springs.subMenuOpen` | damping ratio 0.85, response 0.4 s | measured: UIKit's own spring for UIMenu's sub-menu opening, iOS 26.2 simulator, 2026-10-24 | + | `springs.subMenuBack` | damping ratio 1, response 0.4 s | measured: UIKit's own spring for UIMenu's sub-menu header back, iOS 26.2 simulator, 2026-10-24 | + | `springs.contextMenuLift` | damping ratio 0.8, response 0.32 s | measured: UIContextMenuInteraction's preview lift, iOS 26.2 simulator, 2026-10-23 | + | `springs.contextMenuDismiss` | damping ratio 0.85, response 0.35 s | measured: UIContextMenuInteraction's dismissal, iOS 26.2 simulator, 2026-10-23 | + | `springs.contextMenuMorph` | damping ratio 0.8, response 0.3 s | measured: UIContextMenuInteraction's menu entrance and preview shadow, iOS 26.2 simulator, 2026-10-23 | + | `timing.stateChangeMs` | 250 ms | measured: Core Animation's implicit transaction, iOS 26.2 simulator, 2026-10-23 | + | `timing.curves.viewChange` | cubic-bezier(0.25, 0.1, 0.25, 1) | Apple: `CAMediaTimingFunctionName.default` | + | `timing.curves.exit` | cubic-bezier(0.42, 0, 1, 1) | measured: UIKit's ease-in, iOS 26.2 simulator, 2026-10-23 | + | `timing.curves.entrance` | cubic-bezier(0, 0, 0.58, 1) | measured: UIKit's ease-out, iOS 26.2 simulator, 2026-10-23 | + | `timing.curves.move` | cubic-bezier(0.42, 0, 0.58, 1) | measured: UIKit's ease-in-out, iOS 26.2 simulator, 2026-10-23 | ### Motion diff --git a/internal/interaction-harness/README.md b/internal/interaction-harness/README.md index 31c5149c2d407..27e054e9beb71 100644 --- a/internal/interaction-harness/README.md +++ b/internal/interaction-harness/README.md @@ -1,20 +1,24 @@ # @astryxdesign/interaction-harness Playwright instruments for touch interaction and animation proofs: real touch -input, curve sampling and storyboards. **Internal and unpublished.** Specs live -next to their component as `*.interaction.browser.spec.ts` and run under +input, frame evidence, curve sampling and storyboards. **Internal and +unpublished.** Specs live next to their component as +`*.interaction.browser.spec.ts` and run under [`playwright.interaction.config.ts`](../../playwright.interaction.config.ts) on a phone-sized touch screen (402 × 714 CSS px page, 402 × 874 screen, 3 device pixels per CSS px, touch, mobile viewport rules), in Chromium and WebKit. The harness proves its own instruments in -[`src/harness.interaction.browser.spec.ts`](src/harness.interaction.browser.spec.ts), -on fixture pages written inline. +[`src/harness.interaction.browser.spec.ts`](src/harness.interaction.browser.spec.ts). +The busy-main-thread proof and the rendering-trace counts have a negative +control beside their positive checks (a rAF animation that freezes under a +blocked main thread, a `top` animation that lays out every frame); other +self-tests check an instrument against a known answer. ## Running ```sh -pnpm storybook:build # component specs read stories from the built Storybook +pnpm storybook:build # the specs read stories from the built Storybook pnpm exec playwright install chromium webkit pnpm test:interaction --project chromium # everything, Chromium pnpm test:interaction # one spec, both engines @@ -31,6 +35,12 @@ under a rootful docker daemon the container runs as your user so its node internal/interaction-harness/scripts/webkit-container.mjs [playwright test args…] ``` +WebKit composites through Mesa's software GL there, which compiles each shader +the first time a page draws it (450–700 ms for the first frame of a blurred or +newly animated layer). The script keeps Mesa's shader cache on the host +(`~/.cache/astryx-webkit-shaders/`, or `ASTRYX_WEBKIT_SHADER_CACHE`), so +only the first run on a machine pays for them. + Evidence (storyboards, curve traces) is written to `test-results/interaction-evidence/` (override with `ASTRYX_INTERACTION_EVIDENCE`) and attached to the test report. It is never @@ -38,15 +48,18 @@ committed: a pull request links or attaches it. ## Instruments -| Module | What it gives a spec | -| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `scripts.ts` | Touch scripts as data: `tap`, `drag` (hold, rest), `flick` (exact final-move speed, velocity profiles), `holdThenDrag`, `pinch`, `rotate`, `path`, `merge` (a second finger mid-gesture), `releaseVelocity`, `toTouchEvents`. | -| `touch.ts` | `touchDriver(page).play(script)`: the script as real engine input. `installInputLog` / `readInputLog`: what the page received. | -| `curves.ts` | `holdAnimations`, `seekAnimations`, `sampleCurve` (presented transform / opacity at any t), `waitForAnimation`, `watchAnimations`, `waitForRest`, `releaseAnimations`. | -| `math.ts` | Oracles independent of the code under test: `cubicBezier`, `timingFunction`, `springStep` (SwiftUI response / dampingFraction, initial velocity), `projectDecay`, `rubberBand`, `parseTransform`, `compareCurve`, `timeline`. | -| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event): PNG contact sheet + JSON trace. | -| `output.ts` | `evidenceDir`, `writeEvidence`: where evidence goes and how it is attached to the report. | -| `story.ts` | `phone`, `withStorybook`, `openStory`, `frames`, and the timer gate (`installTimerGate`, `freezeTimers`, `thawTimers`). | +| Module | What it gives a spec | +| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `scripts.ts` | Touch scripts as data: `tap`, `drag` (hold, rest), `flick` (exact final-move speed, velocity profiles), `holdThenDrag`, `pinch`, `rotate`, `path`, `merge` (a second finger mid-gesture), `releaseVelocity`, `toTouchEvents`. | +| `touch.ts` | `touchDriver(page).play(script)`: the script as real engine input. `installInputLog` / `readInputLog`: what the page received. | +| `frames.ts` | `startScreencast` (timestamped composited frames), `markElement` (a marker chip or outline), `trackMarker` (marker box per frame), `sampleRects` (per-rAF rects), `frameStats`. | +| `busy.ts` | `blockMainThread(page, ms)` and `busyWindowProof(tracked, window)`: frames presented, frames that moved, longest stall, a verdict; `presented` adds the compositor's own frames. | +| `trace.ts` | `traceRendering(page, body)`: main-thread counts of UpdateLayoutTree / Layout / PrePaint / Paint / Layerize / UpdateLayer / ScrollLayer / HitTest, `countBetween`, long tasks; `installLongTaskLog` / `readLongTasks`; `startCompositorTrace`: when the compositor presents. | +| `curves.ts` | `holdAnimations`, `seekAnimations`, `sampleCurve` (presented transform / opacity at any t), `waitForAnimation`, `watchAnimations`, `waitForRest`, `releaseAnimations`. | +| `math.ts` | Oracles independent of the code under test: `cubicBezier`, `timingFunction`, `springStep` (SwiftUI response / dampingFraction, initial velocity), `projectDecay`, `rubberBand`, `parseTransform`, `compareCurve`, `timeline`. | +| `storyboard.ts` | `animationStoryboard` (seek to t = 0, 16.7, 33.3 … ms), `touchStoryboard` (a frame per scripted event), `castStoryboard` (real screencast frames): PNG contact sheet + JSON trace. | +| `output.ts` | `evidenceDir`, `writeEvidence`: where evidence goes and how it is attached to the report. | +| `story.ts` | `phone`, `withStorybook`, `openStory`, `frames`, and the timer gate (`installTimerGate`, `freezeTimers`, `thawTimers`). | ## The proofs @@ -60,6 +73,14 @@ dispatch latency, and the events go through the compositor's gesture detection the limits below. Read positions a frame after a move: Chromium delivers `pointermove` aligned to animation frames. +**Stories, not Storybook.** `openStory` turns Storybook's highlight module off +on every page it opens, and checks that its root element is absent. Left on, +the module observes `#storybook-root` and, on every insertion, reads the +computed style of every element on the page inside the inserting task (117 +reads and two forced style recalculations for one exit copy in a Button story), +so a proof would measure Storybook as well as the component. +`openStory(page, origin, id, args, {storybookHighlight: true})` keeps it on. + **Curve sampling.** Pause every running animation (CSS transitions and animations are Web Animations too), seek to a time, read `getComputedStyle`: the presented value at that exact time, independent of frame timing. Finished @@ -97,6 +118,109 @@ set): a short entrance no longer races the first look on a cold page. When the action may be the page's first contact with the harness, take `watchAnimations(locator)` before it and read `watch.started()` after. +**Busy main thread.** Mark the element, start the screencast, start the +animation, wait until it runs (`waitForAnimation`: an animation cannot start +while the thread is held), block the main thread for 1 s (`spec:AST-094` +FR5's verification row), stop. Always +start the gesture or animation first and block mid-flight, never before: +in headless Chromium a touch that starts while the main thread is busy does +not scroll or start a gesture even with only passive listeners (the input +waits for the main thread). Tracking a finger under a busy main thread is +therefore proven on the iOS simulator, not here. The +screencast is captured off the main thread, so a compositor animation shows a +new position every frame of the block, and a main-thread animation shows none +(nothing changed, so no frame is sent at all: `busyWindowProof` counts the +whole window as one stall). `settledWithin` (2 px) stops judging at the easing +tail: the screencast rounds to whole pixels, so a tail that still moves shows +plateaus within a pixel or two of rest. A judged part shorter than four of the +screencast's intervals fails: the animation under test was not moving during +the block. With +`during`, `blockMainThread` starts the loop from a task queued in the frame +that saw the animation, after that frame's rendering update: WebKit hands a +newly started animation to its compositor there, and a loop inside the rAF +callback would freeze even a compositor animation. Call it before the release +that starts a settle and await it after, so the page is already waiting when +the settle begins. An animation counts once it is no longer pending and has +run at least one frame: a `minProgressMs` under 16 is raised to 16, since one +created in the frame the page looks is not on the compositor yet. The `verdict` judges the +frames the screencast delivered, whatever its rate (a loaded machine's can +deliver one frame in five or six), and tolerates nothing a frozen animation +does: at least one frame in the judged part, no stall longer than three of the +screencast's intervals (two dropped frames), and at least 90% of the frames it +delivered in the judged part presenting a new position (one miss is always +allowed; a frame it did not deliver is a gap, for the stall check, unless +it delivered fewer than four frames there: then the judged part's span counts +every frame, so one late frame is not "1 of 1 moved"). Pass +`maxMedianFrameMs` (20 at 60 Hz) only in a proof that claims display rate. +`verdict.failures` names each miss with its numbers, and `summary` carries the +raw numbers for the report. Frames are judged in presentation order: a loaded +machine can deliver them out of order. `castStoryboard` turns the frames of the +block into a filmstrip. + +A just-started animation can reach the screen after the block began (50–100 ms +later in WebKit on a loaded machine). When the cast saw the element at rest +before the block (wait for `cast.frameCount()` to pass 0 before starting the +animation), judging starts at its first presented change: the judged part still +ends with the block and still needs four intervals, and `onScreenFromMs` and the +summary report when it came. An element already moving is judged from the +block's start. WebKit's casts are 2 px quantized: the frames come at the +viewport's size, but the page in them is captured at half resolution, so +positions move in 2 CSS px steps (Chromium's in 1). With under two steps of +motion per frame, a frame delivered a few ms early shows no step and reads as +not moved. The proof reports `stepPx` and `pxPerFrame` and notes motion under +two steps per frame, which never fails it. Move a proof's element at least two +steps a frame: the self-tests move 560 px in 1,600 ms, about 4.8 px a frame. + +On a loaded machine Chromium's screencast can skip frames the compositor +still presents. `startCompositorTrace(page)`, started with the screencast and +stopped after the block, records when the compositor presented each frame +(`Display::FrameDisplayed`); pass it as `presented`. The record is +browser-wide (its frames carry no page identity), so it throws when another +page is open as it starts or stops, or opened while it ran. A gap between two moved +frames with nothing delivered inside it is then measured between the +compositor's own frames: the frames it presented there count as presented +(`recorderSkipped`, in the summary), and a gap it did not present through is +a stall as before. A gap that opens or closes the judged part, or holds a +delivered frame that did not move, is judged from the screencast, so a +frozen element fails whatever else the compositor draws (a rAF animation's +block presents no compositor frame at all). With the record, the 90% counts the +compositor's frames from the first to the last moved frame; the gaps before and +after are the stall check's. WebKit has no such trace: +`stop()` resolves to `[]` and the screencast is judged alone. + +**No Layout / Paint.** `traceRendering` records a Chromium trace around the +interaction; `countBetween` counts the steady middle of an animation (its +first and last frames may commit). A transform/opacity animation does no +Layout or Paint there; a layout-property animation does one per frame. The +harness's own animation recorder stands down while the trace runs, since its +frame loop would make every frame a main-thread one, restyling each running +animation's target and laying out an anchored one: it reads only the element a +CSS animation or transition event names, once, when that animation starts, and +catches up afterwards. A `waitForRest` awaited inside the traced body stands +down too: it waits on timers for the scheduled ends of the element's curves +(looking `frames` times a frame apart before settling on none, so an +animation the action starts a frame or two later is waited for) and polls no +style until the trace ends (in the anchored fixture's steady middle, +24 Layouts before, 0 now). Its one read of the element's animations, as it is +called, restyles the element (2 Layouts and 2 style recalculations in the +anchored fixture when called 200 ms into its animation), so await it right +after the action that starts the motion, before the window a proof counts. So do `waitForAnimation` and a watch's +`started()`: they take the element's animations from the recorder's log, +judge a started one by its own times, and reach the element through a handle, +since each `locator.evaluate` makes a main-thread frame (there too, 24 +Layouts before, 0 now). That holds for script animations and CSS +transitions. On a CSS `@keyframes` animation (and, unwatched, a cancelled CSS +transition), `waitForAnimation` and `started()` read the animation's timing, +which flushes its style and adds Layouts to the trace: wait on one with +`waitForRest`, or after the trace. Taking a watch (`watchAnimations`) inside +a trace reads the watched element's animations, which restyles it (and lays +out an anchored one), so a proof takes its traced watch before the window it +counts. The trace's `longTasks` are the page's own: only a +task the harness marked as its own read (`waitForAnimation` describing +keyframes, `blockMainThread` without `during`) is in `devtoolsTasks` instead, +and only when no page script ran inside it. A test's own `page.evaluate` stays a +long task, whether it calls page code directly or sets it off. + **Storyboards.** Deterministic: frames come from seeking held animations or from stepping scripted input, never from wall-clock capture. The JSON trace carries each frame's measurements and the spec asserts on it; the sheet is @@ -104,16 +228,18 @@ evidence. A component that finishes an animation from a timer (a transitionend backstop, a cleanup render) would do so behind a held storyboard's back: `installTimerGate` before the animation starts and `freezeTimers` while photographing. The page can still cancel a held callback, a held frame callback -gets the time it actually runs, and the harness's own frame waits are not held. +gets the time it actually runs, and the harness's own waits (its frame waits, +`blockMainThread` with `during`, `sampleRects`) are not held. An animation that repeats forever does not set a storyboard's default length; with nothing else to film, pass `times`. A touch filmstrip keeps the scripted event `timeStamp`s (Chromium), but a handler that reads `performance.now()` or `Date.now()` sees one event per photograph, about 100 ms apart, so a clock-timed release flings slower when filmed than when played: a storyboard -cannot show such a gesture; record it in real time instead. Chromium can return -an empty copy of the page's surface right after it changes ("Unable to capture -screenshot"); a storyboard takes that photograph again two frames later, up to -three attempts in all, and lists each retake under `recaptures` in the trace. +cannot show such a gesture; record it in real time instead, with +`startScreencast` and `castStoryboard`. Chromium can return an empty copy of +the page's surface right after it changes ("Unable to capture screenshot"); a +storyboard takes that photograph again two frames later, up to three attempts +in all, and lists each retake under `recaptures` in the trace. ## Engine limits (measured with Playwright 1.61) @@ -126,11 +252,15 @@ three attempts in all, and lists each retake under `recaptures` in the trace. | Lifting one finger while others stay down | yes | pointer events right, but that `touchend` lists no remaining `touches`: skip with `PARTIAL_LIFT_SKIP` | | Click synthesised from a touch | yes | only from `driver.tap` (WebKit's tap gesture), not from a raw touch sequence | | Curve sampling, storyboards, input log | yes | yes | +| Screencast frame evidence (busy main thread) | yes | yes (WebKit's own screencast; frames keep coming while the main thread is blocked) | +| Rendering trace, long tasks | yes | no (Playwright exposes neither) | +| Compositor presentation record | yes | no: `stop()` resolves to `[]`; the busy verdict judges the screencast alone | What WebKit cannot show here — native scroll hand-off, flings, a finger -lifting while another stays down, and anything iOS-only — needs Mobile Safari -itself, on a device or the iOS simulator. Skip such a proof on the driver's -flag, citing the reason: `test.skip(!driver.partialLifts, PARTIAL_LIFT_SKIP)`. +lifting while another stays down, finger tracking under a busy main thread, and +anything iOS-only — needs Mobile Safari itself, on a device or the iOS +simulator. Skip such a proof on the driver's flag, citing the reason: +`test.skip(!driver.partialLifts, PARTIAL_LIFT_SKIP)`. The WebKit input command is reached through Playwright's in-process server object, so it needs a locally launched browser (the default), not `connect()`. diff --git a/internal/interaction-harness/package.json b/internal/interaction-harness/package.json index 40cccb26ca206..942eccb9ed8d5 100644 --- a/internal/interaction-harness/package.json +++ b/internal/interaction-harness/package.json @@ -3,7 +3,7 @@ "version": "0.0.0", "private": true, "type": "module", - "description": "Internal: Playwright helpers for touch interaction and animation proofs (real touch input, curve sampling, storyboards). Never published.", + "description": "Internal: Playwright helpers for touch interaction and animation proofs (real touch input, frame evidence, curve sampling, storyboards). Never published.", "main": "./src/index.ts", "exports": { ".": "./src/index.ts", diff --git a/internal/interaction-harness/scripts/webkit-container.mjs b/internal/interaction-harness/scripts/webkit-container.mjs index cb20fd44aa725..a077eae415ed5 100644 --- a/internal/interaction-harness/scripts/webkit-container.mjs +++ b/internal/interaction-harness/scripts/webkit-container.mjs @@ -16,10 +16,19 @@ * docker daemon the container runs as the invoking user (rootless podman and * rootless docker already map it), so the `test-results/` it writes into the * mounted checkout stay the host user's and the next host run can clear them. + * + * WebKit composites through Mesa's software GL in the image, which compiles + * each shader variant the first time a page draws it: 450–700 ms for the + * first frame of a blurred or newly animated layer, long enough for a short + * animation to end before it is ever presented. Mesa caches what it compiles, + * and the cache is kept on the host (`~/.cache/astryx-webkit-shaders/`, + * or `ASTRYX_WEBKIT_SHADER_CACHE`), so a run starts with every shader an + * earlier run compiled instead of paying for them in its first tests. */ import {spawnSync} from 'node:child_process'; import fs from 'node:fs'; import {createRequire} from 'node:module'; +import os from 'node:os'; import path from 'node:path'; import {fileURLToPath} from 'node:url'; @@ -42,6 +51,21 @@ if (engine == null) { process.exit(1); } const args = process.argv.slice(2); +// Where the container sees the host's shader cache (see the header). +const containerShaderCache = '/astryx-shader-cache'; +// Resolved to an absolute path: a bare relative value would be mounted as a +// named volume rather than a host directory. An empty value means the default. +const shaderCache = process.env.ASTRYX_WEBKIT_SHADER_CACHE + ? path.resolve(process.env.ASTRYX_WEBKIT_SHADER_CACHE) + : path.join( + os.homedir(), + '.cache', + 'astryx-webkit-shaders', + image.replace(/[^\w.-]+/g, '_'), + ); +fs.mkdirSync(shaderCache, {recursive: true}); +// The harness self-test reads this marker inside the container. +fs.writeFileSync(path.join(shaderCache, 'ASTRYX-SHADER-CACHE'), `${image}\n`); const env = [ 'ASTRYX_STORYBOOK_DIR', 'ASTRYX_INTERACTION_EVIDENCE', @@ -79,6 +103,10 @@ const command = [ '--ipc=host', '-v', `${root}:${root}`, + '-v', + `${shaderCache}:${containerShaderCache}`, + '-e', + `MESA_SHADER_CACHE_DIR=${containerShaderCache}`, '-w', root, ...env, diff --git a/internal/interaction-harness/src/busy.skip.fixture.ts b/internal/interaction-harness/src/busy.skip.fixture.ts new file mode 100644 index 0000000000000..409886f12e6f8 --- /dev/null +++ b/internal/interaction-harness/src/busy.skip.fixture.ts @@ -0,0 +1,125 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file busy.skip.fixture.ts + * @input One recorded busy-main-thread run in headless Chromium under + * parallel load: a compositor coast through a 566 ms block + * @output `frames`: the screencast frames the marker was tracked in, as + * [ms after the block started, y in CSS px, or null where the marker was not + * found]; `presented`: when the compositor presented each frame + * (`Display::FrameDisplayed`), ms after the block started; `block`: the + * block's span on the same clock. The screencast skipped frames at about + * +366 and +466 ms while the compositor kept presenting every 16.7 ms. + * @position Fixture for busy.test.ts + */ + +export const block = {from: 0, to: 566.1}; + +export const frames: ReadonlyArray = [ + [-1145.8, null], + [-1114, null], + [-1063.9, null], + [-1031.1, null], + [-996.6, null], + [-964.3, null], + [-930.7, null], + [-896.7, null], + [-864, null], + [-830.6, null], + [-797.8, null], + [-762.7, null], + [-731.3, null], + [-686.5, null], + [-680.8, null], + [-651, null], + [-634.1, null], + [-620.7, null], + [-615.8, null], + [-514.4, null], + [-495, null], + [-472.4, null], + [-465.3, null], + [-448.5, null], + [-430.3, null], + [-414.8, null], + [-398.3, null], + [-379.5, null], + [-364.2, null], + [-348.3, null], + [-330.6, null], + [-315, null], + [-297.6, null], + [-281.9, null], + [-265.1, null], + [-247.6, null], + [-230.8, null], + [-215.3, null], + [-198.6, null], + [-181, null], + [-165.3, null], + [-147.7, null], + [-132, null], + [-115.4, null], + [-98.5, null], + [-81.2, 504], + [-65.2, 500], + [-48.5, 496], + [-30.9, 491], + [-15.2, 487], + [1.3, 484], + [32.8, 480], + [49.8, 476], + [66.3, 473], + [82.8, 469], + [99.7, 466], + [116.1, 463], + [132.9, 460], + [149.5, 457], + [166.3, 454], + [182.9, 451], + [199.4, 448], + [216.2, 446], + [232.8, 443], + [249.4, 441], + [266.1, 438], + [282.7, 436], + [299.5, 434], + [316, 432], + [332.7, 430], + [349.5, 428], + [366.1, 426], + [433.1, 418], + [449.7, 417], + [466.2, 415], + [516.2, 411], + [532.8, 409], + [549.5, 408], + [566.2, 407], + [650.6, 401], + [666.2, 400], + [682.9, 399], + [699.6, 398], + [716.3, 397], + [816.1, 391], + [832.8, 391], + [849.5, 390], + [866.2, 389], + [882.9, 388], + [899.5, 388], + [916.1, 387], + [1033.8, 383], + [1050.1, 383], + [1066.7, 382], + [1083.3, 382], + [1099.5, 381], + [1117.3, 381], + [1134.2, 380], + [1149.5, 380], +]; + +export const presented: readonly number[] = [ + -98.7, -81.3, -65.3, -48.7, -31.1, -15.3, 1.3, 32.5, 49.6, 66.1, 82.7, 99.6, + 115.9, 132.6, 149.3, 166.1, 182.6, 199.3, 216, 232.6, 249.3, 266, 282.6, + 299.2, 315.8, 332.6, 349.3, 366, 381.5, 398.3, 415.1, 432.9, 449.6, 466.1, + 481.5, 498.3, 516, 532.6, 549.4, 566.1, 581.7, 598.4, 615, 631.6, 650.3, 666, +]; diff --git a/internal/interaction-harness/src/busy.test.ts b/internal/interaction-harness/src/busy.test.ts new file mode 100644 index 0000000000000..44a2446ecca18 --- /dev/null +++ b/internal/interaction-harness/src/busy.test.ts @@ -0,0 +1,536 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file busy.test.ts + * @input Synthetic tracked frames (no browser) + * @output The busy-main-thread verdict's limits: dropped frames a loaded + * machine produces pass, a frozen animation fails every criterion, a window + * with nothing moving to judge fails, and out-of-order frames are judged in + * presentation order. + * @position Unit test of ./busy and ./frames' ordering; the browser proof + * with a real compositor animation and a rAF control is in + * harness.interaction.browser.spec.ts. + */ + +import {describe, expect, it} from 'vitest'; +import {busyWindowProof} from './busy'; +import * as skip from './busy.skip.fixture'; +import {inPresentationOrder, type TrackedFrame} from './frames'; + +const FRAME = 1000 / 60; +const BLOCK = {from: 1000, to: 1700}; + +/** A marker moving 3 px per frame from t = 800 to 1900, minus `dropped` frame indices. */ +function moving(dropped: readonly number[] = []): TrackedFrame[] { + const frames: TrackedFrame[] = []; + for (let i = 0; 800 + i * FRAME <= 1900; i += 1) { + if (dropped.includes(i)) { + continue; + } + frames.push({ + t: 800 + i * FRAME, + box: {x: 0, y: 3 * i, width: 10, height: 10}, + } as TrackedFrame); + } + return frames; +} + +/** Index of the first frame at or after `t`. */ +const at = (t: number) => Math.ceil((t - 800) / FRAME); + +/** `moving()` with each frame in `repeats` presenting the previous frame's position. */ +function repeating(repeats: readonly number[]): TrackedFrame[] { + return moving().map((f, i) => + repeats.includes(i) + ? ({...f, box: {...f.box!, y: 3 * (i - 1)}} as TrackedFrame) + : f, + ); +} + +/** A marker moving 3 px per frame from t = 800 to 1900, frames `intervals` apart in turn. */ +function cadenced(intervals: readonly number[]): TrackedFrame[] { + const frames: TrackedFrame[] = []; + for (let i = 0, t = 800; t <= 1900; i += 1) { + frames.push({ + t, + box: {x: 0, y: 3 * i, width: 10, height: 10}, + } as TrackedFrame); + t += intervals[i % intervals.length]; + } + return frames; +} + +describe('busyWindowProof verdict', () => { + it('passes a compositor animation with every frame', () => { + const proof = busyWindowProof(moving(), BLOCK); + expect(proof.verdict.failures).toEqual([]); + expect(proof.longestStallFrames).toBe(1); + }); + + it('passes one and two consecutive dropped frames (3 intervals)', () => { + const k = at(1300); + expect(busyWindowProof(moving([k]), BLOCK).verdict.failures).toEqual([]); + const two = busyWindowProof(moving([k, k + 1]), BLOCK); + expect(two.longestStallFrames).toBe(3); + expect(two.verdict.failures).toEqual([]); + }); + + it('fails three consecutive dropped frames', () => { + const k = at(1300); + const proof = busyWindowProof(moving([k, k + 1, k + 2]), BLOCK); + expect(proof.longestStallFrames).toBe(4); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('fails when more than 10% of the delivered frames do not move', () => { + const k = at(1050); + // 39 frames delivered: up to 3 may show no new position (10% = 3.9). + const scattered = [0, 12, 24].map(d => k + d); + expect( + busyWindowProof(repeating(scattered), BLOCK).verdict.failures, + ).toEqual([]); + const many = [0, 6, 12, 18, 24].map(d => k + d); + expect( + busyWindowProof(repeating(many), BLOCK).verdict.failures.join('\n'), + ).toMatch(/expected frames moved/); + }); + + it('judges a frame the screencast did not deliver as a gap for the stall check, not a miss', () => { + // The same five frames, not delivered: two-interval gaps, and every + // delivered frame moved. + const k = at(1050); + const many = [0, 6, 12, 18, 24].map(d => k + d); + const proof = busyWindowProof(moving(many), BLOCK); + expect(proof.verdict.failures).toEqual([]); + expect(proof.deliveredFrames).toBe(proof.moved); + }); + + it('judges an irregular cast on the frames it delivered', () => { + // A cast cycling 11.7, 11.7 and 17 ms: its median interval would expect + // 56 frames where it delivered 49, every one moved. + const proof = busyWindowProof(cadenced([11.7, 11.7, 17]), BLOCK); + expect(proof.verdict.failures).toEqual([]); + expect(proof.deliveredFrames).toBe(49); + expect(proof.moved).toBe(49); + // Still a stall where it delivers nothing for four frames, and still a + // miss where five delivered frames show no new position. + const gap = cadenced([11.7, 11.7, 17]).filter( + f => f.t < 1300 || f.t > 1360, + ); + expect(busyWindowProof(gap, BLOCK).verdict.failures.join('\n')).toMatch( + /longest stall/, + ); + const unmoved = cadenced([11.7, 11.7, 17]).map((f, i, all) => + [20, 26, 32, 38, 44].includes(i) + ? ({...f, box: {...all[i - 1].box!}} as TrackedFrame) + : f, + ); + expect(busyWindowProof(unmoved, BLOCK).verdict.failures.join('\n')).toMatch( + /expected frames moved/, + ); + }); + + it('judges a window that delivered fewer than four frames against its span', () => { + // A rAF animation frozen by the block, in WebKit: one frame from before + // the block arrives 45 ms into it, then nothing until after it. + const late = moving().filter(f => f.t <= 1000 || f.t >= 1750); + late.push({ + t: 1045, + box: {x: 0, y: 3 * at(1001), width: 10, height: 10}, + } as TrackedFrame); + const proof = busyWindowProof(inPresentationOrder(late), BLOCK); + expect(proof.verdict.failures.join('\n')).toMatch(/expected frames moved/); + }); + + it('lets a short window miss a frame, not three', () => { + // 150 ms judged: nine frames delivered, one miss allowed (10% would be 0.9). + const short = {from: 1000, to: 1190}; + const k = at(1040); + expect(busyWindowProof(repeating([k]), short).verdict.failures).toEqual([]); + expect( + busyWindowProof( + repeating([k, k + 3, k + 6]), + short, + ).verdict.failures.join('\n'), + ).toMatch(/expected frames moved/); + }); + + it('fails a frozen animation on every criterion', () => { + // Frames before and after the block only: nothing changed during it. + const frozen = moving().filter(f => f.t < BLOCK.from || f.t > BLOCK.to); + const proof = busyWindowProof(frozen, BLOCK); + expect(proof.changed).toBe(0); + expect(proof.verdict.pass).toBe(false); + expect(proof.verdict.failures.join('\n')).toMatch(/no frame presented/); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + expect(proof.verdict.failures.join('\n')).toMatch(/expected frames moved/); + }); + + it('fails a window in which the animation under test was not moving', () => { + // Settled before the block: the judgment ends where it starts. + const settled = moving().map(f => + f.t < 950 + ? f + : ({...f, box: {...f.box!, y: 3 * at(950)}} as TrackedFrame), + ); + const before = busyWindowProof(settled, BLOCK, {settledWithin: 2}); + expect(before.verdict.pass).toBe(false); + expect(before.verdict.failures.join('\n')).toMatch( + /judged = -?\d+ frames < 4: the animation under test was not moving/, + ); + // Settled 40 ms into the block: two frames, too few to judge. + const early = moving().map(f => + f.t < 1040 + ? f + : ({...f, box: {...f.box!, y: 3 * at(1040)}} as TrackedFrame), + ); + expect( + busyWindowProof(early, BLOCK, {settledWithin: 2}).verdict.failures.join( + '\n', + ), + ).toMatch(/frames < 4: the animation under test was not moving/); + }); + + it('judges the frames a slower screencast delivers by default, and display rate only when asked', () => { + // One frame in six, as a loaded machine's screencast delivers: every + // delivered frame moved, so the default verdict passes. + const sixth = moving().filter((_, i) => i % 6 === 0); + expect(busyWindowProof(sixth, BLOCK).verdict.failures).toEqual([]); + // Half rate fails only a proof that claims display rate. + const odd = moving().filter((_, i) => i % 2 === 1); + expect(busyWindowProof(odd, BLOCK).verdict.pass).toBe(true); + expect( + busyWindowProof(odd, BLOCK, {maxMedianFrameMs: 20}).verdict.failures.join( + '\n', + ), + ).toMatch(/median frame interval/); + }); +}); + +describe('busyWindowProof settledWithin', () => { + it('ends the judgment at a rounded easing tail, which would read as a stall', () => { + // An ease-out to y = 24 as the screencast reports it: whole pixels, so the + // tail repeats values while it still moves. + const ys = [ + 120, 100, 82, 67, 55, 46, 39, 34, 30, 28, 27, 26, 26, 26, 25, 25, 24, 24, + 24, 24, + ]; + const frames = ys.map( + (y, i) => + ({ + t: 1000 + i * FRAME, + box: {x: 0, y, width: 10, height: 10}, + }) as TrackedFrame, + ); + const window = {from: 1000, to: 1000 + ys.length * FRAME}; + const loose = busyWindowProof(frames, window, { + settledWithin: 2, + marginMs: 0, + }); + expect(loose.verdict.failures).toEqual([]); + const strict = busyWindowProof(frames, window, { + settledWithin: 1, + marginMs: 0, + }); + expect(strict.longestStallFrames).toBeGreaterThanOrEqual(3); + }); + + it('takes the end value from settledAt over a stale last frame', () => { + // The tail reaches y = 24, then the last frame shows a stale 100 (WebKit + // presents the last committed position once an accelerated animation + // ends while the main thread is still blocked). + const ys = [ + 120, 100, 82, 67, 55, 46, 39, 34, 30, 28, 27, 26, 25, 24, 24, 24, 100, + ]; + const frames = ys.map( + (y, i) => + ({ + t: 1000 + i * FRAME, + box: {x: 0, y, width: 10, height: 10}, + }) as TrackedFrame, + ); + const window = {from: 1000, to: 1000 + ys.length * FRAME}; + const stale = busyWindowProof(frames, window, { + settledWithin: 2, + marginMs: 0, + }); + expect(stale.verdict.pass).toBe(false); + const fromPage = busyWindowProof(frames, window, { + settledWithin: 2, + settledAt: 24, + marginMs: 0, + }); + expect(fromPage.verdict.failures).toEqual([]); + expect(fromPage.judged.to).toBe(frames[11].t); + }); +}); + +/** Seen at rest at t = 900, then on screen from `start`, moving 3 px a frame. */ +function startingAt(start: number): TrackedFrame[] { + const frames = [ + {t: 900, box: {x: 0, y: 0, width: 10, height: 10}} as TrackedFrame, + ]; + for (let i = 1; start + (i - 1) * FRAME <= 1900; i += 1) { + frames.push({ + t: start + (i - 1) * FRAME, + box: {x: 0, y: 3 * i, width: 10, height: 10}, + } as TrackedFrame); + } + return frames; +} + +describe('busyWindowProof on-screen start', () => { + it('judges a just-started animation from its first presented change when it was seen at rest before the block, and reports when', () => { + // On screen 87 ms into the block: four intervals after the judged part's start. + const proof = busyWindowProof(startingAt(1087), BLOCK); + expect(proof.verdict.failures).toEqual([]); + expect(proof.onScreenFromMs).toBeCloseTo(87, 5); + expect(proof.judged.from).toBeCloseTo(1087, 5); + expect(proof.summary).toMatch( + /on screen from \+87 ms \(nothing moved before the block\)/, + ); + }); + + it('still fails an animation that was already moving and stalls at the block start', () => { + const k = at(1020); + const proof = busyWindowProof(moving([k, k + 1, k + 2, k + 3]), BLOCK); + expect(proof.onScreenFromMs).toBeNull(); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('still fails a stall at the start when no frame before the block shows it at rest', () => { + const unseen = startingAt(1087).filter(f => f.t >= BLOCK.from); + const proof = busyWindowProof(unseen, BLOCK); + expect(proof.onScreenFromMs).toBeNull(); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('still fails a start the block itself delays until it ends', () => { + const proof = busyWindowProof(startingAt(1720), BLOCK); + expect(proof.onScreenFromMs).toBeNull(); + expect(proof.verdict.failures.join('\n')).toMatch(/no frame presented/); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('still fails a start that leaves fewer than minJudgedFrames inside the block', () => { + const proof = busyWindowProof(startingAt(1640), BLOCK); + expect(proof.onScreenFromMs).toBeCloseTo(640, 5); + expect(proof.verdict.failures.join('\n')).toMatch( + /frames < 4: the animation under test was not moving/, + ); + }); +}); + +describe('busyWindowProof position step', () => { + it("reports the cast's position step and motion per frame, and notes motion under two steps without failing", () => { + // 2.4 px a frame on a 2 px grid, as WebKit's casts report positions. + const grid = moving().map( + (f, i) => + ({ + ...f, + box: {...f.box!, y: 2 * Math.round((2.4 * i) / 2)}, + }) as TrackedFrame, + ); + const coarse = busyWindowProof(grid, BLOCK); + expect(coarse.stepPx).toBe(2); + expect(coarse.pxPerFrame).toBeCloseTo(2.4, 1); + expect(coarse.notes.join('\n')).toMatch( + /under two of the cast's 2 px position steps/, + ); + expect(coarse.summary).toMatch(/2\.4 px\/frame in 2 px steps; note: /); + expect(coarse.verdict.failures).toEqual([]); + // 3 px a frame in whole pixels: resolvable, nothing to note. + const fine = busyWindowProof(moving(), BLOCK); + expect(fine.stepPx).toBe(1); + expect(fine.pxPerFrame).toBeCloseTo(3, 1); + expect(fine.notes).toEqual([]); + }); +}); + +/** The compositor presenting every frame from `from` to `to`, minus `gaps` ([from, to] spans). */ +function drawn( + gaps: ReadonlyArray = [], + from = 700, + to = 2000, +): number[] { + const draws: number[] = []; + for (let t = from; t <= to; t += FRAME) { + if (!gaps.some(([a, b]) => t > a && t < b)) { + draws.push(t); + } + } + return draws; +} + +const skipFrames = (): TrackedFrame[] => + skip.frames.map( + ([t, y]) => + ({ + t, + box: y == null ? null : {x: 0, y, width: 10, height: 10}, + }) as TrackedFrame, + ); + +describe('busyWindowProof with the compositor record', () => { + it('with a compositor record, judges a sparse cast from its first to its last moved frame', () => { + // One screencast frame in six, as on a loaded machine, with the compositor + // presenting every frame: the frames before the first and after the last + // moved one are the stall check's, not misses. + const sparse = moving().filter((_, i) => i % 6 === 0); + expect( + busyWindowProof(sparse, BLOCK, {presented: drawn()}).verdict.failures, + ).toEqual([]); + }); + + it('with a compositor record and an on-screen start, counts the frames it credits as skipped in what it judges', () => { + // On screen 87 ms into the block, then four delivered frames that show + // no new position: that fails on a full cast. + const misses = [15, 20, 25, 30]; + const missing = startingAt(1087).map((f, i, all) => + misses.includes(i) + ? ({...f, box: {...all[i - 1].box!}} as TrackedFrame) + : f, + ); + const full = busyWindowProof(missing, BLOCK, {presented: drawn()}); + expect(full.verdict.failures.join('\n')).toMatch(/expected frames moved/); + // The screencast skipping the five frames after the on-screen frame + // changes nothing: the compositor presented them, and they are judged. + const skip = (frames: TrackedFrame[]) => + frames.filter((_, i) => i < 2 || i > 6); + const skipped = busyWindowProof(skip(missing), BLOCK, { + presented: drawn(), + }); + expect(skipped.recorderSkipped).toBe(5); + expect(skipped.verdict.failures.join('\n')).toMatch( + /expected frames moved/, + ); + // And a run with no misses never reads as more than every frame moved. + const clean = busyWindowProof(skip(startingAt(1087)), BLOCK, { + presented: drawn(), + }); + expect(clean.verdict.failures).toEqual([]); + expect(clean.movedFraction).toBeLessThanOrEqual(1); + }); + + it('reads frames the screencast skipped while the compositor presented them as presented: a recorded run that fails without the record', () => { + const frames = skipFrames(); + const alone = busyWindowProof(frames, skip.block, {settledWithin: 1}); + expect(alone.verdict.failures.join('\n')).toMatch( + /longest stall 6\d\.\d ms = 4 intervals > 3/, + ); + // Every frame the screencast delivered moved: only the stall remains, + // which the record resolves. + expect(alone.verdict.failures.join('\n')).not.toMatch( + /expected frames moved/, + ); + const proof = busyWindowProof(frames, skip.block, { + settledWithin: 1, + presented: skip.presented, + }); + expect(proof.verdict.failures).toEqual([]); + expect(proof.recorderSkipped).toBe(5); + expect(proof.longestStallFrames).toBeLessThanOrEqual(2); + expect(proof.summary).toMatch( + /the screencast skipped 5 frames the compositor presented \(at \+3\d\d ms, \+4\d\d ms\)/, + ); + }); + + it('counts a gap the compositor drew through as skipped, and one it did not draw through as a stall', () => { + const k = at(1300); + const gap: [number, number] = [ + 800 + (k - 1) * FRAME, + 800 + (k + 4) * FRAME, + ]; + const frames = moving([k, k + 1, k + 2, k + 3]); + expect(busyWindowProof(frames, BLOCK).verdict.pass).toBe(false); + const skipped = busyWindowProof(frames, BLOCK, {presented: drawn()}); + expect(skipped.verdict.failures).toEqual([]); + expect(skipped.recorderSkipped).toBe(4); + const stalled = busyWindowProof(frames, BLOCK, {presented: drawn([gap])}); + expect(stalled.recorderSkipped).toBe(0); + expect(stalled.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('still fails an element that does not move while the compositor presents for something else', () => { + const frozen = moving().map(f => + f.t >= BLOCK.from && f.t <= BLOCK.to + ? ({...f, box: {...f.box!, y: 3 * at(BLOCK.from)}} as TrackedFrame) + : f, + ); + const proof = busyWindowProof(frozen, BLOCK, {presented: drawn()}); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + expect(proof.verdict.failures.join('\n')).toMatch(/expected frames moved/); + }); + + it('still fails a frozen animation, whether or not the compositor presents during the block', () => { + const frozen = moving().filter(f => f.t < BLOCK.from || f.t > BLOCK.to); + for (const presented of [drawn([[BLOCK.from, BLOCK.to]]), drawn()]) { + const proof = busyWindowProof(frozen, BLOCK, {presented}); + expect(proof.verdict.failures.join('\n')).toMatch(/no frame presented/); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + } + }); + + it('keeps a gap a stall when the screencast delivered a frame inside it that did not move', () => { + const k = at(1300); + const frames = moving([k, k + 1, k + 3]).map(f => + f.t > 800 + (k + 1.5) * FRAME && f.t < 800 + (k + 2.5) * FRAME + ? ({...f, box: {...f.box!, y: 3 * (k - 1)}} as TrackedFrame) + : f, + ); + const proof = busyWindowProof(frames, BLOCK, {presented: drawn()}); + expect(proof.recorderSkipped).toBe(0); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('still counts single dropped frames the compositor did not present as missed', () => { + const k = at(1050); + const drops = [0, 6, 12, 18, 24, 30].map(d => k + d); + const frames = moving(drops); + const gaps = drops.map( + i => + [800 + (i - 0.5) * FRAME, 800 + (i + 0.5) * FRAME] as [number, number], + ); + expect( + busyWindowProof(frames, BLOCK, { + presented: drawn(gaps), + }).verdict.failures.join('\n'), + ).toMatch(/expected frames moved/); + expect( + busyWindowProof(frames, BLOCK, {presented: drawn()}).verdict.failures, + ).toEqual([]); + }); + + it('judges a gap at the edge of the judged part from the screencast alone', () => { + // Nothing delivered for the block's first 90 ms: a stall at its start, + // whatever the compositor drew, since no moved frame opens the gap. + const frames = moving().filter( + f => f.t < BLOCK.from || f.t > BLOCK.from + 90, + ); + const proof = busyWindowProof(frames, BLOCK, {presented: drawn()}); + expect(proof.recorderSkipped).toBe(0); + expect(proof.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + it('treats an empty record as no record, and says so', () => { + const frames = moving([at(1300), at(1300) + 1, at(1300) + 2, at(1300) + 3]); + const proof = busyWindowProof(frames, BLOCK, {presented: []}); + expect(proof.verdict.failures).toEqual( + busyWindowProof(frames, BLOCK).verdict.failures, + ); + expect(proof.summary).toMatch(/no compositor frames recorded/); + }); +}); + +describe('inPresentationOrder', () => { + it('sorts frames delivered out of order, so motion stays monotonic', () => { + const frames = moving(); + const k = at(1040); + const shuffled = [...frames]; + [shuffled[k], shuffled[k + 1]] = [shuffled[k + 1], shuffled[k]]; + expect(busyWindowProof(shuffled, BLOCK).monotonic).toBe(false); + const proof = busyWindowProof(inPresentationOrder(shuffled), BLOCK); + expect(proof.monotonic).toBe(true); + expect(proof.verdict.failures).toEqual([]); + }); +}); diff --git a/internal/interaction-harness/src/busy.ts b/internal/interaction-harness/src/busy.ts new file mode 100644 index 0000000000000..55fd9d670d52a --- /dev/null +++ b/internal/interaction-harness/src/busy.ts @@ -0,0 +1,579 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file busy.ts + * @input Uses a Playwright `Page` + * @output `blockMainThread` — a synchronous busy loop on the page's main thread + * that reports exactly when it ran — and `busyWindowProof`, which reads frame + * evidence for that window. + * @position The busy-main-thread proof: an animation that runs on the + * compositor keeps advancing at display rate while JavaScript holds the main + * thread; one driven from the main thread (rAF, style writes, layout + * properties) freezes for the whole block. Pair it with ./frames + * (`startScreencast` + `trackMarker`), which observes off the main thread. + * + * SYNC: Keep the README's busy-main-thread section equal to this contract. + */ + +import type {Locator, Page} from '@playwright/test'; +import {frameStats, type FrameStats, type TrackedFrame} from './frames'; + +export interface BusyWindow { + /** Epoch ms when the loop started and ended, on the page's own clock. */ + readonly from: number; + readonly to: number; +} + +/** + * Holds the page's main thread for `ms` (a synchronous loop: no task, frame, + * rAF, event or microtask runs meanwhile). Resolves with the window it held. + * + * Block MID-FLIGHT only: start the gesture or animation first. A touch that + * starts while the main thread is busy does not scroll or begin a gesture in + * headless Chromium, even with only passive listeners, and an animation that + * has not started cannot start (see `waitForAnimation`). Finger tracking + * under a busy main thread is a simulator proof, not a Playwright one. + * + * With `during`, the page itself waits until an animation on that element (or + * inside it) has run `minProgressMs`, then blocks from a task it queues in + * that frame. A round trip from the test would start the block later the + * busier the machine is, leaving less of a short animation to judge. The task + * runs after the frame's rendering update, which is where WebKit hands a newly + * started animation to its compositor: a loop inside the rAF callback itself + * holds that hand-off, and even a compositor animation freezes. To catch a + * settle near its start, call it before the release that starts the settle and + * await it after: the page is already waiting when the animation begins. Any + * animation inside `during` counts, so pick an element that runs nothing + * else meanwhile. + * + * An animation counts once it is no longer pending and has run at least one + * frame (`minProgressMs` below 16 ms is raised to 16). One created in the + * frame the page looks is still pending, not yet on the compositor + * (Chromium), and in WebKit its start time resolves before its layer is + * flushed: a block that started on it froze it. + */ +/** One display frame, ms: the least progress `blockMainThread` waits for. */ +const ONE_FRAME_MS = 16; + +export async function blockMainThread( + page: Page, + ms: number, + { + during, + minProgressMs = 16, + timeout = 5_000, + }: {during?: Locator; minProgressMs?: number; timeout?: number} = {}, +): Promise { + const minProgress = Math.max(minProgressMs, ONE_FRAME_MS); + if (during == null) { + return page.evaluate(duration => { + // The harness's own work: a trace sets it apart (trace.ts HARNESS_READ). + console.timeStamp('harness-read'); + const from = performance.timeOrigin + performance.now(); + const end = performance.now() + duration; + let spins = 0; + while (performance.now() < end) { + spins += 1; + } + return {from, to: performance.timeOrigin + performance.now(), spins}; + }, ms); + } + return during.evaluate( + (element, {duration, minProgress, wait}) => + new Promise((resolve, reject) => { + // The harness's own waits run on the real timers: a frozen timer gate + // (story.ts) holds only the page's. + const gate = ( + window as unknown as { + __harnessTimerGate?: { + frame?: typeof requestAnimationFrame; + timeout?: typeof setTimeout; + }; + } + ).__harnessTimerGate; + const frame = gate?.frame ?? window.requestAnimationFrame.bind(window); + const later = gate?.timeout ?? window.setTimeout.bind(window); + const deadline = performance.now() + wait; + const tick = () => { + const running = element + .getAnimations({subtree: true}) + .some( + a => + a.playState === 'running' && + !a.pending && + Number(a.currentTime ?? 0) >= minProgress, + ); + if (running) { + // The same loop as above, from a task queued in the frame that saw + // the animation: it runs once that frame's rendering update has + // handed the animation to the compositor. + later(() => { + const from = performance.timeOrigin + performance.now(); + const end = performance.now() + duration; + while (performance.now() < end) { + // Busy: nothing else runs. + } + resolve({from, to: performance.timeOrigin + performance.now()}); + }, 0); + } else if (performance.now() > deadline) { + reject(new Error('blockMainThread: no animation started')); + } else { + frame(tick); + } + }; + frame(tick); + }), + {duration: ms, minProgress, wait: timeout}, + ); +} + +export interface BusyVerdict { + readonly pass: boolean; + /** Each criterion the window missed, with its measured value and limit. */ + readonly failures: readonly string[]; +} + +export interface BusyProof extends FrameStats { + readonly window: BusyWindow; + /** + * The part of the window judged: inside the margins, up to `settledWithin`, + * and from `onScreenFromMs` when that is set. + */ + readonly judged: {readonly from: number; readonly to: number}; + /** + * When the tracked value was seen at rest before the judged part (a frame + * there, no change in the 250 ms before it: a just-started animation not on + * screen yet), ms after the window's start of its first presented change + * inside the block, where judging starts; null otherwise. + */ + readonly onScreenFromMs: number | null; + /** + * The cast's position step, CSS px: the largest of 0.5, 1 and 2 that every + * tracked position is a multiple of (WebKit's casts are 2: see the README). + */ + readonly stepPx: number; + /** The tracked value's mean motion per screencast interval in the judged part, px. */ + readonly pxPerFrame: number; + /** Facts worth reading beside the verdict, which never fail it. */ + readonly notes: readonly string[]; + /** + * With a compositor record (`presented`): frames the screencast skipped + * between two moved frames while the compositor presented them, which are + * neither stalls nor misses; 0 without one. + */ + readonly recorderSkipped: number; + /** + * The compositor's frame interval around the window, ms, from `presented`; + * null without a usable record (the screencast alone is judged). + */ + readonly compositorFrameMs: number | null; + /** + * The screencast's frame interval, from the frames around the window as well + * as inside it (a short window has few intervals; a frozen one has none). + */ + readonly cadenceMs: number; + /** + * Whole frame intervals in the judged part at the measured cadence (the + * compositor's, when recorded): the `minJudgedFrames` floor reads it. + */ + readonly expectedFrames: number; + /** Frames the screencast delivered in the judged part, with the marker found. */ + readonly deliveredFrames: number; + /** + * Longest time inside the window with no presented change, the window's own + * edges included. A frozen animation presents no new frame at all (nothing + * on screen changed, so the compositor sends none): its stall is the whole + * window. + */ + readonly longestStallMs: number; + /** Where the longest stall starts, ms after the window's start. */ + readonly longestStallAtMs: number; + /** + * `longestStallMs` in screencast intervals, rounded: 1 is no frame missed, + * 2 is one dropped frame. Rounding absorbs timestamp jitter. + */ + readonly longestStallFrames: number; + /** + * Frames in the window that presented a new position (the first compared + * with the last frame before the window). + */ + readonly moved: number; + /** + * `moved` (and `recorderSkipped`) over the frames judged: the frames the + * screencast delivered in the judged part, or with a compositor record the + * compositor's frames from the first to the last moved frame; + * `expectedFrames` when nothing moved, or fewer than `minJudgedFrames` + * frames were delivered. + */ + readonly movedFraction: number; + /** The judgment against the limits (see `busyWindowProof`). */ + readonly verdict: BusyVerdict; + /** One line with the raw numbers, for an annotation or a failure message. */ + readonly summary: string; +} + +/** + * Frame evidence for a busy window: how many frames the screencast presented + * inside it, how many of them moved the tracked axis, and the longest stall. + * The window is trimmed by `marginMs` at both ends so the frame the block + * started on and the one it ended on are not counted. + * + * The verdict judges the frames the screencast delivered, whatever its rate + * (a loaded machine's screencast can deliver one frame in five or six), and + * tolerates nothing a main-thread animation does (it presents no frame for + * the whole block). It passes when: + * - the judged part presented at least one frame, and, only when a proof + * claims display rate and passes `maxMedianFrameMs`, their median interval + * is at most that; + * - no stall spans more than `maxStallFrames` screencast intervals; + * - at least `minMovedFraction` of the frames judged moved the tracked axis + * (a short window may still miss one): the frames the screencast delivered + * in the judged part, so an irregular cast is judged on what it delivered + * and a frame it did not deliver is a gap for the stall check, not a miss + * (unless it delivered fewer than `minJudgedFrames` there: then every frame + * of the judged part's span counts, so one late frame is not 1/1 moved); + * with a compositor record, the compositor's frames from the first to the + * last moved frame (the gaps before and after are the stall check's); + * - the judged part spans at least `minJudgedFrames` frames. Fewer means the + * animation under test was not moving during the block (it settled before + * the block, or the block caught another animation), so there is nothing to + * judge, and that fails rather than passes. + * + * A just-started animation can reach the screen after the block began (its + * hand-off to the compositor lags under load: 50–100 ms in WebKit). When the + * tracked value was seen at rest before the judged part (a frame there, and no + * change in the 250 ms before it), judging starts at its first presented + * change inside the block, which the + * proof reports as `onScreenFromMs`. The judged part still ends with the + * block and the `minJudgedFrames` floor applies to it, so a start the block + * itself delays fails; an animation that was already moving is judged from + * the block's start, and a stall there fails as before. + * + * `stepPx` and `pxPerFrame` say whether a frame's motion is resolvable at + * all: under two position steps per frame, a frame delivered early can show + * no step and read as not moved, which the proof notes (never a failure). + * + * A loaded machine's screencast can skip frames the compositor still presents + * (Chromium). Given the compositor's record (`presented`, from + * `startCompositorTrace`), a gap between two moved frames with no frame + * delivered inside it is measured between the compositor's own frames: it is + * a stall only where the compositor presented nothing, and the frames it did + * present there are neither stalls nor misses (`recorderSkipped`). A gap + * that opens or closes the judged part, or holds a delivered frame that did + * not move, is judged from the screencast as before, so a frozen element + * fails whatever else the compositor draws. + */ +export function busyWindowProof( + tracked: readonly TrackedFrame[], + window: BusyWindow, + { + axis = 'y', + marginMs = 20, + settledWithin, + settledAt, + maxMedianFrameMs = Infinity, + maxStallFrames = 3, + minMovedFraction = 0.9, + minJudgedFrames = 4, + presented, + }: { + axis?: 'x' | 'y' | 'width' | 'height'; + marginMs?: number; + /** + * Opt-in, for a proof that claims display rate: the most the median frame + * interval may be, ms (20 at 60 Hz). Off by default, since it measures the + * screencast's own rate as much as the animation. + */ + maxMedianFrameMs?: number; + /** The most screencast intervals one stall may span (3 = two dropped frames). */ + maxStallFrames?: number; + /** + * The least fraction of the expected frames that must move. One missed + * frame is always allowed, so a window of under ten frames is not judged + * stricter than a long one. + */ + minMovedFraction?: number; + /** The fewest whole frame intervals the judged part of the window may span. */ + minJudgedFrames?: number; + /** + * Stop judging once the value is within this many px of where it ends. + * The screencast reports whole CSS pixels, so an easing tail that still + * moves shows plateaus (26, 26, 26 against a final 24) that would read + * as a stall: 2 is the usual value. + */ + settledWithin?: number; + /** + * Where the value ends, for `settledWithin`: read it from the page once the + * animation has ended (`waitForRest`). Defaults to the last frame's value, + * which can be stale: once an accelerated animation ends while the main + * thread is still blocked, WebKit presents the layer's last committed + * position until the main thread commits the end state. + */ + settledAt?: number; + /** + * When the compositor presented each frame, epoch ms on the page clock + * (`startCompositorTrace`; Chromium). Empty or absent, the screencast is + * judged alone. + */ + presented?: readonly number[]; + } = {}, +): BusyProof { + let end = window.to - marginMs; + if (settledWithin != null) { + const valued = tracked.flatMap(f => + f.box == null ? [] : [{t: f.t, value: f.box[axis]}], + ); + const final = + settledAt ?? (valued.length > 0 ? valued[valued.length - 1].value : null); + const arrived = valued.find( + f => + f.t >= window.from && + final != null && + Math.abs(f.value - final) <= settledWithin, + ); + if (arrived != null) { + end = Math.min(end, arrived.t); + } + } + const inner = {from: window.from + marginMs, to: end}; + const values = tracked + .filter(f => f.t >= inner.from && f.t <= inner.to) + .map(f => ({t: f.t, value: f.box == null ? null : f.box[axis]})); + const before = [...tracked].reverse().find(f => f.t < inner.from); + let previous: number | null = before?.box == null ? null : before.box[axis]; + const inWindow: number[] = []; + for (const v of values) { + if ( + v.value != null && + previous != null && + Math.abs(v.value - previous) > 0.25 + ) { + inWindow.push(v.t); + } + if (v.value != null) { + previous = v.value; + } + } + // Seen at rest before the judged part (a frame there, and no change in the + // 250 ms before it): the first presented change is the animation's on-screen + // start, and judging starts there. With no frame before, nothing is known, + // and the window is judged from its start. + const prior = tracked.filter(f => f.box != null && f.t < inner.from); + let leadFrom = 0; + prior.forEach((f, i) => { + if (f.t < inner.from - 250) { + leadFrom = i; + } + }); + const lead = prior.flatMap(f => (f.box == null ? [] : [f.box[axis]])); + const movedBefore = lead.some( + (value, i) => i > leadFrom && Math.abs(value - lead[i - 1]) > 0.25, + ); + const onScreenFrom = + prior.length > 0 && !movedBefore && inWindow.length > 0 + ? inWindow[0] + : null; + const judged = {from: onScreenFrom ?? inner.from, to: inner.to}; + const changes = inWindow.filter(t => t > judged.from || onScreenFrom == null); + const edges = [judged.from, ...changes, judged.to]; + const around = tracked + .filter(f => f.t >= inner.from - 250 && f.t <= inner.to + 250) + .map(f => f.t); + const intervals = around + .slice(1) + .map((t, i) => t - around[i]) + .sort((a, b) => a - b); + const stats = frameStats( + tracked.map(f => ({t: f.t, value: f.box == null ? null : f.box[axis]})), + judged, + ); + const cadence = + intervals.length >= 4 + ? intervals[Math.floor(intervals.length / 2)] + : stats.medianFrameMs || 1000 / 60; + // The compositor's own frames, when recorded: its interval, from the frames + // around the judged part. + const draws = [...(presented ?? [])] + .filter(t => t >= judged.from - 250 && t <= judged.to + 250) + .sort((a, b) => a - b); + const drawIntervals = draws + .slice(1) + .map((t, i) => t - draws[i]) + .sort((a, b) => a - b); + const compositorFrameMs = + drawIntervals.length >= 4 + ? drawIntervals[Math.floor(drawIntervals.length / 2)] + : null; + const changed = new Set(inWindow); + const delivered = tracked + .filter(f => f.box != null && f.t > judged.from && f.t < judged.to) + .map(f => f.t); + // Each gap between presented changes, in intervals of the channel that saw + // it: the screencast's, or, for a gap between two moved frames with no + // frame delivered inside it, the compositor's own longest interval there. + const gaps = edges.slice(1).map((to, i) => { + const from = edges[i]; + const ms = to - from; + if ( + compositorFrameMs == null || + !changed.has(from) || + !changed.has(to) || + delivered.some(t => t > from && t < to) + ) { + return {from, ms, frames: Math.round(ms / cadence), skipped: 0}; + } + const span = draws.filter( + t => t > from - compositorFrameMs / 2 && t < to + compositorFrameMs / 2, + ); + const longest = + span.length >= 2 + ? Math.min(ms, Math.max(...span.slice(1).map((t, j) => t - span[j]))) + : ms; + const frames = Math.round(longest / compositorFrameMs); + return { + from, + ms: longest, + frames, + skipped: Math.max(0, Math.round(ms / compositorFrameMs) - frames), + }; + }); + const longestGap = gaps.reduce((a, b) => + b.frames > a.frames || (b.frames === a.frames && b.ms > a.ms) ? b : a, + ); + const longestStallMs = longestGap.ms; + // Where the longest stall starts, from the start of the window. + const longestStallAtMs = longestGap.from - window.from; + const longestStallFrames = longestGap.frames; + const skips = gaps.filter(g => g.skipped > 0); + const recorderSkipped = skips.reduce((sum, g) => sum + g.skipped, 0); + // With a compositor record, frames count in its intervals, and the frames + // it presented where the screencast skipped are presented frames. + const expectedFrames = Math.floor( + (judged.to - judged.from) / (compositorFrameMs ?? cadence), + ); + const moved = changes.length; + // The frames judged for motion: the screencast's own frames in the judged + // part (each `changes` entry is one of them), or, with a compositor record, + // the compositor's frames from the first to the last moved frame (the gaps + // before and after them are the stall check's). After an on-screen start + // they count from the on-screen frame, which `changes` leaves out, so the + // frames credited as skipped after it are judged too. A part that moved + // nothing, or delivered fewer than `minJudgedFrames` frames, is judged + // against its span, so a frozen animation misses here too (one late frame + // from before the block is not 1/1 moved). + const deliveredFrames = tracked.filter( + f => + f.box != null && + (onScreenFrom == null ? f.t >= judged.from : f.t > judged.from) && + f.t <= judged.to, + ).length; + const judgedFrames = + compositorFrameMs != null + ? changes.length > 0 + ? Math.round( + (changes[changes.length - 1] - (onScreenFrom ?? changes[0])) / + compositorFrameMs, + ) + (onScreenFrom == null ? 1 : 0) + : expectedFrames + : deliveredFrames < minJudgedFrames + ? expectedFrames + : deliveredFrames; + const movedFraction = (moved + recorderSkipped) / Math.max(1, judgedFrames); + // The cast's position step, and the motion per interval it has to resolve. + const positions = tracked.flatMap(f => (f.box == null ? [] : [f.box[axis]])); + const stepPx = + [2, 1, 0.5].find(step => + positions.every(p => Math.abs(p / step - Math.round(p / step)) < 0.01), + ) ?? 0.5; + const judgedValues = tracked.flatMap(f => + f.box != null && f.t >= judged.from && f.t <= judged.to + ? [{t: f.t, value: f.box[axis]}] + : [], + ); + const first = judgedValues[0]; + const last = judgedValues[judgedValues.length - 1]; + const pxPerFrame = + first == null || last == null || last.t <= first.t + ? 0 + : Math.abs(last.value - first.value) / ((last.t - first.t) / cadence); + const notes: string[] = []; + if (presented != null && compositorFrameMs == null) { + notes.push( + 'no compositor frames recorded around the window: judged from the screencast alone', + ); + } + if (recorderSkipped > 0) { + notes.push( + `the screencast skipped ${recorderSkipped} frames the compositor presented (at ${skips.map(g => `+${(g.from - window.from).toFixed(0)} ms`).join(', ')})`, + ); + } + if (pxPerFrame > 0 && pxPerFrame < 2 * stepPx) { + notes.push( + `${pxPerFrame.toFixed(1)} px/frame is under two of the cast's ${stepPx} px position steps: a frame delivered early can show no step and read as not moved`, + ); + } + const failures: string[] = []; + if (stats.frames === 0) { + failures.push('no frame presented in the judged part'); + } else if (stats.medianFrameMs > maxMedianFrameMs) { + failures.push( + `median frame interval ${stats.medianFrameMs.toFixed(1)} ms > ${maxMedianFrameMs} ms (${stats.frames} frames presented)`, + ); + } + if (longestStallFrames > maxStallFrames) { + failures.push( + `longest stall ${longestStallMs.toFixed(1)} ms = ${longestStallFrames} intervals > ${maxStallFrames}`, + ); + } + if (expectedFrames < minJudgedFrames) { + failures.push( + `${(judged.to - judged.from).toFixed(0)} ms judged = ${expectedFrames} frames < ${minJudgedFrames}: the animation under test was not moving during the block`, + ); + } + const allowedMisses = Math.max( + 1, + Math.floor((1 - minMovedFraction) * judgedFrames + 1e-9), + ); + if (judgedFrames - moved - recorderSkipped > allowedMisses) { + failures.push( + `${moved} of ${judgedFrames} expected frames moved${recorderSkipped > 0 ? ` (and ${recorderSkipped} the screencast skipped)` : ''} (${(movedFraction * 100).toFixed(0)}% < ${(minMovedFraction * 100).toFixed(0)}%)`, + ); + } + const onScreenFromMs = + onScreenFrom == null ? null : onScreenFrom - window.from; + const summary = + `${(judged.to - judged.from).toFixed(0)} ms judged at ${cadence.toFixed(1)} ms/frame: ${stats.frames} frames presented, ` + + `${moved}/${judgedFrames} moved${recorderSkipped > 0 ? ` and ${recorderSkipped} skipped by the screencast` : ''} (${(movedFraction * 100).toFixed(0)}%), ` + + `median ${stats.medianFrameMs.toFixed(1)} ms, ` + + `longest stall ${longestStallMs.toFixed(1)} ms (${longestStallFrames} intervals) at +${longestStallAtMs.toFixed(0)} ms, ` + + `${stats.monotonic ? 'monotonic' : 'not monotonic'}, ` + + `${pxPerFrame.toFixed(1)} px/frame in ${stepPx} px steps` + + (compositorFrameMs == null + ? '' + : `, compositor presenting every ${compositorFrameMs.toFixed(1)} ms`) + + (onScreenFromMs == null + ? '' + : `, on screen from +${onScreenFromMs.toFixed(0)} ms (nothing moved before the block)`) + + notes.map(note => `; note: ${note}`).join(''); + return { + ...stats, + window, + judged, + onScreenFromMs, + stepPx, + pxPerFrame, + notes, + recorderSkipped, + compositorFrameMs, + cadenceMs: cadence, + longestStallMs, + longestStallAtMs, + expectedFrames, + deliveredFrames, + longestStallFrames, + moved, + movedFraction, + verdict: {pass: failures.length === 0, failures}, + summary, + }; +} diff --git a/internal/interaction-harness/src/curves.ts b/internal/interaction-harness/src/curves.ts index 69b9908066f62..4a36629d1f0a7 100644 --- a/internal/interaction-harness/src/curves.ts +++ b/internal/interaction-harness/src/curves.ts @@ -134,17 +134,61 @@ function animationRecorder(): void { return entry.startedAt == null && entry.animation.playState !== 'idle'; }); }; + // While a rendering trace runs (traceRendering sets + // `__harnessRenderingTrace`), the recorder stands down. Its per-frame + // callback would make every frame a main-thread frame, in which Blink + // restyles each running animation's target, a composited one included (and + // lays out an anchored one), and document.getAnimations() flushes style: + // exactly what the trace counts. So it runs no frame callback, and a CSS + // animation or transition event reads only the animations of the element it + // names, once, at that animation's start (one that ends inside the trace is + // otherwise never seen). The animate() wrapper below still records script + // animations (it reads no style), and traceRendering resumes the recorder + // once the trace ends. + const tracing = () => + (window as unknown as {__harnessRenderingTrace?: boolean}) + .__harnessRenderingTrace === true; + let looping = false; const tick = () => { + if (tracing()) { + looping = false; + return; + } record(); frame(tick); }; + const loop = () => { + if (!looping) { + looping = true; + frame(tick); + } + }; // What already runs when the recorder starts (a spinner on a page the // harness touches for the first time) is seen now, before any watch arms; // first seen a frame later, it would read as started by the watched action. + // (A recorder that starts during a trace reads once here, then waits.) record(); - frame(tick); + loop(); for (const type of ['animationstart', 'transitionrun', 'transitionstart']) { - document.addEventListener(type, record, true); + document.addEventListener( + type, + event => { + if (!tracing()) { + record(); + return; + } + const target = event.target; + if (target instanceof Element) { + // A pseudo-element's animation (`::backdrop`) is in the subtree. + const pseudo = (event as AnimationEvent | TransitionEvent) + .pseudoElement; + for (const animation of target.getAnimations({subtree: !!pseudo})) { + remember(animation); + } + } + }, + true, + ); } // A script animation is recorded as it is made: WebKit can start and finish // a short one between two frames of a fresh page. @@ -159,12 +203,36 @@ function animationRecorder(): void { }; w.__harnessAnimations = { log, - record, + // A read from outside the recorder (a wait's poll) stands down while a + // trace runs, as the frame loop does: document.getAnimations() flushes + // style. CSS start events and the animate() wrapper still log what + // starts, and resume() catches up. + record: () => { + if (!tracing()) { + record(); + } + }, + // Catches up on what started during a trace, then runs its frames again. + resume: () => { + record(); + loop(); + }, // Record first, then stamp strictly after the newest entry: a step that // changed style just before has its transition created by that read, - // and it started before the arm. - arm: () => { - record(); + // and it started before the arm. While a trace runs the page-wide read + // stands down, as record() does; the watched element's own animations + // are still read, which resolves its pending style change. That read + // restyles the element (and lays out an anchored one) in the task that + // takes the watch, so a traced proof takes its watch before the window + // it counts (README: No Layout / Paint). + arm: (element?: Element) => { + if (!tracing()) { + record(); + } else if (element != null) { + for (const animation of element.getAnimations({subtree: true})) { + remember(animation); + } + } const newest = log.length === 0 ? 0 : log[log.length - 1].seenAt; armedAt = Math.max(performance.now(), newest + 0.001); return armedAt; @@ -197,10 +265,47 @@ async function installCurveEnd(page: Page): Promise { await page.evaluate(() => { const w = window as unknown as { __harnessCurveEnd?: (animation: Animation) => number; + __harnessTiming?: ( + effect: AnimationEffect | null | undefined, + ) => ComputedEffectTiming | null; + }; + // An effect's timing as getComputedTiming() reports it. Inside a rendering + // trace it is worked out from getTiming() instead: getComputedTiming() + // updates a running animation on demand, which restyles its target in the + // next frame (and lays out an anchored one), exactly what a trace counts. + w.__harnessTiming ??= effect => { + if (effect == null) { + return null; + } + if ( + (window as unknown as {__harnessRenderingTrace?: boolean}) + .__harnessRenderingTrace !== true + ) { + return effect.getComputedTiming(); + } + const timing = effect.getTiming(); + const duration = + typeof timing.duration === 'number' ? timing.duration : 0; + const iterations = Number(timing.iterations ?? 1); + const delay = Number(timing.delay ?? 0); + const endDelay = Number(timing.endDelay ?? 0); + const activeDuration = + duration === 0 || iterations === 0 ? 0 : duration * iterations; + const derived: ComputedEffectTiming = { + ...timing, + duration, + iterations, + delay, + endDelay, + activeDuration, + endTime: Math.max(delay + activeDuration + endDelay, 0), + }; + return derived; }; if (w.__harnessCurveEnd) { return; } + const timingOf = w.__harnessTiming; const timingKeys = new Set([ 'offset', 'computedOffset', @@ -216,7 +321,7 @@ async function installCurveEnd(page: Page): Promise { ); w.__harnessCurveEnd = animation => { const effect = animation.effect as KeyframeEffect | null; - const timing = effect?.getComputedTiming(); + const timing = timingOf(effect); const delay = Number(timing?.delay ?? 0); const active = Number(timing?.activeDuration ?? 0); if ( @@ -826,6 +931,47 @@ export interface ActiveAnimation { readonly finishedAt: number | null; } +/** + * Runs `fn` on `locator`'s element: through the locator, or, `quietly`, + * through a handle to the element it matches now (none yet: `[]`). Each + * locator.evaluate makes a main-thread frame, which a trace counts; + * `elementHandles()` and a handle's evaluate make none. + */ +async function readElement( + locator: Locator, + quietly: boolean, + fn: (element: Element, arg: A) => R[], + arg: A, + timeout: number, +): Promise { + const run = fn as unknown as (element: Element, arg: unknown) => R[]; + if (!quietly) { + return locator.evaluate(run, arg as unknown, {timeout}); + } + const handles = await locator.elementHandles(); + try { + if (handles.length > 1) { + throw new Error( + `waitForAnimation: ${String(locator)} matches ${handles.length} elements`, + ); + } + return handles.length === 0 + ? [] + : await handles[0].evaluate(run, arg as unknown); + } finally { + await Promise.all(handles.map(handle => handle.dispose())); + } +} + +/** The rejection for a wait whose element is not in the document. */ +function notInDocument(name: string, cause?: unknown): Error { + return new Error( + `${name}: the element is not in the document. An exit that removes ` + + 'its element is read through its container (watch the parent).', + cause === undefined ? undefined : {cause}, + ); +} + /** * Runs a locator read within the wait's own deadline. A locator waits for * its element with no timeout of its own, so one the page removed (an exit @@ -847,11 +993,7 @@ async function withinTimeout( if ((await locator.count()) > 0) { return null; } - throw new Error( - `${name}: the element is not in the document. An exit that removes ` + - 'its element is read through its container (watch the parent).', - {cause: error}, - ); + throw notInDocument(name, error); } throw error; } @@ -876,6 +1018,15 @@ async function withinTimeout( * recorder keeps one entry per `Animation` object, so an animation reversed * or replayed mid-run is misread: a watch taken before the reversal rejects, * and an unwatched wait returns at once with a curve dated before it. + * + * Started inside a `traceRendering` body, it stands down as the recorder + * does, so it adds no frame the trace counts: it takes the element's + * animations from the recorder's log, judges a started one by its own times, + * and reaches the element through a handle to what the locator matches at + * each look. That holds for script animations and CSS transitions. On a CSS + * `@keyframes` animation (and, unwatched, a cancelled CSS transition) it reads + * the animation's timing, which flushes its style and adds Layouts to the + * trace: wait on one with `waitForRest`, or after the trace. */ export async function waitForAnimation( locator: Locator, @@ -891,17 +1042,34 @@ export async function waitForAnimation( } = {}, ): Promise { const deadline = Date.now() + timeout; - await installCurveEnd(locator.page()); + const page = locator.page(); + await installCurveEnd(page); + // Inside a rendering trace the wait stands down (see the page function), + // and it reads the element through handles: each locator.evaluate makes a + // main-thread frame (a Layout of an anchored element), where a handle's + // evaluate adds none. + const traced = await page.evaluate( + () => + (window as unknown as {__harnessRenderingTrace?: boolean}) + .__harnessRenderingTrace === true, + ); for (;;) { const read = await withinTimeout( 'waitForAnimation', locator, deadline, evaluateTimeout => - locator.evaluate( + readElement( + locator, + traced, (element, {minProgress, since: watchedFrom}) => { + // The harness's own read: a trace sets it apart (trace.ts HARNESS_READ). + console.timeStamp('harness-read'); const w = window as unknown as { __harnessCurveEnd: (a: Animation) => number; + __harnessTiming: ( + effect: AnimationEffect | null | undefined, + ) => ComputedEffectTiming | null; __harnessAnimations: { log: { animation: Animation; @@ -916,6 +1084,13 @@ export async function waitForAnimation( }; const curveEnd = w.__harnessCurveEnd; const recorder = w.__harnessAnimations; + // Inside a rendering trace the wait stands down as the recorder does: + // record() reads nothing, the animations come from the recorder's log + // rather than getAnimations(), and a started one runs by its own + // times, since playState flushes a CSS animation's style. + const traced = + (window as unknown as {__harnessRenderingTrace?: boolean}) + .__harnessRenderingTrace === true; recorder.record(); const describe = ( a: Animation, @@ -923,7 +1098,7 @@ export async function waitForAnimation( recordedStart: number | null, ) => { const effect = a.effect as KeyframeEffect | null; - const timing = effect?.getComputedTiming(); + const timing = w.__harnessTiming(effect); // The recorded start stands in once the page has cancelled it. A // run going backward began at its end: its startTime is where its // current time would reach 0, a whole run later. @@ -994,24 +1169,36 @@ export async function waitForAnimation( // finished must not answer for it. Only document-timeline animations // count, as for holdAnimations: a scroll- or view-timeline one moves // only when something scrolls and never progresses on its own. + const runs = (a: Animation) => { + if (!(a.timeline instanceof DocumentTimeline)) { + return false; + } + if (!traced || a.startTime == null) { + return a.playState === 'running'; + } + const time = Number(a.currentTime ?? 0); + return a.playbackRate < 0 + ? time > 0 + : time < Number(w.__harnessTiming(a.effect)?.endTime ?? 0); + }; + const onElement = () => + recorder.log.filter( + e => inside(e.animation) || e.ancestors.includes(element), + ); const live = ( - watchedFrom == null + watchedFrom == null && !traced ? element.getAnimations({subtree: true}) - : recorded + : (watchedFrom == null ? onElement() : recorded) .filter(e => e.finishedAt == null) .map(e => e.animation) - ).filter( - a => - a.playState === 'running' && - a.timeline instanceof DocumentTimeline, - ); + ).filter(runs); const running = live.filter(a => a.startTime != null); // Progress along the curve, on the timeline: a delay presents nothing // (going backward, the end delay plays first), a run going backward // (`reverse()`) progresses from its end, and a faster rate presents // more curve per ms. const progressed = running.filter(a => { - const timing = a.effect?.getComputedTiming(); + const timing = w.__harnessTiming(a.effect); const time = Number(a.currentTime ?? 0); const local = a.playbackRate < 0 @@ -1036,13 +1223,20 @@ export async function waitForAnimation( return e.finishedAt; } const a = e.animation; - if (a.playState !== 'finished' || a.startTime == null) { + if (a.startTime == null) { + return null; + } + const end = Number(w.__harnessTiming(a.effect)?.endTime ?? 0); + // Traced, by its own times; otherwise by its state. + const time = Number(a.currentTime ?? 0); + const done = traced + ? a.playbackRate < 0 + ? time <= 0 + : time >= end + : a.playState === 'finished'; + if (!done) { return null; } - const end = Number( - (a.effect as KeyframeEffect | null)?.getComputedTiming() - .endTime ?? 0, - ); return a.playbackRate < 0 ? Number(a.startTime) : Number(a.startTime) + end / Math.abs(a.playbackRate || 1); @@ -1052,7 +1246,7 @@ export async function waitForAnimation( .map(e => describe(e.animation, finishedAt(e), e.startedAt)); }, {minProgress: minProgressMs, since: since ?? null}, - {timeout: evaluateTimeout}, + evaluateTimeout, ), ); const found = read ?? []; @@ -1061,6 +1255,12 @@ export async function waitForAnimation( return found; } if (Date.now() > deadline) { + // A traced read goes through handles, which find no element rather + // than wait for one: a missing element is reported as missing, as the + // untraced read reports it. + if (traced && (await locator.count()) === 0) { + throw notInDocument('waitForAnimation'); + } throw new Error( 'waitForAnimation: no animation started on the element, none running ' + 'and none recorded since the last wait. If the action is the ' + @@ -1072,13 +1272,31 @@ export async function waitForAnimation( } } -/** Marks now as the point later waits count animations from. */ -async function armRecorder(page: Page): Promise { - return page.evaluate(() => - ( - window as unknown as {__harnessAnimations: {arm: () => number}} - ).__harnessAnimations.arm(), - ); +/** + * Marks now as the point later waits count animations from. With a locator, + * its element's pending style change is resolved first, also inside a trace + * (reached through a handle, which adds no main-thread frame). + */ +async function armRecorder(page: Page, locator?: Locator): Promise { + const handles = locator == null ? [] : await locator.elementHandles(); + try { + if (handles.length === 1) { + return await handles[0].evaluate(element => + ( + window as unknown as { + __harnessAnimations: {arm: (element?: Element) => number}; + } + ).__harnessAnimations.arm(element as Element), + ); + } + return await page.evaluate(() => + ( + window as unknown as {__harnessAnimations: {arm: () => number}} + ).__harnessAnimations.arm(), + ); + } finally { + await Promise.all(handles.map(handle => handle.dispose())); + } } export interface AnimationWatch { @@ -1096,7 +1314,9 @@ export interface AnimationWatch { /** * Starts watching `locator`'s element before an action, so the animation the * action starts is caught even when it ends before the test looks (a short - * entrance on a cold page): + * entrance on a cold page). Taking the watch reads the element's animations, + * which restyles it, inside a rendering trace too: take a traced watch before + * the window the proof counts: * * const watch = await watchAnimations(sheet); * await driver.play(tap(button)); @@ -1106,7 +1326,7 @@ export async function watchAnimations( locator: Locator, ): Promise { await installCurveEnd(locator.page()); - const since = await armRecorder(locator.page()); + const since = await armRecorder(locator.page(), locator); return { started: options => waitForAnimation(locator, {...options, since}), }; @@ -1125,6 +1345,19 @@ export async function watchAnimations( * forward run under a linear or cubic-bézier effect easing); under any other * easing or direction, waitForRest waits the hold out, and a hold longer than * `timeout` rejects with "still moving". + * + * While a rendering trace runs (`traceRendering`), it stands down as the + * animation recorder does: a per-frame callback would make every frame a + * main-thread frame, which the trace counts. It then waits on timers for the + * scheduled ends of the curves on the element instead (a run going backward, + * until it finishes), so motion that no animation drives (a script writing + * styles) is not seen until the trace ends. Its one read of the element's + * animations, as it is called, restyles the element (and lays out an anchored + * one) when an animation is already running there: await it right after the + * action that starts the motion, before the window a proof counts, not in its + * middle (README: No Layout / Paint). Finding no animation, it looks again `frames` times a frame + * apart before settling, so one the action starts a few frames later is + * waited for. */ export async function waitForRest( locator: Locator, @@ -1157,15 +1390,94 @@ export async function waitForRest( backdrop, ].join(' | '); }; - // The page's real frames, even while a timer gate holds its callbacks. - const frame = + // The page's real frames and timers, even while a timer gate holds + // its callbacks: a frozen gate never holds the wait itself. + const gate = ( + window as unknown as { + __harnessTimerGate?: { + frame: typeof requestAnimationFrame; + timeout?: typeof setTimeout; + }; + } + ).__harnessTimerGate; + const frame = gate?.frame ?? requestAnimationFrame; + const timeout = gate?.timeout ?? setTimeout; + // Set by traceRendering while it traces (see the recorder above). + const tracing = () => + (window as unknown as {__harnessRenderingTrace?: boolean}) + .__harnessRenderingTrace === true; + const settle = () => { + // Later waits count animations from here. ( - window as unknown as { - __harnessTimerGate?: {frame: typeof requestAnimationFrame}; - } - ).__harnessTimerGate?.frame ?? requestAnimationFrame; - let last = read(); + window as unknown as {__harnessAnimations?: {arm: () => number}} + ).__harnessAnimations?.arm(); + resolve(); + }; + let last = ''; let still = 0; + const frames = () => { + last = read(); + still = 0; + frame(tick); + }; + // While traced: one look at the animations, then a timer to the latest + // scheduled curve end. One not started yet is looked at again a frame + // later; a trace that has ended hands over to the per-frame check. + // Finding nothing settles only after `frames` looks a frame apart, as + // the per-frame check needs `frames` still frames: an animation the + // action starts a frame or two later is still waited for. + let idle = 0; + const scheduled = () => { + let wait = 0; + for (const a of element.getAnimations({subtree: true})) { + // A scroll- or view-timeline animation moves only when something + // scrolls: it has no scheduled end to wait for (holdAnimations + // skips it the same way). + if (!(a.timeline instanceof DocumentTimeline)) { + continue; + } + if ( + a.pending || + (a.playState === 'running' && a.startTime == null) + ) { + wait = Math.max(wait, 16); + } else if (a.playState === 'running') { + // The curve's time left through the playback rate: forward, to + // its (held) end; backward (`reverse()`), to local time 0, + // where it finishes, as the untraced path waits it out. + const rate = a.playbackRate || 1; + const at = Number(a.currentTime ?? 0); + const left = + rate > 0 ? (curveEnd(a) - at) / rate : at / -rate; + wait = Math.max(wait, left); + } + } + if (wait <= 0) { + idle += 1; + if (idle >= need) { + settle(); + return; + } + wait = 16; + } else { + idle = 0; + } + const room = deadline - performance.now(); + timeout( + () => { + if (performance.now() > deadline) { + reject( + new Error('waitForRest: still moving at the deadline'), + ); + } else if (tracing()) { + scheduled(); + } else { + frames(); + } + }, + Math.min(wait, Math.max(0, room)) + 1, + ); + }; const tick = () => { const now = read(); const moving = element.getAnimations({subtree: true}).some( @@ -1182,20 +1494,20 @@ export async function waitForRest( still = !moving && now === last ? still + 1 : 0; last = now; if (still >= need) { - // Later waits count animations from here. - ( - window as unknown as { - __harnessAnimations?: {arm: () => number}; - } - ).__harnessAnimations?.arm(); - resolve(); + settle(); } else if (performance.now() > deadline) { reject(new Error(`waitForRest: still moving (${now})`)); + } else if (tracing()) { + scheduled(); } else { frame(tick); } }; - frame(tick); + if (tracing()) { + scheduled(); + } else { + frames(); + } }), {ms: timeout, need: frames}, {timeout: evaluateTimeout}, diff --git a/internal/interaction-harness/src/frames.ts b/internal/interaction-harness/src/frames.ts new file mode 100644 index 0000000000000..83841d259c17f --- /dev/null +++ b/internal/interaction-harness/src/frames.ts @@ -0,0 +1,531 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file frames.ts + * @input Uses a Playwright `Page`, a `Locator` for the element under test, and + * the engine's screencast (Chromium CDP `Page.startScreencast`; WebKit's + * `Screencast` domain) + * @output `startScreencast` (timestamped composited frames), `markElement` + * (a marker outline the frames can find), `trackMarker` (per-frame marker + * box), `sampleRects` (per-rAF rect sampling), `frameStats`. + * @position Frame evidence for motion proofs. The screencast is captured off + * the page's main thread, from what the compositor presents, so it keeps + * delivering frames while a script blocks the main thread: that is what makes + * the busy-main-thread proof possible. Rect sampling is the cheap path when + * the main thread is free. + * + * Marker tracking needs no image library: frames are decoded and scanned in a + * scratch page of the same browser (createImageBitmap + canvas), so the + * harness adds no dependency. + * + * SYNC: Keep the README's frame-evidence section equal to what this measures. + */ + +import type {CDPSession, Locator, Page} from '@playwright/test'; +import {webkitPageSession} from './touch'; + +export interface CastFrame { + /** Presentation time, epoch ms (the compositor's frame timestamp). */ + readonly t: number; + /** Base64 PNG/JPEG. */ + readonly data: string; + readonly format: 'png' | 'jpeg'; + /** CSS px of the viewport the frame shows. */ + readonly viewportWidth: number; + readonly viewportHeight: number; +} + +export interface Screencast { + /** + * Frames captured so far: wait for the first one to know the cast saw the + * page before an animation starts (`busyWindowProof` then knows its at-rest + * position). + */ + frameCount(): number; + /** Stops and returns every frame captured, in order. */ + stop(): Promise; +} + +export interface ScreencastOptions { + /** Lossless frames by default: marker colours survive PNG, not JPEG. */ + readonly format?: 'png' | 'jpeg'; + readonly quality?: number; + /** + * Keep one presented frame in this many (default 1, every frame): a slower + * evidence channel, like a loaded machine's screencast, for proving that a + * verdict does not depend on the screencast's rate. + */ + readonly everyNthFrame?: number; +} + +/** + * Starts capturing composited frames. Chromium delivers one per presented + * frame (16.7 ms at 60 Hz), timestamped by the compositor; WebKit's own + * screencast delivers one per composited update (about every 14 ms in the + * Linux build), stamped on CLOCK_MONOTONIC and rebased here onto the wall + * clock by that clock's measured offset, so both engines share the page's + * timeline. Both keep delivering while the page's main thread is blocked, and + * neither sends a frame when nothing on screen changed. + */ +export async function startScreencast( + page: Page, + options: ScreencastOptions = {}, +): Promise { + const name = page.context().browser()?.browserType().name(); + const viewport = page.viewportSize() ?? {width: 0, height: 0}; + if (name === 'chromium') { + return chromiumScreencast(page, viewport, options); + } + if (name === 'webkit') { + return webkitScreencast(page, viewport, options); + } + throw new Error( + `startScreencast: unsupported browser ${name ?? '(unknown)'}`, + ); +} + +async function chromiumScreencast( + page: Page, + viewport: {width: number; height: number}, + {format = 'png', quality = 90, everyNthFrame = 1}: ScreencastOptions, +): Promise { + const cdp: CDPSession = await page.context().newCDPSession(page); + const frames: CastFrame[] = []; + const onFrame = (frame: { + data: string; + sessionId: number; + metadata: {timestamp?: number; deviceWidth: number; deviceHeight: number}; + }) => { + frames.push({ + t: (frame.metadata.timestamp ?? Date.now() / 1000) * 1000, + data: frame.data, + format, + viewportWidth: frame.metadata.deviceWidth || viewport.width, + viewportHeight: frame.metadata.deviceHeight || viewport.height, + }); + cdp + .send('Page.screencastFrameAck', {sessionId: frame.sessionId}) + .catch(() => {}); + }; + cdp.on('Page.screencastFrame', onFrame); + // One CSS px per frame px keeps decoding cheap; everyNthFrame 1 = every presented frame. + await cdp.send('Page.startScreencast', { + format, + quality, + maxWidth: viewport.width, + maxHeight: viewport.height, + everyNthFrame, + }); + return { + frameCount: () => frames.length, + async stop() { + await cdp.send('Page.stopScreencast'); + cdp.off('Page.screencastFrame', onFrame); + await cdp.detach().catch(() => {}); + return inPresentationOrder(frames); + }, + }; +} + +async function webkitScreencast( + page: Page, + viewport: {width: number; height: number}, + {quality = 90, everyNthFrame = 1}: ScreencastOptions, +): Promise { + const session = webkitPageSession(page); + const frames: CastFrame[] = []; + let generation = 0; + let delivered = 0; + const onFrame = (payload: unknown) => { + const frame = payload as {data: string; timestamp: number}; + // WebKit's screencast has no everyNthFrame: keep one frame in N here. + delivered += 1; + if ((delivered - 1) % everyNthFrame !== 0) { + session + .send('Screencast.screencastFrameAck', {generation}) + .catch(() => {}); + return; + } + frames.push({ + // WebKit stamps frames in seconds on the monotonic clock; rebased below. + t: frame.timestamp * 1000, + data: frame.data, + format: 'jpeg', + viewportWidth: viewport.width, + viewportHeight: viewport.height, + }); + session.send('Screencast.screencastFrameAck', {generation}).catch(() => {}); + }; + session.on('Screencast.screencastFrame', onFrame); + const started = (await session.send('Screencast.startScreencast', { + width: viewport.width, + height: viewport.height, + toolbarHeight: 0, + quality, + })) as {generation: number}; + generation = started.generation; + // WebKit stamps frames in seconds on CLOCK_MONOTONIC, which this process + // shares: rebase them by the wall clock's offset from that clock, sampled + // together. (Taking the first frame as the start shifted every frame earlier + // by however long that frame took to come, 60–100 ms on a busy page.) + const wallMinusMonotonic = Date.now() - Number(process.hrtime.bigint()) / 1e6; + return { + frameCount: () => frames.length, + async stop() { + await session.send('Screencast.stopScreencast'); + session.off?.('Screencast.screencastFrame', onFrame); + if (frames.length === 0) { + return frames; + } + return inPresentationOrder(frames).map(f => ({ + ...f, + t: f.t + wallMinusMonotonic, + })); + }, + }; +} + +/** + * Frames sorted by their presentation timestamp. A loaded machine can deliver + * screencast frames out of order (a frame stamped 35 ms arriving after one + * stamped 46 ms); judged in arrival order, a steadily moving element would + * appear to step backwards. + */ +export function inPresentationOrder( + frames: readonly T[], +): T[] { + return [...frames].sort((a, b) => a.t - b.t); +} + +/** The marker colour: magenta never occurs in Astryx themes. */ +export const MARKER_RGB: readonly [number, number, number] = [255, 0, 255]; + +export interface MarkerOptions { + readonly rgb?: readonly [number, number, number]; + /** + * `chip` (default): a small square child pinned inside the element's + * top-left corner, above every sibling (z-index max), so nothing the + * component paints can cover it. The element must be its containing block: + * a static element is made `position: relative` while marked. + * `outline`: an inset outline on the element itself (no DOM insertion), for + * elements that cannot take a child; children with their own layers can + * cover parts of it. + */ + readonly mode?: 'chip' | 'outline'; + /** Chip size or outline width, CSS px. */ + readonly size?: number; + /** Chip inset from the element's top-left corner, CSS px. */ + readonly inset?: number; +} + +/** + * Marks the element so screencast frames can find it: the mark moves, scales + * and fades with the element's own layer and never changes its layout. + * Returns an undo. + */ +export async function markElement( + locator: Locator, + options: MarkerOptions = {}, +): Promise<() => Promise> { + const { + rgb = MARKER_RGB, + mode = 'chip', + size = mode === 'chip' ? 16 : 6, + inset = 8, + } = options; + const id = `m${Math.random().toString(36).slice(2, 8)}`; + await locator.evaluate( + (element, {id: markerId, color, size: px, inset: offset, mode: kind}) => { + const el = element as HTMLElement; + el.setAttribute('data-harness-marker', markerId); + if (kind === 'outline') { + const style = document.createElement('style'); + style.setAttribute('data-harness-marker-style', markerId); + style.textContent = `[data-harness-marker="${markerId}"]{outline:${px}px solid rgb(${color.join(',')}) !important;outline-offset:-${px}px !important;}`; + document.head.append(style); + return; + } + if (getComputedStyle(el).position === 'static') { + el.setAttribute('data-harness-marker-position', el.style.position); + el.style.position = 'relative'; + } + const chip = document.createElement('div'); + chip.setAttribute('data-harness-marker-chip', markerId); + chip.setAttribute('aria-hidden', 'true'); + chip.style.cssText = `position:absolute;top:${offset}px;left:${offset}px;width:${px}px;height:${px}px;background:rgb(${color.join(',')});z-index:2147483647;pointer-events:none;`; + el.append(chip); + }, + {id, color: [...rgb], size, inset, mode}, + ); + return async () => { + await locator.page().evaluate(markerId => { + const el = document.querySelector( + `[data-harness-marker="${markerId}"]`, + ); + if (el?.hasAttribute('data-harness-marker-position')) { + el.style.position = + el.getAttribute('data-harness-marker-position') ?? ''; + el.removeAttribute('data-harness-marker-position'); + } + el?.removeAttribute('data-harness-marker'); + document + .querySelector(`[data-harness-marker-chip="${markerId}"]`) + ?.remove(); + document + .querySelector(`[data-harness-marker-style="${markerId}"]`) + ?.remove(); + }, id); + }; +} + +export interface Box { + readonly x: number; + readonly y: number; + readonly width: number; + readonly height: number; +} + +export interface TrackedFrame { + /** Epoch ms. */ + readonly t: number; + /** The marker's bounding box in CSS px, or null when it is not on screen. */ + readonly box: Box | null; + /** Marker pixels found (a fading element finds fewer at the same tolerance). */ + readonly pixels: number; +} + +/** + * Finds the marker in every frame. Decoding runs in a scratch page of the same + * browser, so neither the page under test nor Node needs an image decoder. + */ +export async function trackMarker( + page: Page, + frames: readonly CastFrame[], + { + rgb = MARKER_RGB, + tolerance = 40, + }: {rgb?: readonly [number, number, number]; tolerance?: number} = {}, +): Promise { + const scratch = await page.context().newPage(); + try { + const out: TrackedFrame[] = []; + // Batches keep each evaluate payload moderate. + for (let i = 0; i < frames.length; i += 12) { + const batch = frames.slice(i, i + 12); + const boxes = await scratch.evaluate( + async ({batch: items, color, tol}) => { + const results: { + box: {x: number; y: number; width: number; height: number} | null; + pixels: number; + }[] = []; + for (const item of items) { + const blob = await ( + await fetch(`data:image/${item.format};base64,${item.data}`) + ).blob(); + const bitmap = await createImageBitmap(blob); + const canvas = new OffscreenCanvas(bitmap.width, bitmap.height); + const context = canvas.getContext('2d'); + if (context == null) { + throw new Error('no 2d context'); + } + context.drawImage(bitmap, 0, 0); + const {data, width, height} = context.getImageData( + 0, + 0, + bitmap.width, + bitmap.height, + ); + let minX = Infinity; + let minY = Infinity; + let maxX = -Infinity; + let maxY = -Infinity; + let pixels = 0; + for (let y = 0; y < height; y += 1) { + for (let x = 0; x < width; x += 1) { + const o = (y * width + x) * 4; + if ( + Math.abs(data[o] - color[0]) <= tol && + Math.abs(data[o + 1] - color[1]) <= tol && + Math.abs(data[o + 2] - color[2]) <= tol + ) { + pixels += 1; + if (x < minX) { + minX = x; + } + if (x > maxX) { + maxX = x; + } + if (y < minY) { + minY = y; + } + if (y > maxY) { + maxY = y; + } + } + } + } + const scale = width / item.viewportWidth; + results.push({ + box: + pixels === 0 + ? null + : { + x: minX / scale, + y: minY / scale, + width: (maxX - minX + 1) / scale, + height: (maxY - minY + 1) / scale, + }, + pixels, + }); + } + return results; + }, + { + batch: batch.map(f => ({ + data: f.data, + format: f.format, + viewportWidth: f.viewportWidth, + })), + color: [...rgb], + tol: tolerance, + }, + ); + boxes.forEach((result, k) => + out.push({t: batch[k].t, box: result.box, pixels: result.pixels}), + ); + } + return out; + } finally { + await scratch.close(); + } +} + +export interface RectSample { + /** Epoch ms of the rAF callback. */ + readonly t: number; + readonly box: Box | null; + readonly transform: string; + readonly opacity: number; +} + +/** + * Samples the element's rect, transform and opacity on every animation frame + * for `durationMs`. Main-thread sampling: it stops while the main thread is + * blocked, which is the point of the screencast path above. + */ +export async function sampleRects( + locator: Locator, + durationMs: number, +): Promise { + return locator.evaluate( + (element, duration) => + new Promise< + { + t: number; + box: {x: number; y: number; width: number; height: number} | null; + transform: string; + opacity: number; + }[] + >(resolve => { + // The harness's own waits run on the real timers: a frozen timer gate + // (story.ts) holds only the page's. + const gate = ( + window as unknown as { + __harnessTimerGate?: { + frame?: typeof requestAnimationFrame; + }; + } + ).__harnessTimerGate; + const frame = gate?.frame ?? window.requestAnimationFrame.bind(window); + const samples: { + t: number; + box: {x: number; y: number; width: number; height: number} | null; + transform: string; + opacity: number; + }[] = []; + const end = performance.now() + duration; + const tick = (now: number) => { + const rect = element.getBoundingClientRect(); + const style = getComputedStyle(element); + samples.push({ + t: performance.timeOrigin + now, + box: {x: rect.x, y: rect.y, width: rect.width, height: rect.height}, + transform: style.transform, + opacity: Number(style.opacity), + }); + if (now < end) { + frame(tick); + } else { + resolve(samples); + } + }; + frame(tick); + }), + durationMs, + ); +} + +export interface FrameStats { + readonly frames: number; + /** Frames whose tracked value differs from the frame before. */ + readonly changed: number; + /** Longest stretch without a change inside the window (an open stall at its end included), ms. */ + readonly longestStallMs: number; + /** Median spacing between frames, ms. */ + readonly medianFrameMs: number; + readonly monotonic: boolean; + readonly first: number | null; + readonly last: number | null; +} + +/** Summarises per-frame values (e.g. the marker's y) inside a time window. */ +export function frameStats( + samples: readonly {readonly t: number; readonly value: number | null}[], + window: {readonly from: number; readonly to: number} = { + from: -Infinity, + to: Infinity, + }, + epsilon = 0.25, +): FrameStats { + const inside = samples.filter( + s => s.t >= window.from && s.t <= window.to && s.value != null, + ) as {t: number; value: number}[]; + let changed = 0; + let increasing = 0; + let decreasing = 0; + let stallStart = inside[0]?.t ?? 0; + let longestStallMs = 0; + for (let i = 1; i < inside.length; i += 1) { + const delta = inside[i].value - inside[i - 1].value; + if (Math.abs(delta) > epsilon) { + changed += 1; + if (delta > 0) { + increasing += 1; + } else { + decreasing += 1; + } + longestStallMs = Math.max(longestStallMs, inside[i].t - stallStart); + stallStart = inside[i].t; + } + } + // A stall still open when the window closes counts too: a frozen animation never changes at all. + if (inside.length > 0) { + longestStallMs = Math.max( + longestStallMs, + inside[inside.length - 1].t - stallStart, + ); + } + const gaps = inside + .slice(1) + .map((s, i) => s.t - inside[i].t) + .sort((a, b) => a - b); + return { + frames: inside.length, + changed, + longestStallMs, + medianFrameMs: gaps.length ? gaps[Math.floor(gaps.length / 2)] : 0, + monotonic: increasing === 0 || decreasing === 0, + first: inside[0]?.value ?? null, + last: inside[inside.length - 1]?.value ?? null, + }; +} diff --git a/internal/interaction-harness/src/harness.interaction.browser.spec.ts b/internal/interaction-harness/src/harness.interaction.browser.spec.ts index 7e47af95b2c81..b1a50ebc8c9e3 100644 --- a/internal/interaction-harness/src/harness.interaction.browser.spec.ts +++ b/internal/interaction-harness/src/harness.interaction.browser.spec.ts @@ -3,32 +3,52 @@ /** * @file harness.interaction.browser.spec.ts * @input Fixture pages written inline (no Storybook), real Chromium/WebKit - * @output Proof that each harness instrument measures what it claims: - * scripted timestamps reach the page; a flick flings natively (Chromium); - * a second finger and a pinch arrive as distinct pointers; sampled curves - * match the oracles; the animation-state waits report a running transition - * and its end, and judge rest by presented values even while an animation - * holds its end, and a watch taken on first contact ignores an animation - * already running and keeps the start of one the page cancels when it ends; individual transform properties are sampled; a sample at + * @output Proof that each harness instrument measures what it claims (the + * busy-main-thread proof and the rendering-trace counts with a negative + * control beside their positive checks): scripted timestamps reach the + * page; a flick flings natively (Chromium); a second finger and a pinch + * arrive as distinct pointers; the screencast shows a compositor animation + * advancing through a blocked main thread while a rAF animation freezes; + * the trace counts zero Layout for a transform animation and one per frame + * for a `top` animation, and none from the harness's own animation recorder + * while an anchored element's composited animation runs (a page that asks + * for a frame every frame lays it out in each); the long-task log records + * the page's own long task but not a DevTools evaluate, which the trace + * counts; sampled curves + * match the oracles, individual transform properties included; a sample at * an end reports the time it was read at; holding takes only running * document-timeline animations, each at its own offset, holds each where it * was read, adds a second hold to the held set without moving the first - * hold's times, drops one the page cancels, and releases only them; - * storyboards write a sheet and a trace that follows the curve, an endless - * animation does not set their length, and a photograph Chromium returns - * empty is taken again two frames later; a touch filmstrip keeps scripted - * event times while a handler clock sees its pacing; the timer gate holds a - * component timer until the filmstrip is done, drops callbacks the page - * cancels, stamps held frames with the time they run, and leaves the - * harness its own frames. + * hold's times, drops one the page cancels, and releases only them; the + * animation-state waits report a running transition and its end, and judge + * rest by presented values even while an animation holds its end, and a + * watch taken on first contact ignores an animation already running and + * keeps the start of one the page cancels when it ends, and the recorder + * reads only at CSS starts while a rendering trace runs and catches up + * after it; + * storyboards write a + * sheet and a trace that follows the curve, an endless animation does not + * set their length, and a photograph Chromium returns empty is taken again + * two frames later; a touch filmstrip keeps scripted event times while a + * handler clock sees its pacing; the timer gate holds a component timer + * until the filmstrip is done, drops callbacks the page cancels, stamps held + * frames with the time they run, and leaves the harness its own frames; a + * cast filmstrip keeps exactly the presented frames of its window, labelled + * from its zero, as PNG. * @position The harness's own test. A component spec trusts these * instruments; this file is why it may. */ +import fs from 'node:fs'; +import path from 'node:path'; import {expect, test, type Page} from '@playwright/test'; import { animationStoryboard, + castStoryboard, + blockMainThread, + busyWindowProof, compareCurve, + countBetween, cubicBezier, curveOf, drag, @@ -37,28 +57,46 @@ import { freezeTimers, holdAnimations, installInputLog, + installLongTaskLog, installTimerGate, + markElement, merge, parseTransform, PARTIAL_LIFT_SKIP, pinch, readInputLog, + readLongTasks, releaseAnimations, sampleCurve, + sampleRects, seekAnimations, springStep, + startCompositorTrace, + startScreencast, Storyboard, tap, thawTimers, timeline, touchDriver, touchStoryboard, + traceRendering, + trackMarker, waitForAnimation, waitForRest, watchAnimations, withFinger, } from './index'; +// An element positioned against an anchor: a composited animation of it lays +// out in every main-thread frame, so it shows any per-frame main-thread work. +const ANCHORED = ` + +
`; + const FIXTURE = ` +
`; + +async function busyFixture(page: Page) { + await page.route('http://busy.test/', route => + route.fulfill({contentType: 'text/html', body: BUSY_FIXTURE}), + ); + await page.goto('http://busy.test/'); +} + // A real navigation (not setContent, whose document.open() erases the window // listeners the input log installs). async function fixture(page: Page) { @@ -242,6 +298,894 @@ test.describe('touch input', () => { }); }); +test.describe('frame evidence', () => { + test('a compositor animation advances every frame through a blocked main thread; a rAF one freezes, whatever the screencast rate', async ({ + page, + }, testInfo) => { + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + + const run = async (driver: 'compositor' | 'raf', everyNthFrame = 1) => { + await box.evaluate((el, how) => { + el.getAnimations().forEach(a => a.cancel()); + el.style.transform = 'none'; + if (how === 'compositor') { + el.animate( + [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}], + {duration: 1600, easing: 'linear'}, + ); + } else { + const start = performance.now(); + const step = (now: number) => { + const p = Math.min(1, (now - start) / 1600); + el.style.transform = `translateY(${560 * p}px)`; + if (p < 1) { + requestAnimationFrame(step); + } + }; + requestAnimationFrame(step); + } + }, driver); + const cast = await startScreencast(page, {everyNthFrame}); + await page.waitForTimeout(200); + const window = await blockMainThread(page, 700); + await page.waitForTimeout(200); + const tracked = await trackMarker(page, await cast.stop()); + return busyWindowProof(tracked, window, {axis: 'y'}); + }; + + const compositor = await run('compositor'); + testInfo.annotations.push({ + type: 'busy-main-thread compositor', + description: compositor.summary, + }); + expect(compositor.verdict.failures, compositor.summary).toEqual([]); + expect(compositor.monotonic).toBe(true); + // Resolvable on the cast's grid: two position steps or more per frame. + expect(compositor.notes, compositor.summary).toEqual([]); + expect(compositor.pxPerFrame).toBeGreaterThanOrEqual(2 * compositor.stepPx); + + const raf = await run('raf'); + testInfo.annotations.push({ + type: 'busy-main-thread rAF control', + description: raf.summary, + }); + // Frozen for the whole block: no presented frame moved (the compositor + // sends none at all when nothing on screen changes), and the verdict that + // passes the compositor animation fails it on every criterion. + expect(raf.changed).toBe(0); + expect(raf.longestStallMs).toBeGreaterThan(600); + expect(raf.verdict.pass).toBe(false); + expect(raf.verdict.failures.join('\n')).toMatch(/longest stall/); + expect(raf.verdict.failures.join('\n')).toMatch(/expected frames moved/); + + // The verdict judges the frames the screencast delivered, not its rate: + // at one frame in six (a loaded machine's ~100 ms), the compositor + // animation still passes and the rAF one still fails. + const slow = await run('compositor', 6); + expect(slow.cadenceMs, slow.summary).toBeGreaterThan(60); + expect(slow.verdict.failures, slow.summary).toEqual([]); + const slowRaf = await run('raf', 6); + expect(slowRaf.verdict.pass, slowRaf.summary).toBe(false); + expect(slowRaf.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + test('frames the screencast skips while the compositor presents them are not a stall; a frozen animation still fails with the compositor record', async ({ + page, + browserName, + }, testInfo) => { + test.skip( + browserName !== 'chromium', + 'compositor traces are Chromium-only (README: engine limits)', + ); + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + const run = async (driver: 'compositor' | 'raf') => { + await box.evaluate((el, how) => { + el.getAnimations().forEach(a => a.cancel()); + el.style.transform = 'none'; + if (how === 'compositor') { + el.animate( + [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}], + {duration: 1600, easing: 'linear'}, + ); + } else { + const start = performance.now(); + const step = (now: number) => { + const p = Math.min(1, (now - start) / 1600); + el.style.transform = `translateY(${560 * p}px)`; + if (p < 1) { + requestAnimationFrame(step); + } + }; + requestAnimationFrame(step); + } + }, driver); + const cast = await startScreencast(page); + const compositor = await startCompositorTrace(page); + await page.waitForTimeout(200); + const window = await blockMainThread(page, 700); + await page.waitForTimeout(200); + const presented = await compositor.stop(); + const tracked = await trackMarker(page, await cast.stop()); + return {window, tracked, presented}; + }; + + const moving = await run('compositor'); + expect(moving.presented.length).toBeGreaterThan(0); + // The screencast as a loaded machine delivers it: four frames in a row + // skipped mid-block, which the compositor still presented. + const mid = moving.tracked.findIndex(f => f.t > moving.window.from + 350); + const skipped = moving.tracked.filter((_, i) => i < mid || i >= mid + 4); + const alone = busyWindowProof(skipped, moving.window, {axis: 'y'}); + expect(alone.verdict.failures.join('\n'), alone.summary).toMatch( + /longest stall/, + ); + const proof = busyWindowProof(skipped, moving.window, { + axis: 'y', + presented: moving.presented, + }); + testInfo.annotations.push({ + type: 'busy-main-thread compositor record', + description: proof.summary, + }); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.recorderSkipped).toBeGreaterThanOrEqual(4); + + // A rAF animation presents no change for the whole block: the record + // excuses nothing, whatever the compositor drew meanwhile. + const frozen = await run('raf'); + const still = busyWindowProof(frozen.tracked, frozen.window, { + axis: 'y', + presented: frozen.presented, + }); + testInfo.annotations.push({ + type: 'busy-main-thread rAF control, compositor record', + description: `${still.summary}; compositor frames inside the block: ${frozen.presented.filter(t => t > frozen.window.from && t < frozen.window.to).length}`, + }); + expect(still.verdict.pass).toBe(false); + expect(still.recorderSkipped).toBe(0); + expect(still.verdict.failures.join('\n')).toMatch(/longest stall/); + }); + + test('a block that starts during a just-started transition still lets the compositor advance it', async ({ + page, + }, testInfo) => { + // `during` blocks in the frame the transition has run 16 ms. In WebKit + // the transition reaches the compositor in that frame's rendering update, + // so a loop inside the rAF callback froze it (2 of 29 frames moved). + // Linear and longer than the block, so every frame of the block moves two + // position steps or more and the transition is still running when it + // ends. The cast shows the box at rest first, so a transition that reaches + // the screen after the block began is judged from there (onScreenFromMs). + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + const cast = await startScreencast(page); + await expect.poll(() => cast.frameCount()).toBeGreaterThan(0); + await box.evaluate(el => { + el.style.transition = 'transform 1600ms linear'; + el.getBoundingClientRect(); + el.style.transform = 'translateY(560px)'; + }); + const window = await blockMainThread(page, 700, { + during: box, + minProgressMs: 16, + }); + await waitForRest(box); + const tracked = await trackMarker(page, await cast.stop()); + const proof = busyWindowProof(tracked, window, {axis: 'y'}); + testInfo.annotations.push({ + type: 'busy-main-thread transition', + description: proof.summary, + }); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.monotonic).toBe(true); + }); + + test('a block never starts on an animation that is not on the compositor yet', async ({ + page, + }, testInfo) => { + // An animation created in the frame the helper looks is still pending (not + // committed to the compositor). A block that started on it then froze it + // in Chromium: no presented frame moved. + await busyFixture(page); + const box = page.locator('#box'); + await markElement(box, {mode: 'outline'}); + await box.evaluate(el => { + el.style.willChange = 'transform'; + }); + const cast = await startScreencast(page); + // A rAF loop registered before the helper's tick runs ahead of it in every + // frame; on its 12th frame it creates the animation, and the helper's tick + // looks in that same frame. + await box.evaluate(el => { + let n = 0; + const loop = () => { + n += 1; + if (n === 12) { + el.animate( + [{transform: 'translateY(0px)'}, {transform: 'translateY(560px)'}], + {duration: 1600, easing: 'linear'}, + ); + return; + } + requestAnimationFrame(loop); + }; + requestAnimationFrame(loop); + }); + const window = await blockMainThread(page, 700, { + during: box, + minProgressMs: 0, + }); + await waitForRest(box); + const tracked = await trackMarker(page, await cast.stop()); + const proof = busyWindowProof(tracked, window, {axis: 'y'}); + testInfo.annotations.push({ + type: 'busy-main-thread pending animation', + description: proof.summary, + }); + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.monotonic).toBe(true); + }); + + test('in the WebKit container, compiled shaders outlive the run', async ({ + browser, + }, testInfo) => { + // The official Playwright image keeps its browsers here. + test.skip( + testInfo.project.name !== 'webkit' || + process.env.PLAYWRIGHT_BROWSERS_PATH !== '/ms-playwright', + 'runs inside webkit-container.mjs', + ); + // A cache inside the container dies with it: every run then compiled each + // shader again, 450–700 ms in the first frame of a blurred layer. + const dir = process.env.MESA_SHADER_CACHE_DIR; + expect(dir, 'MESA_SHADER_CACHE_DIR names the host-kept cache').toBeTruthy(); + // Written on the host by the script before the container starts. + expect( + fs.readFileSync(path.join(dir ?? '', 'ASTRYX-SHADER-CACHE'), 'utf8'), + ).toContain('playwright'); + // Mesa writes what it compiles there once a page draws a blurred layer. + // Earlier runs have filled the cache already, so this run's browser gets + // an empty directory beside them, in the same host-kept mount, and the + // test watches its own writes arrive. + const fresh = fs.mkdtempSync(path.join(dir ?? '', 'self-test-')); + try { + const own = await browser.browserType().launch({ + env: {...process.env, MESA_SHADER_CACHE_DIR: fresh}, + }); + try { + const page = await own.newPage(); + await page.setContent( + '
', + ); + await page.evaluate( + () => + document + .getElementById('glass') + ?.animate([{scale: '1'}, {scale: '0.9'}], {duration: 200}) + .finished, + ); + } finally { + await own.close(); + } + expect(fs.readdirSync(fresh, {recursive: true})).not.toEqual([]); + } finally { + fs.rmSync(fresh, {recursive: true, force: true}); + } + }); + + test('rect sampling reads one rect per animation frame while the main thread is free', async ({ + page, + browserName, + }) => { + await fixture(page); + const box = page.locator('#box'); + await box.evaluate(el => { + el.animate( + [{transform: 'translateX(0px)'}, {transform: 'translateX(200px)'}], + {duration: 600, easing: 'linear'}, + ); + }); + const samples = await sampleRects(box, 400); + const xs = samples.map(s => s.box?.x ?? NaN); + const gaps = samples + .slice(1) + .map((s, i) => s.t - samples[i].t) + .sort((a, b) => a - b); + const cadence = gaps[Math.floor(gaps.length / 2)]; + expect(cadence).toBeLessThan(34); + expect(samples.length).toBeGreaterThanOrEqual(10); + if (browserName === 'chromium') { + // One sample per presented frame at 60 Hz. (Headless WebKit's animation + // frames are irregular, so only the cadence is checked there.) + expect(samples.length).toBeGreaterThanOrEqual(20); + } + expect(xs.every((x, i) => i === 0 || x >= xs[i - 1])).toBe(true); + expect(xs.at(-1) ?? 0).toBeGreaterThan(xs[0] + 80); + }); + + test('the trace counts no Layout for a transform animation and one per frame for `top`', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await fixture(page); + const box = page.locator('#box'); + // Count the animation's steady middle: starting and ending it may lay out once each. + const animate = async (property: 'transform' | 'top') => { + let started = 0; + const trace = await traceRendering(page, async () => { + started = await box.evaluate((el, prop) => { + const keyframes = + prop === 'transform' + ? [ + {transform: 'translateY(0px)'}, + {transform: 'translateY(200px)'}, + ] + : [{top: '360px'}, {top: '560px'}]; + el.animate(keyframes, {duration: 600}); + return performance.timeOrigin + performance.now(); + }, property); + await page.waitForTimeout(700); + }); + return countBetween(trace, started + 100, started + 500); + }; + const transform = await animate('transform'); + const top = await animate('top'); + expect(transform.Layout ?? 0).toBe(0); + expect(transform.Paint ?? 0).toBe(0); + // ~24 frames in 400 ms: one layout per frame. + expect(top.Layout ?? 0).toBeGreaterThan(18); + }); + + test("the trace counts no Layout from the harness's own animation recorder, which stands down while it runs", async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + // The recorder runs, as in any spec that waited for an animation first. + await waitForRest(pop); + const watch = await watchAnimations(pop); + const middle = async (askEveryFrame: boolean) => { + let started = 0; + const trace = await traceRendering(page, async () => { + started = await pop.evaluate((el, every) => { + const w = window as unknown as {stop?: boolean}; + w.stop = false; + if (every) { + const ask = () => { + if (!w.stop) { + requestAnimationFrame(ask); + } + }; + requestAnimationFrame(ask); + } + el.animate( + [ + {transform: 'translateY(0px)', opacity: 1}, + {transform: 'translateY(100px)', opacity: 0.5}, + ], + {duration: 600}, + ); + return performance.timeOrigin + performance.now(); + }, askEveryFrame); + await page.waitForTimeout(700); + await page.evaluate(() => { + (window as unknown as {stop?: boolean}).stop = true; + }); + }); + // The steady middle: starting and ending it may lay out once each. + return countBetween(trace, started + 100, started + 500); + }; + expect((await middle(false)).Layout ?? 0).toBe(0); + // Control: a page that asks for a frame every frame lays it out in each. + expect((await middle(true)).Layout ?? 0).toBeGreaterThan(18); + // Nothing the traces ran went unrecorded. + const started = await watch.started(); + expect(started.map(a => a.durationMs)).toEqual([600, 600]); + }); + + test('a waitForRest awaited inside a traced body stands down too, so the trace counts no Layout from it', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + await waitForRest(pop); + let started = 0; + const trace = await traceRendering(page, async () => { + started = await pop.evaluate(el => { + el.animate( + [ + {transform: 'translateY(0px)', opacity: 1}, + {transform: 'translateY(100px)', opacity: 0.5}, + ], + {duration: 600}, + ); + return performance.timeOrigin + performance.now(); + }); + // A per-frame wait here would make every frame a main-thread frame. + await waitForRest(pop); + }); + expect(countBetween(trace, started + 100, started + 500).Layout ?? 0).toBe( + 0, + ); + // It still waited for the curve to end. + expect(trace.window.to - started).toBeGreaterThanOrEqual(590); + }); + + test('a waitForAnimation or watch.started() awaited inside a traced body stands down too, so the trace counts no Layout from it', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + await waitForRest(pop); + const watch = await watchAnimations(pop); + let started = 0; + let waited: {durationMs: number}[] = []; + let watched: {durationMs: number}[] = []; + const trace = await traceRendering(page, async () => { + started = await pop.evaluate(el => { + el.animate( + [ + {transform: 'translateY(0px)', opacity: 1}, + {transform: 'translateY(100px)', opacity: 0.5}, + ], + {duration: 600}, + ); + return performance.timeOrigin + performance.now(); + }); + // Both poll until 450 ms of the curve has played, across the steady + // middle; a read of the page's animations each time would lay it out. + waited = await waitForAnimation(pop, {minProgressMs: 450}); + watched = await watch.started({minProgressMs: 450}); + await page.waitForTimeout(200); + }); + expect(countBetween(trace, started + 100, started + 500).Layout ?? 0).toBe( + 0, + ); + // Each still answered with the animation once 450 ms of it had played. + expect(waited.map(a => a.durationMs)).toEqual([600]); + expect(watched.map(a => a.durationMs)).toEqual([600]); + expect(trace.window.to - started).toBeGreaterThanOrEqual(590); + }); + + test('a waitForRest awaited inside a traced body waits for an animation the action starts two frames later', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + await waitForRest(pop); + const trace = await traceRendering(page, async () => { + // The action starts the animation two frames later, as a component's + // effect often does. + await pop.evaluate(el => { + requestAnimationFrame(() => + requestAnimationFrame(() => { + (window as unknown as {__started: number}).__started = + performance.timeOrigin + performance.now(); + el.animate( + [ + {transform: 'translateY(0px)'}, + {transform: 'translateY(100px)'}, + ], + {duration: 600}, + ); + }), + ); + }); + await waitForRest(pop); + }); + const started = await page.evaluate( + () => (window as unknown as {__started?: number}).__started ?? null, + ); + // The traced window holds the whole animation. + expect(started).not.toBeNull(); + expect(trace.window.to - (started as number)).toBeGreaterThanOrEqual(590); + }); + + test('a waitForRest awaited inside a traced body waits for a run going backward to its end, and no longer', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + await waitForRest(pop); + // How long a traced waitForRest holds after a 600 ms move starts. + const traced = async (reverse: boolean, endDelay = 0) => { + let started = 0; + const trace = await traceRendering(page, async () => { + started = await pop.evaluate( + (el, {reverse, endDelay}) => { + const run = el.animate( + [ + {transform: 'translateY(0px)'}, + {transform: 'translateY(100px)'}, + ], + {duration: 600, endDelay}, + ); + if (reverse) { + // Its current time falls from its end time to 0. + run.reverse(); + } + return performance.timeOrigin + performance.now(); + }, + {reverse, endDelay}, + ); + await waitForRest(pop); + }); + await waitForRest(pop); + return trace.window.to - started; + }; + // Reversed with a 200 ms end delay: the move plays from 200 to 800 ms. + expect(await traced(true, 200)).toBeGreaterThanOrEqual(790); + // A reversed run is waited for as long as the same run forward, not + // twice as long. + const forward = await traced(false); + const backward = await traced(true); + expect(backward).toBeGreaterThanOrEqual(590); + expect(backward).toBeLessThan(forward * 1.5); + }); + + test('a waitForRest awaited inside a traced body runs on the real timers, so a frozen timer gate cannot hold it', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + const pop = page.locator('#pop'); + await waitForRest(pop); + await installTimerGate(page); + await freezeTimers(page); + let outcome = ''; + await traceRendering(page, async () => { + await pop.evaluate(el => { + el.animate( + [{transform: 'translateY(0px)'}, {transform: 'translateY(100px)'}], + {duration: 300}, + ); + }); + // Held by the gate, the wait would neither resolve nor reach its + // deadline: the race reads that as "held". + outcome = await Promise.race([ + waitForRest(pop, {timeout: 2000}).then( + () => 'rested', + () => 'rejected', + ), + new Promise(resolve => setTimeout(() => resolve('held'), 4000)), + ]); + }); + await thawTimers(page); + expect(outcome).toBe('rested'); + }); + + test('inside a trace, waitForRest rests beside a scroll-timeline animation', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await fixture(page); + const scroller = page.locator('#scroller'); + await waitForRest(scroller); + let outcome = ''; + await traceRendering(page, async () => { + // A row driven by its scroller's timeline: it moves only on scroll. + await scroller.evaluate(el => { + const row = el.firstElementChild as HTMLElement; + row.animate([{opacity: 0.5}, {opacity: 1}], { + timeline: new ScrollTimeline({source: el}), + }); + }); + outcome = await waitForRest(scroller, {timeout: 2000}).then( + () => 'rested', + (error: Error) => error.message, + ); + }); + expect(outcome).toBe('rested'); + }); + + test('a CSS transition and animation that start and end inside a trace are reported, and lay out nothing in between', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await page.route('http://anchored.test/', route => + route.fulfill({contentType: 'text/html', body: ANCHORED}), + ); + await page.goto('http://anchored.test/'); + await page.addStyleTag({ + content: + '@keyframes harness-fade { from { opacity: 1 } to { opacity: 0.6 } }', + }); + const pop = page.locator('#pop'); + // The recorder runs, and a watch is taken before the trace. + await waitForRest(pop); + const watch = await watchAnimations(pop); + let started = 0; + const trace = await traceRendering(page, async () => { + started = await pop.evaluate(el => { + const pop = el as HTMLElement; + pop.style.transition = 'transform 400ms linear'; + void pop.offsetWidth; + pop.style.transform = 'translateY(100px)'; + pop.style.animation = 'harness-fade 400ms linear'; + return performance.timeOrigin + performance.now(); + }); + await page.waitForTimeout(600); + }); + // Its reads at the two starts fall before the steady middle. + expect(countBetween(trace, started + 100, started + 300).Layout ?? 0).toBe( + 0, + ); + const ended = await watch.started({timeout: 1_000}); + expect(ended.map(a => a.kind).sort()).toEqual([ + 'CSSAnimation', + 'CSSTransition', + ]); + expect(ended.every(a => a.finishedAt != null)).toBe(true); + }); + + test('the compositor record refuses another open page, whose frames it could not tell apart', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'the compositor record is a Chromium trace (README: engine limits)', + ); + await fixture(page); + // Another page animating: its presented frames would be credited to + // this one, so the record refuses to start. + const other = await page.context().newPage(); + await other.setContent( + '
', + ); + await expect(startCompositorTrace(page)).rejects.toThrow( + /another page is open as the trace starts/, + ); + await other.close(); + // A page opened mid-trace: stop() refuses too, whether it is still open + // or closed again before the record stops. + const record = await startCompositorTrace(page); + const late = await page.context().newPage(); + await expect(record.stop()).rejects.toThrow( + /another page opened during the record/, + ); + await late.close(); + const brief = await startCompositorTrace(page); + const spinner = await page.context().newPage(); + await spinner.setContent( + '
', + ); + await page.waitForTimeout(200); + await spinner.close(); + await expect(brief.stop()).rejects.toThrow( + /another page opened during the record/, + ); + // In another context too. + const browser = page.context().browser(); + if (browser != null) { + const elsewhere = await startCompositorTrace(page); + const context = await browser.newContext(); + await (await context.newPage()).close(); + await context.close(); + await expect(elsewhere.stop()).rejects.toThrow( + /another page opened during the record/, + ); + } + // Alone, it records. + const alone = await startCompositorTrace(page); + await page.waitForTimeout(100); + await expect(alone.stop()).resolves.toBeInstanceOf(Array); + }); + + test('the long-task log fails closed when it never ran on the document', async ({ + page, + }) => { + // No installLongTaskLog before navigating: an empty answer would read as + // "no long task", so it throws instead. + await fixture(page); + await expect(readLongTasks(page)).rejects.toThrow( + /not running on this document/, + ); + }); + + test("the long-task log records the page's own long task but not a DevTools evaluate, which the trace reports apart", async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'the Long Tasks API and rendering traces are Chromium-only', + ); + await installLongTaskLog(page); + await fixture(page); + // The page's own task: a timer that spins 120 ms. + const own = await page.evaluate( + () => + new Promise<{from: number; to: number}>(resolve => { + setTimeout(() => { + const from = performance.timeOrigin + performance.now(); + const end = performance.now() + 120; + while (performance.now() < end) { + // spin + } + resolve({from, to: performance.timeOrigin + performance.now()}); + }, 0); + }), + ); + await frames(page, 2); + const recorded = await readLongTasks(page, own); + expect(recorded).toHaveLength(1); + expect(recorded[0].durationMs).toBeGreaterThanOrEqual(110); + // A DevTools evaluate that spins 300 ms: the trace reports it apart from + // the page's long tasks, and the log never sees it. + let block = {from: 0, to: 0}; + const trace = await traceRendering(page, async () => { + block = await blockMainThread(page, 300); + }); + await frames(page, 2); + const inBlock = (t: {start: number}) => + t.start < block.to && t.start >= block.from - 20; + expect( + trace.devtoolsTasks.some(t => t.durationMs >= 290 && inBlock(t)), + ).toBe(true); + expect(trace.longTasks.filter(inBlock)).toEqual([]); + expect(await readLongTasks(page, block)).toEqual([]); + }); + + test("a harness read inside a trace is not the page's long task, while a test's own evaluate is, whether it calls the page directly or sets it off", async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + // Enough animated rows that describing their keyframes takes a DevTools + // task over 50 ms. + const rows = Array.from( + {length: 1200}, + (_, i) => `
row ${i}
`, + ).join(''); + await page.setContent( + `
${rows}
`, + ); + const slow = page.locator('#slow'); + await slow.evaluate(el => + el.addEventListener('click', () => { + const end = performance.now() + 60; + while (performance.now() < end) { + // spin + } + }), + ); + // Page code a test can call directly, as through a story's own API. + await page.evaluate(() => { + (window as unknown as {pageWork: (ms: number) => void}).pageWork = ( + ms: number, + ) => { + const end = performance.now() + ms; + while (performance.now() < end) { + // spin + } + }; + }); + await frames(page, 3); + const now = () => + page.evaluate(() => performance.timeOrigin + performance.now()); + const read = {from: 0, to: 0}; + const click = {from: 0, to: 0}; + const direct = {from: 0, to: 0}; + let timer = {from: 0, to: 0}; + const trace = await traceRendering(page, async () => { + read.from = await now(); + await waitForAnimation(page.locator('#list')); + read.to = await now(); + // On a slow machine each frame of 1,200 running rows is a long task of + // the page's own; the windows below hold only their own work. + await page.evaluate(() => document.getElementById('list')?.remove()); + await frames(page, 2); + // The page's own task: a timer that spins 60 ms. + timer = await page.evaluate( + () => + new Promise<{from: number; to: number}>(resolve => { + setTimeout(() => { + const from = performance.timeOrigin + performance.now(); + const end = performance.now() + 60; + while (performance.now() < end) { + // spin + } + resolve({from, to: performance.timeOrigin + performance.now()}); + }, 0); + }), + ); + // Page script run inside a DevTools evaluate: a 60 ms click handler. + click.from = await now(); + await slow.evaluate(el => (el as HTMLElement).click()); + click.to = await now(); + // Page code a test's evaluate calls directly: 60 ms of the page's work. + direct.from = await now(); + await page.evaluate(() => + (window as unknown as {pageWork: (ms: number) => void}).pageWork(60), + ); + direct.to = await now(); + }); + const within = (w: {from: number; to: number}) => (t: {start: number}) => + t.start >= w.from - 20 && t.start < w.to; + // The read itself is set apart. The rows' frames around it may be long + // tasks too, the page's own: they stay in longTasks. + expect(trace.devtoolsTasks.some(within(read))).toBe(true); + expect( + trace.longTasks.some(t => within(timer)(t) && t.durationMs >= 55), + ).toBe(true); + expect( + trace.longTasks.some(t => within(click)(t) && t.durationMs >= 55), + ).toBe(true); + expect( + trace.longTasks.some(t => within(direct)(t) && t.durationMs >= 55), + ).toBe(true); + expect(trace.devtoolsTasks.filter(within(direct))).toEqual([]); + }); +}); + test.describe('curve sampling', () => { test('a CSS transition samples onto its cubic-bezier to 0.05 px', async ({ page, @@ -961,6 +1905,91 @@ test.describe('animation state', () => { expect(started.map(a => a.durationMs)).toEqual([300]); }); + test('while a rendering trace runs the recorder reads only at CSS starts, and catches up after it', async ({ + page, + }) => { + await fixture(page); + const box = page.locator('#box'); + const watch = await watchAnimations(box); + // Counts the recorder's reads of the page's animations (each one flushes + // style): the whole document's, and one element's. Then flags a trace, as + // traceRendering does. + await page.evaluate(() => { + const w = window as unknown as { + __reads: number; + __elementReads: number; + __harnessRenderingTrace?: boolean; + }; + w.__reads = 0; + w.__elementReads = 0; + const read = document.getAnimations.bind(document); + document.getAnimations = () => { + w.__reads += 1; + return read(); + }; + const readElement = Element.prototype.getAnimations; + Element.prototype.getAnimations = function ( + this: Element, + ...args: Parameters + ) { + w.__elementReads += 1; + return readElement.apply(this, args); + }; + w.__harnessRenderingTrace = true; + }); + // During the trace: a CSS transition and a CSS animation that both end + // inside it, and a script animation still running after it. + await page.addStyleTag({ + content: + '@keyframes harness-fade { from { opacity: 1 } to { opacity: 0.6 } }', + }); + await box.evaluate(el => { + const box = el as HTMLElement; + box.style.transition = 'transform 120ms linear'; + void box.offsetWidth; + box.style.transform = 'translateX(100px)'; + box.style.animation = 'harness-fade 120ms linear'; + box.animate([{scale: '1'}, {scale: '0.9'}], {duration: 600}); + }); + await page.waitForTimeout(300); + const reads = () => + page.evaluate(() => (window as unknown as {__reads: number}).__reads); + // No frame loop: no document-wide read. One element read per CSS start + // event (transitionrun, transitionstart, animationstart), not per frame. + expect(await reads()).toBe(0); + expect( + await page.evaluate( + () => (window as unknown as {__elementReads: number}).__elementReads, + ), + ).toBeLessThanOrEqual(3); + // The trace ends: traceRendering clears the flag and resumes the recorder, + // which reads again, every frame. + await page.evaluate(() => { + const w = window as unknown as { + __harnessRenderingTrace?: boolean; + __harnessAnimations: {resume: () => void}; + }; + w.__harnessRenderingTrace = false; + w.__harnessAnimations.resume(); + }); + await frames(page, 3); + expect(await reads()).toBeGreaterThan(2); + // All three are reported, the two CSS ones that ended inside the trace + // with the time they finished. + await waitForRest(box); + const started = await watch.started(); + expect(started.map(a => a.kind).sort()).toEqual([ + 'Animation', + 'CSSAnimation', + 'CSSTransition', + ]); + expect( + started + .filter(a => a.kind !== 'Animation') + .map(a => a.finishedAt != null), + ).toEqual([true, true]); + }); + test('a wait answers with the animation the action started, not one that finished before it', async ({ page, }) => { @@ -1025,6 +2054,28 @@ test.describe('animation state', () => { await box.evaluate(el => el.getAnimations().forEach(a => a.finish())); }); + test('inside a rendering trace too, a watch taken right after a style change does not report that change', async ({ + page, + browserName, + }) => { + test.skip(browserName !== 'chromium', 'rendering traces are Chromium-only'); + await fixture(page); + const box = page.locator('#box'); + await waitForRest(box); + await traceRendering(page, async () => { + // The step: a transition the next style read creates. + await box.evaluate(el => { + el.style.transition = 'opacity 300ms linear'; + el.style.opacity = '0.5'; + }); + const watch = await watchAnimations(box); + await expect(watch.started({timeout: 200})).rejects.toThrow( + /waitForAnimation/, + ); + }); + await box.evaluate(el => el.getAnimations().forEach(a => a.finish())); + }); + test('an exit whose element the page removes at its end is still reported', async ({ page, }) => { @@ -1076,6 +2127,43 @@ test.describe('animation state', () => { } }); + test('inside a rendering trace too, a wait on an element no longer in the document rejects as missing', async ({ + page, + browserName, + }) => { + test.skip( + browserName !== 'chromium', + 'rendering traces are Chromium-only (README: engine limits)', + ); + await fixture(page); + await page.evaluate(() => { + const gone = document.createElement('div'); + gone.id = 'gone'; + document.body.append(gone); + }); + const gone = page.locator('#gone'); + const watch = await watchAnimations(gone); + await page.evaluate(() => document.getElementById('gone')?.remove()); + const outcomes: string[] = []; + await traceRendering(page, async () => { + for (const wait of [ + () => watch.started({timeout: 300}), + () => waitForAnimation(gone, {timeout: 300}), + ]) { + outcomes.push( + await wait().then( + () => 'resolved', + (error: Error) => error.message, + ), + ); + } + }); + expect(outcomes).toHaveLength(2); + for (const outcome of outcomes) { + expect(outcome).toMatch(/not in the document/); + } + }); + test('a watch reports only what started after it', async ({page}) => { await fixture(page); const box = page.locator('#box'); @@ -1429,6 +2517,34 @@ test.describe('storyboards', {tag: '@storyboard'}, () => { } }); + test("the harness's own waits run on the real timers: a frozen timer gate holds neither blockMainThread's block nor sampleRects", async ({ + page, + }) => { + await fixture(page); + await installTimerGate(page); + const box = page.locator('#box'); + await box.evaluate(el => { + el.animate( + [{transform: 'translateX(0px)'}, {transform: 'translateX(200px)'}], + {duration: 1_500, easing: 'linear'}, + ); + }); + await freezeTimers(page); + // Settled, or still held after 5 s. + const within = (work: Promise) => + Promise.race([ + work.then(() => 'settled'), + new Promise(resolve => + setTimeout(() => resolve('held'), 5_000), + ), + ]); + expect( + await within(blockMainThread(page, 100, {during: box, timeout: 2_000})), + ).toBe('settled'); + expect(await within(sampleRects(box, 200))).toBe('settled'); + await thawTimers(page); + }); + test('the timer gate holds a component timer until the filmstrip is done', async ({ page, }, testInfo) => { @@ -1614,4 +2730,58 @@ test.describe('storyboards', {tag: '@storyboard'}, () => { expect(state).toMatchObject({timer: false, frame: false}); expect(state.heldFor).toBeGreaterThanOrEqual(140); }); + + test('a cast filmstrip keeps exactly the presented frames of its window, labelled from its zero', async ({ + page, + }, testInfo) => { + await fixture(page); + const box = page.locator('#box'); + // JPEG frames (WebKit's are always JPEG) take the conversion path. + const cast = await startScreencast(page, {format: 'jpeg'}); + await box.evaluate(el => + el.animate( + [{transform: 'translateX(0px)'}, {transform: 'translateX(200px)'}], + {duration: 600}, + ), + ); + await page.waitForTimeout(500); + const presented = await cast.stop(); + expect(presented.length).toBeGreaterThan(8); + // A window of the middle frames, zeroed on one of them. + const zero = presented[2].t; + const from = presented[3].t; + const to = presented[presented.length - 3].t; + const inside = presented.filter(f => f.t >= from && f.t <= to); + const seen: number[] = []; + const film = await castStoryboard(page, presented, { + name: 'harness-self-test-cast', + zero, + from, + to, + data: t => { + seen.push(t); + return {t}; + }, + }); + expect(film.frames.map(f => f.t)).toEqual(inside.map(f => f.t - zero)); + expect(film.frames.map(f => f.label)).toEqual( + inside.map(f => `t=${(f.t - zero).toFixed(1)} ms`), + ); + expect(seen).toEqual(inside.map(f => f.t)); + // Every cell is a PNG, whatever the screencast delivered. + for (const frame of film.frames) { + expect([...frame.png.subarray(0, 4)]).toEqual([0x89, 0x50, 0x4e, 0x47]); + } + const files = await film.write(testInfo); + expect(files['sheet.png']).toMatch(/sheet\.png$/); + // A window with no presented frame in it films nothing. + const empty = await castStoryboard(page, presented, { + name: 'harness-self-test-cast-empty', + zero, + from: to + 1, + to: to + 2, + }); + expect(empty.frames).toEqual([]); + await expect(empty.write(testInfo)).rejects.toThrow(/no frames/); + }); }); diff --git a/internal/interaction-harness/src/index.ts b/internal/interaction-harness/src/index.ts index 537bfc4d4beca..7a45ff45dfe41 100644 --- a/internal/interaction-harness/src/index.ts +++ b/internal/interaction-harness/src/index.ts @@ -12,10 +12,13 @@ * SYNC: When a module is added, export it here and describe it in README.md. */ +export * from './busy'; export * from './curves'; +export * from './frames'; export * from './math'; export * from './output'; export * from './scripts'; export * from './story'; export * from './storyboard'; export * from './touch'; +export * from './trace'; diff --git a/internal/interaction-harness/src/story.ts b/internal/interaction-harness/src/story.ts index 598f75d852cfe..cb57d59b50be2 100644 --- a/internal/interaction-harness/src/story.ts +++ b/internal/interaction-harness/src/story.ts @@ -6,7 +6,8 @@ * static server over a built Storybook * @output `phone` (the touch-phone browser context every interaction proof * runs in), `withStorybook` (serve the built Storybook for a spec file), and - * `openStory` (load one story and wait until it has rendered). + * `openStory` (load one story, with Storybook's highlight module off, and + * wait until it has rendered). * @position Stories are the reproduction path, as for every other browser * check in this repository: a scenario used for verification is checked in, * not built in a one-off page. @@ -59,13 +60,47 @@ export function withStorybook(test: TestType): () => string { }; } -/** Loads a story in the page and waits for its first render, its fonts and two frames. */ +/** Pages whose Storybook highlight module `openStory` turned off. */ +const highlightOff = new WeakSet(); + +export interface OpenStoryOptions { + /** + * Keeps Storybook's highlight module on. It is off by default: when on, it + * observes `#storybook-root` and, on every insertion, reads the computed + * style of every element on the page inside the inserting task, so a proof + * would measure Storybook as well as the component. + */ + readonly storybookHighlight?: boolean; +} + +/** + * Loads a story in the page and waits for its first render, its fonts and two + * frames. Storybook's highlight module is off (see `OpenStoryOptions`), and + * that is checked: its root element must be absent once the story renders. + */ export async function openStory( page: Page, origin: string, storyId: string, args?: Readonly>, + {storybookHighlight = false}: OpenStoryOptions = {}, ): Promise { + if (storybookHighlight) { + if (highlightOff.has(page)) { + throw new Error( + "openStory: this page already turned Storybook's highlight module off; open the story in a new page to keep it on", + ); + } + } else if (!highlightOff.has(page)) { + highlightOff.add(page); + // Storybook's highlight module returns at once when this is set before + // its preview runs. + await page.addInitScript(() => { + ( + globalThis as unknown as {__STORYBOOK_HIGHLIGHT_INITIALIZED?: boolean} + ).__STORYBOOK_HIGHLIGHT_INITIALIZED = true; + }); + } const params = new URLSearchParams({id: storyId, viewMode: 'story'}); if (args != null) { params.set( @@ -86,6 +121,14 @@ export async function openStory( requestAnimationFrame(() => requestAnimationFrame(() => resolve())), ); }); + if ( + !storybookHighlight && + (await page.locator('#storybook-highlights-root').count()) > 0 + ) { + throw new Error( + "openStory: Storybook's highlight module is on (#storybook-highlights-root exists), so every insertion would also read the style of every element on the page; this Storybook may no longer read __STORYBOOK_HIGHLIGHT_INITIALIZED", + ); + } } /** Resolves after `count` animation frames have been presented in the page. */ @@ -110,8 +153,9 @@ export async function frames(page: Page, count = 2): Promise { // A held callback is queued under its timer or frame id, so the page's own // clearTimeout and cancelAnimationFrame still cancel it while frozen; a held -// frame callback gets the time it actually runs at. `frame` stays the real -// requestAnimationFrame, for the harness's own waits. +// frame callback gets the time it actually runs at. `frame` and `timeout` +// stay the real requestAnimationFrame and setTimeout, for the harness's own +// waits. const TIMER_GATE = `(() => { if (window.__harnessTimerGate) return; const realTimeout = window.setTimeout.bind(window); @@ -119,7 +163,7 @@ const TIMER_GATE = `(() => { const realInterval = window.setInterval.bind(window); const realFrame = window.requestAnimationFrame.bind(window); const realCancelFrame = window.cancelAnimationFrame.bind(window); - const gate = {frozen: false, queue: [], frame: realFrame}; + const gate = {frozen: false, queue: [], frame: realFrame, timeout: realTimeout}; window.__harnessTimerGate = gate; const drop = key => { gate.queue = gate.queue.filter(entry => entry.key !== key); }; window.setTimeout = (fn, ms, ...args) => { diff --git a/internal/interaction-harness/src/storyboard.ts b/internal/interaction-harness/src/storyboard.ts index 2dc22fdc8cf04..a49207c4f81c4 100644 --- a/internal/interaction-harness/src/storyboard.ts +++ b/internal/interaction-harness/src/storyboard.ts @@ -336,7 +336,8 @@ export interface TouchStoryboardOptions extends StoryboardOptions { * photograph per event, about 100 ms between moves instead of the script's * 16.7. Its release velocity is that much lower, so a flick that dismisses * when played can settle back when filmed: a storyboard cannot show such a - * gesture; record it in real time instead. + * gesture; record it in real time instead (`startScreencast`, + * `castStoryboard`). */ export async function touchStoryboard( page: Page, @@ -364,3 +365,65 @@ export async function touchStoryboard( }); return storyboard; } + +/** + * A filmstrip of screencast frames (real presented frames, e.g. the frames of + * a busy-main-thread window), labelled with their time relative to `zero`. + * Evidence for what the compositor presented, not a deterministic storyboard. + */ +export async function castStoryboard( + page: Page, + frames: readonly { + readonly t: number; + readonly data: string; + readonly format: 'png' | 'jpeg'; + }[], + // A cast is already the screen's frames: no clip is applied to them. + options: Omit & { + readonly zero: number; + readonly from?: number; + readonly to?: number; + readonly data?: (t: number) => Readonly>; + }, +): Promise { + const storyboard = new Storyboard(page, options); + for (const frame of frames) { + if (options.from != null && frame.t < options.from) { + continue; + } + if (options.to != null && frame.t > options.to) { + continue; + } + const png = + frame.format === 'png' + ? Buffer.from(frame.data, 'base64') + : await toPng(page, frame.data); + const t = frame.t - options.zero; + storyboard.frames.push({ + label: `t=${t.toFixed(1)} ms`, + t, + data: options.data?.(frame.t) ?? {}, + png, + }); + } + return storyboard; +} + +async function toPng(page: Page, jpegBase64: string): Promise { + const scratch = await page.context().newPage(); + try { + const url = await scratch.evaluate(async data => { + const bitmap = await createImageBitmap( + await (await fetch(`data:image/jpeg;base64,${data}`)).blob(), + ); + const canvas = document.createElement('canvas'); + canvas.width = bitmap.width; + canvas.height = bitmap.height; + canvas.getContext('2d')?.drawImage(bitmap, 0, 0); + return canvas.toDataURL('image/png'); + }, jpegBase64); + return Buffer.from(url.slice(url.indexOf(',') + 1), 'base64'); + } finally { + await scratch.close(); + } +} diff --git a/internal/interaction-harness/src/storybook.interaction.browser.spec.ts b/internal/interaction-harness/src/storybook.interaction.browser.spec.ts new file mode 100644 index 0000000000000..a711feb62cb84 --- /dev/null +++ b/internal/interaction-harness/src/storybook.interaction.browser.spec.ts @@ -0,0 +1,104 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file storybook.interaction.browser.spec.ts + * @input The built Storybook (`withStorybook`) and one of its stories + * @output `openStory` turns Storybook's highlight module off, so an insertion + * in a story carries no `getComputedStyle` scan of the page; with the module + * on (the control), every insertion carries one. Both engines; Chromium's + * trace also counts the style recalculations the scan forces + * @position Self-test for story.ts; it needs the built Storybook, as every + * component proof does + */ + +import {expect, test, type Page} from '@playwright/test'; +import {openStory, withStorybook} from './story'; +import {countBetween, traceRendering} from './trace'; + +const storybook = withStorybook(test); + +/** Counts the page's `getComputedStyle` calls, from before Storybook loads. */ +function countStyleReads(): void { + const w = window as unknown as {__styleReads: number}; + w.__styleReads = 0; + const read = window.getComputedStyle.bind(window); + window.getComputedStyle = ((element: Element, pseudo?: string | null) => { + w.__styleReads += 1; + return read(element, pseudo); + }) as typeof window.getComputedStyle; +} + +/** + * Inserts an exit copy of the story's first element into `#storybook-root`, + * as an exit animation's copy is, lets the insertion's mutation microtasks run, + * and reports what ran between its marks. + */ +async function insertCopy(page: Page) { + return page.evaluate(async () => { + const w = window as unknown as {__styleReads: number}; + const root = document.getElementById('storybook-root') as HTMLElement; + const reads = w.__styleReads; + const from = + performance.timeOrigin + performance.mark('copy:start').startTime; + const copy = (root.firstElementChild as HTMLElement).cloneNode( + true, + ) as HTMLElement; + copy.setAttribute('aria-hidden', 'true'); + root.appendChild(copy); + // A MutationObserver's callback is a microtask the insertion queued: + // these run after it. + await Promise.resolve(); + await Promise.resolve(); + const to = performance.timeOrigin + performance.mark('copy:end').startTime; + const scanned = w.__styleReads - reads; + const elements = document.body.querySelectorAll('*').length; + copy.remove(); + return {from, to, scanned, elements}; + }); +} + +test("openStory turns Storybook's highlight module off: an insertion in a story carries no getComputedStyle scan of the page", async ({ + page, + browserName, +}) => { + // The insertion's reads in every engine, and in Chromium (whose trace the + // harness reads) the style recalculations between its marks. + const measure = async (p: Page) => { + if (browserName !== 'chromium') { + return {copy: await insertCopy(p), styleRecalc: null}; + } + let copy = {from: 0, to: 0, scanned: 0, elements: 0}; + const trace = await traceRendering(p, async () => { + copy = await insertCopy(p); + }); + return { + copy, + styleRecalc: + countBetween(trace, copy.from, copy.to).UpdateLayoutTree ?? 0, + }; + }; + // The harness default. + await page.addInitScript(countStyleReads); + await openStory(page, storybook(), 'core-button--primary'); + expect(await page.locator('#storybook-highlights-root').count()).toBe(0); + const off = await measure(page); + expect(off.copy.scanned).toBe(0); + if (off.styleRecalc != null) { + expect(off.styleRecalc).toBe(0); + } + + // Control: Storybook's module kept on reads the style of every element on + // the page inside the inserting task. + const control = await page.context().newPage(); + await control.addInitScript(countStyleReads); + await openStory(control, storybook(), 'core-button--primary', undefined, { + storybookHighlight: true, + }); + expect(await control.locator('#storybook-highlights-root').count()).toBe(1); + const on = await measure(control); + expect(on.copy.scanned).toBeGreaterThanOrEqual(on.copy.elements); + if (on.styleRecalc != null) { + expect(on.styleRecalc).toBeGreaterThan(0); + } + await control.close(); +}); diff --git a/internal/interaction-harness/src/trace.ts b/internal/interaction-harness/src/trace.ts new file mode 100644 index 0000000000000..ce6a3d853d2cd --- /dev/null +++ b/internal/interaction-harness/src/trace.ts @@ -0,0 +1,466 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file trace.ts + * @input Uses a Playwright `Page` (Chromium: CDP `Tracing`) and the page's + * PerformanceObserver for long tasks + * @output `traceRendering` — counts of the main thread's rendering work + * (Layout, Paint, UpdateLayoutTree, PrePaint, …) between two marks — and + * `installLongTaskLog` / `readLongTasks`. + * @position The "no Layout/Paint during a transform/opacity animation" proof. + * A compositor animation costs the main thread nothing per frame; one that + * animates a layout property, or writes styles from JavaScript each frame, + * shows a Layout (and usually a Paint) per frame. Chromium only: WebKit + * exposes no comparable trace to Playwright. + * + * Counting is scoped to the renderer main thread that ran the page's own + * `performance.mark` calls, and to the time between the start and end marks, + * so other pages, the browser process and setup work are excluded. + * + * SYNC: Keep RENDERING_EVENTS equal to the README's list. + */ + +import type {CDPSession, Page} from '@playwright/test'; + +/** + * The mark a harness function puts on its own page-side work + * (`console.timeStamp('harness-read')` at the start of the function it + * evaluates in the page), so a trace can tell the harness's reads from page + * code a test's own evaluate runs. + */ +export const HARNESS_READ = 'harness-read'; + +/** Trace events that run page script: a listener, a timer, a frame callback. */ +const PAGE_SCRIPT_EVENTS = new Set([ + 'EventDispatch', + 'FunctionCall', + 'TimerFire', + 'FireAnimationFrame', + 'FireIdleCallback', +]); + +/** Main-thread trace events that are rendering work, by their trace names. */ +export const RENDERING_EVENTS = [ + 'UpdateLayoutTree', // style recalculation + 'Layout', + 'PrePaint', + 'Paint', + 'Layerize', + 'UpdateLayer', + 'ScrollLayer', + 'HitTest', +] as const; + +export interface RenderingTrace { + /** Count per trace event name inside the window, on the page's main thread. */ + readonly counts: Readonly>; + /** Shorthand for the four that matter most. */ + readonly layout: number; + readonly paint: number; + readonly styleRecalc: number; + readonly prePaint: number; + /** Window length, ms. */ + readonly durationMs: number; + /** + * The page's own main-thread tasks longer than 50 ms inside the window (long + * tasks), epoch ms. A task that ran a test's own DevTools command (a + * `page.evaluate`) is one, whether it calls page code directly or sets it + * off; only the harness's own marked reads are in `devtoolsTasks`. + */ + readonly longTasks: readonly { + readonly start: number; + readonly durationMs: number; + }[]; + /** + * Main-thread tasks longer than 50 ms inside the window that ran a DevTools + * command the harness marked as its own read (`HARNESS_READ`) and no page + * script, epoch ms: `waitForAnimation` describing every animation's + * keyframes, or `blockMainThread(page, ms)` without `during`. + */ + readonly devtoolsTasks: readonly { + readonly start: number; + readonly durationMs: number; + }[]; + /** Every counted rendering event with its epoch-ms time, for sub-window counts. */ + readonly events: readonly {readonly name: string; readonly t: number}[]; + /** The window, epoch ms on the page's clock. */ + readonly window: {readonly from: number; readonly to: number}; +} + +/** Counts rendering events by name inside `[from, to]` (epoch ms). */ +export function countBetween( + trace: RenderingTrace, + from: number, + to: number, +): Record { + const counts: Record = {}; + for (const e of trace.events) { + if (e.t >= from && e.t <= to) { + counts[e.name] = (counts[e.name] ?? 0) + 1; + } + } + return counts; +} + +interface TraceEvent { + name: string; + cat: string; + ph: string; + ts: number; + dur?: number; + pid: number; + tid: number; + args?: Record; +} + +/** + * Traces the page while `body` runs and counts the rendering work its main + * thread did between a start mark (placed just before `body`) and an end mark + * (just after). The harness's animation recorder, `waitForRest`, + * `waitForAnimation` and a watch's `started()` stand down meanwhile, so a body + * may wait with any of them and add no frame of its own, with one gap: on a + * CSS `@keyframes` animation (and, for an unwatched `waitForAnimation`, a + * cancelled CSS transition) the wait reads its timing, which flushes that + * animation's style, so wait on such an animation with `waitForRest`, or + * after the trace. Taking a watch, or calling `waitForRest` while an + * animation already runs on the element, reads its animations once, which + * restyles it: do either before the window a proof counts. + */ +export async function traceRendering( + page: Page, + body: () => Promise, +): Promise { + const cdp: CDPSession = await page.context().newCDPSession(page); + const events: TraceEvent[] = []; + cdp.on('Tracing.dataCollected', ({value}) => { + events.push(...(value as unknown as TraceEvent[])); + }); + const complete = new Promise(resolve => + cdp.once('Tracing.tracingComplete', () => resolve()), + ); + await cdp.send('Tracing.start', { + transferMode: 'ReportEvents', + traceConfig: { + includedCategories: [ + 'devtools.timeline', + 'disabled-by-default-devtools.timeline', + 'blink.user_timing', + 'toplevel', + ], + excludedCategories: ['*'], + }, + }); + const tag = `harness-${Math.random().toString(36).slice(2, 8)}`; + const mark = (name: string) => + page.evaluate( + n => performance.timeOrigin + performance.mark(n).startTime, + name, + ); + // The harness's animation recorder (curves.ts) runs a callback every frame, + // which makes each frame a main-thread one where Blink restyles every running + // animation's target (and lays out an anchored one), and it reads the page's + // animations, which flushes style: exactly what this counts. It stands down + // while this flag is set, a recorder the body itself starts included, and + // resumes once the trace ends. + const standDown = (on: boolean) => + page.evaluate(value => { + const w = window as unknown as { + __harnessRenderingTrace?: boolean; + __harnessAnimations?: {resume?: () => void}; + }; + w.__harnessRenderingTrace = value; + if (!value) { + w.__harnessAnimations?.resume?.(); + } + }, on); + await standDown(true); + let fromEpoch: number; + let toEpoch: number; + try { + fromEpoch = await mark(`${tag}:start`); + await body(); + toEpoch = await mark(`${tag}:end`); + } finally { + // Its catch-up read lands after the end mark, outside the counted window. + // A failure here would hide the body's own error, which is the one to see. + await standDown(false).catch(() => {}); + } + await cdp.send('Tracing.end'); + await complete; + await cdp.detach().catch(() => {}); + + const start = events.find(e => e.name === `${tag}:start`); + const end = events.find(e => e.name === `${tag}:end`); + if (start == null || end == null) { + throw new Error('traceRendering: the window marks were not traced'); + } + const onMain = (e: TraceEvent) => + e.pid === start.pid && + e.tid === start.tid && + e.ts >= start.ts && + e.ts <= end.ts; + // Trace time (µs, monotonic) -> epoch ms, anchored on the start mark. + const toEpochMs = (ts: number) => fromEpoch + (ts - start.ts) / 1000; + const counts: Record = {}; + const counted: {name: string; t: number}[] = []; + for (const e of events) { + if (!onMain(e) || e.ph === 'E') { + continue; + } + if ((RENDERING_EVENTS as readonly string[]).includes(e.name)) { + counts[e.name] = (counts[e.name] ?? 0) + 1; + counted.push({name: e.name, t: toEpochMs(e.ts)}); + } + } + // A task that ran a DevTools command received it over the DevTools + // session's pipe; page script it set off shows as its listener, timer or + // frame callback. + const main = events.filter(e => onMain(e) && e.ph !== 'E'); + const longTasks: {start: number; durationMs: number}[] = []; + const devtoolsTasks: {start: number; durationMs: number}[] = []; + for (const task of main) { + if (task.name !== 'RunTask' || (task.dur ?? 0) <= 50_000) { + continue; + } + const end = task.ts + (task.dur ?? 0); + const inside = main.filter( + e => e !== task && e.ts >= task.ts && e.ts <= end, + ); + const devtools = inside.some( + e => + e.name === 'Receive mojo message' && + ( + e.args?.chrome_mojo_event_info as + {mojo_interface_tag?: unknown} | undefined + )?.mojo_interface_tag === 'blink.mojom.DevToolsSession', + ); + // Only the harness's own reads are set apart: a test's evaluate may call + // page code directly, which leaves no listener or timer in the trace. + const harnessRead = inside.some( + e => + e.name === 'TimeStamp' && + (e.args?.data as {message?: unknown} | undefined)?.message === + HARNESS_READ, + ); + const pageScript = inside.some(e => PAGE_SCRIPT_EVENTS.has(e.name)); + (devtools && harnessRead && !pageScript ? devtoolsTasks : longTasks).push({ + start: toEpochMs(task.ts), + durationMs: (task.dur ?? 0) / 1000, + }); + } + return { + counts, + layout: counts.Layout ?? 0, + paint: counts.Paint ?? 0, + styleRecalc: counts.UpdateLayoutTree ?? 0, + prePaint: counts.PrePaint ?? 0, + durationMs: (end.ts - start.ts) / 1000, + longTasks, + devtoolsTasks, + events: counted, + window: {from: fromEpoch, to: toEpoch}, + }; +} + +/** + * The browser's open pages. The compositor record is browser-wide: its + * `Display::FrameDisplayed` events carry no page identity. + */ +function openPages(page: Page): Page[] { + const browser = page.context().browser(); + return browser == null + ? page.context().pages() + : browser.contexts().flatMap(context => context.pages()); +} + +/** + * Watches for a page that opens while the record runs, in any context (it + * may close again before the record stops). Returns whether one opened. + */ +async function watchForOtherPages( + page: Page, +): Promise<{stop(): Promise}> { + let opened = false; + const onPage = (other: Page) => { + if (other !== page) { + opened = true; + } + }; + // The page's own context, through Playwright (a persistent context has no + // browser to ask). + page.context().on('page', onPage); + // Every context, through the browser's target discovery: a page in a + // context created during the record is a new target too. + let targets: CDPSession | null = null; + const browser = page.context().browser(); + if (browser != null) { + try { + targets = await browser.newBrowserCDPSession(); + const {targetInfos} = await targets.send('Target.getTargets'); + const known = new Set(targetInfos.map(info => info.targetId)); + targets.on('Target.targetCreated', ({targetInfo}) => { + if (targetInfo.type === 'page' && !known.has(targetInfo.targetId)) { + opened = true; + } + }); + await targets.send('Target.setDiscoverTargets', {discover: true}); + } catch { + targets = null; + } + } + return { + async stop() { + page.context().off('page', onPage); + await targets?.detach().catch(() => {}); + return opened; + }, + }; +} + +function assertOnlyPage(page: Page, when: string): void { + if (openPages(page).some(other => other !== page && !other.isClosed())) { + throw new Error( + `startCompositorTrace: another page is open ${when}; the compositor record is browser-wide (its frames carry no page identity), so another page's frames would be credited to this one`, + ); + } +} + +/** + * Records when the compositor presents each frame, for `busyWindowProof`'s + * `presented`: a Chromium trace of `Display::FrameDisplayed`, on the page + * clock. Start it with the screencast, before the block, and stop it after. + * Engines without a CDP trace (WebKit) resolve `stop()` to [], and the + * verdict then judges the screencast alone. It runs its own trace, so not + * inside `traceRendering`. The record is browser-wide, so it throws when + * another page is open as it starts or stops, or opened while it ran. + */ +export async function startCompositorTrace( + page: Page, +): Promise<{stop(): Promise}> { + let cdp: CDPSession; + try { + cdp = await page.context().newCDPSession(page); + } catch { + return {stop: async () => []}; + } + try { + assertOnlyPage(page, 'as the trace starts'); + } catch (error) { + await cdp.detach().catch(() => {}); + throw error; + } + const others = await watchForOtherPages(page); + const events: TraceEvent[] = []; + cdp.on('Tracing.dataCollected', ({value}) => { + events.push(...(value as unknown as TraceEvent[])); + }); + const complete = new Promise(resolve => + cdp.once('Tracing.tracingComplete', () => resolve()), + ); + await cdp.send('Tracing.start', { + transferMode: 'ReportEvents', + traceConfig: { + includedCategories: [ + 'disabled-by-default-display.framedisplayed', + 'blink.user_timing', + ], + excludedCategories: ['*'], + }, + }); + const tag = `harness-compositor-${Math.random().toString(36).slice(2, 8)}`; + const epoch = await page.evaluate( + n => performance.timeOrigin + performance.mark(n).startTime, + tag, + ); + return { + async stop() { + await cdp.send('Tracing.end'); + await complete; + await cdp.detach().catch(() => {}); + if (await others.stop()) { + throw new Error( + "startCompositorTrace: another page opened during the record; the compositor record is browser-wide (its frames carry no page identity), so that page's frames would be credited to this one", + ); + } + assertOnlyPage(page, 'as the trace stops'); + const mark = events.find(e => e.name === tag); + if (mark == null) { + throw new Error('startCompositorTrace: the clock mark was not traced'); + } + // Trace time (µs, monotonic) -> epoch ms, anchored on the mark. + return events + .filter(e => e.name === 'Display::FrameDisplayed') + .map(e => epoch + (e.ts - mark.ts) / 1000) + .sort((a, b) => a - b); + }, + }; +} + +const LONG_TASK_SCRIPT = `(() => { + if (window.__harnessLongTasks) return; + const tasks = []; + const log = {tasks, supported: false}; + window.__harnessLongTasks = log; + if (typeof PerformanceObserver === 'undefined' || !(PerformanceObserver.supportedEntryTypes || []).includes('longtask')) return; + try { + new PerformanceObserver(list => { + for (const entry of list.getEntries()) tasks.push({start: performance.timeOrigin + entry.startTime, durationMs: entry.duration, name: entry.name}); + }).observe({type: 'longtask', buffered: true}); + log.supported = true; + } catch {} +})()`; + +/** + * Records every long task (> 50 ms) the page runs (install before navigation; + * Chromium), through the Long Tasks API. That sees the tasks of the page's own + * event loop: a timer, an event handler, `blockMainThread(page, ms, {during})`. + * A DevTools `page.evaluate`, such as `blockMainThread(page, ms)` without + * `during`, is not one of them and is never recorded here; `traceRendering` + * reports it in `devtoolsTasks`, apart from the page's `longTasks`. + */ +export async function installLongTaskLog(page: Page): Promise { + await page.addInitScript(LONG_TASK_SCRIPT); +} + +export interface LongTask { + /** Epoch ms. */ + readonly start: number; + readonly durationMs: number; + readonly name: string; +} + +/** + * Long tasks recorded since navigation, optionally only those overlapping a + * window. Fails closed: it throws when the log never ran on this document + * (installed after the page navigated) or the engine has no Long Tasks API + * (WebKit), so an empty answer is evidence of no long task, never of no log. + */ +export async function readLongTasks( + page: Page, + range?: {from: number; to: number}, +): Promise { + const log = await page.evaluate( + () => + ( + window as unknown as { + __harnessLongTasks?: {tasks: LongTask[]; supported: boolean}; + } + ).__harnessLongTasks ?? null, + ); + if (log == null) { + throw new Error( + 'readLongTasks: the long-task log is not running on this document; call installLongTaskLog(page) before navigating', + ); + } + if (!log.supported) { + throw new Error( + 'readLongTasks: this engine has no Long Tasks API, so the log records nothing', + ); + } + const tasks = log.tasks; + return range == null + ? tasks + : tasks.filter( + t => t.start < range.to && t.start + t.durationMs > range.from, + ); +} diff --git a/packages/build/__fixtures__/stylex-consts/App.tsx b/packages/build/__fixtures__/stylex-consts/App.tsx new file mode 100644 index 0000000000000..5e839e5372912 --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/App.tsx @@ -0,0 +1,34 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * Reads `defineConsts` values from other files, as a media-query key and as a + * plain value, from both sides of the layer split and from a file compiled + * with the library class-name prefix. Only the constants are + * imported: once StyleX compiles `stylex.create`, nothing in this file uses + * the imports at runtime any more. + */ + +import * as stylex from '@stylexjs/stylex'; +import {appConsts} from './appConsts.stylex'; +import {libraryConsts} from './library/libraryConsts.stylex'; +import {packageConsts} from './packages/core/packageConsts.stylex'; + +const styles = stylex.create({ + row: { + minHeight: appConsts.rowHeight, + transitionProperty: {default: 'color', [appConsts.narrow]: 'none'}, + }, + panel: { + paddingTop: libraryConsts.inset, + paddingBottom: packageConsts.gap, + maxWidth: {default: null, [libraryConsts.wide]: '640px'}, + }, +}); + +export function App() { + return ( +
+
row
+
+ ); +} diff --git a/packages/build/__fixtures__/stylex-consts/appConsts.stylex.ts b/packages/build/__fixtures__/stylex-consts/appConsts.stylex.ts new file mode 100644 index 0000000000000..5e41328b69880 --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/appConsts.stylex.ts @@ -0,0 +1,9 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +import * as stylex from '@stylexjs/stylex'; + +/** Constants the app defines itself, beside the file that reads them. */ +export const appConsts = stylex.defineConsts({ + narrow: '@media (max-width: 599px)', + rowHeight: '52px', +}); diff --git a/packages/build/__fixtures__/stylex-consts/index.html b/packages/build/__fixtures__/stylex-consts/index.html new file mode 100644 index 0000000000000..ba86fc975ff35 --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/index.html @@ -0,0 +1,12 @@ + + + + + + stylex-consts fixture — defineConsts imported across files + + +
+ + + diff --git a/packages/build/__fixtures__/stylex-consts/library/libraryConsts.stylex.ts b/packages/build/__fixtures__/stylex-consts/library/libraryConsts.stylex.ts new file mode 100644 index 0000000000000..92dda6955e61f --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/library/libraryConsts.stylex.ts @@ -0,0 +1,12 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +import * as stylex from '@stylexjs/stylex'; + +/** + * Constants on the library side of the layer split (the build test points + * `libraryPattern` at this folder), read by an app file on the product side. + */ +export const libraryConsts = stylex.defineConsts({ + wide: '@media (min-width: 900px)', + inset: '13px', +}); diff --git a/packages/build/__fixtures__/stylex-consts/main.tsx b/packages/build/__fixtures__/stylex-consts/main.tsx new file mode 100644 index 0000000000000..5811fe017bb3e --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/main.tsx @@ -0,0 +1,6 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +import {createRoot} from 'react-dom/client'; +import {App} from './App'; + +createRoot(document.getElementById('root')!).render(); diff --git a/packages/build/__fixtures__/stylex-consts/packages/core/packageConsts.stylex.ts b/packages/build/__fixtures__/stylex-consts/packages/core/packageConsts.stylex.ts new file mode 100644 index 0000000000000..b4be85d7a3505 --- /dev/null +++ b/packages/build/__fixtures__/stylex-consts/packages/core/packageConsts.stylex.ts @@ -0,0 +1,13 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +import * as stylex from '@stylexjs/stylex'; + +/** + * Constants as an Astryx package defines them: the path holds `packages/core/`, + * so the babel wrapper compiles this file with the library class-name prefix + * (`astryx`), as it does a real package. The app file reading them compiles + * with its own prefix (`x`). + */ +export const packageConsts = stylex.defineConsts({ + gap: '17px', +}); diff --git a/packages/build/src/vite.build.test.ts b/packages/build/src/vite.build.test.ts index ee860275a2752..b0907675792c1 100644 --- a/packages/build/src/vite.build.test.ts +++ b/packages/build/src/vite.build.test.ts @@ -31,7 +31,7 @@ */ import {describe, it, expect, beforeAll, afterAll} from 'vitest'; -import {build} from 'vite'; +import {build, createLogger} from 'vite'; import react from '@vitejs/plugin-react'; import { mkdirSync, @@ -70,6 +70,21 @@ type Built = { async function buildFixture( name: string, htmlNames: string[] = ['index.html'], + { + libraryPattern = 'packages/core/', + zeroConfig = false, + stylexOverrides, + warnings = [], + }: { + /** Which source paths land in the library layer. */ + libraryPattern?: string; + /** Use `astryxStylex({...})` instead of the legacy `{stylexOptions}` form. */ + zeroConfig?: boolean; + /** Zero-config form only: `stylexOverrides`. */ + stylexOverrides?: Record; + /** Receives every warning the build logs. */ + warnings?: string[]; + } = {}, ): Promise { const root = path.join(FIXTURES, name); const outDir = mkdtempSync(path.join(tmpdir(), `astryx-${name}-`)); @@ -77,6 +92,11 @@ async function buildFixture( await build({ root, logLevel: 'error', + customLogger: { + ...createLogger('error'), + warn: (message: string) => void warnings.push(message), + warnOnce: (message: string) => void warnings.push(message), + }, build: { outDir, emptyOutDir: true, @@ -94,17 +114,27 @@ async function buildFixture( resolve: {alias: {'@astryxdesign/core': CORE_SRC}}, plugins: [ react(), - ...astryxStylex({ - stylexOptions: { - dev: false, - unstable_moduleResolution: {type: 'commonJS', rootDir: REPO_ROOT}, - aliases: { - '@astryxdesign/core/*': [path.join(CORE_SRC, '*')], - '@astryxdesign/core': [CORE_SRC], - }, - }, - libraryPattern: 'packages/core/', - }), + ...(zeroConfig + ? astryxStylex({ + dev: false, + rootDir: REPO_ROOT, + libraryPattern, + stylexOverrides, + }) + : astryxStylex({ + stylexOptions: { + dev: false, + unstable_moduleResolution: { + type: 'commonJS', + rootDir: REPO_ROOT, + }, + aliases: { + '@astryxdesign/core/*': [path.join(CORE_SRC, '*')], + '@astryxdesign/core': [CORE_SRC], + }, + }, + libraryPattern, + })), ], }); } catch (error) { @@ -338,6 +368,67 @@ describe('a production build separates Astryx and product styles by layer', () = }); }); +// `defineConsts` values are compiled into the reading file as placeholders, +// `var(--)`, and the defining file contributes the values; the +// final CSS pass substitutes one for the other. Both halves must reach that +// pass. Storybook builds through the legacy `{stylexOptions}` form, whose +// StyleX instance ran without `treeshakeCompensation`: once `stylex.create` +// was compiled, nothing used the import at runtime, the TypeScript transform +// dropped it, and the defining module was never loaded. A media-query key was +// left as `var(--x…){…}` and failed the build ("Invalid empty selector"); a +// value shipped as a custom property nothing defines. And a constant defined +// on one side of the layer split and read on the other was substituted only +// within its own side, and one an Astryx package defines (compiled with the +// library class-name prefix) never matched an app file's placeholder (written +// with the product prefix). +describe('a production build resolves defineConsts imported across files', () => { + /** A constant reference StyleX left unsubstituted. */ + const PLACEHOLDER = /var\(--(?:astryx|x)[a-z0-9]{4,}\)/; + + for (const zeroConfig of [false, true]) { + describe( + zeroConfig ? 'zero-config form' : 'legacy stylexOptions form (Storybook)', + () => { + let built: Built; + beforeAll(async () => { + built = await buildFixture('stylex-consts', ['index.html'], { + libraryPattern: '__fixtures__/stylex-consts/library/', + zeroConfig, + }); + }, 180_000); + afterAll( + () => built && rmSync(built.outDir, {recursive: true, force: true}), + ); + + it('substitutes a constant read as a value', () => { + expect(built.css).toMatch(/min-height:\s*52px/); + }); + + it('substitutes a constant read as a media-query key', () => { + expect(built.css).toMatch( + /@media \((?:max-width: ?599px|width ?<= ?599px)\)\s*\{[^}]*transition-property:\s*none/, + ); + }); + + it('substitutes a library-side constant read from the product side', () => { + expect(built.css).toMatch(/padding-top:\s*13px/); + expect(built.css).toMatch( + /@media \((?:min-width: ?900px|width ?>= ?900px)\)\s*\{[^}]*max-width:\s*640px/, + ); + }); + + it('substitutes a constant an Astryx package defines under the library prefix', () => { + expect(built.css).toMatch(/padding-bottom:\s*17px/); + }); + + it('leaves no constant placeholder behind', () => { + expect(built.css).not.toMatch(PLACEHOLDER); + }); + }, + ); + } +}); + describe('a production build validates StyleX declarations', () => { it('fails with longhand guidance instead of dropping a border shorthand', async () => { await expect(buildFixture('invalid-border')).rejects.toThrow( diff --git a/packages/build/src/vite.ts b/packages/build/src/vite.ts index 0581b58675edb..edaa88ca55a1d 100644 --- a/packages/build/src/vite.ts +++ b/packages/build/src/vite.ts @@ -136,6 +136,12 @@ export function astryxStylex( const libraryLayer = layers.library ?? 'astryx-base'; const productLayer = layers.product ?? 'product'; + // Library files compile with `stylexPrefix`, product files with StyleX's + // `classNamePrefix` (the babel wrapper routes them). + const classNamePrefixes = [ + stylexPrefix, + (stylexOverrides as {classNamePrefix?: string}).classNamePrefix ?? 'x', + ]; const overrideLightningcssOptions = ( stylexOverrides as {lightningcssOptions?: Record} @@ -284,6 +290,7 @@ export function astryxStylex( libraryPattern, libraryLayer, productLayer, + classNamePrefixes, }), ); }, @@ -302,6 +309,7 @@ export function astryxStylex( libraryLayer, productLayer, lightningcssOptions: stylexOptions.lightningcssOptions, + classNamePrefixes, }), ]; } @@ -330,24 +338,43 @@ function renderSplitLayers( libraryPattern: string; libraryLayer: string; productLayer: string; + /** The StyleX class-name prefixes in play: library, then product. */ + classNamePrefixes: readonly string[]; }, ): string { if (!rulesById || rulesById.size === 0) return ''; const libraryRules: unknown[] = []; const productRules: unknown[] = []; + // `defineConsts` values. A file that reads a constant compiles it to a + // placeholder, `var(--)`, and `processStylexRules` substitutes + // the value from the defining file's rules, but only among the rules it is + // given. The defining file and the reading one can sit on opposite sides of + // the split, so every layer gets every constant, under every class-name + // prefix (see constantRulesForEveryPrefix). Constant rules emit no CSS of + // their own, so repeating them adds nothing to the output. + const collectedConstants: unknown[] = []; for (const [filePath, rules] of rulesById.entries()) { - if (filePath.includes(options.libraryPattern)) { - libraryRules.push(...rules); - } else { - productRules.push(...rules); + const own = filePath.includes(options.libraryPattern) + ? libraryRules + : productRules; + for (const rule of rules) { + if (isConstantRule(rule)) { + collectedConstants.push(rule); + } else { + own.push(rule); + } } } + const constantRules = constantRulesForEveryPrefix( + collectedConstants, + options.classNamePrefixes, + ); const render = (rules: unknown[], layer: string) => rules.length - ? `@layer ${layer} {\n${stylexBabelPlugin.processStylexRules(rules as never, {useLayers: true})}\n}` + ? `@layer ${layer} {\n${stylexBabelPlugin.processStylexRules([...constantRules, ...rules] as never, {useLayers: true})}\n}` : ''; return [ @@ -358,6 +385,56 @@ function renderSplitLayers( .join('\n\n'); } +/** + * Each `defineConsts` rule once under every class-name prefix in play. + * + * StyleX keys a constant by its defining file's prefix, `` (or + * `-` with debug class names), where the hash depends on + * the export and key alone, and a file that reads it writes the placeholder + * with its own prefix. An Astryx package compiles with the library prefix and + * an app with the product one, so an app reading a package's constant writes + * `var(--x)` while the package's rule is keyed `astryx`; the + * copies make every prefix's placeholder resolve. A key StyleX took verbatim + * (`'--name'`) carries no prefix and is kept as it is. + */ +function constantRulesForEveryPrefix( + rules: readonly unknown[], + prefixes: readonly string[], +): unknown[] { + const ordered = [...new Set(prefixes)] + .filter(Boolean) + .sort((a, b) => b.length - a.length); + const out: unknown[] = []; + const keys = new Set(); + const add = (rule: unknown[], key: string) => { + if (keys.has(key)) return; + keys.add(key); + const [, body, ...rest] = rule; + out.push([key, {...(body as object), constKey: key}, ...rest]); + }; + for (const rule of rules as unknown[][]) { + const key = String((rule[1] as {constKey: unknown}).constKey); + add(rule, key); + for (const prefix of ordered) { + const match = new RegExp(`^(|.*-)${prefix}([0-9a-z]+)$`).exec(key); + if (match == null) continue; + for (const other of ordered) { + add(rule, `${match[1]}${other}${match[2]}`); + } + break; + } + } + return out; +} + +/** A `defineConsts` value as StyleX collects it: a rule with `constKey`/`constVal`. */ +function isConstantRule(rule: unknown): boolean { + const body = Array.isArray(rule) + ? (rule[1] as {constKey?: unknown; constVal?: unknown} | undefined) + : undefined; + return body?.constKey != null && body?.constVal != null; +} + /** * The build half of the split. StyleX appends one merged block of CSS to the * build's stylesheet, in its own top-level `@layer priority1…priorityN` — which @@ -387,6 +464,8 @@ function buildLayerSplitPlugin( libraryLayer: string; productLayer: string; lightningcssOptions?: unknown; + /** The StyleX class-name prefixes in play: library, then product. */ + classNamePrefixes: readonly string[]; }, ): Plugin { const stylex = basePlugin as { @@ -586,12 +665,25 @@ function astryxStylexLegacy(options: AstryxVitePluginLegacyOptions): Plugin[] { ).lightningcssOptions; const validatedStylexOptions = { propertyValidationMode: 'throw', + // The unplugin applies its Vite default (true) only to its own StyleX + // instance, which runs after the babel wrapper below has already compiled + // every StyleX call. Without it here, a file that reads only `defineConsts` + // from a `.stylex.ts` module loses that import once `stylex.create` is + // compiled (the TypeScript transform drops it as unused), the module is + // never loaded, and its constants never reach the CSS: a media-query key + // fails the build and a value ships as an undefined custom property. + treeshakeCompensation: true, ...(stylexOptions as any), lightningcssOptions: { targets: LIGHTNINGCSS_TARGETS, ...legacyLightningcssOptions, }, }; + const classNamePrefixes = [ + stylexPrefix, + (validatedStylexOptions as {classNamePrefix?: string}).classNamePrefix ?? + 'x', + ]; const astryxBabelPlugin = path.resolve(__dirname, 'babel.js'); const existingPlugins = validatedStylexOptions.babelConfig?.plugins ?? []; @@ -658,6 +750,7 @@ function astryxStylexLegacy(options: AstryxVitePluginLegacyOptions): Plugin[] { libraryPattern, libraryLayer, productLayer, + classNamePrefixes, }), ); }, @@ -677,6 +770,7 @@ function astryxStylexLegacy(options: AstryxVitePluginLegacyOptions): Plugin[] { lightningcssOptions: ( validatedStylexOptions as {lightningcssOptions?: unknown} ).lightningcssOptions, + classNamePrefixes, }), ]; } diff --git a/packages/core/package.json b/packages/core/package.json index 8bd491e6273cb..59a7b26167438 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -859,6 +859,7 @@ "devDependencies": { "@astryxdesign/a11y-spec": "workspace:*", "@astryxdesign/cli": "workspace:*", + "@astryxdesign/interaction-harness": "workspace:*", "@babel/cli": "catalog:", "@babel/core": "catalog:", "@babel/preset-react": "catalog:", diff --git a/packages/core/src/ContextMenu/ContextMenu.doc.mjs b/packages/core/src/ContextMenu/ContextMenu.doc.mjs index f8ee2c5659074..80d646501d402 100644 --- a/packages/core/src/ContextMenu/ContextMenu.doc.mjs +++ b/packages/core/src/ContextMenu/ContextMenu.doc.mjs @@ -132,8 +132,7 @@ export const docs = { name: 'presentation', type: "'popover' | 'bottom-sheet' | 'adaptive'", description: - 'Presentation policy. `popover` opens at the pointer position, `bottom-sheet` always uses an action sheet, and `adaptive` uses the BottomSheet on a compact touch screen: a coarse primary pointer, and a viewport narrower than the theme `md` breakpoint (768px by default) or shorter than 500px.', - default: "'popover'", + 'Presentation policy. Omitted, the menu opens at the pointer position, and under a finger a held finger opens it on iOS timing (the touch model). `popover` opens at the pointer position on every device, and a held finger opens it at 500 ms (the long press), as before; so does a finger where the primary pointer is fine. `bottom-sheet` always uses an action sheet, and `adaptive` uses the BottomSheet on a compact touch screen: a coarse primary pointer, and a viewport narrower than the theme `md` breakpoint (768px by default) or shorter than 500px.', }, { name: 'isDisabled', @@ -160,7 +159,7 @@ export const docs = { usage: { anatomy, description: - 'A right-click context menu that appears at the cursor position. Use to provide contextual actions for specific elements or regions without cluttering the UI with visible buttons.', + 'A right-click context menu that appears at the cursor position. Use to provide contextual actions for specific elements or regions without cluttering the UI with visible buttons. Under a finger, on every touch device, a held finger opens it with iOS timing: the trigger starts to grow at 150 ms, and the menu opens at 735 ms, or as soon as the finger lifts after 400 ms. A block trigger then lifts out of the page, as on iOS: it grows about 13% over a tinted backdrop, and the menu grows out of the finger to sit beside it, below when it fits and above otherwise.', bestPractices: [ { guidance: true, @@ -187,6 +186,11 @@ export const docs = { description: 'Keep a visible MoreMenu or equivalent trigger for important mobile actions; long-press must not be the only route.', }, + { + guidance: true, + description: + 'Leave a little room around a block trigger: a held finger grows it from its center, by about 6%, and it lifts about 13% larger while the menu is open.', + }, { guidance: false, description: @@ -224,7 +228,7 @@ export const docs = { export const docsZh = { usage: { description: - '右键点击时在光标位置出现的上下文菜单。用于为特定元素或区域提供上下文操作，而不使 UI 杂乱。', + '右键点击时在光标位置出现的上下文菜单。用于为特定元素或区域提供上下文操作，而不使 UI 杂乱。在所有触摸设备上，按住手指即可打开，时序与 iOS 相同：150 毫秒时触发区域开始放大，735 毫秒时菜单打开；若手指在 400 毫秒后抬起，菜单在抬起时立即打开。随后块级触发区域会像 iOS 一样从页面中浮起：它在着色背景上放大约 13%，菜单从手指处展开并位于其旁边，空间足够时在下方，否则在上方。', bestPractices: [ {guidance: true, description: '保持菜单项简洁和面向操作。'}, {guidance: true, description: '有很多项时使用分组和分隔线。'}, @@ -242,6 +246,11 @@ export const docsZh = { description: '重要的移动端操作还应提供可见的 MoreMenu 或同等入口，不要只依赖长按。', }, + { + guidance: true, + description: + '块级触发区域周围请留出少量空间：按住时它会从中心放大约 6%，菜单打开期间它浮起时约大 13%。', + }, { guidance: false, description: '将上下文菜单作为访问重要操作的唯一方式。', @@ -260,7 +269,7 @@ export const docsDense = { usage: { anatomy, description: - 'A right-click context menu that appears at the cursor position. Use to provide contextual actions for specific elements or regions.', + 'A right-click context menu that appears at the cursor position. Use to provide contextual actions for specific elements or regions. Under a finger (any touch device): a held finger opens it at 735 ms, or on a lift after 400 ms; the trigger grows from 150 ms, then a block trigger lifts (about 13%) over a tint with the menu below or above it.', bestPractices: [ {guidance: true, description: 'Keep items concise and action-oriented.'}, { @@ -281,6 +290,11 @@ export const docsDense = { guidance: true, description: 'Keep a visible mobile entry point for important actions.', }, + { + guidance: true, + description: + 'Leave room around a block trigger: a held finger grows it about 6%, and it lifts about 13% larger.', + }, { guidance: false, description: 'Use as the only way to access important actions.', diff --git a/packages/core/src/ContextMenu/ContextMenu.spec.md b/packages/core/src/ContextMenu/ContextMenu.spec.md index 9ad771e40db7f..2deb9962dbe2b 100644 --- a/packages/core/src/ContextMenu/ContextMenu.spec.md +++ b/packages/core/src/ContextMenu/ContextMenu.spec.md @@ -204,16 +204,17 @@ anchor is positioning infrastructure rather than consumer anatomy. ## Verification map -| Contract | Verification | Representative states | Mutation or failure expectation | Audit section | -| ------------------- | ------------------------------------------------------------------------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------- | -| FR1, FR4 | `ContextMenu.test.tsx` pointer, long-press, bottom-sheet, adaptive, compound, and drill-in suites | Pointer data/compound and touch data/compound | Collapsing presentation-specific parts or content owners fails structure, role, or behavior tests. | `audit:ContextMenu/anatomy` | -| FR2 | ContextMenu target tests, source inspection, and theming target inventories | Pointer and touch menu surfaces | Moving `context-menu` to the trigger or removing it from a painted branch fails evidence. | `audit:ContextMenu/theming` | -| FR3 | ContextMenu, DropdownMenu, BottomSheet, and List owner tests | Pointer rows, touch frame, touch list/rows | A composed part loses its owner target or is documented as a new ContextMenu target. | `audit:ContextMenu/theming` | -| Layer relationships | `ContextMenu.test.tsx` nested-layer case and current layer/dismissal architecture records | Outside click, Escape, nested context layers, sheet | A Pointer menu bypassing the shared stack or dismissing with its nested child fails. | `audit:ContextMenu/behavior` | -| FR5 | `ContextMenu.test.tsx` press model case; `module:DropdownMenu/useMenuPress` map | Finger slide across rows, stray click | A row acting on the press row or the stray click fails. | `audit:ContextMenu/behavior` | -| FR6 | `ContextMenu.test.tsx` keyboard case; shared `useListFocus` and `useTypeahead` suites | wrap at both ends | Clamping arrows fail. | `audit:ContextMenu/behavior` | -| Theming anatomy map | `scripts/check-knowledge.mjs` | Canonical anatomy and current target | Missing, extra, prefixed, stale, or unclassified mappings fail repository validation. | `audit:ContextMenu/theming` | -| FR6, AR1 | `ContextMenu.test.tsx` "inline trigger (triggerAs)" suite | `span` in a paragraph, default `div` | A block trigger inside prose, or a lost invocation path, fails. | `audit:ContextMenu/behavior` | +| Contract | Verification | Representative states | Mutation or failure expectation | Audit section | +| ------------------- | --------------------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------- | +| FR1, FR4 | `ContextMenu.test.tsx` pointer, long-press, bottom-sheet, adaptive, compound, and drill-in suites | Pointer data/compound and touch data/compound | Collapsing presentation-specific parts or content owners fails structure, role, or behavior tests. | `audit:ContextMenu/anatomy` | +| FR2 | ContextMenu target tests, source inspection, and theming target inventories | Pointer and touch menu surfaces | Moving `context-menu` to the trigger or removing it from a painted branch fails evidence. | `audit:ContextMenu/theming` | +| FR3 | ContextMenu, DropdownMenu, BottomSheet, and List owner tests | Pointer rows, touch frame, touch list/rows | A composed part loses its owner target or is documented as a new ContextMenu target. | `audit:ContextMenu/theming` | +| Layer relationships | `ContextMenu.test.tsx` nested-layer case and current layer/dismissal architecture records | Outside click, Escape, nested context layers, sheet | A Pointer menu bypassing the shared stack or dismissing with its nested child fails. | `audit:ContextMenu/behavior` | +| FR5 | `ContextMenu.test.tsx` press model case; `module:DropdownMenu/useMenuPress` map | Finger slide across rows, stray click | A row acting on the press row or the stray click fails. | `audit:ContextMenu/behavior` | +| FR6 | `ContextMenu.test.tsx` keyboard case; shared `useListFocus` and `useTypeahead` suites | wrap at both ends | Clamping arrows fail. | `audit:ContextMenu/behavior` | +| Theming anatomy map | `scripts/check-knowledge.mjs` | Canonical anatomy and current target | Missing, extra, prefixed, stale, or unclassified mappings fail repository validation. | `audit:ContextMenu/theming` | +| FR6, AR1 | `ContextMenu.test.tsx` "inline trigger (triggerAs)" suite | `span` in a paragraph, default `div` | A block trigger inside prose, or a lost invocation path, fails. | `audit:ContextMenu/behavior` | +| DEC-2 | `ContextMenuLift.interaction.browser.spec.ts` element-count case; `ContextMenu.test.tsx` lift suite | Touch story at rest and lifted, fine and coarse | A lift element at rest, a copy of the trigger, or an open count beyond the layer, the shadow, the anchor and the placeholder, fails. | `audit:ContextMenu/anatomy` | ## Decision log @@ -236,6 +237,43 @@ Rejected: styling a block `div` as inline because it remains invalid inside a Rejected: `display: contents` because a box-less wrapper has no rect for pointer or keyboard anchoring. +### DEC-2 — The lift adds four elements while open, each with one job + +**Reference:** `component:ContextMenu/DEC-2` + +**Status:** proposed; pending owner review. + +Under a finger, a touch-opened menu on a block trigger lifts the trigger +itself above the tinted page (`spec:AST-094` FR33), through the lift module +(`packages/core/src/liftFromTrigger/`). While the menu is up, the page has four +elements more than under a fine pointer, within the open-layer allowance of +`spec:AST-017` FR60 and DEC-29, and each has one job: + +- the lift layer covers the viewport from the top layer, takes the page's + touches and paints the tint as its `::backdrop`; +- the shadow, a box in the layer, follows the trigger's lift as one + compositor animation (an animated `::backdrop` of the trigger restyles + and lays out the page on every frame); +- an empty box in the layer, at the lifted rect, is what the menu is placed + against (anchor positioning follows transforms, so the trigger's own cursor + anchor would move with the lifted trigger); +- an empty placeholder after the trigger holds the trigger's spot, so the + page does not move while the trigger is in the top layer. + +None of the four is themeable or part of the anatomy, none is mounted at rest, and no content is mounted +twice: the trigger lifts itself, with no copy. The input hold's shield +joins while the lift grows, and the layer runtime's exit copy while the menu +closes. + +Element counts (`document.body.querySelectorAll('*').length`, story +`core-contextmenu--touch-long-press`, Chromium and WebKit): fine pointer 138 +at rest and 138 open; coarse pointer 138 at rest and 142 lifted, whatever the +trigger holds. + +Rejected: a copy of the trigger above the tint, because it mounts the +trigger's content twice and its count grows with the trigger (a list row's +8 to 15 elements). + ## Open questions None. diff --git a/packages/core/src/ContextMenu/ContextMenu.test.tsx b/packages/core/src/ContextMenu/ContextMenu.test.tsx index d3d00b8417338..13b3ade85de6c 100644 --- a/packages/core/src/ContextMenu/ContextMenu.test.tsx +++ b/packages/core/src/ContextMenu/ContextMenu.test.tsx @@ -10,7 +10,15 @@ import {useRef, useState} from 'react'; import {createPortal} from 'react-dom'; -import {describe, it, expect, vi, beforeAll, beforeEach} from 'vitest'; +import { + describe, + it, + expect, + vi, + afterEach, + beforeAll, + beforeEach, +} from 'vitest'; import { render, screen, @@ -21,7 +29,6 @@ import { createEvent, } from '@testing-library/react'; import userEvent from '@testing-library/user-event'; -import {readFileSync} from 'node:fs'; import {ContextMenu} from './ContextMenu'; import {ContextMenuGroup} from './index'; import { @@ -34,9 +41,13 @@ import { import {DropdownMenuItem} from '../DropdownMenu/DropdownMenuItem'; import {DropdownMenuSubMenu} from '../DropdownMenu/DropdownMenuSubMenu'; import {Divider} from '../Divider'; +import {LayerProvider} from '../Layer/LayerProvider'; import {useLayerDismissal} from '../Layer/useLayerDismissal'; import {warmMenuBottomSheet} from '../DropdownMenu/__tests__/warmMenuBottomSheet'; import {COMPACT_TOUCH_PRESENTATION_QUERY} from '../hooks/useAdaptivePresentation'; +import {COMPACT_QUERY, TOUCH_QUERY} from '../utils/touchPresentation'; +import {HAPTIC_VIBRATION_PATTERNS} from '../motion/uikit'; +import {contextMenu} from '../motion/uikit/contextMenu'; // The touch sheet is loaded lazily; resolve its modules once up front, so // ContextMenu's own lazy sheet renders within the cases' waitFor. @@ -345,7 +356,11 @@ describe('ContextMenu', () => { vi.stubGlobal( 'matchMedia', vi.fn().mockImplementation((query: string) => ({ - matches: query === COMPACT_TOUCH_PRESENTATION_QUERY, + // A compact touch screen: a coarse primary pointer, a narrow viewport. + matches: + query === COMPACT_TOUCH_PRESENTATION_QUERY || + query === TOUCH_QUERY || + query === COMPACT_QUERY, media: query, onchange: null, addEventListener: vi.fn(), @@ -528,114 +543,725 @@ describe('ContextMenu', () => { fireEvent.contextMenu(trigger, {clientX: 0, clientY: 0, detail: 0}); expect(HTMLElement.prototype.showPopover).toHaveBeenCalled(); }); +}); - it('opens on touch long-press', () => { - vi.useFakeTimers(); - try { +/** + * A finger on the real gesture host: touch events with explicit contacts, + * stamped on the (faked) `performance.now()` clock the host reads. + */ +interface Contact { + id: number; + x: number; + y: number; +} + +function dispatchTouch( + type: 'touchstart' | 'touchmove' | 'touchend' | 'touchcancel', + target: EventTarget, + changed: Contact[], + all: Contact[], +): Event { + const event = new Event(type, {bubbles: true, cancelable: true}); + const toTouch = (contact: Contact) => ({ + identifier: contact.id, + clientX: contact.x, + clientY: contact.y, + target, + }); + Object.defineProperty(event, 'changedTouches', {value: changed.map(toTouch)}); + Object.defineProperty(event, 'touches', {value: all.map(toTouch)}); + Object.defineProperty(event, 'timeStamp', {value: performance.now()}); + target.dispatchEvent(event); + return event; +} + +function finger(target: EventTarget, id = 1, x = 20, y = 20) { + const contact = {id, x, y}; + return { + down: () => dispatchTouch('touchstart', target, [contact], [contact]), + moveTo: (nx: number, ny: number) => { + contact.x = nx; + contact.y = ny; + return dispatchTouch('touchmove', target, [contact], [contact]); + }, + up: () => dispatchTouch('touchend', target, [contact], []), + }; +} + +const advance = (ms: number) => + act(() => { + vi.advanceTimersByTime(ms); + }); + +/** The primary pointer answers `matches` to `(pointer: coarse)`. */ +function stubPointer(isCoarse: boolean) { + vi.stubGlobal( + 'matchMedia', + vi.fn().mockImplementation((query: string) => ({ + matches: isCoarse && query === '(pointer: coarse)', + media: query, + onchange: null, + addEventListener: vi.fn(), + removeEventListener: vi.fn(), + addListener: vi.fn(), + removeListener: vi.fn(), + dispatchEvent: vi.fn(), + })), + ); +} + +/** + * A finger on iOS in jsdom: a coarse pointer, on an engine that supports + * `-webkit-touch-callout` (WebKit on iOS). Undone by vi.unstubAllGlobals(). + */ +function stubIosTouch() { + vi.stubGlobal( + 'matchMedia', + vi.fn().mockImplementation((query: string) => ({ + matches: query === '(pointer: coarse)', + media: query, + onchange: null, + addEventListener: vi.fn(), + removeEventListener: vi.fn(), + addListener: vi.fn(), + removeListener: vi.fn(), + dispatchEvent: vi.fn(), + })), + ); + vi.stubGlobal('CSS', { + ...(globalThis as {CSS?: object}).CSS, + supports: (condition: string) => + condition.includes('-webkit-touch-callout'), + }); +} + +/** A finger on Android: a coarse pointer, no WebKit touch callout. */ +function stubAndroidTouch() { + stubPointer(true); +} + +// The gesture host lives as long as the document and remembers the latest +// event time; each install of fake timers restarts `performance.now()` near 0, +// so every case moves the faked clock past the ones before it. +let clockEpoch = 0; + +/** + * How many times the menu's popover was shown (a touch-opened menu also shows + * the lift layer and the lifted trigger). + */ +const menuShows = () => + vi.mocked(HTMLElement.prototype.showPopover).mock.contexts.filter( + // The popover that holds the menu: not the lift layer, the hold's shield + // inside it, or the lifted trigger. + el => (el as HTMLElement).querySelector('[role="menu"]') != null, + ).length; + +// One touch model on every touch device (#8316): iOS and Android alike. +describe.each([ + ['iOS', stubIosTouch], + ['Android', stubAndroidTouch], +] as const)( + 'ContextMenu under a finger on %s: UIKit’s long-press timeline', + (_platform, stubTouch) => { + beforeEach(() => { + stubTouch(); + vi.useFakeTimers({ + toFake: [ + 'setTimeout', + 'clearTimeout', + 'setInterval', + 'clearInterval', + 'requestAnimationFrame', + 'cancelAnimationFrame', + 'performance', + 'Date', + ], + }); + clockEpoch += 100_000; + vi.advanceTimersByTime(clockEpoch); + }); + afterEach(() => { + vi.useRealTimers(); + vi.unstubAllGlobals(); + }); + + function renderMenu( + props: { + isDisabled?: boolean; + presentation?: 'popover'; + triggerAs?: 'div' | 'span'; + onTouchStart?: (event: React.TouchEvent) => void; + onTouchEnd?: (event: React.TouchEvent) => void; + } = {}, + ) { render( - +
Long-press me
, ); - const trigger = screen.getByTestId('ctx'); - fireEvent.touchStart(trigger, { - touches: [{clientX: 20, clientY: 20}], - }); - // Not open until the long-press threshold elapses. - expect(HTMLElement.prototype.showPopover).not.toHaveBeenCalled(); + return screen.getByTestId('ctx'); + } + + it('opens for a finger still down at 735 ms (iOS 26.2), not a millisecond before', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(contextMenu.press.menuDisplayMs - 1); + expect(menuShows()).toBe(0); + advance(1); + expect(menuShows()).toBe(1); + press.up(); + }); + + it('lights no row when a touch opens it: focus goes to the menu itself, not its first item', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(contextMenu.press.menuDisplayMs); + // The focus moves on the next frame. + advance(20); + expect(screen.getByRole('menu', {hidden: true})).toHaveFocus(); + expect( + screen.getByRole('menuitem', {name: 'Item 1', hidden: true}), + ).not.toHaveFocus(); + press.up(); + }); + + it('opens at the lift for a finger lifted after the commit (400 ms), as UIKit does', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(529); + expect(menuShows()).toBe(0); act(() => { - vi.advanceTimersByTime(500); + press.up(); }); - expect(HTMLElement.prototype.showPopover).toHaveBeenCalled(); - } finally { - vi.useRealTimers(); - } - }); + expect(menuShows()).toBe(1); + }); - it('composes consumer touch handlers with long-press detection', () => { - vi.useFakeTimers(); - try { + it('opens nothing for a drag past the tolerance after the commit, before the menu', () => { + const trigger = renderMenu(); + const press = finger(trigger, 1, 20, 20); + press.down(); + advance(500); + press.moveTo(20, 50); + advance(500); + press.up(); + advance(100); + expect(menuShows()).toBe(0); + }); + + it('keeps the menu open as the finger lifts: the lift sends no click or compatibility mouse events', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(contextMenu.press.menuDisplayMs); + const lift = press.up(); + // A cancelled touchend is the one signal that stops the browser's click + // and its compatibility mousedown, which would read as a press outside. + expect(lift.defaultPrevented).toBe(true); + expect(screen.getByRole('menu', {hidden: true})).toBeInTheDocument(); + expect(HTMLElement.prototype.hidePopover).not.toHaveBeenCalled(); + }); + + it('still opens at 735 ms beside a consumer onTouchStart, which runs (#7167)', () => { const onTouchStart = vi.fn(); - render( + const trigger = renderMenu({onTouchStart}); + const press = finger(trigger); + press.down(); + expect(onTouchStart).toHaveBeenCalledTimes(1); + advance(contextMenu.press.menuDisplayMs - 1); + expect(menuShows()).toBe(0); + advance(1); + expect(menuShows()).toBe(1); + press.up(); + }); + + it('opens nothing for an early lift whose consumer onTouchEnd prevents default (#7167)', () => { + const onTouchEnd = vi.fn((event: React.TouchEvent) => { + event.preventDefault(); + }); + const trigger = renderMenu({onTouchEnd}); + const press = finger(trigger); + press.down(); + advance(200); + press.up(); + expect(onTouchEnd).toHaveBeenCalledTimes(1); + advance(1000); + expect(menuShows()).toBe(0); + }); + + it('moves the recognizer to the new trigger when triggerAs swaps it: a hold on the new node opens the menu', () => { + const {rerender} = render( + +
Long-press me
+ , + ); + rerender( -
Long-press me
+ triggerAs="span"> + Long-press me , ); + const press = finger(screen.getByTestId('ctx')); + press.down(); + advance(contextMenu.press.menuDisplayMs); + expect(menuShows()).toBe(1); + press.up(); + }); - fireEvent.touchStart(screen.getByTestId('ctx'), { - touches: [{clientX: 20, clientY: 20}], + it('opens nothing for a press lifted before the commit, and leaves its click alone', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(250); + const lift = press.up(); + expect(lift.defaultPrevented).toBe(false); + advance(1000); + expect(menuShows()).toBe(0); + }); + + it('cancels when the finger drifts past the tolerance on one axis (11.5 px)', () => { + const trigger = renderMenu(); + const press = finger(trigger, 1, 20, 20); + press.down(); + advance(50); + press.moveTo(20, 31.6); + advance(1000); + expect(menuShows()).toBe(0); + press.up(); + }); + + it('still opens after a drift inside the tolerance', () => { + const trigger = renderMenu(); + const press = finger(trigger, 1, 20, 20); + press.down(); + advance(50); + press.moveTo(31, 31); // 15.6 px away, 11 px on each axis + advance(contextMenu.press.menuDisplayMs); + expect(menuShows()).toBe(1); + press.up(); + }); + + it('cancels when a second finger lands before the commit', () => { + const trigger = renderMenu(); + const first = {id: 1, x: 20, y: 20}; + const second = {id: 2, x: 60, y: 20}; + dispatchTouch('touchstart', trigger, [first], [first]); + advance(100); + dispatchTouch('touchstart', trigger, [second], [first, second]); + advance(1000); + expect(menuShows()).toBe(0); + dispatchTouch('touchend', trigger, [first, second], []); + }); + + it('lets the long press decide while a finger is down: the browser’s own long-press menu opens nothing', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(380); + // Android Chrome fires contextmenu for a held finger on its own clock. + const native = new MouseEvent('contextmenu', { + bubbles: true, + cancelable: true, + clientX: 20, + clientY: 20, + }); + trigger.dispatchEvent(native); + expect(native.defaultPrevented).toBe(true); + expect(menuShows()).toBe(0); + advance(contextMenu.press.menuDisplayMs - 380); + expect(menuShows()).toBe(1); + press.up(); + }); + + it('plays the click haptic at the commit where the platform vibrates, and none for an early lift', () => { + const vibrate = vi.fn(() => true); + Object.defineProperty(navigator, 'vibrate', { + value: vibrate, + configurable: true, }); - expect(onTouchStart).toHaveBeenCalledOnce(); + try { + const trigger = renderMenu(); + const early = finger(trigger); + early.down(); + advance(250); + early.up(); + advance(500); + expect(vibrate).not.toHaveBeenCalled(); + + const held = finger(trigger); + held.down(); + advance(contextMenu.press.commitMs - 1); + expect(vibrate).not.toHaveBeenCalled(); + advance(1); + // At the commit, before the menu shows. + expect(vibrate).toHaveBeenCalledTimes(1); + expect(menuShows()).toBe(0); + expect(vibrate).toHaveBeenCalledWith([ + ...HAPTIC_VIBRATION_PATTERNS[contextMenu.commitHaptic].value, + ]); + held.up(); + } finally { + // jsdom has no vibration API of its own. + delete (navigator as {vibrate?: unknown}).vibrate; + } + }); + it('holds the page still while a menu a finger opened is up, and lets it go when the menu closes', () => { + const trigger = renderMenu(); + const press = finger(trigger); + press.down(); + advance(contextMenu.press.commitMs); + // Lifted after the commit: the menu opens at the lift. act(() => { - vi.advanceTimersByTime(500); + press.up(); }); - expect(HTMLElement.prototype.showPopover).toHaveBeenCalledOnce(); - } finally { - vi.useRealTimers(); - } - }); + expect(document.body.style.position).toBe('fixed'); + act(() => { + fireEvent.keyDown(screen.getByRole('menu', {hidden: true}), { + key: 'Escape', + }); + }); + expect(document.body.style.position).toBe(''); + }); - it('clears a pending long-press when a consumer prevents default on touch end', () => { - vi.useFakeTimers(); - try { + it('leaves the page alone for a menu a right-click opened', () => { + const trigger = renderMenu(); + fireEvent.contextMenu(trigger, {clientX: 20, clientY: 20}); + expect(menuShows()).toBe(1); + expect(document.body.style.position).toBe(''); + }); + + it('passes touch events through to the consumer’s own handlers', () => { + const onTouchStart = vi.fn(); + const onTouchEnd = vi.fn(); + const trigger = renderMenu({onTouchStart, onTouchEnd}); + const press = finger(trigger); + press.down(); + press.up(); + expect(onTouchStart).toHaveBeenCalledTimes(1); + expect(onTouchEnd).toHaveBeenCalledTimes(1); + }); + + it('makes the trigger a pressable surface of core’s one press machine, which starts and ends its growth', () => { + const trigger = renderMenu(); + expect(trigger).toHaveAttribute('data-astryx-pressable'); + }); + + it('keeps today’s menu with presentation="popover": a held finger opens it at 500 ms through today’s long press, with no growth and no scroll lock', () => { + const trigger = renderMenu({presentation: 'popover'}); + const shows = () => + vi.mocked(HTMLElement.prototype.showPopover).mock.calls.length; + expect(trigger).not.toHaveAttribute('data-astryx-pressable'); + const press = finger(trigger); + press.down(); + advance(499); + expect(shows()).toBe(0); + advance(1); + expect(shows()).toBe(1); + // Today’s menu: the first row takes the focus, nothing grows, the page + // is not locked. + expect(trigger.style.getPropertyValue('scale')).toBe(''); + expect(document.body.style.overflow).not.toBe('hidden'); + press.up(); + // The browser’s own long-press menu (Android Chrome) still opens it. + fireEvent.contextMenu(trigger, {clientX: 20, clientY: 20, detail: 1}); + expect(shows()).toBeGreaterThanOrEqual(1); + }); + + it('with presentation="popover", a drift past 10 px or a lift before 500 ms opens nothing (today’s long press)', () => { + const trigger = renderMenu({presentation: 'popover'}); + const shows = () => + vi.mocked(HTMLElement.prototype.showPopover).mock.calls.length; + let press = finger(trigger); + press.down(); + advance(300); + press.up(); + advance(400); + expect(shows()).toBe(0); + press = finger(trigger); + press.down(); + press.moveTo(31, 20); + advance(600); + expect(shows()).toBe(0); + press.up(); + }); + + it('opens nothing on touch when disabled', () => { + const trigger = renderMenu({isDisabled: true}); + const press = finger(trigger); + press.down(); + advance(1000); + expect(menuShows()).toBe(0); + press.up(); + }); + + it('lifts the block trigger itself, no copy, and closes when a finger lifts on the backdrop (UIKit ends at that lift)', () => { + const onOpenChange = vi.fn(); render( event.preventDefault()}> + onOpenChange={onOpenChange}>
Long-press me
, ); - + const press = finger(screen.getByTestId('ctx')); + press.down(); + advance(contextMenu.press.menuDisplayMs); + press.up(); + expect(onOpenChange).toHaveBeenLastCalledWith(true); + // The trigger is in the top layer itself, and an empty placeholder + // after it holds its spot. const trigger = screen.getByTestId('ctx'); - fireEvent.touchStart(trigger, { - touches: [{clientX: 20, clientY: 20}], + expect(trigger.getAttribute('popover')).toBe('manual'); + expect( + trigger.nextElementSibling?.hasAttribute( + 'data-astryx-context-menu-lift-placeholder', + ), + ).toBe(true); + // The layer takes the tint's touches (its ::backdrop paints the tint). + const backdrop = document.querySelector( + '[data-astryx-context-menu-lift]', + ); + if (backdrop == null) { + throw new Error('no lift layer'); + } + // A lift there of a finger that went down elsewhere is not a tap there. + fireEvent.pointerUp(backdrop, { + isPrimary: true, + pointerId: 7, + pointerType: 'touch', + }); + expect(onOpenChange).toHaveBeenLastCalledWith(true); + fireEvent.pointerDown(backdrop, { + isPrimary: true, + pointerId: 8, + pointerType: 'touch', + }); + fireEvent.pointerUp(backdrop, { + isPrimary: true, + pointerId: 8, + pointerType: 'touch', }); - fireEvent.touchEnd(trigger); + expect(onOpenChange).toHaveBeenLastCalledWith(false); + }); - act(() => { - vi.advanceTimersByTime(500); + it('leaves no timer of the lift behind when it unmounts while the lift is still growing (AST-094 FR24)', () => { + const view = render( + +
Long-press me
+ , + ); + const press = finger(screen.getByTestId('ctx')); + press.down(); + advance(contextMenu.press.menuDisplayMs); + press.up(); + expect( + document.querySelector('[data-astryx-context-menu-lift]'), + ).not.toBeNull(); + // The menu's own focus move (one frame) is not the lift's. + advance(20); + const beforeUnmount = vi.getTimerCount(); + view.unmount(); + expect( + document.querySelector('[data-astryx-context-menu-lift]'), + ).toBeNull(); + expect(beforeUnmount).toBeGreaterThan(0); + expect(vi.getTimerCount()).toBe(0); + }); + + it('mounts nothing of the lift at rest, and the lift layer only while lifted (INV7)', async () => { + render( + + +
Long-press me
+ + , + ); + expect( + document.querySelector('[data-astryx-context-menu-lift]'), + ).toBeNull(); + const press = finger(screen.getByTestId('ctx')); + press.down(); + advance(contextMenu.press.menuDisplayMs); + press.up(); + const layer = document.querySelector( + '[data-astryx-context-menu-lift]', + ); + expect(layer).not.toBeNull(); + // While the lift grows the layer is held (AST-094 FR35): the hold's + // shield is its third child. Once it has grown, the shadow and the box + // the menu is placed against are the layer's only content (the trigger + // lifts itself). + expect(layer?.children).toHaveLength(3); + await act(async () => { + vi.advanceTimersByTime(1000); }); - expect(HTMLElement.prototype.showPopover).not.toHaveBeenCalled(); - } finally { - vi.useRealTimers(); - } + expect(layer?.children).toHaveLength(2); + }); + + it('keeps an inline trigger’s menu at the finger: no lift', () => { + const trigger = renderMenu({triggerAs: 'span'}); + const press = finger(trigger); + press.down(); + advance(contextMenu.press.menuDisplayMs); + expect(menuShows()).toBe(1); + expect( + document.querySelector('[data-astryx-context-menu-lift]'), + ).toBeNull(); + press.up(); + }); + }, +); + +describe('ContextMenu under a fine pointer: the touch model costs nothing', () => { + beforeEach(() => { + stubPointer(false); + vi.useFakeTimers({ + toFake: [ + 'setTimeout', + 'clearTimeout', + 'setInterval', + 'clearInterval', + 'requestAnimationFrame', + 'cancelAnimationFrame', + 'performance', + 'Date', + ], + }); + clockEpoch += 100_000; + vi.advanceTimersByTime(clockEpoch); + }); + afterEach(() => { + vi.useRealTimers(); + vi.unstubAllGlobals(); }); - it('suppresses native text selection on the long-press trigger', () => { - const source = readFileSync( - 'packages/core/src/ContextMenu/ContextMenu.tsx', - 'utf8', + it('opens today’s menu for a finger held 500 ms (today’s long press), with no touch model: no growth, no scroll lock, the first row focused', () => { + // A fine pointer in this context, before any check. + expect(window.matchMedia('(pointer: coarse)').matches).toBe(false); + const onTouchStart = vi.fn(); + render( + +
Right-click me
+ , ); + const trigger = screen.getByTestId('ctx'); + expect(trigger).not.toHaveAttribute('data-astryx-pressable'); + const press = finger(trigger); + press.down(); + expect(onTouchStart).toHaveBeenCalledTimes(1); + advance(499); + expect(HTMLElement.prototype.showPopover).not.toHaveBeenCalled(); + advance(1); + expect(HTMLElement.prototype.showPopover).toHaveBeenCalledTimes(1); + expect(trigger.style.getPropertyValue('scale')).toBe(''); + expect(document.body.style.overflow).not.toBe('hidden'); + advance(20); + expect( + screen.getByRole('menuitem', {name: 'Item 1', hidden: true}), + ).toHaveFocus(); + press.up(); + }); - expect(source).toMatch( - /trigger:[\s\S]*?WebkitTouchCallout: 'none',[\s\S]*?WebkitUserSelect: 'none',[\s\S]*?userSelect: 'none'/, + it('does not hand the finger that opened the menu on to it: a slide onto a row and a lift pick nothing', () => { + const onClick = vi.fn(); + render( + +
Right-click me
+ , + ); + const trigger = screen.getByTestId('ctx'); + const press = finger(trigger); + press.down(); + advance(500); + expect(HTMLElement.prototype.showPopover).toHaveBeenCalledTimes(1); + const row = screen.getByRole('menuitem', {name: 'Item 1', hidden: true}); + const box = row.getBoundingClientRect(); + press.moveTo(box.left + 4, box.top + 4); + press.up(); + advance(50); + expect(onClick).not.toHaveBeenCalled(); + }); + + it('opens the cursor menu on a right-click, as before', () => { + render( + +
Right-click me
+ , ); + const trigger = screen.getByTestId('ctx'); + fireEvent.contextMenu(trigger, {clientX: 20, clientY: 20, detail: 1}); + expect(HTMLElement.prototype.showPopover).toHaveBeenCalledTimes(1); + advance(20); + expect( + screen.getByRole('menuitem', {name: 'Item 1', hidden: true}), + ).toHaveFocus(); }); +}); - it('cancels the long-press when the finger moves past the threshold', () => { - vi.useFakeTimers(); - try { - render( +describe('ContextMenu: the primary pointer alone picks a finger’s rules (spec:AST-094 FR13, FR33)', () => { + beforeEach(() => { + vi.useFakeTimers({ + toFake: [ + 'setTimeout', + 'clearTimeout', + 'setInterval', + 'clearInterval', + 'requestAnimationFrame', + 'cancelAnimationFrame', + 'performance', + 'Date', + ], + }); + clockEpoch += 100_000; + vi.advanceTimersByTime(clockEpoch); + }); + afterEach(() => { + vi.useRealTimers(); + vi.unstubAllGlobals(); + }); + + it('opens for the same held finger at 735 ms where the primary pointer is coarse, and at 500 ms where it is fine', () => { + // The menu's popover only (not the lift layer or its hold's shield). + const shows = menuShows; + const renderTrigger = () => { + const view = render( -
Long-press me
+
Hold me
, ); - const trigger = screen.getByTestId('ctx'); - fireEvent.touchStart(trigger, {touches: [{clientX: 20, clientY: 20}]}); - // Move past MOVE_CANCEL_PX (10px) — treated as a scroll, not a press. - fireEvent.touchMove(trigger, {touches: [{clientX: 20, clientY: 40}]}); - act(() => { - vi.advanceTimersByTime(500); - }); - expect(HTMLElement.prototype.showPopover).not.toHaveBeenCalled(); - } finally { - vi.useRealTimers(); - } + return {trigger: screen.getByTestId('ctx'), unmount: view.unmount}; + }; + // Coarse: the touch model's timeline, no menu at 500 ms. + stubPointer(true); + const coarse = renderTrigger(); + let press = finger(coarse.trigger); + press.down(); + advance(600); + expect(shows()).toBe(0); + advance(contextMenu.press.menuDisplayMs - 600); + expect(shows()).toBe(1); + press.up(); + coarse.unmount(); + vi.mocked(HTMLElement.prototype.showPopover).mockClear(); + // Fine: today's long press, the same finger opens at 500 ms. + stubPointer(false); + const fine = renderTrigger(); + press = finger(fine.trigger); + press.down(); + advance(499); + expect(shows()).toBe(0); + advance(1); + expect(shows()).toBe(1); + press.up(); }); }); diff --git a/packages/core/src/ContextMenu/ContextMenu.tsx b/packages/core/src/ContextMenu/ContextMenu.tsx index 322a6c7c85059..e6ee24b64e769 100644 --- a/packages/core/src/ContextMenu/ContextMenu.tsx +++ b/packages/core/src/ContextMenu/ContextMenu.tsx @@ -4,8 +4,11 @@ /** * @file ContextMenu.tsx * @input Uses React, StyleX, useLayer, shared layer dismissal, BottomSheet, - * useListFocus, and the shared menu-presentation resolver - * @output Exports ContextMenu with cursor-popover and touch-sheet presentations + * useListFocus, the presentation resolver (usePresentation), useLongPress, + * and the touch layer (ContextMenuTouchLayer: the press recognizer, the + * press feedback and the lift presentation) + * @output Exports ContextMenu with cursor-popover, touch-lift and touch-sheet + * presentations * @position Core implementation; consumed by index.ts * * Right-click context menu positioned at the cursor. The cursor point is @@ -16,6 +19,20 @@ * under the cursor. * Reuses DropdownMenu item rendering and keyboard navigation. * + * Under a finger (a coarse primary pointer: one touch model on every touch + * device, iOS and Android alike) the trigger behaves like UIKit's + * UIContextMenuInteraction: a finger held still commits the menu at UIKit's + * moment (`contextMenu.press.commitMs`), with the press feedback, the page + * stays put while that menu is up, and the press sends no click. When the + * menu shows, a block trigger lifts itself as UIKit's preview does + * (ContextMenuLift.tsx, on the lift module) and the menu sits beside it + * (`contextMenu.lift`); an inline trigger keeps the menu at the finger. The + * touch layer, the lift included, is part of ContextMenu, so it runs from the + * first render under the touch model (a phone pays no extra request for it). + * A fine primary pointer keeps the right-click menu and today's long press for + * a finger (500 ms, `useLongPress`), as `presentation="popover"` does; the + * touch recognizer stays off there. + * * Supports two content modes with a single keyboard/focus path: * - **Data-driven**: pass `items` array (converted to components internally) * - **Compound-component**: pass `menuContent` JSX directly @@ -29,6 +46,10 @@ * SYNC: When modified, update these files to stay in sync: * - /packages/core/src/ContextMenu/ContextMenu.doc.mjs * - /packages/core/src/ContextMenu/ContextMenu.test.tsx + * - /packages/core/src/ContextMenu/ContextMenuTouch.ts (the touch layer's one import site) + * - /packages/core/src/ContextMenu/contextMenuPress.ts (touch timeline) + * - /packages/core/src/ContextMenu/contextMenuPressFeedback.ts (press feedback) + * - /packages/core/src/ContextMenu/ContextMenuLift.tsx (lift presentation) * - /packages/core/src/ContextMenu/index.ts * - /apps/storybook/stories/ContextMenu.stories.tsx * - /packages/cli/assets/templates/blocks/components/ContextMenu/ (showcase blocks) @@ -49,6 +70,7 @@ import * as stylex from '@stylexjs/stylex'; import {Button} from '../Button'; import {Heading} from '../Heading'; import {Icon} from '../Icon'; +import {addAnchorName, removeAnchorName} from '../Layer/anchorName'; import {useLayer} from '../Layer/useLayer'; import {LayerDepthProvider} from '../Layer/LayerDepthContext'; import {useLayerDismissal} from '../Layer/useLayerDismissal'; @@ -70,9 +92,20 @@ import { import {useMenuOverflow} from '../DropdownMenu/useMenuOverflow'; import {useMenuDrillIn} from '../DropdownMenu/useMenuDrillIn'; import {useListFocus} from '../hooks/useListFocus'; +import {useLongPress} from '../hooks/useLongPress'; +import {pressableProps} from '../utils/pressFeedback'; import {useMenuPress} from '../hooks/useMenuPress'; import {useTypeahead} from '../hooks/useTypeahead'; -import {useLongPress} from '../hooks/useLongPress'; +import {useScrollLock} from '../hooks/useScrollLock'; +import {useIsomorphicLayoutEffect} from '../hooks/useIsomorphicLayoutEffect'; +import {readPresentation, usePresentation} from '../utils/presentation'; +import {useCompactDefinition} from '../utils/touchPresentation'; +import { + ContextMenuTouchLayer, + type ContextMenuTouchHandle, + type LiftedMenuPlacement, +} from './ContextMenuTouchLayer'; +import type {ContextMenuLiftResult} from './ContextMenuLift'; import {layerAnimations} from '../Layer/layerAnimations.stylex'; import { colorVars, @@ -95,10 +128,7 @@ import type { } from '../DropdownMenu/DropdownMenu'; import {useMergedRefs} from '../hooks/useMergedRefs'; -import { - useAdaptivePresentation, - type AdaptivePresentation, -} from '../hooks/useAdaptivePresentation'; +import type {AdaptivePresentation} from '../hooks/useAdaptivePresentation'; const LazyMenuBottomSheet = lazy(async () => import('../DropdownMenu/MenuBottomSheet').then(module => ({ @@ -106,11 +136,12 @@ const LazyMenuBottomSheet = lazy(async () => })), ); const styles = stylex.create({ - // Trigger wrapper: suppress the iOS long-press callout/selection so the - // long-press opens our context menu instead of the native text/callout UI. - // `position: relative` establishes the containing block for the absolutely - // positioned cursor anchor below, so the anchor point tracks the trigger - // (and scrolls with it) instead of the page. + // Trigger wrapper. `position: relative` establishes the containing block + // for the absolutely positioned cursor anchor below, so the anchor point + // tracks the trigger (and scrolls with it) instead of the page. The iOS + // callout and text selection stay off, as they always have, so a long press + // or a right-click opens our menu instead of selecting the trigger's text + // (spec:AST-017 FR59); the gesture host adds its own lock for a held press. trigger: { position: 'relative', WebkitTouchCallout: 'none', @@ -122,12 +153,15 @@ const styles = stylex.create({ // relative to the trigger's context — it follows the content on scroll and // the browser can auto-flip it against the viewport edges. It owns a box // because WebKit counts an empty anchor box as clipped, and the default - // `position-visibility: anchors-visible` then hides the menu. + // `position-visibility: anchors-visible` then hides the menu. It paints + // nothing. While the trigger is lifted, the menu is placed against the + // lift's own box instead (openLifted). cursorAnchor: { position: 'absolute', width: '1px', height: '1px', pointerEvents: 'none', + visibility: 'visible', }, menu: { boxSizing: 'border-box', @@ -148,6 +182,11 @@ const styles = stylex.create({ transitionTimingFunction: easeVars['--ease-standard'], userSelect: 'none', WebkitTouchCallout: 'none', + // A touch open focuses the menu itself, so no row is lit until the finger + // moves onto one (as DropdownMenu's held finger): an internal focus target, + // not a control, so no browser ring; keyboard focus moves to an item + // instead. + outline: 'none', }, // Scroll ownership by the browser's own signal; see DropdownMenu. touchPinchZoom: { @@ -265,11 +304,13 @@ interface ContextMenuBaseProps extends BaseProps { onOpenChange?: (isOpen: boolean) => void; /** * Presentation policy for the menu. - * - `popover`: open beside the pointer position. + * - omitted: the menu at the pointer; under a finger, a held finger opens it + * on iOS's timeline (the touch model). + * - `popover`: the menu at the pointer on every device, as before; a held + * finger opens it at 500 ms through today's long press. * - `bottom-sheet`: open as an action sheet. * - `adaptive`: use a BottomSheet on compact coarse-pointer viewports and * the cursor-positioned popover elsewhere. - * @default 'popover' */ presentation?: AdaptivePresentation; 'data-testid'?: string; @@ -323,7 +364,7 @@ export function ContextMenu({ label: labelFromProps, isDisabled = false, onOpenChange, - presentation = 'popover', + presentation, triggerAs = 'div', ref, className, @@ -341,8 +382,14 @@ export function ContextMenu({ const t = useTranslator(); const label = labelFromProps ?? t('@astryx.contextMenu.label'); const backLabel = t('@astryx.dropdownMenu.back'); - const resolvedPresentation = useAdaptivePresentation(presentation); - const usesBottomSheet = resolvedPresentation === 'bottom-sheet'; + // The surface, and whether the touch model applies: under a finger unless + // the caller pins `presentation="popover"` (public-component-api INV16). + const {surface, touchModes} = usePresentation(presentation, { + whenOmitted: 'touch', + }); + // The theme's compact definition, for the one-time read at the open. + const compact = useCompactDefinition(); + const usesBottomSheet = surface === 'bottom-sheet'; // Separate content props (union discriminant) from DOM pass-through attrs. // The union means exactly one of items/menuContent exists in rest; destructure // both so triggerProps contains only DOM-safe attributes. @@ -364,6 +411,11 @@ export function ContextMenu({ // — it scrolls with the content instead of sitting at a fixed viewport point. const positionRef = useRef({x: 0, y: 0}); const cursorAnchorRef = useRef(null); + // While a lift is up, the layer's anchor is the lift's box (openLifted), + // and the layer's anchor name, for the close, which is declared before the + // layer. + const liftAnchorRef = useRef(null); + const layerAnchorIdRef = useRef(''); const triggerRef = useRef(null); // Element focused before the menu opened, restored when it closes so focus // does not fall to after Escape or outside-click dismissal. @@ -381,20 +433,53 @@ export function ContextMenu({ const currentTitle = currentSubmenu?.label ?? label; const sheetLabel = typeof currentTitle === 'string' ? currentTitle : label; + // Whether a finger opened the menu that is up (touch invocation, below). + const [isTouchOpen, setIsTouchOpen] = useState(false); + // The mounted touch layer, for its press feedback and whether it is timing + // a finger. + const touchHandleRef = useRef(null); + const setTouchHandle = useCallback( + (handle: ContextMenuTouchHandle | null) => { + touchHandleRef.current = handle; + }, + [], + ); + // The menu closed: a finger's menu is over, and the trigger springs back + // with the close itself (run by every close handler below). + const endTouchOpen = useCallback(() => { + setIsTouchOpen(false); + touchHandleRef.current?.release(); + // The lifted trigger returns, and the menu leaves its lifted placement: + // the layer's anchor is the cursor anchor again. + touchHandleRef.current?.presentation.drop(); + setLiftPlacement(null); + const liftAnchor = liftAnchorRef.current; + liftAnchorRef.current = null; + if (liftAnchor != null) { + removeAnchorName(liftAnchor, layerAnchorIdRef.current); + const anchorEl = cursorAnchorRef.current; + if (anchorEl != null) { + addAnchorName(anchorEl, layerAnchorIdRef.current); + } + } + }, []); + const updateOpenState = useCallback( (nextIsOpen: boolean) => { if (!nextIsOpen) { setSubmenuPath([]); + endTouchOpen(); } setIsOpen(nextIsOpen); onOpenChange?.(nextIsOpen); }, - [onOpenChange], + [endTouchOpen, onOpenChange], ); const layer = useLayer({ mode: 'context', onHide: useCallback(() => { + endTouchOpen(); setIsOpen(false); onOpenChange?.(false); // Restore focus to the element that was focused before opening. @@ -403,13 +488,14 @@ export function ContextMenu({ if (toRestore && document.contains(toRestore)) { toRestore.focus(); } - }, [onOpenChange]), + }, [endTouchOpen, onOpenChange]), onShow: useCallback(() => { setIsOpen(true); onOpenChange?.(true); }, [onOpenChange]), lightDismiss: false, }); + layerAnchorIdRef.current = layer.anchorId; const closeMenu = useCallback(() => { if (usesBottomSheet) { @@ -562,16 +648,56 @@ export function ContextMenu({ // trigger (rather than storing viewport coordinates on the menu itself) is // what makes the menu context-relative: it scrolls with the content and the // browser auto-flips it against the viewport edges via CSS anchor positioning. + // How much the press feedback has scaled a block trigger (from its center): + // its bounding rect is the scaled box, its offset size the unscaled one. + const triggerScale = useCallback( + (rect: DOMRect): {x: number; y: number} => { + const trigger = triggerRef.current; + if (trigger == null || triggerAs === 'span') { + return {x: 1, y: 1}; + } + return { + x: trigger.offsetWidth > 0 ? rect.width / trigger.offsetWidth : 1, + y: trigger.offsetHeight > 0 ? rect.height / trigger.offsetHeight : 1, + }; + }, + [triggerAs], + ); + const openAtLocalPoint = useCallback( - (localX: number, localY: number, focusEl: HTMLElement | null) => { + ( + localX: number, + localY: number, + focusEl: HTMLElement | null, + // A touch open focuses the menu itself: UIKit lights no row until the + // finger moves onto one. Pointer and keyboard opens focus the first + // item (APG menu pattern); arrow keys reach the items from the menu. + focusTarget: 'first' | 'menu' = 'first', + ) => { const rect = triggerRef.current?.getBoundingClientRect(); - const x = rect ? keepAnchorInside(localX, rect.width) : localX; - const y = rect ? keepAnchorInside(localY, rect.height) : localY; + const scale = rect ? triggerScale(rect) : {x: 1, y: 1}; + const x = rect ? keepAnchorInside(localX, rect.width / scale.x) : localX; + const y = rect ? keepAnchorInside(localY, rect.height / scale.y) : localY; positionRef.current = {x, y}; const anchorEl = cursorAnchorRef.current; if (anchorEl) { anchorEl.style.left = `${x}px`; anchorEl.style.top = `${y}px`; + // A trigger the press feedback has grown (from its center) would + // carry the anchor with it, and engines differ on whether anchor + // positioning follows transforms. Cancel the scale at the anchor, so + // its box is the same either way: where the point is on screen. + if (rect != null && (scale.x !== 1 || scale.y !== 1)) { + const centerX = rect.width / scale.x / 2; + const centerY = rect.height / scale.y / 2; + anchorEl.style.transformOrigin = '0 0'; + anchorEl.style.scale = `${1 / scale.x} ${1 / scale.y}`; + anchorEl.style.translate = `${(x - centerX) * (1 / scale.x - 1)}px ${(y - centerY) * (1 / scale.y - 1)}px`; + } else { + anchorEl.style.removeProperty('transform-origin'); + anchorEl.style.removeProperty('scale'); + anchorEl.style.removeProperty('translate'); + } } // Remember the element focused before opening so we can restore it on // close (Escape or outside-click), instead of dropping focus to . @@ -583,10 +709,23 @@ export function ContextMenu({ updateOpenState(true); } else { layer.show(); - requestAnimationFrame(() => focusFirst()); + requestAnimationFrame(() => { + if (focusTarget === 'menu') { + listRef.current?.focus({preventScroll: true}); + } else { + focusFirst(); + } + }); } }, - [layer, focusFirst, updateOpenState, usesBottomSheet], + [ + layer, + focusFirst, + listRef, + updateOpenState, + usesBottomSheet, + triggerScale, + ], ); // The box the cursor anchor's offsets are measured from: the trigger, which @@ -596,6 +735,54 @@ export function ContextMenu({ [], ); + // A client point in the trigger's own coordinates, where the cursor anchor + // lives: its offset from the trigger's box as laid out, before the press + // feedback's scale (from its center), kept inside that box. + const toTriggerLocal = useCallback( + (point: {x: number; y: number}): {x: number; y: number} => { + const rect = triggerRef.current?.getBoundingClientRect(); + if (rect == null) { + return point; + } + const scale = triggerScale(rect); + const width = rect.width / scale.x; + const height = rect.height / scale.y; + const left = rect.left + (rect.width - width) / 2; + const top = rect.top + (rect.height - height) / 2; + return { + x: Math.min(Math.max(point.x - left, 0), width), + y: Math.min(Math.max(point.y - top, 0), height), + }; + }, + [triggerScale], + ); + + // Opens the menu beside the lifted trigger: the layer's anchor moves to the + // lift's box at the lifted rect (an anchor inside the trigger would move + // with the trigger's lift, which anchor positioning follows), and the layer + // places the menu against it. + const openLifted = useCallback( + ({anchor}: ContextMenuLiftResult) => { + const anchorEl = cursorAnchorRef.current; + if (anchorEl != null) { + removeAnchorName(anchorEl, layer.anchorId); + } + addAnchorName(anchor, layer.anchorId); + liftAnchorRef.current = anchor; + triggerFocusRef.current = + document.activeElement instanceof HTMLElement + ? document.activeElement + : triggerRef.current; + layer.show(); + // A touch opened it: the menu itself takes the focus, so no row is lit + // until the finger moves onto one (as openAtLocalPoint's touch path). + requestAnimationFrame(() => { + listRef.current?.focus({preventScroll: true}); + }); + }, + [layer, listRef], + ); + const handleContextMenu = useCallback( (e: React.MouseEvent) => { if (isDisabled) { @@ -627,19 +814,130 @@ export function ContextMenu({ [isDisabled, openAtLocalPoint, getAnchorBaseRect], ); - // Touch long-press invocation (menus-8). iOS Safari never synthesizes a - // `contextmenu` event on long-press, so a context menu is otherwise - // unreachable on touch. Open the menu at the touch point once the press is - // held long enough (see useLongPress for timer/move-cancel/cleanup logic). - // The hook also returns the imperative start/moveTo/cancel API the menu press - // model drives; only its four touch handlers belong on the trigger element. + // The lift presentation (ContextMenuLift.tsx, run by the touch layer): when + // the menu shows under a finger, the trigger's preview lifts above a tinted + // backdrop and the menu sits beside it. + // A lift whose menu waits for its side to render, and the finger it grows + // out of. + const pendingLiftRef = useRef<{ + lifted: ContextMenuLiftResult; + press: {x: number; y: number}; + isShown: boolean; + } | null>(null); + const [liftPlacement, setLiftPlacement] = + useState(null); + // The lifted menu, once its first side (below) has rendered: show it, + // measure it, and move it above before the frame paints if it does not fit + // (a state update in a layout effect renders again before paint). It then + // grows out of the finger, from the side it settled on. + useIsomorphicLayoutEffect(() => { + const pending = pendingLiftRef.current; + if (pending == null || liftPlacement == null) { + return; + } + if (!pending.isShown) { + pending.isShown = true; + openLifted(pending.lifted); + const touch = touchHandleRef.current; + const menuHeight = listRef.current?.offsetHeight ?? 0; + if (touch != null) { + const placement = touch.placeMenu(pending.lifted, menuHeight); + if (placement.side !== liftPlacement.side) { + setLiftPlacement(placement); + return; + } + } + } + pendingLiftRef.current = null; + const menu = listRef.current?.closest('[popover]'); + if (menu != null) { + touchHandleRef.current?.presentation.morphMenu(menu, pending.press); + } + }, [liftPlacement, openLifted, listRef]); + // Touch invocation (menus-8). Under the touch model the touch layer + // (ContextMenuTouchLayer.tsx) runs UIKit's long press and asks for the menu here, at the + // client point where the finger landed. + const openFromFinger = useCallback( + (point: {x: number; y: number}) => { + // Read once, at the open: the render value can be a frame behind a + // pointer change (a trackpad docking). + const opensUnderFinger = readPresentation(presentation, { + whenOmitted: 'touch', + compact, + }).touchModes; + setIsTouchOpen(opensUnderFinger); + // UIKit's presentation under a finger (spec:AST-094 FR33): the preview + // lifts and the menu sits beside it. An inline trigger, a fine pointer, + // `presentation="popover"` and the sheet keep the menu where it opens. + const touch = touchHandleRef.current; + const trigger = triggerRef.current; + if ( + touch != null && + trigger != null && + !usesBottomSheet && + triggerAs !== 'span' && + opensUnderFinger + ) { + // The lift grows on from where the press feedback left the trigger, + // which hands the trigger's scale over to the lift. The feedback + // lets go once the lift has read the page, before it writes, so the + // page restyles once for both. + const pressed = trigger.getBoundingClientRect(); + const lifted = touch.presentation.lift( + { + x: pressed.left, + y: pressed.top, + width: pressed.width, + height: pressed.height, + }, + touch.end, + ); + // A lift that did not happen lets go too (end is idempotent). + touch.end(); + if (lifted != null) { + // The menu's side depends on its height: it renders below first, + // and the layout effect above shows it, measures it, moves it above + // before the frame paints if it does not fit, and then grows it out + // of the finger. + pendingLiftRef.current = {lifted, press: point, isShown: false}; + setLiftPlacement(touch.placeMenu(lifted, 0)); + return; + } + } + // Otherwise the menu opens under the finger (openAtLocalPoint cancels + // the trigger's scale at the anchor). + const local = toTriggerLocal(point); + openAtLocalPoint(local.x, local.y, triggerRef.current, 'menu'); + }, + [ + compact, + openAtLocalPoint, + presentation, + toTriggerLocal, + triggerAs, + usesBottomSheet, + ], + ); + // The page stays put while a menu a finger opened is up, as it does under + // UIKit's context menu. A sheet locks the page itself. (The lifted trigger + // and the menu's anchor are fixed to the viewport in the top layer, so + // pinning the page moves neither.) + useScrollLock(isOpen && isTouchOpen && !usesBottomSheet); + + // Today's long press for a finger outside the touch model: under a fine + // primary pointer, or with `presentation="popover"` (the way back, + // public-component-api INV16). iOS Safari never synthesizes a `contextmenu` + // event on a long press, so the menu opens at the touch point once the + // finger has been held for 500 ms (useLongPress: the timer, the move cancel + // and the cleanup). Under the touch model the recognizer above owns the + // finger instead. const { onTouchStart: handleLongPressTouchStart, onTouchMove: handleLongPressTouchMove, onTouchEnd: handleLongPressTouchEnd, onTouchCancel: handleLongPressTouchCancel, } = useLongPress({ - disabled: isDisabled, + disabled: isDisabled || touchModes, onLongPress: useCallback( (point: {x: number; y: number}) => { const rect = getAnchorBaseRect(); @@ -655,17 +953,6 @@ export function ContextMenu({ [openAtLocalPoint, getAnchorBaseRect], ), }); - - // ContextMenu owns invocation and long-press cleanup. Consumer handlers are - // notified too, but preventing their default does not cancel component-owned - // behavior that the public API does not promise as cancelable. - const handleTriggerContextMenu = useCallback( - (event: React.MouseEvent) => { - handleContextMenu(event); - onContextMenuProp?.(event); - }, - [handleContextMenu, onContextMenuProp], - ); const handleTriggerTouchStart = useCallback( (event: React.TouchEvent) => { handleLongPressTouchStart(event); @@ -695,6 +982,23 @@ export function ContextMenu({ [handleLongPressTouchCancel, onTouchCancelProp], ); + // ContextMenu owns invocation. Consumer handlers are notified too, but + // preventing their default does not cancel component-owned behavior that + // the public API does not promise as cancelable. + const handleTriggerContextMenu = useCallback( + (event: React.MouseEvent) => { + if (touchHandleRef.current?.isTracking() === true) { + // The browser's own long-press menu (Android Chrome) for a finger the + // recognizer is timing: the recognizer decides, not this event. + event.preventDefault(); + } else { + handleContextMenu(event); + } + onContextMenuProp?.(event); + }, + [handleContextMenu, onContextMenuProp], + ); + const popoverXstyle = menuWidth ? styles.popoverCustomWidth(menuWidth) : styles.popover; @@ -834,6 +1138,10 @@ export function ContextMenu({ + {touchModes && !isDisabled ? ( + + ) : null} {usesBottomSheet ? ( @@ -871,11 +1189,21 @@ export function ContextMenu({ ) : ( - layer.render({renderedMenu}, { - placement: 'below', - alignment: 'start', - xstyle: [popoverXstyle, layerAnimations.below], - }) + layer.render( + {renderedMenu}, + liftPlacement != null + ? { + placement: liftPlacement.side, + alignment: liftPlacement.alignment, + offset: liftPlacement.gap, + xstyle: [popoverXstyle], + } + : { + placement: 'below', + alignment: 'start', + xstyle: [popoverXstyle, layerAnimations.below], + }, + ) )} ); diff --git a/packages/core/src/ContextMenu/ContextMenuLift.test.tsx b/packages/core/src/ContextMenu/ContextMenuLift.test.tsx new file mode 100644 index 0000000000000..f407111cfe418 --- /dev/null +++ b/packages/core/src/ContextMenu/ContextMenuLift.test.tsx @@ -0,0 +1,72 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file ContextMenuLift.test.tsx + * @input vitest (jsdom), Testing Library, useContextMenuLift with the lift + * module stubbed so its options can be read + * @output Checks that every spring the context menu's lift plays is an + * internal motion role (spec:AST-094 FR4): the preview's growth, its return, + * the tint, the shadow and the menu + * @position Testing for ContextMenuLift.tsx + */ + +import {render} from '@testing-library/react'; +import {useRef} from 'react'; +import {expect, it, vi} from 'vitest'; +import type {LiftFromTriggerOptions} from '../liftFromTrigger/liftFromTrigger'; +import {motionRoleOf} from '../motion/roles'; +import {useContextMenuLift} from './ContextMenuLift'; + +const seen: {options: LiftFromTriggerOptions | null} = {options: null}; + +vi.mock('../liftFromTrigger/useLiftFromTrigger', () => ({ + useLiftFromTrigger: () => ({ + lift: (_source: HTMLElement, options: LiftFromTriggerOptions) => { + seen.options = options; + return null; + }, + presentSurface: () => {}, + dismissSurface: () => {}, + drop: () => {}, + dispose: () => {}, + isLifted: () => false, + }), +})); + +function Lifted({onReady}: {onReady: (lift: () => void) => void}) { + const ref = useRef(null); + const lift = useContextMenuLift(ref); + onReady(() => { + lift.lift(); + }); + return
card
; +} + +it('plays every spring of the lift through an internal motion role (spec:AST-094 FR4)', () => { + let run: () => void = () => {}; + render( + { + run = lift; + }} + />, + ); + run(); + const motion = seen.options?.motion ?? {}; + expect( + Object.fromEntries( + Object.entries(motion).map(([use, spring]) => [ + use, + motionRoleOf(spring)?.id ?? null, + ]), + ), + ).toEqual({ + lift: 'springs.contextMenuLift', + drop: 'springs.contextMenuMorph', + clear: 'springs.contextMenuDismiss', + shadowIn: 'springs.contextMenuMorph', + shadowOut: 'springs.contextMenuMorph', + surfaceIn: 'springs.contextMenuMorph', + surfaceOut: 'springs.contextMenuMorph', + }); +}); diff --git a/packages/core/src/ContextMenu/ContextMenuLift.tsx b/packages/core/src/ContextMenu/ContextMenuLift.tsx new file mode 100644 index 0000000000000..f5caca0865020 --- /dev/null +++ b/packages/core/src/ContextMenu/ContextMenuLift.tsx @@ -0,0 +1,148 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file ContextMenuLift.tsx + * @input The trigger element, the lift module (liftFromTrigger/), the context-menu + * constants and springs, the app's declared layer inset (LayerProvider) + * @output Exports useContextMenuLift: lift() / drop() to show UIKit's lifted + * preview over a tinted backdrop and to return it to its place, and the + * menu's growth out of the finger beside it, on the lift module with + * UIKit's numbers + * @position The lift presentation of ContextMenu's touch invocation, run by + * the touch layer (ContextMenuTouchLayer.tsx); ContextMenu.tsx places the + * menu beside the preview it lifts + * + * When a context menu shows on iOS 26.2, the long-pressed view lifts out of + * the page (UIKit's preview platter): it grows on from the size the press + * gave it to its lifted size, a shadow rising under it, above a backdrop + * that tints the page (no blur, no zoom), and the menu grows out of the + * finger beside it. + * When the menu closes, the view returns to its place while the backdrop and + * the shadow clear. + * + * The lift module (liftFromTrigger/liftFromTrigger.ts) plays it: the trigger itself lifts into the + * top layer, with one placeholder holding its spot, and nothing of the lift + * is in the DOM at rest. ContextMenu supplies UIKit's numbers: the scale rule + * (`contextMenu.lift`), its measured springs (internal: touch motion has no + * theme surface), the + * tint (the theme's modal overlay, `--color-overlay`; UIKit's own is #16152A + * at 21 %) and the shadow (the theme's `--shadow-high`; UIKit's: black at + * 0.30, radius 20, no offset, shaped by the view's pixels). A block trigger + * lifts; an inline one keeps the cursor menu. The menu grows out of the + * press from `contextMenu.lift.morphStartWidth`, in its own aspect, on + * `springs.contextMenuMorph` (the module's rect morph: translate, scale and opacity only, + * so UIKit's blob and the content's blur and clip are not played). + + * + * SYNC: When modified, update /packages/core/src/ContextMenu/ContextMenu.tsx + * and /packages/core/src/ContextMenu/__tests__/ContextMenuLift.interaction.browser.spec.ts + */ + +import {use, useMemo, useRef, type RefObject} from 'react'; +import {LayerContext, type LayerInset} from '../Layer/LayerContext'; +import type { + LiftFromTriggerMotion, + LiftFromTriggerResult, +} from '../liftFromTrigger/liftFromTrigger'; +import {useLiftFromTrigger} from '../liftFromTrigger/useLiftFromTrigger'; +import type {MorphRect} from '../motion/rectMorph'; +import { + contextMenu, + registerContextMenuTable, +} from '../motion/uikit/contextMenu'; +import {registerSpringsTable, springs} from '../motion/uikit/springs'; + +const LIFT = contextMenu.lift; + +/** + * The lift's springs, as measured, each an internal motion role + * (`spec:AST-094` FR4): the preview grows and the tint fades in on + * `springs.contextMenuLift`, the menu enters and the shadow rises and fades on + * `springs.contextMenuMorph`, the preview returns on `springs.contextMenuMorph` + * while the tint clears on `springs.contextMenuDismiss`. + */ +const MOTION: LiftFromTriggerMotion = { + lift: springs.contextMenuLift, + drop: springs.contextMenuMorph, + clear: springs.contextMenuDismiss, + shadowIn: springs.contextMenuMorph, + shadowOut: springs.contextMenuMorph, + surfaceIn: springs.contextMenuMorph, + surfaceOut: springs.contextMenuMorph, +}; + +/** Where a lift landed: the view's box, the preview's, and the viewport. */ +export type ContextMenuLiftResult = LiftFromTriggerResult; + +export interface ContextMenuLift { + /** + * Lift the trigger from `from` (where the press feedback left it, client + * px; its box by default). `release` runs once the lift has read the page, + * before it writes: the press feedback lets go there. Returns where it + * landed, or null without the popover API. + */ + lift(from?: MorphRect, release?: () => void): ContextMenuLiftResult | null; + /** + * The menu's entrance beside the lifted preview: it grows out of `press` + * (client px) into its frame as it fades in. The lift layer takes no touch + * until the lift and the menu settle (spec:AST-094 FR35). + */ + morphMenu(menu: HTMLElement, press: {x: number; y: number}): void; + /** Return the preview to its place. */ + drop(): void; +} + +export function useContextMenuLift( + triggerRef: RefObject, + { + onBackdropTap, + }: { + /** + * A finger lifted on the backdrop: UIKit ends the presentation at that + * lift. (WebKit sends no mouse events for a tap on the bare backdrop, so + * the document's outside-press handler cannot stand in.) + */ + onBackdropTap?: () => void; + } = {}, +): ContextMenuLift { + const lift = useLiftFromTrigger(); + const onBackdropTapRef = useRef(onBackdropTap); + onBackdropTapRef.current = onBackdropTap; + // The app's declared inset (LayerProvider), which the gutter adds. + const layerInsetRef = useRef(undefined); + layerInsetRef.current = use(LayerContext)?.inset; + return useMemo( + () => ({ + lift(from, release) { + const trigger = triggerRef.current; + if (trigger == null) { + return null; + } + // The springs' tables register before the first motion (idempotent), + // so every spring the lift plays resolves through its role + // (spec:AST-094 FR4). + registerContextMenuTable(); + registerSpringsTable(); + return lift.lift(trigger, { + motion: MOTION, + scale: {maxScale: LIFT.maxScale, scaleUpPoints: LIFT.scaleUpPoints}, + surfaceStartWidth: LIFT.morphStartWidth, + from, + release, + inset: layerInsetRef.current, + onBackdropTap: () => onBackdropTapRef.current?.(), + attribute: 'data-astryx-context-menu-lift', + }); + }, + morphMenu(menu, press) { + lift.presentSurface(menu, press); + }, + drop() { + lift.drop(); + }, + }), + [lift, triggerRef], + ); +} diff --git a/packages/core/src/ContextMenu/ContextMenuTouch.ts b/packages/core/src/ContextMenu/ContextMenuTouch.ts new file mode 100644 index 0000000000000..a5dac1bddbe98 --- /dev/null +++ b/packages/core/src/ContextMenu/ContextMenuTouch.ts @@ -0,0 +1,26 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file ContextMenuTouch.ts + * @input ContextMenu's touch modules + * @output ContextMenu's touch layer, its one import site: the press + * recognizer (contextMenuPress.ts), the press feedback + * (contextMenuPressFeedback.ts) and the lift presentation + * (ContextMenuLift.tsx, contextMenuLiftGeometry.ts) + * @position ContextMenuTouchLayer.tsx, which ContextMenu imports + * under the touch model, imports its touch behaviour from here only; the + * mouse and keyboard paths stay in ContextMenu.tsx + * + * SYNC: When modified, update /packages/core/src/ContextMenu/ContextMenuTouchLayer.tsx + */ + +export {ContextMenuPressRecognizer} from './contextMenuPress'; +export {useContextMenuPressFeedback} from './contextMenuPressFeedback'; +export { + useContextMenuLift, + type ContextMenuLift, + type ContextMenuLiftResult, +} from './ContextMenuLift'; +export {menuPlacement, type MenuPlacement} from './contextMenuLiftGeometry'; diff --git a/packages/core/src/ContextMenu/ContextMenuTouchLayer.tsx b/packages/core/src/ContextMenu/ContextMenuTouchLayer.tsx new file mode 100644 index 0000000000000..468b5de6635d4 --- /dev/null +++ b/packages/core/src/ContextMenu/ContextMenuTouchLayer.tsx @@ -0,0 +1,161 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file ContextMenuTouchLayer.tsx + * @input ContextMenu's touch layer (ContextMenuTouch.ts), the gesture + * foundation and the UIKit context-menu constants + * @output ContextMenuTouchLayer: renders nothing; runs the trigger's press + * recognizer, its press feedback and the lift presentation while + * ContextMenu is under the touch model + * @position Imported by ContextMenu, which renders it only while the touch + * model applies: a fine pointer runs none of it + * + * SYNC: When modified, update /packages/core/src/ContextMenu/ContextMenu.tsx + */ + +import {useEffect, useRef, type RefObject} from 'react'; +import {playHaptic, useGestureRecognizer} from '../gestures'; +import {contextMenu} from '../motion/uikit/contextMenu'; +import { + ContextMenuPressRecognizer, + menuPlacement, + useContextMenuLift, + useContextMenuPressFeedback, + type ContextMenuLift, + type ContextMenuLiftResult, + type MenuPlacement, +} from './ContextMenuTouch'; + +/** Where the lifted menu sits, and its gap from the lifted preview, in px. */ +export interface LiftedMenuPlacement extends MenuPlacement { + readonly gap: number; +} + +/** What ContextMenu reaches of a mounted touch layer. */ +export interface ContextMenuTouchHandle { + /** The menu the press opened closed: the trigger springs back. */ + release(): void; + /** The trigger stops growing at once: the lift takes it from here. */ + end(): void; + /** Whether the recognizer is timing a finger on the trigger. */ + isTracking(): boolean; + /** + * The lift presentation (ContextMenuLift.tsx): the trigger lifted above + * the tint, and the menu's entrance and exit beside it. + */ + readonly presentation: ContextMenuLift; + /** Where the lifted menu sits for its rendered height (contextMenu.lift). */ + placeMenu( + lifted: ContextMenuLiftResult, + menuHeight: number, + ): LiftedMenuPlacement; +} + +export interface ContextMenuTouchLayerProps { + triggerRef: RefObject; + /** False for an inline trigger, which does not grow. */ + isShown: boolean; + /** Changes when the trigger element is replaced (`triggerAs`). */ + triggerKey: unknown; + /** Receives the layer's handle as it mounts, and null as it unmounts. */ + onHandleChange: (handle: ContextMenuTouchHandle | null) => void; + /** Open the menu at a client point (where the finger landed). */ + onOpen: (point: {x: number; y: number}) => void; + /** A finger lifted on the lift's tint: the menu closes. */ + onBackdropTap: () => void; +} + +/** + * UIContextMenuInteraction's long press. iOS Safari never fires `contextmenu` + * for a held finger, so the trigger runs its own recognizer on UIKit's + * timeline (contextMenu.press; the states are described in + * contextMenuPress.ts). At the commit the haptic plays and the recognizer + * claims the finger (the page stops scrolling under it, and its lift sends no + * click and no compatibility mouse events). The menu opens where the finger + * landed: at `menuDisplayMs` if it is still down, or at the lift. A lift + * before the commit, a second finger, a scroll, or a drift past the tolerance + * before the menu shows gets no menu. The press feedback grows the trigger + * while core's press machine believes the press, springs back when it ends + * with no menu, and holds while the menu it opened is up. + */ +export function ContextMenuTouchLayer({ + triggerRef, + isShown, + triggerKey, + onHandleChange, + onOpen, + onBackdropTap, +}: ContextMenuTouchLayerProps): null { + const feedback = useContextMenuPressFeedback(triggerRef, isShown, triggerKey); + // The lift presentation: ContextMenu decides when a press lifts the + // trigger, and drives it through the handle. + const lift = useContextMenuLift(triggerRef, {onBackdropTap}); + const liftRef = useRef(lift); + liftRef.current = lift; + const feedbackRef = useRef(feedback); + feedbackRef.current = feedback; + const onOpenRef = useRef(onOpen); + onOpenRef.current = onOpen; + // Whether the press in flight has opened its menu. + const hasPressOpenedRef = useRef(false); + + // The gesture hook follows the element behind the ref, so a `triggerAs` + // swap moves the recognizer to the new node. + const recognizer = useGestureRecognizer( + triggerRef, + () => new ContextMenuPressRecognizer(), + press => { + const open = () => { + // The press feedback has grown the trigger from its center, and the + // cursor anchor inside it moves with that scale: stop the growth + // where it is; ContextMenu places the anchor under the finger. + feedbackRef.current.hold(); + hasPressOpenedRef.current = true; + onOpenRef.current(press.press ?? press.location()); + }; + switch (press.state) { + case 'began': + hasPressOpenedRef.current = false; + // The click haptic, where a vibration motor is the only haptic. + // iOS Safari can play none here: no tap lands at the commit. + playHaptic(contextMenu.commitHaptic); + break; + case 'changed': + if (press.isMenuDue && !hasPressOpenedRef.current) { + open(); + } + break; + case 'ended': + if (!hasPressOpenedRef.current) { + open(); + } + break; + case 'possible': + case 'failed': + case 'cancelled': + // A press that ends with no menu: the press feedback springs back. + break; + } + }, + ); + + useEffect(() => { + onHandleChange({ + release: () => feedbackRef.current.release(), + end: () => feedbackRef.current.end(), + get presentation() { + return liftRef.current; + }, + placeMenu: (lifted, menuHeight) => ({ + ...menuPlacement(lifted.lifted, menuHeight, lifted.viewport), + gap: contextMenu.lift.menuGap, + }), + isTracking: () => recognizer.numberOfTouches > 0, + }); + return () => onHandleChange(null); + }, [onHandleChange, recognizer]); + + return null; +} diff --git a/packages/core/src/ContextMenu/ContextMenuTouchLayerStatic.test.tsx b/packages/core/src/ContextMenu/ContextMenuTouchLayerStatic.test.tsx new file mode 100644 index 0000000000000..9af1bfeae205b --- /dev/null +++ b/packages/core/src/ContextMenu/ContextMenuTouchLayerStatic.test.tsx @@ -0,0 +1,70 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file ContextMenuTouchLayerStatic.test.tsx + * @input vitest (jsdom), Testing Library, ContextMenu under a coarse pointer + * @output Checks that a finger landing on the first render already runs + * UIKit's timeline: the touch layer is part of ContextMenu, not a load away + * @position Testing for ContextMenu.tsx and ContextMenuTouchLayer.tsx + */ + +import {act, fireEvent, render, screen} from '@testing-library/react'; +import {afterEach, beforeAll, beforeEach, expect, it, vi} from 'vitest'; +import type {ContextMenu as ContextMenuComponent} from './ContextMenu'; + +let ContextMenu: typeof ContextMenuComponent; + +// A cold import transforms ContextMenu's whole module graph; allow for it. +beforeAll(async () => { + ({ContextMenu} = await import('./ContextMenu')); +}, 120000); + +beforeEach(() => { + HTMLElement.prototype.showPopover = vi.fn(); + HTMLElement.prototype.hidePopover = vi.fn(); + vi.stubGlobal( + 'matchMedia', + vi.fn().mockImplementation((query: string) => ({ + matches: query === '(pointer: coarse)', + media: query, + onchange: null, + addEventListener: vi.fn(), + removeEventListener: vi.fn(), + addListener: vi.fn(), + removeListener: vi.fn(), + dispatchEvent: vi.fn(), + })), + ); +}); + +afterEach(() => { + vi.useRealTimers(); + vi.unstubAllGlobals(); +}); + +it('a finger that lands on the first render runs UIKit’s timeline: no menu at 500 ms, the menu at 735 ms', () => { + vi.useFakeTimers(); + render( + +
Hold me
+ , + ); + const trigger = screen.getByTestId('touch'); + // The press recognizer is on the trigger from the first commit. + expect(trigger).toHaveAttribute('data-astryx-pressable'); + fireEvent.pointerDown(trigger, { + pointerId: 1, + pointerType: 'touch', + isPrimary: true, + clientX: 20, + clientY: 20, + }); + fireEvent.touchStart(trigger, { + touches: [{identifier: 1, clientX: 20, clientY: 20}], + }); + act(() => { + vi.advanceTimersByTime(500); + }); + // Today's 500 ms long press never runs under the touch model. + expect(HTMLElement.prototype.showPopover).not.toHaveBeenCalled(); +}); diff --git a/packages/core/src/ContextMenu/__tests__/ContextMenuLift.interaction.browser.spec.ts b/packages/core/src/ContextMenu/__tests__/ContextMenuLift.interaction.browser.spec.ts new file mode 100644 index 0000000000000..5b108cb763454 --- /dev/null +++ b/packages/core/src/ContextMenu/__tests__/ContextMenuLift.interaction.browser.spec.ts @@ -0,0 +1,1031 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file ContextMenuLift.interaction.browser.spec.ts + * @input The built "Touch / long press" ContextMenu story, real touch input + * (Chromium CDP / WebKit's input command), the interaction harness, the + * context-menu constants and springs + * @output Interaction and animation proofs for the lift presentation: a held + * card lifts itself (no copy) to UIKit's size about its center over the + * theme's backdrop colour with the theme's shadow, its spot in the page + * held, with the menu 16 px below it and centered; open adds the lift + * layer, its anchor and the placeholder and nothing else; a card near the + * bottom gets its menu above; a card at the left edge lifts 16 px in with + * its menu flush; the card grows on from the press, and the backdrop fades + * in, on springs.contextMenuLift frame by frame; the menu fades in on + * springs.contextMenuMorph with no blur; closing returns the card to the page as it was + * and clears the backdrop; the card keeps growing on every frame through a + * blocked main thread; under Reduce Motion the card cross-fades in place; + * a storyboard of the lift + * @position Interaction specs of packages/core/src/ContextMenu, beside + * ContextMenuLongPress.interaction.browser.spec.ts (the press timeline) + */ + +import {expect, test, type Page} from '@playwright/test'; +import { + animationStoryboard, + busyWindowProof, + castStoryboard, + holdAnimations, + openStory, + releaseAnimations, + seekAnimations, + startScreencast, + tap, + timeline, + touchDriver, + trackMarker, + withStorybook, + writeEvidence, + type BusyWindow, + type TouchDriver, +} from '@astryxdesign/interaction-harness'; +import {solveSpring, toSpring} from '../../motion/spring'; +import {springs} from '../../motion/uikit'; +import {contextMenu} from '../../motion/uikit/contextMenu'; + +const storybook = withStorybook(test); +const {menuDisplayMs} = contextMenu.press; +const {menuGap} = contextMenu.lift; +// The layer runtime's viewport gutter in these pages (spec:AST-059): the +// spacing-4 step, with no safe area and no app-declared inset. +const GUTTER = 16; + +interface Box { + x: number; + y: number; + width: number; + height: number; +} + +interface LiftStory { + driver: TouchDriver; + center: {x: number; y: number}; +} + +/** + * Opens the story; `moveCard` gets the card's box and returns the offset + * (CSS px) to move it by, for the placement cases. + */ +async function openLiftStory( + page: Page, + moveCard?: ( + box: Box, + viewport: {width: number; height: number}, + ) => { + dx: number; + dy: number; + }, +): Promise { + // One touch model: the engine's own finger (a coarse pointer) lifts. + await openStory(page, storybook(), 'core-contextmenu--touch-long-press'); + const trigger = page.getByTestId('touch-context-menu'); + // The touch layer, the lift included, marks the trigger pressable once it + // runs (from the first render under the touch model). + await expect(trigger).toHaveAttribute('data-astryx-pressable', '', { + timeout: 10_000, + }); + let box = await trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + if (moveCard != null) { + const viewport = page.viewportSize(); + if (viewport == null) { + throw new Error('no viewport'); + } + const {dx, dy} = moveCard(box, viewport); + await trigger.evaluate( + (el, offset) => { + const host = el as HTMLElement; + host.style.position = 'relative'; + host.style.left = `${offset.dx}px`; + host.style.top = `${offset.dy}px`; + }, + {dx, dy}, + ); + box = await trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + } + return { + driver: await touchDriver(page), + center: {x: box.x + box.width / 2, y: box.y + box.height / 2}, + }; +} + +/** + * The card's spot in the page (its placeholder while it is lifted), the + * lifted card, the menu's popover and the backdrop, now. + */ +async function readLift(page: Page) { + return page.evaluate(() => { + const rect = (el: Element | null | undefined) => { + if (el == null) { + return null; + } + const r = el.getBoundingClientRect(); + return {x: r.x, y: r.y, width: r.width, height: r.height}; + }; + const trigger = document.querySelector( + '[data-testid="touch-context-menu"]', + ); + const isLifted = trigger?.matches(':popover-open') ?? false; + const placeholder = document.querySelector( + '[data-astryx-context-menu-lift-placeholder]', + ); + // The tint is the lift layer's ::backdrop; the shadow is a box in it. + const layer = document.querySelector('[data-astryx-context-menu-lift]'); + const tint = layer == null ? null : getComputedStyle(layer, '::backdrop'); + return { + card: rect(placeholder ?? trigger), + cardOpacity: trigger == null ? null : getComputedStyle(trigger).opacity, + preview: isLifted ? rect(trigger) : null, + menu: rect(document.querySelector('[role="menu"]')?.closest('[popover]')), + backdropColor: tint == null ? null : tint.backgroundColor, + // The theme's modal overlay where the card is, for comparison. + overlayColor: (() => { + const scope = trigger?.parentElement; + if (scope == null) { + return null; + } + const probe = document.createElement('span'); + probe.style.backgroundColor = 'var(--color-overlay)'; + scope.append(probe); + const color = getComputedStyle(probe).backgroundColor; + probe.remove(); + return color; + })(), + shadow: (() => { + const shadow = document.querySelector( + '[data-astryx-context-menu-lift-shadow]', + ); + if (shadow == null) { + return null; + } + const style = getComputedStyle(shadow); + return {opacity: Number(style.opacity), boxShadow: style.boxShadow}; + })(), + backdropOpacity: tint == null ? null : Number(tint.opacity), + layerOpen: layer?.matches(':popover-open') ?? false, + }; + }); +} + +/** + * Marks every element matching `selector` (the component creates them) for + * the screencast tracker: an inset outline in the harness's marker colour. + */ +async function markSelector(page: Page, selector: string) { + await page.evaluate(sel => { + const style = document.createElement('style'); + style.textContent = `${sel}{outline:6px solid rgb(255,0,255) !important;outline-offset:-6px !important;}`; + document.head.append(style); + }, selector); +} + +/** + * Holds the main thread `ms` long, `afterMs` after the next touch lands (a + * timer set in the touch's own task, so it runs on the page's clock). Returns + * a reader for the window it held, epoch ms. + */ +async function blockAfterTouch( + page: Page, + afterMs: number, + ms: number, +): Promise<() => Promise> { + await page.evaluate( + ({at, length}) => { + addEventListener( + 'touchstart', + () => { + setTimeout(() => { + const from = performance.timeOrigin + performance.now(); + const end = performance.now() + length; + while (performance.now() < end) { + // Held: no task, frame or event runs. + } + (window as unknown as {__busy: BusyWindow}).__busy = { + from, + to: performance.timeOrigin + performance.now(), + }; + }, at); + }, + {once: true, capture: true, passive: true}, + ); + }, + {at: afterMs, length: ms}, + ); + return async () => + page.evaluate(() => (window as unknown as {__busy: BusyWindow}).__busy); +} + +/** A finger held until the menu shows, then lifted (the menu stays). */ +/** + * Until the lift has grown: the layer runtime's hold, which keeps touches off + * the lift layer while it grows (AST-094 FR35), has ended. + */ +async function untilGrown(page: Page) { + await expect + .poll( + async () => + page.evaluate(() => { + const layer = document.querySelector( + '[data-astryx-context-menu-lift]', + ); + return ( + layer != null && + ![...layer.children].some(child => + child.matches('div[popover]:popover-open'), + ) + ); + }), + {timeout: 4000}, + ) + .toBe(true); +} + +async function liftAndSettle(page: Page, {driver, center}: LiftStory) { + await driver.play(tap(center, {holdMs: menuDisplayMs + 100})); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + // The lift settles within about half a second (Linux WebKit starts a + // page's first animations late, so wait for the shadow to be in, too). + await page.waitForTimeout(800); + await expect + .poll(async () => (await readLift(page)).shadow?.opacity, {timeout: 4000}) + .toBe(1); + await page.waitForTimeout(200); + return readLift(page); +} + +test('under a finger the story has the fine pointer’s element count at rest, and open adds only the lift layer, its shadow and anchor, and the placeholder, whatever the card holds (INV7, AST-017 FR60)', async ({ + page, + browser, +}) => { + const count = async (target: Page) => + target.evaluate(() => document.body.querySelectorAll('*').length); + // The fine pointer's counts, in a context without touch. + const fineContext = await browser.newContext({ + viewport: page.viewportSize() ?? {width: 402, height: 874}, + hasTouch: false, + isMobile: false, + }); + const fine = await fineContext.newPage(); + await openStory(fine, storybook(), 'core-contextmenu--touch-long-press'); + // A fine pointer in this context, before any count. + expect( + await fine.evaluate(() => matchMedia('(pointer: coarse)').matches), + ).toBe(false); + const fineRest = await count(fine); + const fineBox = await fine.getByTestId('touch-context-menu').boundingBox(); + if (fineBox == null) { + throw new Error('no trigger'); + } + await fine.mouse.click( + fineBox.x + fineBox.width / 2, + fineBox.y + fineBox.height / 2, + {button: 'right'}, + ); + await expect(fine.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + const fineOpen = await count(fine); + await fineContext.close(); + + const story = await openLiftStory(page); + const coarseRest = await count(page); + const settled = await liftAndSettle(page, story); + const coarseOpen = await count(page); + const lift = await page.evaluate(() => { + const layer = document.querySelector('[data-astryx-context-menu-lift]'); + const trigger = document.querySelector( + '[data-testid="touch-context-menu"]', + ); + return { + // The layer holds the shadow and the box the menu is placed against, + // nothing else. + layerChildren: layer == null ? -1 : layer.children.length, + hasAnchor: + layer?.querySelector('[data-astryx-context-menu-lift-anchor]') != null, + // The card lifts itself: no copy of it anywhere. + copies: document.querySelectorAll('[data-astryx-context-menu-preview]') + .length, + isLifted: trigger?.matches(':popover-open') ?? false, + placeholderAfterTrigger: + trigger?.nextElementSibling?.hasAttribute( + 'data-astryx-context-menu-lift-placeholder', + ) ?? false, + triggerDescendants: + trigger == null ? -1 : trigger.querySelectorAll('*').length, + }; + }); + const detail = JSON.stringify({ + fineRest, + fineOpen, + coarseRest, + coarseOpen, + lift, + }); + expect(coarseRest, detail).toBe(fineRest); + expect(lift.layerChildren, detail).toBe(2); + expect(lift.hasAnchor, detail).toBe(true); + expect(lift.copies, detail).toBe(0); + expect(lift.isLifted, detail).toBe(true); + expect(lift.placeholderAfterTrigger, detail).toBe(true); + // The layer, the shadow, the anchor and the placeholder: 4, within the 5 + // an open layer may add under touch (AST-017 FR60, DEC-29), however many + // elements the card holds. + expect(coarseOpen - fineOpen, detail).toBe(4); + // A tap on the tint, on the card's own spot clear of the lifted card. + if (settled.card == null) { + throw new Error(detail); + } + await story.driver.play( + tap({ + x: settled.card.x + 10, + y: settled.card.y + settled.card.height + 8, + }), + ); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeHidden(); + await expect.poll(async () => count(page), {timeout: 3000}).toBe(coarseRest); +}); + +test('a held card lifts itself to UIKit’s size about its center over the theme’s dim and shadow, its spot in the page held, its menu 16 px below and centered', async ({ + page, +}) => { + const story = await openLiftStory(page); + // Closed, the lift layer takes no room and no touches. + await expect(page.locator('[data-astryx-context-menu-lift]')).toBeHidden(); + const lift = await liftAndSettle(page, story); + const {card, preview, menu} = lift; + if (card == null || preview == null || menu == null) { + throw new Error(JSON.stringify(lift)); + } + const detail = JSON.stringify(lift); + // 200 × 120 lifts by 26 pt on its longer side: 226 × 135.6. + expect(preview.width, detail).toBeCloseTo(226, 0); + expect(preview.height, detail).toBeCloseTo(135.6, 0); + expect(preview.x + preview.width / 2, detail).toBeCloseTo( + card.x + card.width / 2, + 0, + ); + expect(preview.y + preview.height / 2, detail).toBeCloseTo( + card.y + card.height / 2, + 0, + ); + expect(lift.cardOpacity, detail).toBe('1'); + // The theme's modal overlay (`--color-overlay`), fully faded in. + expect(lift.overlayColor, detail).not.toBeNull(); + expect(lift.backdropColor, detail).toBe(lift.overlayColor); + // The theme's high shadow, faded in under the card. + expect(lift.shadow?.opacity, detail).toBeCloseTo(1, 2); + expect(lift.shadow?.boxShadow, detail).not.toBe('none'); + expect(lift.backdropOpacity, detail).toBeCloseTo(1, 2); + expect(menu.y, detail).toBeCloseTo(preview.y + preview.height + menuGap, 0); + expect(menu.x + menu.width / 2, detail).toBeCloseTo( + preview.x + preview.width / 2, + 0, + ); +}); + +test('a card near the bottom gets its menu above, and stays inside the layer runtime’s gutter', async ({ + page, +}) => { + const story = await openLiftStory(page, (box, viewport) => ({ + dx: 0, + // The card's bottom 10 px from the viewport's. + dy: viewport.height - 10 - (box.y + box.height), + })); + const lift = await liftAndSettle(page, story); + const {preview, menu} = lift; + if (preview == null || menu == null) { + throw new Error(JSON.stringify(lift)); + } + const viewport = page.viewportSize(); + const detail = JSON.stringify({lift, viewport}); + expect(preview.y + preview.height, detail).toBeCloseTo( + (viewport?.height ?? NaN) - GUTTER, + 0, + ); + expect(menu.y + menu.height, detail).toBeCloseTo(preview.y - menuGap, 0); +}); + +test('a card at the left edge lifts 16 px in, and its menu is flush with it', async ({ + page, +}) => { + const story = await openLiftStory(page, box => ({dx: 4 - box.x, dy: 0})); + const lift = await liftAndSettle(page, story); + const {preview, menu} = lift; + if (preview == null || menu == null) { + throw new Error(JSON.stringify(lift)); + } + const detail = JSON.stringify(lift); + expect(preview.x, detail).toBeCloseTo(GUTTER, 0); + expect(menu.x, detail).toBeCloseTo(GUTTER, 0); +}); + +test('in a right-to-left page a card at the left edge keeps its menu flush with that edge (the alignment is logical)', async ({ + page, +}) => { + const story = await openLiftStory(page); + const center = await page.getByTestId('touch-context-menu').evaluate(el => { + document.documentElement.dir = 'rtl'; + const host = el as HTMLElement; + host.style.position = 'relative'; + host.style.left = '0px'; + const box = host.getBoundingClientRect(); + host.style.left = `${4 - box.x}px`; + const moved = host.getBoundingClientRect(); + return {x: moved.x + moved.width / 2, y: moved.y + moved.height / 2}; + }); + const lift = await liftAndSettle(page, {...story, center}); + const {preview, menu} = lift; + if (preview == null || menu == null) { + throw new Error(JSON.stringify(lift)); + } + const detail = JSON.stringify(lift); + expect(preview.x, detail).toBeCloseTo(GUTTER, 0); + expect(menu.x, detail).toBeCloseTo(GUTTER, 0); +}); + +test('the menu shows above the lifted card and takes the finger: its rows are on top', async ({ + page, +}) => { + const story = await openLiftStory(page); + await liftAndSettle(page, story); + const share = await page.getByRole('menuitem', {name: 'Share'}).boundingBox(); + if (share == null) { + throw new Error('no Share row'); + } + const hit = await page.evaluate( + ([x, y]) => + document.elementFromPoint(x, y)?.closest('[role="menuitem"]') + ?.textContent ?? null, + [share.x + share.width / 2, share.y + share.height / 2], + ); + expect(hit).toContain('Share'); +}); + +test('the card itself lifts into the top layer while its spot in the page holds, and is back in the page as it was once the menu closes', async ({ + page, +}) => { + const {driver, center} = await openLiftStory(page); + const trigger = page.getByTestId('touch-context-menu'); + const menu = page.getByRole('menu', {name: 'Photo actions'}); + const before = await trigger.evaluate(el => { + const r = el.getBoundingClientRect(); + return { + style: (el as HTMLElement).style.cssText, + box: [r.x, r.y, r.width, r.height], + next: (() => { + const sibling = el.parentElement?.nextElementSibling; + if (sibling == null) { + return null; + } + const n = sibling.getBoundingClientRect(); + return [n.x, n.y]; + })(), + }; + }); + await driver.play(tap(center, {holdMs: menuDisplayMs + 100})); + await expect(menu).toBeVisible(); + const lifted = await trigger.evaluate(el => { + const placeholder = document.querySelector( + '[data-astryx-context-menu-lift-placeholder]', + ); + const r = placeholder?.getBoundingClientRect(); + const sibling = el.parentElement?.nextElementSibling; + const n = sibling?.getBoundingClientRect(); + return { + isOpen: el.matches(':popover-open'), + spot: r == null ? null : [r.x, r.y, r.width, r.height], + next: n == null ? null : [n.x, n.y], + visibility: getComputedStyle(el).visibility, + }; + }); + const detail = JSON.stringify({before, lifted}); + expect(lifted.isOpen, detail).toBe(true); + expect(lifted.visibility, detail).toBe('visible'); + for (let i = 0; i < 4; i++) { + expect(lifted.spot?.[i] ?? NaN, detail).toBeCloseTo(before.box[i], 0); + } + // Nothing after the card moved. + expect(lifted.next, detail).toEqual(before.next); + // Once the lift has grown (it takes no touch before, AST-094 FR35), a tap + // on the tint, far from the card and the menu. + await untilGrown(page); + await driver.play(tap({x: 24, y: 24})); + await expect(menu).toBeHidden(); + await expect + .poll(async () => trigger.evaluate(el => el.matches(':popover-open')), { + timeout: 3_000, + }) + .toBe(false); + await expect + .poll( + async () => + page.evaluate(() => + document.querySelector('[data-astryx-context-menu-lift]'), + ), + {timeout: 3_000}, + ) + .toBeNull(); + // The card is back as it was: no popover, and nothing of the lift inline. + const after = await trigger.evaluate(el => ({ + popover: el.getAttribute('popover'), + style: (el as HTMLElement).style.cssText, + placeholder: + document.querySelector('[data-astryx-context-menu-lift-placeholder]') != + null, + })); + expect(after.popover).toBeNull(); + expect(after.style).toBe(before.style); + expect(after.placeholder).toBe(false); +}); + +test('the card grows on from the press, and the backdrop fades in, on springs.contextMenuLift, frame by frame', async ({ + page, +}, testInfo) => { + const {driver, center} = await openLiftStory(page); + const playing = driver.play(tap(center, {holdMs: menuDisplayMs + 600})); + const lifted = page.locator('[data-testid="touch-context-menu"][popover]'); + await expect(lifted).toBeAttached({timeout: 5_000}); + await holdAnimations(page); + const samples: {t: number; scale: number; backdrop: number}[] = []; + for (const t of timeline(450)) { + await seekAnimations(page, t); + samples.push({ + t, + ...(await page.evaluate(() => { + const card = document.querySelector( + '[data-testid="touch-context-menu"]', + ); + const layer = document.querySelector('[data-astryx-context-menu-lift]'); + const scale = card == null ? 'none' : getComputedStyle(card).scale; + return { + scale: scale === 'none' ? 1 : Number.parseFloat(scale), + backdrop: + layer == null + ? NaN + : Number(getComputedStyle(layer, '::backdrop').opacity), + }; + })), + }); + } + await releaseAnimations(page); + await playing; + const from = samples[0].scale; + const scale = solveSpring(toSpring(springs.contextMenuLift), { + from, + to: 1.13, + }); + const fade = solveSpring(toSpring(springs.contextMenuLift), {from: 0, to: 1}); + const errors = samples.map(s => ({ + t: s.t, + widthErrorPx: Math.abs(s.scale - scale.value(s.t / 1000)) * 200, + // Opacity stops at 1: the spring's 1.5% overshoot (UIKit's alpha peaks + // at 1.0145 too) presents as 1. + backdropError: Math.abs(s.backdrop - Math.min(1, fade.value(s.t / 1000))), + })); + await writeEvidence( + 'curves/contextmenu-lift', + { + [`${testInfo.project.name}.json`]: JSON.stringify( + {from, samples, errors}, + null, + 1, + ), + }, + testInfo, + ); + // The press left the card at about 1.06 (UIKit: 1.0618 at 733 ms). + expect(from).toBeGreaterThan(1.04); + expect(from).toBeLessThan(1.07); + expect(Math.max(...errors.map(e => e.widthErrorPx))).toBeLessThan(0.3); + expect(Math.max(...errors.map(e => e.backdropError))).toBeLessThan(0.01); + // It overshoots, as UIKit's does (226.39 pt at +279 ms). + expect(Math.max(...samples.map(s => s.scale))).toBeGreaterThan(1.13); +}); + +test('the menu grows out of the finger on springs.contextMenuMorph, translate, scale and opacity only: at the press at first, its frame at rest, nothing left inline (AST-094 FR5)', async ({ + page, +}) => { + const {driver, center} = await openLiftStory(page); + const playing = driver.play(tap(center, {holdMs: menuDisplayMs + 600})); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + // The menu's own animations, timed from the menu's start. + await holdAnimations(page, page.locator('[popover]:has([role="menu"])')); + const read = async (t: number) => { + await seekAnimations(page, t); + return page.evaluate(() => { + const popover = document + .querySelector('[role="menu"]') + ?.closest('[popover]'); + if (popover == null) { + return null; + } + const style = getComputedStyle(popover); + const r = popover.getBoundingClientRect(); + const moving = new Set(); + for (const animation of popover.getAnimations()) { + const frames = + (animation.effect as KeyframeEffect | null)?.getKeyframes() ?? []; + const properties = new Set(frames.flatMap(frame => Object.keys(frame))); + for (const property of properties) { + if ( + !['offset', 'computedOffset', 'easing', 'composite'].includes( + property, + ) && + new Set(frames.map(frame => String(frame[property]))).size > 1 + ) { + moving.add(property); + } + } + } + return { + x: r.x + r.width / 2, + y: r.y + r.height / 2, + width: r.width, + opacity: Number(style.opacity), + filter: style.filter, + moving: [...moving].sort(), + }; + }); + }; + const start = await read(0); + const middle = await read(100); + const end = await read(600); + await releaseAnimations(page); + await playing; + const curve = solveSpring(toSpring(springs.contextMenuMorph), { + from: 0, + to: 1, + }); + const detail = JSON.stringify({center, start, middle, end}); + // At the press, UIKit's blob wide, clear. + expect(start?.width ?? NaN, detail).toBeCloseTo( + contextMenu.lift.morphStartWidth, + 0, + ); + expect(start?.x ?? NaN, detail).toBeCloseTo(center.x, 0); + expect(start?.y ?? NaN, detail).toBeCloseTo(center.y, 0); + expect(start?.opacity, detail).toBeCloseTo(0, 2); + // Its opacity follows the growth's own progress. + expect(middle?.opacity, detail).toBeCloseTo(curve.value(0.1), 1); + expect(end?.opacity, detail).toBeCloseTo(1, 2); + expect(end?.width ?? 0, detail).toBeGreaterThan(150); + for (const sample of [start, middle, end]) { + expect(sample?.filter, detail).toBe('none'); + } + expect(start?.moving, detail).toEqual(['opacity', 'scale', 'translate']); + // Once grown, nothing of the growth is left inline on the menu. + await page.waitForTimeout(300); + expect( + await page.evaluate(() => { + const popover = document + .querySelector('[role="menu"]') + ?.closest('[popover]'); + return [popover?.style.translate, popover?.style.scale]; + }), + ).toEqual(['', '']); +}); + +test('closing returns the card to its place and clears the backdrop', async ({ + page, +}) => { + const story = await openLiftStory(page); + const lifted = await liftAndSettle(page, story); + await page.keyboard.press('Escape'); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeHidden(); + await page.waitForTimeout(900); + const after = await readLift(page); + const detail = JSON.stringify({lifted, after}); + expect(after.preview, detail).toBeNull(); + expect(after.layerOpen, detail).toBe(false); + expect(after.cardOpacity, detail).toBe('1'); + // Back on its spot. + expect(after.card?.x ?? NaN, detail).toBeCloseTo(lifted.card?.x ?? NaN, 0); + expect(after.card?.y ?? NaN, detail).toBeCloseTo(lifted.card?.y ?? NaN, 0); + expect(after.card?.width ?? NaN, detail).toBeCloseTo( + lifted.card?.width ?? NaN, + 0, + ); +}); + +test('a tap on the tint while the lift is still growing is swallowed: the menu finishes and stays open, and nothing beneath gets it (AST-094 FR35)', async ({ + page, +}) => { + const story = await openLiftStory(page); + await page.evaluate(() => { + const w = window as unknown as {__clicked?: string[]}; + w.__clicked = []; + document.addEventListener( + 'click', + event => { + const target = event.target as Element; + w.__clicked?.push( + target.closest('[data-astryx-context-menu-lift]') + ? 'layer' + : target.tagName.toLowerCase(), + ); + }, + {capture: true}, + ); + }); + await story.driver.play(tap(story.center, {holdMs: menuDisplayMs + 100})); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + // At once, while the card grows and the menu fades in: a tap on the tint. + await story.driver.play(tap({x: 12, y: 12})); + await page.waitForTimeout(1200); + const lift = await readLift(page); + const clicked = await page.evaluate( + () => (window as unknown as {__clicked?: string[]}).__clicked ?? [], + ); + const detail = JSON.stringify({lift, clicked}); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + expect(lift.layerOpen, detail).toBe(true); + expect(lift.backdropOpacity, detail).toBeCloseTo(1, 2); + expect( + clicked.filter(target => target !== 'layer'), + detail, + ).toEqual([]); + // Settled, a tap on the tint closes it as before. + await story.driver.play(tap({x: 12, y: 12})); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeHidden(); +}); + +test('a tap on the backdrop closes the menu and reaches nothing under it', async ({ + page, +}) => { + const story = await openLiftStory(page); + const lift = await liftAndSettle(page, story); + if (lift.card == null) { + throw new Error(JSON.stringify(lift)); + } + await page.evaluate(() => { + const w = window as unknown as {__clicked?: string[]}; + w.__clicked = []; + document.addEventListener( + 'click', + event => { + const target = event.target as Element; + w.__clicked?.push( + target.closest('[data-astryx-context-menu-lift]') + ? 'backdrop' + : target.closest('[data-testid="touch-context-menu"]') + ? 'card' + : target.tagName.toLowerCase(), + ); + }, + {capture: true}, + ); + }); + // On the card's own spot, under the backdrop, clear of the lifted card. + await story.driver.play( + tap({x: lift.card.x + 10, y: lift.card.y + lift.card.height + 8}), + ); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeHidden(); + const clicked = await page.evaluate( + () => (window as unknown as {__clicked?: string[]}).__clicked ?? [], + ); + expect(clicked.filter(target => target !== 'backdrop')).toEqual([]); +}); + +test('the lift keeps moving while the main thread is blocked: the card grows on every frame', async ({ + page, + browserName, +}, testInfo) => { + // Measured: Linux WebKit's compositor stalls with its main thread (no + // frames for most of a block, the card frozen). iOS Safari keeps the card + // growing through the same 1 s block: the simulator recording. + test.skip( + browserName !== 'chromium', + 'Linux WebKit’s compositor stalls with its main thread; iOS Safari is the simulator recording', + ); + const {driver, center} = await openLiftStory(page); + await markSelector(page, '[data-testid="touch-context-menu"]'); + // Blocked from 20 ms after the menu's moment for 1 s (spec:AST-094's FR5 + // row), past the card's whole growth; the finger lifts after the block. + const busyWindow = await blockAfterTouch(page, menuDisplayMs + 20, 1000); + const cast = await startScreencast(page); + await driver.play(tap(center, {holdMs: menuDisplayMs + 1300})); + const castFrames = await cast.stop(); + const block = await busyWindow(); + const tracked = await trackMarker(page, castFrames); + // The screencast reports whole pixels: the spring's tail plateaus within + // 2 px of the end (the harness's usual settle). + const proof = busyWindowProof(tracked, block, { + axis: 'width', + settledWithin: 2, + }); + await writeEvidence( + 'busy-main-thread/contextmenu-lift', + { + [`${testInfo.project.name}.json`]: JSON.stringify( + { + block, + proof, + frames: tracked.map(f => ({ + t: +(f.t - block.from).toFixed(1), + w: f.box?.width ?? null, + })), + }, + null, + 1, + ), + }, + testInfo, + ); + const film = await castStoryboard(page, castFrames, { + name: 'contextmenu-lift-busy-main-thread', + zero: block.from, + from: block.from - 40, + to: block.to + 40, + columns: 8, + cellScale: 0.35, + data: t => ({w: tracked.find(f => f.t === t)?.box?.width ?? -1}), + }); + await film.write(testInfo, {block}); + // Frames kept coming and the card kept growing through the block: the + // harness judges each delivered frame (no stall, no frozen card). + expect(proof.verdict.failures, proof.summary).toEqual([]); + expect(proof.frames).toBeGreaterThanOrEqual(proof.expectedFrames - 2); + expect(proof.monotonic).toBe(true); + expect((proof.last ?? 0) - (proof.first ?? 0)).toBeGreaterThanOrEqual(4); +}); + +/** The lifted card's box, its opacity and the menu's filter, as the frame shows them. */ +type ReducedMotionSample = Awaited>; + +async function reducedMotionSample(page: Page) { + return page.evaluate(() => { + const preview = document.querySelector( + '[data-testid="touch-context-menu"]', + ); + const menu = document + .querySelector('[role="menu"]') + ?.closest('[popover]'); + const box = preview?.getBoundingClientRect(); + return { + x: box?.x ?? Number.NaN, + y: box?.y ?? Number.NaN, + width: box?.width ?? Number.NaN, + opacity: preview ? Number(getComputedStyle(preview).opacity) : Number.NaN, + filter: menu ? getComputedStyle(menu).filter : 'none', + menuOpacity: menu ? Number(getComputedStyle(menu).opacity) : Number.NaN, + }; + }); +} + +test('under Reduce Motion the lift cross-fades in place, as UIKit’s reduced path does: the card holds its lifted box in every frame, only its opacity moves, and the menu fades in without blur', async ({ + page, +}) => { + await page.emulateMedia({reducedMotion: 'reduce'}); + const {driver, center} = await openLiftStory(page); + const playing = driver.play(tap(center, {holdMs: menuDisplayMs + 600})); + const lifted = page.locator('[data-testid="touch-context-menu"][popover]'); + await expect(lifted).toBeAttached({timeout: 5_000}); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + // The lift's and the menu's animations, timed from the lift's start (the + // menu starts in the commit after it). + await holdAnimations(page); + const times = timeline(600); + const samples: ReducedMotionSample[] = []; + for (const t of times) { + await seekAnimations(page, t); + samples.push(await reducedMotionSample(page)); + } + await releaseAnimations(page); + await playing; + const detail = JSON.stringify(samples); + const last = samples.at(-1); + expect(last, detail).toBeDefined(); + if (last == null) { + return; + } + for (const sample of samples) { + // No travel, no growth. + expect(Math.abs(sample.x - last.x), detail).toBeLessThan(0.5); + expect(Math.abs(sample.y - last.y), detail).toBeLessThan(0.5); + expect(Math.abs(sample.width - last.width), detail).toBeLessThan(0.5); + // No blur. + expect(sample.filter, detail).toMatch(/^(none|blur\(0(px)?\))$/); + } + // The lifted box is the card's lifted size, not its size on the page. + expect(last.width, detail).toBeGreaterThan( + ( + await page + .locator('[data-astryx-context-menu-lift-placeholder]') + .boundingBox() + )?.width ?? 0, + ); + // Only the opacity moves: from (near) clear to opaque, never back, and + // done within 200 ms (`design:reduced-motion` DR2); the menu's fade too, + // within 200 ms of its own start. + expect(samples[0].opacity, detail).toBeLessThan(0.5); + expect(last.opacity, detail).toBeCloseTo(1, 2); + const menuStart = + times[samples.findIndex(sample => sample.menuOpacity > 0.001)] ?? 0; + for (const [i, t] of times.entries()) { + if (t >= 200 + 1000 / 60) { + expect(samples[i].opacity, detail).toBeGreaterThan(0.999); + } + if (t >= menuStart + 200 + 1000 / 60) { + expect(samples[i].menuOpacity, detail).toBeGreaterThan(0.999); + } + } + expect( + samples.every( + (s, i) => i === 0 || s.opacity >= samples[i - 1].opacity - 1e-4, + ), + detail, + ).toBe(true); +}); + +test('under Reduce Motion closing fades the card out where it is, with no return travel', async ({ + page, +}) => { + await page.emulateMedia({reducedMotion: 'reduce'}); + const {driver, center} = await openLiftStory(page); + await driver.play(tap(center, {holdMs: menuDisplayMs + 600})); + const menu = page.getByRole('menu', {name: 'Photo actions'}); + await expect(menu).toBeVisible(); + await page.waitForTimeout(700); + const open = await reducedMotionSample(page); + // A tap on the dim, far from the card and the menu. + await driver.play(tap({x: 24, y: 24})); + await holdAnimations(page); + const times = timeline(300); + const samples: ReducedMotionSample[] = []; + for (const t of times) { + await seekAnimations(page, t); + samples.push(await reducedMotionSample(page)); + } + await releaseAnimations(page); + const detail = JSON.stringify({open, samples}); + // Gone within 200 ms (`design:reduced-motion` DR2). + for (const [i, t] of times.entries()) { + if (t >= 200 + 1000 / 60 && Number.isFinite(samples[i].opacity)) { + expect(samples[i].opacity, detail).toBeLessThan(0.001); + } + } + for (const sample of samples) { + expect(Math.abs(sample.x - open.x), detail).toBeLessThan(0.5); + expect(Math.abs(sample.y - open.y), detail).toBeLessThan(0.5); + expect(Math.abs(sample.width - open.width), detail).toBeLessThan(0.5); + } + expect(samples.at(-1)?.opacity ?? 1, detail).toBeLessThan(0.5); + expect( + samples.every( + (s, i) => i === 0 || s.opacity <= samples[i - 1].opacity + 1e-4, + ), + detail, + ).toBe(true); +}); + +test( + 'storyboard: the lift, from the menu’s moment', + {tag: '@storyboard'}, + async ({page}, testInfo) => { + const {driver, center} = await openLiftStory(page); + const playing = driver.play(tap(center, {holdMs: menuDisplayMs + 900})); + await expect( + page.locator('[data-testid="touch-context-menu"][popover]'), + ).toBeAttached({timeout: 5_000}); + const viewport = page.viewportSize(); + const {storyboard} = await animationStoryboard(page, { + name: 'contextmenu-lift', + clip: { + x: 0, + y: Math.max(0, center.y - 120), + width: viewport?.width ?? 402, + height: 360, + }, + times: timeline(450, 1000 / 30), + measure: async () => + page.evaluate(() => { + const card = document.querySelector( + '[data-testid="touch-context-menu"]', + ); + const layer = document.querySelector( + '[data-astryx-context-menu-lift]', + ); + return { + width: card?.getBoundingClientRect().width.toFixed(1) ?? '-', + backdrop: + layer == null + ? '-' + : Number(getComputedStyle(layer, '::backdrop').opacity).toFixed( + 3, + ), + }; + }), + columns: 5, + cellScale: 0.5, + }); + await releaseAnimations(page); + await playing; + await storyboard.write(testInfo); + const widths = storyboard.frames.map(f => Number(f.data.width)); + expect(widths.at(-1)).toBeCloseTo(226, 0); + }, +); diff --git a/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.a11y.chromium.spec.ts b/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.a11y.chromium.spec.ts index 29102ddf41af7..56af38af8a87b 100644 --- a/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.a11y.chromium.spec.ts +++ b/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.a11y.chromium.spec.ts @@ -49,20 +49,54 @@ async function triggerCenter(page: Page): Promise<{x: number; y: number}> { } /** - * Resolves once the browser has delivered the `click` that follows a touch - * (the last of its compatibility mouse events) and two frames have run, so - * anything that event closes has closed. + * Resolves two frames after the lift's `touchend`, or after the `click` that + * follows it when the browser sends one (the last of a touch's compatibility + * mouse events), so anything those events close has closed. Under the touch + * model a hold that opens the menu gets no compatibility events, so the lift's + * `touchend` is the last event of the press. */ async function armCompatClick(page: Page): Promise { await page.evaluate(() => { const w = window as unknown as {__compatClick?: Promise}; w.__compatClick = new Promise(resolve => { - document.addEventListener( - 'click', - () => - requestAnimationFrame(() => requestAnimationFrame(() => resolve())), - {once: true, capture: true}, - ); + // Each event restarts the two frames, so a click that follows the + // lift's touchend is waited for too. + let last = 0; + const settle = () => { + const mine = ++last; + requestAnimationFrame(() => + requestAnimationFrame(() => { + if (mine === last) { + resolve(); + } + }), + ); + }; + document.addEventListener('touchend', settle, { + once: true, + capture: true, + }); + document.addEventListener('click', settle, {once: true, capture: true}); + }); + }); +} + +/** + * Resolves two frames after every running animation on the page has + * finished: the presentation's, which ends the layer's input hold. + */ +async function presentationSettled(page: Page): Promise { + await page.evaluate(async () => { + await Promise.all( + document.getAnimations().map(async animation => + animation.finished.then( + () => undefined, + () => undefined, + ), + ), + ); + await new Promise(resolve => { + requestAnimationFrame(() => requestAnimationFrame(() => resolve())); }); }); } @@ -94,7 +128,10 @@ test('the long press that opens the menu leaves it open as the finger lifts', as await compatClickDelivered(page); await expect(menu).toBeVisible(); - // A tap outside still closes it. + // A tap outside still closes it, once the presentation has run: until it + // completes, a tap reaches nothing (the layer's input hold, as UIKit drops + // a touch while a presentation runs). + await presentationSettled(page); await cdp.send('Input.dispatchTouchEvent', { type: 'touchStart', touchPoints: [{x: 5, y: 5, id: 2}], diff --git a/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.interaction.browser.spec.ts b/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.interaction.browser.spec.ts new file mode 100644 index 0000000000000..6c23fb38d1568 --- /dev/null +++ b/packages/core/src/ContextMenu/__tests__/ContextMenuLongPress.interaction.browser.spec.ts @@ -0,0 +1,800 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file ContextMenuLongPress.interaction.browser.spec.ts + * @input The built "Touch / long press" ContextMenu story, real touch input + * (Chromium CDP / WebKit's input command), the interaction harness, the + * context-menu constants and UIKit's measured press frames + * @output Interaction and animation proofs for opening a context menu with a + * held finger on UIKit's timeline, one touch model on every touch device + * (Chromium's finger stands for Android's engine, WebKit's for iOS): the + * menu opens at 735 ms for a finger + * still down, or at a lift after the commit, and not before; under Reduce + * Motion the card does not grow and the timeline holds; the lift sends no + * click; an early lift opens nothing and the card springs back; a second + * finger before the commit opens nothing and the card springs back; a + * drift within the tolerance still opens, past it on one axis (or a drag + * after the commit) opens nothing; the press feedback's presented scale + * follows UIKit's measured frames; it keeps advancing every frame through a + * blocked main thread; it does no Layout or Paint; the page does not scroll + * under the held finger; a storyboard of the press + * @position Interaction specs of packages/core/src/ContextMenu (the + * BottomSheetDetent spec is the exemplar); where the menu then shows is + * ContextMenuLift.interaction.browser.spec.ts's + */ + +import {expect, test, type Locator, type Page} from '@playwright/test'; +import { + blockMainThread, + busyWindowProof, + castStoryboard, + countBetween, + drag, + freezeTimers, + holdAnimations, + installInputLog, + installLongTaskLog, + installTimerGate, + markElement, + merge, + openStory, + readInputLog, + releaseAnimations, + startScreencast, + tap, + thawTimers, + timeline, + touchDriver, + trackMarker, + traceRendering, + waitForAnimation, + withFinger, + withStorybook, + writeEvidence, + animationStoryboard, + type TouchDriver, +} from '@astryxdesign/interaction-harness'; +import {contextMenu} from '../../motion/uikit/contextMenu'; +import { + PRESS_FEEDBACK_START_MS, + pressFeedbackCurve, +} from '../contextMenuPressCurve'; +import {UIKIT_LONG_PRESS, UIKIT_PRESS_VIEW_WIDTH} from './uikitPressFrames'; + +const storybook = withStorybook(test); +const {commitMs, menuDisplayMs} = contextMenu.press; + +interface TouchStory { + driver: TouchDriver; + trigger: Locator; + center: {x: number; y: number}; +} + +/** The touch story under the engine's own finger (a coarse pointer). */ +async function openTouchStory( + page: Page, + story = 'core-contextmenu--touch-long-press', +): Promise { + await installInputLog(page); + await installLongTaskLog(page); + // Note when the menu is asked to show, on the page clock (no per-frame + // work: the rendering proofs share this page). + await page.addInitScript(() => { + const w = window as unknown as { + __menuShownAt?: number | null; + __liftHandledAt?: number | null; + __liftShownAt?: number | null; + __framesBeforeMenu?: number | null; + }; + w.__menuShownAt = null; + w.__liftHandledAt = null; + w.__liftShownAt = null; + w.__framesBeforeMenu = null; + // When the page gets to the lift (the event's own timestamp is the + // hardware's, or the script's): the menu shows in the same task. + addEventListener( + 'touchend', + () => { + w.__liftHandledAt = performance.now(); + }, + {capture: true, passive: true}, + ); + const show = HTMLElement.prototype.showPopover; + HTMLElement.prototype.showPopover = function ( + this: HTMLElement, + ...args: Parameters + ) { + // A finger's lift, where one opens the menu: the frames painted from + // it until the menu opens. + if ( + w.__liftShownAt == null && + this.hasAttribute('data-astryx-context-menu-lift') + ) { + w.__liftShownAt = performance.now(); + let frames = 0; + const count = () => { + if (w.__menuShownAt == null) { + frames += 1; + requestAnimationFrame(count); + } else { + w.__framesBeforeMenu ??= frames; + } + }; + requestAnimationFrame(count); + } + // Only a popover that holds the menu (Storybook shows its own). + if ( + this.matches('[role="menu"]') || + this.querySelector('[role="menu"]') != null + ) { + w.__menuShownAt ??= performance.now(); + } + return show.apply(this, args); + }; + }); + await openStory(page, storybook(), story); + const driver = await touchDriver(page); + const trigger = page.getByTestId('touch-context-menu'); + const box = await trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + return { + driver, + trigger, + center: {x: box.x + box.width / 2, y: box.y + box.height / 2}, + }; +} + +const menuShownAt = async (page: Page) => + page.evaluate( + () => + (window as unknown as {__menuShownAt?: number | null}).__menuShownAt ?? + null, + ); + +/** When a finger's lift started, and the frames painted before the menu. */ +const liftShownAt = async (page: Page) => + page.evaluate(() => { + const w = window as unknown as { + __liftShownAt?: number | null; + __framesBeforeMenu?: number | null; + }; + return {at: w.__liftShownAt ?? null, frames: w.__framesBeforeMenu ?? null}; + }); + +/** The page-clock timestamp of the last `type` event. */ +async function lastInput( + page: Page, + type: 'touchstart' | 'touchend', +): Promise { + const log = await readInputLog(page); + const record = log.filter(entry => entry.type === type).at(-1); + if (record == null) { + throw new Error(`no ${type} reached the page`); + } + return record.timeStamp; +} + +const touchStartedAt = async (page: Page) => lastInput(page, 'touchstart'); + +const presentedScale = async (trigger: Locator) => + trigger.evaluate(element => { + const scale = getComputedStyle(element).scale; + return scale === 'none' ? 1 : Number.parseFloat(scale); + }); + +/** A finger held still for `ms`, then lifted. */ +const hold = (at: {x: number; y: number}, ms: number) => tap(at, {holdMs: ms}); + +test('a finger still down gets the menu at 735 ms (iOS 26.2), not before', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + await driver.play(hold(center, menuDisplayMs + 200)); + const start = await touchStartedAt(page); + const shown = await menuShownAt(page); + const lift = await liftShownAt(page); + expect(shown, 'the menu never showed').not.toBeNull(); + expect(lift.at, 'the trigger never lifted').not.toBeNull(); + // The presentation starts at the timeline's moment: the timer runs at its + // deadline, give or take a task, and the trigger lifts in it. + const after = (lift.at as number) - start; + expect(after).toBeGreaterThanOrEqual(menuDisplayMs); + expect(after).toBeLessThan(menuDisplayMs + 20); + // The menu opens with it, in the commit that follows: by the lift's second + // frame at the latest (it starts to grow from nothing, so either reads as + // one moment). + expect(lift.frames ?? 0).toBeLessThanOrEqual(1); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); +}); + +test('a finger that lands as soon as the card shows gets UIKit’s timeline, however slowly scripts arrive: no menu at 500 ms, the menu at 735 ms', async ({ + page, +}) => { + // A slow phone network: a touch layer fetched on its own would arrive 8 s + // after the page. + await page.route(/ContextMenuTouchLayer[^/]*\.js/u, async route => { + await new Promise(resolve => setTimeout(resolve, 8000)); + await route.continue(); + }); + const {driver, center} = await openTouchStory(page); + await driver.play(hold(center, menuDisplayMs + 200)); + const start = await touchStartedAt(page); + const shown = await menuShownAt(page); + expect(shown, 'the menu never showed').not.toBeNull(); + expect((shown as number) - start).toBeGreaterThanOrEqual(menuDisplayMs); +}); + +test('with presentation="popover" a finger keeps today’s long press: the menu at 500 ms, with no growth and no iOS timeline', async ({ + page, +}) => { + const {driver, trigger, center} = await openTouchStory( + page, + 'core-contextmenu--touch-long-press-popover', + ); + // A coarse primary pointer: only the caller's choice keeps today's menu. + expect( + await page.evaluate(() => matchMedia('(pointer: coarse)').matches), + ).toBe(true); + expect(await trigger.getAttribute('data-astryx-pressable')).toBeNull(); + const playing = driver.play(hold(center, 1000)); + await page.waitForTimeout(350); + const scaleBeforeOpen = await presentedScale(trigger); + // Still held: the menu is up. (On the lift, emulated Chromium sends the + // compatibility mouse events a long press sends no click for on a device, + // and they dismiss today's menu as they do on main; the touch model's + // recognizer claims the finger instead.) + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible({ + timeout: 550, + }); + await playing; + const start = await touchStartedAt(page); + const shown = await menuShownAt(page); + expect(shown, 'the menu never showed').not.toBeNull(); + const after = (shown as number) - start; + // Today's 500 ms, give or take a task and the render that shows the menu; + // far from the touch model's 735 ms. + expect(after).toBeGreaterThanOrEqual(500); + expect(after).toBeLessThan(550); + expect(scaleBeforeOpen).toBe(1); +}); + +test('a finger where the primary pointer is fine keeps today’s long press: the menu at 500 ms, with no growth', async ({ + browser, + page, +}) => { + // A touch screen whose primary pointer is fine (a laptop with a trackpad). + const context = await browser.newContext({ + viewport: page.viewportSize() ?? {width: 402, height: 874}, + hasTouch: true, + isMobile: false, + }); + const fine = await context.newPage(); + try { + await installInputLog(fine); + await openStory(fine, storybook(), 'core-contextmenu--touch-long-press'); + const coarse = await fine.evaluate( + () => matchMedia('(pointer: coarse)').matches, + ); + test.skip( + coarse, + 'this engine reports a coarse primary pointer whenever touch is emulated; the fine-primary finger is covered in jsdom', + ); + const trigger = fine.getByTestId('touch-context-menu'); + const box = await trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + expect(await trigger.getAttribute('data-astryx-pressable')).toBeNull(); + const driver = await touchDriver(fine); + const center = {x: box.x + box.width / 2, y: box.y + box.height / 2}; + const playing = driver.play(hold(center, 1000)); + await fine.waitForTimeout(350); + const scaleBeforeOpen = await presentedScale(trigger); + await expect(fine.getByRole('menu', {name: 'Photo actions'})).toBeVisible({ + timeout: 550, + }); + await playing; + expect(scaleBeforeOpen).toBe(1); + } finally { + await context.close(); + } +}); + +test('under Reduce Motion the card does not grow, and the menu still opens at 735 ms', async ({ + page, +}) => { + await page.emulateMedia({reducedMotion: 'reduce'}); + const {driver, trigger, center} = await openTouchStory(page); + const playing = driver.play(hold(center, menuDisplayMs + 200)); + // Sampled while the finger is down: before the commit, after it, and just + // before the menu. + const scales: number[] = []; + for (const wait of [300, 250, 100]) { + await page.waitForTimeout(wait); + scales.push(await presentedScale(trigger)); + } + await playing; + const start = await touchStartedAt(page); + const shown = await menuShownAt(page); + expect(scales).toEqual([1, 1, 1]); + // No growth played on it (the lift's own entrance only fades, under Reduce + // Motion). + expect( + await trigger.evaluate( + el => + el + .getAnimations() + .filter( + animation => + new Set( + (animation.effect as KeyframeEffect | null) + ?.getKeyframes() + .map(frame => String(frame.scale)), + ).size > 1, + ).length, + ), + ).toBe(0); + expect(shown, 'the menu never showed').not.toBeNull(); + const after = (shown as number) - start; + // The touch's stamp and the page clock are each coarsened (0.1 ms). + expect(after).toBeGreaterThanOrEqual(menuDisplayMs - 0.5); + expect(after).toBeLessThan(menuDisplayMs + 20); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); +}); + +test('the trigger’s selection and callout lock: its own style keeps it off at rest, and the gesture host refuses a selection for a held press', async ({ + page, +}) => { + const {driver, trigger, center} = await openTouchStory(page); + // WebKit spells it `-webkit-user-select` only. + const readLock = async () => + trigger.evaluate(el => { + const host = el as HTMLElement; + const computed = getComputedStyle(host); + return { + inline: + host.style.getPropertyValue('user-select') || + host.style.getPropertyValue('-webkit-user-select'), + computed: + computed.getPropertyValue('user-select') || + computed.getPropertyValue('-webkit-user-select'), + }; + }); + // Whether a selection starting under the finger is refused now: where iOS + // WebKit does not run, the host's lock cancels `selectstart` and writes no + // style (spec:AST-094 FR41). + const refusesSelection = async () => + page.evaluate(({x, y}) => { + const event = new Event('selectstart', {bubbles: true, cancelable: true}); + (document.elementFromPoint(x, y) ?? document.body).dispatchEvent(event); + return event.defaultPrevented; + }, center); + // At rest the trigger's own style keeps selection off (spec:AST-017 + // FR59), and the host refuses nothing. + const rest = {...(await readLock()), refused: await refusesSelection()}; + expect(rest).toEqual({inline: '', computed: 'none', refused: false}); + // A held finger, short of the commit: the host refuses a selection under + // it, with no style written. + const playing = driver.play(hold(center, commitMs - 100)); + await page.waitForTimeout(commitMs / 2); + const held = {...(await readLock()), refused: await refusesSelection()}; + await playing; + await page.waitForTimeout(100); + // Lifted with no menu: back to the rest values. + const lifted = {...(await readLock()), refused: await refusesSelection()}; + expect(held).toEqual({inline: '', computed: 'none', refused: true}); + expect(lifted).toEqual(rest); + expect( + await page.getByRole('menu', {name: 'Photo actions'}).isVisible(), + ).toBe(false); +}); + +test('a touch open lights no row: the menu itself takes the focus, and every row keeps its resting background', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + const playing = driver.play(hold(center, menuDisplayMs + 300)); + const menu = page.getByRole('menu', {name: 'Photo actions'}); + await expect(menu).toBeVisible(); + // The focus moves on the frame after the menu shows. + await page.evaluate( + async () => + new Promise(resolve => { + requestAnimationFrame(() => { + requestAnimationFrame(() => { + resolve(); + }); + }); + }), + ); + await expect(menu).toBeFocused(); + // An internal focus target: no browser ring on it. + expect(await menu.evaluate(el => getComputedStyle(el).outlineStyle)).toBe( + 'none', + ); + const backgrounds = await menu + .getByRole('menuitem') + .evaluateAll(rows => + rows.map(row => getComputedStyle(row).backgroundColor), + ); + expect(new Set(backgrounds).size).toBe(1); + await playing; +}); + +test('a finger lifted after the commit gets the menu at the lift, and sends no click', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + await driver.play(hold(center, 530)); + await page.waitForTimeout(300); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + const {lift, shown} = await page.evaluate(() => { + const w = window as unknown as { + __menuShownAt?: number | null; + __liftHandledAt?: number | null; + }; + return {lift: w.__liftHandledAt ?? null, shown: w.__menuShownAt ?? null}; + }); + expect(lift).not.toBeNull(); + expect(shown).not.toBeNull(); + // In the lift's own task: after it, and before the next frame could paint. + expect((shown as number) - (lift as number)).toBeGreaterThanOrEqual(0); + expect((shown as number) - (lift as number)).toBeLessThan(1000 / 60); + const log = await readInputLog(page); + const end = log.filter(record => record.type === 'touchend').at(-1); + expect(end?.defaultPrevented, JSON.stringify(log)).toBe(true); + expect(log.filter(record => record.type === 'click')).toEqual([]); +}); + +test('the lift after the menu shows keeps it open and sends no click', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + await driver.play(hold(center, menuDisplayMs + 150)); + await page.waitForTimeout(300); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + const log = await readInputLog(page); + expect(log.filter(record => record.type === 'click')).toEqual([]); +}); + +test('a drag past the tolerance after the commit opens nothing', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + await driver.play( + drag( + center, + {x: center.x, y: center.y + 30}, + {durationMs: 50, holdMs: 500, restMs: 400}, + ), + ); + await page.waitForTimeout(200); + expect(await menuShownAt(page)).toBeNull(); +}); + +test('an early lift opens nothing, and the card springs back to its size', async ({ + page, +}) => { + const {driver, trigger, center} = await openTouchStory(page); + await driver.play(hold(center, 280)); + await page.waitForTimeout(commitMs + 200); + expect(await menuShownAt(page)).toBeNull(); + // The return spring settles within ~1.2 s; then the scale is gone. + await expect + .poll(async () => trigger.evaluate(el => (el as HTMLElement).style.scale), { + timeout: 3000, + }) + .toBe(''); + expect(await presentedScale(trigger)).toBe(1); +}); + +test('a second finger landing before the commit cancels: no menu, and the card springs back', async ({ + page, + browserName, +}) => { + test.skip( + browserName !== 'chromium', + 'two-finger input is Chromium-only in Playwright (WebKit drops the first finger)', + ); + const {driver, trigger, center} = await openTouchStory(page); + // The first finger holds; a second lands 200 ms in, before the commit. + await driver.play( + merge( + hold(center, menuDisplayMs + 300), + withFinger(hold({x: center.x + 50, y: center.y}, 400), 1, 200), + ), + ); + await page.waitForTimeout(300); + expect(await menuShownAt(page)).toBeNull(); + expect(await presentedScale(trigger)).toBeCloseTo(1, 3); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeHidden(); +}); + +test('a drift within the tolerance still opens the menu; past it on one axis cancels', async ({ + page, +}) => { + const {driver, center} = await openTouchStory(page); + // (8, 8): 11.3 px away, within the tolerance on each axis (a long press's + // 10 px distance would cancel it). A drift past the browser's own pan slop + // goes to the page instead, as a scroll view's pan takes it on iOS. Lifted + // after the commit: the menu opens at the lift. + await driver.play( + drag( + center, + {x: center.x + 8, y: center.y + 8}, + {durationMs: 60, holdMs: 100, restMs: commitMs}, + ), + ); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + + const other = await openTouchStory(page); + await other.driver.play( + drag( + other.center, + {x: other.center.x, y: other.center.y + 12}, + {durationMs: 60, holdMs: 100, restMs: commitMs}, + ), + ); + await page.waitForTimeout(200); + expect(await menuShownAt(page)).toBeNull(); +}); + +test('the press feedback follows UIKit’s measured press, frame by frame', async ({ + page, +}, testInfo) => { + const {driver, trigger, center} = await openTouchStory(page); + const playing = driver.play(hold(center, commitMs + 300)); + await waitForAnimation(trigger, {minProgressMs: 16}); + await holdAnimations(page, trigger); + // Seek the held curve (its time 0 is PRESS_FEEDBACK_START_MS after the + // touch) to every 60 Hz frame and read the presented scale. + const samples = await trigger.evaluate((element, times) => { + const held = + (window as unknown as {__harnessHeld?: Animation[]}).__harnessHeld ?? []; + return times.map(t => { + for (const animation of held) { + animation.currentTime = t; + } + const scale = getComputedStyle(element).scale; + return {t, value: scale === 'none' ? 1 : Number.parseFloat(scale)}; + }); + }, timeline(560)); + await releaseAnimations(page); + await playing; + const at = (ms: number) => { + const t = ms - PRESS_FEEDBACK_START_MS; + if (t <= 0) { + return 1; + } + // Linear between the seeked frames. + const i = samples.findIndex(s => s.t >= t); + if (i <= 0) { + return samples[Math.max(0, i)].value; + } + const a = samples[i - 1]; + const b = samples[i]; + return a.value + ((b.value - a.value) * (t - a.t)) / (b.t - a.t); + }; + const errors = UIKIT_LONG_PRESS.map(([ms, uikit]) => ({ + ms, + uikit, + web: at(ms), + errorPt: Math.abs(at(ms) - uikit) * UIKIT_PRESS_VIEW_WIDTH, + })); + const rms = Math.sqrt( + errors.reduce((sum, e) => sum + e.errorPt ** 2, 0) / errors.length, + ); + await writeEvidence( + 'curves/contextmenu-press-feedback', + { + [`${testInfo.project.name}.json`]: JSON.stringify( + {rmsPt: rms, errors, samples}, + null, + 1, + ), + }, + testInfo, + ); + testInfo.annotations.push({ + type: 'uikit-parity', + description: `press feedback vs UIKit iOS 26.2 (200 pt card): rms ${rms.toFixed(3)} pt over ${errors.length} frames`, + }); + expect(rms).toBeLessThan(0.1); + expect(Math.max(...errors.map(e => e.errorPt))).toBeLessThan(0.25); +}); + +test('the press feedback keeps advancing every frame while the main thread is blocked', async ({ + page, + browserName, +}, testInfo) => { + test.skip( + browserName !== 'chromium', + 'screencast frame evidence is Chromium-only in Playwright (WebKit: the iOS simulator rig)', + ); + const {driver, trigger, center} = await openTouchStory(page); + // A wider card grows by about a CSS pixel a frame, which the screencast + // resolves; it stays inside the viewport at its largest. + await trigger.evaluate(el => { + (el.firstElementChild as HTMLElement).style.width = '340px'; + }); + await markElement(trigger, {mode: 'outline'}); + const cast = await startScreencast(page); + const playing = driver.play(hold(center, 1100)); + const [feedback] = await waitForAnimation(trigger, {minProgressMs: 40}); + const block = await blockMainThread(page, 450); + await playing; + await page.waitForTimeout(100); + const castFrames = await cast.stop(); + const tracked = await trackMarker(page, castFrames); + const proof = busyWindowProof(tracked, block, {axis: 'width'}); + // What the curve grows by over the judged window, in the marker's pixels. + const touch = + (await page.evaluate(() => performance.timeOrigin)) + + (await touchStartedAt(page)); + const curve = pressFeedbackCurve(); + const scaleAt = (epoch: number) => + curve.at((epoch - touch - PRESS_FEEDBACK_START_MS) / 1000); + const restWidth = (proof.first ?? 0) / scaleAt(block.from + 20); + const expectedGrowth = + restWidth * (scaleAt(block.to - 20) - scaleAt(block.from + 20)); + await writeEvidence( + 'busy-main-thread/contextmenu-press-feedback', + { + [`${testInfo.project.name}.json`]: JSON.stringify( + { + block, + feedback, + proof, + frames: tracked.map(f => ({ + t: +(f.t - block.from).toFixed(1), + w: f.box?.width ?? null, + })), + }, + null, + 1, + ), + }, + testInfo, + ); + const film = await castStoryboard(page, castFrames, { + name: 'contextmenu-press-feedback-busy-main-thread', + zero: block.from, + from: block.from - 40, + to: block.to + 40, + columns: 8, + cellScale: 0.35, + data: t => ({w: tracked.find(f => f.t === t)?.box?.width ?? -1}), + }); + await film.write(testInfo, {block, feedback, expectedGrowth}); + // Frames kept coming at display rate and the card kept growing, as much + // as the curve says. The screencast resolves whole pixels and the card + // grows by about one every other frame, so a pixel may take 3 frames. + expect(proof.medianFrameMs).toBeLessThan(20); + expect(proof.frames).toBeGreaterThanOrEqual(proof.expectedFrames - 2); + expect(proof.monotonic).toBe(true); + expect(proof.longestStallMs).toBeLessThan(4 * (1000 / 60) + 4); + expect((proof.last ?? 0) - (proof.first ?? 0)).toBeGreaterThanOrEqual( + Math.floor(expectedGrowth) - 1, + ); +}); + +test('the press feedback does no Layout or Paint per frame', async ({ + page, + browserName, +}) => { + test.skip(browserName !== 'chromium', 'rendering traces are Chromium-only'); + const {driver, center} = await openTouchStory(page); + const timeOrigin = await page.evaluate(() => performance.timeOrigin); + const trace = await traceRendering(page, async () => { + // Lifted just before the commit: the feedback alone, then its return. + await driver.play(hold(center, commitMs - 20)); + }); + const start = timeOrigin + (await touchStartedAt(page)); + const during = countBetween( + trace, + start + PRESS_FEEDBACK_START_MS + 50, + start + commitMs - 30, + ); + expect(during.Layout ?? 0, JSON.stringify(during)).toBe(0); + expect(during.Paint ?? 0, JSON.stringify(during)).toBe(0); + expect(trace.longTasks).toEqual([]); +}); + +/** The story with somewhere to scroll below it, and its trigger's center. */ +async function openScrollableStory(page: Page) { + const story = await openTouchStory(page); + await page.evaluate(() => { + document.body.style.paddingBottom = '3000px'; + }); + const box = await story.trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + const center = {x: box.x + box.width / 2, y: box.y + box.height / 2}; + // Where the trigger's container sits: the trigger itself is scaled by the + // press feedback. + const top = async () => + story.trigger.evaluate( + el => (el.parentElement ?? el).getBoundingClientRect().y, + ); + return {...story, center, top}; +} + +test('the page does not scroll under a finger held until the menu, nor while the menu is up', async ({ + page, +}) => { + const {driver, center, top} = await openScrollableStory(page); + const before = await top(); + // Held until the menu shows, then dragged up 200 px. + await driver.play( + drag( + center, + {x: center.x, y: center.y - 200}, + {durationMs: 300, holdMs: menuDisplayMs + 50, restMs: 100}, + ), + ); + expect(await top()).toBe(before); + await expect(page.getByRole('menu', {name: 'Photo actions'})).toBeVisible(); + expect(await page.evaluate(() => document.body.style.position)).toBe('fixed'); +}); + +test('a drag that cancels the press after the commit does not scroll the page either', async ({ + page, +}) => { + const {driver, center, top} = await openScrollableStory(page); + const before = await top(); + await driver.play( + drag( + center, + {x: center.x, y: center.y - 200}, + {durationMs: 300, holdMs: commitMs + 100, restMs: 100}, + ), + ); + expect(await menuShownAt(page)).toBeNull(); + expect(await top()).toBe(before); +}); + +test( + 'storyboard: the press feedback up to the menu', + {tag: '@storyboard'}, + async ({page}, testInfo) => { + const {driver, trigger, center} = await openTouchStory(page); + const box = await trigger.boundingBox(); + if (box == null) { + throw new Error('no trigger'); + } + // Gate the page's timers first: the commit and the menu wait while the + // held feedback is photographed, and the finger stays down meanwhile. + await installTimerGate(page); + const playing = driver.play(hold(center, 8000)); + await waitForAnimation(trigger, {minProgressMs: 16}); + await freezeTimers(page); + const {storyboard} = await animationStoryboard(page, { + name: 'contextmenu-press-feedback', + clip: { + x: box.x - 24, + y: box.y - 24, + width: box.width + 48, + height: box.height + 48, + }, + scope: trigger, + // t counts from the visible ramp's start, 188 ms after the touch. + times: timeline(menuDisplayMs - PRESS_FEEDBACK_START_MS, 1000 / 30), + measure: async () => ({ + scale: (await presentedScale(trigger)).toFixed(4), + }), + columns: 6, + cellScale: 0.6, + }); + await releaseAnimations(page); + await thawTimers(page); + await playing; + await storyboard.write(testInfo); + const scales = storyboard.frames.map(f => Number(f.data.scale)); + expect(scales.every((s, i) => i === 0 || s >= scales[i - 1])).toBe(true); + // UIKit's platter at 732.6 ms: 1.0618. + expect(scales.at(-1)).toBeCloseTo(1.0618, 2); + }, +); diff --git a/packages/core/src/ContextMenu/__tests__/uikitLiftFrames.ts b/packages/core/src/ContextMenu/__tests__/uikitLiftFrames.ts new file mode 100644 index 0000000000000..84081cd0cf839 --- /dev/null +++ b/packages/core/src/ContextMenu/__tests__/uikitLiftFrames.ts @@ -0,0 +1,70 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file uikitLiftFrames.ts + * @input UIKit's context-menu presentation measured on iOS 26.2 (iPhone 17 + * Pro simulator, 60 Hz): a 200 × 120 pt card in the middle of the screen + * (x 101, y 377), held 0.9 s, a 3-action menu + * @output UIKIT_LIFT_FRAMES and UIKIT_LIFT_STARTS: per-frame presentation + * values from willDisplay, and when each one starts moving + * @position Fixture for the lift's parity tests + * (packages/core/src/ContextMenu/contextMenuLiftGeometry.test.ts) + * + * Columns: ms after willDisplay; the backdrop's alpha; the preview's width + * (pt; the press had grown it to 212.36); the menu content's opacity; its + * blur (pt). The frame at +27 ms, which still reads the model values, is + * left out, as is the preview's width at +45 ms (not yet presented). + */ + +export const UIKIT_LIFT_FRAMES: ReadonlyArray< + readonly [ + ms: number, + backdrop: number, + previewWidth: number | null, + menuOpacity: number, + menuBlur: number, + ] +> = [ + [45.4, 0.0449, null, 0.138, 3.447], + [62, 0.1504, 215.16, 0.297, 2.812], + [78.7, 0.2832, 217.19, 0.453, 2.188], + [95.4, 0.421, 219.19, 0.593, 1.63], + [112, 0.5505, 220.96, 0.71, 1.159], + [128.7, 0.6644, 222.42, 0.804, 0.783], + [145.4, 0.7596, 223.57, 0.876, 0.496], + [162, 0.8358, 224.45, 0.929, 0.285], + [178.7, 0.8945, 225.09, 0.965, 0.139], + [195.4, 0.9381, 225.53, 0.99, 0.042], + [212, 0.9689, 225.83, 1.004, -0.017], + [228.7, 0.9898, 226.01, 1.012, -0.05], + [245.4, 1.003, 226.11, 1.016, -0.063], + [262, 1.0106, 226.16, 1.016, -0.066], + [278.7, 1.0142, 226.39, 1.015, -0.061], + [295.4, 1.0152, 226.37, 1.013, -0.052], + [312, 1.0145, 226.33, 1.011, -0.043], + [328.7, 1.0129, 226.27, 1.008, -0.033], + [345.4, 1.0109, 226.21, 1.006, -0.025], + [362, 1.0088, 226.16, 1.004, -0.018], + [378.7, 1.0068, 226.12, 1.003, -0.012], + [395.4, 1.0051, 226.08, 1.002, -0.008], + [412, 1.0036, 226.05, 1.001, -0.004], + [428.7, 1.0025, 226.03, 1.001, -0.002], + [445.4, 1.0016, 226.02, 1, -0.001], + [462, 1.0009, 226.01, 1, 0], + [478.7, 1.0005, 226, 1, 0], +]; + +/** The preview's width when the menu shows: the press's 1.0618 × 200 pt. */ +export const UIKIT_LIFT_PREVIEW_FROM = 212.36; + +/** + * When each value starts moving, ms after willDisplay (UIKit commits the + * animations a frame or two after the delegate call): least-squares starts + * for the springs the lift uses. + */ +export const UIKIT_LIFT_STARTS = { + backdrop: 28.5, + previewWidth: 18.5, + menuOpacity: 12.5, + menuBlur: 12.5, +} as const; diff --git a/packages/core/src/ContextMenu/__tests__/uikitPressFrames.ts b/packages/core/src/ContextMenu/__tests__/uikitPressFrames.ts new file mode 100644 index 0000000000000..e7b2fc49df90d --- /dev/null +++ b/packages/core/src/ContextMenu/__tests__/uikitPressFrames.ts @@ -0,0 +1,79 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file uikitPressFrames.ts + * @input Nothing; measured data only + * @output Exports the visible scale of UIKit's context-menu press platter, + * frame by frame, for a long press and for a press lifted before the + * commit, as the reference the press feedback is compared against + * @position Test-only fixture of packages/core/src/ContextMenu (excluded from + * the build with the rest of __tests__); read by the unit and interaction + * tests of the press feedback + * + * UIKit lab (injected host app), iOS 26.2 simulator, iPhone 17 Pro, Oct 23 + * 2026: the presentation width of `_UIContentPlatterView` over a 200 × 120 pt + * view, divided by 200, at each frame's display time in ms from the main + * thread's receipt of the touch. + */ + +/** The measured view's width, to read scale errors in points. */ +export const UIKIT_PRESS_VIEW_WIDTH = 200; + +/** A long press: the menu displayed at 734 ms. */ +export const UIKIT_LONG_PRESS: ReadonlyArray = [ + [199.6, 1], + [215.9, 1.00014], + [232.5, 1.00047], + [249.2, 1.00109], + [265.9, 1.00203], + [282.5, 1.00331], + [299.2, 1.00492], + [315.9, 1.00682], + [332.7, 1.00907], + [349.2, 1.01159], + [366.1, 1.01424], + [382.5, 1.01699], + [399.2, 1.01988], + [415.8, 1.02286], + [432.5, 1.02581], + [449.2, 1.02877], + [466.6, 1.03168], + [483.1, 1.03444], + [499.2, 1.03703], + [515.8, 1.03944], + [532.9, 1.04173], + [554.4, 1.04388], + [565.8, 1.04586], + [583, 1.0477], + [599.2, 1.04944], + [615.8, 1.05111], + [632.5, 1.0527], + [649.2, 1.05425], + [666.5, 1.05577], + [682.5, 1.05727], + [699.2, 1.05878], + [716.1, 1.06029], + [732.6, 1.06183], +]; + +/** When the early press lifted, ms (main-thread receipt). */ +export const UIKIT_EARLY_LIFT_AT = 266.5; + +/** A press lifted before the commit: no menu, the platter springs back. */ +export const UIKIT_EARLY_LIFT: ReadonlyArray = [ + [199.5, 1], + [216.1, 1.00014], + [232.7, 1.00046], + [249.4, 1.00107], + [267.3, 1.002], + [282.8, 1.00327], + [299.5, 1.00438], + [316.1, 1.00493], + [332.8, 1.00504], + [349.5, 1.00483], + [366.1, 1.0044], + [382.8, 1.00382], + [399.4, 1.00318], + [416.1, 1.00252], + [432.8, 1.00189], +]; diff --git a/packages/core/src/ContextMenu/contextMenuLiftGeometry.test.ts b/packages/core/src/ContextMenu/contextMenuLiftGeometry.test.ts new file mode 100644 index 0000000000000..b515bbfa0a507 --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuLiftGeometry.test.ts @@ -0,0 +1,198 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuLiftGeometry.test.ts + * @input vitest, the context-menu constants, and the settled frames of + * UIKit's lifted previews measured on iOS 26.2 (iPhone 17 Pro, 402 × 874) + * @output The lift geometry against UIKit: an icon, a card and a full-width + * row lift to UIKit's sizes about the same center, held inside the + * viewport's gutter; the menu's side and alignment; and the lift's springs + * against UIKit's measured frames + * @position Unit tests of packages/core/src/ContextMenu/contextMenuLiftGeometry.ts + */ + +import {describe, expect, it} from 'vitest'; +import {solveSpring, toSpring} from '../motion/spring'; +import {springs} from '../motion/uikit'; +import {contextMenu} from '../motion/uikit/contextMenu'; +import {liftedBox, liftScale, menuPlacement} from './contextMenuLiftGeometry'; +import { + UIKIT_LIFT_FRAMES, + UIKIT_LIFT_PREVIEW_FROM, + UIKIT_LIFT_STARTS, +} from './__tests__/uikitLiftFrames'; + +const {screenInset, verticalMargin} = contextMenu.lift; +/** + * UIKit's own margins as the gutter (16 pt at the sides, 20 pt plus the safe + * area at the top and bottom), so the frames below are UIKit's. + */ +const uikitGutter = (safeTop: number, safeBottom: number) => ({ + top: safeTop + verticalMargin, + bottom: safeBottom + verticalMargin, + inline: screenInset, +}); +const phone = {width: 402, height: 874, gutter: uikitGutter(0, 0)}; +/** UIKit rounds the lifted size to device pixels (3× here): half of one. */ +const HALF_PIXEL = 1 / 6; + +describe('liftScale', () => { + it('lifts a 60 pt icon to 69 pt (capped at 1.15)', () => { + expect(60 * liftScale(60, 60, phone)).toBeCloseTo(69, 5); + }); + + it('lifts a 200 × 120 card to 226 × 135.67 (26 pt on its larger side)', () => { + const scale = liftScale(200, 120, phone); + expect(200 * scale).toBeCloseTo(226, 5); + expect(Math.abs(120 * scale - 135.67)).toBeLessThan(HALF_PIXEL); + }); + + it('shrinks a full-width 402 × 60 row to 370 × 55.33 (the screen less 16 pt each side)', () => { + const scale = liftScale(402, 60, phone); + expect(402 * scale).toBeCloseTo(370, 5); + expect(Math.abs(60 * scale - 55.33)).toBeLessThan(HALF_PIXEL); + }); +}); + +describe('liftedBox', () => { + it('grows a card about its center, as UIKit’s platter does', () => { + const box = liftedBox({x: 101, y: 377, width: 200, height: 120}, phone); + expect(box.x).toBeCloseTo(88, 5); + expect(box.width).toBeCloseTo(226, 5); + expect(box.y + box.height / 2).toBeCloseTo(437, 5); + }); + + it('centers a full-width row inside the 16 pt side insets', () => { + const box = liftedBox({x: 0, y: 300, width: 402, height: 60}, phone); + expect(box.x).toBeCloseTo(16, 5); + expect(box.x + box.width).toBeCloseTo(386, 5); + }); + + it('holds a card at the left edge at x 16, and one at the right edge at a right edge of 386', () => { + expect( + liftedBox({x: 4, y: 377, width: 200, height: 120}, phone).x, + ).toBeCloseTo(16, 5); + const right = liftedBox({x: 198, y: 377, width: 200, height: 120}, phone); + expect(right.x + right.width).toBeCloseTo(386, 5); + }); + + it('holds a card at the top at y 82 and one at the bottom at a bottom of 820 (UIKit: the safe area and 20 pt)', () => { + const safe = {...phone, gutter: uikitGutter(62, 34)}; + expect( + liftedBox({x: 101, y: 70, width: 200, height: 120}, safe).y, + ).toBeCloseTo(82, 5); + const bottom = liftedBox({x: 101, y: 700, width: 200, height: 120}, safe); + expect(bottom.y + bottom.height).toBeCloseTo(820, 5); + }); + + it("holds a card inside the layer runtime's gutter: the safe area itself where it is past 16, plus an app's bar", () => { + // max(16, safe area) + the app's inset (spec:AST-059), here a 50 px bar. + const runtime = {...phone, gutter: {top: 62, bottom: 34 + 50, inline: 16}}; + expect( + liftedBox({x: 101, y: 40, width: 200, height: 120}, runtime).y, + ).toBeCloseTo(62, 5); + const bottom = liftedBox( + {x: 101, y: 700, width: 200, height: 120}, + runtime, + ); + expect(bottom.y + bottom.height).toBeCloseTo(874 - 84, 5); + }); +}); + +describe('menuPlacement', () => { + const safe = {...phone, gutter: uikitGutter(62, 34)}; + + it('puts the 3-action menu below the middle card, centered, as UIKit did', () => { + const box = liftedBox({x: 101, y: 377, width: 200, height: 120}, safe); + expect(menuPlacement(box, 146, safe)).toEqual({ + side: 'below', + alignment: 'center', + }); + }); + + it('puts the menu above the bottom card, where it does not fit below', () => { + const box = liftedBox({x: 101, y: 700, width: 200, height: 120}, safe); + expect(menuPlacement(box, 146, safe).side).toBe('above'); + }); + + it('aligns the menu with the edge a preview is held against', () => { + const left = liftedBox({x: 4, y: 377, width: 200, height: 120}, safe); + expect(menuPlacement(left, 146, safe).alignment).toBe('start'); + const right = liftedBox({x: 198, y: 377, width: 200, height: 120}, safe); + expect(menuPlacement(right, 146, safe).alignment).toBe('end'); + }); +}); + +describe('the lift’s springs against UIKit’s measured frames (iOS 26.2, 200 × 120 card)', () => { + const at = ( + description: Parameters[0], + from: number, + to: number, + startMs: number, + ms: number, + ) => + ms <= startMs + ? from + : solveSpring(toSpring(description), {from, to}).value( + (ms - startMs) / 1000, + ); + const errors = ( + pick: (row: (typeof UIKIT_LIFT_FRAMES)[number]) => number | null, + model: (ms: number) => number, + ) => + UIKIT_LIFT_FRAMES.flatMap(row => { + const uikit = pick(row); + return uikit == null ? [] : [Math.abs(model(row[0]) - uikit)]; + }); + const rms = (e: number[]) => + Math.sqrt(e.reduce((sum, x) => sum + x * x, 0) / e.length); + + it('the backdrop fades on springs.contextMenuLift (alpha rms < 0.001)', () => { + const e = errors( + row => row[1], + ms => at(springs.contextMenuLift, 0, 1, UIKIT_LIFT_STARTS.backdrop, ms), + ); + expect(rms(e)).toBeLessThan(0.001); + expect(Math.max(...e)).toBeLessThan(0.003); + }); + + it('the preview grows on from the press on springs.contextMenuLift (rms < 0.15 pt)', () => { + const e = errors( + row => row[2], + ms => + at( + springs.contextMenuLift, + UIKIT_LIFT_PREVIEW_FROM, + 226, + UIKIT_LIFT_STARTS.previewWidth, + ms, + ), + ); + expect(rms(e)).toBeLessThan(0.15); + expect(Math.max(...e)).toBeLessThan(0.4); + }); + + it('the menu content fades in on morphSpring (opacity rms < 0.01); its blur is not played, since only transform and opacity move (spec:AST-094 FR5)', () => { + const morphSpring = springs.contextMenuMorph; + const opacity = errors( + row => row[3], + ms => at(morphSpring, 0, 1, UIKIT_LIFT_STARTS.menuOpacity, ms), + ); + expect(rms(opacity)).toBeLessThan(0.01); + expect(Math.max(...opacity)).toBeLessThan(0.03); + }); +}); + +describe('menuPlacement in a right-to-left page', () => { + const viewport = {...phone, direction: 'rtl' as const}; + + it('keeps the menu flush with the edge the preview is held against: the alignment is logical, so the sides swap', () => { + const atLeft = {x: 16, y: 300, width: 226, height: 136}; + const atRight = {x: 402 - 16 - 226, y: 300, width: 226, height: 136}; + expect(menuPlacement(atLeft, 150, viewport).alignment).toBe('end'); + expect(menuPlacement(atRight, 150, viewport).alignment).toBe('start'); + expect( + menuPlacement(atLeft, 150, {...viewport, direction: 'ltr'}).alignment, + ).toBe('start'); + }); +}); diff --git a/packages/core/src/ContextMenu/contextMenuLiftGeometry.ts b/packages/core/src/ContextMenu/contextMenuLiftGeometry.ts new file mode 100644 index 0000000000000..484f7aed8ee52 --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuLiftGeometry.ts @@ -0,0 +1,87 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuLiftGeometry.ts + * @input The context-menu constants and the lift module's geometry + * (liftFromTrigger/liftFromTriggerGeometry.ts) + * @output Exports liftScale, liftedBox and menuPlacement: where UIKit's + * lifted preview lands when the menu shows, and where the menu sits + * against it, with UIKit's numbers + * @position The geometry of ContextMenu's lift presentation (touch); + * ContextMenu.tsx renders it + * + * When a context menu shows on iOS 26.2, the long-pressed view's preview + * lifts above a tinted backdrop: it grows about its center + * (`contextMenu.lift`), stays inside the viewport's gutter, and the menu sits + * a gap away: below it when it fits, else above, centered on it or flush with + * the edge it is held against. The gutter is the layer runtime's one + * definition (spec:AST-059), not UIKit's own margins (16 pt at the sides, 20 + * pt plus the safe area at the top and bottom, `contextMenu.lift`): at the + * sides they agree; near a notch the runtime's leaves up to 20 px less. The + * lift module owns the geometry; this file gives it UIKit's numbers. + * + * SYNC: When modified, update + * /packages/core/src/ContextMenu/contextMenuLiftGeometry.test.ts + */ + +import { + liftFromTriggerRect as liftedRect, + liftFromTriggerScale as liftScaleBy, + surfacePlacement, + type LiftFromTriggerGutter as LiftGutter, + type LiftFromTriggerViewport as LiftViewport, + type SurfacePlacement, +} from '../liftFromTrigger/liftFromTriggerGeometry'; +import type {MorphRect} from '../motion/rectMorph'; +import {contextMenu} from '../motion/uikit/contextMenu'; + +/** A box in client CSS px. */ +export type LiftBox = MorphRect; +export type {LiftGutter, LiftViewport}; +/** Where the menu sits against the lifted preview. */ +export type MenuPlacement = SurfacePlacement; + +/** UIKit's scale rule for the lifted preview (`contextMenu.lift`). */ +const RULE = { + maxScale: contextMenu.lift.maxScale, + scaleUpPoints: contextMenu.lift.scaleUpPoints, +}; + +/** + * The factor a `width` × `height` preview grows by when the menu shows: up to + * `maxScale`, by `scaleUpPoints` on its larger side, and never wider than the + * screen less its insets (which can make it smaller than 1). + */ +export function liftScale( + width: number, + height: number, + viewport: LiftViewport, +): number { + return liftScaleBy(width, height, viewport, RULE); +} + +/** + * The lifted preview's box: `source` scaled by `liftScale` about its center, + * then moved inside the viewport's gutter if it crosses it. + */ +export function liftedBox(source: LiftBox, viewport: LiftViewport): LiftBox { + return liftedRect(source, viewport, RULE); +} + +/** + * The menu's place: below the preview when `menuHeight` fits between it and + * the bottom limit, else above; centered on the preview, or flush with the + * preview's edge when the preview is held against a side of the screen. + */ +export function menuPlacement( + lifted: LiftBox, + menuHeight: number, + viewport: LiftViewport, +): MenuPlacement { + return surfacePlacement( + lifted, + menuHeight, + viewport, + contextMenu.lift.menuGap, + ); +} diff --git a/packages/core/src/ContextMenu/contextMenuPress.test.ts b/packages/core/src/ContextMenu/contextMenuPress.test.ts new file mode 100644 index 0000000000000..253dee360bebe --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuPress.test.ts @@ -0,0 +1,228 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuPress.test.ts + * @input vitest, the gesture arena driven by synthetic touches at explicit + * timestamps (the gesture tests' harness), ContextMenuPressRecognizer + * @output The context menu's touch timeline as a state machine: the press is + * reported when the finger lands, commits at UIKit's moment and not a + * millisecond before, at that moment even when the timer runs late, and + * fails on travel past the long-press tolerance, a lift, a second finger or + * the browser taking the touch; the commit claims the finger and drops its + * click, a press that fails leaves the click alone + * @position Unit tests of packages/core/src/ContextMenu/contextMenuPress.ts + */ + +import {describe, expect, it, vi} from 'vitest'; +import {createHarness} from '../gestures/__tests__/harness'; +import {contextMenu} from '../motion/uikit/contextMenu'; +import {ContextMenuPressRecognizer} from './contextMenuPress'; + +const COMMIT = contextMenu.press.commitMs; +const DISPLAY = contextMenu.press.menuDisplayMs; + +function setup( + options: ConstructorParameters[0] = {}, +) { + const harness = createHarness(); + const press = new ContextMenuPressRecognizer({name: 'press', ...options}); + const states: string[] = []; + press.addTransitionListener((recognizer, _from, to) => { + states.push(`${to}@${recognizer.clockTime}`); + }); + return {harness, press, states}; +} + +describe('ContextMenuPressRecognizer', () => { + it('commits at UIKit’s moment (iOS 26.2: 400 ms, first 60 Hz tick 417 ms), not a millisecond before', () => { + const {harness, press, states} = setup(); + harness.finger(1).down(100, 200, 1000, {recognizers: [press]}); + harness.advance(1000 + COMMIT - 1); + expect(press.state).toBe('possible'); + harness.advance(1000 + COMMIT); + expect(press.state).toBe('began'); + expect(states).toEqual([`began@${1000 + COMMIT}`]); + expect(COMMIT).toBe(400); + }); + + it('commits at the deadline even when the main thread runs the timer late', () => { + const {harness, press, states} = setup(); + harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + // Nothing ran until 900 ms (a blocked main thread): the press still + // began at 400, and the menu was due at 735. + harness.advance(900); + expect(states).toEqual([`began@${COMMIT}`, `changed@${DISPLAY}`]); + expect(press.beganAt).toBe(COMMIT); + }); + + it('reports the press when the finger lands, with its time and place', () => { + const onPressStart = vi.fn((recognizer: ContextMenuPressRecognizer) => ({ + ...recognizer.press, + })); + const {harness, press} = setup({onPressStart}); + harness.finger(1).down(42, 84, 1234.5, {recognizers: [press]}); + expect(onPressStart).toHaveBeenCalledTimes(1); + expect(onPressStart.mock.results[0].value).toEqual({ + time: 1234.5, + x: 42, + y: 84, + }); + }); + + it('claims the finger at the commit and drops the lift’s click', () => { + const {harness, press} = setup(); + const finger = harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + expect(harness.claims).toEqual([]); + harness.advance(COMMIT); + expect(harness.claims).toEqual([1]); + expect(finger.up(COMMIT + 300)).toBe('suppress'); + expect(press.state).toBe('possible'); + expect(press.press).toBeNull(); + }); + + it('makes the menu due at 735 ms while the finger stays (changed, isMenuDue)', () => { + const {harness, press, states} = setup(); + harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.advance(DISPLAY - 1); + expect(press.isMenuDue).toBe(false); + harness.advance(DISPLAY); + expect(press.isMenuDue).toBe(true); + expect(states).toEqual([`began@${COMMIT}`, `changed@${DISPLAY}`]); + expect(DISPLAY).toBe(735); + }); + + it('ends at a lift after the commit, before the menu is due (UIKit shows the menu at the lift)', () => { + const {harness, press, states} = setup(); + const finger = harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.advance(COMMIT); + expect(finger.up(529)).toBe('suppress'); + harness.advance(DISPLAY + 100); + expect(states).toEqual([`began@${COMMIT}`, 'ended@529', 'possible@529']); + }); + + it('is cancelled by a drift past the tolerance after the commit, before the menu', () => { + const {harness, press, states} = setup(); + const finger = harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.advance(COMMIT); + finger.move(0, 5, 480); // within: still committed + finger.move(0, 30, 500); // a 30 px drag at 0.5 s + expect(states).toEqual([`began@${COMMIT}`, 'changed@480', 'cancelled@500']); + harness.advance(DISPLAY + 100); + expect(press.isMenuDue).toBe(false); + }); + + it('lets the finger move freely once the menu is due', () => { + const {harness, press, states} = setup(); + const finger = harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.advance(DISPLAY); + finger.move(0, 80, DISPLAY + 50); + finger.up(DISPLAY + 100); + expect(states).toEqual([ + `began@${COMMIT}`, + `changed@${DISPLAY}`, + `changed@${DISPLAY + 50}`, + `ended@${DISPLAY + 100}`, + `possible@${DISPLAY + 100}`, + ]); + }); + + it.each([ + [7, 7], + [8, 8], + [10, 0], + [11, 11], + [0, -11.5], + ])('still commits after a drift of (%d, %d) pt, as UIKit does', (dx, dy) => { + const {harness, press} = setup(); + harness + .finger(1) + .down(100, 100, 0, {recognizers: [press]}) + .move(100 + dx, 100 + dy, 100); + harness.advance(COMMIT); + expect(press.state).toBe('began'); + }); + + it.each([ + [12, 0], + [0, 12], + [-11.6, 0], + [0, -11.6], + ])( + 'fails on a drift of (%d, %d) pt, past the tolerance on one axis', + (dx, dy) => { + const {harness, press, states} = setup(); + harness + .finger(1) + .down(100, 100, 0, {recognizers: [press]}) + .move(100 + dx, 100 + dy, 250); + expect(press.state).toBe('failed'); + harness.advance(COMMIT); + expect(states).toEqual(['failed@250']); + }, + ); + + it('fails when the finger lifts before the commit, and lets that tap’s click through', () => { + const {harness, press, states} = setup(); + const finger = harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + expect(finger.up(250)).toBe('allow'); + harness.advance(COMMIT); + expect(states).toEqual(['failed@250', 'possible@250']); + }); + + it('fails on a second finger before the commit', () => { + const {harness, press} = setup(); + harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.finger(2).down(50, 0, 120, {recognizers: [press]}); + expect(press.state).toBe('failed'); + }); + + it('ignores a second finger after the commit', () => { + const {harness, press} = setup(); + harness.finger(1).down(0, 0, 0, {recognizers: [press]}); + harness.advance(COMMIT); + harness.finger(2).down(50, 0, COMMIT + 20, {recognizers: [press]}); + expect(press.state).toBe('began'); + }); + + it('fails when the browser takes the finger to scroll', () => { + const {harness, press, states} = setup(); + harness + .finger(1) + .down(0, 0, 0, {recognizers: [press]}) + .takenByBrowser(90); + // The browser owns the finger now: the recognizer lets go of it at once. + expect(states).toEqual(['failed@90', 'possible@90']); + }); + + it('keeps the press while it fails, so the feedback knows which one ended', () => { + const {harness, press} = setup(); + let pressAtFailure: unknown = 'unset'; + press.addTransitionListener((recognizer, _from, to) => { + if (to === 'failed') { + pressAtFailure = recognizer.press; + } + }); + harness + .finger(1) + .down(10, 20, 5, {recognizers: [press]}) + .up(200); + expect(pressAtFailure).toEqual({time: 5, x: 10, y: 20}); + expect(press.press).toBeNull(); + }); + + it('takes a finger or a pen, not a mouse (a mouse opens the menu with its secondary button)', () => { + const mouse = setup(); + mouse.harness + .finger(1) + .down(0, 0, 0, {recognizers: [mouse.press], pointerType: 'mouse'}); + mouse.harness.advance(COMMIT); + expect(mouse.press.state).toBe('possible'); + + const pen = setup(); + pen.harness + .finger(1) + .down(0, 0, 0, {recognizers: [pen.press], pointerType: 'pen'}); + pen.harness.advance(COMMIT); + expect(pen.press.state).toBe('began'); + }); +}); diff --git a/packages/core/src/ContextMenu/contextMenuPress.ts b/packages/core/src/ContextMenu/contextMenuPress.ts new file mode 100644 index 0000000000000..e06afbb5612be --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuPress.ts @@ -0,0 +1,191 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuPress.ts + * @input Touches from the gesture arena; the context-menu constants; the + * long-press detector (useLongPress's createLongPressDetector) + * @output Exports ContextMenuPressRecognizer: the long press that opens a + * context menu on UIContextMenuInteraction's timeline (feedback, commit, + * menu) under a finger, on every touch device, and the press it reports + * @position ContextMenu's touch invocation, on the gesture foundation + * (../gestures); ContextMenu.tsx attaches it to the trigger + * + * A context menu is not UIKit's plain long press (500 ms, 10 pt of travel). + * Its timeline (contextMenu.press), as the states of this recognizer: + * - `possible`: the finger landed (`onPressStart`); the press feedback + * shows from `feedbackOnsetMs`. A lift, a second finger, the browser + * taking the touch to scroll, or a drift past `allowableMovement` on + * either axis: `failed`. + * - `began` at `commitMs`: the click haptic. The recognizer claims the + * touch (the page stops scrolling under it, and the lift sends no click + * and no compatibility mouse events). The menu is not shown yet; a drift + * past `allowableMovement` still ends it with no menu (`cancelled`), and + * a lift shows it at once (`ended` without `isMenuDue`). + * - `changed` with `isMenuDue` at `menuDisplayMs`: the finger is still + * down and the menu shows. From then on the finger moves freely. + * + * The hold runs on useLongPress's detector (createLongPressDetector), the + * one long-press source in core: the commit is its delay, the menu a later + * stage, the drift its tolerance, each passed as the measured value. The + * detector keeps its deadlines on this recognizer's arena (its event clock + * and timers), so the commit lands at its deadline even under a blocked main + * thread. The recognizer is a LongPressGestureRecognizer for the foundation's + * claim and failure rules (a lift, a second finger, the browser taking the + * touch) and the long press's style on the trigger; it never begins on its + * own timer, only at the detector's long press. A press that fails or is cancelled reports it through + * `addTransitionListener`, while `press` still names it. + * + * SYNC: When modified, update /packages/core/src/ContextMenu/ContextMenu.tsx + * and /packages/core/src/ContextMenu/contextMenuPress.test.ts + */ + +import { + isActive, + LongPressGestureRecognizer, + type GestureEventContext, + type GestureTouch, + type LongPressGestureRecognizerOptions, +} from '../gestures'; +import {createLongPressDetector} from '../hooks/useLongPress'; +import {contextMenu} from '../motion/uikit/contextMenu'; + +const {commitMs, menuDisplayMs, allowableMovement} = contextMenu.press; + +/** + * The foundation's own press timer, which this recognizer never lets fire + * (the detector commits it): longer than any hold, and within the range a + * window timer takes. + */ +const NEVER_MS = 2 ** 30; + +/** A finger that may become a context menu: when and where it landed. */ +export interface ContextMenuPress { + /** On the event clock (`performance.now()`, an event's `timeStamp`), ms. */ + readonly time: number; + /** Client CSS px. */ + readonly x: number; + readonly y: number; +} + +export interface ContextMenuPressRecognizerOptions extends Omit< + LongPressGestureRecognizerOptions, + | 'allowableMovement' + | 'minimumPressDuration' + | 'numberOfTapsRequired' + | 'numberOfTouchesRequired' +> { + /** A finger landed and may become a context menu. */ + onPressStart?: (recognizer: ContextMenuPressRecognizer) => void; +} + +export class ContextMenuPressRecognizer extends LongPressGestureRecognizer { + /** The finger in play, from its landing until the gesture resets. */ + press: ContextMenuPress | null = null; + /** + * The finger stayed until `menuDisplayMs`: the menu is due, and the finger + * moves freely from here. + */ + isMenuDue = false; + private readonly onPressStart: + ((recognizer: ContextMenuPressRecognizer) => void) | undefined; + /** The finger the hold follows. */ + private pressTouchId: number | null = null; + /** The hold, on the arena's clock: the commit, then the menu's stage. */ + private readonly detector = createLongPressDetector(() => ({ + delayMs: commitMs, + moveCancelPx: allowableMovement, + stages: [{delayMs: menuDisplayMs, onStage: () => this.showMenu()}], + onLongPress: () => this.commit(), + scheduler: { + now: () => this.now(), + setTimer: (deadline, callback) => { + this.setTimer(deadline, callback); + return () => this.clearTimer(); + }, + }, + })); + + constructor(options: ContextMenuPressRecognizerOptions = {}) { + const {onPressStart, ...rest} = options; + super({ + ...rest, + // A finger or a pen; a mouse opens the menu with its secondary button. + allowedPointerTypes: options.allowedPointerTypes ?? ['touch', 'pen'], + minimumPressDuration: NEVER_MS, + // The drift rule is the detector's tolerance (touchesMoved below). + allowableMovement: Infinity, + }); + this.onPressStart = onPressStart; + } + + /** + * The event clock at this moment: the timestamp of the input (or the + * deadline of the timer) being processed. Read it from a listener to + * place what the transition starts at the time it happened. + */ + get clockTime(): number { + return this.now(); + } + + protected override touchesBegan( + touches: ReadonlyArray, + event: GestureEventContext, + ): void { + super.touchesBegan(touches, event); + if ( + this.press == null && + this.state === 'possible' && + this.numberOfTouches === this.numberOfTouchesRequired + ) { + const [touch] = touches; + this.press = {time: event.time, x: touch.x, y: touch.y}; + this.pressTouchId = touch.id; + // The hold's deadlines count from the touch itself (this replaces the + // foundation's own press timer, which never fires). + this.detector.start({x: touch.x, y: touch.y}, event.time); + this.onPressStart?.(this); + } + } + + protected override touchesMoved(touches: ReadonlyArray): void { + const touch = touches.find(each => each.id === this.pressTouchId); + const hasDrifted = + touch != null && !this.detector.moveTo({x: touch.x, y: touch.y}); + if (hasDrifted && this.state === 'possible') { + this.setState('failed'); + return; + } + if (hasDrifted && isActive(this.state) && !this.isMenuDue) { + this.setState('cancelled'); + return; + } + super.touchesMoved(touches); + } + + /** The detector's long press: the commit claims the finger. */ + private commit(): void { + if ( + this.state === 'possible' && + this.numberOfTouches === this.numberOfTouchesRequired + ) { + this.beganAt = this.now(); + this.setState('began'); + } + } + + /** The detector's later stage: the finger is still down, the menu is due. */ + private showMenu(): void { + if (isActive(this.state)) { + this.isMenuDue = true; + this.setState('changed'); + } + } + + protected override reset(): void { + super.reset(); + this.detector.cancel(); + this.pressTouchId = null; + this.press = null; + this.isMenuDue = false; + } +} diff --git a/packages/core/src/ContextMenu/contextMenuPressCurve.test.ts b/packages/core/src/ContextMenu/contextMenuPressCurve.test.ts new file mode 100644 index 0000000000000..01cc989944e7c --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuPressCurve.test.ts @@ -0,0 +1,226 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuPressCurve.test.ts + * @input vitest, the context-menu constants, the press feedback curves, an + * independent RK4 integration, and per-frame presentation sizes of UIKit's + * press platter measured on iOS 26.2 + * @output The press feedback against UIKit: the size UIKit asks for, the + * closed-form curve against a numerical integration of the same spring, + * continuity where the ramp changes, and the visible size of a long press + * and of a press lifted at 0.25 s against the measured frames + * @position Unit tests of packages/core/src/ContextMenu/contextMenuPressCurve.ts + */ + +import {afterEach, describe, expect, it, vi} from 'vitest'; +import {solveSpring, toSpring} from '../motion/spring'; +import { + contextMenu, + registerContextMenuTable, +} from '../motion/uikit/contextMenu'; +import { + PRESS_FEEDBACK_START_MS, + pressFeedbackCurve, + pressFeedbackMotion, + pressFeedbackTarget, +} from './contextMenuPressCurve'; +import { + UIKIT_EARLY_LIFT, + UIKIT_EARLY_LIFT_AT, + UIKIT_LONG_PRESS, + UIKIT_PRESS_VIEW_WIDTH, +} from './__tests__/uikitPressFrames'; + +const press = contextMenu.press; + +/** The visible scale of a held press `t` ms after the touch. */ +function held(t: number): number { + const curve = pressFeedbackCurve(); + return t <= PRESS_FEEDBACK_START_MS + ? 1 + : curve.at((t - PRESS_FEEDBACK_START_MS) / 1000); +} + +/** + * The visible scale of a press lifted at `lift` ms: the held curve until the + * screen shows the lift, then the feedback spring back to 1 from where the + * curve was (what the animator plays on `fail`). + */ +function liftedAt(lift: number): (t: number) => number { + const curve = pressFeedbackCurve(); + const turn = lift + press.returnLatencyMs; + const s = (turn - PRESS_FEEDBACK_START_MS) / 1000; + const back = solveSpring(toSpring(press.feedbackSpring), { + from: curve.at(s), + to: 1, + velocity: curve.velocity(s), + }); + return t => (t < turn ? held(t) : back.value((t - turn) / 1000)); +} + +function errors( + measured: ReadonlyArray, + model: (t: number) => number, +): {rms: number; max: number} { + const points = measured.map( + ([t, value]) => Math.abs(model(t) - value) * UIKIT_PRESS_VIEW_WIDTH, + ); + return { + rms: Math.sqrt(points.reduce((sum, e) => sum + e * e, 0) / points.length), + max: Math.max(...points), + }; +} + +describe('pressFeedbackTarget', () => { + it('is 1 until the onset, then ramps to 1.04 at the commit', () => { + expect(pressFeedbackTarget(0)).toBe(1); + expect(pressFeedbackTarget(press.feedbackOnsetMs)).toBe(1); + expect(pressFeedbackTarget(press.commitMs)).toBeCloseTo(1.04, 10); + }); + + it('continues on the slower ramp from the same size, and stops when the menu displays', () => { + expect(pressFeedbackTarget(press.commitMs + 1e-9)).toBeCloseTo(1.04, 8); + expect(pressFeedbackTarget(press.menuDisplayMs)).toBeCloseTo( + 1 + (press.feedbackRateAfterCommit * press.menuDisplayMs) / 1000, + 10, + ); + expect(pressFeedbackTarget(5000)).toBe( + pressFeedbackTarget(press.menuDisplayMs), + ); + }); +}); + +describe('pressFeedbackCurve', () => { + it('matches a numerical integration of the feedback spring chasing the ramp', () => { + const {stiffness, damping, mass} = toSpring(press.feedbackSpring); + const curve = pressFeedbackCurve(); + const target = (s: number) => + pressFeedbackTarget(press.feedbackOnsetMs + s * 1000); + let x = 1; + let v = 0; + const h = 1e-4; + let worst = 0; + for (let step = 0; step * h < curve.duration - h; step++) { + const s = step * h; + if (step % 50 === 0) { + worst = Math.max(worst, Math.abs(curve.at(s) - x)); + } + const f = (px: number, pv: number, ps: number): [number, number] => [ + pv, + (stiffness * (target(ps) - px) - damping * pv) / mass, + ]; + const [a1, b1] = f(x, v, s); + const [a2, b2] = f(x + (h / 2) * a1, v + (h / 2) * b1, s + h / 2); + const [a3, b3] = f(x + (h / 2) * a2, v + (h / 2) * b2, s + h / 2); + const [a4, b4] = f(x + h * a3, v + h * b3, s + h); + x += (h / 6) * (a1 + 2 * a2 + 2 * a3 + a4); + v += (h / 6) * (b1 + 2 * b2 + 2 * b3 + b4); + } + expect(worst).toBeLessThan(1e-6); + }); + + it('is continuous in value and velocity where the ramp changes', () => { + const curve = pressFeedbackCurve(); + for (const ms of [press.commitMs, press.menuDisplayMs]) { + const s = (ms - press.feedbackOnsetMs) / 1000; + expect(curve.at(s + 1e-7)).toBeCloseTo(curve.at(s - 1e-7), 6); + expect(curve.velocity(s + 1e-7)).toBeCloseTo(curve.velocity(s - 1e-7), 4); + } + }); + + it('starts from rest at 1 and rests on the last size', () => { + const curve = pressFeedbackCurve(); + expect(curve.at(0)).toBe(1); + expect(curve.velocity(0)).toBe(0); + expect(curve.at(curve.duration)).toBeCloseTo( + pressFeedbackTarget(press.menuDisplayMs), + 3, + ); + expect(Math.abs(curve.velocity(curve.duration))).toBeLessThan(1e-3); + }); + + it('continues from a press still moving', () => { + const curve = pressFeedbackCurve({from: 1.02, velocity: -0.05}); + expect(curve.at(0)).toBeCloseTo(1.02, 10); + expect(curve.velocity(0)).toBeCloseTo(-0.05, 10); + }); +}); + +describe('the press feedback against UIKit (iOS 26.2, measured frames)', () => { + it('a long press: the visible size follows UIKit’s within 0.05 pt rms on a 200 pt view', () => { + const {rms, max} = errors(UIKIT_LONG_PRESS, held); + expect(rms).toBeLessThan(0.05); + expect(max).toBeLessThan(0.12); + }); + + it('a press lifted at 0.25 s: it overshoots on the way back as UIKit’s does, within 0.05 pt rms', () => { + const {rms, max} = errors(UIKIT_EARLY_LIFT, liftedAt(UIKIT_EARLY_LIFT_AT)); + expect(rms).toBeLessThan(0.05); + expect(max).toBeLessThan(0.12); + // The measured peak after the lift: 1.00504 at 332.8 ms. + expect(liftedAt(UIKIT_EARLY_LIFT_AT)(332.8)).toBeCloseTo(1.00504, 4); + }); +}); + +describe('pressFeedbackMotion: the feedback plays the table’s motion (touch motion stays internal)', () => { + // The touch code registers the table before its first motion. + registerContextMenuTable(); + + afterEach(() => { + vi.restoreAllMocks(); + document.body.innerHTML = ''; + }); + + function themed(values: Record) { + vi.spyOn(window, 'getComputedStyle').mockImplementation( + (element: Element) => + ({ + getPropertyValue: (name: string) => + element.hasAttribute('data-astryx-theme') + ? (values[name] ?? '') + : '', + }) as CSSStyleDeclaration, + ); + const scope = document.createElement('div'); + scope.setAttribute('data-astryx-theme', 'test'); + const child = document.createElement('div'); + scope.append(child); + document.body.append(scope); + return child; + } + + it('reads the table’s spring, onset and latency where no theme sets them', () => { + const element = themed({}); + expect(pressFeedbackMotion(element)).toEqual({ + spring: contextMenu.press.feedbackSpring, + onsetMs: contextMenu.press.feedbackOnsetMs, + latencyMs: contextMenu.press.feedbackLatencyMs, + }); + }); + + it('takes nothing from a theme: no ContextMenu value is a motion role', () => { + const element = themed({ + '--motion-context-menu-press-feedback-spring': '0.5s 0', + '--motion-context-menu-press-feedback-latency-ms': '80ms', + '--motion-context-menu-press-feedback-onset-ms': '300ms', + '--motion-context-menu-press-commit-ms': '100ms', + }); + const motion = pressFeedbackMotion(element); + expect(motion).toEqual({ + spring: contextMenu.press.feedbackSpring, + onsetMs: contextMenu.press.feedbackOnsetMs, + latencyMs: contextMenu.press.feedbackLatencyMs, + }); + const curve = pressFeedbackCurve({motion}); + expect(curve.at(0)).toBe(1); + expect(curve.at(0.3)).toBeGreaterThan(1); + }); + + it('starts its ramp at the onset it is given and reaches the commit continuously', () => { + // Nothing until the onset; the ramp then reaches UIKit's commit + // continuously. + expect(pressFeedbackTarget(250, 300)).toBe(1); + expect(pressFeedbackTarget(400, 300)).toBeCloseTo(1.016, 6); + expect(pressFeedbackTarget(400.001, 300)).toBeCloseTo(1.016, 4); + }); +}); diff --git a/packages/core/src/ContextMenu/contextMenuPressCurve.ts b/packages/core/src/ContextMenu/contextMenuPressCurve.ts new file mode 100644 index 0000000000000..aa77f4b4aa388 --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuPressCurve.ts @@ -0,0 +1,182 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file contextMenuPressCurve.ts + * @input The context-menu constants and + * the motion foundation's spring toward a moving target + * @output Exports pressFeedbackTarget (the size UIKit asks the press copy for + * at a moment of the press), PRESS_FEEDBACK_START_MS, pressFeedbackMotion + * (the feedback's spring, onset and latency) and + * pressFeedbackCurve (the size it shows, as a PressFeedbackCurve the + * compositor animator plays) + * @position The press feedback's curve, for ContextMenu's touch invocation + * + * On iOS 26.2 a finger resting on a view with a context menu makes a copy of + * the view grow from its center: UIKit asks for a size that ramps linearly + * (`contextMenu.press.feedbackRate` from the onset, a slower + * `feedbackRateAfterCommit` after the commit, until the menu displays), and + * the copy's visible size chases that ramp through a spring + * (`feedbackSpring`), shown `feedbackLatencyMs` after UIKit starts it. These + * are pure functions of the constants; the curve is closed-form, so the whole + * held press can be computed when the finger lands and played on the + * compositor. None of them is a motion role: touch motion stays internal, + * so no theme retunes them, and `pressFeedbackMotion` reads the table's. + * + * SYNC: When modified, update + * /packages/core/src/ContextMenu/contextMenuPressCurve.test.ts + */ + +import { + settlingDuration, + solveSpringTowardRamp, + toSpring, + type SpringDescription, + type SpringRampCurve, +} from '../motion/spring'; +import {resolveMotion} from '../motion/resolveMotion'; +import {contextMenu} from '../motion/uikit/contextMenu'; + +const { + feedbackOnsetMs, + commitMs, + menuDisplayMs, + feedbackRate, + feedbackRateAfterCommit, + feedbackSpring, + feedbackLatencyMs, +} = contextMenu.press; + +/** The press feedback's motion: its spring, onset and display latency. */ +export interface PressFeedbackMotion { + readonly spring: SpringDescription; + readonly onsetMs: number; + readonly latencyMs: number; +} + +/** The table's motion (UIKit's). */ +const TABLE_MOTION: PressFeedbackMotion = { + spring: feedbackSpring, + onsetMs: feedbackOnsetMs, + latencyMs: feedbackLatencyMs, +}; + +/** + * The press feedback's motion for `element`: the table's spring and latency, + * as its registry resolves them (no ContextMenu value is a motion role, so a + * theme sets none). The onset is the measured one: core's press machine + * decides when a press is believed. + */ +export function pressFeedbackMotion( + element: Element | null | undefined, +): PressFeedbackMotion { + return { + spring: resolveMotion(element, contextMenu.press.feedbackSpring), + onsetMs: feedbackOnsetMs, + latencyMs: resolveMotion(element, contextMenu.press, 'feedbackLatencyMs'), + }; +} + +/** + * The scale UIKit asks the press copy for, `t` ms after the finger landed: + * 1 until the onset, then the first ramp, then (continuously) the second one + * until the menu displays, where it stops. + */ +export function pressFeedbackTarget( + t: number, + onsetMs: number = feedbackOnsetMs, +): number { + if (t <= onsetMs) { + return 1; + } + const atCommit = 1 + (feedbackRate * Math.max(commitMs - onsetMs, 0)) / 1000; + if (t <= commitMs) { + return 1 + (feedbackRate * (t - onsetMs)) / 1000; + } + return ( + atCommit + + (feedbackRateAfterCommit * (Math.min(t, menuDisplayMs) - commitMs)) / 1000 + ); +} + +/** + * A closed-form curve over `duration` seconds: t is seconds from its start, + * the value at `duration` its end state. The shape the compositor animator + * plays (`animator.play`). + */ +export interface PressFeedbackCurve { + readonly duration: number; + at(t: number): number; + /** Units per second. */ + velocity(t: number): number; +} + +/** When the visible curve starts, ms after the finger landed. */ +export const PRESS_FEEDBACK_START_MS = feedbackOnsetMs + feedbackLatencyMs; + +/** + * The scale the press copy shows from `PRESS_FEEDBACK_START_MS` on (t in + * seconds from there), for a finger that stays: the feedback spring chasing + * `pressFeedbackTarget` as the screen shows it (`feedbackLatencyMs` late), + * chained piece by piece where the ramp changes, then settling on the last + * size. It starts from rest at 1 unless the view is still moving from an + * earlier press. + */ +export function pressFeedbackCurve({ + from = 1, + velocity = 0, + motion = TABLE_MOTION, +}: { + from?: number; + velocity?: number; + /** The motion to play (`pressFeedbackMotion`); the table's by default. */ + motion?: PressFeedbackMotion; +} = {}): PressFeedbackCurve { + const {onsetMs} = motion; + const spring = toSpring(motion.spring); + const commit = Math.max(commitMs - onsetMs, 0) / 1000; + const display = Math.max(menuDisplayMs - onsetMs, 0) / 1000; + const first = solveSpringTowardRamp(spring, { + from, + velocity, + target: 1, + slope: feedbackRate, + }); + const second = solveSpringTowardRamp(spring, { + from: first.value(commit), + velocity: first.velocity(commit), + target: pressFeedbackTarget(commitMs, onsetMs), + slope: feedbackRateAfterCommit, + }); + const secondLength = display - commit; + const last = solveSpringTowardRamp(spring, { + from: second.value(secondLength), + velocity: second.velocity(secondLength), + target: pressFeedbackTarget(menuDisplayMs, onsetMs), + slope: 0, + }); + const settle = settlingDuration(spring, {epsilon: 1e-4}); + const pieces: ReadonlyArray<[start: number, curve: SpringRampCurve]> = [ + [0, first], + [commit, second], + [display, last], + ]; + const pieceAt = (t: number): [number, SpringRampCurve] => { + for (let i = pieces.length - 1; i > 0; i--) { + if (t >= pieces[i][0]) { + return pieces[i]; + } + } + return pieces[0]; + }; + return { + duration: display + settle, + at: t => { + const [start, curve] = pieceAt(t); + return curve.value(t - start); + }, + velocity: t => { + const [start, curve] = pieceAt(t); + return curve.velocity(t - start); + }, + }; +} diff --git a/packages/core/src/ContextMenu/contextMenuPressFeedback.ts b/packages/core/src/ContextMenu/contextMenuPressFeedback.ts new file mode 100644 index 0000000000000..7aa44288cf93d --- /dev/null +++ b/packages/core/src/ContextMenu/contextMenuPressFeedback.ts @@ -0,0 +1,260 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file contextMenuPressFeedback.ts + * @input The press feedback's curve (contextMenuPressCurve.ts), the + * context-menu constants, the compositor animator, the trigger element and + * the press paint core's one press machine gives it (utils/pressFeedback.ts, + * `subscribePressPaint`: one shared registry, no observer per trigger) + * @output Exports useContextMenuPressFeedback (hold and release from + * ContextMenu's menu, and end when the lift takes over; the growth + * itself follows the press machine) + * @position The press feedback of ContextMenu's touch invocation + * (pressFeedback.ts believes the press, contextMenuPress.ts opens the menu; + * this shows it) + * + * On iOS 26.2 a finger resting on a view with a context menu makes a copy of + * the view grow from its center: UIKit asks for a size that ramps linearly + * (`contextMenu.press.feedbackRate` from the onset, a slower + * `feedbackRateAfterCommit` after the commit), and the copy's visible size + * chases that ramp through a spring (`feedbackSpring`), shown + * `feedbackLatencyMs` after UIKit starts it. A finger that lifts or wanders + * before the commit sends the size back to 1 on the same spring + * (`returnLatencyMs` after the lift). After the commit the view stays grown + * while the menu is up, and returns on the dismissal spring + * (`springs.contextMenuDismiss`) when it closes. + * + * Here the trigger itself takes the scale (the individual `scale` property, + * from its center), played by the compositor animator. The trigger is a + * pressable surface of core's one press machine, which writes + * `data-astryx-press="on"` once it believes the press (its onset) and takes + * it away when the press ends (a lift, travel, a scroll, a second finger): + * the growth starts at "on" and springs back when it goes, unless the menu + * opened. The whole held curve is computed when it starts and placed on the + * document timeline, so it keeps advancing every frame while the main thread + * is busy. Nothing renders and nothing reads layout during the press. The + * machine's onset (150 ms, on a timer) stands for UIKit's measured + * `feedbackOnsetMs` (the same 150 ms, from the touch). + * + * Not shown: an inline trigger (`triggerAs="span"`), which a transform does + * not apply to, and Reduce Motion (`contextMenu.reducedMotionFeedback`). + * + * SYNC: When modified, update /packages/core/src/ContextMenu/ContextMenu.tsx + */ + +import {useCallback, useEffect, useRef, type RefObject} from 'react'; +import {usePressFeedback} from '../hooks/usePressFeedback'; +import { + PRESSED_ATTRIBUTE, + subscribePressPaint, + type pressableProps, +} from '../utils/pressFeedback'; +import {createAnimator, type Animator} from '../motion/animator'; +import {resolveMotion} from '../motion/resolveMotion'; +import {registerSpringsTable, springs} from '../motion/uikit/springs'; +import { + contextMenu, + registerContextMenuTable, +} from '../motion/uikit/contextMenu'; +import {pressFeedbackCurve, pressFeedbackMotion} from './contextMenuPressCurve'; + +const {feedbackSpring} = contextMenu.press; + +export interface ContextMenuPressFeedback { + /** + * The trigger's marker for core's press machine (utils/pressFeedback.ts): + * spread on the trigger; the growth follows the pressed state it writes. + */ + pressable: typeof pressableProps; + /** The menu the press opened closed: return on the dismissal spring. */ + release(): void; + /** + * The menu showed: the feedback stops where it is. The menu's cursor anchor + * sits inside the trigger and anchor positioning follows its transformed + * box, so a trigger still growing would carry the open menu with it. + */ + hold(): void; + /** + * The lift takes over: the feedback lets go of the trigger at once (its + * scale and compositing layer), and the lift grows it on from the rect the + * press left. + */ + end(): void; +} + +/** + * The press feedback of a ContextMenu trigger. `isShown` is read at each + * press: false for an inline trigger. `triggerKey` changes when the trigger + * element is replaced (`triggerAs`), so the new one is followed. + */ +export function useContextMenuPressFeedback( + triggerRef: RefObject, + isShown: boolean, + triggerKey?: unknown, +): ContextMenuPressFeedback { + const pressable = usePressFeedback(); + const animatorRef = useRef(null); + const isShownRef = useRef(isShown); + isShownRef.current = isShown; + // Whether the trigger carries a scale of ours (to put back and clear). + const isGrownRef = useRef(false); + // The menu the press opened is up: the growth holds until it closes. + const isHeldRef = useRef(false); + + useEffect( + () => () => { + animatorRef.current?.dispose(); + animatorRef.current = null; + }, + [], + ); + + const animatorFor = useCallback((): Animator | null => { + const element = triggerRef.current; + if (element == null) { + return null; + } + // The feedback's springs resolve through their tables (none is a motion + // role): register them before the first motion (idempotent). + registerContextMenuTable(); + registerSpringsTable(); + if (animatorRef.current?.element !== element) { + animatorRef.current?.dispose(); + animatorRef.current = createAnimator(element); + } + return animatorRef.current; + }, [triggerRef]); + + // Back at 1: drop the inline scale, which would otherwise leave the trigger + // a stacking context and a containing block for fixed descendants. + const settleBack = useCallback((finished: Promise) => { + void finished.then(isDone => { + const element = animatorRef.current?.element; + if (isDone && element != null && isGrownRef.current) { + isGrownRef.current = false; + element.style.removeProperty('scale'); + } + }); + }, []); + + const start = useCallback( + (time: number) => { + if (!isShownRef.current || isHeldRef.current) { + return; + } + const animator = animatorFor(); + // Under Reduce Motion the view does not grow (UIKit's + // `reducedMotionFeedback`), and no fade stands in for it. + if ( + animator == null || + (animator.reducedMotion() !== 'off' && + !contextMenu.reducedMotionFeedback) + ) { + return; + } + isGrownRef.current = true; + // The table's spring and latency. `time` is the onset (the press + // machine's "on"). + const motion = pressFeedbackMotion(animator.element); + const at = time + motion.latencyMs; + animator.play( + { + scale: pressFeedbackCurve({ + from: animator.get('scaleX', at), + velocity: animator.getVelocity('scaleX', at), + motion, + }), + }, + {at}, + ); + }, + [animatorFor], + ); + + const fail = useCallback( + (time: number) => { + const animator = animatorRef.current; + if (animator == null || !isGrownRef.current || isHeldRef.current) { + return; + } + settleBack( + animator.animate( + {scale: 1}, + {spring: feedbackSpring}, + { + at: + time + + resolveMotion( + animator.element, + contextMenu.press, + 'returnLatencyMs', + ), + }, + ).finished, + ); + }, + [settleBack], + ); + + const release = useCallback(() => { + isHeldRef.current = false; + const animator = animatorRef.current; + if (animator == null || !isGrownRef.current) { + return; + } + settleBack( + animator.animate({scale: 1}, {spring: springs.contextMenuDismiss}) + .finished, + ); + }, [settleBack]); + + const hold = useCallback(() => { + isHeldRef.current = true; + if (isGrownRef.current) { + animatorRef.current?.stop(['scale']); + } + }, []); + + const end = useCallback(() => { + const animator = animatorRef.current; + if (animator == null || !isGrownRef.current) { + return; + } + isGrownRef.current = false; + // Let go of the trigger at once, its compositing layer included: the + // lift's own motion has it to itself from here (a new press makes a new + // animator). + const element = animator.element; + animatorRef.current = null; + animator.dispose(); + element?.style.removeProperty('scale'); + }, []); + + // Follow the press machine's paint on the trigger: grow from its "on", + // spring back when the press ends with no menu. One registry serves every + // surface (no observer or listener per trigger). + useEffect(() => { + void triggerKey; + const element = triggerRef.current; + if (element == null) { + return; + } + let isPressed = element.getAttribute(PRESSED_ATTRIBUTE) === 'on'; + return subscribePressPaint(element, paint => { + const next = paint === 'on'; + if (next === isPressed) { + return; + } + isPressed = next; + if (next) { + start(performance.now()); + } else { + fail(performance.now()); + } + }); + }, [triggerRef, triggerKey, start, fail]); + + return {pressable, release, hold, end}; +} diff --git a/packages/core/src/Layer/__tests__/inputHold.fixture.ts b/packages/core/src/Layer/__tests__/inputHold.fixture.ts new file mode 100644 index 0000000000000..84cfc1b93f198 --- /dev/null +++ b/packages/core/src/Layer/__tests__/inputHold.fixture.ts @@ -0,0 +1,137 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file inputHold.fixture.ts + * @input Nothing: bundled by esbuild and injected into a blank page + * @output A page whose whole area is a button (`#under`), a modal dialog + * (`#dialog`: text, an action, and a gesture-host tap recognizer on the + * dialog itself, an ancestor of the hold's shield), a light-dismissing + * popover (`#pop`) and a full-viewport layer (`#lift`) whose content and + * tint take the hits while a test may make its root take none, plus + * `window.__hold`: open a layer, hold and + * release it, read what an element at a point is and what lies past a + * hold's shield there, and a log of every + * click, tap recognition and popover toggle + * @position Browser fixture of inputHold.interaction.browser.spec.ts; + * test-only + */ + +import {getGestureHost} from '../../gestures/GestureHost'; +import {TapGestureRecognizer} from '../../gestures/TapGestureRecognizer'; +import {holdInput, isInputShield, type InputHoldScope} from '../inputHold'; + +type LayerId = 'dialog' | 'pop' | 'lift'; + +interface HoldLog { + clicks: string[]; + taps: number; + toggles: string[]; +} + +declare global { + interface Window { + __hold: { + log: HoldLog; + open: (id: LayerId) => void; + hold: (id: LayerId, scope: InputHoldScope) => void; + release: () => void; + hitAt: (x: number, y: number) => string; + hitPast: (x: number, y: number) => string; + }; + } +} + +document.body.style.margin = '0'; +document.body.innerHTML = ` + + +

Delete this item?

+ + +
+ +
+ `; + +function element(id: string): HTMLElement { + const found = document.getElementById(id); + if (found == null) { + throw new Error(`missing #${id}`); + } + return found; +} + +const log: HoldLog = {clicks: [], taps: 0, toggles: []}; + +document.addEventListener( + 'click', + event => { + const target = event.target; + if (target instanceof Element) { + const button = target.closest('button'); + if (button != null) { + log.clicks.push(button.id); + } + } + }, + true, +); +element('pop').addEventListener('toggle', event => { + log.toggles.push(`pop:${event.newState}`); +}); + +// A recognizer on the dialog itself sees every touch inside it, the hold's +// shield included: only the gesture host's touch gate keeps it away. +const tap = new TapGestureRecognizer({name: 'dialogTap'}); +tap.addTransitionListener((_current, _from, to) => { + if (to === 'ended') { + log.taps += 1; + } +}); +getGestureHost(document).attach(element('dialog'), tap); + +let release: (() => void) | null = null; + +window.__hold = { + log, + open(id) { + if (id === 'dialog') { + (element('dialog') as HTMLDialogElement).showModal(); + } else if (id === 'pop') { + element('pop').showPopover(); + } else { + element('lift').style.display = 'block'; + } + }, + hold(id, scope) { + release = holdInput(element(id), { + until: new Promise(() => {}), + scope, + }); + }, + release() { + release?.(); + release = null; + }, + hitAt(x, y) { + return describe(document.elementFromPoint(x, y)); + }, + hitPast(x, y) { + return describe( + document.elementsFromPoint(x, y).find(el => !isInputShield(el)) ?? null, + ); + }, +}; + +function describe(hit: Element | null): string { + if (hit == null) { + return 'nothing'; + } + if (isInputShield(hit)) { + return `shield:${hit.parentElement?.id ?? ''}`; + } + return hit.id !== '' ? hit.id : hit.tagName.toLowerCase(); +} diff --git a/packages/core/src/Layer/__tests__/inputHold.interaction.browser.spec.ts b/packages/core/src/Layer/__tests__/inputHold.interaction.browser.spec.ts new file mode 100644 index 0000000000000..c31bc8fe820a5 --- /dev/null +++ b/packages/core/src/Layer/__tests__/inputHold.interaction.browser.spec.ts @@ -0,0 +1,348 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file inputHold.interaction.browser.spec.ts + * @input The input-hold fixture bundled with esbuild, a phone-sized + * touch screen, and real engine touch input (touchDriver) + * @output Browser proof of what the unit tests cannot see: the hold's shield + * is the hit target over the whole viewport, above a modal dialog in the + * top layer; a tap during a hold reaches no click and no recognizer on the + * layer (the gesture host's touch gate) and leaves a light-dismissing + * popover open; a layer root that takes no hits itself (pointer-events: + * none, or visibility: hidden, its content opting back in) is held all the + * same; keys still act during a presentation's input hold; a dismissal's + * hold makes the content inert and moves focus out of it, so no key acts + * on it either (a focused button that turns inert still takes Enter and + * Space in both engines); each release gives input back; a hit test + * that skips shields (isInputShield) finds what lies beneath, and + * a layer above the held one still wins; a one-finger + * pan on the shield scrolls nothing while a pinch on it still zooms the + * page (spec:AST-094 FR10, FR18, FR35; Chromium, which pinch-zooms from + * injected touches) + * @position Browser tests of packages/core/src/Layer/inputHold.ts, run + * by the interaction project (`pnpm test:interaction inputHold`) + */ + +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {build} from 'esbuild'; +import {expect, test, type Page} from '@playwright/test'; +import { + drag, + pinch, + touchDriver, + type TouchDriver, +} from '@astryxdesign/interaction-harness'; + +type Point = {x: number; y: number}; +type Scope = 'viewport' | 'layer'; +type LayerId = 'dialog' | 'pop' | 'lift'; + +interface HoldApi { + log: {clicks: string[]; taps: number; toggles: string[]}; + open: (id: LayerId) => void; + hold: (id: LayerId, scope: Scope) => void; + release: () => void; + hitAt: (x: number, y: number) => string; + hitPast: (x: number, y: number) => string; +} + +let bundle = ''; + +test.beforeAll(async () => { + const result = await build({ + entryPoints: [ + path.join( + path.dirname(fileURLToPath(import.meta.url)), + 'inputHold.fixture.ts', + ), + ], + bundle: true, + write: false, + format: 'iife', + platform: 'browser', + target: 'es2022', + }); + bundle = result.outputFiles[0].text; +}); + +async function load(page: Page): Promise { + // A real navigation first: touch emulation exposes the Touch Events API to + // documents it creates. + await page.goto('about:blank'); + await page.setContent( + '', + ); + await page.addScriptTag({content: bundle}); + return touchDriver(page); +} + +type HoldWindow = {__hold: HoldApi}; + +const open = async (page: Page, id: LayerId) => + page.evaluate(layer => { + (window as unknown as HoldWindow).__hold.open(layer); + }, id); +const hold = async (page: Page, id: LayerId, scope: Scope) => + page.evaluate( + ([layer, how]) => { + (window as unknown as HoldWindow).__hold.hold(layer, how); + }, + [id, scope] as const, + ); +const release = async (page: Page) => + page.evaluate(() => { + (window as unknown as HoldWindow).__hold.release(); + }); +const hitAt = async (page: Page, point: Point) => + page.evaluate( + at => (window as unknown as HoldWindow).__hold.hitAt(at.x, at.y), + point, + ); +const hitPast = async (page: Page, point: Point) => + page.evaluate( + at => (window as unknown as HoldWindow).__hold.hitPast(at.x, at.y), + point, + ); +const readLog = async (page: Page) => + page.evaluate(() => (window as unknown as HoldWindow).__hold.log); + +async function centre(page: Page, id: string): Promise { + const box = await page.locator(`#${id}`).boundingBox(); + if (box == null) { + throw new Error(`#${id} has no box`); + } + return {x: box.x + box.width / 2, y: box.y + box.height / 2}; +} + +// A tap's click (or its recognition) lands within a few frames. +const settle = async (page: Page) => page.waitForTimeout(250); + +test("during a presentation's input hold on a modal dialog, a tap on it or beside it reaches nothing, keys still act, and the release gives touch back", async ({ + page, +}) => { + const driver = await load(page); + await open(page, 'dialog'); + const action = await centre(page, 'action'); + const message = await centre(page, 'message'); + const beside = {x: 12, y: 12}; + await hold(page, 'dialog', 'viewport'); + // The shield is what a touch hits, on the dialog and over its backdrop. + expect(await hitAt(page, action)).toBe('shield:dialog'); + expect(await hitAt(page, beside)).toBe('shield:dialog'); + await driver.tap(action); + await driver.tap(message); + await settle(page); + // No click, and the recognizer on the dialog saw nothing: the gate. + expect(await readLog(page)).toEqual({clicks: [], taps: 0, toggles: []}); + // Keys still act during a presentation's input hold. + await page.locator('#action').focus(); + await page.keyboard.press('Enter'); + await expect + .poll(async () => (await readLog(page)).clicks) + .toEqual(['action']); + await release(page); + expect(await hitAt(page, action)).toBe('action'); + expect(await hitAt(page, beside)).toBe('dialog'); + await driver.tap(message); + await expect.poll(async () => (await readLog(page)).taps).toBe(1); +}); + +test('while a dismissal holds it, its content is inert, focus and keys leave it, and the shield still takes every tap; the release restores both', async ({ + page, +}) => { + const driver = await load(page); + await open(page, 'dialog'); + const action = await centre(page, 'action'); + const message = await centre(page, 'message'); + // The dialog moved focus to its action when it opened. + expect(await page.evaluate(() => document.activeElement?.id ?? null)).toBe( + 'action', + ); + await hold(page, 'dialog', 'layer'); + // A key reaches nothing: the focused action left the inert content. + await page.keyboard.press('Enter'); + await page.keyboard.press('Space'); + await settle(page); + expect((await readLog(page)).clicks).toEqual([]); + // An inert subtree is no hit target: the shield, in the dialog, is. + expect(await hitAt(page, action)).toBe('shield:dialog'); + const state = async () => + page.evaluate(() => { + const button = document.getElementById('action'); + button?.focus(); + return { + inert: button?.closest('[inert]') != null, + focused: document.activeElement === button, + }; + }); + expect(await state()).toEqual({inert: true, focused: false}); + await driver.tap(action); + await driver.tap(message); + await settle(page); + expect(await readLog(page)).toEqual({clicks: [], taps: 0, toggles: []}); + await release(page); + expect(await state()).toEqual({inert: false, focused: true}); + expect(await hitAt(page, action)).toBe('action'); + await driver.tap(message); + await expect.poll(async () => (await readLog(page)).taps).toBe(1); +}); + +test('a light-dismissing popover stays open when a hold takes a tap outside it; after the release the same tap dismisses it', async ({ + page, +}) => { + const driver = await load(page); + await open(page, 'pop'); + const viewport = page.viewportSize(); + if (viewport == null) { + throw new Error('no viewport'); + } + const outside = {x: viewport.width / 2, y: viewport.height - 40}; + expect(await hitAt(page, outside)).toBe('under'); + await hold(page, 'pop', 'viewport'); + expect(await hitAt(page, outside)).toBe('shield:pop'); + await driver.tap(outside); + await settle(page); + expect( + await page.evaluate(() => + document.getElementById('pop')?.matches(':popover-open'), + ), + ).toBe(true); + expect(await readLog(page)).toEqual({ + clicks: [], + taps: 0, + toggles: ['pop:open'], + }); + await release(page); + await driver.tap(outside); + await expect + .poll(async () => (await readLog(page)).toggles) + .toEqual(['pop:open', 'pop:closed']); +}); + +test('a hit test that skips shields sees past a hold: under a held modal dialog’s shield, its own action', async ({ + page, +}) => { + await load(page); + await open(page, 'dialog'); + const action = await centre(page, 'action'); + await hold(page, 'dialog', 'viewport'); + // The opening finger's press resolves its row past the shield. + expect(await hitAt(page, action)).toBe('shield:dialog'); + expect(await hitPast(page, action)).toBe('action'); + await release(page); + expect(await hitAt(page, action)).toBe('action'); +}); + +test('a layer opened above a held one is what that hit test finds, and past the shield the held layer’s content', async ({ + page, +}) => { + await load(page); + await open(page, 'lift'); + const liftAction = await centre(page, 'liftAction'); + await hold(page, 'lift', 'viewport'); + expect(await hitAt(page, liftAction)).toBe('shield:lift'); + expect(await hitPast(page, liftAction)).toBe('liftAction'); + await open(page, 'pop'); + const popAction = await centre(page, 'popAction'); + expect(await hitAt(page, popAction)).toBe('popAction'); + expect(await hitPast(page, popAction)).toBe('popAction'); +}); + +// A full-viewport layer whose root takes no hits itself and lets its tint +// and content take them (ContextMenu's lift layer is pointer-events: none): +// the shield, inside that root, must not inherit that. +for (const rootStyle of ['pointer-events: none', 'visibility: hidden']) { + test(`a layer root with ${rootStyle} (its content opting back in) is held all the same: a tap reaches neither the content nor the page`, async ({ + page, + }) => { + const driver = await load(page); + await page.evaluate(style => { + const lift = document.getElementById('lift'); + if (lift != null) { + lift.style.cssText += `; ${style}`; + } + }, rootStyle); + await open(page, 'lift'); + const action = await centre(page, 'liftAction'); + const tint = {x: 12, y: 12}; + // Unheld, the content and the tint take the hits. + expect(await hitAt(page, action)).toBe('liftAction'); + expect(await hitAt(page, tint)).toBe('tint'); + await hold(page, 'lift', 'layer'); + expect(await hitAt(page, action)).toBe('shield:lift'); + expect(await hitAt(page, tint)).toBe('shield:lift'); + await driver.tap(action); + await driver.tap(tint); + await settle(page); + expect((await readLog(page)).clicks).toEqual([]); + await release(page); + expect(await hitAt(page, action)).toBe('liftAction'); + }); +} + +// Page zoom stays the browser's while a layer is held (spec:AST-094 FR10, +// FR18 and FR35; WCAG 1.4.4): the shield starts no pan, but a pinch on it +// still zooms the page. +for (const [id, scope] of [ + ['pop', 'viewport'], + ['dialog', 'layer'], +] as const) { + test(`while a ${scope === 'viewport' ? 'presentation' : 'dismissal'} holds ${id === 'pop' ? 'a popover' : 'a modal dialog'}, a one-finger pan scrolls nothing and a pinch still zooms the page; neither reaches the layer or the page`, async ({ + page, + browserName, + }) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling and pinch-zoom from injected touches (Mobile Safari on the simulator)', + ); + const driver = await load(page); + const viewport = page.viewportSize(); + if (viewport == null) { + throw new Error('no viewport'); + } + const middle = {x: viewport.width / 2, y: viewport.height / 2}; + const panUp = drag( + {x: middle.x, y: middle.y + 150}, + {x: middle.x, y: middle.y - 150}, + // A rest before the lift: no fling outlives the pan. + {durationMs: 200, restMs: 150}, + ); + // A page that scrolls under the layer: unheld, the same pan scrolls it. + await page.evaluate(() => { + const spacer = document.createElement('div'); + spacer.style.height = '300vh'; + document.body.append(spacer); + }); + await driver.play(panUp); + await expect + .poll(async () => page.evaluate(() => window.scrollY)) + .toBeGreaterThan(0); + await settle(page); + await page.evaluate(() => window.scrollTo(0, 0)); + await settle(page); + expect(await page.evaluate(() => window.scrollY)).toBe(0); + await open(page, id); + await hold(page, id, scope); + expect(await hitAt(page, middle)).toBe(`shield:${id}`); + await driver.play(panUp); + await settle(page); + expect(await page.evaluate(() => window.scrollY)).toBe(0); + await driver.play( + pinch({ + center: middle, + fromDistance: 60, + toDistance: 220, + durationMs: 200, + }), + ); + await expect + .poll(async () => page.evaluate(() => window.visualViewport?.scale ?? 1)) + .toBeGreaterThan(1.2); + expect(await readLog(page)).toEqual({ + clicks: [], + taps: 0, + toggles: id === 'pop' ? ['pop:open'] : [], + }); + }); +} diff --git a/packages/core/src/Layer/__tests__/layerExitCopy.interaction.browser.spec.ts b/packages/core/src/Layer/__tests__/layerExitCopy.interaction.browser.spec.ts new file mode 100644 index 0000000000000..c6af8c473c385 --- /dev/null +++ b/packages/core/src/Layer/__tests__/layerExitCopy.interaction.browser.spec.ts @@ -0,0 +1,189 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file layerExitCopy.interaction.browser.spec.ts + * @input The exit copy bundled with esbuild onto a blank fixture page: a + * popover layer inside a clipping, transformed panel, an entrance curve + * held halfway, and Chromium and WebKit + * @output Proof that the engine shows the copy where the layer was drawn: + * placed as the layer closes (from script and by the browser's own light + * dismiss), it is in the top layer as a manual popover with no `z-index`, + * uncut by the clipping panel, and its drawn box equals the layer's drawn + * box mid-entrance within half a px; a second copy for the same layer + * replaces the first while another layer's copy stays + * @position Browser proof for layerExitCopy.ts, run by `pnpm + * test:interaction`; jsdom has no layout and no top layer. The refusal + * path (Android Chromium 158 refuses a show during another popover's + * toggle) is proven on the Android gate emulator (pull request evidence): + * these engines do not refuse. + */ + +import {fileURLToPath} from 'node:url'; +import {expect, test, type Page} from '@playwright/test'; +import {build} from 'esbuild'; + +declare global { + interface Window { + ExitCopy: typeof import('../layerExitCopy'); + } +} + +let script = ''; + +test.beforeAll(async () => { + const result = await build({ + entryPoints: [ + fileURLToPath(new URL('../layerExitCopy.ts', import.meta.url)), + ], + bundle: true, + write: false, + format: 'iife', + globalName: 'ExitCopy', + target: 'es2022', + }); + script = result.outputFiles[0].text; +}); + +const ATTRIBUTE = 'data-exit-copy'; + +// A real navigation (not setContent): the page keeps its own window. The +// layer sits in a panel that clips and is transformed: a fixed box placed +// there would be cut and moved; the top layer is neither. +async function open(page: Page) { + const html = ` + +
+

One

+
Other
+
+`; + await page.route('http://exit-copy.test/', async route => + route.fulfill({contentType: 'text/html', body: html}), + ); + await page.goto('http://exit-copy.test/'); +} + +test('a copy placed as a layer closes mid-entrance is in the top layer, where the layer was drawn', async ({ + page, +}) => { + await open(page); + const result = await page.evaluate( + async ({attribute}) => { + const layer = document.getElementById('layer') as HTMLElement; + layer.showPopover(); + // An entrance halfway through: drawn at 60 % about its top-left corner. + const entrance = layer.animate( + [ + {scale: '0.2', transformOrigin: '0 0'}, + {scale: '1', transformOrigin: '0 0'}, + ], + {duration: 1000, fill: 'both'}, + ); + entrance.pause(); + entrance.currentTime = 500; + const drawn = layer.getBoundingClientRect(); + let copy: HTMLElement | null = null; + layer.addEventListener( + 'beforetoggle', + event => { + if ((event).newState === 'closed') { + copy = window.ExitCopy.placeLayerExitCopy(layer, attribute); + } + }, + {once: true}, + ); + layer.hidePopover(); + await new Promise(resolve => requestAnimationFrame(resolve)); + const placed = copy as HTMLElement | null; + if (placed == null) { + return null; + } + const rect = placed.getBoundingClientRect(); + return { + drawn: [drawn.left, drawn.top, drawn.width, drawn.height], + copy: [rect.left, rect.top, rect.width, rect.height], + open: placed.matches(':popover-open'), + popover: placed.getAttribute('popover'), + zIndex: getComputedStyle(placed).zIndex, + layerOpen: layer.matches(':popover-open'), + }; + }, + {attribute: ATTRIBUTE}, + ); + expect(result).not.toBeNull(); + expect(result!.layerOpen).toBe(false); + expect(result!.open).toBe(true); + expect(result!.popover).toBe('manual'); + expect(result!.zIndex).toBe('auto'); + // Mid-entrance the drawn box is not the layout box; the copy holds it. + expect(result!.drawn[2]).toBeLessThan(280); + result!.copy.forEach((value, index) => { + expect(Math.abs(value - result!.drawn[index])).toBeLessThanOrEqual(0.5); + }); +}); + +test('the browser\u2019s own light dismiss gets the copy too, uncut by the clipping panel', async ({ + page, +}) => { + await open(page); + await page.evaluate(attribute => { + const layer = document.getElementById('layer') as HTMLElement; + layer.showPopover(); + layer.addEventListener( + 'beforetoggle', + event => { + if ((event).newState === 'closed') { + window.ExitCopy.placeLayerExitCopy(layer, attribute); + } + }, + {once: true}, + ); + }, ATTRIBUTE); + // A press outside: the browser closes the layer by itself. + await page.mouse.click(360, 700); + const result = await page.evaluate(attribute => { + const copy = document.querySelector(`[${attribute}]`); + if (copy == null) { + return null; + } + const rect = copy.getBoundingClientRect(); + return { + open: copy.matches(':popover-open'), + rect: [rect.left, rect.top, rect.width, rect.height], + layerOpen: document.getElementById('layer')!.matches(':popover-open'), + }; + }, ATTRIBUTE); + expect(result).not.toBeNull(); + expect(result!.layerOpen).toBe(false); + expect(result!.open).toBe(true); + // Where the layer stood, at its full size: the 60 x 40 clipping panel and + // its 30 px transform neither cut nor moved it. + expect(result!.rect).toEqual([40, 120, 280, 200]); +}); + +test('a second copy for the same layer replaces the first; another layer\u2019s copy stays', async ({ + page, +}) => { + await open(page); + const result = await page.evaluate(attribute => { + const layer = document.getElementById('layer') as HTMLElement; + const other = document.getElementById('other') as HTMLElement; + layer.showPopover(); + other.showPopover(); + const otherCopy = window.ExitCopy.placeLayerExitCopy(other, attribute); + const first = window.ExitCopy.placeLayerExitCopy(layer, attribute); + const second = window.ExitCopy.placeLayerExitCopy(layer, attribute); + return { + first: first?.isConnected, + second: second?.isConnected && second.matches(':popover-open'), + other: otherCopy?.isConnected && otherCopy.matches(':popover-open'), + copies: document.querySelectorAll(`[${attribute}]`).length, + }; + }, ATTRIBUTE); + expect(result).toEqual({first: false, second: true, other: true, copies: 2}); +}); diff --git a/packages/core/src/Layer/inputHold.test.tsx b/packages/core/src/Layer/inputHold.test.tsx new file mode 100644 index 0000000000000..836f661c103c3 --- /dev/null +++ b/packages/core/src/Layer/inputHold.test.tsx @@ -0,0 +1,231 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +import {render} from '@testing-library/react'; +import {useRef} from 'react'; +import {afterEach, describe, expect, it, vi} from 'vitest'; +import {GestureHost} from '../gestures/GestureHost'; +import { + holdInput, + isInHeldLayer, + isInputShield, + useInputHold, +} from './inputHold'; + +function deferred() { + let resolve: () => void = () => {}; + let reject: (error: unknown) => void = () => {}; + const promise = new Promise((res, rej) => { + resolve = res; + reject = rej; + }); + return {promise, resolve, reject}; +} + +/** A layer root with a child, in the document. */ +function mountLayer(): {layer: HTMLElement; child: HTMLElement} { + const layer = document.createElement('div'); + const child = document.createElement('button'); + layer.append(child); + document.body.append(layer); + return {layer, child}; +} + +const flush = async () => { + await Promise.resolve(); + await Promise.resolve(); +}; + +afterEach(() => { + document.body.replaceChildren(); + vi.restoreAllMocks(); +}); + +describe('holdInput', () => { + it('holds a presenting layer with a shield over the viewport until its end settles', async () => { + const {layer, child} = mountLayer(); + const outside = document.createElement('div'); + document.body.append(outside); + const end = deferred(); + holdInput(layer, {until: end.promise, scope: 'viewport'}); + const shield = layer.lastElementChild as HTMLElement; + expect(shield).not.toBe(child); + expect(shield.getAttribute('popover')).toBe('manual'); + expect(shield.style.position).toBe('fixed'); + // No pan, and a pinch still zooms the page (spec:AST-094 FR10). + expect(shield.style.touchAction).toBe('pinch-zoom'); + // Its own hit-testing, whatever the layer root's (the browser proof holds + // a root that is pointer-events: none). + expect(shield.style.pointerEvents).toBe('auto'); + expect(shield.style.visibility).toBe('visible'); + expect(isInHeldLayer(child)).toBe(true); + expect(isInHeldLayer(shield)).toBe(true); + expect(isInHeldLayer(outside)).toBe(false); + // Keys and assistive technology stay live while presenting. + expect(layer.hasAttribute('inert')).toBe(false); + end.resolve(); + await flush(); + expect(shield.isConnected).toBe(false); + expect(isInHeldLayer(child)).toBe(false); + }); + + it('swallows a press and a click on the shield before they reach the page, and leaves a touch its start so a pinch can still zoom', () => { + const {layer} = mountLayer(); + holdInput(layer, {until: new Promise(() => {}), scope: 'viewport'}); + const shield = layer.lastElementChild as HTMLElement; + const seen: string[] = []; + for (const type of ['pointerdown', 'click', 'touchstart']) { + document.addEventListener(type, () => seen.push(type)); + } + const press = new Event('pointerdown', {bubbles: true, cancelable: true}); + const click = new MouseEvent('click', {bubbles: true, cancelable: true}); + const touch = new Event('touchstart', {bubbles: true, cancelable: true}); + shield.dispatchEvent(press); + shield.dispatchEvent(click); + shield.dispatchEvent(touch); + expect(seen).toEqual([]); + expect(press.defaultPrevented).toBe(true); + expect(click.defaultPrevented).toBe(true); + // Cancelling a touch's start would cancel the page's pinch-zoom too. + expect(touch.defaultPrevented).toBe(false); + }); + + it("holds a dismissing layer: its content inert, the layer itself live under the shield, each element's inert given back", async () => { + const {layer, child} = mountLayer(); + const already = document.createElement('div'); + already.setAttribute('inert', ''); + layer.append(already); + child.focus(); + expect(document.activeElement).toBe(child); + const end = deferred(); + holdInput(layer, {until: end.promise, scope: 'layer'}); + // Focus leaves the content: a focused element that turns inert would + // keep its key activation. + expect(document.activeElement).not.toBe(child); + const shield = layer.lastElementChild as HTMLElement; + expect(shield.getAttribute('popover')).toBe('manual'); + // The layer stays live so the shield inside it catches every hit: an + // inert subtree is no hit target. + expect(layer.hasAttribute('inert')).toBe(false); + expect(shield.hasAttribute('inert')).toBe(false); + expect(child.hasAttribute('inert')).toBe(true); + expect(isInHeldLayer(child)).toBe(true); + end.reject(new Error('interrupted')); + await flush(); + expect(child.hasAttribute('inert')).toBe(false); + expect(already.hasAttribute('inert')).toBe(true); + expect(shield.isConnected).toBe(false); + expect(isInHeldLayer(child)).toBe(false); + }); + + it('names a live hold’s shield, and only it, so a hit test can skip it', () => { + const {layer, child} = mountLayer(); + const popover = document.createElement('div'); + popover.setAttribute('popover', 'manual'); + document.body.append(popover); + const release = holdInput(layer, { + until: new Promise(() => {}), + scope: 'viewport', + }); + const shield = layer.lastElementChild as HTMLElement; + expect(isInputShield(shield)).toBe(true); + for (const other of [layer, child, popover, null]) { + expect(isInputShield(other)).toBe(false); + } + release(); + expect(isInputShield(shield)).toBe(false); + }); + + it('sees slotted content as held: the flat tree, a slotted node to its slot', () => { + const host = document.createElement('div'); + const slotted = document.createElement('button'); + host.append(slotted); + document.body.append(host); + const root = host.attachShadow({mode: 'open'}); + const layer = document.createElement('div'); + layer.append(document.createElement('slot')); + root.append(layer); + expect(isInHeldLayer(slotted)).toBe(false); + const release = holdInput(layer, { + until: new Promise(() => {}), + scope: 'viewport', + }); + expect(isInHeldLayer(slotted)).toBe(true); + release(); + expect(isInHeldLayer(slotted)).toBe(false); + }); + + it('takes an animator handle, ending with its finished', async () => { + const {layer, child} = mountLayer(); + const end = deferred(); + holdInput(layer, {until: {finished: end.promise}, scope: 'layer'}); + expect(isInHeldLayer(child)).toBe(true); + end.resolve(); + await flush(); + expect(isInHeldLayer(child)).toBe(false); + }); + + it('stacks holds on one layer, either scope, under one shield: it comes back when the last one ends', () => { + const {layer, child} = mountLayer(); + const first = holdInput(layer, { + until: new Promise(() => {}), + scope: 'viewport', + }); + const second = holdInput(layer, { + until: new Promise(() => {}), + scope: 'layer', + }); + expect(layer.querySelectorAll('[popover]')).toHaveLength(1); + expect(child.hasAttribute('inert')).toBe(true); + first(); + first(); + expect(isInHeldLayer(child)).toBe(true); + expect(layer.querySelectorAll('[popover]')).toHaveLength(1); + second(); + expect(isInHeldLayer(child)).toBe(false); + expect(child.hasAttribute('inert')).toBe(false); + expect(layer.querySelectorAll('[popover]')).toHaveLength(0); + }); + + it("registers one touch gate per document, which answers whether a touch's target is held", () => { + const addTouchGate = vi.spyOn(GestureHost.prototype, 'addTouchGate'); + const {layer, child} = mountLayer(); + holdInput(layer, {until: new Promise(() => {}), scope: 'layer'}); + const release = holdInput(layer, { + until: new Promise(() => {}), + scope: 'viewport', + }); + expect(addTouchGate.mock.calls.length).toBeLessThanOrEqual(1); + const registered = + addTouchGate.mock.calls[0]?.[0] ?? + ((target: Node) => isInHeldLayer(target)); + expect(registered(child)).toBe(true); + expect(registered(document.body)).toBe(false); + release(); + }); +}); + +describe('useInputHold', () => { + function Bar({isHeld}: {isHeld: boolean}) { + const ref = useRef(null); + useInputHold(ref, {isHeld, scope: 'layer'}); + return ( +
+ +
+ ); + } + + it('holds the layer while asked, and releases it when no longer asked or unmounted', () => { + const view = render(); + const bar = view.getByTestId('bar'); + const back = view.getByRole('button', {name: 'Back', hidden: true}); + expect(back.hasAttribute('inert')).toBe(true); + expect(isInHeldLayer(back)).toBe(true); + view.rerender(); + expect(back.hasAttribute('inert')).toBe(false); + view.rerender(); + expect(back.hasAttribute('inert')).toBe(true); + view.unmount(); + expect(isInHeldLayer(bar)).toBe(false); + }); +}); diff --git a/packages/core/src/Layer/inputHold.ts b/packages/core/src/Layer/inputHold.ts new file mode 100644 index 0000000000000..8dab031277803 --- /dev/null +++ b/packages/core/src/Layer/inputHold.ts @@ -0,0 +1,328 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file inputHold.ts + * @input A layer's root element, what ends the hold (a promise, or an + * animator handle's `finished`), and its scope; the gesture host's touch + * gates + * @output holdInput, useInputHold, isInHeldLayer and + * isInputShield: input held away from a layer while its + * presentation runs + * @position The Layer module's one input hold, for every component + * whose layer presents or dismisses under a finger + * + * UIKit delivers no touch while a presentation runs: until it completes, a + * tap reaches nothing, and a dismissal drops every input until it ends + * (measured on iOS 26.2 with UIAlertController: taps 400 ms into its + * presentation ignored, 450 ms in handled). A hold covers one layer root + * until what ends it settles: + * + * - `scope: 'viewport'` (a presentation): a touch, a pointer press or a + * click anywhere reaches nothing. A transparent shield inside the layer + * covers the viewport from the top layer, so a tap on it lands inside the + * layer (a light-dismissing layer stays open) and starts no pan (iOS + * drops that input too while it presents), while a pinch on it still + * zooms the page, which stays the browser's (spec:AST-094 FR10, FR18, + * FR35; WCAG 1.4.4); the gesture host holds + * the layer's touches away from every recognizer. Keys and assistive + * technology stay live, as on iOS. + * - `scope: 'layer'` (a dismissal, an inert bar): the same shield, and the + * layer's content is inert (no focus, key or assistive-technology + * activation; a focused element inside it is blurred, since one that turns + * inert keeps its key activation). The layer element itself stays live, so + * the shield inside + * it still catches every hit: an inert subtree is no hit target, and a tap + * on it would otherwise land on whatever lies beneath. A modal dialog + * still receives its own `cancel` (Escape): the caller ignores it while + * held. + * + * The shield sets its own `pointer-events` and `visibility`, so a layer root + * that takes no hits itself (its content or a backdrop child opts back in) + * is held all the same. A hit test that must see past held input (a press + * the finger that opened the layer continues, captured before the hold + * began) skips shields and nothing else: the first element of + * `elementsFromPoint` for which `isInputShield` is false, so an + * overlay really above the layer still wins. The shield is a manual + * popover, so its `toggle` events reach a document listener that logs + * every popover's: such a log keeps to its own popovers. + * + * One hold per layer: holds on a held layer stack, and the layer comes back + * when the last one ends (when what ends it settles, resolved or rejected; + * when its release runs; on unmount, for the hook). Nothing is written to + * the DOM but the shield and the content's `inert` attribute; the host + * learns through one touch gate per document that asks `isInHeldLayer`, + * walking the flat tree as it does (a slotted node to its slot, a shadow + * root to its host). + * + * SYNC: When modified, update inputHold.test.tsx, + * __tests__/inputHold.interaction.browser.spec.ts and the components + * that hold their layer (holdInput and useInputHold callers). + */ + +import {useEffect, type RefObject} from 'react'; +import {getGestureHost} from '../gestures/GestureHost'; +import {parentAcrossShadow} from '../gestures/nativeScroll'; + +/** What a hold covers: the whole viewport, or the layer alone. */ +export type InputHoldScope = 'viewport' | 'layer'; + +/** What ends a hold: a promise, or an animator handle (its `finished`). */ +export type InputHoldEnd = + PromiseLike | {readonly finished: PromiseLike}; + +export interface InputHoldOptions { + /** The hold ends when this settles. */ + until: InputHoldEnd; + /** `'viewport'` while presenting, `'layer'` while dismissing. */ + scope: InputHoldScope; +} + +interface HeldLayer { + viewport: number; + layer: number; + shield: HTMLElement | null; + /** The content a dismissal made inert, and whether each was already. */ + inertContent: Map | null; +} + +const held = new WeakMap(); +const gatedDocuments = new WeakSet(); +const liveShields = new WeakSet(); + +/** + * Whether `node` is the shield of a live hold. A hit test that must see past + * held input takes the first element of `elementsFromPoint` that is not one. + */ +export function isInputShield(node: Node | null): boolean { + return node != null && liveShields.has(node); +} + +/** + * Whether `target` is inside a held layer, walking the flat tree as the + * gesture host does (parentAcrossShadow: a slotted node to its slot, a + * shadow root to its host). + */ +export function isInHeldLayer(target: Node | null): boolean { + for ( + let node: Node | null = target; + node != null; + node = parentAcrossShadow(node) + ) { + if (node instanceof Element && held.has(node)) { + return true; + } + } + return false; +} + +/** One gate per document: the host holds a held layer's touches away. */ +function gate(doc: Document): void { + if (gatedDocuments.has(doc)) { + return; + } + gatedDocuments.add(doc); + getGestureHost(doc).addTouchGate(isInHeldLayer); +} + +const SWALLOWED = [ + 'pointerdown', + 'pointerup', + 'touchstart', + 'touchend', + 'mousedown', + 'mouseup', + 'click', + 'contextmenu', +] as const; + +const swallow = (event: Event) => { + // A touch's start stays uncancelled: cancelling it would cancel the page's + // pinch-zoom too. The shield's `touch-action: pinch-zoom` keeps a finger + // from panning, and a tap's mouse events and click land on the shield, + // which swallows them. + if (event.cancelable && event.type !== 'touchstart') { + event.preventDefault(); + } + event.stopPropagation(); +}; + +/** A transparent top-layer cover of the viewport, inside `layer`. */ +function addShield(layer: Element): HTMLElement { + const shield = layer.ownerDocument.createElement('div'); + shield.setAttribute('popover', 'manual'); + shield.setAttribute('aria-hidden', 'true'); + Object.assign(shield.style, { + position: 'fixed', + inset: '0', + width: '100vw', + height: '100vh', + maxWidth: 'none', + maxHeight: 'none', + margin: '0', + padding: '0', + border: '0', + background: 'transparent', + overflow: 'hidden', + touchAction: 'pinch-zoom', + // The two inherited properties that decide hit-testing, set here so the + // shield takes hits whatever its layer root opts out of (a full-viewport + // layer that is `pointer-events: none` and lets a backdrop child take the + // hits): inherited, they would let a held tap fall through to the page. + pointerEvents: 'auto', + visibility: 'visible', + }); + for (const type of SWALLOWED) { + shield.addEventListener(type, swallow, {capture: true, passive: false}); + } + layer.append(shield); + liveShields.add(shield); + if (typeof shield.showPopover === 'function') { + try { + shield.showPopover(); + } catch { + // Not connected, or popovers unsupported: the shield still covers + // what it can as a fixed element. + } + } + return shield; +} + +/** + * Holds input away from `layer` until `until` settles. Returns the release + * (idempotent), for a hold that ends early. + */ +export function holdInput( + layer: Element, + {until, scope}: InputHoldOptions, +): () => void { + gate(layer.ownerDocument); + let state = held.get(layer); + if (state == null) { + state = {viewport: 0, layer: 0, shield: null, inertContent: null}; + held.set(layer, state); + } + if (state.shield == null) { + state.shield = addShield(layer); + } + if (scope === 'viewport') { + state.viewport += 1; + } else { + state.layer += 1; + if (state.layer === 1) { + state.inertContent = makeContentInert(layer, state.shield); + } + } + let isReleased = false; + const release = () => { + if (isReleased) { + return; + } + isReleased = true; + const current = held.get(layer); + if (current == null) { + return; + } + if (scope === 'viewport') { + current.viewport -= 1; + } else { + current.layer -= 1; + if (current.layer === 0 && current.inertContent != null) { + restoreInert(current.inertContent); + current.inertContent = null; + } + } + if (current.viewport === 0 && current.layer === 0) { + if (current.shield != null) { + liveShields.delete(current.shield); + current.shield.remove(); + } + held.delete(layer); + } + }; + const end = 'finished' in until ? until.finished : until; + void Promise.resolve(end).then(release, release); + return release; +} + +/** + * Makes the layer's content inert, the shield aside, and moves focus out of + * it: an element that turns inert while focused keeps its focus, and with it + * its key activation (Enter and Space still click a focused button), until + * the engine's focus fixup runs, if it runs at all (measured in Chromium and + * WebKit). Blurring it leaves focus on the document, as that fixup does. + */ +function makeContentInert( + layer: Element, + shield: Element, +): Map { + const content = new Map(); + for (const child of layer.children) { + if (child !== shield) { + content.set(child, child.hasAttribute('inert')); + child.setAttribute('inert', ''); + } + } + const focused = deepActiveElement(layer.ownerDocument); + if ( + focused instanceof HTMLElement && + [...content.keys()].some(child => isInFlatTree(child, focused)) + ) { + focused.blur(); + } + return content; +} + +/** The focused element, inside any shadow roots on the way down. */ +function deepActiveElement(doc: Document): Element | null { + let active = doc.activeElement; + while (active?.shadowRoot?.activeElement != null) { + active = active.shadowRoot.activeElement; + } + return active; +} + +/** Whether `node` is `root` or inside it, walking the flat tree. */ +function isInFlatTree(root: Node, node: Node): boolean { + for ( + let current: Node | null = node; + current != null; + current = parentAcrossShadow(current) + ) { + if (current === root) { + return true; + } + } + return false; +} + +function restoreInert(content: Map): void { + for (const [element, wasInert] of content) { + if (!wasInert) { + element.removeAttribute('inert'); + } + } +} + +/** + * Holds input away from the layer behind `ref` while `isHeld` is true (an + * inert bar, a presentation driven by state); released when `isHeld` turns + * false or the component unmounts. + */ +export function useInputHold( + ref: RefObject, + {isHeld, scope}: {isHeld: boolean; scope: InputHoldScope}, +): void { + useEffect(() => { + const layer = ref.current; + if (!isHeld || layer == null) { + return; + } + // Released by the cleanup alone. + return holdInput(layer, { + until: new Promise(() => {}), + scope, + }); + }, [ref, isHeld, scope]); +} diff --git a/packages/core/src/Layer/layerExitCopy.test.ts b/packages/core/src/Layer/layerExitCopy.test.ts new file mode 100644 index 0000000000000..ef0306fc46687 --- /dev/null +++ b/packages/core/src/Layer/layerExitCopy.test.ts @@ -0,0 +1,387 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file layerExitCopy.test.ts + * @input vitest (jsdom), layerExitCopy + * @output Checks the static copy a closing layer leaves for its close to + * play on: out of reach and out of the accessibility tree, no ids or form + * names, only in the top layer (hidden until a refused show succeeds, none + * without the popover API), on the layer's layout box with the transform + * it was drawn with, one per layer, scrolled where the layer was + * @position Testing; the browser proofs are in + * __tests__/layerExitCopy.interaction.browser.spec.ts + */ + +import {afterEach, describe, expect, it, vi} from 'vitest'; +import {placeLayerExitCopy} from './layerExitCopy'; + +const BOX = {left: 41, top: 204, width: 320, height: 253}; +const ATTRIBUTE = 'data-test-exit'; + +/** A layer drawn on BOX (jsdom has no layout). */ +function layer(): HTMLElement { + const host = document.createElement('section'); + document.body.appendChild(host); + const element = document.createElement('div'); + element.id = 'layer'; + element.setAttribute('popover', 'auto'); + element.style.setProperty('position-anchor', '--trigger'); + element.style.setProperty('scale', '0.5 0.4'); + element.style.setProperty('opacity', '0.3'); + element.style.setProperty('transition', 'opacity 1s'); + element.innerHTML = + ''; + host.appendChild(element); + element.getBoundingClientRect = () => ({ + ...BOX, + x: BOX.left, + y: BOX.top, + right: BOX.left + BOX.width, + bottom: BOX.top + BOX.height, + toJSON: () => ({}), + }); + // Its layout box, which a fixed layer's offset box reports in viewport px. + for (const [name, value] of [ + ['offsetLeft', BOX.left], + ['offsetTop', BOX.top], + ['offsetWidth', BOX.width], + ['offsetHeight', BOX.height], + ] as const) { + Object.defineProperty(element, name, {configurable: true, value}); + } + return element; +} + +describe('placeLayerExitCopy', () => { + const originalShow = Object.getOwnPropertyDescriptor( + HTMLElement.prototype, + 'showPopover', + ); + /** The popover API, stood in for (jsdom has none), or taken away. */ + const popoverApi = (present: boolean) => { + if (present) { + const show = vi.fn(); + Object.defineProperty(HTMLElement.prototype, 'showPopover', { + configurable: true, + writable: true, + value: show, + }); + return show; + } + Reflect.deleteProperty(HTMLElement.prototype, 'showPopover'); + return null; + }; + afterEach(() => { + if (originalShow == null) { + Reflect.deleteProperty(HTMLElement.prototype, 'showPopover'); + } else { + Object.defineProperty(HTMLElement.prototype, 'showPopover', originalShow); + } + vi.restoreAllMocks(); + document.body.innerHTML = ''; + }); + + const place = (element: HTMLElement) => { + const copy = placeLayerExitCopy(element, ATTRIBUTE); + if (copy == null) { + throw new Error('no copy'); + } + return copy; + }; + + it('puts a static copy right after the layer, out of reach and out of the accessibility tree', () => { + popoverApi(true); + const element = layer(); + const copy = place(element); + expect(copy.previousElementSibling).toBe(element); + expect(copy.hasAttribute(ATTRIBUTE)).toBe(true); + expect(copy.getAttribute('aria-hidden')).toBe('true'); + expect(copy.hasAttribute('inert')).toBe(true); + expect(copy.id).toBe(''); + expect(copy.querySelectorAll('[id]')).toHaveLength(0); + // The content is the layer's, classes and all. + expect(copy.querySelector('.list')?.textContent).toBe('One'); + expect(copy.style.getPropertyValue('pointer-events')).toBe('none'); + }); + + it('takes the copy out of every form and radio group, so the live selection stays', () => { + popoverApi(true); + const element = layer(); + element.insertAdjacentHTML( + 'beforeend', + '', + ); + const live = element.querySelector('input[value="m"]')!; + const copy = place(element); + expect(copy.querySelectorAll('[name]')).toHaveLength(0); + // Every cloned control names a form that cannot exist, so none has an + // owner, whatever the copy stands inside. + expect( + [...copy.querySelectorAll('input')].map(node => + node.getAttribute('form'), + ), + ).toEqual(['', '', '']); + expect(live.checked).toBe(true); + expect(live.getAttribute('name')).toBe('size'); + }); + + it('leaves a form around the layer valid while the copy stands in it', () => { + popoverApi(true); + const element = layer(); + const form = document.createElement('form'); + element.parentElement!.append(form); + form.append(element); + element.insertAdjacentHTML( + 'beforeend', + '', + ); + // The live layer's own fields are the form's while it is open. + expect(form.checkValidity()).toBe(false); + const copy = place(element); + // The copy stands right after the layer, so inside the form; its + // controls belong to no form, and the live fields are unchanged. + expect(copy.closest('form')).toBe(form); + expect([...form.elements]).toHaveLength(3); + for (const node of copy.querySelectorAll('input, select, textarea')) { + expect((node as HTMLInputElement).form).toBeNull(); + } + }); + + it('keeps one copy per layer: a second takes only that layer\u2019s first away', () => { + popoverApi(true); + const element = layer(); + // Another layer beside it, closing too, with its own copy. + const other = layer(); + element.parentElement!.append(other); + const otherCopy = place(other); + const first = place(element); + const second = place(element); + expect(first.isConnected).toBe(false); + expect(second.isConnected).toBe(true); + expect(otherCopy.isConnected).toBe(true); + expect(document.querySelectorAll(`[${ATTRIBUTE}]`)).toHaveLength(2); + }); + + it('stands on the layer\u2019s box, free of the live layer\u2019s anchoring and motion', () => { + popoverApi(true); + const element = layer(); + const copy = place(element); + expect(copy.style.getPropertyValue('position')).toBe('fixed'); + expect(copy.style.getPropertyValue('left')).toBe('41px'); + expect(copy.style.getPropertyValue('top')).toBe('204px'); + expect(copy.style.getPropertyValue('width')).toBe('320px'); + expect(copy.style.getPropertyValue('height')).toBe('253px'); + for (const property of [ + 'position-anchor', + 'scale', + 'opacity', + 'transition', + ]) { + expect(copy.style.getPropertyValue(property), property).toBe(''); + } + expect(copy.style.getPropertyValue('animation-name')).toBe('none'); + // No alignment against the anchor it left (a phone menu's anchor-center). + expect(copy.style.getPropertyValue('justify-self')).toBe('normal'); + expect(copy.style.getPropertyValue('align-self')).toBe('normal'); + }); + + it('holds where a layer still entering was drawn: its layout box, with the transform it was drawn with', () => { + popoverApi(true); + const element = layer(); + // Mid-entrance: drawn at half size about its top-left corner, so its + // drawn rect is not its layout box. + element.getBoundingClientRect = () => ({ + left: 41, + top: 204, + width: 160, + height: 126.5, + x: 41, + y: 204, + right: 201, + bottom: 330.5, + toJSON: () => ({}), + }); + for (const [name, value] of [ + ['offsetLeft', 41], + ['offsetTop', 204], + ['offsetWidth', 320], + ['offsetHeight', 253], + ] as const) { + Object.defineProperty(element, name, {configurable: true, value}); + } + element.getAnimations = () => + [ + {effect: {target: element, pseudoElement: null}}, + ] as unknown as Animation[]; + const presented: Record = { + scale: '0.5', + 'transform-origin': '0px 0px', + width: '320.4px', + height: '253px', + 'box-sizing': 'border-box', + }; + const original = window.getComputedStyle.bind(window); + vi.spyOn(window, 'getComputedStyle').mockImplementation((node, pseudo) => + node === element + ? ({ + getPropertyValue: (property: string) => presented[property] ?? '', + } as CSSStyleDeclaration) + : original(node, pseudo), + ); + const copy = place(element); + // Laid out where the layer is, at its used size, fractions kept... + expect(copy.style.getPropertyValue('left')).toBe('41px'); + expect(copy.style.getPropertyValue('top')).toBe('204px'); + expect(copy.style.getPropertyValue('width')).toBe('320.4px'); + expect(copy.style.getPropertyValue('height')).toBe('253px'); + // ...and drawn as it was drawn. + expect(copy.style.getPropertyValue('scale')).toBe('0.5'); + expect(copy.style.getPropertyValue('transform-origin')).toBe('0px 0px'); + }); + + it('shows the copy only in the top layer, as a manual popover, with no z-index', () => { + const show = popoverApi(true)!; + const element = layer(); + const copy = place(element); + expect(copy.getAttribute('popover')).toBe('manual'); + expect(show).toHaveBeenCalledTimes(1); + expect(show.mock.contexts[0]).toBe(copy); + expect(copy.style.getPropertyValue('z-index')).toBe(''); + expect(copy.style.getPropertyValue('display')).toBe(''); + // The UA's popover inset does not over-constrain the box. + expect(copy.style.getPropertyValue('right')).toBe('auto'); + expect(copy.style.getPropertyValue('bottom')).toBe('auto'); + }); + + it('waits, hidden, when the browser refuses the show during the closing toggle, and shows once it is over', async () => { + const show = popoverApi(true)!; + // Android Chromium 158 refuses a show inside another popover's toggle. + show.mockImplementationOnce(() => { + throw new DOMException( + 'Invalid to show a popover during another show operation', + 'InvalidStateError', + ); + }); + const element = layer(); + let copy: HTMLElement | null = null; + expect(() => { + copy = place(element); + }).not.toThrow(); + // Not shown yet, and nothing stands in for the top layer: the UA keeps a + // popover that is not open out of view. + expect(copy!.isConnected).toBe(true); + expect(copy!.style.getPropertyValue('display')).toBe(''); + expect(copy!.style.getPropertyValue('z-index')).toBe(''); + expect(show).toHaveBeenCalledTimes(1); + // The script that closed the layer is over: shown. + await Promise.resolve(); + expect(show).toHaveBeenCalledTimes(2); + expect(copy!.isConnected).toBe(true); + }); + + it('tries again in the next frame, and takes the copy away if the browser still refuses', async () => { + const show = popoverApi(true)!; + show.mockImplementation(() => { + throw new DOMException('refused', 'InvalidStateError'); + }); + const frames: FrameRequestCallback[] = []; + vi.spyOn(window, 'requestAnimationFrame').mockImplementation(callback => { + frames.push(callback); + return frames.length; + }); + const element = layer(); + const copy = place(element); + await Promise.resolve(); + expect(frames).toHaveLength(1); + frames[0](0); + expect(show).toHaveBeenCalledTimes(3); + expect(copy.isConnected).toBe(false); + }); + + it('does not reach for the top layer for a copy already taken away', async () => { + const show = popoverApi(true)!; + show.mockImplementationOnce(() => { + throw new DOMException('refused', 'InvalidStateError'); + }); + const copy = place(layer()); + copy.remove(); + await Promise.resolve(); + expect(show).toHaveBeenCalledTimes(1); + }); + + it('places no copy without the popover API: the close is immediate', () => { + popoverApi(false); + const element = layer(); + expect(placeLayerExitCopy(element, ATTRIBUTE)).toBeNull(); + expect(document.querySelector(`[${ATTRIBUTE}]`)).toBeNull(); + }); + + it('starts as the layer looks: values its running animations hold are written onto the copy', () => { + popoverApi(true); + const element = layer(); + const dialog = element.querySelector('[role="dialog"]')!; + const list = element.querySelector('.list')!; + // A compositor curve on the layer and one on its dialog; the list has none. + element.getAnimations = () => + [element, dialog].map(target => ({ + effect: {target, pseudoElement: null}, + })) as unknown as Animation[]; + const presented = new Map>([ + [element, {opacity: '0.7', 'transform-origin': '160px 0px'}], + [dialog, {translate: '0px 12px', opacity: '0.5'}], + ]); + const original = window.getComputedStyle.bind(window); + vi.spyOn(window, 'getComputedStyle').mockImplementation((node, pseudo) => { + const values = presented.get(node); + if (values == null) { + return original(node, pseudo); + } + return { + getPropertyValue: (property: string) => values[property] ?? '', + } as CSSStyleDeclaration; + }); + const copy = place(element); + expect(copy.style.getPropertyValue('opacity')).toBe('0.7'); + expect(copy.style.getPropertyValue('transform-origin')).toBe('160px 0px'); + const copyDialog = copy.querySelector('[role="dialog"]')!; + expect(copyDialog.style.getPropertyValue('translate')).toBe('0px 12px'); + expect(copyDialog.style.getPropertyValue('opacity')).toBe('0.5'); + // Nothing is written where nothing runs. + expect( + copy.querySelector('.list')!.getAttribute('style'), + ).toBe(list.getAttribute('style')); + }); + + it.each(['iframe', 'object', 'embed', 'video', 'audio', 'canvas'])( + 'copies a %s as an empty box of its size, so nothing loads or plays again', + tag => { + popoverApi(true); + const element = layer(); + const live = document.createElement(tag); + live.className = 'media'; + live.style.setProperty('margin', '4px'); + Object.defineProperty(live, 'offsetWidth', {value: 320}); + Object.defineProperty(live, 'offsetHeight', {value: 180}); + element.querySelector('.list')!.appendChild(live); + const copy = place(element); + expect(copy.querySelector(tag)).toBeNull(); + const box = copy.querySelector('.list > .media')!; + expect(box.tagName).toBe('SPAN'); + expect(box.childNodes).toHaveLength(0); + expect(box.style.getPropertyValue('width')).toBe('320px'); + expect(box.style.getPropertyValue('height')).toBe('180px'); + expect(box.style.getPropertyValue('margin')).toBe('4px'); + // The live one is untouched. + expect(element.querySelector(tag)).toBe(live); + }, + ); + + it('keeps scrolled content where it was scrolled to', () => { + popoverApi(true); + const element = layer(); + const list = element.querySelector('.list')!; + list.scrollTop = 120; + const copy = place(element); + expect(copy.querySelector('.list')!.scrollTop).toBe(120); + }); +}); diff --git a/packages/core/src/Layer/layerExitCopy.ts b/packages/core/src/Layer/layerExitCopy.ts new file mode 100644 index 0000000000000..024872cedf2ef --- /dev/null +++ b/packages/core/src/Layer/layerExitCopy.ts @@ -0,0 +1,335 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +'use client'; + +/** + * @file layerExitCopy.ts + * @input A layer's popover element as it closes + * @output Exports `placeLayerExitCopy`: a static, inert copy of the layer put + * where the layer is drawn, in the top layer, for a touch presentation's + * close to play on, and `LayerExitBox` + * @position Layer runtime helper for a touch presentation that animates a + * popover out: the one exit copy a layer may have + * (architecture:layer-runtime INV13, architecture:component-style-authoring + * INV7) + * + * The browser hides a popover the moment it closes (a pick, Escape, its own + * light dismiss), so a close that has to stay on screen plays on a copy taken + * as the popover closes: same place, same size, same inherited theme, and the + * same look, values its running animations hold included (a clone keeps only + * inline styles, and a compositor curve holds its value outside them). It is + * laid out on the layer's own box and carries the transform the layer was + * drawn with, so a copy taken mid-entrance holds where it was drawn. The copy + * has no ids, no form names and no form owner (a cloned checked radio would + * otherwise take the live group's selection, and a named field would + * submit), is `inert` and hidden from assistive technology, and takes no + * pointer events. Embedded content (an iframe, an object or embed, media, a + * canvas) stands in the copy as an empty box of its size: a deep copy would + * load or play it again (an iframe navigates, autoplay restarts, a canvas + * comes back blank). A layer has one copy: placing another for the same layer + * takes the previous one away; other layers' copies stay. + * + * It is shown only in the top layer, as a manual popover, so a clipping, + * transformed or sticky ancestor neither cuts nor moves it; a fixed box or a + * larger `z-index` is no substitute (spec:AST-027 FR8). Where the browser + * refuses the show during the closing popover's own toggle (Android Chromium + * 158: "Invalid to show a popover during another show operation"), the copy + * waits, hidden, and shows once the toggle is over: after the script that + * closed the layer or, failing that, in the next frame before it paints. A + * browser without the popover API, or one that still refuses, gets no copy, + * and the close is immediate. Its owner animates it and removes it when its + * spring settles; nothing on the main thread runs per frame while it moves. + * + * SYNC: When modified, update: + * - /packages/core/src/Layer/layerExitCopy.test.ts + * - /packages/core/src/Layer/__tests__/layerExitCopy.interaction.browser.spec.ts + * - /docs/architecture/layer-runtime.md + */ + +/** A box in viewport px. */ +export interface LayerExitBox { + left: number; + top: number; + width: number; + height: number; +} + +/** + * Properties a running animation may hold, which the copy takes from what + * the layer presents. + */ +const PRESENTED_PROPERTIES = [ + 'translate', + 'rotate', + 'scale', + 'transform', + 'transform-origin', + 'opacity', +] as const; + +/** Inline properties the copy must not keep from the live layer. */ +const LIVE_ONLY_PROPERTIES = [ + 'position-anchor', + 'position-area', + 'position-try-fallbacks', + 'inset', + 'right', + 'bottom', + 'scale', + 'opacity', + 'transition', +] as const; + +/** + * Writes onto `copy` what `element` and its subtree present through running + * animations: the copy starts as the layer looks, not as its inline styles + * say. + */ +function commitPresented(element: HTMLElement, copy: HTMLElement): void { + const view = element.ownerDocument.defaultView; + if (view == null || typeof element.getAnimations !== 'function') { + return; + } + const animated = new Set(); + for (const animation of element.getAnimations({subtree: true})) { + const effect = animation.effect as KeyframeEffect | null; + if (effect?.target != null && effect.pseudoElement == null) { + animated.add(effect.target); + } + } + if (animated.size === 0) { + return; + } + const sources = [element, ...element.querySelectorAll('*')]; + const copies = [copy, ...copy.querySelectorAll('*')]; + sources.forEach((source, index) => { + const target = copies[index]; + if (!animated.has(source) || !(target instanceof view.Element)) { + return; + } + const computed = view.getComputedStyle(source); + const {style} = target as HTMLElement; + for (const property of PRESENTED_PROPERTIES) { + style.setProperty(property, computed.getPropertyValue(property)); + } + }); +} + +/** The properties that move a box without moving its layout. */ +const TRANSFORM_PROPERTIES = [ + 'transform', + 'translate', + 'scale', + 'rotate', +] as const; + +/** Each layer's copy: placing another for the same layer replaces it. */ +/** The elements a `
` can own; `form=""` matches no form's id. */ +const FORM_CONTROLS = + 'button, fieldset, input, object, output, select, textarea'; + +const copies = new WeakMap(); + +/** A length in px, or 0. */ +const px = (value: string): number => parseFloat(value) || 0; + +/** + * Where `element` is laid out, in viewport px: its box without the transform + * a presentation in progress draws it with (the copy carries that transform + * itself). A popover's layer is fixed in the top layer, so its offset box is + * in viewport px. + */ +function layoutBox(element: HTMLElement, view: Window): LayerExitBox { + const style = view.getComputedStyle(element); + const isTransformed = TRANSFORM_PROPERTIES.some(property => { + const value = style.getPropertyValue(property); + return value !== '' && value !== 'none'; + }); + if (!isTransformed) { + const rect = element.getBoundingClientRect(); + return { + left: rect.left, + top: rect.top, + width: rect.width, + height: rect.height, + }; + } + // The used width and height keep their fractions, where the offset size + // rounds (a copy narrower by a fraction could wrap its text differently). + const isBorderBox = style.getPropertyValue('box-sizing') === 'border-box'; + const width = px(style.getPropertyValue('width')); + const height = px(style.getPropertyValue('height')); + return { + left: element.offsetLeft, + top: element.offsetTop, + width: + width === 0 + ? element.offsetWidth + : isBorderBox + ? width + : width + + px(style.getPropertyValue('padding-left')) + + px(style.getPropertyValue('padding-right')) + + px(style.getPropertyValue('border-left-width')) + + px(style.getPropertyValue('border-right-width')), + height: + height === 0 + ? element.offsetHeight + : isBorderBox + ? height + : height + + px(style.getPropertyValue('padding-top')) + + px(style.getPropertyValue('padding-bottom')) + + px(style.getPropertyValue('border-top-width')) + + px(style.getPropertyValue('border-bottom-width')), + }; +} + +/** Content a deep copy would load or play again. */ +const EMBEDDED_CONTENT = 'iframe, object, embed, video, audio, canvas'; + +/** + * An empty box standing where embedded content was drawn: the copied node's + * classes and inline style, sized to the live node's box, with nothing in it. + */ +function emptyBox( + copied: HTMLElement, + live: HTMLElement | undefined, + view: Window, +): HTMLElement { + const box = copied.ownerDocument.createElement('span'); + box.className = copied.className; + const inline = copied.getAttribute('style'); + if (inline != null) { + box.setAttribute('style', inline); + } + const display = live == null ? 'inline' : view.getComputedStyle(live).display; + box.style.setProperty( + 'display', + display === 'inline' || display === '' ? 'inline-block' : display, + ); + box.style.setProperty('box-sizing', 'border-box'); + box.style.setProperty('width', `${live?.offsetWidth ?? 0}px`); + box.style.setProperty('height', `${live?.offsetHeight ?? 0}px`); + return box; +} + +/** + * Copies `element` into a static box where it is drawn, marked with + * `attribute`, inserts it right after the element and shows it in the top + * layer. The copy keeps the element's scroll positions and presented look. + * Returns null where there is no top layer to show it in (no popover API): + * the close is then immediate. + */ +export function placeLayerExitCopy( + element: HTMLElement, + attribute: string, +): HTMLElement | null { + const view = element.ownerDocument.defaultView; + if (view == null || typeof element.showPopover !== 'function') { + return null; + } + // One copy per layer: a close during a close replaces it. + copies.get(element)?.remove(); + const box = layoutBox(element, view); + const copy = element.cloneNode(true) as HTMLElement; + copy.removeAttribute('id'); + for (const node of copy.querySelectorAll('[id]')) { + node.removeAttribute('id'); + } + // Out of every form and radio group: a cloned checked radio with the live + // group's name would take its selection, and a named field would submit. + for (const node of copy.querySelectorAll('[name], [form]')) { + node.removeAttribute('name'); + node.removeAttribute('form'); + } + // The copy stands right after the layer, which renders inline by default, + // so a layer inside a `` would leave the copy's controls owned by + // that form while it closes: a cloned empty `required` field makes the + // form invalid and blocks its submit. `form=""` matches no form's id, so + // each cloned control has no owner wherever the copy stands. + for (const node of copy.querySelectorAll(FORM_CONTROLS)) { + node.setAttribute('form', ''); + } + copy.setAttribute(attribute, ''); + copy.setAttribute('aria-hidden', 'true'); + copy.setAttribute('inert', ''); + copy.setAttribute('popover', 'manual'); + const {style} = copy; + for (const property of LIVE_ONLY_PROPERTIES) { + style.removeProperty(property); + } + commitPresented(element, copy); + // On the layer's box: in the top layer the viewport is its containing + // block. No alignment against an anchor or the viewport (a phone menu + // centered on its trigger with `anchor-center` would otherwise shift it), + // and the UA's popover inset would over-constrain the box (and win in RTL). + style.setProperty('position', 'fixed'); + style.setProperty('margin', '0'); + style.setProperty('left', `${box.left}px`); + style.setProperty('top', `${box.top}px`); + style.setProperty('right', 'auto'); + style.setProperty('bottom', 'auto'); + style.setProperty('width', `${box.width}px`); + style.setProperty('height', `${box.height}px`); + style.setProperty('box-sizing', 'border-box'); + style.setProperty('justify-self', 'normal'); + style.setProperty('align-self', 'normal'); + style.setProperty('pointer-events', 'none'); + style.setProperty('animation-name', 'none'); + style.setProperty('overflow', 'visible'); + // Scrolled content stays where it was scrolled to, once the copy is shown + // (a hidden box keeps no scroll position). + const scrolled: [Element, number, number][] = []; + const sources = element.querySelectorAll('*'); + const targets = copy.querySelectorAll('*'); + sources.forEach((source, index) => { + const target = targets[index]; + if (target != null && (source.scrollTop !== 0 || source.scrollLeft !== 0)) { + scrolled.push([target, source.scrollTop, source.scrollLeft]); + } + }); + // Embedded content would load or play again in a deep copy (an iframe + // navigates, autoplay media restarts, a canvas comes back blank): each one + // stands as an empty box of the same size where it was drawn. + const liveEmbeds = element.querySelectorAll(EMBEDDED_CONTENT); + copy + .querySelectorAll(EMBEDDED_CONTENT) + .forEach((node, index) => { + node.replaceWith(emptyBox(node, liveEmbeds[index], view)); + }); + element.after(copy); + copies.set(element, copy); + const show = (): boolean => { + try { + copy.showPopover(); + } catch { + return false; + } + for (const [target, top, left] of scrolled) { + target.scrollTop = top; + target.scrollLeft = left; + } + return true; + }; + const giveUp = () => { + copy.remove(); + if (copies.get(element) === copy) { + copies.delete(element); + } + }; + if (!show()) { + // Refused during the closing popover's toggle: hidden until it is over. + view.queueMicrotask(() => { + if (!copy.isConnected || show()) { + return; + } + view.requestAnimationFrame(() => { + if (copy.isConnected && !show()) { + giveUp(); + } + }); + }); + } + return copy; +} diff --git a/packages/core/src/__tests__/internalModulesNotExperimental.test.ts b/packages/core/src/__tests__/internalModulesNotExperimental.test.ts new file mode 100644 index 0000000000000..85fe3a821d9f4 --- /dev/null +++ b/packages/core/src/__tests__/internalModulesNotExperimental.test.ts @@ -0,0 +1,88 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file internalModulesNotExperimental.test.ts + * @input vitest, the source of core's internal motion, gesture and keyboard + * modules, and of the package-private utility module (touchPresentation) + * @output Fails when one of them declares `@experimental`: those modules are + * internal and never exported (spec:AST-017 FR21, DEC-27), and the tag + * belongs only to a deliberately experimental public API + * @position Testing; a repository guard over src/motion, src/gestures, + * src/keyboard and src/utils/touchPresentation.ts. The rest of src/utils + * backs the public `@astryxdesign/core/utils` entry, where an experimental + * API may carry the tag + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import {describe, expect, it} from 'vitest'; + +const SRC = path.resolve(__dirname, '..'); +const INTERNAL_DIRS = ['motion', 'gestures', 'keyboard']; +/** Package-private files in a directory that also backs a public entry. */ +const INTERNAL_FILES = ['utils/touchPresentation.ts']; + +/** Every source file under `dir`, tests and declarations left out. */ +function sourcesUnder(dir: string): string[] { + const files: string[] = []; + for (const entry of fs.readdirSync(dir, {withFileTypes: true})) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) { + if (entry.name !== '__tests__') { + files.push(...sourcesUnder(full)); + } + } else if ( + /\.(ts|tsx|mjs|js)$/.test(entry.name) && + !/\.(test|spec)\.[cm]?[jt]sx?$/.test(entry.name) && + !entry.name.endsWith('.d.ts') + ) { + files.push(full); + } + } + return files; +} + +/** The internal files the guard reads, relative to src. */ +function checkedFiles(): string[] { + const files: string[] = []; + for (const dir of INTERNAL_DIRS) { + const root = path.join(SRC, dir); + if (fs.existsSync(root)) { + files.push(...sourcesUnder(root)); + } + } + files.push(...INTERNAL_FILES.map(file => path.join(SRC, file))); + return files.map(file => path.relative(SRC, file)); +} + +describe('internal modules', () => { + it('declare no @experimental: only an experimental public API carries it', () => { + const offenders: string[] = []; + for (const file of checkedFiles()) { + fs.readFileSync(path.join(SRC, file), 'utf8') + .split('\n') + .forEach((line, index) => { + if (/@experimental\b/.test(line)) { + offenders.push(`${file}:${index + 1}`); + } + }); + } + expect(offenders).toEqual([]); + }); + + it('checks the directories and files it names', () => { + expect(fs.existsSync(path.join(SRC, 'motion'))).toBe(true); + expect(fs.existsSync(path.join(SRC, 'gestures'))).toBe(true); + expect(sourcesUnder(path.join(SRC, 'motion')).length).toBeGreaterThan(5); + for (const file of INTERNAL_FILES) { + expect(fs.existsSync(path.join(SRC, file)), file).toBe(true); + } + }); + + it('leaves the public utils entry alone: of src/utils, only its package-private files are checked', () => { + const utils = checkedFiles().filter(file => + file.startsWith('utils' + path.sep), + ); + expect(utils).toEqual([path.join('utils', 'touchPresentation.ts')]); + }); +}); diff --git a/packages/core/src/gestures/GestureArena.ts b/packages/core/src/gestures/GestureArena.ts index 3acc3edc83c66..2f35b6826cc1b 100644 --- a/packages/core/src/gestures/GestureArena.ts +++ b/packages/core/src/gestures/GestureArena.ts @@ -58,6 +58,7 @@ import type { TouchSample, } from './types'; import {DEFAULT_ENVIRONMENT} from './types'; +import {internals} from './recognizerInternals'; export interface GestureClock { now(): number; @@ -119,10 +120,15 @@ interface Timer { callback: () => void; } +/** + * The arena that decides which recognizer gets a touch. + */ export class GestureArena { readonly environment: GestureEnvironment; private readonly clock: GestureClock; private readonly options: GestureArenaOptions; + /** The touches whose move the arena is delivering right now. */ + private movingIds: ReadonlySet = new Set(); private readonly records = new Map(); /** Recognizers involved in the current interaction, in resolution order. */ private active: GestureRecognizer[] = []; @@ -134,6 +140,10 @@ export class GestureArena { /** The latest time processed: input never moves the clock backwards. */ private latestTime = Number.NEGATIVE_INFINITY; private depth = 0; + /** Open `deferResolution` calls: what their stops unblock waits. */ + private deferDepth = 0; + /** A deferred stop left a resolution for the next pass. */ + private owesResolution = false; constructor(options: GestureArenaOptions = {}) { this.options = options; @@ -153,11 +163,25 @@ export class GestureArena { return this.records.get(id)?.isClaimed === true; } + /** @internal The view-level consumer of this touch, if any. */ + consumerOf(id: number): TouchConsumer | null { + return this.records.get(id)?.consumer ?? null; + } + /** The live touch with this id. */ getTouch(id: number): GestureTouch | undefined { return this.records.get(id)?.touch; } + /** + * The recognizers that took touch `id` when it began: each was enabled, + * accepted its pointer, and its `shouldReceiveTouch` agreed. Empty for a + * touch the arena does not track. + */ + participantsOf(id: number): ReadonlyArray { + return this.records.get(id)?.participants ?? []; + } + /** Every touch the arena tracks, lifted ones awaiting a decision included. */ get touches(): ReadonlyArray { return [...this.records.values()].map(record => record.touch); @@ -198,14 +222,15 @@ export class GestureArena { target: input.target, scroll: input.scroll ?? null, takenByBrowser: false, + ...(input.isFeedbackOnly === true ? {isFeedbackOnly: true} : {}), }; const participants: GestureRecognizer[] = []; for (const [depth, recognizer] of input.recognizers.entries()) { - if (!recognizer.wantsTouch(touch)) { + if (!internals(recognizer).wantsTouch(touch)) { continue; } participants.push(recognizer); - recognizer.arena = this; + internals(recognizer).arena = this; if (!this.active.includes(recognizer)) { // Resolution follows `active`, so a recognizer joins ahead of any // that sits further out on this touch's path: when two recognize @@ -241,7 +266,7 @@ export class GestureArena { for (const recognizer of [...this.active]) { const touches = newTouches.get(recognizer); if (touches != null) { - recognizer.deliverBegan(touches, context); + internals(recognizer).deliverBegan(touches, context); } } // A recognizer already tracking a gesture hears of a touch that began @@ -253,7 +278,7 @@ export class GestureArena { .map(record => record.touch) .filter(touch => own == null || !own.includes(touch)); if (elsewhere.length > 0 && recognizer.numberOfTouches > 0) { - recognizer.deliverBeganElsewhere(elsewhere, context); + internals(recognizer).deliverBeganElsewhere(elsewhere, context); } } for (const record of created) { @@ -262,6 +287,41 @@ export class GestureArena { }); } + /** + * A contact began where nothing takes part (plain page content). It is + * never tracked and stays the browser's; recognizers already tracking a + * gesture hear of it, as of any touch off their hit path. + */ + untrackedBegan(input: TouchBeganInput): void { + if (!this.active.some(recognizer => recognizer.numberOfTouches > 0)) { + return; + } + this.run(input.time, () => { + const time = this.clampTime(input.time); + const touch: GestureTouch = { + id: input.id, + pointerType: input.pointerType, + phase: 'began', + x: input.x, + y: input.y, + time, + startX: input.x, + startY: input.y, + startTime: time, + samples: [{x: input.x, y: input.y, time}], + target: input.target, + scroll: input.scroll ?? null, + takenByBrowser: false, + }; + const context = this.context(); + for (const recognizer of [...this.active]) { + if (recognizer.numberOfTouches > 0) { + internals(recognizer).deliverBeganElsewhere([touch], context); + } + } + }); + } + /** Contacts moved. */ moved(inputs: ReadonlyArray): void { if (inputs.length === 0) { @@ -273,8 +333,14 @@ export class GestureArena { return; } const context = this.context(); - for (const recognizer of [...this.active]) { - recognizer.deliverMoved(changed, context); + const previous = this.movingIds; + this.movingIds = new Set(changed.map(touch => touch.id)); + try { + for (const recognizer of [...this.active]) { + internals(recognizer).deliverMoved(changed, context); + } + } finally { + this.movingIds = previous; } for (const touch of changed) { const record = this.records.get(touch.id); @@ -298,7 +364,7 @@ export class GestureArena { const changed = this.update(inputs, 'ended'); const context = this.context(); for (const recognizer of [...this.active]) { - recognizer.deliverEnded(changed, context); + internals(recognizer).deliverEnded(changed, context); } // A lift can decide a waiting recognition (its requirement lifted // undecided and failed): settle those first, so each click's answer is @@ -325,7 +391,7 @@ export class GestureArena { } const context = this.context(); for (const recognizer of [...this.active]) { - recognizer.deliverCancelled(touches, context); + internals(recognizer).deliverCancelled(touches, context); } for (const touch of touches) { const record = this.records.get(touch.id); @@ -360,7 +426,7 @@ export class GestureArena { } const context = this.context(); for (const recognizer of [...this.active]) { - recognizer.deliverTakenByBrowser(touches, context); + internals(recognizer).deliverTakenByBrowser(touches, context); } for (const touch of touches) { const record = this.records.get(touch.id); @@ -387,7 +453,10 @@ export class GestureArena { this.fail(recognizer); return; } - if (recognizer.pending != null && (next === 'began' || next === 'ended')) { + if ( + internals(recognizer).pending != null && + (next === 'began' || next === 'ended') + ) { // Already waiting on a requirement; the recognition stands as asked. return; } @@ -421,6 +490,38 @@ export class GestureArena { } } + /** + * Runs `body`, code outside input that stops recognizers (an Effect that + * turns one off or detaches it), without resolving what those stops + * unblock: the recognitions waiting on them, the touches they held from + * a view, the resets. A pass a microtask later resolves all of it, at the + * last event's time, so the listeners and consumers it reaches hear it + * outside the caller's frame and read the recognized state + * (react-component-runtime INV2). + */ + deferResolution(body: () => void): void { + this.deferDepth += 1; + try { + body(); + } finally { + this.deferDepth -= 1; + } + if (this.deferDepth === 0 && this.owesResolution) { + queueMicrotask(() => { + if (this.owesResolution && this.deferDepth === 0) { + this.run(this.lastEventTime(), () => {}); + } + }); + } + } + + /** The last event's time, or now before any event. */ + private lastEventTime(): number { + return Number.isFinite(this.latestTime) + ? this.latestTime + : this.clock.now(); + } + /** Cancel an active recognizer, or fail a possible one. */ stop(recognizer: GestureRecognizer): void { // A cancel from code between events happens at the last event's time: @@ -444,13 +545,19 @@ export class GestureArena { recognizer: GestureRecognizer, next: 'began' | 'ended', ): void { - if (!recognizer.shouldBegin() || this.isVetoedByConsumer(recognizer)) { + if ( + !internals(recognizer).shouldBegin() || + this.isVetoedByConsumer(recognizer) + ) { this.fail(recognizer); return; } let isWaiting = false; for (const other of this.active) { - if (other === recognizer || !recognizer.requiresFailureOf(other)) { + if ( + other === recognizer || + !internals(recognizer).requiresFailureOf(other) + ) { continue; } if (other.state === 'failed') { @@ -466,7 +573,7 @@ export class GestureArena { } } if (isWaiting) { - recognizer.pending = next; + internals(recognizer).pending = next; this.claim(recognizer); return; } @@ -490,14 +597,16 @@ export class GestureArena { continue; } if ( - other.requiresFailureOf(recognizer) || + internals(other).requiresFailureOf(recognizer) || this.conflicts(recognizer, other) ) { this.fail(other); } } this.claim(recognizer); - if (recognizer.cancelsTouchesInView) { + // A watcher takes nothing from the view either: the press and the click + // under it stay (spec:AST-094 FR11, FR19; FR15 exempts watchers). + if (recognizer.claimsTouches && recognizer.cancelsTouchesInView) { this.cancelViewTouchesOf(recognizer); } } @@ -507,7 +616,14 @@ export class GestureArena { winner: GestureRecognizer, other: GestureRecognizer, ): boolean { - if (!sharesTouches(winner, other) || winner.allowsSimultaneous(other)) { + if ( + !sharesTouches(winner, other) || + internals(winner).allowsSimultaneous(other) || + // A watcher (claimsTouches false) prevents nothing and nothing + // prevents it. + !winner.claimsTouches || + !other.claimsTouches + ) { return false; } return winner.canPrevent(other) && other.canBePrevented(winner); @@ -515,7 +631,7 @@ export class GestureArena { /** UIControl's rule: a control may veto an overlapping ancestor recognizer. */ private isVetoedByConsumer(recognizer: GestureRecognizer): boolean { - for (const id of recognizer.seenTouchIds) { + for (const id of internals(recognizer).seenTouchIds) { const consumer = this.records.get(id)?.consumer; if (consumer?.allowsAncestorRecognizer?.(recognizer) === false) { return true; @@ -532,7 +648,7 @@ export class GestureArena { this.apply(recognizer, 'cancelled'); return; } - recognizer.pending = null; + internals(recognizer).pending = null; this.apply(recognizer, 'failed'); this.reviewPending(); } @@ -541,12 +657,19 @@ export class GestureArena { * A recognition that claimed its touches and then fails or is cancelled (a * waiting recognition that fails, a gesture a second finger or the app * cancels) gives them back to the browser unless another recognition owns - * them, so a finger still resting there can pinch-zoom the page. + * them, so a finger still resting there can pinch-zoom the page. A gesture + * that cancels itself on a finger's move (a long press past its tolerance) + * keeps that finger until it lifts: the drag goes on, and the page never + * takes it over midway (`spec:AST-094` FR14), as UIKit's scroll view never + * takes a touch another recognizer had. */ - private releaseClaim(recognizer: GestureRecognizer): void { + private releaseClaim( + recognizer: GestureRecognizer, + keep: ReadonlySet = new Set(), + ): void { for (const touch of recognizer.trackedTouches) { const record = this.records.get(touch.id); - if (record == null || !record.isClaimed) { + if (record == null || !record.isClaimed || keep.has(touch.id)) { continue; } // A recognized discrete gesture (`ended`, a swipe) still owns its @@ -557,7 +680,7 @@ export class GestureArena { other.claimsTouches && (isActive(other.state) || other.state === 'ended' || - other.pending != null) && + internals(other).pending != null) && other.trackedTouches.some(tracked => tracked.id === touch.id), ); if (!isOwned) { @@ -568,7 +691,7 @@ export class GestureArena { private apply(recognizer: GestureRecognizer, next: GestureState): void { const from = recognizer.state; - recognizer.applyState(next); + internals(recognizer).applyState(next); if (recognizer.state === next && from !== next) { this.options.onTransition?.(recognizer, from, next); } else if (next === 'changed' && from === 'changed') { @@ -577,7 +700,9 @@ export class GestureArena { if (isTerminal(next)) { this.clearTimer(recognizer); } - if (next === 'cancelled' || next === 'failed') { + if (next === 'cancelled') { + this.releaseClaim(recognizer, this.movingIds); + } else if (next === 'failed') { this.releaseClaim(recognizer); } } @@ -588,25 +713,29 @@ export class GestureArena { * settle in one pass. */ private reviewPending(): void { + if (this.deferDepth > 0) { + this.owesResolution = true; + return; + } let changed = true; while (changed) { changed = false; for (const recognizer of [...this.active]) { - const next = recognizer.pending; + const next = internals(recognizer).pending; if (next == null || recognizer.state !== 'possible') { continue; } const blocked = this.active.some( other => other !== recognizer && - recognizer.requiresFailureOf(other) && + internals(recognizer).requiresFailureOf(other) && other.state === 'possible' && other.isInvolved(), ); if (blocked) { continue; } - recognizer.pending = null; + internals(recognizer).pending = null; this.attemptRecognition(recognizer, next); changed = true; } @@ -735,7 +864,7 @@ export class GestureArena { recognizer.delaysTouchesBegan && recognizer.state === 'possible' && recognizer.isInvolved() && - recognizer.seenTouchIds.has(record.touch.id), + internals(recognizer).seenTouchIds.has(record.touch.id), ); } @@ -745,12 +874,12 @@ export class GestureArena { recognizer.delaysTouchesEnded && recognizer.state === 'possible' && recognizer.isInvolved() && - recognizer.seenTouchIds.has(record.touch.id), + internals(recognizer).seenTouchIds.has(record.touch.id), ); } private cancelViewTouchesOf(recognizer: GestureRecognizer): void { - for (const id of recognizer.seenTouchIds) { + for (const id of internals(recognizer).seenTouchIds) { const record = this.records.get(id); if (record == null) { continue; @@ -831,13 +960,26 @@ export class GestureArena { private run(time: number, body: () => void): void { this.depth += 1; try { + if (this.depth === 1 && this.deferDepth === 0 && this.owesResolution) { + // A deferred stop resolves before anything newer: input that comes + // ahead of its microtask (a nested dispatch) sees settled state. + this.owesResolution = false; + this.currentTime = this.clampTime(this.lastEventTime()); + this.settle(); + } this.flushTimers(time); this.currentTime = this.clampTime(time); body(); // Re-entrant calls settle with the outermost one: settling mid-delivery - // would reset recognizers that have not heard the event yet. + // would reset recognizers that have not heard the event yet. Inside + // `deferResolution`, the next pass settles. if (this.depth === 1) { - this.settle(); + if (this.deferDepth > 0) { + this.owesResolution = true; + } else { + this.owesResolution = false; + this.settle(); + } } } finally { this.depth -= 1; @@ -897,7 +1039,7 @@ export class GestureArena { } this.active = this.active.filter(other => other !== recognizer); const from = recognizer.state; - recognizer.resetToPossible(); + internals(recognizer).resetToPossible(); didReset = true; if (from !== 'possible') { this.options.onTransition?.(recognizer, from, 'possible'); @@ -957,8 +1099,8 @@ export class GestureArena { } function sharesTouches(a: GestureRecognizer, b: GestureRecognizer): boolean { - for (const id of a.seenTouchIds) { - if (b.seenTouchIds.has(id)) { + for (const id of internals(a).seenTouchIds) { + if (internals(b).seenTouchIds.has(id)) { return true; } } diff --git a/packages/core/src/gestures/GestureHost.ts b/packages/core/src/gestures/GestureHost.ts new file mode 100644 index 0000000000000..ff9ff1975f205 --- /dev/null +++ b/packages/core/src/gestures/GestureHost.ts @@ -0,0 +1,2890 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file GestureHost.ts + * @input DOM touch, pointer, click, contextmenu, scroll and focus events of + * one document; elements with recognizers and consumers attached + * @output Exports GestureHost (the DOM side of the gesture system) and + * getGestureHost (one host per document) + * @position Browser adapter of packages/core/src/gestures: turns real events + * into GestureArena input, and carries out what the arena decides + * + * ONLY WHERE THE PRIMARY POINTER IS COARSE + * + * - Recognizers attach, and the host registers listeners, only while the + * primary pointer is coarse (`spec:AST-094` FR19, FR24; coarsePointer.ts). + * Where it is fine, an attach is recorded but nothing is installed: no + * document, window or element listener, no `touch-action` or other style, + * nothing delivered, so a finger on a touchscreen laptop gets the + * browser's defaults. The host follows the query: turning coarse installs + * it and styles every registration; turning fine cancels every touch in + * flight and removes it all. A development warning names the first attach + * under a fine pointer. + * - The one carve-out is `perEventPointer` (AttachOptions). A per-event + * recognizer gets its listeners and its element's style on any pointer. A + * per-event press tracker, where the pointer is fine, gets only passive + * `touchstart` and `pointerdown` listeners on its own element and no + * style; the host's document and window listeners are added at a finger's + * press on it and removed once that press's contacts end. + * + * INPUT CHANNELS + * + * - Fingers begin and end with Touch Events: they are the only cancelable + * channel (a pointer event's preventDefault cannot stop a scroll or a + * click), and every event lists all the fingers at once. A + * finger's movement comes from BOTH its `touchmove`s and its + * `pointermove`s (paired at touchstart by target and position), the echo + * of one in the other dropped: Chromium sends no `touchmove` until a + * finger has travelled about 16 px, while a tap fails and a pan begins at + * 10. + * - A finger's events go to the element it landed on even after that + * element leaves the document, and then never reach the document. Each + * tracked finger's target therefore gets its own listeners, and a finger + * the events say is no longer down is cancelled, so a re-rendered or + * removed target never strands a gesture. + * - Mice and pens come from Pointer Events, with coalesced samples for + * velocity. A pen on an iPad also produces `stylus` touches; those are left + * to the pointer path. + * - `pointercancel` on a finger is the browser saying it took that finger + * for its own pan or zoom; so is a `touchmove` that can no longer be + * cancelled where `touch-action` let the browser pan, and a scroll above + * the touch. + * - Where the primary pointer is fine the host listens for nothing + * (`spec:AST-094` FR19, FR24) except an action group + * (`AttachOptions.actionGroup`, `family:action-groups` FR5): one passive + * `pointerdown` listener on the group, and document listeners only while + * its press is held, removed at the release. Only the group's own + * recognizers and consumer take that press. + * + * FINGER SCROLLS + * + * - `watchFingerScrolls(scroller)` / `isFingerScroll(scroller)` tell a + * scroll a finger drives (a finger that landed inside the scroller is down + * while it scrolls, then the momentum after the lift) from one the program + * starts (`scrollTo`, a status-bar tap): the finger's lasts until the + * scroller's own `scrollend`, or, where the browser fires none (mobile + * Safari before 26), until its scroll events go quiet. A component that + * animates differently for the two asks here instead of listening to + * touches itself. + * + * WHAT THE BROWSER IS TOLD + * + * - Each element gets the `touch-action` its recognizers leave the browser + * (touchAction.ts), never wider than the element's own value (a caller's + * explicit `touch-action` stays in force), set inline when they attach and + * restored when they detach. + * - While a finger or a pen is down, text selection and the touch callout + * are off only where the hold does something (`spec:AST-094` FR41): on the + * elements of recognizers that bring a `selectionLock`, and on a non-text + * input under the finger inside them (WebKit's own style makes every + * input selectable); restored when the contact ends. Only where the + * primary pointer is coarse: a mouse never takes this lock, nor a finger + * or a pen where the primary pointer is fine. + * - Each element with a recognizer that can claim a touch while it moves (a + * pan, a swipe, a long press that then drags, a pinch, a rotation) gets a + * NON-passive `touchmove` listener that cancels the event only while a + * recognizer owns one of its touches (a pan claims when it begins, at + * UIKit's 10 pt, `spec:AST-094` FR17). Nothing else is ever cancelled, so a + * touch nobody claims scrolls exactly as it would without the system. The + * known cost: a scroll that BEGINS on such an element waits for a busy main + * thread (measured in iOS Safari; a passive listener would hand every + * diagonal to the scroll instead). Elements whose recognizers only claim + * at the lift (taps), recognizers that never claim (`claimsTouches` false) + * and press consumers get no blocking listener, and the listener a + * finger's target gets at touch-down is passive for them too, so a scroll + * starting on them never waits. + * - The browser took a finger on `pointercancel`, on its first uncancelable + * `touchmove` where it could pan, or on a scroll above it once the finger + * has travelled the slop; a scroll under a finger that has not moved is + * the page's own and leaves the touch alone. + * - A lift whose click must not happen cancels its `touchend` (no click, no + * compatibility mouse events) and arms a one-shot swallower for the next + * trusted click; a click held for a recognizer that may still recognize is + * replayed if they all fail. Keyboard and assistive-technology clicks + * (`detail === 0`) always pass. + * - A `contextmenu` while a recognizer owns a touch is cancelled, so a long + * press never also opens the browser's menu. + * - When the app loses focus (a window blur, a hidden page), every touch in + * flight is cancelled, as in UIKit; a finger the browser keeps alive lifts + * on nothing: its `touchend` is cancelled (`spec:AST-094` FR28). + * + * SSR-safe: nothing touches `window` or `document` until something attaches. + * + * SYNC: When modified, update GestureHost.test.ts and README.md. + */ + +import {isRtlElement} from '../hooks/isRtlElement'; +import {UIKIT_CLICK, UIKIT_PRESS} from '../motion/uikit/gestures'; +import { + browserClock, + GestureArena, + type ArenaBeganInput, + type ClickDisposition, + type GestureArenaOptions, + type GestureClock, +} from './GestureArena'; +import type {GestureRecognizer, SelectionLock} from './GestureRecognizer'; +import {HAPTIC_SWITCH_ATTRIBUTE} from './hapticSwitch'; +import {HoverGestureRecognizer} from './HoverGestureRecognizer'; +import {TapGestureRecognizer} from './TapGestureRecognizer'; +import {createNativeScrollProbe, parentAcrossShadow} from './nativeScroll'; +import { + gestureHostStyle, + intersectTouchAction, + type GestureHostStyle, +} from './touchAction'; +import type { + GestureEnvironment, + GestureTouch, + PointerKind, + TouchBeganInput, + TouchConsumer, + TouchInput, + TouchSample, +} from './types'; +import {internals} from './recognizerInternals'; +import {isCoarsePointer, watchCoarsePointer} from '../utils/coarsePointer'; +import {locksSelectionFor} from './selectionLock'; +import type {PanGestureRecognizer} from './PanGestureRecognizer'; +import { + markTouchAction, + TOUCH_ACTION_ATTRIBUTE, + unmarkTouchAction, +} from './touchActionRule'; + +interface Registration { + readonly recognizers: GestureRecognizer[]; + readonly hovers: HoverGestureRecognizer[]; + consumer: TouchConsumer | null; + /** Whether the element carries the host's shared blocking touchmove handler. */ + hasTouchMoveListener: boolean; + /** Inline values the host replaced, to restore on detach. */ + readonly savedStyle: Map; + /** + * What the element's own stylesheet gives it, read in a frame for the + * derived value the host marked then: `null` where the host's rule is in + * force (the author sets nothing else); the host never allows more than + * the author does. `null` before the read. + */ + authorTouchAction: {derived: string; author: string | null} | null; + manageStyle: boolean; + /** An action group (`family:action-groups` FR5). */ + actionGroup: boolean; + /** The group's passive `pointerdown` listener under a fine pointer. */ + groupDownListener: EventListener | null; +} + +export interface GestureHostOptions { + /** The arena's clock; tests pass a manual one. */ + clock?: GestureClock; + /** + * Where fingers come from: Touch Events where the browser has them + * (`auto`), or Pointer Events only. + */ + inputMode?: 'auto' | 'pointer-events'; + /** Every recognizer transition, for debugging and tests. */ + onTransition?: GestureArenaOptions['onTransition']; + /** + * Whether the primary pointer is coarse (a phone, a tablet). Where it is + * fine, the host registers no listener at all (`spec:AST-094` FR19, FR24), + * except an action group's own (`AttachOptions.actionGroup`). + * @default `(pointer: coarse)` on the document's window + */ + isPrimaryPointerCoarse?: () => boolean; +} + +/** + * How far a finger travels before a page-scroll hold judges its pan, in CSS + * px: enough that a swipe's first sample (a 2 px by 2 px wobble) does not fix + * the wrong axis, and still under WebKit's 10–11 px scroll claim, so the + * verdict comes before the browser commits to a scroll. + */ +const PAGE_HOLD_SLOP = 5; + +/** + * Where the browser sends no `scrollend`, how long a finger-driven scroller + * stays quiet (no scroll event) before its scroll counts as ended, in ms: + * longer than the gap between a momentum scroll's events, short enough that + * a program's scroll right after it is not read as the finger's. + */ +const SCROLL_END_FALLBACK_MS = 150; + +/** + * Decides, once per one-finger touch, whether its pan would scroll the page: + * `origin` is the innermost node under the finger (the composed path's first + * node, so a scroller inside a shadow root is seen), `dx` and `dy` the touch's + * movement once it clears a small slop. True cancels the touch's moves for as + * long as it lasts. + */ +export type PageScrollDecision = ( + origin: EventTarget | null, + dx: number, + dy: number, +) => boolean; + +export interface AttachOptions { + /** Let the host set touch-action (and long-press callout) styles. @default true */ + manageStyle?: boolean; + /** + * The recognizer, or the press tracker, reads each event's pointer itself, + * so it attaches even where the primary pointer is fine, with no warning. + * A recognizer gets the host's listeners and its element's style there, as + * `spec:AST-057`'s row swipe does. A press tracker there gets only passive + * `touchstart` and `pointerdown` listeners on its own element, and no + * style; the host's document and window listeners come for a finger's + * press on it and go when the press ends. It paints that press's feedback + * only (an FR19 exception for press feedback only, pending in + * `spec:AST-094`'s amendment): no control event, nothing kept from the + * browser, which still pans, and no mouse or pen press. Everything else + * stays inactive there. @default false + */ + perEventPointer?: boolean; + /** + * The element is an action group, sibling actions a held pointer slides + * across (`family:action-groups` FR5). Where the primary pointer is fine, + * the host still serves it, and nothing else: one passive `pointerdown` + * listener on the element, and document listeners only while a press is + * held (`spec:AST-094` FR19, FR24). @default false + */ + actionGroup?: boolean; +} + +const STYLE_PROPERTIES: Record = { + touchAction: 'touch-action', + WebkitTouchCallout: '-webkit-touch-callout', + WebkitUserSelect: '-webkit-user-select', + userSelect: 'user-select', +}; + +/** + * Run `task` in the next frame, before it paints: a read there styles what + * the frame must style anyway, and its writes are what the frame shows. + */ +function inNextFrame(view: Window | null, task: () => void): void { + if (view != null && typeof view.requestAnimationFrame === 'function') { + view.requestAnimationFrame(task); + } else { + setTimeout(task, 0); + } +} + +/** + * Whether a recognizer can take a touch while it moves, and so needs the + * blocking `touchmove` listener that stops the browser's pan. A tap claims + * only at the lift (it fails once the finger travels), so it needs none. + */ +function claimsWhileMoving(recognizer: GestureRecognizer): boolean { + return ( + recognizer.claimsTouches && + internals(recognizer).acceptsTouchInput && + !(recognizer instanceof TapGestureRecognizer) + ); +} + +const PASSIVE_CAPTURE = {capture: true, passive: true} as const; + +const ACTIVE_CAPTURE = {capture: true, passive: false} as const; + +type StylusTouch = Touch & {touchType?: string}; + +interface PlacedContact { + readonly x: number; + readonly y: number; + readonly target: unknown; +} + +/** A touchstart and a pointerdown describe one finger: same target, same spot. */ +function isSamePlace(a: PlacedContact, b: PlacedContact): boolean { + return ( + a.target === b.target && + Math.abs(a.x - b.x) <= 1 && + Math.abs(a.y - b.y) <= 1 + ); +} + +function isPointerKind(type: string): type is PointerKind { + return type === 'touch' || type === 'mouse' || type === 'pen'; +} + +/** + * What an event, or one of its touches, landed on. Inside an open shadow root + * the document's listeners see the shadow host, in `event.target` and in a + * touch's `target` alike, while the composed path starts at the element + * itself. A touch whose target is not the event's (another finger) keeps its + * own. + */ +function landedOn( + event: Event, + retargeted: EventTarget | null = event.target, +): EventTarget | null { + if (retargeted !== event.target) { + return retargeted; + } + return event.composedPath()[0] ?? retargeted; +} + +/** + * An event's time on the arena's clock (`performance.now()`). Some engines + * and DOM emulators stamp events in epoch milliseconds instead. + */ +function eventTime(event: Event): number { + const time = event.timeStamp; + if ( + time > 1e12 && + typeof performance !== 'undefined' && + performance.timeOrigin > 0 + ) { + return time - performance.timeOrigin; + } + return time; +} + +/** + * A pan, read by its shape so the host never imports the pan's module (a + * page that constructs no pan does not bundle it, nor the gesture table + * groups only the pan reads). + */ +function isPan( + recognizer: GestureRecognizer, +): recognizer is PanGestureRecognizer { + return ( + typeof (recognizer as {maximumNumberOfTouches?: unknown}) + .maximumNumberOfTouches === 'number' + ); +} + +/** A touch list as an array; a synthetic event may carry none. */ +function touchesOf(list: TouchList | null | undefined): Touch[] { + return list == null ? [] : Array.from(list); +} + +export class GestureHost { + readonly arena: GestureArena; + private readonly doc: Document; + private readonly registry = new Map(); + /** Elements whose caller touch-action waits for a read after paint. */ + private readonly pendingCallerReads = new Set(); + private isCallerReadScheduled = false; + /** The selection lock each contact holds, by host contact id. */ + private readonly lockedBy = new Map void>(); + /** Touch.identifier → arena id, for live fingers. */ + private readonly touchIds = new Map(); + /** + * Fingers cancelled because the app lost focus, kept until the browser + * ends them: a browser that keeps such a touch alive must not let its + * lift click (`spec:AST-094` FR28: nothing acts). + */ + private readonly cancelledFingers = new Set(); + /** Touch.identifier → the element the finger landed on. */ + private readonly touchTargets = new Map(); + /** Finger targets with listeners of their own, and how many fingers each. */ + /** + * Finger targets with listeners of their own: how many fingers each, and + * how many of those a recognizer could claim while they move (only those + * need the blocking `touchmove` listener). + */ + private readonly watched = new Map< + EventTarget, + {count: number; blocking: number; claiming: number} + >(); + /** Fingers whose target listener is the blocking one. */ + private readonly blockingFingers = new Set(); + /** + * Fingers a participant could claim (a recognizer that claims touches, or + * a press consumer): only their target's `touchend` may need cancelling, + * so a touch only watchers follow gets a passive one (`spec:AST-094` + * FR20). + */ + private readonly claimingFingers = new Set(); + /** Touch events already handled (the document's and the target's listeners both see one). */ + private readonly processed = new WeakSet(); + /** pointerId → arena id, for live mouse and pen contacts. */ + private readonly pointerIds = new Map(); + /** Fingers whose first uncancelable `touchmove` was already judged. */ + private readonly judgedUncancelable = new Set(); + /** + * A finger's pointerId → its arena id, once its pointerdown and its + * touchstart are paired. Chromium sends no `touchmove` until a finger has + * travelled its slop (about 16 px), but sends every `pointermove`, so the + * finger's movement comes from both channels; and its `pointercancel` is + * the browser's claim of exactly that finger. + */ + private readonly fingerPointers = new Map(); + /** Finger pointerdowns still waiting for their touchstart. */ + private readonly pendingPointers = new Map(); + /** Fingers still waiting for their pointerdown, by arena id. */ + private readonly unpairedTouches = new Map(); + /** The last position fed per contact, so one channel's echo is dropped. */ + private readonly isPrimaryPointerCoarse: () => boolean; + /** Whether the host follows `(pointer: coarse)` (no read was passed). */ + private readonly followsPointer: boolean; + private readonly lastFed = new Map(); + private readonly hovered = new Map>(); + /** When a finger was last down or lifted, and where it landed. */ + private lastTouch: {readonly target: Node; readonly time: number} | null = + null; + /** The last scroll a finger drove, and when. */ + /** Scrollers a recognizer is scrolling itself this frame (ownScroll). */ + private readonly ownedScrolls = new Map(); + /** Numbers each ownScroll call: only the newest one's timer clears it. */ + private ownScrollCalls = 0; + private lastScroll: {readonly scroller: Node; readonly time: number} | null = + null; + /** Page-scroll holds; while any is held, page pans under a finger are judged. */ + private readonly pageHolds = new Set<{readonly decide: PageScrollDecision}>(); + /** Whether the holds' listeners are on: while held and the pointer is coarse. */ + private isHoldListening = false; + private stopWatchingHoldPointer: (() => void) | null = null; + /** The one finger of the gesture the page-scroll holds are judging. */ + private pageHoldTouch: { + readonly id: number; + readonly x: number; + readonly y: number; + verdict: boolean | null; + } | null = null; + /** Predicates that hold a touch's target away from every participant. */ + private readonly touchGates = new Set<(target: Node) => boolean>(); + /** Scrollers someone watches (watchFingerScrolls), and how many watchers. */ + private readonly scrollWatchers = new Map(); + /** + * Watched scrollers whose scroll a finger is driving: a finger that landed + * inside one was down when it scrolled, and its own `scrollend` (the + * momentum after the lift included) has not come yet. + */ + private readonly fingerScrollers = new Set(); + /** + * Finger-driven scrollers whose `scrollend` came while that finger still + * rested on them: the scroll stays the finger's until the lift. + */ + private readonly endedWhileDown = new Set(); + /** + * Where the browser has no `scrollend`: per finger-driven scroller, the + * quiet spell after its last scroll event that ends the scroll instead. + */ + private readonly scrollEndFallbacks = new Map< + Node, + ReturnType + >(); + /** + * Where each finger now on the screen landed, from the last touch event's + * `touches` (every finger, whether or not a recognizer tracks it). + */ + private fingerTargets: ReadonlyArray = []; + private nextId = 1; + private isInstalled = false; + /** + * Whether the primary pointer is coarse (`spec:AST-094` FR19), read when + * something first attaches and kept current while anything is attached. + */ + private isCoarse = false; + private stopWatchingPointer: (() => void) | null = null; + private hasWarnedFinePointer = false; + /** Recognizers attached with `perEventPointer`: active on any pointer. */ + private readonly perEventRecognizers = new Set(); + /** Press consumers attached with `perEventPointer`: active on any pointer. */ + private readonly perEventConsumers = new Set(); + /** + * Where the primary pointer is fine: the elements of enabled + * perEventPointer press trackers, which listen for a finger's press + * themselves, and whether such a press is in progress (the host's document + * and window listeners last only that long). + */ + private readonly armedElements = new Map void>(); + private isArmedPress = false; + /** + * Whether the installed listeners include the cancelable ones (the + * document's touchend, contextmenu and the window's click): only while a + * participant that can claim is attached (`spec:AST-094` FR20). + */ + private isClaimingInstalled = false; + private readonly inputMode: 'auto' | 'pointer-events'; + private usesTouchEvents = true; + private swallowDeadline = Number.NEGATIVE_INFINITY; + + constructor(doc: Document, options: GestureHostOptions = {}) { + this.doc = doc; + this.inputMode = options.inputMode ?? 'auto'; + const environment: GestureEnvironment = { + viewportSize: () => ({ + width: doc.defaultView?.innerWidth ?? 0, + height: doc.defaultView?.innerHeight ?? 0, + }), + // The writing direction through core's one helper. + isRtl: view => + view instanceof Element && isRtlElement(view as HTMLElement), + // A screen-edge pan tells its edge zone from a wider view by its size + // (spec:AST-094 FR18). + viewSize: view => { + if (!(view instanceof Element)) { + return null; + } + const {width, height} = view.getBoundingClientRect(); + return {width, height}; + }, + }; + this.arena = new GestureArena({ + clock: options.clock ?? browserClock, + environment, + onDeferredClick: (touch, allowed) => this.replayClick(touch, allowed), + onTransition: options.onTransition, + }); + const view = doc.defaultView; + // A test may pass its own read; otherwise the one coarse-pointer gate, + // followed as it changes. + this.followsPointer = options.isPrimaryPointerCoarse == null; + this.isPrimaryPointerCoarse = + options.isPrimaryPointerCoarse ?? (() => isCoarsePointer(view)); + } + + // --- Attachment ------------------------------------------------------------ + + /** Attach a recognizer to an element. Returns the detach function. */ + attach( + element: Element, + recognizer: GestureRecognizer, + options: AttachOptions = {}, + ): () => void { + const registration = this.registrationFor(element); + registration.manageStyle = options.manageStyle ?? registration.manageStyle; + registration.actionGroup = options.actionGroup ?? registration.actionGroup; + recognizer.view = element; + if (options.perEventPointer === true) { + this.perEventRecognizers.add(recognizer); + } else if (!this.isCoarse) { + this.warnFinePointer(); + } + if (recognizer instanceof HoverGestureRecognizer) { + registration.hovers.push(recognizer); + } else { + registration.recognizers.push(recognizer); + } + recognizer.onEnabledChange = () => { + this.refresh(element, registration); + this.syncListeners(); + }; + this.refresh(element, registration); + this.syncListeners(); + return () => { + recognizer.onEnabledChange = null; + this.perEventRecognizers.delete(recognizer); + const index = registration.recognizers.indexOf(recognizer); + if (index >= 0) { + registration.recognizers.splice(index, 1); + } + if (recognizer instanceof HoverGestureRecognizer) { + const hoverIndex = registration.hovers.indexOf(recognizer); + if (hoverIndex >= 0) { + registration.hovers.splice(hoverIndex, 1); + } + recognizer.hoverExited(); + } + // A recognizer whose view goes away mid-gesture is cancelled, as in + // UIKit when a view leaves the window. + if (recognizer.state !== 'possible' || recognizer.isInvolved()) { + recognizer.cancel(); + } + this.refresh(element, registration); + this.cleanup(element, registration); + // A registration that stays may have lost its last enabled member. + this.syncListeners(); + }; + } + + /** Attach the view-level touch consumer of an element (a pressable control). */ + attachConsumer( + element: Element, + consumer: TouchConsumer, + options: Pick = {}, + ): () => void { + const registration = this.registrationFor(element); + registration.consumer = consumer; + if (options.perEventPointer === true) { + this.perEventConsumers.add(consumer); + } else if (!this.isCoarse) { + this.warnFinePointer(); + } + this.refresh(element, registration); + this.syncListeners(); + return () => { + this.perEventConsumers.delete(consumer); + if (registration.consumer === consumer) { + registration.consumer = null; + } + // A keepsTouch consumer's blocking touchmove goes with it, even while + // the element keeps its recognizers. + this.refresh(element, registration); + this.cleanup(element, registration); + // A registration that stays may have lost its last enabled member. + this.syncListeners(); + }; + } + + /** + * Hold the page still under a finger until the returned release runs. A + * one-finger pan that `decide` says would scroll the page, and that no + * recognizer owns, is cancelled for the rest of its touch; a second finger + * (pinch-zoom) never is. Holds stack. Only while one is held does the host + * listen with a non-passive `touchmove`, so a scroll lock costs nothing + * once it is released, and a touch scrolls as without the system. + */ + holdPageScroll(decide: PageScrollDecision): () => void { + const hold = {decide}; + this.pageHolds.add(hold); + if (this.pageHolds.size === 1) { + // Only where the primary pointer is coarse (FR19, FR24), following it. + const view = this.doc.defaultView; + this.stopWatchingHoldPointer = watchCoarsePointer(view, isCoarse => + this.listenForHolds(isCoarse), + ); + this.listenForHolds(isCoarsePointer(view)); + } + let isHeld = true; + return () => { + if (!isHeld) { + return; + } + isHeld = false; + this.pageHolds.delete(hold); + if (this.pageHolds.size === 0) { + this.stopWatchingHoldPointer?.(); + this.stopWatchingHoldPointer = null; + this.listenForHolds(false); + } + }; + } + + /** Turn the page holds' document listeners on or off. */ + private listenForHolds(on: boolean): void { + if (on === this.isHoldListening) { + return; + } + this.isHoldListening = on; + if (on) { + this.doc.addEventListener( + 'touchstart', + this.onHoldTouchStart, + PASSIVE_CAPTURE, + ); + this.doc.addEventListener( + 'touchmove', + this.onHoldTouchMove, + ACTIVE_CAPTURE, + ); + } else { + this.doc.removeEventListener( + 'touchstart', + this.onHoldTouchStart, + PASSIVE_CAPTURE, + ); + this.doc.removeEventListener( + 'touchmove', + this.onHoldTouchMove, + ACTIVE_CAPTURE, + ); + this.pageHoldTouch = null; + } + } + + /** + * Re-derive an element's styles and the host's listeners after its + * recognizers' options or its consumer's enabled state changed. + */ + update(element: Element): void { + if (!this.followsPointer) { + // A read passed in (a test's) is not followed: an update reads it again. + this.isCoarse = this.isPrimaryPointerCoarse(); + } + const registration = this.registry.get(element); + if (registration != null) { + this.refresh(element, registration); + } + this.syncListeners(); + } + + /** + * The Pointer Events `pointerId` of a contact the arena tracks: a mouse's + * or a pen's own, and for a finger driven by touch events the pointer its + * touch was paired with (null before the pairing, or without Pointer + * Events). It hands a held finger to code that follows pointers, as a + * menu does when a long press turns into a press-drag selection. + */ + pointerIdOf(touch: GestureTouch | number): number | null { + const id = typeof touch === 'number' ? touch : touch.id; + for (const [pointerId, arenaId] of this.fingerPointers) { + if (arenaId === id) { + return pointerId; + } + } + for (const [pointerId, arenaId] of this.pointerIds) { + if (arenaId === id) { + return pointerId; + } + } + return null; + } + + /** + * Whether a finger set a scroller containing `node` moving within the last + * `window` ms. A touch there now brakes the scroll rather than pressing + * whatever is under it: a native list highlights nothing for one. + */ + isScrollBraking( + node: Node, + window: number = UIKIT_PRESS.scrollBrake.value, + ): boolean { + const last = this.lastScroll; + return ( + last != null && + this.arena.now() - last.time < window && + last.scroller.contains(node) + ); + } + + /** + * Keeps track of whether the scrolls of `scroller` (an element, or the + * document for the page's own scroll) are a finger's, for isFingerScroll, + * until the returned function is called. The host stays installed + * meanwhile. + */ + watchFingerScrolls(scroller: Node): () => void { + this.scrollWatchers.set( + scroller, + (this.scrollWatchers.get(scroller) ?? 0) + 1, + ); + this.watchPointer(); + this.syncListeners(); + let isWatching = true; + return () => { + if (!isWatching) { + return; + } + isWatching = false; + const count = (this.scrollWatchers.get(scroller) ?? 1) - 1; + if (count > 0) { + this.scrollWatchers.set(scroller, count); + } else { + this.scrollWatchers.delete(scroller); + this.fingerScrollers.delete(scroller); + this.endedWhileDown.delete(scroller); + this.clearScrollEndFallback(scroller); + } + this.uninstallIfIdle(); + }; + } + + /** + * Whether the scroll now moving a watched `scroller` is a finger's: a + * finger that landed inside it was down when it scrolled, and the scroll + * has not ended since (its `scrollend` comes after the momentum that + * follows the lift). A scroll the program starts while no finger is down + * there (`scrollTo`, a status-bar tap) is not. The scroller's own + * `scrollend` closes a finger's scroll; where the browser has no + * `scrollend`, a quiet spell of SCROLL_END_FALLBACK_MS after its last + * scroll event does. + */ + isFingerScroll(scroller: Node): boolean { + return this.fingerScrollers.has(scroller); + } + + private registrationFor(element: Element): Registration { + this.watchPointer(); + let registration = this.registry.get(element); + if (registration == null) { + registration = { + recognizers: [], + hovers: [], + consumer: null, + hasTouchMoveListener: false, + savedStyle: new Map(), + authorTouchAction: null, + manageStyle: true, + actionGroup: false, + groupDownListener: null, + }; + this.registry.set(element, registration); + } + return registration; + } + + /** + * Whether `recognizer` takes input now: where the primary pointer is + * coarse, or with `perEventPointer` (FR19; `spec:AST-057`). + */ + private isActive(recognizer: GestureRecognizer): boolean { + return this.isCoarse || this.perEventRecognizers.has(recognizer); + } + + /** + * Whether a press consumer takes input now (FR19; its per-event opt-in, + * an FR19 exception for press feedback pending in AST-094's amendment). + */ + private isConsumerActive(consumer: TouchConsumer): boolean { + return this.isCoarse || this.perEventConsumers.has(consumer); + } + + /** Whether anything attached takes input now. */ + private needsInput(): boolean { + return ( + this.isCoarse || + this.perEventRecognizers.size > 0 || + this.perEventConsumers.size > 0 + ); + } + + private refresh(element: Element, registration: Registration): void { + if (registration.manageStyle && element instanceof HTMLElement) { + this.applyStyle(element, registration); + } + // Where the primary pointer is fine, no touch listener but a + // perEventPointer recognizer's (FR19). + const needsListener = + registration.recognizers.some( + recognizer => + this.isActive(recognizer) && + recognizer.isEnabled && + claimsWhileMoving(recognizer), + ) || + // A kept press holds its touch only where the pointer is coarse; a + // per-event tracker elsewhere gives press feedback only (FR19). + (registration.consumer?.keepsTouch != null && + registration.consumer.isEnabled?.() !== false && + this.isCoarse); + // The blocking listener sits on the element itself (FR19: a scroll that + // starts elsewhere never waits on it), but every element shares the + // host's one handler, which routes through the arena. + if (needsListener && !registration.hasTouchMoveListener) { + element.addEventListener('touchmove', this.onBlockingTouchMove, { + passive: false, + }); + registration.hasTouchMoveListener = true; + } else if (!needsListener && registration.hasTouchMoveListener) { + element.removeEventListener('touchmove', this.onBlockingTouchMove); + registration.hasTouchMoveListener = false; + } + // An action group under a fine pointer: its one idle listener. + const needsGroupListener = + registration.actionGroup && + !this.isPrimaryPointerCoarse() && + (registration.recognizers.some(r => r.isEnabled) || + (registration.consumer != null && + registration.consumer.isEnabled?.() !== false)); + if (needsGroupListener && registration.groupDownListener == null) { + const listener: EventListener = event => + this.beginGroupPress(event as PointerEvent, registration); + registration.groupDownListener = listener; + element.addEventListener('pointerdown', listener, {passive: true}); + } else if (!needsGroupListener && registration.groupDownListener != null) { + element.removeEventListener( + 'pointerdown', + registration.groupDownListener, + ); + registration.groupDownListener = null; + } + } + + private applyStyle(element: HTMLElement, registration: Registration): void { + // Where the primary pointer is fine, the browser's own styles, but a + // perEventPointer recognizer's (FR19). + // A disabled recognizer leaves the browser everything (FR24). + const active = registration.recognizers.filter( + recognizer => recognizer.isEnabled && this.isActive(recognizer), + ); + let style: GestureHostStyle = + active.length > 0 ? gestureHostStyle(active) : {}; + // A control that keeps its touch (a stepper's hold-repeat) takes taps in + // quick succession: where the pointer is coarse, the browser's double-tap + // zoom goes, as for a tap recognizer (`spec:AST-094` FR18); where it is + // fine, the browser keeps its own value (FR19). + if ( + style.touchAction == null && + this.isCoarse && + registration.consumer?.keepsTouch != null + ) { + style = {...style, touchAction: 'manipulation'}; + } + for (const key of Object.keys( + STYLE_PROPERTIES, + ) as (keyof GestureHostStyle)[]) { + const property = STYLE_PROPERTIES[key]; + let value = style[key]; + if (key === 'touchAction') { + value = this.touchActionToWrite(element, registration, value); + } + if (value != null) { + if (!registration.savedStyle.has(property)) { + registration.savedStyle.set( + property, + element.style.getPropertyValue(property), + ); + } + element.style.setProperty(property, value); + } else if (registration.savedStyle.has(property)) { + const saved = registration.savedStyle.get(property) ?? ''; + registration.savedStyle.delete(property); + if (saved === '') { + element.style.removeProperty(property); + } else { + element.style.setProperty(property, saved); + } + } + } + } + + /** + * The touch-action to write inline, if any. The author's value (a + * trigger's own `touch-action: none`) stays in force and the host never + * allows more (spec:AST-094 FR18, DEC-8): + * - an inline value is the style's own, read with no recalc, and the host + * writes what both allow, inline; + * - otherwise the element is marked with the derived value, under a rule + * any author rule outranks, as the recognizer attaches: a subtree that + * mounts with it is styled once, never restyled for it in a later frame. + * The next frame reads what is in force, styles clean, and the host + * writes inline only what both allow where that is narrower than an + * author rule's value. + */ + private touchActionToWrite( + element: HTMLElement, + registration: Registration, + derived: string | undefined, + ): string | undefined { + if (derived == null) { + unmarkTouchAction(element); + registration.authorTouchAction = null; + return undefined; + } + const property = STYLE_PROPERTIES.touchAction; + const inline = registration.savedStyle.has(property) + ? (registration.savedStyle.get(property) ?? '') + : element.style.getPropertyValue(property); + if (inline !== '') { + unmarkTouchAction(element); + return intersectTouchAction(derived, inline); + } + if (!markTouchAction(element, derived)) { + // No document or shadow root to hold the rule (a detached element): + // nothing is styled there, and the next frame looks again. + this.readCallerTouchActionInNextFrame(element); + return undefined; + } + const read = registration.authorTouchAction; + if (read == null || read.derived !== derived) { + // The rule is in force until the read: the derived value, or the + // author's where the author sets one. + this.readCallerTouchActionInNextFrame(element); + return undefined; + } + if (read.author == null) { + return undefined; + } + const both = intersectTouchAction(derived, read.author); + return both === read.author ? undefined : both; + } + + /** + * Read the touch-action in force on each marked element in the next frame, + * then apply its style again. Batched: one style read serves every pending + * element, and the writes follow the reads. The host has written no + * touch-action inline on them, so a value other than the mark is an author + * rule's. + */ + private readCallerTouchActionInNextFrame(element: HTMLElement): void { + this.pendingCallerReads.add(element); + if (this.isCallerReadScheduled) { + return; + } + this.isCallerReadScheduled = true; + inNextFrame(this.doc.defaultView, () => { + this.isCallerReadScheduled = false; + const pending: [HTMLElement, Registration][] = []; + for (const candidate of this.pendingCallerReads) { + const registration = this.registry.get(candidate); + if (registration != null) { + pending.push([candidate, registration]); + } + } + this.pendingCallerReads.clear(); + const view = this.doc.defaultView; + for (const [candidate, registration] of pending) { + const derived = candidate.getAttribute(TOUCH_ACTION_ATTRIBUTE); + if (derived == null || view == null) { + continue; + } + const inForce = view.getComputedStyle(candidate).touchAction || 'auto'; + registration.authorTouchAction = { + derived, + author: inForce === derived ? null : inForce, + }; + } + for (const [candidate, registration] of pending) { + this.applyStyle(candidate, registration); + } + }); + } + + private cleanup(element: Element, registration: Registration): void { + if ( + registration.recognizers.length > 0 || + registration.hovers.length > 0 || + registration.consumer != null + ) { + // What stays may take nothing (only disabled recognizers): FR24. + this.syncListeners(); + return; + } + if (registration.hasTouchMoveListener) { + element.removeEventListener('touchmove', this.onBlockingTouchMove); + } + if (registration.groupDownListener != null) { + element.removeEventListener( + 'pointerdown', + registration.groupDownListener, + ); + registration.groupDownListener = null; + } + this.registry.delete(element); + this.uninstallIfIdle(); + } + + /** + * Listen only while something could take a touch now (`spec:AST-094` + * FR19, FR24): an enabled recognizer or consumer that takes input (where + * the primary pointer is coarse, or one attached with `perEventPointer`), + * or a finger-scroll watch. + * Otherwise no document listener is registered, so a desktop page and a + * disabled control pay nothing. + */ + private syncListeners(): void { + this.syncArmed(); + if (this.shouldListen()) { + this.install(); + this.syncClaimingListeners(); + } else { + this.uninstall(); + } + } + + /** + * Where the primary pointer is fine, an enabled perEventPointer press + * tracker's element listens for a finger's press itself (passive, on the + * element only): the host's document and window listeners are added at + * that press's start and removed at its end, so an idle tracker leaves + * none (spec:AST-094 FR19's exception, for a press's feedback only). + */ + private syncArmed(): void { + const wanted = new Set(); + if (!this.isCoarse) { + for (const [element, registration] of this.registry) { + const consumer = registration.consumer; + if ( + consumer != null && + this.perEventConsumers.has(consumer) && + consumer.isEnabled?.() !== false + ) { + wanted.add(element); + } + } + } + for (const [element, release] of this.armedElements) { + if (!wanted.has(element)) { + release(); + this.armedElements.delete(element); + } + } + for (const element of wanted) { + if (this.armedElements.has(element)) { + continue; + } + element.addEventListener( + 'touchstart', + this.onArmedPress, + PASSIVE_CAPTURE, + ); + element.addEventListener( + 'pointerdown', + this.onArmedPress, + PASSIVE_CAPTURE, + ); + this.armedElements.set(element, () => { + element.removeEventListener( + 'touchstart', + this.onArmedPress, + PASSIVE_CAPTURE, + ); + element.removeEventListener( + 'pointerdown', + this.onArmedPress, + PASSIVE_CAPTURE, + ); + }); + } + if (wanted.size === 0) { + this.isArmedPress = false; + } + } + + /** A finger's press on an armed element: listen for this press. */ + private readonly onArmedPress = (event: Event): void => { + if ( + this.isInstalled || + (event.type === 'pointerdown' && + (event as PointerEvent).pointerType !== 'touch') + ) { + return; + } + this.isArmedPress = true; + this.install(); + // The document's capture phase is over for this event: hand it on. + if (event.type === 'touchstart') { + this.onTouchStart(event as TouchEvent); + } else { + this.onPointerDown(event as PointerEvent); + } + }; + + /** After a contact ends: an armed press that is over stops listening. */ + private releaseArmedPress(): void { + if (!this.isArmedPress) { + return; + } + queueMicrotask(() => { + if (this.isArmedPress && this.arena.touches.length === 0) { + this.isArmedPress = false; + this.syncListeners(); + } + }); + } + + /** With nothing attached, stop listening and stop following the pointer. */ + private uninstallIfIdle(): void { + this.syncListeners(); + if (this.registry.size === 0 && this.scrollWatchers.size === 0) { + this.uninstall(); + this.stopWatchingPointer?.(); + this.stopWatchingPointer = null; + } + } + + /** Read the primary pointer, and follow it while anything is attached. */ + private watchPointer(): void { + if (this.stopWatchingPointer != null) { + return; + } + const view = this.doc.defaultView; + this.isCoarse = this.isPrimaryPointerCoarse(); + this.stopWatchingPointer = this.followsPointer + ? watchCoarsePointer(view, isCoarse => this.pointerChanged(isCoarse)) + : () => {}; + } + + /** + * The primary pointer turned coarse or fine (a tablet docking a mouse): + * install or remove everything, as if each attach happened now. + */ + private pointerChanged(isCoarse: boolean): void { + if (isCoarse === this.isCoarse) { + return; + } + this.isCoarse = isCoarse; + if (!isCoarse) { + this.cancelEverything(this.arena.now()); + } + for (const [element, registration] of this.registry) { + this.refresh(element, registration); + } + this.syncListeners(); + } + + /** Something attached where the primary pointer is fine: say so once. */ + private warnFinePointer(): void { + if (process.env.NODE_ENV !== 'production' && !this.hasWarnedFinePointer) { + this.hasWarnedFinePointer = true; + console.warn( + 'Astryx: a gesture recognizer or press tracker was attached where ' + + 'the primary pointer is fine. It stays inactive until the pointer ' + + 'is coarse (spec:AST-094 FR19), so attach touch work only under a ' + + 'touch presentation, or pass perEventPointer for a recognizer or ' + + "press tracker that reads each event's pointer itself (spec:AST-057).", + ); + } + } + + /** + * Whether an enabled participant could claim a touch or cancel a lift: a + * consumer, or a recognizer that claims touches. With only watchers + * attached, the host's listeners are all passive (`spec:AST-094` FR20). + */ + private hasClaimer(): boolean { + for (const registration of this.registry.values()) { + const consumer = registration.consumer; + if ( + (consumer != null && + consumer.isEnabled?.() !== false && + this.isConsumerActive(consumer)) || + registration.recognizers.some( + recognizer => + recognizer.isEnabled && + recognizer.claimsTouches && + this.isActive(recognizer), + ) + ) { + return true; + } + } + return false; + } + + /** + * Keeps the document's touchend cancelable, and the contextmenu and click + * listeners on, only while a claimer is attached; with only watchers the + * touchend is passive and the other two are off. + */ + private syncClaimingListeners(): void { + if (!this.isInstalled) { + return; + } + const isClaiming = this.hasClaimer(); + if (isClaiming === this.isClaimingInstalled) { + return; + } + const doc = this.doc; + const view = doc.defaultView; + if (this.inputMode === 'auto') { + doc.removeEventListener('touchend', this.onTouchEnd, true); + doc.addEventListener( + 'touchend', + this.onTouchEnd, + isClaiming ? ACTIVE_CAPTURE : PASSIVE_CAPTURE, + ); + } + if (isClaiming) { + doc.addEventListener('contextmenu', this.onContextMenu, ACTIVE_CAPTURE); + view?.addEventListener('click', this.onClick, ACTIVE_CAPTURE); + } else { + doc.removeEventListener( + 'contextmenu', + this.onContextMenu, + ACTIVE_CAPTURE, + ); + view?.removeEventListener('click', this.onClick, ACTIVE_CAPTURE); + } + this.isClaimingInstalled = isClaiming; + } + + private shouldListen(): boolean { + if (!this.needsInput()) { + return false; + } + // A finger-scroll watch listens only where the pointer is coarse: beside + // a per-event participant under a fine one it would keep every page + // listener installed for its whole life (FR24). + if (this.isCoarse && this.scrollWatchers.size > 0) { + return true; + } + let isArmed = false; + for (const registration of this.registry.values()) { + const consumer = registration.consumer; + if ( + consumer != null && + consumer.isEnabled?.() !== false && + this.isConsumerActive(consumer) + ) { + if (this.isCoarse) { + return true; + } + // A perEventPointer press tracker where the pointer is fine: only + // while a press it armed is in progress (syncArmed). + isArmed = true; + } + if ( + registration.recognizers.some( + recognizer => recognizer.isEnabled && this.isActive(recognizer), + ) || + (this.isCoarse && registration.hovers.some(r => r.isEnabled)) + ) { + return true; + } + } + return isArmed && this.isArmedPress; + } + + // --- Listeners ------------------------------------------------------------- + + private install(): void { + this.watchPointer(); + if (!this.needsInput()) { + return; + } + if (this.isInstalled) { + return; + } + const view = this.doc.defaultView; + this.isInstalled = true; + this.usesTouchEvents = + this.inputMode === 'auto' && view != null && 'ontouchstart' in view; + const doc = this.doc; + // Pointer Events only: a finger comes from its pointer events alone. A + // browser that has Touch Events still sends them for the same finger, and + // reading both would track it twice (a long press needs one touch). + if (this.inputMode === 'auto') { + doc.addEventListener('touchstart', this.onTouchStart, PASSIVE_CAPTURE); + doc.addEventListener('touchmove', this.onTouchMove, PASSIVE_CAPTURE); + // Passive until a claimer is attached (syncClaimingListeners). + doc.addEventListener('touchend', this.onTouchEnd, PASSIVE_CAPTURE); + doc.addEventListener('touchcancel', this.onTouchCancel, PASSIVE_CAPTURE); + } + doc.addEventListener('pointerdown', this.onPointerDown, PASSIVE_CAPTURE); + doc.addEventListener('pointermove', this.onPointerMove, PASSIVE_CAPTURE); + doc.addEventListener('pointerup', this.onPointerUp, PASSIVE_CAPTURE); + doc.addEventListener( + 'pointercancel', + this.onPointerCancel, + PASSIVE_CAPTURE, + ); + doc.addEventListener('pointerout', this.onPointerOut, PASSIVE_CAPTURE); + doc.addEventListener('scroll', this.onScroll, PASSIVE_CAPTURE); + doc.addEventListener('scrollend', this.onScrollEnd, PASSIVE_CAPTURE); + doc.addEventListener('visibilitychange', this.onVisibilityChange); + view?.addEventListener('blur', this.onBlur); + } + + private uninstall(): void { + if (!this.isInstalled) { + return; + } + this.isInstalled = false; + // Unheard from here on: no fingers to keep. + this.fingerTargets = []; + this.isClaimingInstalled = false; + const doc = this.doc; + const view = doc.defaultView; + doc.removeEventListener('touchstart', this.onTouchStart, PASSIVE_CAPTURE); + doc.removeEventListener('touchmove', this.onTouchMove, PASSIVE_CAPTURE); + doc.removeEventListener('touchend', this.onTouchEnd, true); + doc.removeEventListener('touchcancel', this.onTouchCancel, PASSIVE_CAPTURE); + doc.removeEventListener('pointerdown', this.onPointerDown, PASSIVE_CAPTURE); + doc.removeEventListener('pointermove', this.onPointerMove, PASSIVE_CAPTURE); + doc.removeEventListener('pointerup', this.onPointerUp, PASSIVE_CAPTURE); + doc.removeEventListener( + 'pointercancel', + this.onPointerCancel, + PASSIVE_CAPTURE, + ); + doc.removeEventListener('pointerout', this.onPointerOut, PASSIVE_CAPTURE); + doc.removeEventListener('contextmenu', this.onContextMenu, ACTIVE_CAPTURE); + doc.removeEventListener('scroll', this.onScroll, PASSIVE_CAPTURE); + doc.removeEventListener('scrollend', this.onScrollEnd, PASSIVE_CAPTURE); + doc.removeEventListener('visibilitychange', this.onVisibilityChange); + view?.removeEventListener('click', this.onClick, ACTIVE_CAPTURE); + view?.removeEventListener('blur', this.onBlur); + this.cancelEverything(this.arena.now()); + for (const scroller of [...this.scrollEndFallbacks.keys()]) { + this.clearScrollEndFallback(scroller); + } + // Its scroll and scrollend go with the listeners: a finger scroll still + // running would never end, and later program scrolls would read as the + // finger's. + this.fingerScrollers.clear(); + this.endedWhileDown.clear(); + this.lastTouch = null; + this.lastScroll = null; + } + + // --- Hit path -------------------------------------------------------------- + + /** + * The recognizers on the path from `target` to the document, innermost + * first, and (for a contact) its consumer: the innermost element with an + * attached consumer. + */ + private participantsAt( + target: unknown, + pointerType: PointerKind | null, + ): { + recognizers: GestureRecognizer[]; + consumer: TouchConsumer | null; + consumerElement: Element | null; + hovers: HoverGestureRecognizer[]; + } { + const recognizers: GestureRecognizer[] = []; + const hovers: HoverGestureRecognizer[] = []; + let consumer: TouchConsumer | null = null; + let consumerElement: Element | null = null; + if ( + target instanceof Node && + [...this.touchGates].some(gate => gate(target)) + ) { + return {recognizers, consumer, consumerElement, hovers}; + } + let hasConsumerOwner = pointerType == null; + for ( + let node: Node | null = target instanceof Node ? target : null; + node != null; + node = parentAcrossShadow(node) + ) { + if (!(node instanceof Element)) { + continue; + } + const registration = this.registry.get(node); + if (registration != null) { + // Where the primary pointer is fine, only perEventPointer + // recognizers take part (FR19; `spec:AST-057`). + recognizers.push( + ...registration.recognizers.filter(recognizer => + this.isActive(recognizer), + ), + ); + if (this.isCoarse) { + hovers.push(...registration.hovers); + } + } + if (hasConsumerOwner || pointerType == null) { + continue; + } + // Where the primary pointer is fine, only a perEventPointer consumer + // takes part (FR19); an inactive one is as if absent. + if ( + registration?.consumer != null && + this.isConsumerActive(registration.consumer) && + // Where the pointer is fine its press feedback is for a finger only: + // a mouse or a pen there keeps the desktop's own handling (FR19). + (this.isCoarse || pointerType === 'touch') + ) { + consumer = registration.consumer; + consumerElement = node; + hasConsumerOwner = true; + continue; + } + } + return {recognizers, consumer, consumerElement, hovers}; + } + + /** + * Turn text selection and the touch callout off, for as long as this + * finger or pen is down, only on the elements whose hold does something: + * a long press's (a menu, a hold-repeat), those of recognizers created + * with a `selectionLock` (a slider's thumb, a switch's drag, a tab bar's + * zero-delay lens) and that of a press tracker created with one (a + * stepper's hold-repeat). Anything whose hold does nothing, a plain + * press-tracked control or row included, keeps the browser's long-press + * selection (`spec:AST-094` FR41). Restored when the contact ends. Callers + * lock only where the primary pointer is coarse (locksSelectionFor), so + * desktop selection is untouched, and only once the arena has admitted + * the contact: `recognizers` and `consumer` are the ones that took it. + */ + private lockSelection( + id: number, + recognizers: ReadonlyArray, + target: EventTarget | null, + consumer: TouchConsumer | null, + consumerElement: Element | null, + ): void { + // Each recognizer whose hold does something brings its lock; the host + // never imports one (a component without such a hold does not bundle + // it). Only one that took this contact locks for it: a recognizer that + // declined it (its pointer, or its own shouldReceiveTouch) has no hold + // here. + const byLock = new Map>(); + for (const recognizer of recognizers) { + const lock = recognizer.selectionLock; + if (lock != null && recognizer.view instanceof Element) { + const elements = byLock.get(lock) ?? new Set(); + elements.add(recognizer.view); + byLock.set(lock, elements); + } + } + const consumerLock = consumer?.selectionLock; + if (consumerLock != null && consumerElement != null) { + const elements = byLock.get(consumerLock) ?? new Set(); + elements.add(consumerElement); + byLock.set(consumerLock, elements); + } + if (byLock.size > 0) { + const undo = [...byLock].map(([lock, elements]) => + lock(elements, target), + ); + this.lockedBy.set(id, () => { + for (const release of undo) { + release(); + } + }); + } + } + + /** Undo `lockSelection` for a contact that ended; idempotent. */ + private unlockSelection(id: number): void { + this.lockedBy.get(id)?.(); + this.lockedBy.delete(id); + } + + // --- Touch events ---------------------------------------------------------- + + /** + * Hold touches away from every participant while `gate(target)` answers + * true (a presentation running in a layer: its content takes no touch + * until it settles). Returns the release. + */ + addTouchGate(gate: (target: Node) => boolean): () => void { + this.touchGates.add(gate); + return () => { + this.touchGates.delete(gate); + }; + } + + /** + * How many fingers are on the screen: the length of `touches` in the last + * touch event the host's listeners heard (`touchstart`, `touchmove`, + * `touchend`, `touchcancel`), whatever its target, so a finger on an + * element with no recognizer counts too. 0 before any, and while no + * listener is installed. A recognizer that must not begin under a second + * finger reads it in `shouldBegin`. + */ + fingersDown(): number { + return this.fingerTargets.length; + } + + /** Whether this event is new to the host (and mark it handled). */ + private takeEvent(event: Event): boolean { + if (this.processed.has(event)) { + return false; + } + this.processed.add(event); + return true; + } + + private readonly onTouchStart = (event: TouchEvent): void => { + this.noteFingersDown(event); + // Where the host reads a finger from its pointer events (no ontouchstart, + // as after touch emulation turns on), its Touch Events would track it twice. + if (!this.usesTouchEvents || !this.takeEvent(event)) { + return; + } + if (this.fingerTargets.length > 1) { + this.failUnclaimedLoneFingerPans(); + } + const time = eventTime(event); + const inputs: ArenaBeganInput[] = []; + const untracked: TouchBeganInput[] = []; + const stale: number[] = []; + // Locked once the arena knows which recognizers took each finger. + const toLock: { + id: number; + target: EventTarget | null; + consumerElement: Element | null; + }[] = []; + for (const touch of touchesOf(event.changedTouches)) { + if ((touch as StylusTouch).touchType === 'stylus') { + continue; + } + // A finger number still tracked here began again: the touch it named + // ended where no listener of ours could hear it. + this.cancelledFingers.delete(touch.identifier); + const previous = this.forgetFinger(touch.identifier); + if (previous != null) { + stale.push(previous); + } + const target = + touch.target instanceof Node + ? landedOn(event, touch.target) + : landedOn(event); + const {recognizers, consumer, consumerElement} = this.participantsAt( + target, + 'touch', + ); + if (recognizers.length === 0 && consumer == null) { + // Nothing here takes part: the touch stays the page's, but a + // recognizer tracking another finger hears it began (a second + // finger beside an open menu ends its press, spec:AST-094 FR28). + untracked.push({ + id: this.nextId++, + x: touch.clientX, + y: touch.clientY, + time, + pointerType: 'touch', + target, + }); + continue; + } + const id = this.nextId++; + this.touchIds.set(touch.identifier, id); + if (locksSelectionFor('touch', this.doc.defaultView)) { + toLock.push({id, target, consumerElement}); + } + this.watchTarget( + touch.identifier, + target, + recognizers.some(claimsWhileMoving) || + (this.isCoarse && consumer?.keepsTouch != null), + consumer != null || + recognizers.some(recognizer => recognizer.claimsTouches), + ); + this.lastFed.set(id, {x: touch.clientX, y: touch.clientY, time}); + this.pairTouch(id, {x: touch.clientX, y: touch.clientY, target}); + inputs.push({ + id, + x: touch.clientX, + y: touch.clientY, + time, + pointerType: 'touch', + target, + scroll: createNativeScrollProbe(target), + recognizers, + consumer, + // Where the pointer is fine only per-event participants hear it, and + // a press tracker among them gives press feedback only (FR19). + ...(this.isCoarse ? {} : {isFeedbackOnly: true}), + }); + } + if (stale.length > 0) { + this.arena.cancelled(stale, time); + } + this.arena.began(inputs); + for (const {id, target, consumerElement} of toLock) { + this.lockSelection( + id, + this.arena.participantsOf(id), + target, + this.arena.consumerOf(id) ?? null, + consumerElement, + ); + } + for (const input of untracked) { + this.arena.untrackedBegan(input); + } + // Stamped after the consumers judged the touch: the brake asks whether + // an EARLIER finger is still driving a scroll. + const landed = landedOn(event); + if (landed instanceof Node) { + this.lastTouch = {target: landed, time}; + } + }; + + private readonly onTouchMove = (event: TouchEvent): void => { + this.noteFingersDown(event); + // Where the host reads a finger from its pointer events (no ontouchstart, + // as after touch emulation turns on), its Touch Events would track it twice. + if (!this.usesTouchEvents || !this.takeEvent(event)) { + return; + } + this.dropFingersNotDown(event); + const time = eventTime(event); + const ids: number[] = []; + const inputs: TouchInput[] = []; + for (const touch of touchesOf(event.changedTouches)) { + const id = this.touchIds.get(touch.identifier); + if (id == null) { + continue; + } + ids.push(id); + inputs.push({id, x: touch.clientX, y: touch.clientY, time}); + } + if (inputs.length === 0) { + return; + } + if (!event.cancelable) { + // The browser no longer asks the page about these fingers. WebKit + // stops asking when it takes the touch to pan (its pointercancel comes + // with it); Chromium also stops after one unconsumed touchmove, even + // when `touch-action` keeps it from panning the way the finger went (a + // horizontal drag under pan-y). So a finger's FIRST uncancelable move + // counts as the browser's claim only if the browser could pan along + // the finger's main axis since touch-down; later ones add nothing, and + // pointercancel or a scroll still report any real pan. + const placed = new Map(inputs.map(input => [input.id, input])); + const taken = ids.filter(id => { + const touch = this.arena.getTouch(id); + if ( + touch == null || + touch.takenByBrowser || + this.arena.isClaimed(id) || + touch.scroll?.allowsNativePan === false || + this.judgedUncancelable.has(id) + ) { + return false; + } + this.judgedUncancelable.add(id); + const input = placed.get(id); + if (input == null || touch.scroll == null) { + return true; + } + const dx = input.x - touch.startX; + const dy = input.y - touch.startY; + return Math.abs(dx) >= Math.abs(dy) + ? touch.scroll.canScroll(dx, 0) + : touch.scroll.canScroll(0, dy); + }); + if (taken.length > 0) { + this.arena.browserTookOver(taken, time); + } + } + this.feedMoves(inputs); + }; + + private readonly onHoldTouchStart = (event: TouchEvent): void => { + const fingers = touchesOf(event.touches); + const current = this.pageHoldTouch; + // Only a finger already down keeps its record: a new touch that reuses + // the identifier starts afresh. + if ( + current != null && + fingers.some(finger => finger.identifier === current.id) && + !touchesOf(event.changedTouches).some( + finger => finger.identifier === current.id, + ) + ) { + // A second finger joins the held one: nothing is cancelled while two + // are down (a pinch), and the first keeps its verdict for the rest of + // its touch once it is alone again. + return; + } + const finger = fingers[0]; + this.pageHoldTouch = + fingers.length === 1 && finger != null + ? { + id: finger.identifier, + x: finger.clientX, + y: finger.clientY, + verdict: null, + } + : null; + }; + + private readonly onHoldTouchMove = (event: TouchEvent): void => { + const fingers = touchesOf(event.touches); + const current = this.pageHoldTouch; + const finger = fingers[0]; + // A second finger is a pinch: zooming the page stays available. + if ( + fingers.length !== 1 || + finger == null || + current == null || + finger.identifier !== current.id + ) { + return; + } + // A recognizer that owns the touch decides about its moves itself. + const id = this.touchIds.get(current.id); + if (id != null && this.arena.isClaimed(id)) { + return; + } + if (current.verdict == null) { + const dx = finger.clientX - current.x; + const dy = finger.clientY - current.y; + if (Math.max(Math.abs(dx), Math.abs(dy)) < PAGE_HOLD_SLOP) { + return; + } + // The browser picks the scroller once per gesture; so do the holds. + const origin = event.composedPath()[0] ?? event.target; + current.verdict = [...this.pageHolds].some(hold => + hold.decide(origin, dx, dy), + ); + } + if (current.verdict && event.cancelable) { + event.preventDefault(); + } + }; + + /** The one blocking `touchmove` handler every claiming element shares. */ + private readonly onBlockingTouchMove = (event: Event): void => { + this.cancelIfClaimed(event as TouchEvent); + }; + + /** + * Cancel a `touchmove` while a recognizer owns one of its fingers. A touch + * no recognizer owns keeps its moves (`spec:AST-094` FR19): nothing is + * cancelled before a recognizer begins, or for a touch it does not track, + * unless a recognizer holds that finger already (a scroll-edge pan pulling + * away from an edge every scroller under it is at; FR19, DEC-18). + */ + /** + * Under `inputMode: 'pointer-events'` a finger comes from its pointer + * events, so its touches carry no id of ours: the live finger it is, the + * nearest pointer-fed finger on the same target. Its touchmoves can then + * be cancelled once a recognizer claims it (`spec:AST-094` FR19, FR21). + */ + private pointerFingerAt(touch: Touch): number | undefined { + let best: GestureTouch | null = null; + let bestDistance = Number.POSITIVE_INFINITY; + for (const finger of this.arena.touches) { + if ( + finger.pointerType !== 'touch' || + finger.phase === 'ended' || + finger.phase === 'cancelled' || + !(finger.target instanceof Node) || + !(touch.target instanceof Node) || + !( + finger.target === touch.target || + finger.target.contains(touch.target) || + touch.target.contains(finger.target) + ) + ) { + continue; + } + const d = Math.hypot(finger.x - touch.clientX, finger.y - touch.clientY); + if (d < bestDistance) { + best = finger; + bestDistance = d; + } + } + return best?.id; + } + + private cancelIfClaimed(event: TouchEvent): void { + if (!event.cancelable || event.defaultPrevented) { + return; + } + // Only the touches this move reports: another finger a recognizer owns + // does not make this one its own. + const moved = touchesOf(event.changedTouches); + for (const touch of moved) { + const id = + this.touchIds.get(touch.identifier) ?? + (this.inputMode === 'pointer-events' + ? this.pointerFingerAt(touch) + : undefined); + if ( + id != null && + (this.arena.isClaimed(id) || + (this.isCoarse && + this.arena.consumerOf(id)?.keepsTouch?.(id) === true) || + // A recognizer that holds the finger before it begins: a + // scroll-edge pan pulling away from an edge (FR19, DEC-18), only + // while this is the one finger on the screen: with a second one + // down anywhere, even off the recognizer's element, the moves stay + // the browser's, for its pinch (FR10). + (event.touches.length === 1 && + this.arena + .participantsOf(id) + .some(recognizer => internals(recognizer).keepsTouch(id)))) + ) { + event.preventDefault(); + return; + } + } + if (moved.some(touch => (touch as StylusTouch).touchType === 'stylus')) { + for (const id of this.pointerIds.values()) { + if (this.arena.isClaimed(id)) { + // A pen whose pointer a recognizer owns: its stylus touches too, + // never a finger no recognizer owns. + event.preventDefault(); + return; + } + } + } + } + + private readonly onTouchEnd = (event: TouchEvent): void => { + this.releaseArmedPress(); + this.noteFingersDown(event); + // Where the host reads a finger from its pointer events (no ontouchstart, + // as after touch emulation turns on), its Touch Events would track it twice. + if (!this.usesTouchEvents || !this.takeEvent(event)) { + return; + } + this.dropFingersNotDown(event); + const time = eventTime(event); + const inputs: TouchInput[] = []; + let isRefused = false; + for (const touch of touchesOf(event.changedTouches)) { + if (this.cancelledFingers.delete(touch.identifier)) { + isRefused = true; + continue; + } + const id = this.forgetFinger(touch.identifier); + if (id == null) { + continue; + } + inputs.push({id, x: touch.clientX, y: touch.clientY, time}); + } + // The lift is when momentum starts: the clock the brake wants most. + const landed = landedOn(event); + if (landed instanceof Node) { + this.lastTouch = {target: landed, time}; + } + const dispositions = + inputs.length > 0 ? this.arena.ended(inputs) : new Map(); + if (isRefused || needsSuppression(dispositions)) { + if (event.cancelable) { + event.preventDefault(); + } + this.armSwallower(); + } + }; + + private readonly onTouchCancel = (event: TouchEvent): void => { + this.releaseArmedPress(); + this.noteFingersDown(event); + // Where the host reads a finger from its pointer events (no ontouchstart, + // as after touch emulation turns on), its Touch Events would track it twice. + if (!this.usesTouchEvents || !this.takeEvent(event)) { + return; + } + const listed = touchesOf(event.changedTouches); + // A cancel that names no finger is the system taking them all. + const identifiers = + listed.length > 0 + ? listed.map(touch => touch.identifier) + : [...this.touchIds.keys()]; + if (listed.length === 0) { + this.cancelledFingers.clear(); + } + for (const identifier of identifiers) { + this.cancelledFingers.delete(identifier); + } + const ids: number[] = []; + for (const identifier of identifiers) { + const id = this.forgetFinger(identifier); + if (id != null) { + ids.push(id); + } + } + if (ids.length > 0) { + this.arena.cancelled(ids, eventTime(event)); + } + }; + + /** The target's own listeners, for a target that left the document. */ + private readonly onTargetTouchMove = (event: Event): void => { + this.onTouchMove(event as TouchEvent); + this.cancelIfClaimed(event as TouchEvent); + }; + + /** The same for a finger nothing could claim while it moves: never blocks. */ + private readonly onTargetTouchMovePassive = (event: Event): void => { + this.onTouchMove(event as TouchEvent); + }; + + private readonly onTargetTouchEnd = (event: Event): void => { + this.onTouchEnd(event as TouchEvent); + }; + + /** The same, registered passively for a touch only watchers follow. */ + private readonly onTargetTouchEndPassive = (event: Event): void => { + this.onTouchEnd(event as TouchEvent); + }; + + private readonly onTargetTouchCancel = (event: Event): void => { + this.onTouchCancel(event as TouchEvent); + }; + + /** + * Cancel tracked fingers the event says are no longer down. `touches` + * lists every finger on the screen; one missing from it (and not lifting + * in this very event) ended where we could not see it. + */ + private dropFingersNotDown(event: TouchEvent): void { + if ( + event.touches == null || + (this.touchIds.size === 0 && this.cancelledFingers.size === 0) + ) { + return; + } + const down = new Set(); + for (const touch of touchesOf(event.touches)) { + down.add(touch.identifier); + } + for (const touch of touchesOf(event.changedTouches)) { + down.add(touch.identifier); + } + for (const identifier of [...this.cancelledFingers]) { + if (!down.has(identifier)) { + this.cancelledFingers.delete(identifier); + } + } + const gone = [...this.touchIds.keys()].filter( + identifier => !down.has(identifier), + ); + const ids: number[] = []; + for (const identifier of gone) { + const id = this.forgetFinger(identifier); + if (id != null) { + ids.push(id); + } + } + if (ids.length > 0) { + this.arena.cancelled(ids, eventTime(event)); + } + } + + private watchTarget( + identifier: number, + target: EventTarget | null, + isBlocking: boolean, + isClaiming: boolean, + ): void { + if (target == null || typeof target.addEventListener !== 'function') { + return; + } + this.touchTargets.set(identifier, target); + const state = this.watched.get(target); + if (state == null) { + this.watched.set(target, { + count: 1, + blocking: isBlocking ? 1 : 0, + claiming: isClaiming ? 1 : 0, + }); + if (isBlocking) { + this.blockingFingers.add(identifier); + } + if (isClaiming) { + this.claimingFingers.add(identifier); + } + target.addEventListener( + 'touchmove', + isBlocking ? this.onTargetTouchMove : this.onTargetTouchMovePassive, + {passive: !isBlocking}, + ); + target.addEventListener( + 'touchend', + isClaiming ? this.onTargetTouchEnd : this.onTargetTouchEndPassive, + {passive: !isClaiming}, + ); + target.addEventListener('touchcancel', this.onTargetTouchCancel, { + passive: true, + }); + return; + } + const wasBlocking = state.blocking > 0; + const wasClaiming = state.claiming > 0; + state.count += 1; + if (isBlocking) { + state.blocking += 1; + this.blockingFingers.add(identifier); + } + if (isClaiming) { + state.claiming += 1; + this.claimingFingers.add(identifier); + } + this.swapTargetTouchMove(target, wasBlocking, state.blocking > 0); + this.swapTargetTouchEnd(target, wasClaiming, state.claiming > 0); + } + + private unwatchTarget(identifier: number): void { + const target = this.touchTargets.get(identifier); + if (target == null) { + return; + } + this.touchTargets.delete(identifier); + const state = this.watched.get(target); + if (state == null) { + return; + } + const wasBlocking = state.blocking > 0; + const wasClaiming = state.claiming > 0; + state.count -= 1; + if (this.blockingFingers.delete(identifier)) { + state.blocking -= 1; + } + if (this.claimingFingers.delete(identifier)) { + state.claiming -= 1; + } + if (state.count > 0) { + this.swapTargetTouchMove(target, wasBlocking, state.blocking > 0); + this.swapTargetTouchEnd(target, wasClaiming, state.claiming > 0); + return; + } + this.watched.delete(target); + target.removeEventListener( + 'touchmove', + wasBlocking ? this.onTargetTouchMove : this.onTargetTouchMovePassive, + ); + target.removeEventListener( + 'touchend', + wasClaiming ? this.onTargetTouchEnd : this.onTargetTouchEndPassive, + ); + target.removeEventListener('touchcancel', this.onTargetTouchCancel); + } + + /** + * Keeps one `touchmove` listener on a watched target: the blocking one + * while a finger on it could be claimed as it moves, a passive one + * otherwise, so a scroll starting there never waits on the main thread + * (`spec:AST-094` FR19). + */ + private swapTargetTouchMove( + target: EventTarget, + wasBlocking: boolean, + isBlocking: boolean, + ): void { + if (wasBlocking === isBlocking) { + return; + } + target.removeEventListener( + 'touchmove', + wasBlocking ? this.onTargetTouchMove : this.onTargetTouchMovePassive, + ); + target.addEventListener( + 'touchmove', + isBlocking ? this.onTargetTouchMove : this.onTargetTouchMovePassive, + {passive: !isBlocking}, + ); + } + + /** + * Keeps one `touchend` listener on a watched target: the cancelable one + * while a finger on it has a participant that could claim (a lift whose + * click must not happen cancels its `touchend`), a passive one while only + * watchers follow its fingers (`spec:AST-094` FR20). + */ + private swapTargetTouchEnd( + target: EventTarget, + wasClaiming: boolean, + isClaiming: boolean, + ): void { + if (wasClaiming === isClaiming) { + return; + } + target.removeEventListener( + 'touchend', + wasClaiming ? this.onTargetTouchEnd : this.onTargetTouchEndPassive, + ); + target.addEventListener( + 'touchend', + isClaiming ? this.onTargetTouchEnd : this.onTargetTouchEndPassive, + {passive: !isClaiming}, + ); + } + + /** Stop tracking a finger by its Touch.identifier; returns its arena id. */ + private forgetFinger(identifier: number): number | undefined { + const id = this.touchIds.get(identifier); + this.touchIds.delete(identifier); + this.unwatchTarget(identifier); + if (id != null) { + this.forgetContact(id); + } + return id; + } + + private pairTouch(id: number, place: PlacedContact): void { + for (const [pointerId, pending] of this.pendingPointers) { + if (isSamePlace(pending, place)) { + this.pendingPointers.delete(pointerId); + this.fingerPointers.set(pointerId, id); + return; + } + } + this.unpairedTouches.set(id, place); + } + + private pairPointer(pointerId: number, place: PlacedContact): void { + for (const [id, unpaired] of this.unpairedTouches) { + if (isSamePlace(unpaired, place)) { + this.unpairedTouches.delete(id); + this.fingerPointers.set(pointerId, id); + return; + } + } + this.pendingPointers.set(pointerId, place); + } + + /** Feed moves, dropping the echo of a move the other channel delivered. */ + private feedMoves(inputs: ReadonlyArray): void { + const fresh = inputs.filter(input => { + const last = this.lastFed.get(input.id); + if ( + last != null && + (input.time < last.time || + (input.time === last.time && + input.x === last.x && + input.y === last.y)) + ) { + return false; + } + this.lastFed.set(input.id, {x: input.x, y: input.y, time: input.time}); + return true; + }); + if (fresh.length > 0) { + this.arena.moved(fresh); + } + } + + /** Forget a contact's pairing and echo state. */ + private forgetContact(id: number): void { + this.unlockSelection(id); + this.lastFed.delete(id); + this.judgedUncancelable.delete(id); + this.unpairedTouches.delete(id); + for (const [pointerId, paired] of this.fingerPointers) { + if (paired === id) { + this.fingerPointers.delete(pointerId); + } + } + } + + // --- Pointer events -------------------------------------------------------- + + private readonly onPointerDown = (event: PointerEvent): void => { + // A new press: whatever click is still owed to the last one is not this. + this.disarmSwallower(); + if (!isPointerKind(event.pointerType)) { + return; + } + this.exitHover(event.pointerId); + if (event.pointerType === 'touch' && this.usesTouchEvents) { + this.pairPointer(event.pointerId, { + x: event.clientX, + y: event.clientY, + target: landedOn(event), + }); + return; + } + if (event.pointerType === 'mouse' && event.button !== 0) { + return; + } + const landed = landedOn(event); + const {recognizers, consumer, consumerElement} = this.participantsAt( + landed, + event.pointerType, + ); + if (recognizers.length === 0 && consumer == null) { + return; + } + const id = this.nextId++; + this.pointerIds.set(event.pointerId, id); + this.arena.began([ + { + id, + x: event.clientX, + y: event.clientY, + time: eventTime(event), + pointerType: event.pointerType, + target: landed, + scroll: createNativeScrollProbe(landed), + recognizers, + consumer, + ...(this.isCoarse ? {} : {isFeedbackOnly: true}), + }, + ]); + if (locksSelectionFor(event.pointerType, this.doc.defaultView)) { + this.lockSelection( + id, + this.arena.participantsOf(id), + event.target, + this.arena.consumerOf(id) ?? null, + consumerElement, + ); + } + }; + + /** The pointer of the group press being served under a fine pointer. */ + private groupPressPointer: number | null = null; + + /** + * A press on an action group under a fine pointer (`family:action-groups` + * FR5): the group's recognizers and consumer take it, nothing else on the + * path does (`spec:AST-094` FR19), and the document listens only until it + * lifts (FR24). Where the primary pointer is coarse the host's own + * listeners serve the group, as every other recognizer. + */ + private beginGroupPress( + event: PointerEvent, + registration: Registration, + ): void { + if ( + this.isInstalled || + this.groupPressPointer != null || + !isPointerKind(event.pointerType) || + (event.pointerType === 'mouse' && event.button !== 0) + ) { + return; + } + const recognizers = registration.recognizers.filter( + r => r.isEnabled && internals(r).acceptsTouchInput, + ); + const consumer = + registration.consumer != null && + registration.consumer.isEnabled?.() !== false + ? registration.consumer + : null; + if (recognizers.length === 0 && consumer == null) { + return; + } + this.disarmSwallower(); + const id = this.nextId++; + this.pointerIds.set(event.pointerId, id); + this.groupPressPointer = event.pointerId; + this.listenForGroupPress(true); + this.arena.began([ + { + id, + x: event.clientX, + y: event.clientY, + time: eventTime(event), + pointerType: event.pointerType, + target: event.target, + scroll: createNativeScrollProbe(event.target), + recognizers, + consumer, + }, + ]); + } + + /** The document listeners one group press needs while it is held. */ + private listenForGroupPress(isListening: boolean): void { + const doc = this.doc; + const view = doc.defaultView; + if (isListening) { + doc.addEventListener('pointermove', this.onGroupPointer, PASSIVE_CAPTURE); + doc.addEventListener('pointerup', this.onGroupPointer, PASSIVE_CAPTURE); + doc.addEventListener( + 'pointercancel', + this.onGroupPointer, + PASSIVE_CAPTURE, + ); + view?.addEventListener('blur', this.onGroupBlur); + // The click a lift owes is the press's too: claimed presses swallow it. + view?.addEventListener('click', this.onClick, ACTIVE_CAPTURE); + } else { + doc.removeEventListener( + 'pointermove', + this.onGroupPointer, + PASSIVE_CAPTURE, + ); + doc.removeEventListener( + 'pointerup', + this.onGroupPointer, + PASSIVE_CAPTURE, + ); + doc.removeEventListener( + 'pointercancel', + this.onGroupPointer, + PASSIVE_CAPTURE, + ); + view?.removeEventListener('blur', this.onGroupBlur); + } + } + + /** + * A group press's own events, finger, pen or mouse alike, through the + * pointer path: under a fine pointer the host reads no Touch Events, so + * the touch pairing the installed host does for a finger never applies. + */ + private readonly onGroupPointer = (event: PointerEvent): void => { + if (event.pointerId !== this.groupPressPointer) { + return; + } + const id = this.pointerIds.get(event.pointerId); + if (event.type === 'pointermove') { + if (id != null) { + this.arena.moved([ + { + id, + x: event.clientX, + y: event.clientY, + time: eventTime(event), + history: coalescedHistory(event), + }, + ]); + } + return; + } + this.pointerIds.delete(event.pointerId); + if (id != null) { + if (event.type === 'pointerup') { + const dispositions = this.arena.ended([ + {id, x: event.clientX, y: event.clientY, time: eventTime(event)}, + ]); + if (needsSuppression(dispositions)) { + this.armSwallower(); + } + } else { + this.arena.cancelled([id], eventTime(event)); + } + } + this.endGroupPress(); + }; + + private readonly onGroupBlur = (): void => { + const pointer = this.groupPressPointer; + const id = pointer == null ? undefined : this.pointerIds.get(pointer); + if (pointer != null && id != null) { + this.pointerIds.delete(pointer); + this.arena.cancelled([id], this.arena.now()); + } + this.endGroupPress(); + }; + + /** + * The press lifted: its document listeners go now; the click listener stays + * for the click the lift dispatches in the same task, then goes too. + */ + private endGroupPress(): void { + this.groupPressPointer = null; + this.listenForGroupPress(false); + const view = this.doc.defaultView; + const removeClick = () => { + // The click listener stays while a claimer keeps it installed. + if (!this.isClaimingInstalled) { + view?.removeEventListener('click', this.onClick, ACTIVE_CAPTURE); + } + }; + if (view != null && typeof view.setTimeout === 'function') { + view.setTimeout(removeClick, 0); + } else { + removeClick(); + } + } + + private readonly onPointerMove = (event: PointerEvent): void => { + if (event.pointerType === 'touch' && this.usesTouchEvents) { + const id = this.fingerPointers.get(event.pointerId); + if (id != null) { + this.feedMoves([ + { + id, + x: event.clientX, + y: event.clientY, + time: eventTime(event), + history: coalescedHistory(event), + }, + ]); + } + return; + } + const id = this.pointerIds.get(event.pointerId); + if (id != null) { + this.arena.moved([ + { + id, + x: event.clientX, + y: event.clientY, + time: eventTime(event), + history: coalescedHistory(event), + }, + ]); + return; + } + if (event.pointerType !== 'touch' && event.buttons === 0) { + this.updateHover(event); + } + }; + + private readonly onPointerUp = (event: PointerEvent): void => { + this.releaseArmedPress(); + if (event.pointerType === 'touch' && this.usesTouchEvents) { + // The lift itself is the touchend's: it alone can cancel the click. + this.fingerPointers.delete(event.pointerId); + this.pendingPointers.delete(event.pointerId); + return; + } + const id = this.pointerIds.get(event.pointerId); + if (id == null) { + return; + } + this.pointerIds.delete(event.pointerId); + this.unlockSelection(id); + const dispositions = this.arena.ended([ + {id, x: event.clientX, y: event.clientY, time: eventTime(event)}, + ]); + if (needsSuppression(dispositions)) { + this.armSwallower(); + } + }; + + private readonly onPointerCancel = (event: PointerEvent): void => { + this.releaseArmedPress(); + if (event.pointerType === 'touch' && this.usesTouchEvents) { + const id = + this.fingerPointers.get(event.pointerId) ?? + this.touchForCancelledPointer(event); + this.fingerPointers.delete(event.pointerId); + this.pendingPointers.delete(event.pointerId); + if (id != null) { + this.arena.browserTookOver([id], eventTime(event)); + } + return; + } + const id = this.pointerIds.get(event.pointerId); + if (id != null) { + this.pointerIds.delete(event.pointerId); + this.unlockSelection(id); + this.arena.cancelled([id], eventTime(event)); + } + }; + + /** + * Which live finger a touch `pointercancel` is about. The pointer keeps + * the target the finger landed on (implicit capture), as the Touch does; + * among fingers on that target, the nearest. WebKit also numbers the + * pointer like the Touch, which settles what position cannot. + */ + private touchForCancelledPointer(event: PointerEvent): number | null { + let best: GestureTouch | null = null; + let bestDistance = Number.POSITIVE_INFINITY; + for (const touch of this.arena.touches) { + if ( + touch.pointerType !== 'touch' || + touch.phase === 'ended' || + touch.phase === 'cancelled' || + touch.target !== landedOn(event) + ) { + continue; + } + const d = Math.hypot(touch.x - event.clientX, touch.y - event.clientY); + if (d < bestDistance) { + best = touch; + bestDistance = d; + } + } + if (best != null) { + return best.id; + } + return this.touchIds.get(event.pointerId) ?? null; + } + + // --- Hover ------------------------------------------------------------------- + + private updateHover(event: PointerEvent): void { + if (!isPointerKind(event.pointerType)) { + return; + } + const {hovers} = this.participantsAt(landedOn(event), null); + const next = new Set(hovers.filter(hover => hover.isEnabled)); + const previous = this.hovered.get(event.pointerId) ?? new Set(); + if (next.size === 0 && previous.size === 0) { + return; + } + const point = {x: event.clientX, y: event.clientY}; + for (const hover of previous) { + if (!next.has(hover)) { + hover.hoverExited(point); + } + } + for (const hover of next) { + if (previous.has(hover)) { + hover.hoverMoved(point); + } else { + hover.hoverEntered(point, event.pointerType); + } + } + if (next.size === 0) { + this.hovered.delete(event.pointerId); + } else { + this.hovered.set(event.pointerId, next); + } + } + + private exitHover(pointerId: number): void { + const hovers = this.hovered.get(pointerId); + if (hovers == null) { + return; + } + this.hovered.delete(pointerId); + for (const hover of hovers) { + hover.hoverExited(); + } + } + + private readonly onPointerOut = (event: PointerEvent): void => { + if (event.relatedTarget == null) { + this.exitHover(event.pointerId); + } + }; + + // --- Clicks, menus, scrolls, focus ----------------------------------------- + + private armSwallower(): void { + this.swallowDeadline = + browserClock.now() + UIKIT_CLICK.strayClickWindow.value; + } + + private disarmSwallower(): void { + this.swallowDeadline = Number.NEGATIVE_INFINITY; + } + + private readonly onClick = (event: MouseEvent): void => { + if (browserClock.now() > this.swallowDeadline) { + return; + } + if (!event.isTrusted || event.detail === 0) { + return; + } + if ( + event.target instanceof Element && + event.target.hasAttribute(HAPTIC_SWITCH_ATTRIBUTE) + ) { + return; + } + this.disarmSwallower(); + event.preventDefault(); + event.stopImmediatePropagation(); + }; + + private replayClick(touch: GestureTouch, allowed: boolean): void { + if (!allowed) { + return; + } + // The browser's own click was swallowed while recognizers decided; they + // all failed, so the click the touch earned is delivered now. + this.disarmSwallower(); + const target = touch.target; + const view = this.doc.defaultView; + if (!(target instanceof Element) || !target.isConnected || view == null) { + return; + } + target.dispatchEvent( + new view.MouseEvent('click', { + bubbles: true, + cancelable: true, + composed: true, + detail: 1, + clientX: touch.x, + clientY: touch.y, + }), + ); + } + + private readonly onContextMenu = (event: MouseEvent): void => { + for (const touch of this.arena.touches) { + if (this.arena.isClaimed(touch.id)) { + event.preventDefault(); + return; + } + } + }; + + /** + * A recognizer is about to scroll `scroller` itself (a menu's edge + * autoscroll, spec:AST-094 FR30): the scroll that write fires is not a + * finger's and not the browser taking one. It passes until the end of the + * next frame, when that event has fired; a real pan in that window still + * reports itself through pointercancel or touchcancel. + */ + ownScroll(scroller: Node): void { + const call = ++this.ownScrollCalls; + this.ownedScrolls.set(scroller, call); + const view = this.doc.defaultView; + if (view == null) { + return; + } + view.requestAnimationFrame(() => { + view.requestAnimationFrame(() => { + // A newer call keeps it until its own frame has passed. + if (this.ownedScrolls.get(scroller) === call) { + this.ownedScrolls.delete(scroller); + } + }); + }); + } + + private readonly onScroll = (event: Event): void => { + const time = eventTime(event); + const scroller = event.target === this.doc ? this.doc : event.target; + if (!(scroller instanceof Node) || this.ownedScrolls.has(scroller)) { + return; + } + // A watched scroller's scroll is a finger's from a scroll while a finger + // that landed inside it is down until its own scrollend. + this.endedWhileDown.delete(scroller); + if ( + this.scrollWatchers.has(scroller) && + !this.fingerScrollers.has(scroller) && + this.isFingerDownIn(scroller) + ) { + this.fingerScrollers.add(scroller); + } + if (this.fingerScrollers.has(scroller) && !this.hasScrollEnd()) { + this.clearScrollEndFallback(scroller); + this.scrollEndFallbacks.set( + scroller, + setTimeout(() => { + this.scrollEndFallbacks.delete(scroller); + this.endFingerScroll(scroller); + }, SCROLL_END_FALLBACK_MS), + ); + } + // A scroll is a finger's only if that finger was inside this scroller, + // recently; anything else is the program moving something. + const last = this.lastTouch; + if ( + last != null && + time - last.time < UIKIT_PRESS.momentumMemory.value && + scroller.contains(last.target) + ) { + this.lastScroll = {scroller, time}; + } + const taken: number[] = []; + for (const touch of this.arena.touches) { + if ( + touch.takenByBrowser || + touch.phase === 'ended' || + touch.phase === 'cancelled' || + touch.pointerType !== 'touch' || + this.arena.isClaimed(touch.id) + ) { + continue; + } + if ( + touch.target instanceof Node && + scroller.contains(touch.target) && + // Only a pan this finger drove takes the touch: the browser starts + // one only once the finger has travelled its slop. A scroll under a + // finger that has not (scrollIntoView, a transcript pinned to its + // end, scroll anchoring) is the page's own: the touch, and the click + // it earns, stay where they are. + Math.hypot(touch.x - touch.startX, touch.y - touch.startY) >= + UIKIT_PRESS.slop.value + ) { + taken.push(touch.id); + } + } + if (taken.length > 0) { + this.arena.browserTookOver(taken, time); + } + }; + + private noteFingersDown(event: TouchEvent): void { + const fingers: Node[] = []; + for (const touch of touchesOf(event.touches)) { + if ((touch as StylusTouch).touchType !== 'stylus') { + if (touch.target instanceof Node) { + fingers.push(touch.target); + } + } + } + this.fingerTargets = fingers; + // A scroll that ended while its finger rested ends with the lift. + for (const scroller of this.endedWhileDown) { + if (!this.isFingerDownIn(scroller)) { + this.endedWhileDown.delete(scroller); + this.fingerScrollers.delete(scroller); + } + } + } + + /** + * A second finger landed somewhere (wherever: often on a plain row): a + * pan that takes one finger only and has not claimed its touch fails, so + * the two fingers stay the browser's pinch (WCAG 1.4.4) instead of the + * pan claiming them midway. A drag a recognizer has claimed keeps its + * finger: a second finger never cancels or redirects it (`spec:AST-094` + * FR10), and no claim is released here. + */ + private failUnclaimedLoneFingerPans(): void { + for (const recognizer of [...this.arena.involvedRecognizers]) { + if ( + isPan(recognizer) && + recognizer.state === 'possible' && + recognizer.maximumNumberOfTouches === 1 && + recognizer.trackedTouches.length > 0 && + recognizer.trackedTouches.every( + touch => + touch.pointerType === 'touch' && !this.arena.isClaimed(touch.id), + ) + ) { + recognizer.cancel(); + } + } + } + + private isFingerDownIn(scroller: Node): boolean { + return this.fingerTargets.some(target => scroller.contains(target)); + } + + private readonly onScrollEnd = (event: Event): void => { + const scroller = event.target === this.doc ? this.doc : event.target; + if (scroller instanceof Node) { + this.endFingerScroll(scroller); + } + }; + + /** Whether the browser sends `scrollend`. */ + private hasScrollEnd(): boolean { + const view = this.doc.defaultView; + return view != null && 'onscrollend' in view; + } + + private clearScrollEndFallback(scroller: Node): void { + const timer = this.scrollEndFallbacks.get(scroller); + if (timer != null) { + clearTimeout(timer); + this.scrollEndFallbacks.delete(scroller); + } + } + + /** A finger-driven scroll stopped: it ends, or at the lift if it rests. */ + private endFingerScroll(scroller: Node): void { + if (!this.fingerScrollers.has(scroller)) { + return; + } + if (this.isFingerDownIn(scroller)) { + // The finger still rests there: the scroll is its until the lift. + this.endedWhileDown.add(scroller); + } else { + this.fingerScrollers.delete(scroller); + } + } + + private readonly onVisibilityChange = (): void => { + if (this.doc.visibilityState === 'hidden') { + this.cancelEverything(this.arena.now()); + } + }; + + private readonly onBlur = (): void => { + this.cancelEverything(this.arena.now()); + }; + + /** The app lost focus: UIKit cancels every touch in flight. */ + private cancelEverything(time: number): void { + const ids = [...this.pointerIds.values()]; + for (const id of ids) { + this.unlockSelection(id); + } + for (const identifier of [...this.touchIds.keys()]) { + // The browser may keep the touch alive: its lift must not click. + this.cancelledFingers.add(identifier); + const id = this.forgetFinger(identifier); + if (id != null) { + ids.push(id); + } + } + this.pointerIds.clear(); + this.fingerPointers.clear(); + this.pendingPointers.clear(); + this.unpairedTouches.clear(); + this.lastFed.clear(); + if (ids.length > 0) { + this.arena.cancelled(ids, time); + } + for (const pointerId of [...this.hovered.keys()]) { + this.exitHover(pointerId); + } + } +} + +function needsSuppression( + dispositions: Map, +): boolean { + for (const disposition of dispositions.values()) { + if (disposition !== 'allow') { + return true; + } + } + return false; +} + +function coalescedHistory(event: PointerEvent): TouchSample[] { + if (typeof event.getCoalescedEvents !== 'function') { + return []; + } + const coalesced = event.getCoalescedEvents(); + return coalesced.slice(0, -1).map(sample => ({ + x: sample.clientX, + y: sample.clientY, + time: eventTime(sample), + })); +} + +const hosts = new WeakMap(); + +/** The gesture host of a document, created on first use. */ +export function getGestureHost(doc: Document): GestureHost { + let host = hosts.get(doc); + if (host == null) { + host = new GestureHost(doc); + hosts.set(doc, host); + } + return host; +} diff --git a/packages/core/src/gestures/GestureRecognizer.ts b/packages/core/src/gestures/GestureRecognizer.ts index 349ebbbda9ce8..d316b3a7fd9e9 100644 --- a/packages/core/src/gestures/GestureRecognizer.ts +++ b/packages/core/src/gestures/GestureRecognizer.ts @@ -24,7 +24,8 @@ * event, so a listener that writes a transform writes it in the same frame * as the input, with no React render in between. * - * SYNC: When modified, update GestureArena.ts. + * SYNC: When modified, update GestureArena.ts, recognizerInternals.ts and + * README.md. */ import type {GestureArena} from './GestureArena'; @@ -38,6 +39,16 @@ import type { } from './types'; import {DEFAULT_ENVIRONMENT} from './types'; +/** + * Turns text selection and the touch callout off on `elements` (and on a + * non-text input `target` inside one) for one held press; returns the undo. + * gestures/selectionLock.ts's `lockSelection` is the one implementation. + */ +export type SelectionLock = ( + elements: Iterable, + target: EventTarget | null, +) => () => void; + export interface GestureRecognizerOptions { /** A name for debugging and test output. */ name?: string; @@ -51,6 +62,18 @@ export interface GestureRecognizerOptions { cancelsTouchesInView?: boolean; /** Hold the view's touch-began until this recognizer fails. @default false */ delaysTouchesBegan?: boolean; + /** + * For a hold that does something (a slider's thumb, a switch's or a + * segmented control's drag): the lock that turns text selection and the + * touch callout off on this recognizer's element for each held press + * (`spec:AST-094` FR41). Pass `lockSelection` from gestures/selectionLock.ts; + * the host calls it and never imports it, so a component without such a + * hold does not bundle it. Content under a sheet's or a pager's pan stays + * selectable, so there is none by default; a mouse keeps desktop selection + * either way. A long press brings its own when it has a hold (a + * `minimumPressDuration` above 0); `null` brings none. + */ + selectionLock?: SelectionLock | null; /** * Hold the view's touch-ended (and the browser's click) while this * recognizer may still recognize. @default true @@ -60,6 +83,17 @@ export interface GestureRecognizerOptions { allowedPointerTypes?: ReadonlyArray; /** Ignore a touch of a different kind than the ones already tracked. @default true */ requiresExclusiveTouchType?: boolean; + /** + * Whether this recognizer takes its touches from the browser when it + * recognizes. `false` makes a watcher (a tab bar that minimizes while the + * page scrolls): it never claims, so it gets only passive listeners and + * never delays a scroll; it prevents no other recognizer and none prevents + * it (`spec:AST-094` FR15); it cancels no press or click under it; and it + * keeps tracking a touch the browser takes for a scroll, + * reporting through the native scroll and ending at the lift. + * @default true + */ + claimsTouches?: boolean; /** gestureRecognizerShouldBegin: return false to fail instead of beginning. */ shouldBegin?: (recognizer: R) => boolean; /** gestureRecognizer(_:shouldReceive:): return false to never see this touch. */ @@ -118,7 +152,9 @@ export function isActive(state: GestureState): boolean { return state === 'began' || state === 'changed'; } -/** The browser behaviors `touch-action` can keep or remove. */ +/** + * The browser behaviors `touch-action` can keep or remove. + */ export interface NativeTouchBehavior { readonly panX: boolean; readonly panY: boolean; @@ -150,6 +186,8 @@ export abstract class GestureRecognizer { name: string; isEnabled: boolean; cancelsTouchesInView: boolean; + /** See GestureRecognizerOptions.selectionLock. */ + readonly selectionLock: SelectionLock | null; delaysTouchesBegan: boolean; delaysTouchesEnded: boolean; allowedPointerTypes: ReadonlySet | null; @@ -161,9 +199,9 @@ export abstract class GestureRecognizer { view: unknown = null; /** @internal The arena delivering touches to this recognizer, if any. */ - arena: GestureArena | null = null; + protected arena: GestureArena | null = null; /** @internal A recognition waiting for a failure requirement. */ - pending: 'began' | 'ended' | null = null; + protected pending: 'began' | 'ended' | null = null; /** * The delegate functions. Typed against the base so a subclass stays @@ -182,7 +220,7 @@ export abstract class GestureRecognizer { /** Touches currently tracked, by id. */ protected readonly touches = new Map(); /** @internal Every touch id received since the last reset. */ - readonly seenTouchIds = new Set(); + protected readonly seenTouchIds = new Set(); private lastLocation: Point = {x: 0, y: 0}; constructor(options: GestureRecognizerOptions = {}) { @@ -191,6 +229,7 @@ export abstract class GestureRecognizer { this.name = options.name ?? this.constructor.name; this.isEnabled = options.isEnabled ?? true; this.cancelsTouchesInView = options.cancelsTouchesInView ?? true; + this.selectionLock = options.selectionLock ?? null; this.delaysTouchesBegan = options.delaysTouchesBegan ?? false; this.delaysTouchesEnded = options.delaysTouchesEnded ?? true; this.allowedPointerTypes = @@ -212,7 +251,7 @@ export abstract class GestureRecognizer { * it. Every touch recognizer claims; hover has no touches to claim. */ get claimsTouches(): boolean { - return true; + return this.options.claimsTouches ?? true; } get state(): GestureState { @@ -229,7 +268,7 @@ export abstract class GestureRecognizer { } /** @internal Whether this recognizer listens to touch at all. */ - get acceptsTouchInput(): boolean { + protected get acceptsTouchInput(): boolean { return ( this.allowedPointerTypes == null || this.allowedPointerTypes.has('touch') ); @@ -286,7 +325,7 @@ export abstract class GestureRecognizer { } /** @internal Whether this recognizer must wait for `other` to fail. */ - requiresFailureOf(other: GestureRecognizer): boolean { + protected requiresFailureOf(other: GestureRecognizer): boolean { return ( this.failureRequirements.has(other) || this.options.shouldRequireFailureOf?.(this, other) === true || @@ -299,7 +338,7 @@ export abstract class GestureRecognizer { * (`recognizeSimultaneouslyWith` sets both sides; each delegate answers for * its own recognizer), so one delegate alone keeps them exclusive (FR15). */ - allowsSimultaneous(other: GestureRecognizer): boolean { + protected allowsSimultaneous(other: GestureRecognizer): boolean { return ( this.simultaneousWith.has(other) || (this.options.shouldRecognizeSimultaneously?.(this, other) === true && @@ -308,10 +347,20 @@ export abstract class GestureRecognizer { } /** @internal The delegate's should-begin answer. */ - shouldBegin(): boolean { + protected shouldBegin(): boolean { return this.options.shouldBegin?.(this) ?? true; } + /** + * @internal Whether this recognizer holds `touchId` before it begins: the + * host then cancels that finger's moves as for a touch a recognizer owns. + * None does by default (`spec:AST-094` FR19); a scroll-edge pan holds a + * finger pulling away from an edge every scroller under it is at. + */ + protected keepsTouch(_touchId: number): boolean { + return false; + } + /** UIKit's `canPrevent(_:)`: whether recognizing fails `other`. */ canPrevent(_other: GestureRecognizer): boolean { return true; @@ -396,6 +445,10 @@ export abstract class GestureRecognizer { touches: ReadonlyArray, event: GestureEventContext, ): boolean { + if (!this.claimsTouches) { + // A watcher follows the browser's scroll to the lift. + return true; + } this.touchesCancelled(touches, event); return false; } @@ -469,7 +522,7 @@ export abstract class GestureRecognizer { // --- Arena hooks (internal) ---------------------------------------------- /** @internal Whether this recognizer takes `touch` right now. */ - wantsTouch(touch: GestureTouch): boolean { + protected wantsTouch(touch: GestureTouch): boolean { if (this.state !== 'possible' && !isActive(this.state)) { return false; } @@ -480,7 +533,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverBegan( + protected deliverBegan( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -493,7 +546,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverBeganElsewhere( + protected deliverBeganElsewhere( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -504,7 +557,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverMoved( + protected deliverMoved( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -517,7 +570,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverEnded( + protected deliverEnded( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -535,7 +588,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverCancelled( + protected deliverCancelled( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -552,7 +605,7 @@ export abstract class GestureRecognizer { } /** @internal */ - deliverTakenByBrowser( + protected deliverTakenByBrowser( touches: ReadonlyArray, event: GestureEventContext, ): void { @@ -568,12 +621,19 @@ export abstract class GestureRecognizer { } } + /** @internal The host's hook: listeners follow whether anything is enabled. */ + onEnabledChange: (() => void) | null = null; + /** * UIKit's `isEnabled` setter: turning a recognizer off mid-gesture cancels * the gesture (an active one) or fails it (a possible one). */ setEnabled(value: boolean): void { + const changed = this.isEnabled !== value; this.isEnabled = value; + if (changed) { + this.onEnabledChange?.(); + } if (!value && (this.state !== 'possible' || this.isInvolved())) { this.cancel(); } @@ -592,7 +652,7 @@ export abstract class GestureRecognizer { } /** @internal Apply a transition the arena has approved. */ - applyState(next: GestureState): void { + protected applyState(next: GestureState): void { const from = this.stateValue; if (from === next && next !== 'changed') { return; @@ -615,7 +675,7 @@ export abstract class GestureRecognizer { } /** @internal Return to `possible` for the next attempt. */ - resetToPossible(): void { + protected resetToPossible(): void { this.clearTimer(); this.pending = null; this.touches.clear(); diff --git a/packages/core/src/gestures/HoverGestureRecognizer.ts b/packages/core/src/gestures/HoverGestureRecognizer.ts new file mode 100644 index 0000000000000..ff66f5f2c48cb --- /dev/null +++ b/packages/core/src/gestures/HoverGestureRecognizer.ts @@ -0,0 +1,82 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file HoverGestureRecognizer.ts + * @input Hover enter / move / exit reports from a GestureHost + * @output Exports HoverGestureRecognizer (UIHoverGestureRecognizer) + * @position Continuous recognizer of packages/core/src/gestures that takes + * no touches: a mouse or a hovering pen over its view + * + * Began when a hovering pointer (no button pressed) enters the view, changed + * as it moves inside, ended when it leaves. A finger never hovers, so the + * recognizer is silent on touch screens, as UIKit's is on an iPhone. It + * claims nothing and conflicts with nothing: the host drives it directly, + * outside the touch arena. + * + * Governed by architecture:interaction-modality: hover only enhances, so no + * behavior may depend on this recognizer alone, and a touch or keyboard + * path reaches whatever it reveals. + */ + +import { + GestureRecognizer, + isActive, + isTerminal, + type GestureRecognizerOptions, +} from './GestureRecognizer'; +import type {Point, PointerKind} from './types'; + +export class HoverGestureRecognizer extends GestureRecognizer { + private point: Point = {x: 0, y: 0}; + /** The kind of pointer hovering: a mouse, or a pen in range. */ + pointerType: PointerKind | null = null; + + constructor(options: GestureRecognizerOptions = {}) { + super(options); + } + + override get claimsTouches(): boolean { + return false; + } + + override location(): Point { + return this.point; + } + + /** @internal The host saw a hovering pointer enter the view. */ + hoverEntered(point: Point, pointerType: PointerKind): void { + if (!this.isEnabled || isActive(this.state)) { + return; + } + if (isTerminal(this.state)) { + // A hover cancelled mid-way (the recognizer was disabled, or stopped + // from code) starts over at the next enter. + this.resetToPossible(); + } + this.point = point; + this.pointerType = pointerType; + this.applyState('began'); + } + + /** @internal */ + hoverMoved(point: Point): void { + if (!isActive(this.state)) { + return; + } + this.point = point; + this.applyState('changed'); + } + + /** @internal The pointer left, pressed, or went away. */ + hoverExited(point?: Point): void { + if (!isActive(this.state)) { + return; + } + if (point != null) { + this.point = point; + } + this.applyState('ended'); + this.resetToPossible(); + this.pointerType = null; + } +} diff --git a/packages/core/src/gestures/LongPressGestureRecognizer.ts b/packages/core/src/gestures/LongPressGestureRecognizer.ts index b552928573727..0f7ee6f28dc3d 100644 --- a/packages/core/src/gestures/LongPressGestureRecognizer.ts +++ b/packages/core/src/gestures/LongPressGestureRecognizer.ts @@ -21,6 +21,7 @@ */ import {UIKIT_LONG_PRESS, UIKIT_TAP} from '../motion/uikit/gestures'; +import {lockSelection} from './selectionLock'; import {travel} from './geometry'; import { GestureRecognizer, @@ -55,7 +56,20 @@ export class LongPressGestureRecognizer extends GestureRecognizer { beganAt: number | null = null; constructor(options: LongPressGestureRecognizerOptions = {}) { - super(options); + // A long press's hold does something (a menu, a hold-repeat): it locks + // selection for its press (spec:AST-094 FR41). One at 0 ms recognizes at + // touch-down and has no hold, so it brings none, nor does one told null: + // there the browser keeps its native long press. + super({ + ...options, + selectionLock: + options.selectionLock !== undefined + ? options.selectionLock + : (options.minimumPressDuration ?? + UIKIT_LONG_PRESS.minimumPressDuration.value) > 0 + ? lockSelection + : null, + }); this.minimumPressDuration = options.minimumPressDuration ?? UIKIT_LONG_PRESS.minimumPressDuration.value; diff --git a/packages/core/src/gestures/PanGestureRecognizer.ts b/packages/core/src/gestures/PanGestureRecognizer.ts index 136fbcfe094f3..e52527ec65940 100644 --- a/packages/core/src/gestures/PanGestureRecognizer.ts +++ b/packages/core/src/gestures/PanGestureRecognizer.ts @@ -364,6 +364,36 @@ export class PanGestureRecognizer extends GestureRecognizer { this.offset = {x: -swallowed.x, y: -swallowed.y}; } + /** + * A scroll-edge pan holds its finger from the first move while every + * scroller under it is at its edge and the finger pulls away from that + * edge, the pan's own way (`spec:AST-094` FR19, DEC-18): + * the browser would only rubber-band there. It still begins at its + * hysteresis; a move the content can still scroll stays the browser's, + * and so does every move while a second finger is down (a pinch). + */ + protected override keepsTouch(touchId: number): boolean { + const touch = this.touches.get(touchId); + if ( + !this.scrollEdge || + this.state !== 'possible' || + // A lone finger only: two fingers before a claim are the page's pinch + // (spec:AST-094 FR10). + this.touches.size !== 1 || + touch == null || + touch.pointerType !== 'touch' || + touch.takenByBrowser + ) { + return false; + } + const moved = this.raw; + return ( + (moved.x !== 0 || moved.y !== 0) && + this.classify(moved) === 'begin' && + !this.nativeCanScroll(moved.x, moved.y) + ); + } + private nativeCanScroll(dx: number, dy: number): boolean { for (const touch of this.touches.values()) { if (touch.scroll?.canScroll(dx, dy) === true) { diff --git a/packages/core/src/gestures/PinchGestureRecognizer.ts b/packages/core/src/gestures/PinchGestureRecognizer.ts new file mode 100644 index 0000000000000..28352376459b2 --- /dev/null +++ b/packages/core/src/gestures/PinchGestureRecognizer.ts @@ -0,0 +1,144 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file PinchGestureRecognizer.ts + * @input Touches from a GestureArena + * @output Exports PinchGestureRecognizer (UIPinchGestureRecognizer) and its + * options + * @position Continuous two-finger recognizer of packages/core/src/gestures + * + * Tracks the first two fingers; more are ignored. Begins once the distance + * between them has changed by `hysteresis` px. As in UIKit, `scale` is then + * the current distance over the starting distance moved by the hysteresis + * toward the pinch, so it does not jump (settable to re-base it); `velocity` + * is scale per second. Lifting either finger ends an + * active pinch and fails a possible one. + * + * A host gives every element WITHOUT a pinch recognizer `pinch-zoom` in its + * `touch-action`, so the page stays zoomable (WCAG 1.4.4); attaching one + * takes the two-finger gesture for this element only. + */ + +import {UIKIT_PINCH} from '../motion/uikit/gestures'; +import {centroid, distance} from './geometry'; +import { + GestureRecognizer, + isActive, + type GestureRecognizerOptions, + type NativeTouchBehavior, +} from './GestureRecognizer'; +import type {GestureEventContext, GestureTouch, Point} from './types'; +import {VelocityTracker} from './VelocityTracker'; + +export interface PinchGestureRecognizerOptions extends GestureRecognizerOptions { + hysteresis?: number; +} + +export class PinchGestureRecognizer extends GestureRecognizer { + hysteresis: number; + private initialSpan = 0; + private scaleValue = 1; + private readonly tracker = new VelocityTracker(); + private lastTime = 0; + + constructor(options: PinchGestureRecognizerOptions = {}) { + super(options); + this.hysteresis = options.hysteresis ?? UIKIT_PINCH.hysteresis.value; + } + + /** Current span over the span the fingers started at. */ + get scale(): number { + return this.scaleValue; + } + + /** Re-base: from now on the scale reads `value` at the current span. */ + set scale(value: number) { + const span = this.span(); + if (span > 0 && value > 0) { + this.initialSpan = span / value; + this.scaleValue = value; + // The velocity continues from the re-based value, without a jump. + this.tracker.reset(); + this.tracker.add(value, 0, this.lastTime); + } + } + + /** Scale per second (UIKit's filter; measured for pans, assumed here). */ + get velocity(): number { + return this.tracker.velocity().x; + } + + /** The midpoint between the two fingers. */ + focalPoint(): Point { + return centroid(this.touches.values()); + } + + /** Two fingers on this view are this recognizer's, not the browser's zoom. */ + override get nativeTouchBehavior(): NativeTouchBehavior { + return {panX: true, panY: true, pinchZoom: false, doubleTapZoom: false}; + } + + protected override acceptsTouch(touch: GestureTouch): boolean { + return this.numberOfTouches < 2 && super.acceptsTouch(touch); + } + + protected override touchesBegan( + _touches: ReadonlyArray, + event: GestureEventContext, + ): void { + this.lastTime = event.time; + if (this.numberOfTouches === 2) { + this.initialSpan = this.span(); + this.scaleValue = 1; + this.tracker.reset(); + this.tracker.add(1, 0, event.time); + } + } + + protected override touchesMoved( + _touches: ReadonlyArray, + event: GestureEventContext, + ): void { + if (this.numberOfTouches < 2 || this.initialSpan <= 0) { + return; + } + this.lastTime = event.time; + const span = this.span(); + // Once recognized, a pinch waiting on a recognizer it requires to fail + // keeps its re-based start: its moves only follow the fingers. + if (this.state === 'possible' && this.pending == null) { + if (Math.abs(span - this.initialSpan) >= this.hysteresis) { + // UIKit measures from the start moved by the hysteresis, and its + // first velocity spans from the touch-down (scale 1) to here. + this.initialSpan += + Math.sign(span - this.initialSpan) * this.hysteresis; + this.scaleValue = span / this.initialSpan; + this.tracker.add(this.scaleValue, 0, event.time); + this.setState('began'); + } + return; + } + this.scaleValue = span / this.initialSpan; + // Every move of the fingers is a sample: a span that holds while they + // move reads as no change, so the velocity never goes stale. + this.tracker.add(this.scaleValue, 0, event.time, {isMotion: true}); + if (isActive(this.state)) { + this.setState('changed'); + } + } + + protected override touchesEnded(): void { + this.setState(isActive(this.state) ? 'ended' : 'failed'); + } + + protected override reset(): void { + this.initialSpan = 0; + this.scaleValue = 1; + this.tracker.reset(); + } + + private span(): number { + const [a, b] = this.trackedTouches; + return a == null || b == null ? 0 : distance(a, b); + } +} diff --git a/packages/core/src/gestures/PressTracker.ts b/packages/core/src/gestures/PressTracker.ts new file mode 100644 index 0000000000000..06b17ab4ac0fb --- /dev/null +++ b/packages/core/src/gestures/PressTracker.ts @@ -0,0 +1,394 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file PressTracker.ts + * @input Touches the arena delivers to a pressable view (TouchConsumer) + * @output Exports PressTracker (UIControl's touch tracking), its options, the + * control event type, and PRESS_PROFILES (a control's and a table cell's + * tracking and release clock) + * @position View-level consumer of packages/core/src/gestures: what a + * UIControl does with the touches no gesture recognizer took + * + * UIControl semantics, event for event: + * + * - `touchDown` when the finger lands; the highlight comes on in the same + * frame, unless the control sits in a scroller, where it waits + * `contentTouchDelay` (UIScrollView.delaysContentTouches) so a flick + * paints nothing. + * - `touchDragInside` / `touchDragOutside` on every move, `touchDragExit` / + * `touchDragEnter` when the finger crosses the control's bounds EXPANDED + * by `trackingExpansion` (a fingertip dragged a little off a button is + * still on it). The highlight follows inside / outside. + * - `touchUpInside` / `touchUpOutside` at the lift. A tap quicker than the + * scroller delay shows the highlight at the lift for `quickTapFlash`. + * - `touchCancel` when a recognizer took the touch (`cancelsTouchesInView`) + * or the browser took it to scroll: the highlight goes at once, with no + * release, one frame before the content moves. + * + * Two profiles (PRESS_PROFILES) cover what UIKit does: + * + * - `control` (a button, a switch): as above. + * - `cell` (a table or collection row): the highlight waits the + * content-touch delay inside a scroll view and comes on at once outside one + * (`spec:AST-094` FR11), travel past the pan hysteresis cancels the press + * for good (the scroll view took the touch), and a quicker tap highlights + * at the lift straight into the release, with no flash. + * + * And UIControl's iOS 6 rule: a single-finger single tap recognizer on an + * ANCESTOR does not beat the control's own action for a touch on it (the + * default veto; `vetoesAncestorRecognizer` replaces it, as a slider vetoes a + * pan along its track). + * + * The bounds are read once, when the finger lands; nothing reads layout + * during the press. Activation stays the browser's click, which the arena + * swallows for a touch a recognizer took. + */ + +import {UIKIT_CONTROL, UIKIT_PAN, UIKIT_PRESS} from '../motion/uikit/gestures'; +import {timing} from '../motion/uikit/timing'; +import {travel} from './geometry'; +import type {GestureRecognizer, SelectionLock} from './GestureRecognizer'; +import {TapGestureRecognizer} from './TapGestureRecognizer'; +import type {GestureTouch, TouchConsumer} from './types'; + +export type ControlEvent = + | 'touchDown' + | 'touchDragInside' + | 'touchDragOutside' + | 'touchDragEnter' + | 'touchDragExit' + | 'touchUpInside' + | 'touchUpOutside' + | 'touchCancel'; + +export interface PressBounds { + readonly left: number; + readonly top: number; + readonly right: number; + readonly bottom: number; +} + +export interface PressTrackerOptions { + /** The control's bounds in client px, read when a finger lands. */ + getBounds: () => PressBounds | null; + /** The view the tracker belongs to; recognizers on it are not ancestors. */ + view?: unknown; + onControlEvent?: (event: ControlEvent, touch: GestureTouch) => void; + onHighlightChange?: (isHighlighted: boolean) => void; + /** + * A continuous-tracking control (a held stepper) keeps its touch once its + * press is delivered: the host cancels the browser's pan for that finger, + * as UIKit's `touchesShouldCancel(in:)` is false for a tracking control. + * Its element then gets a blocking `touchmove` listener, so a scroll that + * starts on it waits on the main thread: never set it on plain buttons, + * links, cards or list cells. + * @default false + */ + keepsTouch?: boolean; + /** + * For a control whose own record defines a hold action (a stepper's + * hold-repeat): pass `lockSelection` (gestures/selectionLock.ts) to turn + * text selection and the callout off for each press, where the primary + * pointer is coarse (`spec:AST-094` FR41). A control whose hold does + * nothing passes none. @default undefined + */ + selectionLock?: SelectionLock; + trackingExpansion?: number; + /** + * When the highlight waits `contentTouchDelay`: inside a scroller (a + * control or a cell), always, or never. @default 'inScrollers' + */ + highlightDelay?: PressHighlightDelay; + /** + * Travel (px) past which the press is cancelled for good, with no + * re-highlight on the way back (a cell whose scroll view took the touch). + * Unset, the finger is tracked in and out of the expanded bounds. + */ + cancelTravel?: number; + /** + * How long a tap quicker than the delay shows the highlight at the lift, + * ms; 0 highlights and releases at once. @default the quick-tap flash + */ + quickTapFlash?: number; + /** Which ancestor recognizers the control's own action beats. */ + vetoesAncestorRecognizer?: (recognizer: GestureRecognizer) => boolean; + /** @default () => true */ + isEnabled?: () => boolean; + /** + * Whether this touch is braking a scroll the finger set moving (see + * GestureHost.isScrollBraking): a native list highlights nothing for one, + * and the control does not track it at all. + */ + isBraking?: (touch: GestureTouch) => boolean; +} + +const PRESS = UIKIT_PRESS; + +export type PressHighlightDelay = 'inScrollers' | 'always' | 'never'; + +/** + * How a kind of pressable tracks a finger and how long its release takes. + */ +export interface PressProfile { + readonly highlightDelay: PressHighlightDelay; + readonly cancelTravel?: number; + readonly quickTapFlash: number; + /** The release (`fading`) clock a painter uses, ms. */ + readonly releaseMs: number; +} + +export const PRESS_PROFILES = { + /** UIControl: same-frame highlight outside scrollers, 70 pt tracking. */ + control: { + highlightDelay: 'inScrollers', + quickTapFlash: PRESS.quickTapFlash.value, + // The shared press styles release every surface on the deselect clock + // until a control runs its own release (UIButton's 250 ms), so a + // painter's `fading` never outlasts the fade it shows. + releaseMs: timing.press.highlightOutMs, + }, + /** + * A table or collection cell: inside a scroll view it highlights after the + * content-touch delay, outside one at once (AST-094 FR11: the delay applies + * only inside scroll views); cancelled by the scroll. + */ + cell: { + highlightDelay: 'inScrollers', + cancelTravel: UIKIT_PAN.hysteresis.value, + quickTapFlash: 0, + releaseMs: timing.press.highlightOutMs, + }, +} as const satisfies Record; + +export type PressProfileName = keyof typeof PRESS_PROFILES; + +/** UIButton's rule: an ancestor's single-finger single tap. */ +function vetoesSingleTap(recognizer: GestureRecognizer): boolean { + return ( + recognizer instanceof TapGestureRecognizer && + recognizer.numberOfTapsRequired === 1 && + recognizer.numberOfTouchesRequired === 1 + ); +} + +export class PressTracker implements TouchConsumer { + isHighlighted = false; + private isKept = false; + isTracking = false; + isTouchInside = false; + trackingExpansion: number; + + private readonly options: PressTrackerOptions; + private touchId: number | null = null; + private bounds: PressBounds | null = null; + private highlightTimer: ReturnType | null = null; + private flashTimer: ReturnType | null = null; + /** A pending highlight whose finger travelled: it will never paint. */ + private isPendingDead = false; + + constructor(options: PressTrackerOptions) { + if (options.keepsTouch === true) { + // Only the tracked touch whose press was delivered is kept. + this.keepsTouch = touchId => this.isKept && this.touchId === touchId; + } + this.options = options; + this.trackingExpansion = + options.trackingExpansion ?? UIKIT_CONTROL.trackingExpansion.value; + } + + /** + * Whether the control takes touches now: a disabled one gives the host no + * reason to listen (`spec:AST-094` FR24). + */ + isEnabled(): boolean { + return this.options.isEnabled?.() !== false; + } + + touchBegan(touch: GestureTouch): void { + if ( + this.touchId != null || + this.options.isEnabled?.() === false || + this.options.isBraking?.(touch) === true + ) { + return; + } + this.clearFlash(); + this.touchId = touch.id; + this.bounds = this.options.getBounds(); + this.isTracking = true; + this.isTouchInside = true; + this.isPendingDead = false; + this.emit('touchDown', touch); + const delay = this.options.highlightDelay ?? 'inScrollers'; + const delays = + touch.pointerType === 'touch' && + (delay === 'always' || + (delay === 'inScrollers' && touch.scroll?.isInsideScroller === true)); + if (delays) { + this.highlightTimer = setTimeout(() => { + this.highlightTimer = null; + if (this.isTouchInside) { + this.setHighlighted(true); + } + }, PRESS.contentTouchDelay.value); + } else { + this.setHighlighted(true); + } + } + + touchMoved(touch: GestureTouch): void { + if (touch.id !== this.touchId) { + return; + } + const cancelTravel = this.options.cancelTravel; + if (cancelTravel != null && travel(touch) > cancelTravel) { + // A cell's scroll view took the touch: the press is over. + this.touchCancelled(touch); + return; + } + if (this.highlightTimer != null && travel(touch) > PRESS.slop.value) { + // A finger travelling before the delay ran out is scrolling: the + // press it began never paints, as a flick on a list paints nothing. + clearTimeout(this.highlightTimer); + this.highlightTimer = null; + this.isPendingDead = true; + } + const inside = this.contains(touch); + if (inside !== this.isTouchInside) { + this.isTouchInside = inside; + this.emit(inside ? 'touchDragEnter' : 'touchDragExit', touch); + if (this.highlightTimer == null && !this.isPendingDead) { + this.setHighlighted(inside); + } + } + this.emit(inside ? 'touchDragInside' : 'touchDragOutside', touch); + } + + touchEnded(touch: GestureTouch): void { + if (touch.id !== this.touchId) { + return; + } + this.isKept = false; + const inside = this.contains(touch); + const wasPending = this.highlightTimer != null; + this.clearHighlightTimer(); + this.emit(inside ? 'touchUpInside' : 'touchUpOutside', touch); + const flash = this.options.quickTapFlash ?? PRESS.quickTapFlash.value; + if (wasPending && inside) { + // Quicker than the delay: the highlight shows at the lift, for the + // flash, or (a cell) straight into the release. + this.setHighlighted(true); + if (flash > 0) { + this.flashTimer = setTimeout(() => { + this.flashTimer = null; + this.setHighlighted(false); + }, flash); + } else { + this.setHighlighted(false); + } + } else { + this.setHighlighted(false); + } + this.finish(); + } + + touchCancelled(touch: GestureTouch): void { + if (touch.id !== this.touchId) { + return; + } + this.isKept = false; + this.clearHighlightTimer(); + this.emit('touchCancel', touch); + this.setHighlighted(false); + this.finish(); + } + + allowsAncestorRecognizer(recognizer: unknown): boolean { + const candidate = recognizer as GestureRecognizer; + if (candidate.view != null && candidate.view === this.options.view) { + return true; + } + const vetoes = this.options.vetoesAncestorRecognizer ?? vetoesSingleTap; + return !vetoes(candidate); + } + + /** Drop any highlight and timers (the control unmounted or was disabled). */ + dispose(): void { + this.clearHighlightTimer(); + this.clearFlash(); + this.setHighlighted(false); + this.finish(); + } + + private contains(touch: GestureTouch): boolean { + const bounds = this.bounds; + if (bounds == null) { + return true; + } + const margin = this.trackingExpansion; + return ( + touch.x >= bounds.left - margin && + touch.x <= bounds.right + margin && + touch.y >= bounds.top - margin && + touch.y <= bounds.bottom + margin + ); + } + + private emit(event: ControlEvent, touch: GestureTouch): void { + // A finger where the primary pointer is fine gets press feedback only. + if (touch.isFeedbackOnly === true) { + return; + } + this.options.onControlEvent?.(event, touch); + } + + private setHighlighted(value: boolean): void { + if (value && this.touchId != null) { + // Delivered: the control keeps this touch until it ends (also when a + // quick tap's flash is still showing as the press arrives). + this.isKept = true; + } + if (this.isHighlighted === value) { + return; + } + this.isHighlighted = value; + this.options.onHighlightChange?.(value); + } + + /** + * Present only for a continuous-tracking control created with + * `keepsTouch: true` (a held stepper): once its press is delivered, the + * browser's pan must not take the touch. A plain pressable leaves it + * undefined and keeps passive listeners, so a scroll starting on it never + * waits on the main thread. + */ + readonly keepsTouch?: (touchId: number) => boolean; + + /** The selection lock of a control whose hold acts (FR41); see the option. */ + get selectionLock(): SelectionLock | undefined { + return this.options.selectionLock; + } + + private clearHighlightTimer(): void { + if (this.highlightTimer != null) { + clearTimeout(this.highlightTimer); + this.highlightTimer = null; + } + } + + private clearFlash(): void { + if (this.flashTimer != null) { + clearTimeout(this.flashTimer); + this.flashTimer = null; + } + } + + private finish(): void { + // A highlight shown at the lift (a quick tap's flash) keeps nothing. + this.isKept = false; + this.touchId = null; + this.isTracking = false; + this.isTouchInside = false; + this.bounds = null; + } +} diff --git a/packages/core/src/gestures/README.md b/packages/core/src/gestures/README.md new file mode 100644 index 0000000000000..9ca84bb2d8e83 --- /dev/null +++ b/packages/core/src/gestures/README.md @@ -0,0 +1,565 @@ +# Gestures + +A gesture recognizer system with UIKit's semantics, for the touch +presentations of Astryx core components. Components import it from +`../gestures`, and Lab through internal workspace imports; nothing here is +exported from the package. + +UIKit is the reference throughout: the same recognizer states, the same +conflict rules, the same thresholds (from `../motion/uikit/gestures.ts`, each +one sourced or marked as a guess awaiting measurement), and touch handling +that behaves like `UIControl` and `UIScrollView`. The rest of this page says +where the web forces a difference. + +## Using it from a component + +```tsx +import {useRef} from 'react'; +import {PanGestureRecognizer, useGestureRecognizer} from '../gestures'; + +function Sheet() { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'down'}), + pan => { + // Runs inside the input event: write the transform, render nothing. + const element = ref.current; + if (element == null) return; + if (pan.state === 'ended') { + settle(element, pan.translation().y, pan.velocity().y); + } else { + element.style.transform = `translateY(${pan.translation().y}px)`; + } + }, + ); + return
…
; +} +``` + +- The recognizer is created once per component and keeps its identity, so + relationships set up when it is created (`requireToFail`, + `recognizeSimultaneouslyWith`) keep working. +- The listener is read through a ref. Passing a new function on each render + attaches nothing new, and a gesture never causes a React render. +- The recognizer attaches in a layout effect, so the element's `touch-action` + is set before the browser can see a finger. It follows the element behind + the ref: a node rendered after mount, or swapped behind the same ref (a + trigger rendered `as` another element), is attached at that commit. +- `isEnabled: false` mid-gesture ends the gesture without calling the + listener, since the handler that turned it off owns that end (no consumer + callback from an Effect). A handler that wants UIKit's `.cancelled` calls + `recognizer.setEnabled(false)` itself, which reports it at once. A + recognizer that required it to fail (a single tap waiting on a double + tap) resolves just after that render, and its listener reports as usual; + so does one waiting on a recognizer whose node a render replaced. After + unmount the listener hears nothing, not even the cancellation of a + gesture the unmount ended. +- `actionGroup: true` attaches the element as an action group, sibling + actions a held pointer slides across, which the host serves where the + primary pointer is fine too (`family:action-groups` FR5). Changing it + attaches the element again. +- `usePressTracker(ref, options)` gives an element `UIControl` touch tracking + (see below). It follows its element the same way, takes `isBraking` (by + default the host's: a touch that stops a flick presses nothing), and keeps + one tracker for the component's life: it reads `trackingExpansion`, + `highlightDelay`, `cancelTravel` and `quickTapFlash` (and the ref itself) + as they are now, so a change, even mid-press, never drops the press in + progress. Unmounting or losing its element calls neither + `onHighlightChange` nor `onControlEvent`, and neither does disabling the + control under a finger: the press ends, so the handler that disables it + clears its own highlight. + +Outside React, `getGestureHost(document).attach(element, recognizer)` does +the same and returns a detach function. `getGestureHost(document).fingersDown()` +is how many fingers are on the screen (the last touch event's `touches`, +wherever they landed): a recognizer that claims only a lone finger checks it +in `shouldBegin`, so two fingers stay the browser's pinch. + +## Recognizers + +| Recognizer | UIKit | Kind | Notes | +| -------------------------------- | ---------------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `TapGestureRecognizer` | `UITapGestureRecognizer` | discrete | `numberOfTapsRequired`, `numberOfTouchesRequired`; fails past 45 px of travel, on extra fingers, a hold over 1.5 s, a late or far next tap | +| `LongPressGestureRecognizer` | `UILongPressGestureRecognizer` | continuous | `minimumPressDuration` 500 ms, `allowableMovement` 10 px (a distance, not per axis), `numberOfTapsRequired`; a zero duration begins on the touch itself (a selection tracker over a group of actions) | +| `PanGestureRecognizer` | `UIPanGestureRecognizer` | continuous | min/max touches, `direction` (axis or one-way) with `axisRatio`, `lockAxis`, `scrollEdge`; `translation()`, `totalTranslation()`, `velocity()` | +| `SwipeGestureRecognizer` | `UISwipeGestureRecognizer` | discrete | recognized during the motion; one or more directions, `numberOfTouchesRequired`; 50 px along, at most 50 px across, 500 ms | +| `PinchGestureRecognizer` | `UIPinchGestureRecognizer` | continuous | `scale` (settable, to re-base it), `velocity` in scale per second, `focalPoint()` | +| `RotationGestureRecognizer` | `UIRotationGestureRecognizer` | continuous | `rotation` in radians, unwrapped, clockwise positive; `velocity` in rad/s | +| `ScreenEdgePanGestureRecognizer` | `UIScreenEdgePanGestureRecognizer` | continuous | `edges` including `leading` / `trailing` (resolved against the writing direction); fails as soon as the touch lands outside | +| `HoverGestureRecognizer` | `UIHoverGestureRecognizer` | continuous | a mouse or a hovering pen; silent for fingers | + +A recognizer created with `claimsTouches: false` is a watcher (a tab bar +that minimizes while the page scrolls): it never claims, so its element gets +only passive listeners and a scroll never waits for it; it prevents no other +recognizer and none prevents it; and it follows a touch the browser takes +for a scroll to the lift. It adds nothing to the element's derived +`touch-action`. + +Every recognizer also has UIKit's switches: `isEnabled` (use `setEnabled`; +turning it off mid-gesture cancels), `cancelsTouchesInView` (default true), +`delaysTouchesBegan` (default false), `delaysTouchesEnded` (default true), +`allowedPointerTypes`, and `requiresExclusiveTouchType`. + +### What the pan reports + +The pan reads its numbers the way `UIPanGestureRecognizer` does, as measured +in the UIKit lab: + +- It begins once the centroid has moved `hysteresis` (10 px) along either + axis, not as a distance. +- `translation()` leaves out the travel the threshold took: a slow drag reads + zero when the pan begins, and a step that overshoots keeps only the + overshoot. A dragged surface does not jump. `totalTranslation()` is the + movement since the fingers landed, for a surface that wants it. +- `axisRatio` (for a pan limited to an axis): the movement along the axis + must exceed `axisRatio` times the movement across it to begin; movement + across that exceeds the movement along it fails the pan; in between it + waits. +- `velocity()` is UIKit's own filter (see Velocity). The lift updates + neither number; a finger that rested past the stillness interval releases + at 0. + +### States + +``` +possible ─► began ─► changed* ─► ended | cancelled continuous +possible ─► ended discrete ("recognized") +possible ─► failed +``` + +Listeners (UIKit's target-action) run for `began`, `changed`, `ended` and +`cancelled`. `addTransitionListener` also reports `failed` and the return to +`possible`. A recognizer in a terminal state ignores the touches it still +holds and returns to `possible` when all of them have ended. + +## Conflicts: the arena + +A new touch goes to every recognizer on its hit path, innermost first, and +each one that accepts it tracks it until it ends. Then: + +- **Exclusivity.** Two recognizers that share a touch are never active + together unless one allows it (`recognizeSimultaneouslyWith`, or the + `shouldRecognizeSimultaneously` delegate). The first to recognize fails + the other possible ones. A recognizer that tries to begin while a + conflicting one is active fails instead. When two qualify on the same + input, the innermost one wins. +- **Failure requirements.** `a.requireToFail(b)` keeps `a` in `possible` + until `b` fails, then lets `a` recognize. If `b` recognizes, `a` fails. A + recognizer that is not involved in the current touches holds nothing up. + The delegates `shouldRequireFailureOf` and `shouldBeRequiredToFailBy` + express the same thing dynamically. +- **Delegates.** `shouldBegin`, `shouldReceiveTouch`, and `canPrevent` / + `canBePrevented` on subclasses. The arena's plumbing (touch delivery, + approved transitions, a pending recognition) is `protected`: a subclass + overrides the `touches*` handlers and sets `state`, and only the gesture + system drives the rest. + +```ts +// Single tap waits for the double tap to fail, as in UIKit. +single.requireToFail(double); + +// Inner horizontal scroller yields to the back-swipe in the edge band only: +// the edge pan fails at touch-down anywhere else, so nothing waits. +innerPan.requireToFail(edgePan); +``` + +## Touches in the view: `PressTracker` + +A pressable control's view-level touch handling, with `UIControl` +semantics: + +- `touchDown`, `touchDragInside` / `touchDragOutside`, `touchDragEnter` / + `touchDragExit`, `touchUpInside` / `touchUpOutside`, `touchCancel`. +- "Inside" is the control's bounds expanded by `trackingExpansion`, so a + fingertip that slides a little off a button is still on it. +- The highlight comes on in the same frame. Inside a scroller it waits + `contentTouchDelay` (`UIScrollView.delaysContentTouches`), so a flick + paints nothing. A tap quicker than that delay shows the highlight at the + lift for `quickTapFlash`. +- A recognizer that takes the touch (`cancelsTouchesInView`), or the browser + taking it to scroll, cancels the press with no release. +- `UIControl`'s iOS 6 rule applies: an ancestor's single-finger single tap + does not beat the control's own action for a touch on it. + `vetoesAncestorRecognizer` replaces that rule (a slider vetoes a pan along + its track). + +The browser's click stays the control's activation, so keyboard and +assistive-technology activation keep working. + +### Press profiles + +`PRESS_PROFILES` holds the two ways UIKit tracks a press: + +- `control` (a button, a switch): the highlight comes on in the same frame + outside a scroller, the finger is tracked 70 pt beyond the bounds, and + the release takes the list's deselect clock, as every surface's does + until a control runs its own (UIButton's 0.25 s); +- `cell` (a table or collection row): the highlight waits the content-touch + delay inside a scroll view and comes on at once outside one + (`spec:AST-094` FR11), travel past the pan hysteresis cancels the press for + good, a quicker tap highlights at the lift straight into the release, and + the release takes the list's deselect clock. + +`paintPress(element, 'on' | 'held' | 'fading' | 'none', {releaseMs})`, in +`utils/pressFeedback.ts`, is the one writer of `data-astryx-press`. It owns the release timer and writes +only over values the caller may replace (by default none, `on` and +`fading`), so a `held` row survives presses until its owner releases it +with `onlyFrom: ['held']`. + +### Rows and cards + +A list of a thousand rows cannot pay for a hook and four listeners per row. +The document press controller (`utils/pressFeedback.ts`) presses the +innermost enabled `[data-astryx-pressable]` under each touch and paints it, +with no listener per row; a touch that brakes a scroll the finger set moving +presses nothing. + +## What the browser is told + +- **Only where the primary pointer is coarse.** Recognizers attach, and the + host registers listeners, only while `TOUCH_QUERY` matches + (`spec:AST-094` FR19, FR24; `coarsePointer.ts`). Where the primary pointer + is fine, an attach is recorded and nothing else happens: no document, + window or element listener, no style below, nothing delivered, so a + finger on a touchscreen laptop gets the browser's defaults. The host + follows the query (a tablet docking or undocking a mouse): turning coarse + installs it and styles every element; turning fine cancels every touch in + flight and removes it all. A development warning names the first attach + under a fine pointer: attach touch work under a touch presentation. The + one carve-out is a recognizer or press tracker that reads each event's + pointer itself, as `spec:AST-057`'s row swipe does, or a press tracker that + paints press feedback (an FR19 exception for press feedback only, pending + in `spec:AST-094`'s amendment): attached with + `perEventPointer: true` (an option of `attach`, `attachConsumer`, + `useGestureRecognizer` and `usePressTracker`), it attaches on any pointer + with no warning, while everything else beside it stays inactive. A + per-event recognizer gets the host's listeners, its deliveries and its own + element's style. Where the pointer is fine, a per-event press tracker gets + only passive `touchstart` and `pointerdown` listeners on its own element + and no style; the host's document and window listeners are added at a + finger's press on it and removed once that press ends. It paints that + press and does nothing else: it reports no control event, keeps nothing + from the browser, which still pans, and takes no mouse or pen press. + Even where it is coarse, the host listens only while something enabled + could take a touch: a disabled recognizer or consumer registers nothing + (FR24), and after a consumer's `isEnabled()` changes, `host.update(element)` + re-syncs. So a component that activates through a recognizer (a tap that + pages, a press that opens) also activates from the browser's own `click` + where the pointer is fine, and a mouse, a trackpad and the keyboard keep + working; its tests declare a phone's pointer for the touch path + (`coarsePrimaryPointer()` in `__tests__/pointerQuery.ts`). +- **`touch-action`.** Each element gets the intersection of what its + recognizers leave the browser, and never more than the element's own + value allows: a caller's explicit `touch-action` (a trigger's `none`) + stays in force, as the intersection of both. It is set inline on attach + and restored on detach (opt out with `manageStyle: false`): + + | Recognizers on the element | `touch-action` | + | ------------------------------------------------- | ------------------- | + | tap, long press | `manipulation` | + | horizontal pan or swipe | `pan-y pinch-zoom` | + | horizontal edge pan on a strip within its band | `pan-y pinch-zoom` | + | vertical pan or swipe | `pan-x pinch-zoom` | + | one-way pan, pan with `scrollEdge` | left as is (`auto`) | + | edge pan on a view wider than its band | left as is (`auto`) | + | free pan | `pinch-zoom` | + | pinch or rotation | `pan-x pan-y` | + | pinch or rotation together with a free pan | `none` | + | a watcher (`claimsTouches: false`), any direction | left as is (`auto`) | + + A one-way pan leaves the browser its own axis too: a toast dragged away + from its edge scrolls the page. It claims its direction by cancelling + `touchmove` when it begins. + + `pinch-zoom` stays in the value unless a pinch or rotation recognizer is + attached, so the page stays zoomable (WCAG 1.4.4). A long press with a + hold also turns off the iOS callout and text selection on its element for + its press; one at 0 ms (a touch-down detector) or given + `selectionLock: null` leaves them to the browser. + + While a finger or a pen is down, the host turns them off too only where + the hold does something (`spec:AST-094` FR41): on the element of a + recognizer created with a `selectionLock` (pass `lockSelection`; a + slider's thumb, a segmented + control's or a switch's drag) that would take the touch (enabled, and + accepting its pointer), restored when the contact ends. The host calls + the recognizer's lock and never imports one, so a component with no such + hold does not bundle it. A press tracker created with a `selectionLock` + (a control whose own record defines a hold action, such as a stepper's + hold-repeat) gets the same for each press. A press-tracked + control or row whose hold does nothing, and the document press + controller's surfaces (`usePressFeedback` rows), keep the browser's + long-press selection; a tap never selects. A non-text input under the finger inside such an element (a checkbox, a radio, a switch's input) is locked too: WebKit's own stylesheet makes every input selectable, which no ancestor's lock reaches. A text field keeps its selection, since a held finger places its caret. Only where the + primary pointer is coarse: a mouse never takes the lock, nor a finger or a + pen where the primary pointer is fine (a touchscreen laptop, a desktop pen + tablet), so desktop selection is unchanged (`spec:AST-094` FR19); content + under a sheet's or a pager's pan stays selectable. + +- **Claiming.** When a recognizer recognizes, it owns its touches. Each + element with a recognizer that can claim a moving touch (a pan, a swipe, a + long press that then drags, a pinch, a rotation) has a non-passive + `touchmove` listener that cancels the event only while one of its touches + is owned. A touch nobody owns scrolls exactly as it would without the + system. Taps claim only at the lift, and a recognizer that never claims + (`claimsTouches` false, such as a pull that only watches the scroll) never + owns a touch, so an element with only those, or only a press consumer, + gets no such listener (see the known limit below). A press tracker created + with `keepsTouch: true` (opt-in, for a continuous-tracking control such + as a held stepper; never plain buttons, links, cards or list cells) gets + one: once its press is delivered its finger's moves are cancelled, so the + browser's pan never takes the held control (UIKit's + `touchesShouldCancel(in:)` is false for a tracking control). Where the + pointer is coarse its element also gets `touch-action: manipulation`, as a + tap recognizer's does (`spec:AST-094` FR18), so taps in quick succession + never zoom. A scroll that starts on such a control waits on the main + thread; a move before the press is delivered still scrolls. The listener the host + adds to a finger's target at touch-down, for a target that leaves the + document mid-gesture, is passive too unless a recognizer on that touch can + claim while it moves. +- **Claim at the begin.** A pan claims its touch only as it begins, at + UIKit's 10 pt (`spec:AST-094` FR19), before WebKit commits a slow drag to + a scroll at 10–11 px; a drag the browser took first stays the browser's + (FR14). One exception: a `scrollEdge` pan cancels its finger's moves from + the first one while every scroller under the finger is at its edge and + the finger pulls away from it, the pan's own way (the browser could only + rubber-band there); it still begins at 10 pt. It holds only while that is + the one finger on the screen: with a second down anywhere, the moves stay + the browser's, for its pinch (FR10). + Measured in Mobile Safari (iOS 26.2): `touchmove` arrives from the first + pixel; on a scrollable element WebKit commits to scrolling at 10 to 11 px + (the last cancelable `touchmove` at 10 px), and cancelling every + `touchmove` from 8 px keeps the page still for the whole gesture. Under + `pan-y` it scrolls only drags steeper than about 2:1 to 3:1, once the + vertical travel reaches about 10 px. A pan that competes with native + scrolling therefore decides at 10 px or less. +- **The browser's claim.** `pointercancel` on a finger, or a scroll above + the touch once the finger has travelled the slop (the browser starts a pan + no earlier), means the browser took the finger; a scroll under a finger + that has not moved (scrollIntoView, a transcript pinned to its end, scroll + anchoring) is the page's own and leaves the press and its click alone. So + does its first `touchmove` + that can no longer be cancelled, when the browser could pan along the + finger's main axis. (Chromium also stops asking after one unconsumed + `touchmove` when `touch-action` keeps it from panning that way, as for a + horizontal drag under `pan-y`; that is not a claim.) Possible recognizers that share it fail, active ones are + cancelled, and the view's press is cancelled. +- **Clicks.** When a touch's gesture cancelled the view's touches, its + `touchend` is cancelled (no click and no compatibility mouse events) and a + one-shot guard swallows the next trusted click. A click held back for a + recognizer that may still recognize (a double tap between taps) is + swallowed at once and replayed on the original target if the recognizer + fails. Clicks with `detail === 0` (keyboard, assistive technology, code) + always pass. +- **Context menus.** A `contextmenu` while a recognizer owns a touch is + cancelled, so a long press never also opens the browser's menu. +- **Focus.** Losing window focus or hiding the page cancels every touch in + flight, as UIKit does when the app resigns active. + +### Native scrolling + +`scrollEdge` makes a pan share its axis with the native scrollers under the +finger. This is the sheet over its own list: the pan begins only if nothing +under the touch could scroll for that movement, the finger landed at the +edge and pulls away from it. The browser has not committed yet, so beginning +claims the touch and the content never moves. From the first move that +pulls away from such an edge the pan already holds the finger, so its moves +are cancelled before it begins and the page does not rubber-band either. + +Who gets a drag is decided once, when it begins, as in UIKit: a drag that +began as a scroll stays the scroll view's for its whole length (it scrolls +to the top, then rubber-bands), and never changes hands midway +(`spec:AST-094` FR14). + +"Could scroll" follows the browser's own rules: `touch-action` between the +touch and each scroller, scroll chaining stopped by `overscroll-behavior`, +and RTL `scrollLeft` ranges, along the flat tree (content slotted into a +shadow-tree scroller scrolls with it, as the hit path does). + +### Holding the page still + +A caller that needs the page still holds it through the host: +`getGestureHost(document).holdPageScroll(decide)` returns the release +(`useScrollLock` does not use it). While any hold is active, and only where +the primary pointer is coarse, the host listens to the document with a +non-passive `touchmove`. Once per one-finger touch that no recognizer owns, +after the +finger clears a 5 px slop (under WebKit's 10–11 px scroll claim), it asks +`decide(origin, dx, dy)` whether that pan would scroll the page, `origin` +being the innermost node under the finger (the composed path's first node), +and cancels the touch's moves for its whole length if so. A second finger is +never cancelled, so pinch-zoom keeps working. With no hold, the listener is +gone and every touch scrolls as without the system. + +### Multiple fingers + +A drag a recognizer has claimed keeps its finger: a second finger never +cancels or redirects it (`spec:AST-094` FR10), as in UIKit, where a pan +ignores touches beyond its maximum. Before any claim, a second finger landing +anywhere (often on a plain row no recognizer is on) fails a pan limited to +one finger, so the two fingers zoom the page (WCAG 1.4.4) instead of the pan +claiming them midway. + +## Time + +Every input carries its own timestamp (`event.timeStamp`), and timers are +deadlines on that same clock. Before an input is processed, every timer due +by its timestamp runs first, at its own deadline. A long press therefore +begins at exactly touch-down + 500 ms even when the main thread was blocked +and the timer callback ran late. Tests drive the whole system with explicit +timestamps and no real time. + +## Velocity + +`VelocityTracker` reports px per second, UIKit's unit, and is the one +velocity tracker every continuous recognizer uses +(`architecture:interaction-modality` INV10, `spec:AST-100` FR1, FR2; no +alternative estimators). One filter, iOS's measured blend, updated on each +moved sample (a sample at the newest position is not motion): + +- A pan's velocity is 0.2 times the newest interval's velocity plus 0.8 times + the previous interval's (an interval runs from one moved sample to the + next; the touch-down counts as the first). With one interval so far it is + that interval, as UIKit reports it. It reproduces every value UIKit + reported on six recorded touch streams, and like UIKit's it reacts a frame + late to an acceleration. +- A scrolling surface's release (paging, a wheel) is 0.25 times the newest + pan value plus 0.75 times the previous one: a pan's `scrollVelocity()`. + Everything else reads `velocity()`. + +A release reads 0 when the finger stayed still for longer than the stillness +interval (`UIKIT_GESTURES.velocity.stillness`) before lifting, the one +difference from iOS, whose filter never decays and so flings after a rest; +and 0 with fewer than two moved samples. The pan passes its lift's event time +to the tracker. + +## Haptics + +`playHaptic(kind)` vibrates where `navigator.vibrate` exists, with one +pattern per UIKit feedback kind. iOS Safari has no vibration API. It plays a +light impact only when a real tap toggles an ``, and WebKit requires a trusted event for it. `attachTapHaptic(label)` +puts a hidden switch inside a label the control renders as its tap target, +so every real tap on it plays that impact; focus the activation gives the +switch goes straight back. A label tied to its control by `for`, or whose +control is itself a switch, gets no switch; a label wrapping its own control +gets one, which hands each tap on to that control. Nothing can play a haptic later +from script, so mid-drag haptics (detent snaps, picker ticks) are not +possible in iOS Safari. + +## Thresholds + +All numbers live in `../motion/uikit/gestures.ts`. Each one carries either +its source (Apple documentation, or a measurement: the recognizers' own +values read in a UIKit app on the iOS 26.2 simulator and confirmed with +touches, or Mobile Safari measured on the same simulator) or a guess +together with the measurement that will replace it. A unit test fails if an +entry has neither. + +## Touch presentation + +Whether a component presents itself the touch (UIKit) way is answered once, +in `../utils/touchPresentation.ts`: + +- touch behaviours (this system, the press model, UIKit springs and + timings) key on a coarse PRIMARY pointer (`TOUCH_QUERY`); the gesture + host itself installs nothing while it is fine (What the browser is told); +- layout behaviours key on a compact viewport, narrower than the theme's + `md` width point or shorter than 500 px (`COMPACT_QUERY`; behaviour reads + the active theme's points), the closest a size can get to + UIKit's compact size class (a Max-size iPhone in landscape is its one + known miss); +- a presentation that needs both, a sheet in place of a popover, asks for + both (`COMPACT_TOUCH_QUERY`). + +Styles switch on the same conditions by writing the same query as a StyleX +media key (`'@media (pointer: coarse)'`, and the compact-touch query spelled +exactly as `COMPACT_TOUCH_QUERY`, which a test enforces), so the first paint +is right. Behaviour reads +`useTouchPresentation()` (SSR-safe), a component with no interaction to +infer from included (UIKit timings under a finger, its own motion under a +mouse), and a decision taken once reads `readTouchPresentation()`. + +A component that, under a finger, presents a different surface or switches +how an open surface follows input takes a `presentation` prop, and a +caller's explicit value always wins over the touch default +(`architecture:public-component-api` INV16). It resolves that prop with +`usePresentation(presentation, {whenOmitted})` (`../utils/presentation.ts`; +`readPresentation` at one moment): `surface` is the popover or the sheet; +`touchModes` says whether its touch input modes apply (a finger, and not +pinned to `'popover'`), the same on every touch device. An +omitted value means the component's default, `whenOmitted: 'popover'` or +`'touch'` (the popover surface with touch input modes), so it stays +distinguishable from an explicit `'popover'`, which pins the fine surface +and input modes on every device. A style that implements a touch surface or +mode is applied with `touchModes`, not as a bare media key. Touch styling and +touch-only behavior that a component's own record defines keep the touch +presentation above. + +## Known limits + +- **A scroll that begins on a recognizer waits for a busy main thread.** In + Mobile Safari (iOS 26.2, measured), a vertical scroll that starts on an + element with a non-passive `touchmove` listener waits until the current + JavaScript task ends; on a plain row, a pressable row or a tap-only + element it starts at once (within 50 ms) even during a 500 ms task. + Chromium documents the same rule for non-passive listeners. The listener + stays because nothing else keeps UIKit's arbitration: with a passive + listener, or one added during `touchstart` (which Safari ignores for that + touch), Safari hands every diagonal drag on a swipe row to the list's + scroll, 3:1 included. It is never worse than UIKit, where a busy main + thread stalls every scroll. +- **Lists where every row has a horizontal recognizer** (swipe actions on + every row of a mail list): every scroll start waits on the current task in + Safari. Keep list screens' tasks short. A native horizontal `scroll-snap` + reveal inside the row would never wait, but it tracks with scroll physics + rather than UIKit's measured tracking spring; it is a possible opt-in, not + the default. +- **Follow-up:** Chromium keeps 3:1 to 1.5:1 drags as swipes even with a + passive listener (its `touch-action` filter drops a `pan-y` scroll that + starts sideways), so a passive listener there would cost no parity. It + waits on a measured run on Android before any engine-specific choice. + +## Where the web differs from UIKit + +- Once a recognizer owns a touch, the browser cannot take the scroll back + later in the same gesture. UIKit does not either: it decides between a + sheet and its list when the drag begins. +- In Chromium a finger's first `touchmove` arrives only after about 16 px. + Movement below that comes from the finger's pointer events, but the + browser's own decision to scroll happens at that first `touchmove`. + Mobile Safari delivers `touchmove` from the first pixel. +- A tap that holds a plain `', + ); + const consumer = new RecordingConsumer(); + detachers.push(host.attachConsumer(button, consumer)); + const clicks: number[] = []; + button.addEventListener('click', () => clicks.push(1)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', button, [finger], [finger], 0); + // The program scrolls (a transcript pinned to its end, scrollIntoView). + scrollable(list, {scrollTop: 50, scrollHeight: 2000, clientHeight: 300}); + list.dispatchEvent(new Event('scroll')); + expect(consumer.events.some(event => event.startsWith('cancelled'))).toBe( + false, + ); + const end = touch('touchend', button, [finger], [], 120); + expect(end.defaultPrevented).toBe(false); + expect(consumer.events.at(-1)).toMatch(/^ended/); + }); + + it("cancels a finger's moves only when a claimed pen's own stylus touch moves, never a finger nobody owns", () => { + const {pad, list} = mount('
'); + const drawing = new PanGestureRecognizer({name: 'draw'}); + detachers.push( + host.attach(pad, drawing), + host.attach( + list, + new PanGestureRecognizer({name: 'other', direction: 'horizontal'}), + ), + ); + for (const [x, time] of [ + [0, 0], + [5, 16], + [30, 32], + ] as const) { + pointer(time === 0 ? 'pointerdown' : 'pointermove', pad, { + id: 9, + pointerType: 'pen', + x, + y: 0, + buttons: 1, + time, + }); + } + expect(drawing.state).toBe('began'); + // A finger on the list, which no recognizer owns: its moves scroll. + const finger = {id: 4, x: 10, y: 10}; + touch('touchstart', list, [finger], [finger], 40); + const move = touch( + 'touchmove', + list, + [{...finger, y: 40}], + [{...finger, y: 40}], + 56, + ); + expect(move.defaultPrevented).toBe(false); + // The pen's own stylus touch is held: the browser must not pan with it. + const stylus = new Event('touchmove', {bubbles: true, cancelable: true}); + const pen = { + identifier: 77, + clientX: 31, + clientY: 0, + target: pad, + touchType: 'stylus', + }; + Object.defineProperty(stylus, 'changedTouches', {value: [pen]}); + Object.defineProperty(stylus, 'touches', {value: [pen]}); + Object.defineProperty(stylus, 'timeStamp', {value: 60}); + pad.dispatchEvent(stylus); + expect(stylus.defaultPrevented).toBe(true); + pointer('pointerup', pad, { + id: 9, + pointerType: 'pen', + x: 31, + y: 0, + time: 80, + }); + }); + + it('keeps every touchmove listener passive on a finger nothing can claim as it moves', () => { + const {taps, quiet} = mount('
'); + class Watcher extends PanGestureRecognizer { + override get claimsTouches(): boolean { + return false; + } + } + detachers.push( + host.attach(taps, new TapGestureRecognizer({name: 'tap'})), + host.attachConsumer(taps, new RecordingConsumer()), + host.attach(quiet, new Watcher({name: 'watcher'})), + ); + for (const element of [taps, quiet]) { + const added = vi.spyOn(element, 'addEventListener'); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', element, [finger], [finger], 0); + const blocking = added.mock.calls.filter( + ([type, , options]) => + type === 'touchmove' && + (options as AddEventListenerOptions | undefined)?.passive === false, + ); + expect(blocking).toHaveLength(0); + touch('touchend', element, [finger], [], 50); + } + }); + + it('follows the flat tree: content slotted into a shadow scroller scrolls with it', () => { + const {host: shell, item} = mount( + '
row
', + ); + const shadow = shell.attachShadow({mode: 'open'}); + shadow.innerHTML = + '
'; + const scroller = shadow.getElementById('scroller') as HTMLElement; + scrollable(scroller, { + scrollTop: 200, + scrollHeight: 2000, + clientHeight: 300, + }); + const probe = createNativeScrollProbe(item); + expect(probe.canScroll(0, 10)).toBe(true); + expect(probe.isInsideScroller).toBe(true); + }); + + it("writes a touch-action no wider than the caller's own", () => { + vi.useFakeTimers({toFake: ['setTimeout', 'requestAnimationFrame']}); + const restore = touchActionFromData(); + try { + const {trigger, strip} = mount( + '
', + ); + const set = vi.spyOn(trigger.style, 'setProperty'); + const setStrip = vi.spyOn(strip.style, 'setProperty'); + detachers.push( + host.attach(trigger, new LongPressGestureRecognizer({name: 'hold'})), + host.attach( + strip, + new PanGestureRecognizer({name: 'h', direction: 'horizontal'}), + ), + ); + vi.advanceTimersByTime(16); + // manipulation (the hold's) with none, and pan-y pinch-zoom (the + // pan's) with pan-y, allow the caller's own: its rule stays in force. + expect(window.getComputedStyle(trigger).touchAction).toBe('none'); + expect(window.getComputedStyle(strip).touchAction).toBe('pan-y'); + expect(set).not.toHaveBeenCalledWith('touch-action', expect.anything()); + expect(setStrip).not.toHaveBeenCalledWith( + 'touch-action', + expect.anything(), + ); + } finally { + restore(); + vi.useRealTimers(); + } + }); + + it('marks an element with its derived touch-action as it attaches, under a rule any author rule outranks, with no inline write and no style read (spec:AST-094 FR18, DEC-8)', () => { + const restore = touchActionFromData(); + try { + const {box} = mount('
'); + const set = vi.spyOn(box.style, 'setProperty'); + const read = vi.mocked(window.getComputedStyle); + read.mockClear(); + detachers.push( + host.attach(box, new PanGestureRecognizer({direction: 'horizontal'})), + ); + expect(box.getAttribute('data-astryx-touch-action')).toBe( + 'pan-y pinch-zoom', + ); + expect(hostRules(document)).toContain( + ':where([data-astryx-touch-action="pan-y pinch-zoom"])', + ); + expect(set).not.toHaveBeenCalledWith('touch-action', expect.anything()); + expect(read).not.toHaveBeenCalled(); + } finally { + restore(); + } + }); + + it("keeps an author rule in force from the first frame, and writes the intersection inline only where it is narrower than the author's (spec:AST-094 FR18)", () => { + vi.useFakeTimers({toFake: ['setTimeout', 'requestAnimationFrame']}); + const restore = touchActionFromData(); + try { + const {both, strip} = mount( + '
', + ); + const setBoth = vi.spyOn(both.style, 'setProperty'); + const setStrip = vi.spyOn(strip.style, 'setProperty'); + detachers.push( + host.attach(both, new PanGestureRecognizer({direction: 'horizontal'})), + host.attach(strip, new PanGestureRecognizer({direction: 'horizontal'})), + ); + // Before the host reads anything, the author's value stands. + expect(both.getAttribute('data-astryx-touch-action')).toBe( + 'pan-y pinch-zoom', + ); + expect(window.getComputedStyle(both).touchAction).toBe('pan-x pan-y'); + expect(setBoth).not.toHaveBeenCalledWith( + 'touch-action', + expect.anything(), + ); + vi.advanceTimersByTime(16); + // pan-y pinch-zoom with pan-x pan-y allows pan-y: narrower than the + // author's, so it is written. + expect(setBoth).toHaveBeenLastCalledWith('touch-action', 'pan-y'); + // With pan-y it is the author's own value: nothing to write. + expect(setStrip).not.toHaveBeenCalledWith( + 'touch-action', + expect.anything(), + ); + } finally { + restore(); + vi.useRealTimers(); + } + }); + + it('removes the mark when the last recognizer detaches', () => { + const {box} = mount('
'); + const detach = host.attach( + box, + new PanGestureRecognizer({direction: 'horizontal'}), + ); + expect(box.hasAttribute('data-astryx-touch-action')).toBe(true); + detach(); + expect(box.hasAttribute('data-astryx-touch-action')).toBe(false); + }); + + it('adds the rule to the shadow root an element is in', () => { + const {box} = mount('
'); + const shadow = box.attachShadow({mode: 'open'}); + const inner = document.createElement('div'); + shadow.append(inner); + detachers.push(host.attach(inner, new LongPressGestureRecognizer())); + expect(inner.getAttribute('data-astryx-touch-action')).toBe('manipulation'); + expect(hostRules(shadow)).toContain( + ':where([data-astryx-touch-action="manipulation"])', + ); + }); + + it("is never wider than the author's touch-action, at any frame (spec:AST-094 FR18, DEC-8)", () => { + vi.useFakeTimers({toFake: ['setTimeout', 'requestAnimationFrame']}); + const restore = touchActionFromData(); + try { + const {trigger} = mount( + '
', + ); + const set = vi.spyOn(trigger.style, 'setProperty'); + detachers.push( + host.attach(trigger, new LongPressGestureRecognizer({name: 'hold'})), + ); + // The author's none outranks the host's manipulation from the attach + // on, and the read finds nothing to narrow. + expect(window.getComputedStyle(trigger).touchAction).toBe('none'); + vi.advanceTimersByTime(16); + expect(window.getComputedStyle(trigger).touchAction).toBe('none'); + expect(set).not.toHaveBeenCalledWith('touch-action', expect.anything()); + } finally { + restore(); + vi.useRealTimers(); + } + }); + + it('gives an element attached while on screen its touch-action in the attach itself', () => { + const restore = touchActionFromData(); + try { + const {box} = mount('
'); + detachers.push(host.attach(box, new TapGestureRecognizer({name: 'tap'}))); + expect(window.getComputedStyle(box).touchAction).toBe('manipulation'); + } finally { + restore(); + } + }); + + it("reads no computed style while attaching: a caller's inline touch-action applies at once, a stylesheet's is read in the next frame", () => { + vi.useFakeTimers({toFake: ['setTimeout', 'requestAnimationFrame']}); + const restore = touchActionFromData(); + try { + const {trigger, strip} = mount( + '
', + ); + // jsdom keeps no touch-action in a style: this stands in for the + // caller's inline `touch-action: pan-y`. + vi.spyOn(strip.style, 'getPropertyValue').mockImplementation(name => + name === 'touch-action' ? 'pan-y' : '', + ); + const set = vi.spyOn(trigger.style, 'setProperty'); + const setStrip = vi.spyOn(strip.style, 'setProperty'); + const read = vi.mocked(window.getComputedStyle); + read.mockClear(); + detachers.push( + host.attach(trigger, new LongPressGestureRecognizer({name: 'hold'})), + host.attach( + strip, + new PanGestureRecognizer({name: 'h', direction: 'horizontal'}), + ), + ); + // Attaching forces no style recalc over whatever just mounted. + expect(read).not.toHaveBeenCalled(); + // The caller's inline value: pan-y pinch-zoom ∩ pan-y, at once. + expect(setStrip).toHaveBeenLastCalledWith('touch-action', 'pan-y'); + // A stylesheet's value: read once in the next frame, where none is + // already no wider than both allow. + vi.advanceTimersByTime(16); + expect(read).toHaveBeenCalledTimes(1); + expect(set).not.toHaveBeenCalledWith('touch-action', expect.anything()); + } finally { + restore(); + vi.useRealTimers(); + } + }); + + describe('finger scrolls', () => { + it('reads a scroll while a finger that landed inside is down as the finger’s, until the scroller’s scrollend', () => { + const {list, row} = mount('
'); + detachers.push(host.watchFingerScrolls(list)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(true); + // The lift starts the momentum: still the finger's scroll. + touch('touchend', row, [finger], [], 120); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(true); + list.dispatchEvent(new Event('scrollend')); + expect(host.isFingerScroll(list)).toBe(false); + }); + + it('forgets a finger scroll in progress when the host stops listening, so a later program scroll is not the finger’s', () => { + const pointer = pointerQuery(true); + detachers.push(pointer.restore); + const {list, row} = mount('
'); + detachers.push(host.watchFingerScrolls(list)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(true); + // The pointer turns fine mid-scroll: the host stops listening, and its + // scrollend never comes. + pointer.set(false); + pointer.set(true); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + }); + + it('reads a scroll with no finger down as the program’s, even right after a tap', () => { + const {list, row} = mount('
'); + detachers.push(host.watchFingerScrolls(list)); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + // A tap whose click scrolls the list (a scrollTo after the lift). + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + touch('touchend', row, [finger], [], 80); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + }); + + it('reads the page’s own scroll on the document', () => { + const {row} = mount('
'); + detachers.push(host.watchFingerScrolls(document)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + document.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(document)).toBe(true); + touch('touchend', row, [finger], [], 120); + document.dispatchEvent(new Event('scrollend')); + expect(host.isFingerScroll(document)).toBe(false); + }); + + it('does not count a finger down outside the scroller', () => { + const {list, other} = mount( + '
', + ); + detachers.push(host.watchFingerScrolls(list)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', other, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + }); + + it('keeps a scroll that ends while the finger still rests the finger’s until the lift', () => { + const {list, row} = mount('
'); + detachers.push(host.watchFingerScrolls(list)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + list.dispatchEvent(new Event('scrollend')); + expect(host.isFingerScroll(list)).toBe(true); + // Lifting without moving the list again ends it. + touch('touchend', row, [finger], [], 300); + expect(host.isFingerScroll(list)).toBe(false); + }); + + it('ends a finger’s scroll after a quiet spell where the browser sends no scrollend', () => { + vi.useFakeTimers(); + const own = Object.getOwnPropertyDescriptor(window, 'onscrollend'); + const proto = Object.getPrototypeOf(window) as object; + const inherited = Object.getOwnPropertyDescriptor(proto, 'onscrollend'); + Reflect.deleteProperty(window, 'onscrollend'); + Reflect.deleteProperty(proto, 'onscrollend'); + try { + expect('onscrollend' in window).toBe(false); + const {list, row} = mount('
'); + detachers.push(host.watchFingerScrolls(list)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + touch('touchend', row, [finger], [], 120); + // The momentum keeps scrolling: still the finger's. + vi.advanceTimersByTime(100); + list.dispatchEvent(new Event('scroll')); + vi.advanceTimersByTime(100); + expect(host.isFingerScroll(list)).toBe(true); + // Quiet for the fallback spell: the scroll has ended. + vi.advanceTimersByTime(60); + expect(host.isFingerScroll(list)).toBe(false); + // A program's scroll afterwards is not the finger's. + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + } finally { + if (inherited != null) { + Object.defineProperty(proto, 'onscrollend', inherited); + } + if (own != null) { + Object.defineProperty(window, 'onscrollend', own); + } + vi.useRealTimers(); + } + }); + + it('forgets a scroller once nobody watches it', () => { + const {list, row} = mount('
'); + const stop = host.watchFingerScrolls(list); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(true); + stop(); + expect(host.isFingerScroll(list)).toBe(false); + list.dispatchEvent(new Event('scroll')); + expect(host.isFingerScroll(list)).toBe(false); + }); + }); + + it('drives mouse input through Pointer Events', () => { + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + let released = {x: 0, y: 0}; + pan.addListener(r => { + if (r.state === 'ended') { + released = r.velocity(); + } + }); + detachers.push(host.attach(box, pan)); + pointer('pointerdown', box, { + pointerType: 'mouse', + x: 0, + y: 0, + buttons: 1, + time: 0, + }); + for (let t = 8; t <= 96; t += 8) { + pointer('pointermove', box, { + pointerType: 'mouse', + x: t * 2, + y: 0, + buttons: 1, + time: t, + }); + } + pointer('pointerup', box, {pointerType: 'mouse', x: 192, y: 0, time: 104}); + expect(pan.state).toBe('possible'); + expect(released.x).toBeCloseTo(2000, 0); + }); + + it('cancels the context menu while a long press owns the touch', () => { + const {box} = mount('
'); + const press = new LongPressGestureRecognizer({name: 'press'}); + detachers.push(host.attach(box, press)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + const before = new MouseEvent('contextmenu', { + bubbles: true, + cancelable: true, + }); + box.dispatchEvent(before); + expect(before.defaultPrevented).toBe(false); + host.arena.advanceTime(500); + expect(press.state).toBe('began'); + const during = new MouseEvent('contextmenu', { + bubbles: true, + cancelable: true, + }); + box.dispatchEvent(during); + expect(during.defaultPrevented).toBe(true); + }); + + it('holds the click of a tap a double tap may claim, and replays it when the double tap fails', () => { + const {row, button} = mount( + '
', + ); + const double = new TapGestureRecognizer({ + name: 'double', + numberOfTapsRequired: 2, + }); + detachers.push(host.attach(row, double)); + const clicks: {target: EventTarget | null; detail: number}[] = []; + button.addEventListener('click', event => + clicks.push({target: event.target, detail: event.detail}), + ); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', button, [finger], [finger], 0); + const end = touch('touchend', button, [finger], [], 60); + expect(end.defaultPrevented).toBe(true); + expect(clicks).toEqual([]); + const interval = UIKIT_GESTURES.tap.maximumIntervalBetweenTaps.value; + host.arena.advanceTime(60 + interval); + expect(clicks).toEqual([{target: button, detail: 1}]); + }); + + it('collects recognizers across shadow roots', () => { + const {outer} = mount('
'); + const shadow = outer.attachShadow({mode: 'open'}); + shadow.innerHTML = 'x'; + const inner = shadow.getElementById('inner') as HTMLElement; + const tap = new TapGestureRecognizer({name: 'tap'}); + const actions: string[] = []; + tap.addListener(r => actions.push(r.state)); + detachers.push(host.attach(outer, tap)); + const finger = {id: 1, x: 5, y: 5, target: inner}; + touch('touchstart', inner, [finger], [finger], 0); + touch('touchend', inner, [finger], [], 50); + expect(actions).toEqual(['ended']); + }); + + it('gives a recognizer inside a shadow root its finger: a touch reaches the page retargeted to the shadow host, and the composed path names what it landed on', () => { + const {outer} = mount('
'); + const shadow = outer.attachShadow({mode: 'open'}); + shadow.innerHTML = 'x'; + const inner = shadow.getElementById('inner') as HTMLElement; + const tap = new TapGestureRecognizer({name: 'tap'}); + const actions: string[] = []; + tap.addListener(r => actions.push(r.state)); + detachers.push(host.attach(inner, tap)); + // At the document, a touch's target is the shadow host, as the event's. + const finger = {id: 1, x: 5, y: 5, target: outer}; + touch('touchstart', inner, [finger], [finger], 0); + touch('touchend', inner, [finger], [], 50); + expect(actions).toEqual(['ended']); + }); + + it('gives a recognizer inside a shadow root a mouse press too, the pointer event retargeted to the shadow host at the document', () => { + const {outer} = mount('
'); + const shadow = outer.attachShadow({mode: 'open'}); + shadow.innerHTML = 'x'; + const inner = shadow.getElementById('inner') as HTMLElement; + const tap = new TapGestureRecognizer({name: 'tap'}); + const actions: string[] = []; + tap.addListener(r => actions.push(r.state)); + detachers.push(host.attach(inner, tap)); + pointer('pointerdown', inner, { + pointerType: 'mouse', + x: 5, + y: 5, + buttons: 1, + time: 0, + }); + pointer('pointerup', inner, {pointerType: 'mouse', x: 5, y: 5, time: 50}); + expect(actions).toEqual(['ended']); + }); + + it('drives hover recognizers from a hovering mouse', () => { + const {box, other} = mount('
'); + const hover = new HoverGestureRecognizer(); + const states: string[] = []; + hover.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, hover)); + pointer('pointermove', box, {pointerType: 'mouse', x: 1, y: 1, time: 0}); + pointer('pointermove', box, {pointerType: 'mouse', x: 2, y: 1, time: 16}); + pointer('pointermove', other, { + pointerType: 'mouse', + x: 300, + y: 1, + time: 32, + }); + expect(states).toEqual(['began', 'changed', 'ended']); + // A finger never hovers. + pointer('pointermove', box, {pointerType: 'touch', x: 1, y: 1, time: 48}); + expect(states).toHaveLength(3); + }); + + it('cancels every touch in flight when the window loses focus', () => { + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const states: string[] = []; + pan.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, pan)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [{...finger, x: 40}], [{...finger, x: 40}], 16); + window.dispatchEvent(new Event('blur')); + expect(states).toEqual(['began', 'cancelled']); + }); + + it('remembers a finger the window blur cancelled until it lifts, and refuses that lift its click: nothing acts (spec:AST-094 FR28)', () => { + const {box, page} = mount('

text

'); + const tap = new TapGestureRecognizer({name: 'tap'}); + const states: string[] = []; + tap.addTransitionListener((_r, _from, to) => states.push(to)); + detachers.push(host.attach(box, tap)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + window.dispatchEvent(new Event('blur')); + // The browser kept the touch alive: its lift would click the box. + const lift = touch('touchend', box, [finger], [], 80); + expect(lift.defaultPrevented).toBe(true); + expect(states).not.toContain('ended'); + // Only that lift: a later finger with the same identifier keeps its + // click. + touch('touchstart', page, [finger], [finger], 200); + expect(touch('touchend', page, [finger], [], 260).defaultPrevented).toBe( + false, + ); + // A cancelled finger whose lift never came: the next touch with its + // identifier is a new finger, and its lift keeps its click too. + touch('touchstart', box, [finger], [finger], 300); + window.dispatchEvent(new Event('blur')); + touch('touchstart', page, [finger], [finger], 400); + expect(touch('touchend', page, [finger], [], 460).defaultPrevented).toBe( + false, + ); + }); + + it('finishes a gesture whose target left the document mid-gesture', () => { + const {box, child} = mount('
x
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const states: string[] = []; + pan.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, pan)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', child, [finger], [finger], 0); + touch('touchmove', child, [{...finger, x: 40}], [{...finger, x: 40}], 16); + // A re-render replaces the span: its events no longer reach the document. + child.remove(); + const move = touch( + 'touchmove', + child, + [{...finger, x: 60}], + [{...finger, x: 60}], + 32, + ); + expect(move.defaultPrevented).toBe(true); + touch('touchend', child, [{...finger, x: 60}], [], 48); + // The move and the lift after the removal still reached the pan. + expect(states).toEqual(['began', 'changed', 'ended']); + expect(host.arena.touches).toEqual([]); + }); + + it('cancels a finger the events say is no longer down', () => { + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const states: string[] = []; + pan.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, pan)); + const first = {id: 1, x: 10, y: 10}; + touch('touchstart', box, [first], [first], 0); + touch('touchmove', box, [{...first, x: 40}], [{...first, x: 40}], 16); + // Its touchend went to a node outside the document; the next event for + // another finger no longer lists it. + const other = {id: 2, x: 300, y: 300}; + touch('touchmove', document.body, [other], [other], 32); + expect(states).toEqual(['began', 'cancelled']); + }); + + it('treats a reused finger number as a new touch', () => { + const {box} = mount('
'); + const tap = new TapGestureRecognizer({name: 'tap'}); + const states: string[] = []; + tap.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, tap)); + const finger = {id: 4, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + // No touchend reached us; the same identifier lands again. + touch('touchstart', box, [finger], [finger], 500); + touch('touchend', box, [finger], [], 560); + expect(states).toEqual(['ended']); + }); + + it('cancels every finger on a touchcancel that names none', () => { + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const states: string[] = []; + pan.addListener(r => states.push(r.state)); + detachers.push(host.attach(box, pan)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [{...finger, x: 40}], [{...finger, x: 40}], 16); + box.dispatchEvent(new Event('touchcancel', {bubbles: true})); + expect(states).toEqual(['began', 'cancelled']); + }); + + it('reads epoch-stamped events on the arena clock', () => { + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + let released = {x: 0, y: 0}; + pan.addListener(r => { + if (r.state === 'ended') { + released = r.velocity(); + } + }); + detachers.push(host.attach(box, pan)); + const epoch = performance.timeOrigin + 5000; + const finger = {id: 1, x: 0, y: 10}; + touch('touchstart', box, [finger], [finger], epoch); + for (let t = 8; t <= 96; t += 8) { + const at = {...finger, x: t}; + touch('touchmove', box, [at], [at], epoch + t); + } + touch('touchend', box, [{...finger, x: 96}], [], epoch + 100); + expect(released.x).toBeCloseTo(1000, 0); + expect(host.arena.now()).toBeLessThan(1e9); + }); + + it('locks only for a recognizer that would take the contact: enabled, and accepting its pointer kind', () => { + detachers.push(coarsePrimaryPointer()); + const {slider, stylus} = mount( + '
', + ); + const release = vi.fn(); + const lock = vi.fn(() => release); + const drag = new PanGestureRecognizer({ + name: 'drag', + direction: 'horizontal', + selectionLock: lock, + }); + detachers.push(host.attach(slider, drag)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', slider, [finger], [finger], 0); + expect(lock).toHaveBeenCalledTimes(1); + touch('touchend', slider, [finger], [], 40); + expect(release).toHaveBeenCalledTimes(1); + drag.setEnabled(false); + touch('touchstart', slider, [finger], [finger], 100); + touch('touchend', slider, [finger], [], 140); + expect(lock).toHaveBeenCalledTimes(1); + // A pen-only drag locks for a pen: pen acceptance decides, not touch. + const penLock = vi.fn(() => () => {}); + detachers.push( + host.attach( + stylus, + new PanGestureRecognizer({ + name: 'pen', + direction: 'horizontal', + allowedPointerTypes: ['pen'], + selectionLock: penLock, + }), + ), + ); + pointer('pointerdown', stylus, { + id: 7, + pointerType: 'pen', + x: 10, + y: 10, + time: 200, + }); + expect(penLock).toHaveBeenCalledTimes(1); + pointer('pointerup', stylus, { + id: 7, + pointerType: 'pen', + x: 10, + y: 10, + time: 240, + }); + }); + + it('shares one blocking touchmove handler among the elements that claim while moving: 200 pans, one handler, none for a tap (spec:AST-094 FR19)', () => { + const added = vi.spyOn(Element.prototype, 'addEventListener'); + try { + const pans = Array.from({length: 200}, () => { + const element = document.createElement('div'); + document.body.appendChild(element); + detachers.push( + host.attach( + element, + new PanGestureRecognizer({direction: 'horizontal'}), + ), + () => element.remove(), + ); + return element; + }); + const tapTarget = document.createElement('div'); + document.body.appendChild(tapTarget); + detachers.push(host.attach(tapTarget, new TapGestureRecognizer()), () => + tapTarget.remove(), + ); + const blocking = added.mock.calls + .map((call, index) => ({ + call, + element: added.mock.contexts[index] as Element, + })) + .filter( + ({call: [type, , options]}) => + type === 'touchmove' && + typeof options === 'object' && + options.passive === false, + ); + // Each claiming element carries the listener itself, so a scroll that + // starts outside the rows never waits on it... + expect(blocking.map(({element}) => element)).toEqual(pans); + // ...and all of them share the host's one handler. + expect(new Set(blocking.map(({call}) => call[1])).size).toBe(1); + } finally { + added.mockRestore(); + } + }); + + it('keeps native long-press selection on a press-tracked element whose hold does nothing (spec:AST-094 FR41)', () => { + detachers.push(coarsePrimaryPointer()); + const {row, text} = mount( + '
Row
', + ); + detachers.push(host.attachConsumer(row, new RecordingConsumer())); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', text, [finger], [finger], 0); + expect(row.style.getPropertyValue('user-select')).toBe('text'); + touch('touchend', text, [finger], [], 600); + expect(row.style.getPropertyValue('user-select')).toBe('text'); + }); + + it('turns text selection and the callout off while a finger holds an element whose hold opens a menu (spec:AST-094 FR41)', () => { + detachers.push(coarsePrimaryPointer()); + const {trigger} = mount( + '
Hold for the menu
', + ); + const release = vi.fn(); + const lock = vi.fn(() => release); + detachers.push( + host.attach( + trigger, + new LongPressGestureRecognizer({name: 'menu', selectionLock: lock}), + ), + ); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', trigger, [finger], [finger], 0); + // The host calls the long press's lock for this contact alone. + expect(lock).toHaveBeenCalledTimes(1); + expect([ + ...(lock.mock.calls[0] as unknown as [Iterable])[0], + ]).toEqual([trigger]); + expect(release).not.toHaveBeenCalled(); + touch('touchend', trigger, [finger], [], 600); + expect(release).toHaveBeenCalledTimes(1); + }); + + it('a long press brings lockSelection by default', () => { + expect(new LongPressGestureRecognizer().selectionLock).toBe(lockSelection); + expect(new TapGestureRecognizer().selectionLock).toBeNull(); + }); + + it('keeps native long-press selection under a 0 ms long press, a touch-down detector whose hold does nothing (spec:AST-094 FR41)', () => { + detachers.push(coarsePrimaryPointer()); + const {photo} = mount( + '
Photo
', + ); + detachers.push( + host.attach( + photo, + new LongPressGestureRecognizer({minimumPressDuration: 0}), + ), + ); + const set = vi.spyOn(photo.style, 'setProperty'); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', photo, [finger], [finger], 0); + expect(set).not.toHaveBeenCalledWith('user-select', 'none'); + expect(set).not.toHaveBeenCalledWith('-webkit-touch-callout', 'none'); + expect(photo.style.getPropertyValue('user-select')).toBe('text'); + touch('touchend', photo, [finger], [], 1200); + }); + + it('keeps only the finger a kept-press control tracks: a second finger on the held control still scrolls (spec:AST-094 FR19)', () => { + detachers.push(coarsePrimaryPointer()); + const {stepper} = mount('
+
'); + detachers.push( + host.attachConsumer( + stepper, + new PressTracker({getBounds: () => null, keepsTouch: true}), + ), + ); + const held = {id: 1, x: 5, y: 5}; + const heldMoved = {id: 1, x: 5, y: 40}; + const second = {id: 2, x: 8, y: 8}; + const secondMoved = {id: 2, x: 8, y: 60}; + touch('touchstart', stepper, [held], [held], 0); + // The held finger's press is delivered: its moves are kept. + expect( + touch('touchmove', stepper, [heldMoved], [heldMoved], 16) + .defaultPrevented, + ).toBe(true); + // A second finger the tracker is not tracking keeps its own moves. + touch('touchstart', stepper, [second], [heldMoved, second], 32); + expect( + touch('touchmove', stepper, [secondMoved], [heldMoved, secondMoved], 48) + .defaultPrevented, + ).toBe(false); + touch('touchend', stepper, [heldMoved, secondMoved], [], 64); + }); + + it('writes touch-action: manipulation for a kept-press control where the primary pointer is coarse, so rapid taps never zoom, and nothing where it is fine (spec:AST-094 FR18, FR19)', () => { + const pointer = pointerQuery(true); + detachers.push(pointer.restore); + const {stepper, row} = mount( + '
+
Row
', + ); + detachers.push( + host.attachConsumer( + stepper, + new PressTracker({getBounds: () => null, keepsTouch: true}), + ), + ); + detachers.push( + host.attachConsumer(row, new PressTracker({getBounds: () => null})), + ); + expect(stepper.getAttribute('data-astryx-touch-action')).toBe( + 'manipulation', + ); + // A press tracker that keeps no touch gets no mark. + expect(row.hasAttribute('data-astryx-touch-action')).toBe(false); + // Where the pointer turns fine, the browser's own value comes back (FR19). + pointer.set(false); + expect(stepper.hasAttribute('data-astryx-touch-action')).toBe(false); + }); + + it("calls a press tracker's own selection lock for each finger's press, a stepper's hold-repeat, and nowhere else (spec:AST-094 FR41)", () => { + detachers.push(coarsePrimaryPointer()); + const {stepper, row} = mount( + '
+
Row
', + ); + const release = vi.fn(); + const lock = vi.fn( + (_elements: Iterable, _target: EventTarget | null) => release, + ); + detachers.push( + host.attachConsumer(stepper, new LockingConsumer(lock)), + host.attachConsumer(row, new RecordingConsumer()), + ); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', stepper, [finger], [finger], 0); + expect(lock).toHaveBeenCalledTimes(1); + expect([...lock.mock.calls[0][0]]).toEqual([stepper]); + expect(release).not.toHaveBeenCalled(); + touch('touchend', stepper, [finger], [], 600); + expect(release).toHaveBeenCalledTimes(1); + // A mouse press keeps desktop selection. + pointer('pointerdown', stepper, { + pointerType: 'mouse', + x: 5, + y: 5, + buttons: 1, + time: 700, + }); + pointer('pointerup', stepper, { + pointerType: 'mouse', + x: 5, + y: 5, + time: 750, + }); + // A tracker whose hold does nothing brings no lock. + touch('touchstart', row, [finger], [finger], 800); + touch('touchend', row, [finger], [], 900); + expect(lock).toHaveBeenCalledTimes(1); + }); + it('locks for a hold that takes the touch, and nothing where it declines one: a zero-delay hold opts in with its own lock (spec:AST-094 FR41)', () => { + detachers.push(asIosWebKit()); + detachers.push(coarsePrimaryPointer()); + const {bar, lens, search} = mount( + '
Lens
', + ); + // A tab bar's lens: a hold at 0 ms that brings the lock itself, and + // takes only touches on the lens, not on the search tab beside it. + detachers.push( + host.attach( + bar, + new LongPressGestureRecognizer({ + minimumPressDuration: 0, + selectionLock: lockSelection, + shouldReceiveTouch: (_recognizer, contact) => + contact.target instanceof Node && lens.contains(contact.target), + }), + ), + ); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', search, [finger], [finger], 0); + expect(bar.style.getPropertyValue('user-select')).toBe(''); + touch('touchend', search, [finger], [], 100); + touch('touchstart', lens, [finger], [finger], 200); + expect(bar.style.getPropertyValue('user-select')).toBe('none'); + touch('touchend', lens, [finger], [], 300); + expect(bar.style.getPropertyValue('user-select')).toBe(''); + }); + + it('on a coarse primary pointer, keeps selection for a mouse, a press-tracked row and a pan that does not ask, and locks a pan that asks', () => { + detachers.push(asIosWebKit()); + detachers.push(coarsePrimaryPointer()); + const {row, sheet, slider} = mount( + '
', + ); + detachers.push( + host.attachConsumer(row, new RecordingConsumer()), + host.attach(sheet, new PanGestureRecognizer()), + host.attach( + slider, + new PanGestureRecognizer({ + direction: 'horizontal', + selectionLock: lockSelection, + }), + ), + ); + const at = {x: 5, y: 5}; + pointer('pointerdown', row, { + pointerType: 'mouse', + ...at, + buttons: 1, + time: 0, + }); + expect(row.style.getPropertyValue('user-select')).toBe(''); + pointer('pointerup', row, {pointerType: 'mouse', ...at, time: 50}); + // A pen on a touch device (an iPad's pencil) on a row whose hold does + // nothing keeps native selection too (spec:AST-094 FR41). + pointer('pointerdown', row, { + pointerType: 'pen', + ...at, + buttons: 1, + time: 100, + }); + expect(row.style.getPropertyValue('user-select')).toBe(''); + pointer('pointerup', row, {pointerType: 'pen', ...at, time: 150}); + expect(row.style.getPropertyValue('user-select')).toBe(''); + + const finger = {id: 1, ...at}; + touch('touchstart', sheet, [finger], [finger], 200); + expect(sheet.style.getPropertyValue('user-select')).toBe(''); + touch('touchend', sheet, [finger], [], 250); + touch('touchstart', slider, [finger], [finger], 300); + expect(slider.style.getPropertyValue('user-select')).toBe('none'); + touch('touchcancel', slider, [finger], [], 350); + expect(slider.style.getPropertyValue('user-select')).toBe(''); + }); + + it("keeps selection for every contact where the primary pointer is fine: a touchscreen laptop's finger, a desktop pen and a mouse (spec:AST-094 FR19, spec:AST-017 FR59)", () => { + detachers.push(pointerQuery(false).restore); + const {row, slider} = mount('
'); + detachers.push( + host.attachConsumer(row, new RecordingConsumer()), + host.attach( + slider, + new PanGestureRecognizer({ + direction: 'horizontal', + selectionLock: lockSelection, + }), + ), + ); + const at = {x: 5, y: 5}; + for (const [time, pointerType] of [ + [0, 'mouse'], + [100, 'pen'], + [200, 'touch'], + ] as const) { + pointer('pointerdown', row, { + pointerType, + ...at, + buttons: 1, + time, + }); + expect(row.style.getPropertyValue('user-select')).toBe(''); + pointer('pointerup', row, {pointerType, ...at, time: time + 50}); + } + const finger = {id: 1, ...at}; + touch('touchstart', slider, [finger], [finger], 300); + expect(slider.style.getPropertyValue('user-select')).toBe(''); + touch('touchend', slider, [finger], [], 350); + }); + + it('locks a checkbox under the finger inside a locked element, which WebKit makes selectable, but never a text field', () => { + detachers.push(asIosWebKit()); + detachers.push(coarsePrimaryPointer()); + const {row, check, field} = mount( + '
', + ); + // A switch's drag: a hold that does something. + detachers.push( + host.attach( + row, + new PanGestureRecognizer({ + direction: 'horizontal', + selectionLock: lockSelection, + }), + ), + ); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', check, [finger], [finger], 0); + expect(check.style.getPropertyValue('user-select')).toBe('none'); + touch('touchend', check, [finger], [], 100); + expect(check.style.getPropertyValue('user-select')).toBe(''); + touch('touchstart', field, [finger], [finger], 200); + expect(row.style.getPropertyValue('user-select')).toBe('none'); + expect(field.style.getPropertyValue('user-select')).toBe(''); + touch('touchend', field, [finger], [], 300); + // An input outside every locked element is not ours to lock. + const {alone} = mount(''); + detachers.push(host.attach(alone, new TapGestureRecognizer())); + touch('touchstart', alone, [finger], [finger], 400); + expect(alone.style.getPropertyValue('user-select')).toBe(''); + touch('touchend', alone, [finger], [], 450); + }); + + it('ignores touches that land outside every attached element', () => { + const {box, elsewhere} = mount( + '
', + ); + const pan = new PanGestureRecognizer({name: 'pan'}); + detachers.push(host.attach(box, pan)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', elsewhere, [finger], [finger], 0); + const move = touch( + 'touchmove', + elsewhere, + [{...finger, x: 60}], + [{...finger, x: 60}], + 16, + ); + expect(move.defaultPrevented).toBe(false); + expect(host.arena.touches).toEqual([]); + }); +}); + +describe('holdPageScroll', () => { + const at = (id: number, x: number, y: number) => [{id, x, y}]; + + it('holds the page only where the primary pointer is coarse, following the pointer while held (spec:AST-094 FR19, FR24)', () => { + const pointer = pointerQuery(false); + detachers.push(pointer.restore); + const added = vi.spyOn(document, 'addEventListener'); + const removed = vi.spyOn(document, 'removeEventListener'); + detachers.push(() => { + added.mockRestore(); + removed.mockRestore(); + }); + detachers.push(host.holdPageScroll(() => true)); + const touchTypes = (spy: typeof added) => + spy.mock.calls + .map(([type]) => type) + .filter(type => type.startsWith('touch')); + expect(touchTypes(added)).toEqual([]); + pointer.set(true); + expect(touchTypes(added)).toEqual(['touchstart', 'touchmove']); + pointer.set(false); + expect(touchTypes(removed)).toEqual(['touchstart', 'touchmove']); + }); + + it('cancels a one-finger pan the decision says would scroll the page, for the rest of its touch', () => { + const {scrim} = mount('
'); + const decide = vi.fn(() => true); + detachers.push(host.holdPageScroll(decide)); + + touch('touchstart', scrim, at(1, 100, 300), at(1, 100, 300), 0); + const first = touch( + 'touchmove', + scrim, + at(1, 100, 250), + at(1, 100, 250), + 16, + ); + const second = touch( + 'touchmove', + scrim, + at(1, 100, 200), + at(1, 100, 200), + 32, + ); + + expect(first.defaultPrevented).toBe(true); + expect(second.defaultPrevented).toBe(true); + expect(decide).toHaveBeenCalledTimes(1); + expect(decide).toHaveBeenCalledWith(scrim, 0, -50); + }); + + it("judges a new touch afresh when it reuses the last touch's identifier", () => { + const {scrim} = mount('
'); + // The backdrop pan would scroll the page; the list pan scrolls the list. + const decide = vi.fn( + (_origin: EventTarget | null, _dx: number, dy: number) => dy < 0, + ); + detachers.push(host.holdPageScroll(decide)); + touch('touchstart', scrim, at(1, 100, 300), at(1, 100, 300), 0); + expect( + touch('touchmove', scrim, at(1, 100, 250), at(1, 100, 250), 16) + .defaultPrevented, + ).toBe(true); + touch('touchend', scrim, at(1, 100, 250), [], 32); + // The next touch reuses identifier 1 and pans the other way. + touch('touchstart', scrim, at(1, 100, 300), at(1, 100, 300), 48); + expect( + touch('touchmove', scrim, at(1, 100, 350), at(1, 100, 350), 64) + .defaultPrevented, + ).toBe(false); + expect(decide).toHaveBeenCalledTimes(2); + }); + + it('keeps holding the first finger after a second finger touches and lifts', () => { + const {scrim} = mount('
'); + detachers.push(host.holdPageScroll(() => true)); + touch('touchstart', scrim, at(1, 100, 300), at(1, 100, 300), 0); + expect( + touch('touchmove', scrim, at(1, 100, 250), at(1, 100, 250), 16) + .defaultPrevented, + ).toBe(true); + // A second finger lands and lifts: while two are down, nothing is + // cancelled, so the page can zoom. + const both = [...at(1, 100, 250), ...at(2, 200, 300)]; + touch('touchstart', scrim, at(2, 200, 300), both, 32); + expect( + touch('touchmove', scrim, at(2, 200, 260), both, 48).defaultPrevented, + ).toBe(false); + touch('touchend', scrim, at(2, 200, 260), at(1, 100, 250), 64); + // Alone again, the first finger is held for the rest of its touch. + expect( + touch('touchmove', scrim, at(1, 100, 200), at(1, 100, 200), 80) + .defaultPrevented, + ).toBe(true); + }); + + it('judges a pan once it clears the slop, so a first wobble does not fix its axis', () => { + const {strip} = mount('
'); + const decide = vi.fn((_origin: EventTarget | null, dx: number) => dx === 0); + detachers.push(host.holdPageScroll(decide)); + + touch('touchstart', strip, at(1, 100, 300), at(1, 100, 300), 0); + const wobble = touch( + 'touchmove', + strip, + at(1, 102, 302), + at(1, 102, 302), + 16, + ); + const swipe = touch( + 'touchmove', + strip, + at(1, 110, 303), + at(1, 110, 303), + 32, + ); + + expect(wobble.defaultPrevented).toBe(false); + expect(decide).toHaveBeenCalledTimes(1); + expect(decide).toHaveBeenCalledWith(strip, 10, 3); + expect(swipe.defaultPrevented).toBe(false); + }); + + it('lets a pan through when the decision says something under the finger takes it', () => { + const {list} = mount('
'); + detachers.push(host.holdPageScroll(() => false)); + + touch('touchstart', list, at(1, 100, 300), at(1, 100, 300), 0); + expect( + touch('touchmove', list, at(1, 100, 250), at(1, 100, 250), 16) + .defaultPrevented, + ).toBe(false); + }); + + it('never cancels a two-finger gesture, so pinch-zoom keeps working', () => { + const {scrim} = mount('
'); + const decide = vi.fn(() => true); + detachers.push(host.holdPageScroll(decide)); + const both = [ + {id: 1, x: 100, y: 300}, + {id: 2, x: 200, y: 300}, + ]; + + touch('touchstart', scrim, both, both, 0); + const moved = [ + {id: 1, x: 90, y: 300}, + {id: 2, x: 210, y: 300}, + ]; + expect(touch('touchmove', scrim, moved, moved, 16).defaultPrevented).toBe( + false, + ); + expect(decide).not.toHaveBeenCalled(); + }); + + it('hands the decision the innermost node of a touch inside a shadow root', () => { + const {outer} = mount('
'); + const inner = document.createElement('div'); + outer.attachShadow({mode: 'open'}).append(inner); + const decide = vi.fn(() => false); + detachers.push(host.holdPageScroll(decide)); + + touch('touchstart', inner, at(1, 100, 300), at(1, 100, 300), 0); + touch('touchmove', inner, at(1, 100, 250), at(1, 100, 250), 16); + + expect(decide).toHaveBeenCalledWith(inner, 0, -50); + }); + + it('stacks holds and stops judging pans once the last one is released', () => { + const {scrim} = mount('
'); + const releaseFirst = host.holdPageScroll(() => true); + const releaseSecond = host.holdPageScroll(() => true); + const pan = (time: number) => { + touch('touchstart', scrim, at(1, 100, 300), at(1, 100, 300), time); + return touch( + 'touchmove', + scrim, + at(1, 100, 250), + at(1, 100, 250), + time + 16, + ).defaultPrevented; + }; + + releaseFirst(); + expect(pan(0)).toBe(true); + releaseSecond(); + releaseSecond(); + expect(pan(100)).toBe(false); + }); +}); + +describe('touch gates and a second finger', () => { + it('holds a gated target away from every participant until the gate is released', () => { + const {layer, button} = mount( + '
', + ); + const tap = new TapGestureRecognizer({name: 'tap'}); + detachers.push(host.attach(button, tap)); + const release = host.addTouchGate(target => layer.contains(target)); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', button, [finger], [finger], 0); + expect(tap.numberOfTouches).toBe(0); + touch('touchend', button, [finger], [], 40); + release(); + touch('touchstart', button, [finger], [finger], 100); + expect(tap.numberOfTouches).toBe(1); + touch('touchend', button, [finger], [], 140); + }); + + it('fails a one-finger pan that has not claimed when a second finger lands anywhere, so the browser can pinch', () => { + const {row, text} = mount('

plain

'); + const swipe = new PanGestureRecognizer({ + name: 'swipe', + direction: 'horizontal', + maximumNumberOfTouches: 1, + }); + const states: string[] = []; + swipe.addListener(r => states.push(r.state)); + detachers.push(host.attach(row, swipe)); + const first = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [first], [first], 0); + // Under the hysteresis: still possible, nothing claimed. + touch('touchmove', row, [{...first, x: 15}], [{...first, x: 15}], 16); + const second = {id: 2, x: 200, y: 300}; + touch('touchstart', text, [second], [{...first, x: 15}, second], 32); + // The pan can no longer claim the pair midway: the pinch is the browser's. + const pinch = touch( + 'touchmove', + row, + [{...first, x: 60}], + [{...first, x: 60}, second], + 48, + ); + expect(pinch.defaultPrevented).toBe(false); + expect(states).toEqual([]); + }); + + it('keeps a claimed one-finger drag when a second finger lands anywhere (AST-094 FR10)', () => { + const {row, text} = mount('

plain

'); + const swipe = new PanGestureRecognizer({ + name: 'swipe', + direction: 'horizontal', + maximumNumberOfTouches: 1, + }); + detachers.push(host.attach(row, swipe)); + const first = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [first], [first], 0); + touch('touchmove', row, [{...first, x: 40}], [{...first, x: 40}], 16); + expect(swipe.state).toBe('began'); + const second = {id: 2, x: 200, y: 300}; + touch('touchstart', text, [second], [{...first, x: 40}, second], 32); + const drag = touch( + 'touchmove', + row, + [{...first, x: 80}], + [{...first, x: 80}, second], + 48, + ); + expect(swipe.state).toBe('changed'); + expect(drag.defaultPrevented).toBe(true); + }); +}); + +describe('a delivered press keeps its touch', () => { + it('cancels the browser pan once the consumer keeps the touch, and not before', () => { + const {row} = mount('
'); + class Keeper extends RecordingConsumer { + kept = false; + keepsTouch(): boolean { + return this.kept; + } + } + const control = new Keeper(); + detachers.push(host.attachConsumer(row, control)); + const finger = {id: 1, x: 10, y: 10}; + touch('touchstart', row, [finger], [finger], 0); + // Before the press is delivered, a move still scrolls. + const early = touch( + 'touchmove', + row, + [{...finger, y: 14}], + [{...finger, y: 14}], + 40, + ); + expect(early.defaultPrevented).toBe(false); + // Delivered (highlighted past the content delay): the touch is the control's. + control.kept = true; + const late = touch( + 'touchmove', + row, + [{...finger, y: 54}], + [{...finger, y: 54}], + 200, + ); + expect(late.defaultPrevented).toBe(true); + expect(control.events.some(event => event.startsWith('cancelled'))).toBe( + false, + ); + touch('touchend', row, [{...finger, y: 54}], [], 260); + }); +}); + +describe('detaching a keepsTouch press tracker', () => { + it('removes its blocking touchmove listener while the element keeps its recognizers', () => { + const {row} = mount('
'); + const active = new Set(); + const add = row.addEventListener.bind(row); + const remove = row.removeEventListener.bind(row); + vi.spyOn(row, 'addEventListener').mockImplementation( + (type, listener, options) => { + if ( + type === 'touchmove' && + listener != null && + typeof options === 'object' && + options.passive === false + ) { + active.add(listener); + } + add(type, listener, options); + }, + ); + vi.spyOn(row, 'removeEventListener').mockImplementation( + (type, listener, options) => { + if (type === 'touchmove' && listener != null) { + active.delete(listener); + } + remove(type, listener, options); + }, + ); + // A tap decides at the lift: on its own it needs no blocking listener. + detachers.push(host.attach(row, new TapGestureRecognizer({name: 'tap'}))); + expect(active.size).toBe(0); + class Keeper extends RecordingConsumer { + keepsTouch(): boolean { + return true; + } + } + const detach = host.attachConsumer(row, new Keeper()); + expect(active.size).toBe(1); + detach(); + expect(active.size).toBe(0); + }); +}); + +describe('GestureHost.fingersDown', () => { + it('counts every finger on the screen, one on a plain element too', () => { + const {handle, row} = mount( + '
plain
', + ); + expect(host.fingersDown()).toBe(0); + detachers.push( + host.attach(handle, new LongPressGestureRecognizer({name: 'hold'})), + ); + const first = {id: 1, x: 10, y: 10, target: handle}; + const second = {id: 2, x: 200, y: 10, target: row}; + touch('touchstart', handle, [first], [first], 0); + expect(host.fingersDown()).toBe(1); + // The second finger lands on an element no recognizer is attached to. + touch('touchstart', row, [second], [first, second], 16); + expect(host.fingersDown()).toBe(2); + // A lift lowers it. + touch('touchend', row, [second], [first], 32); + expect(host.fingersDown()).toBe(1); + touch('touchend', handle, [first], [], 48); + expect(host.fingersDown()).toBe(0); + }); + + it('drops to what a touchcancel lists', () => { + const {handle, row} = mount( + '
plain
', + ); + detachers.push( + host.attach(handle, new LongPressGestureRecognizer({name: 'hold'})), + ); + const first = {id: 1, x: 10, y: 10, target: handle}; + const second = {id: 2, x: 200, y: 10, target: row}; + touch('touchstart', handle, [first], [first], 0); + touch('touchstart', row, [second], [first, second], 16); + expect(host.fingersDown()).toBe(2); + // The system takes every finger. + touch('touchcancel', handle, [first, second], [], 32); + expect(host.fingersDown()).toBe(0); + }); + + it('reads 0 once the host stops listening', () => { + const {handle} = mount('
'); + const detach = host.attach( + handle, + new LongPressGestureRecognizer({name: 'hold'}), + ); + const finger = {id: 1, x: 10, y: 10, target: handle}; + touch('touchstart', handle, [finger], [finger], 0); + expect(host.fingersDown()).toBe(1); + detach(); + expect(host.fingersDown()).toBe(0); + }); +}); + +describe('only where the primary pointer is coarse (spec:AST-094 FR19, FR24)', () => { + /** Quiets the development warning, and returns its spy. */ + function quietWarning() { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + detachers.push(() => warn.mockRestore()); + return warn; + } + + it('attaches nothing where it is fine: no touch or pointer listener, no style, nothing delivered', () => { + quietWarning(); + detachers.push(pointerQuery(false).restore); + const added = vi.spyOn(EventTarget.prototype, 'addEventListener'); + detachers.push(() => added.mockRestore()); + const {box, row} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const press = new LongPressGestureRecognizer({name: 'press'}); + const consumer = new RecordingConsumer(); + detachers.push( + host.attach(box, pan), + host.attach(box, press), + host.attachConsumer(row, consumer), + ); + expect( + added.mock.calls + .map(([type]) => type) + .filter(type => /^(touch|pointer)/.test(type)), + ).toEqual([]); + // The long press's callout and selection styles stay the browser's. + expect(box.style.getPropertyValue('user-select')).toBe(''); + const finger = {id: 1, x: 5, y: 5}; + const moved = {id: 1, x: 60, y: 5}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [moved], [moved], 16); + touch('touchend', box, [moved], [], 32); + touch('touchstart', row, [finger], [finger], 100); + touch('touchend', row, [finger], [], 150); + expect(pan.state).toBe('possible'); + expect(consumer.events).toEqual([]); + }); + + it('follows the pointer: turning coarse installs it, turning fine cancels the touch in flight and removes it all', () => { + quietWarning(); + const pointer = pointerQuery(false); + detachers.push(pointer.restore); + const {box} = mount('
'); + const pan = new PanGestureRecognizer({name: 'pan'}); + const states: string[] = []; + pan.addTransitionListener((_current, _from, to) => states.push(to)); + detachers.push( + host.attach(box, pan), + host.attach(box, new LongPressGestureRecognizer({name: 'press'})), + ); + expect(box.hasAttribute('data-astryx-touch-action')).toBe(false); + // A tablet undocks its mouse. + pointer.set(true); + expect(box.hasAttribute('data-astryx-touch-action')).toBe(true); + const finger = {id: 1, x: 5, y: 5}; + const moved = {id: 1, x: 40, y: 5}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [moved], [moved], 16); + expect(pan.state).toBe('began'); + // It docks one again mid-drag: the drag ends without acting. + pointer.set(false); + expect(states).toContain('cancelled'); + expect(box.style.getPropertyValue('user-select')).toBe(''); + const count = states.length; + const later = {id: 1, x: 80, y: 5}; + touch('touchmove', box, [later], [later], 32); + touch('touchend', box, [later], [], 48); + expect(states).toHaveLength(count); + }); + + it("attaches a recognizer that reads each event's pointer where it is fine, with its own listener and style and no warning (spec:AST-057)", () => { + const warn = quietWarning(); + detachers.push(pointerQuery(false).restore); + const {box} = mount('
'); + const swipe = new PanGestureRecognizer({ + name: 'swipe', + direction: 'horizontal', + }); + const states: string[] = []; + swipe.addTransitionListener((_current, _from, to) => states.push(to)); + detachers.push(host.attach(box, swipe, {perEventPointer: true})); + expect(box.getAttribute('data-astryx-touch-action')).toBe( + 'pan-y pinch-zoom', + ); + const finger = {id: 1, x: 5, y: 5}; + const moved = {id: 1, x: 60, y: 5}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [moved], [moved], 16); + touch('touchend', box, [moved], [], 32); + expect(states).toEqual(expect.arrayContaining(['began', 'ended'])); + expect(warn).not.toHaveBeenCalled(); + }); + + it('keeps every other recognizer and press consumer beside it inactive where it is fine, and warns for them', () => { + const warn = quietWarning(); + detachers.push(pointerQuery(false).restore); + const {box, row} = mount('
'); + const set = vi.spyOn(row.style, 'setProperty'); + const tap = new TapGestureRecognizer({name: 'tap'}); + const taps: string[] = []; + tap.addTransitionListener((_current, _from, to) => taps.push(to)); + const consumer = new RecordingConsumer(); + detachers.push( + host.attach(box, new PanGestureRecognizer({name: 'swipe'}), { + perEventPointer: true, + }), + host.attach(row, tap), + host.attachConsumer(row, consumer), + ); + expect(set).not.toHaveBeenCalled(); + expect(row.hasAttribute('data-astryx-touch-action')).toBe(false); + const finger = {id: 2, x: 5, y: 5}; + touch('touchstart', row, [finger], [finger], 0); + touch('touchend', row, [finger], [], 50); + expect(taps).toEqual([]); + expect(consumer.events).toEqual([]); + expect(warn).toHaveBeenCalledTimes(1); + }); + + it('where the pointer is fine, a per-event press tracker takes no mouse press, and attaches with no warning beside one that stays inactive (spec:AST-094 FR19)', () => { + const warn = quietWarning(); + detachers.push(pointerQuery(false).restore); + const {stepper, other} = mount( + '
', + ); + const bounds = {left: 0, top: 0, right: 100, bottom: 100}; + const tracker = (events: string[]) => + new PressTracker({ + getBounds: () => bounds, + onControlEvent: event => events.push(event), + onHighlightChange: isHighlighted => + events.push(isHighlighted ? 'highlight' : 'unhighlight'), + }); + const held: string[] = []; + const inactive: string[] = []; + detachers.push( + host.attachConsumer(stepper, tracker(held), {perEventPointer: true}), + ); + expect(warn).not.toHaveBeenCalled(); + detachers.push(host.attachConsumer(other, tracker(inactive))); + expect(warn).toHaveBeenCalledTimes(1); + pointer('pointerdown', stepper, { + pointerType: 'mouse', + x: 5, + y: 5, + buttons: 1, + time: 0, + }); + pointer('pointerup', stepper, { + pointerType: 'mouse', + x: 5, + y: 5, + time: 600, + }); + // Press feedback is for a finger: a mouse keeps the desktop's handling, + // with no paint and no control event. + expect(held).toEqual([]); + pointer('pointerdown', other, { + id: 2, + pointerType: 'mouse', + x: 5, + y: 5, + buttons: 1, + time: 700, + }); + pointer('pointerup', other, { + id: 2, + pointerType: 'mouse', + x: 5, + y: 5, + time: 800, + }); + expect(inactive).toEqual([]); + }); + + it("where the pointer is fine, gives a per-event press tracker's finger its press feedback only: it paints, reports no control event, and the browser keeps the pan (spec:AST-094 FR19)", () => { + quietWarning(); + const pointer = pointerQuery(false); + detachers.push(pointer.restore); + const {stepper} = mount('
'); + const bounds = {left: 0, top: 0, right: 100, bottom: 100}; + const events: string[] = []; + const highlights: boolean[] = []; + const tracker = new PressTracker({ + getBounds: () => bounds, + keepsTouch: true, + onControlEvent: event => events.push(event), + onHighlightChange: isHighlighted => highlights.push(isHighlighted), + }); + detachers.push( + host.attachConsumer(stepper, tracker, {perEventPointer: true}), + ); + const finger = {id: 1, x: 5, y: 5}; + const moved = {id: 1, x: 5, y: 60}; + touch('touchstart', stepper, [finger], [finger], 0); + // Its press is painted, as under a coarse pointer, and reports nothing. + expect(highlights).toContain(true); + expect(events).toEqual([]); + // But it never keeps the touch from the browser: the page still pans. + expect( + touch('touchmove', stepper, [moved], [moved], 16).defaultPrevented, + ).toBe(false); + touch('touchend', stepper, [moved], [], 32); + expect(events).toEqual([]); + // On a coarse primary pointer the same held press reports its events + // and keeps its touch. + pointer.set(true); + touch('touchstart', stepper, [finger], [finger], 100); + expect(events).toEqual(['touchDown']); + expect( + touch('touchmove', stepper, [moved], [moved], 116).defaultPrevented, + ).toBe(true); + touch('touchend', stepper, [moved], [], 132); + }); + + it("where the pointer is fine, an idle per-event press tracker leaves no document or window listener: they last only a finger's press (spec:AST-094 FR19)", async () => { + quietWarning(); + const pointer = pointerQuery(false); + detachers.push(pointer.restore); + const {stepper} = mount('
'); + const listening = new Set(); + const watch = (scope: Document | Window, name: string) => { + const addOriginal = scope.addEventListener.bind(scope); + const removeOriginal = scope.removeEventListener.bind(scope); + const add = vi + .spyOn(scope, 'addEventListener') + .mockImplementation( + ( + type: string, + listener: EventListenerOrEventListenerObject, + options?: boolean | AddEventListenerOptions, + ) => { + listening.add(`${name}:${type}`); + addOriginal(type, listener, options); + }, + ); + const remove = vi + .spyOn(scope, 'removeEventListener') + .mockImplementation( + ( + type: string, + listener: EventListenerOrEventListenerObject, + options?: boolean | EventListenerOptions, + ) => { + listening.delete(`${name}:${type}`); + removeOriginal(type, listener, options); + }, + ); + detachers.push(() => { + add.mockRestore(); + remove.mockRestore(); + }); + }; + watch(document, 'document'); + watch(window, 'window'); + const highlights: boolean[] = []; + detachers.push( + host.attachConsumer( + stepper, + new PressTracker({ + getBounds: () => ({left: 0, top: 0, right: 100, bottom: 100}), + onHighlightChange: value => highlights.push(value), + }), + {perEventPointer: true}, + ), + ); + const pageListeners = () => + [...listening].filter(key => /:(touch|pointer|click)/.test(key)); + // Attached and idle: nothing on the document or the window, even beside + // a finger-scroll watch. + expect(pageListeners()).toEqual([]); + detachers.push(host.watchFingerScrolls(document)); + expect(pageListeners()).toEqual([]); + const finger = {id: 1, x: 5, y: 5}; + touch('touchstart', stepper, [finger], [finger], 0); + // The press is painted, and the host listens for this press only. + expect(highlights).toContain(true); + expect(pageListeners().length).toBeGreaterThan(0); + touch('touchend', stepper, [finger], [], 40); + await Promise.resolve(); + expect(pageListeners()).toEqual([]); + }); + + it('keeps it working when the pointer turns fine, as the rest comes down', () => { + quietWarning(); + const pointer = pointerQuery(true); + detachers.push(pointer.restore); + const {box, row} = mount('
'); + const swipe = new PanGestureRecognizer({ + name: 'swipe', + direction: 'horizontal', + }); + const states: string[] = []; + swipe.addTransitionListener((_current, _from, to) => states.push(to)); + detachers.push( + host.attach(box, swipe, {perEventPointer: true}), + host.attach(row, new LongPressGestureRecognizer({name: 'press'})), + ); + expect(row.getAttribute('data-astryx-touch-action')).toBe('manipulation'); + pointer.set(false); + expect(row.hasAttribute('data-astryx-touch-action')).toBe(false); + // The swipe's own touch-action stays. + expect(box.getAttribute('data-astryx-touch-action')).toBe( + 'pan-y pinch-zoom', + ); + const finger = {id: 3, x: 5, y: 5}; + const moved = {id: 3, x: 60, y: 5}; + touch('touchstart', box, [finger], [finger], 0); + touch('touchmove', box, [moved], [moved], 16); + touch('touchend', box, [moved], [], 32); + expect(states).toEqual(expect.arrayContaining(['began', 'ended'])); + }); + + it('warns once in development when something attaches where it is fine', () => { + const warn = quietWarning(); + detachers.push(pointerQuery(false).restore); + const {a, b} = mount('
'); + detachers.push( + host.attach(a, new PanGestureRecognizer()), + host.attach(b, new TapGestureRecognizer()), + ); + expect(warn).toHaveBeenCalledTimes(1); + expect(String(warn.mock.calls[0][0])).toContain( + 'where the primary pointer is fine', + ); + }); +}); + +describe('an action group under a fine pointer (family:action-groups FR5; spec:AST-094 FR19, FR24)', () => { + const listenerTypes = (spy: ReturnType) => + (spy.mock.calls as unknown[][]) + .map(call => call[0] as string) + .filter(t => /^(touch|pointer|click|blur|scroll|contextmenu)/.test(t)); + + it('keeps one passive pointerdown listener on the group while idle, and none for a recognizer that is not a group', () => { + const {group, other} = mount( + '
', + ); + const added = vi.spyOn(document, 'addEventListener'); + const viewAdded = vi.spyOn(window, 'addEventListener'); + const groupAdded = vi.spyOn(group, 'addEventListener'); + const otherAdded = vi.spyOn(other, 'addEventListener'); + added.mockClear(); + viewAdded.mockClear(); + const fine = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => false, + }); + detachers.push( + fine.attach(group, new TapGestureRecognizer(), {actionGroup: true}), + fine.attach(other, new TapGestureRecognizer()), + ); + expect(listenerTypes(added)).toEqual([]); + expect(listenerTypes(viewAdded)).toEqual([]); + expect(groupAdded.mock.calls).toEqual([ + ['pointerdown', expect.any(Function), {passive: true}], + ]); + expect(otherAdded).not.toHaveBeenCalled(); + }); + + it('adds document listeners only during a press, removes them at the release, and serves the press to the group', () => { + const {group, one} = mount( + '
', + ); + const added = vi.spyOn(document, 'addEventListener'); + const removed = vi.spyOn(document, 'removeEventListener'); + const fine = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => false, + }); + const tap = new TapGestureRecognizer(); + const states: string[] = []; + tap.addListener(r => states.push(r.state)); + detachers.push(fine.attach(group, tap, {actionGroup: true})); + added.mockClear(); + removed.mockClear(); + pointer('pointerdown', one, { + pointerType: 'mouse', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + expect(listenerTypes(added)).toEqual( + expect.arrayContaining(['pointermove', 'pointerup', 'pointercancel']), + ); + expect(listenerTypes(removed)).toEqual([]); + pointer('pointerup', one, {pointerType: 'mouse', x: 10, y: 10, time: 80}); + expect(states).toContain('ended'); + for (const type of ['pointermove', 'pointerup', 'pointercancel']) { + expect(listenerTypes(removed)).toContain(type); + } + }); + + it('serves only the group: a recognizer that is not a group, on the press path, takes no part', () => { + const {outer, group, one} = mount( + '
', + ); + const fine = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => false, + }); + const groupTap = new TapGestureRecognizer(); + const outerTap = new TapGestureRecognizer(); + const outerStates: string[] = []; + outerTap.addListener(r => outerStates.push(r.state)); + detachers.push( + fine.attach(group, groupTap, {actionGroup: true}), + fine.attach(outer, outerTap), + ); + pointer('pointerdown', one, { + pointerType: 'mouse', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + pointer('pointerup', one, {pointerType: 'mouse', x: 10, y: 10, time: 80}); + expect(outerStates).toEqual([]); + }); + + it('serves a finger on the group too, and still answers a mouse afterwards', () => { + const {group, one} = mount( + '
', + ); + const fine = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => false, + }); + const tap = new TapGestureRecognizer(); + const states: string[] = []; + tap.addListener(r => states.push(r.state)); + detachers.push(fine.attach(group, tap, {actionGroup: true})); + pointer('pointerdown', one, { + pointerType: 'touch', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + pointer('pointermove', one, { + pointerType: 'touch', + x: 11, + y: 10, + buttons: 1, + time: 16, + }); + pointer('pointerup', one, {pointerType: 'touch', x: 11, y: 10, time: 60}); + expect(states).toEqual(['ended']); + pointer('pointerdown', one, { + pointerType: 'mouse', + x: 10, + y: 10, + buttons: 1, + time: 400, + }); + pointer('pointerup', one, {pointerType: 'mouse', x: 10, y: 10, time: 460}); + expect(states).toEqual(['ended', 'ended']); + }); + + it('removes the group listener on detach, and a coarse pointer keeps the usual document listeners instead', () => { + const {group} = mount('
'); + const groupRemoved = vi.spyOn(group, 'removeEventListener'); + const fine = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => false, + }); + const detach = fine.attach(group, new TapGestureRecognizer(), { + actionGroup: true, + }); + detach(); + expect(groupRemoved).toHaveBeenCalledWith( + 'pointerdown', + expect.any(Function), + ); + const groupAdded = vi.spyOn(group, 'addEventListener'); + const added = vi.spyOn(document, 'addEventListener'); + added.mockClear(); + const coarse = new GestureHost(document, { + clock, + isPrimaryPointerCoarse: () => true, + }); + detachers.push( + coarse.attach(group, new TapGestureRecognizer(), {actionGroup: true}), + ); + expect(listenerTypes(added)).toContain('pointerdown'); + expect( + groupAdded.mock.calls.filter(call => call[0] === 'pointerdown'), + ).toEqual([]); + }); +}); + +describe("inputMode 'pointer-events'", () => { + /** A clock whose timers the test fires. */ + class TimerClock { + time = 0; + private readonly timers = new Map void>(); + private next = 1; + now(): number { + return this.time; + } + setTimeout(callback: () => void): number { + const id = this.next++; + this.timers.set(id, callback); + return id; + } + clearTimeout(id: number): void { + this.timers.delete(id); + } + /** Runs every timer, and the ones they set, at `time`. */ + fireAll(time: number): void { + this.time = time; + while (this.timers.size > 0) { + const due = [...this.timers.values()]; + this.timers.clear(); + for (const callback of due) { + callback(); + } + } + } + } + + it("tracks a finger once, from its pointer events, so a still finger's long press begins", () => { + const {box} = mount('
'); + const timers = new TimerClock(); + const pointerHost = new GestureHost(document, { + clock: timers, + inputMode: 'pointer-events', + isPrimaryPointerCoarse: () => true, + }); + const press = new LongPressGestureRecognizer(); + const states: string[] = []; + press.addListener(r => states.push(r.state)); + detachers.push(pointerHost.attach(box, press)); + // A browser with Touch Events sends both for one finger. + pointer('pointerdown', box, { + pointerType: 'touch', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + const finger = {id: 7, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + expect(press.numberOfTouches).toBe(1); + timers.fireAll(800); + expect(states).toContain('began'); + pointer('pointerup', box, {pointerType: 'touch', x: 10, y: 10, time: 820}); + touch('touchend', box, [finger], [], 820); + expect(states).toContain('ended'); + }); + + it("cancels a claimed finger's touchmove, matching its touches to its pointer", () => { + const {box} = mount('
'); + const pointerHost = new GestureHost(document, { + clock, + inputMode: 'pointer-events', + isPrimaryPointerCoarse: () => true, + }); + const pan = new PanGestureRecognizer({direction: 'horizontal'}); + detachers.push(pointerHost.attach(box, pan)); + pointer('pointerdown', box, { + pointerType: 'touch', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + const finger = {id: 7, x: 10, y: 10}; + touch('touchstart', box, [finger], [finger], 0); + // Before the pan claims, the browser keeps the move. + const early = {...finger, x: 14}; + pointer('pointermove', box, { + pointerType: 'touch', + x: 14, + y: 10, + buttons: 1, + time: 8, + }); + expect(touch('touchmove', box, [early], [early], 8).defaultPrevented).toBe( + false, + ); + // Past its hysteresis the pan owns the finger: its touchmoves are cancelled. + const claimed = {...finger, x: 40}; + pointer('pointermove', box, { + pointerType: 'touch', + x: 40, + y: 10, + buttons: 1, + time: 32, + }); + expect(pan.state).toBe('began'); + expect( + touch('touchmove', box, [claimed], [claimed], 32).defaultPrevented, + ).toBe(true); + }); +}); + +describe('Touch Events from a browser that exposes no ontouchstart', () => { + it('takes a finger once, from its pointer events, so a one-finger long press recognizes (touch emulation turned on after load)', () => { + // Chromium fixes `ontouchstart` when the page is created; turning touch + // emulation on later sends Touch Events without it. + let owner: object | null = window; + let descriptor: PropertyDescriptor | undefined; + while (owner != null) { + descriptor = Object.getOwnPropertyDescriptor(owner, 'ontouchstart'); + if (descriptor != null) { + break; + } + owner = Object.getPrototypeOf(owner) as object | null; + } + if (owner != null && descriptor != null) { + const from = owner; + const saved = descriptor; + delete (from as {ontouchstart?: unknown}).ontouchstart; + detachers.push(() => Object.defineProperty(from, 'ontouchstart', saved)); + } + expect('ontouchstart' in window).toBe(false); + const {box} = mount('
'); + const press = new LongPressGestureRecognizer({name: 'press'}); + detachers.push(host.attach(box, press)); + const finger = {id: 1, x: 10, y: 10}; + pointer('pointerdown', box, { + id: 7, + pointerType: 'touch', + x: 10, + y: 10, + buttons: 1, + time: 0, + }); + touch('touchstart', box, [finger], [finger], 0); + host.arena.advanceTime(600); + expect(press.state).toBe('began'); + }); +}); diff --git a/packages/core/src/gestures/__tests__/HoverGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/HoverGestureRecognizer.test.ts new file mode 100644 index 0000000000000..6f1ed17faa15b --- /dev/null +++ b/packages/core/src/gestures/__tests__/HoverGestureRecognizer.test.ts @@ -0,0 +1,36 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file HoverGestureRecognizer.test.ts + * @input Hover entries, moves and exits from a mouse or a hovering pen + * @output HoverGestureRecognizer's state machine: enter, move and exit + * @position Unit tests of packages/core/src/gestures/HoverGestureRecognizer.ts + */ + +import {describe, expect, it} from 'vitest'; +import {HoverGestureRecognizer} from '../HoverGestureRecognizer'; + +describe('HoverGestureRecognizer', () => { + it('begins on enter, changes on move and ends on exit', () => { + const hover = new HoverGestureRecognizer(); + const states: string[] = []; + hover.addListener(r => states.push(r.state)); + hover.hoverEntered({x: 1, y: 1}, 'mouse'); + hover.hoverMoved({x: 2, y: 2}); + hover.hoverExited({x: 3, y: 3}); + expect(states).toEqual(['began', 'changed', 'ended']); + expect(hover.state).toBe('possible'); + expect(hover.claimsTouches).toBe(false); + }); + + it('starts over at the next enter after a hover was cancelled mid-way', () => { + const hover = new HoverGestureRecognizer(); + const states: string[] = []; + hover.addListener(r => states.push(r.state)); + hover.hoverEntered({x: 1, y: 1}, 'mouse'); + hover.setEnabled(false); + hover.setEnabled(true); + hover.hoverEntered({x: 4, y: 4}, 'mouse'); + expect(states).toEqual(['began', 'cancelled', 'began']); + }); +}); diff --git a/packages/core/src/gestures/__tests__/LongPressGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/LongPressGestureRecognizer.test.ts index d1c83b9466072..2686bd86f9b83 100644 --- a/packages/core/src/gestures/__tests__/LongPressGestureRecognizer.test.ts +++ b/packages/core/src/gestures/__tests__/LongPressGestureRecognizer.test.ts @@ -9,7 +9,9 @@ import {describe, expect, it} from 'vitest'; import {UIKIT_GESTURES} from '../../motion/uikit/gestures'; +import type {SelectionLock} from '../GestureRecognizer'; import {LongPressGestureRecognizer} from '../LongPressGestureRecognizer'; +import {lockSelection} from '../selectionLock'; import {createHarness} from './harness'; describe('LongPressGestureRecognizer', () => { @@ -139,3 +141,27 @@ describe('LongPressGestureRecognizer', () => { expect(h.log).toContain('press:ended@800'); }); }); + +describe('LongPressGestureRecognizer selection lock (spec:AST-094 FR41)', () => { + it('brings the lock for a hold, none for a touch-down detector at 0 ms, and none when told null', () => { + expect(new LongPressGestureRecognizer().selectionLock).toBe(lockSelection); + expect( + new LongPressGestureRecognizer({minimumPressDuration: 300}).selectionLock, + ).toBe(lockSelection); + // A 0 ms long press recognizes at touch-down: it has no hold to act on, + // so the browser keeps its native long press. + expect( + new LongPressGestureRecognizer({minimumPressDuration: 0}).selectionLock, + ).toBeNull(); + expect( + new LongPressGestureRecognizer({selectionLock: null}).selectionLock, + ).toBeNull(); + const own: SelectionLock = () => () => {}; + expect( + new LongPressGestureRecognizer({ + minimumPressDuration: 0, + selectionLock: own, + }).selectionLock, + ).toBe(own); + }); +}); diff --git a/packages/core/src/gestures/__tests__/PanGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/PanGestureRecognizer.test.ts index 1cf18ec0b6549..04049b935b608 100644 --- a/packages/core/src/gestures/__tests__/PanGestureRecognizer.test.ts +++ b/packages/core/src/gestures/__tests__/PanGestureRecognizer.test.ts @@ -8,15 +8,15 @@ */ import {describe, expect, it} from 'vitest'; -import {UIKIT_GESTURES} from '../../motion/uikit/gestures'; +import {UIKIT_PAN, UIKIT_VELOCITY} from '../../motion/uikit/gestures'; import {PanGestureRecognizer} from '../PanGestureRecognizer'; -import {createHarness, scrollProbe} from './harness'; +import {createHarness} from './harness'; describe('PanGestureRecognizer', () => { it('begins past the hysteresis and leaves the threshold out of the translation', () => { const h = createHarness(); const pan = new PanGestureRecognizer({name: 'pan'}); - const hysteresis = UIKIT_GESTURES.pan.hysteresis.value; + const hysteresis = UIKIT_PAN.hysteresis.value; h.finger(1).down(100, 100, 0, {recognizers: [pan]}); h.finger(1).move(100 + hysteresis - 0.1, 100, 16); expect(pan.state).toBe('possible'); @@ -75,7 +75,7 @@ describe('PanGestureRecognizer', () => { }); it('releases at 0 after a rest longer than the stillness interval (AST-100 FR2)', () => { - const stillness = UIKIT_GESTURES.velocity.stillness.value; + const stillness = UIKIT_VELOCITY.stillness.value; const release = (liftAt: number) => { const h = createHarness(); const pan = new PanGestureRecognizer({name: 'pan'}); diff --git a/packages/core/src/gestures/__tests__/PinchGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/PinchGestureRecognizer.test.ts new file mode 100644 index 0000000000000..bccdf40aca43d --- /dev/null +++ b/packages/core/src/gestures/__tests__/PinchGestureRecognizer.test.ts @@ -0,0 +1,74 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file PinchGestureRecognizer.test.ts + * @input Synthetic touches with explicit timestamps through a GestureArena + * @output PinchGestureRecognizer's state machine: the hysteresis, the scale against the moved starting span, the velocity, extra fingers, re-basing + * @position Unit tests of packages/core/src/gestures/PinchGestureRecognizer.ts + */ + +import {describe, expect, it} from 'vitest'; +import {PinchGestureRecognizer} from '../PinchGestureRecognizer'; +import {createHarness} from './harness'; + +describe('PinchGestureRecognizer', () => { + it('begins after the hysteresis and reports scale and velocity', () => { + const h = createHarness(); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + h.finger(1).down(100, 100, 0, {recognizers: [pinch]}); + h.finger(2).down(200, 100, 0, {recognizers: [pinch]}); + h.finger(2).move(205, 100, 16); + expect(pinch.state).toBe('possible'); + h.finger(2).move(210, 100, 32); + expect(pinch.state).toBe('began'); + // UIKit: against the starting span moved by the hysteresis (100 + 8). + expect(pinch.scale).toBeCloseTo(110 / 108, 5); + for (let t = 48; t <= 160; t += 16) { + h.finger(2).move(210 + (t - 32) * 0.1, 100, t); + } + // 0.1 px/ms over 108 px. + expect(pinch.velocity).toBeCloseTo(100 / 108, 5); + h.finger(1).up(170); + expect(h.log).toContain('pinch:ended@170'); + }); + + it('ignores a third finger and can be re-based', () => { + const h = createHarness(); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + h.finger(1).down(0, 0, 0, {recognizers: [pinch]}); + h.finger(2).down(100, 0, 0, {recognizers: [pinch]}); + h.finger(3).down(50, 50, 5, {recognizers: [pinch]}); + expect(pinch.numberOfTouches).toBe(2); + h.finger(2).move(150, 0, 16); + pinch.scale = 1; + h.finger(2).move(300, 0, 32); + expect(pinch.scale).toBeCloseTo(2, 5); + }); + + it('lets the velocity settle when the span holds while the fingers move', () => { + const h = createHarness(); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + h.finger(1).down(100, 100, 0, {recognizers: [pinch]}); + h.finger(2).down(200, 100, 0, {recognizers: [pinch]}); + let time = 0; + for (let x = 210; x <= 260; x += 10) { + time += 16; + h.finger(2).move(x, 100, time); + } + const spreading = pinch.velocity; + expect(spreading).toBeGreaterThan(0); + // The second finger circles the first at exactly the same distance (160: + // 3-4-5 triangles), so the span holds while it moves. + for (const [dx, dy] of [ + [128, 96], + [96, 128], + [0, 160], + [-96, 128], + [-128, 96], + ]) { + time += 16; + h.finger(2).move(100 + dx, 100 + dy, time); + } + expect(Math.abs(pinch.velocity)).toBeLessThan(spreading * 0.25); + }); +}); diff --git a/packages/core/src/gestures/__tests__/PressTracker.test.ts b/packages/core/src/gestures/__tests__/PressTracker.test.ts new file mode 100644 index 0000000000000..7ea256b92160f --- /dev/null +++ b/packages/core/src/gestures/__tests__/PressTracker.test.ts @@ -0,0 +1,262 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file PressTracker.test.ts + * @input A PressTracker fed through a GestureArena by synthetic touches + * @output UIControl semantics: events, the expanded tracking bounds, the + * highlight timing inside and outside scrollers, cancellation + * @position Unit tests of packages/core/src/gestures/PressTracker.ts + */ + +import {afterEach, beforeEach, describe, expect, it, vi} from 'vitest'; +import {UIKIT_GESTURES} from '../../motion/uikit/gestures'; +import {PanGestureRecognizer} from '../PanGestureRecognizer'; +import {PRESS_PROFILES, PressTracker, type ControlEvent} from '../PressTracker'; +import {TapGestureRecognizer} from '../TapGestureRecognizer'; +import {createHarness, scrollProbe} from './harness'; + +const BOUNDS = {left: 100, top: 100, right: 200, bottom: 140}; +const EXPANSION = UIKIT_GESTURES.control.trackingExpansion.value; +const DELAY = UIKIT_GESTURES.press.contentTouchDelay.value; +const FLASH = UIKIT_GESTURES.press.quickTapFlash.value; + +function setup() { + const events: ControlEvent[] = []; + const highlights: boolean[] = []; + const tracker = new PressTracker({ + getBounds: () => BOUNDS, + onControlEvent: event => events.push(event), + onHighlightChange: value => highlights.push(value), + }); + return {events, highlights, tracker, h: createHarness()}; +} + +describe('PressTracker', () => { + beforeEach(() => { + vi.useFakeTimers(); + }); + afterEach(() => { + vi.useRealTimers(); + }); + + it('a keepsTouch tracker keeps only the touch whose press it delivered: a quick tap leaves the next touch free until its own press (spec:AST-094 FR19)', () => { + const tracker = new PressTracker({ + getBounds: () => BOUNDS, + keepsTouch: true, + highlightDelay: 'always', + }); + const h = createHarness(); + // A tap quicker than the delay: the press shows at the lift, for a flash. + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + expect(tracker.keepsTouch?.(1)).toBe(false); + h.finger(1).up(DELAY / 2); + expect(tracker.keepsTouch?.(1)).toBe(false); + // The next touch is not kept from its first move: its press waits for + // the delay, so a scroll that starts on the control is the browser's. + h.finger(2).down(150, 120, DELAY, {recognizers: [], consumer: tracker}); + expect(tracker.keepsTouch?.(2)).toBe(false); + vi.advanceTimersByTime(DELAY); + expect(tracker.keepsTouch?.(2)).toBe(true); + h.finger(2).up(DELAY * 3); + expect(tracker.keepsTouch?.(2)).toBe(false); + }); + + it('highlights in the same frame outside a scroller and acts on touch up inside', () => { + const {events, highlights, tracker, h} = setup(); + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + expect(highlights).toEqual([true]); + h.finger(1).move(152, 121, 16); + h.finger(1).up(80); + expect(events).toEqual(['touchDown', 'touchDragInside', 'touchUpInside']); + expect(highlights).toEqual([true, false]); + }); + + it('keeps tracking inside the expanded bounds and exits past them', () => { + const {events, highlights, tracker, h} = setup(); + const finger = h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: tracker, + }); + finger.move(200 + EXPANSION, 120, 16); + expect(tracker.isTouchInside).toBe(true); + finger.move(200 + EXPANSION + 1, 120, 32); + expect(tracker.isTouchInside).toBe(false); + finger.move(150, 120, 48); + finger.up(64, 200 + EXPANSION + 10, 120); + expect(events).toEqual([ + 'touchDown', + 'touchDragInside', + 'touchDragExit', + 'touchDragOutside', + 'touchDragEnter', + 'touchDragInside', + 'touchUpOutside', + ]); + expect(highlights).toEqual([true, false, true, false]); + }); + + it('waits the content-touch delay inside a scroller, and flashes a quicker tap', () => { + const {highlights, tracker, h} = setup(); + const probe = scrollProbe(() => true, true); + h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: tracker, + scroll: probe, + }); + expect(highlights).toEqual([]); + vi.advanceTimersByTime(DELAY - 1); + expect(highlights).toEqual([]); + vi.advanceTimersByTime(1); + expect(highlights).toEqual([true]); + h.finger(1).up(400); + expect(highlights).toEqual([true, false]); + + const quick = setup(); + quick.h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: quick.tracker, + scroll: probe, + }); + quick.h.finger(1).up(60); + expect(quick.highlights).toEqual([true]); + vi.advanceTimersByTime(FLASH); + expect(quick.highlights).toEqual([true, false]); + }); + + it('never paints a pending press whose finger travelled (a flick)', () => { + const {highlights, tracker, h} = setup(); + const probe = scrollProbe(() => true, true); + h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: tracker, + scroll: probe, + }); + h.finger(1).move(150, 140, 20); + vi.advanceTimersByTime(DELAY * 2); + expect(highlights).toEqual([]); + }); + + it('is cancelled, with no release, when a recognizer takes the touch', () => { + const {events, highlights, tracker, h} = setup(); + const pan = new PanGestureRecognizer({name: 'pan'}); + h.finger(1).down(150, 120, 0, {recognizers: [pan], consumer: tracker}); + h.finger(1).move(175, 120, 16); + expect(events).toEqual(['touchDown', 'touchCancel']); + expect(highlights).toEqual([true, false]); + }); + + it('is cancelled when the browser takes the touch to scroll', () => { + const {events, tracker, h} = setup(); + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + h.finger(1).takenByBrowser(30); + expect(events).toEqual(['touchDown', 'touchCancel']); + expect(tracker.isTracking).toBe(false); + }); + + it('tracks one finger at a time', () => { + const {events, tracker, h} = setup(); + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + h.finger(2).down(160, 120, 10, {recognizers: [], consumer: tracker}); + h.finger(2).up(20); + h.finger(1).up(30); + expect(events).toEqual(['touchDown', 'touchUpInside']); + }); +}); + +describe('PressTracker cell profile', () => { + beforeEach(() => { + vi.useFakeTimers(); + }); + afterEach(() => { + vi.useRealTimers(); + }); + + function cell() { + const events: ControlEvent[] = []; + const highlights: boolean[] = []; + const tracker = new PressTracker({ + getBounds: () => BOUNDS, + ...PRESS_PROFILES.cell, + onControlEvent: event => events.push(event), + onHighlightChange: value => highlights.push(value), + }); + return {events, highlights, tracker, h: createHarness()}; + } + + it('highlights at once outside a scroller: the delay applies only inside one (AST-094 FR11)', () => { + const {highlights, tracker, h} = cell(); + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + expect(highlights).toEqual([true]); + vi.advanceTimersByTime(DELAY); + expect(highlights).toEqual([true]); + h.finger(1).up(DELAY + 40); + expect(highlights).toEqual([true, false]); + }); + + it('waits the content-touch delay inside a scroller', () => { + const {highlights, tracker, h} = cell(); + h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: tracker, + scroll: scrollProbe(() => true, true), + }); + expect(highlights).toEqual([]); + vi.advanceTimersByTime(DELAY); + expect(highlights).toEqual([true]); + h.finger(1).up(DELAY + 40); + expect(highlights).toEqual([true, false]); + }); + + it('is cancelled for good once the finger travels past the pan hysteresis', () => { + const {events, highlights, tracker, h} = cell(); + const finger = h.finger(1).down(150, 120, 0, { + recognizers: [], + consumer: tracker, + }); + vi.advanceTimersByTime(DELAY); + const limit = UIKIT_GESTURES.pan.hysteresis.value; + finger.move(150, 120 + limit, DELAY + 16); + expect(tracker.isTracking).toBe(true); + finger.move(150, 120 + limit + 1, DELAY + 32); + expect(tracker.isTracking).toBe(false); + // Back inside: no re-highlight, and the lift does nothing. + finger.move(150, 120, DELAY + 48); + finger.up(DELAY + 64); + expect(events).toEqual(['touchDown', 'touchDragInside', 'touchCancel']); + expect(highlights).toEqual([true, false]); + }); + + it('shows a quicker tap at the lift straight into the release, with no flash', () => { + const {events, highlights, tracker, h} = cell(); + h.finger(1).down(150, 120, 0, {recognizers: [], consumer: tracker}); + h.finger(1).up(60); + expect(events).toEqual(['touchDown', 'touchUpInside']); + expect(highlights).toEqual([true, false]); + }); + + it('keeps the control profile on today defaults, with the button release clock', () => { + expect(PRESS_PROFILES.control.highlightDelay).toBe('inScrollers'); + expect(PRESS_PROFILES.control.quickTapFlash).toBe(FLASH); + expect(PRESS_PROFILES.cell.cancelTravel).toBe( + UIKIT_GESTURES.pan.hysteresis.value, + ); + }); +}); + +describe('PressTracker in the arena', () => { + it("lets a control veto an ancestor's single tap (UIControl's rule)", () => { + const h = createHarness(); + const button = new PressTracker({getBounds: () => null, view: 'button'}); + const tap = new TapGestureRecognizer({name: 'tap'}); + tap.view = 'row'; + const double = new TapGestureRecognizer({ + name: 'double', + numberOfTapsRequired: 2, + }); + double.view = 'row'; + h.finger(1).down(0, 0, 0, {recognizers: [tap, double], consumer: button}); + expect(h.finger(1).up(50)).toBe('defer'); + expect(h.log[0]).toBe('tap:failed@50'); + expect(double.state).toBe('possible'); + }); +}); diff --git a/packages/core/src/gestures/__tests__/RotationGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/RotationGestureRecognizer.test.ts new file mode 100644 index 0000000000000..4b472cdb52799 --- /dev/null +++ b/packages/core/src/gestures/__tests__/RotationGestureRecognizer.test.ts @@ -0,0 +1,82 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file RotationGestureRecognizer.test.ts + * @input Synthetic touches with explicit timestamps through a GestureArena + * @output RotationGestureRecognizer's state machine: the hysteresis and unwrapping past half a turn + * @position Unit tests of packages/core/src/gestures/RotationGestureRecognizer.ts + */ + +import {describe, expect, it} from 'vitest'; +import {RotationGestureRecognizer} from '../RotationGestureRecognizer'; +import {createHarness} from './harness'; + +describe('RotationGestureRecognizer', () => { + it('begins after the hysteresis and unwraps past half a turn', () => { + const h = createHarness(); + const rotation = new RotationGestureRecognizer({name: 'rotation'}); + h.finger(1).down(0, 0, 0, {recognizers: [rotation]}); + h.finger(2).down(100, 0, 0, {recognizers: [rotation]}); + const at = (angle: number, time: number) => + h.finger(2).move(100 * Math.cos(angle), 100 * Math.sin(angle), time); + at((3 * Math.PI) / 180, 16); + expect(rotation.state).toBe('possible'); + at((6 * Math.PI) / 180, 32); + expect(rotation.state).toBe('began'); + let time = 32; + for (let degrees = 30; degrees <= 270; degrees += 30) { + time += 16; + at((degrees * Math.PI) / 180, time); + } + expect(rotation.rotation).toBeCloseTo((270 * Math.PI) / 180, 5); + expect(rotation.velocity).toBeGreaterThan(0); + }); + + it('setting rotation (UIKit\u2019s incremental read) never moves the velocity', () => { + const h = createHarness(); + const rotation = new RotationGestureRecognizer({name: 'rotation'}); + h.finger(1).down(0, 0, 0, {recognizers: [rotation]}); + h.finger(2).down(100, 0, 0, {recognizers: [rotation]}); + const at = (degrees: number, time: number) => + h + .finger(2) + .move( + 100 * Math.cos((degrees * Math.PI) / 180), + 100 * Math.sin((degrees * Math.PI) / 180), + time, + ); + let time = 0; + for (let degrees = 10; degrees <= 60; degrees += 10) { + time += 16; + at(degrees, time); + // Read and reset each frame, as an incremental consumer does. + rotation.rotation = 0; + } + expect(rotation.velocity).toBeGreaterThan(0); + }); + + it('lets the velocity settle when the angle holds while the fingers move', () => { + const h = createHarness(); + const rotation = new RotationGestureRecognizer({name: 'rotation'}); + h.finger(1).down(0, 0, 0, {recognizers: [rotation]}); + h.finger(2).down(100, 0, 0, {recognizers: [rotation]}); + let time = 0; + for (const degrees of [10, 20, 40, 60, 80]) { + time += 16; + const a = (degrees * Math.PI) / 180; + h.finger(2).move(100 * Math.cos(a), 100 * Math.sin(a), time); + } + // Straight up from the first finger: exactly a quarter turn. + time += 16; + h.finger(2).move(0, 100, time); + const turning = rotation.velocity; + expect(turning).toBeGreaterThan(0); + // The second finger slides outward along the same line: the angle holds + // exactly while the finger moves. + for (let y = 110; y <= 160; y += 10) { + time += 16; + h.finger(2).move(0, y, time); + } + expect(Math.abs(rotation.velocity)).toBeLessThan(turning * 0.25); + }); +}); diff --git a/packages/core/src/gestures/__tests__/ScreenEdgePanGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/ScreenEdgePanGestureRecognizer.test.ts new file mode 100644 index 0000000000000..0ca3efc69ccb5 --- /dev/null +++ b/packages/core/src/gestures/__tests__/ScreenEdgePanGestureRecognizer.test.ts @@ -0,0 +1,220 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file ScreenEdgePanGestureRecognizer.test.ts + * @input Synthetic touches with explicit timestamps through a GestureArena + * @output ScreenEdgePanGestureRecognizer's state machine (the edge band, RTL edges, inward movement) and its edge-zone touch-action + * @position Unit tests of packages/core/src/gestures/ScreenEdgePanGestureRecognizer.ts + */ + +import {describe, expect, it} from 'vitest'; +import {UIKIT_SCREEN_EDGE_PAN} from '../../motion/uikit/gestures'; +import {ALL_NATIVE_TOUCH} from '../GestureRecognizer'; +import {ScreenEdgePanGestureRecognizer} from '../ScreenEdgePanGestureRecognizer'; +import {createHarness} from './harness'; + +describe('ScreenEdgePanGestureRecognizer', () => { + it('begins from the leading band and fails elsewhere at touch-down', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.finger(1) + .down(10, 300, 0, {recognizers: [edge]}) + .drag(40, 4, 50); + expect(h.log[0]).toMatch(/^edge:began@/); + expect(edge.activeEdge).toBe('left'); + + const middle = createHarness(); + const miss = new ScreenEdgePanGestureRecognizer({name: 'miss'}); + middle.finger(1).down(60, 300, 0, {recognizers: [miss]}); + expect(middle.log).toEqual(['miss:failed@0']); + }); + + it('resolves the leading edge under RTL and requires inward movement', () => { + const h = createHarness(); + h.environment.rtl = true; + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.finger(1) + .down(385, 300, 0, {recognizers: [edge]}) + .drag(-40, 2, 50); + expect(edge.activeEdge).toBe('right'); + expect(h.log[0]).toMatch(/^edge:began@/); + + const outward = createHarness(); + const back = new ScreenEdgePanGestureRecognizer({name: 'back'}); + outward + .finger(1) + .down(10, 300, 0, {recognizers: [back]}) + .drag(-8, 30, 50); + expect(back.state).toBe('failed'); + }); + + it('a drag 5 pt inward and then down the edge is the page\u2019s scroll: it fails and claims nothing (spec:AST-094 FR19)', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.finger(1) + .down(10, 300, 0, {recognizers: [edge]}) + .drag(5, 0, 30) + .drag(0, 20, 50); + expect(edge.state).toBe('failed'); + expect(h.claims).toEqual([]); + }); + + it('leaves a view wider than the edge zone every touch behavior, so its scrolling is untouched (spec:AST-094 FR18)', () => { + expect(new ScreenEdgePanGestureRecognizer().nativeTouchBehavior).toEqual( + ALL_NATIVE_TOUCH, + ); + const page = new ScreenEdgePanGestureRecognizer(); + page.view = viewOfSize(390, 844); + expect(page.nativeTouchBehavior).toEqual(ALL_NATIVE_TOUCH); + // A pan from several edges too, off their zones. + expect( + new ScreenEdgePanGestureRecognizer({edges: ['left', 'right']}) + .nativeTouchBehavior, + ).toEqual(ALL_NATIVE_TOUCH); + }); + + it('takes the axis across its edge in the edge zone, a view no wider than the edge region (spec:AST-094 FR18)', () => { + const leading = new ScreenEdgePanGestureRecognizer(); + leading.view = viewOfSize(leading.edgeRegion, 844); + expect(leading.nativeTouchBehavior).toEqual({ + panX: false, + panY: true, + pinchZoom: true, + doubleTapZoom: false, + }); + const bottom = new ScreenEdgePanGestureRecognizer({edges: ['bottom']}); + bottom.view = viewOfSize(390, 12); + expect(bottom.nativeTouchBehavior).toEqual({ + panX: true, + panY: false, + pinchZoom: true, + doubleTapZoom: false, + }); + // One pixel wider than the band is not the edge zone. + const wide = new ScreenEdgePanGestureRecognizer(); + wide.view = viewOfSize(wide.edgeRegion + 1, 844); + expect(wide.nativeTouchBehavior).toEqual(ALL_NATIVE_TOUCH); + }); + + it('takes a strip authored at the edge region as the edge zone though layout lays it out at 40.203 px', () => { + const strip = new ScreenEdgePanGestureRecognizer(); + // 40.2 px snapped to layout's 1/64 px. + strip.view = viewOfSize(Math.ceil(strip.edgeRegion * 64) / 64, 844); + expect(strip.nativeTouchBehavior).toEqual({ + panX: false, + panY: true, + pinchZoom: true, + doubleTapZoom: false, + }); + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.environment.view = {width: 40.203, height: 844}; + const hysteresis = UIKIT_SCREEN_EDGE_PAN.hysteresis.value; + h.finger(1) + .down(10, 300, 0, {recognizers: [edge]}) + .drag(hysteresis - 1, 0, 30); + expect(edge.state).toBe('possible'); + h.finger(1).drag(2, 0, 50); + expect(h.log[0]).toMatch(/^edge:began@/); + }); + + it('in the edge zone, still decides at its hysteresis', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.environment.view = {width: edge.edgeRegion, height: 844}; + const hysteresis = UIKIT_SCREEN_EDGE_PAN.hysteresis.value; + h.finger(1) + .down(10, 300, 0, {recognizers: [edge]}) + .drag(hysteresis - 1, 0, 30); + expect(edge.state).toBe('possible'); + h.finger(1).drag(2, 0, 50); + expect(h.log[0]).toMatch(/^edge:began@/); + }); +}); + +describe('where an in-band touch decides (spec:AST-094 FR18, DEC-22)', () => { + it('on an element that reaches beyond the band, at 10 px, before the browser commits to a scroll', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + // A stack container: as wide as the screen, the touch 4 px from its edge. + h.environment.view = {width: 390, height: 844}; + h.finger(1) + .down(4, 300, 0, {recognizers: [edge]}) + .drag(9, 0, 30, 9); + expect(edge.state).toBe('possible'); + h.finger(1).drag(1, 0, 10, 1); + expect(h.log[0]).toMatch(/^edge:began@/); + }); + + it('on an element within the band, at 15 px', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + h.environment.view = {width: edge.edgeRegion, height: 844}; + const hysteresis = UIKIT_SCREEN_EDGE_PAN.hysteresis.value; + h.finger(1) + .down(4, 300, 0, {recognizers: [edge]}) + .drag(hysteresis - 1, 0, 30, 14); + expect(edge.state).toBe('possible'); + h.finger(1).drag(1, 0, 10, 1); + expect(h.log[0]).toMatch(/^edge:began@/); + }); + + it('from several edges, at 10 px on an element beyond their bands', () => { + const h = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({ + name: 'edge', + edges: ['left', 'right'], + }); + h.environment.view = {width: 390, height: 844}; + h.finger(1) + .down(4, 300, 0, {recognizers: [edge]}) + .drag(9, 0, 30, 9); + expect(edge.state).toBe('possible'); + h.finger(1).drag(1, 0, 10, 1); + expect(h.log[0]).toMatch(/^edge:began@/); + }); +}); + +describe('a pan from several edges (spec:AST-094 FR18)', () => { + it('outside every edge zone leaves the browser every touch behavior', () => { + const edge = new ScreenEdgePanGestureRecognizer({ + edges: ['left', 'bottom'], + }); + edge.view = viewOfSize(390, 844); + expect(edge.nativeTouchBehavior).toEqual(ALL_NATIVE_TOUCH); + }); + + it('inside a side edge zone claims pan-x only', () => { + const edge = new ScreenEdgePanGestureRecognizer({ + edges: ['left', 'bottom'], + }); + edge.view = viewOfSize(edge.edgeRegion, 844); + expect(edge.nativeTouchBehavior).toEqual({ + panX: false, + panY: true, + pinchZoom: true, + doubleTapZoom: false, + }); + }); + + it('inside a bottom edge zone claims pan-y only', () => { + const edge = new ScreenEdgePanGestureRecognizer({ + edges: ['left', 'bottom'], + }); + edge.view = viewOfSize(390, edge.edgeRegion); + expect(edge.nativeTouchBehavior).toEqual({ + panX: true, + panY: false, + pinchZoom: true, + doubleTapZoom: false, + }); + }); +}); + +/** An element whose box has `width` × `height`, as layout would give it. */ +function viewOfSize(width: number, height: number): HTMLElement { + const view = document.createElement('div'); + view.getBoundingClientRect = () => + ({width, height, top: 0, left: 0, right: width, bottom: height}) as DOMRect; + return view; +} diff --git a/packages/core/src/gestures/__tests__/SwipeGestureRecognizer.test.ts b/packages/core/src/gestures/__tests__/SwipeGestureRecognizer.test.ts new file mode 100644 index 0000000000000..d195d7d2e212e --- /dev/null +++ b/packages/core/src/gestures/__tests__/SwipeGestureRecognizer.test.ts @@ -0,0 +1,108 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file SwipeGestureRecognizer.test.ts + * @input Synthetic touches with explicit timestamps through a GestureArena + * @output SwipeGestureRecognizer's state machine: recognition during the motion, and failure when too slow, off-axis or the wrong way + * @position Unit tests of packages/core/src/gestures/SwipeGestureRecognizer.ts + */ + +import {describe, expect, it} from 'vitest'; +import {SwipeGestureRecognizer} from '../SwipeGestureRecognizer'; +import {createHarness} from './harness'; + +describe('SwipeGestureRecognizer', () => { + it('is recognized during the motion, once the distance is covered', () => { + const h = createHarness(); + const swipe = new SwipeGestureRecognizer({name: 'swipe'}); + const finger = h.finger(1).down(0, 0, 0, {recognizers: [swipe]}); + finger.move(30, 2, 30); + expect(swipe.state).toBe('possible'); + finger.move(50, 4, 50); + expect(swipe.recognizedDirection).toBe('right'); + expect(h.log[0]).toBe('swipe:ended@50'); + }); + + it('fails when too slow, off-axis, or the wrong way', () => { + const slow = createHarness(); + const a = new SwipeGestureRecognizer({name: 'a'}); + slow + .finger(1) + .down(0, 0, 0, {recognizers: [a]}) + .move(30, 0, 400); + slow.advance(500); + expect(a.state).toBe('failed'); + + const diagonal = createHarness(); + const b = new SwipeGestureRecognizer({name: 'b'}); + diagonal + .finger(1) + .down(0, 0, 0, {recognizers: [b]}) + .move(45, 55, 60); + // 55 px across is past the 50 px secondary allowance. + expect(b.state).toBe('failed'); + + const wrong = createHarness(); + const c = new SwipeGestureRecognizer({name: 'c', direction: 'left'}); + wrong + .finger(1) + .down(100, 0, 0, {recognizers: [c]}) + .move(160, 0, 60); + expect(c.state).toBe('failed'); + }); + + it('takes several directions and fingers', () => { + const h = createHarness(); + const swipe = new SwipeGestureRecognizer({ + name: 'swipe', + direction: ['up', 'down'], + numberOfTouchesRequired: 2, + }); + h.finger(1).down(0, 200, 0, {recognizers: [swipe]}); + h.finger(2).down(40, 200, 5, {recognizers: [swipe]}); + h.finger(1).move(0, 140, 50); + h.finger(2).move(40, 140, 50); + expect(swipe.recognizedDirection).toBe('up'); + }); + + it('gives the same verdict for the same path however its events are spaced', () => { + // A straight drag to (55, 40): along 55 past the 50 px distance, 40 px + // across within the 50 px allowance. + const sparse = createHarness(); + const a = new SwipeGestureRecognizer({name: 'a'}); + sparse + .finger(1) + .down(0, 0, 0, {recognizers: [a]}) + .move(55, 40, 40); + expect(a.state).toBe('ended'); + + const dense = createHarness(); + const b = new SwipeGestureRecognizer({name: 'b'}); + const finger = dense.finger(1).down(0, 0, 0, {recognizers: [b]}); + for (let step = 1; step <= 10; step += 1) { + finger.move(5.5 * step, 4 * step, 4 * step); + } + expect(b.state).toBe('ended'); + expect(b.recognizedDirection).toBe('right'); + }); + + it('recognizes a 40\u00b0 swipe at 16 px per event as at small steps', () => { + // At 40\u00b0 the travel along reaches 50 px with 42 px across, inside + // the 50 px allowance; at 16 px per event the next event is already 51 px + // across, so the verdict must come from the path between the events. + const angle = (40 * Math.PI) / 180; + for (const step of [2, 16]) { + const h = createHarness(); + const swipe = new SwipeGestureRecognizer({name: 'swipe'}); + const finger = h.finger(1).down(0, 0, 0, {recognizers: [swipe]}); + for (let d = step, t = 4; d <= 96; d += step, t += 4) { + finger.move(d * Math.cos(angle), d * Math.sin(angle), t); + if (swipe.state !== 'possible') { + break; + } + } + expect(swipe.state).toBe('ended'); + expect(swipe.recognizedDirection).toBe('right'); + } + }); +}); diff --git a/packages/core/src/gestures/__tests__/arena.test.ts b/packages/core/src/gestures/__tests__/arena.test.ts index 8cc06fc5fe270..062eda64833e7 100644 --- a/packages/core/src/gestures/__tests__/arena.test.ts +++ b/packages/core/src/gestures/__tests__/arena.test.ts @@ -24,6 +24,7 @@ import { import type {GestureEventContext, GestureTouch} from '../types'; import {GestureArena, type GestureClock} from '../GestureArena'; import {createHarness, RecordingConsumer} from './harness'; +import {internals} from '../recognizerInternals'; type Step = (recognizer: Scripted, event: GestureEventContext) => void; @@ -75,6 +76,11 @@ class Scripted extends GestureRecognizer { this.setState('failed'); } + /** Cancel itself, as a recognizer that gives up mid-gesture does. */ + cancelSelf(): void { + this.setState('cancelled'); + } + /** Recognize at `deadline` on the event clock, unless the touch lifts. */ recognizeAt(deadline: number): void { this.setTimer(deadline, () => this.recognize()); @@ -171,7 +177,7 @@ describe('arena: who wins', () => { h.finger(1).down(0, 0, 0, {recognizers: [single, double]}); h.finger(1).up(50); expect(single.state).toBe('possible'); - expect(single.pending).toBe('ended'); + expect(internals(single).pending).toBe('ended'); h.advance(400); expect(h.log).toEqual([ 'double:failed@400', @@ -210,7 +216,7 @@ describe('arena: who wins', () => { }); h.finger(1).down(0, 0, 0, {recognizers: [single, double]}); h.finger(1).up(50); - expect(single.pending).toBe('ended'); + expect(internals(single).pending).toBe('ended'); }); it('honors shouldBegin and shouldReceiveTouch', () => { @@ -517,6 +523,36 @@ describe('arena: claims', () => { first.up(80); second.up(80); }); + + it('keeps a moving finger claimed until it lifts when its gesture cancels itself on that move, so the drag never turns into the page scroll midway (FR14)', () => { + const h = createHarness(); + // A long press's shape: it begins while the finger rests, then cancels + // itself on the first move past its tolerance. + const press = new Scripted('press', { + began: (self, event) => self.recognizeAt(event.time + 500), + moved: (self, event) => { + const touch = event.allTouches[0]; + if ( + touch != null && + Math.abs(touch.y - touch.startY) > 10 && + isActive(self.state) + ) { + self.cancelSelf(); + } + }, + }); + const finger = h.finger(1).down(0, 0, 0, {recognizers: [press]}); + h.advance(600); + expect(press.state).toBe('began'); + expect(h.arena.isClaimed(1)).toBe(true); + finger.move(0, -11, 616); + expect(press.state).toBe('cancelled'); + expect(h.arena.isClaimed(1)).toBe(true); + finger.move(0, -60, 632); + expect(h.arena.isClaimed(1)).toBe(true); + finger.up(700); + expect(h.arena.isClaimed(1)).toBe(false); + }); }); /** Records the touches that began off its hit path. */ diff --git a/packages/core/src/gestures/__tests__/arenaPinchSwipeEdge.test.ts b/packages/core/src/gestures/__tests__/arenaPinchSwipeEdge.test.ts new file mode 100644 index 0000000000000..54829fb2aeac3 --- /dev/null +++ b/packages/core/src/gestures/__tests__/arenaPinchSwipeEdge.test.ts @@ -0,0 +1,188 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file arenaPinchSwipeEdge.test.ts + * @input GestureArena with pinches, swipes, edge pans and pans, driven by + * synthetic touches with explicit timestamps + * @output Simultaneous recognition (pinch + pan), the first to recognize + * winning without permission, edge pan vs inner pan, swipe vs pan, and a + * delayed touch delivery + * @position Unit tests of packages/core/src/gestures/GestureArena.ts + */ + +import {describe, expect, it} from 'vitest'; +import {UIKIT_PINCH, UIKIT_SWIPE} from '../../motion/uikit/gestures'; +import {GestureRecognizer} from '../GestureRecognizer'; +import {PanGestureRecognizer} from '../PanGestureRecognizer'; +import {PinchGestureRecognizer} from '../PinchGestureRecognizer'; +import {ScreenEdgePanGestureRecognizer} from '../ScreenEdgePanGestureRecognizer'; +import {SwipeGestureRecognizer} from '../SwipeGestureRecognizer'; +import {createHarness, RecordingConsumer} from './harness'; + +/** + * A recognizer that stays possible, even past its finger's lift, until the + * test fails it: something another recognizer can wait on. + */ +class Held extends GestureRecognizer { + private isWaiting = true; + + override isInvolved(): boolean { + return super.isInvolved() || this.isWaiting; + } + + fail(): void { + this.isWaiting = false; + this.setState('failed'); + } +} + +describe('conflict matrix', () => { + it('pinch + pan recognize simultaneously when allowed', () => { + const h = createHarness(); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + const pan = new PanGestureRecognizer({name: 'pan'}); + pinch.recognizeSimultaneouslyWith(pan); + h.finger(1).down(100, 100, 0, {recognizers: [pinch, pan]}); + h.finger(2).down(200, 100, 0, {recognizers: [pinch, pan]}); + h.finger(1).move(80, 120, 16); + h.finger(2).move(240, 120, 16); + expect(h.log.slice(0, 2)).toEqual(['pinch:began@16', 'pan:began@16']); + // Span 100 -> 160, measured against 100 + 8. + expect(pinch.scale).toBeCloseTo(160 / 108, 5); + // The pan began at the first finger's move, centroid (-10, 10), which + // the hysteresis swallows; the second finger's move brought it to (10, 20). + expect(pan.translation()).toEqual({x: 20, y: 10}); + }); + + it('pinch vs pan without permission: the first to recognize wins', () => { + const h = createHarness(); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + const pan = new PanGestureRecognizer({name: 'pan'}); + h.finger(1).down(100, 100, 0, {recognizers: [pinch, pan]}); + h.finger(2).down(200, 100, 0, {recognizers: [pinch, pan]}); + h.finger(1).move(80, 120, 16); + h.finger(2).move(240, 120, 16); + expect(h.log.slice(0, 2)).toEqual(['pinch:began@16', 'pan:failed@16']); + }); + + it('edge pan vs inner pan: in the band the edge pan wins, elsewhere the inner pan runs undelayed', () => { + const band = createHarness(); + const edge = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + const inner = new PanGestureRecognizer({ + name: 'inner', + direction: 'horizontal', + }); + inner.requireToFail(edge); + // A thin edge view: the edge zone, where the pan takes its axis. + band.environment.view = {width: edge.edgeRegion, height: 844}; + band.finger(1).down(8, 300, 0, {recognizers: [inner, edge]}); + band.finger(1).drag(60, 2, 60, 6); + // The edge pan's own hysteresis is 15 px: the second 10 px step. + expect(band.log.slice(0, 2)).toEqual(['edge:began@20', 'inner:failed@20']); + + const middle = createHarness(); + const edge2 = new ScreenEdgePanGestureRecognizer({name: 'edge'}); + const inner2 = new PanGestureRecognizer({ + name: 'inner', + direction: 'horizontal', + }); + inner2.requireToFail(edge2); + middle.finger(1).down(150, 300, 0, {recognizers: [inner2, edge2]}); + // Failed at touch-down, before any movement: nothing waits on it. + expect(middle.log).toEqual(['edge:failed@0']); + middle.finger(1).drag(60, 2, 60, 6); + expect(middle.log[1]).toBe('inner:began@10'); + }); + + it('swipe vs pan: the pan wins by default, the swipe when the pan requires it to fail', () => { + const plain = createHarness(); + const swipe = new SwipeGestureRecognizer({name: 'swipe'}); + const pan = new PanGestureRecognizer({name: 'pan'}); + plain + .finger(1) + .down(0, 0, 0, {recognizers: [swipe, pan]}) + .drag(80, 0, 60, 6); + expect(plain.log.slice(0, 2)).toEqual(['pan:began@10', 'swipe:failed@10']); + + const ordered = createHarness(); + const swipe2 = new SwipeGestureRecognizer({name: 'swipe'}); + const pan2 = new PanGestureRecognizer({name: 'pan'}); + pan2.requireToFail(swipe2); + ordered + .finger(1) + .down(0, 0, 0, {recognizers: [swipe2, pan2]}) + .drag(80, 0, 60, 8); + expect(ordered.log.slice(0, 2)).toEqual([ + 'swipe:ended@37.5', + 'pan:failed@37.5', + ]); + + const slow = createHarness(); + const swipe3 = new SwipeGestureRecognizer({name: 'swipe'}); + const pan3 = new PanGestureRecognizer({name: 'pan'}); + pan3.requireToFail(swipe3); + slow + .finger(1) + .down(0, 0, 0, {recognizers: [swipe3, pan3]}) + .drag(30, 0, 300, 3); + expect(pan3.state).toBe('possible'); + slow.advance(500); + // The swipe ran out of time; the waiting pan begins with what it has. + expect(slow.log.slice(0, 2)).toEqual(['swipe:failed@500', 'pan:began@500']); + }); +}); + +describe('a recognizer waiting on another', () => { + it('a swipe holds its recognition past its lift and its deadline (spec:AST-094 FR15)', () => { + const h = createHarness(); + const held = new Held({name: 'held'}); + const swipe = new SwipeGestureRecognizer({name: 'swipe'}); + swipe.requireToFail(held); + h.finger(1) + .down(0, 0, 0, {recognizers: [swipe, held]}) + .drag(80, 0, 60, 8) + .up(70); + h.advance(UIKIT_SWIPE.maximumDuration.value + 100); + expect(swipe.state).toBe('possible'); + expect(h.log.some(entry => entry.startsWith('swipe:failed'))).toBe(false); + held.fail(); + expect(h.log).toContain( + `swipe:ended@${UIKIT_SWIPE.maximumDuration.value + 100}`, + ); + expect(swipe.recognizedDirection).toBe('right'); + }); + + it('a pinch keeps its start once it passes the hysteresis, so it begins at the scale its fingers show (spec:AST-094 FR15)', () => { + const h = createHarness(); + const held = new Held({name: 'held'}); + const pinch = new PinchGestureRecognizer({name: 'pinch'}); + pinch.requireToFail(held); + h.finger(1).down(100, 100, 0, {recognizers: [pinch, held]}); + h.finger(2).down(200, 100, 0, {recognizers: [pinch, held]}); + // Span 100 -> 120 -> 140 -> 160 while the pinch waits. + h.finger(2).move(220, 100, 16); + h.finger(2).move(240, 100, 32); + h.finger(2).move(260, 100, 48); + expect(pinch.state).toBe('possible'); + held.fail(); + expect(h.log).toContain('pinch:began@48'); + // Measured against 100 + 8 once, not re-based on every move. + const hysteresis = UIKIT_PINCH.hysteresis.value; + expect(pinch.scale).toBeCloseTo(160 / (100 + hysteresis), 5); + }); +}); + +describe('delivery to the view', () => { + it('holds the began with delaysTouchesBegan until the recognizer fails', () => { + const h = createHarness(); + const consumer = new RecordingConsumer(); + const swipe = new SwipeGestureRecognizer({ + name: 'swipe', + delaysTouchesBegan: true, + }); + h.finger(1).down(0, 0, 0, {recognizers: [swipe], consumer}); + expect(consumer.events).toEqual([]); + h.advance(UIKIT_SWIPE.maximumDuration.value); + expect(consumer.events).toEqual(['began:1']); + }); +}); diff --git a/packages/core/src/gestures/__tests__/arenaTapLongPress.test.ts b/packages/core/src/gestures/__tests__/arenaTapLongPress.test.ts index 91bd0b54033b5..24267edb00919 100644 --- a/packages/core/src/gestures/__tests__/arenaTapLongPress.test.ts +++ b/packages/core/src/gestures/__tests__/arenaTapLongPress.test.ts @@ -16,6 +16,7 @@ import {LongPressGestureRecognizer} from '../LongPressGestureRecognizer'; import {TapGestureRecognizer} from '../TapGestureRecognizer'; import {UIKIT_GESTURES} from '../../motion/uikit/gestures'; import {createHarness, RecordingConsumer} from './harness'; +import {internals} from '../recognizerInternals'; const INTERVAL = UIKIT_GESTURES.tap.maximumIntervalBetweenTaps.value; @@ -31,7 +32,7 @@ describe('conflict matrix', () => { h.finger(1).down(0, 0, 0, {recognizers: [single, double]}); h.finger(1).up(80); expect(single.state).toBe('possible'); - expect(single.pending).toBe('ended'); + expect(internals(single).pending).toBe('ended'); h.advance(80 + INTERVAL); expect(h.log).toEqual([ `double:failed@${80 + INTERVAL}`, @@ -41,6 +42,31 @@ describe('conflict matrix', () => { ]); }); + it('a requirement stopped inside deferResolution leaves the recognition waiting on it to a pass a microtask later', async () => { + const h = createHarness(); + const single = new TapGestureRecognizer({name: 'single'}); + const double = new TapGestureRecognizer({ + name: 'double', + numberOfTapsRequired: 2, + }); + single.requireToFail(double); + h.finger(1).down(0, 0, 0, {recognizers: [single, double]}); + h.finger(1).up(80); + // Code outside input (a React Effect) turns the double tap off: it fails + // at once, at the last event's time, and the single tap waiting on it + // resolves only in the next pass. + h.arena.deferResolution(() => double.setEnabled(false)); + expect(h.log).toEqual(['double:failed@80']); + expect(internals(single).pending).toBe('ended'); + await Promise.resolve(); + expect(h.log).toEqual([ + 'double:failed@80', + 'single:ended@80', + 'single:possible@80', + 'double:possible@80', + ]); + }); + it('tap vs double tap, single requires double to fail: a double tap fails the single', () => { const h = createHarness(); const single = new TapGestureRecognizer({name: 'single'}); @@ -95,7 +121,7 @@ describe('conflict matrix', () => { }); h.finger(1).down(0, 0, 0, {recognizers: [single, double]}); h.finger(1).up(50); - expect(single.pending).toBe('ended'); + expect(internals(single).pending).toBe('ended'); }); }); diff --git a/packages/core/src/gestures/__tests__/browser/bareFixture.ts b/packages/core/src/gestures/__tests__/browser/bareFixture.ts new file mode 100644 index 0000000000000..74d99eaf6fc88 --- /dev/null +++ b/packages/core/src/gestures/__tests__/browser/bareFixture.ts @@ -0,0 +1,97 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file bareFixture.ts + * @input Nothing: bundled by esbuild and injected into a blank page + * @output A page with nothing attached: the document's gesture host and the + * recognizers (pan, screen-edge pan, tap, long press) on `window.__api`, for a test to attach what it needs, and + * `window.__pageTouch`, the count of touch listeners on the window and the + * document right now + * @position Browser fixture of gestures.browser.spec.ts for the checks that + * need a page with no other gesture on it (FR19, FR24); test-only + */ + +import {getGestureHost} from '../../GestureHost'; +import {LongPressGestureRecognizer} from '../../LongPressGestureRecognizer'; +import {PanGestureRecognizer} from '../../PanGestureRecognizer'; +import {ScreenEdgePanGestureRecognizer} from '../../ScreenEdgePanGestureRecognizer'; +import {lockSelection} from '../../selectionLock'; +import {TapGestureRecognizer} from '../../TapGestureRecognizer'; + +declare global { + interface Window { + __pageTouch: number; + __api: { + host: ReturnType; + LongPressGestureRecognizer: typeof LongPressGestureRecognizer; + PanGestureRecognizer: typeof PanGestureRecognizer; + ScreenEdgePanGestureRecognizer: typeof ScreenEdgePanGestureRecognizer; + TapGestureRecognizer: typeof TapGestureRecognizer; + lockSelection: typeof lockSelection; + }; + } +} + +// The touch listeners on the window and the document, as the engine sees +// them: what makes the page pay for touches. +window.__pageTouch = 0; +const live = new Set(); +const listenerIds = new WeakMap(); +let nextListenerId = 1; +function keyOf( + target: EventTarget, + type: string, + listener: EventListenerOrEventListenerObject, + options?: boolean | EventListenerOptions, +): string { + let id = listenerIds.get(listener); + if (id == null) { + id = nextListenerId++; + listenerIds.set(listener, id); + } + const capture = + typeof options === 'boolean' ? options : (options?.capture ?? false); + return [target === window ? 'window' : 'document', type, capture, id].join(); +} +// Playwright's own hit-target interceptors listen on the window too; they +// are the harness's, not the page's. +const isPageTouch = (target: EventTarget, type: string) => + (target === window || target === document) && + type.startsWith('touch') && + !(new Error().stack ?? '').includes('InjectedScript'); +const addListener = EventTarget.prototype.addEventListener; +const removeListener = EventTarget.prototype.removeEventListener; +EventTarget.prototype.addEventListener = function ( + this: EventTarget, + type: string, + listener: EventListenerOrEventListenerObject | null, + options?: boolean | AddEventListenerOptions, +) { + if (listener != null && isPageTouch(this, type)) { + live.add(keyOf(this, type, listener, options)); + window.__pageTouch = live.size; + } + addListener.call(this, type, listener, options); +}; +EventTarget.prototype.removeEventListener = function ( + this: EventTarget, + type: string, + listener: EventListenerOrEventListenerObject | null, + options?: boolean | EventListenerOptions, +) { + if (listener != null && isPageTouch(this, type)) { + live.delete(keyOf(this, type, listener, options)); + window.__pageTouch = live.size; + } + removeListener.call(this, type, listener, options); +}; + +document.body.style.margin = '0'; +window.__api = { + host: getGestureHost(document), + LongPressGestureRecognizer, + PanGestureRecognizer, + ScreenEdgePanGestureRecognizer, + TapGestureRecognizer, + lockSelection, +}; diff --git a/packages/core/src/gestures/__tests__/browser/build-fixture.mjs b/packages/core/src/gestures/__tests__/browser/build-fixture.mjs new file mode 100644 index 0000000000000..0d1be1180c45d --- /dev/null +++ b/packages/core/src/gestures/__tests__/browser/build-fixture.mjs @@ -0,0 +1,46 @@ +#!/usr/bin/env node +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file build-fixture.mjs + * @input fixture.ts beside this file + * @output A standalone HTML page (the fixture bundled inline) for running the + * gesture proofs in any browser, including Mobile Safari on the iOS + * simulator + * @position Test-only helper of packages/core/src/gestures + * + * node packages/core/src/gestures/__tests__/browser/build-fixture.mjs out.html + * + * The page records every recognizer transition and every click on + * `window.__gestures`; read it with the Web Inspector or by evaluating + * `JSON.stringify(window.__gestures)`. + */ + +import fs from 'node:fs'; +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {build} from 'esbuild'; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const out = process.argv[2]; +if (out == null) { + console.error('usage: build-fixture.mjs '); + process.exit(1); +} +const result = await build({ + entryPoints: [path.join(here, 'fixture.ts')], + bundle: true, + write: false, + format: 'iife', + platform: 'browser', + target: 'es2022', +}); +const html = ` + + +Gesture fixture + + +`; +fs.writeFileSync(out, html); +console.log(`wrote ${out} (${html.length} bytes)`); diff --git a/packages/core/src/gestures/__tests__/browser/fixture.ts b/packages/core/src/gestures/__tests__/browser/fixture.ts new file mode 100644 index 0000000000000..7f05a8ee73415 --- /dev/null +++ b/packages/core/src/gestures/__tests__/browser/fixture.ts @@ -0,0 +1,260 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file fixture.ts + * @input Nothing: bundled by esbuild and injected into a blank page + * @output A page of gesture hosts (a horizontal pan inside a scroller, a tap + * target, a long press, a pinch, a sheet body that hands its scroll to a + * pan, a passive control: a press consumer under `pan-y` with no + * recognizer) that records every recognizer transition, every consumer + * event and every click on `window.__gestures` + * @position Browser fixture of gestures.chromium.spec.ts; test-only + */ + +import {getGestureHost} from '../../GestureHost'; +import type {GestureRecognizer} from '../../GestureRecognizer'; +import {LongPressGestureRecognizer} from '../../LongPressGestureRecognizer'; +import {PanGestureRecognizer} from '../../PanGestureRecognizer'; +import {lockSelection} from '../../selectionLock'; +import {PinchGestureRecognizer} from '../../PinchGestureRecognizer'; +import {PressTracker} from '../../PressTracker'; +import {TapGestureRecognizer} from '../../TapGestureRecognizer'; +import {internals} from '../../recognizerInternals'; + +interface Entry { + name: string; + state: string; + time: number; + x?: number; + y?: number; + scale?: number; + beganAt?: number | null; + startedAt?: number; +} + +interface ListenerEntry { + id: string; + type: string; + passive: boolean | null; +} + +declare global { + interface Window { + __gestures: Entry[]; + __listeners: ListenerEntry[]; + /** Every touch listener added to the document or the window, by type. */ + __pageTouchListeners: string[]; + /** + * The same, as the page left them once loaded: the test driver adds its + * own window listeners when it first runs a script. + */ + __pageTouchListenersAtLoad: string[]; + } +} + +window.__gestures = []; + +// Every touch listener added to an element with an id, as the engine sees +// it: which elements make a scroll wait for the page. +window.__listeners = []; +window.__pageTouchListeners = []; +const addListener = EventTarget.prototype.addEventListener; +EventTarget.prototype.addEventListener = function ( + this: EventTarget, + type: string, + listener: EventListenerOrEventListenerObject | null, + options?: boolean | AddEventListenerOptions, +) { + if ((this === document || this === window) && type.startsWith('touch')) { + window.__pageTouchListeners.push(type); + } + if (this instanceof Element && this.id !== '' && type.startsWith('touch')) { + window.__listeners.push({ + id: this.id, + type, + passive: typeof options === 'object' ? (options.passive ?? null) : null, + }); + } + addListener.call(this, type, listener, options); +}; + +document.body.style.margin = '0'; +document.body.innerHTML = ` +
+
+
scroller
+
+ +
+
content
+
+
tap
+
press
+
pinch
+
+
sheet content
+
+
passive control
+
slider: a pan that turns selection off
+
checkbox row
+
switch: a drag that turns selection off
+
card (swipe up)
+
one-finger pan
+
kept press (a stepper's hold)
+
plain area
+
+
feed row (tap)
+
feed content
+
+
page
+
`; + +function element(id: string): HTMLElement { + const found = document.getElementById(id); + if (found == null) { + throw new Error(`missing #${id}`); + } + return found; +} + +const host = getGestureHost(document); + +function record(recognizer: GestureRecognizer, extra: () => Partial) { + recognizer.addTransitionListener((current, _from, to) => { + window.__gestures.push({ + name: current.name, + state: to, + time: internals(current).arena?.now() ?? performance.now(), + ...(to === 'possible' || to === 'failed' ? {} : extra()), + }); + }); +} + +const hpan = new PanGestureRecognizer({name: 'hpan', direction: 'horizontal'}); +record(hpan, () => ({x: hpan.translation().x, y: hpan.translation().y})); +host.attach(element('hpan'), hpan); + +const tap = new TapGestureRecognizer({name: 'tap'}); +record(tap, () => ({})); +host.attach(element('tapper'), tap); + +// A row in a scroller: a scroll the page makes under a still finger keeps +// its tap; a drag that scrolls the feed takes it. +const feedTap = new TapGestureRecognizer({name: 'feedTap'}); +record(feedTap, () => ({})); +host.attach(element('feedRow'), feedTap); + +const press = new LongPressGestureRecognizer({name: 'press'}); +record(press, () => ({ + beganAt: press.beganAt, + startedAt: press.trackedTouches[0]?.startTime, +})); +host.attach(element('presser'), press); + +const pinch = new PinchGestureRecognizer({name: 'pinch'}); +const pinchPan = new PanGestureRecognizer({ + name: 'pinchPan', + maximumNumberOfTouches: 2, +}); +pinch.recognizeSimultaneouslyWith(pinchPan); +record(pinch, () => ({scale: pinch.scale})); +record(pinchPan, () => ({x: pinchPan.translation().x})); +host.attach(element('pinchable'), pinch); +host.attach(element('pinchable'), pinchPan); + +const sheet = new PanGestureRecognizer({ + name: 'sheet', + direction: 'down', + scrollEdge: true, +}); +record(sheet, () => ({y: sheet.translation().y})); +host.attach(element('sheetBody'), sheet); + +// A one-way pan over the scrollable page, as a toast's upward swipe is: it +// claims its touch as it begins, at UIKit's 10 pt (spec:AST-094 FR19). +const card = new PanGestureRecognizer({name: 'card', direction: 'up'}); +record(card, () => ({x: card.translation().x, y: card.translation().y})); +host.attach(element('card'), card); + +// A control with no recognizer of its own (no blocking listener): only a +// press consumer, under pan-y, as a passive switch is. +const passiveEvent = (state: string) => + window.__gestures.push({name: 'passive', state, time: performance.now()}); +host.attach( + element('slider'), + new PanGestureRecognizer({ + name: 'slider', + direction: 'horizontal', + selectionLock: lockSelection, + }), +); + +host.attach( + element('switchRow'), + new PanGestureRecognizer({ + name: 'switchRow', + direction: 'horizontal', + selectionLock: lockSelection, + }), +); + +host.attachConsumer(element('checkRow'), { + touchBegan: () => {}, + touchMoved: () => {}, + touchEnded: () => {}, + touchCancelled: () => {}, +}); + +host.attachConsumer(element('passive'), { + touchBegan: () => passiveEvent('began'), + touchMoved: () => passiveEvent('moved'), + touchEnded: () => passiveEvent('ended'), + touchCancelled: () => passiveEvent('cancelled'), +}); + +// A one-finger pan: a second finger before any claim fails it. +const onePan = new PanGestureRecognizer({ + name: 'onePan', + maximumNumberOfTouches: 1, +}); +record(onePan, () => ({ + x: onePan.translation().x, + y: onePan.translation().y, +})); +host.attach(element('onePan'), onePan); + +// A control that keeps its press through a drag, as a stepper's hold does. +const keptEvent = (state: string) => + window.__gestures.push({name: 'kept', state, time: performance.now()}); +host.attachConsumer( + element('kept'), + new PressTracker({ + getBounds: () => element('kept').getBoundingClientRect(), + keepsTouch: true, + onControlEvent: event => keptEvent(event), + onHighlightChange: isHighlighted => + keptEvent(isHighlighted ? 'highlight' : 'unhighlight'), + }), +); + +// A scroll lock's page hold, held on demand: every one-finger pan would +// scroll the page. +(window as unknown as {__holdPage: () => () => void}).__holdPage = () => + host.holdPageScroll(() => true); + +for (const type of ['click', 'mousedown']) { + document.addEventListener( + type, + event => { + const target = event.target as HTMLElement | null; + window.__gestures.push({ + name: `${type}:${target?.id ?? ''}`, + state: 'dom', + time: event.timeStamp, + }); + }, + true, + ); +} + +window.__pageTouchListenersAtLoad = [...window.__pageTouchListeners]; diff --git a/packages/core/src/gestures/__tests__/browser/gestures.browser.spec.ts b/packages/core/src/gestures/__tests__/browser/gestures.browser.spec.ts new file mode 100644 index 0000000000000..866f934dbd595 --- /dev/null +++ b/packages/core/src/gestures/__tests__/browser/gestures.browser.spec.ts @@ -0,0 +1,1388 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file gestures.browser.spec.ts + * @input The gesture fixture bundled with esbuild, a phone-sized touch + * screen, and real engine touch input: Chromium through CDP with explicit + * timestamps, WebKit through its own input command (real-time pacing) + * @output Browser proof of the negotiation the unit tests cannot see: + * touch-action per element, touchmove cancelled exactly while a recognizer + * owns the finger, native scrolling untouched otherwise, click suppression + * with no ghost click, a drag that began as a scroll staying the browser's, pinch-zoom + * left to the browser, a second finger that never breaks the first, + * timing taken from event timestamps under a blocked main thread, nothing + * attached where the primary pointer is fine or a recognizer is disabled, + * and a held finger's selection lock only where the hold takes the touch + * @position Browser tests of packages/core/src/gestures (see + * playwright.config.ts beside this file). WebKit's Linux build cannot + * scroll, fling or pinch-zoom from injected touches, and synthesises a + * click only from its tap gesture; the checks that need those run in + * Chromium here and in Mobile Safari on the iOS simulator. + */ + +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {build} from 'esbuild'; +import {expect, test, type Page} from '@playwright/test'; + +interface Entry { + name: string; + state: string; + time: number; + x?: number; + y?: number; + scale?: number; + beganAt?: number | null; + startedAt?: number; +} + +let bundle = ''; +let bareBundle = ''; + +test.beforeAll(async () => { + const bundleOf = async (file: string) => { + const result = await build({ + entryPoints: [ + path.join(path.dirname(fileURLToPath(import.meta.url)), file), + ], + bundle: true, + write: false, + format: 'iife', + platform: 'browser', + target: 'es2022', + // A development build, as an app's bundler would make one. + define: {'process.env.NODE_ENV': '"development"'}, + }); + return result.outputFiles[0].text; + }; + bundle = await bundleOf('fixture.ts'); + bareBundle = await bundleOf('bareFixture.ts'); +}); + +type Point = {x: number; y: number}; +type Finger = Point & {id: number}; + +/** The fingers down after a step, and when (ms after the gesture's start). */ +interface Step { + t: number; + down: Finger[]; +} + +interface Driver { + readonly engine: 'chromium' | 'webkit'; + /** Plays the steps as real touch input; returns the page time of t = 0. */ + play(steps: Step[]): Promise; + /** A tap through the engine's tap path (WebKit synthesises click only from it). */ + tap(at: Point): Promise; +} + +const sleep = async (ms: number) => + new Promise(resolve => setTimeout(resolve, ms)); + +/** Each step's change against the previous one: who landed, lifted, moved. */ +function diff(previous: Finger[], next: Finger[]) { + const before = new Set(previous.map(finger => finger.id)); + const after = new Set(next.map(finger => finger.id)); + return { + landed: next.filter(finger => !before.has(finger.id)), + lifted: previous.filter(finger => !after.has(finger.id)), + }; +} + +async function chromiumDriver(page: Page): Promise { + const cdp = await page.context().newCDPSession(page); + const origin = await page.evaluate(() => performance.timeOrigin); + // The fingers down, across calls: a gesture may span several. + let down: Finger[] = []; + const send = async (type: string, points: Finger[], at: number) => + cdp.send('Input.dispatchTouchEvent', { + type: type as 'touchStart', + touchPoints: points.map(point => ({ + x: point.x, + y: point.y, + id: point.id, + })), + timestamp: (origin + at) / 1000, + }); + return { + engine: 'chromium', + async play(steps) { + // CDP stamps every event with the scripted time, so the page sees the + // script's timing exactly, whatever the dispatch latency. + const base = Math.ceil(await page.evaluate(() => performance.now())) + 50; + for (const step of steps) { + const {landed} = diff(down, step.down); + // CDP diffs active points against the previous event: a new point + // is pressed, a missing one released; touchEnd carries none. + const type = + landed.length > 0 + ? 'touchStart' + : step.down.length === 0 + ? 'touchEnd' + : 'touchMove'; + await send(type, step.down, base + step.t); + down = step.down; + } + return base; + }, + async tap(at) { + const base = Math.ceil(await page.evaluate(() => performance.now())) + 50; + await send('touchStart', [{...at, id: 1}], base); + await send('touchEnd', [], base + 60); + }, + }; +} + +interface WebKitSession { + send(method: string, params?: Record): Promise; +} + +/** Playwright's server-side WebKit page session (a locally launched browser). */ +function webkitSession(page: Page): WebKitSession { + const connection = ( + page as unknown as {_connection?: {toImpl?: (o: unknown) => unknown}} + )._connection; + const impl = connection?.toImpl?.(page) as + {delegate?: {_pageProxySession?: WebKitSession}} | undefined; + const session = impl?.delegate?._pageProxySession; + if (session == null) { + throw new Error('WebKit touch input needs a locally launched browser'); + } + return session; +} + +function webkitDriver(page: Page): Driver { + const session = webkitSession(page); + // WebKit's command applies the event's phase to every point it lists: a + // start lists only the fingers that land, an end only those that lift, a + // move every finger down. Integer CSS px; the page sees real time. + const points = (fingers: Finger[]) => + fingers.map(f => ({x: Math.round(f.x), y: Math.round(f.y), id: f.id})); + // The fingers down, across calls: a gesture may span several. + let down: Finger[] = []; + return { + engine: 'webkit', + async play(steps) { + const start = Date.now(); + const base = await page.evaluate(() => performance.now()); + for (const step of steps) { + const wait = start + step.t - Date.now(); + if (wait > 0) { + await sleep(wait); + } + const {landed, lifted} = diff(down, step.down); + if (landed.length > 0) { + await session.send('Input.dispatchTouchEvent', { + type: 'touchStart', + touchPoints: points(landed), + }); + } else if (lifted.length > 0) { + await session.send('Input.dispatchTouchEvent', { + type: 'touchEnd', + touchPoints: points(lifted), + }); + } else { + await session.send('Input.dispatchTouchEvent', { + type: 'touchMove', + touchPoints: points(step.down), + }); + } + down = step.down; + } + return base; + }, + async tap(at) { + await page.touchscreen.tap(Math.round(at.x), Math.round(at.y)); + }, + }; +} + +/** One finger from `from` to `to` in `count` frames of 16 ms, then a lift. */ +function drag(from: Point, to: Point, count = 12, lift = true): Step[] { + const steps: Step[] = [{t: 0, down: [{...from, id: 1}]}]; + for (let i = 1; i <= count; i++) { + const k = i / count; + steps.push({ + t: i * 16, + down: [ + { + x: from.x + (to.x - from.x) * k, + y: from.y + (to.y - from.y) * k, + id: 1, + }, + ], + }); + } + if (lift) { + steps.push({t: (count + 1) * 16, down: []}); + } + return steps; +} + +async function open(page: Page, engine: string): Promise { + // A real navigation first: touch emulation exposes the Touch Events API + // to documents it creates. + await page.goto('about:blank'); + await page.setContent( + '', + ); + await page.addScriptTag({content: bundle}); + return engine === 'webkit' ? webkitDriver(page) : chromiumDriver(page); +} + +/** A page with nothing attached: the test attaches through `window.__api`. */ +async function openBare(page: Page, engine: string): Promise { + await page.goto('about:blank'); + await page.setContent( + '', + ); + await page.addScriptTag({content: bareBundle}); + return engine === 'webkit' ? webkitDriver(page) : chromiumDriver(page); +} + +/** + * The touch-action the host gives an element (its mark, or an inline value + * it wrote) and the page's touch listeners, read after the next frame: the + * host reads an author's value there (FR18), so an absent value is absent + * for good, not merely not yet written. + */ +async function footprint(page: Page, id: string) { + return page.evaluate(async target => { + await new Promise(resolve => { + requestAnimationFrame(() => resolve()); + }); + const element = document.getElementById(target); + return { + touchAction: + element?.getAttribute('data-astryx-touch-action') ?? + element?.style.getPropertyValue('touch-action') ?? + null, + pageTouchListeners: window.__pageTouch, + }; + }, id); +} + +/** + * Whether a selection starting on an element now is refused: where iOS + * WebKit does not run, the selection lock cancels `selectstart` and writes + * no style (`spec:AST-094` FR41). + */ +async function refusesSelection(page: Page, id: string): Promise { + return page.evaluate(target => { + const event = new Event('selectstart', {bubbles: true, cancelable: true}); + (document.getElementById(target) ?? document.body).dispatchEvent(event); + return event.defaultPrevented; + }, id); +} + +async function center(page: Page, id: string): Promise { + const box = await page.locator(`#${id}`).boundingBox(); + if (box == null) { + throw new Error(`#${id} has no box`); + } + return { + x: Math.round(box.x + box.width / 2), + y: Math.round(box.y + box.height / 2), + }; +} + +async function log(page: Page): Promise { + await page.waitForTimeout(50); + return page.evaluate(() => window.__gestures); +} + +async function clearLog(page: Page): Promise { + await page.evaluate(() => { + window.__gestures.length = 0; + }); +} + +async function scrollTop(page: Page, id: string): Promise { + return page.evaluate( + target => document.getElementById(target)?.scrollTop ?? -1, + id, + ); +} + +/** Record whether each touchmove reached the page cancelled. */ +async function logTouchMoves(page: Page): Promise { + await page.evaluate(() => { + const moves: boolean[] = []; + (window as unknown as {__moves: boolean[]}).__moves = moves; + window.addEventListener( + 'touchmove', + event => moves.push(event.defaultPrevented), + {passive: true}, + ); + }); +} + +async function readTouchMoves(page: Page): Promise { + return page.evaluate( + () => (window as unknown as {__moves: boolean[]}).__moves, + ); +} + +const states = (entries: Entry[], name: string) => + entries.filter(entry => entry.name === name).map(entry => entry.state); + +test('derives touch-action per element from its recognizers', async ({ + page, + browserName, +}) => { + await open(page, browserName); + // The host reads each author's value and writes its own in the frame + // after the attach (FR18). + await page.evaluate( + async () => + new Promise(resolve => { + requestAnimationFrame(() => resolve()); + }), + ); + const actions = await page.evaluate(() => + Object.fromEntries( + ['hpan', 'tapper', 'presser', 'pinchable', 'sheetBody'].map(id => [ + id, + getComputedStyle(document.getElementById(id) as HTMLElement) + .touchAction, + ]), + ), + ); + expect(actions).toEqual({ + hpan: 'pan-y pinch-zoom', + tapper: 'manipulation', + presser: 'manipulation', + pinchable: 'none', + sheetBody: 'auto', + }); +}); + +test('only elements whose recognizers claim a moving touch carry a blocking touchmove listener', async ({ + page, + browserName, +}) => { + await open(page, browserName); + const blocking = await page.evaluate(() => + window.__listeners + .filter(entry => entry.type === 'touchmove' && entry.passive === false) + .map(entry => entry.id), + ); + // Pans (the cards', the switch's drag and a one-finger pan too), a long + // press, a pinch, the sheet's pan and a press that keeps its touch can take + // a moving touch; a tap-only element and a plain press consumer never wait. + expect(new Set(blocking)).toEqual( + new Set([ + 'hpan', + 'presser', + 'pinchable', + 'sheetBody', + 'slider', + 'switchRow', + 'card', + 'onePan', + 'kept', + ]), + ); +}); + +test('a horizontal drag on a pan-y element is the recognizer’s: cancelled moves, no scroll, no click', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await logTouchMoves(page); + const start = await center(page, 'hbutton'); + // A 4:1 drag, the shape WebKit would otherwise pan natively. + await driver.play(drag(start, {x: start.x + 120, y: start.y + 30})); + const entries = await log(page); + const hpan = states(entries, 'hpan'); + expect(hpan[0]).toBe('began'); + expect(hpan.slice(-2)).toEqual(['ended', 'possible']); + const last = entries + .filter(entry => entry.name === 'hpan' && entry.state === 'changed') + .at(-1); + // As in UIKit, the 10 px hysteresis is not part of the translation. + expect(last?.x).toBeCloseTo(120 - 10, 0); + // Every touchmove after the claim was cancelled. + const moves = await readTouchMoves(page); + expect(moves.length).toBeGreaterThan(0); + expect(moves.slice(-3).every(Boolean)).toBe(true); + expect(await scrollTop(page, 'scroller')).toBe(0); + expect(entries.some(entry => entry.name.startsWith('click:'))).toBe(false); + expect(entries.some(entry => entry.name.startsWith('mousedown:'))).toBe( + false, + ); +}); + +test('a vertical drag on the same element fails the pan and is never cancelled', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await logTouchMoves(page); + const start = await center(page, 'hpan'); + await driver.play(drag(start, {x: start.x + 4, y: start.y - 120})); + const entries = await log(page); + expect(states(entries, 'hpan')).toEqual(['failed', 'possible']); + expect((await readTouchMoves(page)).some(Boolean)).toBe(false); + if (driver.engine === 'chromium') { + // The browser scrolled it natively. + expect(await scrollTop(page, 'scroller')).toBeGreaterThan(60); + } + expect(entries.some(entry => entry.name.startsWith('click:'))).toBe(false); +}); + +test('a horizontal drag on a passive pan-y control keeps its press: uncancelable moves the browser cannot pan are not a claim', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await page.evaluate(() => { + document.getElementById('passive')?.scrollIntoView({block: 'center'}); + }); + const start = await center(page, 'passive'); + await driver.play( + drag({x: start.x - 120, y: start.y}, {x: start.x + 120, y: start.y + 6}), + ); + const consumed = (await log(page)) + .filter(entry => entry.name === 'passive') + .map(entry => entry.state); + expect(consumed[0]).toBe('began'); + expect(consumed).not.toContain('cancelled'); + expect(consumed.at(-1)).toBe('ended'); +}); + +test('a one-way overlay pan keeps its direction from the page, and leaves the other way to it', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling from injected touches (Mobile Safari on the simulator)', + ); + const driver = await open(page, browserName); + await page.evaluate(() => { + document.getElementById('card')?.scrollIntoView({block: 'center'}); + }); + const before = await page.evaluate(() => window.scrollY); + const start = await center(page, 'card'); + // Up, slowly: the card's pan begins at 10 px and the page never moves. + await driver.play(drag(start, {x: start.x, y: start.y - 40}, 40)); + expect(await page.evaluate(() => window.scrollY)).toBe(before); + const up = (await log(page)).filter(entry => entry.name === 'card'); + expect(up[0]?.state).toBe('began'); + expect(up.filter(entry => entry.state === 'changed').at(-1)?.y).toBe(-30); + // Down: the page's. + await clearLog(page); + await driver.play(drag(start, {x: start.x, y: start.y + 60}, 30)); + expect(await page.evaluate(() => window.scrollY)).toBeLessThan(before - 20); + expect(states(await log(page), 'card')).toEqual(['failed', 'possible']); +}); + +test('a held finger turns text selection off only where the hold does something (a pan that asks), until it lifts; a press-tracked control, a mouse and other pans keep native selection (spec:AST-094 FR41)', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const held: Record = {}; + for (const id of ['passive', 'slider', 'hpan']) { + await page.locator(`#${id}`).scrollIntoViewIfNeeded(); + const at = await center(page, id); + const before = await refusesSelection(page, id); + await driver.play([{t: 0, down: [{id: 1, ...at}]}]); + const during = await refusesSelection(page, id); + await driver.play([{t: 0, down: []}]); + held[id] = [before, during, await refusesSelection(page, id)]; + } + // A control whose hold does nothing keeps the browser's long-press + // selection. + expect(held.passive).toEqual([false, false, false]); + expect(held.slider).toEqual([false, true, false]); + expect(held.hpan).toEqual([false, false, false]); + + const at = await center(page, 'passive'); + await page.mouse.move(at.x, at.y); + await page.mouse.down(); + const withMouse = await refusesSelection(page, 'passive'); + await page.mouse.up(); + expect(withMouse).toBe(false); +}); + +test('a held finger on a checkbox inside an element whose hold does something turns its selection off too, as WebKit makes every input selectable; inside a press-tracked row it stays native (spec:AST-094 FR41)', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const heldSelection = async (id: string) => { + await page.locator(`#${id}`).scrollIntoViewIfNeeded(); + const at = await center(page, id); + const before = await refusesSelection(page, id); + await driver.play([{t: 0, down: [{id: 1, ...at}]}]); + const during = await refusesSelection(page, id); + await driver.play([{t: 0, down: []}]); + return [before, during, await refusesSelection(page, id)]; + }; + expect(await heldSelection('switchInput')).toEqual([false, true, false]); + expect(await heldSelection('check')).toEqual([false, false, false]); +}); + +test('a scroll the page makes under a still finger keeps the touch and its tap', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await page.locator('#feed').scrollIntoViewIfNeeded(); + const at = await center(page, 'feedRow'); + await clearLog(page); + await driver.play([{t: 0, down: [{id: 1, ...at}]}]); + // The page scrolls the feed under the resting finger (scrollIntoView, a + // pinned transcript): the finger never travelled, so the touch stays. + await page.evaluate(() => { + const feed = document.getElementById('feed'); + if (feed != null) { + feed.scrollTop += 6; + } + }); + await page.evaluate(async () => { + await new Promise(resolve => { + requestAnimationFrame(() => resolve()); + }); + }); + await driver.play([{t: 0, down: []}]); + expect(states(await log(page), 'feedTap')).toEqual(['ended', 'possible']); +}); + +test('a drag that scrolls the feed natively is the browser’s: no tap', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling from injected touches (Mobile Safari on the simulator)', + ); + const driver = await open(page, browserName); + await page.locator('#feed').scrollIntoViewIfNeeded(); + await page.evaluate(() => { + const feed = document.getElementById('feed'); + if (feed != null) { + feed.scrollTop = 0; + } + }); + const start = await center(page, 'feedRow'); + await clearLog(page); + await driver.play(drag(start, {x: start.x, y: start.y - 60}, 30)); + const scrolled = await page.evaluate( + () => document.getElementById('feed')?.scrollTop ?? 0, + ); + expect(scrolled).toBeGreaterThan(20); + expect(states(await log(page), 'feedTap')).not.toContain('ended'); +}); + +test('a second finger before any claim fails a one-finger pan and leaves the pinch to the page', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await page.locator('#onePan').scrollIntoViewIfNeeded(); + const at = await center(page, 'onePan'); + await clearLog(page); + await logTouchMoves(page); + const pair = (spread: number) => [ + {id: 1, x: at.x - spread, y: at.y}, + {id: 2, x: at.x + spread, y: at.y}, + ]; + await driver.play([ + {t: 0, down: [{id: 1, x: at.x - 10, y: at.y}]}, + {t: 30, down: pair(10)}, + {t: 60, down: pair(30)}, + {t: 90, down: pair(50)}, + {t: 120, down: pair(70)}, + {t: 150, down: []}, + ]); + const pan = states(await log(page), 'onePan'); + expect(pan).toContain('failed'); + expect(pan).not.toContain('began'); + // No move of the pinch is cancelled: the page may zoom. + const moves = await readTouchMoves(page); + expect(moves.length).toBeGreaterThan(0); + expect(moves.some(Boolean)).toBe(false); +}); + +test('a page hold keeps the page still under a one-finger pan, and leaves a pinch alone', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await page.locator('#plainArea').scrollIntoViewIfNeeded(); + await page.evaluate(() => { + const own = window as unknown as { + __holdPage: () => () => void; + __releaseHold: () => void; + }; + own.__releaseHold = own.__holdPage(); + }); + await logTouchMoves(page); + const start = await center(page, 'plainArea'); + const before = await page.evaluate(() => window.scrollY); + await driver.play(drag(start, {x: start.x, y: start.y - 80}, 30)); + expect((await readTouchMoves(page)).some(Boolean)).toBe(true); + expect(await page.evaluate(() => window.scrollY)).toBe(before); + // A second finger is a pinch: never cancelled. + await logTouchMoves(page); + await driver.play([ + { + t: 0, + down: [ + {id: 1, x: start.x - 10, y: start.y}, + {id: 2, x: start.x + 10, y: start.y}, + ], + }, + { + t: 30, + down: [ + {id: 1, x: start.x - 40, y: start.y}, + {id: 2, x: start.x + 40, y: start.y}, + ], + }, + {t: 60, down: []}, + ]); + expect((await readTouchMoves(page)).some(Boolean)).toBe(false); + await page.evaluate(() => + (window as unknown as {__releaseHold: () => void}).__releaseHold(), + ); +}); + +test('a control that keeps its press through a drag holds the page still and stays pressed', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await page.locator('#kept').scrollIntoViewIfNeeded(); + await clearLog(page); + const start = await center(page, 'kept'); + const before = await page.evaluate(() => window.scrollY); + // The finger rests until the press is delivered: a control in a scroller + // waits before it highlights, and a drag that starts sooner is a scroll. + await driver.play([{t: 0, down: [{id: 1, ...start}]}]); + await page.waitForFunction(() => + window.__gestures.some( + entry => entry.name === 'kept' && entry.state === 'highlight', + ), + ); + await logTouchMoves(page); + await driver.play(drag(start, {x: start.x, y: start.y - 40}, 20, false)); + let kept = states(await log(page), 'kept'); + expect(kept).toContain('touchDown'); + expect(kept).not.toContain('touchCancel'); + expect(kept).not.toContain('unhighlight'); + // Its press was delivered, so every move is cancelled: the page stays. + const moves = await readTouchMoves(page); + expect(moves.length).toBeGreaterThan(0); + expect(moves.every(Boolean)).toBe(true); + expect(await page.evaluate(() => window.scrollY)).toBe(before); + await driver.play([{t: 0, down: []}]); + kept = states(await log(page), 'kept'); + expect(kept).toContain('touchUpInside'); + expect(kept).not.toContain('touchCancel'); +}); + +test('where the primary pointer is fine, the host writes no touch-action and registers no touch listener (spec:AST-094 FR19, FR24)', async ({ + browser, +}) => { + // A desktop: a mouse is the primary pointer. + const context = await browser.newContext({ + viewport: {width: 1280, height: 800}, + hasTouch: false, + isMobile: false, + }); + try { + const page = await context.newPage(); + await page.goto('about:blank'); + await page.setContent(''); + await page.addScriptTag({content: bareBundle}); + await page.evaluate(() => { + const {host, LongPressGestureRecognizer, PanGestureRecognizer} = + window.__api; + document.body.innerHTML = + '
drag
hold
'; + host.attach( + document.getElementById('drag') as HTMLElement, + new PanGestureRecognizer({direction: 'horizontal'}), + ); + host.attach( + document.getElementById('hold') as HTMLElement, + new LongPressGestureRecognizer(), + ); + }); + expect(await footprint(page, 'drag')).toEqual({ + touchAction: '', + pageTouchListeners: 0, + }); + expect((await footprint(page, 'hold')).touchAction).toBe(''); + } finally { + await context.close(); + } +}); + +test("the host's touch-action applies from the attach itself, an author rule outranks it from the first frame, and a shadow root gets it too (spec:AST-094 FR18, DEC-8)", async ({ + page, + browserName, +}) => { + await openBare(page, browserName); + const actions = await page.evaluate(() => { + const {host, PanGestureRecognizer, LongPressGestureRecognizer} = + window.__api; + const author = document.createElement('style'); + author.textContent = '#authored { touch-action: none; }'; + document.head.append(author); + document.body.innerHTML = + '
'; + const plain = document.getElementById('plain') as HTMLElement; + const authored = document.getElementById('authored') as HTMLElement; + const inner = document.createElement('div'); + inner.style.height = '60px'; + (document.getElementById('shadowed') as HTMLElement) + .attachShadow({mode: 'open'}) + .append(inner); + host.attach(plain, new PanGestureRecognizer({direction: 'horizontal'})); + host.attach(authored, new LongPressGestureRecognizer()); + host.attach(inner, new LongPressGestureRecognizer()); + // Read in the attach's own task, before any frame. + return { + plain: getComputedStyle(plain).touchAction, + authored: getComputedStyle(authored).touchAction, + inner: getComputedStyle(inner).touchAction, + }; + }); + expect(actions).toEqual({ + plain: 'pan-y pinch-zoom', + authored: 'none', + inner: 'manipulation', + }); +}); + +test("an author rule in any cascade layer outranks the host's touch-action (spec:AST-094 FR18, DEC-8)", async ({ + page, + browserName, +}) => { + await openBare(page, browserName); + const actions = await page.evaluate(() => { + const {host, PanGestureRecognizer} = window.__api; + // The layer order a package stylesheet declares (spec:AST-080 FR3), and + // author rules in Astryx's base layer and in the product's layer. + const author = document.createElement('style'); + author.textContent = `@layer reset, astryx-base, astryx-theme, product; + @layer astryx-base { #base { touch-action: none; } } + @layer product { #product { touch-action: pan-x; } }`; + document.head.append(author); + document.body.innerHTML = + '
'; + const base = document.getElementById('base') as HTMLElement; + const product = document.getElementById('product') as HTMLElement; + host.attach(base, new PanGestureRecognizer({direction: 'horizontal'})); + host.attach(product, new PanGestureRecognizer({direction: 'horizontal'})); + return { + base: getComputedStyle(base).touchAction, + product: getComputedStyle(product).touchAction, + }; + }); + expect(actions).toEqual({base: 'none', product: 'pan-x'}); +}); + +test('a subtree that mounts with a recognizer is styled once: the host adds no second recalc of it (spec:AST-094 FR18)', async ({ + page, + browserName, +}) => { + test.skip( + browserName !== 'chromium', + 'counts style recalcs in a Chromium trace', + ); + await openBare(page, browserName); + const cdp = await page.context().newCDPSession(page); + const events: {name: string; args?: {elementCount?: number}}[] = []; + cdp.on('Tracing.dataCollected', ({value}) => { + events.push(...(value as unknown as typeof events)); + }); + const complete = new Promise(resolve => { + cdp.once('Tracing.tracingComplete', resolve); + }); + await cdp.send('Tracing.start', { + transferMode: 'ReportEvents', + traceConfig: { + includedCategories: [ + 'devtools.timeline', + 'disabled-by-default-devtools.timeline', + ], + excludedCategories: ['*'], + }, + }); + await page.evaluate(async () => { + const {host, PanGestureRecognizer} = window.__api; + const wheel = document.createElement('div'); + for (let i = 0; i < 300; i += 1) { + const row = document.createElement('div'); + row.textContent = `Row ${i}`; + wheel.append(row); + } + document.body.append(wheel); + host.attach(wheel, new PanGestureRecognizer({direction: 'vertical'})); + for (let i = 0; i < 4; i += 1) { + await new Promise(resolve => { + requestAnimationFrame(resolve); + }); + } + }); + await cdp.send('Tracing.end'); + await complete; + // The subtree's first styling, and no recalc of it after: a touch-action + // written later would restyle every row (each one's effective value). + const subtreeRecalcs = events.filter( + event => + event.name === 'UpdateLayoutTree' && + (event.args?.elementCount ?? 0) >= 300, + ); + expect(subtreeRecalcs).toHaveLength(1); +}); + +test('a recognizer registers no touch listener and writes no touch-action while it is disabled, and does once it is enabled (spec:AST-094 FR24)', async ({ + page, + browserName, +}) => { + await openBare(page, browserName); + await page.evaluate(() => { + const {host, PanGestureRecognizer} = window.__api; + document.body.innerHTML = '
solo
'; + const pan = new PanGestureRecognizer({ + direction: 'horizontal', + isEnabled: false, + }); + (window as unknown as {__solo: typeof pan}).__solo = pan; + host.attach(document.getElementById('solo') as HTMLElement, pan); + }); + expect(await footprint(page, 'solo')).toEqual({ + touchAction: '', + pageTouchListeners: 0, + }); + const setEnabled = async (isEnabled: boolean) => + page.evaluate(next => { + const solo = ( + window as unknown as { + __solo: {setEnabled(value: boolean): void}; + } + ).__solo; + solo.setEnabled(next); + window.__api.host.update(document.getElementById('solo') as HTMLElement); + }, isEnabled); + await setEnabled(true); + const enabled = await footprint(page, 'solo'); + expect(enabled.touchAction).toBe('pan-y pinch-zoom'); + expect(enabled.pageTouchListeners).toBeGreaterThan(0); + await setEnabled(false); + expect(await footprint(page, 'solo')).toEqual({ + touchAction: '', + pageTouchListeners: 0, + }); +}); + +test('a zero-delay hold that brings its own lock turns selection off only for a touch it takes: a touch it declines keeps native selection (spec:AST-094 FR41)', async ({ + page, + browserName, +}) => { + const driver = await openBare(page, browserName); + await page.evaluate(() => { + const {host, LongPressGestureRecognizer, lockSelection} = window.__api; + // A tab bar: its lens is a hold at 0 ms, and the search tab beside it + // is a button the lens's recognizer declines. + document.body.innerHTML = + '
lens
'; + const lens = document.getElementById('lens') as HTMLElement; + host.attach( + document.getElementById('bar') as HTMLElement, + new LongPressGestureRecognizer({ + minimumPressDuration: 0, + selectionLock: lockSelection, + shouldReceiveTouch: (_recognizer, touch) => + touch.target instanceof Node && lens.contains(touch.target), + }), + ); + }); + expect(await refusesSelection(page, 'lens')).toBe(false); + const held: Record = {}; + for (const id of ['search', 'lens']) { + const at = await center(page, id); + await driver.play([{t: 0, down: [{id: 1, ...at}]}]); + const during = await refusesSelection(page, 'lens'); + await driver.play([{t: 0, down: []}]); + held[id] = [during, await refusesSelection(page, 'lens')]; + } + expect(held.search).toEqual([false, false]); + expect(held.lens).toEqual([true, false]); +}); + +test('a tap recognizer replaces the click: no ghost click, no compatibility mouse events', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await driver.tap(await center(page, 'tapper')); + let entries = await log(page); + expect(states(entries, 'tap')).toEqual(['ended', 'possible']); + expect(entries.filter(entry => entry.state === 'dom')).toEqual([]); + + // A tap the pan does not take keeps its click (exactly one). + await clearLog(page); + await driver.tap(await center(page, 'hbutton')); + entries = await log(page); + expect(entries.filter(entry => entry.name === 'click:hbutton')).toHaveLength( + 1, + ); +}); + +test('a finger the window blur cancelled lifts on nothing: its lift gets no click, and a later tap there keeps its own (spec:AST-094 FR28)', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const at = await center(page, 'hbutton'); + await clearLog(page); + await driver.play([{t: 0, down: [{id: 1, ...at}]}]); + // A headless page keeps its focus, so the blur is dispatched to the window. + await page.evaluate(() => window.dispatchEvent(new FocusEvent('blur'))); + await driver.play([{t: 0, down: []}]); + await page.waitForTimeout(100); + const clicks = async () => + (await log(page)).filter(entry => entry.name === 'click:hbutton'); + expect(await clicks()).toEqual([]); + await driver.tap(at); + expect(await clicks()).toHaveLength(1); +}); + +test('scroll edge: at the top the pull is the sheet’s at once', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const start = await center(page, 'sheetBody'); + await driver.play(drag(start, {x: start.x, y: start.y + 96})); + const entries = await log(page); + const sheet = entries.filter(entry => entry.name === 'sheet'); + expect(sheet[0]?.state).toBe('began'); + expect( + sheet.filter(entry => entry.state === 'changed').at(-1)?.y, + ).toBeCloseTo(96 - 10, 0); + expect(await scrollTop(page, 'sheetBody')).toBe(0); +}); + +test("scroll edge: at the top the sheet cancels the pull's moves from the first one, before it begins (spec:AST-094 FR19, DEC-18)", async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await logTouchMoves(page); + const start = await center(page, 'sheetBody'); + await driver.play(drag(start, {x: start.x, y: start.y + 24})); + const moves = await readTouchMoves(page); + // WebKit sends the first move at 2 px, well before the sheet's 10 px begin; + // Chromium sends none inside its own 15 px slop. Every move the page sees + // is cancelled, the first one included. + expect(moves.length).toBeGreaterThan(0); + expect(moves).toEqual(moves.map(() => true)); + const sheet = (await log(page)).filter(entry => entry.name === 'sheet'); + expect(sheet[0]?.state).toBe('began'); + expect(await scrollTop(page, 'sheetBody')).toBe(0); +}); + +test('a drag that began as a scroll stays the browser’s (AST-094 FR14): the content scrolls to its top and the sheet never begins', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling from injected touches (Mobile Safari on the simulator)', + ); + const driver = await open(page, browserName); + await page.evaluate(() => { + (document.getElementById('sheetBody') as HTMLElement).scrollTop = 60; + }); + const start = await center(page, 'sheetBody'); + start.y -= 60; + await driver.play(drag(start, {x: start.x, y: start.y + 200}, 25)); + const entries = await log(page); + const sheet = entries.filter(entry => entry.name === 'sheet'); + // The content takes the whole drag: it scrolls to its top, and the travel + // past that point never becomes the sheet's. + expect(await scrollTop(page, 'sheetBody')).toBe(0); + expect(sheet.some(entry => entry.state === 'began')).toBe(false); +}); + +/** Two fingers that land one after the other and spread apart. */ +function spread(at: Point, from: number, by: number, drift = 0): Step[] { + const steps: Step[] = [ + {t: 0, down: [{x: at.x - from, y: at.y, id: 1}]}, + { + t: 30, + down: [ + {x: at.x - from, y: at.y, id: 1}, + {x: at.x + from, y: at.y, id: 2}, + ], + }, + ]; + for (let i = 1; i <= 12; i++) { + const half = from + (by * i) / 12; + steps.push({ + t: 30 + i * 16, + down: [ + {x: at.x - half + drift * i, y: at.y, id: 1}, + {x: at.x + half + drift * i, y: at.y, id: 2}, + ], + }); + } + steps.push({t: 30 + 13 * 16, down: []}); + return steps; +} + +test('pinch-zoom stays the browser’s on an element without a pinch recognizer', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native pinch-zoom from injected touches (Mobile Safari on the simulator)', + ); + const driver = await open(page, browserName); + await driver.play(spread(await center(page, 'tapper'), 20, 96)); + await page.waitForTimeout(400); + expect( + await page.evaluate(() => window.visualViewport?.scale ?? 1), + ).toBeGreaterThan(1.2); + expect(states(await log(page), 'tap')).not.toContain('ended'); +}); + +test('a pinch recognizer takes the two-finger gesture on its element only, alongside its pan', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + await driver.play(spread(await center(page, 'pinchable'), 20, 40, 1)); + const entries = await log(page); + expect(states(entries, 'pinch')[0]).toBe('began'); + const scale = entries + .filter(entry => entry.name === 'pinch' && entry.state === 'changed') + .at(-1)?.scale; + // The span grew from 40 to 120; as in UIKit, the scale is measured from + // the starting span moved by the 8 px hysteresis. + expect(scale).toBeCloseTo(120 / (40 + 8), 1); + expect(states(entries, 'pinchPan')).toContain('began'); + expect(await page.evaluate(() => window.visualViewport?.scale ?? 1)).toBe(1); +}); + +test('a second finger never breaks a claimed one-finger pan', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const start = await center(page, 'hbutton'); + const other = await center(page, 'tapper'); + const steps: Step[] = [{t: 0, down: [{...start, id: 1}]}]; + for (let i = 1; i <= 4; i++) { + steps.push({t: i * 16, down: [{x: start.x + i * 10, y: start.y, id: 1}]}); + } + const first = {x: start.x + 40, y: start.y, id: 1}; + steps.push({t: 80, down: [first, {...other, id: 2}]}); + for (let i = 1; i <= 6; i++) { + steps.push({ + t: 80 + i * 16, + down: [ + {x: first.x + i * 5, y: first.y, id: 1}, + {x: other.x + i * 15, y: other.y + i * 15, id: 2}, + ], + }); + } + steps.push({ + t: 200, + down: [{x: other.x + 90, y: other.y + 90, id: 2}], + }); + steps.push({t: 216, down: []}); + await driver.play(steps); + const entries = await log(page); + const hpan = entries.filter(entry => entry.name === 'hpan'); + expect(hpan.map(entry => entry.state)).not.toContain('cancelled'); + expect(hpan.map(entry => entry.state)).toContain('ended'); + expect(hpan.filter(entry => entry.state === 'changed').at(-1)?.x).toBeCloseTo( + 70 - 10, + 0, + ); + expect(await page.evaluate(() => window.visualViewport?.scale ?? 1)).toBe(1); +}); + +test('a long press begins at touch-down + 500 ms on the event clock, even when the main thread was blocked', async ({ + page, + browserName, +}) => { + const driver = await open(page, browserName); + const at = await center(page, 'presser'); + const base = await driver.play([{t: 0, down: [{...at, id: 1}]}]); + // Block the main thread well past the press duration. + await page.evaluate(() => { + const end = performance.now() + 800; + while (performance.now() < end) { + // busy + } + }); + await page.waitForTimeout(100); + await driver.play([{t: 0, down: []}]); + const entries = await log(page); + const began = entries.find( + entry => entry.name === 'press' && entry.state === 'began', + ); + if (driver.engine === 'chromium') { + // The touch carries the timestamp CDP gave it (to the browser's 0.1 ms + // timestamp precision). + expect(Math.abs((began?.startedAt ?? 0) - base)).toBeLessThan(0.5); + } + // The press began exactly 500 ms after the touch, though the timer that + // noticed it ran 300 ms late. + expect((began?.beganAt ?? 0) - (began?.startedAt ?? 0)).toBeCloseTo(500, 6); + expect(states(entries, 'press')).toEqual(['began', 'ended', 'possible']); + expect(entries.some(entry => entry.name.startsWith('click:'))).toBe(false); +}); + +test.describe('where the primary pointer is fine', () => { + test.use({hasTouch: false, isMobile: false}); + + test('the host registers no touch listener and writes no style, though the page attaches recognizers (spec:AST-094 FR19)', async ({ + page, + }) => { + const warnings: string[] = []; + page.on('console', message => { + if (message.type() === 'warning') { + warnings.push(message.text()); + } + }); + await page.goto('about:blank'); + await page.setContent( + '', + ); + await page.addScriptTag({content: bundle}); + const seen = await page.evaluate(() => ({ + isCoarse: matchMedia('(pointer: coarse)').matches, + page: window.__pageTouchListenersAtLoad, + elements: window.__listeners.map(entry => entry.id), + // What the host would write: none of it, in any engine's spelling. + styles: ['hpan', 'presser', 'slider', 'card'].map(id => { + const style = document.getElementById(id)?.style; + return ['touch-action', '-webkit-user-select', '-webkit-touch-callout'] + .map(property => style?.getPropertyValue(property) ?? 'missing') + .join('|'); + }), + })); + expect(seen.isCoarse).toBe(false); + expect(seen.page).toEqual([]); + expect(seen.elements).toEqual([]); + expect(seen.styles).toEqual(['||', '||', '||', '||']); + // The page's attaches are named once, in development. + expect( + warnings.filter(text => text.includes('primary pointer is fine')), + ).toHaveLength(1); + }); +}); + +/** + * A page with a scroller of rows and a tap recognizer on one row; every + * transition of the tap goes to `window.__taps`. + */ +async function openTapInScroller(page: Page, engine: string): Promise { + const driver = await openBare(page, engine); + await page.evaluate(() => { + const {host, TapGestureRecognizer} = window.__api; + document.body.innerHTML = ` +
+
row
+
more rows
+
`; + const taps: string[] = []; + (window as unknown as {__taps: string[]}).__taps = taps; + const tap = new TapGestureRecognizer({name: 'tap'}); + tap.addTransitionListener((_current, _from, to) => taps.push(to)); + host.attach(document.getElementById('row') as HTMLElement, tap); + }); + return driver; +} + +for (const travel of [ + {dx: 0, dy: 9, label: '9 px straight', keeps: true}, + {dx: 0, dy: 10, label: '10 px straight', keeps: false}, + {dx: 7, dy: 7, label: '(7, 7), 9.9 px diagonally', keeps: true}, + {dx: 8, dy: 8, label: '(8, 8), 11.3 px diagonally', keeps: false}, +]) { + test(`a finger moved ${travel.label} over a scroll view ${travel.keeps ? 'keeps its tap and scrolls nothing' : 'is a tap or a scroll, never both'} (spec:AST-094 FR18–FR22)`, async ({ + page, + browserName, + }) => { + const driver = await openTapInScroller(page, browserName); + await logTouchMoves(page); + const at = await center(page, 'row'); + await driver.play(drag(at, {x: at.x + travel.dx, y: at.y + travel.dy}, 6)); + await page.waitForTimeout(100); + const taps = await page.evaluate( + () => (window as unknown as {__taps: string[]}).__taps, + ); + const scrolled = (await scrollTop(page, 'list')) > 0; + if (travel.keeps) { + // Under 10 px of travel: a tap, and no scroll took the touch. + expect(taps).toContain('ended'); + expect(scrolled).toBe(false); + } else { + // From 10 px the browser may scroll (WebKit on a phone starts at 10; + // Chromium waits longer): if it does, the scroll takes the touch and + // there is no tap; if it does not, the tap stands. + expect(scrolled && taps.includes('ended')).toBe(false); + } + // A tap never claims a moving touch: no touchmove is cancelled, so the + // browser's scroll is never blocked. + expect((await readTouchMoves(page)).some(Boolean)).toBe(false); + }); +} + +/** + * A screen-edge pan on a strip along the left edge as wide as its band + * (40.2 px, which lays out at 40.203), over a horizontal scroller that + * fills the page; every + * transition of the pan goes to `window.__edge`. + */ +async function openEdgePan(page: Page, engine: string): Promise { + const driver = await openBare(page, engine); + await page.evaluate(() => { + const {host, ScreenEdgePanGestureRecognizer} = window.__api; + document.body.innerHTML = ` +
+
wide
+
+
`; + const edges: string[] = []; + (window as unknown as {__edge: string[]}).__edge = edges; + const pan = new ScreenEdgePanGestureRecognizer({ + name: 'edge', + edges: ['left'], + }); + pan.addTransitionListener((_current, _from, to) => edges.push(to)); + host.attach(document.getElementById('edge') as HTMLElement, pan); + }); + return driver; +} + +for (const run of [ + {travel: 14, begins: false}, + {travel: 15, begins: true}, +]) { + test(`a screen-edge pan landing 40 px from the edge ${run.begins ? 'claims at' : 'has not claimed at'} ${run.travel} px of travel (spec:AST-094 FR18–FR22)`, async ({ + page, + browserName, + }) => { + const driver = await openEdgePan(page, browserName); + await logTouchMoves(page); + await driver.play( + drag({x: 40, y: 150}, {x: 40 + run.travel, y: 150}, 7, false), + ); + await page.waitForTimeout(100); + const edges = await page.evaluate( + () => (window as unknown as {__edge: string[]}).__edge, + ); + expect(edges.includes('began')).toBe(run.begins); + await driver.play([{t: 0, down: []}]); + }); +} + +test('a touch landing 41 px from the edge is outside the band: the edge pan never begins, and the horizontal scroll it starts is not blocked (spec:AST-094 FR18–FR22)', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling from injected touches (Mobile Safari on the simulator)', + ); + const driver = await openEdgePan(page, browserName); + await page.evaluate(() => { + (document.getElementById('hscroll') as HTMLElement).scrollLeft = 600; + }); + await logTouchMoves(page); + await driver.play(drag({x: 41, y: 150}, {x: 201, y: 150}, 12)); + await page.waitForTimeout(300); + const edges = await page.evaluate( + () => (window as unknown as {__edge: string[]}).__edge, + ); + expect(edges).not.toContain('began'); + expect((await readTouchMoves(page)).some(Boolean)).toBe(false); + expect( + await page.evaluate( + () => (document.getElementById('hscroll') as HTMLElement).scrollLeft, + ), + ).toBeLessThan(600); +}); + +/** A watcher's transition, with how far the list had scrolled then. */ +interface WatchTransition { + to: string; + scrollTop: number; +} + +test('a watcher follows a touch the browser scrolls with to the lift, and cancels none of its moves (spec:AST-094 FR20)', async ({ + page, + browserName, +}) => { + test.skip( + browserName === 'webkit', + 'needs native scrolling from injected touches (Mobile Safari on the simulator)', + ); + const driver = await openBare(page, browserName); + await page.evaluate(() => { + const {host, PanGestureRecognizer} = window.__api; + document.body.innerHTML = ` +
+
row
+
more rows
+
`; + const list = document.getElementById('list') as HTMLElement; + const states: WatchTransition[] = []; + (window as unknown as {__watch: WatchTransition[]}).__watch = states; + const watcher = new PanGestureRecognizer({ + name: 'watch', + direction: 'vertical', + claimsTouches: false, + }); + // Each transition with how far the list had scrolled then. + watcher.addTransitionListener((_current, _from, to) => + states.push({to, scrollTop: list.scrollTop}), + ); + host.attach(document.getElementById('list') as HTMLElement, watcher, { + manageStyle: false, + }); + }); + await logTouchMoves(page); + const at = await center(page, 'row'); + await driver.play(drag(at, {x: at.x, y: at.y - 120}, 30)); + await page.waitForTimeout(100); + const states = await page.evaluate( + () => + ( + window as unknown as { + __watch: WatchTransition[]; + } + ).__watch, + ); + // The browser scrolled the list: the watcher took nothing from it. + expect(await scrollTop(page, 'list')).toBeGreaterThan(20); + expect((await readTouchMoves(page)).some(Boolean)).toBe(false); + // And it followed the scroll from its start to the lift: it still + // reported moves once the browser had taken the touch and was scrolling. + expect(states.map(entry => entry.to)).toContain('began'); + expect( + states.some(entry => entry.to === 'changed' && entry.scrollTop > 0), + ).toBe(true); + // It ended there, then the arena reset it to possible, as UIKit resets a + // recognizer after it ends. + expect(states.slice(-2).map(entry => entry.to)).toEqual([ + 'ended', + 'possible', + ]); +}); diff --git a/packages/core/src/gestures/__tests__/browser/playwright.config.ts b/packages/core/src/gestures/__tests__/browser/playwright.config.ts new file mode 100644 index 0000000000000..9664a60b03fb7 --- /dev/null +++ b/packages/core/src/gestures/__tests__/browser/playwright.config.ts @@ -0,0 +1,49 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file playwright.config.ts + * @input Uses @playwright/test + * @output The gesture system's browser proof on a phone-sized touch screen, + * in Chromium (touch input through CDP) and WebKit (its own input command) + * @position Both-engine config for packages/core/src/gestures (CI runs the + * same specs in Chromium through playwright.interaction.config.ts); run with + * `pnpm exec playwright test --config packages/core/src/gestures/__tests__/browser/playwright.config.ts`. + * Where WebKit cannot launch on the host, run its project inside the + * official Playwright image (the repository mounted at the same path): + * `podman run --rm --network=host --ipc=host -v "$PWD:$PWD" -w "$PWD" + * mcr.microsoft.com/playwright:v1.61.1-noble node + * node_modules/@playwright/test/cli.js test --config --project webkit` + */ + +import os from 'node:os'; +import path from 'node:path'; +import {defineConfig, devices} from '@playwright/test'; + +const phone = { + viewport: {width: 390, height: 844}, + deviceScaleFactor: 3, + hasTouch: true, + isMobile: true, +} as const; + +export default defineConfig({ + testDir: '.', + // Traces and screenshots of failures stay out of the source tree. + outputDir: path.join(os.tmpdir(), 'astryx-gesture-proof'), + testMatch: ['*.browser.spec.ts'], + fullyParallel: false, + workers: 1, + timeout: 60_000, + reporter: [['list']], + projects: [ + { + name: 'chromium', + use: { + ...devices['Pixel 7'], + browserName: 'chromium', + ...phone, + }, + }, + {name: 'webkit', use: {browserName: 'webkit', ...phone}}, + ], +}); diff --git a/packages/core/src/gestures/__tests__/browserNegotiation.test.ts b/packages/core/src/gestures/__tests__/browserNegotiation.test.ts new file mode 100644 index 0000000000000..381eb8ef37a3c --- /dev/null +++ b/packages/core/src/gestures/__tests__/browserNegotiation.test.ts @@ -0,0 +1,287 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file browserNegotiation.test.ts + * @input Recognizer sets, touch-action strings, and jsdom elements with + * scripted scroll metrics + * @output What the browser is told (touch-action and long-press styles) and + * what the native scroll probe answers (edges, touch-action, chaining, + * RTL) + * @position Unit tests of touchAction.ts and nativeScroll.ts + */ + +import {afterEach, beforeEach, describe, expect, it, vi} from 'vitest'; +import {LongPressGestureRecognizer} from '../LongPressGestureRecognizer'; +import {createNativeScrollProbe, parseTouchAction} from '../nativeScroll'; +import {PanGestureRecognizer} from '../PanGestureRecognizer'; +import {PinchGestureRecognizer} from '../PinchGestureRecognizer'; +import {RotationGestureRecognizer} from '../RotationGestureRecognizer'; +import {ScreenEdgePanGestureRecognizer} from '../ScreenEdgePanGestureRecognizer'; +import {SwipeGestureRecognizer} from '../SwipeGestureRecognizer'; +import {TapGestureRecognizer} from '../TapGestureRecognizer'; +import {deriveTouchAction, gestureHostStyle} from '../touchAction'; +import {HoverGestureRecognizer} from '../HoverGestureRecognizer'; + +describe('deriveTouchAction', () => { + it('needs nothing without touch recognizers', () => { + expect(deriveTouchAction([])).toBeNull(); + expect(deriveTouchAction([new HoverGestureRecognizer()])).toBeNull(); + expect( + deriveTouchAction([ + new PanGestureRecognizer({allowedPointerTypes: ['mouse']}), + ]), + ).toBeNull(); + }); + + it('removes only double-tap zoom for taps and long presses', () => { + expect(deriveTouchAction([new TapGestureRecognizer()])).toBe( + 'manipulation', + ); + expect(deriveTouchAction([new LongPressGestureRecognizer()])).toBe( + 'manipulation', + ); + }); + + it('gives the browser the other axis of a one-axis pan or swipe, and keeps pinch-zoom', () => { + expect( + deriveTouchAction([new PanGestureRecognizer({direction: 'horizontal'})]), + ).toBe('pan-y pinch-zoom'); + expect( + deriveTouchAction([new PanGestureRecognizer({direction: 'vertical'})]), + ).toBe('pan-x pinch-zoom'); + expect(deriveTouchAction([new PanGestureRecognizer()])).toBe('pinch-zoom'); + expect( + deriveTouchAction([new SwipeGestureRecognizer({direction: 'left'})]), + ).toBe('pan-y pinch-zoom'); + }); + + it('leaves the browser everything for a one-way pan, or one that shares native scrolling', () => { + // The other direction keeps scrolling natively; the pan claims its own + // by cancelling touchmove when it begins. + expect( + deriveTouchAction([new PanGestureRecognizer({direction: 'down'})]), + ).toBeNull(); + // A screen-edge pan from one edge goes one way, inward (spec:AST-094 FR18). + expect( + deriveTouchAction([ + new ScreenEdgePanGestureRecognizer({edges: ['leading']}), + ]), + ).toBeNull(); + expect( + deriveTouchAction([ + new PanGestureRecognizer({direction: 'down', scrollEdge: true}), + ]), + ).toBeNull(); + expect( + deriveTouchAction([ + new PanGestureRecognizer({direction: 'down', scrollEdge: true}), + new TapGestureRecognizer(), + ]), + ).toBe('manipulation'); + }); + + it('takes pinch-zoom only for a pinch or rotation recognizer', () => { + expect(deriveTouchAction([new PinchGestureRecognizer()])).toBe( + 'pan-x pan-y', + ); + expect(deriveTouchAction([new RotationGestureRecognizer()])).toBe( + 'pan-x pan-y', + ); + expect( + deriveTouchAction([ + new PinchGestureRecognizer(), + new PanGestureRecognizer(), + ]), + ).toBe('none'); + }); + + it('intersects every recognizer on the element', () => { + expect( + deriveTouchAction([ + new TapGestureRecognizer(), + new PanGestureRecognizer({direction: 'horizontal'}), + ]), + ).toBe('pan-y pinch-zoom'); + }); + + it('attaches the touch-action alone, never a selection lock (spec:AST-094 FR41)', () => { + // A mouse and a keyboard keep selecting (spec:AST-017 FR59); the lock + // for one held press is the long press's own selectionLock. + expect(gestureHostStyle([new LongPressGestureRecognizer()])).toEqual({ + touchAction: 'manipulation', + }); + }); +}); + +describe('parseTouchAction', () => { + it('reads the pans each value allows', () => { + expect(parseTouchAction('auto')).toEqual({x: true, y: true}); + expect(parseTouchAction('manipulation')).toEqual({x: true, y: true}); + expect(parseTouchAction('none')).toEqual({x: false, y: false}); + expect(parseTouchAction('pan-y pinch-zoom')).toEqual({x: false, y: true}); + expect(parseTouchAction('pan-left')).toEqual({x: true, y: false}); + expect(parseTouchAction('pinch-zoom')).toEqual({x: false, y: false}); + }); +}); + +/** Give an element the scroll metrics jsdom cannot lay out. */ +function scroller( + element: HTMLElement, + metrics: { + scrollTop?: number; + scrollHeight?: number; + clientHeight?: number; + scrollLeft?: number; + scrollWidth?: number; + clientWidth?: number; + }, +): void { + for (const [key, value] of Object.entries(metrics)) { + Object.defineProperty(element, key, {configurable: true, value}); + } +} + +/** + * jsdom drops `touch-action` from styles, so the elements under test carry + * it in `data-touch-action` and computed style reports it from there. + */ +function withTouchActionFromData(): () => void { + const original = window.getComputedStyle.bind(window); + const spy = vi + .spyOn(window, 'getComputedStyle') + .mockImplementation((element: Element, pseudo?: string | null) => { + const style = original(element, pseudo); + const touchAction = element.getAttribute('data-touch-action'); + if (touchAction == null) { + return style; + } + return new Proxy(style, { + get(target, key): string | number | object | null | undefined { + if (key === 'touchAction') { + return touchAction; + } + const value = Reflect.get(target, key) as + string | number | object | null | undefined; + return typeof value === 'function' ? value.bind(target) : value; + }, + }); + }); + return () => spy.mockRestore(); +} + +describe('createNativeScrollProbe', () => { + let restore: () => void = () => {}; + beforeEach(() => { + restore = withTouchActionFromData(); + }); + afterEach(() => { + restore(); + document.body.innerHTML = ''; + document.documentElement.style.overflow = ''; + }); + + function setup(listStyle: string) { + // The page itself does not scroll in these cases. + document.documentElement.style.overflow = 'hidden'; + document.body.innerHTML = `
`; + const list = document.getElementById('list') as HTMLElement; + const row = document.getElementById('row') as HTMLElement; + return {list, row}; + } + + it('answers by edge: at the top a pull down scrolls nothing, a pull up does', () => { + const {list, row} = setup('overflow-y: auto'); + scroller(list, {scrollTop: 0, scrollHeight: 1000, clientHeight: 400}); + const probe = createNativeScrollProbe(row); + expect(probe.canScroll(0, 20)).toBe(false); + expect(probe.canScroll(0, -20)).toBe(true); + expect(probe.isInsideScroller).toBe(true); + scroller(list, {scrollTop: 600}); + expect(probe.canScroll(0, -20)).toBe(false); + expect(probe.canScroll(0, 20)).toBe(true); + }); + + it('finds a scroller in a shadow tree around the slot a light-DOM row is slotted into', () => { + document.documentElement.style.overflow = 'hidden'; + document.body.innerHTML = + '
'; + const host = document.getElementById('host') as HTMLElement; + const row = document.getElementById('row') as HTMLElement; + const shadow = host.attachShadow({mode: 'open'}); + shadow.innerHTML = + '
'; + const list = shadow.getElementById('list') as HTMLElement; + scroller(list, {scrollTop: 0, scrollHeight: 1000, clientHeight: 400}); + const probe = createNativeScrollProbe(row); + expect(probe.isInsideScroller).toBe(true); + expect(probe.canScroll(0, -20)).toBe(true); + }); + + it('puts a control in a fixed bar outside the page it floats over (spec:AST-094 FR11)', () => { + document.body.innerHTML = + '
'; + const root = document.scrollingElement ?? document.documentElement; + scroller(root as HTMLElement, { + scrollTop: 0, + scrollHeight: 3000, + clientHeight: 800, + }); + const probe = createNativeScrollProbe(document.getElementById('row')); + // The page still scrolls under a drag that starts on the bar, but the bar + // is not in a scroll view, so its control gets no content-touch delay. + expect(probe.isInsideScroller).toBe(false); + expect(probe.canScroll(0, -20)).toBe(true); + for (const key of ['scrollTop', 'scrollHeight', 'clientHeight']) { + Reflect.deleteProperty(root, key); + } + }); + + it('honors touch-action between the touch and the scroller', () => { + const {list, row} = setup('overflow-y: auto'); + row.setAttribute('data-touch-action', 'pan-x'); + scroller(list, {scrollTop: 100, scrollHeight: 1000, clientHeight: 400}); + const probe = createNativeScrollProbe(row); + expect(probe.canScroll(0, 20)).toBe(false); + expect(probe.allowsNativePan).toBe(true); + row.setAttribute('data-touch-action', 'none'); + expect(createNativeScrollProbe(row).allowsNativePan).toBe(false); + }); + + it('ignores the wobble of a mostly straight drag', () => { + const {list, row} = setup('overflow-x: auto; overflow-y: hidden'); + scroller(list, {scrollLeft: 50, scrollWidth: 1000, clientWidth: 300}); + const probe = createNativeScrollProbe(row); + expect(probe.canScroll(2, 30)).toBe(false); + expect(probe.canScroll(30, 2)).toBe(true); + }); + + it('follows the RTL scrollLeft range', () => { + const {list, row} = setup( + 'overflow-x: auto; overflow-y: hidden; direction: rtl', + ); + // RTL starts at 0 and runs negative toward the inline end (the left). + scroller(list, {scrollLeft: 0, scrollWidth: 1000, clientWidth: 300}); + const probe = createNativeScrollProbe(row); + expect(probe.canScroll(30, 0)).toBe(true); // reveals content on the left + expect(probe.canScroll(-30, 0)).toBe(false); + scroller(list, {scrollLeft: -700}); + expect(probe.canScroll(30, 0)).toBe(false); + }); + + it('stops the chain at overscroll-behavior: contain', () => { + document.documentElement.style.overflow = ''; + document.body.innerHTML = `
x
`; + const outer = document.getElementById('outer') as HTMLElement; + const inner = document.getElementById('inner') as HTMLElement; + scroller(outer, {scrollTop: 300, scrollHeight: 2000, clientHeight: 500}); + scroller(inner, {scrollTop: 0, scrollHeight: 800, clientHeight: 400}); + const probe = createNativeScrollProbe(document.getElementById('t')); + // The inner list is at its top and contains its overscroll: a pull + // down reaches nothing, though the outer one could scroll. + expect(probe.canScroll(0, 20)).toBe(false); + inner.style.overscrollBehavior = 'auto'; + expect( + createNativeScrollProbe(document.getElementById('t')).canScroll(0, 20), + ).toBe(true); + }); +}); diff --git a/packages/core/src/gestures/__tests__/haptics.test.ts b/packages/core/src/gestures/__tests__/haptics.test.ts new file mode 100644 index 0000000000000..fe2bba423c5af --- /dev/null +++ b/packages/core/src/gestures/__tests__/haptics.test.ts @@ -0,0 +1,182 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file haptics.test.ts + * @input playHaptic, attachTapHaptic and the support probes, with + * navigator.vibrate stubbed or absent, in jsdom + * @output Vibration patterns per feedback kind where the platform vibrates, + * nothing where it does not; the hidden switch iOS Safari plays its tap + * haptic on: hidden from layout, touch and assistive technology, its click + * and change kept inside, removed on release + * @position Unit tests of packages/core/src/gestures/haptics.ts + */ + +import {afterEach, describe, expect, it, vi} from 'vitest'; +import { + HAPTIC_FEEDBACK_KINDS, + HAPTIC_VIBRATION_PATTERNS, +} from '../../motion/uikit/gestures'; +import { + attachTapHaptic, + canVibrate, + HAPTIC_SWITCH_ATTRIBUTE, + playHaptic, + supportsSwitchControl, +} from '../haptics'; + +const original = Object.getOwnPropertyDescriptor(navigator, 'vibrate'); + +function stubVibrate(implementation: (pattern: number[]) => boolean) { + const vibrate = vi.fn(implementation); + Object.defineProperty(navigator, 'vibrate', { + configurable: true, + value: vibrate, + }); + return vibrate; +} + +afterEach(() => { + if (original == null) { + delete (navigator as {vibrate?: unknown}).vibrate; + } else { + Object.defineProperty(navigator, 'vibrate', original); + } + document.body.innerHTML = ''; +}); + +describe('playHaptic', () => { + it('vibrates each feedback kind with its own pattern', () => { + const vibrate = stubVibrate(() => true); + expect(canVibrate()).toBe(true); + for (const kind of HAPTIC_FEEDBACK_KINDS) { + expect(playHaptic(kind)).toBe(true); + expect(vibrate).toHaveBeenLastCalledWith([ + ...HAPTIC_VIBRATION_PATTERNS[kind].value, + ]); + } + }); + + it('plays nothing where the platform cannot vibrate, or refuses', () => { + delete (navigator as {vibrate?: unknown}).vibrate; + expect(canVibrate()).toBe(false); + expect(playHaptic('impactLight')).toBe(false); + stubVibrate(() => { + throw new Error('blocked'); + }); + expect(playHaptic('selection')).toBe(false); + }); +}); + +describe('attachTapHaptic', () => { + it('gives the label a hidden switch that keeps its own events inside', () => { + document.body.innerHTML = ''; + const label = document.getElementById('tap') as HTMLLabelElement; + const onClick = vi.fn(); + const onChange = vi.fn(); + label.addEventListener('click', onClick); + label.addEventListener('change', onChange); + + const release = attachTapHaptic(label); + const input = label.querySelector(`[${HAPTIC_SWITCH_ATTRIBUTE}]`); + expect(input).toBeInstanceOf(HTMLInputElement); + const sw = input as HTMLInputElement; + expect(sw.type).toBe('checkbox'); + expect(sw.hasAttribute('switch')).toBe(true); + expect(sw.getAttribute('aria-hidden')).toBe('true'); + expect(sw.tabIndex).toBe(-1); + // It never takes the touch itself, so a pan starting on it still + // scrolls, and it takes no room. + expect(sw.style.pointerEvents).toBe('none'); + expect(sw.style.position).toBe('absolute'); + + // The switch toggling (the label's activation) does not reach the + // control as a second click or a change. + sw.click(); + expect(onClick).not.toHaveBeenCalled(); + expect(onChange).not.toHaveBeenCalled(); + + release(); + expect(label.querySelector(`[${HAPTIC_SWITCH_ATTRIBUTE}]`)).toBeNull(); + }); + + it('never leaves the focus on its hidden switch', () => { + document.body.innerHTML = + ''; + const before = document.getElementById('before') as HTMLButtonElement; + const label = document.getElementById('tap') as HTMLLabelElement; + const release = attachTapHaptic(label); + const sw = label.querySelector( + `[${HAPTIC_SWITCH_ATTRIBUTE}]`, + )!; + before.focus(); + // Chromium focuses a label's control on its activation. + sw.focus(); + expect(document.activeElement).not.toBe(sw); + expect(document.activeElement).toBe(before); + release(); + }); + + it('keeps a label\u2019s own control: the tap plays and the control still toggles', () => { + document.body.innerHTML = + ''; + const label = document.getElementById('tap') as HTMLLabelElement; + const own = document.getElementById('own') as HTMLInputElement; + const onOwnClick = vi.fn(); + own.addEventListener('click', onOwnClick); + const release = attachTapHaptic(label); + const sw = label.querySelector( + `[${HAPTIC_SWITCH_ATTRIBUTE}]`, + )!; + // The switch takes the label's activation, so a tap plays. + expect(label.control).toBe(sw); + // The checkbox keeps its name: the label's text, through aria-labelledby. + expect(own).toHaveAccessibleName('Remember me'); + sw.click(); + expect(sw.checked).toBe(true); + expect(own.checked).toBe(true); + expect(onOwnClick).toHaveBeenCalledTimes(1); + release(); + expect(label.control).toBe(own); + expect(own).not.toHaveAttribute('aria-labelledby'); + }); + + it('gives a disabled control no feedback and leaves no focus on the switch (FR23)', () => { + document.body.innerHTML = + '' + + ''; + const before = document.getElementById('before') as HTMLButtonElement; + const label = document.getElementById('tap') as HTMLLabelElement; + const own = document.getElementById('own') as HTMLInputElement; + const release = attachTapHaptic(label); + const sw = label.querySelector( + `[${HAPTIC_SWITCH_ATTRIBUTE}]`, + )!; + before.focus(); + sw.focus(); + sw.click(); + // The switch did not toggle (nothing plays), the control did not change, + // and focus is not stranded on the hidden switch. + expect(sw.checked).toBe(false); + expect(own.checked).toBe(false); + expect(document.activeElement).toBe(before); + release(); + }); + + it('leaves a label whose control is already a switch, or tied by for, as it is', () => { + document.body.innerHTML = + '' + + ''; + for (const id of ['a', 'b']) { + const label = document.getElementById(id) as HTMLLabelElement; + const release = attachTapHaptic(label); + expect(label.querySelector(`[${HAPTIC_SWITCH_ATTRIBUTE}]`)).toBeNull(); + release(); + } + }); + + it('reports whether the browser knows the switch control', () => { + // jsdom has no `switch` property on checkbox inputs, like browsers + // before Safari 17.4. + expect(supportsSwitchControl(document)).toBe(false); + }); +}); diff --git a/packages/core/src/gestures/__tests__/harness.ts b/packages/core/src/gestures/__tests__/harness.ts index a54549412f7e0..b3416fc5be785 100644 --- a/packages/core/src/gestures/__tests__/harness.ts +++ b/packages/core/src/gestures/__tests__/harness.ts @@ -4,8 +4,9 @@ * @file harness.ts * @input Nothing * @output Exports the synthetic-input harness the gesture tests share: a - * manual clock, an arena wired to a transition log, and fingers that land, - * move and lift at explicit timestamps + * manual clock, an arena wired to a transition log, fingers that land, + * move and lift at explicit timestamps, and a coarse primary pointer for + * tests that go through the document's host * @position Test-only helper of packages/core/src/gestures (excluded from the * build with the rest of __tests__) */ @@ -40,7 +41,13 @@ export interface Harness { claims: number[]; /** Deferred clicks as they resolved. */ deferredClicks: {id: number; allowed: boolean; time: number}[]; - environment: {width: number; height: number; rtl: boolean}; + environment: { + width: number; + height: number; + rtl: boolean; + /** The size every recognizer's view measures, when a test sets one. */ + view?: {width: number; height: number}; + }; finger(id: number): Finger; /** Advance the clock and run every timer due by `time`. */ advance(time: number): void; @@ -51,7 +58,11 @@ export function createHarness(): Harness { const log: string[] = []; const claims: number[] = []; const deferredClicks: Harness['deferredClicks'] = []; - const environment = {width: 390, height: 844, rtl: false}; + const environment: Harness['environment'] = { + width: 390, + height: 844, + rtl: false, + }; const fingers = new Map(); const env: GestureEnvironment = { viewportSize: () => ({ @@ -59,6 +70,7 @@ export function createHarness(): Harness { height: environment.height, }), isRtl: () => environment.rtl, + viewSize: () => environment.view ?? null, }; const arena: GestureArena = new GestureArena({ clock, @@ -202,3 +214,31 @@ export function scrollProbe( ): NativeScrollProbe { return {canScroll, isInsideScroller, allowsNativePan: true}; } + +/** + * Declare a phone's primary pointer for a test that goes through the + * document's host (`getGestureHost`): the host listens only where + * `(pointer: coarse)` matches (`spec:AST-094` FR19, FR24), and jsdom has no + * `matchMedia`. Returns the undo. + */ +export function coarsePrimaryPointer(): () => void { + const hadOwn = Object.prototype.hasOwnProperty.call(window, 'matchMedia'); + const original = window.matchMedia; + window.matchMedia = (query: string) => ({ + matches: query === '(pointer: coarse)', + media: query, + onchange: null, + addEventListener: () => {}, + removeEventListener: () => {}, + addListener: () => {}, + removeListener: () => {}, + dispatchEvent: () => false, + }); + return () => { + if (hadOwn) { + window.matchMedia = original; + } else { + delete (window as {matchMedia?: unknown}).matchMedia; + } + }; +} diff --git a/packages/core/src/gestures/__tests__/passiveByDefault.test.tsx b/packages/core/src/gestures/__tests__/passiveByDefault.test.tsx new file mode 100644 index 0000000000000..8d553dd3ac358 --- /dev/null +++ b/packages/core/src/gestures/__tests__/passiveByDefault.test.tsx @@ -0,0 +1,88 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file passiveByDefault.test.tsx + * @input Uses vitest, Testing Library, Button, Item, GestureHost and + * PressTracker + * @output Checks that plain pressables never get a blocking touchmove + * listener, and that only a press tracker created with `keepsTouch: true` + * does + * @position Guards the 60 fps scroll goal: a scroll that starts on a plain + * button or a list cell never waits on the main thread + */ + +import {render} from '@testing-library/react'; +import {afterEach, beforeEach, describe, expect, it, vi} from 'vitest'; +import {Button} from '../../Button/Button'; +import {Item} from '../../Item/Item'; +import {GestureHost} from '../GestureHost'; +import {PressTracker} from '../PressTracker'; +import {ManualClock} from './harness'; +import {coarsePrimaryPointer} from './pointerQuery'; + +function touchStart(target: Element): void { + const event = new Event('touchstart', {bubbles: true, cancelable: true}); + const entry = {identifier: 1, clientX: 5, clientY: 5, target}; + Object.defineProperty(event, 'changedTouches', {value: [entry]}); + Object.defineProperty(event, 'touches', {value: [entry]}); + target.dispatchEvent(event); +} + +function blockingTouchMoves(spy: {mock: {calls: unknown[][]}}): number { + return spy.mock.calls.filter( + ([type, , options]) => + type === 'touchmove' && + (options as AddEventListenerOptions | undefined)?.passive === false, + ).length; +} + +afterEach(() => { + vi.restoreAllMocks(); + document.body.innerHTML = ''; +}); + +// A phone: the host attaches only where the primary pointer is coarse. +let restorePointer: () => void; +beforeEach(() => { + restorePointer = coarsePrimaryPointer(); +}); +afterEach(() => { + restorePointer(); +}); + +describe('blocking touchmove listeners', () => { + it('never come with a plain Button or an Item cell', () => { + const spy = vi.spyOn(EventTarget.prototype, 'addEventListener'); + const {getByRole, getByText} = render( +
+
, + ); + touchStart(getByRole('button', {name: 'Save'})); + touchStart(getByText('Row')); + expect(blockingTouchMoves(spy)).toBe(0); + }); + + it('come only with a press tracker that keeps its touch', () => { + document.body.innerHTML = '
'; + const plain = document.getElementById('plain') as HTMLElement; + const stepper = document.getElementById('stepper') as HTMLElement; + const host = new GestureHost(document, {clock: new ManualClock()}); + const onPlain = vi.spyOn(plain, 'addEventListener'); + const onStepper = vi.spyOn(stepper, 'addEventListener'); + const detachPlain = host.attachConsumer( + plain, + new PressTracker({getBounds: () => null}), + ); + const detachStepper = host.attachConsumer( + stepper, + new PressTracker({getBounds: () => null, keepsTouch: true}), + ); + touchStart(plain); + expect(blockingTouchMoves(onPlain)).toBe(0); + expect(blockingTouchMoves(onStepper)).toBe(1); + detachPlain(); + detachStepper(); + }); +}); diff --git a/packages/core/src/gestures/__tests__/pointerQuery.ts b/packages/core/src/gestures/__tests__/pointerQuery.ts new file mode 100644 index 0000000000000..a260bef9bfb5f --- /dev/null +++ b/packages/core/src/gestures/__tests__/pointerQuery.ts @@ -0,0 +1,75 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file pointerQuery.ts + * @input jsdom's window.matchMedia + * @output coarsePrimaryPointer() and pointerQuery(): the primary pointer a + * test needs, until restored + * @position Test helper of packages/core/src/gestures + * + * jsdom's matchMedia models a desktop, whose primary pointer is fine, and + * the gesture host attaches nothing there (`spec:AST-094` FR19). A test of + * touch behaviour models a phone with coarsePrimaryPointer(); a test of the + * gate itself drives pointerQuery() and fires its change. + */ + +import {COARSE_POINTER_QUERY} from '../../utils/coarsePointer'; + +/** The primary pointer as a test sets it. */ +export interface PointerQuery { + /** Turn the primary pointer coarse or fine, firing `change`. */ + set(isCoarse: boolean): void; + /** Put jsdom's matchMedia back. */ + readonly restore: () => void; +} + +/** + * The primary pointer is coarse (`isCoarse`) or fine until `restore` runs; + * `set` changes it as a tablet docking a mouse would. + */ +export function pointerQuery(isCoarse: boolean): PointerQuery { + const matchMedia = window.matchMedia; + const listeners = new Set<() => void>(); + let coarse = isCoarse; + const query = { + get matches() { + return coarse; + }, + media: COARSE_POINTER_QUERY, + onchange: null, + addEventListener: (_type: string, listener: () => void) => { + listeners.add(listener); + }, + removeEventListener: (_type: string, listener: () => void) => { + listeners.delete(listener); + }, + addListener: (listener: () => void) => { + listeners.add(listener); + }, + removeListener: (listener: () => void) => { + listeners.delete(listener); + }, + dispatchEvent: () => false, + } as unknown as MediaQueryList; + window.matchMedia = (media: string) => + media.trim() === COARSE_POINTER_QUERY ? query : matchMedia(media); + return { + set(next) { + if (next === coarse) { + return; + } + coarse = next; + for (const listener of [...listeners]) { + listener(); + } + }, + restore: () => { + window.matchMedia = matchMedia; + }, + }; +} + +/** The primary pointer is coarse (a phone, a tablet) until the restore runs. */ +export function coarsePrimaryPointer(): () => void { + return pointerQuery(true).restore; +} diff --git a/packages/core/src/gestures/__tests__/recognizerInternals.test.ts b/packages/core/src/gestures/__tests__/recognizerInternals.test.ts new file mode 100644 index 0000000000000..84fb080627426 --- /dev/null +++ b/packages/core/src/gestures/__tests__/recognizerInternals.test.ts @@ -0,0 +1,52 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file recognizerInternals.test.ts + * @input Uses vitest, TapGestureRecognizer and recognizerInternals + * @output Checks that a recognizer's arena plumbing is off its public type + * and reachable through internals() + * @position Test for packages/core/src/gestures/recognizerInternals.ts + * + * The `@ts-expect-error` lines are the assertion: `tsc` fails if any of + * these members is public again. + */ + +import {describe, expect, it} from 'vitest'; +import {TapGestureRecognizer} from '../TapGestureRecognizer'; +import {internals} from '../recognizerInternals'; + +describe('recognizer internals', () => { + it('keeps the arena plumbing protected on the published class', () => { + const tap = new TapGestureRecognizer({name: 'tap'}); + function reachesPlumbing(): void { + // @ts-expect-error protected: the arena assigns it + tap.arena = null; + // @ts-expect-error protected + void tap.pending; + // @ts-expect-error protected + void tap.seenTouchIds; + // @ts-expect-error protected + void tap.acceptsTouchInput; + // @ts-expect-error protected + tap.requiresFailureOf(tap); + // @ts-expect-error protected + tap.allowsSimultaneous(tap); + // @ts-expect-error protected + tap.shouldBegin(); + // @ts-expect-error protected + tap.applyState('began'); + // @ts-expect-error protected + tap.resetToPossible(); + // @ts-expect-error protected + tap.deliverCancelled([], {time: 0, allTouches: []}); + } + expect(typeof reachesPlumbing).toBe('function'); + }); + + it('lets the gesture system reach them', () => { + const tap = new TapGestureRecognizer({name: 'tap'}); + expect(internals(tap).arena).toBeNull(); + expect(internals(tap).acceptsTouchInput).toBe(true); + expect(internals(tap).shouldBegin()).toBe(true); + }); +}); diff --git a/packages/core/src/gestures/__tests__/selectionLock.test.ts b/packages/core/src/gestures/__tests__/selectionLock.test.ts new file mode 100644 index 0000000000000..9cc69107c1216 --- /dev/null +++ b/packages/core/src/gestures/__tests__/selectionLock.test.ts @@ -0,0 +1,94 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file selectionLock.test.ts + * @input lockSelection on the elements a hold acts on, in jsdom, with and + * without iOS WebKit's touch callout + * @output A hold that acts starts no selection and no callout, a link's + * callout included (`spec:AST-094` FR41, DEC-9). Where iOS WebKit runs the + * lock is a style; elsewhere `selectstart` is cancelled and no style is + * written + * @position Unit tests for gestures/selectionLock.ts + */ + +import {afterEach, describe, expect, it, vi} from 'vitest'; +import {lockSelection} from '../selectionLock'; + +/** Make `CSS.supports` answer as iOS WebKit does for its touch callout. */ +function asIosWebKit(): () => void { + const original = globalThis.CSS; + Object.defineProperty(globalThis, 'CSS', { + configurable: true, + value: { + ...original, + supports: (query: string) => query === '(-webkit-touch-callout: none)', + }, + }); + return () => { + Object.defineProperty(globalThis, 'CSS', { + configurable: true, + value: original, + }); + }; +} + +function mount(html: string): (id: string) => HTMLElement { + document.body.innerHTML = html; + return id => document.getElementById(id) as HTMLElement; +} + +/** Whether a selection starting on `target` was refused. */ +function refuses(target: Element): boolean { + const event = new Event('selectstart', {bubbles: true, cancelable: true}); + target.dispatchEvent(event); + return event.defaultPrevented; +} + +afterEach(() => { + document.body.innerHTML = ''; + vi.restoreAllMocks(); +}); + +describe('lockSelection where iOS WebKit runs', () => { + it("suppresses a link's touch callout when its hold acts, under the link too (spec:AST-094 FR41, DEC-9)", () => { + const restore = asIosWebKit(); + try { + const get = mount( + 'Next', + ); + const set = vi.spyOn(get('row').style, 'setProperty'); + const release = lockSelection([get('row')], get('label')); + expect(set).toHaveBeenCalledWith('-webkit-touch-callout', 'none'); + expect(set).toHaveBeenCalledWith('user-select', 'none'); + release(); + } finally { + restore(); + } + }); +}); + +describe('lockSelection elsewhere', () => { + it('refuses a selection by cancelling selectstart, with no style write, until the release (spec:AST-094 FR41)', () => { + const get = mount( + '
Row
', + ); + const set = vi.spyOn(get('row').style, 'setProperty'); + const release = lockSelection([get('row')], get('label')); + expect(set).not.toHaveBeenCalled(); + expect(refuses(get('label'))).toBe(true); + // A text field keeps its selection: a held finger places its caret. + expect(refuses(get('field'))).toBe(false); + release(); + expect(refuses(get('label'))).toBe(false); + }); + + it('keeps refusing while any holder holds', () => { + const get = mount('
Row
'); + const first = lockSelection([get('row')], get('label')); + const second = lockSelection([get('row')], get('label')); + first(); + expect(refuses(get('label'))).toBe(true); + second(); + expect(refuses(get('label'))).toBe(false); + }); +}); diff --git a/packages/core/src/gestures/__tests__/useGestureRecognizer.test.tsx b/packages/core/src/gestures/__tests__/useGestureRecognizer.test.tsx new file mode 100644 index 0000000000000..3a24d09914dda --- /dev/null +++ b/packages/core/src/gestures/__tests__/useGestureRecognizer.test.tsx @@ -0,0 +1,1233 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file useGestureRecognizer.test.tsx + * @input React components using useGestureRecognizer / usePressTracker, and + * touch events dispatched on their elements + * @output The hooks keep one recognizer per component, attach before the + * first touch, run the latest listener without rendering per move, and + * detach on unmount + * @position jsdom tests of packages/core/src/gestures/useGestureRecognizer.ts + */ + +import {act, render} from '@testing-library/react'; +import {StrictMode, useEffect, useRef} from 'react'; +import {afterEach, beforeEach, describe, expect, it, vi} from 'vitest'; +import type {SelectionLock} from '../GestureRecognizer'; +import {getGestureHost} from '../GestureHost'; +import {PanGestureRecognizer} from '../PanGestureRecognizer'; +import type {ControlEvent} from '../PressTracker'; +import {lockSelection} from '../selectionLock'; +import {TapGestureRecognizer} from '../TapGestureRecognizer'; +import {useGestureRecognizer, usePressTracker} from '../useGestureRecognizer'; +import {coarsePrimaryPointer, pointerQuery} from './pointerQuery'; + +function touch( + type: string, + target: EventTarget, + contact: {id: number; x: number; y: number}, + time: number, + isLift = false, +): void { + const event = new Event(type, {bubbles: true, cancelable: true}); + const entry = { + identifier: contact.id, + clientX: contact.x, + clientY: contact.y, + target, + }; + Object.defineProperty(event, 'changedTouches', {value: [entry]}); + Object.defineProperty(event, 'touches', {value: isLift ? [] : [entry]}); + Object.defineProperty(event, 'timeStamp', {value: time}); + target.dispatchEvent(event); +} + +let renders = 0; + +function Draggable({ + onPan, + onRecognizer, + isEnabled = true, +}: { + onPan: (recognizer: PanGestureRecognizer) => void; + onRecognizer?: (recognizer: PanGestureRecognizer) => void; + isEnabled?: boolean; +}) { + renders += 1; + const ref = useRef(null); + const pan = useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + onPan, + {isEnabled}, + ); + onRecognizer?.(pan); + return
; +} + +// A phone: the host attaches only where the primary pointer is coarse. +let restorePointer: () => void = () => {}; +beforeEach(() => { + restorePointer = coarsePrimaryPointer(); +}); +afterEach(() => { + restorePointer(); +}); + +/** Wait for the next animation frame, where the host writes touch-action. */ +async function nextFrame(): Promise { + await act(async () => { + await new Promise(resolve => { + requestAnimationFrame(() => resolve()); + }); + }); +} +describe('useGestureRecognizer', () => { + afterEach(() => { + renders = 0; + }); + + it('drives the latest listener with no render per move', () => { + const seen: string[] = []; + const {getByTestId} = render( + { + seen.push(`${recognizer.state}:${recognizer.translation().x}`); + }} + />, + ); + const surface = getByTestId('surface'); + expect(renders).toBe(1); + touch('touchstart', surface, {id: 1, x: 0, y: 0}, 0); + for (let x = 6; x <= 36; x += 6) { + touch('touchmove', surface, {id: 1, x, y: 0}, x * 2); + } + touch('touchend', surface, {id: 1, x: 36, y: 0}, 80, true); + // The 10 px hysteresis is left out of the translation, as in UIKit. + expect(seen).toEqual([ + 'began:2', + 'changed:8', + 'changed:14', + 'changed:20', + 'changed:26', + 'ended:26', + ]); + expect(renders).toBe(1); + }); + + it('keeps one recognizer across renders and reads the newest listener', () => { + const identities = new Set(); + const first = vi.fn(); + const second = vi.fn(); + const {getByTestId, rerender} = render( + identities.add(r)} />, + ); + rerender( + identities.add(r)} />, + ); + expect(identities.size).toBe(1); + const surface = getByTestId('surface'); + touch('touchstart', surface, {id: 2, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 2, x: 30, y: 0}, 16); + expect(first).not.toHaveBeenCalled(); + expect(second).toHaveBeenCalledTimes(1); + touch('touchend', surface, {id: 2, x: 30, y: 0}, 32, true); + }); + + it('cancels a gesture in flight when it is disabled, without calling the listener from the effect', () => { + const states: string[] = []; + // A listener of the recognizer's own, outside the hook, hears the cancel. + const heard: string[] = []; + let pan: PanGestureRecognizer | null = null; + const {getByTestId, rerender} = render( + states.push(r.state)} + onRecognizer={r => { + if (pan == null) { + pan = r; + r.addListener(current => heard.push(current.state)); + } + }} + />, + ); + const surface = getByTestId('surface'); + touch('touchstart', surface, {id: 3, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 3, x: 30, y: 0}, 16); + act(() => { + rerender( + states.push(r.state)} isEnabled={false} />, + ); + }); + // Disabling cancels the gesture; its attachment then goes (FR24), which + // leaves the recognizer at rest. The hook reports nothing from the effect. + expect(heard).toContain('cancelled'); + expect(pan!.state).not.toBe('changed'); + expect(states).toEqual(['began']); + touch('touchend', surface, {id: 3, x: 30, y: 0}, 32, true); + expect(states).toEqual(['began']); + }); + + it('follows the element when the node behind the ref is replaced', () => { + const onPan = vi.fn(); + function Swappable({tag}: {tag: 'div' | 'section'}) { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + onPan, + ); + const Tag = tag; + return ; + } + const {getByTestId, rerender} = render(); + const first = getByTestId('surface'); + rerender(); + const second = getByTestId('surface'); + expect(second).not.toBe(first); + touch('touchstart', first, {id: 6, x: 0, y: 0}, 0); + touch('touchmove', first, {id: 6, x: 30, y: 0}, 16); + touch('touchend', first, {id: 6, x: 30, y: 0}, 32, true); + expect(onPan).not.toHaveBeenCalled(); + touch('touchstart', second, {id: 7, x: 0, y: 0}, 100); + touch('touchmove', second, {id: 7, x: 30, y: 0}, 116); + expect(onPan).toHaveBeenCalled(); + touch('touchend', second, {id: 7, x: 30, y: 0}, 132, true); + }); + + it('reports cancelled to a handler that disables the recognizer itself, from that handler', () => { + const states: string[] = []; + let pan: PanGestureRecognizer | null = null; + const {getByTestId} = render( + states.push(r.state)} + onRecognizer={r => { + pan = r; + }} + />, + ); + const surface = getByTestId('surface'); + touch('touchstart', surface, {id: 6, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 6, x: 30, y: 0}, 16); + // A handler (here, the test) turns it off: UIKit's .cancelled, at once. + (pan as PanGestureRecognizer | null)?.setEnabled(false); + expect(states).toEqual(['began', 'cancelled']); + touch('touchend', surface, {id: 6, x: 30, y: 0}, 32, true); + }); + + it("attaches a recognizer that reads each event's pointer where the primary pointer is fine (spec:AST-057)", () => { + const pointer = pointerQuery(false); + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + try { + const onPan = vi.fn(); + function Swipe() { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + onPan, + {perEventPointer: true}, + ); + return
; + } + const {getByTestId, unmount} = render(); + const surface = getByTestId('surface'); + touch('touchstart', surface, {id: 9, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 9, x: 30, y: 0}, 16); + expect(onPan).toHaveBeenCalled(); + touch('touchend', surface, {id: 9, x: 30, y: 0}, 32, true); + unmount(); + expect(warn).not.toHaveBeenCalled(); + } finally { + warn.mockRestore(); + pointer.restore(); + } + }); + + it('reports a recognizer that required the disabled one to fail after the isEnabled Effect, not inside it, in its recognized state (react-component-runtime INV2)', async () => { + const states: string[] = []; + const onSingle = vi.fn((recognizer: TapGestureRecognizer) => { + states.push(recognizer.state); + }); + function Taps({isDoubleEnabled}: {isDoubleEnabled: boolean}) { + const ref = useRef(null); + const double = useGestureRecognizer( + ref, + () => new TapGestureRecognizer({numberOfTapsRequired: 2}), + undefined, + {isEnabled: isDoubleEnabled}, + ); + const single = useGestureRecognizer( + ref, + () => new TapGestureRecognizer(), + onSingle, + ); + useEffect(() => { + single.requireToFail(double); + }, [single, double]); + return
; + } + const {getByTestId, rerender} = render(); + const surface = getByTestId('taps'); + touch('touchstart', surface, {id: 1, x: 5, y: 5}, 0); + touch('touchend', surface, {id: 1, x: 5, y: 5}, 40, true); + // The single tap waits for the double tap to fail. + expect(onSingle).not.toHaveBeenCalled(); + // Turning the double tap off fails it in the Effect; the single tap + // waiting on it recognizes in a pass just after, so its caller hears it + // outside the Effect, reading the recognized state. + rerender(); + expect(onSingle).not.toHaveBeenCalled(); + await act(async () => { + await Promise.resolve(); + }); + expect(states).toEqual(['ended']); + }); + + it('reports a recognizer that required one a replaced node cancelled after that layout Effect, not inside it (react-component-runtime INV2)', async () => { + const states: string[] = []; + function Nested({innerKey}: {innerKey: string}) { + const outerRef = useRef(null); + const innerRef = useRef(null); + const double = useGestureRecognizer( + innerRef, + () => new TapGestureRecognizer({numberOfTapsRequired: 2}), + ); + const single = useGestureRecognizer( + outerRef, + () => new TapGestureRecognizer(), + recognizer => { + states.push(recognizer.state); + }, + ); + useEffect(() => { + single.requireToFail(double); + }, [single, double]); + return ( +
+
+
+ ); + } + const {getByTestId, rerender} = render(); + const inner = getByTestId('inner'); + touch('touchstart', inner, {id: 2, x: 5, y: 5}, 0); + touch('touchend', inner, {id: 2, x: 5, y: 5}, 40, true); + expect(states).toEqual([]); + // The double tap's node is replaced: detaching it cancels its wait in a + // layout Effect, and the outer single tap recognizes just after. + rerender(); + expect(states).toEqual([]); + await act(async () => { + await Promise.resolve(); + }); + expect(states).toEqual(['ended']); + }); + + it('attaches an action group, which the host serves under a fine pointer: a mouse press on one of its actions reaches the recognizer (family:action-groups FR5)', () => { + // A document whose host was never told the pointer is coarse (jsdom's + // frames have no matchMedia): a fine primary pointer. + const frame = document.createElement('iframe'); + document.body.append(frame); + const frameDocument = frame.contentDocument as Document; + const frameWindow = frameDocument.defaultView as Window & typeof globalThis; + const states: string[] = []; + function Group({actionGroup}: {actionGroup: boolean}) { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new TapGestureRecognizer(), + recognizer => { + states.push(recognizer.state); + }, + {actionGroup}, + ); + return ( +
+ +
+ ); + } + const press = (target: Element, pointerId: number, time: number) => { + for (const [type, buttons] of [ + ['pointerdown', 1], + ['pointerup', 0], + ] as const) { + const event = new frameWindow.PointerEvent(type, { + bubbles: true, + cancelable: true, + composed: true, + pointerId, + pointerType: 'mouse', + clientX: 5, + clientY: 5, + button: 0, + buttons, + }); + Object.defineProperty(event, 'timeStamp', {value: time}); + target.dispatchEvent(event); + } + }; + try { + const {getByTestId, rerender, unmount} = render( + , + {container: frameDocument.body, baseElement: frameDocument.body}, + ); + const action = getByTestId('action'); + // Not a group: under a fine pointer the host serves nothing. + press(action, 1, 0); + expect(states).toEqual([]); + act(() => { + rerender(); + }); + press(action, 2, 1000); + expect(states).toEqual(['ended']); + unmount(); + } finally { + frame.remove(); + } + }); + + it('attaches nothing while disabled: no touch-action, no gesture (FR24)', async () => { + const states: string[] = []; + const {getByTestId, rerender} = render( + states.push(r.state)} />, + ); + const surface = getByTestId('surface'); + // The host writes its touch-action in the next frame (FR18). + await nextFrame(); + const remove = vi.spyOn(surface.style, 'removeProperty'); + act(() => { + rerender( + states.push(r.state)} isEnabled={false} />, + ); + }); + // Disabling detaches: the host gives the element its touch-action back. + expect(remove).toHaveBeenCalledWith('touch-action'); + touch('touchstart', surface, {id: 6, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 6, x: 30, y: 0}, 16); + touch('touchend', surface, {id: 6, x: 30, y: 0}, 32, true); + expect(states).toEqual([]); + act(() => { + rerender( states.push(r.state)} />); + }); + touch('touchstart', surface, {id: 8, x: 0, y: 0}, 100); + touch('touchmove', surface, {id: 8, x: 30, y: 0}, 116); + expect(states).toContain('began'); + touch('touchend', surface, {id: 8, x: 30, y: 0}, 132, true); + }); + + it('follows the element behind the ref when it is replaced (INV6)', () => { + const states: string[] = []; + function Swapped({id}: {id: string}) { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + r => states.push(r.state), + ); + return
; + } + const {getByTestId, rerender} = render(); + act(() => { + rerender(); + }); + const second = getByTestId('second'); + touch('touchstart', second, {id: 9, x: 0, y: 0}, 0); + touch('touchmove', second, {id: 9, x: 30, y: 0}, 16); + expect(states).toContain('began'); + touch('touchend', second, {id: 9, x: 30, y: 0}, 32, true); + }); + + it('detaches on unmount', async () => { + const onPan = vi.fn(); + const {getByTestId, unmount} = render(); + const surface = getByTestId('surface'); + // The host writes its touch-action in the next frame (FR18). + await nextFrame(); + const remove = vi.spyOn(surface.style, 'removeProperty'); + unmount(); + expect(remove).toHaveBeenCalledWith('touch-action'); + touch('touchstart', surface, {id: 4, x: 0, y: 0}, 0); + touch('touchmove', surface, {id: 4, x: 30, y: 0}, 16); + expect(onPan).not.toHaveBeenCalled(); + }); +}); + +function Button({ + onEvent, + cancelTravel, + isBraking, + selectionLock, + isDisabled = false, +}: { + onEvent: (event: ControlEvent) => void; + cancelTravel?: number; + isBraking?: () => boolean; + selectionLock?: SelectionLock; + isDisabled?: boolean; +}) { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => onEvent(event), + cancelTravel, + isBraking, + selectionLock, + isEnabled: () => !isDisabled, + }); + return ( + + ); +} + +describe('usePressTracker', () => { + it('gives the host no listener while the control is disabled (FR24)', () => { + const add = vi.spyOn(document, 'addEventListener'); + const remove = vi.spyOn(document, 'removeEventListener'); + const {rerender, unmount} = render( + + ); + } + function Pair({isDisabled}: {isDisabled: boolean}) { + return ( + <> + + ); + } + const {getByTestId, unmount} = render(); + touch('touchstart', getByTestId('held'), {id: 9, x: 1, y: 1}, 0); + const seenHighlights = [...highlights]; + const seenEvents = [...events]; + unmount(); + expect(highlights).toEqual(seenHighlights); + expect(events).toEqual(seenEvents); + }); + + it('attaches a per-event press tracker where the primary pointer is fine, which takes no mouse press there, beside one that stays inactive', () => { + const pointer = pointerQuery(false); + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}); + try { + const held: ControlEvent[] = []; + const inactive: ControlEvent[] = []; + function Stepper({ + events, + perEventPointer, + testId, + }: { + events: ControlEvent[]; + perEventPointer: boolean; + testId: string; + }) { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => events.push(event), + perEventPointer, + }); + return ( + + ); + } + const {getByTestId} = render( + <> + + + , + ); + const mouse = (type: string, target: Element, id: number) => { + const event = new PointerEvent(type, { + bubbles: true, + cancelable: true, + composed: true, + pointerId: id, + pointerType: 'mouse', + clientX: 5, + clientY: 5, + button: 0, + buttons: type === 'pointerdown' ? 1 : 0, + }); + target.dispatchEvent(event); + }; + const stepper = getByTestId('held'); + boundsOf(stepper, 44, 44); + mouse('pointerdown', stepper, 1); + mouse('pointerup', stepper, 1); + // Its press feedback is for a finger (spec:AST-094 FR19): a mouse + // keeps the desktop's handling. + expect(held).toEqual([]); + const other = getByTestId('other'); + boundsOf(other, 44, 44); + mouse('pointerdown', other, 2); + mouse('pointerup', other, 2); + expect(inactive).toEqual([]); + } finally { + warn.mockRestore(); + pointer.restore(); + } + }); + + it('passes isBraking through to the tracker', () => { + const events: ControlEvent[] = []; + const isBraking = vi.fn(() => true); + function BrakingButton() { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => events.push(event), + isBraking, + }); + return ( + + ); + } + const {getByTestId} = render(); + const button = getByTestId('braking'); + touch('touchstart', button, {id: 8, x: 1, y: 1}, 0); + touch('touchend', button, {id: 8, x: 1, y: 1}, 50, true); + expect(isBraking).toHaveBeenCalled(); + expect(events).not.toContain('touchDown'); + }); + + it("uses the caller's scroll brake", () => { + const events: ControlEvent[] = []; + const {getByTestId} = render( +
+ ); + } + + it.each([ + ['alone', false], + ['with a single tap waiting on it', true], + ])( + 'ends a press that waits on an ancestor double tap after, not inside, the isEnabled Effect that turns it off (%s; react-component-runtime INV2)', + async (_event, hasSingle) => { + const events: (ControlEvent | 'highlight' | 'unhighlight')[] = []; + const {getByTestId, rerender} = render( + events.push(event)} + />, + ); + const button = getByTestId('press'); + touch('touchstart', button, {id: 21, x: 1, y: 1}, 0); + touch('touchend', button, {id: 21, x: 1, y: 1}, 40, true); + const before = [...events]; + // Turning the double tap off fails it in the Effect: whatever that + // arena pass resolves for the press reaches no caller callback. + rerender( + events.push(event)} + />, + ); + expect(events).toEqual(before); + // Just after it, the press ends and un-highlights: nothing is left + // highlighted. + await act(async () => { + await Promise.resolve(); + }); + expect(events.slice(before.length)).toEqual([ + 'touchUpInside', + 'unhighlight', + ]); + }, + ); + + it('tracks the element behind the ref when it is replaced (INV6)', () => { + const events: ControlEvent[] = []; + function Swapped({id}: {id: string}) { + const ref = useRef(null); + usePressTracker(ref, {onControlEvent: event => events.push(event)}); + return + ) : null; +} + +describe('usePressTracker follows its element and its options', () => { + it('tracks a node rendered after mount', () => { + const events: ControlEvent[] = []; + const {getByTestId, rerender} = render( + events.push(event)} />, + ); + rerender( events.push(event)} />); + const stepper = getByTestId('stepper'); + touch('touchstart', stepper, {id: 1, x: 2, y: 2}, 0); + touch('touchend', stepper, {id: 1, x: 2, y: 2}, 60, true); + expect(events).toContain('touchUpInside'); + }); + + it("uses the caller's brake", () => { + const events: ControlEvent[] = []; + const {getByTestId} = render( + true} + onEvent={event => events.push(event)} + />, + ); + const stepper = getByTestId('stepper'); + touch('touchstart', stepper, {id: 1, x: 2, y: 2}, 0); + touch('touchend', stepper, {id: 1, x: 2, y: 2}, 60, true); + expect(events).not.toContain('touchDown'); + expect(events).not.toContain('touchUpInside'); + }); + + it('takes a highlight delay changed after mount from the next touch', () => { + vi.useFakeTimers(); + const highlights: boolean[] = []; + const {getByTestId, rerender} = render( + highlights.push(isOn)} + />, + ); + rerender( + highlights.push(isOn)} + />, + ); + const stepper = getByTestId('stepper'); + touch('touchstart', stepper, {id: 1, x: 2, y: 2}, 0); + // 'always' waits the content-touch delay before highlighting. + expect(highlights).toEqual([]); + touch('touchend', stepper, {id: 1, x: 2, y: 2}, 400, true); + vi.useRealTimers(); + }); + + it('calls no callback from cleanup when it unmounts while highlighted', () => { + const highlights: boolean[] = []; + const {getByTestId, unmount} = render( + highlights.push(isOn)} + />, + ); + const stepper = getByTestId('stepper'); + touch('touchstart', stepper, {id: 1, x: 2, y: 2}, 0); + expect(highlights).toEqual([true]); + unmount(); + expect(highlights).toEqual([true]); + }); +}); + +function boundsOf(element: HTMLElement, width: number, height: number): void { + Object.defineProperty(element, 'getBoundingClientRect', { + configurable: true, + value: () => ({ + left: 0, + top: 0, + right: width, + bottom: height, + width, + height, + x: 0, + y: 0, + toJSON: () => ({}), + }), + }); +} + +function SwappedColumn({ + which, + freshRef, + onEvent, +}: { + which: 'a' | 'b'; + freshRef: boolean; + onEvent: (event: ControlEvent) => void; +}) { + const stableRef = useRef(null); + const fresh = {current: null as HTMLButtonElement | null}; + const ref = freshRef ? fresh : stableRef; + usePressTracker(ref, {onControlEvent: event => onEvent(event)}); + return which === 'a' ? ( + + ) : ( + + ); +} + +describe('usePressTracker hit-tests the element behind its ref now', () => { + for (const freshRef of [false, true]) { + it(`judges a lift against the swapped node's bounds (${freshRef ? 'a new ref object' : 'the same ref'})`, () => { + const events: ControlEvent[] = []; + const {getByTestId, rerender} = render( + events.push(event)} + />, + ); + rerender( + events.push(event)} + />, + ); + const b = getByTestId('b'); + boundsOf(b, 100, 40); + touch('touchstart', b, {id: 1, x: 10, y: 10}, 0); + // Far outside B, past the tracking expansion. + touch('touchmove', b, {id: 1, x: 400, y: 400}, 16); + touch('touchend', b, {id: 1, x: 400, y: 400}, 32, true); + expect(events).toContain('touchDown'); + expect(events).toContain('touchUpOutside'); + expect(events).not.toContain('touchUpInside'); + }); + } +}); + +function SwappedSurface({ + which, + onPan, +}: { + which: 'a' | 'b'; + onPan: (recognizer: PanGestureRecognizer) => void; +}) { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + onPan, + ); + return which === 'a' ? ( +
+ ) : ( +
+ ); +} + +describe('useGestureRecognizer follows a replaced node', () => { + for (const strict of [false, true]) { + it(`cancels a gesture in flight without calling the listener from the effect${strict ? ' (StrictMode)' : ''}`, () => { + const states: string[] = []; + const wrap = (node: React.ReactElement) => + strict ? {node} : node; + const {getByTestId, rerender} = render( + wrap( states.push(r.state)} />), + ); + const a = getByTestId('a'); + touch('touchstart', a, {id: 3, x: 0, y: 0}, 0); + touch('touchmove', a, {id: 3, x: 30, y: 0}, 16); + expect(states).toEqual(['began']); + rerender( + wrap( states.push(r.state)} />), + ); + // The replaced node's cancel comes from a layout effect: no callback. + expect(states).toEqual(['began']); + // The new node drives the same recognizer. + const b = getByTestId('b'); + touch('touchstart', b, {id: 4, x: 0, y: 0}, 100); + touch('touchmove', b, {id: 4, x: 30, y: 0}, 116); + expect(states).toEqual(['began', 'began']); + touch('touchend', b, {id: 4, x: 30, y: 0}, 132, true); + }); + } +}); + +describe('usePressTracker keeps its press through an option change', () => { + for (const strict of [false, true]) { + it(`a press in progress survives a rerender with new options${strict ? ' (StrictMode)' : ''}`, () => { + const events: ControlEvent[] = []; + const highlights: boolean[] = []; + function Held({cancelTravel}: {cancelTravel?: number}) { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => events.push(event), + onHighlightChange: isHighlighted => highlights.push(isHighlighted), + cancelTravel, + }); + return ( + + ); + } + const wrap = (node: React.ReactElement) => + strict ? {node} : node; + const {getByTestId, rerender} = render(wrap()); + const button = getByTestId('held'); + touch('touchstart', button, {id: 6, x: 1, y: 1}, 0); + expect(highlights).toEqual([true]); + rerender(wrap()); + // Still pressed: the tracker was not rebuilt or disposed. + expect(highlights).toEqual([true]); + touch('touchend', button, {id: 6, x: 1, y: 1}, 50, true); + expect(events).toEqual(['touchDown', 'touchUpInside']); + }); + } +}); + +describe('the hooks let go of an element removed while they stay mounted', () => { + it('useGestureRecognizer cancels its gesture without calling back, and works on a new element', () => { + const states: string[] = []; + function Maybe({shown}: {shown: boolean}) { + const ref = useRef(null); + useGestureRecognizer( + ref, + () => new PanGestureRecognizer({direction: 'horizontal'}), + r => states.push(r.state), + ); + return shown ?
: null; + } + const {getByTestId, rerender} = render(); + const first = getByTestId('maybe'); + touch('touchstart', first, {id: 11, x: 0, y: 0}, 0); + touch('touchmove', first, {id: 11, x: 30, y: 0}, 16); + expect(states).toEqual(['began']); + rerender(); + expect(states).toEqual(['began']); + rerender(); + const second = getByTestId('maybe'); + touch('touchstart', second, {id: 12, x: 0, y: 0}, 100); + touch('touchmove', second, {id: 12, x: 30, y: 0}, 116); + expect(states).toEqual(['began', 'began']); + touch('touchend', second, {id: 12, x: 30, y: 0}, 132, true); + }); + + it('usePressTracker ends its press without calling back, and presses a new element', () => { + const events: ControlEvent[] = []; + const highlights: boolean[] = []; + function Maybe({shown}: {shown: boolean}) { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => events.push(event), + onHighlightChange: isHighlighted => highlights.push(isHighlighted), + }); + return shown ? ( + + ) : null; + } + const {getByTestId, rerender} = render(); + touch('touchstart', getByTestId('maybe'), {id: 13, x: 1, y: 1}, 0); + expect(highlights).toEqual([true]); + rerender(); + expect(highlights).toEqual([true]); + expect(events).toEqual(['touchDown']); + rerender(); + const button = getByTestId('maybe'); + touch('touchstart', button, {id: 14, x: 1, y: 1}, 100); + touch('touchend', button, {id: 14, x: 1, y: 1}, 150, true); + expect(events).toEqual(['touchDown', 'touchDown', 'touchUpInside']); + }); +}); + +describe('usePressTracker reads trackingExpansion as it is now', () => { + it('a smaller expansion takes effect without rebuilding the tracker', () => { + const events: ControlEvent[] = []; + function Tracked({expansion}: {expansion: number}) { + const ref = useRef(null); + usePressTracker(ref, { + onControlEvent: event => events.push(event), + trackingExpansion: expansion, + }); + return ( + + ); + } + const {getByTestId, rerender} = render(); + const button = getByTestId('tracked'); + boundsOf(button, 100, 40); + touch('touchstart', button, {id: 15, x: 10, y: 10}, 0); + // 30 px below the bottom edge: inside a 70 px expansion. + touch('touchmove', button, {id: 15, x: 10, y: 70}, 16); + expect(events).not.toContain('touchDragExit'); + rerender(); + // The same point is outside a 10 px expansion, mid-press. + touch('touchmove', button, {id: 15, x: 10, y: 71}, 32); + expect(events).toContain('touchDragExit'); + touch('touchend', button, {id: 15, x: 10, y: 71}, 48, true); + expect(events).toContain('touchUpOutside'); + }); +}); diff --git a/packages/core/src/gestures/__tests__/watcherRecognizer.test.ts b/packages/core/src/gestures/__tests__/watcherRecognizer.test.ts new file mode 100644 index 0000000000000..ef07786786c35 --- /dev/null +++ b/packages/core/src/gestures/__tests__/watcherRecognizer.test.ts @@ -0,0 +1,248 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file watcherRecognizer.test.ts + * @input Uses vitest, the arena harness, GestureHost and the pan recognizer + * @output Checks the `claimsTouches: false` watcher: it never claims or + * blocks a scroll, prevents nothing and is prevented by nothing, and + * follows a touch the browser takes for a scroll to the lift + * @position Test for GestureRecognizerOptions.claimsTouches + */ + +import {afterEach, beforeEach, describe, expect, it, vi} from 'vitest'; +import {GestureHost} from '../GestureHost'; +import {PanGestureRecognizer} from '../PanGestureRecognizer'; +import {TapGestureRecognizer} from '../TapGestureRecognizer'; +import {createHarness, ManualClock, RecordingConsumer} from './harness'; +import {coarsePrimaryPointer} from './pointerQuery'; + +describe('a recognizer that never claims (a watcher)', () => { + let detach: (() => void)[] = []; + // A phone: the host attaches only where the primary pointer is coarse. + let restorePointer: () => void; + beforeEach(() => { + restorePointer = coarsePrimaryPointer(); + }); + afterEach(() => { + detach.forEach(release => release()); + detach = []; + document.body.innerHTML = ''; + restorePointer(); + }); + + it('runs alongside a claiming pan on the same touch, and claims nothing', () => { + const h = createHarness(); + const watcher = new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }); + const sheet = new PanGestureRecognizer({ + name: 'sheet', + direction: 'vertical', + }); + const finger = h + .finger(1) + .down(0, 100, 0, {recognizers: [sheet, watcher]}) + .move(0, 115, 16); + expect(watcher.state).toBe('began'); + expect(sheet.state).toBe('began'); + expect(h.claims).toEqual([1]); + finger.up(32); + }); + + it('cancels no press under it when it begins: a held control keeps its touch (spec:AST-094 FR15, FR19)', () => { + const h = createHarness(); + const watcher = new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }); + const press = new RecordingConsumer(); + const finger = h + .finger(1) + .down(0, 100, 0, {recognizers: [watcher], consumer: press}) + .move(0, 130, 16); + expect(watcher.state).toBe('began'); + expect(press.events).not.toContain('cancelled:1'); + finger.up(32); + expect(press.events).toContain('ended:1'); + }); + + it('follows a touch the browser took for a scroll until the lift', () => { + const h = createHarness(); + const watcher = new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }); + const finger = h + .finger(1) + .down(0, 100, 0, {recognizers: [watcher]}) + .move(0, 115, 16); + expect(watcher.state).toBe('began'); + finger.takenByBrowser(20); + finger.move(0, 160, 32); + expect(watcher.state).toBe('changed'); + finger.up(48); + expect(h.log).toContain('watcher:ended@48'); + }); + + it('gets only passive touchmove listeners from the host', () => { + document.body.innerHTML = '
'; + const page = document.getElementById('page') as HTMLElement; + const host = new GestureHost(document, {clock: new ManualClock()}); + const added = vi.spyOn(page, 'addEventListener'); + detach.push( + host.attach( + page, + new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }), + {manageStyle: false}, + ), + ); + const event = new Event('touchstart', {bubbles: true, cancelable: true}); + const entry = {identifier: 1, clientX: 10, clientY: 10, target: page}; + Object.defineProperty(event, 'changedTouches', {value: [entry]}); + Object.defineProperty(event, 'touches', {value: [entry]}); + page.dispatchEvent(event); + const blocking = added.mock.calls.filter( + ([type, , options]) => + type === 'touchmove' && + (options as AddEventListenerOptions | undefined)?.passive === false, + ); + expect(blocking).toHaveLength(0); + }); + + it('gets only passive target listeners for its touch, touchend included, while a tap on the same kind of touch gets a cancelable touchend (spec:AST-094 FR20)', () => { + document.body.innerHTML = '
'; + const page = document.getElementById('page') as HTMLElement; + const button = document.getElementById('button') as HTMLElement; + const host = new GestureHost(document, {clock: new ManualClock()}); + const onPage = vi.spyOn(page, 'addEventListener'); + const onButton = vi.spyOn(button, 'addEventListener'); + detach.push( + host.attach( + page, + new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }), + {manageStyle: false}, + ), + host.attach(button, new TapGestureRecognizer({name: 'tap'})), + ); + const touchDown = (target: HTMLElement, identifier: number) => { + const event = new Event('touchstart', {bubbles: true, cancelable: true}); + const entry = {identifier, clientX: 10, clientY: 10, target}; + Object.defineProperty(event, 'changedTouches', {value: [entry]}); + Object.defineProperty(event, 'touches', {value: [entry]}); + target.dispatchEvent(event); + }; + const passivity = (spy: typeof onPage, type: string) => + spy.mock.calls + .filter(([listened]) => listened === type) + .map( + ([, , options]) => + (options as AddEventListenerOptions | undefined)?.passive, + ); + touchDown(page, 1); + expect(passivity(onPage, 'touchmove')).toEqual([true]); + expect(passivity(onPage, 'touchend')).toEqual([true]); + expect(passivity(onPage, 'touchcancel')).toEqual([true]); + touchDown(button, 2); + expect(passivity(onButton, 'touchend')).toEqual([false]); + }); + + it('alone, leaves the host only passive document and window listeners; a claiming recognizer brings the cancelable touchend, contextmenu and click (spec:AST-094 FR20)', () => { + document.body.innerHTML = '
'; + const page = document.getElementById('page') as HTMLElement; + const strip = document.getElementById('strip') as HTMLElement; + const host = new GestureHost(document, {clock: new ManualClock()}); + const active = () => { + const listed: string[] = []; + for (const [target, spy] of [ + ['document', onDocument], + ['window', onWindow], + ] as const) { + for (const [type, , options] of spy.mock.calls) { + if ( + typeof options === 'object' && + (options).passive === false + ) { + listed.push(`${target} ${type}`); + } + } + } + return listed; + }; + const onDocument = vi.spyOn(document, 'addEventListener'); + const onWindow = vi.spyOn(window, 'addEventListener'); + detach.push( + host.attach( + page, + new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }), + {manageStyle: false}, + ), + ); + expect(active()).toEqual([]); + detach.push( + host.attach( + strip, + new PanGestureRecognizer({name: 'claimer', direction: 'horizontal'}), + ), + ); + expect(active().sort()).toEqual([ + 'document contextmenu', + 'document touchend', + 'window click', + ]); + }); + + it('leaves the browser every touch behavior: its direction writes no touch-action', () => { + document.body.innerHTML = '
'; + const list = document.getElementById('list') as HTMLElement; + const row = document.getElementById('row') as HTMLElement; + const host = new GestureHost(document, {clock: new ManualClock()}); + const listStyle = vi.spyOn(list.style, 'setProperty'); + detach.push( + host.attach( + list, + new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }), + ), + host.attach( + row, + new PanGestureRecognizer({ + name: 'watcher', + direction: 'vertical', + claimsTouches: false, + }), + ), + // A claiming pan beside it on the same element keeps its own. + host.attach( + row, + new PanGestureRecognizer({name: 'swipe', direction: 'horizontal'}), + ), + ); + expect(list.hasAttribute('data-astryx-touch-action')).toBe(false); + expect(row.getAttribute('data-astryx-touch-action')).toBe( + 'pan-y pinch-zoom', + ); + expect(listStyle).not.toHaveBeenCalledWith( + 'touch-action', + expect.anything(), + ); + }); +}); diff --git a/packages/core/src/gestures/hapticSwitch.ts b/packages/core/src/gestures/hapticSwitch.ts new file mode 100644 index 0000000000000..aa9a669d43c9d --- /dev/null +++ b/packages/core/src/gestures/hapticSwitch.ts @@ -0,0 +1,14 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file hapticSwitch.ts + * @input Nothing + * @output Exports HAPTIC_SWITCH_ATTRIBUTE + * @position Shared by haptics.ts, which marks its hidden switch with it, and + * the gesture host, whose click swallower skips that switch, so the host + * never imports the haptics module (a component that plays no haptic pays + * nothing for it) + */ + +/** Marks the hidden switch, so click handlers can recognize and ignore it. */ +export const HAPTIC_SWITCH_ATTRIBUTE = 'data-astryx-haptic-switch'; diff --git a/packages/core/src/gestures/haptics.ts b/packages/core/src/gestures/haptics.ts new file mode 100644 index 0000000000000..323a6dbcf9f36 --- /dev/null +++ b/packages/core/src/gestures/haptics.ts @@ -0,0 +1,212 @@ +// Copyright (c) Meta Platforms, Inc. and affiliates. + +/** + * @file haptics.ts + * @input A feedback kind, or a label element a real finger will tap + * @output Exports playHaptic (vibration where the platform has it), + * attachTapHaptic (iOS Safari's tap-moment light impact) and the support + * probes + * @position Haptics of packages/core/src/gestures; the kinds and the + * vibration patterns live in the constants table (motion/uikit/gestures.ts) + * + * The web has one haptics API, `navigator.vibrate`, and iOS Safari does not + * implement it. iOS Safari does play a haptic in exactly one place: when a + * REAL tap toggles an ``, by landing on the + * switch or on a `