Control your Mac with camera-based hand gestures. AirPointer uses Apple's Vision framework to track hands locally, then translates gestures into pointer movement, clicks and shortcuts.
This development snapshot is V4 Trackpad (0.4). Gesture accuracy depends on lighting, framing and hand visibility. Synthetic tests cover the command pipeline; they do not establish real-world tracking accuracy.
Licensed under the MIT License. Copyright © 2025–2026 Bruno Schreiber. The original app icon is included under the same license; see dependencies and artwork.
- macOS 26 or later.
- Full Xcode 26 or later with Swift 6.2 or later (Command Line Tools alone are insufficient for the app and its
.iconresource). This snapshot was validated with Xcode 27. - A camera for live gesture tracking; Camera and Accessibility permissions for the corresponding features.
- No external packages, API keys, cloud account or paid Apple Developer membership are required for the local ad hoc build.
Open AirPointer.xcodeproj and select the AirPointer scheme, or run:
./script/build_and_run.sh --build-onlyThe script uses one build job and ad hoc signing. The app is written to DerivedData/Build/Products/Debug/AirPointer.app. Override that location with AIRPOINTER_DERIVED_DATA. If Xcode is installed under a different name or location, set DEVELOPER_DIR to its Contents/Developer directory.
To build and launch:
./script/build_and_run.shThe launch modes quit any existing AirPointer process first. --build-only neither quits nor opens the app. Optional launch modes: --debug, --logs, --telemetry, --verify (process-presence check only). These are development builds, not notarized releases. For distribution, configure your own signing identity and notarization; no developer team or signing credentials are included.
- Open the app and grant Camera access when you want to start tracking. Grant Accessibility access to allow pointer and keyboard actions.
- Start with Diagnostic only — no commands in the gesture menu. It displays recognition states and click counters without sending mouse, keyboard or scroll events.
- Keep your hands visible and verify that the displayed left/right sides match your physical hands. Disable diagnostic mode when ready to try commands in a safe area of the desktop.
| Left-hand gesture | Action |
|---|---|
| Close/relax all four fingers, then move the wrist | Relative pointer movement |
| Open the palm | Pause and reposition; cancel a pending click |
| Raise only the index finger, then close the fingers | Left click |
| Raise index and middle fingers, then close the fingers | Right click |
| Pinch with the other fingers open in Deepwork | Continuous scroll |
The pointer stays still while a click forms. Hold a click pose for about 0.20 seconds, then close for about 0.12 seconds. Sensitivity ranges from 0.5× to 3×; the default is 1.2× and applies to the current session.
In Presentation mode, right-hand double pinch advances, and an open-hand-to-held-fist sequence goes back. The right pinky gesture invokes the configured mute shortcut. Shortcut behavior depends on the selected profile and foreground application; see IntentDispatcher.swift for mappings.
Package.swift is a library-only test harness. It excludes the app entry point and UI and uses synthetic hand fixtures and injected action/permission doubles:
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
swift test --jobs 1 --disable-index-storeThe tests cover relative pointer movement, gesture transitions, clicks, profiles, permissions, recording serialization and command dispatch. They do not open the camera or post real input events. The previous replay test tied to a private recording is not part of this snapshot.
Xcode UI tests launch the app and may show permission prompts; they are separate from the offline command above. Live camera and physical gesture verification remain manual.
Camera frames are processed locally with Vision. No network service or analytics SDK is configured in this source snapshot.
Record frames is a separate, opt-in diagnostic feature: it saves hand joint coordinates, handedness, timing, a creation date and a profile label as .airframes.json, not camera images. Files are stored under AirPointer/Recordings inside the app's Application Support directory (inside its container for sandboxed builds). Treat these recordings as personal data and do not attach them to public issues. Diagnostic-only mode disables recording and discards an active recording when enabled.
The repository excludes recordings, local IDE state, signing material, build products and private diagnostic evidence. See contributing before sharing a bug report.
AirPointer/: SwiftUI app, camera tracking, gesture engines and command dispatch.AirPointer/ProductionSupport/: permission handling.AirPointerTests/: synthetic tests used by SwiftPM and Xcode.AirPointerUITests/: Xcode UI tests.script/build_and_run.sh: local build and launch entry point.