Expose cursor position and modifier keys for computer actions

[NikhilShahi]

Summary

Expose desktop cursor position and pointer modifier keys through existing browserctrl functionality. Keep existing coordinate-based click/scroll APIs intact, with separate current-pointer helpers and the standard ComputerActionResponse for every action.

Changes

• Add cursorPosition() returning the standard ComputerActionResponse; the request executor uses the same concrete response type as every other action.
• Add ComputerAction.CURSOR_POSITION and its request/coordinate variants to the existing general action and response unions.
• Include ScrollAtCursorActionParams in ComputerActionParams; the shared request executor accepts only the main request union.
• Preserve required coordinates on click() and scroll() and required scroll deltas/fields; use clickAtCursor() and scrollAtCursor() for coordinate-free actions.
• Append optional modifier keys to click, drag and scroll, preserving existing positional arguments and no-modifier wire payloads.
• Add local HTTP transport and public typing regression coverage, including guarded cursor-coordinate reads, helper defaults, session lookup, screenshots, modifiers and failed responses without coordinate data.
• Minor-bump 1.0.3 → 1.1.0; no backend changes or new dependencies.

Compatibility

Existing click/scroll signatures and coordinate-based ScrollActionParams fields are preserved. The main request union also accepts coordinate-free scroll payloads; generic scroll-request consumers must check for coordinates before reading them. The general action enum, request union and response-data union gain cursor position, following prior SDK expansion practice: strict exhaustive switches/maps and generic response consumers may need to handle the new variant. All actions return ComputerActionResponse; callers check data && "x" in data before reading cursor coordinates. No dedicated cursor-response type or generic request-executor return type is introduced.

Validation

• yarn lint, yarn typecheck, yarn test, yarn build: passed
• Changed-file prettier --check; git diff --check: passed
• Compiler fixtures verify ScrollAtCursorActionParams is assignable to ComputerActionParams.
• Runtime and compiler fixtures narrow shared data before reading coordinates; type assertions verify cursor methods return ComputerActionResponse and preserve required click/scroll coordinates and scroll deltas.
• Earlier disposable live session verified cursor lookup, screenshots, modifiers and current-pointer actions; live checks were not rerun for this API refinement.

---

Note

Medium Risk
Expands the public computer-action API and request/response unions; callers with exhaustive switches or assumptions about scroll coordinates may need small updates, but existing positional arguments stay compatible.

Overview
Bumps the package to 1.1.0 and extends computer action APIs for desktop pointer control without changing coordinate-based click() / scroll() signatures.

Adds cursorPosition() and types for CURSOR_POSITION, with cursor coordinates returned on the standard ComputerActionResponse.data (x / y). Adds clickAtCursor() and scrollAtCursor() for actions at the current pointer (no x/y on the wire), plus optional modifier keys on click, drag, and scroll (including the new helpers).

Public exports and ComputerActionParams unions gain cursor and scroll-at-cursor variants. New integration tests cover HTTP payloads, session lookup, modifiers, and failures; unit tests lock in the exported type contracts.

^{Reviewed by Cursor Bugbot for commit 53897b1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d2d7df8. Configure here.}