fix(desktop): pin fill_cloud_connector_form to Omi's own MCP endpoint and ChatGPT's connector page

[sauranshbhardwaj]

What changed and why

fill_cloud_connector_form fills the "add a custom connector" form in Claude or ChatGPT so the person can connect Omi's own MCP server. It runs without an approval card, as a documented exception in the approval work for #20938. That exception says the tool only fills a connector form that is already open. While reviewing #20975, the maintainers' reviewer noted that the tool "still takes an arbitrary server_url and can press submit".

That was a fair reading. The URL wasn't pinned, and the ChatGPT path didn't confirm the page, so the exception comment overstated the bound. It held only for the Claude path. This PR tightens the tool so the stated bound holds on both paths. It does not add an approval card.

1. Only Omi's own endpoints

The tool now checks its arguments before any UI step. Nothing is focused, typed or clicked when it refuses.

• server_url must be this build's Omi MCP endpoint, MemoryExportDestination.mcpServerURL. That is the same value the "Connect your AI" setup passes, built from the backend environment for the current app (for example https://api.omi.me/v1/mcp on the stable app). Anything else is refused with server_url_not_omi.
• auth_url and token_url, when given, must be Omi's OAuth endpoints, MemoryExportDestination.mcpAuthorizeURL and mcpTokenURL. Anything else is refused with auth_url_not_omi or token_url_not_omi.
• The match is exact. Both sides must be https, with the same host and port and the same path, and no user info, query or fragment. There is no prefix or substring matching. A trailing slash, an extra path segment, a look-alike host and plain http are all refused.
• The form always gets Omi's own spelling of the URL, never the caller's. Case or a written-out :443 cannot change what is typed.
• A refusal reads Error: <code>: this tool only enters Omi's own endpoints, so <field> must be <url>. Nothing was typed.

A development build whose backend URL is plain http accepts no server_url. That build was never a target for the hosted connectors.

2. ChatGPT: only ChatGPT's connector form

The ChatGPT path now finds the form the way the Claude path does:

• It looks only at supported browsers (the same list the Claude path uses). It no longer scans other apps.
• The window's text must show a chatgpt.com connector settings URL (chatgpt.com/#settings/Connectors… or #settings/Apps…, with or without https://). The host must be exactly chatgpt.com. A look-alike host, a path that only mentions chatgpt.com, plain http, and other ChatGPT pages are refused.
• The form must be inside a dialog in that window that holds the server URL field. Fields are filled only inside that dialog. The old fallback could reach any text field in the window.
• When a field refuses a direct value, the paste fallback runs only while that browser is frontmost and that field has focus.
• Otherwise the tool refuses with chatgpt_connector_form_not_verified and touches nothing.

3. Submit stays opt-in and bound to the form

submit is still off unless the caller asks for it. On ChatGPT the tool re-checks the page and the form before pressing anything. It then presses only the one button inside that dialog whose whole label is the form's own Add or Create label (Add, Create, Create app, and the like). Before, it pressed the first button anywhere in the window whose label merely contained "add", "connect", "create" or "save". If no button or more than one button matches, nothing is pressed, and the result says "Submit skipped".

The Claude path is unchanged apart from receiving the pinned URL.

The argument checks and the pure page and form checks are in a new CloudConnectorFormRequest.swift, an extension of CloudConnectorFormAutomation, so the automation file gets shorter rather than longer.

The exception comment

The comment for this exception in run-tool-capability.test.ts is part of the approval stack (#20973 / #20975) and is not on main, so this PR cannot edit it. Once this lands, that comment should say exactly what is now enforced:

// Scope: not one of the on-device tools issue #20938 gates. Safeguard: it
// enters only Omi's own MCP and OAuth endpoints (exact https match, refused
// before any UI step otherwise), and only into Claude's or ChatGPT's connector
// form already open in a supported browser on claude.ai or chatgpt.com. Risk
// it keeps: with submit it presses that form's own Add/Create button (and
// Claude's Connect for the Omi connector).
fill_cloud_connector_form: "outside #20938's device tools; Omi's endpoints only, into the provider's open connector form",

Product invariants affected

• INV-INT-1 (product/invariants/integrations.md) is named because CloudConnectorFormAutomation.swift matches **/*CloudConnector*. This change only narrows what the existing automation may do. It adds refusals: the endpoint pin, the chatgpt.com page check, the dialog scope and the exact submit label. It adds no new way to find or click an element, and no new OCR. The ChatGPT page check compiles in ChatGPT's connector settings route, just as the Claude path already compiles in claude.ai/customize/connectors. If ChatGPT moves that page, the tool refuses rather than filling an unverified page.

How it was verified

• xcrun swift build --build-tests in desktop/macos/Desktop: passes.
• Focused tests: xcrun swift test --filter 'CloudConnectorFormEndpointPinTests|BrowserAutomationTargetTests|SpatialOverlayRenderGeometryTests|SpatialOverlayDogfoodHarnessTests' ran 60 tests with 0 failures. That is the 11 new tests plus the existing Claude connector, setup-result and overlay tests.
• swift-format lint (pinned wrapper) is clean on the three changed Swift files.
• The agent and the tool manifest are unchanged. generate-tool-surfaces.mjs --check reports "all outputs match".
• Whole Swift suite (xcrun swift test --skip SubscriptionEntitlementServiceTests in desktop/macos/Desktop): 7889 tests, 10 skipped, 2 failures. Both failures are known and unrelated to this change: DesktopAutomationWindowPresentationTests.testQuietPresentationAlsoParksNonactivatingOverlayPanels (display geometry) and RecentScreensMenuContentTests.testDayOldFrameShowsACalendarDateNotAClockTime (an Indian locale date format).
• The pre-push gate, run locally against upstream with this body: PRE_PUSH_BASE_REMOTE=upstream OMI_PR_BODY_FILE=<this body> scripts/pre-push-singleflight upstream. It reports "PR preflight passed: 30 checks" and "Bounded pre-push gate passed".

Packaged-app check (to run by hand). Use a packaged build of this branch with Accessibility granted.

1. Claude: in Omi, open "Connect your AI", choose Claude and click "Do it for me". Expect Claude's add custom connector form to be filled with Omi's URL and client ID. Click Add (or let Omi press it), then Connect, then approve. Expect Claude to show Omi as connected.
2. ChatGPT (if you use it): open ChatGPT's settings in Chrome, Atlas or another supported browser, go to Apps / Connectors (developer mode), and open the create app form. Ask Omi to finish connecting Omi to ChatGPT. Expect the form to be filled with Omi's MCP URL, OAuth client ID, Auth URL and Token URL, and the form's own Create button to be pressed. Nothing outside that form should be typed into or clicked.
3. Optional: with a ChatGPT chat open (not the settings form), ask Omi to fill the ChatGPT connector form. Expect chatgpt_connector_form_not_verified and nothing typed.

Tests

CloudConnectorFormEndpointPinTests (new, 11 tests) runs the tool with a spy in place of its UI steps:

• Non-Omi server_url is refused before any UI step, for both providers: another host, http://, look-alike hosts (omi.me.evil.com, api.omi.me.evil.com), another host with Omi in the path (evil.com/omi.me, evil.com/api.omi.me/v1/mcp), extra path (/v1/mcp/extra, /v1/mcp/sse, a trailing slash), a shorter path, a query, a fragment, user info, another port, a trailing-dot host, no scheme, ftp, and text that is not a URL. Each returns server_url_not_omi, and the UI steps never run.
• Omi's URL is accepted and the pinned spelling is what gets typed, including with surrounding whitespace, upper case, or :443.
• This build's own endpoints are accepted, using the same arguments the ChatGPT setup prompt passes. The pinned endpoints equal MemoryExportDestination's.
• OAuth URLs: a non-Omi auth_url or token_url (another host, http, a look-alike host, extra path, a query, or the two swapped) is refused before any UI step with its own code. Omi's are accepted and pinned.
• An http backend accepts nothing.
• Earlier argument errors are unchanged (unknown provider, empty server_url).
• Claude is unchanged: the same provider, submit flag, fields and values reach the UI steps.
• ChatGPT page: chatgpt.com/#settings/Connectors with or without https://, its Advanced sub-route, and #settings/Apps are accepted. These are refused: another site, Claude's page, chatgpt.com.evil.com, evil.com/chatgpt.com/…, a query that mentions it, notchatgpt.com, http://chatgpt.com, a ChatGPT chat, ChatGPT's general settings, and a window title with no URL.
• ChatGPT form: a dialog needs a server URL field to count as the connector form.
• ChatGPT submit: only an exact Create, Create app or Add counts. These never count: "Add files", "Connect Google Drive", "Save memory", "Created by you", "Address", a bare "Connect" or "Save", two matches, or no buttons.

The existing Claude tests still pass: page-state classification, the state machine, the OCR candidates, the guidance overlays, and the setup result checks.

UI evidence

UI-Evidence: none -- tool argument and page checks only; no UI change

Failure class (fixes)

Failure-Class: none

[Git-on-my-level]

Reviewed the full diff. This is a genuine security tightening of fill_cloud_connector_form, and the implementation holds up:

• CloudConnectorFormRequest.swift: prepareFill now refuses server_url/auth_url/token_url unless they match this build's MemoryExportDestination.mcp*URL, with an exact-match check in isSameHTTPSEndpoint — scheme must be https on both sides, host compared lowercased, port normalized to a 443 default, percentEncodedPath equality, and user info / query / fragment all rejected. No prefix or substring matching, and the form always receives the pinned spelling rather than the caller's. The refusal codes (server_url_not_omi etc.) fire before any UI step, which the spy-injected tests verify directly (uiRequests == []).
• CloudConnectorFormAutomation.swift: the old findBestForm fallback that could scan any regular app is gone; ChatGPT now requires a supported-browser window whose text shows a chatgpt.com/#settings/Connectors-or-Apps URL, and filling happens only inside an AXApplicationDialog that contains a server-URL field (verifiedChatGPTConnectorDialog). Submit re-verifies the dialog and presses only a single exact-label Add/Create button (chatGPTSubmitButtonIndex returns nil on zero or two matches — good). The setField paste fallback is now correctly guarded so Cmd+V only fires while the verified browser is frontmost and the target field holds focus. The Claude path is preserved via advanceClaudeConnectorStateMachine.
• CloudConnectorFormEndpointPinTests.swift: good adversarial coverage — look-alike hosts, evil.com/api.omi.me/... paths, trailing dots, user-info, queries, fragments, non-default ports, swapped OAuth URLs, and loose button labels like "Add files" / "Connect Google Drive" correctly not matching.
• Changelog fragment and the e2e covers entry are in place.

Two things need a human maintainer before merge — both are judgment calls, not defects:

1. This tool still runs without an approval card (the documented Desktop: the new on-device tools (iMessage, Mail, AppleScript) can't run for users — add real per-action approvals #20938 exception), types OAuth client IDs/secrets into a browser form, and can press submit. This PR narrows that exception's blast radius substantially (endpoint pin + page verification + dialog-scoped submit), but whether the tightened bound is now acceptable for a no-approval-card tool is a security sign-off I don't want to make unilaterally.
2. INV-INT-1 is locked and says MUST NOT "add another hardcoded element-anchoring heuristic to CloudConnectorFormAutomation" and "compile volatile provider knowledge into the Swift binary." The new showsChatGPTConnectorSettingsURL route strings and chatGPTSubmitButtonLabels set are exactly that kind of constant, and both new files match the invariant's **/*CloudConnector* globs (the PR body names INV-INT-1 and argues these are narrowing bounds rather than new heuristics — a defensible reading, but it's the invariant owner's call, not mine).

Code quality is high and CI is green at head. Leaving for maintainer review on those two questions rather than blocking on anything in the diff itself.

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

[sauranshbhardwaj]

Thanks. On point 2: you're right that the ChatGPT route strings and the Add/Create label set are provider knowledge in the binary, which INV-INT-1 says not to add. I'd rather not ask for an exception to a locked invariant, so here are two ways to keep the fix inside it:

• A (my preference): keep the endpoint pin, which is Omi's own contract rather than provider knowledge, and stop auto-filling ChatGPT. The tool would refuse the ChatGPT path with a clear reason, and the person would paste Omi's MCP URL into ChatGPT themselves. That removes the new constants and the old loose ChatGPT path together, and leaves less automation to maintain.
• B: keep ChatGPT filling, without the new route or label checks, behind an Allow Once card each time, so the person confirms before anything is typed or pressed.

Either way the Claude path and the endpoint pin stay as they are. Tell me which you'd like, or whether the invariant owner is fine with the current narrowing reading, and I'll update the PR. On point 1, agreed that it's a maintainer's call.