Skip to content

feat(desktop): add propose_message and propose_email draft-and-you-send tools - #21166

Open
sauranshbhardwaj wants to merge 4 commits into
BasedHardware:mainfrom
sauranshbhardwaj:feat/mac-propose-compose
Open

sauranshbhardwaj wants to merge 4 commits into
BasedHardware:mainfrom
sauranshbhardwaj:feat/mac-propose-compose

Conversation

@sauranshbhardwaj

@sauranshbhardwaj sauranshbhardwaj commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

Today "email Sam the deck" on the Mac has no proper path. list_mail_messages reads headers only, send_message covers iMessage only and needs an approval card, and the fallback is run_applescript. iOS already has propose_message: it opens the system compose sheet, filled in, and the person taps Send.

This PR adds two Mac tools in that spirit:

  • propose_message places a draft in the Messages app with the recipients and text filled in.
  • propose_email opens a new email draft in the Mac's default email app with the recipients, subject and body filled in.

The person reviews the draft and presses Send in that window. Omi never sends: the tools use NSSharingService (.composeMessage, .composeEmail) and stop there. There is no AppleScript, no Mail or Messages automation, and no TCC prompt. Nothing is sent until the person presses Send, though Mail may autosave an open draft to the account's Drafts mailbox on the mail server.

Which app. .composeEmail hands the draft to whatever app handles mailto: links. That may be Mail, another mail app such as Outlook, or a browser, which opens its web mail site. The packaged-app test confirmed this: the tester's default was Google Chrome, and the draft opened in Outlook on the web.

  • Results name the app, resolved with NSWorkspace.urlForApplication(toOpen: mailto:), in app and app_is_browser.
  • When the app is a browser, the result says the draft opened on its web mail site and that Omi cannot see whether it was sent.
  • Nothing in the results, guidance or docs assumes Mail.

Routing. In the same live test, "text me at '…'" made the model file a create_desktop_dispatch approval for send_message. On main that dispatch has no card to render it, so the request dead-ended. The guidance is now explicit:

  • propose_message is the default for any request to text, message or iMessage someone;
  • send_message is only for when the person explicitly asks Omi to send it for them;
  • create_desktop_dispatch, send_message and propose_* all say a message or email is never pre-filed as a dispatch.

send_message's behaviour and the dispatch tool itself are unchanged.

This is piece 1 of the follow-through proposal in #21162. That issue is titled "…behind the approval card" and has no maintainer reply yet. This piece deliberately has no Omi approval card (see Policy below), and that is a decision for maintainers to confirm.

The PR goes straight to main. It does not depend on the approval-card stack (#20973 → #20975 → #20997 → #20998).

What is shared with iOS, and what differs

  • Shared:
    • the tool name propose_message;
    • to (an array) and text;
    • the "this does NOT send" contract.
  • Different on the Mac:
    • the Mac never reports sent or cancelled in practice. iOS gets sent or cancelled from its compose sheet. On the Mac (verified on macOS 27), .composeMessage shows no sheet: it places the draft in the Messages app's own composer and reports the handoff within about half a second. Email apps do the same. So Omi cannot see whether the person sends or discards the draft;
    • the Mac returns as soon as the draft is handed off, instead of holding the turn until the person acts;
    • Mac propose_message has no subject;
    • the Mac adds propose_email.

Results

Every result carries sent_by_omi: false and a status:

  • handed_to_email_app: the email app named in app took the draft. macOS reports this before the person presses Send, so it never means sent.
  • handed_to_messages: the draft is in Messages for the person to review and send. Whether they send or discard it cannot be observed.
  • cancelled and open (still on screen after the wait) are produced only if the system shows a sheet or an email app reports a cancel. The code still handles a sheet, but current macOS does not show one. open is also the answer when the turn is stopped after the handoff.
  • not_opened: no window was opened. The reasons are:
    • stopped: the call was stopped before the window opened;
    • person_typing;
    • draft_already_open: the same draft was handed off in the last two minutes;
    • (from send_message, not a propose tool) draft_in_messages: a send to someone whose draft is waiting in Messages; nothing is sent;
    • compose_already_open (only with a sheet);
    • draft_limit_reached;
    • compose_unavailable: canPerform false or "not configured";
    • a validation reason, listed below.

compose_failed is used for any other error from the share service and reports only the error code.

Recipients come from the person, not from what the model read

The compose window has no "Omi wants to…" framing. So the recipient rule from send_message ("never send to a handle the user did not name or confirm") is enforced in code, not only in the prompt.

Every recipient must come from one of:

  • words the person typed, pasted or dictated in this turn. This is the run's originatingUserText with the clock preamble stripped, as isMemoryContentGroundedInUserRequest does. For push-to-talk, the run text is the provider's transcript of what the person said, so dictation needs no separate path;
  • the person's one previous turn in the same main chat, so that "draft it to sam@x.com" followed by "yes, that one" works;
  • a search_contacts result from the same run and owner;
  • the signed-in account's own address.

If a recipient fails that check, the tool returns recipient_needs_confirmation, lists the address, and opens nothing. This covers an address the model read on a page, in an email or on screen.

Limits of the check:

  • Question-card answers never count. On a question card the person clicks a label, but the turn's text is the model-written preparedAnswer. Those turns carry a qri_ turn key in the chat transcript and are left out, both as the current turn and as the previous one. The run text is matched to its chat turn by content; when nothing matches (task chat, voice), the run text stands alone.
  • Pasted text counts as the person's. If they paste a thread and ask for a reply, its addresses are grounded. That is useful, but it means a pasted page can bring an address in, and the person still sees it in the To field.
  • Spoken addresses don't match. "sam at example dot com" is not an address, so voice users reach people through Contacts.
  • Phone numbers match as whole numbers. The digits must be equal, or one side may add a one-to-three-digit country code to a national number of ten or more digits. A date, the digits inside an address, or a number after "order", "invoice", "ticket", "#" and similar words never grounds a phone number.

Validation:

  • a bare name returns recipient_needs_lookup with next_tool: search_contacts;
  • addresses use a narrow ASCII form, which rejects ?, &, %, /, #, .. and lookalike Unicode domains, while punycode is allowed;
  • phone numbers allow 3 to 15 digits;
  • recipients are deduplicated, up to 20 per draft.

No handoff while the person is typing (load-bearing)

Return sends from the Messages composer. The live test confirmed it. A keystroke that lands in the draft can send it before the person has reviewed it, so this guard is a safety control, not a nicety:

  • a draft is handed off only after 3 s with no key press, measured as the system-wide keyDown idle through UserInputPresence;
  • the last idle check runs with no suspension before the handoff;
  • the guard waits at most 8 s, and otherwise returns not_opened / person_typing.

Residual risk: someone who starts typing again just after the handoff can still type into the composer.

"Send it" after a draft

In the live test, the person replied "send" after a draft. The model called propose_message again, which put three copies in Messages. Two fixes:

  • Guidance. propose_message, propose_email and send_message now say that after a draft the person presses Send in Messages or their email app. On "send" or "send it", the model tells them to do that. It never opens the same draft again and never calls send_message or create_desktop_dispatch for it.
  • A guard in send_message. In the next live run, the person typed "Send" after a draft, and the model called send_message with the same recipient and text. On main the kernel's approval check refused it, so nothing was sent, but the guidance alone did not hold. send_message now checks before any send. If the same owner handed a propose_message draft to Messages in the last 10 minutes, and the send's recipient matches any recipient of that draft, it returns not_sent / draft_in_messages and sends nothing.
    • Matching uses the same normalisation as grounding, so spelling and country-code variants match. The text does not matter.
    • A cancelled draft and an email draft do not count.
    • The check is in Swift. With an approval card in front of send_message (the separate approval stack), the card would still appear first. A kernel-side check would be needed to suppress the card.
  • A Swift guard against duplicate drafts. An identical draft for the same owner within two minutes returns draft_already_open and is not handed off again. Identical means the same channel, the same recipients in any order or spelling, and the same text. Only a SHA-256 digest is kept. A cancel or failure report clears it, so the person can ask for the same draft again.

Limits

All limits are per signed-in owner, and a sign-out releases them:

  • One email draft and one message draft per request (run). "Email Sam the deck and text him" works, but a looping or injected model cannot stack compose windows inside one answer. The email app takes its window as soon as it has the draft, so this limit, not the window, is what bounds email.
  • At most three drafts a minute, or one a minute when there is no run ID.
  • The same draft at most once in two minutes (draft_already_open).
  • Only if the system shows a Messages sheet: one open sheet at a time (compose_already_open). A sheet that never reports stops blocking after 5 minutes.

Policy: no Omi approval card, and why

  • The tools get a new bundle, desktop.compose.propose, separate from desktop.messaging.send. Selecting one never implies the other.

  • It is a write bundle (risk tier medium, privacy local_private), but it is platform-confirmed. Apple's window shows the person the exact recipients and text, and nothing is sent until they press Send. An Omi card in front of it would ask twice for the same effect. iOS takes the same position (requires_user_confirmation: "the compose sheet is the approval").

  • evaluateDesktopToolPolicy returns allow, but narrowly:

    • any other write bundle requested beside it keeps its own approval;
    • externalSend or persistentGrant still forces a dispatch;
    • the tools are not in the broker's enforced approval set.
  • Remaining risk without a card: the window looks like the person's own draft. Three things keep that risk bounded:

    • recipients must come from the person;
    • the limits above;
    • the person must still press Send.
  • The tools are also restricted in three ways:

    • Coordinator only: a background or pill worker can't call them.
    • No ask-mode service runs: these runs have read-only authority.
    • Never retried automatically: the invocations are non-idempotent, so they are never_auto_retry.
  • Model guidance:

    • prefer propose_message unless the person explicitly asked Omi to send;
    • use propose_email for email;
    • never draft to a handle the person did not name or confirm;
    • one draft per channel per request;
    • say who the draft is addressed to.

    send_message, run_applescript and search_contacts now point at the propose tools.

Privacy

The arguments are logged by shape only. The executor's own log line records counts and the outcome. Results echo the recipients, which the model supplied, and never the body.

Not in this PR, and questions

  • CC/BCC. NSSharingService has no CC field. A mailto: path would lose the outcome report and open whichever mail app is the default.
  • Attachments.
  • A checked send. Confirming a handed-off draft in the mail account's Sent mailbox or the Messages history is the next piece of Desktop follow-through: from a captured commitment to a checked send, behind the approval card (#9352) #21162.
  • Pills (question for maintainers). Execute-mode pills are leaf runs. They can call send_message (with its card) but not the safer propose_*. Should pills get the propose tools? That would need a decision on raising windows from background work.
  • A pre-existing problem in this area. ProactiveTaskExecute.systemPromptSuffix still tells the agent to drive Messages and Mail through osascript.

Product invariants

  • INV-AGENT-* (Authority). The changes are in desktop/macos/agent/src/runtime/**:

    • the new desktop.compose.propose bundle and its platform-confirmed classification in desktop-tool-policy.ts;
    • the bundle in the evaluate_desktop_tool_policy enums;
    • two manifest entries.

    Run/attempt capability checks are unchanged. propose_* go through the same broker authorization and ledger preparation as every Swift tool, and the executor checks the owner before and after the window. run-tool-capability.test.ts covers the new authorization path.

  • INV-CHAT-1. Nothing changes in the transcript. Proposal results are ordinary tool results in the kernel's invocation ledger. The only Swift state is in memory: the per-run contacts handles, the draft digests, the open-sheet slot and the limits. All of it is owner-scoped and released on sign-out, so it is not a second store.

Testing

  • xcrun swift build -c debug --package-path Desktop: passes.
  • ComposeProposalCoordinatorTests (20), ComposeProposalTests (19), ComposeRecipientGroundingTests (12), ComposeEmailAppTests (9), ComposeDuplicateDraftTests (4) and ComposeSendGuardTests (6): 70 tests, all passing. They use a fake ComposeSurface, a scripted keyboard, an injected sleeper and an injected clock, so no window opens and no test waits on the wall clock. They cover:
    • Results: email handoff; a report arriving mid-wait; no outcome ever sent; every result having a status.
    • Limits: one draft per channel per request even after the email app took the first, one email plus one message allowed in a request; three a minute; one a minute with no run; an open sheet blocking the next request; a late report freeing the slot; the 5-minute stale release; a new owner and sign-out not inheriting the slot.
    • Failure paths: unavailable and refused presentation; the not-configured and failure mappings.
    • Stopping: a call stopped before the window opens opens nothing, at the coordinator and the executor; stopping after the window opens returns open.
    • Typing guard: waiting for quiet; 2.5 s of quiet still counting as typing (3 s threshold); the handoff following the last idle check directly; continued typing opening nothing and leaving the request's draft unused.
    • send_message after a draft: executeSendMessage returns draft_in_messages before reaching Messages; any text to the same person in another spelling is refused; the normal path applies again after 10 minutes and for a different recipient or owner; an email draft or a cancelled draft does not block; every recipient of a group draft is covered.
    • Duplicates: the same draft refused within two minutes (including whitespace-only differences), then allowed again after; recipients matched in any order and spelling; an edited draft allowed; a cancelled draft can be asked for again; another owner is not a duplicate.
    • Grounding:
      • addresses the person typed (with the clock preamble present);
      • an injected page address refused, at grounding level and end to end through the executor, with nothing presented;
      • a question-card answer not counted, as the current turn or as the previous one, including end to end;
      • the previous turn counting, and only one turn back, including a confirmation end to end;
      • task chat and voice text standing alone;
      • pasted text counting;
      • contacts handles scoped to their run and owner, and released;
      • the account's own address;
      • whole-number phone matching (country code added or dropped, short codes exact, no partial or seven-digit tail);
      • dates, order, invoice and ticket numbers, and digits inside addresses never grounding a phone number.
    • Validation: ?cc=, %40, &bcc=, /, #, Cyrillic and fullwidth lookalike domains, .., a leading dot, a hyphen-led label and a one-letter TLD are all rejected; ordinary and punycode addresses are accepted; plus the earlier parsing cases.
    • Logging: the argument log never contains the address, subject or body.
  • ComposeEmailAppTests (9): the Messages handoff result says the person presses Send in Messages and Omi cannot see the send, with no sheet claimed; a duplicate result points at the app and forbids send_message and create_desktop_dispatch. Also, with a fake handler for Mail, Microsoft Outlook and Google Chrome, plus an unknown handler:
    • results name the app and never assume Mail;
    • a browser result says "web mail site" and "can't see whether it was sent";
    • open, failed and unavailable results name the app;
    • Messages results say Messages;
    • classification: a known browser, or the app that opens web links, counts as a browser, and Outlook does not;
    • the executor passes the resolved handler through.
  • scripts/test-tool-surfaces.sh: generate-tool-surfaces --check matches, the full agent vitest suite passes (72 files, 909 tests), and pi-mono-extension passes (143). The vitest cases cover:
    • the policy: allow without dispatch, deny without the bundle, any other write keeps approval, externalSend forces a dispatch;
    • routing: propose_message is the default for "text me at ''"; send_message no longer says "requires an approval dispatch" and says never to file one; create_desktop_dispatch says never to use it for a text, message or email; propose_email never assumes Mail and reports handed_to_email_app. The repo's only routing eval (eval-realtime-routing) covers the realtime voice lane, which does not offer propose_*, so it has no case for this;
    • after a draft: "send" or "send it" means telling the person to press Send; never open the same draft again; never call send_message or create_desktop_dispatch for it; send_message must not send an open draft and names draft_in_messages; propose_message makes no claim about a sheet;
    • the manifest: "does NOT send", "did not name or confirm", recipient_needs_confirmation, the same-run precondition, coordinatorOnly;
    • the broker: authorized without a grant, never_auto_retry, absent from a leaf capability.
  • e2e: the hermetic device-tool-surface flow now asserts that a compose proposal refuses a bare name, an empty draft and an address the person never gave, before any window could open. desktop-flow-lint passes.
  • swift-format and SwiftLint (0 violations) are clean; the desktop test-quality ratchet is at baseline.
  • Whole Swift suite: xcrun swift test --skip SubscriptionEntitlementServiceTests executed 7905 tests with 10 skipped. Four tests failed (5 assertions), all known and unrelated to this change: DesktopAutomationWindowPresentationTests (2 tests, display-dependent window geometry), RecentScreensMenuContentTests.testDayOldFrameShowsACalendarDateNotAClockTime (locale date format) and MeetingScreenEvidencePassTests.testRetryAfterFinalizeJoinsTheRunTheDeadlineLeftInFlight (timing). SubscriptionEntitlementServiceTests was skipped because it crashes whole runs on this machine.

Packaged-app pass (a person, on the packaged build of this branch at e1894d91ed, production servers, macOS 27, placeholder recipients only):

  • ✓ Email draft opened in the default mailto handler and was named correctly. The handler was Google Chrome, so the draft opened in Outlook on the web, filled in, with nothing sent.
  • ✓ The Messages draft was handed to the Messages app's own composer. macOS 27 shows no sheet.
  • ✓ The typing guard refused to hand off while the person was typing (not_opened / person_typing).
  • ✓ A duplicate draft was refused (draft_already_open, twice).
  • ✓ "send" / "send it" after a draft: the guidance and the duplicate guard held. One later send_message attempt was stopped by the kernel's approval check, so nothing was sent. That case is now also refused in Swift (draft_in_messages).
  • Also confirmed: Return sends from the Messages composer. That is why the typing guard matters.

UI evidence

Captured by a person (captured_by: human, source: manual). The metadata is in .agent-artifacts/ui-evidence/desktop-propose-compose/evidence.json.

Email draft filled in, opened by the default mailto handler (Chrome, Outlook on the web)

Message draft placed in the Messages composer (no sheet on macOS 27)

Typing guard: Omi did not open the draft while the person was typing

Failure class (fixes)

No fix: commits; no declaration is required.

Line-Count-Exception: desktop/macos/Desktop/Sources/Providers/ChatToolExecutor.swift | 3961 -> 3965 | the propose_message/propose_email dispatch case with its turn text and run id, and the run and owner passed to search_contacts so its handles can address a draft; parsing, grounding and result mapping live in ChatToolExecutor+ComposeProposals.swift and Services/Compose*.swift

View guided diff

…nd tools

The new tools open a compose window through NSSharingService, filled in with recipients, subject and body, and stop there: the person reviews the draft and presses Send, and Omi has no path that sends it. On current macOS a message draft is placed in the Messages app's own composer and the handoff is reported at once, so Omi cannot see whether it is sent or discarded (a send or cancel report arrives only if the system shows a sheet); email drafts go to whatever app handles mailto links, so results name that app (resolved through NSWorkspace, with app and app_is_browser fields) and, when it is a browser, say the draft opened on its web mail site and that Omi cannot see whether it was sent; nothing assumes Mail. They share the iOS propose_message contract (name, to plus text, does-not-send, cancelled) but never report sent, because the email handoff is reported before Send. propose_message is the default for any request to text or message someone, send_message is only for when the person asks Omi to send for them, and the guidance of send_message, propose_* and create_desktop_dispatch says a message or email is never pre-filed as a dispatch. A new desktop.compose.propose bundle is a platform-confirmed write, so the policy allows it without an Omi approval card; any other write requested beside it keeps its own approval, and the tools are coordinator-only so a background worker cannot raise a window. Because the compose window has no Omi framing, send_message's recipient rule is enforced in code: every recipient must appear in words the person typed, pasted or dictated in this turn or their one previous turn in the same chat (a model-written question-card answer never counts), in a search_contacts result from the same run and owner, or be the account's own address, otherwise recipient_needs_confirmation opens nothing; phone numbers match only as whole numbers and never from dates or reference numbers, a bare name returns recipient_needs_lookup, and addresses use a narrow ASCII form. Return sends from the Messages composer, so a draft is handed off only after three seconds without a key press, with the last idle check immediately before the handoff; the same draft (channel, recipients, text digest) is not handed off again within two minutes for the same owner, which returns draft_already_open, and the guidance tells the model that after a draft the person presses Send themselves, so "send it" never re-opens the draft or calls send_message or create_desktop_dispatch, and send_message itself refuses, before any send, a recipient whose propose_message draft the same owner handed to Messages in the last ten minutes, returning draft_in_messages; each request opens at most one email draft and one message draft and an owner at most three a minute, an open Messages sheet blocks the next draft for up to five minutes, sign-out releases every limit, and a stopped call opens nothing. Results carry sent_by_omi=false and a status of handed_to_email_app, handed_to_messages, cancelled, open or not_opened. Draft content is redacted from the argument log.
…2e probe and document the contract

The hermetic device_tools_probe now checks that a compose proposal refuses a bare name, an empty draft and an address the person never gave before any window could open, and the device-tool-surface flow asserts all three. The device tool surface doc gains the propose_message and propose_email rows and explains the no-card policy, that a message draft is placed in the Messages app's composer where Return sends and the send cannot be observed, which app an email draft opens in (the mailto handler, possibly a browser's web mail) and how results name it, the duplicate-draft guard, the send-it guidance and the send_message check that refuses a send to someone whose draft is waiting in Messages, the routing rule that keeps messages off create_desktop_dispatch, the recipient grounding rule and its limits (question-card answers excluded, pasted text and the previous turn counted, whole-number phone matching, no spoken addresses), the three-second typing guard and why it is load-bearing, the per-channel per-request and per-minute limits, the never-sent statuses, what is shared with and different from iOS, and that an email app may autosave an open draft to the server's Drafts mailbox. It also corrects the stale tool count. Adds the user-facing changelog fragment.
…ools

Adds three screenshots captured by a person on the packaged build of this branch (production servers, macOS 27, placeholder recipients only): the email draft filled in and opened by the default mailto handler (Google Chrome, Outlook on the web), the message draft placed in the Messages app's own composer, and Omi's reply when the typing guard declined to hand off a draft. The sibling evidence.json records each image under the shared UI evidence contract.
@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Thanks @sauranshbhardwaj - this is a carefully built change and I went through the new code in detail. The guard stacking stands out.

Verified highlights

  • ComposeProposalService.swift uses NSSharingService (.composeMessage/.composeEmail) only - no AppleScript, no Mail/Messages automation, no TCC prompt, and no code path that can deliver a message without the person pressing Send in Apple's own window. ComposeProposalOutcome deliberately has no sent case, and every result payload carries sent_by_omi: false (composeOutcomeJSON).
  • ComposeRecipientGrounding.swift is the standout piece: a recipient is grounded only when it appears in words the person typed/pasted/dictated (this turn or the one before), in a same-run search_contacts result, or is the account's own address; model-written question-card answers are explicitly excluded, and phone/date/reference heuristics look sensible. That closes the 'address lifted from a page, email or screen becomes the person's draft' path.
  • ComposeProposalCoordinator.swift rate-limits per signed-in owner (one draft per channel per run, three per minute, two-minute duplicate digest via SHA256 draftDigest, stale-sheet blocking, owner-change teardown), and ChatToolExecutor+ComposeProposals.swift adds sendBlockedByWaitingDraft so 'send it' after a draft cannot double-send through send_message.
  • The typing guard (waitForKeyboardQuiet + UserInputPresence.secondsSinceLastKeyPress) correctly addresses a stray Return in the Messages composer sending the draft from under the person's keystrokes.
  • propose_message/propose_email were added to sensitiveArgumentTools in ChatToolExecutor+DeviceTools.swift, so recipients and draft bodies stay out of logs - good.
  • desktop-tool-policy.ts keeps propose_* out of leaf workers' allowed tools, marks them never_auto_retry and non_idempotent_write (run-tool-capability.test.ts), and selecting desktop.messaging.send does not imply the compose bundle.
  • Tests are thorough (grounding, duplicate, send-guard, email-app naming, coordinator suites, ~1000 lines) and the DeviceToolsProbe/e2e additions keep the probe deterministic and non-sending. The .agent-artifacts/ui-evidence set and changelog fragment follow existing repo convention.

Two decisions I'd like a maintainer to make

  1. Product: whether the Mac gains propose_message/propose_email as the default path for 'text/email X' requests, with send_message demoted to explicit-send-only (the guidance rewrite in omi-tool-manifest.ts and GeneratedToolCapabilities.swift).
  2. Security/policy: this is the first write-capable desktop tool that runs with approvalPolicy 'allow' - no dispatch card - on the argument that Apple's compose window itself is the approval. The design argument is coherent, but accepting that contract for an agent-initiated draft is a policy call that deserves explicit maintainer sign-off. One trade-off to confirm while deciding: a same-run search_contacts result grounds every handle of the matched contact, so the person's confirmation is effectively the recipient shown in the compose window.

CI on the relevant lanes (Desktop Swift Build & Tests, Agent Runtime, Hygiene) is green at this head.

Leaving for human maintainer review: product-direction and approval-policy sign-off needed for the new compose tool surface.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

@Git-on-my-level Git-on-my-level added needs-maintainer-review Needs a human maintainer to sign off before merge security-review Touches auth, provider routing, secrets, or security-sensitive surfaces macOS labels Oct 11, 2026
@sauranshbhardwaj

Copy link
Copy Markdown
Contributor Author

Thanks for the close read. On the two decisions:

  1. Drafts as the default for "text/email X": that's the shape iOS already ships (propose_message with requires_user_confirmation), and Epic: Rebuild Omi Tasks around quiet capture, focused goals, persistent workstreams, and contextual action #9352 §5 says sends must never be silent, so I made drafting the default and kept send_message for an explicit "send it now". Happy to flip it if you'd rather keep send_message first.
  2. No Omi card for a draft: the compose window shows the recipient, subject and body and nothing leaves until the person presses Send there, so a second card in Omi would ask the same question twice. That's the same contract device_tool_surface.dart documents for iOS. The guards around it are the grounding, the rate limits, the duplicate check and the typing guard.

On the trade-off you named: yes, a search_contacts result in the same run grounds every handle of that contact. If you'd prefer it tighter, I can ground only a handle the person actually picked when a contact has more than one, and otherwise ask them which one. iOS asks first too. That's a small change, and I can add it here or as a follow-up.

@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Thanks @sauranshbhardwaj — this is a careful, unusually well-guarded feature, and the e2e probe extension plus changelog fragment are appreciated. Detailed review below; the product/approval-model decision itself is left for a human maintainer.

What this does well, verified against the diff:

  • No send path. ComposeProposalService.swift only configures NSSharingService (.composeMessage/.composeEmail) and reports the delegate callback; there is no AppleScript or automation path, and ComposeProposalOutcome deliberately has no sent case. Every result carries sent_by_omi:false, and ComposeProposalTests.testNoOutcomeEverReportsSent enumerates all ten outcomes across both channels.
  • Recipient grounding is enforced in code, not just prompts. ComposeRecipientGrounding.ungroundedRecipients accepts only addresses from the person's current turn, their one previous typed turn (question-card qri_ answers excluded via ComposeTranscriptTurn), a same-run search_contacts result (recorded by the new hook in executeSearchContacts), or the account's own address. An address read from a page or email returns recipient_needs_confirmation and opens nothing — covered by testAnInjectedAddressOpensNothingAndAsksForConfirmation.
  • Injection-resistant validation. ComposeProposalRequest.emailPattern is a deliberately narrow ASCII form (rejects ?, &, %, /, #, lookalike Unicode domains and header-injection shapes); subject newlines are flattened; the rejection table in ComposeProposalTests covers homographs (sam@exаmple.com) and mailto-parameter-injection forms.
  • Loop and nuisance containment in ComposeProposalCoordinator: one draft per channel per run, at most three per minute, a two-minute duplicate window keyed on a SHA-256 digest (only the digest is retained), owner-scoped with sign-out release, and a typing guard (3 s keyboard quiet, at most 8 s wait) so a keystroke cannot land in the Messages composer, where Return sends.
  • The draft_in_messages guard: executeSendMessage now refuses a send to any recipient whose propose_message draft is waiting in Messages (10-minute window) — ComposeSendGuardTests covers the 599 s/601 s boundary, group drafts, and that cancelled drafts do not count.
  • Privacy: the propose tools were added to sensitiveArgumentTools (redacted argument summaries, tested in testDraftContentNeverReachesTheArgumentLog), and the executor logs shape only — recipient count, subject presence, body length.
  • The runtime policy change is narrow and well tested: desktop.compose.propose is a write bundle that is platform-confirmed (allow without dispatch, matching the iOS contract), while any other write requested beside it keeps user approval, an externalSend flag still forces dispatch, and the tools are coordinatorOnly — the broker test verifies leaf workers cannot reach them.

Judgment calls that need a maintainer, and why this is not a rubber stamp:

  1. propose_message/propose_email get approvalPolicy "allow" — no Omi approval card — on the reasoning that Apple's compose window is the approval. The grounding check makes this defensible, but it is a real loosening of the "every write needs user approval" invariant, and the documented residual risks are non-trivial: an email app may autosave the prefilled draft to the account's Drafts mailbox on the server, addresses inside a pasted thread count as grounded, and someone who resumes typing after the handoff can still type into the composer. These are disclosed honestly in docs/device-tool-surface.md; they still merit a product owner's explicit sign-off.
  2. The PR also re-routes send_message: its manifest guidance is rewritten ("only when the person explicitly asked Omi to send it") and the "always requires an approval dispatch" wording is removed. The Swift-side approval flow is unchanged, but agent behavior will shift toward drafts-by-default for all messaging requests — a UX/product direction decision, not just an implementation detail.

Smaller notes (non-blocking):

  • The send_message capability bullets in GeneratedToolCapabilities.swift state the "only when the person explicitly asked" guidance twice in different wording; harmless, just wordy for the prompt.
  • DeviceToolsProbe uses "+100****0000" as the propose-body-validation fixture recipient; if that is meant as a masked placeholder it reads oddly — confirm it is intentional.

Checks: Desktop Swift Build & Tests, Agent Runtime, PR Metadata Preflight, and Mobile Release Eligibility pass at this head; the rest are skipped by path filter as expected.

Because this changes the approval model for a security-sensitive surface (messaging/email compose on macOS), it needs maintainer sign-off on the allow-without-dispatch decision and the documented residual risks before merge; keeping security-review and needs-maintainer-review.

This feedback is AI-generated on behalf of the Omi maintainers.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

…fictional number in the propose probe

The send_message capability doc repeated the explicitly-asked rule that its prompt guidelines already carry, so the generated capability bullets said it twice in different words; the capability-doc copy is dropped and the regenerated surfaces carry the guideline alone. The propose body-validation probe now uses the fictional +15555550100, which still has to be a valid handle so the empty draft is what gets refused.
@sauranshbhardwaj

Copy link
Copy Markdown
Contributor Author

Thanks, both fixed in one small commit. send_message's capability doc in omi-tool-manifest.ts repeated the "only when the person explicitly asked Omi to send it" rule that its prompt guidelines already give. I dropped the capability-doc copy, so the generator now folds in the guideline alone and the rule appears once in GeneratedToolCapabilities.swift. The surfaces were regenerated with generate-tool-surfaces.mjs, and --check is clean. The propose body-validation check in DeviceToolsProbe now uses the fictional +15555550100 (555-01xx). It still needs a valid-looking handle, because the parser checks the recipient first and the probe has to reach the empty-body refusal, so I added a short comment saying that. Nothing else changed. The two judgment calls are still yours to make: whether the propose tools should go without an Omi approval card, with the Messages or email composer serving as the approval, and whether execute-mode pills should get the propose tools.

@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Reviewed the full diff at bc1f15c. This is a well-built feature: the draft-and-you-send contract (nothing leaves the Mac without a keystroke in Apple's own compose window) is enforced in code, not just in prompts, and the details hold up on close reading. Thanks also for the docs and test coverage.

How the no-send property holds, file by file:

  • ComposeProposalService.swift — SystemComposeSurface uses NSSharingService .composeMessage/.composeEmail and stops there; no AppleScript and no Mail/Messages automation, so no code path can deliver a message. The outcome enum deliberately has no sent case, and every result carries sent_by_omi: false.
  • ComposeProposalRequest.swift — validation is genuinely narrow: emails restricted to a tight ASCII form that excludes ?, &, %, /, # and lookalike Unicode domains (ComposeProposalTests covers header-injection shapes like a@b.com?cc=x%40y.com and sam@exаmple.com); bare names are rejected with recipient_needs_lookup plus a next_tool: search_contacts hint; phone digits are constrained to 3–15 digits (short codes through E.164).
  • ComposeRecipientGrounding.swift — the strongest part of the PR. An address the model lifted from a page, email or screen cannot become the person's draft: recipients ground only against the person's own words (this turn plus one previous turn), same-run search_contacts handles, or the signed-in account's own address. Question-card answers (model-written preparedAnswers on qri_ turns) are correctly excluded in both positions; date-shaped and "order/invoice/ticket/#"-prefixed numbers never ground a phone; phone matching is whole-number with at most a 1–3 digit country-code delta. The known limits (pasted text grounds addresses; spoken "sam at example dot com" cannot match) are honestly documented in device-tool-surface.md.
  • ComposeProposalCoordinator.swift — a single Pending slot claimed before the first suspension; per-owner limits (one draft per channel per run, at most three per minute, a SHA-256 draft digest refusing the same draft for two minutes); owner re-checked on both sides of the window; sign-out drops the slot and all limits. The typing guard (wait for 3 s of keyboard quiet before handoff, since Return sends from the Messages composer) is load-bearing and tested, and person_typing is surfaced rather than silently dropping the request.
  • ChatToolExecutor+DeviceTools.swift / ChatToolExecutor+ComposeProposals.swift — sendBlockedByWaitingDraft is a good catch: after a draft is handed to Messages, "send it" cannot produce a second message via send_message (draft_in_messages, not_sent), using the same recipient normalization as the grounding check. propose_message/propose_email are added to sensitiveArgumentTools so logs keep only shape, not content.
  • desktop-tool-policy.ts — the approval carve-out is narrow and correct: a request is allowed without dispatch only when every write bundle it carries is desktop.compose.propose (approvalPolicyFor plus the requiredBundles.every check in evaluateDesktopToolPolicy); any other write riding along keeps user_approval; and the run-tool-capability tests keep both tools out of leaf workers' allowed set (coordinatorOnly), so a background worker cannot put a window in front of the person.

Two non-blocking observations:

  1. UserInputPresence.secondsSinceLastKeyPress reads system-wide keyDown idle via CGEventSource, so the typing guard also reacts to typing in unrelated apps (typing in a browser delays a Messages draft). That is the conservative direction and probably intended, but worth confirming it is the desired semantic.
  2. In SystemComposeSurface.present, the .message branch calls NSApp.activate() so a system sheet would be visible; since macOS 27 reportedly hands off to Messages without a sheet, that path is currently future-proofing and unexercised on this OS. Fine as is — just noting it.

The model-guidance changes (omi-tool-manifest.ts, control-tool-manifest.ts, GeneratedToolCapabilities.swift, and the regenerated fixtures) read as safe and accurate: propose_* becomes the default for "text/message/email someone", and the send_message/create_desktop_dispatch guidance steers away from the dead-end dispatch routing described in the PR body. The guidance matches the enforced Swift behavior, which is the right direction.

Not approving formally. This adds a no-approval-card write surface (desktop.compose.propose) to the Mac tool policy by design and re-routes send_message away from dispatch approval — a product/policy direction a human maintainer should own before merge, even though the implementation defends it well in code. The compose-window-is-the-approval argument mirrors the contract iOS already ships, which is why this otherwise reads well to me. Keeping security-review and needs-maintainer-review, adding positive-signal.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

@Git-on-my-level Git-on-my-level added the positive-signal Automation verified a genuine fix/quality contribution label Oct 13, 2026
@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Thank you for this exceptionally careful PR — I did a full static review of the new Swift services and the agent-side tool policy, and the quality bar here is genuinely high. Summarizing what I verified and the small number of product decisions left for a human maintainer.

What I verified in the code:

  • ComposeProposalService.swift: the only handoff mechanism is NSSharingService (.composeMessage/.composeEmail) — there is no AppleScript, Mail or Messages automation anywhere behind these tools, so no code path can deliver a message without the person pressing Send in Apple's own window. ComposeProposalOutcome deliberately has no sent case, and the mailto: handler is resolved and named (app, app_is_browser) instead of assuming Mail.
  • ComposeRecipientGrounding.swift: a recipient is grounded only when it appears in words the person typed, pasted or dictated (this turn or the one before), came from a same-run search_contacts lookup, or is the owner's own address. Question-card answers (the qri_ prefix — text the model wrote, the person only clicked) are excluded, and isPhoneShaped rejects dates and order/invoice/tracking-style numbers. An address the model lifted from a page, email or screen is rejected with recipient_needs_confirmation and no window opens. This is exactly the guard this surface needed.
  • ComposeProposalCoordinator.swift: one draft per channel per request, at most three per minute, identical-draft refusal for two minutes, a typing guard (a keystroke landing in the Messages composer could press Return/SEND, so no handoff happens while the person is typing), and owner checks on both sides of the window with state dropped on sign-out.
  • ChatToolExecutor+ComposeProposals.swift: the send guard refuses send_message to a recipient whose handed-off draft is still waiting in Messages (draft_in_messages, sent_by_omi: false); every compose result carries sent_by_omi: false; logging records only counts, not recipients or body text.
  • desktop-tool-policy.ts: desktop.compose.propose is denied unless explicitly selected (the send bundle does not imply it), and it is the only platform-confirmed bundle — the compose window itself is treated as the approval, so no dispatch card double-asks.
  • Tests are thorough (seven new Swift suites covering grounding, coordinator limits, duplicates, email-app naming, the send guard and argument parsing, plus agent-runtime policy/broker/manifest tests), CI is green where it runs, and docs/device-tool-surface.md, the probe and the e2e flow all match the code.

What needs a human maintainer's call before merge:

  1. Consent-model direction: the compose window replaces the dispatch approval card for these tools (approvalPolicy: allow in desktop-tool-policy.ts). That matches the iOS propose_message pattern and I found no way to send without the person acting, but it is a security-sensitive consent-surface change that deserves maintainer sign-off, not just code review.
  2. Existing-tool behavior change: for 10 minutes after a handed-off message draft, send_message to that recipient is refused. Legitimate anti-double-send, but it changes behavior of a shipped tool and the product tradeoff (e.g. a person who genuinely wants Omi to send a fresh copy within that window) should be confirmed.
  3. Routing change: propose_message becomes the default for "text/message someone" requests and create_desktop_dispatch guidance now forbids message approvals through it. The guidance edits look proportionate, but this re-routing is a direction decision.
  4. The iOS-parity claim (iOS already ships propose_message) I could not verify from this diff.

No security issues found — no exfiltration, no hidden send path, no prompt-injection-reachable bypass of the grounding check that I could construct. Keeping security-review and needs-maintainer-review for the consent-model decision above; the positive-signal label is well earned.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macOS needs-maintainer-review Needs a human maintainer to sign off before merge positive-signal Automation verified a genuine fix/quality contribution security-review Touches auth, provider routing, secrets, or security-sensitive surfaces

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants