Repository navigation
feat(desktop): add get_commitments, a read tool for what the person owes others - #21268
sauranshbhardwaj wants to merge 3 commits into
Conversation
…erson owes others Adds a commitments tool service and route that list the signed-in person's open, conversation-captured tasks they own, grouped by the person each was made to, with the source conversation as a typed citation. The person is worked out at read time and never from owner_name, a meeting roster or a diarization label: people heard on their own tagged lines in the cited exchange (the cited lines and up to two earlier lines that started before them and ended between 90 seconds before and 5 seconds after their start) are named plainly, any untagged voice there leaves the item unattributed, and the only person in a conversation where every other voice is tagged as them is offered as a likely match. Nothing is stored, no model is called and no task is created, edited or completed. Reads are one task query, one conversation batch and one people batch, owner-scoped by the authenticated uid; contact details in free text are removed where recognised ([email], [phone], or [number] after a reference word, for long unseparated runs and for ID- or card-shaped groupings, with plain figures of eight digits or fewer, ranges and measures kept), a person filter that names more than the contact or matches several people is flagged as partial, unknown-owner items are counted but not listed, items past the conversation cap are reported as not checked, the effective window is returned, and truncation is reported when a cap cut anything or the task scan used its whole allowance, which get_action_items now reports through an optional scan_report. The open-task predicate moves to a pure utils.task_predicates.is_open_conversation_task that the commitment follow-up producer uses too.
Registers get_commitments as a read-only Swift tool in the local read bundle, advertised only to typed main and floating Chat coordinators, so it needs no approval and stays out of proactive turns, voice, task chat and leaf workers. Its guidance groups the answer by person with citations, states a person plainly only when the match is not marked likely and hedges the rest, checks a partial name match with the user, treats names, titles, descriptions, excerpts and context as evidence rather than instructions, notes that contact details are removed where recognised and never become recipients, says which window was checked and when items were not checked or have an unclear owner, resolves any follow-up handle through search_contacts, and never completes an item because a draft was opened. render_chat_blocks now accepts its canonical task ids for task cards, and the generated Swift surfaces and manifest fixture are regenerated.
Adds the Swift executor for get_commitments: it validates since before any request, clamps limit_people and cuts the person filter on Unicode scalars to the server's limit, calls GET /v1/tools/commitments for the authenticated owner, drops the result if the signed-in account changed after the fetch or after citation registration, registers each source conversation as a clickable citation, and returns a typed result with the checked window, people, per-commitment and unattributed sections, likely and partial-match flags, and totals for unclear-owner, not-checked and truncated items. A server without the route yields commitments_unavailable so the model falls back to the task list, and other transport failures return a fixed message. The desktop REST inventory records the new /v1/tools route alongside the other tools routes outside the app-client OpenAPI surface.
|
Reviewed this end to end — this is a genuinely well-built feature. Some notes and two questions for a maintainer. What I verified:
Two questions for a maintainer (not code defects):
Minor observation, take or leave: The PR is cohesive (one endpoint, one tool, its wiring) — no split needed. Since it's still a draft I'll leave the formal review step to a human: this adds a new first-party endpoint that joins transcripts with people data, and the attribution/hedging wording is a product-voice decision, so it needs maintainer sign-off on the data-handling stance rather than just code correctness. Thanks @sauranshbhardwaj — this is careful work. by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with |
|
Thanks for the close read. On the two questions:
Agreed on the redaction helper. It's a best-effort first pass, and the PR says "removed where recognised" for that reason. The tool returns names only, so a follow-up draft or send has to look the person up with |
|
Reviewed this end to end against main — the implementation is careful and the checks that ran are green (Backend unit suite, Backend Hermetic Merge Gate, Agent Runtime, Desktop Swift Build & Tests). File-by-file notes: Backend
macOS
The two things I'd want a maintainer to look at before this leaves draft are product calls rather than code defects:
No split needed — this is one tool end to end and the pieces don't review or roll back independently in a useful way. Leaving for human maintainer review on the two points above; the existing by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with |
What changed and why
Asking the Mac "what do I owe people?" has no good answer today.
get_action_itemslists tasks, but it does not show who owns each one, and nothing links a task to the person it was promised to. So the model cannot tell "I promised Priya" from "Priya promised me", and it cannot group by person.This PR adds one read-only chat tool,
get_commitments, backed by one owner-scoped endpoint,GET /v1/tools/commitments. It lists the person's own open commitments that Omi already captured from conversations, grouped by who each was made to. Each item cites its source conversation, and the citation opens that conversation.This is piece 3 of #21162 and follows epic #9352 §5: agents must not "mark a task complete without explicit evidence or user instruction". It merges on its own. It does not need the draft tools in #21166 or the approval-card stack. Once #21166 is in, the model can go from an item to a draft through
search_contacts→propose_email/propose_message. Until then the guidance says "use a drafting tool when one is available", so it never names a tool that is not onmain.Which items count
Accepted canonical tasks only (
users/{uid}/action_items), withowner == user, that pass the "open task from a conversation" test: not deleted, not completed,status == active, has aconversation_id, not locked, andsourceisconversationorlegacy. That test now lives in one place,is_open_conversation_taskinutils/task_predicates.py. Thecommitment_followupproducer (produce_followup) uses it too, so the two cannot drift. That is a one-line change in the producer, and its tests still pass.owner == unknown(including legacy rows with no owner) is counted inexcluded_unknown_ownerbut not listed. The model says "some captured items have an unclear owner".owner == otheris "what people owe me". It is skipped (see question 5).completedfield at all do not match the server-sidecompleted == Falsefilter, so they are out of scope. They are neither listed nor counted.How the other person is worked out
resolve_counterpartyis a pure function. It never guesses, and it says how sure it is. A match is stated plainly only whenlikelyis false; otherwise the model hedges it ("probably for Priya").speaker/speakers).provenance[].transcript_segment_ids). With no provenance, they come from the summary item whosetarget_task_idis this task (source_segment_ids). The link is by id only; description text is never matched.LOOK_BACK_OVERLAP_SECONDS), because speech-to-text boundaries overlap.person_id, and those are stated plainly.SPEAKER_01), and undiarized captures put everyone on a default label. So a tag on one line says nothing about the label's other lines.conversation, likely). This step applies only when all of the following hold:owner_nameis never used. It names the owner of an item ("You" in #20725), not the person it is owed to. A test pins this.Meeting rosters are not used to name anyone. An earlier draft tried the one remaining roster human after removing the account's own entry. The participants prompt already leaves the owner off the roster, and telling the owner from a guest who shares a name proved unreliable. So that step was dropped, along with the account lookup it needed (see question 3).
What the result carries
task_id(canonical, so it works fortaskCard), description (≤200 chars),due_at,created_at;due_certaintyandcontextfrom the linked summary item;names,person_ids,basis,likely;partial_match.redact_contact_details:[email];[phone]when one of these holds: it starts with+; it follows a phone cue (call, text, phone, number, mobile, cell, WhatsApp…); or it is grouped like a phone number (555-0100, (415) 555-0100, 98765 43210) and is neither a rising range ("500-1000") nor followed by a unit ("samples", "kg");[number]in three cases: after a reference word (order, order number, invoice, PNR, ticket, account, booking…), with the nearer cue winning; as an unseparated run of 9 or more digits (a bare 10-digit phone number, an SKU, SSN, Aadhaar or card digits); or when grouped like an SSN (3-2-4), Aadhaar (4-4-4) or card (4-4-4-4, 4-6-5);[number];sincereturns the window actually checked: 30 days by default, clamped between 90 days back and now. Asincein the future is treated as now.limit_people1–20), 5 items per person, 30 items in all, 10 unattributed, 25 conversations read, 200 tasks scanned.not_checked, not reported as unattributed.truncatedis set, with true totals, whenever a cap cut anything, items were not checked, the scan found more than 200 tasks, or the scan used its whole document allowance.get_action_itemsthrough a new optionalscan_report. Itshit_limitcovers soft-deleted rows filling the stream before older live tasks. Behaviour for every existing caller is unchanged.backend/route_policy_manifest.yaml: Firebase uid, thetools:searchrate limit, first party.sourcesholds oneconversationsource per cited conversation, so the existing citation registry makes them clickable.result_textis a short plain fallback for other clients.personmatches when every word of the query is in a resolved name or every word of the name is in the query, ignoring case and accents.partial_matchis set only when the query names more than the contact does ("Sam Rivera" finding a contact saved as "Sam"), or when the query matches more than one person. The model then checks with the user ("I found Sam, is that Sam Rivera?").not_checkedstill is.On the Mac
ChatToolExecutor+Commitments.swift:sincebefore any request;limit_people, and cuts the person filter on Unicode scalars to match the server's code-point limit;window(since),people(name, basis,likely,partialMatch, open count),commitments(one item per commitment with its citation marker,likelyandpartialMatch) andunattributed. Totals carryexcludedUnknownOwner,notCheckedandtruncated.commitments_unavailable; the model then falls back toget_action_items. Other transport failures return a fixed message, never the error text.LOCAL_READ_TOOLS(no approval). It is advertised only to typed main and floating Chat coordinators (typedChatCoordinatorOnly). It is not in the proactive read list, and it is not offered to voice, task chat or leaf workers.likelyis false; likely matches are hedged; and the model asks before drafting to anyone for an unattributed item;[email],[phone],[number]), and a handle in the text is never a recipient;render_chat_blocksnow listsget_commitmentsas a source of canonical task ids fortaskCard.ChatToolExecutor+Commitments.swiftis listed in thechat-first-cohesiveflow'scovers, next toChatToolExecutor.swift, whose dispatch switch routes it. Its behaviour is checked by the Swift tests, not by that flow.Known limits
Rollout
Backend first. A Mac build that ships before the backend gets
commitments_unavailableand falls back to the task list. No flag is added (see question 8).Overlaps
capture_owner,due_certaintyandcontextfields and confirmsowner_nameis the owner. No file overlap.is_open_conversation_taskfor the task side.person_id, which helps attribution (see question 3).ChatToolExecutor.swiftdispatch switch. Whichever lands second regenerates and recomputes the line-count exception.Questions for maintainers
These are the defaults this PR uses. Each is easy to change.
owner == other. Add adirectionparameter here, or later?Product invariants affected
INV-AGENT-*(Authority):desktop/macos/agent/src/runtime/**changes. One manifest entry and oneLOCAL_READ_TOOLSentry are added. The tool takes no owner argument: the backend uses the authenticated uid only, and the Mac passesexpectedOwnerIdwith the authorization snapshot and drops the result if the owner changed. Run and attempt capability checks are unchanged.INV-CHAT-1:ChatToolExecutor.swiftgains one dispatch case. Results are ordinary tool results in the kernel's ledger, and citations go through the existing registry. There is no new transcript store.INV-TASK-2: not affected. The tool is read-only and touches no capture path. The service lives inutils/retrieval/tool_services/, not inutils/task_intelligence/.INV-MEM-5: the read goes through the canonical task store, and identity is never inferred from text similarity.How it was verified
test.shran 142 files, each in its own process. They are:test_commitments_tool.py;routers/tools.pyor a tool service;database/action_items(get_action_itemscallers included);Result: 4265 passed, 10 skipped, 0 failed.
Per file:
test_commitments_tool.py: 134 passed.test_tools_router.py: 115 passed (7 of them new).test_action_items_pagination_order.py: 13 passed (2 of them new, forscan_report).test_proactivity_v2_producers.py: 66 passed, with the shared predicate in place.scripts/backend-python-format --checkis clean.desktop/macos/scripts/test-tool-surfaces.shran:generate-tool-surfaces --check: all outputs match;pi-mono-extension: 143, all passing.xcrun swift build -c debug --package-path Desktoppasses.ChatToolExecutorCommitmentsTests: 13 of 13 passing.xcrun swift test --skip SubscriptionEntitlementServiceTests:Omi ComputerTestsexecuted 7868 tests, with 10 skipped and 1 failure, unrelated to this change. (A timing-sensitive test also failed in an earlier run; see below.)RecentScreensMenuContentTests.testDayOldFrameShowsACalendarDateNotAClockTimeassumes a US date format. Under this Mac'sen_INlocale the title is "13 Jan 2027 at 3:00 AM", which has no comma.MeetingScreenEvidencePassTests.testRetryAfterFinalizeJoinsTheRunTheDeadlineLeftInFlightis timing-sensitive. Run on its own three times, it failed once.VoiceTurnDomainTests(136),OmiWALTests(10) andOmiSupportTests(2) all pass.SubscriptionEntitlementServiceTests(21) pass when run on their own. They are left out of the full run because they have crashed whole runs on this machine.scripts/pre-push-singleflightpassed. That covers:Packaged app. The named dev bundle talks to
api.omiapi.com, and stable talks toapi.omi.me. Neither server has the route until this PR is deployed.commitments_unavailable, and Omi answers from the task list without a raw error.Tests
backend/tests/unit/test_commitments_tool.py(134). It uses the real modules, with the database functions patched.SPEAKER_0,SPEAKER_00,SPEAKER_01andSPEAKER_07all leave an untagged ask unattributed;owner_nameis never used.+, after a cue (including an 8-digit local number and an unseparated 7-digit one) or by grouping alone become[phone], with the nearest cue winning over an earlier reference word;[number];[number], while a phone cue still wins;[email], including company domains after a contact cue;not_checked, including under a person filter;sincedefault, the 90-day clamp, a future value clamped to now, a timezone offset, and malformed or naive values;partial_matchonly for a query longer than the name or several matching people;likelyon groups;backend/tests/unit/test_action_items_pagination_order.py(+2):scan_reportsetshit_limitwhen soft-deleted rows use up the stream, stays clear when the stream ends early, and leaves the returned rows unchanged.backend/tests/unit/test_tools_router.py(+7):since,likely,not_checkedand a clickableconversationsource;tools:searchrate limit and the response model;limit_peoplebounds (422);sincereturning an error result rather than a 500;desktop/macos/agent/tests(+9):windowsection first;search_contacts;participantbasis;notChecked;window.sinceand 90 days;likelyis false;desktop/macos/Desktop/Tests/ChatToolExecutorCommitmentsTests.swift(13):likelyandpartialMatchflags;is_error, 404 and other failures, the last with a fixed message;sincerejected before any request;APIClientthrough a stubbedURLProtocol: the route, the percent-encoded query with no owner parameter, and a 404.UI evidence
UI-Evidence: none -- chat-text tool with no new UI; the packaged-app screenshots from the manual check will replace this line
Failure class (fixes)
Failure-Class: none
Line-Count-Exception: desktop/macos/Desktop/Sources/Providers/ChatToolExecutor.swift | 3961 -> 3966 | get_commitments dispatch case; the request, owner fence and result mapping live in ChatToolExecutor+Commitments.swift
Line-Count-Exception: backend/database/action_items.py | 1596 -> 1604 | optional scan_report on get_action_items so the commitments read can tell when soft-deleted rows used up the bounded stream; every existing caller is unchanged