FE-1664: Land remaining voice stack changes

[kostandinang]

🌟 What is the purpose of this PR?

Land the complete GPT-Live Voice stack after its lower layers were merged into feature branches after parent PR #9671 had already entered main.

This recovery PR now contains the squash commits from #9719, #9673, and #9704, plus an ordinary merge of current main. It preserves the reviewed stack without rebasing, force-pushing, or cherry-picking.

🔗 Related links

• FE-1664 (internal)
• FE-1712 (internal)
• Parent merged into main: FE-1663: Experiment Live Full-Duplex Migration #9671
• Petrinaut Voice platform: FE-1664: Refine host-owned Voice sessions in Petrinaut #9719
• Full Brunch integration: FE-1664: Experiment Live Full Brunch Integration #9673
• Stabilization follow-up, now included here: FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704

🚫 Blocked by

• Kostandin’s matched speaker/headphone comparison: confirm no unwanted admissions while silent, no lost quantity/negation corrections during playback, and acceptable semantic-VAD waiting time with natural speech.

🔍 What does this change?

• Makes the reviewed FE-1664: Refine host-owned Voice sessions in Petrinaut #9719, FE-1664: Experiment Live Full Brunch Integration #9673, and FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704 changes reachable from main through one recovery PR.
• Reconciles the Voice stack with the AI experiment work that reached main alongside FE-1663: Experiment Live Full-Duplex Migration #9671.
• Adds the host-owned Voice presentation and the GPT-Live/Brunch transcription and commentary path.
• Includes FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704 stabilization: browser audio processing preferences, medium-eagerness semantic VAD, a 500 ms Speaking-indicator hold, and patient-listening guidance.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• modifies an npm-publishable library and I have added a changeset file(s)

The existing @hashintel/petrinaut patch changeset from #9719 is included.

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

⚠️ Known issues

• Acoustic improvement, human conversational latency, and deterministic feedback-loop prevention are not established until the owner-held speaker/headphone witness is complete.
• The corrected synthetic provider probe retained all three utterances, but two completions took 5–6 seconds.
• Live speech remains best effort; commentary acceptance is not proof of playback.

🐾 Next steps

Complete the owner-held manual witness described in #9704, then merge this PR into main if its acceptance criteria and CI pass.

🛡 What tests cover this?

• Recovery reconciliation: affected dependency graph build, Petrinaut and website typechecks, and Petrinaut and website ESLint passed.
• @hashintel/petrinaut unit suite: 1,166 passed.
• FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704 provider-free verification: 386 tests passed in 11 Live/Realtime suites; the website build/typecheck/lint graph passed all 16 tasks.
• Post-squash review verification: 21 transport tests and 74 Live bridge/transcription tests passed.
• Corrected synthetic provider probe: three committed/completed transcripts in predecessor order, no extra inputs or provider errors, and verified cleanup. This does not establish acoustic behavior.
• Current-head CI: website unit tests passed.
• Current-head CI: website integration tests passed.

❓ How to test this?

1. Run the provider-free Voice suites and website build/typecheck/lint commands documented in FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704.
2. Start the Brunch development pair with PETRINAUT_OPENAI_VOICE_ENABLED=true PETRINAUT_VOICE_PROVIDER=live yarn dev:brunch.
3. Perform FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704’s matched speaker/headphone witness using the same browser, microphone, volume, prompt, and disposable document.
4. Confirm the host-owned Voice states from FE-1664: Refine host-owned Voice sessions in Petrinaut #9719, the canonical Brunch commentary flow from FE-1664: Experiment Live Full Brunch Integration #9673, and the stabilization behavior from FE-1712: Stabilize GPT-Live full-duplex voice feedback #9704.

📹 Demo

See the original PRs for review fixtures and manual test context. A physical speaker/headphone witness remains pending.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hash	Ready	Preview	Sep 15, 2026 3:55pm UTC
hashdotdesign-tokens	Ready	Preview	Sep 15, 2026 3:55pm UTC
petrinaut	Ready	Preview	Sep 15, 2026 3:55pm UTC
petrinaut-docs	Ready	Preview	Sep 15, 2026 3:55pm UTC

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

PR Summary

High Risk
Changes dual WebRTC voice transport, canonical Brunch admission/settlement, and user-facing error handling on a security-sensitive unauthenticated voice path.

Overview
Recovers and lands the Brunch-backed GPT-Live Voice stack on the website: one microphone capture drives parallel Live and gpt-4o-transcribe WebRTC sessions, with a new /api/voice/transcription-session handler alongside the existing live-session route.

Canonical flow: finalized transcription items (committed order, session-scoped IDs) feed LiveBrunchBridge, which admits utterances to Brunch, waits for complete settlements and rendered/snapshot prose, then offers frozen answers to Live via appendCommentary / appendInstructions correlated to client delegations—without auto-retry, replay, or streaming partial Brunch text.

LiveConversationControl wires the bridge to panel transport (settlements, Flue snapshot, response lifecycle, stop-response), maps composer submitted/streaming to dock Thinking, surfaces connection errors on the dock, and routes Brunch/Live append failures to warningMessage (paired with the @hashintel/petrinaut patch: failures behind a warning popover, compact dock layout, setup until connect).

Docs shift from standalone FE-1663 experiment copy to FE-1664 Brunch-backed Live and a slim FE-1712 mission pointer; demo app passes snapshot into voice mode. Large provider-free test suites cover bridge settlement edge cases and dual-connection lifecycle.

^{Reviewed by Cursor Bugbot for commit 096d839. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 096d839. Configure here.}