Skip to content

FE-1833: Speak progress while Brunch works - #9889

Open
kostandinang wants to merge 5 commits into
mainfrom
kd/fe-1833-speak-progress
Open

kostandinang wants to merge 5 commits into
mainfrom
kd/fe-1833-speak-progress

Conversation

@kostandinang

@kostandinang kostandinang commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

🌟 What is the purpose of this PR?

Let Brunch say a short progress line during long Live voice turns, while staying silent through quick answers and fast tool bursts.

🔗 Related links

🚫 Blocked by

🔍 What does this change?

  • Code chooses up to two fixed progress lines per turn and instructs Live to read them verbatim.
  • Stay quiet during speech and approval waits, with timing limits to avoid chatter.
  • Send Live quiet context about the current work, keep progress out of written history, and hold the final summary while progress audio is active.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

  • No publishable library changes.

📜 Does this require a change to the docs?

  • Yes; voice documentation is still pending.

🕸️ Does this require a change to the Turbo Graph?

  • No execution graph changes.

⚠️ Known issues

The recording exposed a final-caption mismatch: part of the acknowledgement appears in the written summary but not its audio. Live also added “I'm” to the second progress line. The model build was blocked by browser-tool lease failures.

🐾 Next steps

Fix the caption mismatch, verify a successful model-building turn, and finish the voice docs before marking ready.

🛡 What tests cover this?

  • Scripted turns, timing boundaries, approval waits, interruptions, summary ordering, and written history.
  • Passed: 1,038 unit tests, 5 integration tests, type checking, lint, formatting, and build.

❓ How to test this?

  1. With working voice credentials, run PETRINAUT_OPENAI_VOICE_ENABLED=true PETRINAUT_VOICE_PROVIDER=live yarn dev:brunch. Open /new on port 4915.
  2. Try quick answers, long model-building and experiment turns, interruptions, and approval waits.
  3. Check verbose [Petrinaut Live trace] logs. Progress should be audible only on long turns, leave the request open, and neither overlap the summary nor appear in written history.

📹 Demo

progress-demo.mp4

Click to play with sound. Real Live session, 70 seconds with long pauses trimmed: first progress at 0:28, second at 0:39, final response at 0:48. The final response reports the failed model build; this is not a successful-build demo.

@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Oct 2, 2026 10:01pm UTC
petrinaut Ready Ready Preview Oct 2, 2026 10:01pm UTC
petrinaut-docs Ready Ready Preview Oct 2, 2026 10:01pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Oct 2, 2026 10:01pm UTC

Request Review

@cursor

cursor Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes Live turn timing, delegation commentary, and settlement ordering in the voice bridge; mistakes could overlap progress with summaries or speak during approval waits.

Overview
Adds spoken progress for long Brunch Live voice turns: a new ProgressPolicy watches assistant tool UI state (plus host mutation approval via isToolAwaitingApproval) and may send up to two fixed commentary lines through Live, with timing gates so quick answers and fast tool bursts stay silent.

LiveBrunchBridge polls progress on a timer, publishes quiet JSON progress via appendThinking, sends null-delegation commentary, and holds the final summarized answer until Live output is quiet (with bounded fallbacks). Captions treat progress as spoken-only so it does not land in written voice history or steal wrap-up captions; Live session instructions now tell the model to read supplied progress verbatim.

The demo wires approval pending state from mutationApproval.coordinator into voice mode; controls forward output activity into liveSpeaking and attribute late append acceptances to the correct turn.

Reviewed by Cursor Bugbot for commit 18488e5. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread apps/petrinaut-website/src/main/app/voice-interview/live-speech-captions.ts Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

kostandinang and others added 5 commits October 2, 2026 21:56
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 18488e5. Configure here.

result.status === "accepted" &&
result.startMs !== undefined
) {
captions.progress(result.startMs);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Progress send failures warn as answers

Medium Severity

Progress lines are sent as commentary with a null delegation, but append failures still use the existing commentary warning that calls the payload an answer. A refused progress send can surface that warning while Brunch is still working and the wrap-up has not been attempted.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 18488e5. Configure here.

This branch was successfully deployed

3 active deployments
Preview – petrinaut-docs — 18488e58 Deployed Oct 2, 2026 by vercel[bot]
Preview – hash — 18488e58 Deployed Oct 2, 2026 by vercel[bot]
Preview – petrinaut — 18488e58 Deployed Oct 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

1 participant