Skip to content

perf(app): stream a reply without re-rendering the whole chat - #76

Merged
httpsphl merged 1 commit into
mainfrom
perf/app-chat
Sep 30, 2026
Merged

httpsphl merged 1 commit into
mainfrom
perf/app-chat

Conversation

@httpsphl

Copy link
Copy Markdown
Owner

Third slice of the performance round. It is independent of #75, and each helps on its own.

What changes

  • Memo on BotRun (compares the items one by one; withItem keeps unchanged items stable), InboundRow, NoticeRow, ChatComposer and ChatView. Each chat.delta now re-renders only the run being written, not every run, tool line and card in the chat.
  • chatRows runs only when the items change. useFiles and onSent keep their identity.
  • BotView's showFile / showBrowser / showScreen are stable (ui/useStable.ts), so the chat skips BotView's re-renders: bot changes, and screen.draft every 250 ms.
  • The draft is split after its last finished block (never inside a code fence, never before an indented line). The finished part is parsed once, and only the tail again as text arrives. Before, the whole draft was re-parsed on every piece.
  • day() and decimal() reuse their Intl formatters per language.

Not done here, for now: batching chat.delta by requestAnimationFrame in the app. #75 already sends at most ~25 per second per bot.

Tests

  • stream.test.tsx counts the renders of a finished run's ToolLines while text streams: 0 extra, against 4 without the memo (I checked that it fails).
  • rows.test.ts: splitDraft with paragraphs, a code fence and an indented line.
  • pnpm check passes. The FakeBotloft preview shows the chat as before.

Manual check (real Claude)

  • Ask a bot for a long answer with paragraphs, a list and a code block. The text appears normally while being written, the code block does not break mid-way, and the final reply replaces the draft without a jump.

Each chat.delta re-rendered every run, tool line, card and the composer
of the open chat, and parsed the whole draft as markdown again, so a long
reply cost more with every piece.

- BotRun, InboundRow, NoticeRow, ChatComposer and ChatView are memoized;
  BotRun compares its items one by one, which withItem keeps stable, so
  only the run being written re-renders.
- chatRows runs only when the items change; useFiles and onSent keep
  their identity, and BotView's show callbacks are stable (useStable),
  so the chat skips the renders of the view around it (bot changes,
  screen drafts every 250 ms).
- The draft is split after its last finished block (not inside a code
  fence): that part is parsed once, only the rest again as text arrives.
- day() and decimal() reuse their Intl formatters per language.
@httpsphl
httpsphl merged commit 7fb1dd2 into main Sep 30, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant