Repository navigation
perf(app): stream a reply without re-rendering the whole chat - #76
Merged
Merged
Conversation
Each chat.delta re-rendered every run, tool line, card and the composer of the open chat, and parsed the whole draft as markdown again, so a long reply cost more with every piece. - BotRun, InboundRow, NoticeRow, ChatComposer and ChatView are memoized; BotRun compares its items one by one, which withItem keeps stable, so only the run being written re-renders. - chatRows runs only when the items change; useFiles and onSent keep their identity, and BotView's show callbacks are stable (useStable), so the chat skips the renders of the view around it (bot changes, screen drafts every 250 ms). - The draft is split after its last finished block (not inside a code fence): that part is parsed once, only the rest again as text arrives. - day() and decimal() reuse their Intl formatters per language.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third slice of the performance round. It is independent of #75, and each helps on its own.
What changes
BotRun(compares the items one by one;withItemkeeps unchanged items stable),InboundRow,NoticeRow,ChatComposerandChatView. Eachchat.deltanow re-renders only the run being written, not every run, tool line and card in the chat.chatRowsruns only when the items change.useFilesandonSentkeep their identity.BotView'sshowFile/showBrowser/showScreenare stable (ui/useStable.ts), so the chat skipsBotView's re-renders: bot changes, andscreen.draftevery 250 ms.day()anddecimal()reuse theirIntlformatters per language.Not done here, for now: batching
chat.deltabyrequestAnimationFramein the app. #75 already sends at most ~25 per second per bot.Tests
stream.test.tsxcounts the renders of a finished run'sToolLineswhile text streams: 0 extra, against 4 without the memo (I checked that it fails).rows.test.ts:splitDraftwith paragraphs, a code fence and an indented line.pnpm checkpasses. TheFakeBotloftpreview shows the chat as before.Manual check (real Claude)