Repository navigation
Let an agent embed a screenshot in its reply, at a size it picks - #320
Merged
Merged
Conversation
Tool-result screenshots already showed up in the transcript as thumbnails on the tool card (#303, #314), but an agent had no way to put one in its own prose — which is where "here's the thing I just built" belongs. Two things were missing. The agent never learned where the capture landed: the MCP result is image bytes, and the path Ness spills them to was derived later, in the transcript extractor. /browser/screenshot now does the spill itself and returns the path. The write is content-hash keyed, so it's the same file the extractor derives from the image block — one copy, not two. And a local path in markdown couldn't render anyway. The renderer's origin is app:// (http://localhost in dev), so an absolute filesystem path resolves against the wrong root, and react-markdown blanks file:// in its default URL transform. Chat markdown now routes srcs that look like local files through the attachment-read IPC that the existing thumbnails use, at full column width. Remote srcs keep their old behaviour, and a blanked javascript: src renders nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An agent that embeds a screenshot had no way to say how big it should be, and the unsized default — a 24rem-tall cap — gave one screenshot about a third of the chat window. CommonMark has no sizing syntax and raw HTML is off in chat (rehype-raw would let any model-authored markup into the renderer), so the size rides in the places an image already carries free text. Three forms, all prior art, because agents guess differently: a pipe in the alt (``, Obsidian), a leading-`=` title (``, markdown-it-imsize), and a fragment on the path (`…png#w=200`). Specs are `400`, `400x300`, `x300`, `50%`, clamped to 16-2000px. A title without the `=` is still a tooltip, and alt text that merely contains a pipe is left whole. The fragment form is the reason this isn't just the first two: it previously resolved to nothing and fell through to a broken <img> that rendered as blank space — the worst available outcome for someone who guessed a reasonable syntax. Default for an unsized embed is now 16rem tall / 32rem wide, down from 24rem tall, with the loading placeholder following it down. Verified against a live transcript in the web client: the title and fragment forms both render at the requested 200px, and the unsized control dropped from ~535x340 to ~357x218. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tool-result screenshots already showed up in the transcript as thumbnails on the tool card (#303, #314), but an agent had no way to put one in its own prose — which is where "here's the thing I just built" belongs.
Getting the screenshot into the reply
The agent never learned where the capture landed. The MCP result is image bytes, and the path Ness spills them to was derived later, in the transcript extractor.
/browser/screenshotnow does the spill itself and returns the path; the bridge appends one terse line to the result. The write is content-hash keyed, so it's the same file the extractor derives from the image block — one copy, not two.A local path in markdown couldn't render anyway. The renderer's origin is
app://(http://localhostin dev), so an absolute filesystem path resolves against the wrong root, and react-markdown blanksfile://in its default URL transform. Chat markdown now routes srcs that look like local files through the attachment-read IPC the existing thumbnails already use. Works for any image on the machine, not justscreenshot_tabcaptures. Remote srcs keep their old behaviour, and a blankedjavascript:src renders nothing.Sizing
CommonMark has no sizing syntax and raw HTML is off in chat (
rehype-rawwould let any model-authored markup into the renderer), so the size rides in the places an image already carries free text. Three forms, all prior art, because agents guess differently:=title (markdown-it-imsize)Specs are
400,400x300,x300,50%, clamped to 16–2000px. A title without the=is still a tooltip; alt text that merely contains a pipe is left whole.The fragment form is why this isn't just the first two: it previously resolved to nothing and fell through to a broken
<img>that rendered as blank space — the worst outcome for someone who guessed a reasonable syntax.Unsized default is 16rem tall / 32rem wide (was 24rem tall), since one screenshot was taking about a third of the chat window.
Still renders as raw text:
with the spec outside the quotes. CommonMark won't parse it as an image at all, so honouring it needs a remark transform — skipped as out of proportion to three working forms.Verification
src/shared/chat-image-src.test.ts— 22 cases over src resolution (posix,file://+ percent-decode, drive letters, remote/relative/unknown-ext rejections) and the three size parsers (clamping, non-size tails, tooltip preservation).renderToStaticMarkup: absolute paths,file://, alt pipes and=titles all arrive intact — including a pipe escaped inside a GFM table cell;javascript:alert(1)arrives blanked and renders nothing.npm run typecheck,npx electron-vite build, and all 1745 tests pass.🤖 Generated with Claude Code