Repository navigation
feat: each bot gets its own browser, watched live in the app - #32
Merged
Merged
Conversation
BrowserState, BrowserFrame, BrowserAction and BrowserView for the app (browser.list, browser.watch, browser.unwatch and their notifications), the browser_sites table, and chat summaries for the browser tools and the owner's site request (spec 21.5, 21.7).
The daemon starts a Microsoft Edge without a window per bot on its first browser_* tool, in a job of its own, with a profile under <home>\browsers. A small DevTools client follows every tab (popups become the active one), waits for pages to settle, accepts dialogs and sends downloads to the bot's folder. A page reader in an isolated world turns the page into text with refs for links, buttons and fields. Ten MCP tools let the bot open, read, click, type, choose, press, scroll, go back, see the screen and close. In Manual, Accept edits and Plan modes the first visit to each site waits for the owner, and files open only from the bot's own folders. The app lists browsers, watches one per connection with latest-wins frames, and hears every action for the cursor. Idle browsers and those of paused or archived bots close. Spec: new section 21; tests run against the real Edge when installed.
A globe button in the bot's header opens the browser panel, with a dot while the bot is browsing. The panel shows the address, the page as the bot sees it (frames from browser.watch), the bot's cursor gliding to each point it acts on, a ring where it clicks and a line about what it did. Closed browsers keep their last page, faded; one that could not start says why. The panel can take all the room the chat can spare. Browser tool lines in the chat get the globe and a button to watch, and a bot asking to use a site gets its own card. The store follows browser.list and browser.changed; FakeBotloft fakes browsers, and the dev preview has Scout searching live. Texts in English, Portuguese and Spanish.
A Manual bot on Haiku asked for each new site, opened and clicked, and answered from the screenshot image. Also records why Edge runs with --disable-extensions and why a new tab's setup goes out at once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Bots can now research and use websites in a browser of their own, and the owner watches it live beside the chat (spec section 21).
Daemon (N1)
--headless=new), started on its firstbrowser_*tool, in its own Job Object, with a profile in<home>\browsers\<bot_id>. A small DevTools client follows every tab (popups become the active one), waits for pages to settle, accepts dialogs and sends downloads to the bot'sdownloads\folder.[e12 link "Sign in"]for links, buttons and fields.browser_open,browser_look,browser_click,browser_type,browser_select,browser_press,browser_scroll,browser_back,browser_screenshot(an image the model sees) andbrowser_close.browser_sites. Auto and Bypass browse freely.file:pages open only from the bot's own folders.browser.list, andbrowser.watch/browser.unwatchper connection.browser.changedandbrowser.actiongo to every app;browser.framegoes only to the connection watching, newest frame wins.[browser] idle_minutes), when their bot or crew pauses, and pastmax_open. Archiving deletes the profile.App (N2)
FakeBotlofthas browsers, and the dev preview shows Scout searching live.Tests
cargo test --workspace(235), clippy-D warnings, fmt,pnpm check(124 tests).crates/botloftd/tests/browser.rsruns against the real Edge (skipped where it is not installed) with a local site: form filling, site allow/deny, files, new tabs, dialogs, live frames, actions and screenshots.Tested with the real Claude Code (2.1.284, Haiku, Manual mode)
Done with a dev daemon and the app preview (
?live):Still to try by hand: the panel inside the Tauri window (
pnpm tauri dev) rather than the browser preview.