[httpsphl]

What

The design for Desktop Use: bots seeing and using the apps open on the owner's screen. It adds spec section 24 and docs/adr/0002-desktop-use.md. No code yet.

The owner made these decisions on 2026-10-04:

• When: the bot may also act while the owner is away, in apps already granted. That option only turns on after a screen that spells out the risks and an "I understand the risks" check.
• Scope: the owner picks one app at a time or the whole desktop.
• Levels: see and act are granted apart.
• How it acts: UI Automation first, which never takes the owner's cursor. Real mouse and keyboard are an option the owner turns on.

The design also covers:

• Modes: auto and bypass_permissions do not open the desktop. A saved grant is always needed.
• Never granted, kept as a list in code:
  • Botloft itself;
  • elevated windows;
  • the secure desktop;
  • terminals and command boxes, since typing there would skip the bot's command permission rules;
  • password managers;
  • password fields, whose value is never read and never typed into.
• No tool to launch programs.
• Titles of windows outside the scope are hidden. The bot sees only the app name.
• Real input: the owner's own (non-injected) input stops the bot at once. An on-screen notice shows while it acts, and Ctrl+Alt+End stops everything.
• One desktop action at a time, across all bots.
• Windows only at first.
• Milestones D1–D6: see, act, live panel, real mouse and keyboard, unattended, whole desktop.
• Section 19 gets four Windows behaviors to check with real tests: UI Automation and PrintWindow while the screen is locked, SetForegroundWindow from the background program, and telling injected input apart from the owner's.

Review

This is a security design, so the owner should read section 24.2, 24.3, 24.8 and 24.10 before D1 starts.