[httpsphl]

What

The daemon part of D3 (spec 24.9). The app's panel comes next, in D3b.

• Each bot's DesktopState, kept in memory:

  • the window it read or acted in last (id, title, app);
  • its last action, as a structured DesktopAction (click, type, select or scroll, the control's name, the option), so the app can word it in the owner's language;
  • when it happened;
  • whether the owner stopped it.

  Every reading, picture and action updates it and sends desktop.changed to all apps.

• Live picture. desktop.watch {botId} returns the state and the newest picture.

  • While any connection watches that bot, its window is pictured 4 times a second (PrintWindow).
  • A desktop.frame goes out only when the picture changed, and only to that connection, newest first, like the browser's frames.
  • Picturing stops with the last watcher (desktop.unwatch, another desktop.watch, or the connection closing).

• Stopping a bot. desktop.stop and desktop.resume mark a bot as stopped or let it go on.

  • A stopped bot's desktop tools refuse at the start, and again after waiting for the owner's answer or for their turn on the desktop.
  • An action already under way ends within its 5 seconds.

• Ctrl+Alt+End. The daemon registers it on startup, on a thread with its own message queue, and it stops every bot on the desktop even with the app closed. If another program holds the shortcut (a dev daemon, for example), the daemon logs a warning and goes on without it.

• Time limits. Every UI Automation session, reading too, now has 5 seconds per call and to connect, and a reading stops at 15 seconds with what it has.

  • Seen on Windows 11: right after a classic combo box took a choice, the window answered slowly, and the next reading, with no limit, hung for minutes.
  • The choice and scroll test waits up to 2 seconds for the window to show them, as the tool does. 12 runs in a row passed.

• Housekeeping:

  • the desktop tests' window and setup moved to tests/common/desktop.rs;
  • the event-to-notification mapping moved to rpc/notify.rs, since rpc/mod.rs had reached 300 lines;
  • FakeBotloft handles the new methods.

• Spec 24.5, 24.9 and 24.12 are updated.

Checks

• cargo fmt, clippy -D warnings and pnpm check pass. cargo test --workspace passed once the time limits were in.
• New real-window tests in tests/desktop_panel.rs:
  • watching first shows no window;
  • after a reading, desktop.changed carries the window and a desktop.frame arrives;
  • stop refuses the tools and resume lets them through;
  • stop_all, which the shortcut calls, stops everyone.
• Pressing the shortcut for real is a manual check, since a test would send keys to the owner's machine.

Manual check on the real app

With a bot using a window, press Ctrl+Alt+End. The bot's next desktop tool should say the owner stopped it.