feat(miniapps): operate live apps through semantic and DOM controls

docs/interactive-capabilities/README.md

@@ -1,9 +1,9 @@
 <!-- generated by scripts/generate-interactive-capabilities.mjs; do not edit -->
 # OpenBitFun 功能与设置目录 / OpenBitFun Features & Settings
 
-OpenBitFun Playbook 当前包含 **23 个功能**和 **21 个设置页**，共 **44 个**用户可理解的条目、**334 项**有源码证据的子能力。每个条目有独立 Markdown，并直接服务于说明书网站、OpenBitFun 全局搜索和 `OpenBitFunControl` 智能体工具。
+OpenBitFun Playbook 当前包含 **23 个功能**和 **21 个设置页**，共 **44 个**用户可理解的条目、**335 项**有源码证据的子能力。每个条目有独立 Markdown，并直接服务于说明书网站、OpenBitFun 全局搜索和 `OpenBitFunControl` 智能体工具。
 
-OpenBitFun Playbook currently contains **23 features**, **21 settings pages**, and **334** source-backed sub-capabilities across **44** user-facing entries. Every entry has its own Markdown page and directly powers the website, in-app global search, and the `OpenBitFunControl` agent tool.
+OpenBitFun Playbook currently contains **23 features**, **21 settings pages**, and **335** source-backed sub-capabilities across **44** user-facing entries. Every entry has its own Markdown page and directly powers the website, in-app global search, and the `OpenBitFunControl` agent tool.
 
 ## 唯一事实源 / Single source of truth
 
@@ -27,20 +27,20 @@ OpenBitFun Playbook currently contains **23 features**, **21 settings pages**, a
 - Generated per-item interaction audit: `docs/interactive-capabilities/technical/product-control-open-audit.json`
 - Generated low-level audit map: `docs/interactive-capabilities/technical/tauri-command-map.json`
 
-说明书、网站、搜索和智能体只看“功能 + 设置 + 子能力”。每项子能力都必须引用已注册 Tauri Command 或可解析的源码标记；这些证据不会进入公开目录。当前 **702** 个 Tauri 命令只用于实现覆盖审计。产品 UI 交互源码会在生成和检查时扫描并校验，但不会保存成随普通 UI 改动频繁变化的版本化快照。
+说明书、网站、搜索和智能体只看“功能 + 设置 + 子能力”。每项子能力都必须引用已注册 Tauri Command 或可解析的源码标记；这些证据不会进入公开目录。当前 **704** 个 Tauri 命令只用于实现覆盖审计。产品 UI 交互源码会在生成和检查时扫描并校验，但不会保存成随普通 UI 改动频繁变化的版本化快照。
 
-Docs, website, search, and agents see only features, settings, and documented sub-capabilities. Every sub-capability must reference a registered Tauri command or a resolvable source marker; evidence is stripped from public projections. The **702** Tauri commands remain implementation-audit evidence only. Product UI interaction sources are scanned and validated during generation and checks, but are not stored as a versioned snapshot that churns with ordinary UI changes.
+Docs, website, search, and agents see only features, settings, and documented sub-capabilities. Every sub-capability must reference a registered Tauri command or a resolvable source marker; evidence is stripped from public projections. The **704** Tauri commands remain implementation-audit evidence only. Product UI interaction sources are scanned and validated during generation and checks, but are not stored as a versioned snapshot that churns with ordinary UI changes.
 
 ## 控制边界 / Control boundary
 
-- 每个子能力都明确标记为直接控制、委托给专用智能体工具、需交互打开或不支持；“打开页面”不会再被统计成“智能体已控制”。当前覆盖：直接 **52**、委托 **66**、需交互 **216**、不支持 **0**。
+- 每个子能力都明确标记为直接控制、委托给专用智能体工具、需交互打开或不支持；“打开页面”不会再被统计成“智能体已控制”。当前覆盖：直接 **52**、委托 **67**、需交互 **216**、不支持 **0**。
 - 稳定行为声明为带 JSON 输入契约的 `operations` 或 `options`，并绑定原生产品控制 Provider；智能体不接触原始 Tauri Command。
-- `OpenBitFunControl list` 和 `search` 都返回带 `nextCursor` 的精简分页结果；目录可持续增长，不靠固定总量上限。完整目录和 334 项子能力都不会写入 system prompt。
+- `OpenBitFunControl list` 和 `search` 都返回带 `nextCursor` 的精简分页结果；目录可持续增长，不靠固定总量上限。完整目录和 335 项子能力都不会写入 system prompt。
 - 目录发现与契约读取不依赖 React 或可见窗口。普通配置型 option 统一由 Product Assembly 的共享 ConfigService 执行器读、写并回读，因此 Desktop、CLI 与 Headless 表面走同一份实现；只有宿主原生 operation/provider option 和界面导航按表面注册适配器，缺失时必须明确返回不可用，禁止静默回退本机。只读智能体只能发现和读取目录。
 
-- Every documented item is classified as direct control, delegated Agent control, interactive opening, or unsupported; opening a page is never counted as direct control. Current coverage is **52 direct**, **66 delegated**, **216 interactive**, and **0 unsupported**.
+- Every documented item is classified as direct control, delegated Agent control, interactive opening, or unsupported; opening a page is never counted as direct control. Current coverage is **52 direct**, **67 delegated**, **216 interactive**, and **0 unsupported**.
 - Stable behavior becomes a typed `operation` or `option` with a JSON input contract and a native product-control provider. Agents never receive raw Tauri commands.
-- `OpenBitFunControl list` and `search` return compact pages with a `nextCursor`; the catalog can grow without a fixed total-size ceiling. Neither the full catalog nor its 334 documented items enters the system prompt.
+- `OpenBitFunControl list` and `search` return compact pages with a `nextCursor`; the catalog can grow without a fixed total-size ceiling. Neither the full catalog nor its 335 documented items enters the system prompt.
 - Discovery and contract lookup do not depend on React or a visible window. Ordinary config-backed options are read, written, and read back by one Product Assembly ConfigService executor shared by Desktop, CLI, and headless surfaces. Only host-native operations/provider options and presentation routes install surface adapters; missing adapters return explicit unavailability without local fallback. Read-only agents may only discover and inspect entries.
 
 ## 防腐化门禁 / Anti-drift gates

docs/interactive-capabilities/capabilities.json

@@ -4,7 +4,7 @@
   "title": "OpenBitFun Playbook",
   "origin": "https://playbook.openbitfun.com",
   "source": "src/shared/interactive-capabilities/catalog.json",
-  "digest": "64bbd3571c458cdd0b67ba9337820e88f9b52d5edf4aefc8320874643dce7a77",
+  "digest": "bff6e24a5bc74bba63fcd2858d7ac58e028d6901e44aece40e5bd885f518f8af",
   "ownerDigest": "4e1ace270088acecbd333d2c8aced0547dddb0fd6bae778bec24f2b0fb48ea91",
   "searchAcceptance": [
     {
@@ -159,10 +159,10 @@
     "features": 23,
     "settings": 21,
     "userFacing": 44,
-    "documentedItems": 334,
+    "documentedItems": 335,
     "controlCoverage": {
       "direct": 52,
-      "delegated": 66,
+      "delegated": 67,
       "interactive": 216,
       "unsupported": 0
     }
@@ -5823,6 +5823,7 @@
         "versions",
         "agent-authoring",
         "agent-market-submit",
+        "agent-operate-app",
         "publishing"
       ],
       "kind": "query",
@@ -6769,6 +6770,51 @@
         "actionId": "surface.miniapps.open"
       }
     },
+    {
+      "id": "feature.miniapps:delegate:agent-operate-app",
+      "capabilityId": "feature.miniapps",
+      "itemIds": [
+        "agent-operate-app"
+      ],
+      "kind": "delegate",
+      "risk": "execute",
+      "executionHost": "workspaceHost",
+      "availability": {
+        "desktop": {
+          "available": true
+        },
+        "cli": {
+          "available": true
+        },
+        "peer": {
+          "available": true,
+          "requiredCapabilities": [
+            "product_control_v1"
+          ]
+        },
+        "remoteControl": {
+          "available": true
+        },
+        "detachedDispatch": {
+          "available": true
+        }
+      },
+      "inputSchema": {
+        "type": "object",
+        "additionalProperties": true
+      },
+      "outputSchema": {
+        "type": "object",
+        "additionalProperties": true
+      },
+      "delegateTools": [
+        "MiniAppBrowser"
+      ],
+      "presentationTarget": {
+        "kind": "action",
+        "actionId": "surface.miniapps.open"
+      }
+    },
     {
       "id": "feature.miniapps:open:publishing",
       "capabilityId": "feature.miniapps",
@@ -23038,6 +23084,25 @@
             ]
           }
         },
+        {
+          "id": "agent-operate-app",
+          "titleZh": "由智能体结合源码控制 Mini App",
+          "titleEn": "Let an agent control a Mini App using its source and live interface",
+          "control": {
+            "kind": "delegate",
+            "tools": [
+              "MiniAppBrowser"
+            ],
+            "workflowZh": [
+              "先用 OpenBitFunControl 的 get 查询 feature.miniapps，并按 schema 用 inspect-app 读取源码，理解业务函数、动作参数和状态；源码更新属于应用改造，不是日常操作的前置步骤",
+              "用 MiniAppBrowser 的 open 连接当前实例，优先 observe_bridge 读取业务状态、act_bridge 执行语义动作并核对结果，保留用户草稿，不改源码、不重载、不切换呈现。旧应用缺少接口时使用可用的 DOM/输入或说明限制；自动活动信息不能代替业务状态"
+            ],
+            "workflowEn": [
+              "Use OpenBitFunControl get for feature.miniapps, then inspect-app source to understand business functions, action parameters, and state. Source updates are app enhancements, not prerequisites for routine operation",
+              "Open the current instance with MiniAppBrowser; prefer observe_bridge for business state and act_bridge for semantic operations and result verification. Preserve drafts without editing source, reloading, or changing presentation. For legacy apps use available DOM/input controls or report the gap; activity is not business state"
+            ]
+          }
+        },
         {
           "id": "publishing",
           "titleZh": "撤回和审查市场发布，并可渲染幻灯片页面",
@@ -23383,6 +23448,8 @@
         "Let an agent initialize, author, and validate a Mini App",
         "在用户明确要求时由智能体提交已完成的 Mini App 到市场审核",
         "Let an agent submit a completed Mini App for market review when explicitly requested",
+        "由智能体结合源码控制 Mini App",
+        "Let an agent control a Mini App using its source and live interface",
         "撤回和审查市场发布，并可渲染幻灯片页面",
         "Withdraw and review market releases, and render slide pages",
         "打开 Mini Apps",
@@ -23408,7 +23475,12 @@
         "仅在用户明确要求对外提交时调用 PublishMiniApp，并提供已完成应用、许可与发布说明",
         "登录、校验或上传返回可恢复状态时，向用户展示具体下一步，不把待授权或待审核状态报告成发布成功",
         "Call PublishMiniApp only after an explicit request to submit externally, with the completed app, license, and release notes",
-        "When authentication, validation, or upload returns a resumable state, show the exact next step and never report pending authorization or review as published"
+        "When authentication, validation, or upload returns a resumable state, show the exact next step and never report pending authorization or review as published",
+        "MiniAppBrowser",
+        "先用 OpenBitFunControl 的 get 查询 feature.miniapps，并按 schema 用 inspect-app 读取源码，理解业务函数、动作参数和状态；源码更新属于应用改造，不是日常操作的前置步骤",
+        "用 MiniAppBrowser 的 open 连接当前实例，优先 observe_bridge 读取业务状态、act_bridge 执行语义动作并核对结果，保留用户草稿，不改源码、不重载、不切换呈现。旧应用缺少接口时使用可用的 DOM/输入或说明限制；自动活动信息不能代替业务状态",
+        "Use OpenBitFunControl get for feature.miniapps, then inspect-app source to understand business functions, action parameters, and state. Source updates are app enhancements, not prerequisites for routine operation",
+        "Open the current instance with MiniAppBrowser; prefer observe_bridge for business state and act_bridge for semantic operations and result verification. Preserve drafts without editing source, reloading, or changing presentation. For legacy apps use available DOM/input controls or report the gap; activity is not business state"
       ],
       "docsUrl": "https://playbook.openbitfun.com/capabilities/feature.miniapps/"
     },

docs/interactive-capabilities/capabilities/feature.miniapps.md

@@ -39,6 +39,8 @@ Install, create, and run lightweight apps inside OpenBitFun, combining UI, data,
   - Let an agent initialize, author, and validate a Mini App
 - **由专用智能体工具控制 / Delegated Agent tool** · `PublishMiniApp` · 在用户明确要求时由智能体提交已完成的 Mini App 到市场审核
   - Let an agent submit a completed Mini App for market review when explicitly requested
+- **由专用智能体工具控制 / Delegated Agent tool** · `MiniAppBrowser` · 由智能体结合源码控制 Mini App
+  - Let an agent control a Mini App using its source and live interface
 - **智能体可定位入口，需交互完成 / Agent opens; interaction required** · 撤回和审查市场发布，并可渲染幻灯片页面
   - Withdraw and review market releases, and render slide pages
 

docs/interactive-capabilities/technical/product-control-open-audit.json

@@ -1,7 +1,7 @@
 {
   "schemaVersion": 1,
   "generatedFrom": "src/shared/interactive-capabilities/catalog.json",
-  "catalogDigest": "64bbd3571c458cdd0b67ba9337820e88f9b52d5edf4aefc8320874643dce7a77",
+  "catalogDigest": "bff6e24a5bc74bba63fcd2858d7ac58e028d6901e44aece40e5bd885f518f8af",
   "count": 216,
   "reasonCounts": {
     "externalAuth": 4,

docs/interactive-capabilities/technical/tauri-command-map.json

@@ -1,13 +1,13 @@
 {
   "schemaVersion": 2,
   "generatedFrom": "src/shared/interactive-capabilities/catalog.json",
-  "catalogDigest": "64bbd3571c458cdd0b67ba9337820e88f9b52d5edf4aefc8320874643dce7a77",
-  "commandCount": 702,
+  "catalogDigest": "bff6e24a5bc74bba63fcd2858d7ac58e028d6901e44aece40e5bd885f518f8af",
+  "commandCount": 704,
   "coverage": {
-    "commandCount": 702,
+    "commandCount": 704,
     "documentedCommandCount": 645,
-    "implementationCommandCount": 57,
-    "implementationDigest": "503b48eebc37750e50e0427d861e884dfa353a81aae32236e4bc2d7e0f5310e2"
+    "implementationCommandCount": 59,
+    "implementationDigest": "0bce9fbda3e8b517e409aec124ca0f2c3b36a7954d996400d9ccf405194dc768"
   },
   "commands": [
     {
@@ -5877,6 +5877,20 @@
       "signature": "fn manage_dialog_queue( runtime: State<'_, DesktopRuntimeContext>, request: openbitfun_runtime_ports::DialogQueueRequest, ) -> Result<openbitfun_runtime_ports::DialogQueueSnapshot, String>",
       "remoteWorkspacePolicy": "RemoteRouted"
     },
+    {
+      "id": "mark_miniapp_web_surface_ready",
+      "moduleId": "miniapp_web_host",
+      "capabilityId": "feature.miniapps",
+      "capabilityIds": [
+        "feature.miniapps"
+      ],
+      "documentedItemIds": [],
+      "visibility": "implementation",
+      "rustPath": "miniapp_web_host::mark_miniapp_web_surface_ready",
+      "sourceFile": "src/apps/desktop/src/miniapp_web_host.rs",
+      "signature": "fn mark_miniapp_web_surface_ready(request: Option<MiniAppWebSurfaceReadyRequest>)",
+      "remoteWorkspacePolicy": "WorkspaceAgnostic"
+    },
     {
       "id": "mark_openbitfun_control_surface_ready",
       "moduleId": "openbitfun_control_host",
@@ -8006,6 +8020,20 @@
       "signature": "fn report_ide_control_result(request: IdeControlResultRequest) -> Result<(), String>",
       "remoteWorkspacePolicy": "LegacyUnaudited"
     },
+    {
+      "id": "report_miniapp_web_result",
+      "moduleId": "miniapp_web_host",
+      "capabilityId": "feature.miniapps",
+      "capabilityIds": [
+        "feature.miniapps"
+      ],
+      "documentedItemIds": [],
+      "visibility": "implementation",
+      "rustPath": "miniapp_web_host::report_miniapp_web_result",
+      "sourceFile": "src/apps/desktop/src/miniapp_web_host.rs",
+      "signature": "fn report_miniapp_web_result( request: ReportMiniAppWebResultRequest, ) -> Result<(), String>",
+      "remoteWorkspacePolicy": "WorkspaceAgnostic"
+    },
     {
       "id": "report_openbitfun_control_result",
       "moduleId": "openbitfun_control_host",

src/apps/desktop/src/lib.rs

@@ -35,6 +35,7 @@ mod macos_window_chrome;
 #[cfg(target_os = "macos")]
 mod macos_window_close;
 mod miniapp_desktop;
+mod miniapp_web_host;
 
 // Keep one macro expansion: on macOS it emits a process-wide Info.plist symbol.
 fn desktop_context() -> tauri::Context<tauri::Wry> {
@@ -1136,6 +1137,7 @@ pub async fn run() {
             openbitfun_webdriver::maybe_start(app_handle.clone());
             builtin_browser_host::install(app_handle.clone());
             openbitfun_control_host::install(app_handle.clone());
+            miniapp_web_host::install(app_handle.clone());
             if !miniapp_background {
                 if let Err(error) = desktop_cards::install(&app_handle) {
                     log::warn!("Desktop cards unavailable: {}", error);
@@ -1492,6 +1494,8 @@ pub async fn run() {
             openbitfun_control_host::mark_openbitfun_control_surface_unready,
             openbitfun_control_host::product_control_invoke,
             openbitfun_control_host::report_openbitfun_control_result,
+            miniapp_web_host::mark_miniapp_web_surface_ready,
+            miniapp_web_host::report_miniapp_web_result,
             get_health_status,
             get_statistics,
             test_ai_connection,