diff --git a/command-center-ai/CHANGELOG.md b/command-center-ai/CHANGELOG.md index 789d0ee6..02dc7bcd 100644 --- a/command-center-ai/CHANGELOG.md +++ b/command-center-ai/CHANGELOG.md @@ -1,5 +1,22 @@ # Changelog +## Unreleased + +Compatibility axes: additive npm API on `AgentIcon` (`activity`), on `CommandCenterAgentIcon` +(optional `poses` and `toolPoses`), on `ChatRunStatusValue` (optional `currentTool`), and new +exports `resolveAgentIconPose`, `AGENT_ICON_RUN_STATUS_POSES` and the `AgentIconActivity` and +`AgentIconToolSignal` types; the read of the platform's additive `commandCenterIcon.poses` and +`toolPoses` and of its pose route; and the content of the packaged skill +`design-agent-conversation-capabilities`. No stylesheet class, browser storage key, skill name or +path, installer behavior, or SDK peer range changes. An agent without poses renders as before. + +- **The working agent shows what it is doing.** When the platform declares poses for an agent + (the platform's ADR-033, amended 2026-10-16), the thread avatar of the agent that is working and + the rail header show the pose for the application whose tool runs, otherwise its `working`, + `thinking` or `error` pose, and the idle icon when a pose is missing. The thread reads the + latest tool call from the message; the rail reads the engine's new `currentTool`. Poses load + ahead where icons animate, and reduced motion shows the chosen pose still. + ## 0.0.14 Compatibility axes: additive npm API on `ChatRailProps` (`resizable`, `width`, `defaultWidth`, diff --git a/command-center-ai/agent_scaffold/skills/contracts/design-agent-conversation-capabilities/SKILL.md b/command-center-ai/agent_scaffold/skills/contracts/design-agent-conversation-capabilities/SKILL.md index a64d0a59..ccec8299 100644 --- a/command-center-ai/agent_scaffold/skills/contracts/design-agent-conversation-capabilities/SKILL.md +++ b/command-center-ai/agent_scaffold/skills/contracts/design-agent-conversation-capabilities/SKILL.md @@ -74,7 +74,9 @@ answers so the text and the tool calls tell the story on their own. theme-following mask or a colour image, with a robot as the fallback. A colour icon may be an animated pixel-art sprite: it moves only in the rail header, the connecting stage and while its Agent works, shows its first frame everywhere else and under reduced motion, and keeps sharp - pixels when the platform marks it pixelated. During a live turn the avatar + pixels when the platform marks it pixelated. While the Agent works, those places show the pose + the Agent declares for the application whose tool runs, or its `working`, `thinking` or `error` + pose. During a live turn the avatar is the robot until the next reload; that is reported to the package's owner. ## The Session Around The Answers diff --git a/command-center-ai/docs/adr/adr-090-agent-icons-in-command-center-surfaces.md b/command-center-ai/docs/adr/adr-090-agent-icons-in-command-center-surfaces.md index ff98b2c3..5f1961c7 100644 --- a/command-center-ai/docs/adr/adr-090-agent-icons-in-command-center-surfaces.md +++ b/command-center-ai/docs/adr/adr-090-agent-icons-in-command-center-surfaces.md @@ -5,6 +5,8 @@ - Amended: 2026-09-16 for the unified Agent shortcut - Amended: 2026-10-16 for animated and pixel-art icons (section 6), following the platform's ADR-033 amendment of the same day +- Amended: 2026-10-16 for poses that follow the agent's run and tools (section 7), following the + platform's per-tool animations - Moved: 2026-09-23 into the chat package with its code (ADR 096, step 4); the code paths below are the package's - Related: @@ -127,6 +129,28 @@ frame 0 is its still, and may mark a colour icon `imageRendering: "pixelated"`. - **Pixel art stays sharp.** A colour icon marked `pixelated` is drawn with CSS `image-rendering: pixelated`. Mask icons are always smooth. +### 7. Poses that follow the run and its tools + +Amended 2026-10-16. The projection may carry, per agent, `poses` (pose name to its delivery URL) +and `toolPoses` (a declared application to a pose); the platform delivers a pose at +`/api/v1/command-center/agents//icon//`. Three names also follow the run: +`thinking`, `working` and `error`. + +- **Where.** Only the places that animate follow the run: the thread avatar of the agent that is + working and the rail header. `AgentIcon` takes `activity` (`thinking`, the latest tool and its + phase, a failed run); every other place keeps the idle icon's frame 0. +- **Which pose** (`resolveAgentIconPose`, `src/engine/agent-icon-pose.ts`): while a tool runs, the + rule of its application (`__call_tool` or `__list_tools`), otherwise `working`; + after a failed tool or run, `error`; while the agent works, `thinking`; otherwise, or when the + pose is not declared, the idle icon. +- **Signals.** The thread reads the latest tool call of the active message from its parts (as the + working preview already does). The rail header reads `useChatRunStatus().currentTool`, which + the engine follows from the stream's `tool-call-start` and `tool-result` chunks; it is null + outside a run and optional on the public type. +- **Loading.** A place that animates loads the agent's poses ahead of use through the same byte + cache; until a pose's bytes are ready the idle icon stays. Each pose gets its frame 0 still, so + reduced motion shows the chosen pose still. + ## Non-goals - Uploading or editing icons from the Command Center; the repository workflow owns that. @@ -147,6 +171,10 @@ frame 0 is its still, and may mark a colour icon `imageRendering: "pixelated"`. scaling. - `src/engine/apng-still.ts`: frame 0 of an APNG as a static PNG; tested with the platform's own sanitizer output beside the cache tests. +- `src/engine/agent-icon-pose.ts`: the pose resolver; `AgentIcon` (`activity`), `ChatThread` + (the working agent), `ChatRail` (the header) and the engine's `currentTool`; tested for the + resolver, the projection's poses, pose choice and loading, and a thread whose running + application tool shows its pose. - `ChatEngineProvider` mounts the provider and exposes `activeAgentUid`; the package's `ChatThread.tsx` (`ActorAvatar`, the connecting stage prop) and `AgentConnectingState.tsx` use the component, and so do the application's own agent surfaces in section 4. diff --git a/command-center-ai/src/backend/command-center-agent-icons-api.test.ts b/command-center-ai/src/backend/command-center-agent-icons-api.test.ts index 6c09f3dd..3858c263 100644 --- a/command-center-ai/src/backend/command-center-agent-icons-api.test.ts +++ b/command-center-ai/src/backend/command-center-agent-icons-api.test.ts @@ -35,11 +35,29 @@ describe("command center agent icons api", () => { rendering: "mask", imageRendering: "auto", url: "https://api.test/icon/a1/", + poses: {}, + toolPoses: {}, }); expect(icons.get("a5")?.rendering).toBe("color"); expect(normalizeCommandCenterAgentIcons({ not: "an array" }).size).toBe(0); }); + it("reads pose urls and keeps only tool rules whose pose exists", () => { + const icon = normalizeCommandCenterAgentIcons([ + { + agentUid: "a1", + commandCenterIcon: { + rendering: "color", + url: "https://api.test/icon/", + poses: { reading: "https://api.test/icon/reading/", broken: 7 }, + toolPoses: { tutorial_api: "reading", orders: "missing" }, + }, + }, + ]).get("a1"); + expect(icon?.poses).toEqual({ reading: "https://api.test/icon/reading/" }); + expect(icon?.toolPoses).toEqual({ tutorial_api: "reading" }); + }); + it("keeps pixelated scaling only for colour icons", () => { const icons = normalizeCommandCenterAgentIcons([ { agentUid: "pixel", commandCenterIcon: { rendering: "color", imageRendering: "pixelated", url: "https://api.test/p/" } }, diff --git a/command-center-ai/src/backend/command-center-agent-icons-api.ts b/command-center-ai/src/backend/command-center-agent-icons-api.ts index c7a53b35..e15db868 100644 --- a/command-center-ai/src/backend/command-center-agent-icons-api.ts +++ b/command-center-ai/src/backend/command-center-agent-icons-api.ts @@ -24,6 +24,10 @@ export interface CommandCenterAgentIcon { /** `pixelated` only for color icons; absent or `auto` scales smoothly. */ imageRendering?: CommandCenterAgentIconImageRendering; url: string; + /** Delivery URL of each declared pose sprite, keyed by pose name. */ + poses?: Readonly>; + /** The pose a declared application's tools show while they run, keyed by application. */ + toolPoses?: Readonly>; } export type CommandCenterAgentIconMap = ReadonlyMap; @@ -41,6 +45,19 @@ function readString(value: unknown) { return typeof value === "string" && value.trim() ? value.trim() : null; } +function readStringMap(value: unknown) { + const entries: Record = {}; + if (value && typeof value === "object" && !Array.isArray(value)) { + for (const [key, entry] of Object.entries(value as Record)) { + const text = readString(entry); + if (key && text) { + entries[key] = text; + } + } + } + return entries; +} + export function normalizeCommandCenterAgentIcons(payload: unknown): Map { const icons = new Map(); if (!Array.isArray(payload)) { @@ -66,7 +83,11 @@ export function normalizeCommandCenterAgentIcons(payload: unknown): Map pose in poses), + ); + icons.set(agentUid, { agentUid, rendering, imageRendering, url, poses, toolPoses }); } return icons; } diff --git a/command-center-ai/src/engine/ChatEngineProvider.tsx b/command-center-ai/src/engine/ChatEngineProvider.tsx index 37ae18e0..698a34eb 100644 --- a/command-center-ai/src/engine/ChatEngineProvider.tsx +++ b/command-center-ai/src/engine/ChatEngineProvider.tsx @@ -60,6 +60,7 @@ import { cancelChatSession } from "../backend/session-cancel-api.js"; import { fetchSessionHistory } from "../backend/session-history-api.js"; import { describeStreamedToolCallStatus, + describeToolActivity, followStreamedToolCall, type StreamedToolCall, } from "../backend/tool-activity.js"; @@ -73,6 +74,7 @@ import { type AgentSessionDetailSnapshot, } from "../session-detail/model.js"; import { useAgentSessionDetail } from "../session-detail/useAgentSessionDetail.js"; +import type { AgentIconToolSignal } from "./agent-icon-pose.js"; import { AgentIconsProvider } from "./agent-icons-context.js"; import { ChatEngineContext, ChatRunStatusContext } from "./engine-context.js"; import { LocalChatEngineProvider, type LocalChatEngineProviderProps } from "./LocalChatEngineProvider.js"; @@ -280,6 +282,11 @@ interface SessionRuntimeAccessUiMeta { // useChatRunStatus(). export interface ChatRunStatusValue { hasVisibleAssistantOutput: boolean; + /** + * The tool the running turn uses, from the stream: the declared application whose tool runs, + * if any, and its phase. Null outside a run; absent from engines that do not track it. + */ + currentTool?: AgentIconToolSignal | null; runStatus: ChatRunStatus; runStatusDetail: string | null; thinkingSummary: string | null; @@ -729,6 +736,7 @@ function PlatformChatEngineProvider({ const [currentSessionId, setCurrentSessionId] = useState(null); const [runStatus, setRunStatus] = useState("idle"); const [runStatusDetail, setRunStatusDetail] = useState(null); + const [currentTool, setCurrentTool] = useState(null); const [thinkingSummary, setThinkingSummary] = useState(null); const [availableModels, setAvailableModels] = useState([]); const [availableModelsError, setAvailableModelsError] = useState(null); @@ -3646,6 +3654,7 @@ function PlatformChatEngineProvider({ onRequestStart: async () => { const queuedSend = queueSendingRef.current; setRunStatus("queued"); + setCurrentTool(null); setRunStatusDetail( queuedSend ? `Sending queued message ${queuedSend.position} of ${queuedSend.total}` @@ -3754,6 +3763,12 @@ function PlatformChatEngineProvider({ return; } const status = describeStreamedToolCallStatus(call); + const application = describeToolActivity(call.toolName, undefined, null).application?.name ?? null; + setCurrentTool((previous) => + previous?.phase === "running" && previous.application === application + ? previous + : { application, phase: "running" }, + ); setRunStatus("thinking"); setRunStatusDetail(status); setThinkingSummary(status); @@ -3761,6 +3776,13 @@ function PlatformChatEngineProvider({ return; } if (type === "tool-result") { + const details = + data.result && typeof data.result === "object" && "details" in data.result + ? (data.result as { details?: { is_error?: unknown; failure?: unknown } }).details + : null; + const failed = + data.isError === true || details?.is_error === true || details?.failure === "timeout"; + setCurrentTool((previous) => (previous ? { ...previous, phase: failed ? "failed" : "done" } : previous)); setRunStatus("thinking"); setRunStatusDetail("Thinking..."); setThinkingSummary(null); @@ -4444,12 +4466,16 @@ function PlatformChatEngineProvider({ ); const runStatusValue = useMemo( () => ({ + currentTool: + runStatus === "queued" || runStatus === "thinking" || runStatus === "responding" + ? currentTool + : null, hasVisibleAssistantOutput, runStatus, runStatusDetail, thinkingSummary, }), - [hasVisibleAssistantOutput, runStatus, runStatusDetail, thinkingSummary], + [currentTool, hasVisibleAssistantOutput, runStatus, runStatusDetail, thinkingSummary], ); return ( diff --git a/command-center-ai/src/engine/agent-icon-pose.test.ts b/command-center-ai/src/engine/agent-icon-pose.test.ts new file mode 100644 index 00000000..ac68c8e8 --- /dev/null +++ b/command-center-ai/src/engine/agent-icon-pose.test.ts @@ -0,0 +1,61 @@ +import { describe, expect, it } from "vitest"; + +import type { CommandCenterAgentIcon } from "../backend/command-center-agent-icons-api.js"; +import { decodeToolSignal, encodeToolSignal, resolveAgentIconPose } from "./agent-icon-pose.js"; + +const icon: CommandCenterAgentIcon = { + agentUid: "a1", + rendering: "color", + imageRendering: "pixelated", + url: "https://api.test/icon/", + poses: { + thinking: "https://api.test/icon/thinking/", + working: "https://api.test/icon/working/", + error: "https://api.test/icon/error/", + reading: "https://api.test/icon/reading/", + }, + toolPoses: { tutorial_api: "reading" }, +}; + +describe("resolveAgentIconPose", () => { + it("shows the application's pose while its tool runs, and working for other tools", () => { + expect(resolveAgentIconPose(icon, { thinking: true, tool: { application: "tutorial_api", phase: "running" } })).toBe( + "reading", + ); + expect(resolveAgentIconPose(icon, { thinking: true, tool: { application: "orders", phase: "running" } })).toBe( + "working", + ); + expect(resolveAgentIconPose(icon, { thinking: true, tool: { application: null, phase: "running" } })).toBe( + "working", + ); + }); + + it("follows the run: error after a failed tool or run, thinking otherwise, idle when not working", () => { + expect(resolveAgentIconPose(icon, { thinking: true, tool: { application: "tutorial_api", phase: "failed" } })).toBe( + "error", + ); + expect(resolveAgentIconPose(icon, { thinking: false, tool: null, failed: true })).toBe("error"); + expect(resolveAgentIconPose(icon, { thinking: true, tool: { application: "tutorial_api", phase: "done" } })).toBe( + "thinking", + ); + expect(resolveAgentIconPose(icon, null)).toBeNull(); + expect(resolveAgentIconPose(icon, { thinking: false, tool: null })).toBeNull(); + }); + + it("falls back to idle when a pose is not declared", () => { + const bare: CommandCenterAgentIcon = { agentUid: "a2", rendering: "color", url: "https://api.test/icon/" }; + expect(resolveAgentIconPose(bare, { thinking: true, tool: { application: "tutorial_api", phase: "running" } })).toBeNull(); + expect(resolveAgentIconPose(bare, { thinking: true, tool: null })).toBeNull(); + }); + + it("round-trips tool signals through their string form", () => { + const signal = { application: "tutorial_api", phase: "running" } as const; + expect(decodeToolSignal(encodeToolSignal(signal))).toEqual(signal); + expect(decodeToolSignal(encodeToolSignal({ application: null, phase: "failed" }))).toEqual({ + application: null, + phase: "failed", + }); + expect(decodeToolSignal("")).toBeNull(); + expect(decodeToolSignal("waiting:x")).toBeNull(); + }); +}); diff --git a/command-center-ai/src/engine/agent-icon-pose.ts b/command-center-ai/src/engine/agent-icon-pose.ts new file mode 100644 index 00000000..ad73cd9d --- /dev/null +++ b/command-center-ai/src/engine/agent-icon-pose.ts @@ -0,0 +1,75 @@ +import type { CommandCenterAgentIcon } from "../backend/command-center-agent-icons-api.js"; + +/** + * Which pose an agent's icon shows while it works (ADR 090, amended 2026-10-16). + * + * The platform delivers, per agent, pose sprites by name and rules mapping a declared + * application to a pose. Three pose names also follow the run status. + */ +export const AGENT_ICON_RUN_STATUS_POSES = { + thinking: "thinking", + working: "working", + error: "error", +} as const; + +/** The tool an agent is using, as the thread or the stream sees it. */ +export interface AgentIconToolSignal { + /** The declared application whose tool runs (`__call_tool`), or null. */ + application: string | null; + phase: "running" | "done" | "failed"; +} + +/** What the agent is doing now; absent when the place does not follow the run. */ +export interface AgentIconActivity { + /** The agent is working on a turn. */ + thinking: boolean; + /** The latest tool call of that turn, or null. */ + tool: AgentIconToolSignal | null; + /** The run failed. */ + failed?: boolean; +} + +function declared(icon: CommandCenterAgentIcon, pose: string | null | undefined) { + return pose && icon.poses?.[pose] ? pose : null; +} + +/** + * The pose to draw, or null for the idle icon: the application's rule while its tool runs, + * otherwise `working` while any tool runs, `error` after a failed tool or run, `thinking` while + * the agent works, and idle when the pose is not declared. + */ +export function resolveAgentIconPose( + icon: CommandCenterAgentIcon, + activity: AgentIconActivity | null | undefined, +): string | null { + if (!activity) { + return null; + } + const { tool } = activity; + if (tool?.phase === "running") { + const rule = tool.application ? icon.toolPoses?.[tool.application] : null; + return declared(icon, rule) ?? declared(icon, AGENT_ICON_RUN_STATUS_POSES.working); + } + if (tool?.phase === "failed" || activity.failed) { + return declared(icon, AGENT_ICON_RUN_STATUS_POSES.error); + } + return activity.thinking ? declared(icon, AGENT_ICON_RUN_STATUS_POSES.thinking) : null; +} + +/** A tool signal as a string, so selectors can return a stable value. */ +export function encodeToolSignal(signal: AgentIconToolSignal | null): string { + return signal ? `${signal.phase}:${signal.application ?? ""}` : ""; +} + +export function decodeToolSignal(value: string): AgentIconToolSignal | null { + if (!value) { + return null; + } + const separator = value.indexOf(":"); + const phase = value.slice(0, separator); + const application = value.slice(separator + 1); + if (phase !== "running" && phase !== "done" && phase !== "failed") { + return null; + } + return { application: application || null, phase }; +} diff --git a/command-center-ai/src/index.ts b/command-center-ai/src/index.ts index e93ec2b4..1ba8affd 100644 --- a/command-center-ai/src/index.ts +++ b/command-center-ai/src/index.ts @@ -273,6 +273,12 @@ export { type AgentConnectionStep, } from "./ui/AgentConnectingState.js"; export { AgentIcon } from "./ui/AgentIcon.js"; +export { + AGENT_ICON_RUN_STATUS_POSES, + resolveAgentIconPose, + type AgentIconActivity, + type AgentIconToolSignal, +} from "./engine/agent-icon-pose.js"; // Model provider settings (ADR 096, step 5) export { diff --git a/command-center-ai/src/ui/AgentIcon.client.test.tsx b/command-center-ai/src/ui/AgentIcon.client.test.tsx index 4483045a..3daff996 100644 --- a/command-center-ai/src/ui/AgentIcon.client.test.tsx +++ b/command-center-ai/src/ui/AgentIcon.client.test.tsx @@ -6,6 +6,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { type CommandCenterAgentIcon } from "../backend/command-center-agent-icons-api.js"; import { createChatBackendConnection } from "../backend/connection.js"; import { clearAgentIconCache } from "../engine/agent-icon-cache.js"; +import type { AgentIconActivity } from "../engine/agent-icon-pose.js"; import { AgentIconAuthContext, type AgentIconLookup, AgentIconsContext } from "../engine/agent-icons-context.js"; // What the chat engine's icon provider supplies: the platform connection and the person's token. @@ -23,6 +24,13 @@ const APNG = Uint8Array.from(atob("iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0A const icons: Record = { masked: { agentUid: "masked", rendering: "mask", url: "https://api.test/agents/masked/icon/" }, colored: { agentUid: "colored", rendering: "color", url: "https://api.test/agents/colored/icon/" }, + posing: { + agentUid: "posing", + rendering: "color", + url: "https://api.test/agents/posing/icon/", + poses: { reading: "https://api.test/agents/posing/icon/reading/" }, + toolPoses: { tutorial_api: "reading" }, + }, animated: { agentUid: "animated", rendering: "color", @@ -61,9 +69,20 @@ describe("AgentIcon", () => { clearAgentIconCache(); vi.unstubAllGlobals(); }); - function render(agentUid: string | null, withProvider = true, motion?: "still" | "animate") { + function render( + agentUid: string | null, + withProvider = true, + motion?: "still" | "animate", + activity?: AgentIconActivity | null, + ) { const element = ( - } motion={motion} /> + } + motion={motion} + activity={activity} + /> ); act(() => { root.render( @@ -164,4 +183,32 @@ describe("AgentIcon", () => { expect(image.dataset.agentIconMotion).toBeUndefined(); expect(image.style.imageRendering).toBe(""); }); + + describe("poses", () => { + const requested = () => fetchMock.mock.calls.map(([input]) => String(input)); + + it("shows the pose of the application whose tool runs", async () => { + render("posing", true, "animate", { + thinking: true, + tool: { application: "tutorial_api", phase: "running" }, + }); + await vi.waitFor(() => + expect(container.querySelector('img[data-agent-icon-pose="reading"]')).not.toBeNull(), + ); + expect(requested()).toContain("https://api.test/agents/posing/icon/reading/"); + }); + + it("loads the poses ahead where it animates, and keeps the idle icon without activity", async () => { + render("posing", true, "animate"); + await vi.waitFor(() => expect(requested()).toContain("https://api.test/agents/posing/icon/reading/")); + await vi.waitFor(() => expect(container.querySelector('img[data-agent-icon="color"]')).not.toBeNull()); + expect(container.querySelector("[data-agent-icon-pose]")).toBeNull(); + }); + + it("loads no pose where it stays still", async () => { + render("posing"); + await vi.waitFor(() => expect(container.querySelector('img[data-agent-icon="color"]')).not.toBeNull()); + expect(requested()).toEqual(["https://api.test/agents/posing/icon/"]); + }); + }); }); diff --git a/command-center-ai/src/ui/AgentIcon.tsx b/command-center-ai/src/ui/AgentIcon.tsx index 7646e5ec..2816fc3c 100644 --- a/command-center-ai/src/ui/AgentIcon.tsx +++ b/command-center-ai/src/ui/AgentIcon.tsx @@ -1,6 +1,8 @@ -import type { ReactNode } from "react"; +import { useEffect, type ReactNode } from "react"; import { useCommandCenterViewport } from "@dev-mainsequence/command-center-sdk/layout"; +import { ensureAgentIcon } from "../engine/agent-icon-cache.js"; +import { resolveAgentIconPose, type AgentIconActivity } from "../engine/agent-icon-pose.js"; import { useAgentIconAuth, useAgentIconImage, @@ -22,22 +24,42 @@ import { cx } from "./class-names.js"; * large single places pass (the rail header, the connecting stage, the agent that is working), * and always shows the still when the person asks for reduced motion. `imageRendering: * "pixelated"` keeps pixel art sharp when it is scaled. + * + * `activity` says what the agent is doing; with it the icon shows the matching pose sprite (the + * application's tool rule, `working`, `thinking`, `error`) and falls back to the idle icon. A + * place that animates also loads the agent's poses ahead of use. */ export function AgentIcon({ agentUid, className, fallback, motion = "still", + activity = null, }: { agentUid?: string | null; className?: string; fallback: ReactNode; motion?: "still" | "animate"; + activity?: AgentIconActivity | null; }) { const lookup = useAgentIconLookup(); const icon = lookup(agentUid); const auth = useAgentIconAuth(); - const { objectUrl, stillObjectUrl } = useAgentIconImage(icon?.url ?? null, auth); + const pose = icon ? resolveAgentIconPose(icon, activity) : null; + const poseUrl = pose ? icon?.poses?.[pose] ?? null : null; + const idle = useAgentIconImage(icon?.url ?? null, auth); + const posed = useAgentIconImage(poseUrl, auth); + // A pose still loading keeps the idle icon in place. + const { objectUrl, stillObjectUrl } = posed.objectUrl ? posed : idle; + const poseUrls = motion === "animate" && icon?.poses ? Object.values(icon.poses).join("\n") : ""; + useEffect(() => { + if (!poseUrls || !auth?.connection) { + return; + } + for (const url of poseUrls.split("\n")) { + ensureAgentIcon({ connection: auth.connection, url, token: auth.token, tokenType: auth.tokenType }); + } + }, [auth, poseUrls]); const { reducedMotion } = useCommandCenterViewport(); if (!icon || !objectUrl) { return <>{fallback}; @@ -70,6 +92,7 @@ export function AgentIcon({ className={cx("ms-chat-agent-icon ms-chat-agent-icon--image", className)} data-agent-icon="color" data-agent-icon-motion={stillObjectUrl ? (animates ? "animated" : "still") : undefined} + data-agent-icon-pose={posed.objectUrl ? pose ?? undefined : undefined} draggable={false} src={stillObjectUrl && !animates ? stillObjectUrl : objectUrl} style={icon.imageRendering === "pixelated" ? { imageRendering: "pixelated" } : undefined} diff --git a/command-center-ai/src/ui/ChatRail.tsx b/command-center-ai/src/ui/ChatRail.tsx index 0da8ee2d..cfb5b970 100644 --- a/command-center-ai/src/ui/ChatRail.tsx +++ b/command-center-ai/src/ui/ChatRail.tsx @@ -1,9 +1,11 @@ -import { useId, useRef, type CSSProperties, type ReactNode } from "react"; +import { useContext, useId, useMemo, useRef, type CSSProperties, type ReactNode } from "react"; import { Expand, Sparkles, X } from "lucide-react"; import { Button } from "@dev-mainsequence/command-center-sdk/controls"; +import type { AgentIconActivity } from "../engine/agent-icon-pose.js"; import { useOptionalChatEngine } from "../engine/ChatEngineProvider.js"; +import { ChatRunStatusContext } from "../engine/engine-context.js"; import { AgentIcon } from "./AgentIcon.js"; import { cx } from "./class-names.js"; import { useChatRailResize } from "./useChatRailResize.js"; @@ -83,6 +85,19 @@ export function ChatRail({ }: ChatRailProps) { const engine = useOptionalChatEngine(); const railAgentUid = agentUid === undefined ? (engine?.activeAgentUid ?? null) : agentUid; + // Outside an engine there is no run, so the header shows the idle icon. + const runStatus = useContext(ChatRunStatusContext); + const status = runStatus?.runStatus; + const currentTool = runStatus?.currentTool ?? null; + const railActivity = useMemo( + () => + status === "queued" || status === "thinking" || status === "responding" + ? { thinking: true, tool: currentTool } + : status === "error" + ? { thinking: false, tool: null, failed: true } + : null, + [currentTool, status], + ); const overlay = mode === "overlay"; const railRef = useRef(null); const railId = useId(); @@ -148,6 +163,7 @@ export function ChatRail({ agentUid={railAgentUid} className="ms-chat-rail__tile-icon" motion="animate" + activity={railActivity} fallback={icon ?? } /> diff --git a/command-center-ai/src/ui/ChatThread.icons.client.test.tsx b/command-center-ai/src/ui/ChatThread.icons.client.test.tsx index 31feffbe..057994e9 100644 --- a/command-center-ai/src/ui/ChatThread.icons.client.test.tsx +++ b/command-center-ai/src/ui/ChatThread.icons.client.test.tsx @@ -133,16 +133,24 @@ const iconAuth = { tokenType: "Bearer", }; -function Harness({ messages }: { messages: ThreadMessageLike[] }) { +function Harness({ + iconLookup = lookup, + isRunning = false, + messages, +}: { + iconLookup?: AgentIconLookup; + isRunning?: boolean; + messages: ThreadMessageLike[]; +}) { const runtime = useExternalStoreRuntime({ messages, - isRunning: false, + isRunning, onNew: async () => {}, convertMessage: (message) => message, }); return ( - + @@ -226,4 +234,46 @@ describe("ChatThread agent icons", () => { const viewerTurn = roots().find((element) => element.dataset.actorKey === `user:${VIEWER_UID}`)!; expect(viewerTurn.querySelector("[data-agent-icon]")).toBeNull(); }); + + it("draws the working agent in the pose its running application tool declares", async () => { + const posingLookup: AgentIconLookup = (uid) => + uid === SESSION_AGENT_UID + ? { + agentUid: SESSION_AGENT_UID, + rendering: "color", + url: "https://api.test/agents/session/icon/", + poses: { reading: "https://api.test/agents/session/icon/reading/" }, + toolPoses: { tutorial_api: "reading" }, + } + : null; + const running = [ + thread[2], + withProvenance( + { + id: "a_2", + role: "assistant", + content: [ + { + type: "tool-call" as const, + toolCallId: "call_1", + toolName: "tutorial_api__call_tool", + args: { tool: "search_docs", arguments: { query: "jobs" } }, + }, + ], + status: { type: "running" }, + } as ThreadMessageLike, + { origin: "agent", channel: "command-center", targetAgentUid: SESSION_AGENT_UID }, + ), + ]; + act(() => { + root.render(); + }); + await vi.waitFor(() => + expect(container.querySelector('img[data-agent-icon-pose="reading"]')).not.toBeNull(), + ); + expect(fetchMock.mock.calls.map(([input]) => String(input))).toContain( + "https://api.test/agents/session/icon/reading/", + ); + }); }); + diff --git a/command-center-ai/src/ui/ChatThread.tsx b/command-center-ai/src/ui/ChatThread.tsx index 812c6ebb..b11ad733 100644 --- a/command-center-ai/src/ui/ChatThread.tsx +++ b/command-center-ai/src/ui/ChatThread.tsx @@ -92,6 +92,11 @@ import { type AgentRuntimeStatus, } from "./AgentConnectingState.js"; import { AgentIcon } from "./AgentIcon.js"; +import { + decodeToolSignal, + encodeToolSignal, + type AgentIconActivity, +} from "../engine/agent-icon-pose.js"; import { AssistantErrorRetry, AssistantMessageActions, @@ -342,6 +347,21 @@ function getToolPhase(part: { return "done"; } +/** The latest tool call of a message as an encoded signal, for the agent's pose. */ +function getLatestToolSignal(parts: ReadonlyArray): string { + for (let index = parts.length - 1; index >= 0; index -= 1) { + const part = parts[index]; + if (part.type === "tool-call" && typeof part.toolName === "string" && part.toolName.trim()) { + const activity = describeToolActivity(part.toolName, part.result, part.args); + return encodeToolSignal({ + application: activity.application?.name ?? null, + phase: getToolPhase(part), + }); + } + } + return ""; +} + function getThinkingPreview(parts: ReadonlyArray) { for (let index = parts.length - 1; index >= 0; index -= 1) { const part = parts[index]; @@ -606,12 +626,15 @@ function getActorAvatarVariant(actor: MessageActor, context: MessageActorContext function ActorAvatar({ actor, active = false, + activity = null, className, size = "md", variant, }: { actor: MessageActor; active?: boolean; + /** What the agent is doing while active, for its pose. */ + activity?: AgentIconActivity | null; className?: string; size?: "md" | "sm"; variant: ActorAvatarVariant; @@ -651,6 +674,7 @@ function ActorAvatar({ agentUid={actor.uid} className={cx(size === "md" ? "ms-chat-icon-lg" : "ms-chat-icon-xs", active && "ms-chat-pulse")} motion={active ? "animate" : "still"} + activity={active ? activity : null} fallback={ + getLatestToolSignal(s.message.parts as ReadonlyArray), + ); + const agentActivity = useMemo( + () => (isActiveAssistant ? { thinking: true, tool: decodeToolSignal(toolSignal) } : null), + [isActiveAssistant, toolSignal], + ); return ( diff --git a/command-center-ai/src/ui/README.md b/command-center-ai/src/ui/README.md index 0cf68708..c12b20ad 100644 --- a/command-center-ai/src/ui/README.md +++ b/command-center-ai/src/ui/README.md @@ -35,7 +35,9 @@ The thread: ([ADR 090](../../docs/adr/adr-090-agent-icons-in-command-center-surfaces.md)). It reads the projection and the credentials from the engine's icon provider, so outside the engine it draws the fallback. An animated icon shows its first frame unless `motion="animate"` is passed, and - always under reduced motion; a pixel-art icon is drawn with sharp pixels. + always under reduced motion; a pixel-art icon is drawn with sharp pixels. With `activity` it shows + the agent's pose for what it is doing: its application tool's pose, `working`, `thinking` or + `error`. The frame Command Center draws around the thread, so every application looks like Command Center ([the rail guide](../../docs/rail-and-expanded-rail.md)):