Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .memory/timed-ask-user.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Timed ask-user waits — 2026-09-27

Branch: `feature/timed-ask-user`. Plan: `docs/plans/timed-ask-user-plan.md`.

The desktop coordinator owns optional deadlines, `expiresAt`, explicit expired responses, and a bounded recent-prompt set. Attended desktop requests without a timeout continue waiting; unattended requests use the Remote-aligned cap. Timed-out prompts now remain in the desktop composer after generation settles, letting a late answer become a follow-up or queued follow-up.

The CLI adapter must forward the tool's `timeoutSeconds` to the terminal selection UI. The CLI uses one absolute deadline across the complete questionnaire and multi-select steps, passing each `select` only the remaining milliseconds plus the tool-call abort signal. Pi `ExtensionUIContext.select` supports both options. If multi-select exhausts every option, preserve the accumulated choices before leaving the loop.

Relevant validation: `npm run test:ask-user-question`; `npm run test:cli`; `npm --prefix packages/cli run type-check`.
6 changes: 6 additions & 0 deletions .papercuts/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -1407,6 +1407,12 @@ because their native file-mutator test binary had not been built. Run
## 2026-09-26 PR #121 merge of #251 (Remote contract revision 14)
- A PR that adds to the Remote contract has to renumber when main bumps `contractRevision`. The conflicts show up in 7 files: both fixtures, the TS/iOS/Android fixture assertions and the iOS fixture CodingKeys. After resolving, `cmp` the Android copy against the shared fixture. Plan docs that name the revision also go stale.

## 2026-09-27 timed ask-user (feature/timed-ask-user)
- The worktree-isolation guard refuses compound shell commands (python heredoc plus a runner, `cat >> <<EOF` then npx, and `$HOME` inside gradlew env). Write scripts to the scratchpad and run them as a separate plain command, and spell out absolute SDK paths.
- The Android gradle run printed only "Unable to locate a Java Runtime" to the log, and its background task still reported exit 0. Check the log, not the task status.
- Pi terminal `select` supports `signal` and `timeout` options. Pass the shrinking remaining timeout on every selection under one questionnaire deadline; otherwise a sequence of questions or multi-select choices can outlive the advertised limit.
- Pi's multi-select loop has no final `Done` choice once every option is selected. Save accumulated choices on that loop-exhaustion path or a completed selection can disappear from the response.

## 2026-09-27 Live subagent context window
- A fresh workflow worktree has no `node_modules`, so `tsc` reports hundreds of misleading pi-ai type errors. Run `npm ci` before the first type-check.
- The worktree guard refuses a heredoc and a `python3` run in the same Bash command. Write the script to the scratchpad and run it as a separate command.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1826,6 +1826,9 @@ class AidenChatViewModel(
if (question.id != promptId || !question.canRespond) return
if (!question.expiresAt.isAfter(Instant.now())) {
_pendingQuestion.value = null
// The Mac stopped waiting at expiresAt and the agent continued on
// its own judgement; say so instead of silently dropping the answer.
_presentedError.value = "This question expired, so Aiden continued with its best judgement. Send your answer as a message if it should change course."
return
}
if (!canRespondToQuestions()) {
Expand Down
2 changes: 1 addition & 1 deletion docs/aiden-remote-api-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ Selection nonces are a separate type. Workspace creation atomically revalidates
approvals but lacks that mutation grant still receives the bounded approval
with `canAllow: false` and may deny it; an attempted allow fails closed.
- `GET /streams/{streamId}/approval`: current bounded approval snapshot, or `null` after resolution.
- `GET /streams/{streamId}/question`: current bounded question-prompt snapshot, or `null` after resolution. Requires `chat:read` and is advertised only when `/server.features` contains `chat-question-prompts-v1`; any authenticated device may read its own stream's snapshot, while a response still requires the `questions:respond` grant. The snapshot carries `promptId`, `streamId`, `chatId`, `toolCallId`, the bounded `questions` array (1–4 questions, each with `question`, `header`, `multiSelect`, and 2–4 `options` of `label`/`description`), and `expiresAt`. Question text is assistant-authored and already bounded by the host grammar; no host-only data is projected.
- `GET /streams/{streamId}/question`: current bounded question-prompt snapshot, or `null` after resolution. Requires `chat:read` and is advertised only when `/server.features` contains `chat-question-prompts-v1`; any authenticated device may read its own stream's snapshot, while a response still requires the `questions:respond` grant. The snapshot carries `promptId`, `streamId`, `chatId`, `toolCallId`, the bounded `questions` array (1–4 questions, each with `question`, `header`, `multiSelect`, and 2–4 `options` of `label`/`description`), and `expiresAt`. `expiresAt` is the agent's own answer deadline, never more than five minutes after the prompt reached the stream; when it passes, the host tells the agent to proceed with its best judgement and the prompt resolves as expired. Clients should present an answer given after that point as an ordinary follow-up message. Question text is assistant-authored and already bounded by the host grammar; no host-only data is projected.
- `POST /questions/{promptId}/respond`: resolve one pending prompt owned by the authenticated device. Requires `questions:respond` and an `Idempotency-Key`. The exact body is `{ "cancelled": boolean, "answers": [...] }` where `cancelled: true` dismisses the entire prompt and each answer addresses one question index as `{ "questionIndex": n, "kind": "option" | "custom" | "multi", ... }`—`option` carries one selected `answer` label, `custom` carries free `answer` text (bounded to the shared custom-answer limit), and `multi` carries a unique `selected` label array for `multiSelect` questions. The host semantically validates answers against the stored prompt before settlement: wrong indices, unselectable options, single answers on multi-select questions, or malformed kinds return `invalid_request`. Success returns `{ "promptId", "resolvedAt" }`. A second response for a settled prompt returns `question_already_resolved`; an expired, cancelled, or terminated prompt returns `question_expired`. The idempotency ledger replays the original outcome for a repeated request UUID and rejects conflicting payload reuse. The stream leaves its waiting state and resumes generation once the host consumes the answer.

Turn start returns `turnId`, `streamId`, accepted state, and canonical appended message. The generation owner is the authenticated device/stream, not a socket. Disconnect never resends the prompt or cancels the turn. Restart during an active remote turn records one explicit interrupted terminal state and never retries the provider call.
Expand Down
1 change: 1 addition & 0 deletions docs/plans/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This directory is the source of truth for Aiden's implementation plans. The engi
| Plan | Status | Current state |
| -------------------------------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Aiden CLI](aiden-cli-plan.md) | Active | Phases 0–5 implementation complete; macOS/Linux CLI (57 tests each), Linux native helpers, complete shared subagent suites, root TypeScript/lint, and Android client checks pass. Physical iPhone acceptance is pending an unlocked device. Phase 6 adds QR remote pairing, scheduled-run notifications (mobile push + desktop), shared desktop memory, daemon autostart, and prebuilt binaries; see the [checklist](aiden-cli-parity-checklist.md). |
| [Timed ask-user waits](timed-ask-user-plan.md) | Implemented for review | Optional `timeoutSeconds` on `ask_user_question`; unattended (Remote) runs always expire within 5 min and resolve with an explicit best-judgement result. Late desktop answers become a Send/Queue follow-up offer; Remote `expiresAt` carries the real deadline; iOS/Android say when a question expired. PR CI pending. |
| [Rich link previews for Chats and Bots](rich-link-previews-plan.md) | Implemented for review | Shared regular Chat and Bot transcripts now show provider-aware inline icons and bounded hover/focus cards. User HTTP(S) text is autolinked, assistant streaming and persisted Markdown share the opt-in renderer, and URL-derived previews perform no network requests. |
| [Simulator Devices](simulator-devices-plan.md) | Partial | Environment **Simulator** tab plus `device_*` agent tools, ported from T3 Code (MIT, `1c127066`). iOS only; consent-gated pinned `expo-device-hub@0.12.0` + `agent-device@0.21.12`; token-authenticated loopback proxy. Phases 0–3 done (spike, flagged tab shell, main-process toolchain/host/proxy/service/IPC, live stream viewer with controls and screenshot-to-chat; `test:devices` 96 pass; fake-hub Electron E2E; real-Mac acceptance passed). Phase 4 (agent `device_*` tools), Phase 5 (simulators on paired Macs over Aiden Remote; [plan](simulator-devices-phase-5-peers.md)) and Phase 6 (procedural 3D device frames; [plan](simulator-devices-phase-6-3d.md)) done. Phase 7 done: Settings → **Simulator** (consent switches, pinned/installed helper versions, prune, remove installed tools) and [`docs/devices.md`](../devices.md); onboarding skipped while the flag is off. Agent guidance now prefers `device_*`/`agent-device` for the watched device but allows shell `xcrun simctl`/`xcodebuild`/`adb` for builds, installs, logs, port forwarding, and diagnostics (T3 #13908). Remaining: real-Mac acceptance for Phases 4–6 before the flag defaults on; SSH hosts are a later follow-up. |
| [Scoped MCP resources](mcp-scoped-resources-plan.md) | Implemented | Per-server resource inventory/templates/read; MCP93/Bots448/scheduled151/onboarding56 and both independent reviews pass. PR CI pending. |
Expand Down
32 changes: 32 additions & 0 deletions docs/plans/timed-ask-user-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Timed ask-user waits with late-reply handling

Status: Implemented for review.

Source: the DeepSeek harness comparison page and the 2026-09-24..27 digests. They point out that an `ask_user_question` wait can block an unattended run forever, and that an answer arriving after the agent has moved on is silently dropped.

## Contract

- `ask_user_question` accepts an optional `timeoutSeconds` (integer, clamped to 30–3600 s).
- The main-owned `AskUserQuestionCoordinator` owns the deadline. It stamps `expiresAt` (ISO 8601) on the prompt and settles the wait itself when the deadline passes. A response that arrives after the deadline is also settled as a timeout, even if the timer has not fired yet.
- A timed-out response is `{ cancelled: true, answers: [], timedOut: true }`. `timedOut` is main-only: owner responses cannot set it. Because it is also `cancelled`, existing consumers such as the vendored advisor picker fall back to their default without code changes.
- The tool result tells the agent that no answer was received. It says to proceed with its best judgement or any default it stated, to name the assumption it made, and not to ask the same question again.
- Deadline policy (`resolveAskUserQuestionTimeoutMs`):
- Attended desktop owners have no deadline unless the tool asks for one.
- Unattended (Remote-owned) runs always get one, capped at 300 s, which is the existing Remote question lifetime. The wire contract is unchanged, so there is no protocol revision bump.
- `chat:answerQuestionnaire` returns `{ status: "answered" | "expired" }`. A prompt that expired recently (the last 64 are remembered) returns `expired` instead of an error.

## Surfaces

- Desktop:
- Once `expiresAt` passes, the composer shows an "expired" notice.
- If the agent finishes after that deadline, the expired question remains visible in the composer so the user can still answer it.
- An answer submitted after expiry is not lost. It becomes a "late answer" notice that offers **Send as follow-up**, or **Queue follow-up** while a response is streaming, plus **Discard**.
- The follow-up text restates each question with its answer.
- Remote: the stream uses the prompt's `expiresAt`, capped at the Remote lifetime, for `question_required` and the pending-question snapshot, so clients see the real agent deadline.
- iOS and Android: when the local expiry trips, the question is no longer dropped silently. Both clients show that Aiden continued with its best judgement and suggest sending the answer as a message.
- CLI TUI: one timeout covers the entire questionnaire, including all steps of a multi-select question. Each terminal selection receives the remaining time and the tool-call cancellation signal.

## Follow-ups

- Native late-answer follow-up UX (composer prefill) and a live-expiry card state on iOS and Android.
- An optional deadline for detached or background desktop renderer runs, for example scheduled tasks rendered in a window.
3 changes: 3 additions & 0 deletions ios/AidenOnTheGo/Features/Remote/AidenChatFeature.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2927,6 +2927,9 @@ final class AidenChatViewModel {
let question = pendingQuestion, question.id == promptID, question.canRespond else { return }
guard question.expiresAt > Date() else {
pendingQuestion = nil
// The Mac stopped waiting at expiresAt and the agent continued on
// its own judgement; say so instead of silently dropping the answer.
presentedError = String(localized: "This question expired, so Aiden continued with its best judgement. Send your answer as a message if it should change course.")
return
}
guard let streamID = activeStreamID,
Expand Down
19 changes: 16 additions & 3 deletions main/handlers/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { llmClient } from "../services/llm-client.js";
import { chatGenerationOwner } from "../services/chat-generation-owner.js";
import { isSafeSubagentIdentifier } from "../../renderer/shared/subagent-runs.js";
import { parseChatRunInput } from "../../renderer/shared/chat-run-input.js";
import type { AskUserQuestionAnswerStatus } from "../../renderer/shared/ask-user-question.js";
import { parseParams } from "./chat-params.js";
import { geminiLiveService } from "../services/gemini-live/service-main.js";
import { MAX_CHAT_MESSAGE_CONTENT_BYTES } from "../../renderer/shared/chat-message-contract.js";
Expand Down Expand Up @@ -162,12 +163,24 @@ export function registerChatGenerationHandlers(): void {

ipcMain.handle(
"chat:answerQuestionnaire",
async (event, promptId: unknown, response: unknown) => {
if (typeof promptId !== "string" || !promptId) return;
async (
event,
promptId: unknown,
response: unknown,
): Promise<{ status: AskUserQuestionAnswerStatus } | undefined> => {
if (typeof promptId !== "string" || !promptId) return undefined;
const owner = chatGenerationOwner(event);
if (!llmClient.answerQuestionnaire(promptId, response, owner.documentId)) {
const outcome = llmClient.answerQuestionnaireWithOutcome(
promptId,
response,
owner.documentId,
);
if (outcome === "rejected") {
throw new Error("This renderer document does not own that questionnaire.");
}
// "expired" tells the renderer the agent already moved on, so it can
// offer the answer as a follow-up message instead of losing it.
return { status: outcome };
},
);
}
43 changes: 43 additions & 0 deletions main/services/aiden-remote-streams.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1592,6 +1592,49 @@ test("remote questions expire and are dropped on terminal state", async () => {
);
});

test("remote questions expire at the agent deadline, never later than the Remote lifetime", () => {
let now = Date.parse("2026-09-27T10:00:00.000Z");
const settled: string[] = [];
const service = new AidenRemoteStreamService({
now: () => now,
cancel: () => true,
approve: () => true,
respondQuestion: (promptId) => {
settled.push(promptId);
return true;
},
});
const owner = service.create("device-1", "stream-1", "chat-1", "turn-1");
owner.owner.send("chat:questionnaire", {
...QUESTION_PROMPT,
expiresAt: "2026-09-27T10:01:00.000Z",
});
assert.equal(
service.pendingQuestion("device-1", "stream-1")?.expiresAt,
"2026-09-27T10:01:00.000Z",
);
now = Date.parse("2026-09-27T10:01:00.000Z");
assert.throws(
() => service.questionChatId("device-1", "q-prompt-1"),
(error: unknown) => (error as { code?: string }).code === "question_expired",
);
// The host is asked to settle it, which the coordinator records as a timeout.
assert.deepEqual(settled, ["q-prompt-1"]);

const owner2 = service.create("device-1", "stream-2", "chat-1", "turn-2");
owner2.owner.send("chat:questionnaire", {
...QUESTION_PROMPT,
promptId: "q-prompt-2",
streamId: "stream-2",
expiresAt: "2026-09-27T12:00:00.000Z",
});
assert.equal(
service.pendingQuestion("device-1", "stream-2")?.expiresAt,
new Date(now + 5 * 60 * 1_000).toISOString(),
);
owner2.owner.send("chat:done", { chat: { messages: [{ id: "a-1", role: "assistant" }] } });
});

test("questionnaire publishes reject unbound prompts and duplicate stream prompts", () => {
const service = new AidenRemoteStreamService({
now: () => 1_000,
Expand Down
13 changes: 11 additions & 2 deletions main/services/aiden-remote-streams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1084,17 +1084,26 @@ export class AidenRemoteStreamService {
if (this.questions.has(promptId) || this.pendingQuestionForStream(stream.streamId)) {
throw new Error("A questionnaire is already pending on this stream.");
}
const expiresAt = this.options.now() + QUESTION_LIFETIME_MS;
// The coordinator stamps the agent-facing deadline on the prompt. Use it
// so the phone card expires at the instant the agent stops waiting, but
// never extend past the Remote question lifetime.
const now = this.options.now();
const agentDeadline =
typeof payload.expiresAt === "string" ? Date.parse(payload.expiresAt) : Number.NaN;
const expiresAt = Number.isFinite(agentDeadline)
? Math.min(Math.max(agentDeadline, now), now + QUESTION_LIFETIME_MS)
: now + QUESTION_LIFETIME_MS;
const expiry = setTimeout(() => {
const current = this.questions.get(promptId);
if (!current || current.expiresAt !== expiresAt) return;
// The host settles a past-deadline prompt as timed out, not closed.
this.resolveQuestion(promptId, {
version: ASK_USER_QUESTION_VERSION,
promptId,
cancelled: true,
answers: [],
});
}, QUESTION_LIFETIME_MS);
}, expiresAt - now);
expiry.unref?.();
this.questions.set(promptId, {
streamId: stream.streamId,
Expand Down
Loading
Loading