feat(tasks): pause the wall and idle clocks for human waits, answerably

[asto18089]

Summary

Re-lands the human-wait clock pause that #75 originally carried before its history was squashed, rebuilt on the fixes the fresh 12-agent audit demanded. Split out of #75 deliberately: it is task-lifecycle behavior change rather than a review fix, and as first landed it regressed the TUI.

A durable task whose turn hits an approval or user-input prompt was still bounded by the 30-minute task wall_time, which interrupted the turn at expiry and denied the pending decision on the human's behalf. The window is now excluded from the wall clock (mirroring the engine-side turn budget), the idle clock pauses and restarts fresh when the window closes, and the exclusion is bounded by a 24h fail-safe cap with a dedicated HumanWaitTimeout terminal reason.

Three design holes from the first landing are closed:

• Answerability — the pause applies only where a host can actually deliver the decision. The runtime API's task threads are served by HTTP decide_approval/submit_user_input and pause; the TUI's private task runtime has no such channel, so a prompt there never arms the window and the wall clock releases the worker at its own deadline. Before, it parked the worker for the 24h cap — the TUI regression that pulled this feature out of fix: land the timeout-audit review fixes for the merged #63 #75.
• Aggregate bound — the cap bounds the windows summed across the whole task, not only each open window, so a task chaining short prompts cannot sum its way past the cap once per prompt.
• Park-poll backoff — while a window is parked, the journal poll backs off from the 200ms catch-up cadence (doubling, capped at 2s); an arriving decision broadcasts on the subscription and wakes the loop early, so the backoff only removes the empty polls, never delays an answer.

Docs: SUBAGENTS.md / zh describe the pause and the answerability scope.

Testing

Red-green pins for every claim, each verified by mutation (ungating the answerability check, removing the aggregate check, re-applying the dead 200ms clamp, re-hoisting the aggregate check above shutdown/cancel, or dropping the closed-window subtraction from the parked wake — each fails its pin):

• unanswerable_surface_keeps_the_wall_clock_running_through_a_prompt — wall WallTimeout fires through a pending prompt, and no HumanWaitStarted is published.
• chained_short_waits_still_terminate_at_the_fail_safe_cap — two 120ms windows against a 200ms cap cannot chain past the fail-safe: the guard terminalizes as HumanWaitTimeout once the windows sum past it. With back-to-back windows the open-window check observes the crossing mid-window; the between-windows arm itself is pinned by execution_guard_spent_aggregate_cap_terminalizes_between_windows.
• The existing pins from the first landing re-verified: the wall survives a 2x-budget wait and completes on the late answer; the fail-safe cap releases an unanswered prompt; the supervisor guard follows the window signals.

Full suite on this head: cargo test -p codewhale-tui --lib 11980/0 with RUST_MIN_STACK=33554432; fmt clean. Head 320307876, stacked on #75's squashed head 1f69acdf8; this PR's base is #75's head branch round6-followup, so Files changed carries only this PR's four commits. Merge order is enforced by the stack: after #75 merges, retarget this PR to pinvou3-clean (GitHub does it automatically if round6-followup is deleted at merge time; this repo usually keeps branches, so retarget by hand otherwise) — the full check/gate CI, whose pull_request trigger is filtered to pinvou3-clean, runs there.

Review-fix wave (ac2ecca64)

A fresh from-scratch review pass of the re-land found and fixed three things:

• The park-poll backoff shipped inert. evaluate() clamped the loop wait to the 200ms catch-up cadence unconditionally, so the doubling and PARKED_POLL_CAP never took effect. The clamp now applies only on the non-parked path, and the parked cap wait excludes the already-closed windows, so the aggregate budget — not the open window alone — bounds the sleep. Deterministic pins now cover the cadence (EVENT_CATCHUP_POLL * 4 at 2s parked), the 2s ceiling, and the aggregate-budget wake; each fails under its mutation.
• The aggregate check jumped the queue. It fired before shutdown/cancel, mislabeling a canceled or shutting-down task as human_wait_timeout. It now sits in the pending chain after them (pins assert both outrank a spent cap). The window-closing event arm is also gated like the opening arm, so unanswerable surfaces no longer emit unpaired HumanWaitEnded.
• The answerability flag is now a named config field. TaskManagerConfig::human_waits_answerable (default false, the TUI's safe default) replaces the positional bool; the runtime API opts in via server_task_config, which pins both the default (from_runtime_defaults_to_unanswerable_human_waits) and the opt-in (server_task_config_opts_tasks_into_answerable_human_waits).

Docs: SUBAGENTS.md / zh also disclose the worker-hold cost — a parked task keeps its worker slot, so several simultaneously parked tasks can stall the durable-task queue until their answers or the fail-safe cap.

Review-hardening wave (320307876)

A second from-scratch review pass (eight domain reviewers plus independent mutation re-verification) confirmed the behavior and tightened the claims:

• The chained-waits test above was claiming the wrong mechanism — under a mutation deleting the between-windows aggregate arm it still passed, because window one's spend is already in the aggregate total and the open-window check fires mid-window. It is renamed and recommented to say what it actually pins; the between-windows arm stays pinned by execution_guard_spent_aggregate_cap_terminalizes_between_windows, which does go red under that mutation (re-verified).
• The backoff pins now assert literal durations (800ms, 2s) instead of restating the constants, so drifting EVENT_CATCHUP_POLL or PARKED_POLL_CAP off their documented 200ms / 2s values fails them instead of silently tracking the constants (both drifts re-verified red).
• The supervised wall-exclusion executor delay grew from 900ms to 2s, clearing the thin 500ms scheduling-starvation margin on loaded CI.
• The shared evaluate() comment now names both wake paths (the turn loop's event subscription and the supervisor's forwarded task event), and the human-wait event/config docs state that a custom executor emitting the HumanWaitStarted/Ended pair self-declares its waits answerable, independent of human_waits_answerable.

No-Issue: re-land of the human-wait half of the #75 fix wave; no separate tracking issue exists.

Signed-off-by trail: every commit DCO-signed.

Wiring-pin wave (b11a92cdb)

The fresh review's one MAJOR: the server opt-in had no pin below the config builder, so reverting run_http_server's call site or dropping the flag's forward into EngineTaskExecutor kept the whole suite green — the one mutation the red-green standard did not cover. Fixed by moving run_http_server's task-runtime wiring into open_server_task_runtime, the single wiring point, and pinning it with server_task_runtime_wires_answerable_human_waits_into_the_executor, which reads the flag where it lands — the executor field the manager runs tasks on — through the same entry the server itself uses. Verified by mutation: inlining the builder there and hardcoding the forward to false each turn only this pin red (the builder pin stays green, it never saw those hops); dropping the builder's assignment turns both red. Head is now b11a92cdb, stacked as before.

[JensenChen28]

已复核当前 head。实现把可应答宿主的审批/用户输入等待同时接入 turn guard 与 worker supervisor，暂停 wall/idle 计时，并用单次及累计 24h fail-safe cap 释放永久无人应答的任务；TUI 私有运行面保持默认不启用，避免不可应答提示占用 worker。事件配对、取消/关机优先级、累计上限与 parked poll backoff 的边界处理一致，文档也披露了等待期间继续占用 worker slot。git diff --check 通过；本地相关测试共实际执行 16 个（human-wait、backoff、aggregate cap、unanswerable surface 等），全部通过；当前 DCO、Gitleaks、check、gate、CodeQL 等门禁均通过。

[asto18089]

Fresh from-scratch review (nine domain reviewers + mainline verification, no reliance on the earlier waves). Head audited: 320307876; parent #75 advanced 17 commits to 1ee9b8036 after this branch was cut, so I rebased the four commits onto the new tip locally: clean, all four patch-ids identical (git range-diff all =), no conflicts. Not pushing the rewrite — the content is byte-identical, so a force-push would only invalidate the SHAs this body and the earlier review waves reference. After #75 merges, retarget to pinvou3-clean as planned.

Verdict: 0 BLOCKER, 1 MAJOR (a test gap, not a code bug), 8 MINOR. The design is sound and the code is correct as far as nine adversarial passes could break it — the one MAJOR is that the PR's own mutation-verified-pin standard is not met for its headline wiring.

What was verified, briefly:

• The load-bearing premise holds end to end. Task turns run on the same RuntimeThreadManager the server serves; decide_approval/submit_user_input resolve manager-wide pending approvals and task threads are ordinary threads in that registry, with no interactive/task scoping anywhere in the delivery path. The TUI private runtime genuinely has no delivery channel (its event bus has exactly three subscribers: the two SSE endpoints and the drive loop). Every production EngineTaskExecutor/TaskManagerConfig construction site was enumerated; the answerability value is correct at each.
• The window-pair contract holds against every current producer. All six journal event names exist and are emitted on the paths the comment claims, including forced-deny on interrupt/shutdown/engine-death and terminal-turn settlement of user inputs; closers are journaled before the terminal turn event; prompts are serial per turn by construction (tool_plan_can_join_parallel_batch excludes interactive plans from parallel batches), so the first-wins/any-closes merge is safe today. The broadcast subscription is wake-only — the journal cursor is authoritative, so Lagged cannot lose a decision and a cross-thread decided cannot close this turn's window (events are filtered by thread and turn id before the match).
• The guard arithmetic is correct. Window spans are subsets of [started_at, now] on one monotonic clock, so the saturating subs in engine_wall_elapsed are structurally unreachable no-ops; no chaining schedule evades the cap (evaluate runs once per loop iteration regardless of event batching); backoff shift/mul overflow is impossible; Duration += overflow is unreachable by ~14 orders of magnitude. HumanWaitTimeout is wired through every exhaustive match (as_str, status, receipt, preserve_timeout_reason, finish summary).
• Loop mechanics hold. The parked wait is a genuine three-arm select! with the subscription, journal append precedes broadcast (no wake-then-miss), and the supervisor's biased select polls event_rx before both timers. Cancel of a parked task is prompt (~0.5–7 s end to end), shutdown latency is bounded (~2 s sleep + 5 s grace, no deadlock), and the backoff is a ~10x cut in empty journal fetches during parks.

MAJOR

M1 — The server opt-in wiring is unpinned end to end; a call-site revert keeps the whole suite green. I verified by mutation: reverting run_http_server back to inline TaskManagerConfig::from_runtime (dropping the opt-in) → 11984 passed, 0 failed. The propagation hop cfg.human_waits_answerable → EngineTaskExecutor::new → drive_engine_turn (task_manager.rs:1498, :1053) is likewise unobserved — no test constructs a manager through start_with_runtime_manager. The builder-level pins (server_task_config_opts_tasks_into_answerable_human_waits, from_runtime_defaults_to_unanswerable_human_waits) are both real, but the headline production behavior — server tasks pause — is the one mutation nothing catches, which falls short of the body's "red-green pins for every claim, each verified by mutation" bar. Cheap fix: rebuild human_wait_is_excluded_from_the_task_wall_clock through the full chain — TaskManager::start_with_runtime_manager with a config from server_task_config, real runtime manager, emit_event_for_test to open the prompt — and assert the wall survives a pending prompt. That one test goes red if any of the three hops regresses.

MINOR

1. Closing-event journal-append failures are swallowed (runtime_threads.rs:9313/9350/9469/9496/9527/6911; :4038-4048 restores the claim without publishing), so a partial store failure can leave the window open with both clocks paused on an otherwise-healthy task until the 24h cap. The comment's "the window cannot stick open" overstates; lean on its own final cap clause.
2. The supervisor wall arm lacks the idle arm's drain-and-recheck (task_manager.rs:2133-2162 drains queued events only for IdleTimeout): a queued-but-unapplied HumanWaitStarted cannot beat a wall firing in the arming gap. Ms-scale in practice, but the idle path already has the exact protection — mirroring it for the wall arm (or draining on any interrupt) is small.
3. The trailing drain violates its own stated invariant: it skips only ToolHeartbeat, so a tail HumanWait* falls into apply_execution_event and takes the manager-wide state lock for the record no-op arm (task_manager.rs:2216-2227 vs :2393) — precisely what the drain comment says liveness-only ticks must not do. Extend the skip to the pair. Harmless today (≤2 events, post-executor, not urgent), parity-only.
4. Docs nits: "two workers by default" is true for the runtime API — the only surface where parking happens — but unqualified (the TUI private queue defaults to up to 4 via event_loop.rs:655 clamp(1, 4)); the human_wait_timeout terminal reason is never named, so an operator can't connect the doc to the task record; the idle clock pausing and restarting fresh on window close is real and pinned but unmentioned; zh 工作者/工作者槽位 is a new coinage where the zh tree otherwise keeps "worker" untranslated.
5. CHANGELOG: [Unreleased] does carry behavior-change entries by precedent (#39, telemetry), though the parent #75 stack adds none — mixed practice; either add a line or state the release-time archival intent.
6. Coverage bundle (each cheap, none blocking): early-wake under backoff is unpinned (deleting the subscription.recv() arm keeps everything green — a ≥2s park with a decision arriving mid-sleep would pin it); HumanWaitTimeout vs wall/idle precedence with several budgets exceeded simultaneously; the supervisor-side cap path (terminal_reason: "human_wait_timeout" on the record); unpaired HumanWaitEnded/double-begin idempotency; the persist-urgent classification of the pair.
7. Commit hygiene: the revert-of-revert 2b0fb7531 carries the bare default message — a bisect lands on the known-broken first-landing tree with no in-commit explanation; 03848af9e (test:) also rewrites production doc comments (the evaluate() wake-paths comment, the self-declaration sentence) — disclosed in the message, but the type is imprecise.
8. One out-of-scope comment sentence: the debounce-starvation note added in handle_execution_event documents pre-existing behavior and partially duplicates the parent's 37498eef2 disclosure at the flush arm. Trim it or keep it knowingly.

Notes (no action demanded)

approval.timeout close arm is dead on fresh journals (no current producer; legacy replay only) — fine as defense. RecvError::Closed on the wake subscription would hot-spin but is unreachable (the manager outlives the loop). The aggregate spend is per-run, not persisted across restart — consistent with how wall_time already behaves, and disclosed in the variant doc. docs/RUNTIME_API.md:1134 ("no wall-clock cap") is engine-layer and remains true; a one-line cross-reference would prevent misreading. Auto-approve journals zero-length window pairs — benign. While parked with a tool in flight the supervisor still wakes per heartbeat (~2s), so the backoff's savings are mostly the turn loop's. The FLEET.md stale max_steps claim is parent-internal — belongs to #75, not here.

Verification on the rebased head

cargo test -p codewhale-tui --lib 11984/0 (13 ignored, 142s, RUST_MIN_STACK=33554432); cargo fmt --check clean; clippy under the release.yml flag set: zero diagnostics in touched files (8 pre-existing errors, all in files this PR does not touch: core/engine/tests.rs, tools/{tasks,web_search,js_execution}.rs, shell_dispatcher.rs). Mutations: M1 above (suite blind — the MAJOR); ungating the answerability arms → unanswerable_surface_keeps_the_wall_clock_running_through_a_prompt red ✓; re-applying the unconditional 200ms clamp → the three execution_guard_parked* pins red ✓; killing the between-windows aggregate arm → execution_guard_spent_aggregate_cap_terminalizes_between_windows red while chained_short_waits_still_terminate_at_the_fail_safe_cap stays green ✓ — exactly the mechanism split the body discloses.

No-Issue consideration is already handled by the body. Fix M1 (and whatever MINORs are worth their churn) as one more commit on this branch; happy to re-verify the diff afterwards.

[asto18089]

MAJOR fixed and pushed: b11a92cdb (fast-forward on human-wait-pragma, no history rewrite — the previously referenced heads stand).

run_http_server's task-runtime wiring now lives in open_server_task_runtime (crates/tui/src/runtime_api.rs), the single wiring point, and the new pin server_task_runtime_wires_answerable_human_waits_into_the_executor drives that same entry and reads the flag where it lands — the executor field the manager runs tasks on — via a cfg(test) as_engine_executor seam on TaskExecutor (custom executors keep the None default). The test isolates CODEWHALE_TASKS_DIR under lock_test_env, so it touches no real state.

Mutation evidence, each applied to the committed fix:

• inlining the builder inside open_server_task_runtime (dropping the opt-in) → the new pin red, server_task_config_opts_tasks_into_answerable_human_waits green — the exact blindness the MAJOR described, now closed;
• hardcoding false in start_with_runtime_manager's forward → the new pin red, builder pin green;
• dropping the builder's = true assignment → both pins red.

Full suite on the pushed head: cargo test -p codewhale-tui --lib 11985/0 with RUST_MIN_STACK=33554432; fmt clean; clippy under the release.yml flag set shows zero diagnostics in the two touched files. PR body updated with the wave section.

Scope note: the review's 8 MINORs are untouched, pending a call on which are worth their churn. The rebase onto #75's advanced tip still happens at retarget time (after #75 merges), as before.