Skip to content

fix(adapters): count subagent transcript writes as idle-notice activity; name the waited-on tool - #896

Merged
pbean merged 4 commits into
mainfrom
fix/858-subagent-idle-activity
Oct 13, 2026
Merged

pbean merged 4 commits into
mainfrom
fix/858-subagent-idle-activity

Conversation

@pbean

@pbean pbean commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The #680 idle detector stats only the parent transcript. Claude Code writes nothing to the parent <sid>.jsonl while a foreground subagent runs, so a session blocked on a subagent journals a false session-idle. Under bmad-loop that is the normal case, because claude.toml sets CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1 and every subagent is foreground.

This PR does two things:

  1. Subagent writes count as activity. A declared subagent_transcript_dir profile field names a subagent transcript tree. A stat-only recursive walk of that tree feeds the idle clock only.
  2. The notice names what it is waiting on. Every session-idle on a claude-jsonl session carries in_tool, the unanswered tool call (for example Bash). The TUI renders · idle 71m · in Bash.

Two corrections to the issue's suggested fix

  • The walk is recursive. Workflow agents live at <sid>/subagents/workflows/wf_<id>/agent-*.jsonl, the reason tokens.tally already uses rglob on the same anchor. Sibling *.meta.json files are not transcripts.
  • The subagent signal is a separate key, not folded into idle.last_key. The wait loop's sample_transcript reads last_key[1] as the parent's byte size. That value is the A dev session waiting on an interactive permission prompt is recorded as crashed, and retry burns the last attempt #727 growth test and the scan offset for _transcript_has_assistant_activity. A combined key would credit subagent bytes as parent growth and scan from a wrong offset. The tree gets its own _IdleTracker.sub_key, OR-ed only into the idle clock's change test.

Design

subagent_transcript_dir (CLIProfile, appended field):

  • None (the default) derives the value from usage_parser: {stem}/subagents for claude-jsonl, else off. "" turns it off, and any other string is an explicit template.
  • Derived, not hard-coded, so a user's pre-existing claude overlay (overlays replace a profile wholesale) gets the fix without re-declaring anything. claude.toml is untouched.
  • Validation: {stem} is required (without it, sibling sessions' trees would mask idleness). The value must be relative with no .., and it is refused on hookless profiles.
  • The walk is an explicit os.scandir stack. It skips symlinks and never opens a file (a FIFO cannot block). A missing root reads as empty, and the walk is uncapped. The key is an aggregate (count, total_bytes, newest_mtime_ns).
  • A walk fault degrades the tick to parent-only, the pre-fix behaviour. It is crumbed as subagent-scan-failed / subagent-scan-recovered.

in_tool on session-idle:

  • A closed slug chosen by code: a Claude built-in name from _IDLE_TOOL_SLUGS, mcp (MCP server names stay out of the journal), other, none (nothing open), or unknown (the read raised). Never transcript text. Declared benign in the journal field guard.
  • Computed once per stretch, just before the session-idle append. A silent transcript holds still, so the state read then holds for the whole stretch: a tool_result landing moves the file and closes the stretch. The reader reads the parent plus every subagent-tree jsonl and names the newest open call by entry timestamp. A parent's open Agent therefore loses to its subagent's later Bash, which is the innermost wait.
  • claude-jsonl only (the tool_use/tool_result shape is Claude's). Every other parser omits the field, so an absent field means "this CLI cannot tell".
  • A fault reads unknown, never none. It is crumbed as tool-scan-failed / tool-scan-recovered.
  • It labels instead of suppressing. A running trunk check and a hung docker run leave identical transcripts (session-end journaled 2h19 after session_timeout_min fires when the session is wedged inside a tool call #157). Suppressing the notice during a tool call would hide exactly the hung-tool case [BUG] Stall detection cannot see a session idling inside a tool call: pane-log re-arm keeps the grace alive through sleep #680 exists for. The label tells an operator which one to suspect. session-active and the heartbeat are unchanged.

Evidence

A read-only scan of local ~/.claude/projects transcripts (Claude Code 2.1.273–2.1.296) found 196 foreground agents that would trip a false session-idle at the 600 s default. The longest false stretch was 9572 s, with zero parent writes inside any ≥600 s gap. Classifying every ≥600 s silence by the unanswered tool call when the gap began:

Unanswered call Gaps
Agent 205 (the subagent fold fixes these)
AskUserQuestion / ExitPlanMode 43
Bash 33
other tools 5
none (turn over) 350

So the waited-on call is recoverable from the transcript, and the remaining gaps get a label.

Residuals

  • An overlay that changes usage_parser away from claude-jsonl must set subagent_transcript_dir itself, and it gets no in_tool label.

Out of scope

  • Codex child rollouts. They can only be identified by reading each file's first line and they live in shared day dirs, so no template can express them. The observed parent gap stayed under 600 s. Noted on DW-547.
  • Gemini / Copilot / Antigravity subagent storage. Unverified; tracked as DW-548.
  • A SubagentStart/SubagentStop hook latch. "A subagent is open" is not "it is working", and the latch would reopen a completion hazard.

Test plan

  • tests/test_profile.py: derived default, explicit "" disables, malformed values refused (each check ablated), hookless refusal, claude's root matches the tokens.tally anchor.
  • tests/test_generic_tmux.py, subagent fold: direct and workflow-nested writes keep a static parent active; a tree appearing mid-run closes the stretch; a silent tree still opens one stretch; re-point rebaselines the tree; subagent growth is not A dev session waiting on an interactive permission prompt is recorded as crashed, and retry burns the last attempt #727 work; scan faults crumb once and degrade to the parent; the streak survives a re-point; the fold is off without a layout; a declared dir folds under usage_parser = "none"; final-interval activity closes the stretch; direct _subagent_tree_key walk rules.
  • tests/test_generic_tmux.py, in_tool: names the open call; none when nothing is open; innermost wait across parent and tree; MCP maps to mcp with no server name; omitted for gemini/codex/copilot; read faults and tree-walk faults read unknown with one crumb pair; session-active fields unchanged; the tolerant reader skips torn and garbage lines, skips a FIFO, and propagates a read fault.
  • tests/test_tui_data.py / tests/test_tui_app.py: idle_tool derivation, its close by session-active, non-string values read as None; header renders · in Bash / · in unknown, and nothing for none or an unlabelled stretch.
  • Every negative assertion was ablated locally and fails with its gating code removed.
  • uv run pytest -q -n logical: 14413 passed, 113 skipped. uv run pyright: 0 errors. trunk check: clean.

Closes #858

Summary by CodeRabbit

  • New Features
    • The TUI now identifies which tool an agent is waiting on during idle periods.
    • Idle tracking accounts for activity in configured subagent transcripts, so active subagent work is not shown as idle.
  • Documentation
    • Updated the profile and TUI guides with subagent transcript configuration and idle-indicator details.

t added 2 commits October 12, 2026 20:13
…ty (#858)

A parent blocked on a foreground subagent writes nothing to its own
transcript, so the #680 idle detector journaled a false session-idle.

- Add the CLIProfile field `subagent_transcript_dir` (appended): None
  derives from usage_parser (`{stem}/subagents` for claude-jsonl), ""
  turns it off; an explicit template must use exactly `{stem}`, stay
  relative, and is refused on hookless profiles.
- Walk that tree stat-only and recursively (`_subagent_tree_files` /
  `_subagent_tree_key`: count, total bytes, newest mtime) and OR its
  movement into the idle clock's change test. `idle.last_key` stays the
  parent's key, since it is the #727 scan offset.
- A raising walk degrades the tick to parent-only, crumbed as
  subagent-scan-failed / subagent-scan-recovered.
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8e502fa1-a265-4d21-bfcd-c7069860b7fa
📥 Commits

Reviewing files that changed from the base of the PR and between 283b5be and cfe6adf.

📒 Files selected for processing (4)
  • src/bmad_loop/adapters/generic.py
  • src/bmad_loop/adapters/profile.py
  • tests/test_generic_tmux.py
  • tests/test_profile.py

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

Walkthrough

Idle tracking now includes profile-declared subagent transcript trees. Claude session-idle journal entries can identify the awaited tool, and the TUI displays that label with the idle duration.

Changes

Subagent-aware session idle tracking

Layer / File(s) Summary
Subagent transcript profile configuration
src/bmad_loop/adapters/profile.py, tests/test_profile.py, docs/adapter-authoring-guide.md
Profiles accept subagent_transcript_dir. Claude profiles default to {stem}/subagents; explicit templates are validated and can be disabled with an empty string.
Subagent transcript activity sampling
src/bmad_loop/adapters/generic.py, src/bmad_loop/policy.py, tests/test_generic_tmux.py, docs/tui-guide.md, CHANGELOG.md
Idle detection stats regular JSONL files recursively under the configured tree. Changes in that tree count as activity without changing the parent transcript’s work offset. Missing paths are skipped; scan faults fall back to parent-only tracking and record failure/recovery breadcrumbs.
Idle tool labels and TUI display
src/bmad_loop/adapters/generic.py, src/bmad_loop/tui/*, tests/test_generic_tmux.py, tests/test_portability_guard.py, tests/test_tui_*, CHANGELOG.md
Claude idle entries can include in_tool, based on the newest unanswered tool call across parent and subagent transcripts. The TUI displays the label for the current idle stretch. Other usage parsers omit the field.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Low

Sequence Diagram(s)

sequenceDiagram
  participant GenericAdapter
  participant CLIProfile
  participant TranscriptFiles
  participant RunJournal
  participant TUI
  GenericAdapter->>CLIProfile: Resolve subagent_transcript_root
  GenericAdapter->>TranscriptFiles: Stat parent and subagent transcripts
  TranscriptFiles-->>GenericAdapter: Activity keys and open tool calls
  GenericAdapter->>RunJournal: Record session-idle with in_tool when available
  TUI->>RunJournal: Read current session entries
  RunJournal-->>TUI: session-idle and in_tool
  TUI->>TUI: Display idle duration and tool label
Loading

Suggested reviewers: dracic

Merge Risk: ⚪ Minimal · up to cfe6a

No material issue identified here prevents merging after normal checks.

Pre-merge checks · Passed 5
Passed Passed checks (5 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely identifies both primary changes: counting subagent transcript writes as idle activity and naming the awaited tool.
Linked Issues check Passed Issue #858 requires foreground subagent transcript writes to prevent false session-idle notices. The PR adds subagent_transcript_dir, derives {stem}/subagents for claude-jsonl, recursively sta…
Out of Scope Changes check Passed The profile field, recursive scan, fault handling, in_tool journal field, TUI label, documentation, and tests all support issue #858 or the idle-notice behavior described by the PR. The available ch…
Docstring Coverage Passed Docstring coverage is 82.93% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 41 functions across 9 files. (1 skipped: 1 …
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the transcript trail
And counts each subagent’s tale
A tool name joins the idle line
While busy files keep time just fine
The TUI glows; the clocks align

Comment @coderabbitai help to get the list of available commands.

@pbean

pbean commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T04:36:32.640166Z cfe6adf Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 972f17b1cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/bmad_loop/adapters/generic.py Outdated
Comment on lines +698 to +701
elif item.get("type") == "tool_result":
call_id = item.get("tool_use_id")
if isinstance(call_id, str):
open_calls.pop(call_id, None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve tool-event chronology across transcript files

When a forked Claude subagent copies the parent's still-open tool_use, the parent can later contain its corresponding tool_result while the child retains the copied use. Because files are processed parent-first, this line removes the call while reading the parent, but the subsequently scanned child re-adds the older copied use, so a later idle stretch can incorrectly report in_tool: Agent instead of none. Merge events by timestamp, or prevent an older use from overriding a newer cross-file result.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in git rev-parse --short=8 HEAD. A tool_result now closes its call in every file, whatever order the files are read in. The scan collects results across all transcripts first, then decides which calls are still open.

For the record: real fork transcripts do copy the parent's Agent tool_use under the same id and timestamp. Today the child closes that copy itself with a synthetic "Fork started" result about 10ms later, so the stale label could only appear in that window. The fix removes the dependence on read order anyway. Regression test: test_open_tool_call_closes_a_forked_copy_from_the_parent (it fails against the old per-file close).

…le (#858)

A forked subagent's transcript copies the parent's open Agent tool_use
under the same id. The open-call scan closed calls per file in read
order, so a parent-side tool_result could be undone by the child's copy
read afterwards. Collect results across all files before deciding which
calls are still open.
@pbean

pbean commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 283b5be163

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"transcript's filename stem), with no other field, format spec or "
f"conversion: got {subdir!r}"
)
if is_absolute_path(subdir) or has_parent_ref(subdir):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject NULs in subagent transcript templates

A profile can set subagent_transcript_dir = "{stem}\u0000/subagents": TOML accepts the escape, and it passes the placeholder, absolute-path, and parent-reference checks here. On the first idle sample, _subagent_tree_files() passes the rendered path to os.scandir(), which raises ValueError: embedded null character; _sample_subagent_tree() catches only OSError, so the exception escapes the wait loop and crashes the run instead of being rejected as a ProfileError. Reject NUL characters at this profile boundary.

AGENTS.md reference: AGENTS.md:L86-L87

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cfe6adf: validate_profile now rejects a NUL in subagent_transcript_dir with a ProfileError before parsing the template. Test: a "{stem}\u0000/x" row in test_malformed_subagent_transcript_dir_is_a_profile_error (ablated: it fails without the check).

Comment thread src/bmad_loop/adapters/generic.py Outdated
Comment on lines +674 to +676
if not path.is_file():
continue
with path.open("rb") as stream:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Open transcript files without a FIFO race

When a transcript path is replaced concurrently after is_file() returns but before open(), the replacement can be a FIFO with no writer, causing this synchronous open("rb") to block indefinitely. Because _open_tool_call() runs inside the session wait loop, that prevents subsequent Stop, window-death, timeout, and hard-stop handling despite the stated FIFO guard. Open nonblocking and validate the resulting descriptor with fstat() so a raced non-regular file degrades through the existing observable fault path.

AGENTS.md reference: AGENTS.md:L86-L87

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cfe6adf: _open_tool_call keeps the is_file() prefilter, then opens with os.open(O_RDONLY | O_NONBLOCK) and skips the file unless fstat says S_ISREG. A FIFO swapped in between neither blocks nor gets read. New test test_open_tool_call_skips_a_fifo_swapped_in_after_the_check (ablated: dropping O_NONBLOCK and dropping the fstat check each fail it).

…#858)

Reject a NUL in subagent_transcript_dir at profile load: TOML accepts a
\u0000 escape, and the rendered path made the tree walk raise ValueError,
which escaped the wait loop instead of degrading like an OSError.

Open transcripts in _open_tool_call non-blocking and re-check the opened
descriptor with fstat, so a FIFO swapped in after the is_file check
neither blocks the wait loop nor gets read.
@pbean

pbean commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: cfe6adf618

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@pbean
pbean merged commit 7834dfa into main Oct 13, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] #680 idle detector flags a session idle while a foreground subagent works

1 participant