Skip to content

fix: heartbeat sessions are normal sessions — idle display and normal residency with durable wake - #1967

Merged
sethkarten merged 17 commits into
mainfrom
fix/heartbeat-idle-classification
Sep 3, 2026
Merged

fix: heartbeat sessions are normal sessions — idle display and normal residency with durable wake#1967
sethkarten merged 17 commits into
mainfrom
fix/heartbeat-idle-classification

Conversation

@snimu

@snimu snimu commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

A session with an armed heartbeat sat in the agents view's Running section forever — even when it had been quiet for hours — and its worker was pinned resident around the clock, because the daemon treated "has a heartbeat" as both a running signal and an idle-eviction veto. This PR makes heartbeat sessions completely normal sessions: truthful status in the view, and normal residency with a durable wake when the next beat comes due.

Part 1 — display. The contract is now: running = doing work now (an executing heartbeat turn still lands there through its normal prompt turn); idle = resident and quiet, including armed heartbeats between firings; inactive = not resident. A heartbeat count is never a running signal. The heartbeat-to-running rule is deleted from every place it lived — the roster classifier, the unified agents-view classifier, ancestor propagation, subagent count projections, and the daemon's agent-message summaries. hasActiveHeartbeat stays on the wire for badges and labels (no protocol change): idle rows label the next firing (heartbeat · next 5m), paused-only heartbeats render a dimmed ♥ badge, and the delete confirmation warns when the targeted row still has an armed heartbeat.

Part 2 — residency. The residency half of the same rule (hasRegisteredHeartbeat vetoing idle eviction, child passivation, and empty-draft eviction) is deleted too; heartbeat sessions passivate under the one existing idle rule, idleEvictionMinutes (default 90 minutes, "off" supported). Ownership is split by residency: workers keep all firing and delivery for resident trees (existing scheduler revival, claim locks, and catch-up semantics unchanged), and the supervisor owns exactly one new concern — waking a tree that is resident nowhere. It recomputes a single unref'd wake timer (the soonest due job) from durable truth only — the spawn ledger's family() plus each session's scheduled-jobs.json artifact — at boot and whenever heartbeats or residency change; on fire it relaunches the root worker through the same create path clients use, and the fresh worker's scheduler runs the due job. Failed relaunches back off 60 seconds instead of hot-looping, and the schedule stays disarmed while an update restart is being prepared. The supervisor never dispatches or delivers, so firing ownership never overlaps.

Client-owned (ephemeral) sessions stay private: stopping the worker cancels the scheduled jobs of its root and ledger descendants — schedules die with the registration exactly like their roster rows — and a failed cancel keeps the stop tombstone as durable intent, re-derived from the persisted descriptors at the next boot (public sessions are unaffected: heartbeat_set / cron_add promote owned sessions before their jobs land). One eligibility gate: the wake enumerator sees only the supervisor's sessions root, so an idle public worker whose root file lives outside that root keeps its residency while it has a registered heartbeat or cron job — passivating it would orphan schedules no wake scan can see. And a freshly launched root registers its passive descendants' scheduled-job artifacts on creation (no hydration; a corrupt artifact skips only itself), so a relaunched tree schedules its passivated children too.

Passivated heartbeats stay usable: heartbeats_list appends passive scheduled jobs from the same durable enumeration the wake uses, roster rows keep their registration marks across eviction and rank first within the Inactive section, the badge aggregate matches jobs by stable session id and file (the active id goes stale after passivation), and heartbeat_manage mutates a passive job's scheduled-jobs store directly — no wake just to pause or stop it.

Validation: 14 display pins fail on main and pass here (next-fire label, dimmed paused badge, delete guard included); 6 residency pins fail on the display-only half and pass here, plus 4 review-round pins (client-owned schedule cancellation, ledger-topology wake, update-restart disarm, passive heartbeat management); resident-idle firing is already pinned by the unchanged daemon-mode heartbeat tests. daemon-mode (184), daemon-supervisor-* (including the process suite), session-action-store, daemon-session-list, cron-jobs, all heartbeat suites, and the full agents-view/roster/subagent display suites pass in a sanitized env; root npm run check green.

Linear: RES-1251 (display) https://linear.app/primeintellect/issue/RES-1251/sessions-with-armed-heartbeats-show-as-running-forever-instead-of-idle
Linear: RES-1250 (residency) https://linear.app/primeintellect/issue/RES-1250/armed-heartbeats-pin-workers-resident-247-supervisor-owned-scheduler


Note

Medium Risk
Changes daemon supervisor scheduling, idle eviction, and session wake paths alongside agents-view status semantics; incorrect wake or eviction could miss heartbeats or drop schedules, though behavior is heavily test-covered.

Overview
Armed heartbeats no longer count as “running.” Roster and agents-view classification, ancestor propagation, and subagent counts now treat only real work (streaming, tools, children, etc.) as Running; quiet sessions with active heartbeats sit in Idle with badges and labels like heartbeat · next 5m. Paused-only jobs get a dim ♥ badge; delete confirmations warn when a row still has an armed heartbeat.

Residency matches normal idle rules. hasRegisteredHeartbeat is removed from idle eviction, child passivation, and empty-session eviction. The supervisor instead scans durable scheduled-job artifacts and the spawn ledger, arms a single wake timer, and creates/reuses a root worker when the next active job is due (with backoff and disarm during update-restart). Workers whose session files sit outside the enumerable sessions root stay resident when they have schedules the wake scan cannot see (hasWakeBlindSchedule).

Passivated trees stay operable: heartbeats_list / cron_list merge passive jobs; heartbeat_manage and cron_cancel can hit durable stores without hydration; badge aggregation keys off session id/file when active ids go stale; passivated roster rows keep registration marks; new roots register passive descendants’ cron artifacts. Client-owned workers cancel scheduled jobs for the whole tree on stop, with tombstoned descriptors retried on boot if cancel fails.

Reviewed by Cursor Bugbot for commit 281cfbf. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Treat heartbeat sessions as normal sessions for idle display and residency, add durable passive wake

  • Removes the special-case logic that treated an active heartbeat as sufficient to classify a session or subagent as running. classifyAgentStatus, classifySessionRosterStatus, and the agents-view/interactive classifiers now derive status from actual work, residency, and queued children only.
  • Adds a supervisor-level durable scheduled-session wake system. The supervisor enumerates scheduled-job artifacts from passive (workerless) RLM session trees, arms a timer for the next due job, and creates or reuses a root worker when the job fires. This replaces the old approach of pinning heartbeat workers in memory to keep schedules alive.
  • Introduces a wake-blind flag on WorkerEvictionSnapshot for workers whose session file lives outside the enumerable sessions root. These workers are protected from idle and detach eviction because the supervisor cannot re-wake them.
  • Extends cron_list, heartbeats_list, heartbeat_manage, and cron_cancel command handlers to operate on passive scheduled jobs directly via durable artifacts, without hydrating or waking a worker.
  • Client-owned worker stop now cancels the full session-tree scheduled jobs before descriptor deletion; failed cancellation leaves a persisted stop tombstone retried on next boot.
  • Behavioral Change: sessions with only a registered heartbeat are now eligible for idle/detach eviction when otherwise inactive; agents-view parent rows no longer show running status due to descendant heartbeats; SessionEvictionSnapshot no longer carries a per-session heartbeat-registration field and classifyAgentStatus no longer accepts a heartbeat activity flag.

Macroscope summarized 281cfbf.

An armed heartbeat between firings is residency, not work, so it no
longer counts as a running signal anywhere: the roster classifier, the
unified agents-view classifier, the ancestor-propagation overwrite, and
the subagent count projections all drop the heartbeat clause. A firing
heartbeat still lands in Running through its executing turn, and
hasActiveHeartbeat stays on the wire for badges and labels.

Idle rows with an armed heartbeat now label the next fire time
(heartbeat - next 5m) when the aggregate knows it. Paused-only
heartbeats, previously invisible, render a dimmed heartbeat badge
without affecting the section. The delete confirmation warns when the
targeted session or subagent still has an armed heartbeat; the daemon
already cancels scheduled jobs in both delete paths.

ENG-5847
…me due

An armed heartbeat no longer pins a session resident forever. The
hasRegisteredHeartbeat veto is deleted from the idle-eviction and child
passivation policies (and from the empty-draft detach rule): heartbeat
sessions follow the one existing idle rule (idleEvictionMinutes,
default 90m) like every other session.

The schedule survives passivation through a single wake owner split by
residency: workers keep owning all firing and delivery for resident
trees (their scheduler already revives passive children on due jobs),
and the supervisor owns exactly one new concern - waking a session tree
that is resident nowhere. It recomputes wake timers from durable truth
(the saved-session catalog plus each session's scheduled-jobs artifact)
at boot and on every heartbeat/residency change, and on fire relaunches
the root worker through the same create path clients use; the fresh
worker's scheduler then runs the due job with the existing catch-up
semantics. Workers now also register passive descendants' scheduled-job
artifacts when a root becomes resident, so a relaunched tree schedules
its passivated children's heartbeats without hydrating them.

Passivated sessions keep their heartbeats visible: the supervisor's
heartbeats_list appends passive scheduled jobs, roster rows keep their
registration marks across eviction, and the agents-view badge aggregate
matches jobs by stable session id and file, not just the stale active
id.

ENG-5846
@snimu snimu changed the title fix: classify sessions with armed heartbeats as idle, not running fix: heartbeat sessions are normal sessions — idle display and normal residency with durable wake Sep 2, 2026
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-mode.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
…opology

Review round on the residency half:

- Client-owned schedules now die with the registration, exactly like
  their roster rows: stopping an ephemeral worker cancels the scheduled
  jobs of its root and ledger descendants, so a private session is never
  listed in heartbeats_list or relaunched by the wake. Public sessions
  are unaffected because heartbeat_set/cron_add promote owned sessions
  before their jobs land.
- The wake enumeration now reads topology from the RLM spawn ledger's
  family (roots plus ledger descendants, fork headers stripped) instead
  of the saved-session catalog, which missed nested subagent transcripts
  entirely and misassigned passivated forks to their former parent
  through header lineage.
- The wake schedule stays disarmed while an update restart is being
  prepared instead of spinning a zero-delay timer on overdue jobs; the
  failed-preparation transition re-arms it once, and a successful
  restart re-arms at the next boot.
- One corrupt descendant artifact no longer strands the remaining
  descendants' registrations in a fresh worker.
- Managing a passivated heartbeat now works: heartbeat_manage resolves
  the job through the same durable enumeration and mutates its
  scheduled-jobs store directly, without waking the session.

ENG-5846
Comment pass over both halves: every added comment either deleted or cut
to one load-bearing line. Test pass: the paused/active heartbeat eviction
test duplicated the sweep pin's heartbeat eviction (the distinction has
no code path anymore) and the next-fire label pin merged into the
heartbeat aggregation test it shares fixtures with.
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
…he wake

The ephemeral-stop cancel derived descendant session ids from filenames,
while scheduled-job artifacts key on the persisted session id: a child
whose id differs from its file name kept armed jobs past its owner's
departure. The cancel now walks the same ledger family() rows the wake
enumeration uses, so both sides share one id resolution.

The cancel also ran after the worker left the worker map, leaving a
window (or, after a swallowed failure, forever) in which the tree looked
non-resident with active jobs and could be woken publicly. It now runs
before the map removal - every enumeration observed during the awaited
cancel still sees the tree covered - and a failed cancel parks the root
in pendingEphemeralCancels: enumeration retries the cancel first and
keeps the tree excluded until it lands.

ENG-5846
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
…tale cancel intents

Unscoped cron_cancel probed only resident workers, so a passivated
session's still-firing job could not be stopped without reopening the
session; it now falls back to the passive enumeration and cancels
through the per-job store API, broadcasting so the wake timer drops it.
Unscoped cron_list merges the same passive jobs after the resident
sweep (worker copies win). With these, no unscoped job surface in the
protocol misses passive sessions.

A parked ephemeral-cancel intent also outlived its premise: reopening
the tree after a failed cancel would let the enumeration retry destroy
the fresh worker's new schedules. The retry now drops the intent when a
worker covers the tree again - the store is worker-owned from that
moment and resident trees are excluded from enumeration anyway.

RES-1250
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
One malformed scheduled-jobs.json aborted collectPassiveScheduledJobs
entirely, so every other passive session's jobs vanished from listings
and their wakes never armed. Ref: RES-1250
A failed ephemeral cancel was parked only in memory, so a supervisor
restart lost the intent and the next boot recovered the private tree's
jobs wake-eligible. The stop now deletes the persisted tombstone only
after the cancel settles; boot finishes the cancel from the tombstone,
with the same covered-tree exemption that protects a reopened session's
new schedules. Ref: RES-1251
Resident workers honor includeInactive but the passive merge only
contributed active/paused jobs, so passivation silently shrank an
inclusive listing. The wake path keeps considering active jobs only.
Ref: RES-1250
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
promoteOwnedWorker can clear ownerClientId while the cancel awaits the
ledger family read, so a successful promotion had its now-public
schedules cancelled out from under it. The guard runs in the same
synchronous turn as the store walk; promotion commits are synchronous
single-mutation blocks, so a promote lands either wholly before the
guard (cancel skipped) or wholly after the walk. Ref: RES-1251
… its jobs

Passivity was judged by the top ancestor alone, so a job owned by a
mid-tree session with its own resident worker still counted passive and
the wake launched a competing root worker over an active subtree. The
walk now checks findWorkerBySessionFile at every hop; only a fully
uncovered chain stays wake-eligible. Ref: RES-1250
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
The round-5 stillWanted guard only ran when the family read succeeded:
a promotion landing during a throwing read was still parked, and the
enumeration retry cancelled the now-public schedules unguarded. The
catch now settles instead of parking when ownership is gone, and the
retry passes stillWanted from the parked worker's descriptor - the same
object a late promotion mutates and persists, which the boot gate
already reads. Ref: RES-1251
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
The in-memory park/retry map duplicated a truth that already lives on
disk: the kept tombstoned client-owned descriptor. Enumeration now
re-derives retry intents (and the still-pending tree exclusion) from
descriptorDir alone, and the covering-worker exemption moves inside the
destructive walk itself, checked per tree member - chain-complete where
the old root-only retry check missed a replacement worker reopened on a
child file - so stop-time, retry, and boot cancels share one guard.
Promotion self-corrects in memory and on disk because promoteOwnedWorker
replaces and persists the descriptor. Ref: RES-1251
The wake scan only enumerates the supervisor's sessions root, so a
public worker whose root file lives in a custom session dir would
passivate and never wake for its schedules. Eviction eligibility now
matches wake enumerability: such a worker with a registered heartbeat
or cron job stays resident, exactly the pre-PR behavior for the blind
set. create.config.sessionDir is client-reachable (daemon CLI
--session-dir). Ref: RES-1250
Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1723f5e. Configure here.

Comment thread packages/coding-agent/src/modes/daemon/daemon-supervisor.ts Outdated
VincentBailly pushed a commit to VincentBailly/prime-agent that referenced this pull request Sep 2, 2026
… section

Fork-local workaround. Do not merge upstream. Delete it once upstream
classifies finished agents correctly.

Finished agents stay in the agents-view Running section forever. The
supervisor ledger only re-publishes a roster row when the summarizer
verdict text changes, so a finished agent keeps activity "working"
(label "classifying") and a frozen rosterStatus "running". This guard
trusts "running" only when a hard busy signal still backs it.

Greppable marker sites (both carry the marker LOCAL PATCH(agents-view-done-in-running)):
  packages/coding-agent/src/modes/agents-view/agents-view-state.ts
  packages/coding-agent/test/agents-view-done-in-running.test.ts

Drop test, one command: delete the guard block in agents-view-state.ts, then run

  npx vitest --run packages/coding-agent/test/agents-view-done-in-running.test.ts

If it still passes without the guard, upstream fixed the bug: delete the
guard block AND the whole test file.

Upstream refs: PrimeIntellect-ai#1873, PrimeIntellect-ai#1872, PrimeIntellect-ai#1967.
Caused by 8d5722e / 1d2e91d.
snimu added a commit that referenced this pull request Sep 2, 2026
The descendant tally read row sections after
propagateHeartbeatStateToAncestors had promoted idle ancestors to
running, so a heartbeat-active grandchild counted its promoted parent
too. The tally now runs before the promotion pass and counts
intrinsically busy rows only; once that propagation pass is deleted
(#1967) the ordering is a no-op.

RES-1253
The gate lived at the sweep's candidate build only, so the fenced
recompute after the mutation drain and the last-detach empty eviction
could still passivate a custom-dir worker into the blind spot. The fact
moves into the WorkerEvictionSnapshot (hasWakeBlindSchedule, refused by
canEvictWorker) so both sweep sites read one policy, and the last-detach
path - a different input shape built from empty-draft summaries, not
eviction snapshots - consults the same isWakeBlindScheduledWorker helper
in its own eligibility predicate. Call-site check deleted. Ref: RES-1250
…ptor

workerEvictionSnapshot now computes the required hasWakeBlindSchedule field, which walks descriptor.sessionFile ?? createCommand.sessionPath. createCommand is required on DaemonWorkerDescriptor and every real descriptor carries it; the fixture impersonated one through an unknown-cast without it. Fix the fixture, not the type contract.
@sethkarten
sethkarten self-requested a review September 3, 2026 15:19
@sethkarten
sethkarten merged commit d72beaf into main Sep 3, 2026
25 checks passed
@sethkarten
sethkarten deleted the fix/heartbeat-idle-classification branch September 3, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants