Proposal: suspension ownership (replaces ResumeLedger)

PR #41, branch feat/fast-logpoint-40, on top of 5d54490. Status: design for adversarial review, not implemented.

1. Problem

JDI suspend counts are fungible. EventSet.resume(), VirtualMachine.resume() and ThreadReference.resume() decrement counts. None of them names the stop it releases. Today the tools resume blindly: jdwp_resume and jdwp_resume_until_event call vm.resume(), while jdwp_resume_thread and the step tools call thread.resume().

An event set waiting for the dispatcher still holds a suspension of its own. A blind tool resume can use that suspension up. The set's own resume then releases a newer stop: the late-resume steal, pre-existing on main through JDI's dequeue filter.

The waiting rule of 9d77e6a only inferred that a suspension was used up. The ResumeLedger of 5d54490 counted tool resumes. Both guess which suspension a resume consumed, and both are wrong when a thread carries several suspensions (review of 5d54490, finding 3). Example:

• The user holds an exception stop on W, which suspends only W's thread.
• A queued stop-all set from T adds a second suspension to W.
• One jdwp_resume takes W from two suspensions to one.
• The ledger marks the queued set as spent. W stays frozen and no stop is reported.

2. Principle

Every suspension the debugger creates has exactly one owner, and only the owner releases it, exactly once. Tools never resume blindly. They release the stops the user holds.

This is the model IntelliJ's SuspendManager uses (suspend contexts, resumed individually; threads resumed out of a suspend-all context are re-suspended before the context's vm.resume()).

Owners:

Consequence: a queued set's suspension can only be released by its own resume. Its thread cannot run, and cannot stop again, until the dispatcher settles the set. So the dispatcher always resumes what it owes. The waiting rule, the claim, spent and ResumeLedger all go.

3. SuspensionRegistry (new @Service, replaces ResumeLedger)

One monitor guards all state and spans every JDWP resume/suspend it issues. The monitor is never held across event processing or an invocation.

State, reset on every listener start:

• routed: entries for sets handed to the dispatcher (queued or in flight). Each entry is {EventSet, policy, reporting thread, released: threads resumed out of a stop-all set}.
• inFlight: the entry being processed. It may carry releaseAll or a set of threads whose release was requested.
• holds: an ordered list. Each hold is either an event hold {entry} or a manual hold {thread}.

Listener and dispatcher operations:

• route(set) → entry: the listener thread, when it hands a set to the dispatcher.
• begin(entry): the dispatcher, before process().
• finish(entry, stop): the dispatcher, after process().
  • Not a stop: release the set.
  • A stop with releaseAll requested meanwhile: release the set.
  • A stop with only some threads requested: make it a hold, then apply those thread releases (below).
  • Otherwise: make it a hold.
  • Clear inFlight.

Tool operations:

• releaseAll() (jdwp_resume, jdwp_resume_until_event):
  • Release every hold, oldest first, and clear holds.
  • If a set is in flight, mark it releaseAll. Its stop, published before processing ended (the classpath pre-warm runs after the latch), is released when processing ends.
  • Queued sets are not touched.
  • Returns what it released or deferred.
• releaseThread(T) (jdwp_resume_thread, the step tools):
  • A thread-policy event hold of T, or a manual hold of T: release it and remove it.
  • A stop-all event hold (any reporting thread) whose released does not contain T: T.resume(), then add T to released. The hold stays for the other threads.
  • An in-flight set that suspended T: request T's release, applied in finish. T may be the thread the evaluation runs on, so it cannot be resumed earlier.
  • A queued set is not touched.
• suspendThread(T) (jdwp_suspend_thread): T.suspend(), then add a manual hold.

Releasing a set:

• A stop-all set with a non-empty released: first suspend() each released thread, so the set's vm.resume() nets zero on it. Then EventSet.resume().
• Any other set: EventSet.resume().

JDI exceptions on dead threads are caught per thread.

4. Issue #43 folded in

Without a blind vm.resume(), the user's escape hatch for #43 is gone. In #43, another thread's stop-all set stalls an evaluation in flight. So #43 must be fixed:

When the listener routes a stop-all set, every thread currently inside an MCP-driven evaluation (EvaluationGuard) gets one resume() and is added to the entry's released. This cancels exactly the increment the set added on that thread:

• inside an invocation, the thread had count 0, the set made it 1, and the resume brings it back to 0, so the invocation continues;
• between invocations, the thread had count 1 (its own set), the set made it 2, and the resume brings it back to 1.

When the set is later released, the thread is re-suspended first, so the net effect on it stays zero.

Sets raised by an evaluating thread itself are still suppressed inline.

EvaluationGuard keys are thread IDs. The listener resolves them to ThreadReferences with vm.allThreads() only when a stop-all set is routed while an evaluation runs.

5. Tool semantics (user-visible changes)

Fallback: when nothing is held and nothing is routed (no queued or in-flight set), jdwp_resume, jdwp_resume_until_event and jdwp_resume_thread keep the old blind resume. That covers suspensions the registry never saw. It is safe exactly when no set owns a suspension. The remaining race is a set routed between the check and the resume: the same narrow window JDI's own filter has.

Side effect, a fix: after stepping from a stop-all breakpoint, one jdwp_resume now resumes every thread. Before, the other threads kept one suspension per step.

6. Removed

• ResumeLedger, the Claim, spent, the waiting map and the "later set waiting" rule.
• The SUSPEND_NONE exclusion of 5d54490 is moot: nothing is inferred from later sets. A none set is routed and settled like any other, and its resume is a no-op.

7. Invariants (to test)

1. A queued or in-flight set is released exactly once, by the dispatcher, or by finish when a release was requested.
2. A tool never decrements a suspension it does not hold.
3. After a stop-all hold is released, every thread released out of it has the same count as before the hold existed.
4. One jdwp_resume while the user holds stops on several threads, and several stop-all and thread-policy sets are queued, leaves:
   • every held thread running;
   • every queued set's threads suspended until each set is processed;
   • every count back to its pre-event value once all sets are settled.
5. An evaluation in flight continues when another thread's stop-all set is routed (A stop-all breakpoint on another thread stalls an in-flight logpoint/condition evaluation #43).

8. Known limits

• An MCP-driven evaluation that starts after a stop-all set is held, on a thread that set suspended: the invocation keeps count ≥ 1 and stalls until the set is released. This is pre-existing. SYNC dispatch means the user cannot resume meanwhile.
• A set routed between the fallback's "nothing routed" check and its blind resume (see § 5).
• Holds of a thread that died are dropped when their release fails.

9. Tests

• Registry unit tests over a fake suspend-count model (a test double that tracks per-thread counts for vm.resume, thread.resume/suspend, EventSet.resume by policy), asserting final counts rather than mock calls:
  • review 3 finding 3 (W case);
  • releaseThread out of a stop-all hold, then releaseAll;
  • a step chain;
  • a manual hold;
  • in-flight release (releaseAll and releaseThread during processing, with the stop published before finish);
  • the fallback.
• Listener tests:
  • a queued set is always resumed by the dispatcher after a tool resume (findings 2/3 of review 2);
  • A stop-all breakpoint on another thread stalls an in-flight logpoint/condition evaluation #43: a stop-all set routed during a blocked evaluation releases the evaluating thread, and the evaluation completes;
  • setSuspendingNothingIsNoStop stays green.
• JDWPTools tests adapted: resume / step / suspend tools go through the registry. Existing vm.resume() / thread.resume() verifications hold through the fallback when nothing is held.
• Live FastLogpointLiveTest 15/15.

10. Docs

• docs/event-pipeline.md § 1: ownership replaces the late-resume rules.
• docs/event-pipeline.md § 9: resume via the registry.
• docs/threading-and-safety.md: the registry monitor; A stop-all breakpoint on another thread stalls an in-flight logpoint/condition evaluation #43 resolved.
• docs/architecture.md and CLAUDE.md: SuspensionRegistry replaces ResumeLedger.
• Tool descriptions of jdwp_resume, jdwp_resume_thread, jdwp_suspend_thread, jdwp_resume_until_event.
• The java-debug skill text, if it describes resume semantics.
• Issue Event set spent while queued is still processed as a stop #44 becomes moot: a queued set's suspension can no longer be spent. Close it with a pointer to this commit. Issue A stop-all breakpoint on another thread stalls an in-flight logpoint/condition evaluation #43 is closed by § 4.

The SUSPEND_NONE exclusion closes prior finding 1. Prior findings 2–3 remain reproducible during processing.

1. MAJOR — Resumes during processing can still release a newer stop.

   Location: JdiEventListener.java:484, ResumeLedger.java:102.

   spent is sampled before processing. The final synchronized resume does not revalidate ownership; it assumes every intervening tool resume missed the reporting thread inside an invocation. Processing includes periods before, between, and after invocations.

   Failing scenario: A is a SUSPEND_EVENT_THREAD logpoint on T. Its evaluation finishes, and the dispatcher observes no later set in waiting. Before it acquires the ledger for its final resume, jdwp_resume consumes A’s suspension. T hits breakpoint B, which the listener fully routes. A then calls eventSet.resume() unconditionally, releasing B. B is subsequently reported although T runs. Both sets were routed; this does not require the documented event-to-routing residual.

   The cross-thread version also survives: while SUSPEND_ALL A is processing outside an invocation, a VM resume consumes it and U hits SUSPEND_ALL B. A’s thread-only fallback releases T from B’s suspension.

   Fix: Track whether an intervening resume actually consumed the claimed suspension, accounting for actual invocation resume/restoration transitions. Revalidate that ownership under the same monitor as the final resume. The broad evaluation guard also covers compilation and cannot establish that a thread was running.

   Test gap: Pause after the waiting check, resume through the ledger, fully route B, then finish A. Assert B’s suspension remains. Existing tests exercise the favorable case where the resume occurs inside the mocked invocation.

   Confidence: High.

2. MAJOR — A queued set can be declared spent while an MCP worker is invoking on its thread.

   Location: ResumeLedger.java:93, JDWPTools.java:1159.

   The assertion that nobody can invoke on a queued set’s thread is false. Evaluation tools accept an explicit thread ID and require suspension; they do not require the listener to have published that stop.

   Failing scenario: The dispatcher is occupied on U. Logpoint A suspends T and is claimed and queued. An MCP worker calls jdwp_evaluate_expression(T, ...) and enters a single-threaded invocation. Another worker calls jdwp_resume while T’s suspend count is zero. The ledger increments its VM counter although that resume cannot release T’s original suspension. The invocation returns and suspends T again. After the worker finishes, A is processed and its logpoint evaluated, but spent(claim) prevents its owed resume. T remains suspended indefinitely with no stopping event reported. The invocation’s resume/re-suspend behavior follows the JDI contract.

   This differs from issue Event set spent while queued is still processed as a stop #44: A’s suspension was never spent.

   Fix: Include worker-driven invocations in suspension accounting, including invocations started after a claim was queued. Count effective consumption of suspension debt rather than treating every VM-resume command as consumption for every claim.

   Test gap: Run a worker invocation on an already queued thread, resume the VM during that invocation, then assert automatic cleanup leaves T’s suspend count zero.

   Confidence: High.

3. MAJOR — One VM resume incorrectly settles multiple suspension increments on the same thread.

   Location: ResumeLedger.java:94.

   Any VM-counter change invalidates every earlier claim, regardless of overlapping suspension scopes. One resume decrements a thread’s pending suspend count only once. See JDI resume semantics.

   Failing scenario: While the dispatcher is busy compiling another logpoint:

   • A suspends T with SUSPEND_EVENT_THREAD and is claimed.
   • B on U suspends all threads and is claimed. T now has suspend count 2.
   • Both requests are cleared, and jdwp_resume runs once. T’s count becomes 1.
   • The dispatcher drops both deleted requests. Both claims are marked spent, so neither resumes anything.

   T retains a suspension forever, and neither deleted event reports a stop. This uses a VM resume, so it is outside the documented non-reporting-thread-resume residual.

   Fix: Represent remaining suspension debt per affected thread. A single VM resume can consume at most one increment per thread; overlapping claims cannot all become fully spent from that command.

   Test gap: Model overlapping thread-policy and all-policy suspensions and assert final suspend counts. Mockito call verification cannot detect this lost decrement.

   Confidence: High.

4. MAJOR — The suppression path bypasses ownership accounting and can double-resume.

   Location: JdiEventListener.java:631.

   Suppressed sets receive no claim and resume directly, without synchronization against tool resumes.

   Failing scenario: Logpoint A invokes code on T. Nested breakpoint B suspends T, and the listener begins suppressing B. Pause the listener after recording suppression but before line 631. A tool VM resume releases B, allowing A’s invocation to finish. The dispatcher correctly resumes A’s restored suspension, and T reaches ordinary breakpoint C. The listener’s delayed B.resume() then releases C. C is later reported although T runs.

   The tool resume occurs after suppression has begun, not before routing. This also remains reproducible independently of finding 1: A’s suspension really was restored by its invocation.

   Fix: Claim suppressed suspensions and settle them through an atomic ledger operation that recognizes intervening consumption. Suppression performs no invocation, so it does not need the processing-window exception.

   Test gap: Delay a suppressed set’s final resume, issue a tool resume, let the outer invocation complete and T reach C, then assert C remains suspended.

   Confidence: High.

Verdict: do not merge.

1. BLOCKER — An in-flight release can discard a stop that has not been published yet.

   Design: §3, releaseAll() and finish(); §5, jdwp_resume_until_event.

   Defect: inFlight does not distinguish processing from a published stop. Marking every in-flight entry releaseAll grants a resume request authority over a future user stop.

   Failing scenario: W has a held exception stop. B on T is processing a condition and has not fired the latch. jdwp_resume_until_event arms its latch, releases W, and marks B releaseAll. B’s condition succeeds; B publishes its stop and wakes the caller. finish() immediately resumes B. The caller receives a stop that disappears without another resume request.

   Proposed change: Give publication an explicit transition under the registry monitor. A release request may cover only holds and stops already published at that transition. Bind deferred releases to entry identity and publication generation. Publication racing with resume must resolve either as “released existing stop” or “new held stop.”

   Confidence: High.

2. BLOCKER — Ownership omits suspension temporarily borrowed by an invocation.

   Design: §§2–4; §3’s unconditional non-stop release.

   Defect: A logically owned suspension need not currently exist in the native suspend count. HotSpot’s single-threaded invocation decrements once on entry and increments once on completion. Calling resume() while that count is zero does not discharge the later restoration. OpenJDK invocation implementation.

   Failing scenario: With compiler configuration already warm, logpoint A suspends T and waits behind dispatcher work on U. An MCP evaluation explicitly targets T and enters an invocation: T’s count becomes zero. The dispatcher reaches A, sees EvaluationGuard, suppresses its handling, and settles A with resume() at zero. The worker invocation returns and restores count one. A has been removed, but T remains suspended without an owner or reported stop. Fast-logpoint arming’s “already evaluating, rearm and return” path has the same exposure.

   A second variant needs no queued evaluation: a resume deferred during published-stop processing returns; the next serialized MCP call starts evaluating that thread; the dispatcher then performs the deferred release during the worker’s invocation.

   Proposed change: Add invocation leases tied to suspension contexts. Every settlement path must respect borrowed suspension, including dispatcher auto-resume. For a smaller implementation, reject evaluations on unsettled entries and pin accepted holds until invocation restoration completes. Cover evaluator, prewarm, force-load, to_string, and arming invocations.

   Confidence: High.

3. BLOCKER — The fallback reintroduces the late-resume failure, including JDI’s invisible dequeue resume.

   Design: §5 fallback; §8’s claimed remaining window.

   Defect: An empty registry does not establish that no set owns a suspension. A suspending packet can already be waiting inside JDI before route() exists. The exposure lasts for the entire unread interval, not merely a check-to-route instruction window.

   Failing scenario: T hits A; its packet remains unread. A is cleared. With no registry entries, the fallback resumes T. T reaches B and stops again. JDI subsequently builds A’s set, filters its deleted request, and automatically resumes it—consuming B’s suspension. A never reaches the registry. This automatic resume is present in OpenJDK’s EventSetImpl.build().

   Proposed change: Remove implicit blind recovery from ordinary resume tools. Report “no registered stop to release.” Handle initial VMStart through explicit attach/startup coordination, so removing the fallback does not strand an initial stop that arrives after connect returns. Unknown suspension recovery should be an explicitly destructive recovery operation, such as reconnect, with different guarantees.

   Confidence: High.

4. MAJOR — Deferring all in-flight releases until finish() removes a necessary deadlock escape.

   Design: §3’s deferred releases; §4; §8.

   Defect: Processing may require another thread to run before it can finish. Deferring that other thread’s release creates a circular dependency.

   Failing scenario: A SUSPEND_ALL breakpoint on T is published. Prewarm invokes on T and blocks acquiring a classloader or application lock held by U, which A suspended. releaseAll() only marks A for later release. releaseThread(U) also waits for finish(), although U is not the invocation target. Prewarm cannot finish until U runs; U cannot run until prewarm finishes. No additional event is needed, so §4 cannot help.

   Proposed change: Track the actual invocation target and permit immediate, accounted partial release of other threads from a published context. Defer only debt currently borrowed by an invocation. Removing speculative prewarm after publication would substantially reduce this problem. Deferred replies must identify the blocked invocation rather than promise eventual execution.

   Confidence: High.

5. MAJOR — The A stop-all breakpoint on another thread stalls an in-flight logpoint/condition evaluation #43 fix only handles evaluations that were registered before routing.

   Design: §4; §8’s first limit.

   Defect: Evaluation entry is not coordinated with existing held, queued, or in-flight stop-all contexts. The documented limit therefore extends beyond already-held sets.

   Failing scenario: T has a thread-policy stop A. B on U suspends all threads and is routed while T is not evaluating. T now has count two. An MCP evaluation starts on T. Its invocation decrements to one and stalls. B can subsequently become held, leaving the serialized MCP worker unable to issue a release.

   Proposed change: Make evaluation admission a registry operation coordinated with routing. Either arrange temporary exemptions from every additional applicable context or fail before invoking, naming the blocking contexts. A count greater than one must not become an indefinitely blocked MCP call.

   Confidence: High.

6. MAJOR — Resolving evaluating IDs through vm.allThreads() can miss the evaluation target.

   Design: §4’s final paragraph.

   Defect: JDI does not guarantee that allThreads() includes virtual threads. Their inclusion is implementation/configuration dependent. JDI allThreads() contract.

   Failing scenario: A virtual thread T is executing a dispatcher-driven logpoint invocation. Another thread raises a stop-all set. T is in EvaluationGuard but absent from allThreads(), so §4 never resumes it. The dispatcher remains blocked waiting for T and cannot process the queued set.

   Proposed change: Store the actual ThreadReference in a session-scoped evaluation lease when entering the guard. Retain the captured ID for exception-safe cleanup. Do not reconstruct known evaluation targets by enumerating the VM.

   Confidence: High.

7. MAJOR — Evaluation exemptions become permanent user resumes.

   Design: §4’s reuse of released; §§2, 5, and 7.

   Defect: released conflates “the user requested this thread run” with “temporarily allow this invocation to finish.” Nothing restores the latter when evaluation ends.

   Failing scenario: T evaluates thread-policy logpoint A. Stop-all breakpoint B on U is routed and exempts T. A finishes and auto-resumes. B is then reported and held, but T continues application execution throughout that stop. Inspection of the supposedly stopped VM can observe changing shared state.

   There is also a liveness boundary: keeping HotSpot’s global suspend-all state means newly started threads inherit suspension. An exempted invocation that starts V and joins it can still deadlock while B waits behind that invocation. OpenJDK suspend-all bookkeeping.

   Proposed change: Separate temporary evaluation exemptions from explicit user releases. Restore temporary debt at coordinated evaluation exit if its context remains active. Define the policy for invocation-created threads and dependencies; resuming only existing evaluation targets does not establish general evaluation liveness. If permanent exemptions are intentional, expose them in stop replies and abandon the whole-VM-stop guarantee.

   Confidence: High.

8. MAJOR — Resetting singleton state on start() does not isolate connection generations.

   Design: §3’s reset rule; §§8–10.

   Defect: The current listener uses bounded shutdown and shutdownNow(); an old dispatcher can finish afterward. Clearing shared registry fields does not revoke its authority.

   Failing scenario: Old-session A pauses immediately before finish(). Reconnect disposes that VM, resets the registry, and starts new-session B. A then finishes, potentially adding an old hold or clearing B’s inFlight. Subsequent resume accounting uses corrupted state. Unscoped evaluation IDs can likewise collide across VMs.

   Proposed change: Use a per-connection registry/session object captured by listener tasks and invocation leases. Closing it must invalidate later callbacks; old completion must never mutate the new registry. Explicitly define terminal ownership on death/dispose. For jdwp_reset and request clearing, preserve physical suspension debt until settlement while invalidating obsolete publication work—deleting a request must not delete its hold.

   Confidence: High.

9. MAJOR — A published stop is still unsafe for inspection and stepping.

   Design: §3’s in-flight handling; §5’s tool changes.

   Defect: Redirecting resume calls does not coordinate the tools’ existing suspension checks, frame reads, or step-request installation with post-publication prewarm.

   Failing scenario: jdwp_resume_until_event returns a breakpoint on T. Prewarm invokes on T, temporarily taking its count to zero. The next serialized jdwp_resume_thread or step call sees !thread.isSuspended() and returns before reaching the registry. A worker evaluation can instead race prewarm’s frame invalidation or attempt another invocation on the same target.

   Proposed change: Expose logical stop readiness separately from physical isSuspended(). Queue an entire step operation—including request creation and origin capture—until its context is ready, or return an explicit busy result before creating the request. Gate evaluation and frame inspection similarly. Prefer completing or removing speculative target work before advertising an inspectable stop.

   Confidence: High.

10. MINOR — The invariants and fake model cannot validate the stated guarantees.

Design: §§7 and 9.

Defect: Invariant 4 simultaneously requires previously held threads to run and queued stop-all scopes to retain their suspensions. Those requirements conflict. Invariant 3’s historical count comparison also ignores legitimate later stops.

Failing scenario: W has held exception A and queued stop-all B. Releasing A correctly leaves W suspended by B. Separately, T released from stop-all A can reach later stop C; settling A must preserve C, rather than restore T’s historical pre-A count.

Proposed change: Assert that native debt equals outstanding owners, adjusted for invocation loans and temporary exemptions. Extend the model with invocation entry/restoration, publication generations, global suspend-all state, newborn threads, and session disposal. Include both orderings of publication/release and interleavings with the balanced dump_locks probe.

Keep the compensation rule relative: re-suspending T before A’s VM resume must preserve a later C. Retain the actual VM resume to discharge global suspend-all bookkeeping. Skip dead-thread compensation individually without abandoning the remaining live threads’ context. Also specify whether repeated manual suspends accumulate holds and whether releaseThread releases one or all applicable user holds; replies must distinguish released, deferred, still blocked, and absent.

Confidence: High.

The smaller implementation is a session-scoped context registry with an atomic publication transition, invocation leases, and no ordinary blind fallback. Remove post-publication prewarm, initially reject evaluations on unsettled contexts, and defer complete step operations until their context is ready.

Verdict: implement with changes.