Reserved address space is not a hole in the walk's coverage - #183
Merged
Merged
Conversation
A user-mode heap walk answered `coverage: Partial` on every healthy live process. What held it there was not damage: it was the tails of subsegments and page ranges, reserved and never committed, filed as `Unreadable` because *would not read* was the only thing the walk could observe. A signal that fires on everything says nothing, which cost the one signal the walker has for "we could not see something". Three states hide behind one failed read — reserved, committed and paged out, committed and present — and the allocator's own records answer only the first split, from three structures that move between builds (`CommittedPageCount`, a VS `CommitBitmap`, an LFH commit state at `CommitStateOffset`). The memory manager answers all of it in one call and answers about the target rather than about what one allocator believes, so `DebugEngine::virtual_region` wraps `IDebugDataSpaces2::QueryVirtual` and `PoolState::Uncommitted` joins `Unreadable`, with `PoolState::is_coverage_gap` the single definition of which gap costs a walk its `complete`. Only a positive `MEM_RESERVE`/`MEM_FREE` excuses a gap. A failed query, a run that cannot advance, a state this crate does not name and a source that cannot be asked are each `None`, and `None` keeps the conservative reading — the excuse is never granted by an absence of evidence. Kernel sessions are not asked at all, so the pool walk is unchanged there, which for paged pool is the truth. A free chunk whose middle the allocator decommitted is the same question reached another way: it runs past the committed extent it starts in, and `walk_vs` emitted no span for it and cleared `complete` with no diagnostic. A span is geometry and state, both known there, so where the tail holds nothing the chunk is now reported; where it is memory the process has it is still refused, and now the walk names the chunk it dropped. The query's output buffer must be 16-byte aligned, which `MEMORY_BASIC_INFORMATION64` is not: the engine's live path copies its answer out with three `movaps` stores and faults inside dbgeng otherwise, while the dump path copies field by field and never complains. Testing it on a dump proves nothing about it. Measured on a live 26200 process (`sihost`, four Segment Heaps, 20,426 chunks, 2026-09-24): all 33 gaps `MEM_RESERVE`, an allocated chunk and a free one both `MEM_COMMIT`, and an answer that had been `Partial` is `Complete`. Checked from the other end on a thin dump of the same process: an address whose page the dump does not carry still answers `Committed`, the read still fails, and the walk still counts it. windbg-mcp FOLLOWUPS item 98. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUhLUt9rB6zd42Y25btB3h
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A user-mode heap walk answered
coverage: Partialon every healthy live process. Not damage: the tails of subsegments and page ranges — reserved and never committed — read exactly the way a paged-out page does, and would not read was the only thing the walk could observe. A signal that fires on everything says nothing, so this cost the walker the one signal it has for we could not see something.Three states, one observation
The allocator's own records answer only the first split, and from three different structures that move between builds: a page range descriptor's
CommittedPageCount, a VS subsegment'sCommitBitmap, an LFH subsegment's commit state atCommitStateOffset. The memory manager answers all of it in one call, and answers about the target rather than about what one allocator believes.So
DebugEngine::virtual_regionwrapsIDebugDataSpaces2::QueryVirtualas a typedVirtualRegion/VirtualState, andPoolState::UncommittedjoinsUnreadable, withPoolState::is_coverage_gapas the single definition of which gap costs a walk itscomplete.Only a positive
MEM_RESERVE/MEM_FREEexcuses a gap. A failed query, a run that cannot advance, a state the crate does not name, and a source that cannot be asked at all are eachNone, andNonekeeps the conservative reading — the excuse is never granted by an absence of evidence. Kernel sessions are not asked (QueryVirtualis a user-mode question), so the kernel pool walk is unchanged, which for paged pool is the truth.The second half: a chunk with a decommitted middle
A free chunk whose middle the allocator decommitted runs past the committed extent it starts in.
walk_vsemitted no span for it and clearedcomplete— with no diagnostic, so nothing in the answer said why. A span is geometry and state, both of which are known there (header read, size out of that header and past the subsegment bound, state from the free tree); the only thing missing is the chunk's contents, which no span carries. Where the tail holds nothing the chunk is now reported; where it is memory the process has it is still refused, and now the walk names the chunk it dropped.The alignment trap
QueryVirtual's output buffer must be 16-byte aligned, whichMEMORY_BASIC_INFORMATION64is not. The engine's live-target path copies the 48-byte answer out with threemovapsstores and takes an access violation inside dbgeng; the dump path copies field by field and never complains. The same call answered 22 queries from an 8-aligned buffer against a full dump before it was ever pointed at a live process — testing this on a dump proves nothing about it. Measured on 26200, 2026-09-24:dbgeng!Ordinal367+0x14f96,movaps xmmword ptr [rbx],xmm0.Measured
Live 26200 process (
sihost, four Segment Heaps, 20,426 chunks, 2026-09-24), through the newexamples/heap_coverage.rs:All 33 gaps confirmed
MEM_RESERVEby a query independent of the one the walk made; both controlsMEM_COMMIT. Before this change the same target wasPartialwith 45–47 unreadable gaps and five chunks with decommitted middles.The dump direction, checked from the other end on a thin dump of the same process — this is the case that must not be swept up:
A committed page the dump does not carry still answers
Committed, the read still fails, and the walk still counts it against coverage.Tests
Four new cases in
src/pool/snapshot.rsplus one insrc/pool/index.rs, all differential — one fixture past three memory managers, so the only thing that varies is the answer:test_only_the_memory_manager_decides_whether_a_gap_costs_coverage— says-nothing-there / says-committed / cannot-say, and one query per run rather than per page.test_a_gap_with_nothing_behind_it_is_not_worth_a_diagnostictest_a_gap_straddling_the_commit_boundary_is_split_at_ittest_a_commit_run_that_cannot_advance_is_no_answer_at_all— deleting the guard hangs rather than fails, verified, which is why the guard is there.test_a_chunk_running_into_nothing_is_still_a_chunkMutation-checked: making
Uncommittedcount against coverage fails two of them; making thewalk_vsgate unconditional fails two more.PoolMemory::committed_rundefaults toNone, so every fixture written before this keeps its old meaning and the ones exercising the new state have to say so.435 tests pass;
cargo fmt --all -- --checkclean;cargo clippy --all-targets -- -D warningsreports the same 12 pre-existing findings asmainon this toolchain and none from this change.glslang/windbg-mcp FOLLOWUPS item 98.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MUhLUt9rB6zd42Y25btB3h