Repository navigation
openvmm: refuse virtio device buffers outside the e820 RAM ranges - #974
Open
Pedro Henrique Penna (ppenna) wants to merge 3 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 3 commits into
Pedro Henrique Penna (ppenna) wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The pin changes unsafe guest-memory access and all microVM virtio DMA paths, requiring maintainer review while the prerequisite PR remains unmerged and exact-pin cross-backend boot/restore validation is pending.
No open issues 🎉
What changed in this PR
Documents the OpenVMM pin update that restricts microVM virtio device DMA to permitted RAM ranges, addressing issue #554’s memory-layout enforcement requirements.
Changes:
- Updates the upstream roadmap with the new pin, five commits, and their upstreaming dependencies.
- Explains DMA restrictions, refused-buffer behavior, and device reset notifications in the architecture guide.
| File | Description |
|---|---|
doc/openvmm-upstream-roadmap.md |
Records the new pin and maps commits 353–357. |
doc/design/2.3-memory-layout.md |
Documents virtio DMA enforcement and refusal reporting. |
Review effort: Balanced
Pin microsoft/openvmm#4794 at bd2b27a2ef9339bd16692c93facb8d84952dc9f8, five commits on 832d5bb48, the previous pin, which is the head of feature/microvm/dev. The microVM memory layout requires that DMA ranges lie wholly inside a RAM range of the e820 map and cross neither the MMIO aperture nor a reserved boot structure, but each microVM virtio device received the whole guest memory. A descriptor failed only when its buffer left guest memory, as in the aperture, and then only for the part that did, so a guest could point a device buffer at the shared virtio interrupt-status page at 0x30000, which the devices themselves update, or at the ISA hole. The guest can write those pages directly, so this did not weaken isolation, but the devices enforced less than the layout states. The first commit adds guest memory views, which admit only the accesses that lie wholly inside a set of page-aligned ranges and refuse any other before touching memory, on every access path, page locks included, while sharing the mappings of the memory that they restrict, so that admitted accesses keep their zero-copy paths. The second makes a virtio queue check each descriptor's whole buffer, and each indirect table, against the device's memory before handing the chain to the device, and refuse the chain as malformed when the memory does not admit it, so that a device reads and writes none of a buffer that lies even partly outside the admitted ranges. Rings already lie in subranges that the memory refuses as a whole when a queue starts. The third lets an MMIO transport restrict its device to a set of ranges and report each refusal by setting DEVICE_NEEDS_RESET, with a configuration-change notification once the driver has set DRIVER_OK, as virtio 1.2 provides for a device in an error state. A refused ring fails DRIVER_OK and leaves DEVICE_NEEDS_RESET alone set. A reset clears the mark. The transport keeps the whole memory for the shared interrupt-status word. The fourth returns the RAM ranges of the microVM e820 map from the loader that builds it, and the fifth restricts every microVM virtio device to them: the layout's RAM, which after a restore includes the RAM that the restore activates, less the interrupt-status page and the ISA hole. The protocol vectors and built-in CPU profiles that nvx-rs copies are unchanged. Refs #554 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the five commits that the previous commit pins as positions 355-359 of the microVM chain, and move the NVX pin to bd2b27a2e. Positions 355-359 restrict the DMA of the microVM virtio devices to the RAM ranges of the e820 map: guest memory views, the virtio queue's whole-buffer check, the MMIO transport's restriction and needs-reset report, the loader's e820 RAM ranges, and the microVM worker that applies them. Position 355 changes only guestmem, which no other commit of the chain changes, so it can go upstream on its own, as positions 350-352 can, and position 356 adds only the queue's check on top of it. Positions 357-359 build on the fork's shared interrupt-status transport, MP-table loader, and microVM worker; say so in the table's remarks. The pin is now 202 commits past the mapped tip. Positions 353 and 354 are no longer only a proposal: microsoft/openvmm#4775 merged them, so position 354, 832d5bb48, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4794 as the proposal of positions 355-359. Refs #554 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Section 2.3 requires that loader writes and DMA ranges lie wholly inside a RAM range of the e820 map and cross neither the MMIO aperture nor a reserved boot structure, but said nothing about who enforces it. The Linux direct loader always kept its writes inside RAM; with the pinned OpenVMM, the virtio devices now enforce the rule for DMA too. Note the enforcement after the requirement: each device reaches only the RAM ranges of the e820 map and the RAM that a restore activates beyond them, and refuses a queue whose rings, or a descriptor whose buffer, lies even partly outside them, as in the interrupt-status page, the ISA hole, or across the aperture. It reads and writes none of it, stops serving the queue, and sets DEVICE_NEEDS_RESET, notifying the driver through a configuration change once DRIVER_OK is set. A refused ring fails DRIVER_OK instead, leaving DEVICE_NEEDS_RESET alone set. A device reset clears the mark, and only the transports update the interrupt-status page. Fixes #554 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-554-refuse-dma-outside-ram
branch
from
October 29, 2026 00:19
642e1a7 to
8457a99
Compare
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The pin changes unsafe guest-memory access and virtio DMA/reset behavior across KVM, MSHV, and WHP, requiring human review and cross-backend validation while exact-head CI remains pending.
No open issues 🎉
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #554. Pins microsoft/openvmm#4794 at
bd2b27a2ef9339bd16692c93facb8d84952dc9f8, which restricts the DMA of every microVM virtio device to the RAM ranges of the e820 map, and notes the enforcement in section 2.3 of the architecture guide. A device now refuses a queue whose rings, or a descriptor whose buffer, lies even partly in the shared virtio interrupt-status page, the ISA hole, or the MMIO aperture: it reads and writes none of it, and its transport marks it as needing reset.Based on
devat26860ef86. microsoft/openvmm#4794 is five commits directly on832d5bb48, the head offeature/microvm/devand the revision thatdevpins since #948 merged, and the pin is that pull request's head commit itself, so this branch's three commits are its whole diff.Change
677fe92c6openvmmopenvmm832d5bb48→bd2b27a2e: the five commits of microsoft/openvmm#47945d917731edocdoc/openvmm-upstream-roadmap.mdguestmem, so it can go upstream on its own, while positions 357-359 build on fork-only code; moves the NVX pin; and records that microsoft/openvmm#4775 merged positions 353 and 354 intofeature/microvm/dev8457a9916docdoc/design/2.3-memory-layout.mdNo other NVX change is needed: OpenVMM's control-session protocol vectors and built-in CPU profiles are unchanged at the pin.
OpenVMM changes
2607a7d55).GuestMemory::restrictreturns a view that admits only the accesses that lie wholly inside a set of page-aligned ranges and refuses any other before touching memory, on every access path, page locks and subranges included, with an optional refusal hook. The view shares the mappings of the memory that it restricts, so admitted accesses keep their zero-copy paths; memory that is not a view pays one moreNonecheck per access.GuestMemory::check_admittedchecks a range without touching memory.35af55902). A virtio queue checks each descriptor's whole buffer, and each indirect table, before handing the chain to the device, and refuses the chain withQueueError::BufferNotAdmitted, as malformed: the chain is not consumed and the queue retires its fetch side. A device thus reads and writes none of a buffer that lies even partly outside RAM, rather than the part inside. Rings lie in subranges that the view refuses as a whole when a queue starts.b7bec57cd).VirtioMmioDevice::restrict_dmagives the device the view; the transport keeps the whole memory for the shared interrupt-status word. A refusal setsDEVICE_NEEDS_RESET, with a configuration-change notification once the driver has setDRIVER_OK, as virtio 1.2 section 2.1.2 provides. A refused ring failsDRIVER_OKand leavesDEVICE_NEEDS_RESETalone set. A device reset clears the mark.09a68a784,bd2b27a2e). The loader returns the RAM entries of the map that it builds, and the microVM worker restricts every virtio device to them: the layout's RAM, which after a snapshot restore includes the RAM that the restore activates, less the interrupt-status page and the ISA hole.The issue's tasks map onto this as follows:
virtiotests refuse buffers in, and crossing into or out of, the interrupt-status page, the ISA hole, and the aperture, indirect tables and rings there, and check through the MMIO transport that the device writes none of a refused buffer, that the interrupt-status word changes only by the configuration-change bit, and that a reset clears the mark.guestmemandloadertests cover the view and the e820 RAM ranges.Guests that keep their buffers in RAM see no change. Only the microVM profile restricts device DMA.
Validation
d647d4de4,cargo clippy --all-targets -- -D warningsandcargo doc --no-depsforguestmem(with and withoutbitmap),virtio,loader, andopenvmm_core;cargo nextest run --profile agentforguestmem(7, and 9 withbitmap),virtio(147),loader(55),openvmm_core(118), andvirtio_blk,virtio_net,virtiofs, andvirtio_console(262 passed, 1 skipped); andcargo xtask fmt, all clean. After the rebase onto832d5bb48, atbd2b27a2e:cargo xtask fmt, clippy forguestmem,virtio,loader,virtiofs, andopenvmm_core, and nextest forguestmem,virtio,loader, andvirtiofs(322) andopenvmm_core(118), all clean.8457a9916:python scripts\nvx.py verify(consistent); theopenvmm_vectorsandopenvmm_cpu_profilescomparisons with--ignored(3 and 1 passed), against the pinned revision;python scripts\check_markdown_links.py; andgit diff --check.26860ef86after openvmm: let a platform restore attach an aggregate to a dormant filesystem slot #948 moveddev's pin to832d5bb48: [DO NOT REVIEW] microVM: refuse virtio device buffers outside the e820 RAM ranges openvmm#4794 was rebased onto that revision without conflicts (git range-diffshows its five patches unchanged, and [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775 changes none of their files), and this branch's three commits were made again ondev. The previous head,642e1a708, passed the full CI matrix with the same OpenVMM patches onb92daa350(run 40776227171, attempt 2): every microVM suite on Linux/KVM, Linux/MSHV, and Windows/WHP, which boot guests that use every microVM virtio device, the OpenVMM unit tests and VMM tests at the pin, and the benchmarks. Its first attempt's only failure was the Ubuntusnapshot-corescenario on Linux/KVM, which failed to parse a time ABI marker line,...last_sample_error=nonel_suppress)"; echo "NVX-""CANCELED-CAPTURE-DONE", into which the guest tty had echoed the scenario's next command line:canceled_capture_checkstypes two command lines at once, so the second line's echo can interleave with the first command's output. The same scenario passed on MSHV and WHP, and on Linux/KVM on rerun.bd2b27a2e, and this pull request's at head8457a9916f30211ff5e4c0613217ff496d8bb731.Merge microsoft/openvmm#4794 into
feature/microvm/dev, by fast-forwarding that branch tobd2b27a2e, before merging this pull request.