Skip to content

openvmm: refuse virtio device buffers outside the e820 RAM ranges - #974

Open
Pedro Henrique Penna (ppenna) wants to merge 3 commits into
devfrom
agents/issue-554-refuse-dma-outside-ram
Open

Pedro Henrique Penna (ppenna) wants to merge 3 commits into
devfrom
agents/issue-554-refuse-dma-outside-ram

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #554. Pins microsoft/openvmm#4794 at bd2b27a2ef9339bd16692c93facb8d84952dc9f8, which restricts the DMA of every microVM virtio device to the RAM ranges of the e820 map, and notes the enforcement in section 2.3 of the architecture guide. A device now refuses a queue whose rings, or a descriptor whose buffer, lies even partly in the shared virtio interrupt-status page, the ISA hole, or the MMIO aperture: it reads and writes none of it, and its transport marks it as needing reset.

Based on dev at 26860ef86. microsoft/openvmm#4794 is five commits directly on 832d5bb48, the head of feature/microvm/dev and the revision that dev pins since #948 merged, and the pin is that pull request's head commit itself, so this branch's three commits are its whole diff.

Change

Commit Files Change
677fe92c6 openvmm openvmm 832d5bb48 → bd2b27a2e: the five commits of microsoft/openvmm#4794
5d917731e doc doc/openvmm-upstream-roadmap.md Records them as positions 355-359, unassigned, with a remark that position 355 changes only guestmem, so it can go upstream on its own, while positions 357-359 build on fork-only code; moves the NVX pin; and records that microsoft/openvmm#4775 merged positions 353 and 354 into feature/microvm/dev
8457a9916 doc doc/design/2.3-memory-layout.md Notes after the layout's DMA rule that the virtio devices enforce it, how a device refuses a ring or a buffer outside RAM, and how it reports the refusal

No other NVX change is needed: OpenVMM's control-session protocol vectors and built-in CPU profiles are unchanged at the pin.

OpenVMM changes

  • Guest memory views (2607a7d55). GuestMemory::restrict returns a view that admits only the accesses that lie wholly inside a set of page-aligned ranges and refuses any other before touching memory, on every access path, page locks and subranges included, with an optional refusal hook. The view shares the mappings of the memory that it restricts, so admitted accesses keep their zero-copy paths; memory that is not a view pays one more None check per access. GuestMemory::check_admitted checks a range without touching memory.
  • Whole-descriptor refusal (35af55902). A virtio queue checks each descriptor's whole buffer, and each indirect table, before handing the chain to the device, and refuses the chain with QueueError::BufferNotAdmitted, as malformed: the chain is not consumed and the queue retires its fetch side. A device thus reads and writes none of a buffer that lies even partly outside RAM, rather than the part inside. Rings lie in subranges that the view refuses as a whole when a queue starts.
  • Restriction and reporting in the MMIO transport (b7bec57cd). VirtioMmioDevice::restrict_dma gives the device the view; the transport keeps the whole memory for the shared interrupt-status word. A refusal sets DEVICE_NEEDS_RESET, with a configuration-change notification once the driver has set DRIVER_OK, as virtio 1.2 section 2.1.2 provides. A refused ring fails DRIVER_OK and leaves DEVICE_NEEDS_RESET alone set. A device reset clears the mark.
  • The e820 RAM ranges (09a68a784, bd2b27a2e). The loader returns the RAM entries of the map that it builds, and the microVM worker restricts every virtio device to them: the layout's RAM, which after a snapshot restore includes the RAM that the restore activates, less the interrupt-status page and the ISA hole.

The issue's tasks map onto this as follows:

  • View or descriptor check. Both: the view enforces the RAM ranges on every access path, and the queue applies it to each descriptor's whole buffer before the device sees the chain.
  • Reporting. Marking the device as needing reset. An error completion cannot serve every device: the refused buffer may be the one that would carry the error, such as a virtio-blk status byte or a virtio-fs reply header, and virtio-net and virtio-console have no error completion.
  • Coverage. OpenVMM's virtio tests refuse buffers in, and crossing into or out of, the interrupt-status page, the ISA hole, and the aperture, indirect tables and rings there, and check through the MMIO transport that the device writes none of a refused buffer, that the interrupt-status word changes only by the configuration-change bit, and that a reset clears the mark. guestmem and loader tests cover the view and the e820 RAM ranges.
  • Architecture guide. Section 2.3 notes the enforcement.

Guests that keep their buffers in RAM see no change. Only the microVM profile restricts device DMA.

Validation

  • OpenVMM, on Windows: at d647d4de4, cargo clippy --all-targets -- -D warnings and cargo doc --no-deps for guestmem (with and without bitmap), virtio, loader, and openvmm_core; cargo nextest run --profile agent for guestmem (7, and 9 with bitmap), virtio (147), loader (55), openvmm_core (118), and virtio_blk, virtio_net, virtiofs, and virtio_console (262 passed, 1 skipped); and cargo xtask fmt, all clean. After the rebase onto 832d5bb48, at bd2b27a2e: cargo xtask fmt, clippy for guestmem, virtio, loader, virtiofs, and openvmm_core, and nextest for guestmem, virtio, loader, and virtiofs (322) and openvmm_core (118), all clean.
  • NVX at 8457a9916: python scripts\nvx.py verify (consistent); the openvmm_vectors and openvmm_cpu_profiles comparisons with --ignored (3 and 1 passed), against the pinned revision; python scripts\check_markdown_links.py; and git diff --check.
  • Every commit is signed, both these and [DO NOT REVIEW] microVM: refuse virtio device buffers outside the e820 RAM ranges openvmm#4794's.
  • Not run locally: NVX's microVM scenarios, since the Windows Hypervisor Platform is disabled on the Windows host on which this was prepared.
  • Rebuilt on 26860ef86 after openvmm: let a platform restore attach an aggregate to a dormant filesystem slot #948 moved dev's pin to 832d5bb48: [DO NOT REVIEW] microVM: refuse virtio device buffers outside the e820 RAM ranges openvmm#4794 was rebased onto that revision without conflicts (git range-diff shows its five patches unchanged, and [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775 changes none of their files), and this branch's three commits were made again on dev. The previous head, 642e1a708, passed the full CI matrix with the same OpenVMM patches on b92daa350 (run 40776227171, attempt 2): every microVM suite on Linux/KVM, Linux/MSHV, and Windows/WHP, which boot guests that use every microVM virtio device, the OpenVMM unit tests and VMM tests at the pin, and the benchmarks. Its first attempt's only failure was the Ubuntu snapshot-core scenario on Linux/KVM, which failed to parse a time ABI marker line, ...last_sample_error=nonel_suppress)"; echo "NVX-""CANCELED-CAPTURE-DONE", into which the guest tty had echoed the scenario's next command line: canceled_capture_checks types two command lines at once, so the second line's echo can interleave with the first command's output. The same scenario passed on MSHV and WHP, and on Linux/KVM on rerun.
  • Pending CI: [DO NOT REVIEW] microVM: refuse virtio device buffers outside the e820 RAM ranges openvmm#4794's checks at bd2b27a2e, and this pull request's at head 8457a9916f30211ff5e4c0613217ff496d8bb731.

Merge microsoft/openvmm#4794 into feature/microvm/dev, by fast-forwarding that branch to bd2b27a2e, before merging this pull request.

Copilot AI balanced review requested due to automatic review settings October 28, 2026 21:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The pin changes unsafe guest-memory access and all microVM virtio DMA paths, requiring maintainer review while the prerequisite PR remains unmerged and exact-pin cross-backend boot/restore validation is pending.

No open issues 🎉

What changed in this PR

Documents the OpenVMM pin update that restricts microVM virtio device DMA to permitted RAM ranges, addressing issue #554’s memory-layout enforcement requirements.

Changes:

  • Updates the upstream roadmap with the new pin, five commits, and their upstreaming dependencies.
  • Explains DMA restrictions, refused-buffer behavior, and device reset notifications in the architecture guide.
File Description
doc/​openvmm-upstream-roadmap.md Records the new pin and maps commits 353–357.
doc/​design/​2.3-memory-layout.md Documents virtio DMA enforcement and refusal reporting.

Review effort: Balanced

Pin microsoft/openvmm#4794 at bd2b27a2ef9339bd16692c93facb8d84952dc9f8,
five commits on 832d5bb48, the previous pin, which is the head of
feature/microvm/dev.

The microVM memory layout requires that DMA ranges lie wholly inside a
RAM range of the e820 map and cross neither the MMIO aperture nor a
reserved boot structure, but each microVM virtio device received the
whole guest memory. A descriptor failed only when its buffer left guest
memory, as in the aperture, and then only for the part that did, so a
guest could point a device buffer at the shared virtio interrupt-status
page at 0x30000, which the devices themselves update, or at the ISA
hole. The guest can write those pages directly, so this did not weaken
isolation, but the devices enforced less than the layout states.

The first commit adds guest memory views, which admit only the accesses
that lie wholly inside a set of page-aligned ranges and refuse any
other before touching memory, on every access path, page locks
included, while sharing the mappings of the memory that they restrict,
so that admitted accesses keep their zero-copy paths.

The second makes a virtio queue check each descriptor's whole buffer,
and each indirect table, against the device's memory before handing the
chain to the device, and refuse the chain as malformed when the memory
does not admit it, so that a device reads and writes none of a buffer
that lies even partly outside the admitted ranges. Rings already lie in
subranges that the memory refuses as a whole when a queue starts.

The third lets an MMIO transport restrict its device to a set of ranges
and report each refusal by setting DEVICE_NEEDS_RESET, with a
configuration-change notification once the driver has set DRIVER_OK,
as virtio 1.2 provides for a device in an error state. A refused ring
fails DRIVER_OK and leaves DEVICE_NEEDS_RESET alone set. A reset clears
the mark. The transport keeps the whole memory for the shared
interrupt-status word.

The fourth returns the RAM ranges of the microVM e820 map from the
loader that builds it, and the fifth restricts every microVM virtio
device to them: the layout's RAM, which after a restore includes the
RAM that the restore activates, less the interrupt-status page and the
ISA hole.

The protocol vectors and built-in CPU profiles that nvx-rs copies are
unchanged.

Refs #554

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the five commits that the previous commit pins as positions
355-359 of the microVM chain, and move the NVX pin to bd2b27a2e.

Positions 355-359 restrict the DMA of the microVM virtio devices to the
RAM ranges of the e820 map: guest memory views, the virtio queue's
whole-buffer check, the MMIO transport's restriction and needs-reset
report, the loader's e820 RAM ranges, and the microVM worker that
applies them. Position 355 changes only guestmem, which no other commit
of the chain changes, so it can go upstream on its own, as positions
350-352 can, and position 356 adds only the queue's check on top of it.
Positions 357-359 build on the fork's shared interrupt-status transport,
MP-table loader, and microVM worker; say so in the table's remarks. The
pin is now 202 commits past the mapped tip.

Positions 353 and 354 are no longer only a proposal:
microsoft/openvmm#4775 merged them, so position 354, 832d5bb48, is the
head of feature/microvm/dev. Say so in the scope, and name
microsoft/openvmm#4794 as the proposal of positions 355-359.

Refs #554

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Section 2.3 requires that loader writes and DMA ranges lie wholly inside
a RAM range of the e820 map and cross neither the MMIO aperture nor a
reserved boot structure, but said nothing about who enforces it. The
Linux direct loader always kept its writes inside RAM; with the pinned
OpenVMM, the virtio devices now enforce the rule for DMA too.

Note the enforcement after the requirement: each device reaches only
the RAM ranges of the e820 map and the RAM that a restore activates
beyond them, and refuses a queue whose rings, or a descriptor whose
buffer, lies even partly outside them, as in the interrupt-status page,
the ISA hole, or across the aperture. It reads and writes none of it,
stops serving the queue, and sets DEVICE_NEEDS_RESET, notifying the
driver through a configuration change once DRIVER_OK is set. A refused
ring fails DRIVER_OK instead, leaving DEVICE_NEEDS_RESET alone set. A
device reset clears the mark, and only the transports update the
interrupt-status page.

Fixes #554

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/issue-554-refuse-dma-outside-ram branch from 642e1a7 to 8457a99 Compare October 29, 2026 00:19
Copilot AI balanced review requested due to automatic review settings October 29, 2026 00:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The pin changes unsafe guest-memory access and virtio DMA/reset behavior across KVM, MSHV, and WHP, requiring human review and cross-backend validation while exact-head CI remains pending.

No open issues 🎉

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

microVM: refuse device buffers outside the e820 RAM ranges

2 participants