[Pedro Henrique Penna (ppenna)]

Fixes #540. Pins microsoft/openvmm#4775 at 832d5bb48dadd2cc5221f2c8b8905e6b80d13d53, which lets the restore of a microVM platform snapshot attach an aggregate of host directories and files, with its access policy and owner, to the filesystem slot that the capture left dormant, validated as at a cold boot, before any vCPU runs. It adds a microVM scenario that restores one platform snapshot with such an aggregate on KVM, MSHV, and WHP.

Based on dev at ba30ff904. microsoft/openvmm#4775, two commits directly on b92daa350, the revision that dev pins, has landed: its head, 832d5bb48, is the head of feature/microvm/dev. This branch's four commits are its whole diff.

Change

Commit	Files	Change
205233511 openvmm	openvmm	b92daa350 → 832d5bb48: the two commits of microsoft/openvmm#4775
464b89155 doc	doc/openvmm-upstream-roadmap.md	Records them as positions 353 and 354, unassigned like positions 346-352 because position 354 changes the dormant-slot restore of PR18's position 113, which position 293 limited to one --mount; moves the NVX pin; and records that microsoft/openvmm#4773 merged positions 350-352 into feature/microvm/dev
3bdcee72d doc	doc/run.md	A Platform filesystem exports section for direct OpenVMM restores; the virtio-fs mapping section, whose nvx-rs run rule still holds, links it
c241bbbba scripts	scripts/nvx_tools/microvm_tests.py, scripts/nvx_tools/microvm_test_scripts/platform-aggregate-export.sh, scripts/test_microvm_tests.py, scripts/nvx_tools/microvm_shards.json, doc/ci.md, doc/usage.md	New default-suite scenario platform-aggregate-export; filesystem-snapshot also refuses an aggregate on its untiered dormant slot; shards replanned

OpenVMM changes

• The device takes a dormant slot's state into an aggregate (057a2eaa6). A capture that leaves the slot dormant saves a dormant device state, without an attachment, a policy, objects, or an initialized FUSE session, which the device restores into any attached device. A new unit test checks it for an aggregate of a read-write directory and a read-only file, and that the device then saves the aggregate rather than the dormant slot. Only a test changes.
• A platform restore attaches an aggregate to the dormant slot (832d5bb48). At the restore of a platform snapshot whose slot is dormant, --mount-aggregate with its --mount-child options, and their --mount-deny, --mount-allow, --mount-write, and --mount-owner policy, attach an aggregate. The restore builds it with a cold boot's code, so it validates the children's kinds, modes, names, overlaps, and limits, the policy paths, and the owner as a cold boot does, and refuses what a cold boot refuses, before any vCPU runs. The slot's device keeps the stable ID, tag, MMIO range, interrupt, features, and queues that the snapshot recorded, and the guest keeps its captured command line, which names no filesystem: OpenVMM exposes no layout, and the guest learns of the export over its own control channel (nvx-rs: aggregate exports for clones #857). The restore contract still records the dormant slot. Only the platform tier changes: the dormant slot of a workload-start, instance-checkpoint, or untiered snapshot still binds only a --mount, a snapshot that predates the dormant slot binds nothing, and a snapshot that captured a filesystem still requires exactly that filesystem. --mount, and a restore without an attachment, behave as before.

Tests

• platform-aggregate-export captures a platform snapshot with sandbox blocks and without a filesystem, and restores it with the aggregate of the filesystem-shares scenario: a read-write workspace and a read-only tool cache, each with a denied path, a read-write data directory whose root exposes only its allowed paths and whose notes alone are writable, and a read-only and a read-write file, each without its directory. The clone keeps its captured command line, so its guest checks that no virtfs_ or nvx_share= token appeared, mounts the aggregate itself, finds the five children in the read-only root that only root can enter, checks each child's kind, mode, and policy, and binds a directory and a file child; the host then requires exactly the writes that the policy admits. Before the guest runs, OpenVMM must refuse the restores whose aggregates a cold boot refuses: overlapping directories, a file inside a directory child, a directory as a file and a file as a directory, a repeated name, a policy path of a file child, one outside every child, and a relative one, and on a Windows host caller ownership. A refused restore must print none of the guest's lines, and no restore may change the snapshot or the shared trees. It runs with the Alpine sandbox-control guest only, like platform-filesystem, and joins shard 2 on KVM and MSHV and shard 1 on WHP, each planned with the scenario's time in the bare-metal default suite below.
• filesystem-snapshot also requires OpenVMM to refuse an aggregate for the dormant slot of its untiered snapshot, which still binds a --mount.
• Unit tests cover the guest's checks, the capture's and the restore's arguments, each refusal's aggregate against the host's trees, the Windows-only refusal, a missing report, a guest failure, writes beyond the policy, and the snapshot's and the trees' integrity. [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775 adds unit tests for the device's restore of a dormant slot's state into an aggregate; for a dormant-slot restore that resolves an aggregate of directory and file children, with mixed modes and denied, allowed, and writable paths, to a cold boot's configuration, roots, and identities; for its refusal of what a cold boot refuses, with a cold boot's errors; for --mount-owner; for snapshots without the dormant slot and of a single directory; and for the tier rule at every tier.

Of #540's acceptance cases, the scenario covers a restore of a dormant slot with an aggregate of directory and file children, with mixed modes and policies, and the refusal of invalid aggregates, on every backend. Snapshots with a captured attachment keep their exact-match rules: filesystem-shares and filesystem-snapshot, which restore captured aggregates and directories, pass unchanged in the default suite, as do OpenVMM's exact-match unit tests. A current OpenVMM cannot capture a snapshot without the dormant slot's version, so OpenVMM's unit tests cover that refusal, and the tier rule's unit test covers workload and instance snapshots, as filesystem-snapshot does for an untiered one.

Evidence

At 114dbb7be, this branch before its latest rebase, on the bare-metal hosts prometheus32 (Linux/KVM), prometheus30 (Linux/MSHV), and prometheus28 (Windows/WHP), each in a separate worktree with OpenVMM built there from 832d5bb48 and the guest of development release v0.1.0-dev.3beaf1646086 on KVM and MSHV or v0.1.0-dev.be07e2f8ada4 on WHP, both of which pin b92daa350:

Check	KVM	MSHV	WHP
platform-aggregate-export with the fix	Passes (3.4 s)	Passes (4.3 s)	Passes (8.8 s)
filesystem-snapshot with the fix	Passes (4.5 s)	Passes (4.8 s)	Passes (8.2 s)

With the unpatched OpenVMM b92daa350 of v0.1.0-dev.3beaf1646086 on KVM, the capture succeeds and the clone's restore fails at a dormant-slot snapshot can attach only a --mount on restore. CI run 40767318100 passed at 114dbb7be, where platform-aggregate-export ran in shard 2 on KVM and MSHV and in shard 1 on WHP.

Before the rebases, the same hosts ran these checks at 02838f0ae, each with OpenVMM built there from d7892ffa8 and the guest of development release v0.1.0-dev.1ee6489241a8:

Check	KVM	MSHV	WHP
platform-aggregate-export with the fix	Passes (3.4 s)	Passes (4.1 s)	Passes (9.1 s)
filesystem-snapshot with the fix	Passes (4.8 s)	Passes (5.0 s)	Passes (8.8 s)
Default microVM suite with the fix	41/41 (47 executions), 509.1 s	41/41 (47 executions), 488.7 s	41/41 (47 executions), 655.6 s
platform-aggregate-export in the default suite	3.4 s	4.5 s	9.1 s

With that release's unpatched OpenVMM d2338ea22 on MSHV, the capture succeeds and the clone's restore fails at a dormant-slot snapshot can attach only a --mount on restore.

In both runs, every clone reported NVX-POST-RESTORE-OK: tier=platform and NVX-PLATFORM-AGGREGATE-OK, and every refusal failed before the guest ran, with: microVM filesystem host paths must not overlap (overlapping directories, and a file inside a directory child); invalid --mount-child 0, caused by the root's not being a regular file or a plain directory; invalid --mount-aggregate or --mount-child options, caused by a child name that is not unique; --mount-child 4 exposes the file ...; microVM denied path resolves outside the filesystem export roots; with --mount-aggregate, --mount-deny requires an absolute host path: secrets; on WHP, --mount-owner caller requires a Linux host; and for filesystem-snapshot's untiered snapshot, --mount-aggregate on a dormant filesystem slot requires a platform snapshot.

On prometheus28, nvx-rs download fails with curl: (43) A libcurl function was given a bad argument under the host's curl 8.8.0, so in both runs its guest artifacts came from nvx-rs download on another Windows machine, with curl 8.21.0, and their SHA-256 digests matched after the copy.

Equivalence and validation

• Hosts. The hosts ran the default suite at 02838f0ae, on dev at d77908efc, and platform-aggregate-export and filesystem-snapshot again at 4cf5d4afe, on dev at dce536f1d with OpenVMM d7892ffa8, where the scenario passes in 3.4 s on KVM, 4.0 s on MSHV, and 8.8 s on WHP, and those two scenarios again at 114dbb7be, as Evidence shows. 4cf5d4afe differs from 02838f0ae in the scenario only by the review fix: the integrity check after the refused restores also compares the workspace. It also differs in the shard partition, which the hosts' unsharded runs don't read and which is planned with their in-suite times, in the scenario commit's message, and in its base: dev gained nvx-rs: capture templates into their store, and publish them #887 and tests: capture templates in VMs through the runtime adapter #933, which change none of this branch's files.
• Rebase onto b92daa350. 114dbb7be is 4cf5d4afe rebased onto dev at 1853252ca, and [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775's head 832d5bb48 is d7892ffa8 rebased onto b92daa350. [DO NOT REVIEW] disk_blob: bound HTTP blob requests and retry transient failures openvmm#4773 moved feature/microvm/dev, and dev's pin, to b92daa350 with three commits that change only disk_blob, so the gitlink and roadmap commits are rebuilt on that pin, and the roadmap's positions 350 and 351 become 353 and 354. git range-diff shows [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775's two patches, and this branch's run.md and scenario patches, unchanged; dev's other new commits change nvx-rs, the runtime adapter, the performance scripts, and the network text of doc/run.md and doc/usage.md, none of this branch's lines. The scenario commit's message now names the release whose unpatched OpenVMM the negative control ran, v0.1.0-dev.1ee6489241a8, rather than calling it the latest.
• Rebase onto ba30ff904. This head is 114dbb7be rebased onto dev at ba30ff904, which still pins b92daa350. guest: write back the host shares before every capture #976 appends a paragraph on filesystem-snapshot's write-back of shared mappings to doc/ci.md where this branch appends the scenario's, so the scenario's paragraph now precedes it, and changes run_filesystem_snapshot's live capture, which Git merges with this branch's refusal for the dormant slot; run: pin the refusal of share targets that would break a kernel token #970 and nvx-rs: reject unbalanced sandbox disk path brackets #971 change other lines of doc/run.md and doc/usage.md. The other patches are unchanged.
• OpenVMM, on [DONT REVIEW] microVM: let a platform restore attach an aggregate to a dormant filesystem slot openvmm#4775's head, on Windows. cargo clippy --all-targets and cargo doc --no-deps for virtiofs and openvmm_entry, cargo nextest run --profile agent for both (339 passed), and cargo xtask fmt --fix (clean). The protocol vectors and CPU profiles that nvx-rs copies are unchanged: openvmm_vectors and openvmm_cpu_profiles pass with --ignored.
• NVX, on Windows. At this head: nvx.py verify, the Markdown link check, ruff check, ruff format --check, compileall, pyright for Linux and Windows, test_microvm_tests.py, test_markdown_links.py, and test_module_diagrams.py (418 tests, 12 skipped), and nvx.py test-microvm --help, which lists the scenario. At 114dbb7be, the validate-nvx unittest list without test_setup_azure.py (1,320 tests, 54 skipped). The Git sh.exe runs of test_host_loopback_allow_script_checks_publication_and_traffic exceed their 10 s limit intermittently on this machine, on this branch and on dev alike, and the case passes on a re-run. At 321888b05, before the review fix and the rebases: the 18 shell cases of test_setup_azure.py, which need a Linux bash, fail the same way on dev here, and test_adversarial.py, test_hosts.py, and the other CLI --help checks pass.
• Not run locally. Those shell cases; shellcheck and shfmt, which run in Docker, although no checked shell script changed; check-nvx-rs, since nvx-rs doesn't change; and the OpenVMM VMM tests, which cannot reach an aggregate through the management RPC. CI runs all of them.

Notes

• doc/design isn't edited: draft doc: describe the managed layered root and platform-clone activation #591 rewrites those files, and nvx-rs: aggregate exports for clones #857, which this unblocks, makes nvx-rs attach a clone's aggregate at its restore.
• The management RPC still describes a single directory, so it attaches no aggregate; the aggregate is a CLI restore option, as --restore-network-policy is.

Exact NVX head submitted to CI: c241bbbba97a74a8a8778ed11b92df57d7f9758b.

[Copilot]

🟡 Changes recommended

The refused-restore integrity check omits the workspace, allowing token overwrites to go undetected; workspace comparison and a regression case for mutation during refusal are needed.

1 open issue

• Verify workspace integrity after refused restores · New

What changed in this PR

This PR integrates OpenVMM support for attaching aggregate filesystem exports to dormant slots during platform snapshot restore, addressing #540 and enabling clone exports planned in #857.

Changes:

• Advances the OpenVMM pin and documents platform-only aggregate attachment rules.
• Adds a default-suite scenario covering mixed directory/file exports, access policies, invalid attachments, and snapshot integrity.
• Adds unit coverage and rebalances the KVM, MSHV, and WHP test shards.

File	Description
scripts/​test_microvm_tests.py	Tests scenario arguments, refusals, guest reports, and integrity checks.
scripts/​nvx_tools/​microvm_tests.py	Adds the aggregate restore scenario and untiered-snapshot refusal check.
scripts/​nvx_tools/​microvm_test_scripts/​platform-aggregate-export.sh	Checks guest mounts, child types, access policies, and bind mounts.
scripts/​nvx_tools/​microvm_shards.json	Includes the scenario and rebalances platform shards.
doc/​usage.md	Lists the scenario’s sandbox-control guest requirement.
doc/​run.md	Documents direct OpenVMM platform aggregate restores.
doc/​openvmm-upstream-roadmap.md	Records the new pin and its two OpenVMM commits.
doc/​ci.md	Describes aggregate restore coverage and rejection cases.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The OpenVMM pin changes snapshot-restore admission and pre-vCPU filesystem-policy enforcement across KVM, MSHV, and WHP, requiring human confirmation of the restore boundary and exact-head platform CI evidence.

No open issues 🎉

1 resolved since last review

• Verify workspace integrity after refused restores

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Runtime evidence targets d7892ffa8 rather than the reviewed pin 832d5bb48, leaving source equivalence and current-pin KVM/MSHV/WHP restore validation unconfirmed while relevant CI is still running.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

There are no active review issues requiring resolution.

No open issues 🎉

_{Review effort: Balanced}