nvx-rs: aggregate exports for clones by esaurez · Pull Request #992 · microsoft/nvx

Enrique Saurez (esaurez) · 2026-10-29T02:22:15Z

Chain: #951 → #954 → #957 → #959 → #960 → #955 → #958 → #962 → #963 → #925 → #926 → #978 → this PR, on dev; #879, #881, #885, #886, #877, #932, #884, #887, #933, #880, #936, #937, #939, #940, #941, #888, #949, and #950 have merged.

Stacked on #978.

Fixes #857.

What and why

Until now, a clone exported one directory alone, under stage 1's interim rule (#846):

ClonePlan::new refused an export plan of more than one directory or file, or of a file, since OpenVMM bound only one directory, with --mount, to a template's dormant filesystem slot.
The restore bound that directory with --mount, and the clone's mapping table named paths in it without the child's index, with an empty source for the directory itself.

A cold launch, meanwhile, exports an aggregate. filesystem::plan makes one child per outermost mapped directory and one per mapped file outside them, up to 256, each read-only or read-write, with OpenVMM's hidden, re-exposed, and writable paths and an owner mode. The launch attaches it with --mount-aggregate and --mount-child, and its table's sources name each path by its child's index.

OpenVMM now lets a platform restore attach an aggregate to a dormant slot (#540). With this PR, a clone exports what a cold launch exports, attached and mapped as a cold launch attaches and maps it.

Host: `nvx::host::template`

Clone plan: ClonePlan::new takes any export plan that filesystem::plan could make for the container root (CloneExport, formerly DirectoryExport). Before anything is opened, it still refuses:
- a plan that ExportPlan::check refuses: mappings without an export, an export without mappings, more than 256 directories and files, or a target that the container root keeps;
- an export one of whose directories or files lies inside the template store, or one of whose directories holds it, which a guest could reach a leased payload through;
- an export that lets the guest write in a directory through which the store's path leads, or one of whose directories is or holds the clone's sandbox directory, where the host writes the clone's records;
- an export one of whose files lies in the sandbox directory itself, where it may be one of the host's records, such as the clone's binding: new with aggregates, since a clone's export could hold no file before. A file in a directory of the caller's inside the sandbox directory stays allowed.
Each check covers every child.
Restore: the restore attaches the export to the dormant slot with a cold launch's arguments, ExportPlan::openvmm_arguments: --mount-aggregate, a --mount-child of each directory and file with its mode, and file for a file, then the hidden, re-exposed, and writable paths, and --mount-owner caller for the caller owner. Without an export, the slot stays dormant.
Mapping table: the restore sequence sends a cold launch's table, ExportPlan::guest_table, whose sources name each path by its child's index. No source is empty.
Activation record: the record names the export as an aggregate, with its plan's identity and its number of mappings. The record's encoding already had the aggregate shape, so it does not change, and the guest mounts and maps the export as before.
Launch: before any VMM starts, the launch verifies the export with ExportPlan::verify, as before, which now covers every child: each child's kind, the read-only mappings outside the parts that the guest may write, and the pinned objects that the guest could move. A file that a directory replaced, which would grant the whole directory, fails the clone with request, invalid-mappings, launch.

Protocol and guest: no change

The protocol and the guest still take an empty source, which names the root of a directory export, and MAPS|ROOT_SOURCE stays a version 1 vector. Only the rationale changes: NVX's hosts, whose exports are aggregates, send no empty source, and a host whose export is one directory sends one only to a guest that deploys with it. These texts change:

doc/design/4.4-host-guest-control-interface.md: 4.4.1, a mapping request's sources, and 4.4.4, version 1's exception for MAPS|ROOT_SOURCE;
nvx-rs/tests/vectors/README.md: version 1's exceptions;
nvx-rs/README.md: the nvx::host::template and nvx::guest::mappings sections.

Runtime adapter

Allowed paths: launch, start, and run take --allow PATH beside --deny, which HostFsSharing::plan passes to the planner as the request's allowed paths, so that a hidden directory can show paths in it again.
Step hook at a start: start takes RUNTIME_ADAPTER_STEP_HOOK at checked alone: after NVX's ClonePlan checked the clone against the leased generation, before the start records the clone, and before the clone's launch checks its export again. A hook at another step is a usage error of a start, and launch's four steps are unchanged.
Clones: the clone order sends a cold launch's table. The module documentation and the README's Clones section describe the aggregate.

Both seams are test-only and additive.

Tests

nvx-rs unit tests, template::clone:

a_clone_with_an_export_records_its_plan_and_maps_from_its_children (was ..._from_its_directory): an aggregate of two directories and two files, read-only and read-write, with a hidden path and a path shown again. The record names an aggregate, and the table's sources are 0, 1, 2, 3, and 0/work.
exports_that_no_cold_launch_attaches_are_refused_before_anything_is_opened (was exports_that_a_dormant_filesystem_slot_does_not_bind_...): the refusals of two directories and of a file go, an export of 257 children is refused, and an aggregate serves a clone.
exports_that_overlap_the_template_store_are_refused_before_anything_is_opened: a file of the store, beside a directory outside it, is refused.
exports_that_let_the_guest_turn_the_template_store_s_path_are_refused: a writable directory through which the store's path leads is refused as an aggregate's second child, while a writable file beside the path's link is allowed.
files_in_the_sandbox_s_directory_are_refused (new): the clone's binding, read-only or writable, beside a directory outside the sandbox, is refused, while a file in a directory of the caller's inside the sandbox is allowed.
the_export_of_a_clone_is_attached_and_mapped_as_a_cold_launch_attaches_and_maps_it (was the_export_of_a_clone_binds_its_directory_with_its_policy_and_owner): the arguments and the table, child by child, are a cold launch's.
launch:
- a_clone_with_an_export_binds_it_to_its_template_s_dormant_filesystem_slot checks the aggregate's arguments and table;
- a_clone_whose_exported_objects_changed_since_its_plan_starts_nothing adds a file of an aggregate that a directory replaced.
restore: a_clone_with_an_export_gets_its_mapping_table_once_ready_and_its_start_once_it_applied_it checks that the table names each path by its child's index. The root mapping with an empty source goes.

Runtime adapter, without VMs:

host_session: a_clone_s_order_maps_its_export_s_children_by_their_indices replaces a_clone_s_order_maps_the_root_of_its_export_with_an_empty_source.
layered_launch: --allow in the plan, its refusal outside every --deny, and --allow needs --share.
clone_start: --allow needs --share for start and run.
step_hook: a_start_takes_a_step_hook_at_checked_alone.

Runtime adapter VM scenarios (scenarios.txt, still 154):

vm_clone a_clone_maps_an_aggregate_of_directories_and_files_beneath_its_root_before_its_sandbox_starts replaces a_clone_maps_its_export_beneath_its_root_before_its_sandbox_starts. One template restores a clone with an aggregate of:
- a read-only directory, with a read-write directory in it;
- a read-only file and a read-write file;
- a read-write directory;
- a hidden file, and a hidden directory that shows a file again.
It checks what the guest and the workload read, write, and list through each mapping, what the host holds before and after the stop, and, on Linux, that the VMM attaches the export with a cold launch's --mount-aggregate, --mount-child, --mount-deny, --mount-allow, and --mount-write arguments.
vm_clone a_start_refuses_an_export_whose_mapped_objects_changed_before_any_vmm_starts replaces a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts. With the start paused at checked, a directory replaces a read-only file; then another directory replaces a pinned one inside a read-write directory. Each start fails with request, invalid-mappings, launch, planned, none, no-vm: no VMM starts, nothing is recorded, and the lease is released.
vm_clone clones_at_once_see_only_their_own_exports: clones with aggregates run beside a clone without an export, whose VMM gets none of OpenVMM's export options, so its slot stays dormant.
vm_clone a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver: the aggregate beside the NIC's resolver.

Guide changes (#591's numbering)

At 950c1fe, the guide already describes a new directory export or aggregate on a dormant filesystem slot at restore. These refinements are not made in #591; they are listed here for it:

2.1.4 Restore-Time Configuration: a platform clone's export is the aggregate that a cold launch of the same shares attaches, never a directory export: up to 256 directories and files, each with its own mode and path policy, and the export's ownership mode. Before any VMM starts, the host verifies it again, and a child of another kind, or a pinned object that moved, refuses the clone.
3.6.2 Export Forms: a dormant slot gains an aggregate at restore. Its guest learns the form from the activation record rather than from boot tokens, then binds each child at its target, as on a cold boot.
3.6.9 Restore: the new export of a dormant slot passes the checks of a cold boot's export, an aggregate's included, before any processor runs.
3.6.10 Dormant Slot and Saved-State Versions: NVX's hosts bind an aggregate to the dormant slot, whose mapping table names each path by its child's index, as on a cold boot.
The guide at 950c1fe holds no interim rule (one directory per clone), so there is nothing to remove from it.

Validation

At c15b66a, after each of the 3 commits passed its own targeted checks on Windows and Linux:

Windows: nvx-rs's cargo fmt, Clippy with -D warnings (all features, --lib, and --lib --no-default-features), tests, rustdoc with -D warnings, and MSRV build; the runtime adapter's fmt, Clippy, and tests in its protocol-only, host, and guest configurations; the scripts' tests and validate-nvx's Windows list; test-runtime-adapter --check-only; and the Markdown links.
Linux: nvx-rs's tests (2045 passed); the root-only guest::mappings, guest::clone::activation::apply, and guest_tools tests; the guest binaries, macOS checks, and MSRV builds; the adapter's musl guest; and validate-nvx's Linux list.
CI: ci.yml run 40873697513 on this branch, with every job green, KVM, MSHV, and WHP included, at its first attempt.
nvx.py test-runtime-adapter, with OpenVMM 832d5bb4 from release v0.1.0-dev.2d14547b669f and a guest initramfs and guest agent built from c15b66a: the 154 scenarios of 11 targets, vm_clone's 34 included, passed on WHP (with RUST_TEST_THREADS=1) and on MSHV.

Copilot

🔵 Needs a closer look

The filesystem-isolation changes span hypervisor backends, but KVM coverage and full runtime-adapter VM suites remain pending and cannot be verified in this source-only review.

No open issues 🎉

What changed in this PR

This PR removes the single-directory restriction on clone exports for #857, using the OpenVMM restore support introduced by #948. Clones now attach and map aggregates like cold launches, without changing the guest protocol or activation-record encoding.

Changes:

Reuse aggregate export arguments and child-indexed mapping tables, and identify exports as aggregates in activation records.
Add runtime-adapter --allow paths and a test-only start hook at checked.
Update documentation and tests for mixed directory/file exports, path policies, changed-object rejection, and clones without exports.

File	Description
`tests/runtime-adapter/tests/vm_clone.rs`	Exercises aggregate exports, path policies, isolation, and pre-launch rejection.
`tests/runtime-adapter/tests/step_hook.rs`	Tests that starts accept hooks only at `checked`.
`tests/runtime-adapter/tests/layered_launch.rs`	Tests allowed paths and invalid `--allow` requests.
`tests/runtime-adapter/tests/host_session.rs`	Checks child-indexed clone mapping messages.
`tests/runtime-adapter/tests/clone_start.rs`	Adds `--allow` validation cases for starts and runs.
`tests/runtime-adapter/src/layered.rs`	Passes allowed paths to the filesystem planner.
`tests/runtime-adapter/src/hook.rs`	Documents the clone-start hook boundary.
`tests/runtime-adapter/src/clone.rs`	Updates clone export documentation for aggregates.
`tests/runtime-adapter/src/bin/runtime-adapter-host.rs`	Adds `--allow` parsing and the checked-stage start hook.
`tests/runtime-adapter/scenarios.txt`	Updates registered scenario names.
`tests/runtime-adapter/README.md`	Documents aggregate clones, allowed paths, hooks, and tests.
`nvx-rs/tests/vectors/README.md`	Clarifies compatibility rules for empty mapping sources.
`nvx-rs/src/host/template/mod.rs`	Updates the template API overview for aggregate exports.
`nvx-rs/src/host/template/clone/tests/restore.rs`	Checks restore mappings against the aggregate table.
`nvx-rs/src/host/template/clone/tests/launch.rs`	Tests aggregate arguments and file-replacement rejection.
`nvx-rs/src/host/template/clone/tests.rs`	Covers aggregate planning, limits, store overlap, and cold-launch parity.
`nvx-rs/src/host/template/clone/record.rs`	Marks activation-record exports as aggregates.
`nvx-rs/src/host/template/clone/launch.rs`	Documents aggregate restore arguments and verification failures.
`nvx-rs/src/host/template/clone/failure.rs`	Removes obsolete single-directory failure examples.
`nvx-rs/src/host/template/clone/export.rs`	Reuses aggregate argument and mapping-table generation.
`nvx-rs/src/host/template/clone.rs`	Integrates `CloneExport` and updates clone API contracts.
`nvx-rs/README.md`	Updates template and guest-mapping API descriptions.
`doc/design/4.4-host-guest-control-interface.md`	Clarifies aggregate sources and empty-source compatibility.

_{Review effort: Balanced}

The aggregate exports of clones (#857) hide paths and show paths inside them again. The shares of a launch, a start, and a run take --allow beside --deny, which Shares::plan passes to NVX's planner as the request's allowed paths, so that the adapter's VM scenarios can export them. A start takes RUNTIME_ADAPTER_STEP_HOOK at checked alone: once NVX's ClonePlan checked the clone against the leased generation, and holds the files that its checks opened, before the start records the clone, and before the clone's launch checks its export again, so that a VM scenario can change a shared object there. A hook at another step is a usage error of a start. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: afd23653-4313-4e60-ab6b-350dffc1890b

A template is a platform snapshot, and OpenVMM lets a platform restore attach an aggregate to a dormant filesystem slot, which it checks as a cold boot's (#540, OpenVMM 832d5bb4). So a clone takes any export plan that filesystem::plan makes for the container root, as a cold launch does: up to 256 host directories and files, each read-only or read-write, with OpenVMM's hidden, re-exposed, and writable paths, and the plan's owner, none of which may lie inside the template store or, for a directory, hold it. The restore renders the export as a cold launch renders it, with --mount-aggregate and a --mount-child of each directory and file, and the clone's mapping table is a cold launch's, whose sources name each path by its child's index. The checks that keep a clone's guest from the template store and from the host's records in the clone's sandbox apply to each directory and file of the aggregate. A file holds no directory, and lets the guest write its contents alone, but a file in the sandbox's directory itself may be one of the host's records there, so the plan refuses one, read-only or writable, as it refuses a directory that is or holds the sandbox's. This replaces stage 1's interim rule (#846), which took an export of one directory alone, bound it with --mount, and sent sources without the child's index. The activation record names the export as an aggregate, which the guest mounts and maps as it did. The protocol and the guest still take an empty source, which names the root of a directory export; the design document, the vectors' README, and the README now say that NVX's hosts, whose exports are aggregates, send none. The adapter's clone order sends a cold launch's table, and its documentation describes the aggregate. Its VM scenarios follow in the next commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: afd23653-4313-4e60-ab6b-350dffc1890b

A clone's VM scenario now restores a template with an aggregate of host directories and files (#857): a read-only directory with a read-write one inside it, a read-only file, a read-write file, and a read-write directory, with a hidden file, and a hidden directory that shows a file again, which the start shares with --allow. On Linux, the scenarios check that the clone's VMM attaches its export as a cold launch attaches one, with --mount-aggregate and a --mount-child of each directory and file, and that a clone without an export passes none of OpenVMM's options of an export, which leaves the template's filesystem slot dormant. A new scenario changes a mapped object once NVX's plan checked the clone, while the step hook pauses the start at checked: a directory replaces a read-only file, or another directory a pinned one inside a read-write directory. The clone's launch checks the export again, and refuses it, before any VMM starts. The scenario of stage 1's interim rule, which refused an export of two directories, or of a file alone, goes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: afd23653-4313-4e60-ab6b-350dffc1890b

Copilot

🔵 Needs a closer look

Pending validation of restore-time aggregate attachment, export isolation, and dormant-slot behavior requires maintainer confirmation of exact-head VM scenarios on KVM, MSHV, and WHP.

No open issues 🎉

_{Review effort: Balanced}

Enrique Saurez (esaurez) self-assigned this Oct 29, 2026

Enrique Saurez (esaurez) requested a balanced review from Copilot October 29, 2026 02:22

Copilot started reviewing on behalf of Enrique Saurez (esaurez) October 29, 2026 02:22

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

Enrique Saurez and others added 3 commits October 29, 2026 04:42

Enrique Saurez (esaurez) force-pushed the esaurez/857-aggregate-exports-20261028 branch from 5a62b14 to c15b66a Compare October 29, 2026 12:24

Enrique Saurez (esaurez) changed the base branch from esaurez/857-base-snapshot-20261028 to esaurez/850-network-policies-20261028 October 29, 2026 12:24

Enrique Saurez (esaurez) requested a balanced review from Copilot October 29, 2026 12:25

Copilot started reviewing on behalf of Enrique Saurez (esaurez) October 29, 2026 12:26

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

Enrique Saurez (esaurez) removed their assignment Oct 29, 2026

Navigation Menu

Repository navigation

nvx-rs: aggregate exports for clones - #992

nvx-rs: aggregate exports for clones#992
Enrique Saurez (esaurez) wants to merge 3 commits into
esaurez/850-network-policies-20261028from
esaurez/857-aggregate-exports-20261028

Enrique Saurez (esaurez) commented Oct 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Enrique Saurez (esaurez) commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why

Host: nvx::host::template

Protocol and guest: no change

Runtime adapter

Tests

Guide changes (#591's numbering)

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Enrique Saurez (esaurez) commented Oct 29, 2026 •

edited

Loading

Host: `nvx::host::template`