diff --git a/doc/run.md b/doc/run.md index 13a3b6959..2a5c31ee7 100644 --- a/doc/run.md +++ b/doc/run.md @@ -56,8 +56,10 @@ nvx-rs/target/release/nvx-rs run \ ``` Networking requires the explicit `portable` capability profile. It uses the -same in-process data plane on KVM, MSHV, and WHP; omitting either `--net` or -`--network-profile portable` is rejected before OpenVMM starts. +same in-process data plane on KVM, MSHV, and WHP; a cold boot that omits +either `--net` or `--network-profile portable` is rejected before OpenVMM +starts. A restore takes its addresses from the snapshot instead; see +[Networked snapshots](#networked-snapshots). Select an explicit processor count after building the matching specialized guest kernel: @@ -369,11 +371,12 @@ Most `nvx-rs run` options pass through unchanged: `--machine`, `--processors`, | `--dry-run` | No equivalent; this only prints the generated command | Always include `--single-process`. When restoring, omit `--memory`, `--kernel`, -and `--initrd`. Every restore gives the guest fresh entropy through the time -ABI's restore packet, so `--restore-entropy` is no longer needed; OpenVMM still -accepts it without effect. Do not pass `--guest ubuntu` during restore; the -captured RAM and machine contract already identify the restored guest. For -example: +`--initrd`, and `--net`, whose addresses the snapshot keeps; see +[Networked snapshots](#networked-snapshots). Every restore gives the guest +fresh entropy through the time ABI's restore packet, so `--restore-entropy` is +no longer needed; OpenVMM still accepts it without effect. Do not pass +`--guest ubuntu` during restore; the captured RAM and machine contract already +identify the restored guest. For example: ```bash ./bin/openvmm \ @@ -507,6 +510,34 @@ explicit `--host-loopback allow`, because its listeners are live host attachments, and so does a restore, unless it installs a new policy on a [NIC template](#platform-nic-templates). +### Networked snapshots + +A snapshot captured with `--net` keeps its NIC's addresses and the digest of +its network policy. Restore it with `--network-profile portable`, without +`--net`, and with the network policy options of the capture, which OpenVMM +checks against that digest before the guest runs. `nvx-rs run` passes them to +OpenVMM as given, and refuses `--net` on a restore: + +```bash +nvx-rs/target/release/nvx-rs run \ + --restore-snapshot /var/lib/nvx/snapshot \ + --network-profile portable \ + --network-egress deny \ + --network-egress-allow 10.0.0.1/32:tcp:443 +``` + +The restore fails unless its policy matches the captured one: the default +egress, the same rules, whether `--network-egress-allow` and +`--network-egress-deny` give them or a policy file compiles to them, +`--host-loopback`, and `--network-proxy`. A restore therefore cannot change +the policy, and, like a capture, it rejects `--network-ingress allow`, explicit +`--host-loopback allow`, and `--host-loopback-forward`, whose published ports +are live host attachments. The restored guest keeps the network configuration +that its boot applied, such as its DNS server; flows from before the capture +do not survive the restore, and new traffic opens fresh ones. A snapshot +captured without `--net` takes no network option at restore. Only a +[platform NIC template](#platform-nic-templates) takes a new policy at restore. + ### Platform NIC templates A platform snapshot whose NIC was captured with a complete deny-all policy, @@ -539,8 +570,8 @@ policy names no DNS server and starts no `nvx-ingress`, so it learns its DNS server and whether to report its listeners from its host after the restore. OpenVMM rejects the option for a cold boot, for a snapshot of another tier or without a NIC, and for a NIC captured with any other policy, and every other -networked restore requires the captured policy. `nvx-rs run` does not pass -the option. +[networked restore](#networked-snapshots) requires the captured policy. +`nvx-rs run` does not pass the option. ## Migration from the retired profile diff --git a/doc/usage.md b/doc/usage.md index 25faa6fbd..1b9820c0a 100644 --- a/doc/usage.md +++ b/doc/usage.md @@ -167,8 +167,8 @@ nvx-rs/target/release/nvx-rs run | `--mount-allow HOST_PATH` | none | Expose one existing file or directory inside a `--mount-deny` path again; the hidden directories on the way list only the entries that lead to it. Repeat to allow multiple paths. With several mappings, or a shared file, the path must be absolute; a shared file takes none. See [Access policy](run.md#access-policy). | | `--mount-write HOST_PATH` | none | Make one existing file or directory one of the only writable parts of an `rw` mapping; the guest's writes anywhere else in it fail with `EROFS`. Repeat to declare multiple writable paths. With several mappings, or a shared file, the path must be absolute; a shared file takes none. Active snapshot restore requires the same denied, allowed, and writable paths. | | `--mount-owner {vmm,caller}` | `vmm` | Select the host identity of the guest's operations on every `--mount` directory or file. `caller` performs each one as the guest caller's UID and GID and squashes guest root to the owner of the shared directory or file; it requires a Linux host. See [Run](run.md#file-ownership). | -| `--net IPV4/PREFIX` | none | Enable virtio-net with the static guest IPv4 address and prefix. The guest also gets the IPv6 address that embeds it in `fd00::/96`, such as `fd00::a00:2/120` for `10.0.0.2/24`. | -| `--network-profile {portable}` | none | Select the required cross-platform network behavior contract; must be specified with `--net`. | +| `--net IPV4/PREFIX` | none | Enable virtio-net with the static guest IPv4 address and prefix. The guest also gets the IPv6 address that embeds it in `fd00::/96`, such as `fd00::a00:2/120` for `10.0.0.2/24`. A restore takes the addresses from the snapshot and rejects this option. | +| `--network-profile {portable}` | none | Select the required cross-platform network behavior contract; must be specified with `--net` on a fresh boot, and without it to restore a snapshot captured with `--net`. | | `--network-egress {allow,deny}` | `allow` | Set the default guest egress policy. | | `--network-ingress {allow,deny}` | `deny` | Set the default host ingress policy. `allow` publishes every TCP and UDP port on which the guest listens to clients on the host's private networks, and to the host itself only with `--host-loopback allow`; see [Private-network ingress](run.md#private-network-ingress). Snapshot capture and restore reject `allow`. | | `--network-egress-allow CIDR[:PROTOCOL[:PORT[-PORT]]]` | none | Allow matching guest egress; repeat to add rules. `CIDR` is an IPv4 or IPv6 address or network, and a rule matches only its own family. `PROTOCOL` is `tcp`, `udp`, or `icmp` and matches every port when `PORT` is omitted; only `tcp` and `udp` take a `PORT` or an inclusive `PORT-PORT` range, such as `8000-8010`. | @@ -176,10 +176,10 @@ nvx-rs/target/release/nvx-rs run | `--network-egress-policy-file PATH` | none | Load bounded IPv4 and IPv6 ranges and rule-local CIDR exclusions from JSON. Requires explicit `--network-egress`; cannot be mixed with explicit allow/deny rule flags. | | `--host-loopback {allow,deny}` | existing mapping | Control access between the guest and host loopback services. Explicit `allow` is bidirectional: the guest reaches host loopback through its gateway, and every TCP and UDP port on which the guest listens is published on host loopback for the host alone, or with `--network-ingress allow` admits the host as well; see [Host-loopback access](run.md#host-loopback-access). Snapshot capture and restore reject `allow`. | | `--network-proxy IPV4:TCP-PORT` | none | Allow one explicit host TCP proxy endpoint. | -| `--host-loopback-forward PROTOCOL:HOST_PORT:GUEST_PORT` | none | Publish one TCP or UDP localhost port to the guest; repeat to add forwards. | +| `--host-loopback-forward PROTOCOL:HOST_PORT:GUEST_PORT` | none | Publish one TCP or UDP localhost port to the guest; repeat to add forwards. Snapshot capture and restore reject forwards. | | `--outcome-report PATH` | none | Write a bounded local JSON outcome report. | | `--cmdline TEXT` | empty | Append kernel parameters; `nvx_*` and `tsc=` tokens are reserved. | -| `--restore-snapshot PATH` | none | Restore the immutable machine contract and saved state from a snapshot directory. | +| `--restore-snapshot PATH` | none | Restore the immutable machine contract and saved state from a snapshot directory. A snapshot captured with `--net` needs `--network-profile portable` and the network policy options of its capture, which the restore cannot change; see [Networked snapshots](run.md#networked-snapshots). | | `--restore-processors {1,2,4,8}` | none | Bring this contiguous processor prefix online before restore readiness. Requires an opt-in microVM snapshot and cannot exceed `--processors` capacity. | | `--restore-memory-mib MIB` | none | Select the 128 MiB-aligned RAM target for an expansion-capable snapshot restore, in the range of `--memory-mib`. | | `--restore-ready-path PATH` | none | Publish one restore-readiness event to an existing Unix socket or Windows named pipe. | diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 4fbff00ec..7ef986137 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -608,17 +608,24 @@ message. `nvx-rs run` ([Usage](../doc/usage.md#run)) has the options, with their names, defaults, and value checks, and the checks, messages, outputs, and exit -statuses of the `nvx.py run` that it replaced. It prints `>> COMMAND`, -OpenVMM's command line, quoted as `sandbox run` quotes it, and unless -`--dry-run` is given, it runs OpenVMM with the standard streams, so that the -guest's console and the terminal settings that OpenVMM restores behave as -they do for OpenVMM itself, and exits with OpenVMM's status. +statuses of the `nvx.py run` that it replaced, except as +[Other differences from nvx.py](#other-differences-from-nvxpy) lists. It +prints `>> COMMAND`, OpenVMM's command line, quoted as `sandbox run` quotes +it, and unless `--dry-run` is given, it runs OpenVMM with the standard +streams, so that the guest's console and the terminal settings that OpenVMM +restores behave as they do for OpenVMM itself, and exits with OpenVMM's +status. - A cold boot runs the kernel and the initramfs of `--guest`, `alpine` by default, with `--memory-mib` or else the guest's default, 128 MiB for Alpine and 512 MiB for Ubuntu and Azure Linux, which `Guest::default_memory_mib` gives. `--restore-snapshot` restores a snapshot instead, without them, and with the `--restore-*` options, which need it. +- A cold boot takes `--net` and `--network-profile` together. A restore takes + the addressing of its NIC from the snapshot, as OpenVMM does, so it refuses + `--net`, and passes `--network-profile` alone, which OpenVMM requires, with + the network policy that the snapshot was captured with, to restore a + snapshot that has a NIC. - One directory `--mount` is OpenVMM's `--mount`. Several, or a shared file, are the children of an aggregate at `/run/nvx/shares`, whose guest targets must be absolute, must not overlap, and must survive the kernel command line, @@ -879,6 +886,11 @@ guest agents of NVX advertise bits 0, 4, and 6 in sandbox mode. of `run`'s standard handles and the scratch image's claim alone. - `start` and `exec` refuse, and `stop` terminates, a VM that another runtime identity started; see [Runtime identity](#runtime-identity). +- `run --restore-snapshot` refuses `--net`, whose addressing a restored NIC + takes from the snapshot, and passes `--network-profile` without it, as + OpenVMM requires to restore a snapshot that has a NIC. `nvx.py run` took the + two only together, so it could not restore such a snapshot: OpenVMM refused + its `--net`, or required the `--network-profile` that it omitted. ### Test hooks @@ -890,7 +902,9 @@ one-shot runs, which run `/bin/echo`, `/bin/sleep`, `/bin/true`, or status. It also runs the microVMs of `nvx-rs run`, without a guest: a cold boot, or the restore of a snapshot directory, prints that it booted or restored the microVM, writes an outcome report if asked, and exits with status 0, and a -restore whose snapshot directory is missing fails with status 1. Kernel +restore whose snapshot directory is missing fails with status 1. Like OpenVMM, +it refuses with status 2 a restore with `--net`, whose NIC keeps the addressing +that its snapshot saved, and a cold boot with `--network-profile` alone. Kernel command-line tokens make it fail: `fake_exit_on_start=STATUS`, `fake_exit_status=STATUS`, `fake_guest_features=MASK`, and `fake_legacy_guest=1`, among others; see `nvx::testing::fake_openvmm`. Like diff --git a/nvx-rs/src/host/cli/network.rs b/nvx-rs/src/host/cli/network.rs index 7f9463b93..08c9f76e9 100644 --- a/nvx-rs/src/host/cli/network.rs +++ b/nvx-rs/src/host/cli/network.rs @@ -118,8 +118,9 @@ impl NetworkOptions { } /// Appends the OpenVMM options of these network options and `rules` to `command`, as - /// `_extend_network_arguments` does, for `run`; `sandbox` lowers them through its typed network - /// policy instead. + /// `_extend_network_arguments` does, for `run`, except that `--network-profile` goes without + /// `--net` too, as a restore passes it; `sandbox` lowers them through its typed network policy + /// instead. pub(crate) fn extend_arguments(&self, command: &mut Vec, rules: &NativeRules) { let mut push = |name: &str, value: &str| { command.push(name.to_owned()); @@ -127,10 +128,9 @@ impl NetworkOptions { }; if let Some(net) = &self.net { push("--net", net); - push( - "--network-profile", - self.network_profile.unwrap_or_default(), - ); + } + if let Some(profile) = self.network_profile { + push("--network-profile", profile); } if let Some(egress) = self.network_egress { push("--network-egress", egress); diff --git a/nvx-rs/src/host/cli/run.rs b/nvx-rs/src/host/cli/run.rs index 69883e7b5..71bbcee8c 100644 --- a/nvx-rs/src/host/cli/run.rs +++ b/nvx-rs/src/host/cli/run.rs @@ -7,6 +7,11 @@ //! `nvx.py run` with their names, defaults, types, and choices, and ends a usage error with //! status 2. //! +//! A cold boot takes `--net` and `--network-profile` together. A restore takes the addressing of +//! its NIC from the snapshot, as OpenVMM does, so it refuses `--net`, and passes +//! `--network-profile` alone, which OpenVMM requires, with the network policy that the snapshot +//! was captured with, to restore a snapshot that has a NIC. +//! //! Its shares differ from those of `sandbox`. `--mount-deny`, `--mount-allow`, and `--mount-write` //! name host paths, which OpenVMM attributes to the share that holds them, and which must be //! absolute with several shares or a shared file. One directory share uses `--mount`, several @@ -287,14 +292,21 @@ fn push(command: &mut Vec, name: &str, value: impl Into) { command.push(value.into()); } -/// Checks `options` in the order of `command_run`, before anything that it looks up. +/// Checks `options` in the order of `command_run`, before anything that it looks up, except that +/// a restore refuses `--net` and takes `--network-profile` alone. fn check(options: &Options) -> Result { let refused = |message: String| Err(Exit::error(ErrorCode::Unsupported, message)); let network = &options.network; - if network.net.is_some() != network.network_profile.is_some() { + let restore = options.restore_snapshot.is_some(); + if restore && network.net.is_some() { + return refused( + "--net is not accepted for restore; the snapshot already fixes the network addressing" + .to_owned(), + ); + } + if !restore && network.net.is_some() != network.network_profile.is_some() { return refused("--net and --network-profile must be specified together".to_owned()); } - let restore = options.restore_snapshot.is_some(); for (given, option) in [ (options.restore_ready_path.is_some(), "--restore-ready-path"), (options.restore_processors.is_some(), "--restore-processors"), diff --git a/nvx-rs/src/testing/fake_openvmm.rs b/nvx-rs/src/testing/fake_openvmm.rs index 46e2b8608..9340d56a2 100644 --- a/nvx-rs/src/testing/fake_openvmm.rs +++ b/nvx-rs/src/testing/fake_openvmm.rs @@ -131,6 +131,10 @@ //! on Windows, they take the access of the destination's directory, while OpenVMM gives them its //! own, which the store's unit tests apply. //! +//! Like OpenVMM, the fake takes `--net` only with `--network-profile portable`, and only to boot a +//! guest: a restore takes the addressing of its NIC from its snapshot, so it refuses `--net`, and +//! takes `--network-profile` alone, which a boot refuses. +//! //! Every `--mount-deny`, `--mount-allow`, and `--mount-write` path of a launch must exist in a //! shared directory. Like OpenVMM, the fake attributes each one to the share whose root contains //! it once it has resolved both, so a root that is relative to the working directory, as @@ -1337,9 +1341,18 @@ fn parse(program: &str, arguments: &[String]) -> Result { { return Err("--host-loopback-forward requires explicit --host-loopback allow".to_owned()); } - if values.contains_key("--net") { + // OpenVMM attaches a NIC at boot with `--net`, and restores one with the addressing that its + // snapshot saved, which `--network-profile` selects without `--net`. + let net = values.contains_key("--net"); + if net || values.contains_key("--network-profile") { expect("--network-profile", "portable")?; } + if net && restore.is_some() { + return Err("restore takes microVM network addressing from saved state".to_owned()); + } + if !net && restore.is_none() && values.contains_key("--network-profile") { + return Err("--network-profile portable requires --net or --restore-snapshot".to_owned()); + } let knob = |name: &str| { tokens .iter() @@ -2724,6 +2737,27 @@ mod tests { options.args_dump, temporary.path().join("fake-openvmm-share.json") ); + // A restored NIC keeps the addressing that its snapshot saved, so a restore takes the + // network profile and policy without `--net`. + let networked = run_with(&[ + "--restore-snapshot", + "{dir}/snapshot", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-egress-allow", + "10.0.0.1/32:tcp:443", + "--host-loopback", + "deny", + ]); + let (temporary, options) = parse_run(&networked); + let options = options.unwrap(); + assert_eq!(options.restore, Some(temporary.path().join("snapshot"))); + assert!(options.network.requested); + assert_eq!(options.network.mode, "rules"); + assert_eq!(options.network.allow, 1); + assert_eq!(options.network.host_loopback, "deny"); } #[test] @@ -2760,6 +2794,18 @@ mod tests { restore(&["--restore-memory", "1G"]), "--restore-memory must be given in MiB", ), + ( + restore(&["--net", "10.0.0.2/24", "--network-profile", "portable"]), + "restore takes microVM network addressing from saved state", + ), + ( + restore(&["--network-profile", "open"]), + "--network-profile must be portable, not open", + ), + ( + boot(&["--network-profile", "portable"]), + "--network-profile portable requires --net or --restore-snapshot", + ), ( boot(&["--restore-processors", "2"]), "--restore-processors requires --restore-snapshot", diff --git a/nvx-rs/tests/cli/dry_run.rs b/nvx-rs/tests/cli/dry_run.rs index 25df155b0..b6b3aef41 100644 --- a/nvx-rs/tests/cli/dry_run.rs +++ b/nvx-rs/tests/cli/dry_run.rs @@ -900,6 +900,63 @@ fn run_dry_runs() -> Vec { run(&["--network-egress", "deny"]), cold_boot().with("--network-egress", ["deny"]), ), + // A restore takes the addressing of its NIC from the snapshot, so it passes the profile + // and the policy without `--net`. + dry_run( + "run: restore of a networked snapshot", + run(&[ + "--restore-snapshot", + "{root/snapshot}", + "--network-profile", + "portable", + ]), + restore().with("--network-profile", ["portable"]), + ), + dry_run( + "run: restore with a network policy", + run(&[ + "--restore-snapshot", + "{root/snapshot}", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-egress-allow", + "10.0.0.0/8:tcp:443", + "--network-egress-deny", + "10.1.0.0/16", + "--host-loopback", + "deny", + "--network-proxy", + "10.0.0.1:3128", + ]), + restore() + .with("--network-profile", ["portable"]) + .with("--network-egress", ["deny"]) + .with("--network-egress-allow", ["10.0.0.0/8:tcp:443"]) + .with("--network-egress-deny", ["10.1.0.0/16"]) + .with("--host-loopback", ["deny"]) + .with("--network-proxy", ["10.0.0.1:3128"]), + ), + dry_run( + "run: restore with an egress policy file", + run(&[ + "--restore-snapshot", + "{root/snapshot}", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-egress-policy-file", + "{root/policy.json}", + ]), + restore() + .with("--network-profile", ["portable"]) + .with("--network-egress", ["deny"]) + .with("--network-egress-allow", COMPILED_ALLOW) + .with("--network-egress-deny", COMPILED_DENY), + ) + .with_file("policy.json", POLICY), dry_run( "run: host loopback forward", run(&["--host-loopback-forward", "tcp:8080:80"]), diff --git a/nvx-rs/tests/cli/one_shot.rs b/nvx-rs/tests/cli/one_shot.rs index 44a4713de..22694bf1b 100644 --- a/nvx-rs/tests/cli/one_shot.rs +++ b/nvx-rs/tests/cli/one_shot.rs @@ -440,6 +440,41 @@ fn run_launches() -> Vec { status: 4, network: NO_NETWORK, }), + // A restore passes the network profile and policy of a snapshot that has a NIC without + // `--net`, which OpenVMM refuses on a restore. + launch( + "run: restore of a networked snapshot", + run(&[ + "--restore-snapshot", + "{root/snapshot}", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-egress-allow", + "10.0.0.0/8:tcp:443", + "--host-loopback", + "deny", + "--outcome-report", + "{root/reports/openvmm-run.json}", + ]), + restore() + .with("--network-profile", ["portable"]) + .with("--network-egress", ["deny"]) + .with("--network-egress-allow", ["10.0.0.0/8:tcp:443"]) + .with("--host-loopback", ["deny"]) + .with("--microvm-report", ["{root/reports/openvmm-run.json}"]), + 0, + restored, + ) + .with_file(SNAPSHOT) + .recorded_in(Some("fake-openvmm-reports.json")) + .reporting(Report { + path: "reports/openvmm-run.json", + category: "success", + status: 0, + network: ("applied", "rules", 1, 0, "deny"), + }), launch( "run: network outcome report", run(&[ diff --git a/nvx-rs/tests/cli/validation.rs b/nvx-rs/tests/cli/validation.rs index 74b0201a7..7b19e9717 100644 --- a/nvx-rs/tests/cli/validation.rs +++ b/nvx-rs/tests/cli/validation.rs @@ -710,6 +710,31 @@ fn run_refusals() -> Vec { run_dry(&["--network-profile", "portable"]), &["--net", "--network-profile"], ), + // A restored NIC keeps the addressing that its snapshot saved. + refusal( + "run: net of a restore", + "unsupported", + run_dry(&[ + "--restore-snapshot", + "{root/snapshot}", + "--net", + "10.0.0.2/24", + "--network-profile", + "portable", + ]), + &["--net", "restore", "network addressing"], + ), + refusal( + "run: net without profile of a restore", + "unsupported", + run_dry(&[ + "--restore-snapshot", + "{root/snapshot}", + "--net", + "10.0.0.2/24", + ]), + &["--net", "restore", "network addressing"], + ), refusal( "run: ready path without snapshot", "unsupported", diff --git a/nvx-rs/tests/fake_openvmm.rs b/nvx-rs/tests/fake_openvmm.rs index a07217991..7f7881aa5 100644 --- a/nvx-rs/tests/fake_openvmm.rs +++ b/nvx-rs/tests/fake_openvmm.rs @@ -641,6 +641,29 @@ fn runs_boot_or_restore_a_microvm_on_the_terminal() { text(&output.stderr), "nvx-fake-openvmm: a restore takes no --memory\n" ); + + // As OpenVMM does, a restore takes the network profile and policy of a snapshot that has a NIC + // without `--net`, whose addressing the snapshot saved, and refuses `--net`. + let networked = [ + "--restore-snapshot", + &snapshot, + "--network-profile", + "portable", + "--network-egress", + "deny", + ]; + let output = run_microvm(root, &networked); + assert_eq!(output.status.code(), Some(0), "{}", text(&output.stderr)); + assert_eq!( + text(&output.stdout), + "nvx-fake-openvmm: restored the microVM\n" + ); + let output = run_microvm(root, &[&networked[..], &["--net", "10.0.0.2/24"]].concat()); + assert_eq!(output.status.code(), Some(2)); + assert_eq!( + text(&output.stderr), + "nvx-fake-openvmm: restore takes microVM network addressing from saved state\n" + ); } #[test]