diff --git a/doc/design/1.5-compatibility-and-versioning-model.md b/doc/design/1.5-compatibility-and-versioning-model.md index e21b9b863..27b42cc65 100644 --- a/doc/design/1.5-compatibility-and-versioning-model.md +++ b/doc/design/1.5-compatibility-and-versioning-model.md @@ -73,13 +73,14 @@ activation ranges. Layer-role presence and device geometry stay fixed: a platform template accepts a substitute layer only with the same recorded logical length and block sizes. -A template's compatibility key also records its network, as OpenVMM's -arguments to which its class lowers it: the guest's address and the policy of -its NIC, which every restore of the template passes unchanged, so a template -serves the clones of its own network policy alone. The key's encoding, version -1, includes that network and has not been released, so a template store that -an earlier build created is not supported: delete it and create a new one. -There is no migration. +A template's compatibility key also records its NIC, if it has one: the +guest's address and the identity of the complete deny-all policy under which +the capture attached it. Every restore of the template replaces that policy +with its clone's own, so a template serves the clones of every network policy +at its guest's address, and its key records none of them. The key's encoding, +version 1, has not been released, so a template store that an earlier build +created is not supported: delete it and create a new one. There is no +migration. ## 1.5.3 Persisted State diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 2fbd85d2d..d7877a97c 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -188,7 +188,9 @@ kernel's view and compares names as the kernel does in both. at ASCII whitespace, at most one token, with one IPv4 or IPv6 address in the form that OpenVMM writes, and `write_resolver` writes the resolver file that names it, `nameserver ADDRESS`, as NVX's init writes the initramfs's -`/etc/resolv.conf`. +`/etc/resolv.conf`. `gateway` reads the IPv4 gateway of the guest's NIC from +OpenVMM's `virtnet_gw=` token, `GATEWAY_TOKEN`, as strictly: the address to +which the announcer of a clone's listeners sends its announcements. ### `nvx::protocol::export` @@ -572,9 +574,10 @@ are restored. template's capture, without I/O: the placeholder `distro`, which its `DistroBinding` binds, and whose length is the capacity of every clone's image, over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's -transaction directory, with the dormant filesystem slot, and no NIC or one under -the template's policy, which allows neither ingress nor host-loopback access -both ways; `TemplateResources`, with the processors, 1, 2, 4, or 8, and those +transaction directory, with the dormant filesystem slot, and no NIC or one at +the guest's address of the adapter's network, under the complete deny-all +policy, whatever the network's own, which each clone's restore replaces with the +clone's; `TemplateResources`, with the processors, 1, 2, 4, or 8, and those online at boot, as many or fewer, and RAM and a `--memory-capacity` in blocks of `MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, with `maxcpus=` and the processors online at boot when they are fewer, so that the capture's @@ -595,17 +598,19 @@ identity is unbound, a fresh scratch, and the dormant filesystem slot. `CaptureClass` holds the invariant inputs of a capture, known before it: the runtime, the guest's build, the backend, the requested CPU profile, the processors, those online at boot, and RAM, the geometries of the disks, the -network, and the adapter's other `Invariants`. +network, of which it takes what the template captures alone, its NIC at the +guest's address under the complete deny-all policy, and the adapter's other +`Invariants`. `TemplateKey::new` checks that the capture made the machine that its class requested, and derives the template's compatibility key, from metadata alone: the runtime and contract versions through which its clones are restored, the backend, CPU profile and effective CPUID, topology, rates, resources, kernel -command line, disks, and devices of its machine, and its network, as its class -lowered it to OpenVMM's arguments, the guest's address and the policy of its -NIC, which every restore of the template passes unchanged, so that a template -serves the clones of its network policy alone. A capture's own metadata, such as -its time anchor and host, stays out of the key. +command line, disks, and devices of its machine, with the NIC's addresses and +the identity of the complete deny-all policy under which the capture attached +it, which each clone's restore replaces with the clone's own, so that a template +with a NIC serves the clones of every network policy at its guest's address. A +capture's own metadata, such as its time anchor and host, stays out of the key. A key and a class each have one canonical, versioned encoding, whose domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts @@ -613,9 +618,10 @@ to and from the `nvx::activation::Identity` by which a clone's activation record names its template's key, or a `ClassIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, `tests/vectors/template-key-v1.txt`, pin both encodings and identities. The -key's encoding, version 1, includes the template's network and has not been -released, so a template store that an earlier build created is not supported: -delete it and create a new one. There is no migration. +key's encoding, version 1, leaves the network policy out, since templates +capture their NIC under the complete deny-all policy, and has not been released, +so a template store that an earlier build created is not supported: delete it +and create a new one. There is no migration. `Store` keeps the generations of templates that a host published, apart from every sandbox's store, at a location that the adapter chooses: a private @@ -664,11 +670,12 @@ and no launch. `ClonePlan::new` plans a clone under the lock that acquired its lease, before the launch that the lease reserves, from metadata alone: the leased generation, its key, which must speak the current contracts, the boot and control contracts -of the runtime that captured it included, and its snapshot; the template's -network exactly, as its key records it, or none for a template without a NIC, so -that a template serves the clones of its network policy, at its guest's address, -which OpenVMM restores from the snapshot, alone; a processor target of 1, 2, 4, -or 8 from the template's processors online at boot to its capacity; a memory +of the runtime that captured it included, and its snapshot; a network that +attaches a NIC if and only if the template has one, at the guest's address of +the template's NIC, which OpenVMM restores from the snapshot, so that a template +with a NIC serves the clones of every network policy at its guest's address; a +processor target of 1, 2, 4, or 8 from the template's processors online at boot +to its capacity; a memory target in whole blocks of `MEMORY_BLOCK_SIZE` from the template's RAM to its capacity; the clone's image, which the check of its `DistroBinding` opens; its upper, either the @@ -705,10 +712,12 @@ partition table the guest checks with `nvx::gpt` as at a cold launch; a tmpfs, or the scratch image's binding identity with the journal policy; the export, a directory, with its plan's identity, and its mappings, the plan's binds, from 1 to 4,096, or no export and no mappings, which leaves the slot dormant; the -template NIC's policy identity, as its key records it, unpublished listeners, -and the guest's proxy URL; and the processor and memory targets. The identity of -a binding or of an export plan is the SHA-256 digest of a domain and of its -serde form, never of a file's content. +clone's network, with its identity, the DNS server that OpenVMM serves the guest +at a gateway under its policy, if any, whether the guest publishes its +listeners, as under a policy that allows ingress or host-loopback access both +ways, and the guest's proxy URL; and the processor and memory targets. The +identity of a binding, of an export plan, or of a network is the SHA-256 digest +of a domain and of its serde form, never of a file's content. The image must present the logical and physical block sizes of the template's placeholder and be no longer than it, since a restore presents every clone's @@ -732,8 +741,9 @@ writes the clone's activation record in its canonical encoding, the VMM's restore-ready event, at the socket `restore.sock` on Linux, and at a named pipe of a random name on Windows; and it runs `supervisor::launch` with OpenVMM's restore of the clone's devices, with `--restore-absent-scratch` for a -clone in RAM, and of the template's network, as its -key records it, unchanged, its export bound to the dormant slot, `--mount` with +clone in RAM, and of its network, whose policy `--restore-network-policy` +installs on the template's NIC in place of the complete deny-all policy, its +export bound to the dormant slot, `--mount` with the directory, and its hidden, re-exposed, and writable paths and its owner, `--restore-snapshot` with the generation's snapshot in place, which no clone copies or changes, so that clones of one generation restore it at once, @@ -1261,9 +1271,9 @@ expects one, or a GPT disk whose GUID it checks if the record expects one, its upper, a tmpfs or the scratch with the record's journal policy, and its overlay mode, and with the `RootTemplate` that the guest's own configuration holds, which its template captured: the runtime directory, the block devices, the -timeout, and the host filesystem device's virtio-fs tag, but waits for no device -past the activation's deadline, which the adapter passes, so that the -application ends by then. +timeout, the host filesystem device's virtio-fs tag, and the gateway of the +template's NIC, if it has one, but waits for no device past the activation's +deadline, which the adapter passes, so that the application ends by then. For a GPT disk, it first asks the kernel to reread the disk's partition table with `BLKRRPART`, since the kernel created the block devices of partitions for @@ -1282,19 +1292,25 @@ filesystem device with `Root::mount_export`, read-write, at `hostfs/root` in the runtime directory, outside the root, since the record names no mode and OpenVMM limits each child to its own access; with its network's DNS server, it writes a resolver file that names it, mode 0444, to `RESOLVER_FILE` in the runtime -directory. It then binds the runtime's files for every execution with one -`Entry::bind_runtime` in the caller's mount namespace: the machine ID's file -read-only over the root's `/etc/machine-id` and the resolver file over its +directory; and with a network that publishes the workload's listeners, it starts +their announcer, `/sbin/nvx-ingress inherit GATEWAY_IPV4`, which announces from +its standard input, a UDP socket that the application bound, as root, to a +privileged source port. It then binds the runtime's files for every execution +with one `Entry::bind_runtime` in the caller's mount namespace: the machine ID's +file read-only over the root's `/etc/machine-id` and the resolver file over its `/etc/resolv.conf`, or, with neither, nothing; the entry checks the record's hostname, which every execution's entry takes with `Entry::hostname`. It returns the `Applied` root and workload, from which `take_export` hands the adapter the export's `SourceRoot`, to map the host's mapping table from once the clone is -ready, and `proxy` the network's proxy endpoint, which the adapter tells its -workloads as at a cold boot, and whose `teardown` closes the export's root and +ready, `proxy` the network's proxy endpoint, which the adapter tells its +workloads as at a cold boot, and `announcer` the announcer, which the adapter +watches through its pidfd, and must end the clone once it ended, since the +gateway then keeps publishing the listeners that it last announced until the VM +ends, and whose `teardown` stops the announcer, closes the export's root, and removes the binds before it tears the root down, as a drop does, or an `ApplyError`, `Export` or `Network` among them, after which nothing that it mounted or bound stays mounted unless the error reports the unmounts that -failed, and applies none of the record's listener publication. +failed, and no announcer that it started runs. Once the adapter passed the activation what the session received while the application ran, which came before Ready, `Activation::complete` opens the fence @@ -2462,7 +2478,7 @@ disk's block device. | `nvx-exit [CODE]` | Stops the VM with an 8-bit exit code through OpenVMM's control port 0x604 (x86_64) | | `nvx-device-io file ...`, `nvx-device-io network ...`, `nvx-device-io map PATH TEXT` | The benchmark helper that measures file and UDP round-trip I/O and prints one JSON line, and reports a file benchmark's progress on stderr; `map` writes TEXT at the start of a file through a shared mapping and holds the dirty mapping until its standard input ends, for the microVM scenarios (x86_64) | | `nvx-managed-agent CONTROL_TTY ROOTFS HOSTNAME UID GID USER HOME` | Serves the managed control session and supervises sequential workloads; `--exec-config-fd FD -- WORKLOAD ...` is the direct agent's internal launch helper | -| `nvx-ingress GATEWAY_IPV4` | Announces reachable TCP listeners and settled UDP sockets to the portable-network gateway; `test DIRECTORY` runs synthetic procfs snapshots | +| `nvx-ingress [inherit] GATEWAY_IPV4` | Announces reachable TCP listeners and settled UDP sockets to the portable-network gateway, from a privileged source port that it binds, or, with `inherit`, from its standard input, a socket that a clone's application bound to one; `test DIRECTORY` runs synthetic procfs snapshots | | `nvx-time [--report-only] COMMAND` | Enforces NVX time ABI v1: runs its conformance checks, repairs the wall clock after a restore, publishes the time state, and runs the discipline and violation-watcher daemon; `test NAME ...` runs its decoder and process test hooks (x86_64) | | `nvx-time-probe latency\|warp\|clock ...` | Diagnoses clock-read latency, cross-CPU TSC warp and offset, the clocks, CPUID, clock events, kernel messages, and optionally MSRs (x86_64) | diff --git a/nvx-rs/src/guest/clone.rs b/nvx-rs/src/guest/clone.rs index 833917bab..caf0a354d 100644 --- a/nvx-rs/src/guest/clone.rs +++ b/nvx-rs/src/guest/clone.rs @@ -152,19 +152,20 @@ //! tmpfs or the scratch, with the record's journal policy, and its overlay mode, and with the //! inputs that the guest takes from its own configuration, which its template captured, a //! [`RootTemplate`]: the runtime directory, the block devices, the host filesystem device's -//! virtio-fs tag, and how long the assembly waits for each block device, but never past the -//! activation's deadline, which the adapter passes, so that the application ends by then. For a GPT -//! disk, it first asks the kernel to reread the disk's partition table, once the distro device -//! appeared, by the same deadline: the kernel created the block devices of partitions from the -//! template's placeholder, when it found it, and creates those of the clone's disk then, from the -//! same read-only bytes that the assembly reads and checks, as at a cold boot. The upper must be -//! fresh: the assembly refuses one whose `upper` or `work` directory exists and is not empty, since -//! a clone's root starts as its image alone, while a cold sandbox's scratch keeps its upper across -//! boots; the check reads at most one entry of each, and no file. If the record has a workload -//! identity, it then checks its account in the assembled root, before Ready, as NVX's init checks -//! it at boot: the identity is [`Identity::new`](super::entry::Identity::new) of the record's user -//! and group IDs, without supplementary groups or capabilities, in existing-account mode, and every -//! execution runs as it. Without one, the adapter gives each execution its own identity, which +//! virtio-fs tag, the gateway of the template's NIC, if it has one, and how long the assembly waits +//! for each block device, but never past the activation's deadline, which the adapter passes, so +//! that the application ends by then. For a GPT disk, it first asks the kernel to reread the disk's +//! partition table, once the distro device appeared, by the same deadline: the kernel created the +//! block devices of partitions from the template's placeholder, when it found it, and creates those +//! of the clone's disk then, from the same read-only bytes that the assembly reads and checks, as +//! at a cold boot. The upper must be fresh: the assembly refuses one whose `upper` or `work` +//! directory exists and is not empty, since a clone's root starts as its image alone, while a cold +//! sandbox's scratch keeps its upper across boots; the check reads at most one entry of each, and +//! no file. If the record has a workload identity, it then checks its account in the assembled +//! root, before Ready, as NVX's init checks it at boot: the identity is +//! [`Identity::new`](super::entry::Identity::new) of the record's user and group IDs, without +//! supplementary groups or capabilities, in existing-account mode, and every execution runs as it. +//! Without one, the adapter gives each execution its own identity, which //! [`Entry::check`](super::entry::Entry::check) checks. If the record asks for a generated machine //! ID, it then generates one, once, with //! [`MachineId::generate`](super::entry::MachineId::generate), after the repair's reseed, so that @@ -174,18 +175,25 @@ //! `hostfs/root` in the runtime directory, outside the root, where only the guest's root user //! reaches it: the record names no mode, and OpenVMM limits each of the export's children to its //! own access. If the record's network names a DNS server, it writes a resolver file that names -//! it to [`RESOLVER_FILE`] in the runtime directory. It then binds the runtime's files into the -//! root for every execution, with one [`Entry::bind_runtime`](super::entry::Entry::bind_runtime), -//! in the caller's mount namespace: the generated machine ID's file, read-only over the root's -//! `/etc/machine-id`, and the resolver file, read-only over the root's `/etc/resolv.conf`, or, -//! with neither, nothing. The entry checks the record's hostname first, which every execution's -//! entry then takes. It returns the [`Applied`] root and workload, which the adapter keeps, with -//! the export's root, from which the adapter maps the host's mapping table once the clone is -//! ready, and the network's proxy endpoint, which the adapter tells its workloads as at a cold -//! boot, and whose [`teardown`](Applied::teardown) closes the export's root and removes the binds -//! before it tears the root down, or an [`ApplyError`], after which nothing that it mounted or -//! bound stays mounted, unless the error holds the unmounts that failed. It applies none of the -//! record's other settings: the publication of the workload's listeners. +//! it to [`RESOLVER_FILE`] in the runtime directory. If the network publishes the workload's +//! listeners, it starts their announcer, with +//! [`Announcer::start`](super::ingress::Announcer::start), to the template's gateway: the +//! template captured its NIC under a policy that publishes nothing, so that no announcer ran in +//! it. It then binds the runtime's files into the root for every execution, with one +//! [`Entry::bind_runtime`](super::entry::Entry::bind_runtime), in the caller's mount namespace: +//! the generated machine ID's file, read-only over the root's `/etc/machine-id`, and the +//! resolver file, read-only over the root's `/etc/resolv.conf`, or, with neither, nothing. The +//! entry checks the record's hostname first, which every execution's entry then takes. It returns +//! the [`Applied`] root and workload, which the adapter keeps, with the export's root, from which +//! the adapter maps the host's mapping table once the clone is ready, the network's proxy +//! endpoint, which the adapter tells its workloads as at a cold boot, and the announcer, which the +//! adapter watches, and must end the clone once the announcer ended: the gateway then keeps +//! publishing the listeners that the announcer last announced until the VM ends, and learns of no +//! later change of them. The [`teardown`](Applied::teardown) of the [`Applied`] stops the +//! announcer, closes the export's root, and removes the binds before it tears the root down. An +//! application that fails returns an [`ApplyError`] instead, after which nothing that it mounted or +//! bound stays mounted, unless the error holds the unmounts that failed, and no announcer that it +//! started runs. //! //! # Example //! @@ -271,8 +279,10 @@ //! application, [`apply`](fn@apply), needs the guest's root user, in the guest's initial mount //! namespace, as [`root::assemble`] does: for a GPT disk, it asks the kernel to reread the disk's //! partition table, which needs `CAP_SYS_ADMIN`; it mounts the root, reads the account databases -//! below it, and, if it fails, tears it down. It starts no process, and generates at most one ID, -//! a machine ID that the record asks for. It mounts the host export of a record that has one, and +//! below it, and, if it fails, tears it down. It starts no process but the announcer of a record +//! whose network publishes the workload's listeners, whose socket it binds to a privileged source +//! port, and whose status it collects through the [`Applied`], and it generates at most one ID, a +//! machine ID that the record asks for. It mounts the host export of a record that has one, and //! writes the machine ID's and the resolver's files in the runtime directory, whose binds over //! the root's `/etc/machine-id` and `/etc/resolv.conf` need Linux 6.8 or later. diff --git a/nvx-rs/src/guest/clone/activation/apply.rs b/nvx-rs/src/guest/clone/activation/apply.rs index 6de3b9522..7218ddef9 100644 --- a/nvx-rs/src/guest/clone/activation/apply.rs +++ b/nvx-rs/src/guest/clone/activation/apply.rs @@ -8,7 +8,7 @@ mod tests; use std::fmt; use std::fs; use std::io; -use std::net::IpAddr; +use std::net::{IpAddr, Ipv4Addr}; use std::os::unix::fs::{OpenOptionsExt, PermissionsExt}; use std::path::{Path, PathBuf}; use std::time::{Duration, Instant}; @@ -17,11 +17,14 @@ use std::time::{Duration, Instant}; use super::super::Devices; #[cfg(doc)] use super::Activation; -use crate::activation::{self, ActivationRecord, Configuration, MachineIdMode, WorkloadIdentity}; +use crate::activation::{ + self, ActivationRecord, Configuration, MachineIdMode, Publication, WorkloadIdentity, +}; use crate::gpt::Expected; #[cfg(doc)] use crate::guest::entry::AccountMode; use crate::guest::entry::{self, Entry, Identity, MachineId, RuntimeBinds}; +use crate::guest::ingress::{Announcer, StartError}; use crate::guest::mappings::{self, Access, SourceRoot}; use crate::guest::root::{self, Assembly, AssemblyError, Device, Root, TeardownError}; use crate::protocol::cmdline::write_resolver; @@ -39,11 +42,12 @@ pub const RESOLVER_FILE: &str = "resolv.conf"; /// The mode of the resolver file, whatever the umask: every execution may read it. const RESOLVER_MODE: u32 = 0o444; -/// The inputs of a clone's root assembly that its guest takes from its own configuration, which -/// its template captured, so that every clone of the template has the same: the runtime -/// directory, the block devices, which their MMIO windows name, with the sizes that they must -/// have, how long the assembly waits for each, and the host filesystem device, which its -/// virtio-fs tag names. [`apply`](fn@apply) takes the rest from the activation record. +/// The inputs of a clone's activation that its guest takes from its own configuration, which its +/// template captured, so that every clone of the template has the same: the runtime directory, +/// the block devices, which their MMIO windows name, with the sizes that they must have, how long +/// the assembly of the root waits for each, the host filesystem device, which its virtio-fs tag +/// names, and the IPv4 gateway of the template's NIC, if it has one. [`apply`](fn@apply) takes +/// the rest from the activation record. #[derive(Clone, Debug, PartialEq, Eq, Hash)] pub struct RootTemplate { /// The runtime directory, below which the root is mounted, as [`Assembly::runtime`] tells, @@ -62,6 +66,11 @@ pub struct RootTemplate { /// nothing mounted, as [`Devices::filesystem_tag`] names it, and through which the host /// attaches the export of a record that has one. pub filesystem_tag: String, + /// The IPv4 gateway of the template's NIC, which the template's kernel command line names, as + /// [`gateway`](crate::protocol::cmdline::gateway) reads it, if the template has a NIC: the + /// gateway to which the announcer of the workload's listeners sends its announcements, for a + /// record that publishes them. + pub gateway: Option, } /// A clone's root and workload, which [`apply`](fn@apply) activated: the [`Root`] that it @@ -69,7 +78,8 @@ pub struct RootTemplate { /// export, if the record has one, the runtime's binds that it made on the root, in the caller's /// mount namespace, the identity in which every execution runs, if the record has a workload /// identity, the workload's hostname and machine ID, the record's, the resolver file that names -/// the DNS server of the record's network, if it names one, and the endpoint of its proxy, if any. +/// the DNS server of the record's network, if it names one, the endpoint of its proxy, if any, and +/// the announcer of the workload's listeners, if the record's network publishes them. /// /// Without a workload identity, the adapter gives each execution its own identity, which /// [`Entry::check`] checks when the adapter prepares the execution's entry, as at a cold boot. @@ -78,11 +88,17 @@ pub struct RootTemplate { /// runtime's binds, which each execution's mount namespace copies from the caller's. The adapter /// [takes](Self::take_export) the export's root, from which it maps the host's mapping table once /// the clone is ready, and tells its workloads the [proxy's endpoint](Self::proxy) as at a cold -/// boot. [`teardown`](Self::teardown) closes the export's root, if the adapter did not take it, -/// removes the binds, then tears the root down, and dropping it does the same, ignoring failures, -/// as dropping a [`Root`] does. +/// boot. The adapter watches the [announcer](Self::announcer) through its pidfd, and must end the +/// clone once the announcer ended: the gateway then keeps publishing the listeners that the +/// announcer last announced until the VM ends, and learns of no later change of them. +/// [`teardown`](Self::teardown) stops the announcer, closes the export's root, if the adapter did +/// not take it, removes the binds, then tears the root down, and dropping it does the same, +/// ignoring failures, as dropping a [`Root`] does. #[derive(Debug)] pub struct Applied { + /// The announcer of the workload's listeners, if the record's network publishes them, which a + /// drop stops first, as a teardown does. + announcer: Option, /// The root of the host export, which keeps the export's mount busy, so that a drop closes it /// before the root drops, unless the adapter took it. export: Option, @@ -155,6 +171,20 @@ impl Applied { self.proxy.as_deref() } + /// Returns the announcer of the workload's listeners, which the application started, if the + /// record's network publishes them, so that the adapter polls its [pidfd](Announcer::pidfd), + /// and ends the clone once the announcer ended, as it must: the gateway then keeps publishing + /// the listeners that the announcer last announced, and learns of no later change of them. + pub fn announcer(&self) -> Option<&Announcer> { + self.announcer.as_ref() + } + + /// Returns the announcer of the workload's listeners, as [`announcer`](Self::announcer) + /// does, so that the adapter collects its status once it ended. + pub fn announcer_mut(&mut self) -> Option<&mut Announcer> { + self.announcer.as_mut() + } + /// Takes the root of the host export, if the record has an export, and the adapter did not /// take it before: the [`SourceRoot`] from which the adapter maps the host's mapping table /// with [`Root::map`] once the clone is ready, as at a cold boot. @@ -166,11 +196,12 @@ impl Applied { self.export.take() } - /// Closes the root of the host export, unless the adapter [took](Self::take_export) it, - /// removes the runtime's binds, then tears the root down, as [`Root::teardown`] does, which - /// unmounts the host export too, in the caller's mount namespace, which must be the one where - /// the application made the binds: the export's root and the binds keep the root's mounts - /// busy, and a teardown detaches no busy mount. + /// Stops the announcer of the workload's listeners, if the application started one, as + /// [`Announcer::stop`] does, ignoring its failure, closes the root of the host export, unless + /// the adapter [took](Self::take_export) it, removes the runtime's binds, then tears the root + /// down, as [`Root::teardown`] does, which unmounts the host export too, in the caller's mount + /// namespace, which must be the one where the application made the binds: the export's root + /// and the binds keep the root's mounts busy, and a teardown detaches no busy mount. /// /// It attempts both the removal and the teardown, whatever failed before, and fails with the /// binds that stay mounted, and the unmounts of the root's teardown that failed, whose mounts @@ -180,24 +211,30 @@ impl Applied { export, mut binds, root, + announcer, .. } = self; - teardown_with(&mut LiveSystem, export, binds.take(), root) + teardown_with(&mut LiveSystem, announcer, export, binds.take(), root) } /// Returns the assembled root, the runtime's binds on it, and the identity in which every /// execution runs, if any, for an adapter that keeps them apart: it then removes the binds /// with [`RuntimeBinds::remove`] before it tears the root down, in the same mount namespace, - /// since dropping them leaves them mounted. It closes the root of the host export, unless the - /// adapter [took](Self::take_export) it. + /// since dropping them leaves them mounted. It stops the announcer of the workload's + /// listeners, if the application started one, and closes the root of the host export, unless + /// the adapter [took](Self::take_export) it. pub fn into_parts(self) -> (Root, RuntimeBinds, Option) { let Self { export, mut binds, root, identity, + announcer, .. } = self; + if let Some(announcer) = announcer { + let _ = announcer.stop(); + } drop(export); (root, binds.take(), identity) } @@ -274,8 +311,8 @@ impl std::error::Error for AppliedTeardownError {} /// the record expects of it, if any, and the bound of its first partition; its upper, a tmpfs, /// or the scratch, with the record's journal policy; and its overlay mode; and, from /// `template`, the runtime directory, the devices, and the timeout. It takes the workload's -/// hostname and machine-ID mode, whether it has an export, and its network's DNS server and -/// proxy endpoint from the record too; +/// hostname and machine-ID mode, whether it has an export, and its network's DNS server, +/// proxy endpoint, and publication of the workload's listeners from the record too; /// 2. for a GPT disk, asks the kernel to reread the disk's partition table, once the distro /// device appeared, so that the block devices of the disk's partitions are those of the /// clone's disk, rather than those of the template's placeholder, which the kernel created @@ -322,15 +359,22 @@ impl std::error::Error for AppliedTeardownError {} /// new file, of mode 0444, whatever the umask, so that every execution may read it. A file /// there already, or one that cannot be created or written, fails with /// [`ApplyError::Network`], once the application tore the root down; -/// 8. binds the runtime's files into the root, for every execution, with one +/// 8. if the record's network publishes the workload's listeners, starts their announcer, to +/// the gateway of [`RootTemplate::gateway`], with [`Announcer::start`]: it binds a UDP socket +/// to a privileged source port, and spawns `/sbin/nvx-ingress inherit GATEWAY_IPV4` with the +/// socket as its standard input, and waits until the program runs, but for no announcement. +/// The template captured its NIC under a policy that publishes nothing, so no announcer ran +/// in it. A template without a gateway, a socket that cannot be bound, and a program that does +/// not run fail with [`ApplyError::Network`], once the application tore the root down; +/// 9. binds the runtime's files into the root, for every execution, with one /// [`Entry::bind_runtime`], in the caller's mount namespace, of which each execution's starts /// as a copy: a generated machine ID's file, read-only over the root's `/etc/machine-id`, then /// the resolver file, read-only over the root's `/etc/resolv.conf`, and, with the image's own /// machine ID and without a resolver file, nothing. The entry checks the record's hostname /// first, which every execution's entry then takes, through [`Entry::hostname`]. A failure to /// bind the resolver file fails with [`ApplyError::Network`], and any other with -/// [`ApplyError::View`], once the binds that it made were removed, and the application tore -/// the root down. +/// [`ApplyError::View`], once the binds that it made were removed, the application stopped +/// the announcer that it started, and tore the root down. /// /// The activation answers a retry of the Activate without applying the record again, so every /// execution, and every retry, sees the same machine ID and resolver file. A failure after the @@ -338,12 +382,12 @@ impl std::error::Error for AppliedTeardownError {} /// finally, and the adapter ends the VM. /// /// It returns the root of the host export, the [`SourceRoot`] from which the adapter maps the -/// host's mapping table once the clone is ready, as at a cold boot, and the endpoint of the -/// network's proxy, which it leaves to the adapter, which tells its workloads as at a cold boot. -/// It applies none of the record's other settings: the publication of the workload's listeners. -/// It starts no process, generates at most one ID, after the activation accepted the record, and -/// never powers the VM off: a failure, which the adapter reports to [`Activation::complete`], -/// fails the activation finally, and the adapter ends the VM. +/// host's mapping table once the clone is ready, as at a cold boot, the endpoint of the network's +/// proxy, which it leaves to the adapter, which tells its workloads as at a cold boot, and the +/// announcer that it started, which the adapter watches. It starts no other process, generates at +/// most one ID, after the activation accepted the record, and never powers the VM off: a failure, +/// which the adapter reports to [`Activation::complete`], fails the activation finally, and the +/// adapter ends the VM. /// /// It needs the guest's root user, in the guest's initial mount namespace, as /// [`root::assemble`] does, and the binds of a generated machine ID and of a resolver file need @@ -363,6 +407,7 @@ pub fn apply( machine_id: activated.machine_id, resolver: activated.resolver, proxy: activated.proxy, + announcer: activated.announcer, }) } @@ -424,7 +469,9 @@ pub enum ApplyError { }, /// The record's network settings could not be applied, as this error tells: the resolver /// file that names the network's DNS server could not be written, or bound over the root's - /// `/etc/resolv.conf`. The application then tore the root down. + /// `/etc/resolv.conf`, or the announcer of the workload's listeners, which the network + /// publishes, could not start. The application then stopped an announcer that it started, + /// and tore the root down. Network { /// Why the network settings could not be applied. error: NetworkError, @@ -505,6 +552,12 @@ pub enum NetworkError { /// [`entry::Error::Rollback`] of one, after which the entry removed the binds that it made, /// unless the error names those that stay. Bind(entry::Error), + /// The network publishes the workload's listeners, but the template names no IPv4 gateway, + /// [`RootTemplate::gateway`], to which their announcer would send its announcements. + Gateway, + /// The announcer of the workload's listeners, which the network publishes, could not start, + /// as [`Announcer::start`] tells. + Announcer(StartError), } impl fmt::Display for NetworkError { @@ -512,6 +565,11 @@ impl fmt::Display for NetworkError { match self { Self::Write(error) => write!(formatter, "the resolver file cannot be written: {error}"), Self::Bind(error) => error.fmt(formatter), + Self::Gateway => formatter.write_str( + "the network publishes the workload's listeners, but the template names no IPv4 \ + gateway to announce them to", + ), + Self::Announcer(error) => error.fmt(formatter), } } } @@ -520,8 +578,11 @@ impl std::error::Error for NetworkError { fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { match self { Self::Write(error) => Some(error), - // The entry's error names the resolver file already, so it stands for this one. + // The entry's error names the resolver file already, so it stands for this one, as + // the start's error names the announcer. Self::Bind(error) => error.source(), + Self::Announcer(error) => error.source(), + Self::Gateway => None, } } } @@ -538,6 +599,10 @@ trait System { /// The runtime's binds that a bind made: [`RuntimeBinds`], in a VM. type Binds; + /// The announcer of the workload's listeners that a start returned: an [`Announcer`], in a + /// VM. + type Announcer; + /// Asks the kernel to reread the partition table of the GPT disk on the distro device /// `distro`, once it appeared, waiting for it for as long as `timeout` allows, but until /// `deadline` at the latest, if any, as [`root::reread_partitions`] does. @@ -579,6 +644,13 @@ trait System { /// [`write_resolver_file`] does. fn write_resolver(&mut self, path: &Path, server: IpAddr) -> io::Result<()>; + /// Starts the announcer of the workload's listeners, which announces them to `gateway`, as + /// [`Announcer::start`] does. + fn announce(&mut self, gateway: Ipv4Addr) -> Result; + + /// Stops `announcer`, as [`Announcer::stop`] does, ignoring its failure. + fn stop(&mut self, announcer: Self::Announcer); + /// Binds the runtime's files of `view` into `root`, for every execution, in the caller's /// mount namespace, as [`Entry::bind_runtime`] does with the view's [entry](View::entry). fn bind(&mut self, root: &Self::Root, view: &View) -> Result; @@ -597,6 +669,7 @@ impl System for LiveSystem { type Root = Root; type Export = SourceRoot; type Binds = RuntimeBinds; + type Announcer = Announcer; fn reread( &mut self, @@ -640,6 +713,14 @@ impl System for LiveSystem { write_resolver_file(path, server) } + fn announce(&mut self, gateway: Ipv4Addr) -> Result { + Announcer::start(gateway) + } + + fn stop(&mut self, announcer: Announcer) { + let _ = announcer.stop(); + } + fn bind(&mut self, root: &Root, view: &View) -> Result { view.entry(root.path()).bind_runtime() } @@ -688,10 +769,10 @@ impl View { /// What an application activated through a system, which an [`Applied`] holds in a VM: the /// root, the root of the host export, if any, the runtime's binds on the root, the identity in /// which every execution runs, if any, the workload's hostname and machine ID, the resolver file, -/// if any, and the proxy's endpoint, if any. +/// if any, the proxy's endpoint, if any, and the announcer of the workload's listeners, if any. #[derive(Debug)] #[cfg_attr(test, derive(PartialEq, Eq))] -struct Activated { +struct Activated { /// The assembled root. root: R, /// The root of the host export, if the record has an export. @@ -708,10 +789,17 @@ struct Activated { resolver: Option, /// The endpoint of the proxy, if the record's network names one. proxy: Option, + /// The announcer of the workload's listeners, if the record's network publishes them. + announcer: Option, } /// What an application through a system of type `S` activated. -type ActivatedBy = Activated<::Root, ::Export, ::Binds>; +type ActivatedBy = Activated< + ::Root, + ::Export, + ::Binds, + ::Announcer, +>; /// Applies `record` with `template` by `deadline`, if any, through `system`, as /// [`apply`](fn@apply) does, and returns what it activated. @@ -778,6 +866,22 @@ fn apply_with( } None => None, }; + let publishes = network.is_some_and(|network| network.publication == Publication::Published); + let announcer = if publishes { + let started = match template.gateway { + Some(gateway) => system.announce(gateway).map_err(NetworkError::Announcer), + None => Err(NetworkError::Gateway), + }; + match started { + Ok(announcer) => Some(announcer), + Err(error) => { + let teardown = discard(system, export, root); + return Err(ApplyError::Network { error, teardown }); + } + } + } else { + None + }; let view = View { identity: identity.clone().unwrap_or_else(|| Identity::new(0, 0)), hostname: workload.hostname.clone(), @@ -787,6 +891,9 @@ fn apply_with( let binds = match system.bind(&root, &view) { Ok(binds) => binds, Err(error) => { + if let Some(announcer) = announcer { + system.stop(announcer); + } let teardown = discard(system, export, root); return Err(if is_resolver(&error) { let error = NetworkError::Bind(error); @@ -805,6 +912,7 @@ fn apply_with( machine_id: view.machine_id, resolver: view.resolver, proxy: network.and_then(|network| network.proxy.clone()), + announcer, }) } @@ -832,15 +940,19 @@ fn is_resolver(error: &entry::Error) -> bool { } } -/// Closes `export`, the root of the host export, if any, removes `binds`, then tears `root` down, -/// through `system`, as [`Applied::teardown`] does, attempting the removal and the teardown -/// whatever failed before. +/// Stops `announcer`, if any, closes `export`, the root of the host export, if any, removes +/// `binds`, then tears `root` down, through `system`, as [`Applied::teardown`] does, attempting the +/// removal and the teardown whatever failed before. fn teardown_with( system: &mut S, + announcer: Option, export: Option, binds: S::Binds, root: S::Root, ) -> Result<(), AppliedTeardownError> { + if let Some(announcer) = announcer { + system.stop(announcer); + } if let Some(export) = export { system.close(export); } diff --git a/nvx-rs/src/guest/clone/activation/apply/tests.rs b/nvx-rs/src/guest/clone/activation/apply/tests.rs index 8e0faed6b..494f8cc1c 100644 --- a/nvx-rs/src/guest/clone/activation/apply/tests.rs +++ b/nvx-rs/src/guest/clone/activation/apply/tests.rs @@ -1,8 +1,9 @@ //! Tests of the application of a clone's activation record, through a fake system, which records //! every call of the application, and whose assembly returns a temporary directory that stands //! for the root, the image's view through the overlay, whose account databases the application -//! reads and checks for real, and which generates no machine ID, writes no resolver file, and -//! mounts, binds, and unmounts nothing; and, with the clone's activation, through an adapter of +//! reads and checks for real, and which generates no machine ID, writes no resolver file, starts +//! no announcer, and mounts, binds, and unmounts nothing; and, with the clone's activation, +//! through an adapter of //! the tests' own, which takes what the host sends, applies the record of the Activate that the //! activation accepted through the fake system, and spawns an execution for each request that //! the activation serves. @@ -75,6 +76,8 @@ const HOSTNAME: &[u8] = b"clone"; const DNS: IpAddr = IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1)); /// The endpoint of the proxy of the tests' networks. const PROXY: &str = "http://10.0.0.1:3128"; +/// The IPv4 gateway of the tests' template's NIC. +const GATEWAY: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1); /// Returns whether a failure's cause is the one that a test expects. type Expected = fn(&ActivationCause) -> bool; @@ -105,6 +108,10 @@ enum Effect { /// The application wrote a resolver file that names this DNS server to a new file at this /// path. Resolver(PathBuf, IpAddr), + /// The application started the announcer of the workload's listeners, to this gateway. + Announce(Ipv4Addr), + /// The announcer of the workload's listeners was stopped. + Stop, /// The application bound the runtime's files of this view into the root, for every /// execution. Bind(View), @@ -135,6 +142,9 @@ struct FakeSystem { unmountable: bool, /// The kind of the error with which each write of a resolver file fails, if it does. unwritable: Option, + /// The error with which each start of the announcer of the workload's listeners fails, if it + /// does. + unannounced: Option StartError>, /// The error with which each bind of the runtime's files fails, if it does, as when the /// image's `/etc/machine-id` or `/etc/resolv.conf` is a directory. unbindable: Option entry::Error>, @@ -161,6 +171,7 @@ impl FakeSystem { generating: None, unmountable: false, unwritable: None, + unannounced: None, unbindable: None, stuck: false, effects: Vec::new(), @@ -219,6 +230,15 @@ impl FakeSystem { } } + /// Returns a system whose image holds the tests' account databases, and each of whose starts + /// of the announcer of the workload's listeners fails with the error that `error` returns. + fn unannounced(error: fn() -> StartError) -> Self { + Self { + unannounced: Some(error), + ..Self::new() + } + } + /// Returns a system whose image holds the tests' account databases, and each of whose binds /// of the runtime's files fails with the error that `error` returns, once the entry checked /// its inputs. @@ -254,6 +274,7 @@ impl System for FakeSystem { type Root = PathBuf; type Export = (); type Binds = (); + type Announcer = Ipv4Addr; fn reread( &mut self, @@ -339,6 +360,19 @@ impl System for FakeSystem { } } + fn announce(&mut self, gateway: Ipv4Addr) -> Result { + self.effects.push(Effect::Announce(gateway)); + match self.unannounced { + Some(error) => Err(error()), + None => Ok(gateway), + } + } + + fn stop(&mut self, announcer: Ipv4Addr) { + assert_eq!(announcer, GATEWAY); + self.effects.push(Effect::Stop); + } + fn bind(&mut self, root: &PathBuf, view: &View) -> Result<(), entry::Error> { assert_eq!(*root, self.root()); self.effects.push(Effect::Bind(view.clone())); @@ -406,8 +440,9 @@ struct Adapter { system: FakeSystem, /// What the application activated: the root, the export's root, if any, the runtime's binds /// on the root, the identity in which every execution runs, if any, the workload's hostname - /// and machine ID, the resolver file, if any, and the proxy's endpoint, if any. - applied: Option>, + /// and machine ID, the resolver file, if any, the proxy's endpoint, if any, and the gateway of + /// the announcer of the workload's listeners, if any. + applied: Option>, /// What the adapter's latest Ready reported. ready: Option, } @@ -490,7 +525,8 @@ impl Adapter { } /// Returns the tests' template: its runtime directory, its devices at NVX's MMIO windows, with -/// the sizes that they must have, its timeout, and its host filesystem device's tag. +/// the sizes that they must have, its timeout, its host filesystem device's tag, and its NIC's +/// gateway. fn template() -> RootTemplate { RootTemplate { runtime: PathBuf::from(RUNTIME), @@ -504,6 +540,7 @@ fn template() -> RootTemplate { }, timeout: DEVICE_TIMEOUT, filesystem_tag: TAG.to_owned(), + gateway: Some(GATEWAY), } } @@ -645,8 +682,17 @@ fn network(dns: Option, proxy: Option<&str>) -> Network { } } +/// Returns a network as [`network`] does, but without published listeners. +fn unpublished(dns: Option, proxy: Option<&str>) -> Network { + Network { + publication: Publication::Unpublished, + ..network(dns, proxy) + } +} + /// Returns the record of the tests' Activate, as [`with_generated_id`] returns it, but with an -/// aggregate export, of three mappings, and a network that names [`DNS`] and [`PROXY`]. +/// aggregate export, of three mappings, and a network that names [`DNS`] and [`PROXY`], and +/// publishes the workload's listeners. fn with_everything() -> ActivationRecord { let mut configuration = with_generated_id().configuration().clone(); configuration.export = Some(export(ExportKind::Aggregate)); @@ -698,7 +744,7 @@ fn activated( system: &FakeSystem, identity: Option, machine_id: MachineId, -) -> Activated { +) -> Activated { Activated { root: system.root(), export: None, @@ -708,6 +754,7 @@ fn activated( machine_id, resolver: None, proxy: None, + announcer: None, } } @@ -720,6 +767,7 @@ fn applied_everything() -> Vec { Effect::Generate(machine_id_path()), Effect::MountExport(TAG.to_owned(), Access::ReadWrite), Effect::Resolver(resolver_path(), DNS), + Effect::Announce(GATEWAY), Effect::Bind(full_view()), ] } @@ -735,10 +783,11 @@ fn apply_error(failure: &ActivationFailure) -> &ApplyError { #[test] fn no_mount_spawn_or_id_generation_happens_before_the_gate() { // The application reaches the system only through its calls, which the fake records in - // full: none starts a process, and the adapter spawns each execution itself, once the - // activation serves its request. Every record asks for a generated machine ID, has an - // export, and names a DNS server, which only the application of an accepted record - // generates, mounts, and writes. + // full: none starts a process but the announcer of the workload's listeners, and the adapter + // spawns each execution itself, once the activation serves its request. Every record asks + // for a generated machine ID, has an export, names a DNS server, and publishes the workload's + // listeners, which only the application of an accepted record generates, mounts, writes, and + // announces. let base = Instant::now(); let record = with_everything().encode(); let settings = with_everything().configuration().clone(); @@ -828,8 +877,8 @@ fn no_mount_spawn_or_id_generation_happens_before_the_gate() { adapter.assert_untouched(&failure, |cause| matches!(cause, ActivationCause::Expired)); // Only a valid Activate mounts the root, reads its account databases, generates the machine - // ID, mounts the export, writes the resolver file, and binds the runtime's files, before - // Ready, and an execution spawns only once Ready opened the fence. + // ID, mounts the export, writes the resolver file, starts the announcer, and binds the + // runtime's files, before Ready, and an execution spawns only once Ready opened the fence. let mut adapter = Adapter::restricted(FakeSystem::new(), base); adapter.take(base, ActivationInput::Flow).unwrap(); assert!(adapter.system.effects.is_empty()); @@ -977,8 +1026,7 @@ fn assembly_and_identity_inputs_are_taken_exactly_from_the_record() { // The record's other settings, which the application leaves alone, change no input: its // bindings, its export's shape and plan, the number of its mappings, which the adapter's - // order takes, its network's policy and the publication of its listeners, its resources, - // and its restore section. + // order takes, its network's policy, its resources, and its restore section. let mut other = with_everything().configuration().clone(); other.image = activation::Image::Erofs { binding: activation::Identity::from_bytes([0x66; activation::Identity::LEN]), @@ -991,7 +1039,6 @@ fn assembly_and_identity_inputs_are_taken_exactly_from_the_record() { other.mappings = 7; other.network = Some(Network { policy: activation::Identity::from_bytes([0xaa; activation::Identity::LEN]), - publication: Publication::Unpublished, ..network(Some(DNS), Some(PROXY)) }); other.resources = ExpectedResources { @@ -1331,24 +1378,37 @@ fn a_machine_id_or_a_view_that_fails_fails_the_activation_once_the_root_is_torn_ #[test] fn the_runtime_s_binds_are_removed_before_the_root_is_torn_down() { // The binds keep the root's mount busy, so they go first, and the export's root, which keeps - // the export's mount busy, before them, unless the adapter took it. + // the export's mount busy, before them, unless the adapter took it, and the announcer before + // anything. let mut system = FakeSystem::new(); let root = system.root(); - teardown_with(&mut system, None, (), root).unwrap(); + teardown_with(&mut system, None, None, (), root).unwrap(); assert_eq!(system.effects, [Effect::Remove, Effect::Teardown]); let mut system = FakeSystem::new(); let root = system.root(); - teardown_with(&mut system, Some(()), (), root).unwrap(); + teardown_with(&mut system, None, Some(()), (), root).unwrap(); assert_eq!( system.effects, [Effect::Close, Effect::Remove, Effect::Teardown] ); + let mut system = FakeSystem::new(); + let root = system.root(); + teardown_with(&mut system, Some(GATEWAY), Some(()), (), root).unwrap(); + assert_eq!( + system.effects, + [ + Effect::Stop, + Effect::Close, + Effect::Remove, + Effect::Teardown + ] + ); // A removal that fails leaves those binds mounted, and the teardown is attempted all the // same; the error reports both. let mut system = FakeSystem::stuck(); let root = system.root(); - let error = teardown_with(&mut system, Some(()), (), root).unwrap_err(); + let error = teardown_with(&mut system, None, Some(()), (), root).unwrap_err(); assert_eq!( system.effects, [Effect::Close, Effect::Remove, Effect::Teardown] @@ -1425,7 +1485,7 @@ fn a_network_s_dns_server_is_written_to_a_resolver_file_that_every_execution_see let ipv6 = IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0xa00, 1)); for dns in [DNS, ipv6] { let mut configuration = accepted().configuration().clone(); - configuration.network = Some(network(Some(dns), Some(PROXY))); + configuration.network = Some(unpublished(Some(dns), Some(PROXY))); let record = clone_record(configuration); let mut system = FakeSystem::new(); let applied = apply_with(&mut system, &record, &template(), None).unwrap(); @@ -1455,7 +1515,7 @@ fn a_network_s_dns_server_is_written_to_a_resolver_file_that_every_execution_see // nothing. for proxy in [None, Some(PROXY)] { let mut configuration = accepted().configuration().clone(); - configuration.network = Some(network(None, proxy)); + configuration.network = Some(unpublished(None, proxy)); let record = clone_record(configuration); let mut system = FakeSystem::new(); let applied = apply_with(&mut system, &record, &template(), None).unwrap(); @@ -1513,36 +1573,43 @@ fn a_resolver_file_names_its_dns_server_once_and_every_execution_may_read_it() { #[test] fn an_export_or_a_network_that_fails_fails_the_activation_once_the_root_is_torn_down() { - // The export's mount fails with the export's class, and the resolver file's write and bind - // with the network's, while the machine ID's bind fails with the view's, each once the - // application closed the export's root, if it mounted the export, and tore the root down, - // which unmounts the export. + // The export's mount fails with the export's class, and the resolver file's write and bind, + // and the announcer's start, with the network's, while the machine ID's bind fails with the + // view's, each once the application stopped the announcer, if it started it, closed the + // export's root, if it mounted the export, and tore the root down, which unmounts the export. let base = Instant::now(); let bytes = with_everything().encode(); - let failed = |done: usize, closed: bool| { + let failed = |done: usize, stopped: bool, closed: bool| { let mut effects = applied_everything()[..done].to_vec(); + if stopped { + effects.push(Effect::Stop); + } if closed { effects.push(Effect::Close); } effects.push(Effect::Teardown); effects }; - let cases: [(FakeSystem, Vec, Failed); 5] = [ - (FakeSystem::unmountable(), failed(4, false), |error| { - matches!( - error, - ApplyError::Export { - error: root::Error::Mount { - mount: root::Mount::Export { tag }, - .. - }, - teardown: None, - } if tag == TAG - ) - }), + let cases: [(FakeSystem, Vec, Failed); 6] = [ + ( + FakeSystem::unmountable(), + failed(4, false, false), + |error| { + matches!( + error, + ApplyError::Export { + error: root::Error::Mount { + mount: root::Mount::Export { tag }, + .. + }, + teardown: None, + } if tag == TAG + ) + }, + ), ( FakeSystem::unwritable(io::ErrorKind::AlreadyExists), - failed(5, true), + failed(5, false, true), |error| { matches!( error, @@ -1553,9 +1620,22 @@ fn an_export_or_a_network_that_fails_fails_the_activation_once_the_root_is_torn_ ) }, ), + ( + FakeSystem::unannounced(|| StartError::NoPrivilegedPort), + failed(6, false, true), + |error| { + matches!( + error, + ApplyError::Network { + error: NetworkError::Announcer(StartError::NoPrivilegedPort), + teardown: None, + } + ) + }, + ), ( FakeSystem::unbindable(|| entry::Error::Resolver(MappingError::TargetKind)), - failed(6, true), + failed(7, true, true), |error| { matches!( error, @@ -1572,7 +1652,7 @@ fn an_export_or_a_network_that_fails_fails_the_activation_once_the_root_is_torn_ error: Box::new(entry::Error::Resolver(MappingError::TargetKind)), teardown: mappings::TeardownError::of("/etc/machine-id", MappingError::MountPoint), }), - failed(6, true), + failed(7, true, true), |error| { matches!( error, @@ -1585,7 +1665,7 @@ fn an_export_or_a_network_that_fails_fails_the_activation_once_the_root_is_torn_ ), ( FakeSystem::unbindable(|| entry::Error::MachineId(MappingError::TargetKind)), - failed(6, true), + failed(7, true, true), |error| { matches!( error, @@ -1612,6 +1692,73 @@ fn an_export_or_a_network_that_fails_fails_the_activation_once_the_root_is_torn_ } } +#[test] +fn a_network_that_publishes_the_workload_s_listeners_starts_their_announcer_to_its_gateway() { + // A network that publishes the workload's listeners has the application start their + // announcer, to the template's gateway, once it wrote the resolver file, and before it binds + // the runtime's files, and hand it to the adapter, which watches it; a network that publishes + // none, and a clone without a network, start none. + let mut configuration = accepted().configuration().clone(); + configuration.network = Some(network(None, Some(PROXY))); + let published = clone_record(configuration.clone()); + let mut system = FakeSystem::new(); + let applied = apply_with(&mut system, &published, &template(), None).unwrap(); + let bound = Effect::Bind(view(Some(workload()), MachineId::Image)); + let assembled = [ + Effect::Assemble(accepted_assembly()), + Effect::Account(workload()), + ]; + assert_eq!( + system.effects, + [&assembled[..], &[Effect::Announce(GATEWAY), bound.clone()]].concat() + ); + assert_eq!( + applied, + Activated { + proxy: Some(PROXY.to_owned()), + announcer: Some(GATEWAY), + ..activated(&system, Some(workload()), MachineId::Image) + } + ); + configuration.network = Some(unpublished(None, Some(PROXY))); + for record in [clone_record(configuration), accepted()] { + let mut system = FakeSystem::new(); + let applied = apply_with(&mut system, &record, &template(), None).unwrap(); + assert_eq!( + system.effects, + [&assembled[..], std::slice::from_ref(&bound)].concat() + ); + assert_eq!(applied.announcer, None); + } + + // A template without a gateway, as one without a NIC, cannot announce: the record that + // publishes listeners fails with the network's class, once the root was torn down, and one + // that publishes none applies. + let gatewayless = RootTemplate { + gateway: None, + ..template() + }; + let mut system = FakeSystem::new(); + let error = apply_with(&mut system, &published, &gatewayless, None).unwrap_err(); + assert!( + matches!( + error, + ApplyError::Network { + error: NetworkError::Gateway, + teardown: None, + } + ), + "{error}" + ); + assert_eq!( + system.effects, + [&assembled[..], &[Effect::Teardown]].concat() + ); + let mut system = FakeSystem::new(); + let applied = apply_with(&mut system, &accepted(), &gatewayless, None).unwrap(); + assert_eq!(applied.announcer, None); +} + /// A tmpfs that stands for a clone's root, mounted at `root` below a temporary directory, in a /// private mount namespace of the calling thread, beside a directory that stands for the /// template's runtime directory, `runtime`. @@ -1914,6 +2061,30 @@ fn application_errors_name_their_cause() { source.source().unwrap().to_string(), MappingError::TargetKind.to_string() ); + let error = ApplyError::Network { + error: NetworkError::Gateway, + teardown: None, + }; + assert_eq!( + error.to_string(), + "the network settings cannot be applied: the network publishes the workload's listeners, \ + but the template names no IPv4 gateway to announce them to" + ); + assert!(error.source().unwrap().source().is_none()); + let denied = io::Error::from(Errno::EACCES); + let error = ApplyError::Network { + error: NetworkError::Announcer(StartError::Bind(Errno::EACCES.into())), + teardown: None, + }; + assert_eq!( + error.to_string(), + format!( + "the network settings cannot be applied: the announcer's socket cannot be bound to a \ + privileged source port: {denied}" + ) + ); + let source = error.source().unwrap(); + assert_eq!(source.source().unwrap().to_string(), denied.to_string()); // The adapter hands the error to the activation, whose failure names it. let base = Instant::now(); diff --git a/nvx-rs/src/guest/ingress/announcer.rs b/nvx-rs/src/guest/ingress/announcer.rs new file mode 100644 index 000000000..e8353c1e3 --- /dev/null +++ b/nvx-rs/src/guest/ingress/announcer.rs @@ -0,0 +1,329 @@ +//! The announcer of a clone's listeners, which the clone's application starts, as the +//! [module documentation](super) describes. + +use std::ffi::{CStr, CString}; +use std::fmt; +use std::io; +use std::net::Ipv4Addr; +use std::os::fd::BorrowedFd; +use std::process::ExitStatus; + +use super::bind_privileged; +use crate::guest::process::Child; +use crate::guest::spawn::{Command, LaunchFailure}; +use crate::guest::sys::{self, Errno, ProcessId}; + +/// The path of the announcer's program in NVX's guests, which NVX's init runs at a cold boot, and +/// [`Announcer::start`] in a clone. +pub const PROGRAM: &CStr = c"/sbin/nvx-ingress"; + +/// The announcer of the guest's listeners, `nvx-ingress inherit GATEWAY_IPV4`, which +/// [`start`](Self::start) started as a child of the caller, whose status it collects for the +/// caller. +/// +/// The announcer runs until it fails, or until [`stop`](Self::stop) ends it: its +/// [pidfd](Self::pidfd) polls readable once it ended, and [`try_wait`](Self::try_wait) then +/// collects its status. Dropping it ends it as `stop` does, ignoring failures. +#[derive(Debug)] +pub struct Announcer { + /// The announcer's process, which a pidfd names. + child: Child, +} + +impl Announcer { + /// Starts the announcer of the guest's listeners, which announces them to `gateway`, the + /// IPv4 gateway of the guest's NIC, as [`PROGRAM`] does at a cold boot: + /// + /// 1. it creates an IPv4 UDP socket, and binds it to the highest free privileged source + /// port, from 1023 down to 512, as the program binds its own at a cold boot, which needs + /// the guest's root user, or `CAP_NET_BIND_SERVICE`: the gateway takes announcements only + /// from such a port, which no workload without that privilege can bind; + /// 2. it spawns [`PROGRAM`] as `nvx-ingress inherit GATEWAY_IPV4`, with the socket as its + /// standard input, the caller's standard output and error, every signal's default + /// disposition, and none blocked, in a session of its own, without a controlling terminal, + /// and waits until the program runs, as its launch report tells, but no longer: nothing + /// waits for the program's first scan, nor for any announcement. + /// + /// Fails with a [`StartError`] if the socket cannot be created or bound, or the program does + /// not start, once it collected the status of a child that did not run the program. + pub fn start(gateway: Ipv4Addr) -> Result { + let socket = sys::UdpSocket::ipv4().map_err(|errno| StartError::Socket(errno.into()))?; + match bind_privileged(|port| socket.bind_any(port)) { + Ok(Some(_)) => Self::spawn(PROGRAM, gateway, socket), + Ok(None) => Err(StartError::NoPrivilegedPort), + Err(errno) => Err(StartError::Bind(errno.into())), + } + } + + /// Spawns `program` as `nvx-ingress inherit GATEWAY_IPV4`, with `socket` as its standard + /// input, as [`start`](Self::start) does once it bound the socket. + fn spawn( + program: &CStr, + gateway: Ipv4Addr, + socket: sys::UdpSocket, + ) -> Result { + let arguments = vec![ + c"nvx-ingress".to_owned(), + c"inherit".to_owned(), + CString::new(gateway.to_string()).expect("an address has no NUL"), + ]; + let spawned = Command::new(program.to_owned(), arguments) + .stdin(socket) + .new_session() + .reset_signals() + .spawn() + .map_err(StartError::Spawn)?; + let id = spawned.id(); + // Nothing but this start collects the status of the child, which keeps its process ID + // until then. + let launched = spawned.into_report().wait(); + let started = match launched { + Ok(None) => Child::with_pidfd(id).map_err(StartError::Spawn), + Ok(Some(failure)) => Err(StartError::Launch(failure)), + Err(error) => Err(StartError::Spawn(error)), + }; + match started { + Ok(child) => Ok(Self { child }), + Err(error) => { + if let Some(process) = ProcessId::from_positive(i64::from(id)) { + // A child whose launch failed has exited already. + let _ = sys::kill_process(process); + } + if let Ok(mut child) = Child::new(id) { + let _ = child.wait(); + } + Err(error) + } + } + } + + /// Returns the announcer's process ID. + pub fn id(&self) -> u32 { + self.child.id() + } + + /// Lends the announcer's pidfd, which polls readable once the announcer ended. + pub fn pidfd(&self) -> BorrowedFd<'_> { + self.child + .pidfd() + .expect("an announcer is named by its pidfd") + } + + /// Collects the announcer's status if it ended, without blocking, and returns it, or `None` + /// while it runs. Once it has a status, this returns it again. + pub fn try_wait(&mut self) -> io::Result> { + self.child.try_wait() + } + + /// Ends the announcer with `SIGKILL`, unless it ended already, and returns its status, which + /// it collects. + pub fn stop(mut self) -> io::Result { + self.end() + } + + /// Ends the announcer, as [`stop`](Self::stop) does. + fn end(&mut self) -> io::Result { + if self.child.status().is_none() { + match sys::kill_through_pidfd(self.pidfd(), false) { + // An announcer that ended, but whose status nothing collected yet, takes the + // signal as a zombie does. + Ok(()) | Err(Errno::ESRCH) => {} + Err(errno) => return Err(errno.into()), + } + } + self.child.wait() + } +} + +impl Drop for Announcer { + fn drop(&mut self) { + let _ = self.end(); + } +} + +/// Why [`Announcer::start`] failed. +#[derive(Debug)] +#[non_exhaustive] +pub enum StartError { + /// The announcer's UDP socket could not be created, as this error tells. + Socket(io::Error), + /// The announcer's socket could not be bound to a privileged source port, for another reason + /// than the port's use, as this error tells: as without the privilege to bind one. + Bind(io::Error), + /// Every privileged source port, from 1023 down to 512, is in use. + NoPrivilegedPort, + /// The announcer could not be spawned, or its launch could not be followed, as this error + /// tells. + Spawn(io::Error), + /// The announcer's launch failed at a stage, as when its program does not exist, which its + /// exec stage then tells. + Launch(LaunchFailure), +} + +impl fmt::Display for StartError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + let program = PROGRAM.to_string_lossy(); + match self { + Self::Socket(error) => { + write!( + formatter, + "the announcer's socket cannot be created: {error}" + ) + } + Self::Bind(error) => write!( + formatter, + "the announcer's socket cannot be bound to a privileged source port: {error}" + ), + Self::NoPrivilegedPort => formatter.write_str( + "no privileged source port, from 1023 down to 512, is free for the announcer's \ + socket", + ), + Self::Spawn(error) => write!(formatter, "{program} cannot be started: {error}"), + Self::Launch(failure) => write!(formatter, "{program} did not start: {failure}"), + } + } +} + +impl std::error::Error for StartError { + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { + match self { + Self::Socket(error) | Self::Bind(error) | Self::Spawn(error) => Some(error), + Self::Launch(failure) => Some(failure), + Self::NoPrivilegedPort => None, + } + } +} + +#[cfg(test)] +mod tests { + use std::fs; + use std::os::unix::ffi::OsStrExt; + use std::os::unix::fs::PermissionsExt; + use std::os::unix::process::ExitStatusExt; + use std::path::Path; + use std::time::{Duration, Instant}; + + use super::*; + use crate::guest::spawn::LaunchStage; + + /// How long a test waits for an announcer to do what it expects. + const PATIENCE: Duration = Duration::from_secs(10); + + /// The gateway of the tests' announcers. + const GATEWAY: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1); + + /// Writes an executable shell script at `path` whose body is `script`, which stands for the + /// announcer's program, and returns its path. + fn program(directory: &Path, name: &str, script: &str) -> CString { + let path = directory.join(name); + fs::write(&path, format!("#!/bin/sh\n{script}\n")).unwrap(); + fs::set_permissions(&path, fs::Permissions::from_mode(0o755)).unwrap(); + CString::new(path.as_os_str().as_bytes()).unwrap() + } + + /// Returns a socket that stands for the announcer's, bound to a port that needs no privilege. + fn socket() -> sys::UdpSocket { + let socket = sys::UdpSocket::ipv4().unwrap(); + socket.bind_any(0).unwrap(); + socket + } + + /// Returns whether `announcer`'s pidfd polls readable within `milliseconds`. + fn ended(announcer: &Announcer, milliseconds: u16) -> bool { + sys::readable_or_hung_up(&[announcer.pidfd()], milliseconds).unwrap() == [true] + } + + #[test] + fn an_announcer_runs_its_program_with_its_socket_until_it_is_stopped() { + let directory = tempfile::tempdir().unwrap(); + let report = directory.path().join("report"); + let script = format!( + "{{ printf '%s\\n' \"$0\" \"$@\"; stat -L -c %F /dev/stdin; }} >{0}.tmp\n\ + mv {0}.tmp {0}\nexec sleep 60", + report.display() + ); + let program = program(directory.path(), "announcer", &script); + let announcer = Announcer::spawn(&program, GATEWAY, socket()).unwrap(); + let deadline = Instant::now() + PATIENCE; + while !report.exists() { + assert!(Instant::now() < deadline, "the announcer reported nothing"); + std::thread::sleep(Duration::from_millis(10)); + } + // The program takes the gateway after `inherit`, and the socket as its standard input. + let reported = fs::read_to_string(&report).unwrap(); + let lines: Vec<_> = reported.lines().collect(); + assert_eq!(lines[1..], ["inherit", "10.0.0.1", "socket"], "{reported}"); + assert!(!ended(&announcer, 0)); + let status = announcer.stop().unwrap(); + assert_eq!(status.signal(), Some(nix::libc::SIGKILL)); + } + + #[test] + fn an_announcer_that_ends_polls_readable_and_keeps_its_status() { + let directory = tempfile::tempdir().unwrap(); + let program = program(directory.path(), "announcer", "exit 3"); + let mut announcer = Announcer::spawn(&program, GATEWAY, socket()).unwrap(); + assert!(ended(&announcer, 10_000)); + let status = announcer.try_wait().unwrap().unwrap(); + assert_eq!(status.code(), Some(3)); + assert_eq!(announcer.try_wait().unwrap(), Some(status)); + // Its stop signals nothing, and returns the status again. + assert_eq!(announcer.stop().unwrap(), status); + } + + #[test] + fn a_dropped_announcer_ends() { + let directory = tempfile::tempdir().unwrap(); + let program = program(directory.path(), "announcer", "exec sleep 60"); + let announcer = Announcer::spawn(&program, GATEWAY, socket()).unwrap(); + let process = ProcessId::from_positive(i64::from(announcer.id())).unwrap(); + drop(announcer); + // The drop ended it and collected its status, so its process ID names no process. + assert!(!sys::process_exists(process)); + } + + #[test] + fn an_announcer_whose_program_does_not_run_does_not_start() { + let directory = tempfile::tempdir().unwrap(); + let missing = + CString::new(directory.path().join("missing").as_os_str().as_bytes()).unwrap(); + let error = Announcer::spawn(&missing, GATEWAY, socket()).unwrap_err(); + let StartError::Launch(failure) = &error else { + panic!("{error}"); + }; + assert_eq!(failure.stage(), LaunchStage::Exec); + assert_eq!(failure.raw_os_error(), nix::libc::ENOENT); + assert!( + error + .to_string() + .starts_with("/sbin/nvx-ingress did not start: the exec launch stage failed: "), + "{error}" + ); + } + + #[test] + fn start_errors_name_their_cause() { + let denied = io::Error::from(Errno::EACCES); + for (error, expected) in [ + ( + StartError::Socket(io::Error::from(Errno::EMFILE)), + "the announcer's socket cannot be created: ", + ), + ( + StartError::Bind(denied), + "the announcer's socket cannot be bound to a privileged source port: ", + ), + ( + StartError::NoPrivilegedPort, + "no privileged source port, from 1023 down to 512, is free for the announcer's \ + socket", + ), + ( + StartError::Spawn(io::Error::from(Errno::EAGAIN)), + "/sbin/nvx-ingress cannot be started: ", + ), + ] { + assert!(error.to_string().starts_with(expected), "{error}"); + } + } +} diff --git a/nvx-rs/src/guest/ingress/mod.rs b/nvx-rs/src/guest/ingress/mod.rs index 5dbe5d6d9..31e3497d2 100644 --- a/nvx-rs/src/guest/ingress/mod.rs +++ b/nvx-rs/src/guest/ingress/mod.rs @@ -4,6 +4,19 @@ //! it, as root, when the kernel command line carries `virtnet_ingress=allow` //! or `virtnet_ingress=host-loopback`. //! +//! A clone of a template restores the template's NIC, which the template +//! captured under a policy that publishes nothing, so its init started no +//! announcer, with a policy of its own, which may publish the clone's +//! listeners. The clone's application then starts the announcer itself, with +//! [`Announcer::start`]: as root, it binds the announcer's UDP socket to a +//! privileged source port, and runs `nvx-ingress inherit GATEWAY_IPV4`, which +//! announces from that socket, its standard input, rather than from one of its +//! own. A failure to bind the socket, or to run the program, is the start's +//! own, before the clone is ready, and the caller watches the announcer that +//! started, and must end the clone once it ended: the gateway then keeps +//! publishing the listeners that it last announced, and learns of no later +//! change of them. +//! //! Every 100 ms it reads the kernel's socket tables. A TCP socket in the //! LISTEN state is a listener, and so is an unconnected UDP socket that //! persists for a second, which skips short-lived client sockets, such as @@ -44,7 +57,10 @@ use super::parse; use super::status::{CommandError, run_main}; use super::sys::{self, Errno, FileDescriptor, Mode, OpenFlags}; +pub use announcer::{Announcer, PROGRAM, StartError}; + mod address; +mod announcer; mod collect; mod diag; mod encode; @@ -171,37 +187,57 @@ struct UdpSocket { inode: u64, } -/// Runs `nvx-ingress` with the process's arguments. Test mode exits with 0 after the last -/// snapshot, or with 1 when a snapshot fails. The announcer runs until it fails, and exits with 1 -/// when it cannot open or bind its socket, or with 2 for a usage error. +/// Runs `nvx-ingress` with the process's arguments: `GATEWAY_IPV4`, the announcer, from a socket +/// of its own; `inherit GATEWAY_IPV4`, the announcer, from the socket that it inherited as its +/// standard input, which must be an IPv4 datagram socket bound to a privileged source port from +/// 512 to 1023; or `test DIRECTORY`. Test mode exits with 0 after the last snapshot, or with 1 +/// when a snapshot fails. The announcer runs until it fails, and exits with 1 when it cannot open +/// or bind its socket, or take the one that it inherited, or with 2 for a usage error. pub fn main() -> ExitCode { run_main(|args| run(args, &mut sys::Stdout::new(), &mut io::stderr())) } /// Runs test mode or the announcer, as `args` select. fn run(args: &[CString], out: &mut dyn Write, err: &mut dyn Write) -> Result<(), CommandError> { - match args { + let announced = match args { [_, mode, directory] if mode.as_bytes() == b"test" => { - test_mode::run(directory, out).map_err(|_| CommandError::Operation) + return test_mode::run(directory, out).map_err(|_| CommandError::Operation); + } + [_, mode, gateway] if mode.as_bytes() == b"inherit" => sender::run_inherited(gateway), + [_, gateway] => sender::run(gateway), + _ => return Err(usage(err)), + }; + match announced { + Ok(()) => Ok(()), + Err(sender::Error::InvalidGateway) => Err(usage(err)), + Err(error) => { + error.report(err); + Err(CommandError::Operation) } - [_, gateway] => match sender::run(gateway) { - Ok(()) => Ok(()), - Err(sender::Error::InvalidGateway) => Err(usage(err)), - Err(error) => { - error.report(err); - Err(CommandError::Operation) - } - }, - _ => Err(usage(err)), } } /// Prints the usage message and returns the usage error. fn usage(err: &mut dyn Write) -> CommandError { - sys::report(err, &[b"usage: nvx-ingress GATEWAY_IPV4\n"]); + sys::report(err, &[b"usage: nvx-ingress [inherit] GATEWAY_IPV4\n"]); CommandError::Usage } +/// Binds a socket, through `bind`, to the highest free privileged source port, from +/// [`FIRST_SOURCE_PORT`] down to [`LAST_SOURCE_PORT`], and returns it, or `None` if every one of +/// them is in use. Only a port in use moves the search to the next one: any other failure of +/// `bind` is the search's. +fn bind_privileged(mut bind: impl FnMut(u16) -> Result<(), Errno>) -> Result, Errno> { + for port in (LAST_SOURCE_PORT..=FIRST_SOURCE_PORT).rev() { + match bind(port) { + Ok(()) => return Ok(Some(port)), + Err(Errno::EADDRINUSE) => {} + Err(errno) => return Err(errno), + } + } + Ok(None) +} + /// Returns `root` followed by `suffix`, a path that starts with `/`, if the C program's /// `snprintf(path, size, "%s%s", root, suffix)` fit a buffer of `size` bytes with its NUL. /// diff --git a/nvx-rs/src/guest/ingress/sender.rs b/nvx-rs/src/guest/ingress/sender.rs index 2b404bc2e..c183a8686 100644 --- a/nvx-rs/src/guest/ingress/sender.rs +++ b/nvx-rs/src/guest/ingress/sender.rs @@ -1,4 +1,5 @@ -//! Binds the privileged UDP source port and runs the live scan/send loop. +//! Binds the privileged UDP source port, or takes the socket that the announcer inherited bound to +//! one, and runs the live scan/send loop. use std::ffi::CStr; use std::io::Write; @@ -9,6 +10,7 @@ use std::time::Duration; use super::collect::{self, Collector, DiagnosticInput, IPV6_DUAL_STACK_BY_DEFAULT}; use super::{ ANNOUNCEMENT_PORT, FIRST_SOURCE_PORT, LAST_SOURCE_PORT, REFRESH_INTERVAL_MS, SCAN_INTERVAL_MS, + bind_privileged, }; use crate::guest::sys::{self, Errno}; @@ -26,6 +28,9 @@ trait System { fn open_socket(&mut self) -> Result<(), Errno>; /// Binds the owned socket to one candidate privileged port. fn bind(&mut self, port: u16) -> Result<(), Errno>; + /// Takes the socket that the process inherited as its standard input, and returns the port + /// to which it is bound. + fn inherit_socket(&mut self) -> Result; /// Reads the monotonic clock in milliseconds. fn now_ms(&mut self) -> u64; /// Collects the announcement for one scan. @@ -75,6 +80,13 @@ impl System for RealSystem { .bind_any(port) } + fn inherit_socket(&mut self) -> Result { + let socket = sys::UdpSocket::inherited(sys::take_inherited_descriptor(c"0")?)?; + let port = socket.port()?; + self.socket = Some(socket); + Ok(port) + } + /// Returns the C program's whole milliseconds of the monotonic clock, or 0 if the clock /// fails, which Linux's monotonic clock does not. fn now_ms(&mut self) -> u64 { @@ -118,6 +130,12 @@ pub(super) enum Error { /// Every privileged source port from `FIRST_SOURCE_PORT` down to `LAST_SOURCE_PORT` was /// already in use. NoPrivilegedPort, + /// The standard input that the announcer inherited is not an IPv4 datagram socket, or the + /// port to which it is bound cannot be read. + Inherited(Errno), + /// The socket that the announcer inherited is bound to a port outside `LAST_SOURCE_PORT` to + /// `FIRST_SOURCE_PORT`, or to none, 0, from which the gateway would take no announcement. + UnprivilegedPort(u16), } impl Error { @@ -132,16 +150,44 @@ impl Error { Self::NoPrivilegedPort => { sys::report(err, &[b"nvx-ingress: no privileged source port is free\n"]) } + Self::Inherited(errno) => report_errno(err, b"nvx-ingress: inherited socket: ", errno), + Self::UnprivilegedPort(port) => sys::report( + err, + &[ + b"nvx-ingress: the inherited socket's port ", + port.to_string().as_bytes(), + b" is not a privileged source port\n", + ], + ), } } } -/// Parses the gateway and runs the live sender forever. +/// Where the sender's socket comes from. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Source { + /// The sender opens its socket, and binds it to the highest free privileged source port. + Bound, + /// The sender takes the socket that it inherited as its standard input, which its starter + /// bound to a privileged source port. + Inherited, +} + +/// Parses the gateway and runs the live sender forever, from a socket of its own. pub(super) fn run(gateway: &CStr) -> Result<(), Error> { let Some(destination) = parse_gateway(gateway) else { return Err(Error::InvalidGateway); }; - run_with(&mut RealSystem::new(), destination, None) + run_with(&mut RealSystem::new(), destination, Source::Bound, None) +} + +/// Parses the gateway and runs the live sender forever, from the socket that it inherited as its +/// standard input, bound to a privileged source port. +pub(super) fn run_inherited(gateway: &CStr) -> Result<(), Error> { + let Some(destination) = parse_gateway(gateway) else { + return Err(Error::InvalidGateway); + }; + run_with(&mut RealSystem::new(), destination, Source::Inherited, None) } /// Parses `text` as the C program's `inet_pton(AF_INET, ...)` did: as four decimal octets of at @@ -151,16 +197,29 @@ fn parse_gateway(text: &CStr) -> Option { text.to_str().ok()?.parse().ok() } -/// Runs the sender against an injectable system, optionally stopping after a test scan count. +/// Runs the sender against an injectable system, with a socket from `source`, optionally stopping +/// after a test scan count. fn run_with( system: &mut impl System, destination: Ipv4Addr, + source: Source, scan_limit: Option, ) -> Result<(), Error> { system.ignore_signals(); let ipv6_dual_stack = system.ipv6_dual_stack(); - system.open_socket().map_err(Error::Socket)?; - let own_port = bind_privileged_port(system)?; + let own_port = match source { + Source::Bound => { + system.open_socket().map_err(Error::Socket)?; + bind_privileged_port(system)? + } + Source::Inherited => { + let port = system.inherit_socket().map_err(Error::Inherited)?; + if !(LAST_SOURCE_PORT..=FIRST_SOURCE_PORT).contains(&port) { + return Err(Error::UnprivilegedPort(port)); + } + port + } + }; let mut last_payload = Vec::new(); let mut last_sent = 0_u64; @@ -182,18 +241,12 @@ fn run_with( } } -/// Binds the system's socket to the highest free privileged source port, from -/// `FIRST_SOURCE_PORT` down to `LAST_SOURCE_PORT`, and returns it. Only a port in use moves the -/// search to the next one. +/// Binds the system's socket to the highest free privileged source port, as [`bind_privileged`] +/// does, and returns it. fn bind_privileged_port(system: &mut impl System) -> Result { - for port in (LAST_SOURCE_PORT..=FIRST_SOURCE_PORT).rev() { - match system.bind(port) { - Ok(()) => return Ok(port), - Err(Errno::EADDRINUSE) => {} - Err(errno) => return Err(Error::Bind(errno)), - } - } - Err(Error::NoPrivilegedPort) + bind_privileged(|port| system.bind(port)) + .map_err(Error::Bind)? + .ok_or(Error::NoPrivilegedPort) } /// Writes one diagnostic that describes `errno` with `nix`'s text. @@ -217,6 +270,10 @@ mod tests { ipv6_dual_stack: bool, /// Optional socket-creation failure. socket_error: Option, + /// The port of the inherited socket, or the failure to take it. + inherited: Result, + /// Whether the sender took its inherited socket. + took_inherited: bool, /// Scripted candidate-port bind outcomes. bind_results: VecDeque>, /// Bind outcome after the scripted queue is empty. @@ -244,6 +301,8 @@ mod tests { ignored_signals: false, ipv6_dual_stack: true, socket_error: None, + inherited: Err(Errno::ENOTSOCK), + took_inherited: false, bind_results: VecDeque::new(), default_bind: Ok(()), bound_ports: Vec::new(), @@ -275,6 +334,11 @@ mod tests { self.bind_results.pop_front().unwrap_or(self.default_bind) } + fn inherit_socket(&mut self) -> Result { + self.took_inherited = true; + self.inherited + } + fn now_ms(&mut self) -> u64 { self.times.pop_front().expect("a time for every test scan") } @@ -350,7 +414,7 @@ mod tests { system.bind_results = VecDeque::from([Err(Errno::EADDRINUSE), Err(Errno::EADDRINUSE), Ok(())]); assert_eq!( - run_with(&mut system, loopback_destination(), Some(1)), + run_with(&mut system, loopback_destination(), Source::Bound, Some(1)), Ok(()) ); assert!(system.ignored_signals); @@ -381,8 +445,8 @@ mod tests { system.bind_results.push_back(Err(error)); } let mut err = Vec::new(); - let failure = - run_with(&mut system, loopback_destination(), Some(1)).expect_err("must fail"); + let failure = run_with(&mut system, loopback_destination(), Source::Bound, Some(1)) + .expect_err("must fail"); failure.report(&mut err); let errno = socket_error.or(bind_error).unwrap(); let mut expected = prefix.to_vec(); @@ -399,8 +463,8 @@ mod tests { let mut system = FakeSystem::one_scan(); system.default_bind = Err(Errno::EADDRINUSE); let mut err = Vec::new(); - let failure = - run_with(&mut system, loopback_destination(), Some(1)).expect_err("must fail"); + let failure = run_with(&mut system, loopback_destination(), Source::Bound, Some(1)) + .expect_err("must fail"); failure.report(&mut err); assert_eq!(system.bound_ports.len(), 512); assert_eq!( @@ -410,6 +474,71 @@ mod tests { assert_eq!(err, b"nvx-ingress: no privileged source port is free\n"); } + /// Verifies that the sender takes the socket that it inherited, bound to a privileged source + /// port, rather than opening and binding one, and refuses one of another port or none. + #[test] + fn an_inherited_socket_is_taken_only_at_a_privileged_source_port() { + for port in [LAST_SOURCE_PORT, 700, FIRST_SOURCE_PORT] { + let mut system = FakeSystem::one_scan(); + system.socket_error = Some(Errno::EMFILE); + system.default_bind = Err(Errno::EACCES); + system.inherited = Ok(port); + assert_eq!( + run_with( + &mut system, + loopback_destination(), + Source::Inherited, + Some(1) + ), + Ok(()) + ); + assert!(system.took_inherited && system.ignored_signals); + assert!(system.bound_ports.is_empty()); + assert_eq!(system.collections, [(port, 100, true)]); + assert_eq!(system.sends, [b"packet".to_vec()]); + } + for (inherited, expected) in [ + ( + Ok(0), + b"nvx-ingress: the inherited socket's port 0 is not a privileged source port\n" + .to_vec(), + ), + ( + Ok(511), + b"nvx-ingress: the inherited socket's port 511 is not a privileged source port\n" + .to_vec(), + ), + ( + Ok(1024), + b"nvx-ingress: the inherited socket's port 1024 is not a privileged source port\n" + .to_vec(), + ), + ( + Err(Errno::ENOTSOCK), + [ + &b"nvx-ingress: inherited socket: "[..], + Errno::ENOTSOCK.desc().as_bytes(), + b"\n", + ] + .concat(), + ), + ] { + let mut system = FakeSystem::one_scan(); + system.inherited = inherited; + let failure = run_with( + &mut system, + loopback_destination(), + Source::Inherited, + Some(1), + ) + .expect_err("must fail"); + let mut err = Vec::new(); + failure.report(&mut err); + assert_eq!(err, expected, "{inherited:?}"); + assert!(system.bound_ports.is_empty() && system.sends.is_empty()); + } + } + /// Verifies retry-after-failure, changed-payload, and periodic refresh sends. #[test] fn changed_packets_refreshes_and_failed_sends_control_sender_state() { @@ -424,7 +553,7 @@ mod tests { ]); system.send_results = VecDeque::from([false, true, true, true]); assert_eq!( - run_with(&mut system, loopback_destination(), Some(5)), + run_with(&mut system, loopback_destination(), Source::Bound, Some(5)), Ok(()) ); assert_eq!( diff --git a/nvx-rs/src/guest/mod.rs b/nvx-rs/src/guest/mod.rs index f75ad0e47..99024c075 100644 --- a/nvx-rs/src/guest/mod.rs +++ b/nvx-rs/src/guest/mod.rs @@ -24,7 +24,9 @@ //! Two of them also offer their steps as typed calls, which any guest init or agent runs in its //! own process and their programs run on: on x86-64, `time`'s `TimeAbi` runs the time ABI's //! cold-boot step, capture, and restore finish, and [`reseed`] reseeds the kernel's random number -//! generator with a restore's fresh entropy, which starts with the new [`GenerationId`]. The +//! generator with a restore's fresh entropy, which starts with the new [`GenerationId`]. And +//! [`ingress`] starts its program, the announcer of the guest's listeners, from a socket that it +//! binds for it, as a clone's application does. The //! libraries need none of NVX's control protocol, managed agent, or workload identity, and NVX's //! managed agent opens its control terminal, applies its host mapping table, and starts, drives, //! collects, and settles its workloads through them. diff --git a/nvx-rs/src/guest/sys/socket.rs b/nvx-rs/src/guest/sys/socket.rs index f1769459c..2af1911cf 100644 --- a/nvx-rs/src/guest/sys/socket.rs +++ b/nvx-rs/src/guest/sys/socket.rs @@ -3,7 +3,9 @@ use std::net::{Ipv4Addr, SocketAddrV4}; use std::os::fd::{AsRawFd, OwnedFd}; -use nix::sys::socket::{AddressFamily, MsgFlags, SockFlag, SockType, SockaddrIn}; +use nix::sys::socket::{ + AddressFamily, MsgFlags, SockFlag, SockType, SockaddrIn, SockaddrLike, SockaddrStorage, sockopt, +}; use super::{Errno, FileDescriptor}; @@ -18,7 +20,7 @@ pub(crate) fn connect_datagram(peer: SocketAddrV4) -> Result Result { + let socket = OwnedFd::from(descriptor); + if nix::sys::socket::getsockopt(&socket, sockopt::SockType)? != SockType::Datagram { + return Err(Errno::EPROTOTYPE); + } + let address: SockaddrStorage = nix::sys::socket::getsockname(socket.as_raw_fd())?; + if address.family() != Some(AddressFamily::Inet) { + return Err(Errno::EAFNOSUPPORT); + } + Ok(Self(socket)) + } + + /// Returns the port to which the socket is bound, or 0 if it is bound to none. + pub(crate) fn port(&self) -> Result { + nix::sys::socket::getsockname::(self.0.as_raw_fd()) + .map(|address| address.port()) + } + /// Binds the socket to `port` on every local IPv4 address. pub(crate) fn bind_any(&self, port: u16) -> Result<(), Errno> { nix::sys::socket::bind( @@ -55,8 +79,18 @@ impl UdpSocket { } } +/// Gives up the socket to its new owner, which `crate::guest::spawn` hands to a child as one of +/// its standard streams. +impl From for OwnedFd { + fn from(socket: UdpSocket) -> Self { + socket.0 + } +} + #[cfg(test)] mod tests { + use std::os::fd::AsFd; + use super::*; #[test] @@ -74,6 +108,41 @@ mod tests { assert_eq!(&buffer[..length], b"announce"); } + #[test] + fn an_inherited_descriptor_is_taken_only_as_an_ipv4_datagram_socket() { + let duplicate = |descriptor: &dyn AsFd| { + FileDescriptor::from_owned(descriptor.as_fd().try_clone_to_owned().unwrap()) + }; + // A bound socket keeps its port, and one that is bound to none has port 0. + let bound = std::net::UdpSocket::bind("0.0.0.0:0").unwrap(); + let port = bound.local_addr().unwrap().port(); + let inherited = UdpSocket::inherited(duplicate(&bound)).unwrap(); + assert_eq!(inherited.port(), Ok(port)); + let unbound = UdpSocket::ipv4().unwrap(); + let inherited = UdpSocket::inherited(duplicate(&unbound.0)).unwrap(); + assert_eq!(inherited.port(), Ok(0)); + // A file, a stream socket, and a datagram socket of IPv6 are refused. + let file = tempfile::tempfile().unwrap(); + let stream = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + let ipv6 = nix::sys::socket::socket( + AddressFamily::Inet6, + SockType::Datagram, + SockFlag::SOCK_CLOEXEC, + None, + ) + .unwrap(); + for (descriptor, expected) in [ + (&file as &dyn AsFd, Errno::ENOTSOCK), + (&stream, Errno::EPROTOTYPE), + (&ipv6, Errno::EAFNOSUPPORT), + ] { + assert_eq!( + UdpSocket::inherited(duplicate(descriptor)).unwrap_err(), + expected + ); + } + } + #[test] fn a_used_port_is_eaddrinuse() { let used = std::net::UdpSocket::bind("0.0.0.0:0").unwrap(); diff --git a/nvx-rs/src/host/network/arguments.rs b/nvx-rs/src/host/network/arguments.rs index bd2f2cc66..9789f0fa2 100644 --- a/nvx-rs/src/host/network/arguments.rs +++ b/nvx-rs/src/host/network/arguments.rs @@ -1,10 +1,10 @@ //! The OpenVMM options that enforce a network posture. -use std::net::Ipv4Addr; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; -use super::address::Interval; +use super::address::{Family, Interval}; use super::loopback::check_loopback_forwards; -use super::lowering; +use super::lowering::{self, NativeProtocol, Rule}; use super::rule::parse_rules; use super::{ Access, DNS_PORT, HostLoopback, MAX_HOST_LOOPBACK_PREFIX, NetworkPolicy, NetworkProxy, @@ -41,30 +41,6 @@ pub fn openvmm_arguments( let Some(policy) = policy else { return Ok(Vec::new()); }; - lower_policy(policy, guest_network, false) -} - -/// Returns the OpenVMM options of `policy` for a guest at `guest_network`, as -/// [`openvmm_arguments`] renders them, but those of an attached network device for a policy that -/// lets nothing through too, for which [`openvmm_arguments`] attaches none: the options of the -/// same policy with one more allow rule that lowers to no OpenVMM rule, such as one that excepts -/// its whole destination, and which [`openvmm_arguments`] renders with a device. -/// -/// Fails as [`openvmm_arguments`] does. -pub(crate) fn attached_arguments( - policy: &NetworkPolicy, - guest_network: &str, -) -> Result> { - lower_policy(policy, guest_network, true) -} - -/// Returns the OpenVMM options of `policy` for a guest at `guest_network`, as -/// [`openvmm_arguments`] renders them, and, if `attached`, as [`attached_arguments`] does. -fn lower_policy( - policy: &NetworkPolicy, - guest_network: &str, - attached: bool, -) -> Result> { if let Some(proxy) = &policy.proxy { return proxy_arguments(policy, proxy, guest_network); } @@ -86,8 +62,7 @@ fn lower_policy( shorter, which leaves an address at which OpenVMM presents host peers to the guest" ))); } - if !attached - && policy.egress == Access::Deny + if policy.egress == Access::Deny && policy.allow.rules.is_empty() && policy.ingress == Access::Deny && policy.host_loopback == HostLoopback::Deny @@ -138,6 +113,90 @@ fn lower_policy( Ok(arguments) } +/// Returns the OpenVMM options with which a template's capture attaches its NIC to a guest at +/// `guest_network`: the portable profile under the complete deny-all policy, egress denied +/// without rules, ingress and host-loopback access denied, without a proxy or forwards. +/// +/// It is the one policy of a snapshot that OpenVMM lets a restore replace, with +/// `--restore-network-policy` and the options of the new policy: under it, the guest learned no +/// DNS server and announces no listeners, and the NIC's endpoint held no flows, listeners, or +/// forwards, so nothing that the snapshot holds depends on the policy beyond the NIC's identity. +pub(crate) fn template_arguments(guest_network: &str) -> Vec { + [ + "--net", + guest_network, + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-ingress", + "deny", + "--host-loopback", + "deny", + ] + .map(str::to_owned) + .to_vec() +} + +/// Returns the DNS server that OpenVMM names to a guest under `policy`, whose NIC's IPv4 gateway +/// is `gateway`, and whose IPv6 gateway is `ipv6_gateway` if its NIC has an IPv6 identity, as +/// OpenVMM chooses it: the IPv4 gateway if the policy lets the guest reach its DNS service, TCP +/// or UDP port 53, and otherwise the IPv6 gateway if it lets the guest reach that one's, and none +/// if it lets the guest reach neither, as a proxy-only policy does. A deny rule that matches the +/// traffic denies it, and otherwise an allow rule that matches it, or the egress default, allows +/// it. Denied host-loopback access leaves the gateway's DNS service to the egress policy. +/// +/// OpenVMM names the server on the kernel command line of a VM that boots with the policy, and a +/// clone whose restore installs the policy learns it from its activation record instead. +/// +/// Fails as [`openvmm_arguments`] does for rules that it cannot parse. +pub(crate) fn dns_server( + policy: &NetworkPolicy, + gateway: Ipv4Addr, + ipv6_gateway: Option, +) -> Result> { + if policy.proxy.is_some() { + return Ok(None); + } + let allow = parse_rules(&policy.allow)?.concat(); + let deny = parse_rules(&policy.deny)?.concat(); + let reaches = |server: IpAddr| { + [NativeProtocol::Tcp, NativeProtocol::Udp] + .into_iter() + .any(|protocol| { + let selected = |rule: &Rule| selects(rule, server, protocol, DNS_PORT); + !deny.iter().any(selected) + && (allow.iter().any(selected) || policy.egress == Access::Allow) + }) + }; + Ok([Some(IpAddr::V4(gateway)), ipv6_gateway.map(IpAddr::V6)] + .into_iter() + .flatten() + .find(|&server| reaches(server))) +} + +/// Returns whether `rule` selects traffic of `protocol` to `port` of `address`, as OpenVMM's rule +/// selects it: an address of the rule's family among its addresses, and any traffic for a rule of +/// every protocol, or traffic of its protocol to one of its ports, or to any for a rule without +/// ports. +fn selects(rule: &Rule, address: IpAddr, protocol: NativeProtocol, port: u16) -> bool { + let (family, address) = match address { + IpAddr::V4(address) => (Family::V4, u128::from(u32::from(address))), + IpAddr::V6(address) => (Family::V6, u128::from(address)), + }; + rule.family == family + && rule + .addresses + .iter() + .any(|&(first, last)| first <= address && address <= last) + && rule.protocol.is_none_or(|selected| { + selected == protocol + && rule + .ports + .is_none_or(|(first, last)| (first..=last).contains(&port)) + }) +} + /// Returns the OpenVMM options of a policy with a proxy, which must be proxy-only, for a guest at /// `guest_network`: everything denied but the proxy, which the guest reaches at its gateway. fn proxy_arguments( diff --git a/nvx-rs/src/host/network/loopback.rs b/nvx-rs/src/host/network/loopback.rs index 257651008..d188e143e 100644 --- a/nvx-rs/src/host/network/loopback.rs +++ b/nvx-rs/src/host/network/loopback.rs @@ -6,6 +6,7 @@ use std::collections::HashSet; use super::address::Family; use super::arguments::guest_subnet; use super::lowering::Rule; +#[cfg(feature = "cli")] use super::rule::parse_port; use super::{Access, MAX_LOOPBACK_FORWARDS, NetworkPolicy}; use crate::error::{Error, Result}; @@ -85,6 +86,7 @@ impl LoopbackForward { /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) and a message that /// starts with `value`. + #[cfg(feature = "cli")] pub(crate) fn parse(value: &str) -> Result { let refused = |reason: &str| Err(Error::unsupported(format!("{value}: {reason}"))); let fields: Vec<&str> = value.split(':').collect(); diff --git a/nvx-rs/src/host/network/mod.rs b/nvx-rs/src/host/network/mod.rs index 97e66f854..1a9dffaeb 100644 --- a/nvx-rs/src/host/network/mod.rs +++ b/nvx-rs/src/host/network/mod.rs @@ -24,6 +24,13 @@ //! a policy with a proxy denies egress without rules, ingress, and host-loopback access, and has //! no forwards. [`NetworkProxy::url`] gives the proxy's URL as the guest sees it; how workloads //! learn it is the caller's choice. +//! +//! A template's capture attaches its NIC under the complete deny-all policy instead, whatever the +//! network's own: egress denied without rules, ingress and host-loopback access denied, without a +//! proxy or forwards. It is the one policy of a snapshot that OpenVMM lets a restore replace, so +//! each restore of the template installs its clone's own policy, and the clone's activation +//! record gives the guest what OpenVMM gives a VM that boots with the policy on its kernel command +//! line: its DNS server, and whether its listeners are published. mod address; mod arguments; @@ -33,8 +40,8 @@ pub mod policy; mod proxy; mod rule; -pub(crate) use self::arguments::attached_arguments; pub use self::arguments::openvmm_arguments; +pub(crate) use self::arguments::{dns_server, template_arguments}; pub use self::loopback::{ForwardProtocol, HostLoopback, LoopbackForward}; pub use self::proxy::{NetworkProxy, ProxyScheme}; pub use self::rule::{Access, Destination, EgressRule, EgressRules, PortSelector, Protocol}; @@ -138,6 +145,15 @@ impl NetworkPolicy { self.deny.rules.push(rule); self } + + /// Returns whether OpenVMM publishes the guest's listeners under the policy: to the host's + /// private networks with ingress allowed, or on host loopback with host-loopback access + /// allowed both ways. While it does, the guest announces its listeners to the gateway, and + /// refuses its workloads `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW`, with which they could forge + /// an announcement. + pub fn publishes_listeners(&self) -> bool { + self.ingress == Access::Allow || self.host_loopback == HostLoopback::Allow + } } #[cfg(test)] diff --git a/nvx-rs/src/host/network/proxy.rs b/nvx-rs/src/host/network/proxy.rs index f74b54498..a81cbeed2 100644 --- a/nvx-rs/src/host/network/proxy.rs +++ b/nvx-rs/src/host/network/proxy.rs @@ -109,6 +109,7 @@ impl NetworkProxy { /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) and a message that /// starts with `endpoint`. + #[cfg(feature = "cli")] pub(crate) fn parse_endpoint(endpoint: &str, gateway: Ipv4Addr) -> Result { let refused = |reason: String| Err(Error::unsupported(format!("{endpoint}: {reason}"))); let Some((address, port)) = endpoint diff --git a/nvx-rs/src/host/network/rule.rs b/nvx-rs/src/host/network/rule.rs index 4ecd05837..bc0aeeba4 100644 --- a/nvx-rs/src/host/network/rule.rs +++ b/nvx-rs/src/host/network/rule.rs @@ -2,7 +2,10 @@ //! the parsing of the rules of an action into the rules that the lowering takes. use super::lowering::{self, PortValue, Rule, RuleProtocol}; -use crate::error::{Error, Result}; +#[cfg(feature = "cli")] +use crate::error::Error; +use crate::error::Result; +#[cfg(feature = "cli")] use crate::net::cidr; /// Allow or deny decision of a network posture. @@ -138,6 +141,7 @@ impl EgressRule { /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) and a message that /// starts with `value`. + #[cfg(feature = "cli")] pub(crate) fn parse_native(value: &str) -> Result { let refused = |reason: &str| Err(Error::unsupported(format!("{value}: {reason}"))); let (destination, selector) = split_native_rule(value); @@ -199,6 +203,7 @@ impl EgressRule { /// splits it: an IPv4 address holds no `:`, so its first `:` starts the selector; a network with a /// prefix ends at the first `:` after its `/`; and an IPv6 address without a prefix ends at the /// last `:` that leaves an IPv6 address before it, since protocol names are not hexadecimal. +#[cfg(feature = "cli")] fn split_native_rule(value: &str) -> (&str, Option<&str>) { let split_at = |index: usize| (&value[..index], Some(&value[index + 1..])); if let Some(slash) = value.find('/') { diff --git a/nvx-rs/src/host/network/tests.rs b/nvx-rs/src/host/network/tests.rs index 0aaab1937..a53f05318 100644 --- a/nvx-rs/src/host/network/tests.rs +++ b/nvx-rs/src/host/network/tests.rs @@ -1,7 +1,7 @@ //! Tests of the OpenVMM options of network postures, and of the parsers of their rules, //! forwards, and proxies. -use std::net::Ipv4Addr; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use super::*; use crate::ErrorKind; @@ -14,43 +14,6 @@ fn rules(arguments: &[String], option: &str) -> Vec { .collect() } -#[test] -fn an_attached_lowering_attaches_the_device_of_a_policy_that_lets_nothing_through() { - // An allow rule that excepts its whole destination lowers to no OpenVMM rule, but a policy - // with one attaches its device, closed, as the attached lowering of the same policy - // without that rule renders it. - let excluded = EgressRule { - to: vec![Destination { - cidr: "192.0.2.0/24".to_owned(), - except: vec!["192.0.2.0/24".to_owned()], - }], - ports: Vec::new(), - }; - let denied = NetworkPolicy::new(Access::Deny).with_deny(EgressRule::to("1.2.3.4")); - for closed in [NetworkPolicy::new(Access::Deny), denied] { - assert!( - openvmm_arguments(Some(&closed), "10.0.0.2/24") - .unwrap() - .is_empty() - ); - let excepted = closed.clone().with_allow(excluded.clone()); - let attached = attached_arguments(&closed, "10.0.0.2/24").unwrap(); - assert_eq!( - attached, - openvmm_arguments(Some(&excepted), "10.0.0.2/24").unwrap() - ); - assert_eq!(attached[..2], ["--net", "10.0.0.2/24"]); - } - // The attached lowering of any other policy is its lowering. - let proxied = NetworkPolicy::proxy_only(NetworkProxy::new(3128)); - for policy in [NetworkPolicy::new(Access::Allow), proxied] { - assert_eq!( - attached_arguments(&policy, "10.0.0.2/24").unwrap(), - openvmm_arguments(Some(&policy), "10.0.0.2/24").unwrap() - ); - } -} - #[test] fn default_postures_map_to_portable_profile_options() { assert!(openvmm_arguments(None, "10.0.0.2/24").unwrap().is_empty()); @@ -1023,3 +986,142 @@ fn loopback_forwards_parse_as_openvmm_parses_them() { assert_eq!(error.message(), format!("{value}: {reason}")); } } + +#[test] +fn templates_attach_their_nic_under_the_complete_deny_all_policy() { + assert_eq!( + template_arguments("10.0.0.2/24"), + [ + "--net", + "10.0.0.2/24", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-ingress", + "deny", + "--host-loopback", + "deny", + ] + ); +} + +#[test] +fn the_dns_server_is_the_first_gateway_whose_dns_service_the_policy_reaches() { + let gateway = Ipv4Addr::new(10, 0, 0, 1); + let ipv6_gateway: Ipv6Addr = "fd00::a00:1".parse().unwrap(); + let v4 = Some(IpAddr::V4(gateway)); + let v6 = Some(IpAddr::V6(ipv6_gateway)); + let allow = || NetworkPolicy::new(Access::Allow); + let deny = || NetworkPolicy::new(Access::Deny); + let rule = |cidr: &str| EgressRule::to(cidr); + let cases = [ + ("egress allowed", allow(), v4, v4), + ("egress denied", deny(), None, None), + ("a proxy-only policy", proxy_only(), None, None), + ( + "an allow rule of TCP port 53 at the gateway", + deny().with_allow(rule("10.0.0.1/32").on_port(Protocol::Tcp, 53)), + v4, + v4, + ), + ( + "an allow rule of UDP to every port", + deny().with_allow(rule("0.0.0.0/0").on_protocol(Protocol::Udp)), + v4, + v4, + ), + ( + "an allow rule of a range of ports that holds 53", + deny().with_allow(rule("10.0.0.0/24").on_port_range(Protocol::Any, 50, 60)), + v4, + v4, + ), + ( + "an allow rule of every protocol at the gateway", + deny().with_allow(rule("10.0.0.1")), + v4, + v4, + ), + ( + "an allow rule of another port at the gateway", + deny().with_allow(rule("10.0.0.1/32").on_port(Protocol::Tcp, 8080)), + None, + None, + ), + ( + "an allow rule of ICMP at the gateway", + deny().with_allow(rule("10.0.0.1/32").on_protocol(Protocol::Icmp)), + None, + None, + ), + ( + "an allow rule of another destination", + deny().with_allow(rule("10.0.0.2/32").on_port(Protocol::Udp, 53)), + None, + None, + ), + ( + "a deny rule of UDP alone, which leaves TCP", + allow().with_deny(rule("10.0.0.1/32").on_port(Protocol::Udp, 53)), + v4, + v4, + ), + ( + "a deny rule of the IPv4 gateway that an allow rule overlaps", + deny() + .with_allow(rule("0.0.0.0/0").on_port(Protocol::Any, 53)) + .with_allow(rule("::/0").on_port(Protocol::Any, 53)) + .with_deny(rule("10.0.0.0/8")), + v6, + None, + ), + ( + "an allow rule of the IPv6 gateway alone", + deny().with_allow(rule("fd00::a00:1/128").on_port(Protocol::Udp, 53)), + v6, + None, + ), + ( + "an exception that removes the IPv4 gateway", + deny().with_allow(EgressRule { + to: vec![Destination { + cidr: "10.0.0.0/24".to_owned(), + except: vec!["10.0.0.0/30".to_owned()], + }], + ports: Vec::new(), + }), + None, + None, + ), + ]; + for (name, policy, dual_stack, ipv4_only) in cases { + assert_eq!( + dns_server(&policy, gateway, Some(ipv6_gateway)).unwrap(), + dual_stack, + "{name}" + ); + assert_eq!( + dns_server(&policy, gateway, None).unwrap(), + ipv4_only, + "{name}" + ); + } + // Host-loopback access, ingress, and forwards leave the egress policy's choice. + let mut open = deny(); + open.ingress = Access::Allow; + open.host_loopback = HostLoopback::Allow; + open.host_loopback_forwards = vec![LoopbackForward { + protocol: ForwardProtocol::Tcp, + host_port: 18080, + guest_port: 8080, + }]; + assert_eq!( + dns_server(&open, gateway, Some(ipv6_gateway)).unwrap(), + None + ); + // A rule that does not parse fails as the lowering fails. + let invalid = deny().with_allow(rule("10.0.0.0/33")); + let error = dns_server(&invalid, gateway, None).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported); +} diff --git a/nvx-rs/src/host/sandbox/plan.rs b/nvx-rs/src/host/sandbox/plan.rs index 4e7531539..e9905a956 100644 --- a/nvx-rs/src/host/sandbox/plan.rs +++ b/nvx-rs/src/host/sandbox/plan.rs @@ -104,7 +104,7 @@ mod form; use std::fmt; -use std::net::Ipv4Addr; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use std::path::{Path, PathBuf}; use serde::{Deserialize, Serialize}; @@ -327,6 +327,31 @@ impl SandboxNetwork { pub fn openvmm_arguments(&self) -> Result> { network::openvmm_arguments(Some(&self.policy), &self.guest_network) } + + /// Returns the OpenVMM arguments with which a template's capture attaches its NIC for the + /// network: the guest's address under the complete deny-all policy, whatever the network's + /// own policy, which each restore of the template replaces with its clone's; or none for a + /// policy that lets nothing through, for which [`openvmm_arguments`](Self::openvmm_arguments) + /// attaches no NIC either, so that a template of the network has none. + /// + /// Fails as [`openvmm_arguments`](Self::openvmm_arguments) does. + pub fn template_arguments(&self) -> Result> { + Ok(if self.openvmm_arguments()?.is_empty() { + Vec::new() + } else { + network::template_arguments(&self.guest_network) + }) + } + + /// Returns the DNS server that OpenVMM names to the guest under the policy, given the IPv6 + /// gateway of the guest's NIC if the NIC has an IPv6 identity, as [`network::dns_server`] + /// chooses it: the gateway, of either family, whose DNS service the policy lets the guest + /// reach, if any. + /// + /// Fails as [`openvmm_arguments`](Self::openvmm_arguments) does. + pub(crate) fn dns_server(&self, ipv6_gateway: Option) -> Result> { + network::dns_server(&self.policy, self.gateway, ipv6_gateway) + } } /// Returns the gateway of `guest_network`, a static guest address that OpenVMM accepts, as diff --git a/nvx-rs/src/host/template/clone.rs b/nvx-rs/src/host/template/clone.rs index 0e4e3ea64..c11db28dc 100644 --- a/nvx-rs/src/host/template/clone.rs +++ b/nvx-rs/src/host/template/clone.rs @@ -78,11 +78,10 @@ pub struct CloneRequest { /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the clone's image, its /// `distro`, an EROFS layer or a GPT disk, over its upper: in RAM, a tmpfs, as /// [`SandboxUpper::RAM`](crate::host::sandbox::SandboxUpper::RAM) plans it, or on the - /// caller's scratch image. Its network must be the template's, which every restore of the - /// template attaches unchanged, as [`ClonePlan::new`] checks: the network that the template's - /// class lowered to OpenVMM's arguments, as its key records it, - /// [`TemplateKey::network_arguments`], or for a template without a NIC none, or one that lets - /// nothing through, which attaches none either. + /// caller's scratch image. Its network is the clone's own, whose policy the restore enforces + /// on the template's NIC in place of the template's complete deny-all policy, at the guest's + /// address that the NIC has, [`TemplateKey::guest_network`], or for a template without a NIC + /// none, or one that lets nothing through, which attaches none either. pub adapter: SandboxPlan, /// The binding of the image. pub binding: DistroBinding, @@ -198,15 +197,13 @@ pub enum CloneUpper { /// online at boot, and serves the target of its capacity alone. /// - The memory target must be whole blocks of [`MEMORY_BLOCK_SIZE`], and of the template's own /// memory blocks, from the template's RAM to its RAM capacity. -/// - The network of the adapter's plan must be the template's: lowered to OpenVMM's arguments, as +/// - The network of the adapter's plan must attach a NIC, as /// [`SandboxNetwork::openvmm_arguments`](crate::host::sandbox::SandboxNetwork::openvmm_arguments) -/// lowers it, it must be the network that the template's key records, -/// [`TemplateKey::network_arguments`], the guest's address and the policy of the template's NIC, -/// or none for a template without a NIC. A template serves the clones of its network policy -/// alone, and the restore passes the template's network unchanged, as the key records it, so no -/// clone has ingress, host-loopback access to its listeners, or forwards, which no template's -/// policy allows: a template's capture refuses a policy that allows ingress, or host-loopback -/// access both ways, which forwards need. +/// lowers it, if and only if the template has one, at the guest's address of the template's NIC, +/// which its key records, [`TemplateKey::guest_network`]. A template captures its NIC under the +/// complete deny-all policy, which the restore replaces with the policy of the clone's network, +/// so a template serves the clones of every network policy at its guest's address: egress +/// rules, a proxy, ingress, host-loopback access, and host-loopback forwards alike. /// - The export, if the request has one, must be a plan that [`filesystem::plan`] could have made /// for the container root, as [`ExportPlan::check`] finds it, of exactly one child, a /// directory: a template's filesystem slot is dormant, and a restore binds one directory to it, @@ -234,7 +231,7 @@ pub enum CloneUpper { /// to a leased payload, say. A request without an export leaves the slot dormant. /// - The activation record, which the clone's guest applies in place of the configuration that /// the template captured, names the restore, with a fresh transaction ID, the clone's image, -/// upper, export, and settings, the template's network, and the resources that the clone must +/// upper, export, and settings, its network, and the resources that the clone must /// have, as [`activation`](Self::activation) describes. Its settings must keep the limits of a /// record, as [`Configuration::check`] checks them. /// - The capability is fresh: the plan draws it for the clone's launch, [`launch`](Self::launch). @@ -319,8 +316,8 @@ pub struct ClonePlan { activation: ActivationRecord, /// The capability of the clone's launch. capability: [u8; CAPABILITY_LEN], - /// The clone's devices, which its restore attaches with the template's network: the adapter's - /// plan of its image, over the file of its `scratch` block device. + /// The clone's devices, which its restore attaches: the adapter's plan of its image and + /// network, over the file of its `scratch` block device. devices: SandboxPlan, /// The clone's export, which its restore binds to the template's dormant filesystem slot, if /// it has one. @@ -406,17 +403,15 @@ impl ClonePlan { /// - with [`ErrorKind::Unsupported`] for a request that the template cannot serve: a template /// whose key names other contracts than this build speaks, the boot and control contracts /// of the runtime that captured it included, [`Evidence::IncompatibleHost`]; a plan whose - /// network is not the template's, as the template's key records it, - /// [`TemplateKey::network_arguments`]: a network for a template without a NIC, none for a - /// template with a NIC, or one with another guest network than the template's NIC, whose - /// address OpenVMM restores from the snapshot, or another policy, - /// [`Evidence::InvalidSettings`]; and a processor target other than 1, 2, 4, or 8, or + /// network does not lower to OpenVMM's arguments, as + /// [`SandboxNetwork::openvmm_arguments`](crate::host::sandbox::SandboxNetwork::openvmm_arguments) + /// finds it, a plan with a network for a template without a NIC, or without one for a + /// template with a NIC, or one at another guest's address than the template's NIC, as its + /// key records it, [`TemplateKey::guest_network`], [`Evidence::InvalidSettings`]; and a + /// processor target other than 1, 2, 4, or 8, or /// outside the template's processors online at boot and its processor capacity, and a /// memory target that is not whole blocks of [`MEMORY_BLOCK_SIZE`] and of the template's /// memory blocks, from the template's RAM to its RAM capacity, [`Evidence::InvalidTargets`]; - /// - with [`ErrorKind::Io`] for a template whose command line names the DNS server of its - /// guest's resolver in another form than OpenVMM writes: more than once, without a NIC, or - /// other than as one IP address in its canonical spelling, [`Evidence::MalformedGeneration`]; /// - with [`ErrorKind::Unsupported`] for settings that no activation record holds, as /// [`Configuration::check`] finds them, such as a GPT disk's first-partition bound of 0, a /// hostname that is empty, longer than @@ -684,16 +679,16 @@ impl ClonePlan { /// the export, if the request has one, a directory, with the identity of its plan, and the /// number of its mappings, the binds of the plan, from 1 to /// [`MAX_MAPPINGS`](crate::activation::MAX_MAPPINGS), or no export and no mappings; the - /// network of the template's NIC, if it has one, with the identity of the policy that OpenVMM - /// enforces there, as the template's key records it, the DNS server with which the template's - /// guest configured its resolver at boot, if the template's - /// [command line](TemplateKey::command_line) names one, unpublished listeners, and the URL of - /// the policy's proxy as the guest sees it, if any; and the processor target, which the - /// restore brings online, and the memory target, as the resources that the clone must have. + /// clone's network, if the template has a NIC, with the identity of the network, whose policy + /// the restore enforces on the NIC, the DNS server that OpenVMM serves the guest at a gateway + /// under that policy, if any, whether the guest publishes its listeners, as it does under a + /// policy that allows ingress or host-loopback access both ways, and the URL of the policy's + /// proxy as the guest sees it, if any; and the processor target, which the restore brings + /// online, and the memory target, as the resources that the clone must have. /// - /// The identity of a binding, or of an export plan, is the SHA-256 digest of a domain of its - /// kind and of its serde form, a few hundred bytes of metadata: the plan reads no content to - /// compute it. + /// The identity of a binding, of an export plan, or of a network is the SHA-256 digest of a + /// domain of its kind and of its serde form, a few hundred bytes of metadata: the plan reads + /// no content to compute it. pub fn activation(&self) -> &ActivationRecord { &self.activation } @@ -725,11 +720,12 @@ impl fmt::Debug for ClonePlan { /// Returns the devices of the clone of `request`: the adapter's plan of its image over its upper, /// which renders the `scratch` block device of an upper on a scratch image alone, since the -/// restore declares that of an upper in RAM absent. The restore attaches the template's network, -/// which its key records, rather than the plan's, which the plan's check found to be the same. +/// restore declares that of an upper in RAM absent, and of its network, whose policy the restore +/// enforces on the template's NIC. fn devices_of(request: &CloneRequest) -> SandboxPlan { let adapter = &request.adapter; SandboxPlan::adapter(adapter.distro().clone(), adapter.upper().clone()) + .with_network(adapter.network().cloned()) } /// Refuses a request that no clone takes, as [`ClonePlan::new`] describes, without reading or @@ -935,48 +931,37 @@ fn check_template( )), )); } - // A template serves the clones of its own network alone: the guest's address and the policy of - // its NIC, which every restore passes unchanged, or no NIC, which a network that lets nothing - // through attaches no more than none does. OpenVMM restores the guest's address on the NIC - // from the snapshot, so a plan of another address would run on the template's. + // A template captures its NIC under the complete deny-all policy, which every restore replaces + // with the policy of its clone's network, so it serves the clones of every network at its + // guest's address, or, without a NIC, those of none, or of one that lets nothing through, + // which attaches no NIC either. let requested = match request.adapter.network() { - None => Vec::new(), - Some(network) => network - .openvmm_arguments() - .attribute(Evidence::InvalidSettings)?, + None => None, + Some(network) => { + let attached = !network + .openvmm_arguments() + .attribute(Evidence::InvalidSettings)? + .is_empty(); + attached.then(|| network.guest_network()) + } }; - let template = key.network_arguments(); - if requested != template { - let guest = request - .adapter - .network() - .map(|network| network.guest_network()); - let nic = key.guest_network(); - let problem = if template.is_empty() { - format!( - "the adapter's plan of a clone attaches a network, but the template's \ - generation {name} has no NIC" - ) - } else if requested.is_empty() { - format!( - "the template's generation {name} has a NIC, whose network every clone of it \ - restores, but the adapter's plan of the clone has none" - ) - } else if let Some(guest) = guest - && let Some(nic) = nic.as_deref() - && guest != nic - { - format!( - "the adapter's plan of a clone gives its guest the network {guest}, but the \ - template's generation {name} restores its NIC on {nic}, from its snapshot" - ) - } else { - format!( - "the network of the adapter's plan of a clone is not the one that the template's \ - generation {name} restores its NIC with: a template serves the clones of its \ - own network policy, at its own guest's address, alone" - ) - }; + let problem = match (requested, key.guest_network()) { + (Some(_), None) => Some(format!( + "the adapter's plan of a clone attaches a network, but the template's generation \ + {name} has no NIC" + )), + (None, Some(_)) => Some(format!( + "the template's generation {name} has a NIC, which its clones restore with the \ + network of the adapter's plan, but the plan has none" + )), + (Some(requested), Some(template)) if requested != template => Some(format!( + "the adapter's plan of a clone attaches a network at {requested}, but the NIC of the \ + template's generation {name} is at {template}: a template serves the clones of its \ + own guest's address alone" + )), + _ => None, + }; + if let Some(problem) = problem { return Err(Fault::new( Evidence::InvalidSettings, Error::unsupported(problem), diff --git a/nvx-rs/src/host/template/clone/failure.rs b/nvx-rs/src/host/template/clone/failure.rs index dbdef5610..56f4ea1e9 100644 --- a/nvx-rs/src/host/template/clone/failure.rs +++ b/nvx-rs/src/host/template/clone/failure.rs @@ -65,7 +65,11 @@ use crate::protocol::status; /// profile, TSC, or LAPIC timer failed on an [incompatible host](Self::IncompatibleHost). Any /// other code, such as one of the snapshot's time contract, keeps what the host saw, since it does /// not tell a defect of the generation from one of this host's OpenVMM, and the failure's message -/// keeps the line. +/// keeps the line. One error that has no code tells the host's fault by its text alone: a VMM +/// whose error's chain names a forward of the clone's network whose port of host loopback OpenVMM +/// cannot bind, as one that another socket holds, which OpenVMM binds as it restores the clone's +/// NIC, before its restore-ready event, failed to [bind a forward](Self::ForwardBind). The failure's +/// message keeps that cause of the chain after the line. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] #[serde(rename_all = "kebab-case")] #[non_exhaustive] @@ -130,7 +134,9 @@ pub enum Evidence { /// event, or a control socket whose path a Unix socket's address, or OpenVMM's option /// `listen=`, cannot hold. EndpointBind, - /// A forward of the clone's network cannot be bound. No clone's network forwards a port yet. + /// A forward of the clone's network cannot be bound: OpenVMM cannot bind its port of host + /// loopback, as when another socket holds it, as it restores the clone's NIC, and exits before + /// its restore-ready event, as [the attribution](Self#attribution) describes. ForwardBind, /// The host ran out of a resource, such as its storage, its memory, or its descriptors. ResourceExhaustion, @@ -164,9 +170,8 @@ pub enum Evidence { /// [`ACTIVATION_ORDER`](crate::protocol::status::ACTIVATION_ORDER) tell. Protocol, /// The leased generation is malformed or inconsistent: the store indexes it with another key - /// than the lease names, its key or snapshot is malformed, its key names its guest's DNS - /// server in a form that OpenVMM does not write, or its payload is not a directory of the - /// current user with mode 0700. + /// than the lease names, its key or snapshot is malformed, or its payload is not a directory + /// of the current user with mode 0700. MalformedGeneration, /// A defect of the generation's capture or repair, which a diagnostic attributes to the /// generation. No clone's restore keeps the guest's diagnostics yet, so a status of the repair @@ -890,17 +895,31 @@ const INCOMPATIBLE_HOST_CODES: [&str; 13] = [ "E_LAPIC_RATE_MISMATCH", ]; +/// The text with which a cause of OpenVMM's fatal error names a forward of a clone's network whose +/// port of host loopback OpenVMM cannot bind as it restores the clone's NIC: the forward's +/// protocol, `Tcp` or `Udp`, its address, and its port follow, then `)`, as the excerpt of +/// OpenVMM's source `tests/vectors/openvmm/port_forward_socket_creation.rs` spells the error. +/// OpenVMM gives the error no code. +pub(super) const FORWARD_BIND_TEXT: &str = "failed to create socket for port forward ("; + /// Returns the evidence of `line`, the line with which OpenVMM reported the error that ended a -/// clone's VMM before its restore-ready event, as [the attribution of -/// evidence](Evidence#attribution) describes: [`Evidence::IncompatibleHost`] for an error whose -/// time ABI code says that the host cannot present the template's CPU or time, and none for any -/// other. +/// clone's VMM before its restore-ready event, followed by the cause of the error's chain that +/// names a forward that OpenVMM cannot bind, if it has one, as the launch reads them from the end +/// of the VMM's log, as [the attribution of evidence](Evidence#attribution) describes: +/// [`Evidence::IncompatibleHost`] for an error whose time ABI code says that the host cannot +/// present the template's CPU or time, [`Evidence::ForwardBind`] for one that names a forward that +/// OpenVMM cannot bind, [`FORWARD_BIND_TEXT`], and none for any other. pub(super) fn fatal_evidence(line: &str) -> Option { - let code = line.strip_prefix(FATAL_PREFIX)?.strip_prefix('[')?; - let code = &code[..code.find(']')?]; - INCOMPATIBLE_HOST_CODES - .contains(&code) - .then_some(Evidence::IncompatibleHost) + let reported = line.strip_prefix(FATAL_PREFIX)?; + let code = reported + .strip_prefix('[') + .and_then(|code| code.get(..code.find(']')?)); + if code.is_some_and(|code| INCOMPATIBLE_HOST_CODES.contains(&code)) { + return Some(Evidence::IncompatibleHost); + } + reported + .contains(FORWARD_BIND_TEXT) + .then_some(Evidence::ForwardBind) } #[cfg(test)] @@ -1240,6 +1259,39 @@ mod tests { } } + #[test] + fn openvmm_s_fatal_error_is_evidence_of_a_forward_bind_when_it_names_a_forward_it_cannot_bind() + { + // OpenVMM spells the error as its source, which the excerpt copies, writes it. + let excerpt = + include_str!("../../../../tests/vectors/openvmm/port_forward_socket_creation.rs"); + let attribute = format!("#[error(\"{FORWARD_BIND_TEXT}{{details}})\")]"); + assert!( + excerpt.lines().any(|line| line.trim() == attribute), + "{attribute} in {excerpt}" + ); + // The cause of the chain follows the line that leads the error, or is the line's own. + for line in [ + "fatal error: failed to launch vm worker: failed to create socket for port forward \ + (Tcp 127.0.0.1:18080)", + "fatal error: failed to create socket for port forward (Udp 127.0.0.1:5353)", + "fatal error: [E_MANIFEST_TIME] failed to launch vm worker: failed to create socket \ + for port forward (Tcp 127.0.0.1:18080)", + ] { + assert_eq!(fatal_evidence(line), Some(Evidence::ForwardBind), "{line}"); + } + // A code of an incompatible host prevails, and a line that is not the fatal error tells + // nothing. + let incompatible = "fatal error: [E_CPU_GENERATION] failed to launch vm worker: failed to \ + create socket for port forward (Tcp 127.0.0.1:18080)"; + assert_eq!( + fatal_evidence(incompatible), + Some(Evidence::IncompatibleHost) + ); + let quoted = "warning: failed to create socket for port forward (Tcp 127.0.0.1:18080)"; + assert_eq!(fatal_evidence(quoted), None); + } + #[test] fn host_failures_that_ran_out_of_a_resource_are_resource_exhaustion() { let failed = |kind: io::ErrorKind| { diff --git a/nvx-rs/src/host/template/clone/launch.rs b/nvx-rs/src/host/template/clone/launch.rs index 842c9f6cc..656d2b808 100644 --- a/nvx-rs/src/host/template/clone/launch.rs +++ b/nvx-rs/src/host/template/clone/launch.rs @@ -130,10 +130,11 @@ impl ClonePlan { /// which OpenVMM opens alike, and for an upper in RAM, `--restore-absent-scratch`, which /// declares the template's scratch absent, so that OpenVMM attaches it with no file, at the /// geometry that the template recorded, as the template's guest negotiated it: its reads return - /// zeros and its writes fail, and the clone's guest never uses it; and the template's - /// network, as its key records it, [`TemplateKey::network_arguments`], the policy of the - /// template's NIC, unchanged, if it has one, without `--net` and the guest's address, which - /// OpenVMM takes from the snapshot. The clone's export, if it has one, follows, bound to the + /// zeros and its writes fail, and the clone's guest never uses it; and the clone's network, if + /// the template has a NIC: `--restore-network-policy`, with which the policy of the clone's + /// network replaces the template's complete deny-all policy on the NIC, and the options of that + /// policy, without `--net` and the guest's address, which OpenVMM takes from the snapshot. The + /// clone's export, if it has one, follows, bound to the /// template's dormant filesystem slot: `--mount` with /// [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), the exported directory, and its /// mode, `rw` if a mapping writes it, and `ro` otherwise, then its hidden, re-exposed, and @@ -282,7 +283,7 @@ impl ClonePlan { devices, export, } = self; - let mut devices = device_arguments(&devices, &key) + let mut devices = device_arguments(&devices) .attribute(Evidence::InvalidSettings) .map_err(unrecorded)?; let table = match &export { @@ -617,27 +618,28 @@ impl ClonePlan { } } -/// Returns OpenVMM's arguments of the devices of a clone's restore: those of the clone's devices, -/// `devices`, which [`SandboxPlan::openvmm_arguments`] renders, with `--restore-absent-scratch` -/// for an upper in RAM, whose plan renders no `scratch` block device, then the template's network, -/// as `key` records it, [`TemplateKey::network_arguments`], without `--net` and the guest's -/// address, which OpenVMM takes from the snapshot, and refuses beside it. The options of the -/// network's policy stay: OpenVMM compares the policy that they name with the one of the -/// snapshot's NIC. -fn device_arguments(devices: &SandboxPlan, key: &TemplateKey) -> Result> { +/// Returns OpenVMM's arguments of the devices of a clone's restore, `devices`: those that +/// [`SandboxPlan::openvmm_arguments`] renders, with `--restore-network-policy` in place of `--net` +/// and the guest's address, which OpenVMM takes from the snapshot, and refuses beside it, so that +/// the policy of the clone's network replaces the template's complete deny-all policy, then +/// `--restore-absent-scratch` for an upper in RAM, whose plan renders no `scratch` block device. +fn device_arguments(devices: &SandboxPlan) -> Result> { let mut arguments = devices.openvmm_arguments()?; + if let Some(at) = arguments.iter().position(|argument| argument == "--net") { + arguments.splice( + at..(at + 2).min(arguments.len()), + ["--restore-network-policy".to_owned()], + ); + } if devices.upper().scratch_path().is_none() { arguments.push("--restore-absent-scratch".to_owned()); } - // A key's network attaches its NIC first, with `--net` and the guest's address. - arguments.extend_from_slice(key.network_arguments().get(2..).unwrap_or_default()); Ok(arguments) } /// What OpenVMM's arguments of a clone's restore name besides its control endpoint. struct Restore<'a> { - /// The arguments of the clone's devices and of the template's network, as - /// [`device_arguments`] renders them. + /// The arguments of the clone's devices, as [`device_arguments`] renders them. devices: Vec, /// The generation's key. key: &'a TemplateKey, diff --git a/nvx-rs/src/host/template/clone/record.rs b/nvx-rs/src/host/template/clone/record.rs index 2e218d7f6..f717b30b8 100644 --- a/nvx-rs/src/host/template/clone/record.rs +++ b/nvx-rs/src/host/template/clone/record.rs @@ -3,11 +3,11 @@ //! clone's guest applies in place of the configuration that its template captured, as //! [`nvx::activation`](crate::activation) describes it. //! -//! A record names the bindings of the clone's image and scratch image, and the plan of its export, -//! by their identities: the SHA-256 digest of a domain of each kind of object and of the object's -//! serde form, which its callers record. The digest covers metadata alone, a few hundred bytes of -//! a binding, and the host paths and guest targets of an export plan, never the content of a -//! file. +//! A record names the bindings of the clone's image and scratch image, the plan of its export, and +//! its network, by their identities: the SHA-256 digest of a domain of each kind of object and of +//! the object's serde form, which its callers record. The digest covers metadata alone, a few +//! hundred bytes of a binding or of a network, and the host paths and guest targets of an export +//! plan, never the content of a file. A clone's launch computes each once. use serde::Serialize; use sha2::{Digest, Sha256}; @@ -20,6 +20,7 @@ use crate::activation::{ }; use crate::error::{Error, Result}; use crate::host::filesystem::ExportPlan; +use crate::host::sandbox::SandboxNetwork; use crate::host::sandbox::distro::{DistroBinding, DistroFormat}; use crate::host::sandbox::scratch::ScratchBinding; use crate::host::template::key::TemplateKey; @@ -37,6 +38,10 @@ const SCRATCH_DOMAIN: &[u8] = b"nvx scratch binding\0"; /// [`DISTRO_DOMAIN`] does for an image's binding. const EXPORT_DOMAIN: &[u8] = b"nvx export plan\0"; +/// What the identity of a clone's network digests before the network's serde form, as +/// [`DISTRO_DOMAIN`] does for an image's binding. +const NETWORK_DOMAIN: &[u8] = b"nvx network policy\0"; + /// Returns the activation record of the clone that `request` asks for, of the template whose key /// is `key`, which `lease` leases, with a fresh transaction ID. /// @@ -54,12 +59,15 @@ const EXPORT_DOMAIN: &[u8] = b"nvx export plan\0"; /// - the export, if the request has one: a directory, which stage 1's rule allows a clone alone, /// with the identity of its plan, and the number of its mappings, the plan's binds, from 1 to /// [`MAX_MAPPINGS`](crate::activation::MAX_MAPPINGS); or no export, and no mappings; -/// - the network of the template's NIC, if it has one: the identity of the policy that OpenVMM -/// enforces there, the digest that OpenVMM computed and the template's key records; the DNS -/// server with which the template's guest configured its resolver at boot, which the template's -/// command line names, [`TemplateKey::dns_server`], if it names one; unpublished listeners, -/// since a template's policy allows neither ingress nor host-loopback access both ways; and the -/// URL of the policy's proxy as the guest sees it, if it has one; +/// - the clone's network, if the template has a NIC, whose policy the restore enforces there in +/// place of the template's complete deny-all policy: the identity of the network, its guest's +/// address and policy; the DNS server that OpenVMM serves the guest at a gateway under the +/// policy, as [`SandboxNetwork::dns_server`] finds it with the IPv6 gateway of the template's +/// NIC, [`TemplateKey::ipv6_gateway`], if it serves one, since the template's guest configured +/// no resolver at boot under the complete deny-all policy; published listeners under a policy +/// that publishes them, [`NetworkPolicy::publishes_listeners`], which allows ingress or +/// host-loopback access both ways, and unpublished ones otherwise; and the URL of the policy's +/// proxy as the guest sees it, if it has one; /// - the expected resources: the request's processor target, which the restore brings online, and /// its memory target. /// @@ -67,11 +75,12 @@ const EXPORT_DOMAIN: &[u8] = b"nvx export plan\0"; /// no record holds, as [`Configuration::check`] finds it, such as a GPT disk's first-partition /// bound of 0, a hostname that is empty or longer than /// [`MAX_HOSTNAME_BYTES`](crate::activation::MAX_HOSTNAME_BYTES), or a workload identity whose -/// UID or GID is 0, [`Evidence::InvalidActivate`]; with [`ErrorKind::Io`](crate::ErrorKind::Io) -/// for a key whose command line names its DNS server in a form that OpenVMM does not write, as -/// [`TemplateKey::dns_server`] finds it, [`Evidence::MalformedGeneration`]; and with +/// UID or GID is 0, [`Evidence::InvalidActivate`], and for a network whose policy does not lower +/// to OpenVMM's arguments, [`Evidence::InvalidSettings`]; and with /// [`ErrorKind::Other`](crate::ErrorKind::Other) if the transaction ID cannot be drawn, or the -/// identity of a binding or of the export's plan computed, the host's evidence. +/// identity of a binding, of the export's plan, or of the network computed, the host's evidence. +/// +/// [`NetworkPolicy::publishes_listeners`]: crate::host::network::NetworkPolicy::publishes_listeners pub(super) fn build( request: &CloneRequest, lease: &Lease, @@ -97,16 +106,22 @@ pub(super) fn build( journal: request.journal, }, }; - let dns = key.dns_server().attribute(Evidence::MalformedGeneration)?; - let network = key.network_policy_digest().map(|policy| Network { - policy: Identity::from_bytes(policy), - dns, - publication: Publication::Unpublished, - proxy: request - .adapter - .network() - .and_then(|network| network.proxy_url()), - }); + // The plan's check found that the network attaches a NIC if and only if the template has one. + let network = match request.adapter.network().filter(|_| key.has_network()) { + None => None, + Some(network) => Some(Network { + policy: network_identity(network).on_host()?, + dns: network + .dns_server(key.ipv6_gateway()) + .attribute(Evidence::InvalidSettings)?, + publication: if network.policy().publishes_listeners() { + Publication::Published + } else { + Publication::Unpublished + }, + proxy: network.proxy_url(), + }), + }; let (export, mappings) = match &request.export { None => (None, 0), Some(plan) => ( @@ -167,8 +182,15 @@ pub(super) fn export_identity(plan: &ExportPlan) -> Result { identity(EXPORT_DOMAIN, plan, "plan of the clone's export") } -/// Returns the SHA-256 digest of `domain` and of the serde form of `object`, a clone's binding or -/// export plan, which messages call `what`. +/// Returns the identity of `network`, a clone's network, as the [module](self) describes it: it +/// covers the guest's address and the whole policy, its egress, rules, ingress, host-loopback +/// access and forwards, and proxy. +pub(super) fn network_identity(network: &SandboxNetwork) -> Result { + identity(NETWORK_DOMAIN, network, "network of the clone") +} + +/// Returns the SHA-256 digest of `domain` and of the serde form of `object`, a clone's binding, +/// export plan, or network, which messages call `what`. fn identity(domain: &[u8], object: &impl Serialize, what: &str) -> Result { let form = serde_json::to_vec(object).map_err(|error| { Error::other(format!("cannot write the {what} in its serde form")).with_source(error) diff --git a/nvx-rs/src/host/template/clone/records.rs b/nvx-rs/src/host/template/clone/records.rs index 8deacfbdd..03e7a0873 100644 --- a/nvx-rs/src/host/template/clone/records.rs +++ b/nvx-rs/src/host/template/clone/records.rs @@ -974,7 +974,10 @@ fn read_private(path: &Path, description: &str, limit: u64) -> Result Option { let mut log = open_regular(path).ok()?; let length = log.metadata().ok()?.len(); @@ -983,11 +986,37 @@ fn read_fatal(path: &Path) -> Option { let mut tail = Vec::new(); log.take(FATAL_TAIL_BYTES).read_to_end(&mut tail).ok()?; let text = String::from_utf8_lossy(&tail); - let line = text - .lines() - .rev() - .find(|line| line.starts_with(failure::FATAL_PREFIX))?; - Some(line.chars().take(MAX_FATAL_CHARS).collect()) + let lines: Vec<&str> = text.lines().collect(); + let at = lines + .iter() + .rposition(|line| line.starts_with(failure::FATAL_PREFIX))?; + let mut fatal: String = lines[at].chars().take(MAX_FATAL_CHARS).collect(); + if !fatal.contains(failure::FORWARD_BIND_TEXT) { + let forward = lines[at + 1..] + .iter() + .map(|line| cause(line)) + .find(|cause| cause.starts_with(failure::FORWARD_BIND_TEXT)); + if let Some(forward) = forward { + fatal.push_str(": "); + fatal.extend(forward.chars().take(MAX_FATAL_CHARS)); + } + } + Some(fatal) +} + +/// Returns the cause of an error's chain that `line` lists, as OpenVMM writes the chain after the +/// line that leads its fatal error, ` N: CAUSE` for each of several causes, or ` CAUSE` for +/// one alone: without its indentation, and its number, if it has one. +fn cause(line: &str) -> &str { + let line = line.trim_start(); + match line.split_once(": ") { + Some((number, cause)) + if !number.is_empty() && number.bytes().all(|b| b.is_ascii_digit()) => + { + cause + } + _ => line, + } } /// What the sandbox of a clone keeps of it, as [`CloneRecords::read`] reads it: the lease that its @@ -1516,4 +1545,38 @@ mod tests { fs::write(&path, "the VMM exited\nwarning: fatal error: quoted\n").unwrap(); assert_eq!(read_fatal(&path), None); } + + #[test] + fn the_cause_that_names_a_forward_that_openvmm_cannot_bind_follows_the_fatal_line() { + let (_directory, private) = private_dir(); + let path = private.join("openvmm.log"); + let fatal = "fatal error: failed to launch vm worker"; + let forward = "failed to create socket for port forward (Tcp 127.0.0.1:18080)"; + let expected = format!("{fatal}: {forward}"); + // OpenVMM lists the causes of its error's chain after the line, numbered when there are + // several, and alone otherwise. + for chain in [ + format!( + " 0: failed to resolve the network endpoint\n 1: {forward}\n 2: Address \ + already in use (os error 98)\n" + ), + format!(" {forward}\n"), + ] { + let log = format!("a line of the VMM's log\n{fatal}\n\nCaused by:\n{chain}"); + fs::write(&path, &log).unwrap(); + assert_eq!( + read_fatal(&path).as_deref(), + Some(expected.as_str()), + "{log}" + ); + } + // A line that names the forward itself keeps its text alone. + let named = format!("fatal error: {forward}"); + fs::write(&path, format!("{named}\n\nCaused by:\n {forward}\n")).unwrap(); + assert_eq!(read_fatal(&path).as_deref(), Some(named.as_str())); + // A forward that a line before the fatal error names is no cause of it. + let log = format!("warning: {forward}\n{fatal}\n\nCaused by:\n 0: another cause\n"); + fs::write(&path, log).unwrap(); + assert_eq!(read_fatal(&path).as_deref(), Some(fatal)); + } } diff --git a/nvx-rs/src/host/template/clone/tests.rs b/nvx-rs/src/host/template/clone/tests.rs index b61a60197..4897ff725 100644 --- a/nvx-rs/src/host/template/clone/tests.rs +++ b/nvx-rs/src/host/template/clone/tests.rs @@ -6,8 +6,9 @@ mod launch; mod restore; +use std::collections::HashSet; use std::io::Read; -use std::net::{IpAddr, Ipv4Addr}; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use std::time::Duration; use sha2::{Digest, Sha256}; @@ -20,7 +21,8 @@ use crate::activation::{ }; use crate::host::filesystem; use crate::host::network::{ - Access as NetworkAccess, EgressRule, NetworkPolicy, NetworkProxy, ProxyScheme, + Access as NetworkAccess, EgressRule, ForwardProtocol, HostLoopback, LoopbackForward, + NetworkPolicy, NetworkProxy, Protocol, ProxyScheme, }; use crate::host::paths; use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, PolicyVerdict}; @@ -274,6 +276,43 @@ fn network(access: NetworkAccess) -> SandboxNetwork { SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(access)).unwrap() } +/// Returns the fixture of a template with a NIC, at `10.0.0.2/24`, whose class's network allows +/// egress. +fn network_fixture() -> Fixture { + Fixture::with(|class, spec, physical| { + *class = network_class(); + *spec = ManifestSpec::with_network(); + let (distro, scratch) = ( + geometry(IMAGE_LEN, physical), + geometry(SCRATCH_LEN, physical), + ); + set_geometries(class, spec, distro, scratch); + }) +} + +/// Returns the policies of networks that attach a NIC, a template's of each: egress allowed; +/// egress denied but to some destinations; a proxy alone; ingress allowed; and host-loopback +/// access both ways, with a forward. +fn attaching() -> Vec { + let mut ingress = NetworkPolicy::new(NetworkAccess::Deny); + ingress.ingress = NetworkAccess::Allow; + let mut forwarded = NetworkPolicy::new(NetworkAccess::Allow); + forwarded.host_loopback = HostLoopback::Allow; + forwarded.host_loopback_forwards.push(LoopbackForward { + protocol: ForwardProtocol::Tcp, + host_port: 18080, + guest_port: 8080, + }); + vec![ + NetworkPolicy::new(NetworkAccess::Allow), + NetworkPolicy::new(NetworkAccess::Deny) + .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)), + NetworkPolicy::proxy_only(NetworkProxy::new(3128).with_scheme(ProxyScheme::Http)), + ingress, + forwarded, + ] +} + /// Returns the request `request` with the export `export`. fn with_export(request: CloneRequest, export: &ExportPlan) -> CloneRequest { CloneRequest { @@ -1998,7 +2037,7 @@ fn requests_that_no_clone_takes_are_refused_before_anything_is_opened() { } #[test] -fn networks_that_the_template_does_not_have_are_refused() { +fn templates_serve_the_networks_of_every_policy_at_their_nics_address_alone() { let fixture = Fixture::new(); let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); let mut acquired = fixture.acquire("sandbox"); @@ -2023,72 +2062,37 @@ fn networks_that_the_template_does_not_have_are_refused() { let closed = with_network(request, Some(network(NetworkAccess::Deny))); drop(fixture.plan(&mut acquired, &closed, NO_CONTENT).unwrap()); - // A template with a NIC serves the clones that attach its network. - let fixture = Fixture::with(|class, spec, physical| { - *class = network_class(); - *spec = ManifestSpec::with_network(); - let (distro, scratch) = ( - geometry(IMAGE_LEN, physical), - geometry(SCRATCH_LEN, physical), - ); - set_geometries(class, spec, distro, scratch); - }); + // A template with a NIC serves no clone without a network. + let fixture = network_fixture(); let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); let mut acquired = fixture.acquire("sandbox"); let request = tmpfs_request(&image, &binding); let generation = fixture.generation.generation(); - assert_eq!( - refused( - fixture.plan(&mut acquired, &request, NO_CONTENT), - Evidence::InvalidSettings - ), - ( - ErrorKind::Unsupported, - format!( - "the template's generation {generation} has a NIC, whose network every clone of \ - it restores, but the adapter's plan of the clone has none" - ) - ) - ); - assert_untouched(&acquired); - // OpenVMM restores the guest's address on the NIC from the snapshot, so a plan of another - // address, or of another prefix, is refused. - for other in ["10.0.0.3/24", "10.0.0.2/16"] { - let moved = with_network( - request.clone(), - Some(SandboxNetwork::new(other, NetworkPolicy::new(NetworkAccess::Allow)).unwrap()), - ); + let closed = with_network(request.clone(), Some(network(NetworkAccess::Deny))); + for request in [&request, &closed] { assert_eq!( refused( - fixture.plan(&mut acquired, &moved, NO_CONTENT), + fixture.plan(&mut acquired, request, NO_CONTENT), Evidence::InvalidSettings ), ( ErrorKind::Unsupported, format!( - "the adapter's plan of a clone gives its guest the network {other}, but the \ - template's generation {generation} restores its NIC on 10.0.0.2/24, from its \ - snapshot" + "the template's generation {generation} has a NIC, which its clones restore \ + with the network of the adapter's plan, but the plan has none" ) ) ); assert_untouched(&acquired); } - // Nor does it serve the clones of another policy at its address: a template serves the clones - // of its own network alone. - let proxy = NetworkProxy::new(3128).with_scheme(ProxyScheme::Http); - for other in [ - network(NetworkAccess::Allow) - .policy() - .clone() - .with_deny(EgressRule::to("192.0.2.0/24")), - NetworkPolicy::new(NetworkAccess::Deny).with_allow(EgressRule::to("192.0.2.0/24")), - NetworkPolicy::proxy_only(proxy), + // Nor the clones of a network at another address. + for (other, policy) in [ + ("10.0.0.3/24", NetworkPolicy::new(NetworkAccess::Allow)), + ("10.0.0.2/16", NetworkPolicy::new(NetworkAccess::Allow)), ] { - let other = with_network( - request.clone(), - Some(SandboxNetwork::new("10.0.0.2/24", other).unwrap()), - ); + let other = SandboxNetwork::new(other, policy).unwrap(); + let address = other.guest_network().to_owned(); + let other = with_network(request.clone(), Some(other)); assert_eq!( refused( fixture.plan(&mut acquired, &other, NO_CONTENT), @@ -2097,16 +2101,22 @@ fn networks_that_the_template_does_not_have_are_refused() { ( ErrorKind::Unsupported, format!( - "the network of the adapter's plan of a clone is not the one that the \ - template's generation {generation} restores its NIC with: a template serves \ - the clones of its own network policy, at its own guest's address, alone" + "the adapter's plan of a clone attaches a network at {address}, but the NIC \ + of the template's generation {generation} is at 10.0.0.2/24: a template \ + serves the clones of its own guest's address alone" ) ) ); assert_untouched(&acquired); } - let attached = with_network(request, Some(network(NetworkAccess::Allow))); - drop(fixture.plan(&mut acquired, &attached, NO_CONTENT).unwrap()); + // It serves the clones of every policy at its guest's address, whatever the policy of its + // class's network, since each clone's restore replaces the complete deny-all policy of the + // template's NIC with its own. + for policy in attaching() { + let attached = SandboxNetwork::new("10.0.0.2/24", policy).unwrap(); + let attached = with_network(request.clone(), Some(attached)); + drop(fixture.plan(&mut acquired, &attached, NO_CONTENT).unwrap()); + } } #[test] @@ -2380,92 +2390,54 @@ fn the_activation_record_of_a_gpt_clone_names_its_disk_and_the_bound_of_its_firs } #[test] -fn the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_policy_and_dns_server() { - let fixture = Fixture::with(|class, spec, physical| { - *class = network_class(); - *spec = ManifestSpec::with_network(); - let (distro, scratch) = ( - geometry(IMAGE_LEN, physical), - geometry(SCRATCH_LEN, physical), - ); - set_geometries(class, spec, distro, scratch); - }); +fn the_activation_record_of_a_clone_of_a_template_with_a_nic_names_its_own_network() { + let fixture = network_fixture(); let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); let mut acquired = fixture.acquire("sandbox"); - // The policy's identity is the digest that OpenVMM computed, as the template's key records - // it, [7; 32] in this manifest, and the network names no proxy without one. - let attached = with_network( - tmpfs_request(&image, &binding), - Some(network(NetworkAccess::Allow)), - ); - let plan = fixture.plan(&mut acquired, &attached, NO_CONTENT).unwrap(); - let expected = Network { - policy: Identity::from_bytes([7; 32]), - dns: None, - publication: Publication::Unpublished, - proxy: None, - }; - assert_eq!( - plan.activation().configuration().network.as_ref(), - Some(&expected) - ); - drop((plan, acquired, fixture)); - // A proxy's URL is the one that the guest sees, at its gateway, for a clone of a template - // whose policy has the proxy. - let proxy = NetworkProxy::new(3128).with_scheme(ProxyScheme::Http); - let proxied = SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::proxy_only(proxy)).unwrap(); - let fixture = Fixture::with(|class, spec, physical| { - *class = CaptureClass { - network: Some(proxied.clone()), - ..network_class() - }; - *spec = ManifestSpec::with_network(); - let (distro, scratch) = ( - geometry(IMAGE_LEN, physical), - geometry(SCRATCH_LEN, physical), - ); - set_geometries(class, spec, distro, scratch); - }); - let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); - let mut acquired = fixture.acquire("sandbox"); - let proxied = with_network(tmpfs_request(&image, &binding), Some(proxied)); - let plan = fixture.plan(&mut acquired, &proxied, NO_CONTENT).unwrap(); - assert_eq!( - plan.activation().configuration().network, - Some(Network { - proxy: Some("http://10.0.0.1:3128".to_owned()), - ..expected.clone() - }) - ); - drop((plan, acquired, fixture)); - // The DNS server with which the template's guest configured its resolver at boot, as its - // command line asks, is the clone's. - let fixture = Fixture::with(|class, spec, physical| { - *class = network_class(); - *spec = ManifestSpec::with_network(); - spec.contract() - .command_line - .push_str(" virtnet_dns=10.0.0.1"); - let (distro, scratch) = ( - geometry(IMAGE_LEN, physical), - geometry(SCRATCH_LEN, physical), + let gateway = IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1)); + // The template's NIC has the IPv6 gateway fd00::a00:1. + let ipv6_gateway = IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0xa00, 1)); + let mut ipv6_dns = NetworkPolicy::new(NetworkAccess::Deny); + for protocol in [Protocol::Tcp, Protocol::Udp] { + ipv6_dns = ipv6_dns.with_allow(EgressRule::to("fd00::a00:1/128").on_port(protocol, 53)); + } + let mut outbound = NetworkPolicy::new(NetworkAccess::Allow); + outbound.host_loopback = HostLoopback::Outbound; + let mut policies = vec![ipv6_dns, outbound]; + policies.extend(attaching()); + let (unpublished, published) = (Publication::Unpublished, Publication::Published); + let expected = [ + (Some(ipv6_gateway), unpublished, None), + (Some(gateway), unpublished, None), + (Some(gateway), unpublished, None), + (None, unpublished, None), + (None, unpublished, Some("http://10.0.0.1:3128")), + (None, published, None), + (Some(gateway), published, None), + ]; + assert_eq!(policies.len(), expected.len()); + // Each clone's record names its own network, whatever the policy of the template's class: + // the identity of the network, the DNS server that OpenVMM serves the guest at a gateway + // under its policy, whether the guest publishes its listeners, and the proxy's URL as the + // guest sees it, at its gateway. + let mut identities = HashSet::new(); + for (policy, (dns, publication, proxy)) in policies.into_iter().zip(expected) { + let network = SandboxNetwork::new("10.0.0.2/24", policy).unwrap(); + let request = with_network(tmpfs_request(&image, &binding), Some(network.clone())); + let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); + let recorded = plan.activation().configuration().network.clone(); + assert_eq!( + recorded, + Some(Network { + policy: digest_of(b"nvx network policy\0", &network), + dns, + publication, + proxy: proxy.map(str::to_owned), + }), + "{network:?}" ); - set_geometries(class, spec, distro, scratch); - }); - let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); - let mut acquired = fixture.acquire("sandbox"); - let attached = with_network( - tmpfs_request(&image, &binding), - Some(network(NetworkAccess::Allow)), - ); - let plan = fixture.plan(&mut acquired, &attached, NO_CONTENT).unwrap(); - assert_eq!( - plan.activation().configuration().network, - Some(Network { - dns: Some(IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1))), - ..expected - }) - ); + assert!(identities.insert(recorded.unwrap().policy)); + } } #[test] diff --git a/nvx-rs/src/host/template/clone/tests/launch.rs b/nvx-rs/src/host/template/clone/tests/launch.rs index 50fe94aae..f5ec80dc8 100644 --- a/nvx-rs/src/host/template/clone/tests/launch.rs +++ b/nvx-rs/src/host/template/clone/tests/launch.rs @@ -388,6 +388,10 @@ impl VmmDir { /// - `fatal:CODE`, with `exit` or `close`, writes the line with which OpenVMM reports a fatal /// error of the time ABI code `CODE`, and the error's chain, on its standard error, after a /// warning that names another code, before it exits; + /// - `forward:PORT`, with `exit` or `close`, writes the line with which OpenVMM reports a fatal + /// error without a code, and the error's chain, whose second cause names the forward of host + /// loopback's TCP port `PORT` that OpenVMM cannot bind, on its standard error, after a + /// warning that names that forward too, before it exits; /// - `other` sends a version of the event that OpenVMM does not send; /// - `partial` sends the first bytes of the event, then closes its end of the endpoint; /// - `flush` flushes the event as OpenVMM does, which waits on Windows until the host read @@ -467,6 +471,18 @@ fn restore_vmm_role() { ) .unwrap(); } + if let Some(port) = words.iter().find_map(|word| word.strip_prefix("forward:")) { + // OpenVMM gives the error no code, and numbers the causes of its chain. + writeln!( + std::io::stderr(), + "warning: the test's forward (Tcp 127.0.0.1:{port}) is not yet bound\n\ + fatal error: failed to launch vm worker\n\nCaused by:\n \ + 0: failed to resolve the test's network endpoint\n \ + 1: failed to create socket for port forward (Tcp 127.0.0.1:{port})\n \ + 2: the test's port is held" + ) + .unwrap(); + } write_report(&words, &arguments); return; } @@ -1000,24 +1016,25 @@ fn the_control_socket_of_a_clone_goes_once_its_vm_has_ended_and_for_good_with_it } #[test] -fn a_clone_of_a_template_with_a_nic_restores_its_policy_without_its_address() { - let fixture = Fixture::with(|class, spec, physical| { - *class = network_class(); - *spec = ManifestSpec::with_network(); - let (distro, scratch) = ( - geometry(IMAGE_LEN, physical), - geometry(SCRATCH_LEN, physical), - ); - set_geometries(class, spec, distro, scratch); - }); +fn a_clone_of_a_template_with_a_nic_restores_its_own_policy_in_place_of_the_templates() { + let fixture = network_fixture(); complete_snapshot(&fixture); let (image, binding) = fixture.image("distro.erofs", IMAGE_LEN); let mut acquired = fixture.acquire("sandbox"); - let attached = network(NetworkAccess::Allow); + // The class's network allows egress; the clone's denies it but to one destination, and + // publishes the guest's listeners on host loopback, with a forward. + let mut policy = NetworkPolicy::new(NetworkAccess::Deny) + .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)); + policy.host_loopback = HostLoopback::Allow; + policy.host_loopback_forwards.push(LoopbackForward { + protocol: ForwardProtocol::Tcp, + host_port: 18080, + guest_port: 8080, + }); + let attached = SandboxNetwork::new("10.0.0.2/24", policy).unwrap(); let request = with_network(tmpfs_request(&image, &binding), Some(attached.clone())); let plan = fixture.plan(&mut acquired, &request, NO_CONTENT).unwrap(); let runtime = plan.key().runtime().clone(); - let template = plan.key().network_arguments().to_vec(); let mut vmm = VmmDir::new(); let dir = vmm.path().to_owned(); let made = RefCell::new(None); @@ -1030,21 +1047,20 @@ fn a_clone_of_a_template_with_a_nic_restores_its_policy_without_its_address() { .launch_with(&mut acquired, Path::new(OPENVMM), &runtime, hooks) .unwrap(); vmm.reference = Some(restoring.starting().process().reference().clone()); - // The devices end with the template's network, as its key records it, the arguments of the - // network's policy, without the guest's address, which OpenVMM takes from the snapshot, and - // refuses beside it: after the image, and the clone's scratch, which its upper in RAM - // declares absent. - let mut policy = attached.openvmm_arguments().unwrap(); - assert_eq!(template, policy); - assert_eq!(policy[..2], ["--net", "10.0.0.2/24"]); - policy.drain(..2); + // After the image, the devices end with `--restore-network-policy` and the options of the + // clone's own policy, without `--net` and the guest's address, which OpenVMM takes from the + // snapshot, and refuses beside it, then with `--restore-absent-scratch`, which declares the + // scratch of the clone's upper in RAM absent. + let mut expected = attached.openvmm_arguments().unwrap(); + assert_eq!(expected[..2], ["--net", "10.0.0.2/24"]); + expected.splice(..2, ["--restore-network-policy".to_owned()]); + expected.push("--restore-absent-scratch".to_owned()); let arguments = made.take().unwrap().arguments; let devices = arguments .iter() .position(|argument| argument == "--single-process") .unwrap(); - assert_eq!(arguments[4], "--restore-absent-scratch"); - assert_eq!(arguments[5..devices], policy); + assert_eq!(arguments[4..devices], expected); assert!(!arguments.iter().any(|argument| argument == "--net")); } diff --git a/nvx-rs/src/host/template/clone/tests/restore.rs b/nvx-rs/src/host/template/clone/tests/restore.rs index c33c19030..24823fce2 100644 --- a/nvx-rs/src/host/template/clone/tests/restore.rs +++ b/nvx-rs/src/host/template/clone/tests/restore.rs @@ -913,8 +913,9 @@ fn a_vmm_that_exits_or_sends_anything_but_its_event_ends_the_launch() { let (fixture, image) = fixture(); // A VMM that exits before its event, as OpenVMM does once it cannot restore the snapshot, ends // the wait at once, and loses the clone's status, unless OpenVMM's fatal error tells that the - // host cannot present the template's CPU or time; another version of the event, and part of - // it, are refused, and the VMM that sent them is terminated. + // host cannot present the template's CPU or time, or bind a forward of the clone's network; + // another version of the event, and part of it, are refused, and the VMM that sent them is + // terminated. let cases = [ ( "exit", @@ -947,6 +948,15 @@ fn a_vmm_that_exits_or_sends_anything_but_its_event_ends_the_launch() { Evidence::IncompatibleHost, Teardown::Exited, ), + // The launch of its VM worker fails too where OpenVMM cannot bind a forward of the + // clone's network as it restores the clone's NIC, because another socket holds its port. + ( + "forward", + "close forward:18080", + ErrorKind::Exited, + Evidence::ForwardBind, + Teardown::Exited, + ), ( "other", "other flush", @@ -975,15 +985,25 @@ fn a_vmm_that_exits_or_sends_anything_but_its_event_ends_the_launch() { assert!(launched.before.elapsed() < TIMEOUT, "{plan}"); assert!(control.calls.is_empty(), "{plan}"); assert_failed(&error, &acquired, restore, evidence, teardown); - // The record keeps the line of OpenVMM's fatal error, and not the warning before it. - let fatal = plan - .split_whitespace() - .find_map(|word| word.strip_prefix("fatal:")); - if let Some(code) = fatal { - let line = - format!("OpenVMM reported: fatal error: [{code}] failed to launch vm worker"); + // The record keeps the line of OpenVMM's fatal error, and not the warning before it, with + // the cause of the error's chain that names the forward that OpenVMM cannot bind. + let line = plan.split_whitespace().find_map(|word| { + if let Some(code) = word.strip_prefix("fatal:") { + Some(format!("[{code}] failed to launch vm worker")) + } else { + word.strip_prefix("forward:").map(|port| { + format!( + "failed to launch vm worker: failed to create socket for port forward \ + (Tcp 127.0.0.1:{port})" + ) + }) + } + }); + if let Some(line) = line { + let line = format!("OpenVMM reported: fatal error: {line}"); assert!(error.message().ends_with(&line), "{error}"); assert!(error.record().message.ends_with(&line), "{error}"); + assert_eq!(error.outcome(), evidence.outcome(), "{error}"); } assert_launch_ended(&fixture, &mut acquired, &vmm); } diff --git a/nvx-rs/src/host/template/key.rs b/nvx-rs/src/host/template/key.rs index 056826faa..ec3e4f6f3 100644 --- a/nvx-rs/src/host/template/key.rs +++ b/nvx-rs/src/host/template/key.rs @@ -3,7 +3,7 @@ use std::collections::BTreeMap; use std::fmt; -use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; +use std::net::{Ipv4Addr, Ipv6Addr}; use sha2::{Digest, Sha256}; @@ -18,10 +18,6 @@ use super::manifest::{ use crate::activation; use crate::contract; use crate::error::{Error, Result}; -use crate::host::network::{ - Access, EgressRule, HostLoopback, LoopbackForward, MAX_LOOPBACK_FORWARDS, MAX_RULES, - NetworkPolicy, NetworkProxy, -}; use crate::host::platform::parse_hex32; use crate::host::sandbox::SandboxNetwork; use crate::host::sandbox::mount::MOUNT_TAG; @@ -38,16 +34,6 @@ const CLASS_DOMAIN: &[u8] = b"nvx capture class\0"; /// The generation of a CPU profile that a host derives, rather than a built-in one. const HOST_GENERATION: &str = "host"; -/// The prefix of the command line's token that names the DNS server of the guest's resolver, -/// which OpenVMM writes once at most, with an IP address in its canonical spelling. -const DNS_TOKEN: &str = "virtnet_dns="; - -/// Most texts of a key's network: the options of OpenVMM that a network lowers to, each with its -/// value, as [`SandboxNetwork::openvmm_arguments`] renders them. A network has at most six options -/// of its own, from `--net` to `--network-proxy`, at most [`MAX_RULES`] rules of each of the two -/// actions of its egress, and at most [`MAX_LOOPBACK_FORWARDS`] host-loopback forwards. -const MAX_NETWORK_ARGUMENTS: usize = 2 * (6 + 2 * MAX_RULES + MAX_LOOPBACK_FORWARDS); - /// The versions of the contracts of a template's clones that its runtime identity does not /// record: the snapshot format, the saved device state, the time ABI, the restore packet, and the /// activation record. @@ -421,8 +407,12 @@ pub struct CaptureClass { /// The dormant scratch device. pub scratch: BlockGeometry, /// The network: `None` or a network that OpenVMM attaches no device for, as - /// [`SandboxNetwork::openvmm_arguments`] lowers it, for a template without a NIC, and otherwise - /// the network of its fixed NIC and the policy that OpenVMM enforces there. + /// [`SandboxNetwork::openvmm_arguments`] lowers it, for a template without a NIC, and + /// otherwise a network at the address of the template's fixed NIC. Of the network, the class + /// takes only what the template captures: its NIC, at the guest's address, under the + /// complete deny-all policy, as [`SandboxNetwork::template_arguments`] lowers it, whatever + /// the network's own policy, which each clone's restore replaces with its own. So the classes + /// of two networks at one address, whose policies both attach a NIC, are one. pub network: Option, /// The invariant settings beyond the machine. pub invariants: Invariants, @@ -468,11 +458,12 @@ impl CaptureClass { Ok(()) } - /// Returns OpenVMM's arguments of the network, none without a NIC. + /// Returns OpenVMM's arguments of the template's NIC, as + /// [`SandboxNetwork::template_arguments`] lowers the network, none without a NIC. fn network_arguments(&self) -> Result> { match &self.network { None => Ok(Vec::new()), - Some(network) => network.openvmm_arguments(), + Some(network) => network.template_arguments(), } } @@ -486,10 +477,10 @@ impl CaptureClass { /// and activation contract versions as `u64`s; the backend, as OpenVMM spells it, and the /// CPU profile, as texts; the processors and those online at boot as `u32`s; the RAM and its /// capacity as `u64`s; the `distro` and scratch geometries, each a `u64` length and two - /// `u32` block sizes; the network, as the list of the texts of OpenVMM's arguments that - /// [`SandboxNetwork::openvmm_arguments`] lowers it to, empty without a NIC; and the invariant - /// settings, as a list of names and values, each a text, in the order of their names. The - /// [module](super) describes the encoding of each kind of value. + /// `u32` block sizes; the network, as the list of the texts of OpenVMM's arguments of the + /// template's NIC, which [`SandboxNetwork::template_arguments`] lowers it to, empty without a + /// NIC; and the invariant settings, as a list of names and values, each a text, in the order + /// of their names. The [module](super) describes the encoding of each kind of value. /// /// Fails as [`check`](Self::check) does. pub fn to_bytes(&self) -> Result> { @@ -549,20 +540,16 @@ impl CaptureClass { /// interrupt, features, and queues, and the interrupt delivery of the virtio devices; the /// filesystem slot's version and tag; whether there is a NIC, and the fixed NIC's profile, MAC /// addresses, addresses, prefixes, and gateways, with the identity of the network policy that -/// OpenVMM enforces there; and the template's [`Invariants`], its other invariant isolation and -/// resource settings. -/// - **Network:** the template's network as its class lowers it to OpenVMM's arguments, -/// [`SandboxNetwork::openvmm_arguments`], none for a template without a NIC: the guest's -/// address and the network policy as this crate writes them, which a clone's plan compares with -/// the network of the clone's own plan, and which every restore of the template passes -/// unchanged. A template serves the clones of its network policy alone, so a host keeps one -/// template of each policy that its clones take. +/// OpenVMM enforced there at the capture, the complete deny-all policy, which every restore of +/// the template replaces with its clone's own; and the template's [`Invariants`], its other +/// invariant isolation and resource settings. /// /// A capture's own metadata is not part of it, so the generations of one key differ in it: when /// the capture was taken, its time contract's anchor, host, boot, and lineage counter, and the /// lengths of its artifacts. [`TemplateManifest`] keeps them. Nothing of a clone is part of it -/// either: its image, upper, export, the network settings that its activation record names, -/// identity, and activation targets. +/// either: its image, upper, export, network policy and settings, identity, and activation +/// targets. So a template with a NIC serves the clones of every network policy at its guest's +/// address. /// /// The key comes from metadata alone: the artifacts' identities that their install metadata /// records, the class's inputs, and the capture's manifest. It never reads or digests a binary, @@ -585,22 +572,20 @@ pub struct TemplateKey { filesystem_tag: String, /// The invariant settings beyond the machine. invariants: Invariants, - /// The network, as the class lowered it to OpenVMM's arguments: empty without a NIC. - network: Vec, } impl TemplateKey { /// Version of the canonical encoding of a key, its first two bytes. /// - /// Version 1 holds the template's network. It has not been released, so a template store that - /// an earlier build created is not supported: delete it and create a new one. There is no - /// migration. + /// Version 1 holds the template's NIC, if it has one, with the identity of the complete + /// deny-all policy under which the capture attached it, but no network policy of its clones. + /// It has not been released, so a template store that an earlier build created is not + /// supported: delete it and create a new one. There is no migration. pub const VERSION: u16 = 1; /// Most bytes of a key's canonical encoding: what the largest key that a manifest of at most - /// [`MAX_MANIFEST_LEN`](super::MAX_MANIFEST_LEN) bytes can describe takes, with the most - /// arguments that a network lowers to, with room to spare. - pub const MAX_LEN: usize = 576 * 1024; + /// [`MAX_MANIFEST_LEN`](super::MAX_MANIFEST_LEN) bytes can describe takes, with room to spare. + pub const MAX_LEN: usize = 256 * 1024; /// Returns the key of the template that a capture of `class` made, whose manifest is /// `manifest`. @@ -610,15 +595,16 @@ impl TemplateKey { /// NIC, and the NIC's address and prefix, and a CPU profile that the class's request selects. /// A manifest whose machine has no processors online at boot activates all of them at boot, /// which a class whose processors are all online at boot requests. The key takes the class's - /// runtime, guest build, invariant settings, and network, as the class lowers it to OpenVMM's - /// arguments, and this crate's versions of the other contracts. + /// runtime, guest build, and invariant settings, and this crate's versions of the other + /// contracts. /// /// The key also takes the identity of the network policy that OpenVMM enforced on the NIC, - /// which this check leaves to OpenVMM: OpenVMM derives it from the network arguments of the - /// class by an encoding of its own, which this crate does not reproduce. A capture that - /// enforced another policy than its class requested makes a key of its own, which no restore - /// with the class's network arguments serves, since OpenVMM refuses a restore whose network, - /// its policy included, is not the one that the snapshot records. + /// which this check leaves to OpenVMM: OpenVMM derives it from the arguments of the + /// template's NIC, under the complete deny-all policy, as + /// [`SandboxNetwork::template_arguments`] lowers the class's network, by an encoding of its + /// own, which this crate does not reproduce. A capture that enforced another policy makes a + /// key of its own, which no restore serves, since OpenVMM lets a restore replace the complete + /// deny-all policy alone. /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) as /// [`CaptureClass::check`] does, and with [`ErrorKind::Io`](crate::ErrorKind::Io) for a @@ -687,11 +673,11 @@ impl TemplateKey { ); } } - let network = class.network_arguments()?; + let attached = !class.network_arguments()?.is_empty(); let requested_network = class .network .as_ref() - .filter(|_| !network.is_empty()) + .filter(|_| attached) .map(|network| network.guest_network().to_owned()); // A network that `SandboxNetwork` accepts has the canonical form that this writes. let found_network = machine.nic.as_ref().map(guest_network_of); @@ -709,7 +695,6 @@ impl TemplateKey { machine: machine.clone(), filesystem_tag: MOUNT_TAG.to_owned(), invariants: class.invariants.clone(), - network, }) } @@ -793,65 +778,21 @@ impl TemplateKey { /// Returns the guest's network on the template's NIC, its IPv4 address and prefix length, in /// the canonical form of a [`SandboxNetwork`], such as `10.0.0.2/24`, which OpenVMM restores - /// from the snapshot with the NIC, or `None` for a template without a NIC. - pub(crate) fn guest_network(&self) -> Option { + /// from the snapshot with the NIC, or `None` for a template without a NIC. The template serves + /// the clones of every network policy at that address. + pub fn guest_network(&self) -> Option { self.machine.nic.as_ref().map(guest_network_of) } - /// Returns the template's network as its class lowered it to OpenVMM's arguments, - /// [`SandboxNetwork::openvmm_arguments`]: `--net` with the guest's address, and the options - /// that enforce the policy of the template's NIC, which every restore of the template passes, - /// but `--net` and the address, which OpenVMM takes from the snapshot; none for a template - /// without a NIC. A clone's plan requires the same arguments of the network of the clone's own - /// plan, so the template serves the clones of its network policy alone. - pub fn network_arguments(&self) -> &[String] { - &self.network - } - - /// Returns the SHA-256 digest of the canonical encoding of the network policy that OpenVMM - /// enforces on the template's NIC, which OpenVMM computed and the manifest records, or `None` - /// for a template without a NIC. - pub(crate) fn network_policy_digest(&self) -> Option<[u8; 32]> { - self.machine.nic.as_ref().map(|nic| nic.policy.digest) - } - - /// Returns the address of the DNS server with which the template's guest configured its - /// resolver at boot: the one that the `virtnet_dns=` token of the command line names, which - /// OpenVMM adds when the policy of the template's NIC lets the guest reach the gateway's DNS - /// server, or `None` for a command line without the token. - /// - /// Fails with [`ErrorKind::Io`](crate::ErrorKind::Io) for a command line that OpenVMM does - /// not write: one with more than one such token, with a token for a template without a NIC, - /// or with a token that does not name one IP address in its canonical spelling. - pub(crate) fn dns_server(&self) -> Result> { - let mut tokens = self - .machine - .command_line - .split_ascii_whitespace() - .filter_map(|token| token.strip_prefix(DNS_TOKEN)); - let Some(token) = tokens.next() else { - return Ok(None); - }; - let malformed = |problem: &str| { - Err(Error::io(format!( - "the command line of the template's key is malformed: {problem}" - ))) - }; - if tokens.next().is_some() { - return malformed(&format!("it holds more than one {DNS_TOKEN} token")); - } - if self.machine.nic.is_none() { - return malformed(&format!( - "it names a DNS server, {token:?}, for a template without a NIC" - )); - } - match token.parse::() { - Ok(server) if server.to_string() == token => Ok(Some(server)), - _ => malformed(&format!( - "its {DNS_TOKEN} token names {token:?}, which is not an IP address in its \ - canonical spelling" - )), - } + /// Returns the gateway of the IPv6 identity of the template's NIC, or `None` for a template + /// without a NIC or whose NIC has no IPv6 identity. + pub(crate) fn ipv6_gateway(&self) -> Option { + self.machine + .nic + .as_ref()? + .ipv6 + .as_ref() + .map(|ipv6| ipv6.gateway) } /// Returns the tag of the filesystem slot, on which a clone's export is bound. @@ -899,9 +840,6 @@ impl TemplateKey { /// the network policy's mode, a text, digest, 32 bytes, whether it is required, a boolean, /// and encoding version, a `u32`; and the invariant settings, a list of names and values, /// each a text, in the order of their names. - /// 8. Network: the texts of OpenVMM's arguments to which the class lowered the network, a list, - /// empty for a template without a NIC, and otherwise starting with `--net` and the NIC's - /// IPv4 address and prefix length, `ADDRESS/PREFIX`. pub fn to_bytes(&self) -> Vec { let mut writer = Writer::default(); writer.u16(Self::VERSION); @@ -910,16 +848,13 @@ impl TemplateKey { self.contracts.write(&mut writer); write_machine(&mut writer, &self.machine, &self.filesystem_tag); self.invariants.write(&mut writer); - writer.list(&self.network, |writer, argument| writer.text(argument)); writer.finish() } /// Reads a key from its canonical encoding, as [`to_bytes`](Self::to_bytes) writes it. /// /// The bytes must be the key's one encoding, of version [`VERSION`](Self::VERSION), with no - /// value of another form than a key derived from a manifest has, a network that is the - /// arguments to which a network of a guest at the address of the machine's NIC lowers, if it - /// has one, and none otherwise, and no byte left over. + /// value of another form than a key derived from a manifest has, and no byte left over. /// /// Fails with [`ErrorCode::FormatUnsupported`](crate::ErrorCode::FormatUnsupported) for a key /// of another version, and with [`ErrorKind::Io`](crate::ErrorKind::Io) for bytes that are @@ -952,12 +887,6 @@ impl TemplateKey { let contracts = Contracts::read(reader)?; let (machine, filesystem_tag) = read_machine(reader)?; let invariants = Invariants::read(reader)?; - let network = reader.list(MAX_NETWORK_ARGUMENTS, "the network", |reader| { - Ok(reader - .text(MAX_TEXT_LEN, "an argument of the network")? - .to_owned()) - })?; - check_network(machine.nic.as_ref(), &network)?; Ok(Self { runtime, guest_build, @@ -965,7 +894,6 @@ impl TemplateKey { machine, filesystem_tag, invariants, - network, }) }; let key = read(&mut reader).map_err(malformed)?; @@ -1217,94 +1145,5 @@ fn read_machine(reader: &mut Reader<'_>) -> Result<(Machine, String), CanonicalE Ok((machine, tag)) } -/// Checks that `network`, the texts of OpenVMM's arguments of a key's network, are those of a -/// network of the NIC `nic` of the key's machine, or none for a machine without a NIC: exactly the -/// arguments to which [`SandboxNetwork::openvmm_arguments`] lowers a network of a guest at the -/// NIC's address and prefix length, which start with `--net` and that address. -/// -/// The check reads the policy back from the arguments, as `nvx-rs sandbox` reads its network -/// options, and lowers it again with its NIC attached, which renders a lowering unchanged: a -/// policy whose allow rules lower to no OpenVMM rule attaches its NIC even if it lets nothing -/// through, and reads back without them. A key whose network no policy lowers to is thus -/// malformed, rather than the network of a template that every clone's plan mismatches. -fn check_network(nic: Option<&Nic>, network: &[String]) -> Result<(), CanonicalError> { - let Some(nic) = nic else { - return if network.is_empty() { - Ok(()) - } else { - Err(CanonicalError::new( - "the network attaches a NIC that the machine does not have", - )) - }; - }; - let address = guest_network_of(nic); - match network { - [option, attached, ..] if option == "--net" && *attached == address => {} - _ => { - return Err(CanonicalError::new(format!( - "the network does not attach the machine's NIC at {address}" - ))); - } - } - let lowered = crate::net::guest_gateway(&address) - .and_then(|gateway| read_policy(&network[2..], gateway)) - .and_then(|policy| crate::host::network::attached_arguments(&policy, &address).ok()); - if lowered.as_deref() == Some(network) { - Ok(()) - } else { - Err(CanonicalError::new(format!( - "the network is not the arguments to which a network policy of a guest at {address} \ - lowers" - ))) - } -} - -/// Reads `arguments`, the options of OpenVMM that follow a network's `--net` and its value, with -/// their values, for a guest whose gateway is `gateway`, into the policy that they name, as -/// `nvx-rs sandbox` reads its network options: the rules of each action as OpenVMM reads its -/// native rules, and, without `--host-loopback`, the host-loopback access that OpenVMM keeps by -/// default. Returns `None` for an option that no lowering writes, or a value that it does not -/// take; the caller compares the policy's lowering with the arguments. -fn read_policy(arguments: &[String], gateway: Ipv4Addr) -> Option { - let options = arguments.chunks_exact(2); - if !options.remainder().is_empty() { - return None; - } - let access = |value: &str| match value { - "allow" => Some(Access::Allow), - "deny" => Some(Access::Deny), - _ => None, - }; - let mut policy = NetworkPolicy::new(Access::Deny); - policy.host_loopback = HostLoopback::Outbound; - for option in options { - let [name, value] = option else { - return None; - }; - match name.as_str() { - "--network-profile" => {} - "--network-egress" => policy.egress = access(value)?, - "--network-ingress" => policy.ingress = access(value)?, - "--host-loopback" => policy.host_loopback = access(value)?.into(), - "--network-egress-allow" => policy - .allow - .rules - .push(EgressRule::parse_native(value).ok()?), - "--network-egress-deny" => policy - .deny - .rules - .push(EgressRule::parse_native(value).ok()?), - "--host-loopback-forward" => policy - .host_loopback_forwards - .push(LoopbackForward::parse(value).ok()?), - "--network-proxy" => { - policy.proxy = Some(NetworkProxy::parse_endpoint(value, gateway).ok()?); - } - _ => return None, - } - } - Some(policy) -} - #[cfg(test)] mod tests; diff --git a/nvx-rs/src/host/template/key/tests.rs b/nvx-rs/src/host/template/key/tests.rs index adaf02420..20352d027 100644 --- a/nvx-rs/src/host/template/key/tests.rs +++ b/nvx-rs/src/host/template/key/tests.rs @@ -5,10 +5,7 @@ use std::collections::HashSet; use super::*; use crate::ErrorCode; -use crate::host::network::{ - Access, Destination, EgressRule, ForwardProtocol, NetworkPolicy, NetworkProxy, Protocol, - ProxyScheme, -}; +use crate::host::network::{Access, EgressRule, HostLoopback, NetworkPolicy, NetworkProxy}; use crate::host::template::test_support::{ self as support, MIB, ManifestSpec, artifact, network_class, }; @@ -245,18 +242,6 @@ fn each_part_of_the_key_changes_it() { ("policy encoding", |_, spec| { spec.network().policy_encoding_version = 3; }), - // The network, as the class lowered it: another policy makes another key, even where the - // manifest is the same, as OpenVMM's identity of the policy would not be. - ("egress rule", |class, _| { - let policy = - NetworkPolicy::new(Access::Allow).with_deny(EgressRule::to("192.0.2.0/24")); - class.network = Some(SandboxNetwork::new("10.0.0.2/24", policy).unwrap()); - }), - ("egress default", |class, _| { - let policy = - NetworkPolicy::new(Access::Deny).with_allow(EgressRule::to("192.0.2.0/24")); - class.network = Some(SandboxNetwork::new("10.0.0.2/24", policy).unwrap()); - }), ("invariant setting", |class, _| { class.invariants = Invariants::new() .with("isolation.seccomp", "strict") @@ -375,134 +360,25 @@ fn a_key_holds_what_its_capture_made() { (class.distro, class.scratch) ); assert!(made.has_network()); - assert!(!key(&support::class(), &ManifestSpec::template()).has_network()); - assert_eq!(made.filesystem_tag(), "microvm"); - assert_eq!(made.invariants(), &Invariants::new()); - assert_eq!( - made.network_arguments(), - class.network.unwrap().openvmm_arguments().unwrap() - ); -} - -#[test] -fn a_key_holds_the_network_that_its_class_lowered() { - // A NIC's network attaches the guest at its address, then enforces its policy. - assert_eq!( - network_key().network_arguments(), - [ - "--net", - "10.0.0.2/24", - "--network-profile", - "portable", - "--network-egress", - "allow", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - ] - ); - let proxy = NetworkProxy::new(3128).with_scheme(ProxyScheme::Http); - let proxied = CaptureClass { - network: Some( - SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::proxy_only(proxy)).unwrap(), - ), - ..network_class() - }; - let made = key(&proxied, &ManifestSpec::with_network()); - assert_eq!(made.network_arguments()[..2], ["--net", "10.0.0.2/24"]); + assert_eq!(made.guest_network().as_deref(), Some("10.0.0.2/24")); assert_eq!( - made.network_arguments()[made.network_arguments().len() - 2..], - ["--network-proxy", "10.0.0.1:3128"] + made.ipv6_gateway(), + Some(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0xa00, 1)) ); - assert_ne!(made.identity(), network_key().identity()); - // A template without a NIC holds no network, the identity of the network that lets nothing - // through, whether its class has no network or one that attaches no NIC. - let none = key(&support::class(), &ManifestSpec::template()); - assert!(none.network_arguments().is_empty()); - let closed = CaptureClass { - network: Some( - SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(Access::Deny)).unwrap(), - ), - ..support::class() - }; - assert_eq!(key(&closed, &ManifestSpec::template()), none); - assert_eq!( - closed.identity().unwrap(), - support::class().identity().unwrap() - ); - // Each key reads back with its network. - for made in [network_key(), made, none] { - assert_eq!(TemplateKey::from_bytes(&made.to_bytes()).unwrap(), made); - } -} - -#[test] -fn a_key_names_the_dns_server_that_its_command_line_names() { - // Neither manifest of the tests names a DNS server. - assert_eq!(network_key().dns_server().unwrap(), None); let template = key(&support::class(), &ManifestSpec::template()); - assert_eq!(template.dns_server().unwrap(), None); - let with_tokens = |class: CaptureClass, mut spec: ManifestSpec, tokens: &str| { - spec.contract().command_line.push_str(tokens); - key(&class, &spec) - }; - let with_dns = |tokens| with_tokens(network_class(), ManifestSpec::with_network(), tokens); - for (tokens, server) in [ - ( - " virtnet_dns=10.0.0.1", - IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1)), - ), - ( - " virtnet_dns=fd00::a00:1", - IpAddr::V6(Ipv6Addr::new(0xfd00, 0, 0, 0, 0, 0, 0xa00, 1)), - ), - ] { - assert_eq!(with_dns(tokens).dns_server().unwrap(), Some(server)); - } - // OpenVMM writes one token at most, with one address in its canonical spelling, and only - // for a NIC. - let noncanonical = |value: &str| { - format!( - "its virtnet_dns= token names {value:?}, which is not an IP address in its canonical \ - spelling" - ) - }; - let without_nic = with_tokens( - support::class(), - ManifestSpec::template(), - " virtnet_dns=10.0.0.1", + assert!(!template.has_network()); + assert_eq!( + (template.guest_network(), template.ipv6_gateway()), + (None, None) ); - for (made, problem) in [ - ( - with_dns(" virtnet_dns=10.0.0.1 virtnet_dns=10.0.0.1"), - "it holds more than one virtnet_dns= token".to_owned(), - ), - (with_dns(" virtnet_dns="), noncanonical("")), - ( - with_dns(" virtnet_dns=010.0.0.1"), - noncanonical("010.0.0.1"), - ), - ( - with_dns(" virtnet_dns=fd00:0::a00:1"), - noncanonical("fd00:0::a00:1"), - ), - ( - with_dns(" virtnet_dns=10.0.0.1,fd00::a00:1"), - noncanonical("10.0.0.1,fd00::a00:1"), - ), - ( - without_nic, - "it names a DNS server, \"10.0.0.1\", for a template without a NIC".to_owned(), - ), - ] { - let error = made.dns_server().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert_eq!( - error.message(), - format!("the command line of the template's key is malformed: {problem}") - ); - } + let mut spec = ManifestSpec::with_network(); + let network = spec.network(); + network.ipv6.clear(); + network.ipv6_prefix_length = 0; + network.ipv6_gateway.clear(); + assert_eq!(key(&network_class(), &spec).ipv6_gateway(), None); + assert_eq!(made.filesystem_tag(), "microvm"); + assert_eq!(made.invariants(), &Invariants::new()); } /// Asserts that the capture of `class` whose manifest is `spec` does not match its class, with @@ -711,9 +587,8 @@ fn each_field_of_a_class_changes_its_identity() { }, |class| class.network = None, |class| { - let policy = - NetworkPolicy::new(Access::Allow).with_deny(EgressRule::to("192.0.2.0/24")); - class.network = Some(SandboxNetwork::new("10.0.0.2/24", policy).unwrap()); + let policy = NetworkPolicy::new(Access::Allow); + class.network = Some(SandboxNetwork::new("10.0.0.3/24", policy).unwrap()); }, |class| { class.invariants = Invariants::new() @@ -738,6 +613,25 @@ fn each_field_of_a_class_changes_its_identity() { let mut none = network_class(); none.network = None; assert_eq!(denied.identity().unwrap(), none.identity().unwrap()); + // The template captures its NIC under the complete deny-all policy, which each clone's restore + // replaces, so the policy of a network that attaches one is no part of its class. + for policy in [ + NetworkPolicy::new(Access::Allow).with_deny(EgressRule::to("192.0.2.0/24")), + NetworkPolicy::new(Access::Deny).with_allow(EgressRule::to("192.0.2.0/24")), + NetworkPolicy { + ingress: Access::Allow, + ..NetworkPolicy::new(Access::Deny) + }, + NetworkPolicy { + host_loopback: HostLoopback::Allow, + ..NetworkPolicy::new(Access::Deny) + }, + NetworkPolicy::proxy_only(NetworkProxy::new(3128)), + ] { + let mut class = network_class(); + class.network = Some(SandboxNetwork::new("10.0.0.2/24", policy).unwrap()); + assert_eq!(class.identity().unwrap(), base, "{:?}", class.network); + } // The identity digests the canonical encoding after the class's domain. let class = network_class(); let mut digest = Sha256::new(); @@ -779,15 +673,6 @@ fn key_refusal(bytes: &[u8], code: ErrorCode) -> String { #[test] fn other_encodings_of_a_key_are_refused() { - // Every key ends with its network, a list that is empty for a template without a NIC: bytes - // without that list decode as no key, and as no key of a template without a NIC either. - let closed = key(&support::class(), &ManifestSpec::template()).to_bytes(); - let (unlisted, count) = closed.split_at(closed.len() - 4); - assert_eq!(count, [0; 4]); - assert_eq!( - key_refusal(unlisted, ErrorCode::Io), - "the template key is malformed: the network is truncated" - ); let key = network_key(); let bytes = key.to_bytes(); for version in [0u16, 2, u16::MAX] { @@ -818,13 +703,41 @@ fn other_encodings_of_a_key_are_refused() { assert!(key_refusal(&oversized, ErrorCode::Io).contains("is longer than the")); } +#[test] +fn a_key_that_holds_its_network_s_lowering_after_its_invariants_is_refused() { + // A build that kept each network policy's template apart wrote version 1 with one more part: + // OpenVMM's arguments to which the class lowered the network, a list of texts after the + // invariant settings, empty for a template without a NIC. Its keys are malformed here, with + // or without a NIC, rather than read without that part, so that a store that it created is + // never misread. + for (made, lowered) in [ + ( + network_key(), + vec!["--net", "10.0.0.2/24", "--network-profile", "portable"], + ), + ( + key(&support::class(), &ManifestSpec::template()), + Vec::new(), + ), + ] { + let mut writer = Writer::default(); + writer.list(&lowered, |writer, argument| writer.text(argument)); + let part = writer.finish(); + let bytes = [made.to_bytes(), part.clone()].concat(); + assert!(part.len() > 1, "{part:?}"); + assert_eq!( + key_refusal(&bytes, ErrorCode::Io), + format!( + "the template key is malformed: {} bytes follow the value", + part.len() + ) + ); + } +} + #[test] fn keys_of_machines_that_no_manifest_describes_are_refused() { type KeyChange = fn(&mut TemplateKey); - // A network that no policy of a guest at the NIC's address lowers to, though it attaches the - // NIC: every clone's plan would mismatch it, whatever its network. - const UNLOWERED: &str = "the network is not the arguments to which a network policy of a \ - guest at 10.0.0.2/24 lowers"; let cases: &[(KeyChange, &str)] = &[ ( |key| key.machine.command_line.push_str(" quiet"), @@ -858,75 +771,6 @@ fn keys_of_machines_that_no_manifest_describes_are_refused() { |key| key.machine.cpu.revision = 0, "the machine has the invalid CPU profile vendor", ), - ( - |key| key.network.clear(), - "the network does not attach the machine's NIC at 10.0.0.2/24", - ), - ( - |key| key.network[1] = "10.0.0.3/24".to_owned(), - "the network does not attach the machine's NIC at 10.0.0.2/24", - ), - ( - |key| drop(key.network.drain(..2)), - "the network does not attach the machine's NIC at 10.0.0.2/24", - ), - // The address alone. - (|key| key.network.truncate(2), UNLOWERED), - // Ingress before egress, which no lowering orders so. - ( - |key| { - key.network.swap(4, 6); - key.network.swap(5, 7); - }, - UNLOWERED, - ), - // Another profile than OpenVMM's portable one. - (|key| key.network[3] = "strict".to_owned(), UNLOWERED), - // A rule that OpenVMM would not read. - ( - |key| { - key.network - .extend(["--network-egress-allow", "nowhere"].map(str::to_owned)); - }, - UNLOWERED, - ), - // A rule that a lowering would write in another form. - ( - |key| { - key.network - .extend(["--network-egress-deny", "192.0.2.7/24"].map(str::to_owned)); - }, - UNLOWERED, - ), - // An option without its value. - ( - |key| key.network.push("--network-egress-deny".to_owned()), - UNLOWERED, - ), - // An option that no lowering writes. - ( - |key| { - key.network - .extend(["--network-dns", "10.0.0.1"].map(str::to_owned)); - }, - UNLOWERED, - ), - // The address twice. - ( - |key| { - key.network - .extend(["--net", "10.0.0.2/24"].map(str::to_owned)) - }, - UNLOWERED, - ), - // A proxy, which a policy that allows egress cannot have. - ( - |key| { - key.network - .extend(["--network-proxy", "10.0.0.1:3128"].map(str::to_owned)); - }, - UNLOWERED, - ), ]; for (change, expected) in cases { let mut changed = network_key(); @@ -937,13 +781,6 @@ fn keys_of_machines_that_no_manifest_describes_are_refused() { "{message:?} lacks {expected:?}" ); } - // A network for a machine without a NIC. - let mut changed = key(&support::class(), &ManifestSpec::template()); - changed.network = network_key().network; - assert_eq!( - key_refusal(&changed.to_bytes(), ErrorCode::Io), - "the template key is malformed: the network attaches a NIC that the machine does not have" - ); // Invariant settings out of the order of their names, which a key never writes. let mut class = network_class(); class.invariants = Invariants::new() @@ -951,18 +788,12 @@ fn keys_of_machines_that_no_manifest_describes_are_refused() { .unwrap() .with("b", "2") .unwrap(); - let made = key(&class, &ManifestSpec::with_network()); - let mut bytes = made.to_bytes(); - // The settings end where the network, which follows them, starts. - let mut network = Writer::default(); - network.list(made.network_arguments(), |writer, argument| { - writer.text(argument) - }); - let end = bytes.len() - network.finish().len(); + let mut bytes = key(&class, &ManifestSpec::with_network()).to_bytes(); + let len = bytes.len(); let tail = [ 1, 0, 0, 0, b'b', 1, 0, 0, 0, b'2', 1, 0, 0, 0, b'a', 1, 0, 0, 0, b'1', ]; - bytes[end - tail.len()..end].copy_from_slice(&tail); + bytes[len - tail.len()..].copy_from_slice(&tail); assert_eq!( key_refusal(&bytes, ErrorCode::Io), "the template key is malformed: the invariant settings are not in the strict order of \ @@ -977,67 +808,6 @@ fn keys_of_machines_that_no_manifest_describes_are_refused() { ); } -#[test] -fn keys_read_back_the_network_of_each_policy() { - let excluded = EgressRule { - to: vec![Destination { - cidr: "192.0.2.0/24".to_owned(), - except: vec!["192.0.2.0/24".to_owned()], - }], - ports: Vec::new(), - }; - let mut open = NetworkPolicy::new(Access::Allow); - open.ingress = Access::Allow; - open.host_loopback = HostLoopback::Outbound; - let mut forwarded = NetworkPolicy::new(Access::Allow) - .with_deny(EgressRule::to("2001:db8::/32").on_port(Protocol::Udp, 53)); - forwarded.host_loopback = HostLoopback::Allow; - forwarded.host_loopback_forwards = vec![ - LoopbackForward { - protocol: ForwardProtocol::Tcp, - host_port: 8080, - guest_port: 80, - }, - LoopbackForward { - protocol: ForwardProtocol::Udp, - host_port: 5353, - guest_port: 53, - }, - ]; - let policies = [ - open, - NetworkPolicy::new(Access::Deny) - .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)) - .with_allow(EgressRule::to("198.51.100.7").on_port_range(Protocol::Udp, 5000, 5010)) - .with_allow(EgressRule::to("::/0").on_protocol(Protocol::Icmp)) - .with_deny(EgressRule::to("192.0.2.7")), - forwarded, - NetworkPolicy::proxy_only(NetworkProxy::new(3128).with_scheme(ProxyScheme::Https)), - // A closed policy whose allow rule excepts its whole destination, which lowers to no - // OpenVMM rule, attaches its NIC all the same, with no allow rule, with or without deny - // rules: its arguments read back as a policy that lets nothing through. - NetworkPolicy::new(Access::Deny).with_allow(excluded.clone()), - NetworkPolicy::new(Access::Deny) - .with_allow(excluded) - .with_deny(EgressRule::to("198.51.100.0/24")), - ]; - for policy in policies { - let network = SandboxNetwork::new("10.0.0.2/24", policy).unwrap(); - let lowered = network.openvmm_arguments().unwrap(); - let class = CaptureClass { - network: Some(network), - ..network_class() - }; - let made = key(&class, &ManifestSpec::with_network()); - assert_eq!(made.network_arguments(), lowered); - assert_eq!( - TemplateKey::from_bytes(&made.to_bytes()).unwrap(), - made, - "{lowered:?}" - ); - } -} - #[test] fn invariant_settings_have_one_form() { let settings = Invariants::new() @@ -1256,7 +1026,7 @@ fn the_conformance_vectors_decode_and_encode() { ); } let invalid = vectors("INVALID_KEY"); - assert_eq!(invalid.len(), 34); + assert_eq!(invalid.len(), 23); for (name, bytes, _) in invalid { let error = TemplateKey::from_bytes(&bytes).expect_err(name); assert_eq!(error.code(), ErrorCode::Io, "{name}: {error}"); diff --git a/nvx-rs/src/host/template/mod.rs b/nvx-rs/src/host/template/mod.rs index 917a9687f..573921701 100644 --- a/nvx-rs/src/host/template/mod.rs +++ b/nvx-rs/src/host/template/mod.rs @@ -37,8 +37,9 @@ //! memory target against the template, and its export, if it has one, a plan of one directory, //! which the template's dormant filesystem slot binds, and refuses a request that the template //! cannot serve, such as an image longer than the template's placeholder, the capacity of every -//! clone's image, or of other block sizes, a network other than the template's, which its key -//! records, or an export of several directories or of a file, before any VMM starts. +//! clone's image, or of other block sizes, a network that attaches a NIC where the template has +//! none, none where it has one, or one at another guest address than the template's, or an +//! export of several directories or of a file, before any VMM starts. //! - [`ClonePlan::launch`] launches the clone, as the launch that its lease reserves, once the //! objects of its export are still those that its plan found: OpenVMM restores the generation's //! snapshot in place with the clone's devices and export, and the clone's activation record @@ -80,8 +81,7 @@ //! - A list is its count, a `u32`, then its items in order. //! //! [`TemplateKey::from_bytes`] refuses any other encoding, and bytes left after it. The vectors in -//! `tests/vectors/template-key-v1.txt` fix version 1 of a key's encoding, which holds the -//! template's network, as its class lowered it, and of a class's. +//! `tests/vectors/template-key-v1.txt` fix version 1 of a key's encoding and of a class's. mod canonical; mod clone; diff --git a/nvx-rs/src/host/template/plan.rs b/nvx-rs/src/host/template/plan.rs index 6b3fd4c07..c3b6ebda0 100644 --- a/nvx-rs/src/host/template/plan.rs +++ b/nvx-rs/src/host/template/plan.rs @@ -6,8 +6,9 @@ use std::path::{Path, PathBuf}; use super::manifest::PROCESSOR_COUNTS; use super::store::{SNAPSHOT_NAME, StorageGeneration}; use crate::error::{Error, Result}; -use crate::host::network::{Access, HostLoopback, NetworkPolicy}; use crate::host::paths; +#[cfg(doc)] +use crate::host::sandbox::SandboxNetwork; use crate::host::sandbox::distro::DistroBinding; use crate::host::sandbox::plan::bound_image; use crate::host::sandbox::{SandboxPlan, SandboxUpper, reject_container_path, reject_disk_path}; @@ -84,10 +85,13 @@ pub struct TemplateResources { /// directory, which sets the device's length and block sizes, and which the template does not /// keep. /// - The filesystem slot, which every cold boot exposes, stays dormant: the plan exports nothing. -/// - The plan's network gives the template its fixed NIC, with its policy, or none, for a -/// template whose clones have no network. The policy allows neither ingress nor host-loopback -/// access both ways, which its forwards need: through them, OpenVMM publishes the guest's -/// listeners as live host attachments, which no snapshot holds. +/// - The plan's network gives the template its fixed NIC, at the network's guest address, under +/// the complete deny-all policy, whatever the network's own, as +/// [`SandboxNetwork::template_arguments`] lowers it: egress denied without rules, ingress and +/// host-loopback access denied, without a proxy or forwards. It is the one policy that a +/// restore may replace, so each clone's restore installs the clone's own policy on the NIC. A +/// plan without a network, or with one that lets nothing through, which attaches no NIC, gives +/// the template none. /// - Its [`TemplateResources`]: its processor capacity, of which those online at boot are online /// when it is captured, and its RAM, with the capacity to which a restore may expand it, /// OpenVMM's `--memory-capacity`. @@ -105,8 +109,9 @@ pub struct TemplateResources { /// console, reboot, and panic settings, `nr_cpus=` with the processor capacity, /// `nvx_snapshot_tier=platform`, the virtio devices and the control tty that the guest discovers, /// and the NIC's addresses. The arguments name no workload identity, lifecycle, share, or -/// published listener, for which OpenVMM would write other tokens. The command line therefore -/// holds no image identity, hostname, share, proxy, entrypoint, or other tenant data, and every +/// published listener, for which OpenVMM would write other tokens, and under the NIC's deny-all +/// policy, OpenVMM names no DNS server either. The command line therefore holds no image +/// identity, hostname, share, proxy, DNS server, entrypoint, or other tenant data, and every /// clone of the template has the same. /// /// # The capture @@ -159,9 +164,12 @@ pub struct TemplateResources { /// ``` #[derive(Clone, Debug, PartialEq, Eq)] pub struct TemplatePlan { - /// The capture's devices: an adapter's plan of the placeholder over the dormant scratch device, - /// with the NIC, if any. + /// The capture's devices but its NIC: an adapter's plan of the placeholder over the dormant + /// scratch device, without a network. plan: SandboxPlan, + /// OpenVMM's arguments of the template's NIC, under the complete deny-all policy, or none for + /// a template without a NIC. + nic: Vec, /// The binding of the placeholder. binding: DistroBinding, /// The length of the dormant scratch device, in bytes. @@ -181,11 +189,11 @@ pub struct TemplatePlan { impl TemplatePlan { /// Lowers `adapter`, an adapter's plan, which [`SandboxPlan::adapter`] makes, into the plan of /// a template's capture: of its placeholder, which `binding` binds, over a dormant scratch - /// device of `scratch_length` bytes, with its network, the processors and RAM of `resources`, - /// the capture's transaction directory `transaction`, spelled with its components alone, as - /// [`Path::components`] reads them, and its bound-block storage generation `generation`, - /// which is apart from the other [namespaces](super::Store#four-namespaces) of a template's - /// generations. + /// device of `scratch_length` bytes, with the NIC of its network, the processors and RAM of + /// `resources`, the capture's transaction directory `transaction`, spelled with its components + /// alone, as [`Path::components`] reads them, and its bound-block storage generation + /// `generation`, which is apart from the other [namespaces](super::Store#four-namespaces) of a + /// template's generations. /// /// The plan's upper is no part of the template, which attaches its dormant scratch device in /// the upper's place, whatever the upper. @@ -201,10 +209,10 @@ impl TemplatePlan { /// logical blocks that OpenVMM presents; /// processors other than 1, 2, 4, or 8, or processors online at boot other than 1, 2, 4, or /// 8, or more than the processors; RAM or a RAM capacity that is not a nonzero multiple - /// of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM; a network policy that allows ingress, - /// or host-loopback access both ways; and a transaction directory that is not valid Unicode, - /// that is relative, which OpenVMM would resolve against the directory in which it runs, or - /// whose path holds a comma or a semicolon, which OpenVMM's block device options would split. + /// of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM; and a transaction directory that is + /// not valid Unicode, that is relative, which OpenVMM would resolve against the directory in + /// which it runs, or whose path holds a comma or a semicolon, which OpenVMM's block device + /// options would split. /// /// It reads, opens, and creates nothing. pub fn new( @@ -260,15 +268,15 @@ impl TemplatePlan { ))); } check_resources(&resources)?; - if let Some(network) = adapter.network() { - check_policy(network.policy())?; - } + let nic = match adapter.network() { + Some(network) => network.template_arguments()?, + None => Vec::new(), + }; let transaction = transaction_directory(transaction)?; let scratch = transaction.join(SCRATCH_NAME); reject_disk_path(&scratch, &scratch.display().to_string())?; let destination = transaction.join(SNAPSHOT_NAME); - let plan = SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch_at(scratch)) - .with_network(adapter.network().cloned()); + let plan = SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch_at(scratch)); // A template whose processors are all online at boot needs no limit, and records none, // as OpenVMM records none without the parameter. let command_line = if resources.boot_processors < resources.processors { @@ -281,6 +289,7 @@ impl TemplatePlan { }; Ok(Self { plan, + nic, binding: binding.clone(), scratch_length, resources, @@ -291,12 +300,20 @@ impl TemplatePlan { }) } - /// Returns the capture's devices: an adapter's plan of the placeholder over the dormant - /// scratch device, with the template's NIC, if any. + /// Returns the capture's devices but its NIC: an adapter's plan of the placeholder over the + /// dormant scratch device, without a network, whose NIC [`nic_arguments`](Self::nic_arguments) + /// attaches. pub fn plan(&self) -> &SandboxPlan { &self.plan } + /// Returns OpenVMM's arguments of the template's NIC, at the guest address of the adapter's + /// network, under the complete deny-all policy, as [`SandboxNetwork::template_arguments`] + /// lowers them, or none for a template without a NIC. + pub fn nic_arguments(&self) -> &[String] { + &self.nic + } + /// Returns the binding of the placeholder. pub fn binding(&self) -> &DistroBinding { &self.binding @@ -358,8 +375,9 @@ impl TemplatePlan { /// Returns OpenVMM's arguments of the capture: those of its devices, as /// [`SandboxPlan::openvmm_arguments`] renders them for [`plan`](Self::plan), the microVM - /// machine, the placeholder as the read-only `distro` block device, the dormant scratch device - /// as the `scratch` one, and the NIC, if any; then those of + /// machine, the placeholder as the read-only `distro` block device, and the dormant scratch + /// device as the `scratch` one; then those of the NIC, if any, + /// [`nic_arguments`](Self::nic_arguments); then those of /// [`capture_arguments`](Self::capture_arguments). /// /// The arguments name the block devices by their paths, which OpenVMM opens itself. A capture @@ -371,6 +389,7 @@ impl TemplatePlan { /// Fails as [`SandboxPlan::openvmm_arguments`] does. pub fn openvmm_arguments(&self) -> Result> { let mut arguments = self.plan.openvmm_arguments()?; + arguments.extend_from_slice(&self.nic); arguments.extend(self.capture_arguments()); Ok(arguments) } @@ -448,23 +467,6 @@ fn check_resources(resources: &TemplateResources) -> Result<()> { Ok(()) } -/// Refuses a network policy that a template's capture cannot take: one that allows ingress, or -/// host-loopback access both ways, which its forwards need, through which OpenVMM publishes the -/// guest's listeners as live host attachments, which no snapshot holds. -fn check_policy(policy: &NetworkPolicy) -> Result<()> { - let allowed = if policy.ingress == Access::Allow { - "ingress" - } else if policy.host_loopback == HostLoopback::Allow { - "host-loopback access both ways" - } else { - return Ok(()); - }; - Err(Error::unsupported(format!( - "a template's network policy cannot allow {allowed}: OpenVMM publishes the guest's \ - listeners as live host attachments, which no snapshot holds" - ))) -} - /// Returns the transaction directory `path`, spelled with its components alone, as /// [`Path::components`] reads them, which must be valid Unicode and absolute. fn transaction_directory(path: &Path) -> Result { diff --git a/nvx-rs/src/host/template/plan/tests.rs b/nvx-rs/src/host/template/plan/tests.rs index 963b057c5..369f99186 100644 --- a/nvx-rs/src/host/template/plan/tests.rs +++ b/nvx-rs/src/host/template/plan/tests.rs @@ -7,7 +7,10 @@ use serde_json::json; use super::*; use crate::ErrorKind; -use crate::host::network::{EgressRule, ForwardProtocol, LoopbackForward, NetworkProxy, Protocol}; +use crate::host::network::{ + Access, EgressRule, ForwardProtocol, HostLoopback, LoopbackForward, NetworkPolicy, + NetworkProxy, Protocol, +}; use crate::host::sandbox::distro::{BINDING_VERSION, DISTRO_POLICY_REVISION}; use crate::host::sandbox::{SandboxDistro, SandboxNetwork, SandboxSettings}; use crate::host::seal::SEAL_VERSION; @@ -81,7 +84,7 @@ const DESTINATION: &str = if cfg!(windows) { const GENERATION: &str = "000102030405060708090a0b0c0d0e0f"; /// The options of templates' arguments, each of which takes one value. -const OPTIONS: [&str; 17] = [ +const OPTIONS: [&str; 14] = [ "--machine", "--microvm-sandbox-block", "--net", @@ -89,9 +92,6 @@ const OPTIONS: [&str; 17] = [ "--network-egress", "--network-ingress", "--host-loopback", - "--network-egress-allow", - "--network-egress-deny", - "--network-proxy", "--processors", "--memory", "--memory-capacity", @@ -175,31 +175,50 @@ fn outbound() -> NetworkPolicy { policy } -/// Returns a network whose policy allows ingress. -fn listening() -> Option { +/// Returns a policy that allows ingress. +fn listening() -> NetworkPolicy { let mut policy = NetworkPolicy::new(Access::Deny); policy.ingress = Access::Allow; - Some(network(policy)) + policy } -/// Returns the networks of templates: none; one under a policy that lets nothing through, which -/// attaches no NIC; and NICs that allow egress, egress to some destinations, egress and -/// host-loopback access from the guest, and a proxy alone. -fn networks() -> Vec> { +/// Returns a policy that allows egress, and host-loopback access both ways, with a forward of a +/// TCP port of host loopback to the guest. +fn forwarded() -> NetworkPolicy { + let mut policy = NetworkPolicy::new(Access::Allow); + policy.host_loopback = HostLoopback::Allow; + policy.host_loopback_forwards.push(LoopbackForward { + protocol: ForwardProtocol::Tcp, + host_port: 18080, + guest_port: 8080, + }); + policy +} + +/// Returns the policies of networks that attach a NIC: they allow egress, egress to some +/// destinations, egress and host-loopback access from the guest, a proxy alone, ingress, and +/// host-loopback access both ways, with a forward. +fn attaching() -> Vec { vec![ - None, - Some(network(NetworkPolicy::new(Access::Deny))), - Some(network(NetworkPolicy::new(Access::Allow))), - Some(network( - NetworkPolicy::new(Access::Deny) - .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)) - .with_deny(EgressRule::to("192.0.2.128/25")), - )), - Some(network(outbound())), - Some(network(NetworkPolicy::proxy_only(NetworkProxy::new(3128)))), + NetworkPolicy::new(Access::Allow), + NetworkPolicy::new(Access::Deny) + .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)) + .with_deny(EgressRule::to("192.0.2.128/25")), + outbound(), + NetworkPolicy::proxy_only(NetworkProxy::new(3128)), + listening(), + forwarded(), ] } +/// Returns the networks of templates: none; one under a policy that lets nothing through, which +/// attaches no NIC; and those under each policy of [`attaching`]. +fn networks() -> Vec> { + let mut networks = vec![None, Some(network(NetworkPolicy::new(Access::Deny)))]; + networks.extend(attaching().into_iter().map(|policy| Some(network(policy)))); + networks +} + /// Returns the resources of `processors`, all online at boot, with 512 MiB of RAM and a capacity /// of 2 GiB. fn resources(processors: u32) -> TemplateResources { @@ -356,49 +375,34 @@ fn templates_render_their_devices_resources_and_capture_options() { // No NIC, without a network or with one that lets nothing through. for network in [None, Some(network(NetworkPolicy::new(Access::Deny)))] { let template = template(adapter(network)); + assert!(template.nic_arguments().is_empty()); assert_eq!(template.capture_arguments(), capture); assert_eq!( template.openvmm_arguments().unwrap(), [devices.clone(), capture.clone()].concat() ); } - // One fixed NIC, with the template's policy. - let nic = strings(&["--net", "10.0.0.2/24", "--network-profile", "portable"]); - for (policy, enforced) in [ - ( - NetworkPolicy::new(Access::Allow), - strings(&[ - "--network-egress", - "allow", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - ]), - ), - ( - outbound(), - strings(&["--network-egress", "allow", "--network-ingress", "deny"]), - ), - ( - NetworkPolicy::proxy_only(NetworkProxy::new(3128)), - strings(&[ - "--network-egress", - "deny", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - "--network-proxy", - "10.0.0.1:3128", - ]), - ), - ] { + // One fixed NIC, at the network's guest address, under the complete deny-all policy, whatever + // the network's own, which each clone's restore replaces. + let nic = strings(&[ + "--net", + "10.0.0.2/24", + "--network-profile", + "portable", + "--network-egress", + "deny", + "--network-ingress", + "deny", + "--host-loopback", + "deny", + ]); + for policy in attaching() { let template = template(adapter(Some(network(policy)))); + assert_eq!(template.nic_arguments(), nic); assert_eq!(template.capture_arguments(), capture); assert_eq!( template.openvmm_arguments().unwrap(), - [devices.clone(), nic.clone(), enforced, capture.clone()].concat() + [devices.clone(), nic.clone(), capture.clone()].concat() ); } // RAM and its capacity in MiB. @@ -434,7 +438,7 @@ fn templates_keep_the_placeholder_and_the_files_of_their_capture_in_its_transact assert_eq!(plan.upper().scratch_path(), Some(Path::new(SCRATCH))); assert_eq!(plan.settings(), None); assert_eq!(plan.export(), None); - assert_eq!(plan.network(), network.as_ref()); + assert_eq!(plan.network(), None); assert_eq!(template.binding(), &binding()); assert_eq!(template.image_capacity(), GIB); assert_eq!(template.scratch_path(), Path::new(SCRATCH)); @@ -587,11 +591,16 @@ fn the_command_line_holds_only_tokens_of_openvmms_platform_allowlist() { "nvx_snapshot_tier={}", value(&arguments, "--snapshot-tier") )); + // Under the NIC's complete deny-all policy, OpenVMM writes no DNS server and no + // published listeners. if arguments.iter().any(|argument| argument == "--net") { - let network = template.plan().network().unwrap(); - let (address, _) = network.guest_network().split_once('/').unwrap(); + let guest_network = value(&arguments, "--net"); + let (address, _) = guest_network.split_once('/').unwrap(); tokens.push(format!("virtnet_ip={address}")); - tokens.push(format!("virtnet_gw={}", network.gateway())); + tokens.push(format!( + "virtnet_gw={}", + crate::net::guest_gateway(guest_network).unwrap() + )); } for token in &tokens { assert!(openvmm::admits(token), "{token} of {template:?}"); @@ -622,7 +631,7 @@ fn templates_hold_no_tenant_data() { for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { assert!(!command_line.contains(text), "{text} in {command_line}"); } - for text in [IDENTITY, "tenant", "/sandboxes", r"\sandboxes"] { + for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { assert!( arguments.iter().all(|argument| !argument.contains(text)), "{text} in {arguments:?}" @@ -638,12 +647,20 @@ fn templates_hold_no_tenant_data() { let option = option.as_str(); assert!(OPTIONS.contains(&option), "{option} in {arguments:?}"); assert!(!value.starts_with("--"), "{option} {value}"); - if ["--network-ingress", "--host-loopback"].contains(&option) { + if ["--network-egress", "--network-ingress", "--host-loopback"].contains(&option) { assert_eq!(value, "deny", "{option} in {arguments:?}"); } } let devices = template.plan().openvmm_arguments().unwrap(); - assert_eq!([devices, template.capture_arguments()].concat(), arguments); + assert_eq!( + [ + devices, + template.nic_arguments().to_vec(), + template.capture_arguments() + ] + .concat(), + arguments + ); } } @@ -841,29 +858,6 @@ fn templates_refuse_what_no_capture_takes() { ); } - // Policies through which OpenVMM publishes the guest's listeners. - let mut both_ways = NetworkPolicy::new(Access::Allow); - both_ways.host_loopback = HostLoopback::Allow; - let mut forwarded = both_ways.clone(); - forwarded.host_loopback_forwards.push(LoopbackForward { - protocol: ForwardProtocol::Tcp, - host_port: 18080, - guest_port: 8080, - }); - for (network, allowed) in [ - (listening(), "ingress"), - (Some(network(both_ways)), "host-loopback access both ways"), - (Some(network(forwarded)), "host-loopback access both ways"), - ] { - assert_eq!( - refusal(Inputs::of(adapter(network)).lower()), - format!( - "a template's network policy cannot allow {allowed}: OpenVMM publishes the \ - guest's listeners as live host attachments, which no snapshot holds" - ) - ); - } - // Transaction directories that OpenVMM would resolve against its own directory. let mut relative = vec!["templates/tx-1", "."]; if cfg!(windows) { @@ -958,7 +952,7 @@ fn templates_check_their_inputs_in_order() { SandboxSettings::default(), ) .unwrap() - .with_network(listening()), + .with_network(Some(network(listening()))), binding: binding_of("erofs", IDENTITY, 1000), scratch_length: 0, resources: TemplateResources { @@ -969,12 +963,22 @@ fn templates_check_their_inputs_in_order() { }, transaction: "templates".to_owned(), }; - let steps: [(&str, Fix); 12] = [ + let steps: [(&str, Fix); 11] = [ ("a template lowers an adapter's plan", |inputs| { - inputs.adapter = adapter_at("placeholder.vhd", gpt(), SandboxUpper::RAM, listening()); + inputs.adapter = adapter_at( + "placeholder.vhd", + gpt(), + SandboxUpper::RAM, + Some(network(listening())), + ); }), ("is not the erofs layer", |inputs| { - inputs.adapter = adapter_at("placeholder.vhd", erofs(), SandboxUpper::RAM, listening()); + inputs.adapter = adapter_at( + "placeholder.vhd", + erofs(), + SandboxUpper::RAM, + Some(network(listening())), + ); }), ( "the placeholder of a template's capture must be absolute", @@ -983,12 +987,12 @@ fn templates_check_their_inputs_in_order() { &PLACEHOLDER.replace(".img", ".vhd"), erofs(), SandboxUpper::RAM, - listening(), + Some(network(listening())), ); }, ), ("disk paths ending in .vhd", |inputs| { - inputs.adapter = adapter(listening()); + inputs.adapter = adapter(Some(network(listening()))); }), ("of 1000 bytes cannot be attached", |inputs| { inputs.binding = binding(); @@ -1005,9 +1009,6 @@ fn templates_check_their_inputs_in_order() { ("needs nonzero RAM", |inputs| { inputs.resources = resources(2) }), - ("cannot allow ingress", |inputs| { - inputs.adapter = adapter(None); - }), ("must be absolute", |inputs| { inputs.transaction = format!("{TRANSACTION},1"); }), diff --git a/nvx-rs/src/host/template/store/capture.rs b/nvx-rs/src/host/template/store/capture.rs index 7a95ee07d..969b7db0d 100644 --- a/nvx-rs/src/host/template/store/capture.rs +++ b/nvx-rs/src/host/template/store/capture.rs @@ -94,8 +94,12 @@ const CAPTURES_WHAT: &str = "the captures of the template store"; /// The adapter's plan is lowered as [`TemplatePlan::new`] lowers it, with the processors, those /// online at boot among them, and RAM of the class, and a scratch of the class's scratch length; /// its upper is no part of the template. The class and the plan must describe one VM: a -/// placeholder whose binding records the class's `distro` length, and a network of the plan that -/// lowers to the OpenVMM arguments of the class's network. +/// placeholder whose binding records the class's `distro` length, and a network of the plan whose +/// NIC, as [`SandboxNetwork::template_arguments`] lowers it for a template, is the class's: none, +/// or one at the same guest's address, under the complete deny-all policy, whatever the policies +/// of the two networks. +/// +/// [`SandboxNetwork::template_arguments`]: crate::host::sandbox::SandboxNetwork::template_arguments #[derive(Debug, Clone)] pub struct Capture { /// The class of the capture, which names its slot, and from which the capture takes its @@ -220,7 +224,7 @@ impl Inputs { /// /// Fails with [`ErrorKind::Unsupported`] as [`CaptureClass::identity`] fails, and for a /// placeholder whose binding records another length than the class's `distro`, and a plan - /// whose network lowers to other arguments than the class's; and with + /// whose network attaches another NIC than the class's, as a template attaches it; and with /// [`ErrorKind::Unavailable`] for an artifact whose path cannot be resolved. fn check(capture: &Capture) -> Result { let class = &capture.class; @@ -236,15 +240,15 @@ impl Inputs { } let planned = match capture.adapter.network() { None => Vec::new(), - Some(network) => network.openvmm_arguments()?, + Some(network) => network.template_arguments()?, }; let requested = match &class.network { None => Vec::new(), - Some(network) => network.openvmm_arguments()?, + Some(network) => network.template_arguments()?, }; if planned != requested { return Err(Error::unsupported(format!( - "the adapter's plan attaches the network {planned:?}, but the class requests \ + "the adapter's plan attaches the NIC {planned:?}, but the class requests \ {requested:?}" ))); } diff --git a/nvx-rs/src/protocol/cmdline.rs b/nvx-rs/src/protocol/cmdline.rs index 8fcd57c21..aff2a0423 100644 --- a/nvx-rs/src/protocol/cmdline.rs +++ b/nvx-rs/src/protocol/cmdline.rs @@ -1,7 +1,7 @@ //! The kernel command line as Linux and the guest's init read it: the parameters into which the //! kernel splits it, the virtio-mmio devices that its `virtio_mmio.device` parameters describe, -//! and the tokens into which the guest's init splits it; and the guest's DNS server, which -//! OpenVMM names in it. +//! and the tokens into which the guest's init splits it; and the guest's DNS server and the IPv4 +//! gateway of its NIC, which OpenVMM names in it. //! //! A reader that splits a command line by rules of its own may find parameters that the kernel //! never sees, or miss ones that it applies: the kernel splits at more bytes than ASCII @@ -30,20 +30,24 @@ //! check of the user's kernel parameters, in `nvx::host::sandbox`, which splits them both as the //! kernel and as the init does, and compares their names as the kernel does. //! -//! The kernel leaves OpenVMM's `virtnet_dns=` token to the guest, which reads it from the whole -//! command line, as OpenVMM checks it: +//! The kernel leaves OpenVMM's `virtnet_dns=` and `virtnet_gw=` tokens to the guest, which reads +//! them from the whole command line, as OpenVMM checks them: //! -//! - [`dns_server`] reads the DNS server that the token names, strictly, as OpenVMM renders and -//! reads it: at most one token, with one IPv4 or IPv6 address. +//! - [`dns_server`] reads the DNS server that the `virtnet_dns=` token names, strictly, as OpenVMM +//! renders and reads it: at most one token, with one IPv4 or IPv6 address. //! - [`write_resolver`] writes the resolver file that names that server, as NVX's init writes the //! initramfs's `/etc/resolv.conf` from the token. +//! - [`gateway`] reads the IPv4 gateway that the `virtnet_gw=` token names, as strictly: at most +//! one token, with one IPv4 address. //! //! Like the rest of [`protocol`](super), it needs nothing outside the standard library, so the //! host, the guest, and test doubles share it. +use std::fmt::Display; use std::io; use std::iter::FusedIterator; -use std::net::IpAddr; +use std::net::{IpAddr, Ipv4Addr}; +use std::str::FromStr; use super::{ProtocolError, violation}; @@ -55,6 +59,11 @@ pub const VIRTIO_MMIO_DEVICE: &str = "virtio_mmio.device"; /// network policy lets the guest reach the gateway's DNS service. pub const DNS_TOKEN: &str = "virtnet_dns="; +/// Prefix of the token through which OpenVMM names the IPv4 gateway of the guest's NIC, +/// `virtnet_gw=`, which it adds to a microVM's command line with the NIC's other tokens: the +/// gateway's address, to which the guest's announcer of its listeners sends its announcements. +pub const GATEWAY_TOKEN: &str = "virtnet_gw="; + /// Returns whether the kernel takes `byte` for whitespace, as its `isspace` does: the space, tab, /// newline, vertical tab, form feed, carriage return, and Latin-1's no-break space, 0xa0. pub const fn is_space(byte: u8) -> bool { @@ -292,29 +301,55 @@ pub fn init_tokens(command_line: &[u8]) -> impl Iterator> { /// such as `010.0.0.1`, `fd00:0::a00:1`, or `FD00::A00:1`. OpenVMM also checks, before the guest /// boots, that the address is one of the gateway's, which the guest leaves to it. pub fn dns_server(command_line: &[u8]) -> Result, ProtocolError> { - let mut server = None; + address_token(command_line, DNS_TOKEN, "IPv4 or IPv6") +} + +/// Returns the IPv4 gateway that the `virtnet_gw=` token of `command_line`, such as the guest's +/// `/proc/cmdline`, names, or `None` if no token names one, as for a guest without a NIC. +/// +/// The reader takes the command line's tokens as [`dns_server`] takes them, and accepts the token +/// only as OpenVMM renders it: at most one token, whose value is one IPv4 address in its canonical +/// text, as Rust's [`Ipv4Addr`] writes it, such as `10.0.0.1`. It refuses with a +/// [`ProtocolError`] a repeated token, an empty value, an IPv6 address, which OpenVMM names with +/// `virtnet_gw6=` instead, and another spelling of an address, such as `010.0.0.1`. +pub fn gateway(command_line: &[u8]) -> Result, ProtocolError> { + address_token(command_line, GATEWAY_TOKEN, "IPv4") +} + +/// Returns the address of the `family` that the token of `command_line` that starts with +/// `prefix`, such as `virtnet_dns=`, names, or `None` if no token names one, as [`dns_server`] +/// reads its token: at most one, whose value is one address in its canonical text. +fn address_token( + command_line: &[u8], + prefix: &str, + family: &str, +) -> Result, ProtocolError> { + let name = prefix.strip_suffix('=').unwrap_or(prefix); + let mut found = None; for token in command_line.split(u8::is_ascii_whitespace) { - let Some(value) = token.strip_prefix(DNS_TOKEN.as_bytes()) else { + let Some(value) = token.strip_prefix(prefix.as_bytes()) else { continue; }; - if server.is_some() { - return Err(violation("the virtnet_dns kernel parameter is repeated")); + if found.is_some() { + return Err(violation(format!( + "the {name} kernel parameter is repeated" + ))); } let address = std::str::from_utf8(value).ok().and_then(|text| { - text.parse::() + text.parse::() .ok() .filter(|address| address.to_string() == text) }); let Some(address) = address else { return Err(violation(format!( - "the virtnet_dns kernel parameter must name one IPv4 or IPv6 address as OpenVMM \ - renders it, not '{}'", + "the {name} kernel parameter must name one {family} address as OpenVMM renders \ + it, not '{}'", value.escape_ascii() ))); }; - server = Some(address); + found = Some(address); } - Ok(server) + Ok(found) } /// Writes to `writer` a resolver file, in the format of `/etc/resolv.conf`, that names `server` as diff --git a/nvx-rs/src/protocol/cmdline/tests.rs b/nvx-rs/src/protocol/cmdline/tests.rs index 8ea3a4c80..4a6e65948 100644 --- a/nvx-rs/src/protocol/cmdline/tests.rs +++ b/nvx-rs/src/protocol/cmdline/tests.rs @@ -555,6 +555,56 @@ fn the_dns_server_is_one_address_in_one_token() { ); } +#[test] +fn the_gateway_is_one_ipv4_address_in_one_token_as_openvmm_renders_it() { + // OpenVMM names the IPv4 gateway of a guest with a NIC among its network tokens, and its IPv6 + // gateway with another token, and the kernel appends a newline to `/proc/cmdline`. + let (before, after) = NETWORK; + let command_line = format!("console=hvc0 {before} virtnet_dns=10.0.0.1 {after}\n"); + assert_eq!( + gateway(command_line.as_bytes()), + Ok(Some(Ipv4Addr::new(10, 0, 0, 1))) + ); + for command_line in [ + &b""[..], + b"console=hvc0 virtnet_gw6=fd00::a00:1", + b"virtnet_gw", + b"xvirtnet_gw=10.0.0.1", + ] { + assert_eq!( + gateway(command_line), + Ok(None), + "{}", + command_line.escape_ascii() + ); + } + for command_line in [ + &b"virtnet_gw="[..], + b"virtnet_gw=010.0.0.1", + b"virtnet_gw=10.0.0.1/24", + b"virtnet_gw=10.0.0.1,10.0.0.2", + b"virtnet_gw=10.0.0.1 virtnet_gw=10.0.0.1", + b"virtnet_gw=10.0.0.1\xa0", + ] { + assert!( + gateway(command_line).is_err(), + "{}", + command_line.escape_ascii() + ); + } + assert_eq!( + gateway(b"virtnet_gw=10.0.0.1 virtnet_gw=10.0.0.2") + .unwrap_err() + .message(), + "the virtnet_gw kernel parameter is repeated" + ); + assert_eq!( + gateway(b"virtnet_gw=fd00::a00:1").unwrap_err().message(), + "the virtnet_gw kernel parameter must name one IPv4 address as OpenVMM renders it, not \ + 'fd00::a00:1'" + ); +} + #[test] fn a_resolver_file_names_the_server_as_the_init_writes_it() { for (server, file) in [ diff --git a/nvx-rs/src/protocol/status.rs b/nvx-rs/src/protocol/status.rs index 69a5d3aa6..3b5d15f1d 100644 --- a/nvx-rs/src/protocol/status.rs +++ b/nvx-rs/src/protocol/status.rs @@ -75,7 +75,8 @@ pub const ACTIVATION_WORKLOAD: u8 = 55; /// The host export cannot be mounted in the clone. pub const ACTIVATION_EXPORT: u8 = 56; /// The record's network settings cannot be applied in the clone: its resolver or its proxy -/// endpoint. +/// endpoint, or the announcer of the workload's listeners, which its network publishes, cannot +/// start, or ended. pub const ACTIVATION_NETWORK: u8 = 57; /// The guest's own order misused the activation: it completed an activation that did not run, /// or started the sandbox before the clone was ready. diff --git a/nvx-rs/tests/guest_tools.rs b/nvx-rs/tests/guest_tools.rs index c1ff75f01..140780cde 100644 --- a/nvx-rs/tests/guest_tools.rs +++ b/nvx-rs/tests/guest_tools.rs @@ -77,6 +77,21 @@ where /// test once `timeout` passes. Its output must fit in the pipes, which nothing reads until it /// exits. fn run_within(name: &str, arguments: I, timeout: Duration) -> (Option, Vec, Vec) +where + I: IntoIterator, + S: AsRef, +{ + run_with_input(name, arguments, Stdio::null(), timeout) +} + +/// Runs the guest binary `name` with `arguments` as [`run_within`] does, but with `stdin` as its +/// standard input. +fn run_with_input( + name: &str, + arguments: I, + stdin: Stdio, + timeout: Duration, +) -> (Option, Vec, Vec) where I: IntoIterator, S: AsRef, @@ -85,7 +100,7 @@ where let _spawning = SPAWNING.read().unwrap(); Command::new(binary(name)) .args(arguments) - .stdin(Stdio::null()) + .stdin(stdin) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() @@ -1032,15 +1047,20 @@ fn ingress_test_mode_preserves_udp_settling_between_snapshots() { #[test] fn ingress_rejects_bad_invocations_and_test_snapshots() { - let usage = b"usage: nvx-ingress GATEWAY_IPV4\n".to_vec(); - assert_eq!( - run("nvx-ingress", std::iter::empty::<&str>()), - (Some(2), Vec::new(), usage.clone()) - ); - assert_eq!( - run("nvx-ingress", ["not-an-ipv4-address"]), - (Some(2), Vec::new(), usage) - ); + let usage = b"usage: nvx-ingress [inherit] GATEWAY_IPV4\n".to_vec(); + for arguments in [ + &[][..], + &["not-an-ipv4-address"], + &["inherit"], + &["inherit", "not-an-ipv4-address"], + &["inherit", "127.0.0.1", "extra"], + ] { + assert_eq!( + run("nvx-ingress", arguments), + (Some(2), Vec::new(), usage.clone()), + "{arguments:?}" + ); + } let directory = tempfile::tempdir().unwrap(); assert_eq!( @@ -1075,6 +1095,59 @@ fn ingress_rejects_bad_invocations_and_test_snapshots() { ); } +#[test] +fn ingress_inherits_only_an_ipv4_datagram_socket_bound_to_a_privileged_source_port() { + // The announcer that a clone starts announces from the socket that it inherits as its + // standard input, which only root can bind to a privileged source port, from which alone the + // gateway takes announcements: it refuses any other at once, before it scans anything. + let refused = |stdin: Stdio, expected: String| { + assert_eq!( + run_with_input( + "nvx-ingress", + ["inherit", "127.0.0.1"], + stdin, + Duration::from_secs(10) + ), + (Some(1), Vec::new(), expected.into_bytes()) + ); + }; + let unprivileged = std::net::UdpSocket::bind("127.0.0.1:0").unwrap(); + let port = unprivileged.local_addr().unwrap().port(); + assert!(port >= 1024, "{port}"); + refused( + Stdio::from(std::os::fd::OwnedFd::from(unprivileged)), + format!( + "nvx-ingress: the inherited socket's port {port} is not a privileged source port\n" + ), + ); + let unbound = nix::sys::socket::socket( + nix::sys::socket::AddressFamily::Inet, + nix::sys::socket::SockType::Datagram, + nix::sys::socket::SockFlag::empty(), + None, + ) + .unwrap(); + refused( + Stdio::from(unbound), + "nvx-ingress: the inherited socket's port 0 is not a privileged source port\n".to_owned(), + ); + let stream = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + refused( + Stdio::from(std::os::fd::OwnedFd::from(stream)), + format!( + "nvx-ingress: inherited socket: {}\n", + Errno::EPROTOTYPE.desc() + ), + ); + refused( + Stdio::null(), + format!( + "nvx-ingress: inherited socket: {}\n", + Errno::ENOTSOCK.desc() + ), + ); +} + #[cfg(target_arch = "x86_64")] /// Returns the first CPU that this process may run on. fn first_allowed_cpu() -> usize { diff --git a/nvx-rs/tests/openvmm_vectors.rs b/nvx-rs/tests/openvmm_vectors.rs index eee533c5f..803910b40 100644 --- a/nvx-rs/tests/openvmm_vectors.rs +++ b/nvx-rs/tests/openvmm_vectors.rs @@ -2,9 +2,9 @@ //! protocol vectors and the excerpts of OpenVMM's sources in `tests/vectors/openvmm/`, and the //! versions of OpenVMM's contracts in `nvx::contract`. //! -//! The protocol's tests read the copies, and the unit tests of `nvx::host::template` compile the -//! excerpts, because the `nvx-rs` job cannot check out the submodule, whose deploy key fork pull -//! requests lack. This test is ignored by default: the NVX CLI jobs, which check out the +//! The protocol's tests read the copies, and the unit tests of `nvx::host::template` compile or +//! read the excerpts, because the `nvx-rs` job cannot check out the submodule, whose deploy key +//! fork pull requests lack. This test is ignored by default: the NVX CLI jobs, which check out the //! submodule, run it, and so does an OpenVMM pin promotion. use std::fs; @@ -21,7 +21,7 @@ const FILES: [&str; 2] = [ /// The excerpts of OpenVMM's sources that `tests/vectors/openvmm/` copies: for each, its file /// there, and the file of the OpenVMM submodule whose lines it copies, unchanged. -const EXCERPTS: [(&str, &str); 2] = [ +const EXCERPTS: [(&str, &str); 3] = [ ( "microvm_control_tty_command_line.rs", "openvmm/openvmm_defs/src/microvm.rs", @@ -30,6 +30,10 @@ const EXCERPTS: [(&str, &str); 2] = [ "platform_command_line_token_is_invariant.rs", "openvmm/openvmm_helpers/src/snapshot/microvm.rs", ), + ( + "port_forward_socket_creation.rs", + "vm/devices/net/net_consomme/src/resolver.rs", + ), ]; /// The versions of `nvx::contract` that are OpenVMM's: for each, the file of the OpenVMM diff --git a/nvx-rs/tests/vectors/README.md b/nvx-rs/tests/vectors/README.md index f46b077ad..cbc2c9e07 100644 --- a/nvx-rs/tests/vectors/README.md +++ b/nvx-rs/tests/vectors/README.md @@ -12,8 +12,8 @@ yet breaks those peers. Only bytes that no codec produced catch such a change. | [`nvxc-v1.txt`](nvxc-v1.txt) | Version 1 of the control protocol: `NVXC` frames, their `EXEC`, `MAPS`, and features payloads, and `NVXS` records of NVX's host and guest | | [`nvxl-v1.txt`](nvxl-v1.txt) | Version 1 of the `NVXL` listener announcement | | [`activation-v1.txt`](activation-v1.txt) | Version 1 of the activation record of a layered root, and what Ready reports about an activation | -| [`template-key-v1.txt`](template-key-v1.txt) | Version 1 of the canonical encodings of a template's compatibility key, which holds the template's network as its capture class lowered it, and of a capture class, with their identities | -| [`openvmm/`](openvmm) | OpenVMM's vectors and cases of the `NVXS` session, and the excerpts of its sources that define the kernel command-line tokens of its platform tier, unchanged | +| [`template-key-v1.txt`](template-key-v1.txt) | Version 1 of the canonical encodings of a template's compatibility key and of a capture class, with their identities | +| [`openvmm/`](openvmm) | OpenVMM's vectors and cases of the `NVXS` session, and the excerpts of its sources that define the kernel command-line tokens of its platform tier and the error of a port forward that it cannot bind, unchanged | [4.4 Host-Guest Control Interface](../../../doc/design/4.4-host-guest-control-interface.md) and @@ -66,11 +66,12 @@ the key that the vector's bytes decode to. name the export's root. Only the clone of a template whose export is a directory sends one, to the template's own guest, which deploys with the host and whose build the template's key names, so the C agent of earlier - releases, which refuses it, never receives one. And the keys of - `template-key-v1.txt` came to end with the template's network in place: the - key's encoding has not been released, so a template store that an earlier - build created is not supported, and is deleted and created again, since - nothing migrates it. + releases, which refuses it, never receives one. And `CLASS|NETWORK` of + `template-key-v1.txt` came to encode its network in place as the template + captures it, under the complete deny-all policy: the encodings of a key and + of a class have not been released, so a template store that an earlier build + created is not supported, and is deleted and created again, since nothing + migrates it. - The bytes come from the protocol's definition, never from the code that they check, and so do the values of each vector. @@ -86,7 +87,7 @@ refusals of the records' payloads and headers. The tests of `nvx::protocol::nvxs` check that each vector decodes and encodes to its bytes, and that each case is accepted or refused as it says. -`openvmm/` also holds two excerpts of OpenVMM's sources at that commit, whole +`openvmm/` also holds three excerpts of OpenVMM's sources at that commit, whole lines, unchanged: `platform_command_line_token_is_invariant.rs`, the function of `openvmm/openvmm_helpers/src/snapshot/microvm.rs` that admits the kernel command-line tokens of a capture at OpenVMM's platform tier, and @@ -96,7 +97,14 @@ the function reads. The tests of `nvx::host::template` compile them, and check each token of a template's kernel command line with OpenVMM's own function. The runtime adapter in `tests/runtime-adapter/` compiles them too, as the allowlist with which the guest agent of a template's VM checks its kernel command line, -and its tests pin the set of tokens that they admit. +and its tests pin the set of tokens that they admit. The third, +`port_forward_socket_creation.rs`, is the variant of the error of +`vm/devices/net/net_consomme/src/resolver.rs` with which OpenVMM reports a port +forward whose socket on the host it cannot bind, as when a clone's VMM restores +its NIC with a forward whose port another socket holds. OpenVMM gives that +error no code, so the launch of a clone attributes it to the host by its text, +and the tests of `nvx::host::template` read the excerpt, and check that the +text is OpenVMM's. The tests cannot call OpenVMM's broker or its allowlist: OpenVMM's workspace needs a later Rust release than `nvx`'s minimum, and the `nvx-rs` job cannot diff --git a/nvx-rs/tests/vectors/openvmm/port_forward_socket_creation.rs b/nvx-rs/tests/vectors/openvmm/port_forward_socket_creation.rs new file mode 100644 index 000000000..9262a8d00 --- /dev/null +++ b/nvx-rs/tests/vectors/openvmm/port_forward_socket_creation.rs @@ -0,0 +1,6 @@ + #[error("failed to create socket for port forward ({details})")] + SocketCreation { + #[source] + source: std::io::Error, + details: String, + }, diff --git a/nvx-rs/tests/vectors/template-key-v1.txt b/nvx-rs/tests/vectors/template-key-v1.txt index f00e6e20e..283e97a0c 100644 --- a/nvx-rs/tests/vectors/template-key-v1.txt +++ b/nvx-rs/tests/vectors/template-key-v1.txt @@ -1,8 +1,7 @@ # Conformance vectors of version 1 of the canonical encodings of a template's compatibility key, # nvx::host::template::TemplateKey, and of a capture class, nvx::host::template::CaptureClass, # which the module nvx::host::template describes. README.md describes the line format and the -# rules that the vectors follow. A key ends with the template's network, as its capture class -# lowered it to OpenVMM's arguments, after the invariant settings. +# rules that the vectors follow. # # KEY|NAME|BYTES|IDENTITY # The canonical encoding of a key, and its identity: the SHA-256 digest of "nvx template key", a @@ -23,98 +22,74 @@ # ABI 1, restore packet 4, and activation contract 1. # The key of the template of a KVM microVM of 2 processors, all online at boot, and 512 MiB # of RAM, of a capacity of 2 GiB, with a distro of 1 GiB, a scratch of 64 MiB, the dormant -# filesystem slot, a console, a control console, and no NIC, and so no network, on the CPU -# profile intel.icelake-sp.v1, whose digest is 32 bytes of 5a and whose effective CPUID is the -# bytes 00 to 57: the capture of tests/fixtures/template/platform-kvm. -KEY|TEMPLATE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d000000000000000000|218ead116d918d236ba226cbd9c86753927de98283a4d202ff6d69cad87ea2af +# filesystem slot, a console, a control console, and no NIC, on the CPU profile +# intel.icelake-sp.v1, whose digest is 32 bytes of 5a and whose effective CPUID is the bytes +# 00 to 57: the capture of tests/fixtures/template/platform-kvm. +KEY|TEMPLATE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0000000000|81901eda9602b4f2099c44415b7a38b035ff215b9bc50f860c8c70ca2c6f593f # The same with a fixed NIC at 10.0.0.2/24, with its IPv6 identity, under an allow-list # policy whose digest is 32 bytes of 07, and the invariant settings isolation.seccomp=strict -# and limits.pids=512; its class's network allows egress, and denies ingress and host-loopback -# access: --net 10.0.0.2/24 --network-profile portable --network-egress allow -# --network-ingress deny --host-loopback deny. -KEY|NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e70696473030000003531320a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d65677265737305000000616c6c6f77110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79|061c60bcc639e2edcc7d6974446f9cbc78122cf02d0195ee4077e953ad56674e +# and limits.pids=512. +KEY|NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e7069647303000000353132|f7418d9454a78af64bc5bc46e82206ccd085baf64cf7140e5ae4faf04b7c1ebe # TEMPLATE on the host profile intel.host.v1. -KEY|HOST_PROFILE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c04000000686f7374010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d000000000000000000|7ff9e24a8864510509c68ffaf00cac54c6a9ec391bf45f663536cd095220dea4 +KEY|HOST_PROFILE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c04000000686f7374010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0000000000|93afa8bdde64df606707f5534c58420b92888acbbb6c8245e83006768c3ac940 # A machine of one processor and one device, and no NIC, which the invalid vectors change. -KEY|MINIMAL|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000|e7037b740fb3d00fdb4a9af7b4f0e22a7323c8a744fa697b924360e90d47816f +KEY|MINIMAL|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000|d79a8fefc401da8568586cdc595a93a7e6c988aefbc62b03038901940211c1f1 # The class of the capture of TEMPLATE: the CPU profile auto, and 2 processors, all online at # boot. CLASS|TEMPLATE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d040000006175746f02000000020000000000002000000000000000800000000000000040000000000002000000100000000000040000000000020000000200000000000000000000|b53b0169db5d983b31f24bec900cd0b962d26f12480d90803f21ae1d5c3ff334 -# The class of the capture of NETWORK, whose network allows egress, and denies ingress and -# host-loopback access. -CLASS|NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d040000006175746f02000000020000000000002000000000000000800000000000000040000000000002000000100000000000040000000000020000000200000a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d65677265737305000000616c6c6f77110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e7069647303000000353132|3ed4ab3d6536faa5872354282d702ae63ed89d14bb3c12dc9102d79ea5b38c6d +# The class of the capture of NETWORK, whose NIC the capture attaches under the complete +# deny-all policy, whatever the policy of the class's network, here one that allows egress. +CLASS|NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d040000006175746f02000000020000000000002000000000000000800000000000000040000000000002000000100000000000040000000000020000000200000a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d6567726573730400000064656e79110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e7069647303000000353132|2f32edae7e9eb532625af600232c68aa78b92e02c8116cb49f857f13ff5fcc7c # MINIMAL in other versions of the encoding. -INVALID_KEY_VERSION|VERSION_0|0000470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 -INVALID_KEY_VERSION|VERSION_2|0200470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 -INVALID_KEY_VERSION|VERSION_65535|ffff470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY_VERSION|VERSION_0|0000470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 +INVALID_KEY_VERSION|VERSION_2|0200470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 +INVALID_KEY_VERSION|VERSION_65535|ffff470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # No bytes. INVALID_KEY|EMPTY| # MINIMAL without its last byte. -INVALID_KEY|TRUNCATED|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000000000 +INVALID_KEY|TRUNCATED|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00000000 # MINIMAL and a byte after it. -INVALID_KEY|TRAILING_BYTE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00000000000000000000 +INVALID_KEY|TRAILING_BYTE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000 # A command line whose digest does not match it. -INVALID_KEY|COMMAND_LINE_DIGEST|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663300000000000000000000000000000000000000000000000000000000000000000001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|COMMAND_LINE_DIGEST|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663300000000000000000000000000000000000000000000000000000000000000000001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A backend that does not exist. -INVALID_KEY|UNKNOWN_BACKEND|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d31060000000000000001000000000000000100000000000000040000000000000001000000000000000300000078656e03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|UNKNOWN_BACKEND|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d31060000000000000001000000000000000100000000000000040000000000000001000000000000000300000078656e03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A device range of an address space that does not exist. -INVALID_KEY|UNKNOWN_ADDRESS_SPACE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e000000000100000004000000706f7274200000000000000002000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|UNKNOWN_ADDRESS_SPACE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e000000000100000004000000706f7274200000000000000002000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # An interrupt delivery mode that is not UTF-8. -INVALID_KEY|INTERRUPT_MODE_NOT_UTF8|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000013000000656467652d7368617265642dff737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|INTERRUPT_MODE_NOT_UTF8|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000013000000656467652d7368617265642dff737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A NIC whose presence tag is 2. -INVALID_KEY|PRESENCE_TAG_2|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d020000000000000000 +INVALID_KEY|PRESENCE_TAG_2|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0200000000 # A NIC whose policy's enforcement is 2. -INVALID_KEY|BOOLEAN_2|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c697374070707070707070707070707070707070707070707070707070707070707070702040000000000000000000000 +INVALID_KEY|BOOLEAN_2|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c6973740707070707070707070707070707070707070707070707070707070707070707020400000000000000 # Invariant settings out of the order of their names. -INVALID_KEY|UNSORTED_SETTINGS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00020000000b0000006c696d6974732e70696473030000003531321100000069736f6c6174696f6e2e736563636f6d700600000073747269637400000000 +INVALID_KEY|UNSORTED_SETTINGS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00020000000b0000006c696d6974732e70696473030000003531321100000069736f6c6174696f6e2e736563636f6d7006000000737472696374 # An invariant setting named twice. -INVALID_KEY|DUPLICATE_SETTINGS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00020000000b0000006c696d6974732e70696473030000003531320b0000006c696d6974732e706964730300000035313200000000 +INVALID_KEY|DUPLICATE_SETTINGS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00020000000b0000006c696d6974732e70696473030000003531320b0000006c696d6974732e7069647303000000353132 # An invariant setting whose name holds an uppercase letter. -INVALID_KEY|UPPERCASE_SETTING|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00010000000b0000004c696d6974732e706964730300000035313200000000 +INVALID_KEY|UPPERCASE_SETTING|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00010000000b0000004c696d6974732e7069647303000000353132 # More APIC IDs than a microVM has processors. -INVALID_KEY|TOO_MANY_APIC_IDS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b010000000100000001000000010000000900000000000000010000000200000003000000040000000500000006000000070000000800000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|TOO_MANY_APIC_IDS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b010000000100000001000000010000000900000000000000010000000200000003000000040000000500000006000000070000000800000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # 3 processors, which a microVM cannot have. -INVALID_KEY|THREE_PROCESSORS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b010000000100000003000000010000000300000000000000010000000200000000ca9a3b0000000000e1f505000000000000000003000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|THREE_PROCESSORS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b010000000100000003000000010000000300000000000000010000000200000000ca9a3b0000000000e1f505000000000000000003000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # RAM ranges that do not hold the RAM. -INVALID_KEY|MEMORY_RANGES|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000400000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|MEMORY_RANGES|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000400000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A machine without devices. -INVALID_KEY|NO_DEVICES|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|NO_DEVICES|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000000000001000000070000006d6963726f766d0000000000 # A device ID twice. -INVALID_KEY|DUPLICATE_DEVICE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000200000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000009000000706172746974696f6e05000000636c6f636b06000000766d74696d6500000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|DUPLICATE_DEVICE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000200000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000009000000706172746974696f6e05000000636c6f636b06000000766d74696d6500000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A filesystem slot that is not the dormant slot of version 1. -INVALID_KEY|SLOT_VERSION_0|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000000000000070000006d6963726f766d000000000000000000 +INVALID_KEY|SLOT_VERSION_0|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000000000000070000006d6963726f766d0000000000 # A filesystem slot without a tag. -INVALID_KEY|EMPTY_TAG|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e0000000000000000000000000000000000000000000100000000000000000000000000000000 +INVALID_KEY|EMPTY_TAG|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000000000000000000000 # A distro whose logical block size is 256. -INVALID_KEY|DISTRO_BLOCK_SIZE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000010000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|DISTRO_BLOCK_SIZE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000010000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # An OpenVMM identity that holds a space. -INVALID_KEY|ARTIFACT_WITH_SPACE|0100470000007368613235363a20616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|ARTIFACT_WITH_SPACE|0100470000007368613235363a20616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # A CPU profile of revision 0. -INVALID_KEY|CPU_PROFILE_REVISION_0|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610000000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 +INVALID_KEY|CPU_PROFILE_REVISION_0|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610000000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 # An effective CPUID that is not whole entries of 44 bytes. -INVALID_KEY|CPUID_LENGTH|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112b000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000000000000 -# MINIMAL with a network, for a machine without a NIC. -INVALID_KEY|NETWORK_WITHOUT_NIC|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d000000000002000000050000002d2d6e65740b00000031302e302e302e322f3234 -# NETWORK without its network. -INVALID_KEY|NIC_WITHOUT_NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e706964730300000035313200000000 -# NETWORK with a network that attaches the guest at another address than its NIC's. -INVALID_KEY|NETWORK_OF_ANOTHER_ADDRESS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e70696473030000003531320a000000050000002d2d6e65740b00000031302e302e302e332f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d65677265737305000000616c6c6f77110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79 -# NETWORK with a network that does not start with --net and the guest's address. -INVALID_KEY|NETWORK_WITHOUT_ADDRESS|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e706964730300000035313208000000110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d65677265737305000000616c6c6f77110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79 -# NETWORK with an argument of 257 bytes. -INVALID_KEY|LONG_ARGUMENT|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e70696473030000003531320a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d65677265737305000000616c6c6f77110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b010100006464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464 -# NETWORK with a network of --net and the guest's address alone, to which no network policy -# lowers. -INVALID_KEY|NETWORK_OF_ADDRESS_ALONE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e706964730300000035313202000000050000002d2d6e65740b00000031302e302e302e322f3234 -# NETWORK with its --network-ingress before its --network-egress, an order in which no -# network policy lowers. -INVALID_KEY|NETWORK_OUT_OF_ORDER|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e70696473030000003531320a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65110000002d2d6e6574776f726b2d696e67726573730400000064656e79100000002d2d6e6574776f726b2d65677265737305000000616c6c6f770f0000002d2d686f73742d6c6f6f706261636b0400000064656e79 -# TEMPLATE, NETWORK, HOST_PROFILE, and MINIMAL without their network part, the last part of -# every key, which a key of a template without a NIC holds as an empty list. -INVALID_KEY|TEMPLATE_WITHOUT_NETWORK_PART|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0000000000 -INVALID_KEY|NETWORK_WITHOUT_NETWORK_PART|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c0a0000006963656c616b652d7370010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000f20100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030303030303a31302076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633220766972746e65745f69703d31302e302e302e3220766972746e65745f6d61736b3d3235352e3235352e3235352e3020766972746e65745f67773d31302e302e302e3120766972746e65745f6970363d666430303a3a6130303a322f31323020766972746e65745f6777363d666430303a3a6130303a315e7d49c200c9d60e74547b549917b665501635035894c15c2d7780210f95caf0000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000c0000006e65743a6d6963726f766d300a00000076697274696f2d6e65741500000076697274696f2d6e65742d333438393636303932380b00000076697274696f2d6d6d696f01000000040000006d6d696f000000d0000000000010000000000000010a000000020000002000000001000000020000000200000000010000000100000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0108000000706f727461626c655254000000025254000000010a000002180a00000101fd00000000000000000000000a00000278fd00000000000000000000000a0000010a000000616c6c6f772d6c69737407070707070707070707070707070707070707070707070707070707070707070104000000020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e7069647303000000353132 -INVALID_KEY|HOST_PROFILE_WITHOUT_NETWORK_PART|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d05000000696e74656c04000000686f7374010000005a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a58000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f50515253545556570100000001000000020000000100000002000000000000000100000000752b7d0000000000c2eb0b00000000fa000000020000000000000000000020000000000000008000000000000000080000000001000000000000000000000000000020000000000100000000000020000000000000006000000000530100006561726c79636f6e3d78653920636f6e736f6c653d68766331207265626f6f743d742070616e69633d2d31206e76785f73616e64626f783d31206e76785f636f6e6669673d307864303031303030302c3635353336206e725f637075733d32206e76785f736e617073686f745f746965723d706c6174666f726d2076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030313030303a362076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030323030303a372076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030333030303a342076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030363030303a31312076697274696f5f6d6d696f2e6465766963653d30783130303040307864303030373030303a33206e76785f636f6e74726f6c5f7474793d6876633217f80f8858431ca75fcc7ca694ea68637e017ede02db15e18d0ed4581067858d000000400000000000020000001000000000000400000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000001000000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000003000000767030040000007663707509000000706172746974696f6e00000000000000000000000000000000000000000006000000766d74696d6505000000636c6f636b06000000766d74696d65000000000000000000000000000000000000000000030000007069630300000070696303000000706963000000000200000004000000706d696f2000000000000000020000000000000004000000706d696fa00000000000000002000000000000000000000000000000000000000006000000696f6170696306000000696f6170696306000000696f617069630000000001000000040000006d6d696f0000c0fe00000000001000000000000000000000000000000000000000050000006c61706963050000006c6170696309000000706172746974696f6e0000000001000000040000006d6d696f0000e0fe00000000001000000000000000000000000000000000000000030000007069740300000070697403000000706974000000000100000004000000706d696f400000000000000004000000000000000100000000000000000000000000000000030000007274630300000072746303000000727463000000000100000004000000706d696f7000000000000000020000000000000001080000000000000000000000000000000d0000006d6963726f766d2d706f72746205000000706f7274620d0000006d6963726f766d2d706f727462000000000100000004000000706d696fe900000000000000020000000000000000000000000000000000000000100000006d6963726f766d2d73687574646f776e0800000073687574646f776e100000006d6963726f766d2d73687574646f776e000000000100000004000000706d696f0406000000000000010000000000000000000000000000000000000000180000006d6963726f766d2d736e617073686f742d7265717565737410000000736e617073686f742d72657175657374180000006d6963726f766d2d736e617073686f742d72657175657374000000000100000004000000706d696f05060000000000000100000000000000000000000000000000000000000b00000066733a6d6963726f766d300900000076697274696f2d66731300000076697274696f66732d333438393636353032340b00000076697274696f2d6d6d696f01000000040000006d6d696f001000d000000000001000000000000001060000000200000000000030030000000200000002000000000100000001000017000000636f6e736f6c653a6d6963726f766d2d76697274696f300e00000076697274696f2d636f6e736f6c651900000076697274696f2d636f6e736f6c652d333438393636393132300b00000076697274696f2d6d6d696f01000000040000006d6d696f002000d000000000001000000000000001070000000200000001000030030000000200000002000000000100000001000012000000626c6b3a73616e64626f783a64697374726f0a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393637333231360b00000076697274696f2d6d6d696f01000000040000006d6d696f003000d0000000000010000000000000010400000002000000640600300300000001000000010000000001000013000000626c6b3a73616e64626f783a736372617463680a00000076697274696f2d626c6b1500000076697274696f2d626c6b2d333438393638353530340b00000076697274696f2d6d6d696f01000000040000006d6d696f006000d0000000000010000000000000010b00000002000000440600300300000001000000010000000001000018000000636f6e736f6c653a6d6963726f766d2d636f6e74726f6c301600000076697274696f2d636f6e74726f6c2d636f6e736f6c652100000076697274696f2d636f6e74726f6c2d636f6e736f6c652d333438393638393630300b00000076697274696f2d6d6d696f01000000040000006d6d696f007000d000000000001000000000000001030000000200000001000030030000000200000002000000000100000001000001000000070000006d6963726f766d0000000000 -INVALID_KEY|MINIMAL_WITHOUT_NETWORK_PART|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 +INVALID_KEY|CPUID_LENGTH|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112b000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d0000000000 diff --git a/tests/runtime-adapter/README.md b/tests/runtime-adapter/README.md index 0e7938782..bd3fc848b 100644 --- a/tests/runtime-adapter/README.md +++ b/tests/runtime-adapter/README.md @@ -640,7 +640,9 @@ credit that it needs, never keeps the others' output from the host. session creates, and, as PID 1 of its PID namespace, its end ends every process of the namespace. The guest's network device, its addresses, and the announcer of its listeners stay with NVX's `/init`, which configures them - before PID 1 starts; see [Networks](#networks). + before PID 1 starts, but for the announcer of a clone, which the clone's + application starts; see [Networks](#networks) and + [Template mode](#template-mode). - Each execution runs in a cgroup of its own below the workload cgroup of the root configuration, `execution-N`, where N counts the cgroups that the worker prepared. The worker prepares it with @@ -806,10 +808,12 @@ after the repair's acknowledgement of the restore: CPU count and the memory size that the repair found, before anything has a side effect. 3. It applies the record with `nvx::guest::clone::apply`, with - `worker::capture::root_template` of the template's configuration: its - runtime directory, the `distro` and scratch block devices, its device - timeout, and the tag of the host filesystem device, - `worker::capture::FILESYSTEM_TAG`. The application assembles the root, an + `worker::capture::root_template` of the template's configuration and of its + NIC's IPv4 gateway, which `template::boot` read from the template's kernel + command line: its runtime directory, the `distro` and scratch block devices, + its device timeout, the tag of the host filesystem device, + `worker::capture::FILESYSTEM_TAG`, and the gateway, if the template has a + NIC. The application assembles the root, an EROFS image or a GPT disk over a tmpfs or the scratch, in the record's overlay mode, waiting for each device until the activation's deadline at the latest, so that it ends by then, and checks the account of the record's @@ -827,10 +831,19 @@ after the repair's acknowledgement of the restore: the worker of a VM that boots cold mounts the one that its kernel command line names. If the record's network names a DNS server, it writes a resolver file that names it, of mode 0444, in `nvx::guest::clone::RESOLVER_FILE` of - the runtime directory. It then binds the generated machine ID read-only over - the root's `/etc/machine-id`, and the resolver file read-only over its - `/etc/resolv.conf`, in the worker's mount namespace, for every execution; - with the image's own machine ID and without a DNS server, it binds nothing. + the runtime directory. If the record's network publishes the workload's + listeners, it starts their announcer, + `/sbin/nvx-ingress inherit GATEWAY_IPV4`, with + `nvx::guest::ingress::Announcer::start`: it binds a UDP socket to a + privileged source port, from which alone OpenVMM's gateway takes + announcements, and runs the program with that socket as its standard input, + to the template's gateway, waiting until the program runs, but for no + announcement. The template, captured under a policy that publishes nothing, + runs no announcer of its own. It then binds the generated machine ID + read-only over the root's `/etc/machine-id`, and the resolver file read-only + over its `/etc/resolv.conf`, in the worker's mount namespace, for every + execution; with the image's own machine ID and without a DNS server, it binds + nothing. 4. Once it took what the session received while the application ran, which came before `Ready`, it completes the activation, which opens the fence, and answers `Ready`, which reports the activation after the guest agent's build, @@ -846,13 +859,18 @@ after the repair's acknowledgement of the restore: execution takes the record's hostname. The host tells the workloads the endpoint of the network's proxy, as after a cold boot: the host program's `exec` adds it to each execution's environment, so the worker leaves each - execution's environment exactly its start's. The application does not apply - the publication of the record's listeners, so a clone's executions enter the - root with listeners published, the entry's default, which refuses the most. - On the stop, the worker closes the export's root and removes the - application's binds, which keep the root's mounts busy, before it tears the - root down, through `nvx::guest::clone::Applied::teardown`. It also answers - the [`activation` probe](#probes), with when the events of its activation and + execution's environment exactly its start's. A clone's executions enter the + root with the publication of listeners that the record's network names: + published under a policy that publishes them, which refuses + `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW`, and unpublished otherwise, a clone + without a network included. The worker polls the pidfd of the announcer that + the application started with the session's stream: once the announcer ended, + before the clone was ready or after, the clone ends with status 57, as one + whose announcer could not start. On the stop, the worker stops the announcer, + closes the export's root, and removes the application's binds, which keep the + root's mounts busy, before it tears the root down, through + `nvx::guest::clone::Applied::teardown`. It also answers the + [`activation` probe](#probes), with when the events of its activation and order happened after the repair's acknowledgement of the restore. Until the clone is ready, its session takes nothing but that `Activate`: @@ -880,7 +898,7 @@ ends with one of these statuses, after a diagnostic on standard error: | 1 | The control tty cannot be opened or put in raw mode, the attachment or the acknowledgement of the broker's reset cannot be sent, or the broker closes the session or sends anything but a reset first, or, in a clone, its session failed or the broker closed it | | 13 | The bootstrap of the guest for the template's root failed, before the attachment | | 15 | In a clone, the root's teardown failed once the host stopped the worker | -| 20 | The kernel command line names the platform tier, but holds another parameter, a double quote, or `--`, or the worker cannot read it | +| 20 | The kernel command line names the platform tier, but holds another parameter, a double quote, `--`, or a gateway that OpenVMM does not render, or the worker cannot read it | | 21 | The fence of workload admission cannot be closed | | 22 | Something is mounted from the template's devices or from an overlay, or the mount table cannot be checked | | 23 | The time ABI's boot checks did not complete | @@ -904,7 +922,7 @@ ends with one of these statuses, after a diagnostic on standard error: | 54 | The root's assembly failed, its upper not fresh among its causes, or the kernel did not reread the partition table of the record's GPT disk before it | | 55 | The clone's workload cannot be activated: the image refuses the record's workload identity, as it holds no account of the identity, or not its group, or its account databases cannot be read, or the machine ID that the record asks for cannot be generated, or the record's hostname or that machine ID cannot be set for every execution | | 56 | The record's host export cannot be mounted | -| 57 | The record's network settings cannot be applied: the resolver file that names its DNS server cannot be written, or set for every execution | +| 57 | The record's network settings cannot be applied: the resolver file that names its DNS server cannot be written, or set for every execution, or the announcer of the listeners that its network publishes cannot start, as without the template's gateway; or that announcer ended | | 58 | The worker's own order misused the activation: it completed an activation that did not run, or started the sandbox before the clone was ready, or again | | 193 | A conformance check of the time ABI failed in the clone's repair, such as the finish's, which needs a running time daemon | | 195 | The time ABI's part of the clone's repair failed, such as the validation of the restore packet, or the record of the restore | @@ -1385,7 +1403,8 @@ command's options choose of the template, and `Request::capture` returns NVX's processors, those online at boot, 1 unless the command names more, the fewest that the template's clones take, the RAM and its capacity, the geometries of the placeholder and of the dormant scratch, as OpenVMM presents - a sandbox block device, which `capture::geometry` gives, and the network; + a sandbox block device, which `capture::geometry` gives, and the network, as + the capture attaches its NIC, which no network's policy changes; - NVX's artifacts, with the initramfs that puts the guest agent in place of NVX's agents, which the command writes into a directory of its own in the system's temporary directory, private to the current user, as a launch @@ -1411,16 +1430,19 @@ compatibility key, whose kernel command line holds the platform tier's tokens alone: neither the placeholder's identity nor a root's tokens. The template has a NIC exactly when its network lets something through: a network that denies egress, ingress, and host-loopback access, without a proxy, attaches none, as -no network does. The key holds the network as NVX lowers it, OpenVMM's -arguments of the guest's address and of its policy, none without a NIC, so each -network policy has a template of its own, which serves the -[clones](#clones) of that policy alone. The key's encoding, version 1, includes -that network and has not been released, so a template store that an earlier -build created is not supported: delete it and create a new one. There is no -migration. NVX refuses a network whose policy no -template takes, one that -allows ingress or host-loopback access both ways, before any VM starts, and the -command refuses a class that no capture takes, such as one of 3 processors, or +no network does. The capture attaches the NIC at the network's guest address, +under OpenVMM's complete deny-all policy, whatever the network's own: egress +denied without rules, ingress and host-loopback access denied, without a proxy +or forwards, the one policy of a snapshot that a restore may replace. So the +template's guest learns no DNS server, and starts no announcer of its +listeners, and the key holds what the capture made of the network alone: +whether there is a NIC, its addresses, and the identity of its deny-all +policy. The networks of every policy at one guest address therefore share one +class, and one template, which serves the [clones](#clones) of each of them. +The key's encoding, version 1, holds the NIC that way and has not been +released, so a template store that an earlier build created is not supported: +delete it and create a new one. There is no migration. The command refuses a +class that no capture takes, such as one of 3 processors, or of more online at boot than in all, before it creates the store. A capture that fails leaves the store's generations as they were, and the command's diagnostic names the log of the VMM @@ -1496,7 +1518,8 @@ its executions run, its overlay mode, its scratch's journal policy, the bound of a GPT disk's first partition, the machine ID that its executions see, the host paths that it shares with its root, as a launch's [shares](#shares), and its [network](#networks), as a launch reads it, which -must be the template's. +must be at the guest's address of the template's NIC, under a policy of its +own. `Request::plan` plans the image over the upper, with the network, as a launch plans a layered root, `Request::export` plans the export of the shares as a launch plans it, @@ -1509,14 +1532,19 @@ clone, which a read-only bind holds at the root's `/etc/machine-id` for every execution, and the guest's defaults of the other settings, unless the request names them; for a GPT disk, the bound of its first partition is 64 KiB unless the request names one, as a launch's guest takes it by default. For a template -with a NIC, the record names the network: the identity of its policy, the DNS -server with which the template's guest configured its resolver at boot, if -any, which the guest writes to a resolver file, of mode 0444, that a -read-only bind holds at the root's `/etc/resolv.conf` for every execution, -beside a generated machine ID, and the URL of the policy's proxy, if any, -which every execution of the clone finds in its environment, as an execution -of a launch does; see [Template mode](#template-mode). Without a DNS server, -the root keeps the image's own `/etc/resolv.conf`. +with a NIC, the record names the clone's network: the identity of its policy, +a digest of its serde form; the DNS server that OpenVMM serves the guest under +the policy, if any, which NVX finds as OpenVMM does, since the template's guest +learned none, and which the guest writes to a resolver file, of mode 0444, that +a read-only bind holds at the root's `/etc/resolv.conf` for every execution, +beside a generated machine ID; whether the policy publishes the workload's +listeners, as it does with ingress, or with host-loopback access both ways, for +which the clone's guest starts their announcer, and refuses +`CAP_NET_BIND_SERVICE` and `CAP_NET_RAW` to its executions; and the URL of the +policy's proxy, if any, which every execution of the clone finds in its +environment, as an execution of a launch does; see +[Template mode](#template-mode). Without a DNS server, the root keeps the +image's own `/etc/resolv.conf`. The guest of a GPT clone rereads the disk's partition table before it assembles the root, since its kernel created the partitions of the template's placeholder. @@ -1560,11 +1588,11 @@ measures its downtime, is `ineligible`, by the evidence `tombstone` or NVX's `ClonePlan` then checks the clone against the generation from the metadata of its files alone, under the sandbox's lock, which the lease holds: -its network, which must be the one that the template's key holds, the guest's -address and the policy of the template's NIC, which every restore of the -template attaches unchanged, or, for a template without a NIC, none, or one -that lets nothing through, so that a clone of another network policy finds no -template in it, +its network, which must attach a NIC exactly when the template has one, at the +guest's address of the template's NIC, under a policy of its own, which the +clone's restore installs on the NIC, or, for a template without a NIC, none, or +one that lets nothing through, so that a clone at another guest address finds +no template in it, the bindings of its images, which no one else may change, the geometry of its image, which must have the placeholder's block sizes, and be no longer than the placeholder, whose length is the capacity that the restore presents, @@ -1593,8 +1621,10 @@ placeholder, over the scratch image, or, for a clone in RAM, with the template's scratch absent, which OpenVMM attaches with no file, so that no file of the host backs it, with the export's directory on the template's filesystem slot, -and the template's NIC, if it has one, under the template's network, -and the restore sequence activates the clone through the +and the template's NIC, if it has one, under the policy of the clone's +network, which `--restore-network-policy` installs in place of the template's +complete deny-all policy, and the restore sequence activates the clone through +the adapter's clone order, `host::CloneOrder`, which sends the clone's activation record in an `Activate`, checks the guest agent's build in its `Ready`, sends the export's mapping table, which the record announces, in order, as a launch @@ -2776,12 +2806,7 @@ artifacts, and creates a template of it in a template store of its own: | `a_template_without_a_network_has_no_nic` | 2 processors, 1 of them online at boot, 256 MiB of RAM with a capacity of 512 MiB, a 32 MiB scratch, and no network | A captured template without a NIC, whose key records its 1 processor online at boot, which `maxcpus=1` limits | | `a_template_whose_processors_are_all_online_at_boot_limits_none` | 2 processors, both online at boot | A captured template whose key records no processors online at boot, without `maxcpus=` | | `a_network_that_lets_nothing_through_has_no_nic` | A network that denies egress, ingress, and host-loopback access, without a proxy | A captured template without a NIC | -| `an_egress_network_has_its_nic` | Egress allowed | A captured template with a NIC | -| `an_allow_rule_has_its_nic` | Egress denied but for one TCP rule to the gateway, with outbound host-loopback access | A captured template with a NIC | -| `outbound_host_loopback_has_its_nic` | Egress allowed, with outbound host-loopback access | A captured template with a NIC | -| `a_proxy_only_network_has_its_nic` | A proxy-only network | A captured template with a NIC | -| `ingress_is_refused_before_any_vm_starts` | Ingress allowed | NVX's refusal, with status 1, a store without a generation, and no VMM's log or transaction in any slot | -| `host_loopback_both_ways_is_refused_before_any_vm_starts` | Host-loopback access both ways | NVX's refusal, with status 1, a store without a generation, and no VMM's log or transaction in any slot | +| `the_networks_of_every_policy_at_one_guest_s_address_share_one_template` | Host-loopback access both ways, with a forward of port 18080 of host loopback to the guest's port 8080; then, at the same guest's address, each network that lets something through: egress allowed, with outbound host-loopback access or without, one TCP rule to the gateway at port 8080, with outbound host-loopback access, a proxy at port 3128 of host loopback, ingress allowed, host-loopback access both ways, and that forward | A captured template with a NIC, under the complete deny-all policy, whose key's command line names the NIC's addresses, but neither a DNS server nor a publication of the guest's listeners; then each creation `existing`, with the same generation, key, and class, without a capture | | `a_class_that_no_capture_takes_is_refused_before_the_store` | 3 processors, then 2 processors with 4 online at boot | NVX's refusal of each class, with status 1, before the store exists | | `a_refresh_captures_and_retires_the_generations_that_it_supersedes` | A creation, then two refreshes, the second while a clone leases the second generation, then a creation once the clone released it | `list` and `inspect` of the first generation; each refresh captures a new generation, although the store holds an eligible one, and its eviction retires the first generation, and leaves the leased second one, tombstoned, with its payload; the creation serves the third without a capture, and its eviction deletes the second | | `a_removal_reports_busy_while_leases_keep_the_generation` | A creation, two clones' leases of its generation, a removal, a second creation, then a removal after each release | `list` and `inspect` with the leases; the removal tombstones the generation, which keeps its payload and takes no new lease, and reports busy, with status 7 and its leases; the second creation captures a new generation, and leaves the first to its leases; a removal reports busy while a lease remains, and deletes the generation once none does; a second removal finds it retired, one of a generation that the store never published fails, and a generation that no lease keeps goes at once | @@ -2796,8 +2821,10 @@ processors and those online at boot, none for a template whose processors are all online at boot, the RAM and its capacity, the geometries of the placeholder and of the scratch, the NIC, and a kernel command line of the platform tier's tokens alone, with the processors, `maxcpus=` with those online at boot -exactly when they are fewer, and, exactly with a NIC, the NIC's addresses, -which names neither the placeholder's identity nor a root's tokens. Its class's +exactly when they are fewer, and, exactly with a NIC, the NIC's addresses, but +neither a DNS server nor a publication of the guest's listeners, which OpenVMM +names to no guest under the NIC's deny-all policy, and which names neither the +placeholder's identity nor a root's tokens. Its class's slot then holds no transaction and records no launch, and a second creation prints `existing`, with the same generation, key, and class, without a capture: the slot's log of the VMM stays as the capture left it. @@ -2857,10 +2884,11 @@ directory that the starts authorize: | `a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts` | Sharing two host directories, and sharing a file alone | Status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid mappings, which quarantines nothing; the lease released, and the host's directories unchanged | | `a_clone_whose_mapping_fails_ends_before_its_sandbox_starts` | Over a scratch image, sharing a host directory whose second mapping puts a directory onto a file of the image | Status 8, a `Ready`, but no start, and the record of a failure of an unknown cause, at the start, at the clone's `Ready`, by the evidence of the protocol, once the VM exited with the guest's success, or NVX terminated it, with the guest's report that the table failed after its first mapping; nothing quarantined, and the lease released, with the records; the host's directory unchanged | | `a_template_without_a_nic_serves_clones_without_a_network` | Two at once, without a network, and with one that lets nothing through; then, once they stopped, one of each network with a NIC of the template scenarios | The first two started, with no network in their records, on Linux no argument of a network of their VMMs, no network device, and the image's own `/etc/resolv.conf`, an empty file of mode 0644, which no bind holds, and root may write; then for each of the others, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid settings, which quarantines nothing; the lease released | -| `a_template_serves_the_clones_of_its_own_network_policy_alone` | Of a template of each network with a NIC of the template scenarios: one of the template's network, then, once it stopped, one of each other network, one of the template's policy at the guest's address `192.0.2.3/24`, one of a network that lets nothing through, and one without a network | The first started, with one network device, on Linux the VMM's arguments of the template's policy, as its key holds them, without `--net`, and a record that names the policy, the gateway's DNS server for the networks with egress allowed, the proxy's URL for the proxy-only one, and no publication; from an execution, a TCP connection to the gateway at the port of each of three listeners of host loopback, which opens, and arrives at the listener, where the policy lets it through, and fails, with nothing arriving, elsewhere: the allow rule's listener alone, the proxy's alone, every one with egress and outbound host-loopback access allowed, and none with egress allowed alone; then for each of the others, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid settings, which quarantines nothing; the lease released; each template's policy differs from the others' | -| `a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only` | Of a template of a network with egress allowed, two at once, with a generated machine ID, and with the image's own; then, once they stopped, a GPT disk whose layer holds a regular file `etc`, with the image's own machine ID | Each of the first two started, with a record that names the gateway's DNS server, one network device, and at `/etc/resolv.conf`, a file of mode 0444 that names the server, held by a bind that is read-only, `nosuid`, and `nodev`, after the bind of the generated machine ID, below the root, whose file system refuses a write as root, with nothing in the upper there; and a TCP connection to the server's DNS port, which the gateway's DNS proxy accepts without the host's resolver; then status 8, since no bind can hold the resolver configuration below a file, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of invalid settings, with the guest's status 57; nothing quarantined, and the lease released, with the records | -| `a_clone_s_executions_find_the_proxy_of_its_policy` | Of a template of a proxy-only network, one, with an execution as user ID 0 that runs `env`, then one that names its own `HTTPS_PROXY` | A record that names the proxy's URL, `http://192.0.2.1:PORT`, at the port of the proxy's listener, and no DNS server; the first execution with its own entry, then the URL in `http_proxy`, `https_proxy`, `HTTP_PROXY`, and `HTTPS_PROXY`; the second with exactly its own environment; one network device, and the image's own `/etc/resolv.conf`, which no bind holds | -| `a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver` | Of a template of a network with egress allowed, one with a generated machine ID, sharing a host directory read-write | Started, with one mapping, and a record that names the export and the gateway's DNS server; on Linux, the VMM's one `--mount`, read-write, and its arguments of the template's policy; below the root, the binds of the machine ID and of the resolver configuration, then the export's mapping; the export's directory, and a write through it, which the host's directory holds after the stop | +| `one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address` | Of a template of host-loopback access both ways, with a forward: in turn, one of each network with a NIC, of egress allowed, with outbound host-loopback access or without, of one TCP rule to the gateway, with outbound host-loopback access, of a proxy, of ingress allowed, of host-loopback access both ways, and of that access with a forward; then one at the guest's address `192.0.2.3/24`, one of a network that lets nothing through, and one without a network | Each of the first started, with one network device, on Linux the VMM's `--restore-network-policy`, then the arguments of the clone's own policy, without `--net`, and a record that names the policy, the gateway's DNS server for the networks with egress allowed, the proxy's URL for the proxy-only one, and a publication of the guest's listeners for those with ingress or host-loopback access both ways; an execution's TCP connections to the gateway, at the ports of the host's listeners on host loopback, that reach the allow rule's alone, the proxy's alone, every one with egress and outbound host-loopback access, and none otherwise, as the host's listeners confirm; a listener of the guest, as nobody, that the host reaches at its port of host loopback, with a line each way, exactly with host-loopback access both ways, once the clone's announcer announced it, and through the forward's port; and executions as user ID 0 that ask for `CAP_NET_BIND_SERVICE` or `CAP_NET_RAW` refused with `EPERM` where the listeners are published, and started elsewhere, as one that asks for `CAP_CHOWN` everywhere; then for each of the others, status 8, with NVX's reason, and the record of a failure of the request, at the plan, without a VM, by the evidence of invalid settings, which quarantines nothing; the lease released; each clone's policy differs from the others' | +| `a_forward_whose_port_another_socket_holds_fails_the_clone_as_the_host_s` | Of a template of a network with egress allowed, one of host-loopback access both ways, with a forward of a port of host loopback that a listener of the host holds; then, once the listener closed, one of the same network | Status 8, with OpenVMM's cause, that it cannot create the forward's socket, and the record of a failure of the host, at the restore, once the VMM exited before its restore-ready event, by the evidence of the forward's bind, which quarantines nothing; the lease released, with the records; then a clone that starts and stops | +| `a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only` | Of a template of a proxy-only network, two at once, of a network with egress allowed, with a generated machine ID, and with the image's own; then, once they stopped, a GPT disk whose layer holds a regular file `etc`, with the image's own machine ID | Each of the first two started, with a record that names the gateway's DNS server, one network device, and at `/etc/resolv.conf`, a file of mode 0444 that names the server, held by a bind that is read-only, `nosuid`, and `nodev`, after the bind of the generated machine ID, below the root, whose file system refuses a write as root, with nothing in the upper there; and a TCP connection to the server's DNS port, which the gateway's DNS proxy accepts without the host's resolver; then status 8, since no bind can hold the resolver configuration below a file, and the record of a failure of the request, at the activation, once the VM exited, by the evidence of invalid settings, with the guest's status 57; nothing quarantined, and the lease released, with the records | +| `a_clone_s_executions_find_the_proxy_of_its_policy` | Of a template of a network with egress allowed, one of a proxy-only network, with an execution as user ID 0 that runs `env`, then one that names its own `HTTPS_PROXY` | A record that names the proxy's URL, `http://192.0.2.1:PORT`, at the port of the proxy's listener, and no DNS server; the first execution with its own entry, then the URL in `http_proxy`, `https_proxy`, `HTTP_PROXY`, and `HTTPS_PROXY`; the second with exactly its own environment; one network device, and the image's own `/etc/resolv.conf`, which no bind holds | +| `a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver` | Of a template of a network with egress allowed, one with a generated machine ID, sharing a host directory read-write | Started, with one mapping, and a record that names the export and the gateway's DNS server; on Linux, the VMM's one `--mount`, read-write, and its `--restore-network-policy`, then the arguments of the clone's policy; below the root, the binds of the machine ID and of the resolver configuration, then the export's mapping; the export's directory, and a write through it, which the host's directory holds after the stop | | `a_selection_that_names_no_template_leases_nothing` | A start that names a template store that does not exist, then, once `remove` retired the generation, which no lease kept, a start of it | Status 10, with the record of a selection that names no template, by the evidence `no-store`, then `retired`, without a lease or a VM; and a store that indexes no generation | | `a_run_runs_one_execution_in_its_clone_then_tears_the_clone_down` | A run over a scratch image, as nobody, whose execution names its environment and its working directory, and ends with status 3, then a run in RAM, as nobody, whose execution names an identity of its own | Each with status 0, in one command: the preflight's `staged`, the clone started as a start starts one, the execution, and a stop that releases the lease, without a VM; the first execution's output, of the Ubuntu image's root, as nobody, in its working directory, with its environment, and its status; the second refused with `EINVAL`; no lease left, and the payload with nothing but the capture's files | | `a_run_whose_clone_does_not_start_runs_nothing` | A run that names a template store that does not exist, one with a clock fixture past the restore window, and one over a scratch image of zeros | Once the preflight staged the images, status 10, with the record of a selection that names no template, by the evidence `no-store`, then of an `expired` generation, without a lease or a VM; then status 8, with the record of a failure of the request at the activation, once the VM exited, the lease released, and no execution or stop | @@ -2868,12 +2896,16 @@ directory that the starts authorize: | `a_run_s_execution_finds_its_shares_beneath_the_root` | A run over a scratch image, sharing a host directory read-write, whose execution, as user ID 0, reads a file of the directory and writes another | Status 0, its clone started with one mapping and stopped as a run's; the file's contents in the execution's output; and the written file in the host's directory once the run tore the clone down | | `a_run_whose_clone_s_vm_exits_during_its_execution_ends_with_the_clone_s_failure` | A run over a scratch image, sharing a host directory read-write, whose execution, as user ID 0, writes a file there, then waits, while the test terminates the clone's VMM | Status 8, its clone started with one mapping, the diagnostic of the executions' failure, `stopped=exited`, and the record of a failure of an unknown cause, at the run, by the evidence of the clone's lost status, without a status of the guest's, once the VM exited, which quarantines nothing, and no execution; the lease released, with the records; then another clone that starts and stops | -Every network that the scenarios give a template is one that a template takes: -a network that lets nothing through, egress allowed, with outbound -host-loopback access or without, one rule that allows TCP to the gateway at the -port of a listener of host loopback, or a proxy at the port of another. The -tests bind those listeners, and a third, which no network names, on host IPv4 -loopback, at ports that the host chooses, and hold them until they end. A +The scenarios' networks let nothing through, or allow egress, with outbound +host-loopback access or without, or one rule that allows TCP to the gateway at +the port of a listener of host loopback, with outbound host-loopback access, or +have a proxy at the port of another, or allow ingress, or host-loopback access +both ways, with a forward or without. The tests bind those listeners, and a +third, which no network names, on host IPv4 loopback, at ports that the host +chooses, and hold them until they end. The forward's port of host loopback, and +the guest's port of the listener that the scenarios run in the guest, to which +it forwards, are ports that no socket of the host held when the tests drew +them. A clone's start names its network by a file that the test writes in the template's directory. @@ -3016,17 +3048,12 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_layered` | `unstaged_images_are_refused_before_spawn` | Images that the store does not stage are refused before the VM spawns, with nothing of a launch left, and boot once staged | | `vm_template` | `a_class_that_no_capture_takes_is_refused_before_the_store` | `create` of a class that no capture takes, of 3 processors, or of more online at boot than in all, fails with status 1 before the store exists | | `vm_template` | `a_network_that_lets_nothing_through_has_no_nic` | `create` of a network that lets nothing through captures a template without a NIC | -| `vm_template` | `a_proxy_only_network_has_its_nic` | `create` of a proxy-only network captures a template with a NIC | | `vm_template` | `a_refresh_captures_and_retires_the_generations_that_it_supersedes` | `refresh` captures a new generation although the store holds an eligible one, and its eviction retires the one that it supersedes, or tombstones it while a clone leases it, as `list` and `inspect` report | | `vm_template` | `a_refresh_refuses_a_class_that_no_capture_takes_before_the_store` | `refresh` of a class that no capture takes fails with status 1 before the store exists | | `vm_template` | `a_removal_reports_busy_while_leases_keep_the_generation` | `remove` tombstones a leased generation, keeps its payload, and reports busy, with status 7 and its leases, until no lease remains, then deletes it | | `vm_template` | `a_template_whose_processors_are_all_online_at_boot_limits_none` | `create` captures a template whose processors are all online at boot without `maxcpus=`, and whose key records none online at boot | | `vm_template` | `a_template_without_a_network_has_no_nic` | `create` captures a template without a NIC, whose key holds what it asked for, its processor online at boot among it, and serves it again without a capture | -| `vm_template` | `an_allow_rule_has_its_nic` | `create` of an allow rule captures a template with a NIC | -| `vm_template` | `an_egress_network_has_its_nic` | `create` of a network with egress allowed captures a template with a NIC | -| `vm_template` | `host_loopback_both_ways_is_refused_before_any_vm_starts` | `create` refuses host-loopback access both ways with status 1, before any VM starts, with no generation | -| `vm_template` | `ingress_is_refused_before_any_vm_starts` | `create` refuses ingress with status 1, before any VM starts, with no generation | -| `vm_template` | `outbound_host_loopback_has_its_nic` | `create` with outbound host-loopback access captures a template with a NIC | +| `vm_template` | `the_networks_of_every_policy_at_one_guest_s_address_share_one_template` | `create` of a network of any policy captures one template, whose NIC the capture attaches under the complete deny-all policy, and the creations of the networks of every other policy at the same guest's address serve it without a capture | | `vm_clone` | `a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only` | A clone of a network with egress allowed binds a resolver file that names the gateway's DNS server read-only at `/etc/resolv.conf`, and a TCP connection to the server's DNS port opens, which the gateway's DNS proxy accepts; one whose resolver cannot be bound ends with status 57 | | `vm_clone` | `a_clone_maps_its_export_beneath_its_root_before_its_sandbox_starts` | A clone maps its export's mappings beneath its root before its sandbox starts, each with its access, a denied file hidden, and writes reach the host's directory through the writable ones alone | | `vm_clone` | `a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver` | A clone of a template with a NIC maps its export below the binds of its machine ID and of its resolver configuration | @@ -3039,10 +3066,11 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_clone` | `a_clone_whose_mapping_fails_ends_before_its_sandbox_starts` | A clone whose guest fails a mapping ends before its sandbox starts, with the failure's record, and leaves the host's directory unchanged | | `vm_clone` | `a_clone_whose_view_cannot_be_set_ends_before_it_is_ready` | A generated machine ID that no bind can hold ends the clone before it is ready, with status 55, while the image's own needs no bind | | `vm_clone` | `a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released` | A clone whose VM exits once a stop found it running fails, with status 8 and the record of a failure by the evidence of its lost status, quarantines nothing, and releases its lease | +| `vm_clone` | `a_forward_whose_port_another_socket_holds_fails_the_clone_as_the_host_s` | A forward whose port of host loopback another socket holds ends the clone before its restore-ready event, with status 8 and the record of a failure of the host, by the evidence of the forward's bind, and the template serves the network once the port is free | | `vm_clone` | `a_generation_is_eligible_by_the_clock_of_its_capture` | The store checks eligibility by the host's monotonic clock on the boot of the capture, and by UTC after a reboot, and refuses an expired or an early generation with status 10, without a lease | | `vm_clone` | `a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with` | NVX's plan refuses a processor target below the template's processors online at boot before any VMM starts, and the generation serves its clones from those up | | `vm_clone` | `a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity` | A host program of another runtime ends the clone through its verified identity, without a word to its guest, with status 8, and releases its lease | -| `vm_clone` | `a_run_in_a_template_of_a_network_finds_its_policy_s_proxy` | A run's execution, in a clone of a proxy-only template, finds the proxy's URL in its environment, and a run of another network runs nothing, with status 8 | +| `vm_clone` | `a_run_in_a_template_of_a_network_finds_its_policy_s_proxy` | A run's execution, in a clone of a proxy-only template, finds the proxy's URL in its environment, and a run of a network that lets nothing through runs nothing, with status 8 | | `vm_clone` | `a_run_runs_one_execution_in_its_clone_then_tears_the_clone_down` | A run stages its images, starts a clone, runs one execution in its root, with the execution's environment and working directory, then stops the clone and releases its lease, whatever the execution did | | `vm_clone` | `a_run_s_execution_finds_its_shares_beneath_the_root` | A run's execution reads and writes through its share, which the clone mapped beneath its root, and the host's directory keeps what it wrote | | `vm_clone` | `a_run_whose_clone_does_not_start_runs_nothing` | A run whose selection the store refuses ends with status 10, and one whose clone fails with status 8, without an execution or a stop | @@ -3050,7 +3078,6 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_clone` | `a_selection_that_names_no_template_leases_nothing` | A start of a template store that does not exist, or of a generation that `remove` retired, ends with status 10 and the refusal's record, without a lease or a VM | | `vm_clone` | `a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts` | An export of two directories, or of a file alone, is refused with status 8 before any VMM starts | | `vm_clone` | `a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts` | Images, scratch images, and memory targets that the template cannot serve are refused with status 8 before any VMM starts, unstaged images with status 1, and a generation that the store does not hold with status 10 | -| `vm_clone` | `a_template_serves_the_clones_of_its_own_network_policy_alone` | A template with a NIC serves the clones of its own network alone, whose restores attach its NIC under its policy, and NVX's plan refuses the others with status 8 before any VMM starts | | `vm_clone` | `a_template_without_a_nic_serves_clones_without_a_network` | A template without a NIC serves clones without a network, or with one that lets nothing through, and refuses those of a network with a NIC with status 8 | | `vm_clone` | `a_tombstone_leaves_the_running_clone_of_its_generation_to_its_lease` | A tombstone refuses new clones of its generation with status 10, while the running clone serves on, and releases its lease once it stops | | `vm_clone` | `clones_at_once_keep_their_own_uppers_and_machine_ids` | Clones at once keep their own uppers, and their own machine IDs, generated, distinct, and bound read-only, or the image's own | @@ -3059,6 +3086,7 @@ layered roots, captures of templates, and the starts and runs of their clones. | `vm_clone` | `one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity` | One generation serves clones of each supported processor count from the template's processors online at boot to its capacity, whose repair brings the target's CPUs online, and NVX's plan refuses an unsupported count and one beyond the capacity before any VMM starts | | `vm_clone` | `one_generation_serves_clones_of_each_upper_and_memory_target` | One generation serves clones over a scratch image and in RAM, at each memory target up to its capacity, each with its own lease, and leaves its snapshot unchanged | | `vm_clone` | `one_generation_serves_clones_of_either_format_over_either_upper` | A template of a GPT placeholder serves EROFS and GPT clones of other layouts, shorter than the placeholder, at its length, over either upper, those in RAM with no scratch file, and a first partition over its bound ends a clone with status 54 | +| `vm_clone` | `one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address` | One template with a NIC serves the clones of every network policy at its guest's address, whose restores install their own policies on its NIC, which lets through what each allows, publishes the guest's listeners, through the clone's announcer, and forwards ports where it says, and NVX's plan refuses another address, or no NIC, with status 8 before any VMM starts | | `vm_clone` | `the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store` | The lease of a start that ended before it recorded its clone is released only through its own template store, which `--template` names | ### Running the VM tests diff --git a/tests/runtime-adapter/scenarios.txt b/tests/runtime-adapter/scenarios.txt index 7e036652f..fea23052f 100644 --- a/tests/runtime-adapter/scenarios.txt +++ b/tests/runtime-adapter/scenarios.txt @@ -129,17 +129,12 @@ vm_layered unstaged_images_are_refused_before_spawn vm_template a_class_that_no_capture_takes_is_refused_before_the_store vm_template a_network_that_lets_nothing_through_has_no_nic -vm_template a_proxy_only_network_has_its_nic vm_template a_refresh_captures_and_retires_the_generations_that_it_supersedes vm_template a_refresh_refuses_a_class_that_no_capture_takes_before_the_store vm_template a_removal_reports_busy_while_leases_keep_the_generation vm_template a_template_whose_processors_are_all_online_at_boot_limits_none vm_template a_template_without_a_network_has_no_nic -vm_template an_allow_rule_has_its_nic -vm_template an_egress_network_has_its_nic -vm_template host_loopback_both_ways_is_refused_before_any_vm_starts -vm_template ingress_is_refused_before_any_vm_starts -vm_template outbound_host_loopback_has_its_nic +vm_template the_networks_of_every_policy_at_one_guest_s_address_share_one_template vm_clone a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only vm_clone a_clone_maps_its_export_beneath_its_root_before_its_sandbox_starts @@ -153,6 +148,7 @@ vm_clone a_clone_whose_activation_fails_ends_before_it_is_ready vm_clone a_clone_whose_mapping_fails_ends_before_its_sandbox_starts vm_clone a_clone_whose_view_cannot_be_set_ends_before_it_is_ready vm_clone a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released +vm_clone a_forward_whose_port_another_socket_holds_fails_the_clone_as_the_host_s vm_clone a_generation_is_eligible_by_the_clock_of_its_capture vm_clone a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with vm_clone a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity @@ -164,7 +160,6 @@ vm_clone a_run_whose_clone_s_vm_exits_during_its_execution_ends_with_the_clone_s vm_clone a_selection_that_names_no_template_leases_nothing vm_clone a_start_refuses_an_export_that_a_dormant_filesystem_slot_cannot_bind_before_any_vmm_starts vm_clone a_start_refuses_what_the_template_cannot_serve_before_any_vmm_starts -vm_clone a_template_serves_the_clones_of_its_own_network_policy_alone vm_clone a_template_without_a_nic_serves_clones_without_a_network vm_clone a_tombstone_leaves_the_running_clone_of_its_generation_to_its_lease vm_clone clones_at_once_keep_their_own_uppers_and_machine_ids @@ -173,4 +168,5 @@ vm_clone each_command_reattaches_to_the_clone_from_a_new_process vm_clone one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity vm_clone one_generation_serves_clones_of_each_upper_and_memory_target vm_clone one_generation_serves_clones_of_either_format_over_either_upper +vm_clone one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address vm_clone the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-guest.rs b/tests/runtime-adapter/src/bin/runtime-adapter-guest.rs index c6e49517e..fe242e039 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-guest.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-guest.rs @@ -201,24 +201,26 @@ fn capture_template( let boot = operations .command_line() .and_then(|command_line| template::boot(&command_line).map_err(|error| error.to_string())); - match boot { + let gateway = match boot { Ok(Boot::Cold) => return None, - Ok(Boot::Template) => {} + Ok(Boot::Template { gateway }) => gateway, Err(problem) => { eprintln!("runtime-adapter-guest: {problem}"); return Some(COMMAND_LINE_STATUS); } - } - Some(match capture::capture_on_terminal(tty, operations) { - Ok(templated) => { - eprintln!("runtime-adapter-guest: {templated}"); - templated.status() - } - Err(error) => { - eprintln!("runtime-adapter-guest: {error}"); - SESSION_STATUS - } - }) + }; + Some( + match capture::capture_on_terminal(tty, operations, gateway) { + Ok(templated) => { + eprintln!("runtime-adapter-guest: {templated}"); + templated.status() + } + Err(error) => { + eprintln!("runtime-adapter-guest: {error}"); + SESSION_STATUS + } + }, + ) } /// Runs the executions of the worker's `entries` probe in the root `root`, each placed in the cgroup diff --git a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs index 4a209e04f..a4ccff068 100644 --- a/tests/runtime-adapter/src/bin/runtime-adapter-host.rs +++ b/tests/runtime-adapter/src/bin/runtime-adapter-host.rs @@ -83,8 +83,11 @@ //! clones take, since each clone's restore brings its own processor target online, up to N, MIB //! mebibytes of RAM, 256 by default, and a RAM capacity of MIB mebibytes, its RAM by default, to //! which a restore may expand it, and a dormant scratch of BYTES bytes, 64 MiB by default. It has -//! one NIC under the policy of the network FILE, which the creation reads as a launch reads its -//! network, unless the policy lets nothing through, and none without a network. The creation +//! one NIC at the guest's address of the network FILE, which the creation reads as a launch reads +//! its network, under the complete deny-all policy, whatever the network's own, which each +//! clone's restore replaces with the clone's, unless the network's policy lets nothing through, +//! and none without a network: the networks of every policy at one guest's address share one +//! class, and so one template. The creation //! reads the network before anything else, then stages the placeholder as `stage` stages an //! image, verifying it explicitly, which hashes it only when a manifest is beside it, and checks //! the class before it creates the store. A capture boots the template's VM through NVX's @@ -129,9 +132,10 @@ //! generation G of the template store TEMPLATES, an `nvx::host::template::Store`, in the sandbox //! store STORE, which it creates if needed; see `runtime_adapter::clone`. The clone's VM has the //! network of the file FILE, if given, which the command reads first, as `launch` reads its -//! network: it must be the template's, the guest's address and the policy of the template's NIC, -//! which every restore of the template attaches, or for a template without a NIC none, or one -//! that lets nothing through. SHARES, as `launch` takes them, share +//! network: it must be at the guest's address of the template's NIC, under a policy of its own, +//! which the clone's restore installs on the NIC in place of the template's complete deny-all +//! policy, or for a template without a NIC none, or one that lets nothing through. SHARES, as +//! `launch` takes them, share //! host paths with the clone's root: the command plans their export as a launch does, with //! NVX's `filesystem::plan`, and refuses shares that it cannot export before anything but its //! network. It then reads the generation from the store's index, and, unless N and MIB @@ -144,7 +148,8 @@ //! `Store::acquire`, which refuses a generation that a tombstone quarantines, or that is not //! eligible then, and NVX's `ClonePlan` checks the clone against the generation, from the //! metadata of its files -//! alone: its network, which must be the one that the template's key records, its image, the +//! alone: its network, which must attach a NIC exactly when the template's key records one, at +//! the guest's address of the template's NIC, under a policy of its own, its image, the //! raw file PATH, an EROFS filesystem, `erofs`, or a GPT disk, `gpt`, of //! which the guest expects the identity UUID, no longer than the template's placeholder, //! whatever the placeholder's format, and of its block sizes, its upper, the ext4 scratch image @@ -167,11 +172,15 @@ //! the clone's root before it is ready, the overlay mode MODE and the scratch's journal policy //! POLICY, as a launch takes them, for a GPT disk, the bound BYTES of its first partition, 64 //! KiB unless given, as a launch's guest takes it by default, the export, by the identity of -//! its plan, with the number of its mappings, and for a template with a NIC its network: the -//! identity of its policy, the DNS server with which the template's guest configured its -//! resolver at boot, if any, whose address the clone's guest writes to a resolver file that it -//! binds read-only over the root's `/etc/resolv.conf`, and the URL of the policy's proxy, if -//! any, which every execution then finds in its environment, as a launch's executions do. Every +//! its plan, with the number of its mappings, and for a template with a NIC the clone's network: +//! the identity of its policy, the DNS server that OpenVMM serves the guest under the policy, if +//! any, which NVX finds as OpenVMM does, since the template's guest learned none, and whose +//! address the clone's guest writes to a resolver file that it binds read-only over the root's +//! `/etc/resolv.conf`, whether the policy publishes the workload's listeners, as it does with +//! ingress, or with host-loopback access both ways, for which the clone's guest starts their +//! announcer and refuses `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW` to its executions, and the +//! URL of the policy's proxy, if any, which every execution then finds in its environment, as a +//! launch's executions do. Every //! execution of the clone sees the hostname //! `runtime-adapter`, and the machine ID that `--machine-id` chooses: the image's own, `image`, //! by default, or one that the guest generates once for the clone, `generated`, which a diff --git a/tests/runtime-adapter/src/capture.rs b/tests/runtime-adapter/src/capture.rs index 22ffbb406..6e7c7aa3b 100644 --- a/tests/runtime-adapter/src/capture.rs +++ b/tests/runtime-adapter/src/capture.rs @@ -20,7 +20,9 @@ //! RAM and its capacity; //! - the geometries of the placeholder and of the dormant scratch, as OpenVMM presents a sandbox //! block device, which [`geometry`] gives; -//! - the network, which the adapter's plan renders too, and no other invariant setting. +//! - the network, of which the class takes the NIC alone, at the guest's address, under the +//! complete deny-all policy, whatever the network's own, which each clone's restore replaces +//! with the clone's, and no other invariant setting. //! //! The capture's VM boots NVX's kernel with the initramfs of the adapter's VMs, which puts the //! guest agent in place of NVX's agents; see [`crate::initramfs`]. NVX gives it the kernel command @@ -94,7 +96,8 @@ pub struct Request { pub memory_capacity: u64, /// The length of the dormant scratch, in bytes. pub scratch_length: u64, - /// The network of the template's fixed NIC, or `None` for a template without a NIC. + /// The network at whose guest's address the template has its fixed NIC, under the complete + /// deny-all policy, or `None` for a template without a NIC. pub network: Option, } diff --git a/tests/runtime-adapter/src/clone.rs b/tests/runtime-adapter/src/clone.rs index 77fa06b10..d44e25edd 100644 --- a/tests/runtime-adapter/src/clone.rs +++ b/tests/runtime-adapter/src/clone.rs @@ -8,15 +8,16 @@ //! first partition, the machine ID that its executions see, the image's own or one that its //! guest generates, the host paths that it shares with its root, as a launch shares them, //! `sharing::HostFsSharing`, and its network, as a launch reads it, `layered::read_network`, -//! which must be the template's. +//! which must be at the guest's address of the template's NIC, under a policy of its own. //! [`Request::plan`] plans the image over the upper, with the network, as a launch plans a layered //! root, [`Request::export`] plans the export of the shares as a launch plans it, and //! [`Request::clone_request`] returns NVX's `nvx::host::template::CloneRequest` of both, with the //! bindings that the sandbox's store stages for its images, as `staged::Staged` records them: //! staging hashes, and a start never does. NVX's `ClonePlan` then checks the request against the //! leased generation before any VMM starts, from the metadata of the clone's files alone, takes -//! only the template's network, as the template's key records it, and only an export of one -//! directory, which the clone's restore binds to the template's dormant filesystem slot. +//! only a network at the guest's address of the template's NIC, whose policy the clone's restore +//! installs on the NIC in place of the template's complete deny-all policy, and only an export of +//! one directory, which the clone's restore binds to the template's dormant filesystem slot. //! //! A clone's [record](CloneRecord) names the template store and the generation that the clone //! leases, beside the plan and the bindings, as the record of a launch names what it launched, so @@ -98,9 +99,9 @@ pub struct Request { /// The host paths that the clone shares with its root, as a launch shares them, if any. pub sharing: HostFsSharing, /// The network of the clone's VM, as a launch reads it, `layered::read_network`, if it has - /// one: the template's network, the guest's address and the policy of the template's NIC, which - /// every restore of the template attaches, or for a template without a NIC none, or one that - /// lets nothing through. + /// one: a network at the guest's address of the template's NIC, under a policy of its own, + /// which the clone's restore installs on the NIC, or for a template without a NIC none, or one + /// that lets nothing through. pub network: Option, } diff --git a/tests/runtime-adapter/src/template.rs b/tests/runtime-adapter/src/template.rs index 1f3203741..7ed9e6c5c 100644 --- a/tests/runtime-adapter/src/template.rs +++ b/tests/runtime-adapter/src/template.rs @@ -15,7 +15,9 @@ //! quotes from a parameter, and hands what follows `--` to init rather than taking it as //! parameters, though init may still read it from `/proc/cmdline`, so `boot` also refuses a //! template's command line that holds either, as OpenVMM refuses them in the command line of a VM -//! with a control console. +//! with a control console. It then reads the IPv4 gateway of the template's NIC, if the template +//! has one, with `nvx::protocol::cmdline::gateway`, which refuses a gateway that OpenVMM would not +//! render: the announcer of each clone that publishes its listeners announces them to it. //! //! The worker of a template's VM, and with it the VM, ends with [`COMMAND_LINE_STATUS`] once //! `boot` refused its command line, with the bootstrap's status, 13, once the bootstrap of the @@ -27,8 +29,10 @@ //! documents too, or, once the clone was ready, as the worker of a VM that boots cold does. A //! committed capture ends the template's VM before the worker returns. +use std::net::Ipv4Addr; use std::{fmt, str}; +use nvx::protocol::ProtocolError; use nvx::protocol::cmdline::{self, Parameter}; /// Returns whether `token`, a parameter of a kernel command line as the command line spells it, @@ -98,7 +102,11 @@ pub enum Boot { Cold, /// A template's VM, which captures before it assembles any root: its kernel command line /// names the platform tier, and holds that tier's tokens alone. - Template, + Template { + /// The IPv4 gateway of the template's NIC, which the command line names, if the template + /// has a NIC. + gateway: Option, + }, } /// Why [`boot`] refused the kernel command line of a template's VM. @@ -111,22 +119,26 @@ pub enum CommandLineError { Quote, /// A `--`, after which the kernel hands the rest of the command line to init. Delimiter, + /// A gateway that OpenVMM would not render, as `nvx::protocol::cmdline::gateway` tells: a + /// `virtnet_gw=` token that repeats, or that does not name one IPv4 address in its canonical + /// text. + Gateway(ProtocolError), } impl fmt::Display for CommandLineError { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter - .write_str("the kernel command line names the platform snapshot tier, but holds ")?; + formatter.write_str("the kernel command line names the platform snapshot tier, but ")?; match self { Self::Token(token) => write!( formatter, - "`{}`, which the tier's allowlist does not hold", + "holds `{}`, which the tier's allowlist does not hold", token.escape_ascii() ), - Self::Quote => formatter.write_str("a double quote, which the kernel drops"), + Self::Quote => formatter.write_str("holds a double quote, which the kernel drops"), Self::Delimiter => { - formatter.write_str("`--`, after which the kernel hands the rest to init") + formatter.write_str("holds `--`, after which the kernel hands the rest to init") } + Self::Gateway(error) => formatter.write_str(error.message()), } } } @@ -135,11 +147,12 @@ impl std::error::Error for CommandLineError {} /// Returns how a VM boots, as its kernel command line, `command_line`, such as `/proc/cmdline`, /// tells, as the [module documentation](self) describes: a command line names the platform tier -/// if the kernel finds [`PLATFORM_TIER`] among its parameters. +/// if the kernel finds [`PLATFORM_TIER`] among its parameters, and a template's names the gateway +/// of its NIC, if it has one. /// /// Fails with a [`CommandLineError`] if the command line names the platform tier, but holds a /// double quote, or else a `--`, or else a parameter that [`is_platform_token`] does not accept, -/// the first in the command line's order. +/// the first in the command line's order, or else a gateway that OpenVMM would not render. pub fn boot(command_line: &[u8]) -> Result { let template = cmdline::parameters(command_line) .any(|parameter| token(¶meter) == PLATFORM_TIER.as_bytes()); @@ -156,13 +169,14 @@ pub fn boot(command_line: &[u8]) -> Result { { return Err(CommandLineError::Delimiter); } - match cmdline::parameters(command_line) + if let Some(token) = cmdline::parameters(command_line) .map(|parameter| token(¶meter)) .find(|token| !is_platform_token(token)) { - Some(token) => Err(CommandLineError::Token(token)), - None => Ok(Boot::Template), + return Err(CommandLineError::Token(token)); } + let gateway = cmdline::gateway(command_line).map_err(CommandLineError::Gateway)?; + Ok(Boot::Template { gateway }) } /// Returns `parameter` as the command line spells it, without quotes: its name, then `=` and its @@ -354,10 +368,16 @@ mod tests { &format!(" {NETWORK} nvx_control_tty"), 1, ); + // A template with a NIC names its gateway, which the announcers of its clones take. + assert_eq!( + boot(network.as_bytes()), + Ok(Boot::Template { + gateway: Some(Ipv4Addr::new(10, 0, 2, 2)) + }) + ); for command_line in [ TEMPLATE.to_owned(), TEMPLATE.trim_end().to_owned(), - network, // A template that brings fewer processors online at boot than its capacity. TEMPLATE.replacen(" nvx_control_tty", " maxcpus=1 nvx_control_tty", 1), // The tier alone, and each token whole once more, which OpenVMM leaves to its own @@ -369,7 +389,7 @@ mod tests { ] { assert_eq!( boot(command_line.as_bytes()), - Ok(Boot::Template), + Ok(Boot::Template { gateway: None }), "{command_line:?}" ); } @@ -379,7 +399,38 @@ mod tests { *byte = 0xa0; } } - assert_eq!(boot(&latin1), Ok(Boot::Template)); + assert_eq!(boot(&latin1), Ok(Boot::Template { gateway: None })); + } + + #[test] + fn template_mode_refuses_a_gateway_that_openvmm_would_not_render() { + for (gateway, problem) in [ + ( + "virtnet_gw=10.0.2.2 virtnet_gw=10.0.2.2", + "the virtnet_gw kernel parameter is repeated", + ), + ( + "virtnet_gw=010.0.2.2", + "the virtnet_gw kernel parameter must name one IPv4 address as OpenVMM renders \ + it, not '010.0.2.2'", + ), + ( + "virtnet_gw=fd00::2", + "the virtnet_gw kernel parameter must name one IPv4 address as OpenVMM renders \ + it, not 'fd00::2'", + ), + ] { + let command_line = format!("{} {gateway}\n", TEMPLATE.trim_end()); + let refused = boot(command_line.as_bytes()).unwrap_err(); + assert!( + matches!(&refused, CommandLineError::Gateway(error) if error.message() == problem), + "{refused:?}" + ); + assert_eq!( + refused.to_string(), + format!("the kernel command line names the platform snapshot tier, but {problem}") + ); + } } #[test] diff --git a/tests/runtime-adapter/src/worker.rs b/tests/runtime-adapter/src/worker.rs index 0c45c1e67..9c810cc15 100644 --- a/tests/runtime-adapter/src/worker.rs +++ b/tests/runtime-adapter/src/worker.rs @@ -88,11 +88,15 @@ //! one, once, after the repair's reseed. If the record has an export, it mounts the host export //! out of the root, through the template's host filesystem device, `capture::FILESYSTEM_TAG`, as //! the worker of a VM that boots cold mounts the one that its command line names, and if the -//! record's network names a DNS server, it writes a resolver file that names it. It then binds -//! the generated machine ID read-only over the root's `/etc/machine-id`, and the resolver file -//! over its `/etc/resolv.conf`, in the worker's mount namespace, which each execution's copies. -//! The template bootstrapped the guest for its root with its own configuration before its -//! capture, so the clone has its cgroups. +//! record's network names a DNS server, it writes a resolver file that names it. If the record's +//! network publishes the workload's listeners, it starts their announcer, +//! `/sbin/nvx-ingress inherit GATEWAY_IPV4`, from a UDP socket that it binds to a privileged +//! source port, to the gateway that the template's kernel command line names, as NVX's `/init` +//! starts it at a cold boot whose policy publishes them: the template, captured under a policy +//! that publishes nothing, runs none. It then binds the generated machine ID read-only over the +//! root's `/etc/machine-id`, and the resolver file over its `/etc/resolv.conf`, in the worker's +//! mount namespace, which each execution's copies. The template bootstrapped the guest for its +//! root with its own configuration before its capture, so the clone has its cgroups. //! 4. Once it took what the session received while the application ran, which came before `Ready`, //! it completes the activation, which opens the fence, and answers `Ready`, which reports that //! the activation applied, with the record's transaction, after the guest agent's build, to the @@ -105,19 +109,24 @@ //! the stop. The record's workload identity, if it has one, is the whole sandbox's, as a root //! configuration's is, so that executions name none, every execution takes the record's //! hostname, and a clone's executions run in the cgroups of the template's bootstrap. On the -//! stop, the worker closes the export's root and removes the binds of the application before it -//! tears the root down, since they keep the root's mounts busy, through +//! stop, the worker stops the announcer, closes the export's root, and removes the binds of the +//! application before it tears the root down, since they keep the root's mounts busy, through //! `nvx::guest::clone::Applied::teardown`. It also answers the probe //! `activation::ACTIVATION_PROBE`, with when the events of the clone's activation and order //! happened, after the repair's acknowledgement of the restore. //! -//! The application leaves out two of the record's settings. The endpoint of its network's proxy, +//! The application leaves out one of the record's settings. The endpoint of its network's proxy, //! which `nvx::guest::clone::Applied::proxy` returns, is the host's to tell the workloads, as //! after a cold boot: the host program's `exec` gives each execution the proxy's URL in its //! environment, from the plan of the VM's network, so that an execution's environment stays -//! exactly its start's. And the publication of the record's listeners is not applied, so a -//! clone's executions enter the root with listeners published, the entry's default, which refuses -//! the most. +//! exactly its start's. A clone's executions enter the root with the publication of listeners +//! that the record's network names, published under a policy that publishes them, and unpublished +//! otherwise, a clone without a network included. The worker polls the pidfd of the announcer of +//! a record that publishes them with the session's stream, in its loop, and nothing else: once +//! the announcer ended, before the clone is ready or after, the gateway keeps publishing the +//! listeners that it last announced, and learns of no later change of them, so the activation +//! fails finally with `nvx::protocol::status`'s `ACTIVATION_NETWORK`, as an application whose +//! announcer could not start does. //! //! Until the clone is ready, its session takes nothing but that `Activate`: another reset, any //! other message, a `Stop` and a `Ping` included, an `Activate` that the activation refuses, one @@ -208,7 +217,8 @@ //! describes it: in new mount, PID, and UTS namespaces, with the root moved over `/`, with the //! sandbox's hostname, the entry's default, `nvx-sandbox`, after a cold boot, and the record's in //! a clone, and no time status, and with the publication of listeners that the root -//! configuration names, published unless it names them unpublished, as the entry's default is. +//! configuration names, published unless it names them unpublished, as the entry's default is, +//! after a cold boot, and that the record's network names in a clone, unpublished without one. //! After a cold boot, it sees the image's own machine ID and `/etc/resolv.conf`, so that the //! worker binds nothing into the root, and in a clone, the machine ID that the record names, //! which the clone's application bound for every execution if the record asks for a generated @@ -229,7 +239,8 @@ //! controlling terminal, and the process group that the session creates. As PID 1 of its PID //! namespace, its end ends every process of the namespace. The guest's network device, its //! addresses, and the announcer of its listeners stay with NVX's `/init`, which configures them -//! before PID 1 starts. +//! before PID 1 starts, but for the announcer of a clone, which the clone's application starts +//! and the worker supervises. //! - Each execution runs in a cgroup of its own below the workload cgroup, `execution-N`, where N //! counts the cgroups that the worker prepared. The worker prepares it with //! `nvx::guest::cgroup::ExecutionCgroup::prepare`, and the program joins it as it spawns, @@ -289,7 +300,7 @@ pub mod probes; use std::fs::{self, File, OpenOptions}; use std::io::{self, Read, Write}; -use std::os::fd::AsFd; +use std::os::fd::{AsFd, BorrowedFd}; use std::os::unix::fs::OpenOptionsExt; use std::os::unix::process::ExitStatusExt; use std::path::{Path, PathBuf}; @@ -309,6 +320,8 @@ use nvx::guest::cgroup::Layout; #[cfg(target_arch = "x86_64")] use nvx::guest::clone::{self, Activation, ApplyError, RootTemplate}; use nvx::guest::entry::{self, Entry, Identity, Prepared, Publication}; +#[cfg(target_arch = "x86_64")] +use nvx::guest::ingress::Announcer; use nvx::guest::mappings::{Mapping, SourceRoot}; use nvx::guest::nvxs::GuestConnectionError; use nvx::guest::process::Child; @@ -322,6 +335,7 @@ use nvx::protocol::nvxs::GuestSession; use nvx::protocol::root::{ DISTRO_MMIO_BASE, ImageFormat, RootLayout, RootUpper, SCRATCH_MMIO_BASE, }; +use nvx::protocol::status; use crate::GUEST_BUILD; use crate::exec::HostMessage; @@ -450,17 +464,29 @@ pub trait RootOperations: EntryOperations { /// Runs `probe` on the assembled root, and returns what it found. fn probe(&mut self, probe: &Probe) -> Result; - /// Tears the assembled root down, once it removed the runtime's binds that a clone's - /// application, through `apply`, made on it, which keep the root's mount busy. + /// Tears the assembled root down, once it stopped the announcer that a clone's application, + /// through `apply`, started, if any, and removed the runtime's binds that the application made + /// on it, which keep the root's mount busy. fn teardown(&mut self) -> Result<(), String>; + /// Returns the pidfd of the announcer of the workload's listeners that a clone's application, + /// through `apply`, started for a record whose network publishes them, which polls readable + /// once the announcer ended, or `None` without one, or once its status was collected. + fn announcer(&self) -> Option>; + + /// Collects the status of the announcer of the workload's listeners, if it ended, without + /// blocking, and returns how it ended, as a problem, or `None` while it runs, or without one. + fn announcer_ended(&mut self) -> Option; + /// Applies `record`, the record of the Activate that a clone's activation accepted, to the /// clone, as `nvx::guest::clone::apply` does, with `template`, the inputs of the root's /// assembly that the clone's template captured, by `deadline`, the activation's, if any, until /// which it waits for each device at the latest, and keeps the root that it assembled, with the - /// runtime's binds that it made on it, which the [teardown](Self::teardown) removes first, and - /// the root of the host export that it mounted, if the record has one, from which - /// [`map`](Self::map) maps the host's mapping table: in place of a root configuration's + /// runtime's binds that it made on it, which the [teardown](Self::teardown) removes first, the + /// root of the host export that it mounted, if the record has one, from which + /// [`map`](Self::map) maps the host's mapping table, and the announcer of the workload's + /// listeners that it started, if the record's network publishes them, whose + /// [pidfd](Self::announcer) the worker polls: in place of a root configuration's /// [layout](Self::command_line), assembly, [export](Self::mount_export), and account check. /// The template's VM [bootstrapped](Self::bootstrap) the guest before its capture. /// @@ -501,7 +527,8 @@ pub enum Worked { /// closed. Abandoned, /// A clone's activation failed, finally: before the clone was ready, which left the worker's - /// fence of workload admission closed, or once it was ready. + /// fence of workload admission closed, or once it was ready, as once the announcer of the + /// workload's listeners that its application started ended. Unactivated { /// The status of `nvx::protocol::status` of the failure's class, with which the worker /// ends. @@ -770,7 +797,20 @@ fn serve( if backlogged && !connection.is_backlogged() { continue; } - wait(&connection, &executions, phase.deadline())?; + let announcer = operations.announcer(); + // Once a clone's announcer ended, whenever it ends, the gateway keeps publishing the + // listeners that it last announced, and learns of no later change of them: the clone's + // activation fails finally with the network's class, which ends the clone. + if wait(&connection, &executions, phase.deadline(), announcer)? + && let Some(problem) = operations.announcer_ended() + { + executions.clear(); + let worked = Worked::Unactivated { + status: status::ACTIVATION_NETWORK, + problem, + }; + return end(&mut connection, worked); + } } } @@ -840,15 +880,18 @@ fn open_tty(tty: &Path) -> Result { }) } -/// Waits until the session's stream or an execution's descriptor polls ready, the earliest -/// deadline of the executions passes, or `deadline`, the session's own, if it has one; at once if -/// the connection holds bytes that the worker takes next. The worker waits for the host's messages -/// unless the connection is overloaded, and for the executions' output unless it is backlogged. +/// Waits until the session's stream, an execution's descriptor, or `announcer`, the pidfd of the +/// announcer of a clone's listeners, if any, polls ready, the earliest deadline of the executions +/// passes, or `deadline`, the session's own, if it has one; at once if the connection holds bytes +/// that the worker takes next. The worker waits for the host's messages unless the connection is +/// overloaded, and for the executions' output unless it is backlogged. Returns whether the +/// announcer's pidfd polled readable, once the announcer ended. fn wait( connection: &Connection, executions: &Executions, deadline: Option, -) -> Result<(), ServeError> { + announcer: Option>, +) -> Result { let overloaded = connection.is_overloaded(); let mut events = PollFlags::empty(); if !overloaded { @@ -859,6 +902,10 @@ fn wait( } let mut descriptors = vec![PollFd::new(connection.descriptor(), events)]; executions.poll_descriptors(&mut descriptors, connection.is_backlogged()); + let announcing = announcer.map(|pidfd| { + descriptors.push(PollFd::new(pidfd, PollFlags::POLLIN)); + descriptors.len() - 1 + }); let deadline = [executions.deadline(), deadline] .into_iter() .flatten() @@ -882,7 +929,11 @@ fn wait( } }; match poll::poll(&mut descriptors, timeout) { - Ok(_) | Err(Errno::EINTR) => Ok(()), + Ok(_) | Err(Errno::EINTR) => Ok(announcing.is_some_and(|index| { + descriptors[index] + .revents() + .is_some_and(|events| !events.is_empty()) + })), Err(error) => Err(ServeError::Session(GuestConnectionError::Io(error.into()))), } } @@ -1324,9 +1375,10 @@ impl RootOperations for LibraryRoot { } /// Runs `nvx::guest::root::Root::teardown`, once it closed the host export's root, which keeps - /// the export busy, and, for a clone, `nvx::guest::clone::Applied::teardown`, which removes - /// the runtime's binds that the application made on the root first, since they keep the root's - /// mount busy too. A root whose teardown failed keeps its mounts. + /// the export busy, and, for a clone, `nvx::guest::clone::Applied::teardown`, which stops the + /// announcer that the application started, if any, and removes the runtime's binds that the + /// application made on the root first, since they keep the root's mount busy too. A root whose + /// teardown failed keeps its mounts. fn teardown(&mut self) -> Result<(), String> { self.export = None; match self.root.take() { @@ -1335,9 +1387,39 @@ impl RootOperations for LibraryRoot { } } + /// Lends the pidfd of the announcer that a clone's application started, through + /// `nvx::guest::clone::Applied::announcer`. + fn announcer(&self) -> Option> { + match &self.root { + #[cfg(target_arch = "x86_64")] + Some(Assembled::Clone(applied)) => applied.announcer().map(Announcer::pidfd), + _ => None, + } + } + + /// Collects the status of the announcer that a clone's application started, with + /// `nvx::guest::ingress::Announcer::try_wait`. + fn announcer_ended(&mut self) -> Option { + match &mut self.root { + #[cfg(target_arch = "x86_64")] + Some(Assembled::Clone(applied)) => match applied.announcer_mut()?.try_wait() { + Ok(None) => None, + Ok(Some(status)) => Some(format!( + "the announcer of the workload's listeners ended: {status}" + )), + Err(error) => Some(format!( + "the status of the announcer of the workload's listeners cannot be collected: \ + {error}" + )), + }, + _ => None, + } + } + /// Runs `nvx::guest::clone::apply`, and keeps the root and the runtime's binds that it made, - /// and the root of the host export that it mounted, if the record has one, from which the - /// worker maps the host's mapping table, as after a cold boot. + /// the announcer that it started, if any, and the root of the host export that it mounted, if + /// the record has one, from which the worker maps the host's mapping table, as after a cold + /// boot. #[cfg(target_arch = "x86_64")] fn apply( &mut self, diff --git a/tests/runtime-adapter/src/worker/activation.rs b/tests/runtime-adapter/src/worker/activation.rs index 5aac7dfbc..f1057d7ef 100644 --- a/tests/runtime-adapter/src/worker/activation.rs +++ b/tests/runtime-adapter/src/worker/activation.rs @@ -28,7 +28,7 @@ use std::io::{Read, Write}; use std::os::fd::AsFd; use std::time::{Duration, Instant}; -use nvx::activation::{ActivationRecord, ReadyActivation}; +use nvx::activation::{self, ActivationRecord, ReadyActivation}; use nvx::guest::clone::{ Activation, ActivationAction, ActivationCause, ActivationFailure, ActivationInput, ApplyError, RootTemplate, @@ -84,8 +84,10 @@ pub const ACTIVATION_PROBE: &str = "activation"; /// - [`ACTIVATION_EXPORT`](status::ACTIVATION_EXPORT): the host export of the record cannot be /// mounted, as [`ApplyError::Export`] tells; /// - [`ACTIVATION_NETWORK`](status::ACTIVATION_NETWORK): the resolver file that names the DNS -/// server of the record's network cannot be written or set for every execution, as -/// [`ApplyError::Network`] tells; +/// server of the record's network cannot be written or set for every execution, or the +/// announcer of the workload's listeners, which the network publishes, cannot start, as +/// [`ApplyError::Network`] tells. The worker also ends a clone whose announcer ended with it, +/// whether the clone was ready or not, outside the activation; /// - [`ACTIVATION_ORDER`](status::ACTIVATION_ORDER): the worker's own order completed an /// activation that did not run, or started the sandbox before the clone was ready, or again. /// @@ -141,6 +143,9 @@ struct Applied { mappings: u32, /// The hostname of every execution: the record's. hostname: Vec, + /// The publication of the listeners of every execution: that of the record's network, and + /// unpublished without one. + publication: Publication, /// The number of retries of the Activate that joined the application, which its completion /// answers too. joined: usize, @@ -287,8 +292,8 @@ impl CloneState { /// Applies `record`, the record of the Activate that the activation accepted, through /// `operations`, with the template's inputs, by the activation's deadline, and keeps the - /// application's result, with the record's hostname and the number of its mappings, until the - /// worker [settles](Self::settle) the activation with it. + /// application's result, with the record's hostname, its publication of listeners, and the + /// number of its mappings, until the worker [settles](Self::settle) the activation with it. fn apply(&mut self, operations: &mut dyn RootOperations, record: &ActivationRecord) { // The application's waits for the devices end by the activation's deadline, past which // the completion fails the activation, whatever the application's result. @@ -297,10 +302,21 @@ impl CloneState { let configuration = record.configuration(); // A record announces at most `nvx::activation::MAX_MAPPINGS` mappings, which fit a `u32`. let mappings = configuration.mappings as u32; + // The executions enter the root with the record's publication of listeners, as a launch's + // executions enter it with its root configuration's: published, which refuses + // `CAP_NET_BIND_SERVICE` and `CAP_NET_RAW`, under a network that publishes them, and + // unpublished otherwise, a clone without a network included. + let publication = match configuration.network.as_ref() { + Some(network) if network.publication == activation::Publication::Published => { + Publication::Published + } + _ => Publication::Unpublished, + }; self.applied = Some(Applied { result, mappings, hostname: configuration.workload.hostname.clone(), + publication, joined: 0, }); } @@ -310,8 +326,9 @@ impl CloneState { /// returns the worker's step: `Ready`, which reports the activation, for the `Activate` and /// for each retry that joined its application, once the clone is ready, after which /// `executions` run in the cgroups that the template's bootstrap prepared, as the record's - /// workload identity, if it has one, with the record's hostname, and the worker's order awaits - /// the record's mappings; or the end, once the activation failed. + /// workload identity, if it has one, with the record's hostname and publication of listeners, + /// and the worker's order awaits the record's mappings; or the end, once the activation + /// failed. pub(super) fn settle( &mut self, operations: &mut dyn RootOperations, @@ -326,10 +343,8 @@ impl CloneState { Ok(activation) => { self.timeline.ready = Some(Instant::now()); let identity = identity.map(|identity| (identity.uid(), identity.gid())); - // The executions' listeners are published, the entry's default, which refuses - // the most, as a root configuration that names no publication has them. let sandbox = - Sandbox::new(identity, Publication::Published).with_hostname(applied.hostname); + Sandbox::new(identity, applied.publication).with_hostname(applied.hostname); executions.run_in(operations.cgroups(), sandbox); self.order = Some(Order::new(applied.mappings)); Step::Continue(vec![ready(activation); 1 + applied.joined]) @@ -442,6 +457,7 @@ mod tests { }; use nvx::guest::clone::{AdmissionFence, NetworkError}; use nvx::guest::entry; + use nvx::guest::ingress::StartError; use nvx::guest::mappings::MappingError; use nvx::guest::root::{self, Assembly, Device, Error as RootError, Role}; use nvx::protocol::root::DISTRO_MMIO_BASE; @@ -632,6 +648,17 @@ mod tests { }; let network = activating().complete(now(), Err(unbound.into())); cases.push((failed(network), status::ACTIVATION_NETWORK)); + for error in [ + NetworkError::Gateway, + NetworkError::Announcer(StartError::NoPrivilegedPort), + ] { + let unannounced = ApplyError::Network { + error, + teardown: None, + }; + let network = activating().complete(now(), Err(unannounced.into())); + cases.push((failed(network), status::ACTIVATION_NETWORK)); + } let other = activating().complete(now(), Err("another failure of the application".into())); cases.push((failed(other), status::ACTIVATION_ROOT)); // The worker's own order. @@ -675,7 +702,10 @@ mod tests { let template = template_config(); assert!(ACTIVATION_TIMEOUT >= 2 * template.device_timeout); - assert_eq!(root_template(&template).timeout, template.device_timeout); + assert_eq!( + root_template(&template, None).timeout, + template.device_timeout + ); } #[test] diff --git a/tests/runtime-adapter/src/worker/capture.rs b/tests/runtime-adapter/src/worker/capture.rs index 195296c1b..cd0050170 100644 --- a/tests/runtime-adapter/src/worker/capture.rs +++ b/tests/runtime-adapter/src/worker/capture.rs @@ -44,7 +44,8 @@ //! clone and of the fence, whose deadline ends [`ACTIVATION_TIMEOUT`] after the repair's //! acknowledgement, it answers the reset that the restore queued for the session that the snapshot //! holds, takes the host's `Activate`, and applies its record with [`root_template`] of the -//! template's configuration, then serves the host's order as after a cold boot, as the +//! template's configuration and its NIC's gateway, then serves the host's order as after a cold +//! boot, as the //! [worker's documentation](super#clones) describes. The clone ends as that session does. Once its //! activation failed, or the broker closed the stream before the clone was ready, the worker waits //! for the tty to send what it wrote for at most [`UNACTIVATED_DRAIN`], so that a tty that sends @@ -52,6 +53,7 @@ use std::fmt; use std::io::{self, Read, Write}; +use std::net::Ipv4Addr; use std::path::{Path, PathBuf}; use std::time::Instant; @@ -86,13 +88,16 @@ pub fn template_config() -> RootConfig { RootConfig::default() } -/// Returns the inputs of a clone's root's assembly that its template captured, from `config`, the -/// template's own root configuration: its runtime directory; the distro and the scratch block -/// devices, at the MMIO windows from which the worker of a VM that boots cold assembles its root, -/// with the sizes that the configuration names, if any; how long the assembly waits for each; -/// and the host filesystem device, by [`FILESYSTEM_TAG`], through which the host attaches the -/// export of a clone's record. The record of the clone's `Activate` holds the rest. -pub fn root_template(config: &RootConfig) -> RootTemplate { +/// Returns the inputs of a clone's activation that its template captured, from `config`, the +/// template's own root configuration, and `gateway`, the IPv4 gateway of the template's NIC, which +/// its kernel command line names, if it has one: its runtime directory; the distro and the scratch +/// block devices, at the MMIO windows from which the worker of a VM that boots cold assembles its +/// root, with the sizes that the configuration names, if any; how long the assembly waits for +/// each; the host filesystem device, by [`FILESYSTEM_TAG`], through which the host attaches the +/// export of a clone's record; and the gateway, to which the announcer of the listeners of a +/// clone whose record publishes them announces them. The record of the clone's `Activate` holds +/// the rest. +pub fn root_template(config: &RootConfig, gateway: Option) -> RootTemplate { RootTemplate { runtime: PathBuf::from(&config.runtime), distro: Device { @@ -105,6 +110,7 @@ pub fn root_template(config: &RootConfig) -> RootTemplate { }, timeout: config.device_timeout, filesystem_tag: FILESYSTEM_TAG.to_owned(), + gateway, } } @@ -303,6 +309,7 @@ enum Capture { } /// Captures the template's VM on its control tty at `tty`, with the root operations `operations`, +/// whose NIC's IPv4 gateway is `gateway`, if it has a NIC, as its kernel command line names it, /// as the [module documentation](self) describes, and returns how the worker ended: once the /// bootstrap failed, or the capture ended without a snapshot, or failed, or in a clone, which a /// restore of the snapshot started, once its repair failed, or the session that the worker served @@ -318,6 +325,7 @@ enum Capture { pub fn capture_on_terminal( tty: &Path, operations: &mut dyn RootOperations, + gateway: Option, ) -> Result { guest::on_terminal_within(tty, |terminal| { let config = template_config(); @@ -356,7 +364,7 @@ pub fn capture_on_terminal( // The activation's deadline starts at the repair's acknowledgement of the restore. let acknowledged = repaired.acknowledged_at(); let activation = Activation::new(&repaired, fence, ACTIVATION_TIMEOUT, Instant::now()); - let template = root_template(&config); + let template = root_template(&config, gateway); let worked = work_clone( open_tty(tty)?, session, @@ -510,9 +518,10 @@ mod tests { // names another. let config = template_config(); assert_eq!(config, RootConfig::default()); - let template = root_template(&config); + let template = root_template(&config, None); assert_eq!(template.runtime, Path::new(&config.runtime)); assert_eq!(template.timeout, config.device_timeout); + assert_eq!(template.gateway, None); // The clone's root comes from the devices whose mounts the template's capture checks, and // its export, if its record has one, through the host filesystem device that the capture // found unmounted. @@ -526,11 +535,14 @@ mod tests { scratch_sectors: Some(16), ..config }; - let template = root_template(&sized); + // A template with a NIC hands its clones its gateway. + let gateway = std::net::Ipv4Addr::new(10, 0, 0, 1); + let template = root_template(&sized, Some(gateway)); assert_eq!( (template.distro.sectors, template.scratch.sectors), (Some(8), Some(16)) ); + assert_eq!(template.gateway, Some(gateway)); assert_eq!( (template.distro.mmio, template.scratch.mmio), (DISTRO_MMIO_BASE, SCRATCH_MMIO_BASE) diff --git a/tests/runtime-adapter/tests/guest_session.rs b/tests/runtime-adapter/tests/guest_session.rs index e7e996936..6d4002665 100644 --- a/tests/runtime-adapter/tests/guest_session.rs +++ b/tests/runtime-adapter/tests/guest_session.rs @@ -29,9 +29,11 @@ //! activation, retries of the `Activate`, the order that the worker serves once the clone is //! ready, the status of each class of the activation's failures, which the worker answers with //! nothing, before any side effect where the class allows, its deadline, also once the stream -//! takes nothing more, the record's workload identity, as which every execution runs, and the +//! takes nothing more, the record's workload identity, as which every execution runs, the //! mapping table of the record's export, which the worker applies from the export that the -//! application mounted, while it leaves the endpoint of the network's proxy to the host. +//! application mounted, while it leaves the endpoint of the network's proxy to the host, and the +//! announcer of the listeners of a record whose network publishes them, whose end ends the clone, +//! which the fake stands for with a pipe that the test closes. //! //! The worker's executions run behind the test broker too, whose host side drives the engine's //! peers through the host program's own `exec::HostExecution`: the tests cover several executions @@ -56,7 +58,7 @@ use std::collections::BTreeMap; use std::fs::{self, File, OpenOptions}; use std::io::{self, BufRead, BufReader, Read, Write}; use std::net::Shutdown; -use std::os::fd::OwnedFd; +use std::os::fd::{AsFd, BorrowedFd, OwnedFd}; use std::os::unix::fs::OpenOptionsExt; use std::os::unix::net::UnixStream; use std::os::unix::process::{CommandExt, ExitStatusExt}; @@ -70,6 +72,7 @@ use std::time::{Duration, Instant}; use nix::errno::Errno; use nix::fcntl::OFlag; +use nix::poll::{self, PollFd, PollFlags, PollTimeout}; use nix::pty; use nix::sys::signal::{self, Signal}; use nix::sys::wait::{self, Id, WaitPidFlag, WaitStatus}; @@ -1137,6 +1140,10 @@ enum Operation { /// its record's export, if any. #[cfg(target_arch = "x86_64")] Resolver, + /// A clone's start of the announcer of its workload's listeners, which its record's network + /// publishes, once it wrote the resolver file, if any. + #[cfg(target_arch = "x86_64")] + Announce, /// The host export's mount: the one that the kernel command line names, or a clone's record's. MountExport, /// A probe. @@ -1201,6 +1208,14 @@ struct FakeRoot { /// The workload cgroup of the test's own, below which the worker runs its executions in /// cgroups of their own, where the test can create cgroups. workload: Option, + /// The read end of a pipe whose write end the test holds, which the application takes as the + /// announcer that it starts, if the test made one with [`ending`](Self::ending). + ending: Option, + /// The announcer that a clone's application started, for a record whose network publishes + /// the workload's listeners, until the worker collected its end: the read end of a pipe, + /// which polls readable, hung up, once its write end closes, as an announcer's pidfd polls + /// readable once the announcer ended, with that write end, unless the test holds it. + announcer: Option<(OwnedFd, Option)>, } impl FakeRoot { @@ -1239,6 +1254,8 @@ impl FakeRoot { #[cfg(target_arch = "x86_64")] deadlines: Vec::new(), workload, + ending: None, + announcer: None, } } @@ -1264,6 +1281,14 @@ impl FakeRoot { self } + /// Makes the announcer that a clone's application starts end once the test drops the returned + /// write end of its pipe. + fn ending(mut self) -> (Self, OwnedFd) { + let (read, write) = announcer_pipe(); + self.ending = Some(read); + (self, write) + } + /// Makes each probe, and each of a clone's applications, wait until the test opens the gate, /// which the returned sender opens for one at a time. fn gated(mut self) -> (Self, Sender<()>) { @@ -1346,9 +1371,26 @@ impl RootOperations for FakeRoot { fn teardown(&mut self) -> Result<(), String> { self.calls.push(Call::Teardown); + self.announcer = None; self.result(Operation::Teardown) } + fn announcer(&self) -> Option> { + self.announcer.as_ref().map(|(read, _)| read.as_fd()) + } + + /// Returns the announcer's end once the write end of its pipe closed, which collects it. + fn announcer_ended(&mut self) -> Option { + let (read, _) = self.announcer.as_ref()?; + let mut polls = [PollFd::new(read.as_fd(), PollFlags::POLLIN)]; + poll::poll(&mut polls, PollTimeout::ZERO).unwrap(); + let ended = polls[0].revents().is_some_and(|events| !events.is_empty()); + ended.then(|| { + self.announcer = None; + "the test ended the announcer".to_owned() + }) + } + /// Applies the record as `nvx::guest::clone::apply` does, without a root, once the test opened /// the gate, if any: for a GPT disk, it stands for the kernel's reread of the disk's partition /// table first, which fails as the kernel does while a partition is open if the test fails @@ -1358,9 +1400,12 @@ impl RootOperations for FakeRoot { /// stands for the root that it assembled. It then stands for the generation of the machine ID /// that the record asks for, if any, the mount of the record's export through the template's /// host filesystem device, if it has one, the write of the resolver file that names the DNS - /// server of the record's network, if it names one, and the setting of the record's hostname - /// and machine ID for every execution, which fail if the test fails [`Operation::MachineId`], - /// [`Operation::MountExport`], [`Operation::Resolver`], and [`Operation::View`]. + /// server of the record's network, if it names one, the start of the announcer of the + /// workload's listeners, if the network publishes them, and the setting of the record's + /// hostname and machine ID for every execution, which fail if the test fails + /// [`Operation::MachineId`], [`Operation::MountExport`], [`Operation::Resolver`], + /// [`Operation::Announce`], and [`Operation::View`]. A failure after the start stops the + /// announcer. #[cfg(target_arch = "x86_64")] fn apply( &mut self, @@ -1368,8 +1413,9 @@ impl RootOperations for FakeRoot { template: &nvx::guest::clone::RootTemplate, deadline: Option, ) -> Result, nvx::guest::clone::ApplyError> { - use nvx::activation::MachineIdMode; + use nvx::activation::{self, MachineIdMode}; use nvx::guest::clone::{ApplyError, NetworkError}; + use nvx::guest::ingress::StartError; use nvx::guest::root::{DeviceStep, Error, Mount, Role}; let applied = (record.clone(), template.clone()); @@ -1436,11 +1482,34 @@ impl RootOperations for FakeRoot { teardown: None, })?; } - self.result(Operation::View) - .map_err(|problem| ApplyError::View { + if network.is_some_and(|network| network.publication == activation::Publication::Published) + { + if template.gateway.is_none() { + return Err(ApplyError::Network { + error: NetworkError::Gateway, + teardown: None, + }); + } + self.result(Operation::Announce) + .map_err(|problem| ApplyError::Network { + error: NetworkError::Announcer(StartError::Spawn(io::Error::other(problem))), + teardown: None, + })?; + self.announcer = Some(match self.ending.take() { + Some(read) => (read, None), + None => { + let (read, write) = announcer_pipe(); + (read, Some(write)) + } + }); + } + self.result(Operation::View).map_err(|problem| { + self.announcer = None; + ApplyError::View { error: entry::Error::Root(io::Error::other(problem)), teardown: None, - })?; + } + })?; Ok(identity) } } @@ -1474,6 +1543,13 @@ const PASSWD: &str = "root:x:0:0:root:/root:/bin/sh\n\ /// The `/etc/group` of the [accounts' root](accounts). const GROUP: &str = "root:x:0:\nstaff:x:50:\nworkload:x:1000:\n"; +/// Returns a pipe that stands for the pidfd of a clone's announcer: its read end, which polls +/// readable once the write end closes, and its write end. Both close on exec, so that no program +/// that a test spawns holds the write end, and keeps the announcer from ending. +fn announcer_pipe() -> (OwnedFd, OwnedFd) { + nix::unistd::pipe2(OFlag::O_CLOEXEC).unwrap() +} + /// Returns the directory that stands for the assembled root of every fake root when the worker /// prepares an entry: one of the tests' own, below the target directory, for the tests' user, /// with the account databases [`PASSWD`] and [`GROUP`], and nothing else. @@ -4075,6 +4151,9 @@ mod clones { memory: 1 << 30, }; + /// The IPv4 gateway of the NIC of the clones' template. + const GATEWAY: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1); + /// Returns the record of a clone's Activate of the transaction `transaction`, whose workload /// runs as `identity`, if any: an EROFS image over a scratch, without an export, a mapping, or /// a network, which expects [`RESOURCES`]. @@ -4144,6 +4223,15 @@ mod clones { }) } + /// Returns the record of `record`'s transaction with the network of [`networked`], without a + /// proxy, but which publishes the workload's listeners. + fn published(record: &ActivationRecord) -> ActivationRecord { + changed(&networked(record, None), |configuration| { + let network = configuration.network.as_mut().unwrap(); + network.publication = activation::Publication::Published; + }) + } + /// Returns the activation of a clone that the worker repaired, with [`RESOURCES`], behind the /// worker's fence, which the capture's barrier handed over closed, and whose deadline ends /// `timeout` after `acknowledged`, as from the repair's acknowledgement of the restore. @@ -4153,10 +4241,10 @@ mod clones { Activation::with_resources(RESOURCES, Some(acknowledged), fence, timeout, acknowledged) } - /// Returns the inputs of the assembly of the clones' roots that their template captured: - /// those of the template's own configuration. + /// Returns the inputs of the activation of the clones that their template captured: those of + /// the template's own configuration, and the gateway of its NIC, [`GATEWAY`]. fn template() -> RootTemplate { - capture::root_template(&capture::template_config()) + capture::root_template(&capture::template_config(), Some(GATEWAY)) } /// Returns the guest's role of the session that the template's VM attached before its @@ -4450,8 +4538,9 @@ mod clones { // The application fails once the kernel refused to reread a GPT disk's partition table, // and refuses a workload identity whose account the image lacks, or whose primary group // it lacks, a machine ID that it cannot generate, and a hostname or machine ID that it - // cannot set for every execution, an export that it cannot mount, and a resolver file - // that it cannot write, before Ready. + // cannot set for every execution, an export that it cannot mount, a resolver file that it + // cannot write, and an announcer of the workload's listeners that it cannot start, before + // Ready. let gpt = changed(&clone, |configuration| { configuration.image = activation::Image::Gpt { binding: activation::Identity::from_bytes([0x11; 32]), @@ -4493,6 +4582,11 @@ mod clones { FakeRoot::failing(Operation::MountExport, "the export's device is missing"), status::ACTIVATION_EXPORT, ), + ( + published(&clone), + FakeRoot::failing(Operation::Announce, "no privileged port is free"), + status::ACTIVATION_NETWORK, + ), ( networked(&clone, None), FakeRoot::failing(Operation::Resolver, "a resolver file exists already"), @@ -4808,10 +4902,12 @@ mod clones { } /// Returns the entrance of a clone's execution as `identity`, with the hostname `hostname`, - /// its record's, and published listeners. + /// its record's, and unpublished listeners, as a record whose network publishes none, or + /// that has none, has them. fn cloned(identity: entry::Identity, hostname: &[u8]) -> Entrance { Entrance { hostname: hostname.to_vec(), + publication: Publication::Unpublished, ..entrance(identity) } } @@ -4852,6 +4948,130 @@ mod clones { assert_eq!(entrances(&root), [cloned(identity, b"another-clone")]); } + #[test] + fn a_clone_s_executions_enter_the_root_with_the_publication_of_its_record_s_network() { + // A record's network publishes the guest's listeners under a policy that allows ingress + // or host-loopback access both ways, and then the entry refuses at stage 0 the + // capabilities that could forge a listener announcement, as after a cold boot whose + // configuration publishes them. A record whose network publishes none, or that has + // none, leaves them to the execution, as the other capabilities of the allowlist. + let capabilities = [ + Capability::NET_BIND_SERVICE, + Capability::NET_RAW, + Capability::CHOWN, + ]; + for (record, publication) in [ + (record(1, None), Publication::Unpublished), + (networked(&record(1, None), None), Publication::Unpublished), + (published(&record(1, None)), Publication::Published), + ] { + let mut host = Host::activated(FakeRoot::booting(&layout()), &record); + for (id, capability) in (1..).zip(capabilities) { + let identity = exec::Identity { + capabilities: 1 << capability.number(), + ..exec::Identity::new(0, 0) + }; + host.start( + start_of(id, shell("exit 0")).with_identity(identity), + Vec::new(), + ); + } + host.run_until(|host| (1..=3).all(|id| host.outcome(id).is_some())); + for (id, capability) in (1..).zip(capabilities) { + if publication == Publication::Published && capability != Capability::CHOWN { + let failure = LaunchFailure::new(0, Errno::EPERM as i32); + let outcome = host.outcome(id).unwrap(); + assert_eq!(outcome.launch_failure(), Some(failure), "{capability}"); + } else { + check_lossless(&host, id, Status::Exited(0)); + } + } + let (root, worked) = host.stop(); + assert_eq!(worked.unwrap(), Worked::Stopped); + let expected = capabilities.map(|capability| Entrance { + publication, + ..cloned( + entry::Identity::new(0, 0).capabilities(Capabilities::EMPTY.with(capability)), + b"clone", + ) + }); + assert_eq!(entrances(&root), expected, "{publication:?}"); + } + } + + #[test] + fn a_clone_whose_announcer_ends_ends_with_the_network_s_status() { + // The application started the announcer of the listeners that the record's network + // publishes, whose end, once the clone is ready, before its sandbox started or after, and + // with an execution running, ends the clone with the network's status, once the worker + // dropped its executions: the gateway would keep publishing the listeners that the + // announcer last announced, and learn of no later change of them. + let record = published(&record(1, None)); + for started in [false, true] { + let (root, ending) = FakeRoot::booting(&layout()).ending(); + let (mut broker, working) = start_clone(root, ACTIVATION_TIMEOUT); + broker.reset(RESTORED); + let replies = broker.exchange(&[activate(&record)], OUTER_MAX_PAYLOAD, 1); + assert_eq!(replies, [activated(&record)]); + // The host waits for Ready before it starts the sandbox: until the activation + // settled, any message but a retry of the Activate fails it. + if started { + let replies = broker.exchange(&[Message::Start], OUTER_MAX_PAYLOAD, 1); + assert_eq!(replies, [Message::Started]); + } + drop(ending); + let (root, worked) = joined(working); + let worked = worked.unwrap(); + assert_eq!( + worked, + Worked::Unactivated { + status: status::ACTIVATION_NETWORK, + problem: "the test ended the announcer".to_owned(), + } + ); + assert_eq!(root.calls, [applied(&record)], "{started}"); + assert!(root.announcer.is_none()); + } + let (root, ending) = FakeRoot::booting(&layout()).ending(); + let mut host = Host::activated(root, &record); + let running = start_of(1, shell("echo up; exec sleep 60")); + host.start(running.with_identity(exec::Identity::new(0, 0)), Vec::new()); + host.run_until(|host| host.executions[&1].output(Stream::Stdout) == b"up\n"); + drop(ending); + let (root, worked) = joined(host.working.take().unwrap()); + assert_eq!(worked.unwrap().status(), status::ACTIVATION_NETWORK); + assert_eq!(root.calls[0], applied(&record)); + + // A template without a gateway cannot announce the listeners of a record that publishes + // them, which fails before Ready with the network's status. + let root = FakeRoot::booting(&layout()); + let (broker, guest) = UnixStream::pair().unwrap(); + let mut broker = Broker::new(broker); + let session = captured_session(); + let acknowledged = Instant::now(); + let activation = repaired(acknowledged, ACTIVATION_TIMEOUT); + let gatewayless = RootTemplate { + gateway: None, + ..template() + }; + let working = thread::spawn(move || { + let mut root = root; + let worked = worker::work_clone( + guest, + session, + &mut root, + activation, + gatewayless, + Some(acknowledged), + ); + (root, worked) + }); + broker.reset(RESTORED); + broker.send_data(&activate(&record).encode().unwrap(), OUTER_MAX_PAYLOAD); + let root = unactivated(&mut broker, working, status::ACTIVATION_NETWORK); + assert!(root.announcer.is_none()); + } + /// Returns the facts of the guest's `report`. fn reported(report: &Message) -> Facts { let Message::Report(body) = report else { diff --git a/tests/runtime-adapter/tests/vm_clone.rs b/tests/runtime-adapter/tests/vm_clone.rs index 8af71080b..c8133457f 100644 --- a/tests/runtime-adapter/tests/vm_clone.rs +++ b/tests/runtime-adapter/tests/vm_clone.rs @@ -121,31 +121,43 @@ //! through, which attaches no NIC either: neither has a network device, nor a network in its //! record, and each keeps the image's own `/etc/resolv.conf`, which no bind holds. NVX's plan //! refuses a clone of a network with a NIC there before any VMM starts. -//! - A template of each network with a NIC that a template takes serves the clones of its own -//! network alone: their restores attach the template's NIC under its policy, unchanged, which -//! lets through what the policy allows alone, and their records name the policy, the DNS server -//! with which the template's guest configured its resolver at boot, where the policy lets the -//! guest reach the gateway's, and the proxy, if any. Of the listeners of host loopback that the -//! tests bind, a clone reaches at its gateway the allow rule's alone, the proxy's alone, every -//! one with egress and outbound host-loopback access allowed, and none without host-loopback -//! access. NVX's plan refuses, before any VMM starts, a clone of another network, of the -//! template's policy at another address of the guest, of a network that lets nothing through, -//! and one without a network. Each template's key holds a policy of its own. +//! - One template with a NIC, which its capture attached under the complete deny-all policy, +//! whatever the policy of the network that its creation named, serves the clones of every +//! network at its guest's address: each clone's restore installs the clone's own policy on the +//! template's NIC, with `--restore-network-policy`, and the clone's record names the policy, the +//! DNS server that OpenVMM serves the guest under it, where it lets the guest reach the +//! gateway's, the proxy, if any, and whether it publishes the workload's listeners, which a +//! policy does with ingress, or with host-loopback access both ways. The NIC lets through what +//! the clone's policy allows alone: of the listeners of host loopback that the tests bind, a +//! clone reaches at its gateway the allow rule's alone, the proxy's alone, every one with egress +//! and outbound host-loopback access allowed, and none otherwise. The host reaches a listener of +//! the clone's guest at its port of host loopback exactly where the policy allows host-loopback +//! access both ways, once the announcer that the clone's guest started announced it, and through +//! a forward of the policy, while the guest refuses the executions that ask for +//! `CAP_NET_BIND_SERVICE` or `CAP_NET_RAW` exactly where the policy publishes the listeners. +//! NVX's plan refuses, before any VMM starts, a clone of a network at another address of the +//! guest, of a network that lets nothing through, and one without a network. Each clone's +//! record names a policy of its own. +//! - A clone of a network whose forward names a port of host loopback that another socket holds +//! ends before its restore-ready event, since OpenVMM cannot bind the port as it restores the +//! clone's NIC, which NVX attributes to the host, by the evidence of the forward's bind, and the +//! start releases its lease. Once the port is free, the template serves a clone of the network. //! - A clone of a network that lets the guest reach the gateway's DNS server finds a resolver //! file that names it, of mode 0444, which a read-only bind holds at `/etc/resolv.conf`, after //! the bind of a generated machine ID, as mount tables list them, and which no write changes, -//! and reaches the server through the template's NIC: a TCP connection to its DNS port opens, -//! which the gateway's DNS proxy accepts itself. A clone whose resolver configuration cannot be -//! bound, since its image holds a regular file `etc`, ends before it is ready, with the status -//! of its network settings. +//! and reaches the server through the template's NIC, although the template's guest learned no +//! DNS server: a TCP connection to its DNS port opens, which the gateway's DNS proxy accepts +//! itself. A clone whose resolver configuration cannot be bound, since its image holds a regular +//! file `etc`, ends before it is ready, with the status of its network settings. //! - The executions of a clone of a proxy-only network find the URL of its proxy, which its record //! names, in their environment, as a launch's executions do, and the clone keeps the image's own //! `/etc/resolv.conf`. //! - A clone of a template with a NIC maps its export once it is ready, below the binds of its //! resolver configuration and of a generated machine ID. //! - A run of a clone of a proxy-only network gives its execution the URL of the network's proxy, -//! as the executions of a started clone find it, and a run of another network, which NVX's plan -//! refuses before any VMM starts, once the preflight staged the clone's images, runs nothing. +//! as the executions of a started clone find it, and a run of a network that lets nothing +//! through, which NVX's plan refuses before any VMM starts, once the preflight staged the +//! clone's images, runs nothing. //! - A run's execution finds the run's shares beneath the clone's root, since the restore //! sequence mapped them before the clone's sandbox started, and the host's directory keeps what //! it wrote through them once the run tore the clone down. @@ -168,12 +180,16 @@ use std::collections::{BTreeMap, BTreeSet}; use std::env; use std::ffi::{OsStr, OsString}; use std::fs::{self, File}; -use std::io; +use std::hash::{BuildHasher, RandomState}; +use std::io::{self, Read, Write}; use std::mem; -use std::net::{IpAddr, Ipv4Addr, TcpListener}; +use std::net::{ + IpAddr, Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr, TcpListener, TcpStream, UdpSocket, +}; use std::num::NonZeroU64; use std::path::{Path, PathBuf}; use std::sync::LazyLock; +use std::sync::atomic::{AtomicBool, Ordering}; use std::thread; use std::time::{Duration, Instant, SystemTime}; @@ -338,12 +354,32 @@ const GATEWAY: &str = "192.0.2.1"; /// The host's side of the scenarios' networks, which the tests bind the first time that they name /// one of its ports, and hold until they end. static HOST_SIDE: LazyLock = LazyLock::new(HostSide::bind); +/// The first and the last of the ports that the host draws for a listener of a clone's guest and +/// for a forward: the dynamic range. +const PORTS: (u16, u16) = (49152, 65535); +/// The number of ports that the host draws for each before it gives up. Windows excludes ports per +/// protocol, in blocks, and gives consecutive ports to consecutive binds of port 0, so the host +/// draws each port independently. +const PORT_ATTEMPTS: usize = 16; /// How long the host waits at a service of its side for a connection that a clone opened there, /// where the clone's network lets it through. const ARRIVAL: Duration = Duration::from_secs(10); /// How long the host waits at a service of its side for a connection where the clone's network /// must not let it through. const NEGATIVE_OBSERVATION: Duration = Duration::from_millis(250); +/// The line with which a listener of a clone's guest greets its client. +const GREETING: &str = "RUNTIME-ADAPTER-GUEST"; +/// The line with which the host answers the greeting. +const ANSWER: &str = "RUNTIME-ADAPTER-HOST"; +/// How long, in seconds, a listener of a clone's guest waits for the host where the host must +/// reach it. +const LISTENING: u32 = 30; +/// How long, in seconds, a listener of a clone's guest waits for the host where the host must not +/// reach it, all of which the host spends trying to reach it. +const UNPUBLISHED_LISTENING: u32 = 5; +/// The error number with which the guest refuses an execution that asks for a capability that the +/// publication of the workload's listeners forbids: `EPERM`. +const EPERM: i32 = 1; /// The proxy's settings of an execution that names its own, which it keeps. const OWN_PROXY: &str = "HTTPS_PROXY=http://proxy.example:8080"; /// The `map` probe of a clone's resolver configuration: what the root's `/etc/resolv.conf` shows. @@ -370,6 +406,17 @@ const DNS_REACH: &str = "server=$(sed -n 's/^nameserver //p' /etc/resolv.conf); const REACH: &str = "gateway=$1; shift; for target in \"$@\"; do \ nc -z -w 2 \"$gateway\" \"${target#*:}\" /dev/null 2>&1; \ echo \"${target%%:*} $?\"; done"; +/// The program of the executions that listen at the guest's TCP port `$1` for one client, for at +/// most `$2` seconds, `/bin/sh -c LISTENER sh PORT SECONDS LINE`, which greets the client with the +/// line `$3`, prints what it sends until it closes its side, then prints `listener STATUS`: 0 once +/// the client closed, and 124 once the time ran out without one. +const LISTENER: &str = + "printf '%s\\n' \"$3\" | timeout \"$2\" nc -l -p \"$1\"; echo \"listener $?\""; +/// The capabilities, by their numbers, of the executions that check what the publication of a +/// clone's listeners refuses, each with whether a publication refuses it: `CAP_NET_BIND_SERVICE` +/// and `CAP_NET_RAW`, with which an execution could take the announcer's privileged source port or +/// forge its datagrams, and `CAP_CHOWN`, which none refuses. +const PUBLISHED_CAPABILITIES: [(&str, bool); 3] = [("10", true), ("13", true), ("0", false)]; /// How much longer than the platform's image the GPT placeholder that [`Template::of_disk`] /// captures is, so that the image of each of the template's clones, the platform's image or a GPT /// disk of its length or shorter, is shorter than the capacity that the clone's restore presents. @@ -492,9 +539,9 @@ impl Disk { } /// The network of a scenario's template or clone, which the host program reads from a file that -/// the test writes, in the serde form of a plan's network, as `vm_template` writes it: the guest at -/// an address of its own, with egress denied, and no host-loopback access, unless the network says -/// otherwise. +/// the test writes, in the serde form of a plan's network: the guest at an address of its own, with +/// egress denied, and no host-loopback access, unless the network says otherwise. A network names +/// the ports of the [`HOST_SIDE`]. #[derive(Clone, Copy, Debug, PartialEq, Eq)] enum Network { /// Everything denied, without a proxy, which attaches no NIC. @@ -510,11 +557,29 @@ enum Network { Outbound, /// A proxy-only network, whose proxy is [`Service::Proxy`], on host loopback. Proxy, + /// Ingress allowed, so that OpenVMM publishes the guest's listeners to the host's private + /// networks, which do not include the host itself without host-loopback access. + Ingress, + /// Host-loopback access in both directions, with egress denied: OpenVMM publishes the guest's + /// listeners on host loopback, at their ports, and the guest reaches nothing, since its own + /// connections follow the egress policy. + HostLoopback, + /// Host-loopback access in both directions, with egress denied, and a forward from the port + /// [`HostSide::forward`] of host loopback to the guest's port [`HostSide::listener`]. + Forward, } impl Network { - /// The networks with a NIC that a template takes, which `vm_template` captures. - const WITH_NIC: [Self; 4] = [Self::Egress, Self::AllowRule, Self::Outbound, Self::Proxy]; + /// The networks with a NIC. + const WITH_NIC: [Self; 7] = [ + Self::Egress, + Self::AllowRule, + Self::Outbound, + Self::Proxy, + Self::Ingress, + Self::HostLoopback, + Self::Forward, + ]; /// Returns the network's name, which names its files. fn name(self) -> &'static str { @@ -524,6 +589,9 @@ impl Network { Self::AllowRule => "rule", Self::Outbound => "outbound", Self::Proxy => "proxy", + Self::Ingress => "ingress", + Self::HostLoopback => "loopback", + Self::Forward => "forward", } } @@ -558,24 +626,37 @@ impl Network { let port = HOST_SIDE.port(Service::Proxy); form["proxy"] = json!({"port": port, "scheme": "http"}); } + Self::Ingress => form["ingress"] = json!("allow"), + Self::HostLoopback => form["hostLoopback"] = json!("allow"), + Self::Forward => { + form["hostLoopback"] = json!("allow"); + form["hostLoopbackForwards"] = json!([{ + "protocol": "tcp", + "hostPort": HOST_SIDE.forward, + "guestPort": HOST_SIDE.listener, + }]); + } } form } - /// Returns OpenVMM's arguments with which the restore of a clone of a template of the network, - /// with the guest at `guest`, attaches the template's NIC, as NVX lowers the network: all of - /// them but `--net` and the guest's address, which OpenVMM takes from the snapshot, and none - /// for a network that lets nothing through. + /// Returns OpenVMM's arguments with which the restore of a clone of the network, with the + /// guest at `guest`, installs the network's policy on the template's NIC, as NVX lowers the + /// network: `--restore-network-policy`, in place of `--net` and the guest's address, which + /// OpenVMM takes from the snapshot, then the policy's options, and none for a network that + /// lets nothing through. fn restored(self, guest: &str) -> Vec { let network = serde_json::from_value::(self.form(guest)).unwrap(); let mut arguments = network.openvmm_arguments().unwrap(); - arguments.drain(..arguments.len().min(2)); + if !arguments.is_empty() { + arguments.splice(..2, ["--restore-network-policy".to_owned()]); + } arguments } - /// Returns the DNS server that the network's clones' records name, with which the template's - /// guest configured its resolver at boot: the gateway's, for a network that lets the guest - /// reach it. + /// Returns the DNS server that the network's clones' records name, which OpenVMM serves the + /// guest under the network's policy: the gateway's, for a network that lets the guest reach + /// it. fn dns(self) -> Option { matches!(self, Self::Egress | Self::Outbound).then(|| GATEWAY.parse().unwrap()) } @@ -587,15 +668,34 @@ impl Network { .then(|| format!("http://{GATEWAY}:{}", HOST_SIDE.port(Service::Proxy))) } + /// Returns whether the network publishes the guest's listeners, as its clones' records say: + /// with ingress, or with host-loopback access both ways. + fn publication(self) -> Publication { + match self { + Self::Ingress | Self::HostLoopback | Self::Forward => Publication::Published, + Self::Deny | Self::Egress | Self::AllowRule | Self::Outbound | Self::Proxy => { + Publication::Unpublished + } + } + } + + /// Returns whether OpenVMM publishes the guest's listeners on host loopback, at their ports, + /// where the host reaches them: with host-loopback access both ways. + fn on_host_loopback(self) -> bool { + matches!(self, Self::HostLoopback | Self::Forward) + } + /// Returns the services of the host's side that the network lets a clone reach at its - /// gateway: none without host-loopback access, the allow rule's alone, every one with egress - /// and outbound host-loopback access allowed, and the proxy alone for a proxy-only network. + /// gateway: the allow rule's alone, every one with egress and outbound host-loopback access + /// allowed, the proxy alone for a proxy-only network, and none otherwise, since the guest's own + /// connections follow the egress policy, and reach host loopback only with host-loopback + /// access. fn reaches(self) -> &'static [Service] { match self { - Self::Deny | Self::Egress => &[], Self::AllowRule => &[Service::Rule], Self::Outbound => &Service::ALL, Self::Proxy => &[Service::Proxy], + Self::Deny | Self::Egress | Self::Ingress | Self::HostLoopback | Self::Forward => &[], } } } @@ -628,8 +728,10 @@ impl Service { } /// The host's side of the scenarios' networks: the listener of each [`Service`], on host IPv4 -/// loopback, at a port that the host chose, which the networks that the tests write name. The -/// listeners hold their ports until the tests end, so that no other socket takes one. +/// loopback, at a port that the host chose, which the networks that the tests write name, and the +/// ports of a listener of a clone's guest and of a forward, which no socket of the host held when +/// the host drew them. The listeners hold their ports until the tests end, so that no other socket +/// takes one. struct HostSide { /// The listener of [`Service::Proxy`]. proxy: TcpListener, @@ -637,16 +739,32 @@ struct HostSide { rule: TcpListener, /// The listener of [`Service::Other`]. other: TcpListener, + /// The port at which a listener of a clone's guest listens, and OpenVMM publishes it on host + /// loopback, under a network that publishes the guest's listeners there. + listener: u16, + /// The port of host IPv4 loopback that a [`Network::Forward`] forwards to the guest's port + /// [`listener`](Self::listener). + forward: u16, } impl HostSide { - /// Binds the listener of each service. + /// Binds the listener of each service, and draws the ports of a listener of a clone's guest + /// and of a forward. fn bind() -> Self { let bind = || TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).unwrap(); + let (proxy, rule, other) = (bind(), bind(), bind()); + let mut taken = [&proxy, &rule, &other] + .map(|listener| listener.local_addr().unwrap().port()) + .to_vec(); + let listener = free_port(&taken); + taken.push(listener); + let forward = free_port(&taken); Self { - proxy: bind(), - rule: bind(), - other: bind(), + proxy, + rule, + other, + listener, + forward, } } @@ -684,6 +802,46 @@ impl HostSide { } } +/// Returns a port that the host draws from [`PORTS`], independently each time, at most +/// [`PORT_ATTEMPTS`] times, but none of `taken`, and none that a socket holds: OpenVMM publishes a +/// listener of the guest at the same port of host IPv4 and IPv6 loopback, for TCP and UDP, after +/// it checks, on Windows, every host address of each family there, and binds a forward's port on +/// host IPv4 loopback. Windows lets a socket bind every host address beside one that binds loopback +/// alone, and the reverse, so a port is free only where a TCP and a UDP socket bind both, for each +/// family that the host has. +fn free_port(taken: &[u16]) -> u16 { + let mut addresses = vec![ + IpAddr::V4(Ipv4Addr::UNSPECIFIED), + IpAddr::V4(Ipv4Addr::LOCALHOST), + ]; + if TcpListener::bind((Ipv6Addr::LOCALHOST, 0)).is_ok() { + addresses.extend([ + IpAddr::V6(Ipv6Addr::UNSPECIFIED), + IpAddr::V6(Ipv6Addr::LOCALHOST), + ]); + } + let (first, last) = PORTS; + let random = RandomState::new(); + let mut refused = Vec::new(); + for attempt in 0..PORT_ATTEMPTS { + let offset = random.hash_one(attempt) % u64::from(last - first + 1); + let port = first + u16::try_from(offset).unwrap(); + if taken.contains(&port) { + refused.push(format!("{port}: the host's side holds it")); + continue; + } + let bound = addresses.iter().try_for_each(|&address| { + TcpListener::bind((address, port))?; + UdpSocket::bind((address, port)).map(drop) + }); + match bound { + Ok(()) => return port, + Err(error) => refused.push(format!("{port}: {error}")), + } + } + panic!("no port drawn from {first} to {last} was free: {refused:?}") +} + /// Writes the network `network`, with the guest at `guest`, in a file of its own in `directory`, /// and returns the file's path. fn write_network(directory: &Path, network: Network, guest: &str) -> PathBuf { @@ -1332,6 +1490,81 @@ impl Cloned { } } + /// Runs a listener of the clone's guest, as nobody, at the guest's port [`HostSide::listener`], + /// with [`LISTENER`], while the host tries to reach it through `address`, until the listener's + /// execution ends, and checks that the host reached it, and exchanged a line each way with it, + /// if `reached`, or else that the host never did, while the listener waited all of its time. + fn assert_listener(&mut self, address: SocketAddr, reached: bool) { + let port = HOST_SIDE.listener.to_string(); + let seconds = if reached { + LISTENING + } else { + UNPUBLISHED_LISTENING + }; + let seconds = seconds.to_string(); + let execution = [ + "--identity", + NOBODY, + "--", + "/bin/sh", + "-c", + LISTENER, + "listener", + &port, + &seconds, + GREETING, + ]; + let stop = AtomicBool::new(false); + let (ran, exchanged) = thread::scope(|scope| { + let client = scope.spawn(|| reach_guest(address, &stop)); + let ran = { + let _stop = Stop(&stop); + self.exec(&execution) + }; + (ran, client.join().unwrap()) + }); + assert_eq!(exchanged, reached, "{address}: {ran:?}"); + // The host program escapes each line feed. + let stdout = if reached { + format!("{ANSWER}\\nlistener 0\\n") + } else { + "listener 124\\n".to_owned() + }; + let facts = + ["launch", "status", "stdout", "stderr"].map(|key| &ran[&format!("exec.1.{key}")]); + assert_eq!( + facts, + ["started", "exit 0", stdout.as_str(), ""], + "{address}: {ran:?}" + ); + } + + /// Checks that the clone's guest refuses an execution, as user ID 0, that asks for a + /// capability with which it could take the announcer's privileged source port or forge its + /// datagrams, at stage 0 with `EPERM`, before its program starts, exactly where `published`, + /// as the [`PUBLISHED_CAPABILITIES`] say, and runs the others. + fn assert_publication(&mut self, published: bool) { + let mut executions = Vec::new(); + for (index, (capability, _)) in PUBLISHED_CAPABILITIES.into_iter().enumerate() { + if index > 0 { + executions.push(";"); + } + executions.extend(["--identity", "0:0", "--capabilities", capability]); + executions.extend(["--", "/bin/true"]); + } + let ran = self.exec(&executions); + let refused = format!("failed 0 {EPERM}"); + for (number, (capability, refuses)) in (1..).zip(PUBLISHED_CAPABILITIES) { + let facts = ["launch", "status"].map(|key| &ran[&format!("exec.{number}.{key}")]); + let expected = if published && refuses { + [refused.as_str(), "none"] + } else { + ["started", "exit 0"] + }; + assert_eq!(facts, expected, "{capability} of {published}: {ran:?}"); + } + } + /// Returns the configuration of the clone's activation record, which NVX keeps in the clone's /// sandbox, in its canonical encoding, while the clone holds its lease. fn configuration(&self) -> Configuration { @@ -1369,16 +1602,19 @@ impl Cloned { } /// Checks the arguments with which NVX started the clone's VMM, on Linux, as - /// [`assert_vmm_mount`](Self::assert_vmm_mount) does: `policy`, the arguments of the policy of - /// the template's NIC, in order, as the template's key holds them, and no other argument of a - /// network, so no `--net`, since OpenVMM takes the NIC and the guest's address from the - /// snapshot, nor any argument of a network for a template without a NIC. + /// [`assert_vmm_mount`](Self::assert_vmm_mount) does: `policy`, `--restore-network-policy`, + /// then the options of the policy that the clone's restore installs on the template's NIC, in + /// order, as NVX lowers the clone's network, and no other argument of a network, so no + /// `--net`, since OpenVMM takes the NIC and the guest's address from the snapshot, nor any + /// argument of a network for a template without a NIC. fn assert_vmm_network(&self, policy: &[String]) { #[cfg(target_os = "linux")] { let arguments = self.vmm_arguments(); let of_network = |argument: &&String| { - argument.starts_with("--net") || argument.as_str() == "--host-loopback" + argument.starts_with("--net") + || argument.starts_with("--host-loopback") + || argument.as_str() == "--restore-network-policy" }; assert_eq!( arguments.iter().filter(of_network).count(), @@ -3844,6 +4080,50 @@ fn a_clone_whose_mapping_fails_ends_before_its_sandbox_starts() { assert_payload(&template, false); } +/// Tries to exchange a line each way with a listener of a clone's guest through `address`, until +/// it did, or until `stop`, and returns whether it did: it reads the listener's greeting, answers +/// it, closes its side, and reads to the end. A connection that ends without the greeting, as +/// OpenVMM's do while the guest does not listen yet, or where it admits no connection of the host, +/// is retried. +fn reach_guest(address: SocketAddr, stop: &AtomicBool) -> bool { + while !stop.load(Ordering::SeqCst) { + if let Ok(mut stream) = TcpStream::connect_timeout(&address, Duration::from_secs(1)) { + stream + .set_read_timeout(Some(Duration::from_secs(5))) + .unwrap(); + let mut greeting = Vec::new(); + let mut byte = [0]; + while !greeting.ends_with(b"\n") && matches!(stream.read(&mut byte), Ok(1)) { + greeting.push(byte[0]); + } + if greeting == format!("{GREETING}\n").as_bytes() { + stream.write_all(format!("{ANSWER}\n").as_bytes()).unwrap(); + stream.shutdown(Shutdown::Write).unwrap(); + let mut rest = Vec::new(); + stream.read_to_end(&mut rest).unwrap(); + assert!(rest.is_empty(), "the guest's listener then sent {rest:?}"); + return true; + } + assert!( + greeting.is_empty(), + "the guest's listener greeted the host with {greeting:?}" + ); + } + thread::sleep(Duration::from_millis(100)); + } + false +} + +/// Sets its flag once it drops, so that a thread that waits for the flag ends even when the check +/// that the thread serves panics. +struct Stop<'a>(&'a AtomicBool); + +impl Drop for Stop<'_> { + fn drop(&mut self) { + self.0.store(true, Ordering::SeqCst); + } +} + /// Returns the facts that the [`MAP_RESOLVER`] probe found of a root's `/etc/resolv.conf`: the /// mounts below the root, and the options of the topmost mount at the file, what it shows, and /// whether a write of it is allowed. @@ -3918,28 +4198,20 @@ fn a_template_without_a_nic_serves_clones_without_a_network() { #[test] #[ignore = "boots VMs; see the module documentation for what they need"] -fn a_template_serves_the_clones_of_its_own_network_policy_alone() { - let other = "the network of the adapter's plan of a clone is not the one that the template's \ - generation 1 restores its NIC with"; - let address = format!( - "the adapter's plan of a clone gives its guest the network {OTHER_GUEST_NETWORK}, but the \ - template's generation 1 restores its NIC on {GUEST_NETWORK}, from its snapshot" - ); - let none = "the template's generation 1 has a NIC, whose network every clone of it restores, \ - but the adapter's plan of the clone has none"; +fn one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address() { + // The template's capture attached its NIC under the complete deny-all policy, whatever the + // policy of the network that its creation named, which publishes the guest's listeners, and + // forwards a port of host loopback. + let template = Template::in_network("cl-pol", Network::Forward); let mut policies = Vec::new(); for network in Network::WITH_NIC { - let template = Template::in_network(&format!("cl-p-{}", network.name()), network); - - // The template serves a clone of its own network: its NIC, which the clone's restore - // attached under the template's policy, unchanged, and a record that names the policy, - // the DNS server with which the template's guest configured its resolver at boot, where - // the policy lets the guest reach the gateway's, and the proxy, if any, and that - // publishes no listener. The NIC lets through what the policy allows alone: the clone - // reaches, at its gateway, the services of the host's side that the policy lets it reach, - // and no other. + // The restore of a clone of each network installs the network's policy on the template's + // NIC, and the clone's record names the policy, the DNS server that OpenVMM serves the + // guest under it, where it lets the guest reach the gateway's, the proxy, if any, and + // whether it publishes the workload's listeners. let own = template.network_options(network, GUEST_NETWORK); - let mut cloned = start_with(&template, "cl-own", &own, 0); + let name = format!("cl-{}", network.name()); + let mut cloned = start_with(&template, &name, &own, 0); cloned.assert_started(); let recorded = cloned .configuration() @@ -3947,52 +4219,134 @@ fn a_template_serves_the_clones_of_its_own_network_policy_alone() { .expect("the record names the network"); assert_eq!( (&recorded.dns, &recorded.proxy, recorded.publication), - (&network.dns(), &network.proxy(), Publication::Unpublished), + (&network.dns(), &network.proxy(), network.publication()), "{network:?}" ); policies.push((network, recorded.policy)); cloned.assert_vmm_network(&network.restored(GUEST_NETWORK)); assert_eq!(cloned.script(NETWORK_VIEW)[0], "devices 1", "{network:?}"); + + // The NIC lets through what the clone's policy allows alone: the clone reaches, at its + // gateway, the services of the host's side that the policy lets it reach, and no other. cloned.assert_reaches(network.reaches()); - cloned.stop(); - // A clone of any other network finds no template there: of each other network with a - // NIC, of the template's policy at another address of the guest, of a network that lets - // nothing through, which attaches no NIC, and without a network. NVX's plan refuses each - // before any VMM starts, and names both addresses where the guest's address alone - // differs. - let mut cases = Network::WITH_NIC - .into_iter() - .filter(|requested| *requested != network) - .map(|requested| (Some((requested, GUEST_NETWORK)), other)) - .collect::>(); - cases.extend([ - (Some((network, OTHER_GUEST_NETWORK)), address.as_str()), - (Some((Network::Deny, GUEST_NETWORK)), none), - (None, none), - ]); - for (requested, problem) in cases { - let options = requested.map_or_else(Vec::new, |(requested, guest)| { - template.network_options(requested, guest) - }); - let refused = start_with(&template, "cl-r", &options, CLONE_FAILED); - refused.assert_unserved(problem); + // The host reaches a listener of the clone's guest at its port of host loopback exactly + // where the policy allows host-loopback access both ways, once the announcer that the + // clone's guest started for the policy announced it, since the template's guest started + // none, and through the policy's forward. + let loopback = SocketAddr::from((Ipv4Addr::LOCALHOST, HOST_SIDE.listener)); + cloned.assert_listener(loopback, network.on_host_loopback()); + if network == Network::Forward { + let forward = SocketAddr::from((Ipv4Addr::LOCALHOST, HOST_SIDE.forward)); + cloned.assert_listener(forward, true); } - assert!(template.leases().is_empty()); - assert_payload(&template, false); + + // While the policy publishes the guest's listeners, the guest refuses the capabilities + // with which an execution could take the announcer's port or forge its datagrams. + cloned.assert_publication(network.publication() == Publication::Published); + cloned.stop(); } - // Each template's key holds a policy of its own, which its clones' records name. + + // NVX's plan refuses, before any VMM starts, a clone of a network at another address of the + // guest than the template's NIC's, of a network that lets nothing through, which attaches no + // NIC, and one without a network. + let elsewhere = format!( + "the adapter's plan of a clone attaches a network at {OTHER_GUEST_NETWORK}, but the NIC \ + of the template's generation 1 is at {GUEST_NETWORK}: a template serves the clones of \ + its own guest's address alone" + ); + let none = "the template's generation 1 has a NIC, which its clones restore with the network \ + of the adapter's plan, but the plan has none"; + let cases = [ + ( + Some((Network::Egress, OTHER_GUEST_NETWORK)), + elsewhere.as_str(), + ), + (Some((Network::Deny, GUEST_NETWORK)), none), + (None, none), + ]; + for (requested, problem) in cases { + let options = requested.map_or_else(Vec::new, |(requested, guest)| { + template.network_options(requested, guest) + }); + let refused = start_with(&template, "cl-r", &options, CLONE_FAILED); + refused.assert_unserved(problem); + } + + // Each clone's record names a policy of its own. for (index, (network, policy)) in policies.iter().enumerate() { for (other, other_policy) in &policies[index + 1..] { assert_ne!(policy, other_policy, "{network:?} and {other:?}"); } } + assert!(template.leases().is_empty()); + assert_payload(&template, false); +} + +#[test] +#[ignore = "boots VMs; see the module documentation for what they need"] +fn a_forward_whose_port_another_socket_holds_fails_the_clone_as_the_host_s() { + let template = Template::in_network("cl-busy", Network::Egress); + + // A socket of the host holds the port of host loopback that the forward of the clone's network + // names, so that OpenVMM cannot bind it as it restores the clone's NIC. + let busy = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).unwrap(); + let port = busy.local_addr().unwrap().port(); + let mut form = Network::HostLoopback.form(GUEST_NETWORK); + form["hostLoopbackForwards"] = json!([{ + "protocol": "tcp", + "hostPort": port, + "guestPort": HOST_SIDE.listener, + }]); + let network = template.root().join("network-busy.json"); + fs::write(&network, form.to_string()).unwrap(); + let options = ["--network".to_owned(), network.display().to_string()]; + + // The clone's VMM exits before its restore-ready event, and NVX attributes the failure to the + // host, by the evidence of the forward's bind, at the restore, which quarantines nothing. The + // start's diagnostic names the forward, and the start released the lease, with the records of + // the clone's launch. + let failed = start_with(&template, "cl-b", &options, CLONE_FAILED); + let printed = &failed.started; + assert!(printed.contains_key("pid"), "{printed:?}"); + assert_eq!( + failure(printed), + [ + "host", + "forward-bind", + "restore", + "spawned", + "none", + "exited" + ], + "{printed:?}" + ); + assert_eq!(printed["quarantined"], "no", "{printed:?}"); + assert_eq!(printed["released"], printed["lease"], "{printed:?}"); + let stderr = failed.sandbox.stderr(); + let cause = format!("failed to create socket for port forward (Tcp 127.0.0.1:{port})"); + assert!(stderr.contains(&cause), "{cause:?} in {stderr}"); + assert!(failed.sandbox.record().is_none()); + let store = SandboxStore::open(&failed.sandbox.store).unwrap(); + assert!(CloneRecords::read(&store).unwrap().is_none()); + assert!(template.leases().is_empty()); + + // Once the port is free, the template serves a clone of the same network. + drop(busy); + let mut cloned = start_with(&template, "cl-f", &options, 0); + cloned.assert_started(); + cloned.stop(); + assert!(template.leases().is_empty()); + assert_payload(&template, false); } #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only() { - let template = Template::in_network("cl-dns", Network::Egress); + // A template of a proxy-only network serves the clones of a network with egress allowed: its + // capture attached its NIC under the complete deny-all policy, under which its guest learned + // no DNS server. + let template = Template::in_network("cl-dns", Network::Proxy); let own = template.network_options(Network::Egress, GUEST_NETWORK); let mut generated = own.clone(); generated.extend(["--machine-id".to_owned(), "generated".to_owned()]); @@ -4008,8 +4362,8 @@ fn a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only() { }); let resolver = format!("nameserver {GATEWAY}"); for (cloned, mounts) in &mut clones { - // The record names the gateway's DNS server, with which the template's guest configured - // its resolver at boot. + // The record names the gateway's DNS server, which OpenVMM serves the guest under the + // clone's policy. let recorded = cloned .configuration() .network @@ -4071,7 +4425,8 @@ fn a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only() { #[test] #[ignore = "boots VMs; see the module documentation for what they need"] fn a_clone_s_executions_find_the_proxy_of_its_policy() { - let template = Template::in_network("cl-proxy", Network::Proxy); + // A template of a network with egress allowed serves the clones of a proxy-only network. + let template = Template::in_network("cl-proxy", Network::Egress); let own = template.network_options(Network::Proxy, GUEST_NETWORK); let mut cloned = start_with(&template, "cl-p", &own, 0); cloned.assert_started(); @@ -4138,7 +4493,7 @@ fn a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver() { // The record names the export, with its one mapping, and the network's DNS server. On Linux, // the restore bound the export's directory to the template's dormant filesystem slot, and - // attached the template's NIC under its policy. + // installed the clone's policy on the template's NIC. let configuration = cloned.configuration(); let recorded = configuration.export.map(|export| export.kind); assert_eq!( @@ -4221,8 +4576,8 @@ fn a_run_in_a_template_of_a_network_finds_its_policy_s_proxy() { let printed = &refused.started; assert_eq!(printed["preflight"], "staged", "{printed:?}"); refused.assert_unserved( - "the template's generation 1 has a NIC, whose network every clone of it restores, but \ - the adapter's plan of the clone has none", + "the template's generation 1 has a NIC, which its clones restore with the network of the \ + adapter's plan, but the plan has none", ); assert!( !printed diff --git a/tests/runtime-adapter/tests/vm_template.rs b/tests/runtime-adapter/tests/vm_template.rs index c9f786c5a..726a22d96 100644 --- a/tests/runtime-adapter/tests/vm_template.rs +++ b/tests/runtime-adapter/tests/vm_template.rs @@ -9,13 +9,15 @@ //! its key, and its snapshot, and whose key holds what the scenario asked for: the runtime of the //! adapter's VMs, with the guest agent's build, the backend, the processors and those online at //! boot, the RAM and its capacity, the geometries of the placeholder and of the dormant scratch, a -//! NIC exactly when the network lets something through, and a kernel command line of the platform +//! NIC exactly when the network lets something through, which the capture attaches under the +//! complete deny-all policy, whatever the network's own, and a kernel command line of the platform //! tier's tokens alone, with `maxcpus=` for a template whose processors are not all online at -//! boot, which names neither the placeholder's identity nor a root. The slot of the +//! boot, and the NIC's addresses, but neither a DNS server nor a publication of the guest's +//! listeners, which names neither the placeholder's identity nor a root. The slot of the //! template's class then holds no transaction, and records no launch, and a second creation serves -//! the generation without a capture. NVX refuses a network whose policy no template takes before -//! any VM starts, and the host program refuses a class that no capture takes before it creates the -//! store. +//! the generation without a capture, as does the creation of a network of any other policy at the +//! same guest's address, whose class is the same. The host program refuses a class that no capture +//! takes before it creates the store. //! //! Other scenarios run the host program's operations on the generations of a store: `refresh`, //! which captures a generation of a class whose store holds an eligible one already, and retires @@ -85,10 +87,15 @@ const GATEWAY: &str = "192.0.2.1"; const PROXY_PORT: u16 = 3128; /// The port of the gateway to which the allow rule lets TCP through. const RULE_PORT: u16 = 8080; +/// The port of host loopback that a forward forwards to the guest's [`RULE_PORT`]. The capture +/// binds no forward, so nothing holds it. +const FORWARD_PORT: u16 = 18080; /// The network of a scenario's template, which the creation reads from a file that the test /// writes, in the serde form of a plan's network: the guest at [`GUEST_NETWORK`], with egress -/// denied and no host-loopback access unless the network says otherwise. +/// denied and no host-loopback access unless the network says otherwise. The capture attaches the +/// NIC of a network that lets something through under the complete deny-all policy, whatever the +/// network's own. #[derive(Clone, Copy, Debug, PartialEq, Eq)] enum Network { /// No network, which attaches no network device. @@ -104,13 +111,27 @@ enum Network { Outbound, /// A proxy-only network, whose proxy is at [`PROXY_PORT`] of host loopback. Proxy, - /// Ingress allowed, which no template takes: OpenVMM would publish the guest's listeners. + /// Ingress allowed, under which OpenVMM would publish the guest's listeners. Ingress, - /// Host-loopback access in both directions, which no template takes either. + /// Host-loopback access in both directions, under which OpenVMM would publish the guest's + /// listeners on host loopback. HostLoopback, + /// Host-loopback access in both directions, with a forward from [`FORWARD_PORT`] of host + /// loopback to the guest's [`RULE_PORT`]. + Forward, } impl Network { + /// The networks that let something through, each at [`GUEST_NETWORK`]. + const WITH_NIC: [Self; 7] = [ + Self::Egress, + Self::AllowRule, + Self::Outbound, + Self::Proxy, + Self::Ingress, + Self::HostLoopback, + Self::Forward, + ]; /// Returns the network's serde form, or `None` without a network. fn form(self) -> Option { let mut form = json!({ @@ -141,6 +162,14 @@ impl Network { Self::Proxy => form["proxy"] = json!({"port": PROXY_PORT, "scheme": "http"}), Self::Ingress => form["ingress"] = json!("allow"), Self::HostLoopback => form["hostLoopback"] = json!("allow"), + Self::Forward => { + form["hostLoopback"] = json!("allow"); + form["hostLoopbackForwards"] = json!([{ + "protocol": "tcp", + "hostPort": FORWARD_PORT, + "guestPort": RULE_PORT, + }]); + } } Some(form) } @@ -412,8 +441,9 @@ fn assert_captured(templates: &Templates, created: &Printed, template: &Template // The kernel command line, which every clone restores: the platform tier's tokens alone, // which OpenVMM's capture admits, with the processors, those online at boot when they are - // fewer, and the NIC's addresses exactly when the template has a NIC, but neither the - // placeholder's identity nor a root's tokens. + // fewer, and the NIC's addresses exactly when the template has a NIC, but neither a DNS server + // nor a publication of the guest's listeners, which OpenVMM names to no guest under the NIC's + // complete deny-all policy, nor the placeholder's identity, nor a root's tokens. let command_line = key.command_line(); let tokens: Vec> = cmdline::parameters(command_line.as_bytes()) .map(|parameter| token(¶meter)) @@ -445,11 +475,23 @@ fn assert_captured(templates: &Templates, created: &Printed, template: &Template [limit] => assert!(limited && **limit == expected.as_bytes(), "{command_line}"), _ => panic!("{command_line} limits the processors online at boot more than once"), } - assert_eq!( - tokens.iter().any(|token| token.starts_with(b"virtnet_ip=")), - template.network.has_nic(), - "{command_line}" - ); + for address in ["virtnet_ip=", "virtnet_gw="] { + assert_eq!( + tokens + .iter() + .any(|token| token.starts_with(address.as_bytes())), + template.network.has_nic(), + "{address} in {command_line}" + ); + } + for absent in ["virtnet_dns=", "virtnet_ingress="] { + assert!( + !tokens + .iter() + .any(|token| token.starts_with(absent.as_bytes())), + "{absent} in {command_line}" + ); + } for tenant in [templates.identity.as_str(), "nvx_layer", "nvx_scratch"] { assert!(!command_line.contains(tenant), "{tenant} in {command_line}"); } @@ -497,26 +539,6 @@ fn captures(name: &str, template: Template) { assert_eq!(fs::read(&log).unwrap(), captured); } -/// Creates `template` in a template store of its own, in the sandbox of the test `name`, and -/// checks that NVX refuses its network with `problem` before any VM starts: the store indexes no -/// generation, and no VMM ran in any slot. -fn refuses(name: &str, template: Template, problem: &str) { - let mut templates = Templates::new(name); - templates.create(&template, FAILED); - let stderr = templates.sandbox.stderr(); - assert!(stderr.contains(problem), "{stderr}"); - assert!(templates.store().index().unwrap().generations().is_empty()); - let slots = fs::read_dir(templates.sandbox.store.join(CAPTURES_NAME)) - .into_iter() - .flatten(); - for slot in slots { - let slot = slot.unwrap().path(); - for name in [LOG_NAME, TRANSACTION_NAME] { - assert!(!slot.join(name).exists(), "{}", slot.display()); - } - } -} - #[test] #[ignore = "boots a VM; see the module documentation for what it needs"] fn a_template_without_a_network_has_no_nic() { @@ -551,46 +573,32 @@ fn a_network_that_lets_nothing_through_has_no_nic() { #[test] #[ignore = "boots a VM; see the module documentation for what it needs"] -fn an_egress_network_has_its_nic() { - captures("tpl-egress", Template::new(Network::Egress)); -} - -#[test] -#[ignore = "boots a VM; see the module documentation for what it needs"] -fn an_allow_rule_has_its_nic() { - captures("tpl-rule", Template::new(Network::AllowRule)); -} - -#[test] -#[ignore = "boots a VM; see the module documentation for what it needs"] -fn outbound_host_loopback_has_its_nic() { - captures("tpl-outbound", Template::new(Network::Outbound)); -} - -#[test] -#[ignore = "boots a VM; see the module documentation for what it needs"] -fn a_proxy_only_network_has_its_nic() { - captures("tpl-proxy", Template::new(Network::Proxy)); -} - -#[test] -#[ignore = "needs NVX's artifacts; see the module documentation"] -fn ingress_is_refused_before_any_vm_starts() { - refuses( - "tpl-ingress", - Template::new(Network::Ingress), - "a template's network policy cannot allow ingress", - ); -} +fn the_networks_of_every_policy_at_one_guest_s_address_share_one_template() { + // A creation of a network that would publish the guest's listeners, and forwards a port of + // host loopback, captures a template whose NIC its capture attached under the complete + // deny-all policy. + let mut templates = Templates::new("tpl-nic"); + let template = Template::new(Network::Forward); + let created = templates.create(&template, 0); + assert_captured(&templates, &created, &template); + let log = templates.slot(&created["class"]).join(LOG_NAME); + let captured = fs::read(&log).unwrap(); -#[test] -#[ignore = "needs NVX's artifacts; see the module documentation"] -fn host_loopback_both_ways_is_refused_before_any_vm_starts() { - refuses( - "tpl-loopback", - Template::new(Network::HostLoopback), - "a template's network policy cannot allow host-loopback access both ways", - ); + // A creation of each network that lets something through, at the same guest's address, finds + // the same class, whatever its policy, and serves the generation without a capture. + for network in Network::WITH_NIC { + let existing = templates.create(&Template::new(network), 0); + assert_eq!(existing["created"], "existing", "{network:?}: {existing:?}"); + for printed in ["generation", "key", "class"] { + assert_eq!( + existing[printed], created[printed], + "{network:?}: {printed}" + ); + } + } + assert_eq!(templates.store().index().unwrap().generations().len(), 1); + // No VMM ran since the capture's, whose log the slot keeps. + assert_eq!(fs::read(&log).unwrap(), captured); } #[test]